RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,771 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

2243 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 7b515a0485 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-03-25 22:13:49 +05:00
Andrey Rakhmatullin a97dc55c71 Add/improve docs. 2021-03-25 20:49:12 +05:00
Pratik Mahankal f0e1a33225
Sort the list of Request.meta alphabetically #5061 (#5065) 2021-03-23 22:16:50 +05:00
Adrián Chaves 0dad0fce72
Use pip<20.3 to fix ReadTheDocs builds (#5052) 2021-03-19 11:13:05 +01:00
Adrián Chaves 1d83697990
Merge pull request #4769 from scrapy/http2 
Implement experimental HTTP/2 support
 2021-03-18 22:18:12 +01:00
Mikhail Korobov ab037ab7c9
Merge pull request #5016 from deepang17/scrapy-4996 
DOCS: Cover scrapy-bench in the documentation
 2021-03-12 04:18:56 +05:00
Eugenio Lacuesta 0c16088230
headers_received signal (#4897) 2021-03-11 11:52:35 -03:00
Adrián Chaves 5b2d3e17a3
Merge branch 'master' into http2 2021-03-09 14:59:17 +01:00
deepang17 2d41c77e6a Merge branch 'master' of https://github.com/scrapy/scrapy into scrapy-4996 2021-03-09 17:33:05 +05:30
deepang17 3d88ac605b FIX: Updated benchmarking.rst 2021-03-09 17:19:34 +05:30
Andrey Rahmatullin 954c4b48e6
Merge pull request #3667 from Gallaecio/long-allowed-domains 
Add a FAQ entry on how to deal with long lists of allowed domains
 2021-03-05 20:16:37 +05:00
deepang17 36f1dbf665 DOCS: Covered scrapy-bench 2021-03-02 22:12:44 +05:30
deepang17 b25616d107 DOCS: Cover scrapy-bench in the documentation 2021-02-28 16:26:46 +05:30
deepang17 7a54580679 DOCS:Cover scrapy-bench in the documentation 2021-02-28 15:33:09 +05:30
Adrián Chaves f689615e8d Close files in the PerYearXmlExportPipeline documentation example 2021-02-24 12:54:56 +01:00
Wehzie a36f952198
fixed typo "an quotes.json" -> "a quotes.json" (#5005) 2021-02-24 08:15:44 +01:00
Andrey Rakhmatullin 58f8481301 Update docs. 2021-02-05 16:21:56 +05:00
Adrián Chaves 4c801551fa Document that the bytes_received signal is not yet implemented for HTTP/2 2021-02-03 21:11:46 +01:00
Adrián Chaves 1a7bde0d8e Document that HTTP/2 server pushes are ignored 2021-02-03 10:55:11 +01:00
Adrián Chaves c8d8b180bf Merge remote-tracking branch 'upstream/master' into http2 2021-02-03 09:37:35 +01:00
Adrián Chaves 2ce8e0c742 Document the (hard-coded) maximum HTTP/2 frame size accepted from servers 2021-02-03 09:09:53 +01:00
Adrián Chaves f30f53b3cc Scrapinghub → Zyte 2021-02-02 15:03:20 +01:00
M Ikram Ullah Khan 0a1e2fefab
Docs: fix typo in news.rst (#4942) 2021-01-04 10:30:23 -03:00
Eugenio Lacuesta e494a3f733
protocol attribute for h2 responses 2020-12-31 11:50:15 -03:00
Eugenio Lacuesta d698b5147b
Merge branch 'master' into http2 2020-12-31 11:13:25 -03:00
Adrián Chaves d0af008608
Merge pull request #4878 from elacuesta/response-protocol-attribute 
Response.protocol attribute
 2020-12-16 15:27:40 +01:00
gunadhya db10aaf9eb
Update links in installation guide (#4899) 2020-12-03 10:56:36 +01:00
etimoz 95d39d5cb4 removed wrong super argument in overriding serialize_fields code example 2020-11-29 13:24:04 +01:00
Adrián Chaves 63becd1bc8
Update news.rst 2020-11-17 21:58:08 +01:00
Adrián Chaves 15d301e968
Cover Scrapy 2.4.1 in the release notes (#4884) 
Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
 2020-11-17 09:16:08 +01:00
Eugenio Lacuesta 5e9a99e6a1
Reponse.protocol as string 2020-11-11 11:15:29 -03:00
Eugenio Lacuesta 2242412556
Docs: placeholder for versionadded directive 2020-11-11 10:50:54 -03:00
Eugenio Lacuesta 61d089485c
Docs: sort versionadded directives 2020-11-11 01:31:21 -03:00
Eugenio Lacuesta 587b4dd71f
Docs for the Response.protocol attribute 2020-11-11 01:22:33 -03:00
Eugenio Lacuesta 114229eb4a
Docs: add a note about asyncio.set_event_loop 2020-11-06 13:29:14 -03:00
Adrián Chaves 47eac83740 Set a release date for Scrapy 2.4.0 2020-10-11 22:11:14 +02:00
Adrián Chaves 015c82b974
Scrapy 2.4 release notes (#4808) 2020-10-11 22:09:45 +02:00
Andrey Rahmatullin 9f8c3938cc
Merge pull request #4823 from elacuesta/cookies-revert-header 
Do not process cookies from headers
 2020-10-08 20:20:32 +05:00
Andrey Rahmatullin 45c06cfd80
Merge pull request #4831 from starrify/downloadermw-support-zstd 
Adding support for zstd in HttpCompressionMiddleware
 2020-10-08 14:08:18 +05:00
Adrián Chaves 269fe35d6d
Merge branch 'master' into http2 2020-10-06 21:05:10 +02:00
OfirD1 1a597d5e3d
moved the sentence about processing pending requests when a spider is closed onto a generic note. 2020-10-06 21:54:42 +03:00
Eugenio Lacuesta 137c8ba6ee
Docs: mention limitation about Cookie header 2020-10-06 10:50:17 -03:00
P. Chen da3171d4f7
Using the `zstandard` package than `zstd` for supporting frames both with and without the content size info 
See also: https://github.com/sergey-dryabzhinsky/python-zstd/issues/53
 2020-10-05 23:18:58 +01:00
P. Chen c6c3f2ce66
Updating the doc entry for the HTTP compress downloader middleware on zstd 2020-10-05 21:10:40 +01:00
Adrián Chaves 004b40a719
as soon as → as long as (#4825) 2020-10-02 19:53:55 -03:00
Mikhail Korobov 4f27c5f82b
Merge pull request #4768 from maranqz/feature/4606-exporter-from-FEEDS 
Pass info from FEEDS to ItemExporter
 2020-10-02 00:10:05 +05:00
Habeeb Shopeju f47b120e2b
Documentation of link extractor usage (#4775) 
* Added description when using link extractor outside crawlspiders and created reference documentation for scrapy.link.Link class

* Added link.rst to toctree

* Corrected spelling errors, moved docs to Link doctstring to use autoclass

* Moved link docs to link_extractors

* Update docs/topics/link-extractors.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update link.py

Improvements to URL description

* Update link.py

* Update link.py

Fixed line length Flake issue

* Update link.py

Fixed trailing whitespace

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-10-01 23:50:11 +05:00
Andrey Rahmatullin 5a38639359
Merge pull request #4809 from madeny/patch-1 
Correct some typos
 2020-09-28 18:26:43 +05:00
madeny eff96038c7
Correct some typos 
This won't be an issue if **your** spider doesn't rely on cookies.
 2020-09-26 22:50:38 +02:00
Mirwaisse Djanbaz 3989f64baa
dependecies → dependencies (#4801) 2020-09-21 14:40:00 +02:00