RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,607 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

101 Commits

Author SHA1 Message Date
pankaj1707k c1bbb299d7
Add and run pre-commit hook 'blacken-docs' 
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
 2023-02-01 16:30:57 +05:30
Adrián Chaves 06c8f673af
2.7 release notes (#5680) 
* Fix the display name of documented fingerprinter class methods

* Initial draft for the Scrapy 2.7 release notes

* Update VERSION and PREVIOUS_VERSION references

* Clarify the restrictions lifted for item field output names

* Fix the description of the BOM bug fix

* Fix the note about changes in MIME sniffing

* Fix typo

* Extend highlights

* Fyx typo
 2022-10-17 18:04:29 +05:00
Adrián Chaves b95c634b86 Document how to enforce Scrapy versions on Scrapy components 2022-03-17 22:23:25 +01:00
Adrián Chaves fd08bb6cd9 Refactor the asynchronous process_spider_output documentation 2022-03-16 18:10:57 +01:00
Andrey Rakhmatullin f789547551 Implement spider middleware iterable upgrade/downgrade. 2022-01-11 18:28:32 +05:00
Andrey Rakhmatullin c105a7a85f Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-10-22 21:52:47 +05:00
Andrey Rakhmatullin 66d8a71920 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-10-11 20:24:22 +05:00
Adrián Chaves f108800222
Update spider-middleware.rst 2021-10-02 13:25:15 +02:00
Deepanshu verma de70b3c58b
Update spider-middleware.rst 
added empty line
 2021-10-02 12:12:58 +05:30
Deepanshu de2043f9c1 updated docs/topics/spider-middleware.rst 2021-10-01 20:20:00 +05:30
Deepanshu 74f146bbe0 Document update URLLENGTH_LIMIT 2021-10-01 01:47:05 +05:30
Adrián Chaves 3c57825b0f
Update docs/topics/spider-middleware.rst 2021-09-26 13:41:26 +02:00
Reza (Milad) Maanijou dfdb779756
Apply review comments 2021-09-26 12:45:44 +03:30
Reza (Milad) Maanijou 1829dd774c
Update spider-middleware.rst 2021-09-25 20:22:09 +03:30
maanijou ac9175964d Improve documentation for spider middlewares 2021-09-12 17:59:20 +02:00
Adrián Chaves 8a2768561f
Merge branch 'master' into asyncio-parse-asyncgen-proper-rebased 2021-07-16 14:32:15 +02:00
Marlena Chatzigrigoriou d7deba7e89
Document all import paths and use the shortest in examples (#5099) 2021-07-14 10:34:28 +02:00
Andrey Rakhmatullin ffc6f525ce Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-04-14 17:34:58 +05:00
Andrey Rakhmatullin 849472535e Update docs. 2021-04-02 20:20:35 +05:00
Mikhail Korobov f0c8d31193
Merge pull request #4694 from Jgaldos/improve-httpstatus-all-meta 
Improve http status all on http error middleware
 2021-04-01 22:43:45 +05:00
Andrey Rakhmatullin a97dc55c71 Add/improve docs. 2021-03-25 20:49:12 +05:00
Jose Galdos a41c205928 Update httpstatus documentation. 2020-08-31 08:58:43 -05:00
Adrián Chaves 3f0a677c04
Cover version directive usage in the documentation policy (#4310) 
* Cover version directives in the documentation policy

* Remove version directives in preparation for Scrapy 2.0

* Update the policy based on the deprecation policy

* Only remove version directives after 3 years
 2020-08-27 23:56:58 +05:00
Eugenio Lacuesta ec98dabfab
Support for dataclass and attrs items (#3881) 2020-06-14 11:45:27 +02:00
Aditya f37b1bdc56 [docs] update redirect links to python3 2020-03-20 05:31:41 +05:30
Aditya ca08e04198 [docs] update redirect links python2 -> python3 2020-03-20 02:31:35 +05:30
Adrián Chaves 6ea62cacc7 Add a FAQ entry about splitting items in item pipelines 2019-07-08 13:58:55 +02:00
Konstantin Lopuhin b5c552d17f
Merge pull request #2061 from elacuesta/process_spider_exception_generator 
[MRG+1] process_spider_exception on generators
 2019-04-01 10:43:19 +03:00
Adrián Chaves d346b8cb0f
Merge pull request #3626 from Gallaecio/middlewares-from-crawler 
[MRG+1] Document that the main entry point of downloader and spider middlewar…
 2019-03-15 08:58:35 +01:00
Adrián Chaves 75d6f56c8a Switch from ` to `` where inline code formatting is desired 2019-03-01 16:56:58 +01:00
Adrián Chaves 50bf4c60c4 Document that the main entry point of downloader and spider middlewares is from_crawler() 2019-02-13 17:39:20 +01:00
Eugenio Lacuesta 40449fa0eb Update docs, add tests, remove FIXME comment 2018-08-03 18:20:25 -03:00
Eugenio Lacuesta e96b7782bc Merge branch 'master' into process_spider_exception_generator_experiment 2018-07-15 18:17:37 -03:00
Daniel Graña f8f5f463e6
Merge pull request #3253 from rpkilby/depth-stats 
[MRG+1] Update depth middleware stats (fixes #3245)
 2018-07-11 12:00:08 -03:00
Eugenio Lacuesta df75a0942e Update docs 2018-07-01 13:35:14 -03:00
Eugenio Lacuesta ab48837f09 Merge branch 'master' 2018-06-24 20:31:37 -03:00
Ryan P Kilby 2dfc5d128b Update DEPTH_STATS refs to DEPTH_STATS_VERBOSE 2018-05-17 11:00:21 -04:00
grammy-jiang cb76b88331 fix a mistake in topic spider-middleware.rst 2018-04-04 05:56:05 -04:00
Jesse Bakker 0b14cb44aa Added from_crawler to middleware docs 2017-11-23 15:25:43 +01:00
djunzu 8288f78a39 Add note about request.meta['depth'] in DepthMiddleware 2017-10-16 21:34:37 -02:00
Paul Tremberth bc200d1155 Rename setting to REFERRER_POLICY (with 2 Rs) 2017-03-01 17:51:23 +01:00
Paul Tremberth 537683f945 Add autoclass directives to document built-in policies 2017-03-01 17:51:23 +01:00
Paul Tremberth 3dc09eeceb Use table for referrer policy options 2017-03-01 17:51:23 +01:00
Paul Tremberth 605935f015 Edit text 2017-03-01 17:51:23 +01:00
Paul Tremberth eb07285a63 Reword warning on no-referrer-when-downgrade policy 2017-03-01 17:51:23 +01:00
Paul Tremberth 03ff19d188 Update docs for new "referrer_policy" Request.meta key 2017-03-01 17:51:23 +01:00
Paul Tremberth e249abc32b Update docs 2017-03-01 17:50:39 +01:00
Paul Tremberth c86f568b9c Update docs with "strict-..." policies 2017-03-01 17:50:39 +01:00
Paul Tremberth c9c59db489 Update documentation about REFERER_POLICY setting 2017-03-01 17:50:39 +01:00
Eugenio Lacuesta 706ed0e049 Spider middleware: process_spider_exception on generators 2017-03-01 12:01:53 -03:00