RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,464 Commits 49 Branches 132 Tags 174 MiB
Commit Graph

79 Commits

Author SHA1 Message Date
Adrián Chaves 3f0a677c04
Cover version directive usage in the documentation policy (#4310) 
* Cover version directives in the documentation policy

* Remove version directives in preparation for Scrapy 2.0

* Update the policy based on the deprecation policy

* Only remove version directives after 3 years
 2020-08-27 23:56:58 +05:00
Eugenio Lacuesta ec98dabfab
Support for dataclass and attrs items (#3881) 2020-06-14 11:45:27 +02:00
Aditya f37b1bdc56 [docs] update redirect links to python3 2020-03-20 05:31:41 +05:30
Aditya ca08e04198 [docs] update redirect links python2 -> python3 2020-03-20 02:31:35 +05:30
Adrián Chaves 6ea62cacc7 Add a FAQ entry about splitting items in item pipelines 2019-07-08 13:58:55 +02:00
Konstantin Lopuhin b5c552d17f
Merge pull request #2061 from elacuesta/process_spider_exception_generator 
[MRG+1] process_spider_exception on generators
 2019-04-01 10:43:19 +03:00
Adrián Chaves d346b8cb0f
Merge pull request #3626 from Gallaecio/middlewares-from-crawler 
[MRG+1] Document that the main entry point of downloader and spider middlewar…
 2019-03-15 08:58:35 +01:00
Adrián Chaves 75d6f56c8a Switch from ` to `` where inline code formatting is desired 2019-03-01 16:56:58 +01:00
Adrián Chaves 50bf4c60c4 Document that the main entry point of downloader and spider middlewares is from_crawler() 2019-02-13 17:39:20 +01:00
Eugenio Lacuesta 40449fa0eb Update docs, add tests, remove FIXME comment 2018-08-03 18:20:25 -03:00
Eugenio Lacuesta e96b7782bc Merge branch 'master' into process_spider_exception_generator_experiment 2018-07-15 18:17:37 -03:00
Daniel Graña f8f5f463e6
Merge pull request #3253 from rpkilby/depth-stats 
[MRG+1] Update depth middleware stats (fixes #3245)
 2018-07-11 12:00:08 -03:00
Eugenio Lacuesta df75a0942e Update docs 2018-07-01 13:35:14 -03:00
Eugenio Lacuesta ab48837f09 Merge branch 'master' 2018-06-24 20:31:37 -03:00
Ryan P Kilby 2dfc5d128b Update DEPTH_STATS refs to DEPTH_STATS_VERBOSE 2018-05-17 11:00:21 -04:00
grammy-jiang cb76b88331 fix a mistake in topic spider-middleware.rst 2018-04-04 05:56:05 -04:00
Jesse Bakker 0b14cb44aa Added from_crawler to middleware docs 2017-11-23 15:25:43 +01:00
djunzu 8288f78a39 Add note about request.meta['depth'] in DepthMiddleware 2017-10-16 21:34:37 -02:00
Paul Tremberth bc200d1155 Rename setting to REFERRER_POLICY (with 2 Rs) 2017-03-01 17:51:23 +01:00
Paul Tremberth 537683f945 Add autoclass directives to document built-in policies 2017-03-01 17:51:23 +01:00
Paul Tremberth 3dc09eeceb Use table for referrer policy options 2017-03-01 17:51:23 +01:00
Paul Tremberth 605935f015 Edit text 2017-03-01 17:51:23 +01:00
Paul Tremberth eb07285a63 Reword warning on no-referrer-when-downgrade policy 2017-03-01 17:51:23 +01:00
Paul Tremberth 03ff19d188 Update docs for new "referrer_policy" Request.meta key 2017-03-01 17:51:23 +01:00
Paul Tremberth e249abc32b Update docs 2017-03-01 17:50:39 +01:00
Paul Tremberth c86f568b9c Update docs with "strict-..." policies 2017-03-01 17:50:39 +01:00
Paul Tremberth c9c59db489 Update documentation about REFERER_POLICY setting 2017-03-01 17:50:39 +01:00
Eugenio Lacuesta 706ed0e049 Spider middleware: process_spider_exception on generators 2017-03-01 12:01:53 -03:00
Takehiro Shiozaki fcb3daf4fa fix typo 2017-02-06 14:03:41 +09:00
Jose Ricardo e12e364a40 Add details to the spider middlewares docs 
Document the effects of the middleware order in a more detailed way.
 2016-10-18 12:29:30 -02:00
nyov 5876b9aa30 Update documentation links 2016-03-03 16:28:33 +00:00
Νικόλαος-Διγενής Καραγιάννης 1cffa99e0d tests+doc for subdomains in offsite middleware 2016-01-26 12:49:43 +02:00
Jakob de Maeyer e66f649894 Bring back _BASE settings 2015-11-11 17:39:56 +01:00
Jakob de Maeyer 26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Julia Medina d3f576a816 Move scrapy/spider.py to scrapy/spiders/__init__.py 2015-05-09 04:20:09 -03:00
Julia Medina 180272c092 Move scrapy/contrib/spidermiddleware to scrapy/spidermiddlewares 2015-04-29 21:26:35 -03:00
Pablo Hoffman bb4c922d85 Merge pull request #1081 from scrapy/dict-items 
Allow spiders to return dicts.
 2015-03-27 15:19:27 -03:00
Mikhail Korobov 817dbc6cbd DOC mention dicts in documentation; explain better what are Items for 2015-03-19 05:16:14 +05:00
Shadab Zafar 5a58d64131 Fix some redirection links in documentation 
Fixes #606
 2015-03-18 19:41:26 -03:00
Mikhail Korobov baf5c59386 Merge pull request #1071 from eliasdorneles/updating-request-meta-special-keys 
updating list of Request.meta special keys
 2015-03-13 16:38:19 +05:00
Elias Dorneles f7031c08ff updating list of Request.meta special keys 2015-03-10 22:29:07 -03:00
Mikhail Korobov 283d6a5344 DOC a couple more references are fixed 2015-01-19 22:07:03 +05:00
Mikhail Korobov 73e6b35622 DOC fix a reference 2015-01-19 22:02:46 +05:00
Mikhail Korobov e435b3e3a3 DOC simplify extension docs 2014-09-21 00:19:24 +06:00
Mikhail Korobov 2d3803672b DOC use top-level shortcuts in docs 2014-04-15 01:09:35 +06:00
Nikolaos-Digenis Karagiannis 4335420f40 SpiderMW doc typo: SWP request, response 2014-03-06 16:09:37 +02:00
Mikhail Korobov a27d91f0a6 Rename BaseSpider to Spider. See GH-495. 2013-12-30 19:46:41 +06:00
Pablo Hoffman f87be371a2 better names for HANDLE_* settings, and added doc 2013-11-21 14:33:17 -02:00
Steven Almeroth f62b6660d4 doc: fix typo in spider middleware 2013-03-02 19:46:31 -06:00
Chris Tilden aae6aed4fb fixes spelling errors in documentation 2013-01-22 14:52:18 -08:00