RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,312 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

75 Commits

Author SHA1 Message Date
Adrián Chaves 6ea62cacc7 Add a FAQ entry about splitting items in item pipelines 2019-07-08 13:58:55 +02:00
Konstantin Lopuhin b5c552d17f
Merge pull request #2061 from elacuesta/process_spider_exception_generator 
[MRG+1] process_spider_exception on generators
 2019-04-01 10:43:19 +03:00
Adrián Chaves d346b8cb0f
Merge pull request #3626 from Gallaecio/middlewares-from-crawler 
[MRG+1] Document that the main entry point of downloader and spider middlewar…
 2019-03-15 08:58:35 +01:00
Adrián Chaves 75d6f56c8a Switch from ` to `` where inline code formatting is desired 2019-03-01 16:56:58 +01:00
Adrián Chaves 50bf4c60c4 Document that the main entry point of downloader and spider middlewares is from_crawler() 2019-02-13 17:39:20 +01:00
Eugenio Lacuesta 40449fa0eb Update docs, add tests, remove FIXME comment 2018-08-03 18:20:25 -03:00
Eugenio Lacuesta e96b7782bc Merge branch 'master' into process_spider_exception_generator_experiment 2018-07-15 18:17:37 -03:00
Daniel Graña f8f5f463e6
Merge pull request #3253 from rpkilby/depth-stats 
[MRG+1] Update depth middleware stats (fixes #3245)
 2018-07-11 12:00:08 -03:00
Eugenio Lacuesta df75a0942e Update docs 2018-07-01 13:35:14 -03:00
Eugenio Lacuesta ab48837f09 Merge branch 'master' 2018-06-24 20:31:37 -03:00
Ryan P Kilby 2dfc5d128b Update DEPTH_STATS refs to DEPTH_STATS_VERBOSE 2018-05-17 11:00:21 -04:00
grammy-jiang cb76b88331 fix a mistake in topic spider-middleware.rst 2018-04-04 05:56:05 -04:00
Jesse Bakker 0b14cb44aa Added from_crawler to middleware docs 2017-11-23 15:25:43 +01:00
djunzu 8288f78a39 Add note about request.meta['depth'] in DepthMiddleware 2017-10-16 21:34:37 -02:00
Paul Tremberth bc200d1155 Rename setting to REFERRER_POLICY (with 2 Rs) 2017-03-01 17:51:23 +01:00
Paul Tremberth 537683f945 Add autoclass directives to document built-in policies 2017-03-01 17:51:23 +01:00
Paul Tremberth 3dc09eeceb Use table for referrer policy options 2017-03-01 17:51:23 +01:00
Paul Tremberth 605935f015 Edit text 2017-03-01 17:51:23 +01:00
Paul Tremberth eb07285a63 Reword warning on no-referrer-when-downgrade policy 2017-03-01 17:51:23 +01:00
Paul Tremberth 03ff19d188 Update docs for new "referrer_policy" Request.meta key 2017-03-01 17:51:23 +01:00
Paul Tremberth e249abc32b Update docs 2017-03-01 17:50:39 +01:00
Paul Tremberth c86f568b9c Update docs with "strict-..." policies 2017-03-01 17:50:39 +01:00
Paul Tremberth c9c59db489 Update documentation about REFERER_POLICY setting 2017-03-01 17:50:39 +01:00
Eugenio Lacuesta 706ed0e049 Spider middleware: process_spider_exception on generators 2017-03-01 12:01:53 -03:00
Takehiro Shiozaki fcb3daf4fa fix typo 2017-02-06 14:03:41 +09:00
Jose Ricardo e12e364a40 Add details to the spider middlewares docs 
Document the effects of the middleware order in a more detailed way.
 2016-10-18 12:29:30 -02:00
nyov 5876b9aa30 Update documentation links 2016-03-03 16:28:33 +00:00
Νικόλαος-Διγενής Καραγιάννης 1cffa99e0d tests+doc for subdomains in offsite middleware 2016-01-26 12:49:43 +02:00
Jakob de Maeyer e66f649894 Bring back _BASE settings 2015-11-11 17:39:56 +01:00
Jakob de Maeyer 26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Julia Medina d3f576a816 Move scrapy/spider.py to scrapy/spiders/__init__.py 2015-05-09 04:20:09 -03:00
Julia Medina 180272c092 Move scrapy/contrib/spidermiddleware to scrapy/spidermiddlewares 2015-04-29 21:26:35 -03:00
Pablo Hoffman bb4c922d85 Merge pull request #1081 from scrapy/dict-items 
Allow spiders to return dicts.
 2015-03-27 15:19:27 -03:00
Mikhail Korobov 817dbc6cbd DOC mention dicts in documentation; explain better what are Items for 2015-03-19 05:16:14 +05:00
Shadab Zafar 5a58d64131 Fix some redirection links in documentation 
Fixes #606
 2015-03-18 19:41:26 -03:00
Mikhail Korobov baf5c59386 Merge pull request #1071 from eliasdorneles/updating-request-meta-special-keys 
updating list of Request.meta special keys
 2015-03-13 16:38:19 +05:00
Elias Dorneles f7031c08ff updating list of Request.meta special keys 2015-03-10 22:29:07 -03:00
Mikhail Korobov 283d6a5344 DOC a couple more references are fixed 2015-01-19 22:07:03 +05:00
Mikhail Korobov 73e6b35622 DOC fix a reference 2015-01-19 22:02:46 +05:00
Mikhail Korobov e435b3e3a3 DOC simplify extension docs 2014-09-21 00:19:24 +06:00
Mikhail Korobov 2d3803672b DOC use top-level shortcuts in docs 2014-04-15 01:09:35 +06:00
Nikolaos-Digenis Karagiannis 4335420f40 SpiderMW doc typo: SWP request, response 2014-03-06 16:09:37 +02:00
Mikhail Korobov a27d91f0a6 Rename BaseSpider to Spider. See GH-495. 2013-12-30 19:46:41 +06:00
Pablo Hoffman f87be371a2 better names for HANDLE_* settings, and added doc 2013-11-21 14:33:17 -02:00
Steven Almeroth f62b6660d4 doc: fix typo in spider middleware 2013-03-02 19:46:31 -06:00
Chris Tilden aae6aed4fb fixes spelling errors in documentation 2013-01-22 14:52:18 -08:00
Pablo Hoffman be206ca5ab added process_start_requests method to spider middlewares 2012-08-31 16:41:50 -03:00
Pablo Hoffman 4ec99117d3 fixed minor doc typo 2012-08-30 11:56:30 -03:00
stav f1802289cd small doc typo change to get the fork rolling 2012-04-11 12:05:39 -05:00
Pablo Hoffman 8933e2f2be added REFERER_ENABLED setting, to control referer middleware 2012-03-22 16:35:14 -03:00