RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #2963 from djunzu/mention_request_meta_depth_on_depthmiddleware_doc 

Add note about request.meta['depth'] in DepthMiddleware
This commit is contained in:
Mikhail Korobov 2017-10-26 16:15:36 +05:00 committed by GitHub
parent 73c40eb324 8288f78a39
commit 496fc60b25
1 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
docs/topics/spider-middleware.rst
View File

@ -188,9 +188,13 @@ DepthMiddleware
.. class:: DepthMiddleware
DepthMiddleware is a scrape middleware used for tracking the depth of each
Request inside the site being scraped. It can be used to limit the maximum
depth to scrape or things like that.
DepthMiddleware is used for tracking the depth of each Request inside the
site being scraped. It works by setting `request.meta['depth'] = 0` whenever
there is no value previously set (usually just the first Request) and
incrementing it by 1 otherwise.
It can be used to limit the maximum depth to scrape, control Request
priority based on their depth, and things like that.
The :class:`DepthMiddleware` can be configured through the following
settings (see the settings documentation for more info):