Merge pull request #2670 from qhuang872/master

[MRG+1] Update spiders.rst
This commit is contained in:
Daniel Graña 2017-05-18 16:31:01 -03:00 committed by GitHub
commit b9d3b447a5
1 changed files with 3 additions and 0 deletions

View File

@ -80,6 +80,9 @@ scrapy.Spider
allowed to crawl. Requests for URLs not belonging to the domain names
specified in this list (or their subdomains) won't be followed if
:class:`~scrapy.spidermiddlewares.offsite.OffsiteMiddleware` is enabled.
Let's say your target url is ``https://www.example.com/1.html``,
then add ``'example.com'`` to the list.
.. attribute:: start_urls