mirror of https://github.com/scrapy/scrapy.git
Merge pull request #2670 from qhuang872/master
[MRG+1] Update spiders.rst
This commit is contained in:
commit
b9d3b447a5
|
|
@ -80,6 +80,9 @@ scrapy.Spider
|
|||
allowed to crawl. Requests for URLs not belonging to the domain names
|
||||
specified in this list (or their subdomains) won't be followed if
|
||||
:class:`~scrapy.spidermiddlewares.offsite.OffsiteMiddleware` is enabled.
|
||||
|
||||
Let's say your target url is ``https://www.example.com/1.html``,
|
||||
then add ``'example.com'`` to the list.
|
||||
|
||||
.. attribute:: start_urls
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue