RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Include loading settings in `Running multiple spiders in the same process` section 

The example in the documentation doesn't take into account the project settings
This commit is contained in:
Ricardo Amendoeira 2021-03-29 01:48:28 +01:00 committed by GitHub
parent 63becd1bc8
commit b247fa9982
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/topics/practices.rst
View File

@ -118,6 +118,7 @@ Here is an example that runs multiple spiders simultaneously:
::
import scrapy
from scrapy.utils.project import get_project_settings
from scrapy.crawler import CrawlerProcess
class MySpider1(scrapy.Spider):
@ -128,7 +129,8 @@ Here is an example that runs multiple spiders simultaneously:
# Your second spider definition
...
process = CrawlerProcess()
settings = get_project_settings()
process = CrawlerProcess(settings)
process.crawl(MySpider1)
process.crawl(MySpider2)
process.start() # the script will block here until all crawling jobs are finished