RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Lower the concurrency settings in the default project template. (#6918) 

* Lower the concurrency settings in the default project template.

* Simplify the concurrency settings template.
This commit is contained in:
Andrey Rakhmatullin 2025-06-27 17:40:31 +05:00 committed by GitHub
parent eaae59fbef
commit f454465b14
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
scrapy/templates/project/module/settings.py.tmpl
View File

@ -21,16 +21,10 @@ ADDONS = {}
# Obey robots.txt rules
ROBOTSTXT_OBEY = True
# Configure maximum concurrent requests performed by Scrapy (default: 16)
#CONCURRENT_REQUESTS = 32
# Configure a delay for requests for the same website (default: 0)
# See https://docs.scrapy.org/en/latest/topics/settings.html#download-delay
# See also autothrottle settings and docs
#DOWNLOAD_DELAY = 3
# The download delay setting will honor only one of:
#CONCURRENT_REQUESTS_PER_DOMAIN = 16
#CONCURRENT_REQUESTS_PER_IP = 16
# Concurrency and throttling settings
#CONCURRENT_REQUESTS = 16
CONCURRENT_REQUESTS_PER_DOMAIN = 1
DOWNLOAD_DELAY = 1
# Disable cookies (enabled by default)
#COOKIES_ENABLED = False