RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge 415e120e36 into ad43bf0c56

This commit is contained in:
JosephFarrell 2026-08-15 11:31:49 -05:00 committed by GitHub
parent ad43bf0c56 415e120e36
commit e92e082d1b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 7 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
docs/topics/settings.rst
View File

@ -587,6 +587,11 @@ Default: ``16``
The maximum number of concurrent (i.e. simultaneous) requests that will be
performed by the Scrapy downloader. Use ``0`` for no limit.
High values may exceed the limit that your operating system sets on the number
of open file descriptors per process, causing errors. Some systems allow that
limit to be raised or removed, e.g. with the `ulimit`_ shell built-in or, for
services, with systemd’s `LimitNOFILE`_; otherwise, lower this setting.
.. setting:: CONCURRENT_REQUESTS_PER_DOMAIN
CONCURRENT_REQUESTS_PER_DOMAIN
@ -2415,3 +2420,5 @@ case to see how to enable and use them.
.. _Amazon web services: https://aws.amazon.com/
.. _Google Cloud Storage: https://cloud.google.com/storage/
.. _LimitNOFILE: https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#LimitNOFILE=
.. _ulimit: https://www.gnu.org/software/bash/manual/bash.html#index-ulimit