RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update the docs

This commit is contained in:
Adrián Chaves 2023-11-23 09:11:14 +01:00
parent e0b66c021a
commit 1087bb7b2e
2 changed files with 20 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/topics/request-response.rst
View File

@ -702,6 +702,7 @@ Those are:
* :reqmeta:`download_fail_on_dataloss`
* :reqmeta:`download_latency`
* :reqmeta:`download_maxsize`
* :reqmeta:`download_warnsize`
* :reqmeta:`download_timeout`
* ``ftp_password`` (See :setting:`FTP_PASSWORD` for more info)
* ``ftp_user`` (See :setting:`FTP_USER` for more info)

36
docs/topics/settings.rst
View File

@ -873,40 +873,42 @@ The amount of time (in secs) that the downloader will wait before timing out.
Request.meta key.
.. setting:: DOWNLOAD_MAXSIZE
.. reqmeta:: download_maxsize
DOWNLOAD_MAXSIZE
----------------
Default: ``1073741824`` (1024MB)
Default: ``1073741824`` (1 GiB)
The maximum response size (in bytes) that downloader will download.
The maximum response body size (in bytes) allowed. Bigger responses are
aborted and ignored.
If you want to disable it set to 0.
This applies both before and after compression. If decompressing a response
body would exceed this limit, decompression is aborted and the response is
ignored.
.. reqmeta:: download_maxsize
Use ``0`` to disable this limit.
.. note::
This size can be set per spider using :attr:`download_maxsize`
spider attribute and per-request using :reqmeta:`download_maxsize`
Request.meta key.
This limit can be set per spider using the :attr:`download_maxsize` spider
attribute and per request using the :reqmeta:`download_maxsize` Request.meta
key.
.. setting:: DOWNLOAD_WARNSIZE
.. reqmeta:: download_warnsize
DOWNLOAD_WARNSIZE
-----------------
Default: ``33554432`` (32MB)
Default: ``33554432`` (32 MiB)
The response size (in bytes) that downloader will start to warn.
If the size of a response exceeds this value, before or after compression, a
warning will be logged about it.
If you want to disable it set to 0.
Use ``0`` to disable this limit.
.. note::
This size can be set per spider using :attr:`download_warnsize`
spider attribute and per-request using :reqmeta:`download_warnsize`
Request.meta key.
This limit can be set per spider using the :attr:`download_warnsize` spider
attribute and per request using the :reqmeta:`download_warnsize` Request.meta
key.
.. setting:: DOWNLOAD_FAIL_ON_DATALOSS