Remove docs mentioning deprecated spider attrs. (#7176)

This commit is contained in:
Andrey Rakhmatullin 2025-12-15 13:51:56 +05:00 committed by GitHub
parent d8583a89c7
commit 588f3d4f65
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 19 additions and 22 deletions

View File

@ -163,8 +163,8 @@ information on which commands must be run from inside projects, and which not.
Also keep in mind that some commands may have slightly different behaviours Also keep in mind that some commands may have slightly different behaviours
when running them from inside projects. For example, the fetch command will use when running them from inside projects. For example, the fetch command will use
spider-overridden behaviours (such as the ``user_agent`` attribute to override spider-overridden behaviours (such as the ``custom_settings`` attribute to
the user-agent) if the url being fetched is associated with some specific override settings) if the url being fetched is associated with some specific
spider. This is intentional, as the ``fetch`` command is meant to be used to spider. This is intentional, as the ``fetch`` command is meant to be used to
check how spiders are downloading pages. check how spiders are downloading pages.

View File

@ -290,13 +290,12 @@ DownloadTimeoutMiddleware
.. class:: DownloadTimeoutMiddleware .. class:: DownloadTimeoutMiddleware
This middleware sets the download timeout for requests specified in the This middleware sets the download timeout for requests specified in the
:setting:`DOWNLOAD_TIMEOUT` setting or :attr:`download_timeout` :setting:`DOWNLOAD_TIMEOUT` setting.
spider attribute.
.. note:: .. note::
You can also set download timeout per-request using You can also set download timeout per-request using the
:reqmeta:`download_timeout` Request.meta key; this is supported :reqmeta:`download_timeout` :attr:`.Request.meta` key; this is supported
even when DownloadTimeoutMiddleware is disabled. even when DownloadTimeoutMiddleware is disabled.
HttpAuthMiddleware HttpAuthMiddleware
@ -1212,9 +1211,8 @@ UserAgentMiddleware
.. class:: UserAgentMiddleware .. class:: UserAgentMiddleware
Middleware that allows spiders to override the default user agent. Middleware that sets the ``User-Agent`` header.
In order for a spider to override the default user agent, its ``user_agent`` The header value is taken from the :setting:`USER_AGENT` setting.
attribute must be set.
.. _DBM: https://en.wikipedia.org/wiki/Dbm .. _DBM: https://en.wikipedia.org/wiki/Dbm

View File

@ -1007,9 +1007,8 @@ The amount of time (in secs) that the downloader will wait before timing out.
.. note:: .. note::
This timeout can be set per spider using :attr:`download_timeout` This timeout can be per-request using the :reqmeta:`download_timeout`
spider attribute and per-request using :reqmeta:`download_timeout` :attr:`.Request.meta` key.
Request.meta key.
.. setting:: DOWNLOAD_MAXSIZE .. setting:: DOWNLOAD_MAXSIZE
.. reqmeta:: download_maxsize .. reqmeta:: download_maxsize
@ -1028,9 +1027,10 @@ ignored.
Use ``0`` to disable this limit. Use ``0`` to disable this limit.
This limit can be set per spider using the :attr:`download_maxsize` spider .. note::
attribute and per request using the :reqmeta:`download_maxsize` Request.meta
key. This limit can be set per-request using the :reqmeta:`download_maxsize`
:attr:`.Request.meta` key.
.. setting:: DOWNLOAD_WARNSIZE .. setting:: DOWNLOAD_WARNSIZE
.. reqmeta:: download_warnsize .. reqmeta:: download_warnsize
@ -1045,9 +1045,10 @@ warning will be logged about it.
Use ``0`` to disable this limit. Use ``0`` to disable this limit.
This limit can be set per spider using the :attr:`download_warnsize` spider .. note::
attribute and per request using the :reqmeta:`download_warnsize` Request.meta
key. This limit can be set per-request using the :reqmeta:`download_warnsize`
:attr:`.Request.meta` key.
.. setting:: DOWNLOAD_FAIL_ON_DATALOSS .. setting:: DOWNLOAD_FAIL_ON_DATALOSS

View File

@ -355,11 +355,9 @@ Otherwise, you would cause iteration over a ``start_urls`` string
resulting in each character being seen as a separate url. resulting in each character being seen as a separate url.
A valid use case is to set the http auth credentials A valid use case is to set the http auth credentials
used by :class:`~scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware` used by :class:`~scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware`::
or the user agent
used by :class:`~scrapy.downloadermiddlewares.useragent.UserAgentMiddleware`::
scrapy crawl myspider -a http_user=myuser -a http_pass=mypassword -a user_agent=mybot scrapy crawl myspider -a http_user=myuser -a http_pass=mypassword
Spider arguments can also be passed through the Scrapyd ``schedule.json`` API. Spider arguments can also be passed through the Scrapyd ``schedule.json`` API.
See `Scrapyd documentation`_. See `Scrapyd documentation`_.