Remove docs mentioning deprecated spider attrs. (#7176)

This commit is contained in:
Andrey Rakhmatullin 2025-12-15 13:51:56 +05:00 committed by GitHub
parent d8583a89c7
commit 588f3d4f65
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 19 additions and 22 deletions

View File

@ -163,8 +163,8 @@ information on which commands must be run from inside projects, and which not.
Also keep in mind that some commands may have slightly different behaviours
when running them from inside projects. For example, the fetch command will use
spider-overridden behaviours (such as the ``user_agent`` attribute to override
the user-agent) if the url being fetched is associated with some specific
spider-overridden behaviours (such as the ``custom_settings`` attribute to
override settings) if the url being fetched is associated with some specific
spider. This is intentional, as the ``fetch`` command is meant to be used to
check how spiders are downloading pages.

View File

@ -290,13 +290,12 @@ DownloadTimeoutMiddleware
.. class:: DownloadTimeoutMiddleware
This middleware sets the download timeout for requests specified in the
:setting:`DOWNLOAD_TIMEOUT` setting or :attr:`download_timeout`
spider attribute.
:setting:`DOWNLOAD_TIMEOUT` setting.
.. note::
You can also set download timeout per-request using
:reqmeta:`download_timeout` Request.meta key; this is supported
You can also set download timeout per-request using the
:reqmeta:`download_timeout` :attr:`.Request.meta` key; this is supported
even when DownloadTimeoutMiddleware is disabled.
HttpAuthMiddleware
@ -1212,9 +1211,8 @@ UserAgentMiddleware
.. class:: UserAgentMiddleware
Middleware that allows spiders to override the default user agent.
Middleware that sets the ``User-Agent`` header.
In order for a spider to override the default user agent, its ``user_agent``
attribute must be set.
The header value is taken from the :setting:`USER_AGENT` setting.
.. _DBM: https://en.wikipedia.org/wiki/Dbm

View File

@ -1007,9 +1007,8 @@ The amount of time (in secs) that the downloader will wait before timing out.
.. note::
This timeout can be set per spider using :attr:`download_timeout`
spider attribute and per-request using :reqmeta:`download_timeout`
Request.meta key.
This timeout can be per-request using the :reqmeta:`download_timeout`
:attr:`.Request.meta` key.
.. setting:: DOWNLOAD_MAXSIZE
.. reqmeta:: download_maxsize
@ -1028,9 +1027,10 @@ ignored.
Use ``0`` to disable this limit.
This limit can be set per spider using the :attr:`download_maxsize` spider
attribute and per request using the :reqmeta:`download_maxsize` Request.meta
key.
.. note::
This limit can be set per-request using the :reqmeta:`download_maxsize`
:attr:`.Request.meta` key.
.. setting:: DOWNLOAD_WARNSIZE
.. reqmeta:: download_warnsize
@ -1045,9 +1045,10 @@ warning will be logged about it.
Use ``0`` to disable this limit.
This limit can be set per spider using the :attr:`download_warnsize` spider
attribute and per request using the :reqmeta:`download_warnsize` Request.meta
key.
.. note::
This limit can be set per-request using the :reqmeta:`download_warnsize`
:attr:`.Request.meta` key.
.. setting:: DOWNLOAD_FAIL_ON_DATALOSS

View File

@ -355,11 +355,9 @@ Otherwise, you would cause iteration over a ``start_urls`` string
resulting in each character being seen as a separate url.
A valid use case is to set the http auth credentials
used by :class:`~scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware`
or the user agent
used by :class:`~scrapy.downloadermiddlewares.useragent.UserAgentMiddleware`::
used by :class:`~scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware`::
scrapy crawl myspider -a http_user=myuser -a http_pass=mypassword -a user_agent=mybot
scrapy crawl myspider -a http_user=myuser -a http_pass=mypassword
Spider arguments can also be passed through the Scrapyd ``schedule.json`` API.
See `Scrapyd documentation`_.