From 9381ad893dfea0e8dac73fc78cb609b6017bde9a Mon Sep 17 00:00:00 2001 From: Adham 3llam <102502789+Ad7amstein@users.noreply.github.com> Date: Mon, 22 Dec 2025 11:00:39 +0200 Subject: [PATCH] Remove outdated version marker directives (< 2.8.0) from documentation (#7198) --- docs/topics/asyncio.rst | 2 -- docs/topics/commands.rst | 5 ----- docs/topics/coroutines.rst | 12 ----------- docs/topics/downloader-middleware.rst | 4 ---- docs/topics/exceptions.rst | 2 -- docs/topics/exporters.rst | 3 --- docs/topics/feed-exports.rst | 26 ---------------------- docs/topics/items.rst | 4 ---- docs/topics/link-extractors.rst | 4 ---- docs/topics/loaders.rst | 3 --- docs/topics/media-pipeline.rst | 10 --------- docs/topics/request-response.rst | 31 --------------------------- docs/topics/settings.rst | 13 ----------- docs/topics/signals.rst | 6 ------ docs/topics/spider-middleware.rst | 6 ------ docs/topics/spiders.rst | 3 --- 16 files changed, 134 deletions(-) diff --git a/docs/topics/asyncio.rst b/docs/topics/asyncio.rst index ca5bc9a5c..36f9a1a53 100644 --- a/docs/topics/asyncio.rst +++ b/docs/topics/asyncio.rst @@ -4,8 +4,6 @@ asyncio ======= -.. versionadded:: 2.0 - Scrapy has partial support for :mod:`asyncio`. After you :ref:`install the asyncio reactor `, you may use :mod:`asyncio` and :mod:`asyncio`-powered libraries in any :doc:`coroutine `. diff --git a/docs/topics/commands.rst b/docs/topics/commands.rst index b8bf41402..8d1351eb9 100644 --- a/docs/topics/commands.rst +++ b/docs/topics/commands.rst @@ -233,9 +233,6 @@ genspider * Syntax: ``scrapy genspider [-t template] `` * Requires project: *no* -.. versionadded:: 2.6.0 - The ability to pass a URL instead of a domain. - Creates a new spider in the current folder or in the current project's ``spiders`` folder, if called from inside a project. The ```` parameter is set as the spider's ``name``, while ```` is used to generate the ``allowed_domains`` and ``start_urls`` spider's attributes. Usage example:: @@ -509,8 +506,6 @@ Supported options: * ``--output`` or ``-o``: dump scraped items to a file - .. versionadded:: 2.3 - .. skip: start Usage example:: diff --git a/docs/topics/coroutines.rst b/docs/topics/coroutines.rst index ed2b25252..9c324a031 100644 --- a/docs/topics/coroutines.rst +++ b/docs/topics/coroutines.rst @@ -4,8 +4,6 @@ Coroutines ========== -.. versionadded:: 2.0 - Scrapy :ref:`supports ` the :ref:`coroutine syntax ` (i.e. ``async def``). @@ -28,10 +26,6 @@ hence use coroutine syntax (e.g. ``await``, ``async for``, ``async with``): If you are using any custom or third-party :ref:`spider middleware `, see :ref:`sync-async-spider-middleware`. - .. versionchanged:: 2.7 - Output of async callbacks is now processed asynchronously instead of - collecting all of it first. - - The :meth:`process_item` method of :ref:`item pipelines `. @@ -53,8 +47,6 @@ hence use coroutine syntax (e.g. ``await``, ``async for``, ``async with``): See also :ref:`sync-async-spider-middleware` and :ref:`universal-spider-middleware`. - .. versionadded:: 2.7 - - The :meth:`~scrapy.spidermiddlewares.SpiderMiddleware.process_start` method of :ref:`spider middlewares `, which *must* be defined as an :term:`asynchronous generator`. @@ -307,8 +299,6 @@ You can also send multiple requests in parallel: Mixing synchronous and asynchronous spider middlewares ====================================================== -.. versionadded:: 2.7 - The output of a :class:`~scrapy.Request` callback is passed as the ``result`` parameter to the :meth:`~scrapy.spidermiddlewares.SpiderMiddleware.process_spider_output` method @@ -396,8 +386,6 @@ option. Otherwise, it's better to choose the second option. Universal spider middlewares ---------------------------- -.. versionadded:: 2.7 - To allow writing a spider middleware that supports asynchronous execution of its ``process_spider_output`` method in Scrapy 2.7 and later (avoiding :ref:`asynchronous-to-synchronous conversions `) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 44c7ac17b..c78045c4b 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -917,10 +917,6 @@ Default: ``[]`` Meta tags within these tags are ignored. -.. versionchanged:: 2.0 - The default value of :setting:`METAREFRESH_IGNORE_TAGS` changed from - ``["script", "noscript"]`` to ``[]``. - .. versionchanged:: 2.11.2 The default value of :setting:`METAREFRESH_IGNORE_TAGS` changed from ``[]`` to ``["noscript"]``. diff --git a/docs/topics/exceptions.rst b/docs/topics/exceptions.rst index 0b572ff95..cc690d609 100644 --- a/docs/topics/exceptions.rst +++ b/docs/topics/exceptions.rst @@ -83,8 +83,6 @@ This exception is raised to indicate an unsupported feature. StopDownload ------------- -.. versionadded:: 2.2 - .. exception:: StopDownload(fail=True) Raised from a :class:`~scrapy.signals.bytes_received` or :class:`~scrapy.signals.headers_received` diff --git a/docs/topics/exporters.rst b/docs/topics/exporters.rst index d3186ef5c..74256eef4 100644 --- a/docs/topics/exporters.rst +++ b/docs/topics/exporters.rst @@ -163,9 +163,6 @@ BaseItemExporter populate their respective instance attributes: :attr:`fields_to_export`, :attr:`export_empty_fields`, :attr:`encoding`, :attr:`indent`. - .. versionadded:: 2.0 - The *dont_fail* parameter. - .. method:: export_item(item) Exports the given item. This method must be implemented in subclasses. diff --git a/docs/topics/feed-exports.rst b/docs/topics/feed-exports.rst index 2184f2d0e..a5537c39e 100644 --- a/docs/topics/feed-exports.rst +++ b/docs/topics/feed-exports.rst @@ -236,8 +236,6 @@ This storage backend uses :ref:`delayed file delivery `. Google Cloud Storage (GCS) -------------------------- -.. versionadded:: 2.3 - The feeds are stored on `Google Cloud Storage`_. - URI scheme: ``gs`` @@ -303,8 +301,6 @@ feed URI, allowing item delivery to start way before the end of the crawl. Item filtering ============== -.. versionadded:: 2.6.0 - You can filter items that you want to allow for a particular feed by using the ``item_classes`` option in :ref:`feeds options `. Only items of the specified types will be added to the feed. @@ -344,8 +340,6 @@ ItemFilter Post-Processing =============== -.. versionadded:: 2.6.0 - Scrapy provides an option to activate plugins to post-process feeds before they are exported to feed storages. In addition to using :ref:`builtin plugins `, you can create your own :ref:`plugins `. @@ -425,8 +419,6 @@ These are the settings used for configuring the feed exports: FEEDS ----- -.. versionadded:: 2.1 - Default: ``{}`` A dictionary in which every key is a feed URI (or a :class:`pathlib.Path` @@ -479,8 +471,6 @@ as a fallback value if that key is not provided for a specific feed definition: - ``batch_item_count``: falls back to :setting:`FEED_EXPORT_BATCH_ITEM_COUNT`. - .. versionadded:: 2.3.0 - - ``encoding``: falls back to :setting:`FEED_EXPORT_ENCODING`. - ``fields``: falls back to :setting:`FEED_EXPORT_FIELDS`. @@ -489,20 +479,14 @@ as a fallback value if that key is not provided for a specific feed definition: If undefined or empty, all items are exported. - .. versionadded:: 2.6.0 - - ``item_filter``: a :ref:`filter class ` to filter items to export. :class:`~scrapy.extensions.feedexport.ItemFilter` is used be default. - .. versionadded:: 2.6.0 - - ``indent``: falls back to :setting:`FEED_EXPORT_INDENT`. - ``item_export_kwargs``: :class:`dict` with keyword arguments for the corresponding :ref:`item exporter class `. - .. versionadded:: 2.4.0 - - ``overwrite``: whether to overwrite the file if it already exists (``True``) or append to its content (``False``). @@ -522,8 +506,6 @@ as a fallback value if that key is not provided for a specific feed definition: - :ref:`topics-feed-storage-stdout`: ``False`` (overwriting is not supported) - .. versionadded:: 2.4.0 - - ``store_empty``: falls back to :setting:`FEED_STORE_EMPTY`. - ``uri_params``: falls back to :setting:`FEED_URI_PARAMS`. @@ -532,8 +514,6 @@ as a fallback value if that key is not provided for a specific feed definition: The plugins will be used in the order of the list passed. - .. versionadded:: 2.6.0 - .. setting:: FEED_EXPORT_ENCODING FEED_EXPORT_ENCODING @@ -700,8 +680,6 @@ format in :setting:`FEED_EXPORTERS`. E.g., to disable the built-in CSV exporter FEED_EXPORT_BATCH_ITEM_COUNT ---------------------------- -.. versionadded:: 2.3.0 - Default: ``0`` If assigned an integer number higher than ``0``, Scrapy generates multiple output files @@ -771,15 +749,11 @@ The function signature should be as follows: If :setting:`FEED_EXPORT_BATCH_ITEM_COUNT` is ``0``, ``batch_id`` is always ``1``. - .. versionadded:: 2.3.0 - - ``batch_time``: UTC date and time, in ISO format with ``:`` replaced with ``-``. See :setting:`FEED_EXPORT_BATCH_ITEM_COUNT`. - .. versionadded:: 2.3.0 - - ``time``: ``batch_time``, with microseconds set to ``0``. :type params: dict diff --git a/docs/topics/items.rst b/docs/topics/items.rst index 3588d033e..f05746934 100644 --- a/docs/topics/items.rst +++ b/docs/topics/items.rst @@ -80,8 +80,6 @@ Example: Dataclass objects ----------------- -.. versionadded:: 2.2 - :func:`~dataclasses.dataclass` allows the defining of item classes with field names, so that :ref:`item exporters ` can export all fields by default even if the first scraped object does not have values for all of them. @@ -112,8 +110,6 @@ Example: attr.s objects -------------- -.. versionadded:: 2.2 - :func:`attr.s` allows the defining of item classes with field names, so that :ref:`item exporters ` can export all fields by default even if the first scraped object does not have values for all of them. diff --git a/docs/topics/link-extractors.rst b/docs/topics/link-extractors.rst index f9744ed16..7e3517937 100644 --- a/docs/topics/link-extractors.rst +++ b/docs/topics/link-extractors.rst @@ -76,10 +76,6 @@ LxmlLinkExtractor If not given, it will default to :data:`scrapy.linkextractors.IGNORED_EXTENSIONS`. - .. versionchanged:: 2.0 - :data:`~scrapy.linkextractors.IGNORED_EXTENSIONS` now includes - ``7z``, ``7zip``, ``apk``, ``bz2``, ``cdr``, ``dmg``, ``ico``, - ``iso``, ``tar``, ``tar.gz``, ``webm``, and ``xz``. :type deny_extensions: list :param restrict_xpaths: is an XPath (or list of XPath's) which defines diff --git a/docs/topics/loaders.rst b/docs/topics/loaders.rst index b7ce33616..a43edb28c 100644 --- a/docs/topics/loaders.rst +++ b/docs/topics/loaders.rst @@ -174,9 +174,6 @@ with the data to be parsed, and return a parsed value. So you can use any function as input or output processor. The only requirement is that they must accept one (and only one) positional argument, which will be an iterable. -.. versionchanged:: 2.0 - Processors no longer need to be methods. - .. note:: Both input and output processors must receive an iterable as their first argument. The output of those functions can be anything. The result of input processors will be appended to an internal list (in the Loader) diff --git a/docs/topics/media-pipeline.rst b/docs/topics/media-pipeline.rst index fa17dfda6..8636e29c9 100644 --- a/docs/topics/media-pipeline.rst +++ b/docs/topics/media-pipeline.rst @@ -212,8 +212,6 @@ Where: FTP server storage ------------------ -.. versionadded:: 2.0 - :setting:`FILES_STORE` and :setting:`IMAGES_STORE` can point to an FTP server. Scrapy will automatically upload the files to the server. @@ -547,9 +545,6 @@ See here the methods that you can override in your custom Files Pipeline: By default the :meth:`file_path` method returns ``full/.``. - .. versionadded:: 2.4 - The *item* parameter. - .. method:: FilesPipeline.get_media_requests(item, info) As seen on the workflow, the pipeline will get the URLs of the images to @@ -590,8 +585,6 @@ See here the methods that you can override in your custom Files Pipeline: * ``status`` - the file status indication. - .. versionadded:: 2.2 - It can be one of the following: * ``downloaded`` - file was downloaded. @@ -705,9 +698,6 @@ See here the methods that you can override in your custom Images Pipeline: By default the :meth:`file_path` method returns ``full/.``. - .. versionadded:: 2.4 - The *item* parameter. - .. method:: ImagesPipeline.thumb_path(self, request, thumb_id, response=None, info=None, *, item=None) This method is called for every item of :setting:`IMAGES_THUMBS` per downloaded item. It returns the diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rst index 8a907e377..cb2410207 100644 --- a/docs/topics/request-response.rst +++ b/docs/topics/request-response.rst @@ -32,10 +32,6 @@ Request objects :type url: str :param callback: sets :attr:`callback`, defaults to ``None``. - - .. versionchanged:: 2.0 - The *callback* parameter is no longer required when the *errback* - parameter is specified. :type callback: Callable[Concatenate[Response, ...], Any] | None :param method: the HTTP method of this request. Defaults to ``'GET'``. @@ -116,10 +112,6 @@ Request objects :class:`scrapy.Request.cookies ` parameter. This is a known current limitation that is being worked on. - .. versionadded:: 2.6.0 - Cookie values that are :class:`bool`, :class:`float` or :class:`int` - are casted to :class:`str`. - :type cookies: dict or list :param encoding: the encoding of this request (defaults to ``'utf-8'``). @@ -134,10 +126,6 @@ Request objects :type dont_filter: bool :param errback: sets :attr:`errback`, defaults to ``None``. - - .. versionchanged:: 2.0 - The *callback* parameter is no longer required when the *errback* - parameter is specified. :type errback: Callable[[Failure], Any] | None :param flags: Flags sent to the request, can be used for logging or similar purposes. @@ -448,8 +436,6 @@ To change how request fingerprints are built for your requests, use the REQUEST_FINGERPRINTER_CLASS ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. versionadded:: 2.7 - Default: :class:`scrapy.utils.request.RequestFingerprinter` A :ref:`request fingerprinter class ` or its @@ -988,15 +974,6 @@ Response objects For instance: "HTTP/1.0", "HTTP/1.1", "h2" :type protocol: :class:`str` - .. versionadded:: 2.0.0 - The ``certificate`` parameter. - - .. versionadded:: 2.1.0 - The ``ip_address`` parameter. - - .. versionadded:: 2.5.0 - The ``protocol`` parameter. - .. attribute:: Response.url A string containing the URL of the response. @@ -1062,8 +1039,6 @@ Response objects .. attribute:: Response.cb_kwargs - .. versionadded:: 2.0 - A shortcut to the :attr:`~scrapy.Request.cb_kwargs` attribute of the :attr:`Response.request` object (i.e. ``self.request.cb_kwargs``). @@ -1082,8 +1057,6 @@ Response objects .. attribute:: Response.certificate - .. versionadded:: 2.0.0 - A :class:`twisted.internet.ssl.Certificate` object representing the server's SSL certificate. @@ -1091,8 +1064,6 @@ Response objects .. attribute:: Response.ip_address - .. versionadded:: 2.1.0 - The IP address of the server from which the Response originated. This attribute is currently only populated by the HTTP 1.1 download @@ -1101,8 +1072,6 @@ Response objects .. attribute:: Response.protocol - .. versionadded:: 2.5.0 - The protocol that was used to download the response. For instance: "HTTP/1.0", "HTTP/1.1" diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index 64e5af250..27a8764f3 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -182,8 +182,6 @@ Setting values must be :ref:`picklable `. Import paths and classes ======================== -.. versionadded:: 2.4.0 - When a setting references a callable object to be imported by Scrapy, such as a class or a function, there are two different ways you can specify that object: @@ -667,8 +665,6 @@ DNS in-memory cache size. DNS_RESOLVER ------------ -.. versionadded:: 2.0 - Default: ``'scrapy.resolver.CachingThreadedResolver'`` The class to be used to resolve DNS names. The default ``scrapy.resolver.CachingThreadedResolver`` @@ -1798,8 +1794,6 @@ For available choices, see :setting:`SCHEDULER_MEMORY_QUEUE`. SCRAPER_SLOT_MAX_ACTIVE_SIZE ---------------------------- -.. versionadded:: 2.0 - Default: ``5_000_000`` Soft limit (in bytes) for response data being processed. @@ -1974,8 +1968,6 @@ in the ``project`` subdirectory. TWISTED_REACTOR --------------- -.. versionadded:: 2.0 - Default: ``"twisted.internet.asyncioreactor.AsyncioSelectorReactor"`` Import path of a given :mod:`~twisted.internet.reactor`. @@ -2068,11 +2060,6 @@ If this setting is set ``None``, Scrapy will use the existing reactor if one is already installed, or install the default reactor defined by Twisted for the current platform. -.. versionchanged:: 2.7 - The :command:`startproject` command now sets this setting to - ``twisted.internet.asyncioreactor.AsyncioSelectorReactor`` in the generated - ``settings.py`` file. - .. versionchanged:: 2.13 The default value was changed from ``None`` to ``"twisted.internet.asyncioreactor.AsyncioSelectorReactor"``. diff --git a/docs/topics/signals.rst b/docs/topics/signals.rst index aa27e62dd..efe3bb8fc 100644 --- a/docs/topics/signals.rst +++ b/docs/topics/signals.rst @@ -423,8 +423,6 @@ request_left_downloader .. signal:: request_left_downloader .. function:: request_left_downloader(request, spider) - .. versionadded:: 2.0 - Sent when a :class:`~scrapy.Request` leaves the downloader, even in case of failure. @@ -442,8 +440,6 @@ bytes_received .. signal:: bytes_received .. function:: bytes_received(data, request, spider) - .. versionadded:: 2.2 - Sent by the HTTP 1.1 and S3 download handlers when a group of bytes is received for a specific request. This signal might be fired multiple times for the same request, with partial data each time. For instance, @@ -472,8 +468,6 @@ headers_received .. signal:: headers_received .. function:: headers_received(headers, body_length, request, spider) - .. versionadded:: 2.5 - Sent by the HTTP 1.1 and S3 download handlers when the response headers are available for a given request, before downloading any additional content. diff --git a/docs/topics/spider-middleware.rst b/docs/topics/spider-middleware.rst index 0f81e409e..799d2f99c 100644 --- a/docs/topics/spider-middleware.rst +++ b/docs/topics/spider-middleware.rst @@ -125,10 +125,6 @@ one or more of these methods: :class:`~scrapy.Request` objects and :ref:`item objects `. - .. versionchanged:: 2.7 - This method may be defined as an :term:`asynchronous generator`, in - which case ``result`` is an :term:`asynchronous iterable`. - Consider defining this method as an :term:`asynchronous generator`, which will be a requirement in a future version of Scrapy. However, if you plan on sharing your spider middleware with other people, consider @@ -148,8 +144,6 @@ one or more of these methods: .. method:: process_spider_output_async(response, result) :async: - .. versionadded:: 2.7 - If defined, this method must be an :term:`asynchronous generator`, which will be called instead of :meth:`process_spider_output` if ``result`` is an :term:`asynchronous iterable`. diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst index b4dfb3757..699b141c4 100644 --- a/docs/topics/spiders.rst +++ b/docs/topics/spiders.rst @@ -547,9 +547,6 @@ Crawling rules callbacks for new requests when writing :class:`CrawlSpider`-based spiders; unexpected behaviour can occur otherwise. - .. versionadded:: 2.0 - The *errback* parameter. - CrawlSpider example ~~~~~~~~~~~~~~~~~~~