Assorted docs fixes, part 2. (#7725)

* Assorted docs fixes, part 2.

* Second pass.

* Address feedback.

docs/faq.rst

@@ -384,8 +384,9 @@ How to deal with ``<class 'ValueError'>: filedescriptor out of range in select()
 ----------------------------------------------------------------------------------------------
 
 This issue `has been reported`_ to appear when running broad crawls in macOS, where the default
-Twisted reactor is :class:`twisted.internet.selectreactor.SelectReactor`. Switching to a
+Twisted reactor was :class:`twisted.internet.selectreactor.SelectReactor` at that time.
-different reactor is possible by using the :setting:`TWISTED_REACTOR` setting.
+If you have switched to this reactor using the :setting:`TWISTED_REACTOR` setting you can switch
+to a different one in the same way.
 
 
 .. _faq-stop-response-download:

docs/intro/install.rst

@@ -230,8 +230,8 @@ Installing Scrapy with PyPy on Windows is not tested.
 You can check that Scrapy is installed correctly by running ``scrapy bench``.
 If this command gives errors such as
 ``TypeError: ... got 2 unexpected keyword arguments``, this means
-that setuptools was unable to pick up one PyPy-specific dependency.
+that the ``PyPyDispatcher`` dependency wasn't installed. To fix this issue, run
-To fix this issue, run ``pip install 'PyPyDispatcher>=2.1.0'``.
+``pip install 'PyPyDispatcher>=2.1.0'``.
 
 
 .. _intro-install-troubleshooting:

docs/intro/overview.rst

@@ -83,7 +83,7 @@ While this enables you to do very fast crawls (sending multiple concurrent
 requests at the same time, in a fault-tolerant way) Scrapy also gives you
 control over the politeness of the crawl through :ref:`a few settings
 <topics-settings-ref>`. You can do things like setting a download delay between
-each request, limiting the amount of concurrent requests per domain or per IP, and
+each request, limiting the amount of concurrent requests per domain, and
 even :ref:`using an auto-throttling extension <topics-autothrottle>` that tries
 to figure these settings out automatically.
 

docs/topics/asyncio.rst

@@ -5,7 +5,7 @@ asyncio
 =======
 
 Scrapy supports :mod:`asyncio` natively. New projects created with
-:command:`scrapy startproject` have asyncio enabled by default, and you can use
+:command:`startproject` have asyncio enabled by default, and you can use
 :mod:`asyncio` and :mod:`asyncio`-powered libraries in any :doc:`coroutine
 <coroutines>`.
 
@@ -18,7 +18,7 @@ no additional setup is needed.
 Configuring the asyncio reactor
 ===============================
 
-New projects generated with :command:`scrapy startproject` have the asyncio
+New projects generated with :command:`startproject` have the asyncio
 reactor configured by default. No manual setup is needed.
 
 The :setting:`TWISTED_REACTOR` setting controls which Twisted reactor Scrapy

docs/topics/autothrottle.rst

@@ -75,7 +75,7 @@ AutoThrottle algorithm adjusts download delays based on the following rules:
 .. _download-latency:
 
 In Scrapy, the download latency is measured as the time elapsed between
-establishing the TCP connection and receiving the HTTP headers.
+sending the request and receiving the HTTP headers.
 
 Note that these latencies are very hard to measure accurately in a cooperative
 multitasking environment because Scrapy may be busy processing a spider

docs/topics/commands.rst

@@ -309,11 +309,25 @@ Usage examples::
       * parse_item
 
     $ scrapy check
-    [FAILED] first_spider:parse_item
+    F.F.
-    >>> 'RetailPricex' field is missing
+    ======================================================================
+    FAIL: [first_spider] parse (@returns post-hook)
+    ----------------------------------------------------------------------
+    Traceback (most recent call last):
+      ...
+    scrapy.exceptions.ContractFail: Returned 92 requests, expected 0..4
 
-    [FAILED] first_spider:parse
+    ======================================================================
-    >>> Returned 92 requests, expected 0..4
+    FAIL: [first_spider] parse_item (@scrapes post-hook)
+    ----------------------------------------------------------------------
+    Traceback (most recent call last):
+      ...
+    scrapy.exceptions.ContractFail: Missing fields: RetailPricex
+
+    ----------------------------------------------------------------------
+    Ran 4 contracts in 0.174s
+
+    FAILED (failures=2)
 
 .. skip: end
 
@@ -377,7 +391,7 @@ Supported options:
 
 * ``--spider=SPIDER``: bypass spider autodetection and force use of specific spider
 
-* ``--headers``: print the response's HTTP headers instead of the response's body
+* ``--headers``: print the request's and response's HTTP headers instead of the response's body
 
 * ``--no-redirect``: do not follow HTTP 3xx redirects (default is to follow them)
 
@@ -387,15 +401,19 @@ Usage examples::
     [ ... html content here ... ]
 
     $ scrapy fetch --nolog --headers http://www.example.com/
-    {'Accept-Ranges': ['bytes'],
+    > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
-     'Age': ['1263   '],
+    > Accept-Language: en
-     'Connection': ['close     '],
+    > User-Agent: Scrapy/2.16.0 (+https://scrapy.org)
-     'Content-Length': ['596'],
+    > Accept-Encoding: gzip, deflate, br
-     'Content-Type': ['text/html; charset=UTF-8'],
+    >
-     'Date': ['Wed, 18 Aug 2010 23:59:46 GMT'],
+    < Date: Wed, 08 Jul 2026 06:15:01 GMT
-     'Etag': ['"573c1-254-48c9c87349680"'],
+    < Content-Type: text/html
-     'Last-Modified': ['Fri, 30 Jul 2010 15:30:18 GMT'],
+    < Server: cloudflare
-     'Server': ['Apache/2.2.3 (CentOS)']}
+    < Last-Modified: Wed, 01 Jul 2026 17:50:18 GMT
+    < Allow: GET, HEAD
+    < Cf-Cache-Status: HIT
+    < Age: 8184
+    < Cf-Ray: a17cf3b80eddf141-DME
 
 .. command:: view
 
@@ -605,7 +623,10 @@ shouldn't matter to the user running the command, but when the user :ref:`needs
 a non-default Twisted reactor <disable-asyncio>`, it may be important.
 
 Scrapy decides which of these two classes to use based on the value of the
-:setting:`TWISTED_REACTOR` setting. If the setting value is the default one
+:setting:`TWISTED_REACTOR` and :setting:`TWISTED_REACTOR_ENABLED` settings.
+With :setting:`TWISTED_REACTOR_ENABLED` set to ``False`` it will use
+:class:`~scrapy.crawler.AsyncCrawlerProcess`. Otherwise, if the
+:setting:`TWISTED_REACTOR` value is the default one
 (``'twisted.internet.asyncioreactor.AsyncioSelectorReactor'``),
 :class:`~scrapy.crawler.AsyncCrawlerProcess` will be used, otherwise
 :class:`~scrapy.crawler.CrawlerProcess` will be used. The :ref:`spider settings

docs/topics/coroutines.rst

@@ -16,7 +16,7 @@ Supported callables
 The following callables may be defined as coroutines using ``async def``, and
 hence use coroutine syntax (e.g. ``await``, ``async for``, ``async with``):
 
--   The :meth:`~scrapy.spiders.Spider.start` spider method, which *must* be
+-   The :meth:`~scrapy.Spider.start` spider method, which *must* be
     defined as an :term:`asynchronous generator`.
 
     .. versionadded:: 2.13
@@ -204,13 +204,15 @@ This means you can use many useful Python libraries providing such code:
 Common use cases for asynchronous code include:
 
 * requesting data from websites, databases and other services (in
-  :meth:`~scrapy.spiders.Spider.start`, callbacks, pipelines and
+  :meth:`~scrapy.Spider.start`, callbacks, pipelines and
   middlewares);
 * storing data in databases (in pipelines and middlewares);
 * delaying the spider initialization until some external event (in the
   :signal:`spider_opened` handler);
-* calling asynchronous Scrapy methods like :meth:`ExecutionEngine.download`
+* calling asynchronous Scrapy methods like
-  (see :ref:`the screenshot pipeline example<ScreenshotPipeline>`).
+  :meth:`ExecutionEngine.download_async()
+  <scrapy.core.engine.ExecutionEngine.download_async>` (see :ref:`the
+  screenshot pipeline example <ScreenshotPipeline>`).
 
 .. _aio-libs: https://github.com/aio-libs
 

docs/topics/download-handlers.rst

@@ -89,7 +89,7 @@ the following API:
         If ``True``, the handler will only be instantiated when the first
         request handled by it needs to be downloaded.
 
-    .. method:: download_request(request: Request) -> Response:
+    .. method:: download_request(request: Request) -> Response
         :async:
 
         Download the given request and return a response.

docs/topics/downloader-middleware.rst

@@ -591,8 +591,8 @@ In order to use your storage backend, set:
 HTTPCache middleware settings
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The :class:`HttpCacheMiddleware` can be configured through the following
+:class:`~scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware` can be
-settings:
+configured through the following settings:
 
 .. setting:: HTTPCACHE_ENABLED
 
@@ -868,9 +868,9 @@ OffsiteMiddleware
    .. reqmeta:: allow_offsite
 
    If the request has the :attr:`~scrapy.Request.dont_filter` attribute set to
-   ``True`` or :attr:`Request.meta` has ``allow_offsite`` set to ``True``, then
+   ``True`` or :attr:`Request.meta <scrapy.Request.meta>` has ``allow_offsite``
-   the OffsiteMiddleware will allow the request even if its domain is not listed
+   set to ``True``, then the OffsiteMiddleware will allow the request even if
-   in allowed domains.
+   its domain is not listed in allowed domains.
 
 RedirectMiddleware
 ------------------
@@ -1015,17 +1015,6 @@ RetryMiddleware
    A middleware to retry failed requests that are potentially caused by
    temporary problems such as a connection timeout or HTTP 500 error.
 
-Failed pages are collected on the scraping process and rescheduled at the
-end, once the spider has finished crawling all regular (non failed) pages.
-
-The :class:`RetryMiddleware` can be configured through the following
-settings (see the settings documentation for more info):
-
-* :setting:`RETRY_ENABLED`
-* :setting:`RETRY_TIMES`
-* :setting:`RETRY_HTTP_CODES`
-* :setting:`RETRY_EXCEPTIONS`
-
 .. reqmeta:: dont_retry
 
 If :attr:`Request.meta <scrapy.Request.meta>` has ``dont_retry`` key

docs/topics/exceptions.rst

@@ -12,7 +12,8 @@ Exceptions
 Built-in Exceptions reference
 =============================
 
-Here's a list of all exceptions included in Scrapy and their usage.
+Here's a list of all exceptions included in Scrapy and their usage, except for
+the :ref:`download handler exceptions <download-handlers-exceptions>`.
 
 
 CloseSpider
@@ -71,7 +72,8 @@ remain disabled. Those components include:
 -   Downloader middlewares
 -   Spider middlewares
 
-The exception must be raised in the component's ``__init__`` method.
+The exception must be raised in the component's ``__init__()`` or
+``from_crawler()``  method.
 
 NotSupported
 ------------

docs/topics/feed-exports.rst

@@ -166,7 +166,7 @@ The feeds are stored in the local filesystem.
 -   Required external libraries: none
 
 Note that for the local filesystem storage (only) you can omit the scheme if
-you specify an absolute path like ``/tmp/export.csv`` (Unix systems only).
+you specify a path (e.g. ``/tmp/export.csv``).
 Alternatively you can also use a :class:`pathlib.Path` object.
 
 .. _topics-feed-storage-ftp:
@@ -762,8 +762,8 @@ The function signature should be as follows:
    :param spider: source spider of the feed items
    :type spider: scrapy.Spider
 
-   .. caution:: The function should return a new dictionary, modifying
+   .. caution:: The function must return a new dictionary instead of modifying
-                the received ``params`` in-place is deprecated.
+                the received ``params`` in-place.
 
 For example, to include the :attr:`name <scrapy.Spider.name>` of the
 source spider in the feed URI:

docs/topics/item-pipeline.rst

@@ -121,7 +121,7 @@ Write items to MongoDB
 
 In this example we'll write items to MongoDB_ using pymongo_.
 MongoDB address and database name are specified in Scrapy settings;
-MongoDB collection is named after item class.
+MongoDB collection is specified in a class attribute.
 
 The main point of this example is to show how to :ref:`get the crawler
 <from-crawler>` and how to clean up the resources properly.

docs/topics/items.rst

@@ -23,7 +23,8 @@ Item Types
 
 Scrapy supports the following types of items, via the `itemadapter`_ library:
 :ref:`dictionaries <dict-items>`, :ref:`Item objects <item-objects>`,
-:ref:`dataclass objects <dataclass-items>`, and :ref:`attrs objects <attrs-items>`.
+:ref:`dataclass objects <dataclass-items>`, :ref:`attrs objects <attrs-items>`
+and :ref:`Pydantic models <pydantic-items>`.
 
 .. _itemadapter: https://github.com/scrapy/itemadapter
 
@@ -61,8 +62,8 @@ its ``__init__`` method.
 :class:`Item` also allows the defining of field metadata, which can be used to
 :ref:`customize serialization <topics-exporters-field-serialization>`.
 
-:mod:`trackref` tracks :class:`Item` objects to help find memory leaks
+:mod:`scrapy.utils.trackref` tracks :class:`Item` objects to help find memory
-(see :ref:`topics-leaks-trackrefs`).
+leaks (see :ref:`topics-leaks-trackrefs`).
 
 Example:
 
@@ -262,7 +263,7 @@ Creating items
 
     >>> product = Product(name="Desktop PC", price=1000)
     >>> print(product)
-    Product(name='Desktop PC', price=1000)
+    {'name': 'Desktop PC', 'price': 1000}
 
 
 Getting field values
@@ -381,7 +382,7 @@ Creating items from dicts:
 .. code-block:: pycon
 
     >>> Product({"name": "Laptop PC", "price": 1500})
-    Product(price=1500, name='Laptop PC')
+    {'name': 'Laptop PC', 'price': 1500}
 
     >>> Product({"name": "Laptop PC", "lala": 1500})  # warning: unknown field in dict
     Traceback (most recent call last):

docs/topics/jobs.rst

@@ -151,8 +151,8 @@ Where:
 -   :class:`~scrapy.pqueues.ScrapyPriorityQueue` creates the ``{priority}{s?}``
     directories.
 
--   :class:`scrapy.squeues.PickleLifoDiskQueue`, a subclass of
+-   :class:`scrapy.squeues.PickleFifoDiskQueue`, a subclass of
-    :class:`queuelib.LifoDiskQueue` that uses :mod:`pickle` to serialize
+    :class:`queuelib.FifoDiskQueue` that uses :mod:`pickle` to serialize
     :class:`dict` representations of :class:`scrapy.Request` objects, creates
     the ``info.json`` and ``q{00000}`` files.
 

docs/topics/leaks.rst

@@ -62,9 +62,9 @@ Debugging memory leaks with ``trackref``
 
 .. skip: start
 
-:mod:`trackref` is a module provided by Scrapy to debug the most common cases of
+:mod:`scrapy.utils.trackref` is a module provided by Scrapy to debug the most
-memory leaks. It basically tracks the references to all live Request,
+common cases of memory leaks. It basically tracks the references to all live
-Response, Item, Spider and Selector objects.
+Request, Response, Item, Spider and Selector objects.
 
 You can enter the telnet console and inspect how many objects (of the classes
 mentioned above) are currently alive using the ``prefs()`` function which is an
@@ -164,7 +164,7 @@ Too many spiders?
 -----------------
 
 If your project has too many spiders executed in parallel,
-the output of :func:`prefs` can be difficult to read.
+the output of ``prefs()`` can be difficult to read.
 For this reason, that function has a ``ignore`` argument which can be used to
 ignore a particular class (and all its subclasses). For
 example, this won't show any live references to spiders:

docs/topics/loaders.rst

@@ -264,7 +264,7 @@ metadata. Here is an example:
     >>> il.add_value("name", ["Welcome to my", "<strong>website</strong>"])
     >>> il.add_value("price", ["&euro;", "<span>1000</span>"])
     >>> il.load_item()
-    {'name': 'Welcome to my website', 'price': '1000'}
+    Product(name='Welcome to my website', price='1000')
 
 .. skip: end
 

docs/topics/media-pipeline.rst

@@ -41,11 +41,10 @@ this:
 2. The item is returned from the spider and goes to the item pipeline.
 
 3. When the item reaches the :class:`FilesPipeline`, the URLs in the
-   ``file_urls`` field are scheduled for download using the standard
+   ``file_urls`` field are downloaded using the standard Scrapy downloader
-   Scrapy scheduler and downloader (which means the scheduler and downloader
+   (which means the downloader middlewares are used, but the spider middlewares
-   middlewares are reused), but with a higher priority, processing them before other
+   aren't). The item remains "locked" at that particular pipeline stage until
-   pages are scraped. The item remains "locked" at that particular pipeline stage
+   the files have finished downloading (or failed for some reason).
-   until the files have finish downloading (or fail for some reason).
 
 4. When the files are downloaded, another field (``files``) will be populated
    with the results. This field will contain a list of dicts with information
@@ -548,10 +547,9 @@ See here the methods that you can override in your custom Files Pipeline:
 
    .. method:: FilesPipeline.get_media_requests(item, info)
 
-      As seen on the workflow, the pipeline will get the URLs of the images to
+      As seen on the workflow, the pipeline will get the requests for the files
-      download from the item. In order to do this, you can override the
+      to download from the item by calling this method. You can override it to
-      :meth:`~get_media_requests` method and return a Request for each
+      change what requests are returned:
-      file URL:
 
       .. code-block:: python
 
@@ -591,8 +589,9 @@ See here the methods that you can override in your custom Files Pipeline:
           * ``downloaded`` - file was downloaded.
           * ``uptodate`` - file was not downloaded, as it was downloaded recently,
             according to the file expiration policy.
-          * ``cached`` - file was already scheduled for download, by another item
+          * ``cached`` - file was taken from a cache (the response has a
-            sharing the same file.
+            ``"cached"`` flag, e.g. from
+            :class:`~scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware`).
 
       The list of tuples received by :meth:`~item_completed` is
       guaranteed to retain the same order of the requests returned from the
@@ -619,9 +618,6 @@ See here the methods that you can override in your custom Files Pipeline:
               (False, Failure(...)),
           ]
 
-      By default the :meth:`get_media_requests` method returns ``None`` which
-      means there are no files to download for the item.
-
    .. method:: FilesPipeline.item_completed(results, item, info)
 
       The :meth:`FilesPipeline.item_completed` method called when all file

docs/topics/practices.rst

@@ -17,8 +17,10 @@ Run Scrapy from a script
 You can use the :ref:`API <topics-api>` to run Scrapy from a script, instead of
 the typical way of running Scrapy via ``scrapy crawl``.
 
-Remember that Scrapy is built on top of the Twisted
+Remember that Scrapy requires a Twisted reactor or (with
-asynchronous networking library, so you need to run it inside the Twisted reactor.
+:setting:`TWISTED_REACTOR_ENABLED` set to ``False``) an asyncio event loop, so
+you need to run one of those in your script for it to work (helpers described
+below can do it for you).
 
 The first utility you can use to run your spiders is
 :class:`scrapy.crawler.AsyncCrawlerProcess` or

docs/topics/request-response.rst

@@ -1101,8 +1101,8 @@ Response objects
 
         The IP address of the server from which the Response originated.
 
-        This attribute is currently only populated by the HTTP 1.1 download
+        This attribute is currently only populated by the HTTP download
-        handler, i.e. for ``http(s)`` responses. For other handlers,
+        handlers, i.e. for ``http(s)`` responses. For other handlers,
         :attr:`ip_address` is always ``None``.
 
     .. attribute:: Response.protocol

docs/topics/settings.rst

@@ -418,7 +418,9 @@ Default: ``None``
 
 Import path of a given ``asyncio`` event loop class.
 
-If the asyncio reactor is enabled (see :setting:`TWISTED_REACTOR`) this setting can be used to specify the
+If the asyncio reactor is enabled (see :setting:`TWISTED_REACTOR`) or when
+:ref:`running Scrapy without a reactor <asyncio-without-reactor>` this setting
+can be used to specify the
 asyncio event loop to be used with it. Set the setting to the import path of the
 desired asyncio event loop class. If the setting is set to ``None`` the default asyncio
 event loop will be used.
@@ -554,6 +556,8 @@ performed to any single domain.
 See also: :ref:`topics-autothrottle` and its
 :setting:`AUTOTHROTTLE_TARGET_CONCURRENCY` option.
 
+It is possible to change this setting per domain by using
+:setting:`DOWNLOAD_SLOTS`.
 
 .. setting:: DEFAULT_DROPITEM_LOG_LEVEL
 
@@ -917,9 +921,8 @@ desired.
 
     This delay can be set per spider using :attr:`download_delay` spider attribute.
 
-It is also possible to change this setting per domain, although it requires
+It is possible to change this setting per domain by using
-non-trivial code. See the implementation of the :ref:`AutoThrottle
+:setting:`DOWNLOAD_SLOTS`.
-<topics-autothrottle>` extension for an example.
 
 .. setting:: DOWNLOAD_BIND_ADDRESS
 
@@ -1358,14 +1361,19 @@ FORCE_CRAWLER_PROCESS
 Default: ``False``
 
 If ``False``, :ref:`Scrapy commands that need a CrawlerProcess
-<topics-commands-crawlerprocess>` will decide between using
+<topics-commands-crawlerprocess>`, when :setting:`TWISTED_REACTOR_ENABLED`
+is set to ``True``, will decide between using
 :class:`scrapy.crawler.AsyncCrawlerProcess` and
 :class:`scrapy.crawler.CrawlerProcess` based on the value of the
 :setting:`TWISTED_REACTOR` setting, but ignoring its value in :ref:`per-spider
 settings <spider-settings>`.
 
 If ``True``, these commands will always use
-:class:`~scrapy.crawler.CrawlerProcess`.
+:class:`~scrapy.crawler.CrawlerProcess` when :setting:`TWISTED_REACTOR_ENABLED`
+is set to ``True``.
+
+When :setting:`TWISTED_REACTOR_ENABLED` is set to ``False``,
+:class:`~scrapy.crawler.AsyncCrawlerProcess` will be used in all cases.
 
 Set this to ``True`` if you want to set :setting:`TWISTED_REACTOR` to a
 non-default value in :ref:`per-spider settings <spider-settings>`.
@@ -1746,6 +1754,9 @@ The randomization policy is the same used by `wget`_ ``--random-wait`` option.
 
 If :setting:`DOWNLOAD_DELAY` is zero this option has no effect.
 
+It is possible to change this setting per domain by using
+:setting:`DOWNLOAD_SLOTS`.
+
 .. _wget: https://www.gnu.org/software/wget/manual/wget.html
 
 .. setting:: REACTOR_THREADPOOL_MAXSIZE
@@ -2159,6 +2170,9 @@ Default: ``"twisted.internet.asyncioreactor.AsyncioSelectorReactor"``
 
 Import path of a given :mod:`~twisted.internet.reactor`.
 
+.. note::
+    This setting has no effect when :setting:`TWISTED_REACTOR_ENABLED` is ``False``.
+
 Scrapy will install this reactor if no other reactor is installed yet, such as
 when the ``scrapy`` CLI program is invoked or when using the
 :class:`~scrapy.crawler.AsyncCrawlerProcess` class or the

docs/topics/signals.rst

@@ -454,7 +454,7 @@ bytes_received
 .. signal:: bytes_received
 .. function:: bytes_received(data, request, spider)
 
-    Sent by the HTTP 1.1 and S3 download handlers when a group of bytes is
+    Sent by some download handlers when a group of bytes is
     received for a specific request. This signal might be fired multiple
     times for the same request, with partial data each time. For instance,
     a possible scenario for a 25 kb response would be two signals fired
@@ -482,7 +482,7 @@ headers_received
 .. signal:: headers_received
 .. function:: headers_received(headers, body_length, request, spider)
 
-    Sent by the HTTP 1.1 and S3 download handlers when the response headers are
+    Sent by some download handlers when the response headers are
     available for a given request, before downloading any additional content.
 
     Handlers for this signal can stop the download of a response while it

docs/topics/spiders.rst

@@ -614,7 +614,7 @@ XMLFeedSpider
 
         A string which defines the iterator to use. It can be either:
 
-           - ``'iternodes'`` - a fast iterator based on regular expressions
+           - ``'iternodes'`` - a fast iterator based on ``lxml``
 
            - ``'html'`` - an iterator which uses :class:`~scrapy.Selector`.
              Keep in mind this uses DOM parsing and must load all DOM in memory

docs/topics/stats.rst

@@ -10,8 +10,8 @@ Collector, and can be accessed through the :attr:`~scrapy.crawler.Crawler.stats`
 attribute of the :ref:`topics-api-crawler`, as illustrated by the examples in
 the :ref:`topics-stats-usecases` section below.
 
-However, the Stats Collector is always available, so you can always import it
+The Stats Collector API is always available, so you can always use it (to
-in your module and use its API (to increment or set new stat keys), regardless
+increment or set new stat keys), regardless
 of whether the stats collection is enabled or not. If it's disabled, the API
 will still work but it won't collect anything. This is aimed at simplifying the
 stats collector usage: you should spend no more than one line of code for
@@ -21,9 +21,6 @@ using the Stats Collector from.
 Another feature of the Stats Collector is that it's very efficient (when
 enabled) and extremely efficient (almost unnoticeable) when disabled.
 
-The Stats Collector keeps a stats table per open spider which is automatically
-opened when the spider is opened, and closed when the spider is closed.
-
 .. _topics-stats-usecases:
 
 Common Stats Collector uses
@@ -87,13 +84,13 @@ Get all stats:
 Available Stats Collectors
 ==========================
 
+.. currentmodule:: scrapy.statscollectors
+
 Besides the basic :class:`StatsCollector` there are other Stats Collectors
 available in Scrapy which extend the basic Stats Collector. You can select
 which Stats Collector to use through the :setting:`STATS_CLASS` setting. The
 default Stats Collector used is the :class:`MemoryStatsCollector`.
 
-.. currentmodule:: scrapy.statscollectors
-
 MemoryStatsCollector
 --------------------
 
@@ -102,7 +99,7 @@ MemoryStatsCollector
     A simple stats collector that keeps the stats of the last scraping run (for
     each spider) in memory, after they're closed. The stats can be accessed
     through the :attr:`spider_stats` attribute, which is a dict keyed by spider
-    domain name.
+    name.
 
     This is the default Stats Collector used in Scrapy.
 

docs/topics/telnetconsole.rst

@@ -107,8 +107,8 @@ Here are some example tasks you can do with the telnet console:
 View engine status
 ------------------
 
-You can use the ``est()`` method of the Scrapy engine to quickly show its state
+You can use the ``est()`` method provided by the console to quickly show the
-using the telnet console::
+engine status::
 
     telnet localhost 6023
     >>> est()

scrapy/core/scheduler.py

@@ -289,11 +289,11 @@ class Scheduler(BaseScheduler):
 
         :param dqclass: A class to be used as persistent request queue.
                         The value for the :setting:`SCHEDULER_DISK_QUEUE` setting is used by default.
-        :type dqclass: class
+        :type dqclass: type
 
         :param mqclass: A class to be used as non-persistent request queue.
                         The value for the :setting:`SCHEDULER_MEMORY_QUEUE` setting is used by default.
-        :type mqclass: class
+        :type mqclass: type
 
         :param logunser: A boolean that indicates whether or not unserializable requests should be logged.
                         The value for the :setting:`SCHEDULER_DEBUG` setting is used by default.
@@ -306,7 +306,7 @@ class Scheduler(BaseScheduler):
 
         :param pqclass: A class to be used as priority queue for requests.
                         The value for the :setting:`SCHEDULER_PRIORITY_QUEUE` setting is used by default.
-        :type pqclass: class
+        :type pqclass: type
 
         :param crawler: The crawler object corresponding to the current crawl.
         :type crawler: :class:`scrapy.crawler.Crawler`

scrapy/settings/__init__.py

@@ -288,10 +288,10 @@ class BaseSettings(MutableMapping[str, Any]):
         -   ``['one', 'two']`` if set to ``'["one", "two"]'`` or ``'one,two'``
 
         :param name: the setting name
-        :type name: string
+        :type name: str
 
         :param default: the value to return if no setting is found
-        :type default: any
+        :type default: object
         """
         value = self.get(name, default)
         if value is None: