RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

DOC -> Add missing documentation for CloseSpider & CoreStats (#7421) 

* DOC -> Add missing documentation for CloseSpider & CoreStats

* Apply the suggestion from the PR review.

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
This commit is contained in:
SpiliosDmk 2026-05-20 11:27:24 +03:00 committed by GitHub
parent abe9c63841
commit 4a16550859
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 18 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
docs/topics/extensions.rst
View File

@ -136,6 +136,19 @@ Core Stats extension
Enable the collection of core statistics, provided the stats collection is Enable the collection of core statistics, provided the stats collection is
enabled (see :ref:`topics-stats`). enabled (see :ref:`topics-stats`).
The following stats are collected:
* ``start_time``: start date/time of the crawl (:class:`~datetime.datetime`).
* ``finish_time``: end date/time of the crawl (:class:`~datetime.datetime`).
* ``elapsed_time_seconds``: total crawl duration in seconds (:class:`float`).
* ``finish_reason``: the closing reason string (e.g. ``"finished"``,
``"closespider_timeout"``).
* ``item_scraped_count``: total number of items that passed all pipelines.
* ``item_dropped_count``: total number of items dropped by a pipeline.
* ``item_dropped_reasons_count/<ExceptionName>``: per-exception drop count
(e.g. ``item_dropped_reasons_count/DropItem``).
* ``response_received_count``: total number of HTTP responses received.
.. _topics-extensions-ref-telnetconsole: .. _topics-extensions-ref-telnetconsole:
Log Count extension Log Count extension
@ -247,6 +260,7 @@ settings:
* :setting:`CLOSESPIDER_TIMEOUT_NO_ITEM` * :setting:`CLOSESPIDER_TIMEOUT_NO_ITEM`
* :setting:`CLOSESPIDER_ITEMCOUNT` * :setting:`CLOSESPIDER_ITEMCOUNT`
* :setting:`CLOSESPIDER_PAGECOUNT` * :setting:`CLOSESPIDER_PAGECOUNT`
* :setting:`CLOSESPIDER_PAGECOUNT_NO_ITEM`
* :setting:`CLOSESPIDER_ERRORCOUNT` * :setting:`CLOSESPIDER_ERRORCOUNT`
.. note:: .. note::
@ -260,12 +274,11 @@ settings:
CLOSESPIDER_TIMEOUT CLOSESPIDER_TIMEOUT
""""""""""""""""""" """""""""""""""""""
Default: ``0`` Default: ``0.0``
An integer which specifies a number of seconds. If the spider remains open for If the spider remains open for more than this number of seconds, it will be
more than that number of seconds, it will be automatically closed with the automatically closed with the reason ``closespider_timeout``. If zero (or non
reason ``closespider_timeout``. If zero (or non set), spiders won't be closed by set), spiders won't be closed by timeout.
timeout.
.. setting:: CLOSESPIDER_TIMEOUT_NO_ITEM .. setting:: CLOSESPIDER_TIMEOUT_NO_ITEM