5.2 KiB
Available Signals
System Message: ERROR/3 (<stdin>, line 3)
Unknown directive type "module".
.. module:: scrapy.core.signals :synopsis: Signals definitions
Scrapy uses signals extensively to notify when certain actions occur. You can catch some of those signals in your Scrapy project or extension to perform additional tasks or extend Scrapy to add functionality not provided out of the box.
Even though signals provide several arguments, the handlers which catch them don't have to receive all of them.
For more information about working when see the documentation of pydispatcher (library used to implement signals).
Here's a list of signals used in Scrapy and their meaning, in alphabetical order.
System Message: ERROR/3 (<stdin>, line 25)
Unknown directive type "signal".
.. signal:: domain_closed
System Message: ERROR/3 (<stdin>, line 26)
Unknown directive type "function".
.. function:: domain_closed(domain, spider, status)
Sent right after a spider/domain has been closed.
domain is a string which contains the domain of the spider which has been closed spider is the spider which has been closed status is a string which can have two values: 'finished' if the domain has finished successfully, or 'cancelled' if the domain was cancelled (for example, by hitting Ctrl-C, by calling the engine stop() method or by explicitly closing the domain).
System Message: ERROR/3 (<stdin>, line 37)
Unknown directive type "signal".
.. signal:: domain_open
System Message: ERROR/3 (<stdin>, line 38)
Unknown directive type "function".
.. function:: domain_open(domain, spider)
Sent right before a spider has been opened for crawling.
domain is a string which contains the domain of the spider which is about to be opened spider is the spider which is about to be opened
System Message: ERROR/3 (<stdin>, line 46)
Unknown directive type "signal".
.. signal:: domain_opened
System Message: ERROR/3 (<stdin>, line 47)
Unknown directive type "function".
.. function:: domain_opened(domain, spider)
Sent right after a spider has been opened for crawling.
domain is a string with the domain of the spider which has been opened spider is the spider which has been opened
System Message: ERROR/3 (<stdin>, line 54)
Unknown directive type "signal".
.. signal:: domain_idle
System Message: ERROR/3 (<stdin>, line 55)
Unknown directive type "function".
.. function:: domain_idle(domain, spider)
- Sent when a domain has no further:
- requests waiting to be downloaded
- requests scheduled
- items being processed in the item pipeline
domain is a string with the domain of the spider which has gone idle spider is the spider which has gone idle
If any handler of this signals raises a :exception:`DontCloseDomain` the domain won't be closed at this time and will wait until another idle signal is sent. Otherwise (if no handler raises :exception:`DontCloseDomain`) the domain will be closed immediately after all handlers of domain_idle have finished, and a :signal:`domain_closed` will thus be sent.
System Message: ERROR/3 (<stdin>, line 65); backlink
Unknown interpreted text role "exception".System Message: ERROR/3 (<stdin>, line 65); backlink
Unknown interpreted text role "exception".System Message: ERROR/3 (<stdin>, line 65); backlink
Unknown interpreted text role "signal".System Message: ERROR/3 (<stdin>, line 71)
Unknown directive type "signal".
.. signal:: engine_started
System Message: ERROR/3 (<stdin>, line 72)
Unknown directive type "function".
.. function:: engine_started()
Sent when the Scrapy engine is started (for example, when a crawling process has started).
System Message: ERROR/3 (<stdin>, line 77)
Unknown directive type "signal".
.. signal:: engine_stopped
System Message: ERROR/3 (<stdin>, line 78)
Unknown directive type "function".
.. function:: engine_stopped()
Sent when the Scrapy engine is stopped (for example, when a crawling process has finished).
System Message: ERROR/3 (<stdin>, line 83)
Unknown directive type "signal".
.. signal:: request_received
System Message: ERROR/3 (<stdin>, line 84)
Unknown directive type "function".
.. function:: request_received(request, spider, response)
Sent when the engine receives a :class:`~scrapy.http.Request` from a spider.
System Message: ERROR/3 (<stdin>, line 86); backlink
Unknown interpreted text role "class".request is the :class:`~scrapy.http.Request` received spider is the spider which generated the request response is the :class:`~scrapy.http.Response` fed to the spider which generated the request
System Message: ERROR/3 (<stdin>, line 88); backlink
Unknown interpreted text role "class".System Message: ERROR/3 (<stdin>, line 88); backlink
Unknown interpreted text role "class".System Message: ERROR/3 (<stdin>, line 93)
Unknown directive type "signal".
.. signal:: request_uploaded
System Message: ERROR/3 (<stdin>, line 94)
Unknown directive type "function".
.. function:: request_uploaded(request, spider)
Sent right after the download has sent a :class:`~scrapy.http.Request`.
System Message: ERROR/3 (<stdin>, line 96); backlink
Unknown interpreted text role "class".request is the :class:`~scrapy.http.Request` uploaded/sent spider is the spider which generated the request
System Message: ERROR/3 (<stdin>, line 98); backlink
Unknown interpreted text role "class".System Message: ERROR/3 (<stdin>, line 101)
Unknown directive type "signal".
.. signal:: response_received
System Message: ERROR/3 (<stdin>, line 102)
Unknown directive type "function".
.. function:: response_received(response, spider)
response is the :class:`~scrapy.http.Response` received spider is the spider for which the response is intended
System Message: ERROR/3 (<stdin>, line 104); backlink
Unknown interpreted text role "class".Sent when the engine receives a new :class:`~scrapy.http.Response` from the downloader.
System Message: ERROR/3 (<stdin>, line 107); backlink
Unknown interpreted text role "class".System Message: ERROR/3 (<stdin>, line 110)
Unknown directive type "signal".
.. signal:: response_downloaded
System Message: ERROR/3 (<stdin>, line 111)
Unknown directive type "function".
.. function:: response_downloaded(response, spider)
Sent by the downloader right after a HTTPResponse is downloaded.
response is the HTTPResponse downloaded spider is the spider for which the response is intended
System Message: ERROR/3 (<stdin>, line 118)
Unknown directive type "signal".
.. signal:: item_scraped
System Message: ERROR/3 (<stdin>, line 119)
Unknown directive type "function".
.. function:: item_scraped(item, spider, response)
Sent when the engine receives a new scraped item from the spider, and right before the item is sent to the :ref:`topics-item-pipeline`.
System Message: ERROR/3 (<stdin>, line 121); backlink
Unknown interpreted text role "ref".item is the item scraped spider is the spider which scraped the item response is the :class:`~scrapy.http.Response` from which the item was scraped
System Message: ERROR/3 (<stdin>, line 124); backlink
Unknown interpreted text role "class".System Message: ERROR/3 (<stdin>, line 129)
Unknown directive type "signal".
.. signal:: item_passed
System Message: ERROR/3 (<stdin>, line 130)
Unknown directive type "function".
.. function:: item_passed(item, spider, response, pipe_output)
Sent after an item has passed al the :ref:`topics-item-pipeline` stages without being dropped.
System Message: ERROR/3 (<stdin>, line 132); backlink
Unknown interpreted text role "ref".item is the item which passed the pipeline spider is the spider which scraped the item response is the :class:`~scrapy.http.Response` from which the item was scraped pipe_output is the output of the item pipeline. Typically, this points to the same item object, unless some pipeline stage created a new item.
System Message: ERROR/3 (<stdin>, line 135); backlink
Unknown interpreted text role "class".System Message: ERROR/3 (<stdin>, line 141)
Unknown directive type "signal".
.. signal:: item_dropped
System Message: ERROR/3 (<stdin>, line 142)
Unknown directive type "function".
.. function:: item_dropped(item, spider, response, exception)
Sent after an item has dropped from the :ref:`topics-item-pipeline` when some stage raised a :exception:`DropItem` exception.
System Message: ERROR/3 (<stdin>, line 144); backlink
Unknown interpreted text role "ref".System Message: ERROR/3 (<stdin>, line 144); backlink
Unknown interpreted text role "exception".item is the item dropped from the :ref:`topics-item-pipeline` spider is the spider which scraped the item response is the :class:`~scrapy.http.Response` from which the item was scraped exception is the (:exception:`DropItem` child) exception that caused the item to be dropped
System Message: ERROR/3 (<stdin>, line 147); backlink
Unknown interpreted text role "ref".System Message: ERROR/3 (<stdin>, line 147); backlink
Unknown interpreted text role "class".System Message: ERROR/3 (<stdin>, line 147); backlink
Unknown interpreted text role "exception".