RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Expanded doc for idle signal

This commit is contained in:
Iván de Prado Alonso 2021-06-29 13:40:43 +01:00
parent f35970778b
commit e94d3ac173
2 changed files with 6 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/topics/signals.rst
View File

@ -269,7 +269,11 @@ spider_idle
prevent the spider from being closed.
Alternatively, you may raise a :exc:`~scrapy.exceptions.CloseSpider`
exception to provide a custom spider closing reason.
exception to provide a custom spider closing reason. An
idle handler is the perfect place to put some code that assesses
the final spider results and update the final closing reason
accordingly (e.g. setting it to 'too_few_results' instead of
'finished').
This signal does not support returning deferreds from its handlers.

2
scrapy/core/engine.py
View File

@ -326,7 +326,7 @@ class ExecutionEngine:
Called when a spider gets idle, i.e. when there are no remaining requests to download or schedule.
It can be called multiple times. If a handler for the spider_idle signal raises a DontCloseSpider
exception, the spider is not closed until the next loop and this function is guaranteed to be called
(at least) once again. A handler can raise CloseSpider to provide a custom closing reason
(at least) once again. A handler can raise CloseSpider to provide a custom closing reason.
"""
assert self.spider is not None # typing
expected_ex = (DontCloseSpider, CloseSpider)