RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Document coroutines for signals.

This commit is contained in:
Burak Can Kahraman 2021-12-30 15:48:53 +03:00
parent d6a384b3cf
commit a9dfd85ea6
2 changed files with 11 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/topics/coroutines.rst
View File

@ -1,3 +1,5 @@
.. _topics-coroutines:
========== ==========
Coroutines Coroutines
========== ==========

20
docs/topics/signals.rst
View File

@ -51,12 +51,12 @@ Deferred signal handlers
======================== ========================
Some signals support returning :class:`~twisted.internet.defer.Deferred` Some signals support returning :class:`~twisted.internet.defer.Deferred`
objects from their handlers, allowing you to run asynchronous code that or :term:`awaitable objects <awaitable>` from their handlers, allowing
does not block Scrapy. If a signal handler returns a you to run asynchronous code that does not block Scrapy. If a signal
:class:`~twisted.internet.defer.Deferred`, Scrapy waits for that handler returns one of these objects, Scrapy waits for that asynchronous
:class:`~twisted.internet.defer.Deferred` to fire. operation to finish.
Let's take an example:: Let's take an example using :ref:`coroutines <topics-coroutines>`::
class SignalSpider(scrapy.Spider): class SignalSpider(scrapy.Spider):
name = 'signals' name = 'signals'
@ -68,17 +68,15 @@ Let's take an example::
crawler.signals.connect(spider.item_scraped, signal=signals.item_scraped) crawler.signals.connect(spider.item_scraped, signal=signals.item_scraped)
return spider return spider
def item_scraped(self, item): async def item_scraped(self, item):
# Send the scraped item to the server # Send the scraped item to the server
d = treq.post( response = await treq.post(
'http://example.com/post', 'http://example.com/post',
json.dumps(item).encode('ascii'), json.dumps(item).encode('ascii'),
headers={b'Content-Type': [b'application/json']} headers={b'Content-Type': [b'application/json']}
) )
# The next item will be scraped only after return response
# deferred (d) is fired
return d
def parse(self, response): def parse(self, response):
for quote in response.css('div.quote'): for quote in response.css('div.quote'):
@ -89,7 +87,7 @@ Let's take an example::
} }
See the :ref:`topics-signals-ref` below to know which signals support See the :ref:`topics-signals-ref` below to know which signals support
:class:`~twisted.internet.defer.Deferred`. :class:`~twisted.internet.defer.Deferred` and :term:`awaitable objects <awaitable>`.
.. _topics-signals-ref: .. _topics-signals-ref: