Include signal example

This commit is contained in:
Darshan Chaudhary 2015-10-29 15:40:07 +05:30 committed by darshanime
parent 8dc400c66d
commit a2e6452554
3 changed files with 33 additions and 2 deletions

View File

@ -7,7 +7,7 @@ Downloading and processing files and images
.. currentmodule:: scrapy.pipelines.images
Scrapy provides reusable :doc:`item pipelines </topics/item-pipeline>` for
downloading fies attached to a particular item (for example, when you scrape
downloading files attached to a particular item (for example, when you scrape
products and also want to download their images locally). These pipelines share
a bit of functionality and structure (we refer to them as media pipelines), but
typically you'll either use the Files Pipeline or the Images Pipeline.

View File

@ -16,6 +16,37 @@ deliver the arguments that the handler receives.
You can connect to signals (or send your own) through the
:ref:`topics-api-signals`.
Here is a simple example showing how you can catch signals and perform some action:
::
from scrapy import signals
from scrapy import Spider
class DmozSpider(Spider):
name = "dmoz"
allowed_domains = ["dmoz.org"]
start_urls = [
"http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
"http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/",
]
@classmethod
def from_crawler(cls, crawler, *args, **kwargs):
spider = super(DmozSpider, cls).from_crawler(crawler, *args, **kwargs)
crawler.signals.connect(spider.spider_closed, signal=signals.spider_closed)
return spider
def spider_closed(self, spider):
spider.logger.info('Spider closed: %s', spider.name)
def parse(self, response):
pass
Deferred signal handlers
========================

View File

@ -1,4 +1,4 @@
"""Helper functions which doesn't fit anywhere else"""
"""Helper functions which don't fit anywhere else"""
import re
import hashlib
from importlib import import_module