docs: adds another spider example that yields multiples requests/items from a single callback

This commit is contained in:
Daniel Grana 2010-02-18 16:51:05 -02:00
parent 57d60eae39
commit 91f4d6dc51
1 changed files with 25 additions and 0 deletions

View File

@ -169,6 +169,31 @@ Let's see an example::
SPIDER = MySpider()
Another example returning multiples Requests and Items from a single callback::
from scrapy.selector import HtmlXPathSelector
from scrapy.spider import BaseSpider
from scrapy.http import Request
from myproject.items import MyItem
class MySpider(BaseSpider):
domain_name = 'http://www.example.com'
start_urls = [
'http://www.example.com/1.html',
'http://www.example.com/2.html',
'http://www.example.com/3.html',
]
def parse(self, response):
hxs = HtmlXPathSelector(response)
for h3 in hxs.select('//h3').extract():
yield MyItem(title=h3)
for url in hxs.select('//a/@href').extract():
yield Request(url, callback=self.parse)
SPIDER = MySpider()
.. module:: scrapy.contrib.spiders
:synopsis: Collection of generic spiders