From 50fa46d1830d69f559a51b7a311909464e4c3d22 Mon Sep 17 00:00:00 2001 From: Capi Etheriel Date: Sun, 9 Jun 2013 04:03:20 -0300 Subject: [PATCH] Document CrawlSpider.parse_start_urls method --- docs/topics/spiders.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst index cb255065f..4e969498d 100644 --- a/docs/topics/spiders.rst +++ b/docs/topics/spiders.rst @@ -261,6 +261,14 @@ CrawlSpider described below. If multiple rules match the same link, the first one will be used, according to the order they're defined in this attribute. + This spider also exposes an overrideable method: + + .. method:: parse_start_url(response) + + This method is called for the start_urls responses. It allows to parse + the initial responses and must return either a + :class:`~scrapy.item.Item` object, a :class:`~scrapy.http.Request` + object, or an iterable containing any of them. Crawling rules ~~~~~~~~~~~~~~