renamed "parse_item" method of XMLFeedSpider to "parse_node", keeping backwards compatibility

This commit is contained in:
Pablo Hoffman 2009-08-24 10:54:34 -03:00
parent 31693eb90f
commit 74f706b356
2 changed files with 14 additions and 19 deletions

View File

@ -256,10 +256,9 @@ Let's now take a look at an example CrawlSpider with rules::
This spider would start crawling example.com's home page, collecting category
links, and item links, parsing the latter with the
:meth:`XMLFeedSpider.parse_item` method. For each item response, some data will
be extracted from the HTML using XPath, and a :class:`~scrapy.item.Item` will
be filled with it.
links, and item links, parsing the latter with the ``parse_item`` method. For
each item response, some data will be extracted from the HTML using XPath, and
a :class:`~scrapy.item.Item` will be filled with it.
XMLFeedSpider
-------------
@ -327,7 +326,7 @@ XMLFeedSpider
the response body before parsing it. This method receives a response and
returns response (it could be the same or another one).
.. method:: parse_item(response, selector)
.. method:: parse_node(response, selector)
This method is called for the nodes matching the provided tag name
(``itertag``). Receives the response and an XPathSelector for each node.
@ -336,8 +335,6 @@ XMLFeedSpider
:class:`~scrapy.http.Request` object, or an iterable containing any of
them.
.. warning:: This method will soon change its name to ``parse_node``
.. method:: process_results(response, results)
This method is called for each result (item or request) returned by the
@ -362,7 +359,7 @@ These spiders are pretty easy to use, let's have at one example::
iterator = 'iternodes' # This is actually unnecesary, since it's the default value
itertag = 'item'
def parse_item(self, response, node):
def parse_node(self, response, node):
log.msg('Hi, this is a <%s> node!: %s' % (self.itertag, ''.join(node.extract())))
item = Item()
@ -382,10 +379,9 @@ CSVFeedSpider
.. class:: CSVFeedSpider
.. warning:: The API of the CSVFeedSpider is not yet stable. Use with caution.
This spider is very similar to the XMLFeedSpider, although it iterates through
rows, instead of nodes. It also has other two different attributes:
This spider is very similar to the XMLFeedSpider, except that it iterates
over rows, instead of nodes. The method that gets called in each iteration
is :meth:`parse_row`.
.. attribute:: CSVFeedSpider.delimiter
@ -397,9 +393,6 @@ CSVFeedSpider
A list of the rows contained in the file CSV feed which will be used for
extracting fields from it.
In this spider, the method that gets called in each row iteration ``parse_row``
instead of ``parse_item`` (like in :class:`XMLFeedSpider`).
.. method:: CSVFeedSpider.parse_row(response, row)
Receives a response and a dict (representing each row) with a key for each

View File

@ -43,8 +43,10 @@ class XMLFeedSpider(InitSpider):
"""
return response
def parse_item(self, response, selector):
def parse_node(self, response, selector):
"""This method must be overriden with your custom spider functionality"""
if hasattr(self, 'parse_item'): # backward compatibility
return self.parse_item(response, selector)
raise NotImplemented
def parse_nodes(self, response, nodes):
@ -56,7 +58,7 @@ class XMLFeedSpider(InitSpider):
"""
for selector in nodes:
ret = self.parse_item(response, selector)
ret = self.parse_node(response, selector)
if isinstance(ret, (BaseItem, Request)):
ret = [ret]
if not isinstance(ret, (list, tuple)):
@ -65,8 +67,8 @@ class XMLFeedSpider(InitSpider):
yield result_item
def parse(self, response):
if not hasattr(self, 'parse_item'):
raise NotConfigured('You must define parse_item method in order to scrape this XML feed')
if not hasattr(self, 'parse_node'):
raise NotConfigured('You must define parse_node method in order to scrape this XML feed')
response = self.adapt_response(response)
if self.iterator == 'iternodes':