mirror of https://github.com/scrapy/scrapy.git
renamed "parse_item" method of XMLFeedSpider to "parse_node", keeping backwards compatibility
This commit is contained in:
parent
31693eb90f
commit
74f706b356
|
|
@ -256,10 +256,9 @@ Let's now take a look at an example CrawlSpider with rules::
|
|||
|
||||
|
||||
This spider would start crawling example.com's home page, collecting category
|
||||
links, and item links, parsing the latter with the
|
||||
:meth:`XMLFeedSpider.parse_item` method. For each item response, some data will
|
||||
be extracted from the HTML using XPath, and a :class:`~scrapy.item.Item` will
|
||||
be filled with it.
|
||||
links, and item links, parsing the latter with the ``parse_item`` method. For
|
||||
each item response, some data will be extracted from the HTML using XPath, and
|
||||
a :class:`~scrapy.item.Item` will be filled with it.
|
||||
|
||||
XMLFeedSpider
|
||||
-------------
|
||||
|
|
@ -327,7 +326,7 @@ XMLFeedSpider
|
|||
the response body before parsing it. This method receives a response and
|
||||
returns response (it could be the same or another one).
|
||||
|
||||
.. method:: parse_item(response, selector)
|
||||
.. method:: parse_node(response, selector)
|
||||
|
||||
This method is called for the nodes matching the provided tag name
|
||||
(``itertag``). Receives the response and an XPathSelector for each node.
|
||||
|
|
@ -336,8 +335,6 @@ XMLFeedSpider
|
|||
:class:`~scrapy.http.Request` object, or an iterable containing any of
|
||||
them.
|
||||
|
||||
.. warning:: This method will soon change its name to ``parse_node``
|
||||
|
||||
.. method:: process_results(response, results)
|
||||
|
||||
This method is called for each result (item or request) returned by the
|
||||
|
|
@ -362,7 +359,7 @@ These spiders are pretty easy to use, let's have at one example::
|
|||
iterator = 'iternodes' # This is actually unnecesary, since it's the default value
|
||||
itertag = 'item'
|
||||
|
||||
def parse_item(self, response, node):
|
||||
def parse_node(self, response, node):
|
||||
log.msg('Hi, this is a <%s> node!: %s' % (self.itertag, ''.join(node.extract())))
|
||||
|
||||
item = Item()
|
||||
|
|
@ -382,10 +379,9 @@ CSVFeedSpider
|
|||
|
||||
.. class:: CSVFeedSpider
|
||||
|
||||
.. warning:: The API of the CSVFeedSpider is not yet stable. Use with caution.
|
||||
|
||||
This spider is very similar to the XMLFeedSpider, although it iterates through
|
||||
rows, instead of nodes. It also has other two different attributes:
|
||||
This spider is very similar to the XMLFeedSpider, except that it iterates
|
||||
over rows, instead of nodes. The method that gets called in each iteration
|
||||
is :meth:`parse_row`.
|
||||
|
||||
.. attribute:: CSVFeedSpider.delimiter
|
||||
|
||||
|
|
@ -397,9 +393,6 @@ CSVFeedSpider
|
|||
A list of the rows contained in the file CSV feed which will be used for
|
||||
extracting fields from it.
|
||||
|
||||
In this spider, the method that gets called in each row iteration ``parse_row``
|
||||
instead of ``parse_item`` (like in :class:`XMLFeedSpider`).
|
||||
|
||||
.. method:: CSVFeedSpider.parse_row(response, row)
|
||||
|
||||
Receives a response and a dict (representing each row) with a key for each
|
||||
|
|
|
|||
|
|
@ -43,8 +43,10 @@ class XMLFeedSpider(InitSpider):
|
|||
"""
|
||||
return response
|
||||
|
||||
def parse_item(self, response, selector):
|
||||
def parse_node(self, response, selector):
|
||||
"""This method must be overriden with your custom spider functionality"""
|
||||
if hasattr(self, 'parse_item'): # backward compatibility
|
||||
return self.parse_item(response, selector)
|
||||
raise NotImplemented
|
||||
|
||||
def parse_nodes(self, response, nodes):
|
||||
|
|
@ -56,7 +58,7 @@ class XMLFeedSpider(InitSpider):
|
|||
"""
|
||||
|
||||
for selector in nodes:
|
||||
ret = self.parse_item(response, selector)
|
||||
ret = self.parse_node(response, selector)
|
||||
if isinstance(ret, (BaseItem, Request)):
|
||||
ret = [ret]
|
||||
if not isinstance(ret, (list, tuple)):
|
||||
|
|
@ -65,8 +67,8 @@ class XMLFeedSpider(InitSpider):
|
|||
yield result_item
|
||||
|
||||
def parse(self, response):
|
||||
if not hasattr(self, 'parse_item'):
|
||||
raise NotConfigured('You must define parse_item method in order to scrape this XML feed')
|
||||
if not hasattr(self, 'parse_node'):
|
||||
raise NotConfigured('You must define parse_node method in order to scrape this XML feed')
|
||||
|
||||
response = self.adapt_response(response)
|
||||
if self.iterator == 'iternodes':
|
||||
|
|
|
|||
Loading…
Reference in New Issue