mirror of https://github.com/scrapy/scrapy.git
provide documentation for nested loaders
This commit is contained in:
parent
425e35ee90
commit
88c92cb68b
|
|
@ -432,6 +432,14 @@ ItemLoader objects
|
|||
<topics-loaders-processors>` to get the final value to assign to each
|
||||
item field.
|
||||
|
||||
.. method:: nested_loader(xpath=selector, css=selector)
|
||||
|
||||
Create a nested loader with either an xpath selector or css selector.
|
||||
The supplied selector is applied relative to selector associated
|
||||
with this :class:`ItemLoader`. The nested loader shares the :class:`Item`
|
||||
with the parent :class:`ItemLoader` so calls to :meth:`add_xpath`,
|
||||
:meth:`add_value`, :meth:`replace_value`, etc. will behave as expected.
|
||||
|
||||
.. method:: get_collected_values(field_name)
|
||||
|
||||
Return the collected values for the given field.
|
||||
|
|
@ -490,6 +498,52 @@ ItemLoader objects
|
|||
:attr:`default_selector_class`. This attribute is meant to be
|
||||
read-only.
|
||||
|
||||
.. _topics-loaders-nested:
|
||||
|
||||
Nested Loaders
|
||||
==============
|
||||
|
||||
When parsing related values from a subsection of a document, it can be
|
||||
useful to create nested loaders. Imagine you're extracting details from
|
||||
a footer of a page that looks something like:
|
||||
|
||||
Example::
|
||||
|
||||
<footer>
|
||||
<a class="social" href="http://facebook.com/whatever">Like Us</a>
|
||||
<a class="social" href="http://twitter.com/whatever">Follow Us</a>
|
||||
<a class="email" href="mailto:whatever@example.com">Email Us</a>
|
||||
</footer>
|
||||
|
||||
Without nested loaders, you need to specify the full xpath (or css) for each value
|
||||
that you wish to extract.
|
||||
|
||||
Example::
|
||||
|
||||
loader = ItemLoader(item=Item())
|
||||
# load stuff not in the footer
|
||||
loader.add_xpath('social', '//footer/a[@class = "social"]/@href')
|
||||
loader.add_xpath('email', '//footer/a[@class = "email"]/@href')
|
||||
loader.load_item()
|
||||
|
||||
Instead, you can create a nested loader with the footer selector and add values
|
||||
relative to the footer. The functionality is the same but you avoid repeating
|
||||
the footer selector.
|
||||
|
||||
Example::
|
||||
|
||||
loader = ItemLoader(item=Item())
|
||||
# load stuff not in the footer
|
||||
footer_loader = loader.nested_loader(xpath='//footer')
|
||||
footer_loader.add_xpath('social', 'a[@class = "social"]/@href')
|
||||
footer_loader.add_xpath('email', 'a[@class = "email"]/@href')
|
||||
# no need to call footer_loader.load_item()
|
||||
loader.load_item()
|
||||
|
||||
You can nest loaders arbitrarilly and they work with either xpath or css selectors.
|
||||
As a general guideline, use nested loaders when they make your code simpler but do
|
||||
not go overboard with nesting or your parser can become difficult to read.
|
||||
|
||||
.. _topics-loaders-extending:
|
||||
|
||||
Reusing and extending Item Loaders
|
||||
|
|
|
|||
Loading…
Reference in New Issue