From 88c92cb68b5a6f9b0c50c801f92dacc4fc0fa52a Mon Sep 17 00:00:00 2001 From: Daniel Collins Date: Sat, 29 Aug 2015 14:23:25 -0700 Subject: [PATCH] provide documentation for nested loaders --- docs/topics/loaders.rst | 54 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/docs/topics/loaders.rst b/docs/topics/loaders.rst index 8841711ea..0c20f32ef 100644 --- a/docs/topics/loaders.rst +++ b/docs/topics/loaders.rst @@ -432,6 +432,14 @@ ItemLoader objects ` to get the final value to assign to each item field. + .. method:: nested_loader(xpath=selector, css=selector) + + Create a nested loader with either an xpath selector or css selector. + The supplied selector is applied relative to selector associated + with this :class:`ItemLoader`. The nested loader shares the :class:`Item` + with the parent :class:`ItemLoader` so calls to :meth:`add_xpath`, + :meth:`add_value`, :meth:`replace_value`, etc. will behave as expected. + .. method:: get_collected_values(field_name) Return the collected values for the given field. @@ -490,6 +498,52 @@ ItemLoader objects :attr:`default_selector_class`. This attribute is meant to be read-only. +.. _topics-loaders-nested: + +Nested Loaders +============== + +When parsing related values from a subsection of a document, it can be +useful to create nested loaders. Imagine you're extracting details from +a footer of a page that looks something like: + +Example:: + + + +Without nested loaders, you need to specify the full xpath (or css) for each value +that you wish to extract. + +Example:: + + loader = ItemLoader(item=Item()) + # load stuff not in the footer + loader.add_xpath('social', '//footer/a[@class = "social"]/@href') + loader.add_xpath('email', '//footer/a[@class = "email"]/@href') + loader.load_item() + +Instead, you can create a nested loader with the footer selector and add values +relative to the footer. The functionality is the same but you avoid repeating +the footer selector. + +Example:: + + loader = ItemLoader(item=Item()) + # load stuff not in the footer + footer_loader = loader.nested_loader(xpath='//footer') + footer_loader.add_xpath('social', 'a[@class = "social"]/@href') + footer_loader.add_xpath('email', 'a[@class = "email"]/@href') + # no need to call footer_loader.load_item() + loader.load_item() + +You can nest loaders arbitrarilly and they work with either xpath or css selectors. +As a general guideline, use nested loaders when they make your code simpler but do +not go overboard with nesting or your parser can become difficult to read. + .. _topics-loaders-extending: Reusing and extending Item Loaders