mirror of https://github.com/scrapy/scrapy.git
relocated experimental newitems/loaders doc, and added example for extending fields metadata
--HG-- rename : docs/experimental/newitem-loader.rst => docs/experimental/loaders.rst rename : docs/experimental/newitem.rst => docs/experimental/newitems.rst
This commit is contained in:
parent
d95e99f585
commit
e585c6cac4
|
|
@ -19,5 +19,5 @@ it's properly merged) . Use at your own risk.
|
|||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
newitem
|
||||
newitem-loader
|
||||
newitems
|
||||
loaders
|
||||
|
|
|
|||
|
|
@ -8,12 +8,12 @@ Item Loaders
|
|||
:synopsis: Item Loader class
|
||||
|
||||
Item Loaders (or Loaders, for short) provide a convenient mechanism for
|
||||
populating scraped :ref:`Items <topics-newitem>`. Even though Items can be
|
||||
populating scraped :ref:`Items <topics-newitems>`. Even though Items can be
|
||||
populated using their own dictionary-like API, the Loaders provide a much more
|
||||
convenient API for populating them from a scraping process, by automating some
|
||||
common tasks like parsing the raw extracted data before assigning it.
|
||||
|
||||
In other words, :ref:`Items <topics-newitem>` provide the *container* of
|
||||
In other words, :ref:`Items <topics-newitems>` provide the *container* of
|
||||
scraped data, while Loaders provide the mechanism for *populating* that
|
||||
container.
|
||||
|
||||
|
|
@ -34,8 +34,8 @@ Then, you start adding values to the Loader, typically collecting them using
|
|||
same item field, the Loader will know how to "join" those values later using a
|
||||
Reducer.
|
||||
|
||||
Here's a typical Loader usage in a :ref:`Spider <topics-spiders>` the
|
||||
:ref:`Product item defined in the Items chapter <topics-newitem-declaring>`.::
|
||||
Here is a typical Loader usage in a :ref:`Spider <topics-spiders>` using the
|
||||
:ref:`Product item defined in the Items chapter <topics-newitems-declaring>`.::
|
||||
|
||||
from scrapy.item.loader import Loader
|
||||
from scrapy.xpath import HtmlXPathSelector
|
||||
|
|
@ -1,8 +1,8 @@
|
|||
.. _topics-newitem:
|
||||
.. _topics-newitems:
|
||||
|
||||
=====
|
||||
Items
|
||||
=====
|
||||
=================
|
||||
Items (version 2)
|
||||
=================
|
||||
|
||||
.. module:: scrapy.newitem
|
||||
:synopsis: Item and Field classes
|
||||
|
|
@ -17,7 +17,7 @@ their available fields.
|
|||
|
||||
.. _dictionary-like: http://docs.python.org/library/stdtypes.html#dict
|
||||
|
||||
.. _topics-newitem-declaring:
|
||||
.. _topics-newitems-declaring:
|
||||
|
||||
Declaring Items
|
||||
===============
|
||||
|
|
@ -204,6 +204,15 @@ For example::
|
|||
discount_percent = Field(default=0)
|
||||
discount_expiration_date = Field()
|
||||
|
||||
You can also extend field metadata by using the previous field metadata and
|
||||
appending more values, or changing existing values, like this::
|
||||
|
||||
class SpecificProduct(Product):
|
||||
name = Field(Product.fields['name'], default='product')
|
||||
|
||||
That adds (or replaces) the ``default`` metadata key for the ``name`` field,
|
||||
keeping all the previously existing metadata values.
|
||||
|
||||
Item objects
|
||||
============
|
||||
|
||||
|
|
@ -219,7 +228,7 @@ Item objects
|
|||
A dictionary containing *all declared fields* for this Item, not only
|
||||
those populated. The keys are the field names and the values are the
|
||||
:class:`Field` objects used in the :ref:`Item declaration
|
||||
<topics-newitem-declaring>`.
|
||||
<topics-newitems-declaring>`.
|
||||
|
||||
.. _dict API: http://docs.python.org/library/stdtypes.html#dict
|
||||
|
||||
|
|
@ -231,7 +240,7 @@ Field objects
|
|||
The :class:`Field` class is just an alias to the built-in `dict`_ class and
|
||||
doesn't provide any extra functionality or attributes. In other words,
|
||||
:class:`Field` objects are plain-old Python dicts. A separate class is used
|
||||
to support the :ref:`item declaration syntax <topics-newitem-declaring>`
|
||||
to support the :ref:`item declaration syntax <topics-newitems-declaring>`
|
||||
based on class attributes.
|
||||
|
||||
.. _dict: http://docs.python.org/library/stdtypes.html#dict
|
||||
Loading…
Reference in New Issue