mirror of https://github.com/scrapy/scrapy.git
Improve the docs about scrapy parse --pipelines (#7876)
This commit is contained in:
parent
639fac78b3
commit
91b70e4db4
|
|
@ -507,7 +507,7 @@ Supported options:
|
|||
* ``--cbkwargs``: additional keyword arguments that will be passed to the callback.
|
||||
This must be a valid json string. Example: --cbkwargs='{"foo" : "bar"}'
|
||||
|
||||
* ``--pipelines``: process items through pipelines
|
||||
* ``--pipelines``: :ref:`process items through pipelines <test-item-pipeline>`
|
||||
|
||||
* ``--rules`` or ``-r``: use :class:`~scrapy.spiders.CrawlSpider`
|
||||
rules to discover the callback (i.e. spider method) to use for parsing the
|
||||
|
|
|
|||
|
|
@ -330,6 +330,36 @@ passes through ``PricePipeline`` before it reaches the :ref:`feed exports
|
|||
.. _books.toscrape.com: https://books.toscrape.com/
|
||||
|
||||
|
||||
.. _test-item-pipeline:
|
||||
|
||||
Testing an item pipeline
|
||||
========================
|
||||
|
||||
To send the items from a single URL through your item pipelines, use the
|
||||
:command:`parse` command with the ``--pipelines`` option::
|
||||
|
||||
scrapy parse --pipelines "https://books.toscrape.com/"
|
||||
|
||||
To test specific item data instead, add a callback that builds an item out of
|
||||
its keyword arguments:
|
||||
|
||||
.. skip: next
|
||||
.. code-block:: python
|
||||
|
||||
class BooksSpider(scrapy.Spider):
|
||||
# ...
|
||||
|
||||
def parse_item(self, response, **fields):
|
||||
yield BookItem(**fields)
|
||||
|
||||
and pass those keyword arguments in the command line::
|
||||
|
||||
scrapy parse --pipelines -c parse_item --cbkwargs '{"title": "Test", "price": 10}' "https://books.toscrape.com/"
|
||||
|
||||
Pass any URL that your spider handles; it is downloaded even though the
|
||||
callback ignores it.
|
||||
|
||||
|
||||
Common pitfalls
|
||||
===============
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue