Improve the docs about scrapy parse --pipelines (#7876)

This commit is contained in:
Adrian 2026-08-04 17:30:51 +02:00 committed by GitHub
parent 639fac78b3
commit 91b70e4db4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 31 additions and 1 deletions

View File

@ -507,7 +507,7 @@ Supported options:
* ``--cbkwargs``: additional keyword arguments that will be passed to the callback.
This must be a valid json string. Example: --cbkwargs='{"foo" : "bar"}'
* ``--pipelines``: process items through pipelines
* ``--pipelines``: :ref:`process items through pipelines <test-item-pipeline>`
* ``--rules`` or ``-r``: use :class:`~scrapy.spiders.CrawlSpider`
rules to discover the callback (i.e. spider method) to use for parsing the

View File

@ -330,6 +330,36 @@ passes through ``PricePipeline`` before it reaches the :ref:`feed exports
.. _books.toscrape.com: https://books.toscrape.com/
.. _test-item-pipeline:
Testing an item pipeline
========================
To send the items from a single URL through your item pipelines, use the
:command:`parse` command with the ``--pipelines`` option::
scrapy parse --pipelines "https://books.toscrape.com/"
To test specific item data instead, add a callback that builds an item out of
its keyword arguments:
.. skip: next
.. code-block:: python
class BooksSpider(scrapy.Spider):
# ...
def parse_item(self, response, **fields):
yield BookItem(**fields)
and pass those keyword arguments in the command line::
scrapy parse --pipelines -c parse_item --cbkwargs '{"title": "Test", "price": 10}' "https://books.toscrape.com/"
Pass any URL that your spider handles; it is downloaded even though the
callback ignores it.
Common pitfalls
===============