diff --git a/docs/topics/commands.rst b/docs/topics/commands.rst index 343193627..50da4593a 100644 --- a/docs/topics/commands.rst +++ b/docs/topics/commands.rst @@ -507,7 +507,7 @@ Supported options: * ``--cbkwargs``: additional keyword arguments that will be passed to the callback. This must be a valid json string. Example: --cbkwargs='{"foo" : "bar"}' -* ``--pipelines``: process items through pipelines +* ``--pipelines``: :ref:`process items through pipelines ` * ``--rules`` or ``-r``: use :class:`~scrapy.spiders.CrawlSpider` rules to discover the callback (i.e. spider method) to use for parsing the diff --git a/docs/topics/item-pipeline.rst b/docs/topics/item-pipeline.rst index 951c0f485..c1313635c 100644 --- a/docs/topics/item-pipeline.rst +++ b/docs/topics/item-pipeline.rst @@ -330,6 +330,36 @@ passes through ``PricePipeline`` before it reaches the :ref:`feed exports .. _books.toscrape.com: https://books.toscrape.com/ +.. _test-item-pipeline: + +Testing an item pipeline +======================== + +To send the items from a single URL through your item pipelines, use the +:command:`parse` command with the ``--pipelines`` option:: + + scrapy parse --pipelines "https://books.toscrape.com/" + +To test specific item data instead, add a callback that builds an item out of +its keyword arguments: + +.. skip: next +.. code-block:: python + + class BooksSpider(scrapy.Spider): + # ... + + def parse_item(self, response, **fields): + yield BookItem(**fields) + +and pass those keyword arguments in the command line:: + + scrapy parse --pipelines -c parse_item --cbkwargs '{"title": "Test", "price": 10}' "https://books.toscrape.com/" + +Pass any URL that your spider handles; it is downloaded even though the +callback ignores it. + + Common pitfalls ===============