RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Make architecture overview references a little more clear on the docs 

Expliciting what actually happens by adding links to the respective methods
that are invoked in each processing phase.
This commit is contained in:
Jose Ricardo 2016-10-18 11:48:58 -02:00
parent bebcd5081c
commit ea7bd39529
1 changed files with 8 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
docs/topics/architecture.rst
View File

@ -41,25 +41,26 @@ this:
4. The :ref:`Engine <component-engine>` sends the Requests to the
:ref:`Downloader <component-downloader>`, passing through the
:ref:`Downloader Middleware <component-downloader-middleware>`
(requests direction).
:ref:`Downloader Middlewares <component-downloader-middleware>` (see
:meth:`~scrapy.downloadermiddlewares.DownloaderMiddleware.process_request`).
5. Once the page finishes downloading the
:ref:`Downloader <component-downloader>` generates a Response (with
that page) and sends it to the Engine, passing through the
:ref:`Downloader Middleware <component-downloader-middleware>`
(response direction).
:ref:`Downloader Middlewares <component-downloader-middleware>` (see
:meth:`~scrapy.downloadermiddlewares.DownloaderMiddleware.process_response`).
6. The :ref:`Engine <component-engine>` receives the Response from the
:ref:`Downloader <component-downloader>` and sends it to the
:ref:`Spider <component-spiders>` for processing, passing
through the :ref:`Spider Middleware <component-spider-middleware>`
(input direction).
through the :ref:`Spider Middleware <component-spider-middleware>` (see
:meth:`~scrapy.spidermiddlewares.SpiderMiddleware.process_spider_input`).
7. The :ref:`Spider <component-spiders>` processes the Response and returns
scraped items and new Requests (to follow) to the
:ref:`Engine <component-engine>`, passing through the
:ref:`Spider Middleware <component-spider-middleware>` (output direction).
:ref:`Spider Middleware <component-spider-middleware>` (see
:meth:`~scrapy.spidermiddlewares.SpiderMiddleware.process_spider_output`).
8. The :ref:`Engine <component-engine>` sends processed items to
:ref:`Item Pipelines <component-pipelines>`, then send processed Requests to