mirror of https://github.com/scrapy/scrapy.git
Address feedback.
This commit is contained in:
parent
51410512b7
commit
219c2cd0c6
|
|
@ -210,8 +210,9 @@ Common use cases for asynchronous code include:
|
|||
* delaying the spider initialization until some external event (in the
|
||||
:signal:`spider_opened` handler);
|
||||
* calling asynchronous Scrapy methods like
|
||||
:meth:`scrapy.core.engine.ExecutionEngine.download_async` (see
|
||||
:ref:`the screenshot pipeline example<ScreenshotPipeline>`).
|
||||
:meth:`ExecutionEngine.download_async()
|
||||
<scrapy.core.engine.ExecutionEngine.download_async>` (see :ref:`the
|
||||
screenshot pipeline example <ScreenshotPipeline>`).
|
||||
|
||||
.. _aio-libs: https://github.com/aio-libs
|
||||
|
||||
|
|
|
|||
|
|
@ -591,7 +591,7 @@ In order to use your storage backend, set:
|
|||
HTTPCache middleware settings
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The :class:`~scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware` can be
|
||||
:class:`~scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware` can be
|
||||
configured through the following settings:
|
||||
|
||||
.. setting:: HTTPCACHE_ENABLED
|
||||
|
|
@ -1015,9 +1015,6 @@ RetryMiddleware
|
|||
A middleware to retry failed requests that are potentially caused by
|
||||
temporary problems such as a connection timeout or HTTP 500 error.
|
||||
|
||||
Failed pages are rescheduled with the priority adjusted according to
|
||||
:setting:`RETRY_PRIORITY_ADJUST`.
|
||||
|
||||
.. reqmeta:: dont_retry
|
||||
|
||||
If :attr:`Request.meta <scrapy.Request.meta>` has ``dont_retry`` key
|
||||
|
|
|
|||
|
|
@ -41,11 +41,10 @@ this:
|
|||
2. The item is returned from the spider and goes to the item pipeline.
|
||||
|
||||
3. When the item reaches the :class:`FilesPipeline`, the URLs in the
|
||||
``file_urls`` field are scheduled for download using the standard
|
||||
Scrapy downloader (which means the downloader middlewares are used, but
|
||||
the spider middlewares aren't). The item remains "locked" at that particular
|
||||
pipeline stage until the files have finished downloading (or failed for some
|
||||
reason).
|
||||
``file_urls`` field are downloaded using the standard Scrapy downloader
|
||||
(which means the downloader middlewares are used, but the spider middlewares
|
||||
aren't). The item remains "locked" at that particular pipeline stage until
|
||||
the files have finished downloading (or failed for some reason).
|
||||
|
||||
4. When the files are downloaded, another field (``files``) will be populated
|
||||
with the results. This field will contain a list of dicts with information
|
||||
|
|
|
|||
Loading…
Reference in New Issue