From 7d68b084a4a0a63d5769c51c5cde5e9c1108ddb9 Mon Sep 17 00:00:00 2001 From: Mikhail Korobov Date: Tue, 23 Sep 2014 02:59:35 +0600 Subject: [PATCH] DOC document download_timeout Request.meta key and download_timeout spider attribute. --- docs/topics/downloader-middleware.rst | 9 ++++++++- docs/topics/request-response.rst | 10 ++++++++++ docs/topics/settings.rst | 6 ++++++ 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index e4e490f61..e96ec8852 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -263,7 +263,14 @@ DownloadTimeoutMiddleware .. class:: DownloadTimeoutMiddleware This middleware sets the download timeout for requests specified in the - :setting:`DOWNLOAD_TIMEOUT` setting. + :setting:`DOWNLOAD_TIMEOUT` setting or :attr:`download_timeout` + spider attribute. + +.. note:: + + You can also set download timeout per-request using + :reqmeta:`download_timeout` Request.meta key; this is supported + even when DownloadTimeoutMiddleware is disabled. HttpAuthMiddleware ------------------ diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rst index b3e9d9a1d..68d87d04f 100644 --- a/docs/topics/request-response.rst +++ b/docs/topics/request-response.rst @@ -229,6 +229,7 @@ Those are: * :reqmeta:`redirect_urls` * :reqmeta:`bindaddress` * :reqmeta:`dont_obey_robotstxt` +* :reqmeta:`download_timeout` .. reqmeta:: bindaddress @@ -237,6 +238,15 @@ bindaddress The IP of the outgoing IP address to use for the performing the request. +.. reqmeta:: download_timeout + +download_timeout +---------------- + +The amount of time (in secs) that the downloader will wait before timing out. +See also: :setting:`DOWNLOAD_TIMEOUT`. + + .. _topics-request-response-ref-request-subclasses: Request subclasses diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index 89ee7605a..9000f0242 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -416,6 +416,12 @@ Default: ``180`` The amount of time (in secs) that the downloader will wait before timing out. +.. note:: + + This timeout can be set per spider using :attr:`download_timeout` + spider attribute and per-request using :reqmeta:`download_timeout` + Request.meta key. + .. setting:: DUPEFILTER_CLASS DUPEFILTER_CLASS