From 7d8f922df987e238d009e14f410888da460add93 Mon Sep 17 00:00:00 2001 From: Pablo Hoffman Date: Mon, 18 Oct 2010 22:36:30 -0200 Subject: [PATCH] Added documentation for CLOSESPIDER_ERRORCOUNT setting. Refs #254 --- docs/topics/extensions.rst | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/docs/topics/extensions.rst b/docs/topics/extensions.rst index e30c389e4..8452d486b 100644 --- a/docs/topics/extensions.rst +++ b/docs/topics/extensions.rst @@ -276,7 +276,12 @@ Closes a spider automatically when some conditions are met, using a specific closing reason for each condition. The conditions for closing a spider can be configured through the following -settings. Other conditions will be supported in the future. +settings: + +* :setting:`CLOSESPIDER_TIMEOUT` +* :setting:`CLOSESPIDER_ITEMPASSED` +* :setting:`CLOSESPIDER_PAGECOUNT` +* :setting:`CLOSESPIDER_ERRORCOUNT` .. setting:: CLOSESPIDER_TIMEOUT @@ -305,7 +310,9 @@ non set), spiders won't be closed by number of passed items. .. setting:: CLOSESPIDER_PAGECOUNT CLOSESPIDER_PAGECOUNT -"""""""""""""""""""""" +""""""""""""""""""""" + +.. versionadded:: 0.11 Default: ``0`` @@ -314,8 +321,20 @@ crawls more than that, the spider will be closed with the reason ``closespider_pagecount``. If zero (or non set), spiders won't be closed by number of crawled responses. +.. setting:: CLOSESPIDER_ERRORCOUNT + +CLOSESPIDER_ERRORCOUNT +"""""""""""""""""""""" + .. versionadded:: 0.11 +Default: ``0`` + +An integer which specifies the maximum number of errors to receive before +closing the spider. If the spider generates more than that number of errors, +it will be closed with the reason ``closespider_errorcount``. If zero (or non +set), spiders won't be closed by number of errors. + StatsMailer extension ~~~~~~~~~~~~~~~~~~~~~