From 7c797968a31ecacc24e17b81e57aad44de803389 Mon Sep 17 00:00:00 2001 From: Adrian Date: Sun, 9 Aug 2026 13:53:33 +0200 Subject: [PATCH] Docs: clarify the handling of exceptions raised in errbacks (#7898) --- docs/topics/request-response.rst | 4 ++++ docs/topics/spider-middleware.rst | 5 +++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rst index 75158440b..f810146e4 100644 --- a/docs/topics/request-response.rst +++ b/docs/topics/request-response.rst @@ -770,6 +770,10 @@ is raise while processing it. It receives a :exc:`~twisted.python.failure.Failure` as first parameter and can be used to track connection establishment timeouts, DNS errors etc. +If an errback raises an exception, Scrapy logs it and sends the +:signal:`spider_error` signal, unless the exception is the one that the errback +received, which Scrapy logs as a download error instead. + Here's an example spider logging all errors and catching some specific errors if needed: diff --git a/docs/topics/spider-middleware.rst b/docs/topics/spider-middleware.rst index 78b211dac..db85906c1 100644 --- a/docs/topics/spider-middleware.rst +++ b/docs/topics/spider-middleware.rst @@ -145,8 +145,9 @@ one or more of these methods: .. method:: process_spider_exception(response, exception) - This method is called when a spider or :meth:`process_spider_output` - method (from a previous spider middleware) raises an exception. + This method is called when a spider callback or a + :meth:`process_spider_output` method (from a previous spider + middleware) raises an exception. :meth:`process_spider_exception` should return either ``None`` or an iterable of :class:`~scrapy.Request` or :ref:`item `