From b30ca379b6785c7ceb75e12285fe7865b4f607d1 Mon Sep 17 00:00:00 2001 From: Eugenio Lacuesta Date: Wed, 13 Mar 2019 11:02:51 +0000 Subject: [PATCH] Rule.process_request: docs --- docs/topics/spiders.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst index 742a88659..24b6f7ec9 100644 --- a/docs/topics/spiders.rst +++ b/docs/topics/spiders.rst @@ -402,10 +402,12 @@ Crawling rules of links extracted from each response using the specified ``link_extractor``. This is mainly used for filtering purposes. - ``process_request`` is a callable, or a string (in which case a method from - the spider object with that name will be used) which will be called with - every request extracted by this rule, and must return a request or None (to - filter out the request). + ``process_request`` is a callable (or a string, in which case a method from + the spider object with that name will be used) which will be called for + every request extracted by this rule. This callable should take a Request object + as first positional argument and, optionally, the Response object from which the + Request originated as second positional argument. It must return a request or None + (to filter out the request). CrawlSpider example ~~~~~~~~~~~~~~~~~~~