From 9fb0f8454eed974f5d2ed1808142169364145348 Mon Sep 17 00:00:00 2001 From: Eugenio Lacuesta Date: Tue, 9 Jul 2019 15:30:22 -0300 Subject: [PATCH] Extend docs about Crawling Rules --- docs/topics/spiders.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst index 84a3a8fbe..869a61441 100644 --- a/docs/topics/spiders.rst +++ b/docs/topics/spiders.rst @@ -386,7 +386,9 @@ Crawling rules the specified link extractor. This callback receives a :class:`~scrapy.http.Response` as its first argument and must return either a single instance or an iterable of :class:`~scrapy.item.Item`, ``dict`` and/or :class:`~scrapy.http.Request` objects - (or any subclass of them). + (or any subclass of them). As mentioned above, the received :class:`~scrapy.http.Response` + object will contain the text of the link that produced the :class:`~scrapy.http.Request` + in its ``meta`` dictionary (under the ``link_text`` key) .. warning:: When writing crawl spider rules, avoid using ``parse`` as callback, since the :class:`CrawlSpider` uses the ``parse`` method