From 8bbaea9892003769672204a8ff4e989b5aab5ceb Mon Sep 17 00:00:00 2001 From: databender Date: Mon, 16 Aug 2021 16:57:43 +0530 Subject: [PATCH 1/8] updated documentation for python version for reppy --- docs/topics/downloader-middleware.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 80c6c2c37..222dda685 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1073,6 +1073,8 @@ In order to use this parser: * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser`` +* only works with python 3.8 and earlier + .. _rerp-parser: Robotexclusionrulesparser From 1a8b98843aee548a52faa36f5360a81a1624e208 Mon Sep 17 00:00:00 2001 From: databender Date: Mon, 16 Aug 2021 17:00:05 +0530 Subject: [PATCH 2/8] updated documentation for python version for reppy --- docs/topics/downloader-middleware.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 222dda685..2c00ad45d 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1073,7 +1073,7 @@ In order to use this parser: * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser`` -* only works with python 3.8 and earlier +* Only works with python 3.8 and earlier .. _rerp-parser: From cc1cb2de0c6e91393ceb6872c174aa2ac06c07ac Mon Sep 17 00:00:00 2001 From: databender Date: Mon, 16 Aug 2021 17:21:47 +0530 Subject: [PATCH 3/8] updated suggested changes --- docs/topics/downloader-middleware.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 2c00ad45d..fa211fb75 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1068,12 +1068,13 @@ Native implementation, provides better speed than Protego. In order to use this parser: +.. warning:: Does not support Python 3.9+ + * Install `Reppy `_ by running ``pip install reppy`` * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser`` -* Only works with python 3.8 and earlier .. _rerp-parser: From 013ac90f6129a9e8b862d66ca2ffa8f0f1fd674e Mon Sep 17 00:00:00 2001 From: umair ansari Date: Mon, 16 Aug 2021 18:00:06 +0530 Subject: [PATCH 4/8] Update docs/topics/downloader-middleware.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Adrián Chaves --- docs/topics/downloader-middleware.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index fa211fb75..8323bc564 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1068,10 +1068,10 @@ Native implementation, provides better speed than Protego. In order to use this parser: -.. warning:: Does not support Python 3.9+ - * Install `Reppy `_ by running ``pip install reppy`` + .. warning:: Does not support Python 3.9+ + * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser`` From ebddb77a331c6290e64e449ef9847e33b323a146 Mon Sep 17 00:00:00 2001 From: databender Date: Mon, 16 Aug 2021 18:08:26 +0530 Subject: [PATCH 5/8] updated suggested changes after review --- docs/topics/downloader-middleware.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index fa211fb75..4d7c87404 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1068,10 +1068,10 @@ Native implementation, provides better speed than Protego. In order to use this parser: -.. warning:: Does not support Python 3.9+ - * Install `Reppy `_ by running ``pip install reppy`` +.. warning:: Does not support Python 3.9+ + * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser`` From bcf38a67194f25db66334af18bdf49b34c6a0c39 Mon Sep 17 00:00:00 2001 From: databender Date: Wed, 18 Aug 2021 14:48:47 +0530 Subject: [PATCH 6/8] added upstream issue for not supported python version --- docs/topics/downloader-middleware.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 4d7c87404..089d5683a 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1070,7 +1070,9 @@ In order to use this parser: * Install `Reppy `_ by running ``pip install reppy`` -.. warning:: Does not support Python 3.9+ + .. warning:: `Upstream issue #122 + `_ prevents reppy usage in + Python 3.9+. * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser`` From d623ed15d1a79a91b55aa7aae2d942aac94abfdf Mon Sep 17 00:00:00 2001 From: databender Date: Wed, 18 Aug 2021 14:51:03 +0530 Subject: [PATCH 7/8] indentation updated --- docs/topics/downloader-middleware.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 089d5683a..928a59bf1 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1070,9 +1070,9 @@ In order to use this parser: * Install `Reppy `_ by running ``pip install reppy`` - .. warning:: `Upstream issue #122 - `_ prevents reppy usage in - Python 3.9+. +.. warning:: `Upstream issue #122 + `_ prevents reppy usage in + Python 3.9+. * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser`` From 572d347b3bc0149042f04ea83aff4a4f8fc7a831 Mon Sep 17 00:00:00 2001 From: databender Date: Wed, 18 Aug 2021 16:17:52 +0530 Subject: [PATCH 8/8] warning view updated --- docs/topics/downloader-middleware.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 928a59bf1..99d57bda9 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1070,9 +1070,8 @@ In order to use this parser: * Install `Reppy `_ by running ``pip install reppy`` -.. warning:: `Upstream issue #122 - `_ prevents reppy usage in - Python 3.9+. + .. warning:: `Upstream issue #122 + `_ prevents reppy usage in Python 3.9+. * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser``