diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 80c6c2c37..99d57bda9 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -1070,9 +1070,13 @@ In order to use this parser: * Install `Reppy `_ by running ``pip install reppy`` + .. warning:: `Upstream issue #122 + `_ prevents reppy usage in Python 3.9+. + * Set :setting:`ROBOTSTXT_PARSER` setting to ``scrapy.robotstxt.ReppyRobotParser`` + .. _rerp-parser: Robotexclusionrulesparser