Merge pull request #5231 from umairnsr87/umairnsr87/issues/5226

updated documentation for python version for reppy
This commit is contained in:
Andrey Rahmatullin 2021-08-18 21:08:32 +05:00 committed by GitHub
commit bdf8355ce1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 0 deletions

View File

@ -1070,9 +1070,13 @@ In order to use this parser:
* Install `Reppy <https://github.com/seomoz/reppy/>`_ by running ``pip install reppy``
.. warning:: `Upstream issue #122
<https://github.com/seomoz/reppy/issues/122>`_ prevents reppy usage in Python 3.9+.
* Set :setting:`ROBOTSTXT_PARSER` setting to
``scrapy.robotstxt.ReppyRobotParser``
.. _rerp-parser:
Robotexclusionrulesparser