* Update project.py removed one 'hack', seems irrelevant.
As mentioned by @Gallaecio in issue #3871, the 'hack' is cleared. I also double checked whether the environment variable "SCRAPY_PICKLED_SETTINGS_TO_OVERRIDE" was ever set in our codebase and it turns out we didn't set it or used it anywhere else.So I guess the 'hack' was not used in the current version. Also the name of this environment variable rather doesn't suggest it was a boolean(it is used in an 'if' condition which has perplexed me )
* Update project.py
* Update project.py
How about this?
* Update project.py
* Update project.py
* Update scrapy/utils/project.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Update scrapy/utils/project.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Update project.py
* Drop py34 support
* Travis experiments
* More Travis experiments
* Bump Twisted version for py35+ (stretch)
* Remove Debian build
* Remove pinned lxml for Py34
* Fix merge error
* Remove unused tox env
* Add environment with pinned versions for py36
* Bump minimum Twisted version in py27; Envs with pinned versions for py27 and py35
* Add botocore as extra dep for py27 tests
* Update requirements-py2.txt
* Add botocore and Pillow as extra dependencies
This should be the proper ordering.This is the explanation.
If 'not portrange' is True ,it is guaranteed that `not hasattr(portrange, '__iter__')` is also True the converse of this is not always true.(for example, consider portrange=None, for such case we were executing the logic for `not hasattr(portrange, '__iter__')` . ).Such case is eliminated by this PR.
Make the robots.txt parser configurable through the new ROBOTSTXT_PARSER setting, support the Reppy and Robotexclusionrulesparser parsers, and allow implementing custom robots.txt parsers.
* Update _monkeypatches.py
The workarounds are not required assuming the bugs regarding urlparse are absent in Python versions >2.7. We already exit the program if Python version<2.7 in the __init__.py(line 17).The monkeypatches are deployed after this check at line 27 in the __init__.py .
* Update _monkeypatches.py
Added the second workaround.
* Update _monkeypatches.py
* Update _monkeypatches.py
OmarFarrag
Mikhail Korobov
Anubhav Patel
Tobias Hernstig
Tobias Hernstig
Andrey Rakhmatullin
Eugenio Lacuesta
Shivam Sandbhor
Adrián Chaves
Marc Hernández
elacuesta
Pengyu Chen
tpeng
Daniel Graña
Renne Rocha