RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,363 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

94 Commits

Author SHA1 Message Date
linchiwei123 4dc09f09aa
Update setup.py 2020-08-05 22:23:19 +08:00
Adrián Chaves d29bec60d7
Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest) (#4504) 
* Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest)

* Log a detailed error message to discover why MockServer is not working

* Go for all lines!

* Disable tests based on mitmproxy while running on PyPy

* Fix test_get_func_args for PyPy 3.6+

* Make testPayloadDefaultCiphers work regardless of OpenSSL default ciphers

* Crossing fingers…

* Rename: testPayloadDefaultCiphers → testPayloadDisabledCipher

* Test the PyPy version currently documented as the minimum required version

* Fix the PYPY_VERSION tag

* Update the documentation about supported PyPy versions

* Also test the latest 3.5 Python version with PyPy

* Fix the PYPY_VERSION value for the latest 3.5 version

* Use pinned dependencies for asyncio and PyPy tests against oldest supported Python versions

* Fix PyPy installation for the pypy3-pinned Tox environment

* Try installing Cython

* Maybe PyPy requires lxml 3.6.0?

* install.rst: minor clarification

* lxml 4.0.0 is required on PyPy

* Require setuptools 18.5+

* Revert "Require setuptools 18.5+"

This reverts commit 017ec33ac2.

* Maintain lxml as a dependency if setuptools < 18.5 is used
 2020-07-17 02:19:24 +05:00
Júlio César Batista 38496a00b7
Use the itemlaoders library (#4516) 2020-07-15 12:08:36 +02:00
Eugenio Lacuesta ec98dabfab
Support for dataclass and attrs items (#3881) 2020-06-14 11:45:27 +02:00
Eugenio Lacuesta 8b549392f9
Bump minimum Python version to 3.5.2 (#4615) 2020-06-11 14:53:59 +02:00
Vostretsov Nikita 16f2cb4a83 project URLs in machine-readable format for showing in pypi 2020-03-26 12:57:39 +00:00
Eugenio Lacuesta 0c4e5b68ea
Remove six from requirements and setup files 2019-11-03 12:30:34 -03:00
Andrey Rahmatullin 229e722a03 Initial Python 2 removal (#4091) 2019-10-31 10:46:02 +01:00
Roy 9b47dc6a70 [travis, setup] Adding official python 3.8 support 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 11:24:52 +00:00
Roy Healy 179dc916ff
Update setup.py 
Updating setup.py to remove python 3.8 support for now
 2019-10-27 16:48:20 +00:00
Roy 4e939ca75d [setup.py] Added python 3.8 fields 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-21 18:33:18 +01:00
Vostretsov Nikita c7f2bdfdbe add protego to install_requires 2019-09-10 08:58:52 +00:00
elacuesta 5dbeece8da [MRG+1] Drop py34 support - Update CI envs (#3892) 
* Drop py34 support

* Travis experiments

* More Travis experiments

* Bump Twisted version for py35+ (stretch)

* Remove Debian build

* Remove pinned lxml for Py34

* Fix merge error

* Remove unused tox env

* Add environment with pinned versions for py36

* Bump minimum Twisted version in py27; Envs with pinned versions for py27 and py35

* Add botocore as extra dep for py27 tests

* Update requirements-py2.txt

* Add botocore and Pillow as extra dependencies
 2019-08-07 12:36:52 +05:00
Renne Rocha a25e09ecdd Added constrain on lxml version based on Python version 2019-07-30 23:24:41 -03:00
Adrián Chaves 0da972339b Require Twisted<=19.2.0 for Python 3.4 2019-06-11 14:11:38 +02:00
Mikhail Korobov cdd04dfb1d declare Python 3.7 support in setup.py 2018-12-26 13:13:49 +05:00
Mikhail Korobov 25ac4691b4 require parsel 1.5+ 2018-09-12 17:57:27 +05:00
Eugenio Lacuesta 0d87e77afe Bump parsel dependency 2018-02-08 15:03:57 -03:00
Mikhail Korobov 1058169f0e setup.py: mention that we support PyPy. See GH-2213. 2017-12-25 15:31:07 +05:00
Hugo 44623687ab Drop support for EOL Python 3.3 2017-12-19 17:59:05 +02:00
Aditya 9d9d83a8c3 Use https links wherever possible 2017-10-28 16:24:40 +05:30
Konstantin Lopuhin b0a9236357 Use environment markers for custom PyPy requirements 2017-06-19 19:16:50 +03:00
Mikhail Korobov e1ceaf3b5f require w3lib 1.17+ 2017-02-15 00:32:44 +05:00
Paul Tremberth 1295c17a26 Bump parsel requirement to at least parsel v1.1 2017-02-02 17:47:22 +01:00
Paul Tremberth b3406677b9 Update classifiers in setup.py 2017-01-09 14:40:02 +01:00
Paul Tremberth 78d34cc44c Merge pull request #2454 from crasker/fix/low_parsel_version_number 
[MRG+1] Fix/low parsel version number
 2016-12-19 17:11:17 +01:00
zhouyc 2b3abdb700 update parsel version which will cause attribute error 2016-12-19 16:42:31 +08:00
nyov c8cf1a303d Bump Twisted dependency to 13.1.0 (released June 2013) 2016-12-01 22:02:11 +00:00
Ashish Kulkarni bb3b806467 Use w3lib.url.canonicalize_url() from w3lib 1.15.0 
Also remove code/imports which are now unused due to this change.

fixes #2157
 2016-08-16 17:42:16 +05:30
Paul Tremberth c8bff23751 Bump w3lib version dependency in setup.py 2016-04-29 10:29:37 +02:00
nyov 3787fec460 Remove duplicate code now handled by newer w3lib 
see f3029a6a10
 2016-03-30 14:58:03 +00:00
Mikhail Korobov 17d3bec699 declare Python 3 support in setup.py 2016-03-01 16:34:13 +05:00
Elias Dorneles 26ebccd37a upgrade parsel and use its function to instantiate root for finding form 2015-08-11 14:09:34 -03:00
Elias Dorneles 2fe6d128f5 upgrade parsel and using promoted root attribute 2015-08-11 14:09:16 -03:00
Elias Dorneles 17d7347a36 update minimal parsel version, add deprecated classes for csstranslator modules, fix test 2015-08-11 14:09:00 -03:00
Elias Dorneles ce21884a97 migrating scrapy Selector to use Parsel 2015-08-11 14:08:43 -03:00
Mikhail Korobov 75d3d03382 add service_identity to scrapy install_requires 2015-08-11 20:14:22 +05:00
Daniel Graña 776064aee1 PY3 port signals 2015-07-25 18:04:55 +02:00
Elias Dorneles 3d3633f3d2 favoring web scraping over screen scraping in the descriptions 2015-04-25 11:20:20 -03:00
Pablo Hoffman 5846d6154c emphasize web crawling over screen scraping on scrapy description. closes #586 2015-03-18 00:24:16 -03:00
Daniel Graña 0254f58763 Merge branch 'moderm-setuppy' 2014-08-08 18:09:43 -03:00
Daniel Graña 029c51acef There is a trove classifier for Scrapy framework! 
Added by https://bitbucket.org/pypa/pypi/issue/179
 2014-08-05 14:43:13 -03:00
Paul Tremberth 480cfa1994 Update w3lib requirement to 1.8.0 2014-08-04 16:01:28 +02:00
Daniel Graña fcd34b6561 set zip_safe=False 2014-08-01 16:26:08 -03:00
Daniel Graña 1fc4e59cf4 do not ship tests package 2014-08-01 16:23:37 -03:00
Daniel Graña a8f45dc6dd Modernize setup.py 2014-08-01 15:43:33 -03:00
Daniel Graña bfe02b019e Revert "limit Twisted support to pre-14.0.0 while #718 is fixed" 
This reverts commit 65f69e1609.
 2014-06-25 04:22:18 -03:00
Daniel Graña 65f69e1609 limit Twisted support to pre-14.0.0 while #718 is fixed 2014-06-02 13:27:20 -03:00
alexanderlukanin13 ab6496647c Added "six>=1.5.2" to requirements 2014-01-15 13:05:00 +06:00
Daniel Graña cfe588103c include_package_data is required to build wheels from published sources 2013-11-28 16:25:57 -02:00