RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,814 Commits 50 Branches 132 Tags 142 MiB
Commit Graph

171 Commits

Author SHA1 Message Date
Eugenio Lacuesta d8d1dc5b50
Ignore typing warning in test 2021-04-12 10:43:02 -03:00
Eugenio Lacuesta 1d200258a5
Adjust h2 version requirement (#5066) 2021-03-26 14:45:26 +01:00
Adrián Chaves 308a58aa27
Update CI to support Twisted 21.2.0 (#5027) 2021-03-19 18:39:44 +01:00
Adrián Chaves a390b934de Do not install mitmproxy in Python 3.9 2021-03-18 23:53:58 +01:00
Adrián Chaves 1d83697990
Merge pull request #4769 from scrapy/http2 
Implement experimental HTTP/2 support
 2021-03-18 22:18:12 +01:00
vinayak 42e4dbb23d
Support Python 3.9 (#4759) 
* Update .travis.yml

* Update .travis.yml

* updage travis.yml

* Make 3.9 support official

* Upgrade mitmproxy for Python 3.9

* Restore the Pylint job

* Undo unintended change to mitmproxy requirement

* Enable Python 3.9 in GitHub Actions

* Work around reppy’s Python version limitation

* Disable tests in Windows / Python 3.9 due to a Twisted bug

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-03-18 09:40:03 -03:00
Adrián Chaves 5b2d3e17a3
Merge branch 'master' into http2 2021-03-09 14:59:17 +01:00
Adrián Chaves 4fe26ae970 Limit tests to Twisted < 21 2021-03-02 19:46:34 +01:00
Adrián Chaves bb72bba178 tox: apply upper constraints to all non-pinned package installations 2021-02-09 09:13:25 +01:00
Adrián Chaves 45345ba6b5 Use constraints.txt to limit pip resolver backtracking 2021-02-09 09:13:25 +01:00
Adrián Chaves 8527b53e14 Revert "Use --use-deprecated=legacy-resolver" 
This reverts commit 7b11b74c77.
 2021-02-05 13:06:27 +01:00
Adrián Chaves 7b11b74c77 Use --use-deprecated=legacy-resolver 
Let’s see how test results change
 2021-02-04 11:08:01 +01:00
Adrián Chaves c8d8b180bf Merge remote-tracking branch 'upstream/master' into http2 2021-02-03 09:37:35 +01:00
Eugenio Lacuesta 80db569aea
Migrate CI to GitHub actions (#4924) 2021-01-01 19:13:39 -03:00
Eugenio Lacuesta d698b5147b
Merge branch 'master' into http2 2020-12-31 11:13:25 -03:00
Adrián Chaves 798a818caf Move apt-get command from Tox to Travis CI 2020-12-21 13:35:40 +01:00
Adrián Chaves 0dff5781bc Blind attempt to fix the build of the cryptography-provided OpenSSL 2020-12-21 11:13:14 +01:00
Adrián Chaves 0567fdc56e Merge remote-tracking branch 'upstream/master' into tox-pip20.2 2020-12-21 11:02:05 +01:00
Adrián Chaves f6879c681e SCRAPY_PINNED → _SCRAPY_PINNED 2020-11-23 23:53:03 +01:00
Adrián Chaves 0dc3e6350c Add a test to check the Twisted version in pinned environments 2020-11-23 22:10:45 +01:00
Adrián Chaves 6ef3dc2029 Use the new pip resolver for Tox environments with pinned dependencies 2020-11-17 22:28:20 +01:00
Adrián Chaves 034d61e6cb Restrict pip’s --use-feature=2020-resolver to the extra-deps environment 2020-11-11 16:46:03 +01:00
Adrián Chaves 7327145bf3 Remove mitmproxy from pinned environments 2020-10-30 21:34:15 +01:00
Adrián Chaves 3e5bc77737 Remove pinning from the PyPy job 2020-10-30 10:31:33 +01:00
Adrián Chaves 13bcdc9f88 Restore pinned dependencies in tox.ini 2020-10-30 10:28:42 +01:00
dswij 9461414b14 minor changes to remove unnecessary lines 2020-10-07 11:26:53 +07:00
Adrián Chaves 269fe35d6d
Merge branch 'master' into http2 2020-10-06 21:05:10 +02:00
dswij 9f02df20c5 Remove PyDispatcher from general requirements 2020-10-07 01:10:23 +07:00
dswij ce6884d517
Update tox.ini 2020-10-06 19:51:42 +07:00
dswij 371bb80868 Explicitly declare PyDispatcher as dependencies 2020-10-06 19:45:05 +07:00
dswij 95b2e94496 fix comment error on tox and re-add cache for travis 2020-10-02 01:05:45 +07:00
dswij cc81f9ed06 add download setting for tox 2020-10-02 00:56:59 +07:00
dswij f7201b1427 travis and deps 2020-10-01 19:59:23 +07:00
dswij 0ea6ff1136 travis pip version and deps 2020-10-01 16:14:36 +07:00
dswij 66201737a0 fix travis and deps 2020-10-01 15:47:19 +07:00
dswij f4629fe2cc Update travis-pip and tox deps conflict for pip20.2 2020-10-01 14:58:14 +07:00
dswij 9186e5a686 add pip 20.2 test for tox 2020-09-29 00:39:36 +07:00
Adrián Chaves 07c1d9c25b Remove boto3 as a dependency for tests 2020-09-21 23:32:55 +02:00
Adrián Chaves 8a3ba34a75 Merge remote-tracking branch 'upstream/master' into http2 2020-08-31 16:43:32 +02:00
Adrián Chaves 195f738bba
Update Python version references after dropping support for 3.5 (#4742) 
* Update Python version references after dropping support for 3.5

* Remove outdated test

* Undo change affecting collect_asyncgen

* Undo change to be handled by #4743

* Remove unused import

* Remove unused import

* Update tests/requirements-py3.txt

Co-authored-by: Mikhail Korobov <kmike84@gmail.com>

Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
 2020-08-27 15:43:43 +05:00
Aditya 5e36f539e2 chore: remove typing-extensions dependency 2020-08-27 15:12:22 +05:30
Aditya 1432161477 fix: bump min typing-extensions version to 3.7.4 
- typing-extensions>=3.7.4 only supports TypedDict
 2020-08-24 15:40:01 +05:30
Aditya 26d344b762 Merge branch 'http2' of https://github.com/scrapy/scrapy into h2-client-protocol 2020-08-24 14:58:23 +05:30
Eugenio Lacuesta f1250177dc
Remove Python 3.5 from CI (#4743) 2020-08-22 09:33:35 +02:00
Aditya f9f008e935 test: add typing-extensions 2020-08-16 17:04:40 +05:30
Aditya d97cf973dd Merge branch 'master' of https://github.com/scrapy/scrapy into h2-client-protocol 2020-08-16 11:31:10 +05:30
Adrián Chaves aefd43a6c6 Upgrade minimum dependencies for Python 3.6 support 2020-08-11 12:52:54 +02:00
Mikhail Korobov 5e2d1bd187
Merge pull request #4434 from BroodingKangaroo/ISSUE-4250-add_batch_deliveries 
Feed exports: add batch deliveries
 2020-07-29 23:03:28 +05:00
Adrián Chaves d29bec60d7
Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest) (#4504) 
* Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest)

* Log a detailed error message to discover why MockServer is not working

* Go for all lines!

* Disable tests based on mitmproxy while running on PyPy

* Fix test_get_func_args for PyPy 3.6+

* Make testPayloadDefaultCiphers work regardless of OpenSSL default ciphers

* Crossing fingers…

* Rename: testPayloadDefaultCiphers → testPayloadDisabledCipher

* Test the PyPy version currently documented as the minimum required version

* Fix the PYPY_VERSION tag

* Update the documentation about supported PyPy versions

* Also test the latest 3.5 Python version with PyPy

* Fix the PYPY_VERSION value for the latest 3.5 version

* Use pinned dependencies for asyncio and PyPy tests against oldest supported Python versions

* Fix PyPy installation for the pypy3-pinned Tox environment

* Try installing Cython

* Maybe PyPy requires lxml 3.6.0?

* install.rst: minor clarification

* lxml 4.0.0 is required on PyPy

* Require setuptools 18.5+

* Revert "Require setuptools 18.5+"

This reverts commit 017ec33ac2.

* Maintain lxml as a dependency if setuptools < 18.5 is used
 2020-07-17 02:19:24 +05:00
Adrián Chaves c4f9250220
Merge branch 'master' into ISSUE-4250-add_batch_deliveries 2020-07-16 17:44:18 +02:00