RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

10393 Commits

Author SHA1 Message Date
Andrey Rakhmatullin fa8de2d396 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-18 23:06:06 +05:00
Andrey Rakhmatullin 40eab1d473 Drop a duplicate import. 2021-02-18 19:56:12 +05:00
Andrey Rahmatullin fa3ebb1ced
Merge pull request #4987 from wRAR/asyncdef-tests 
Making sure we can run async def tests
 2021-02-18 18:34:25 +05:00
Andrey Rakhmatullin 58706c6598 Merge branch 'asyncdef-tests' into asyncio-parse-asyncgen-proper-rebased 2021-02-18 17:20:25 +05:00
Andrey Rakhmatullin 49af7c4c8b Drop pytest-twisted, use Scrapy code to install the reactor. 2021-02-18 17:10:30 +05:00
Eugenio Lacuesta 4418f78941
Simplify check for negotiated protocol 
negotiatedProtocol's type is Optional[bytes]

See https://github.com/twisted/twisted/blob/twisted-20.3.0/src/twisted/protocols/tls.py#L563-L587
and https://www.pyopenssl.org/en/20.0.1/api/ssl.html#OpenSSL.SSL.Connection.get_alpn_proto_negotiated

Note that OpenSSL.SSL.Connection.get_next_proto_negotiated is deprecated:
https://www.pyopenssl.org/en/20.0.0/changelog.html#backward-incompatible-changes
 2021-02-17 18:36:52 -03:00
Eugenio Lacuesta e80f37bd3f
Test http2 agent for unsupported scheme 2021-02-17 16:34:29 -03:00
Eugenio Lacuesta ac82a4a8a0
Merge remote-tracking branch 'upstream/master' into http2 2021-02-17 16:33:57 -03:00
Andrey Rakhmatullin abbbfbbb38 Add tests for deferred_f_from_coro_f. 2021-02-12 22:41:16 +05:00
Andrey Rakhmatullin a7557f2a49 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-11 16:09:14 +05:00
Adrián Chaves ea92b49aaa
Merge pull request #4986 from wRAR/uvloop-py36 
Skip uvloop 0.15.0+ on py36.
 2021-02-11 12:07:28 +01:00
Andrey Rakhmatullin 54fd371481 Skip uvloop 0.15.0+ on py36. 2021-02-11 14:24:11 +05:00
Andrey Rakhmatullin 67cff0e8a9 Silence pylint "naked raise" error. 2021-02-10 22:44:14 +05:00
Adrián Chaves 15b501c089 Do not force string interpolation while logging 2021-02-10 18:10:57 +01:00
Andrey Rakhmatullin 1cd953e1b0 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-09 13:28:36 +05:00
Adrián Chaves 9ac5b1d021 Adjust test constraints 2021-02-09 09:13:25 +01:00
Adrián Chaves bb72bba178 tox: apply upper constraints to all non-pinned package installations 2021-02-09 09:13:25 +01:00
Adrián Chaves 45345ba6b5 Use constraints.txt to limit pip resolver backtracking 2021-02-09 09:13:25 +01:00
Adrián Chaves de82ca8547
Merge pull request #4982 from wRAR/refactor-spidermiddlewaremanager 
Refactor SpiderMiddlewareManager.scrape_response.
 2021-02-08 21:27:21 +01:00
Andrey Rakhmatullin 1e9b52c3e0 Refactor SpiderMiddlewareManager.scrape_response. 2021-02-08 22:02:03 +05:00
Andrey Rakhmatullin 5cf403295d Remove a duplicate definition. 2021-02-05 19:40:14 +05:00
Adrián Chaves 8527b53e14 Revert "Use --use-deprecated=legacy-resolver" 
This reverts commit 7b11b74c77.
 2021-02-05 13:06:27 +01:00
Adrián Chaves 7afcd634ab Remove unused import 2021-02-05 13:04:54 +01:00
Andrey Rakhmatullin 58f8481301 Update docs. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 2152a2a508 Add main infrastructure for async callbacks. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 2a1e9359ca Add parallel_async. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 92f2c9e308 Move spider middlewares to process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d66d52d3ed Add process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d658552f23 Add only_not_asyncio. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 7e9f498e00 Add MutableAsyncChain. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin acff1eb496 Add aiter_errback. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 76abcedaf4 Add as_async_generator. 2021-02-05 16:21:56 +05:00
Adrián Chaves 7b11b74c77 Use --use-deprecated=legacy-resolver 
Let’s see how test results change
 2021-02-04 11:08:01 +01:00
Adrián Chaves 0e4b291701 HTTP/2: fix canceling a request before a connection has been established 2021-02-03 21:28:04 +01:00
Adrián Chaves 248800328c Fix test_pinned_twisted_version 2021-02-03 21:13:43 +01:00
Adrián Chaves 4c801551fa Document that the bytes_received signal is not yet implemented for HTTP/2 2021-02-03 21:11:46 +01:00
Adrián Chaves 1773eaf5dc Move lists closer to their introducing paragraph 2021-02-03 11:43:18 +01:00
Adrián Chaves 1a7bde0d8e Document that HTTP/2 server pushes are ignored 2021-02-03 10:55:11 +01:00
Adrián Chaves c8d8b180bf Merge remote-tracking branch 'upstream/master' into http2 2021-02-03 09:37:35 +01:00
Adrián Chaves 536e749ecc HTTP/2: remove verbose protocol-handling logging 2021-02-03 09:22:02 +01:00
Adrián Chaves d1024566d8 setup.py: Twisted → Twisted[http2] 2021-02-03 09:13:45 +01:00
Adrián Chaves 2ce8e0c742 Document the (hard-coded) maximum HTTP/2 frame size accepted from servers 2021-02-03 09:09:53 +01:00
Mikhail Korobov 904a50138b
Merge pull request #4973 from Gallaecio/zyte 
Scrapinghub → Zyte
 2021-02-03 01:10:52 +05:00
Adrián Chaves f30f53b3cc Scrapinghub → Zyte 2021-02-02 15:03:20 +01:00
Mikhail Korobov 28262d4b24
Merge pull request #4956 from Gallaecio/readme-contributors 
Mention contributors in the README
 2021-01-14 19:58:52 +05:00
Adrián Chaves 6e7ae789f9
Reuse the text from https://scrapy.org/ 2021-01-14 11:59:38 +01:00
M Ikram Ullah Khan 0a1e2fefab
Docs: fix typo in news.rst (#4942) 2021-01-04 10:30:23 -03:00
Eugenio Lacuesta 80db569aea
Migrate CI to GitHub actions (#4924) 2021-01-01 19:13:39 -03:00
Eugenio Lacuesta e494a3f733
protocol attribute for h2 responses 2020-12-31 11:50:15 -03:00
Eugenio Lacuesta d698b5147b
Merge branch 'master' into http2 2020-12-31 11:13:25 -03:00