RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,710 Commits 49 Branches 132 Tags 144 MiB
Commit Graph

10710 Commits

Author SHA1 Message Date
Adrián Chaves 386e2a51ae tests/test_downloader_handlers_http2.py: fix style issue 2021-02-24 10:41:01 +01:00
Adrián Chaves 12064d799b HTTP/2: improve header handling 2021-02-24 10:37:38 +01:00
Wehzie a36f952198
fixed typo "an quotes.json" -> "a quotes.json" (#5005) 2021-02-24 08:15:44 +01:00
Adrián Chaves 5101094207 HTTP/2: test a CONNECT request 2021-02-24 07:33:39 +01:00
Adrián Chaves 5ba31cd1a2 HTTP/2 stream close reason handling: Use else + assert instead of elif 2021-02-23 11:57:33 +01:00
Adrián Chaves bd29f32dee HTTP/2: do not make conn_lost_deferred optional 2021-02-23 06:42:28 +01:00
Adrián Chaves 7605f19ec4 HTTP/2: test 2 concurrent requests to the same domain 2021-02-23 05:54:48 +01:00
Eugenio Lacuesta 6326178bc5
http2: acceptable protocol update, tests (#4994) 2021-02-22 16:50:51 +01:00
Adrián Chaves 1f7665c4cf Silence a PyLint check on a mistake made for testing purposes 2021-02-22 16:48:10 +01:00
Adrián Chaves 722a33a2ac Fix style issues 2021-02-22 16:42:38 +01:00
Adrián Chaves 80f5003c88 Add tests for get_retry_request 2021-02-22 16:38:38 +01:00
Adrián Chaves 6ab990181c Document get_retry_requests 2021-02-22 14:48:03 +01:00
Adrián Chaves ec836dcc92 Solve style issues 2021-02-22 14:15:28 +01:00
Adrián Chaves 3f5a1956d8 Merge remote-tracking branch 'upstream/master' into retry-request 2021-02-22 14:10:26 +01:00
Adrián Chaves 825462615a get_retry_request: set the default retry reason to “unspecified” 2021-02-22 14:09:48 +01:00
Adrián Chaves 5fc27b1e6f Remove RetrySpiderMixin and retry_request 2021-02-22 14:09:06 +01:00
Andrey Rakhmatullin f9a5385146 Revert "Move spider middlewares to process_iterable_helper." 
This reverts commit 92f2c9e308.
 2021-02-19 21:19:21 +05:00
Andrey Rakhmatullin fa8de2d396 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-18 23:06:06 +05:00
Andrey Rakhmatullin 40eab1d473 Drop a duplicate import. 2021-02-18 19:56:12 +05:00
Andrey Rahmatullin fa3ebb1ced
Merge pull request #4987 from wRAR/asyncdef-tests 
Making sure we can run async def tests
 2021-02-18 18:34:25 +05:00
Andrey Rakhmatullin 58706c6598 Merge branch 'asyncdef-tests' into asyncio-parse-asyncgen-proper-rebased 2021-02-18 17:20:25 +05:00
Andrey Rakhmatullin 49af7c4c8b Drop pytest-twisted, use Scrapy code to install the reactor. 2021-02-18 17:10:30 +05:00
Eugenio Lacuesta 4418f78941
Simplify check for negotiated protocol 
negotiatedProtocol's type is Optional[bytes]

See https://github.com/twisted/twisted/blob/twisted-20.3.0/src/twisted/protocols/tls.py#L563-L587
and https://www.pyopenssl.org/en/20.0.1/api/ssl.html#OpenSSL.SSL.Connection.get_alpn_proto_negotiated

Note that OpenSSL.SSL.Connection.get_next_proto_negotiated is deprecated:
https://www.pyopenssl.org/en/20.0.0/changelog.html#backward-incompatible-changes
 2021-02-17 18:36:52 -03:00
Eugenio Lacuesta e80f37bd3f
Test http2 agent for unsupported scheme 2021-02-17 16:34:29 -03:00
Eugenio Lacuesta ac82a4a8a0
Merge remote-tracking branch 'upstream/master' into http2 2021-02-17 16:33:57 -03:00
Andrey Rakhmatullin abbbfbbb38 Add tests for deferred_f_from_coro_f. 2021-02-12 22:41:16 +05:00
Andrey Rakhmatullin a7557f2a49 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-11 16:09:14 +05:00
Adrián Chaves ea92b49aaa
Merge pull request #4986 from wRAR/uvloop-py36 
Skip uvloop 0.15.0+ on py36.
 2021-02-11 12:07:28 +01:00
Andrey Rakhmatullin 54fd371481 Skip uvloop 0.15.0+ on py36. 2021-02-11 14:24:11 +05:00
Andrey Rakhmatullin 67cff0e8a9 Silence pylint "naked raise" error. 2021-02-10 22:44:14 +05:00
Adrián Chaves 15b501c089 Do not force string interpolation while logging 2021-02-10 18:10:57 +01:00
Andrey Rakhmatullin 1cd953e1b0 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-09 13:28:36 +05:00
Adrián Chaves 9ac5b1d021 Adjust test constraints 2021-02-09 09:13:25 +01:00
Adrián Chaves bb72bba178 tox: apply upper constraints to all non-pinned package installations 2021-02-09 09:13:25 +01:00
Adrián Chaves 45345ba6b5 Use constraints.txt to limit pip resolver backtracking 2021-02-09 09:13:25 +01:00
Adrián Chaves de82ca8547
Merge pull request #4982 from wRAR/refactor-spidermiddlewaremanager 
Refactor SpiderMiddlewareManager.scrape_response.
 2021-02-08 21:27:21 +01:00
Andrey Rakhmatullin 1e9b52c3e0 Refactor SpiderMiddlewareManager.scrape_response. 2021-02-08 22:02:03 +05:00
Andrey Rakhmatullin 5cf403295d Remove a duplicate definition. 2021-02-05 19:40:14 +05:00
Adrián Chaves 8527b53e14 Revert "Use --use-deprecated=legacy-resolver" 
This reverts commit 7b11b74c77.
 2021-02-05 13:06:27 +01:00
Adrián Chaves 7afcd634ab Remove unused import 2021-02-05 13:04:54 +01:00
Andrey Rakhmatullin 58f8481301 Update docs. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 2152a2a508 Add main infrastructure for async callbacks. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 2a1e9359ca Add parallel_async. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 92f2c9e308 Move spider middlewares to process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d66d52d3ed Add process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d658552f23 Add only_not_asyncio. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 7e9f498e00 Add MutableAsyncChain. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin acff1eb496 Add aiter_errback. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 76abcedaf4 Add as_async_generator. 2021-02-05 16:21:56 +05:00
Adrián Chaves 7b11b74c77 Use --use-deprecated=legacy-resolver 
Let’s see how test results change
 2021-02-04 11:08:01 +01:00