RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,775 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

1647 Commits

Author SHA1 Message Date
Mikhail Korobov f0c8d31193
Merge pull request #4694 from Jgaldos/improve-httpstatus-all-meta 
Improve http status all on http error middleware
 2021-04-01 22:43:45 +05:00
Mikhail Korobov d0e23486b4
Merge pull request #4902 from Gallaecio/retry-request 
Implement a retry request function
 2021-04-01 22:23:56 +05:00
Eugenio Lacuesta 72e8cea8af
Avoid exceptions in is_generator_with_return_value (#4935) 2021-03-22 11:51:11 -03:00
Adrián Chaves ec5a7918ec
Include Content-Length in HTTP/1.1 responses (#5057) 2021-03-22 11:25:40 +01:00
Mikhail Korobov 0e7ee1256c
Merge pull request #5036 from dmiwell/urllength-dont-skip-silently 
UrlLengthMiddleware: don't skip silently
 2021-03-20 00:55:33 +05:00
Adrián Chaves 308a58aa27
Update CI to support Twisted 21.2.0 (#5027) 2021-03-19 18:39:44 +01:00
Adrián Chaves 8e73e1dfb5 upper-constraints.txt: restrict botocore further 2021-03-18 23:42:29 +01:00
Adrián Chaves 1d83697990
Merge pull request #4769 from scrapy/http2 
Implement experimental HTTP/2 support
 2021-03-18 22:18:12 +01:00
Dmitriy Pomazunovskiy 9cc4513bd6 simpler stats access 2021-03-15 21:38:03 +06:00
Dmitriy Pomazunovskiy 0f254a6afb Test from_settings 2021-03-12 17:11:50 +06:00
Dmitriy Pomazunovskiy d4b2b61255 Use from_settings for backward compatibility 2021-03-12 16:59:37 +06:00
Dmitriy Pomazunovskiy 6e5ea7924c Log skipped urls by length to INFO, add skipped stats 2021-03-12 11:08:41 +06:00
Eugenio Lacuesta 0c16088230
headers_received signal (#4897) 2021-03-11 11:52:35 -03:00
Adrián Chaves 5b2d3e17a3
Merge branch 'master' into http2 2021-03-09 14:59:17 +01:00
Eugenio Lacuesta 9e62355271
Allow logger/stats customization in get_retry_request 2021-03-02 12:12:22 -03:00
Adrián Chaves 386e2a51ae tests/test_downloader_handlers_http2.py: fix style issue 2021-02-24 10:41:01 +01:00
Adrián Chaves 12064d799b HTTP/2: improve header handling 2021-02-24 10:37:38 +01:00
Adrián Chaves 5101094207 HTTP/2: test a CONNECT request 2021-02-24 07:33:39 +01:00
Adrián Chaves 7605f19ec4 HTTP/2: test 2 concurrent requests to the same domain 2021-02-23 05:54:48 +01:00
Eugenio Lacuesta 6326178bc5
http2: acceptable protocol update, tests (#4994) 2021-02-22 16:50:51 +01:00
Adrián Chaves 1f7665c4cf Silence a PyLint check on a mistake made for testing purposes 2021-02-22 16:48:10 +01:00
Adrián Chaves 722a33a2ac Fix style issues 2021-02-22 16:42:38 +01:00
Adrián Chaves 80f5003c88 Add tests for get_retry_request 2021-02-22 16:38:38 +01:00
Andrey Rakhmatullin 49af7c4c8b Drop pytest-twisted, use Scrapy code to install the reactor. 2021-02-18 17:10:30 +05:00
Eugenio Lacuesta e80f37bd3f
Test http2 agent for unsupported scheme 2021-02-17 16:34:29 -03:00
Eugenio Lacuesta ac82a4a8a0
Merge remote-tracking branch 'upstream/master' into http2 2021-02-17 16:33:57 -03:00
Andrey Rakhmatullin abbbfbbb38 Add tests for deferred_f_from_coro_f. 2021-02-12 22:41:16 +05:00
Andrey Rakhmatullin 54fd371481 Skip uvloop 0.15.0+ on py36. 2021-02-11 14:24:11 +05:00
Adrián Chaves 9ac5b1d021 Adjust test constraints 2021-02-09 09:13:25 +01:00
Adrián Chaves bb72bba178 tox: apply upper constraints to all non-pinned package installations 2021-02-09 09:13:25 +01:00
Adrián Chaves 45345ba6b5 Use constraints.txt to limit pip resolver backtracking 2021-02-09 09:13:25 +01:00
Adrián Chaves 248800328c Fix test_pinned_twisted_version 2021-02-03 21:13:43 +01:00
Adrián Chaves c8d8b180bf Merge remote-tracking branch 'upstream/master' into http2 2021-02-03 09:37:35 +01:00
Eugenio Lacuesta 80db569aea
Migrate CI to GitHub actions (#4924) 2021-01-01 19:13:39 -03:00
Eugenio Lacuesta e494a3f733
protocol attribute for h2 responses 2020-12-31 11:50:15 -03:00
Eugenio Lacuesta d698b5147b
Merge branch 'master' into http2 2020-12-31 11:13:25 -03:00
Eugenio Lacuesta b83a1a6fbf Disable test under pypy 2020-12-21 17:21:08 +01:00
Adrián Chaves 0567fdc56e Merge remote-tracking branch 'upstream/master' into tox-pip20.2 2020-12-21 11:02:05 +01:00
Adrián Chaves d0af008608
Merge pull request #4878 from elacuesta/response-protocol-attribute 
Response.protocol attribute
 2020-12-16 15:27:40 +01:00
Adrián Chaves f6879c681e SCRAPY_PINNED → _SCRAPY_PINNED 2020-11-23 23:53:03 +01:00
Adrián Chaves 0dc3e6350c Add a test to check the Twisted version in pinned environments 2020-11-23 22:10:45 +01:00
Eugenio Lacuesta 18b05af877
Remove tests/test_utils_http.py 2020-11-23 16:18:58 -03:00
Eugenio Lacuesta 51ca4d0138
Remove deprecated scrapy.utils.gz.is_gzipped function 2020-11-23 15:47:08 -03:00
Eugenio Lacuesta 2405df49f1
Add tests for Response.protocol=None 2020-11-16 12:50:33 -03:00
joaquin garmendia 85604e1078
Add failed and success count stats to feedstorage backends (#4850) 2020-11-11 21:16:01 +01:00
Eugenio Lacuesta 5e9a99e6a1
Reponse.protocol as string 2020-11-11 11:15:29 -03:00
Eugenio Lacuesta 5b6b56240c
Test Response.protocol attribute 2020-11-11 01:08:37 -03:00
Eugenio Lacuesta c20b34269f
Remove unnecessary pytest-azurepipelines package (#4876) 2020-11-10 15:35:09 -03:00
Mikhail Korobov 91a8451c98
Merge pull request #4874 from stummjr/fix-missing-fstring-prefix-genspider 
Fix incorrect output on the genspider command
 2020-11-10 17:23:28 +05:00
Eugenio Lacuesta b20cfef1e5
Remove unnecessary line from test 2020-11-09 13:58:52 -03:00