RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,629 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

1591 Commits

Author SHA1 Message Date
Andrey Rakhmatullin c51ec1ae1c Drop process_iterable_helper, add _process_iterable_universal. 2021-03-23 22:47:30 +05:00
Andrey Rakhmatullin 40eab1d473 Drop a duplicate import. 2021-02-18 19:56:12 +05:00
Andrey Rakhmatullin 58706c6598 Merge branch 'asyncdef-tests' into asyncio-parse-asyncgen-proper-rebased 2021-02-18 17:20:25 +05:00
Andrey Rakhmatullin 49af7c4c8b Drop pytest-twisted, use Scrapy code to install the reactor. 2021-02-18 17:10:30 +05:00
Andrey Rakhmatullin abbbfbbb38 Add tests for deferred_f_from_coro_f. 2021-02-12 22:41:16 +05:00
Andrey Rakhmatullin a7557f2a49 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-11 16:09:14 +05:00
Andrey Rakhmatullin 54fd371481 Skip uvloop 0.15.0+ on py36. 2021-02-11 14:24:11 +05:00
Andrey Rakhmatullin 5cf403295d Remove a duplicate definition. 2021-02-05 19:40:14 +05:00
Andrey Rakhmatullin 2152a2a508 Add main infrastructure for async callbacks. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d66d52d3ed Add process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 7e9f498e00 Add MutableAsyncChain. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin acff1eb496 Add aiter_errback. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 76abcedaf4 Add as_async_generator. 2021-02-05 16:21:56 +05:00
Eugenio Lacuesta 80db569aea
Migrate CI to GitHub actions (#4924) 2021-01-01 19:13:39 -03:00
Eugenio Lacuesta b83a1a6fbf Disable test under pypy 2020-12-21 17:21:08 +01:00
Adrián Chaves 0567fdc56e Merge remote-tracking branch 'upstream/master' into tox-pip20.2 2020-12-21 11:02:05 +01:00
Adrián Chaves d0af008608
Merge pull request #4878 from elacuesta/response-protocol-attribute 
Response.protocol attribute
 2020-12-16 15:27:40 +01:00
Adrián Chaves f6879c681e SCRAPY_PINNED → _SCRAPY_PINNED 2020-11-23 23:53:03 +01:00
Adrián Chaves 0dc3e6350c Add a test to check the Twisted version in pinned environments 2020-11-23 22:10:45 +01:00
Eugenio Lacuesta 18b05af877
Remove tests/test_utils_http.py 2020-11-23 16:18:58 -03:00
Eugenio Lacuesta 51ca4d0138
Remove deprecated scrapy.utils.gz.is_gzipped function 2020-11-23 15:47:08 -03:00
Eugenio Lacuesta 2405df49f1
Add tests for Response.protocol=None 2020-11-16 12:50:33 -03:00
joaquin garmendia 85604e1078
Add failed and success count stats to feedstorage backends (#4850) 2020-11-11 21:16:01 +01:00
Eugenio Lacuesta 5e9a99e6a1
Reponse.protocol as string 2020-11-11 11:15:29 -03:00
Eugenio Lacuesta 5b6b56240c
Test Response.protocol attribute 2020-11-11 01:08:37 -03:00
Eugenio Lacuesta c20b34269f
Remove unnecessary pytest-azurepipelines package (#4876) 2020-11-10 15:35:09 -03:00
Mikhail Korobov 91a8451c98
Merge pull request #4874 from stummjr/fix-missing-fstring-prefix-genspider 
Fix incorrect output on the genspider command
 2020-11-10 17:23:28 +05:00
Eugenio Lacuesta b20cfef1e5
Remove unnecessary line from test 2020-11-09 13:58:52 -03:00
Eugenio Lacuesta 7e98a76ac4
Use deferred_from_coro in asyncio test 2020-11-09 12:17:15 -03:00
Valdir Stumm Junior a2c4a7f920 Add missing f-string prefix to genspider output 2020-11-08 20:14:59 -03:00
Adrián Chaves 4b28da4333 Fix syntax error 2020-11-06 16:46:22 +01:00
Adrián Chaves 1941f607ca Skip 2 additional tests with older Twisted versions 2020-11-06 16:25:56 +01:00
Eugenio Lacuesta 3095d39740
Test: disable asyncio reactor on Windows for Py>=3.8 2020-11-06 12:16:18 -03:00
Adrián Chaves fea5a11899 Also skip test_asyncdef_asyncio on old Twisted versions 2020-11-06 13:00:42 +01:00
Adrián Chaves ea851b910e Clean up Twisted version check 2020-11-06 13:00:31 +01:00
Eugenio Lacuesta 5b5478ae9d
Call asyncio.get_event_loop when installing the asyncio reactor 2020-11-05 14:01:34 -03:00
Adrián Chaves 906626cf0b Skip MiddlewareUsingCoro::test_asyncdef on asyncio and old Twisted 2020-11-04 21:50:12 +01:00
Adrián Chaves 8e7b756727 Solve Flake8-reported issues 2020-11-04 21:26:55 +01:00
Adrián Chaves 7327145bf3 Remove mitmproxy from pinned environments 2020-10-30 21:34:15 +01:00
Georgiy Zatserklianyi a5872a0fad
Fix output file overwrite with -O (FeedExporter updated) (#4859) 2020-10-30 19:36:39 +01:00
Eugenio Lacuesta 585e4a8aee
Replace local server address 2020-10-15 09:07:34 -03:00
Eugenio Lacuesta 868826b346
CachingHostnameResolver tests 2020-10-09 10:38:07 -03:00
Andrey Rahmatullin 9f8c3938cc
Merge pull request #4823 from elacuesta/cookies-revert-header 
Do not process cookies from headers
 2020-10-08 20:20:32 +05:00
Andrey Rahmatullin 45c06cfd80
Merge pull request #4831 from starrify/downloadermw-support-zstd 
Adding support for zstd in HttpCompressionMiddleware
 2020-10-08 14:08:18 +05:00
P. Chen 156bb0a1d4
Fixing the minor typo on test file path in tests/test_downloadermiddleware_httpcompression.py 2020-10-06 19:53:40 +01:00
Adrián Chaves e40788153c Do not consider about: URLs invalid 2020-10-06 19:13:29 +02:00
Andrey Rahmatullin b1255b016a
Merge pull request #4746 from Gallaecio/fix_iternodes 
Fix iternodes
 2020-10-06 20:14:04 +05:00
Andrey Rahmatullin 1aeda66435
Merge pull request #4804 from Gallaecio/mock-s3 
Use mocking for S3 tests that currently need server credentials
 2020-10-06 18:23:01 +05:00
P. Chen 6032a9a310
Minor adjustment to the test case in tests/test_downloadermiddleware_httpcompression.py 2020-10-05 23:55:48 +01:00
P. Chen 50e1f35d1f
Adding test cases for the zstd content encoding 2020-10-05 23:43:12 +01:00