RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,537 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

8537 Commits

Author SHA1 Message Date
Eugenio Lacuesta 2405df49f1
Add tests for Response.protocol=None 2020-11-16 12:50:33 -03:00
Eugenio Lacuesta b0368228d7
Add exception to catch 2020-11-11 11:18:03 -03:00
Eugenio Lacuesta 5e9a99e6a1
Reponse.protocol as string 2020-11-11 11:15:29 -03:00
Eugenio Lacuesta 2242412556
Docs: placeholder for versionadded directive 2020-11-11 10:50:54 -03:00
Eugenio Lacuesta 61d089485c
Docs: sort versionadded directives 2020-11-11 01:31:21 -03:00
Eugenio Lacuesta 0fb7bcb2cf
Style adjustment 2020-11-11 01:26:08 -03:00
Eugenio Lacuesta 587b4dd71f
Docs for the Response.protocol attribute 2020-11-11 01:22:33 -03:00
Eugenio Lacuesta 5b6b56240c
Test Response.protocol attribute 2020-11-11 01:08:37 -03:00
Eugenio Lacuesta 99cc853d69
Response.protocol attribute 2020-11-11 00:44:11 -03:00
Eugenio Lacuesta c20b34269f
Remove unnecessary pytest-azurepipelines package (#4876) 2020-11-10 15:35:09 -03:00
Mikhail Korobov 91a8451c98
Merge pull request #4874 from stummjr/fix-missing-fstring-prefix-genspider 
Fix incorrect output on the genspider command
 2020-11-10 17:23:28 +05:00
Mikhail Korobov 27b07c692c
Merge pull request #4872 from elacuesta/asyncio_get_event_loop 
Call asyncio.get_event_loop when installing the asyncio reactor
 2020-11-10 17:22:20 +05:00
Eugenio Lacuesta b20cfef1e5
Remove unnecessary line from test 2020-11-09 13:58:52 -03:00
Eugenio Lacuesta 7e98a76ac4
Use deferred_from_coro in asyncio test 2020-11-09 12:17:15 -03:00
Valdir Stumm Junior a2c4a7f920 Add missing f-string prefix to genspider output 2020-11-08 20:14:59 -03:00
Eugenio Lacuesta 114229eb4a
Docs: add a note about asyncio.set_event_loop 2020-11-06 13:29:14 -03:00
Eugenio Lacuesta 3095d39740
Test: disable asyncio reactor on Windows for Py>=3.8 2020-11-06 12:16:18 -03:00
Eugenio Lacuesta 5b5478ae9d
Call asyncio.get_event_loop when installing the asyncio reactor 2020-11-05 14:01:34 -03:00
Eugenio Lacuesta c292957cb1
Run Windows tests on GitHub actions (#4869) 2020-11-05 15:15:58 +01:00
Georgiy Zatserklianyi a5872a0fad
Fix output file overwrite with -O (FeedExporter updated) (#4859) 2020-10-30 19:36:39 +01:00
Andrey Rahmatullin 75f35f558f
Merge pull request #4803 from elacuesta/instantiate-resolution-receiver 
CachingHostnameResolver
 2020-10-20 13:55:07 +05:00
Eugenio Lacuesta 585e4a8aee
Replace local server address 2020-10-15 09:07:34 -03:00
Adrián Chaves c340e72988 Bump version: 2.3.0 → 2.4.0 2020-10-11 22:12:45 +02:00
Adrián Chaves 47eac83740 Set a release date for Scrapy 2.4.0 2020-10-11 22:11:14 +02:00
Adrián Chaves 015c82b974
Scrapy 2.4 release notes (#4808) 2020-10-11 22:09:45 +02:00
Eugenio Lacuesta 868826b346
CachingHostnameResolver tests 2020-10-09 10:38:07 -03:00
Eugenio Lacuesta 8fe5876597
HostResolution implementation 2020-10-09 10:38:07 -03:00
Eugenio Lacuesta b55c911ddc
Fix CachingHostnameResolver 2020-10-09 10:38:03 -03:00
Andrey Rahmatullin da426fb3cf
Merge pull request #4839 from elacuesta/pytest_xfail_strict 
Add xfail_strict=true to pytest.ini
 2020-10-09 00:39:05 +05:00
Eugenio Lacuesta 13ae17aecc
Add xfail_strict=true to pytest.ini 2020-10-08 14:04:52 -03:00
Andrey Rahmatullin 9f8c3938cc
Merge pull request #4823 from elacuesta/cookies-revert-header 
Do not process cookies from headers
 2020-10-08 20:20:32 +05:00
Andrey Rahmatullin 45c06cfd80
Merge pull request #4831 from starrify/downloadermw-support-zstd 
Adding support for zstd in HttpCompressionMiddleware
 2020-10-08 14:08:18 +05:00
Andrey Rahmatullin 8fc4e2e011
Merge pull request #4836 from OfirD1/patch-1 
moved the sentence about processing pending requests when a spider is closed onto a generic note.
 2020-10-07 16:00:38 +05:00
Andrey Rahmatullin ded9a5a085
Merge pull request #4835 from Gallaecio/about-url-support 
Do not consider about: URLs invalid
 2020-10-07 14:50:28 +05:00
OfirD1 1a597d5e3d
moved the sentence about processing pending requests when a spider is closed onto a generic note. 2020-10-06 21:54:42 +03:00
P. Chen 156bb0a1d4
Fixing the minor typo on test file path in tests/test_downloadermiddleware_httpcompression.py 2020-10-06 19:53:40 +01:00
P. Chen 2e734e6b35
Minor update on the import order in scrapy/downloadermiddlewares/httpcompression.py 2020-10-06 19:51:05 +01:00
Adrián Chaves e40788153c Do not consider about: URLs invalid 2020-10-06 19:13:29 +02:00
Andrey Rahmatullin b1255b016a
Merge pull request #4746 from Gallaecio/fix_iternodes 
Fix iternodes
 2020-10-06 20:14:04 +05:00
Andrey Rahmatullin 9b1f86b613
Use f-strings 2020-10-06 18:50:55 +05:00
Eugenio Lacuesta 137c8ba6ee
Docs: mention limitation about Cookie header 2020-10-06 10:50:17 -03:00
Andrey Rahmatullin 1aeda66435
Merge pull request #4804 from Gallaecio/mock-s3 
Use mocking for S3 tests that currently need server credentials
 2020-10-06 18:23:01 +05:00
P. Chen 6032a9a310
Minor adjustment to the test case in tests/test_downloadermiddleware_httpcompression.py 2020-10-05 23:55:48 +01:00
P. Chen 50e1f35d1f
Adding test cases for the zstd content encoding 2020-10-05 23:43:12 +01:00
P. Chen da3171d4f7
Using the `zstandard` package than `zstd` for supporting frames both with and without the content size info 
See also: https://github.com/sergey-dryabzhinsky/python-zstd/issues/53
 2020-10-05 23:18:58 +01:00
P. Chen c6c3f2ce66
Updating the doc entry for the HTTP compress downloader middleware on zstd 2020-10-05 21:10:40 +01:00
P. Chen 892dd9da57
Adding support for zstd in HttpCompressionMiddleware 2020-10-05 21:03:01 +01:00
Adrián Chaves 004b40a719
as soon as → as long as (#4825) 2020-10-02 19:53:55 -03:00
Eugenio Lacuesta 797a6690c0
Tests: use classes instead of paths in settings (#4817) 2020-10-01 23:11:11 -03:00
Mikhail Korobov 4f27c5f82b
Merge pull request #4768 from maranqz/feature/4606-exporter-from-FEEDS 
Pass info from FEEDS to ItemExporter
 2020-10-02 00:10:05 +05:00