RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

10393 Commits

Author SHA1 Message Date
Mikhail Korobov 61653418ff
Merge pull request #4090 from Gallaecio/documentation-build 
Fix references to Python types in parameter type fields
 2020-08-14 00:58:06 +05:00
Mikhail Korobov cd0d3fd48d
Merge pull request #4705 from Gallaecio/deprecation-policy 
Cover our deprecation policy in the documentation
 2020-08-14 00:27:37 +05:00
Mikhail Korobov 1f44464a4a
Merge pull request #4723 from Gallaecio/windows 
test_utils_iterators.py: support Windows the right way
 2020-08-14 00:23:55 +05:00
Adrián Chaves 65e0abaea5 Document FEED_URI_PARAMS 2020-08-13 09:05:51 +02:00
Adrián Chaves 24ba5a71ac Maybe the problem is not in the code after all 2020-08-13 06:35:09 +02:00
Adrián Chaves 5f4df622a1 test_utils_iterators.py: support Windows the right way 2020-08-13 05:41:06 +02:00
Adrián Chaves 4c0afb606c Update permission expectations 2020-08-12 17:45:26 +02:00
Adrián Chaves 125a058340 Do not let umask affect the permissions of startproject-generated files 2020-08-12 17:07:21 +02:00
Adrián Chaves b1de55d37d Fix marker syntax 2020-08-12 12:34:40 +02:00
Adrián Chaves 8e393a0b21 Do not change the mitmproxy version for no-3.6 Python versions 2020-08-12 12:29:51 +02:00
Adrián Chaves 394631fc0a Restore 3.5 support for mitmproxy-based tests 2020-08-12 12:08:09 +02:00
Ajay Mittur 1c4b4cc6b0
Support defining file path based on item in media pipelines (#4686) 2020-08-11 14:12:44 +02:00
Adrián Chaves 5138f9a965 Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2020-08-11 13:42:05 +02:00
Adrián Chaves b2f4df5cb7 Merge remote-tracking branch 'upstream/master' into documentation-build 2020-08-11 13:16:14 +02:00
Adrián Chaves aefd43a6c6 Upgrade minimum dependencies for Python 3.6 support 2020-08-11 12:52:54 +02:00
Adrián Chaves 90f85a2b9b Enable Travis CI 2020-08-11 10:20:30 +02:00
Aditya c67d6dea31 fix: H2 docs, NotImplementedError for H2 Tunnel 2020-08-11 04:39:41 +05:30
Andrey Rahmatullin 0cf1340c29
Merge pull request #4663 from ajaymittur28/scrapy-check-test 
Add Tests to `scrapy check` command
 2020-08-10 16:18:29 +05:00
Andrey Rahmatullin 702de0480b
Merge pull request #4564 from jacty/patch-1 
Unnecessary update when value is None
 2020-08-10 16:14:42 +05:00
Aditya e0c3019d90 fix: ScrapyProxyH2Agent 
- add required test cases

BREAKING CHANGES
Presently the tests (in test_downloader_handlers.py)
1. test_download_without_proxy
2. test_download_with_proxy_https_timeout

collide with each other when run together. However, if both of the tests
are ran individually then both pass.
 2020-08-09 16:19:35 +05:30
Jose Galdos 13181ba788 Improve http status all on http error middleware 2020-08-06 20:43:16 -05:00
Adrián Chaves 1cc8d5829f Remove unneeded try-except 
Exceptions only happen when find_spec gets a 2nd parameter.
 2020-08-06 13:52:47 +02:00
Andrey Rahmatullin cad84458ac
Merge pull request #4714 from linchiwei123/fix 
Remove duplicated install_requires
 2020-08-06 12:17:04 +05:00
linchiwei123 4dc09f09aa
Update setup.py 2020-08-05 22:23:19 +08:00
Andrey Rahmatullin 1f0722c87e
Merge pull request #4701 from kshitijcode/weakkeycache-cleanup-4684 
Code cleanup scrapy.utils.python.WeakKeyCache #4684
 2020-08-05 17:48:41 +05:00
Kshitij Sharma 983b7ddf2e aesthetic fixes 2020-08-05 16:13:52 +05:30
Kshitij Sharma b35d1f2b2c deleted tester.py 2020-08-05 09:14:04 +05:30
Kshitij Sharma 9d84289109 deprecated weakkeycache by specifying in __init__ 2020-08-05 09:11:59 +05:30
Marc Hernández 336f19f5cc
Change super syntax (#4707) 2020-08-04 20:42:01 +02:00
Adrián Chaves 4ee538e44b
Update unicode references from Python 2 times in the documentation (#4703) 2020-08-04 20:34:11 +02:00
Adrián Chaves 1278e76d90 Bump version: 2.2.0 → 2.3.0 2020-08-04 20:07:02 +02:00
Adrián Chaves 3600582f56
Cover Scrapy 2.2.1 and 2.3 in the release notes (#4708) 2020-08-04 20:05:56 +02:00
Mikhail Korobov 015b71d89f
Merge pull request #4704 from Gallaecio/python2-u-prefixes 
Remove the u prefix from strings
 2020-07-31 01:23:00 +05:00
Artur Shellunts 5ec66be4a4
Merge branch 'master' into obsolete-s3feednokeys 2020-07-30 22:18:00 +02:00
Aditya d707f8b5d9 docs: mention H2DownloadHandler in settings.rst 2020-07-30 18:06:21 +05:30
Adrián Chaves 6f4ccec567 Cover our deprecation policy in the documentation 2020-07-30 14:03:14 +02:00
Adrián Chaves 890b2138a6 Remove the u prefix from strings 2020-07-30 13:39:30 +02:00
Aditya a3fecaf07f test: fix host-name H2DownloadHandler tests 2020-07-30 15:45:27 +05:30
Kshitij Sharma 49337bd2ae Code cleanup scrapy.utils.python.WeakKeyCache #4684 and fixing ci alerts 2020-07-30 12:25:21 +05:30
Kshitij Sharma 33ddc3d4f3 Trigger CI 2020-07-30 09:47:08 +05:30
Mikhail Korobov 5e2d1bd187
Merge pull request #4434 from BroodingKangaroo/ISSUE-4250-add_batch_deliveries 
Feed exports: add batch deliveries
 2020-07-29 23:03:28 +05:00
Kshitij Sharma 403bc7020a Code cleanup scrapy.utils.python.WeakKeyCache #4684 and fixing ci alerts 2020-07-29 18:05:33 +05:30
Aditya 19f2b4b53d refactor: AcceptableProtocolsContextFactory 
- rename H2WrappedContextFactory to AcceptableProtocolsContextFactory
- AcceptableProtocolsContextFactory accepts an argument
acceptable_protocols which can be used to override the context factory
priority list of protocols during ALPN or NPN
 2020-07-29 17:25:59 +05:30
Aditya e8342996f6 test: H2DownloadHandler 
Following tests are skipped as Content-Length header not matching the
data received is considered as a ProtocolError
- test_download_broken_content_cause_data_loss
- test_download_broken_chunked_content_cause_data_loss
- test_download_broken_content_allow_data_loss
- test_download_broken_chunked_content_allow_data_loss
- test_download_broken_content_allow_data_loss_via_setting
- test_download_broken_chunked_content_allow_data_loss_via_setting

BREAKING CHANGES
The following tests currently fail
- test_content_length_zero_bodyless_post_request_headers
- test_host_header_seted_in_request_headers
- test_download_with_maxsize_very_large_file
 2020-07-29 13:51:01 +05:30
Aditya 92bec38591 feat: MethodNotAllowed405, Content-Length header 
- add tests to check for Content-Length header
- raise MethodNotAllowed405 when remote send 'HTTP/2.0 405 Method Not
Allowed'
 2020-07-29 13:49:14 +05:30
Kshitij Sharma e7a58fe157 Code cleanup scrapy.utils.python.WeakKeyCache #4684 2020-07-29 10:16:18 +05:30
Eugenio Lacuesta 5265853937
Use ItemAdapter.field_names when writing header in CsvItemExporter (#4668) 2020-07-28 11:15:14 +02:00
BroodingKangaroo a6c1d79b7c pep8 tiny changes 2020-07-28 11:53:05 +03:00
Mikhail Korobov ce0c25fc85
Merge pull request #4690 from elacuesta/typing-setup-remove-monkeypatches 
Remove monkeypatches module from mypy section in setup.cfg
 2020-07-23 17:56:08 +05:00
Eugenio Lacuesta 8fae3d5bb7
Remove monkeypatches module from mypy section in setup.cfg 2020-07-22 16:08:35 -03:00