RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,400 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

8400 Commits

Author SHA1 Message Date
Adrián Chaves e70975f0bb
Allow overwriting feeds (#4512) 
Co-authored-by: Yuval Hager <yhager@yhager.com>
 2020-08-17 15:10:08 +02:00
Mikhail Korobov 282a6d4fc1
Merge pull request #4727 from wRAR/mitmproxy-ca 
Use a longer key in mitmproxy-ca.pem.
 2020-08-17 14:06:58 +05:00
Mikhail Korobov 61459de476
Merge pull request #4688 from ashellunts/obsolete-s3feednokeys 
Remove obsolete S3FeedStorage instancing without AWS credentials
 2020-08-17 13:58:32 +05:00
Adrián Chaves 792d29380e
Merge Twisted’s deprecated HTTPClientFactory into ScrapyHTTPClientFactory (#4165) 
* Port code from Twisted’s deprecated HTTPClientFactory into ScrapyHTTPClientFactory

* Mention that ScrapyHTTPClientFactory has Twisted code
 2020-08-17 10:54:58 +02:00
Mikhail Korobov 5ac135f036
Merge pull request #4718 from Gallaecio/support-pinned-3.6 
Upgrade minimum dependencies for Python 3.6 support
 2020-08-17 13:54:45 +05:00
Grammy Jiang 55edf8d3b8
Add typing hint to httpcache downloadermiddlewares (#4243) 2020-08-17 10:50:52 +02:00
Ajay Mittur a8e08d51cd
Check if file is already present on running `scrapy genspider` and terminate if so (#4623) 2020-08-17 10:45:52 +02:00
Eugenio Lacuesta 2aa4f3cbf9
Conditional request attribute binding for responses (#4632) 2020-08-17 10:39:59 +02:00
Andrey Rakhmatullin 756c368a6b Use a longer key in mitmproxy-ca.pem. 2020-08-14 22:09:24 +05:00
Andrey Rahmatullin acb3b4433a
Merge pull request #4724 from Gallaecio/feed-uri-params 
Document FEED_URI_PARAMS
 2020-08-14 14:47:55 +05:00
Mikhail Korobov 61653418ff
Merge pull request #4090 from Gallaecio/documentation-build 
Fix references to Python types in parameter type fields
 2020-08-14 00:58:06 +05:00
Mikhail Korobov cd0d3fd48d
Merge pull request #4705 from Gallaecio/deprecation-policy 
Cover our deprecation policy in the documentation
 2020-08-14 00:27:37 +05:00
Mikhail Korobov 1f44464a4a
Merge pull request #4723 from Gallaecio/windows 
test_utils_iterators.py: support Windows the right way
 2020-08-14 00:23:55 +05:00
Adrián Chaves 65e0abaea5 Document FEED_URI_PARAMS 2020-08-13 09:05:51 +02:00
Adrián Chaves 24ba5a71ac Maybe the problem is not in the code after all 2020-08-13 06:35:09 +02:00
Adrián Chaves 5f4df622a1 test_utils_iterators.py: support Windows the right way 2020-08-13 05:41:06 +02:00
Adrián Chaves b1de55d37d Fix marker syntax 2020-08-12 12:34:40 +02:00
Adrián Chaves 8e393a0b21 Do not change the mitmproxy version for no-3.6 Python versions 2020-08-12 12:29:51 +02:00
Adrián Chaves 394631fc0a Restore 3.5 support for mitmproxy-based tests 2020-08-12 12:08:09 +02:00
Ajay Mittur 1c4b4cc6b0
Support defining file path based on item in media pipelines (#4686) 2020-08-11 14:12:44 +02:00
Adrián Chaves b2f4df5cb7 Merge remote-tracking branch 'upstream/master' into documentation-build 2020-08-11 13:16:14 +02:00
Adrián Chaves aefd43a6c6 Upgrade minimum dependencies for Python 3.6 support 2020-08-11 12:52:54 +02:00
Andrey Rahmatullin 0cf1340c29
Merge pull request #4663 from ajaymittur28/scrapy-check-test 
Add Tests to `scrapy check` command
 2020-08-10 16:18:29 +05:00
Andrey Rahmatullin 702de0480b
Merge pull request #4564 from jacty/patch-1 
Unnecessary update when value is None
 2020-08-10 16:14:42 +05:00
Andrey Rahmatullin cad84458ac
Merge pull request #4714 from linchiwei123/fix 
Remove duplicated install_requires
 2020-08-06 12:17:04 +05:00
linchiwei123 4dc09f09aa
Update setup.py 2020-08-05 22:23:19 +08:00
Andrey Rahmatullin 1f0722c87e
Merge pull request #4701 from kshitijcode/weakkeycache-cleanup-4684 
Code cleanup scrapy.utils.python.WeakKeyCache #4684
 2020-08-05 17:48:41 +05:00
Kshitij Sharma 983b7ddf2e aesthetic fixes 2020-08-05 16:13:52 +05:30
Kshitij Sharma b35d1f2b2c deleted tester.py 2020-08-05 09:14:04 +05:30
Kshitij Sharma 9d84289109 deprecated weakkeycache by specifying in __init__ 2020-08-05 09:11:59 +05:30
Marc Hernández 336f19f5cc
Change super syntax (#4707) 2020-08-04 20:42:01 +02:00
Adrián Chaves 4ee538e44b
Update unicode references from Python 2 times in the documentation (#4703) 2020-08-04 20:34:11 +02:00
Adrián Chaves 1278e76d90 Bump version: 2.2.0 → 2.3.0 2020-08-04 20:07:02 +02:00
Adrián Chaves 3600582f56
Cover Scrapy 2.2.1 and 2.3 in the release notes (#4708) 2020-08-04 20:05:56 +02:00
Mikhail Korobov 015b71d89f
Merge pull request #4704 from Gallaecio/python2-u-prefixes 
Remove the u prefix from strings
 2020-07-31 01:23:00 +05:00
Artur Shellunts 5ec66be4a4
Merge branch 'master' into obsolete-s3feednokeys 2020-07-30 22:18:00 +02:00
Adrián Chaves 6f4ccec567 Cover our deprecation policy in the documentation 2020-07-30 14:03:14 +02:00
Adrián Chaves 890b2138a6 Remove the u prefix from strings 2020-07-30 13:39:30 +02:00
Kshitij Sharma 49337bd2ae Code cleanup scrapy.utils.python.WeakKeyCache #4684 and fixing ci alerts 2020-07-30 12:25:21 +05:30
Kshitij Sharma 33ddc3d4f3 Trigger CI 2020-07-30 09:47:08 +05:30
Mikhail Korobov 5e2d1bd187
Merge pull request #4434 from BroodingKangaroo/ISSUE-4250-add_batch_deliveries 
Feed exports: add batch deliveries
 2020-07-29 23:03:28 +05:00
Kshitij Sharma 403bc7020a Code cleanup scrapy.utils.python.WeakKeyCache #4684 and fixing ci alerts 2020-07-29 18:05:33 +05:30
Kshitij Sharma e7a58fe157 Code cleanup scrapy.utils.python.WeakKeyCache #4684 2020-07-29 10:16:18 +05:30
Eugenio Lacuesta 5265853937
Use ItemAdapter.field_names when writing header in CsvItemExporter (#4668) 2020-07-28 11:15:14 +02:00
BroodingKangaroo a6c1d79b7c pep8 tiny changes 2020-07-28 11:53:05 +03:00
Mikhail Korobov ce0c25fc85
Merge pull request #4690 from elacuesta/typing-setup-remove-monkeypatches 
Remove monkeypatches module from mypy section in setup.cfg
 2020-07-23 17:56:08 +05:00
Eugenio Lacuesta 8fae3d5bb7
Remove monkeypatches module from mypy section in setup.cfg 2020-07-22 16:08:35 -03:00
Artur Shellunts 430d22e46e Remove not used import warnings 2020-07-21 23:39:04 +02:00
nyov 2829cd4268 Allow use without credentials 2020-07-21 23:38:02 +02:00
nyov 98e8086d1b Adapt S3FeedStorage testcase 2020-07-21 23:38:02 +02:00