RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,411 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

8411 Commits

Author SHA1 Message Date
Iván de Prado 3e726b9df7 Support for delegated methods as callbacks 
It can be useful to structure the spiders code around some helper classes.
 2020-08-25 11:31:20 +01:00
Andrey Rahmatullin 64e0ea4ee5
Merge pull request #4735 from wRAR/ciphers-tests 
Re-enable TLS 1.2 in cipher tests.
 2020-08-25 11:48:31 +05:00
Eugenio Lacuesta f1250177dc
Remove Python 3.5 from CI (#4743) 2020-08-22 09:33:35 +02:00
Mikhail Korobov ba0e7f3c85
Merge pull request #4747 from Gallaecio/pylint-update 
Skip checks introduced in Pylint 2.6.0
 2020-08-21 23:18:51 +05:00
Adrián Chaves 7c076122eb Skip checks introduced in Pylint 2.6.0 2020-08-21 17:06:54 +02:00
Andrey Rahmatullin 07b0c591e0
Merge pull request #4745 from Yogendra0Sharma/fix4741 
Removed appveyor.xml no longer needed
 2020-08-21 12:36:13 +05:00
yogendra0sharma 2fbfe2c214 Removed appveyor.xml no longer needed 2020-08-21 12:18:15 +05:30
Grisha Temchenko d68aab992e
Smarter generator check for combined return/yield statements (#4721) 2020-08-20 10:22:07 -03:00
Hugo van Kemenade a57db9e302
Bitbucket no longer supports Mercurial repositories (#4738) 2020-08-19 17:45:24 +02:00
sakshamb2113 42383cc267
Add a setting to customize the asyncio event loop (#4414) 2020-08-19 09:18:14 +02:00
Andrey Rakhmatullin d9e69bfb51 Re-enable TLS 1.2 in cipher tests. 2020-08-17 19:46:24 +05:00
Adrián Chaves e70975f0bb
Allow overwriting feeds (#4512) 
Co-authored-by: Yuval Hager <yhager@yhager.com>
 2020-08-17 15:10:08 +02:00
Mikhail Korobov 282a6d4fc1
Merge pull request #4727 from wRAR/mitmproxy-ca 
Use a longer key in mitmproxy-ca.pem.
 2020-08-17 14:06:58 +05:00
Mikhail Korobov 61459de476
Merge pull request #4688 from ashellunts/obsolete-s3feednokeys 
Remove obsolete S3FeedStorage instancing without AWS credentials
 2020-08-17 13:58:32 +05:00
Adrián Chaves 792d29380e
Merge Twisted’s deprecated HTTPClientFactory into ScrapyHTTPClientFactory (#4165) 
* Port code from Twisted’s deprecated HTTPClientFactory into ScrapyHTTPClientFactory

* Mention that ScrapyHTTPClientFactory has Twisted code
 2020-08-17 10:54:58 +02:00
Mikhail Korobov 5ac135f036
Merge pull request #4718 from Gallaecio/support-pinned-3.6 
Upgrade minimum dependencies for Python 3.6 support
 2020-08-17 13:54:45 +05:00
Grammy Jiang 55edf8d3b8
Add typing hint to httpcache downloadermiddlewares (#4243) 2020-08-17 10:50:52 +02:00
Ajay Mittur a8e08d51cd
Check if file is already present on running `scrapy genspider` and terminate if so (#4623) 2020-08-17 10:45:52 +02:00
Eugenio Lacuesta 2aa4f3cbf9
Conditional request attribute binding for responses (#4632) 2020-08-17 10:39:59 +02:00
Andrey Rakhmatullin 756c368a6b Use a longer key in mitmproxy-ca.pem. 2020-08-14 22:09:24 +05:00
Andrey Rahmatullin acb3b4433a
Merge pull request #4724 from Gallaecio/feed-uri-params 
Document FEED_URI_PARAMS
 2020-08-14 14:47:55 +05:00
Mikhail Korobov 61653418ff
Merge pull request #4090 from Gallaecio/documentation-build 
Fix references to Python types in parameter type fields
 2020-08-14 00:58:06 +05:00
Mikhail Korobov cd0d3fd48d
Merge pull request #4705 from Gallaecio/deprecation-policy 
Cover our deprecation policy in the documentation
 2020-08-14 00:27:37 +05:00
Mikhail Korobov 1f44464a4a
Merge pull request #4723 from Gallaecio/windows 
test_utils_iterators.py: support Windows the right way
 2020-08-14 00:23:55 +05:00
Adrián Chaves 65e0abaea5 Document FEED_URI_PARAMS 2020-08-13 09:05:51 +02:00
Adrián Chaves 24ba5a71ac Maybe the problem is not in the code after all 2020-08-13 06:35:09 +02:00
Adrián Chaves 5f4df622a1 test_utils_iterators.py: support Windows the right way 2020-08-13 05:41:06 +02:00
Adrián Chaves b1de55d37d Fix marker syntax 2020-08-12 12:34:40 +02:00
Adrián Chaves 8e393a0b21 Do not change the mitmproxy version for no-3.6 Python versions 2020-08-12 12:29:51 +02:00
Adrián Chaves 394631fc0a Restore 3.5 support for mitmproxy-based tests 2020-08-12 12:08:09 +02:00
Ajay Mittur 1c4b4cc6b0
Support defining file path based on item in media pipelines (#4686) 2020-08-11 14:12:44 +02:00
Adrián Chaves b2f4df5cb7 Merge remote-tracking branch 'upstream/master' into documentation-build 2020-08-11 13:16:14 +02:00
Adrián Chaves aefd43a6c6 Upgrade minimum dependencies for Python 3.6 support 2020-08-11 12:52:54 +02:00
Andrey Rahmatullin 0cf1340c29
Merge pull request #4663 from ajaymittur28/scrapy-check-test 
Add Tests to `scrapy check` command
 2020-08-10 16:18:29 +05:00
Andrey Rahmatullin 702de0480b
Merge pull request #4564 from jacty/patch-1 
Unnecessary update when value is None
 2020-08-10 16:14:42 +05:00
Andrey Rahmatullin cad84458ac
Merge pull request #4714 from linchiwei123/fix 
Remove duplicated install_requires
 2020-08-06 12:17:04 +05:00
linchiwei123 4dc09f09aa
Update setup.py 2020-08-05 22:23:19 +08:00
Andrey Rahmatullin 1f0722c87e
Merge pull request #4701 from kshitijcode/weakkeycache-cleanup-4684 
Code cleanup scrapy.utils.python.WeakKeyCache #4684
 2020-08-05 17:48:41 +05:00
Kshitij Sharma 983b7ddf2e aesthetic fixes 2020-08-05 16:13:52 +05:30
Kshitij Sharma b35d1f2b2c deleted tester.py 2020-08-05 09:14:04 +05:30
Kshitij Sharma 9d84289109 deprecated weakkeycache by specifying in __init__ 2020-08-05 09:11:59 +05:30
Marc Hernández 336f19f5cc
Change super syntax (#4707) 2020-08-04 20:42:01 +02:00
Adrián Chaves 4ee538e44b
Update unicode references from Python 2 times in the documentation (#4703) 2020-08-04 20:34:11 +02:00
Adrián Chaves 1278e76d90 Bump version: 2.2.0 → 2.3.0 2020-08-04 20:07:02 +02:00
Adrián Chaves 3600582f56
Cover Scrapy 2.2.1 and 2.3 in the release notes (#4708) 2020-08-04 20:05:56 +02:00
Mikhail Korobov 015b71d89f
Merge pull request #4704 from Gallaecio/python2-u-prefixes 
Remove the u prefix from strings
 2020-07-31 01:23:00 +05:00
Artur Shellunts 5ec66be4a4
Merge branch 'master' into obsolete-s3feednokeys 2020-07-30 22:18:00 +02:00
Adrián Chaves 6f4ccec567 Cover our deprecation policy in the documentation 2020-07-30 14:03:14 +02:00
Adrián Chaves 890b2138a6 Remove the u prefix from strings 2020-07-30 13:39:30 +02:00
Kshitij Sharma 49337bd2ae Code cleanup scrapy.utils.python.WeakKeyCache #4684 and fixing ci alerts 2020-07-30 12:25:21 +05:30