RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,236 Commits 49 Branches 132 Tags 160 MiB
Commit Graph

610 Commits

Author SHA1 Message Date
OmarFarrag 0e8770a2f4 test for files pipeline ftp store 2019-09-06 15:47:57 +02:00
OmarFarrag 2047124b35 Follow PEP8 .. Remove unnecessary comments 2019-08-22 16:18:14 +02:00
OmarFarrag bd22b25ef4 Make `stat_file` thread safe .. Refactor file storing.. Support act/psv 2019-08-22 01:30:15 +02:00
Mikhail Korobov b2c013feca
Merge pull request #3878 from elacuesta/mergedict_to_chainmap 
Deprecate scrapy.utils.datatypes.MergeDict in favor of collections.ChainMap
 2019-07-17 14:31:50 +05:00
Eugenio Lacuesta d7074d86d2
Change condition to raise deprecation warning 2019-07-16 14:13:45 -03:00
Eugenio Lacuesta 7e3a602d56
Suggested changes 2019-07-15 12:37:09 -03:00
Eugenio Lacuesta 62f3e22481
Deprecate scrapy.utils.datatypes.MergeDict in favor of collections.ChainMap 2019-07-13 22:51:38 -03:00
Eugenio Lacuesta eced544d64
Prevent even more DeprecationWarnings 2019-07-13 22:16:23 -03:00
Eugenio Lacuesta 40086dabb8 Prevent more DeprecationWarnings 2019-07-13 22:14:00 -03:00
Eugenio Lacuesta 27e63e6890
Monkey patch configparser module to ease the dropping of py2 support 2019-07-13 20:10:06 -03:00
Adrián Chaves 9cab3f247d
Merge pull request #3859 from scrapy/more-deprecations 
More deprecations
 2019-07-08 09:46:48 +02:00
Mikhail Korobov cb4477db3e deprecate scrapy.utils.markup 
it was an import-only shim for w3lib.html
 2019-07-08 11:18:40 +05:00
Mikhail Korobov 64ff3cd6aa deprecate scrapy.utils.multipart 
It was a shim for w3lib.form, but w3lib.form is deprecated as well.
 2019-07-08 11:18:15 +05:00
Mikhail Korobov 1130711cc1 deprecate scrapy.utils.http 
ChunkedTransferMiddleware is deprecated, so decode_chunked_transfer
can be deprecated as well.
 2019-07-08 11:17:30 +05:00
nyov 7a398b7086 Deprecation removals for Scrapy 1.7 
Removing deprecations of 2015 and prior (pre-1.1)
 2019-07-06 13:41:56 +00:00
Eugenio Lacuesta 428309ba1a Merge remote-tracking branch 'origin/master' into callback_kwargs 2019-06-26 11:03:31 -03:00
Mikhail Korobov c81d120bde
Merge pull request #3768 from Gallaecio/meta-ignore-tags 
Implement the METAREFRESH_IGNORE_TAGS setting
 2019-06-25 23:25:58 +03:00
Daniel Graña b53ff59a22
Merge pull request #3790 from andrewbaxter/master 
[MRG+1] Account for mangling when serializing requests with private callbacks
 2019-06-17 12:33:21 -03:00
Andrew Baxter 9c81721c40 Add tests for private method name mangling 2019-06-05 23:43:56 +09:00
Andrew Baxter bcad8947e8 Support inherited private method names 2019-06-03 20:41:02 +09:00
Andrew Baxter 9af91a26b0 Replace regex usage 2019-05-28 01:40:26 +09:00
Andrew Baxter 72b7d3e90a Make the regex align to the spec better; add unit tests for name variations 2019-05-27 23:30:23 +09:00
Andrew Baxter 144afcee79 Use regex to check for private methods 2019-05-25 00:52:00 +09:00
Andrew Baxter e667ca7682 Account for mangling when serializing requests with private callbacks 2019-05-24 21:45:53 +09:00
Adrián Chaves 611249bb7f Implement the METAREFRESH_IGNORE_TAGS setting 2019-05-08 12:52:29 +02:00
Matthijs Vos f6485e6697 Restore alphabetic order and two lines between import and code 2019-05-03 13:55:18 +02:00
Matthijs Vos 7809c0b14e Revert another non-change comment 2019-04-20 09:25:01 +02:00
Matthijs Vos 935387aaea Revert some non-changes 2019-04-18 22:10:23 +02:00
Matthijs Vos 07adca34e1 Fix 'Too many values to unpack' 2019-04-10 13:01:46 +02:00
Matthijs Vos d27c2c68ba Wrap scrapy check in environment 2019-04-10 12:56:50 +02:00
Kostis Anagnostopoulos 3a493b6066 fix: do not catch system exceptions like KeyboardInterrupt 2019-04-05 11:52:00 +02:00
Adrián Chaves 7acf4eec79 Deprecate the scrapy.utils.gz.is_gzipped function 2019-04-02 18:36:03 +02:00
Konstantin Lopuhin b5c552d17f
Merge pull request #2061 from elacuesta/process_spider_exception_generator 
[MRG+1] process_spider_exception on generators
 2019-04-01 10:43:19 +03:00
Eugenio Lacuesta c43a23152b Merge remote-tracking branch 'upstream/master' into callback_kwargs 2019-03-27 14:42:41 -03:00
Eugenio Lacuesta 6760bca74b Rename Request.kwargs to Request.cb_kwargs 2019-03-15 22:42:29 +00:00
Adrián Chaves 2bd5790d9b
Merge pull request #3629 from johndela1/master 
[MRG+1] rel_has_nofollow: remove redundant if statement
 2019-03-15 07:53:23 +01:00
Mikhail Korobov 5dc94db847
Merge branch 'master' into backward 2019-03-14 22:21:09 +05:00
Adrián Chaves 75d6f56c8a Switch from ` to `` where inline code formatting is desired 2019-03-01 16:56:58 +01:00
Adrián Chaves 858f5be747 backwards → backward (adj.) 2019-03-01 16:10:23 +01:00
John de la Garza b02d26fae8 rel_has_nofollow: remove redundant if statement 2019-02-15 16:54:33 -08:00
Eugenio Lacuesta a67f1ce512 Serialize Request kwargs 2019-01-03 18:19:12 -03:00
Eugenio Lacuesta a05eaeed73 Simplify MutableChain 2018-10-11 11:31:51 -03:00
Eugenio Lacuesta 58f5565357 Move MutableChain to scrapy.utils.python 2018-10-11 11:23:12 -03:00
Eugenio Lacuesta 917c1fde68 Merge branch 'master' into enhancement/alternate-feedexport-constructors 2018-07-20 12:09:50 -03:00
Eugenio Lacuesta 98d74d1083 Requested changes 2018-07-20 12:08:49 -03:00
Konstantin Lopuhin f4f39057cb Make csviter work on python 3.7 
PEP 479 does not allow for StopIteration in generators.  Instead,
handle it explicitly, also use a for loop which looks simpler.
 2018-07-09 12:46:45 +03:00
Mikhail Korobov f531b66822 SpiderManager shim is removed 2018-07-06 03:28:01 +05:00
Mikhail Korobov d4c7cc848b remove backwards compatibility shims for relocated modules 2018-07-06 03:19:43 +05:00
Fredrik Bergenlid 6a2d2c3b77 Improve gunzip performance for big files 2018-06-01 21:38:07 +02:00
rhoboro 5254ac393b added test for gcs policy 2018-04-03 18:06:34 +09:00