RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,464 Commits 49 Branches 132 Tags 132 MiB
Commit Graph

124 Commits

Author SHA1 Message Date
Andrey Rahmatullin 5e997587d9
Remove dead boto2 code, deprecate is_botocore() (#4776) 2020-09-20 15:06:46 +02:00
Ammar Najjar 7597193dbe
Merge branch 'master' into 4307-use-f-strings 
Resolve Conflicts:
	tests/test_middleware.py
 2020-08-26 13:46:46 +02:00
nyov cf50561b86
Allow passing classes directly in Settings (#3873) 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-08-26 13:08:14 +02:00
Ammar Najjar 58ca8bbf6d Use f-strings (#4307) 2020-08-26 07:11:05 +02:00
Adrián Chaves e70975f0bb
Allow overwriting feeds (#4512) 
Co-authored-by: Yuval Hager <yhager@yhager.com>
 2020-08-17 15:10:08 +02:00
Mikhail Korobov 61459de476
Merge pull request #4688 from ashellunts/obsolete-s3feednokeys 
Remove obsolete S3FeedStorage instancing without AWS credentials
 2020-08-17 13:58:32 +05:00
Artur Shellunts 5ec66be4a4
Merge branch 'master' into obsolete-s3feednokeys 2020-07-30 22:18:00 +02:00
Adrián Chaves 890b2138a6 Remove the u prefix from strings 2020-07-30 13:39:30 +02:00
Artur Shellunts 430d22e46e Remove not used import warnings 2020-07-21 23:39:04 +02:00
nyov 2829cd4268 Allow use without credentials 2020-07-21 23:38:02 +02:00
nyov 98e8086d1b Adapt S3FeedStorage testcase 2020-07-21 23:38:02 +02:00
nyov ece4fa6c7c Fix ignored testcase: boto is never installed 2020-07-21 23:38:01 +02:00
BroodingKangaroo 3e0492741d Another try to fix test errors on Windows 2020-07-19 00:10:29 +03:00
BroodingKangaroo 86f7ac2f2b Try to fix error at Windows 2020-07-17 18:22:47 +03:00
Adrián Chaves c4f9250220
Merge branch 'master' into ISSUE-4250-add_batch_deliveries 2020-07-16 17:44:18 +02:00
Mikhail Korobov 07470e1a3c
Merge pull request #3608 from ejulio/feat-685 
[MRG+1] Fix for #685 Add Google Cloud Storage Feed Export
 2020-07-16 18:11:19 +05:00
BroodingKangaroo 6454d456d2 Make check of placeholder less strict 2020-07-03 10:24:13 +03:00
Adrián Chaves 3199048520 Complete Azure Pipelines CI setup 2020-07-03 00:56:24 +02:00
BroodingKangaroo 1e245046ed Change setting name. Add leading zeroes to batch_id. Minor fixes. 2020-07-02 12:38:08 +03:00
BroodingKangaroo 7b1d3c35ea Minor updates 2020-07-01 12:24:31 +03:00
BroodingKangaroo 05c2587c6a Docs update and tiny fixes 2020-06-28 09:45:45 +03:00
BroodingKangaroo 88a52198b9 Add batch_item_count support in FEEDS setting 2020-06-27 12:45:08 +03:00
BroodingKangaroo c3cee74fd4 Change default value of FEED_STORAGE_BATCH_ITEM_COUNT to 0 2020-06-26 18:45:21 +03:00
BroodingKangaroo 77333666f1 resolve conflicts 2020-06-26 14:55:46 +03:00
Eugenio Lacuesta c7f1c7e3f3
Merge branch 'master' into feat-685 2020-06-19 21:30:54 -03:00
Stanislau Hluboki a4bfd5ab6f Fix duplicated feed logs 2020-06-19 17:51:56 +03:00
Stas Glubokiy 3d027fb578
Fix missing storage.store calls in FeedExporter.close_spider (#4626) 2020-06-17 17:08:14 +02:00
BroodingKangaroo 677e619d37 Fix too long lines 2020-05-21 14:57:03 +03:00
BroodingKangaroo 5f6c1dceb1 Merge remote-tracking branch 'upstream/master' into ISSUE-4250-add_batch_deliveries 2020-05-21 12:43:04 +03:00
BroodingKangaroo 10ae1a284f Minor fixes 2020-05-15 22:50:54 +03:00
Eugenio Lacuesta 3eeecb4273
Merge branch 'master' into flake8-max-line-length 2020-05-15 11:11:20 -03:00
BroodingKangaroo 111a58fe3d Merge remote-tracking branch 'upstream/master' into ISSUE-4250-add_batch_deliveries 2020-05-14 09:53:14 +03:00
BroodingKangaroo 2327ecead0 Rename FEED_STORAGE_BATCH_SIZE to FEED_STORAGE_BATCH_ITEM_COUNT 2020-05-13 22:50:04 +03:00
Eugenio Lacuesta 6f8758624c
Flake8: remove F841 2020-05-11 13:50:34 -03:00
BroodingKangaroo dad2ea7522 Change time_id to batch_time 2020-05-02 01:21:03 +03:00
BroodingKangaroo 3f9874fac9 Add test s3 export 2020-05-01 11:52:16 +03:00
BroodingKangaroo f0f1be76d1 Using time_id instead of time as a timestamp 2020-04-27 09:56:57 +03:00
BroodingKangaroo ec76445dd6 Update tests 2020-04-18 09:29:23 +03:00
Eugenio Lacuesta 7926da4bd0
Merge branch 'master' into flake8-max-line-length 2020-04-16 16:05:09 -03:00
BroodingKangaroo 5980ae72c6 Some minor fixes and refactoring 2020-04-16 10:13:39 +03:00
BroodingKangaroo 16836e9e77 Merge branch 'ISSUE-4250-add_batch_deliveries' of https://github.com/BroodingKangaroo/scrapy into ISSUE-4250-add_batch_deliveries 2020-04-16 00:41:21 +03:00
BroodingKangaroo 963580463b Update tests 2020-04-15 23:39:46 +03:00
BroodingKangaroo 39d0d13d3f Add partial deliveries tests 2020-04-15 23:38:27 +03:00
Eugenio Lacuesta 1bd8f392c9
Initial removal of twisted.internet.defer.returnValue 2020-04-13 06:12:30 -03:00
Eugenio Lacuesta 182394bcec
E501 compliance (tests) 2020-03-22 23:29:30 -03:00
Eugenio Lacuesta 34d607194b
Merge branch 'master' into feat-685 2020-03-18 21:24:36 -03:00
elacuesta dfbe1d9507
Remove object base class (#4430) 2020-03-16 20:12:46 +01:00
Eugenio Lacuesta ada37c5409
Export to multiple formats in a single crawl 2020-03-05 09:33:57 -03:00
Adrián Chaves e1be078eaa Fix Flake8-reported issues 2020-02-12 17:38:06 +01:00
Adrián Chaves 96bb3b5144
Merge branch 'master' into feat-685 2020-02-10 19:45:09 +01:00