RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,947 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

69 Commits

Author SHA1 Message Date
elacuesta dfbe1d9507
Remove object base class (#4430) 2020-03-16 20:12:46 +01:00
Eugenio Lacuesta ada37c5409
Export to multiple formats in a single crawl 2020-03-05 09:33:57 -03:00
Eugenio Lacuesta 5bab3c0261
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-22 12:12:29 -03:00
Adrián Chaves a2bf340bab Remove unused imports 2019-11-21 14:21:17 +01:00
Eugenio Lacuesta 05785c1c17
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-19 11:01:34 -03:00
Andrey Rakhmatullin a138fb05d4 Replace to_native_str calls with to_unicode. 2019-11-14 13:45:39 +05:00
Andrey Rakhmatullin 92ffd2f249 Simplify some more imports. 2019-11-14 13:45:39 +05:00
Andrey Rahmatullin c91390558d
Merge pull request #4074 from purvaudai/master 
Added Pathlib.Path support: Issue #3731
 2019-11-12 21:43:28 +05:00
purvaudai 4b8b0345e5 Mades Changes as per review 2019-11-12 18:17:15 +05:30
purvaudai 50eaabe1fc Added Test 2019-11-11 20:00:26 +05:30
purvaudai f39ff49458 Added Test 2019-11-11 18:54:21 +05:30
purvaudai 970c3be160 Added Test 2019-11-11 18:34:15 +05:30
purvaudai 9e6e2dde2b Adding test 2019-11-11 16:10:37 +05:30
purvaudai 0042c389eb Adding test 2019-11-11 15:57:58 +05:30
purvaudai 084a1cda6d Adding test 2019-11-11 15:41:00 +05:30
Eugenio Lacuesta 5ab0f189ce
Remove six.moves occurrences from tests 2019-11-03 00:01:09 -03:00
Eugenio Lacuesta c0bfaef37a
Remove __future__ imports from tests 2019-11-02 23:27:04 -03:00
Andrey Rakhmatullin 397e883556 Replace to_native_str calls with to_unicode. 2019-10-31 15:22:53 +05:00
Andrey Rakhmatullin 75b1d051d9 Simplify some more imports. 2019-10-31 15:22:45 +05:00
Adrián Chaves 1d5c270ce8
Fix dangling file descriptor in FeedExporter when FEED_STORE_EMPTY is False (#4023) 2019-10-22 15:12:52 +02:00
purvaudai 4184bac068 Added Pathlib.Path test 2019-10-22 16:57:14 +05:30
purvaudai 85f56a92f0 Added Pathlib.Path test 2019-10-22 16:43:17 +05:30
purvaudai 7031e3a124 Added Pathlib.Path test 2019-10-22 16:31:14 +05:30
purvaudai cd4c211f4b Added Pathlib.Path test 2019-10-22 15:38:06 +05:30
purvaudai a776554282 Added Pathlib.Path test 2019-10-22 15:31:55 +05:30
purvaudai ad96d6ef59 Added Pathlib.Path test correctly 2019-10-22 14:53:59 +05:30
purvaudai 2ee38e8ddb Added Pathlib.Path test 2019-10-22 14:43:47 +05:30
Pengyu Chen 7b755a41a1
Added: Properly handling quoted passwords in FEED_URI for FTP 2019-08-06 15:19:57 +01:00
Luiz Francisco Rodrigues da Silva df68c4b9b1 Add a setting to use active mode in FTPFeedStorage (#3829) 
Add a setting to use active mode in FTPFeedStorage

FTP servers can be configured in active and passive mode, by default
Python and Scrapy use passive mode and there was no way to use active.

This commit adds a setting FEED_STORAGE_FTP_ACTIVE to allow a feed
to be exported to a FTP server configured in active mode.
 2019-07-10 14:25:52 +02:00
nyov 5442c2d3c3 Updating S3FeedStorage instancing without AWS key. 2019-07-06 13:41:56 +00:00
Victor Torres fda1d04b81
Merge branch 'master' into feed-storage-s3-acl 2019-03-15 13:26:45 -03:00
Adrián Chaves 858f5be747 backwards → backward (adj.) 2019-03-01 16:10:23 +01:00
Victor Torres 9b8ba4c383 try to import botocore before runing some tests 2019-02-14 16:20:56 -02:00
Victor Torres ea8be627d1 botocore is not supported on debian jessie 2019-02-13 20:32:36 -02:00
Victor Torres dc0b643832 refactoring tests to avoid mocking private method 2019-02-13 19:44:50 -02:00
Victor Torres 1eac2a163c simplifying how we deal with threads.deferToThread calls 2019-02-08 16:50:39 -02:00
Victor Torres f824f5b2d1 testing public method store instead of private method _store_in_thread 
need to mock deferToThread function
 2019-02-08 15:51:23 -02:00
Victor Torres 079af889e7 also testing without botocore 2019-02-08 15:51:23 -02:00
Victor Torres 126207fb7b PEP8: use short name for mock method 2019-02-08 15:51:23 -02:00
Victor Torres ad83ffdf1f refactoring 2019-02-08 15:51:23 -02:00
Daniel Graña 38608bc249 Use ignore_errors option from rmtree 2018-08-15 11:59:09 -03:00
Daniel Grana a304d6b692 Workaround to pass tests/test_feedexporter.py under windows 2018-08-15 02:23:07 -07:00
Jakob de Maeyer 034152961d Fix Feedexport test in Windows by using proper file URI 2018-08-15 01:35:01 -03:00
Eugenio Lacuesta 917c1fde68 Merge branch 'master' into enhancement/alternate-feedexport-constructors 2018-07-20 12:09:50 -03:00
Eugenio Lacuesta 98d74d1083 Requested changes 2018-07-20 12:08:49 -03:00
Vostretsov Nikita b364d27247 [MRG+1] Automatic port selection for servicies in unit tests (#3210) 
* ability to pass port as a parameter

* try to find free ports

* use environment variables to pass mock server address

* get mock server address from environment variables

* ability to select ports for proxy in runtime

* use common method for URLs from mock server

* https support

* get mock server address

* get mock address

* replace hand-written mechanism by kernel-based one

* use ephemeral ports in mockserver

* strip EOL from addresses

* use ephemeral port in proxy

* no need to restore environment as it is restored in tearDown

* decode bytes

* use mockserver address as a variable

* ability to pass address as variable

* per test-case mockserver

* use base class

* remove obsolete environment manipulation

* return usage of proxy for http cases

* common method for broking proxy auth credentials

* python version-independent url methods
 2018-05-23 21:25:50 +03:00
Jakob de Maeyer 4d77c3084e Add from_crawler constructor to S3FeedStorage 2017-07-25 09:41:51 +02:00
Jakob de Maeyer 815d6160cf Add from_crawler constructor for feed exporters and storages 2017-07-25 09:41:29 +02:00
Eugenio Lacuesta 63b8caf5de Feed exports: rewrite indentation test without .strip() 2017-05-09 11:58:53 -03:00
Eugenio Lacuesta c7bb2fa8ce Feed exports: consistent and backwards compatible behaviour on indent 2017-03-07 11:56:00 -03:00