RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,291 Commits 49 Branches 132 Tags 148 MiB
Commit Graph

9291 Commits

Author SHA1 Message Date
Andrey Rakhmatullin b44bd6f825 Remove Python 2-only tests. 2019-10-31 15:03:09 +05:00
Adrián Chaves 15c55d0c1d
Remove LevelDB support (#4112) 2019-10-31 10:47:29 +01:00
Andrey Rahmatullin 229e722a03 Initial Python 2 removal (#4091) 2019-10-31 10:46:02 +01:00
Benjamin Ooghe-Tabanou 6d6da78eda Add a keep_fragments parameter to the request_fingerprint function (#4104) 2019-10-30 09:13:36 +01:00
Amardeep Bhowmick 66cbceeb0a Fix redirection error when the Location header value starts with 3 slashes (#4042) 2019-10-30 09:09:12 +01:00
Adrián Chaves be2e910dd0 Bump version: 1.7.0 → 1.8.0 2019-10-29 12:57:02 +01:00
Adrián Chaves 94f060fcc8
Cover Scrapy 1.8.0 in the release notes (#3952) 2019-10-29 12:53:46 +01:00
Andrey Rahmatullin 18b808b2e9
Merge pull request #4092 from further-reading/master 
Add Python 3.8 official support
 2019-10-29 16:30:58 +05:00
Roy 93e3dc1b82 [test_downloadermiddleware_httpcache.py] Cleaning text 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 16:12:03 +00:00
Roy b73d217de5 [test_downloadermiddleware_httpcache.py] Fixing pytest mark behaviour 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 12:55:54 +00:00
Roy 74909030a5 [tox.ini] Removing obsolete py37 extra deps enviornment 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 12:52:36 +00:00
Roy c51fb959e2 [test_downloadermiddleware_httpcache] Fixing pytest skip behaviour 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 12:36:54 +00:00
Roy 4432136fff [test_downloadermiddleware_httpcache] Fixing pytest skip behaviour 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 12:22:21 +00:00
Roy 9b47dc6a70 [travis, setup] Adding official python 3.8 support 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 11:24:52 +00:00
Roy 16bb3ac20d [test_downloadermiddleware_httpcache] Using skipif approach 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 11:24:09 +00:00
Roy Healy 3af2abb75b
Merge pull request #1 from Gallaecio/py38 
Mark the LevelDB storage backend as deprecated
 2019-10-28 10:35:04 +00:00
Adrián Chaves 3d0df419c4 Mark the LevelDB storage backend as deprecated 2019-10-28 11:24:47 +01:00
Roy Healy b5a00262ec
Update .travis.yml 
Reverting change to 3.8 extra dependency environment.
 2019-10-28 09:59:33 +00:00
elacuesta 7731814cc2 ItemLoader: improve handling of initial item (#4036) 2019-10-28 10:53:53 +01:00
Adrián Chaves bb91f9c78c Cover Scrapy 1.7.4 in the release notes 2019-10-28 09:58:13 +01:00
Roy 20ea912513 [test_downloadermiddleware_httpcache] Making xfails more informative 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 18:52:01 +00:00
Roy 70b2854590 [test_downloadermiddleware_httpcache] Making xfails more informative 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 18:51:26 +00:00
Roy b3df0a8415 [test_downloadermiddleware_httpcache] Adding xfails to impacted tests following hack fix 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 18:28:47 +00:00
Roy 11942c436c [test_downloadermiddleware_httpcache] Trying hack to handle systemerror whjen importing leveldb 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 18:02:13 +00:00
Roy deacd34c8d [test_downloadermiddleware_httpcache] Attempting to add xfail for leveldb related tests 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 17:39:47 +00:00
Roy Healy 4068797558
Update test_downloadermiddleware_httpcache.py 
Adding xfail denoting that leveldb is not supported in 3.8
 2019-10-27 17:02:17 +00:00
Roy Healy 179dc916ff
Update setup.py 
Updating setup.py to remove python 3.8 support for now
 2019-10-27 16:48:20 +00:00
Adrián Chaves 02577f55a0 Have PyLint cover all Python files in the repository 2019-10-24 13:25:11 +02:00
Kevin Lloyd Bernal 84fe4011b0 update docs of scrapy.loader.ItemLoader.item 2019-10-23 20:39:53 +08:00
Adrián Chaves 3b6f7ac9f2 Use pylint 2019-10-22 19:44:11 +02:00
Adrián Chaves c623a16a22 Remove unused method from scrapy.pqueues._SlotPriorityQueues 2019-10-22 17:52:34 +02:00
Ammar Najjar bf5c1a3dec
docs: use "constructor" for "from_crawler" 
Issue #4086
 2019-10-22 15:56:46 +02:00
Ammar Najjar 7fba8434f3
use instantiation for "Crawler" 
Issue #4086

Co-Authored-By: Mikhail Korobov <kmike84@gmail.com>
 2019-10-22 15:55:52 +02:00
Victor Torres f701f5b0db fix #2552 by improving request schema check on its initialization 2019-10-22 10:48:02 -03:00
Ammar Najjar 7a84a4bdba
docs: use "constructor" for from_settings() & rom_crawler() factory methods 
Issue #4086
 2019-10-22 15:31:34 +02:00
Adrián Chaves d96b9f860b Use object as type for parameters that allow any value 2019-10-22 15:24:59 +02:00
Adrián Chaves 5479e7ecc7 Indicate that lists of emails may be provided as a single string or as a list of strings 2019-10-22 15:24:44 +02:00
Adrián Chaves 1d5c270ce8
Fix dangling file descriptor in FeedExporter when FEED_STORE_EMPTY is False (#4023) 2019-10-22 15:12:52 +02:00
Ammar Najjar d21e1034f0
docs: correct point,comma and plural replacements 
Issue #4086
 2019-10-22 13:29:38 +02:00
purvaudai 4184bac068 Added Pathlib.Path test 2019-10-22 16:57:14 +05:30
purvaudai 85f56a92f0 Added Pathlib.Path test 2019-10-22 16:43:17 +05:30
purvaudai 7031e3a124 Added Pathlib.Path test 2019-10-22 16:31:14 +05:30
purvaudai cd09646438 Added Pathlib.Path test 2019-10-22 16:19:41 +05:30
WinterComes 5d75ed4cba Remove an old note about contracts (#4093) 2019-10-22 12:19:07 +02:00
purvaudai cd4c211f4b Added Pathlib.Path test 2019-10-22 15:38:06 +05:30
illgitthat 07822935ec Updating link for miniconda (#4089) 2019-10-22 12:05:34 +02:00
purvaudai a776554282 Added Pathlib.Path test 2019-10-22 15:31:55 +05:30
purvaudai 0b7d8a51b4 Added Pathlib.Path test 2019-10-22 15:12:53 +05:30
purvaudai 4226791481 Added Pathlib.Path test 2019-10-22 15:07:13 +05:30
purvaudai ad96d6ef59 Added Pathlib.Path test correctly 2019-10-22 14:53:59 +05:30