RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,505 Commits 49 Branches 132 Tags 162 MiB
Commit Graph

5505 Commits

Author SHA1 Message Date
Konstantin Lopuhin 8330776c21 fix error reporting in test: we can fail in process_request too, so result should always be defined 2016-01-15 15:16:12 +03:00
Konstantin Lopuhin 4398d95a02 skip this file on py3 again - it has one compression test, sould be done separately 2016-01-15 14:54:12 +03:00
Konstantin Lopuhin ae6758ee97 Merge branch 'py3-http-downloaders' into py3-downloader-middleware 2016-01-15 14:39:27 +03:00
Konstantin Lopuhin a4ca1668d8 add https test for http10 handler (no luck with testing https with http11 so far) 2016-01-15 14:20:19 +03:00
Konstantin Lopuhin 96fcf4cea4 add a check that byte url is not accepted in http.Response on py3 2016-01-15 13:27:28 +03:00
Paul Tremberth 131f463247 Allow failures for Python 3.5 for now 2016-01-15 11:17:52 +01:00
Konstantin Lopuhin 87849780bc some py3 fixes for RFC2616Policy 2016-01-15 13:13:10 +03:00
Konstantin Lopuhin ea0471e33a py3: fix LeveldbCacheStorage - using bytes as keys and values in leveldb 2016-01-15 12:22:41 +03:00
Konstantin Lopuhin ddc91dda27 py3: fix _BaseTest in httpcache 2016-01-15 11:49:28 +03:00
Konstantin Lopuhin e7ed1fd70d py3 compat in httpcache - headers are bytes 2016-01-15 11:48:07 +03:00
Konstantin Lopuhin dbf6cc73d9 py3: add leveldb to py33 test env, fix anydbm module name on py3 2016-01-15 11:46:56 +03:00
Konstantin Lopuhin 94ab7bee6c py3: body to bytes in tests, unskip test file 2016-01-15 11:26:01 +03:00
Konstantin Lopuhin 99f1f2ad1d unskip tests and modules ported to py3 2016-01-14 19:00:48 +03:00
Konstantin Lopuhin 81a90c3af6 unskip part of test_download_gzip_response on py3, file a twisted issue for the remaining part 2016-01-14 18:47:06 +03:00
Konstantin Lopuhin 2aa6c92ffc py3 fixes in tests.mockserver 2016-01-14 17:52:50 +03:00
Konstantin Lopuhin f46a9d595d skip ftp tests on py3 for now 2016-01-14 17:31:58 +03:00
Konstantin Lopuhin 4950f5988e py3: pass http proxy tests 2016-01-14 17:24:08 +03:00
Konstantin Lopuhin c6f14a39de py3: fix http10 downloader - unicode host expected here 2016-01-14 16:50:16 +03:00
Konstantin Lopuhin 6b79fffa9a py3: pass all of HttpTestCase 2016-01-14 16:37:18 +03:00
Konstantin Lopuhin 3509378b8b py3: pass first http downloader test, simple crawler works now, yay! 2016-01-14 16:29:19 +03:00
Konstantin Lopuhin 32bb5b682a fix import of test_downloader_handlers.py: use @implementer, move failing on py3 imports into corresponding tests 2016-01-14 16:11:16 +03:00
Konstantin Lopuhin 5c2241ccc7 py3: fix webclient tests after making ScrapyHTTPClientFactory use bytes as in twisted 2016-01-14 15:30:28 +03:00
Konstantin Lopuhin 8df35bcac6 rm note to self: to be discussed in PR 2016-01-14 14:52:44 +03:00
Konstantin Lopuhin ac2cf191d1 py3: remove comments, utf-8 is fine here: as twisted ultimately uses urllib.parse.quote that assepts bytes and assumes utf-8 2016-01-14 14:50:58 +03:00
Konstantin Lopuhin e5fb609438 make ScrapyHTTPClientFactory comply to twisted HTTPClientFactory protocol - use bytes (encoding are likely wrong at this stage) 2016-01-14 14:45:02 +03:00
Konstantin Lopuhin 6a412d2503 all tests pass in test_webclient.py on py3 - removing from py3-ignores 2016-01-14 13:48:48 +03:00
Konstantin Lopuhin 0178356178 py3 fix testRedirect: url is bytes here 2016-01-14 13:46:46 +03:00
Konstantin Lopuhin 30c7b4e4cc py3 compat in test_timeoutTriggering cleanup 2016-01-14 13:44:14 +03:00
Konstantin Lopuhin 88f55312af py3 fix in testFactoryInfo - factory attirbutes are bytes in twisted 2016-01-14 13:43:37 +03:00
Konstantin Lopuhin b5f9bc8499 py3 test fixes in test_webclient - expect bytes as page body 2016-01-14 13:04:45 +03:00
Konstantin Lopuhin ae4aa2c3b2 py3 test fix: putChild expects bytes as path 2016-01-14 13:04:10 +03:00
Paul Tremberth 85b0e6c9c7 Travis: run tox with Python 3.5 + add Python 3.5 tests 2016-01-14 10:50:51 +01:00
Konstantin Lopuhin 325b6af6c2 fix ScrapyHTTPPageGetterTests for py3 - we expect bytes here 2016-01-14 12:29:59 +03:00
Konstantin Lopuhin 945674eb8f pass test_externalUnicodeInterference - the logic for py3 is clearly inverse of what was expected in this test, as scrapy Request url must be unicode 2016-01-14 12:25:54 +03:00
Konstantin Lopuhin 1d5ab67183 pass test_getPageHead on py3 2016-01-14 12:04:26 +03:00
Konstantin Lopuhin 73ff87c1dc decode body from utf-8, as scrapy stores body as bytes, and twisted has already converted to unicode 2016-01-14 12:03:08 +03:00
Konstantin Lopuhin 9f2be23a39 webclient tests, py3: fix setUp, pass test_getPage 2016-01-14 11:42:23 +03:00
Konstantin Lopuhin f3889b0bce py3 compat: encode delimiter, method and path in ScrapyHTTPPageGetter 2016-01-14 11:41:49 +03:00
Paul Tremberth a93d49a64c Add Python 3.5 tox env and Python 3.4 tests in Travis CI 2016-01-13 13:09:16 +01:00
Paul Tremberth 1347015a80 Refactored test code 2016-01-13 12:32:28 +01:00
Paul Tremberth d4c7d72b2b Add tests for input type in xmliter calls 2016-01-13 12:13:47 +01:00
Elias Dorneles 57f99fc34e Merge pull request #1644 from yarikoptic/master 
[MRG+1] BF: robustify _monkeypatches check for twisted - str() name first (Closes #1634)
 2016-01-13 08:59:48 -02:00
Paul Tremberth 9fad25f3d1 Use explicit Unicode and bytes for XML body in tests 2016-01-13 11:42:41 +01:00
Paul Tremberth 2f2c2e8096 Merge remote-tracking branch 'origin/master' into xmliter-unicode 2016-01-13 11:42:01 +01:00
Mikhail Korobov f01fd07642 Merge pull request #1661 from redapple/py3-iterators 
[MRG+1] PY3: port utils/iterators
 2016-01-13 10:03:02 +05:00
Paul Tremberth d7d4ef67a6 Changes following comments 2016-01-12 11:08:49 +01:00
Paul Tremberth 6ddd814738 Support unicode tags in xml iterators (fixes #1665) 2016-01-12 10:48:45 +01:00
Paul Tremberth d4872940db PY3: port utils/iterators 2016-01-07 10:51:27 +01:00
Elias Dorneles 95e8ff8ba1 Merge pull request #1660 from stummjr/patch-4 
[MRG+1] Update deprecated examples
 2016-01-06 10:43:08 -02:00
Valdir Stumm Jr 2abc9bc901 Update deprecated examples 
* update the scrapy.org example to deal with the new layout.
* replaced slashdot.org by reddit.com, because it seems that slashdot is blocking requests.
 2016-01-06 10:29:45 -02:00