RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,165 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

1368 Commits

Author SHA1 Message Date
Vostretsov Nikita 4be19e443e name signla catcher in accord with signal name 2020-02-06 13:46:23 +00:00
Vostretsov Nikita 4a91a5427d fix typo 2020-02-06 13:44:51 +00:00
Adrián Chaves 576663e5a7 Make METAREFRESH_IGNORE_TAGS an empty list by default 2020-02-06 10:43:20 +01:00
Vostretsov Nikita 3769f75386 pep8 E302 2020-02-05 12:08:08 +00:00
Vostretsov Nikita 9916f6e556 tests for new signal 2020-02-05 11:32:54 +00:00
Eugenio Lacuesta ad70497416
Remove unnecessary parentheses in class definition 2020-02-04 13:30:13 -03:00
Eugenio Lacuesta e0ef8ad2d6
CrawlerRunner test for Response.ip_address 2020-02-03 15:52:15 -03:00
Eugenio Lacuesta 4851efdfb0
Flake8 adjustments 2020-02-03 14:50:54 -03:00
Eugenio Lacuesta bb8f7dc609
Mock DNS server 2020-02-03 14:50:14 -03:00
Eugenio Lacuesta a2ae380efc
Remove unnecessary commas 2020-02-03 13:23:52 -03:00
Eugenio Lacuesta b9e3a6201b
Merge branch 'master' into response_ip_address 2020-02-03 04:07:44 -03:00
Mikhail Korobov 22f7934fcc
Merge pull request #4269 from wRAR/asyncio-parse 
Support for async def callbacks.
 2020-01-31 23:50:19 +05:00
Andrey Rakhmatullin 7687564c73 Merge remote-tracking branch 'origin/master' into asyncio-signals 2020-01-30 17:42:03 +05:00
Andrey Rakhmatullin cc825c21de Test returning items from an async def callback. 2020-01-30 16:17:06 +05:00
Eugenio Lacuesta 6f02a8dccb
Add source parameter to bytes_received signal 2020-01-29 15:51:25 -03:00
Eugenio Lacuesta 2c9643d38c
Test: bytes_received signal fired multiple times 2020-01-29 14:17:06 -03:00
Evgeny Dorofeev 4e56571a19 [HttpCompressionMiddleware] fix delimiter for Accept-Encoding header 2020-01-29 15:49:22 +03:00
Eugenio Lacuesta 613fd41f44
bytes_received signal: improve test performance 2020-01-27 13:25:06 -03:00
Eugenio Lacuesta dbe20a863f
bytes_received signal: send spider argument 2020-01-27 12:28:54 -03:00
Adrián Chaves 89483ce9f7
Fix Flake8 issues 2020-01-27 12:08:56 -03:00
Eugenio Lacuesta bda37e38bd
[Tests] bytes_received signal 2020-01-27 12:08:56 -03:00
Eugenio Lacuesta cab449b195
Typo fix 2020-01-27 12:08:34 -03:00
Eugenio Lacuesta e8da7e2966
Test DNS resolution using CrawlerProcess 2020-01-26 17:53:39 -03:00
Eugenio Lacuesta 80925ab845
Get server IP address for HTTP/1.1 responses 2020-01-26 16:57:04 -03:00
Mikhail Korobov 8b8df31961
Merge pull request #4227 from elacuesta/name-resolver 
Name resolver with IPv6 support
 2020-01-25 05:03:13 +05:00
Mikhail Korobov 5f407cf657
Merge pull request #3961 from OmarFarrag/ftp_files#3928 
Add FTPFileStore to FilesPipeline
 2020-01-25 04:58:23 +05:00
Mikhail Korobov 7a62bd310c
Merge pull request #4126 from elacuesta/from_crawler_downloader_handlers 
Download handlers: from_crawler factory method, take crawler in __init__
 2020-01-25 04:52:56 +05:00
OmarFarrag 40e0a11aa8 Fix Flake8 errors 2020-01-24 15:51:48 +02:00
OmarFarrag fc98aa6b67
Merge branch 'master' into ftp_files#3928 2020-01-24 14:52:40 +02:00
Mikhail Korobov f80c7776ae
Merge pull request #4008 from elacuesta/docs_request_errback 
Request: remove restriction about errback without callback
 2020-01-23 23:12:44 +05:00
Mikhail Korobov c0a7dfbc01
Merge pull request #4057 from elacuesta/response_follow_all 
Response.follow_all
 2020-01-23 02:15:24 +05:00
Eugenio Lacuesta dee420a69c
Fix name resolvers tests 2020-01-16 23:48:16 -03:00
Eugenio Lacuesta d487498cff
Update name resolvers tests 2020-01-16 22:02:01 -03:00
Eugenio Lacuesta 90e3bd8715
[test] Name resolvers 2020-01-16 20:32:40 -03:00
Eugenio Lacuesta 6ce1ad3107
[test] Spider middleware: catch exceptions right after the spider callback 2020-01-10 04:20:37 -03:00
Andrey Rakhmatullin a91a13b443 Support for async def callbacks. 2020-01-09 15:59:19 +05:00
Andrey Rakhmatullin 3faef2d082 Add async def support to signal handlers that already supported Deferreds. 2020-01-09 15:17:45 +05:00
Andrey Rakhmatullin 7d85984880 Use get_from_asyncio_queue in the pipeline test. 2020-01-09 14:49:16 +05:00
Andrey Rakhmatullin bdef948aae Mark the asyncio pipelines test as only_asyncio. 2020-01-09 14:19:02 +05:00
Andrey Rakhmatullin bfdd552a32 Add a test for pipelines using asyncio. 2020-01-09 14:19:02 +05:00
Andrey Rakhmatullin 1f9cef787d Add async def support to pipelines. 2020-01-09 14:19:02 +05:00
Eugenio Lacuesta 2e405d2d5c
Merge branch 'master' into response_follow_all 2020-01-05 00:33:19 -03:00
Andrey Rakhmatullin 2b9254c2bd Add a test function that uses asyncio.Queue(). 2019-12-31 17:54:41 +05:00
Andrey Rakhmatullin 16787f5bf4 Merge middleware tests back as we don't need to set the setting anymore. 2019-12-30 12:02:19 +05:00
Andrey Rakhmatullin 5cf1ac0005 Move the asyncio downloader mw test to a separate class. 2019-12-30 11:46:45 +05:00
Andrey Rakhmatullin 3603644552 Add a non-asyncio async def middleware test. 2019-12-30 11:46:45 +05:00
Andrey Rakhmatullin 21f50c795a Add async def support to downloader middlewares. 2019-12-30 11:46:45 +05:00
Mikhail Korobov bb991cd303
Merge pull request #4010 from scrapy/asyncio-base 
Base support for asyncio
 2019-12-30 00:51:28 +05:00
Andrey Rakhmatullin dc1ee09481 Rename ASYNCIO_ENABLED to ASYNCIO_REACTOR, change the logic accordingly. 2019-12-27 21:56:28 +05:00
Andrey Rakhmatullin 37ac47ff80 Fix a deprecation warning. 2019-12-26 20:46:54 +05:00
Eugenio Lacuesta 982a66f9fb
[test] Download handler: avoid passing settings if not necessary 2019-12-23 20:28:17 -03:00
Eugenio Lacuesta a6ec89251e
Downloader handlers: crawler=None in __init__ 2019-12-23 10:47:08 -03:00
Eugenio Lacuesta 8a567e98bb
Remove unnecessary __init__ methods in downloader handler tests 2019-12-23 10:36:46 -03:00
Eugenio Lacuesta 7e6387de40
Use create_instance in FTPDownloadHandler/DataURIDownloadHandler tests 2019-12-23 10:02:58 -03:00
Eugenio Lacuesta fa21d8687a
Use create_instance in S3DownloadHandler tests 2019-12-23 10:00:25 -03:00
Eugenio Lacuesta 9e5d945ef2
Use create_instance in downloader handler tests 2019-12-23 09:55:47 -03:00
Eugenio Lacuesta 931b7e68d3
Update FileDownloadHandler test 2019-12-23 09:50:28 -03:00
Eugenio Lacuesta 310cc081d0
Merge branch 'master' into from_crawler_downloader_handlers 2019-12-21 17:24:45 -03:00
Andrey Rakhmatullin 8de80f59db Raise an exception if ASYNCIO_ENABLED but the reactor is wrong. 2019-12-21 13:08:29 +05:00
Mikhail Korobov 5577d4d2be
Merge pull request #3869 from elacuesta/detect_return_in_generator_callbacks 
[MRG+1][Py3] Check for 'return' with arguments inside generators
 2019-12-20 22:46:46 +05:00
Andrey Rakhmatullin e342de5038 Remove a stray newline. 2019-12-20 19:37:50 +05:00
Andrey Rakhmatullin 20289be810 Merge remote-tracking branch 'origin/master' into asyncio-base 2019-12-20 19:31:20 +05:00
Mikhail Korobov fb3fb17c28
Merge pull request #4045 from Gallaecio/documentation-coverage 
Deprecate FilteringLinkExtractor direct usage; add API docs for scrapy.linkextractors
 2019-12-20 02:01:45 +05:00
Mikhail Korobov 3bcb14fe4d
Merge pull request #4164 from Gallaecio/speed-up-tests-2 
Improve the performance of the DOWNLOAD_DELAY test
 2019-12-18 23:56:56 +05:00
Marc Hernandez Cabot a5de2c64e6 fix W291, W292, W293 (whitespaces) 2019-12-18 16:33:46 +01:00
elacuesta 916382e109 Add errback parameter to scrapy.spiders.crawl.Rule (#4000) 
* Add errback parameter to scrapy.spiders.crawl.Rule

* CrawlSpider: optimize by reducing iterations

* [test] Rule.errback

* [doc] Rule.errback

* [doc] Use autoclass in docs/topics/spiders.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Rule.process_links takes a list

* Fix aesthetic issue reported by Flake8
 2019-12-18 20:05:33 +05:00
Andrey Rakhmatullin 7ccb169a27 Split a long test in test_engine.py into three. 2019-12-18 19:41:16 +05:00
Mikhail Korobov e90f276903
Merge pull request #4198 from wRAR/deprecate-noconnect 
Deprecate the HTTPS proxy noconnect mode.
 2019-12-18 16:11:24 +05:00
Adrián Chaves 1f689b50c6 Merge remote-tracking branch 'origin/documentation-coverage' into documentation-coverage 2019-12-18 12:09:34 +01:00
Adrián Chaves 174769a3f0 Use a better name for the LxmlLinkExtractor subclassing test 2019-12-18 12:09:03 +01:00
Andrey Rakhmatullin ac302c3f61 Fix a flake8 problem. 2019-12-18 15:43:05 +05:00
Andrey Rakhmatullin bb2ff13e4c Skip Http10ProxyTestCase.test_download_with_proxy_https_noconnect 2019-12-18 15:39:08 +05:00
Andrey Rakhmatullin 2d92a39003 Restore test_download_with_proxy_https_noconnect, check for a warning there. 2019-12-18 12:07:08 +05:00
Adrián Chaves 607815dfc2
Merge branch 'master' into documentation-coverage 2019-12-17 16:54:27 +01:00
Andrey Rahmatullin bb3f164280
Merge pull request #4236 from wRAR/pipeline-tests 
Add simple tests for pipelines.
 2019-12-17 19:38:06 +05:00
Marc Hernández 63cf5c75c8 Fix E502: backslash is redundant between brackets (#4238) 2019-12-17 13:53:15 +01:00
Andrey Rakhmatullin 5980d3bbff Add simple tests for pipelines. 2019-12-16 22:41:28 +05:00
Eugenio Lacuesta 451e7a616e
Scan callbacks/errbacks for return statements with values different than None 2019-12-16 11:30:40 -03:00
Adrián Chaves a4ef9750f9 Fix Flake8-reported issues 2019-12-13 14:32:06 +01:00
Adrián Chaves b5c4c2cae8
Keep 2 spaces between code and inline comments (#4195) 2019-12-13 14:20:48 +01:00
Andrey Rakhmatullin bfb78b8dea Add CrawlerProcess tests for ASYNCIO_ENABLED. 2019-12-13 18:12:07 +05:00
Andrey Rakhmatullin 3560123090 Rename ASYNCIO_SUPPORT to ASYNCIO_ENABLED. 2019-12-13 18:07:47 +05:00
Andrey Rakhmatullin 352ddbb336 Merge remote-tracking branch 'origin/master' into asyncio-base 2019-12-13 18:07:23 +05:00
Andrey Rahmatullin 02cdc53fb8
Add a test for a CrawlerProcess script. (#4218) 
* Add a test for a CrawlerProcess script.

* Add tests/CrawlerProcess to collect_ignore.

* Remove an extra line.

* Fix/improve conftest.py.
 2019-12-13 18:04:05 +05:00
Eugenio Lacuesta 21a2f8c6f8
Merge branch 'master' into response_follow_all 2019-12-10 17:30:59 -03:00
Adrián Chaves 1fc2b140c1
Merge branch 'master' into documentation-coverage 2019-12-05 14:43:36 +01:00
Andrey Rahmatullin 076f0764b7
Merge pull request #4121 from scrapy/remove-six-code 
Remove six-related code and __future__ imports
 2019-12-05 18:20:14 +05:00
Mikhail Korobov 250da28952
Merge pull request #4170 from mabelvj/4133-handle-start_url 
Raise error when start_url found instead of start_urls.
 2019-12-05 17:47:03 +05:00
Andrey Rakhmatullin ed34ce14c0 Add the ASYNCIO_SUPPORT setting, reshuffle other logic accordingly. 2019-12-04 21:32:56 +05:00
Andrey Rakhmatullin cc19ab5439 Add tests that check asyncio support. 2019-12-04 21:32:56 +05:00
Andrey Rakhmatullin b04b541372 Install the asyncio reactor only in scrapy.cmdline. 2019-12-04 21:32:56 +05:00
Andrey Rakhmatullin 8d8fbddbde Switch to the released version of pytest-twisted. 2019-12-04 21:32:56 +05:00
Andrey Rakhmatullin 9777639533 Run tests using the asyncio reactor. 2019-12-04 21:32:56 +05:00
Andrey Rakhmatullin 1b35260625 Add a test for downloader middlewares using Deferreds. 2019-12-04 21:32:56 +05:00
Eugenio Lacuesta 62778cf23f
Request: remove restriction about errback without callback 2019-12-04 08:53:38 -03:00
Eugenio Lacuesta 2a9f5a0aef
Skip invalid links when passing SelectorLists to Response.follow_all 2019-12-03 16:01:42 -03:00
Eugenio Lacuesta e43f37fff3
Pass args/kwargs in S3DownloadHandler.from_crawler, update tests 2019-12-02 17:36:31 -03:00
Eugenio Lacuesta 3d77f74e40
Download handlers: from_crawler factory method, take crawler instead of settings in __init__ 2019-12-02 17:36:31 -03:00
Andrey Rakhmatullin 63546cbf3e Deprecate the HTTPS proxy noconnect mode. 2019-11-27 22:42:52 +05:00
Adrián Chaves 6d9ed6146d
Merge branch 'master' into remove-six-code 2019-11-25 10:34:21 +01:00
Eugenio Lacuesta 03af8885ff
LogFormatter.download_error 2019-11-23 20:02:44 -03:00
Eugenio Lacuesta 4756e7c587
LogFormatter.spider_error 2019-11-23 19:33:29 -03:00
Eugenio Lacuesta facb926542
Remove quotes from item_error message 2019-11-23 19:16:41 -03:00
Eugenio Lacuesta 7a7d13b112
Rename LogFormatter.error to item_error 2019-11-23 19:12:03 -03:00
Eugenio Lacuesta 40b5cfc0a4
Item loaders: allow single-argument processors (unbound methods) 2019-11-22 20:47:22 -03:00
Eugenio Lacuesta 6fa8f06b64
Merge remote-tracking branch 'upstream/master' into response_follow_all 2019-11-22 13:05:06 -03:00
Eugenio Lacuesta 5bab3c0261
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-22 12:12:29 -03:00
Mikhail Korobov 16e0636dcf
Merge pull request #4186 from Gallaecio/lgtm 
Remove unused imports
 2019-11-22 12:28:29 +05:00
Mabel Villalba 070b3a4e84
Merge branch 'master' into 4133-handle-start_url 2019-11-21 17:10:31 +01:00
Mabel Villalba 1718e450ef [start_url] Fixes #4133: Raise AttributeError error when empty 'start_urls' and 'start_url' found. Added test. 2019-11-21 17:06:47 +01:00
Mikhail Korobov d2b73b8e21
Merge pull request #4114 from scrapy/remove-py2-tests 
Remove py2 tests
 2019-11-21 18:22:10 +05:00
Adrián Chaves a2bf340bab Remove unused imports 2019-11-21 14:21:17 +01:00
Eugenio Lacuesta 714e0d3ac9
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-20 12:41:38 -03:00
Mikhail Korobov 65e2fb7d89
Merge pull request #4153 from Gallaecio/lgtm 
MutableChain: return self from __iter__
 2019-11-20 19:46:20 +05:00
Mikhail Korobov e96ca89a1d
Merge pull request #4174 from Gallaecio/flake8 
Add missing blank lines between functions and classes
 2019-11-20 19:34:04 +05:00
Eugenio Lacuesta 6781d2f5b2
Update sample file references 2019-11-20 09:58:25 -03:00
Eugenio Lacuesta 48223c107a
Merge remote-tracking branch 'upstream/master' into response_follow_all 2019-11-20 09:41:42 -03:00
Eugenio Lacuesta b602c61e1c
[Test] Rename outdated sample files 2019-11-20 09:38:54 -03:00
Eugenio Lacuesta 6f6426559d
Merge remote-tracking branch 'upstream/remove-py2-tests' into remove-six-code 2019-11-19 11:04:46 -03:00
Eugenio Lacuesta 05785c1c17
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-19 11:01:34 -03:00
Andrey Rahmatullin 78ad01632f Fix flake8 problems in PR #3989 (#4176) 2019-11-19 10:43:30 +01:00
Andrey Rakhmatullin beb7d80d6a Add a comment about the noconnect tests. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 922a66cf07 Fix or ignore flake8 problems. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin a7b640991d Rename tests/py3-ignores.txt to tests/ignores.txt. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 8b730a3670 Use self.proc.communicate() after killing mitmdump. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 5970d00eb9 Only xfail test_https_connect_tunnel_error on 3.6+. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 5080180c75 Improve the test_https_tunnel_without_leak_proxy_authorization_header change. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin c4ef950efd Use an older mitmproxy for py3.5. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 3ec6960732 Fix test_proxy_connect.py for py3.5. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin c327ad9ba6 Remove an unused six import. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin cbb6d0c6a7 Mark failing proxy tests. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin f066257e95 Restore tests/test_proxy_connect.py and update it to modern mitmproxy. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin e18014d84d Remove Python 2-only tests. 2019-11-19 13:52:49 +05:00
Andrey Rahmatullin 25cd7ac7ab
Merge pull request #4094 from victor-torres/invalid-url 
Improve URL schema validation on scrapy.Request initialization
 2019-11-19 13:50:08 +05:00
Andrey Rahmatullin 3408b757c1
Merge pull request #3989 from elacuesta/logformatter_error_method 
LogFormatter improvements
 2019-11-19 13:44:43 +05:00
Adrián Chaves f261cf65e9 Add missing blank lines between functions and classes 
Also fixed 2 unrelated Flake8 issues
 2019-11-18 17:16:09 +01:00
Adrián Chaves 99d8b05a0b Deprecate scrapy.utils.python.MutableChain.next 2019-11-18 10:58:47 +01:00
Marc Hernandez Cabot 0e252f5a13 fix E711 and E713 2019-11-15 19:12:43 +01:00
Mikhail Korobov 550d8046fd
Merge pull request #4157 from Gallaecio/flake8 
Add missing whitespace after ‘,’, ‘;’ or ‘:’
 2019-11-14 23:42:06 +05:00
Andrey Rakhmatullin 3b2289ad01 Rename test_non_str_url_py2 to test_bytes_url. 2019-11-14 22:53:28 +05:00
Andrey Rakhmatullin fe3a121f13 Use kwargs when calling get_func_args. 2019-11-14 22:50:53 +05:00
Adrián Chaves 058bdda0af Improve the performance of the DOWNLOAD_DELAY test 2019-11-14 16:53:20 +01:00
Andrey Rakhmatullin e291460db6 Fix flake8-detected errors. 2019-11-14 15:24:37 +05:00
Adrián Chaves 1a4a77d49f Remove Python 2 check from MutableChainTest 2019-11-14 10:24:31 +01:00
Andrey Rakhmatullin 87c23ba22d Remove Py2-only code that checks sys.version_info. 2019-11-14 13:45:39 +05:00
Andrey Rakhmatullin a138fb05d4 Replace to_native_str calls with to_unicode. 2019-11-14 13:45:39 +05:00
Andrey Rakhmatullin 92ffd2f249 Simplify some more imports. 2019-11-14 13:45:39 +05:00
Andrey Rakhmatullin 85e79ae792 Remove cStringIO imports. 2019-11-14 13:45:39 +05:00
Andrey Rakhmatullin 1d7c8cb0b1 Remove six.PY2 and six.PY3 conditionals. 2019-11-14 13:45:37 +05:00
Adrián Chaves 33ef24c757 Add missing whitespace after ‘,’, ‘;’ or ‘:’ 2019-11-13 10:52:05 +01:00
Andrey Rahmatullin c911e80209
Merge pull request #4088 from ammarnajjar/4086-constructor-initializer-docs 
docs: use __init__ method instead of constructor
 2019-11-12 21:48:16 +05:00
Andrey Rahmatullin c91390558d
Merge pull request #4074 from purvaudai/master 
Added Pathlib.Path support: Issue #3731
 2019-11-12 21:43:28 +05:00
purvaudai 4b8b0345e5 Mades Changes as per review 2019-11-12 18:17:15 +05:30
Andrey Rahmatullin 93385e647a
Merge pull request #4123 from elacuesta/utils-local-cache-limit 
Fix LocalCache limit issue, add tests
 2019-11-12 16:05:22 +05:00
purvaudai 50eaabe1fc Added Test 2019-11-11 20:00:26 +05:30
purvaudai f39ff49458 Added Test 2019-11-11 18:54:21 +05:30
purvaudai 970c3be160 Added Test 2019-11-11 18:34:15 +05:30
purvaudai 9e6e2dde2b Adding test 2019-11-11 16:10:37 +05:30
purvaudai 0042c389eb Adding test 2019-11-11 15:57:58 +05:30
purvaudai 084a1cda6d Adding test 2019-11-11 15:41:00 +05:30
Eugenio Lacuesta 07fa768d33
Merge branch 'remove-six-code' of github.com:scrapy/scrapy into remove-six-code 2019-11-08 22:17:12 -03:00
Eugenio Lacuesta b6bbb28197
PEP8 adjustments 2019-11-08 22:13:03 -03:00
Eugenio Lacuesta 342e3b5bd5
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-08 12:29:15 -03:00
Adrián Chaves 1df5755699 Set the bases for testing code examples from the documentation 2019-11-08 16:00:10 +01:00
elacuesta 44f19df311
[test] Update mitmproxy version 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-11-08 11:32:50 -03:00
Marc Hernández c377c14e32 Fix W391 Blank line at end of file (#4137) 2019-11-07 17:47:35 +01:00
elacuesta fe31695ba0
Remove unused import (urllib.parse.unquote) 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-11-05 15:36:19 -03:00
Eugenio Lacuesta 613c66a034
Do not override built-in max function 2019-11-05 09:45:51 -03:00
elacuesta fed9fbe62d
Update tests/test_utils_datatypes.py 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-11-04 15:34:27 -03:00
Eugenio Lacuesta 439a3e59b8
Fix scrapy.utils.datatypes.LocalCache limit issue 2019-11-04 10:35:58 -03:00
Eugenio Lacuesta 00b793dc59
Remove elluding six occurrences 2019-11-03 12:26:38 -03:00
Eugenio Lacuesta 5797aefd4c
Remove six.assertCountEqual 2019-11-03 12:18:35 -03:00
Eugenio Lacuesta 586b25d27e
Remove six types 2019-11-03 12:10:37 -03:00
Eugenio Lacuesta 7f3cb05d8e
Remove metaclass-related six code 2019-11-03 12:03:02 -03:00
Eugenio Lacuesta cca1959c07
Merge remote-tracking branch 'upstream/remove-py2-tests' into remove-six-code 2019-11-03 01:20:08 -03:00
Eugenio Lacuesta d72444b9c8
Remove more six imports 2019-11-03 01:11:28 -03:00
Eugenio Lacuesta eaeaa40b99
Remove six.PY* checks 2019-11-03 01:08:08 -03:00
Eugenio Lacuesta 5d8abdde59
Remove six.text_type from tests 2019-11-03 01:01:23 -03:00
Eugenio Lacuesta 54a786b102
Remove six imports 2019-11-03 00:58:47 -03:00
Eugenio Lacuesta 1aba513693
Remove six.iter* occurrences 2019-11-03 00:26:44 -03:00
Eugenio Lacuesta 5ab0f189ce
Remove six.moves occurrences from tests 2019-11-03 00:01:09 -03:00
Eugenio Lacuesta c0bfaef37a
Remove __future__ imports from tests 2019-11-02 23:27:04 -03:00
Andrey Rakhmatullin 350aa67c3d Rename tests/py3-ignores.txt to tests/ignores.txt. 2019-11-01 19:52:57 +05:00
Andrey Rakhmatullin 4b0cdf7f3e Use self.proc.communicate() after killing mitmdump. 2019-11-01 19:51:52 +05:00
Andrey Rakhmatullin 3c9963ab04 Only xfail test_https_connect_tunnel_error on 3.6+. 2019-11-01 19:47:12 +05:00
Andrey Rakhmatullin e0c5c72496 Improve the test_https_tunnel_without_leak_proxy_authorization_header change. 2019-11-01 19:46:19 +05:00
Andrey Rakhmatullin 5eb01b617d Use an older mitmproxy for py3.5. 2019-10-31 23:21:30 +05:00
Andrey Rakhmatullin a5eb59b92d Fix test_proxy_connect.py for py3.5. 2019-10-31 23:21:14 +05:00
Adrián Chaves f02c3d1dcf
Use communicate() instead of wait() after killing the mock server (#4095) 2019-10-31 13:31:33 +01:00
Andrey Rakhmatullin 7299e91b1f Remove Py2-only code that checks sys.version_info. 2019-10-31 15:22:53 +05:00
Andrey Rakhmatullin 397e883556 Replace to_native_str calls with to_unicode. 2019-10-31 15:22:53 +05:00
Andrey Rakhmatullin 75b1d051d9 Simplify some more imports. 2019-10-31 15:22:45 +05:00
Andrey Rakhmatullin 3ac4b430ae Remove an unused six import. 2019-10-31 15:20:28 +05:00
Andrey Rakhmatullin cea2f5e244 Remove cStringIO imports. 2019-10-31 15:07:54 +05:00
Andrey Rakhmatullin bbd9f4be90 Remove six.PY2 and six.PY3 conditionals. 2019-10-31 15:07:10 +05:00
Andrey Rakhmatullin 186f9d88ac Fix the skip message for test_download_gzip_response. 2019-10-31 15:03:44 +05:00
Andrey Rakhmatullin 439e37fc7b Mark failing proxy tests. 2019-10-31 15:03:44 +05:00
Andrey Rakhmatullin b0d6f4917d Restore tests/test_proxy_connect.py and update it to modern mitmproxy. 2019-10-31 15:03:44 +05:00
Andrey Rakhmatullin b44bd6f825 Remove Python 2-only tests. 2019-10-31 15:03:09 +05:00
Adrián Chaves 15c55d0c1d
Remove LevelDB support (#4112) 2019-10-31 10:47:29 +01:00