RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,618 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

1118 Commits

Author SHA1 Message Date
Mikhail Korobov e90f276903
Merge pull request #4198 from wRAR/deprecate-noconnect 
Deprecate the HTTPS proxy noconnect mode.
 2019-12-18 16:11:24 +05:00
Andrey Rakhmatullin ac302c3f61 Fix a flake8 problem. 2019-12-18 15:43:05 +05:00
Andrey Rakhmatullin bb2ff13e4c Skip Http10ProxyTestCase.test_download_with_proxy_https_noconnect 2019-12-18 15:39:08 +05:00
Andrey Rakhmatullin 2d92a39003 Restore test_download_with_proxy_https_noconnect, check for a warning there. 2019-12-18 12:07:08 +05:00
Andrey Rahmatullin bb3f164280
Merge pull request #4236 from wRAR/pipeline-tests 
Add simple tests for pipelines.
 2019-12-17 19:38:06 +05:00
Marc Hernández 63cf5c75c8 Fix E502: backslash is redundant between brackets (#4238) 2019-12-17 13:53:15 +01:00
Andrey Rakhmatullin 5980d3bbff Add simple tests for pipelines. 2019-12-16 22:41:28 +05:00
Adrián Chaves b5c4c2cae8
Keep 2 spaces between code and inline comments (#4195) 2019-12-13 14:20:48 +01:00
Andrey Rahmatullin 02cdc53fb8
Add a test for a CrawlerProcess script. (#4218) 
* Add a test for a CrawlerProcess script.

* Add tests/CrawlerProcess to collect_ignore.

* Remove an extra line.

* Fix/improve conftest.py.
 2019-12-13 18:04:05 +05:00
Andrey Rahmatullin 076f0764b7
Merge pull request #4121 from scrapy/remove-six-code 
Remove six-related code and __future__ imports
 2019-12-05 18:20:14 +05:00
Mikhail Korobov 250da28952
Merge pull request #4170 from mabelvj/4133-handle-start_url 
Raise error when start_url found instead of start_urls.
 2019-12-05 17:47:03 +05:00
Andrey Rakhmatullin 63546cbf3e Deprecate the HTTPS proxy noconnect mode. 2019-11-27 22:42:52 +05:00
Adrián Chaves 6d9ed6146d
Merge branch 'master' into remove-six-code 2019-11-25 10:34:21 +01:00
Eugenio Lacuesta 40b5cfc0a4
Item loaders: allow single-argument processors (unbound methods) 2019-11-22 20:47:22 -03:00
Eugenio Lacuesta 5bab3c0261
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-22 12:12:29 -03:00
Mikhail Korobov 16e0636dcf
Merge pull request #4186 from Gallaecio/lgtm 
Remove unused imports
 2019-11-22 12:28:29 +05:00
Mabel Villalba 070b3a4e84
Merge branch 'master' into 4133-handle-start_url 2019-11-21 17:10:31 +01:00
Mabel Villalba 1718e450ef [start_url] Fixes #4133: Raise AttributeError error when empty 'start_urls' and 'start_url' found. Added test. 2019-11-21 17:06:47 +01:00
Mikhail Korobov d2b73b8e21
Merge pull request #4114 from scrapy/remove-py2-tests 
Remove py2 tests
 2019-11-21 18:22:10 +05:00
Adrián Chaves a2bf340bab Remove unused imports 2019-11-21 14:21:17 +01:00
Eugenio Lacuesta 714e0d3ac9
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-20 12:41:38 -03:00
Mikhail Korobov 65e2fb7d89
Merge pull request #4153 from Gallaecio/lgtm 
MutableChain: return self from __iter__
 2019-11-20 19:46:20 +05:00
Mikhail Korobov e96ca89a1d
Merge pull request #4174 from Gallaecio/flake8 
Add missing blank lines between functions and classes
 2019-11-20 19:34:04 +05:00
Eugenio Lacuesta 6f6426559d
Merge remote-tracking branch 'upstream/remove-py2-tests' into remove-six-code 2019-11-19 11:04:46 -03:00
Eugenio Lacuesta 05785c1c17
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-19 11:01:34 -03:00
Andrey Rahmatullin 78ad01632f Fix flake8 problems in PR #3989 (#4176) 2019-11-19 10:43:30 +01:00
Andrey Rakhmatullin beb7d80d6a Add a comment about the noconnect tests. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 922a66cf07 Fix or ignore flake8 problems. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin a7b640991d Rename tests/py3-ignores.txt to tests/ignores.txt. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 8b730a3670 Use self.proc.communicate() after killing mitmdump. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 5970d00eb9 Only xfail test_https_connect_tunnel_error on 3.6+. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 5080180c75 Improve the test_https_tunnel_without_leak_proxy_authorization_header change. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin c4ef950efd Use an older mitmproxy for py3.5. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin 3ec6960732 Fix test_proxy_connect.py for py3.5. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin c327ad9ba6 Remove an unused six import. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin cbb6d0c6a7 Mark failing proxy tests. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin f066257e95 Restore tests/test_proxy_connect.py and update it to modern mitmproxy. 2019-11-19 13:52:49 +05:00
Andrey Rakhmatullin e18014d84d Remove Python 2-only tests. 2019-11-19 13:52:49 +05:00
Andrey Rahmatullin 25cd7ac7ab
Merge pull request #4094 from victor-torres/invalid-url 
Improve URL schema validation on scrapy.Request initialization
 2019-11-19 13:50:08 +05:00
Andrey Rahmatullin 3408b757c1
Merge pull request #3989 from elacuesta/logformatter_error_method 
LogFormatter improvements
 2019-11-19 13:44:43 +05:00
Adrián Chaves f261cf65e9 Add missing blank lines between functions and classes 
Also fixed 2 unrelated Flake8 issues
 2019-11-18 17:16:09 +01:00
Adrián Chaves 99d8b05a0b Deprecate scrapy.utils.python.MutableChain.next 2019-11-18 10:58:47 +01:00
Marc Hernandez Cabot 0e252f5a13 fix E711 and E713 2019-11-15 19:12:43 +01:00
Mikhail Korobov 550d8046fd
Merge pull request #4157 from Gallaecio/flake8 
Add missing whitespace after ‘,’, ‘;’ or ‘:’
 2019-11-14 23:42:06 +05:00
Andrey Rakhmatullin 3b2289ad01 Rename test_non_str_url_py2 to test_bytes_url. 2019-11-14 22:53:28 +05:00
Andrey Rakhmatullin fe3a121f13 Use kwargs when calling get_func_args. 2019-11-14 22:50:53 +05:00
Andrey Rakhmatullin e291460db6 Fix flake8-detected errors. 2019-11-14 15:24:37 +05:00
Adrián Chaves 1a4a77d49f Remove Python 2 check from MutableChainTest 2019-11-14 10:24:31 +01:00
Andrey Rakhmatullin 87c23ba22d Remove Py2-only code that checks sys.version_info. 2019-11-14 13:45:39 +05:00
Andrey Rakhmatullin a138fb05d4 Replace to_native_str calls with to_unicode. 2019-11-14 13:45:39 +05:00