RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,166 Commits 49 Branches 132 Tags 132 MiB
Commit Graph

8166 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 2f00a66237 Forbid running ExecutionEngine._next_request in parallel. 2020-06-03 13:52:51 +05:00
Andrey Rakhmatullin 9b16ba8a59 Rename isasyncgen to _isasyncgen. 2020-05-27 20:54:02 +05:00
Andrey Rakhmatullin 548281960b Remove process_iterable_helper from this PR. 2020-05-22 20:13:40 +05:00
Andrey Rakhmatullin 3bacf8e421 Fix test_utils_middlewares for 3.5. 2020-05-21 16:09:03 +05:00
Andrey Rakhmatullin 5b4adffabf Remove a duplicate test. 2020-05-21 15:23:47 +05:00
Andrey Rakhmatullin 8bf5f8915e Make test_utils_asyncgen actually run. 2020-05-21 15:20:05 +05:00
Andrey Rakhmatullin 0620895b7f Add simple tests for process_iterable_helper. 2020-05-21 15:17:22 +05:00
Andrey Rakhmatullin f6b985eaa9 Fix process_async_iterable_helper. 2020-05-21 15:17:15 +05:00
Andrey Rakhmatullin cab1207ce6 Add scrapy/utils/py36.py to collect_ignore. 2020-05-19 12:08:37 +05:00
Andrey Rakhmatullin 81483a7d98 Merge remote-tracking branch 'origin/master' into asyncio-startrequests-asyncgen 2020-05-19 11:43:10 +05:00
Andrey Rakhmatullin 824b98f152 Add utils.defer.isasyncgen(). 2020-05-19 11:38:10 +05:00
Andrey Rakhmatullin abc9aa8737 Restore and deprecate utils.py36 2020-05-19 11:37:51 +05:00
Andrey Rakhmatullin 33279aea04 Small conftest and docs improvements. 2020-05-19 11:37:51 +05:00
Mikhail Korobov febe82a907
Merge pull request #4486 from ilias-ant/add-file-status-on-media-pipelines-file-info 
Add status (downloaded, uptodate) to files information
 2020-05-18 22:30:30 +05:00
Mikhail Korobov bcc40c4077
better deprecation warning for Response.body_as_unicode() (#4579) 2020-05-18 19:12:31 +02:00
Mikhail Korobov 314adf6c38
Merge pull request #4237 from elacuesta/flake8-max-line-length 
Set flake8-max-line-length to 119
 2020-05-16 03:26:45 +05:00
Mikhail Korobov 14612fc3d3
Merge pull request #4534 from elacuesta/deprecate-baseitem 
Deprecate scrapy.item.BaseItem
 2020-05-16 03:24:06 +05:00
Adrián Chaves 36c3c9713e
Run tests with Python 3.5.0 (#4518) 
* Run tests with Python 3.5.0

* Use Ubuntu Trusty 14.04 to test Python 3.5.0

* Use mitmproxy<3.0.0 on Python 3.5.0

* Skip tests requiring mitmproxy in Python 3.5.0

* Change the minimum Python version from 3.5 to 3.5.1

* Do not prevent Scrapy from working with Python 3.5.0

* Force system Python 3.5.1

* Do not install a non-system Python in Python 3.5.1 jobs

* Switch to Trusty to be able to test Xenial’s Python version ¯\_(ツ)_/¯

* Add missing trusty

* Stop breaking old PyPy

* Allow installing Scrapy on Python 3.5.0
 2020-05-15 22:37:56 +05:00
Eugenio Lacuesta 3eeecb4273
Merge branch 'master' into flake8-max-line-length 2020-05-15 11:11:20 -03:00
Marc Hernández fffb0a5b6a
replace set constructor with set literals (#4573) 2020-05-15 15:23:09 +02:00
Eugenio Lacuesta ee13c3e95f
Merge branch 'master' into deprecate-baseitem 2020-05-14 13:31:16 -03:00
Eugenio Lacuesta 3d58e5e3be
Merge pull request #4545 from elacuesta/flake8-remove-e128 
Flake8: Remove E128
 2020-05-14 13:28:50 -03:00
Eugenio Lacuesta 89d0c98f7a
Merge remote-tracking branch 'upstream/master' into flake8-remove-e128 2020-05-14 11:35:30 -03:00
Eugenio Lacuesta 602bb13bdc
Merge pull request #4558 from elacuesta/flake8-remove-e741 
Flake8: Remove E741
 2020-05-14 11:26:59 -03:00
Eugenio Lacuesta df8a1d1c01
Flake8: Remove E741 2020-05-14 10:40:50 -03:00
Eugenio Lacuesta cfe4bf7e50
Merge pull request #4572 from noviluni/add_missed_flake8_rules 
Flake8: remove E306 and F523
 2020-05-14 10:38:55 -03:00
Adrián Chaves e31b6ccc45
Merge branch 'master' into flake8-remove-e128 2020-05-13 22:39:45 +02:00
Adrián Chaves 7dac99bb12
Merge branch 'master' into deprecate-baseitem 2020-05-13 20:35:03 +02:00
marc 8971878c1d fix new detected flake8 cases 2020-05-13 14:11:10 +02:00
Andrey Rahmatullin 519f752d6d
Merge pull request #4566 from elacuesta/flake8-remove-f841 
Flake8: remove F841
 2020-05-13 11:46:48 +05:00
Aditya Kumar 8d1269bcbc
Cover chompjs in documentation (#4562) 2020-05-12 20:42:28 +02:00
Aditya Kumar 97532a9144
test(spiderloader): no duplicate spider names (#4560) 2020-05-12 17:10:09 +02:00
willbeaufoy cf9be5344a
Prevent create_instance() returning None (#4532) 
Currently create_instance() can return None if an extension is
incorrectly implemented, but the extension will still show up as
enabled in the logs. This can cause confusion, as in the linked bug.

This change prevents this occurring by throwing an error if
create_instance() will return None.
 2020-05-11 20:35:25 +02:00
nsirletti cb8140a42a
Deprecate Response.body_as_unicode() (#4555) 
Co-authored-by: Nicolas Sirletti <n.sirletti@gmail.com>
 2020-05-11 20:20:31 +02:00
Eugenio Lacuesta 6f8758624c
Flake8: remove F841 2020-05-11 13:50:34 -03:00
Eugenio Lacuesta 25e9bc2d0d
Merge branch 'master' into flake8-remove-e128 2020-05-11 13:23:57 -03:00
Eugenio Lacuesta abfdc1b542
Update docstring for Item class 2020-05-11 13:20:06 -03:00
Eugenio Lacuesta fb7ba6964d
Merge branch 'master' into deprecate-baseitem 2020-05-11 10:21:56 -03:00
Mikhail Korobov 892467cb8a
Merge pull request #4541 from elacuesta/pickle-adjustments 
Pickle: use protocol 4, update tests
 2020-05-11 15:16:14 +05:00
Mikhail Korobov b183579564
Merge pull request #4205 from elacuesta/bytes_received_signal 
Add bytes_received signal
 2020-05-11 15:09:55 +05:00
Eugenio Lacuesta f689e917bf
Update Item docstring 2020-05-08 19:44:05 -03:00
Eugenio Lacuesta 9485724d98
Merge branch 'master' into deprecate-baseitem 2020-05-08 19:32:47 -03:00
Eugenio Lacuesta be39f274b8
Merge branch 'master' into flake8-remove-e128 2020-05-08 18:33:29 -03:00
Mikhail Korobov 794ab19660
Merge pull request #3727 from Gallaecio/pylint 
Use pylint
 2020-05-09 02:22:54 +05:00
Mikhail Korobov 333910f6c8
Merge pull request #4554 from elacuesta/linkextractor-remove-lambdas 
Remove lambda attributes in default link extractor
 2020-05-09 02:11:59 +05:00
Mikhail Korobov 707f591767
Merge pull request #4557 from elacuesta/flake8-remove-e1xx 
Flake8: remove E111, E114, E116, E117, E121
 2020-05-09 02:08:27 +05:00
Eugenio Lacuesta c2c3054ac1
Flake8: remove E121 2020-05-08 16:43:47 -03:00
Eugenio Lacuesta 83ce82f400
Flake8: remove E114 and E117 (unused) 2020-05-08 16:43:28 -03:00
Eugenio Lacuesta 1a157f2e26
Flake8: remove E116 2020-05-08 16:42:56 -03:00
Eugenio Lacuesta 81d0b2f61a
Flake8: remove E111 2020-05-08 16:42:50 -03:00