RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,291 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

9291 Commits

Author SHA1 Message Date
BroodingKangaroo 677e619d37 Fix too long lines 2020-05-21 14:57:03 +03:00
BroodingKangaroo 5f6c1dceb1 Merge remote-tracking branch 'upstream/master' into ISSUE-4250-add_batch_deliveries 2020-05-21 12:43:04 +03:00
Eugenio Lacuesta fbe915d9cb
StopDownload: check partial response contents 2020-05-20 20:20:30 -03:00
Eugenio Lacuesta 9514393bd1
Reduce amount of lines 2020-05-20 15:28:54 -03:00
Eugenio Lacuesta 276721a5dc
Stop response download 2020-05-20 14:46:26 -03:00
Mikhail Korobov 6f3e3411b1
Merge pull request #4587 from Gallaecio/4510 
logging.rst: remove unused, misleading import
 2020-05-20 22:23:42 +05:00
Kyle Altendorf eb8493f09c
Incompatible with pytest 5.4 and 5.4.1 2020-05-20 09:21:34 -04:00
Adrián Chaves f418c6d5f7 logging.rst: remove unused, misleading import 2020-05-19 20:04:47 +02:00
Mikhail Korobov 6e79216425
Merge pull request #4585 from nsirletti/fix-readthedocs-build 
Make doc downloadable on ReadTheDocs.org
 2020-05-19 20:45:20 +05:00
nsirletti c2a0cca0fe Suppress Sphinx warnings about MIME types other than .rst 2020-05-19 02:17:11 +02:00
nsirletti afb2e5011f Allow doc to be downloadable on readthedocs.org 2020-05-19 02:17:11 +02:00
BroodingKangaroo a7d070f3bb Change log level to error 2020-05-18 22:25:29 +03:00
Mikhail Korobov febe82a907
Merge pull request #4486 from ilias-ant/add-file-status-on-media-pipelines-file-info 
Add status (downloaded, uptodate) to files information
 2020-05-18 22:30:30 +05:00
Mikhail Korobov bcc40c4077
better deprecation warning for Response.body_as_unicode() (#4579) 2020-05-18 19:12:31 +02:00
Eugenio Lacuesta a22f97052e
Cleanup import in selector module 2020-05-15 21:18:26 -03:00
Eugenio Lacuesta a915af2e45
Remove monkeypatches module, E402 flake8 code 2020-05-15 20:41:51 -03:00
Mikhail Korobov 314adf6c38
Merge pull request #4237 from elacuesta/flake8-max-line-length 
Set flake8-max-line-length to 119
 2020-05-16 03:26:45 +05:00
Mikhail Korobov 14612fc3d3
Merge pull request #4534 from elacuesta/deprecate-baseitem 
Deprecate scrapy.item.BaseItem
 2020-05-16 03:24:06 +05:00
Jay Rajput 604fe33bad
Update scrapy/commands/__init__.py 
Changed typo in a comment for BaseRunSpiderCommand

Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
 2020-05-16 01:53:49 +05:30
BroodingKangaroo 10ae1a284f Minor fixes 2020-05-15 22:50:54 +03:00
Jay Rajput cd32a9d9f4 Merge branch 'common_commands' of https://github.com/jay24rajput/scrapy into common_commands 2020-05-16 00:27:02 +05:30
Jay Rajput 4cdd00e21f Changed BaseRunSpiderCommands to BaseRunSpiderCommand 2020-05-16 00:25:57 +05:30
Adrián Chaves 36c3c9713e
Run tests with Python 3.5.0 (#4518) 
* Run tests with Python 3.5.0

* Use Ubuntu Trusty 14.04 to test Python 3.5.0

* Use mitmproxy<3.0.0 on Python 3.5.0

* Skip tests requiring mitmproxy in Python 3.5.0

* Change the minimum Python version from 3.5 to 3.5.1

* Do not prevent Scrapy from working with Python 3.5.0

* Force system Python 3.5.1

* Do not install a non-system Python in Python 3.5.1 jobs

* Switch to Trusty to be able to test Xenial’s Python version ¯\_(ツ)_/¯

* Add missing trusty

* Stop breaking old PyPy

* Allow installing Scrapy on Python 3.5.0
 2020-05-15 22:37:56 +05:00
BroodingKangaroo 1cdcf8b08b Minor fixes 2020-05-15 20:14:07 +03:00
Eugenio Lacuesta 3eeecb4273
Merge branch 'master' into flake8-max-line-length 2020-05-15 11:11:20 -03:00
Marc Hernández fffb0a5b6a
replace set constructor with set literals (#4573) 2020-05-15 15:23:09 +02:00
Adrián Chaves 7701c39655
Merge branch 'master' into common_commands 2020-05-15 13:44:45 +02:00
BroodingKangaroo 0627bf476e Merge remote-tracking branch 'upstream/master' into ISSUE-4250-add_batch_deliveries 2020-05-14 20:59:22 +03:00
Eugenio Lacuesta ee13c3e95f
Merge branch 'master' into deprecate-baseitem 2020-05-14 13:31:16 -03:00
Eugenio Lacuesta 3d58e5e3be
Merge pull request #4545 from elacuesta/flake8-remove-e128 
Flake8: Remove E128
 2020-05-14 13:28:50 -03:00
Eugenio Lacuesta 89d0c98f7a
Merge remote-tracking branch 'upstream/master' into flake8-remove-e128 2020-05-14 11:35:30 -03:00
Eugenio Lacuesta 602bb13bdc
Merge pull request #4558 from elacuesta/flake8-remove-e741 
Flake8: Remove E741
 2020-05-14 11:26:59 -03:00
Eugenio Lacuesta df8a1d1c01
Flake8: Remove E741 2020-05-14 10:40:50 -03:00
Eugenio Lacuesta cfe4bf7e50
Merge pull request #4572 from noviluni/add_missed_flake8_rules 
Flake8: remove E306 and F523
 2020-05-14 10:38:55 -03:00
BroodingKangaroo 69c005f013 Documentation indent fix 2020-05-14 10:35:56 +03:00
BroodingKangaroo 111a58fe3d Merge remote-tracking branch 'upstream/master' into ISSUE-4250-add_batch_deliveries 2020-05-14 09:53:14 +03:00
BroodingKangaroo 8662d3587d Documentation and code refactoring 2020-05-14 00:05:36 +03:00
Adrián Chaves e31b6ccc45
Merge branch 'master' into flake8-remove-e128 2020-05-13 22:39:45 +02:00
BroodingKangaroo 2327ecead0 Rename FEED_STORAGE_BATCH_SIZE to FEED_STORAGE_BATCH_ITEM_COUNT 2020-05-13 22:50:04 +03:00
Adrián Chaves 7dac99bb12
Merge branch 'master' into deprecate-baseitem 2020-05-13 20:35:03 +02:00
marc 8971878c1d fix new detected flake8 cases 2020-05-13 14:11:10 +02:00
Andrey Rahmatullin 519f752d6d
Merge pull request #4566 from elacuesta/flake8-remove-f841 
Flake8: remove F841
 2020-05-13 11:46:48 +05:00
Jacty 33ab0a3663
Update __init__.py 2020-05-13 06:11:07 +08:00
Aditya Kumar 8d1269bcbc
Cover chompjs in documentation (#4562) 2020-05-12 20:42:28 +02:00
Aditya Kumar 97532a9144
test(spiderloader): no duplicate spider names (#4560) 2020-05-12 17:10:09 +02:00
Jay Rajput 07e125f4c9 Travis CI fixes in __init__.py 2020-05-12 16:31:56 +05:30
Jay Rajput 28d223dd87
Update __init__.py 2020-05-12 15:28:22 +05:30
Jay Rajput e01c30f0d5
Update scrapy/commands/__init__.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-05-12 01:05:20 +05:30
willbeaufoy cf9be5344a
Prevent create_instance() returning None (#4532) 
Currently create_instance() can return None if an extension is
incorrectly implemented, but the extension will still show up as
enabled in the logs. This can cause confusion, as in the linked bug.

This change prevents this occurring by throwing an error if
create_instance() will return None.
 2020-05-11 20:35:25 +02:00
nsirletti cb8140a42a
Deprecate Response.body_as_unicode() (#4555) 
Co-authored-by: Nicolas Sirletti <n.sirletti@gmail.com>
 2020-05-11 20:20:31 +02:00