RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,209 Commits 49 Branches 132 Tags 149 MiB
Commit Graph

8209 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 351a400cf8 Merge remote-tracking branch 'origin/master' into asyncio-startrequests-asyncgen 2020-06-03 17:00:59 +05:00
Andrey Rakhmatullin 2f00a66237 Forbid running ExecutionEngine._next_request in parallel. 2020-06-03 13:52:51 +05:00
Andrey Rahmatullin 63929e75e3
Merge pull request #4552 from jay24rajput/common_commands 
Code sharing between crawl and runspider command
 2020-06-03 12:06:12 +05:00
Eugenio Lacuesta 91e505edfe
Return single element from coroutine callback (#4609) 2020-06-02 10:32:14 +02:00
Mikhail Korobov 1d2b692666
Merge pull request #4596 from sharmarohit/remove-unneeded-escape-sequence 
remove unneeded escape sequence
 2020-06-02 02:30:33 +05:00
Adrián Chaves c86a1035dd
Merge pull request #4563 from willbeaufoy/hoverxref 
Extend hoverxref_roles to custom crossrefs (#4495)
 2020-06-01 07:18:13 +02:00
Bulat Khabibullin 5cef927944
Implement TextResponse.json() (#4574) 2020-06-01 06:57:23 +02:00
Mikhail Korobov 6aab3badfa
Merge pull request #4568 from elacuesta/remove-monkeypatches-flake8-e402 
Remove _monkeypatches, import-related Flake8 codes
 2020-05-30 00:07:50 +05:00
Mikhail Korobov 13215cfb71
Merge pull request #4559 from elacuesta/cancel-request-download 
Stop response download from signal handler
 2020-05-27 21:46:00 +05:00
Andrey Rahmatullin ff699eb1f8
Merge pull request #4599 from elacuesta/LocalWeakReferencedCache-getitem-fix 
Fix KeyError in LocalWeakReferencedCache
 2020-05-27 21:42:04 +05:00
Andrey Rahmatullin 7bf375098e
Merge pull request #2400 from elacuesta/keep_cookie_header 
[MRG+1] CookiesMiddleware: keep cookies from 'Cookie' request header, fix encoding
 2020-05-27 21:41:43 +05:00
Andrey Rakhmatullin 9b16ba8a59 Rename isasyncgen to _isasyncgen. 2020-05-27 20:54:02 +05:00
Eugenio Lacuesta 44d5801b05
Fix KeyError in LocalWeakReferencedCache 2020-05-27 11:15:33 -03:00
Eugenio Lacuesta 492197e477
StopDownload: store response in the exception instead of the failure 2020-05-27 10:45:07 -03:00
Eugenio Lacuesta 634ad5ebca
Merge remote-tracking branch 'upstream/master' into cancel-request-download 2020-05-27 10:27:45 -03:00
Will Beaufoy 898bdd3feb Update docs README with build instructions using tox 
None of the existing commands built the docs properly for me (I had to
revert the changes in 901892d to docs/conf.py to get them to build
properly, and even then no tooltips displayed).

Building them with tox worked for me, but other developers say they
can still use the original method, so the docs now contain both.
 2020-05-26 19:09:23 +01:00
Mikhail Korobov b82a480e91
Merge pull request #4588 from altendky/patch-1 
Incompatible with pytest 5.4 and 5.4.1
 2020-05-26 01:38:53 +05:00
Rohit Sharma 0cc0e51ca3 remove uneeded escape sequence 
removed uneeded wscape sequences from method arguments in the docs
folder
 2020-05-24 13:38:44 -07:00
Kyle Altendorf 5b2af85289
Link to issue describing troublesome pytest versions 2020-05-22 15:09:58 -04:00
Mikhail Korobov 3e854a69de
Merge pull request #4593 from Gallaecio/4514 
Fix the description of the ScreenshotPipeline example item pipeline
 2020-05-22 23:12:58 +05:00
Adrián Chaves 08756cd87a Fix the description of the ScreenshotPipeline example item pipeline 2020-05-22 18:06:54 +02:00
Andrey Rakhmatullin 548281960b Remove process_iterable_helper from this PR. 2020-05-22 20:13:40 +05:00
Eugenio Lacuesta f6ed5edc31
CookiesMiddleware: keep cookies from 'Cookie' request header 2020-05-21 11:07:41 -03:00
Andrey Rakhmatullin 3bacf8e421 Fix test_utils_middlewares for 3.5. 2020-05-21 16:09:03 +05:00
Andrey Rakhmatullin 5b4adffabf Remove a duplicate test. 2020-05-21 15:23:47 +05:00
Andrey Rakhmatullin 8bf5f8915e Make test_utils_asyncgen actually run. 2020-05-21 15:20:05 +05:00
Andrey Rakhmatullin 0620895b7f Add simple tests for process_iterable_helper. 2020-05-21 15:17:22 +05:00
Andrey Rakhmatullin f6b985eaa9 Fix process_async_iterable_helper. 2020-05-21 15:17:15 +05:00
Eugenio Lacuesta fbe915d9cb
StopDownload: check partial response contents 2020-05-20 20:20:30 -03:00
Eugenio Lacuesta 9514393bd1
Reduce amount of lines 2020-05-20 15:28:54 -03:00
Eugenio Lacuesta 276721a5dc
Stop response download 2020-05-20 14:46:26 -03:00
Mikhail Korobov 6f3e3411b1
Merge pull request #4587 from Gallaecio/4510 
logging.rst: remove unused, misleading import
 2020-05-20 22:23:42 +05:00
Kyle Altendorf eb8493f09c
Incompatible with pytest 5.4 and 5.4.1 2020-05-20 09:21:34 -04:00
Adrián Chaves f418c6d5f7 logging.rst: remove unused, misleading import 2020-05-19 20:04:47 +02:00
Mikhail Korobov 6e79216425
Merge pull request #4585 from nsirletti/fix-readthedocs-build 
Make doc downloadable on ReadTheDocs.org
 2020-05-19 20:45:20 +05:00
Andrey Rakhmatullin cab1207ce6 Add scrapy/utils/py36.py to collect_ignore. 2020-05-19 12:08:37 +05:00
Andrey Rakhmatullin 81483a7d98 Merge remote-tracking branch 'origin/master' into asyncio-startrequests-asyncgen 2020-05-19 11:43:10 +05:00
Andrey Rakhmatullin 824b98f152 Add utils.defer.isasyncgen(). 2020-05-19 11:38:10 +05:00
Andrey Rakhmatullin abc9aa8737 Restore and deprecate utils.py36 2020-05-19 11:37:51 +05:00
Andrey Rakhmatullin 33279aea04 Small conftest and docs improvements. 2020-05-19 11:37:51 +05:00
nsirletti c2a0cca0fe Suppress Sphinx warnings about MIME types other than .rst 2020-05-19 02:17:11 +02:00
nsirletti afb2e5011f Allow doc to be downloadable on readthedocs.org 2020-05-19 02:17:11 +02:00
Mikhail Korobov febe82a907
Merge pull request #4486 from ilias-ant/add-file-status-on-media-pipelines-file-info 
Add status (downloaded, uptodate) to files information
 2020-05-18 22:30:30 +05:00
Mikhail Korobov bcc40c4077
better deprecation warning for Response.body_as_unicode() (#4579) 2020-05-18 19:12:31 +02:00
Eugenio Lacuesta a22f97052e
Cleanup import in selector module 2020-05-15 21:18:26 -03:00
Eugenio Lacuesta a915af2e45
Remove monkeypatches module, E402 flake8 code 2020-05-15 20:41:51 -03:00
Mikhail Korobov 314adf6c38
Merge pull request #4237 from elacuesta/flake8-max-line-length 
Set flake8-max-line-length to 119
 2020-05-16 03:26:45 +05:00
Mikhail Korobov 14612fc3d3
Merge pull request #4534 from elacuesta/deprecate-baseitem 
Deprecate scrapy.item.BaseItem
 2020-05-16 03:24:06 +05:00
Jay Rajput 604fe33bad
Update scrapy/commands/__init__.py 
Changed typo in a comment for BaseRunSpiderCommand

Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
 2020-05-16 01:53:49 +05:30
Jay Rajput cd32a9d9f4 Merge branch 'common_commands' of https://github.com/jay24rajput/scrapy into common_commands 2020-05-16 00:27:02 +05:30