RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,061 Commits 49 Branches 132 Tags 182 MiB
Commit Graph

9061 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 3577c00569 Merge remote-tracking branch 'origin/master' into crawler-process-reactor-later 2022-01-03 13:59:26 +05:00
Andrey Rahmatullin c5ab58056c
Set WindowsSelectorEventLoopPolicy on Windows (#5315) 2021-12-31 20:38:10 +01:00
Andrey Rakhmatullin 6eaceec735 Implement docs suggestions. 2021-12-31 20:14:24 +05:00
Andrey Rakhmatullin 60800ee5f2 Merge remote-tracking branch 'origin/master' into crawler-process-reactor-later 2021-12-31 20:12:30 +05:00
Andrey Rahmatullin 57dc58123b
Remove the experimental note about asyncio (#5332) 2021-12-31 13:15:08 +01:00
Paweł Miech e4bdd1cb95
downloader.webclient: make reactor import local (#5357) 2021-12-31 11:57:12 +01:00
Andrey Oskin b81938684b
Docs: correct process repetition start step (#5356) 
The process repeats from step 3, the scheduler feeds request to the engine. Steps 1 and 2 are not parts of the loop as their incarnations steps 7 and 8 are parts of the loop.
 2021-12-31 11:49:18 +01:00
Andrey Rakhmatullin 0e78b68650 Merge remote-tracking branch 'origin/master' into crawler-process-reactor-later 2021-12-31 15:46:25 +05:00
Andrey Rakhmatullin 64261d9e38 Slight refactoring. 2021-12-31 15:45:59 +05:00
Andrey Rahmatullin 7380888cad
Fix a warning message. (#5359) 2021-12-30 10:55:16 -03:00
Andrey Rakhmatullin a986792def Add more docs for TWISTED_REACTOR. 2021-12-24 19:43:14 +05:00
Andrey Rakhmatullin 940cc0776f Add docs about TWISTED_REACTOR and other per-process settings. 2021-12-24 17:12:50 +05:00
Andrey Rakhmatullin d4565318c7 Fix a reactor test on Windows. 2021-12-23 17:40:31 +05:00
Andrey Rakhmatullin df04e934e6 Merge remote-tracking branch 'origin/master' into crawler-process-reactor-later 2021-12-23 17:25:56 +05:00
Andrey Rakhmatullin 9c4bfb4836 Remove an unused import. 2021-12-23 17:17:36 +05:00
Andrey Rakhmatullin 0025134382 Completely skip WindowsRunSpiderCommandTest outside Windows. 2021-12-23 16:45:17 +05:00
Andrey Rakhmatullin ebcafdf4a9 Add tests for TWISTED_REACTOR in custom_settings. 2021-12-23 16:35:26 +05:00
Andrey Rakhmatullin 041699b54c Remove tests that want to modify the test process reactor. 2021-12-23 16:14:47 +05:00
Andrey Rakhmatullin 60c8838554 Move installing the reactor from CrawlerProcess to Crawler. 2021-12-23 16:07:18 +05:00
Andrey Rakhmatullin 46ef9cf771 Don't install non-working shutdown handlers in `scrapy shell`. 2021-12-23 16:05:17 +05:00
yogender26 d6a384b3cf
corrrection of coma (#5347) 2021-12-22 23:39:05 +01:00
Andrey Rahmatullin 9ec60e8e4a
Merge pull request #5320 from zessx/5319-oib-base-tag 2021-12-21 17:49:31 +05:00
Andrey Rahmatullin 0862b179e3
Merge pull request #5333 from wRAR/pylint-2.12 
Fix and pin pylint.
 2021-12-16 18:54:27 +05:00
Andrey Rakhmatullin 6483dfdbe1 Move install_shutdown_handlers() from __init__() to start(). 2021-12-01 19:53:39 +05:00
Andrey Rakhmatullin eb62906c3e Extract utils.log.log_reactor_info(). 2021-12-01 17:40:41 +05:00
Andrey Rakhmatullin 6ec66c96fb Fix and pin pylint. 2021-11-26 12:25:45 +05:00
Alex c316ca45a5
Use augmented assignment statements (#5322) 2021-11-16 10:20:56 +01:00
Samuel Marchal 75ed765476 Test coverage for open_in_browser base tag injection (#5319) 2021-11-15 16:44:56 +01:00
Samuel Marchal f2c800c5c9 Improve open_in_browser base tag injection (#5319) 2021-11-15 16:44:56 +01:00
Andrey Rahmatullin 28eba610e2
Re-enable Windows tests for Python 3.9 and 3.10. (#5316) 2021-11-15 08:24:54 +01:00
Andrey Rahmatullin 9828cb3f0b
Merge pull request #5310 from azzamsa/more-test 
test: `test_format_engine_status`
 2021-11-08 20:13:00 +05:00
azzamsa 55cce25a79 test: `test_format_engine_status` 2021-11-05 03:02:02 +07:00
Andrey Rahmatullin ed50a81f50
Merge pull request #5299 from azzamsa/fix-tests 
fix: `CodeBlockParser` has been renamed to `PythonCodeBlockParser`
 2021-11-04 19:00:14 +05:00
azzamsa 67994d1ddd fix: `CodeBlockParser` has been renamed to `PythonCodeBlockParser` 2021-10-27 22:19:07 +07:00
Andrey Rahmatullin 3225de7257
Merge pull request #5292 from azzamsa/tox-patch 
refactor: use `pytest` command as the recommended entry point
 2021-10-25 11:34:59 +05:00
azzamsa 51adf71b1b refactor: use `pytest` command as the recommended entry point 
`pytest` is recommended command since pytest 3.0.
There is a possibility for `py.test` to be deprecated or even removed.

https://github.com/pytest-dev/pytest/issues/1629
 2021-10-24 10:52:56 +07:00
Andrey Rahmatullin 144d1eb834
Add Deferred-to-Future helpers (#5288) 2021-10-22 18:46:01 +02:00
Eugenio Lacuesta cfff79cee6
Make Python 3.10 support official (#5265) 2021-10-18 22:09:17 +02:00
Andrey Rahmatullin afa5881ada
Merge pull request #5279 from erikkemperman/option-logfile-truncate 
Add LOG_FILE_APPEND to settings
 2021-10-18 11:51:42 +05:00
Andrey Rahmatullin 975f15003e
Merge pull request #5281 from raphaelts3/add-pronounced-docs 
Add how Scrapy is pronounced to the docs
 2021-10-18 11:49:55 +05:00
raphaelts3 027ecd8686
Update docs/intro/overview.rst 
Co-authored-by: azzamsa <17734314+azzamsa@users.noreply.github.com>
 2021-10-16 10:52:54 -03:00
Raphael Tomé Santana aec7146e2f Add how Scrapy is pronounced to the docs 2021-10-15 20:38:53 -03:00
Adrián Chaves d774d6a9c4
Remove unused variable 2021-10-15 17:25:22 +02:00
Adrián Chaves 98ee3ddffe
Freeze flake8 2021-10-15 17:18:46 +02:00
Erik Kemperman ca320feb2a
Add LOG_FILE_APPEND to settings 2021-10-15 16:36:41 +02:00
Andrey Rahmatullin 5b13bfd14a
Merge pull request #5269 from Ankur19/allow-non-text-response-types-in-ItemLoader 
Fix bug #5145 - Removing Selector for Response's that are not Http or Xml
 2021-10-14 16:19:54 +05:00
Andrey Rahmatullin 808fb63273
Merge pull request #5276 from imba-tjd/patch-1 
docs: fix typo
 2021-10-14 12:51:30 +05:00
谭九鼎 3243aa2cd5
docs: fix typo 2021-10-14 10:18:26 +08:00
Jake Herbst d08199f631
Removing unnecessary line from docs to prevent test failure (#5274) 2021-10-12 19:20:09 +02:00
Andrey Rahmatullin 1ae53a9d7c
Merge pull request #5256 from rydwhelchel/documentation-changes 
docs: restructed phrasing for clarity
 2021-10-12 16:49:31 +05:00