RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,018 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

8 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 3186ccf5d5
Reactorless tests 1: the test env (#6952) 
* Reactorless tests.

* Updates after merging master.

* Update a new import.

* Updates after merging master.

* Update testenv:no-reactor-pinned.

* Foundations for the reactorless mode.

* Fixes after merging master.

* Updates after merging reactorless-base.

* Add simple subprocess tests for reactorless AsyncCrawler*.

* More reactorless tests.

* Refactor AsyncCrawlerProcess.start().

* More checks.

* Fix test_reactorless_import_hook.

* More tests.

* Call install_reactor() before asyncio.run().

* Cleanup.

* Rephrase.

* Fix a test.

* Fixes.

* Rephrasing.

* Set TELNETCONSOLE_ENABLED=False in the reactorless mode.

* Fix a new import.

* Clarify --reactor marks.

* Cleanup.

* More docs/comments.

* Update test_fallback_workflow().

* Update TestDeferredFFromCoroF.

* Typing improvements.
 2026-02-02 19:25:30 +05:00
Andrey Rakhmatullin 2347138ba4
Unified download handler exceptions (#7208) 
* Add DownloadTimeoutError.

* Don't expect ConnectionAborted in tests.

* Add DownloadCancelledError.

* Add ResponseDataLoss.

* Add more download handler tests.

* Add DownloadConnectionRefusedError.

* Add UnsupportedURLScheme.

* Add CannotResolveHostError.

* Add DownloadFailedError.

* Remove wrapped Twisted exceptions from lists.

* Update a test expectation.

* Extract wrap_twisted_exceptions().

* Wrap TxTimeoutError.

* Rename ResponseDataLoss and UnsupportedURLScheme.
 2026-01-21 23:09:54 +05:00
Andrey Rakhmatullin d1bd8eb49f
Async API for download handlers. (#7164) 2025-12-31 19:03:16 +01:00
Andrey Rakhmatullin 8c8f4ff033
Extract in-process test HTTP servers into separate processes (#6960) 
* Switch TestEngineBase to mockserver.

* Split mockserver.py.

* Stop using WrappingFactory.

* Remove keyfile/certfile fields from TestHttpBase.

* Move webclient-only resources to test_webclient.py.

* Refactor MockServer.url().

* Migrate TestHttpBase to mockserver.

* Migrate TestSimpleHttpsBase to a separate mockserver.

* Rename TestHttpMockServerBase.

* Migrate TestHttpProxyBase to a separate mockserver.

* Refactor HTTP mockservers.
 2025-07-25 12:42:27 +05:00
Andrey Rakhmatullin 6b2997af90
Migrate to pytest-twisted (#6938) 
* Migrate to pytest-twisted (WIP)

* Some typing fixes.

* Make --reactor=asyncio the default again.

* Try installing the correct event loop policy in tests on Windows.

* Make reactor_pytest a normal fixture.

* Fix test warnings.

* Fix FTPDownloadHandler teardown.

* Cleanups, typing.

* More cleanup.

* Update only_asyncio/only_not_asyncio mark messages.
 2025-07-06 21:27:17 +05:00
Andrey Rakhmatullin 8b3c3ea4ae
Refactor test functions that were returning a Deferred directly. (#6889) 2025-06-30 09:19:23 +02:00
Andrey Rakhmatullin 8ae418df44
Rewrite download handler tests to coroutines. (#6846) 2025-06-02 16:02:08 +02:00
Andrey Rakhmatullin b41aea4873
Restructure download handler tests. (#6821) 
* Restructure download handler tests.

* Typo.

* Use mixins to reduce boilerplate.
 2025-05-27 18:19:47 +05:00