b2b2d0b015
Rename TWISTED_ENABLED to TWISTED_REACTOR_ENABLED. ( #7394 )
2026-04-02 10:20:54 +02:00
03d105ac92
Add warnings to components not working without a reactor ( #7355 )
2026-03-25 09:21:08 +01:00
6fe27ba33e
Add more no-reactor tests ( #7259 )
...
* Generic changes and scrapy bench.
* scrapy check.
* scrapy crawl.
* scrapy fetch.
* scrapy parse.
* scrapy runspider.
* scrapy shell.
* Skip httpx tests on default-reactor.
* Review requires_reactor marks.
* Make test functions that require an event loop async def.
* Improve test_pending_asyncio_tasks().
* Add Mac OS DNS error.
* Refactor most of test_scheduler.py.
* Finish refactoring DownloaderAwareSchedulerTestMixin.
* Refactor test_engine_loop.py.
* Add the no-reactor-extra-deps tox env, run no-reactor on macos.
* Skip unhandled CancelledError when shutting down the engine.
* Fix typing and pre-commit checks.
* Fix typing problems in master.
---------
Co-authored-by: Adrian <adrian@zyte.com>
2026-03-24 15:29:22 +05:00
fc30c47f38
Add HttpxDownloader. ( #7239 )
2026-02-12 09:30:44 +01:00
c8aa429c9b
Reactorless tests 2: enable pytest-asyncio. ( #7233 )
...
* Move no-reactor to pytest-asyncio.
* Unify the style.
* Skip a test that installs a reactor in no-reactor.
* Fix pinned env deps.
* Strict pytest-asyncio mode.
2026-02-02 19:36:23 +01:00
3186ccf5d5
Reactorless tests 1: the test env ( #6952 )
...
* Reactorless tests.
* Updates after merging master.
* Update a new import.
* Updates after merging master.
* Update testenv:no-reactor-pinned.
* Foundations for the reactorless mode.
* Fixes after merging master.
* Updates after merging reactorless-base.
* Add simple subprocess tests for reactorless AsyncCrawler*.
* More reactorless tests.
* Refactor AsyncCrawlerProcess.start().
* More checks.
* Fix test_reactorless_import_hook.
* More tests.
* Call install_reactor() before asyncio.run().
* Cleanup.
* Rephrase.
* Fix a test.
* Fixes.
* Rephrasing.
* Set TELNETCONSOLE_ENABLED=False in the reactorless mode.
* Fix a new import.
* Clarify --reactor marks.
* Cleanup.
* More docs/comments.
* Update test_fallback_workflow().
* Update TestDeferredFFromCoroF.
* Typing improvements.
2026-02-02 19:25:30 +05:00
54d8562fb0
Assorted test improvements ( #7232 )
...
* Await some missed deferreds.
* Simplify test_engine_stop_download_*.
* Fix the header value.
* Extend TestHttpWithCrawlerBase.
* Make test_tls_logging more universal.
* Add more tests for header handling.
* Clarify certificate and ip_address integration tests.
* Make test_download_with_maxsize_very_large_file() more universal.
* Fix test_coroutine_asyncio().
* Typo.
2026-02-02 09:37:39 +01:00
3ec6ae05c1
Fix omitting repeated dataloss warnings in HTTP11DownloadHandler. ( #7222 )
2026-01-22 00:41:27 +05:00
2347138ba4
Unified download handler exceptions ( #7208 )
...
* Add DownloadTimeoutError.
* Don't expect ConnectionAborted in tests.
* Add DownloadCancelledError.
* Add ResponseDataLoss.
* Add more download handler tests.
* Add DownloadConnectionRefusedError.
* Add UnsupportedURLScheme.
* Add CannotResolveHostError.
* Add DownloadFailedError.
* Remove wrapped Twisted exceptions from lists.
* Update a test expectation.
* Extract wrap_twisted_exceptions().
* Wrap TxTimeoutError.
* Rename ResponseDataLoss and UnsupportedURLScheme.
2026-01-21 23:09:54 +05:00
d1bd8eb49f
Async API for download handlers. ( #7164 )
2025-12-31 19:03:16 +01:00
11073c8680
Deprecate spider attributes that can be replaced by settings, round 2 ( #7039 )
...
* Move duplicate code to utils
* move new function to end
* draft
* update docs
* Update tests
* Update test name
* rename test
* rollback signature
* leftover
* sort
* Rollback some changes
* Rollback download_delay warning
* Rollback
* fix checks
* Remove unused imports
* Add pragma: no cover.
---------
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2025-12-01 01:26:17 +05:00
8c8f4ff033
Extract in-process test HTTP servers into separate processes ( #6960 )
...
* Switch TestEngineBase to mockserver.
* Split mockserver.py.
* Stop using WrappingFactory.
* Remove keyfile/certfile fields from TestHttpBase.
* Move webclient-only resources to test_webclient.py.
* Refactor MockServer.url().
* Migrate TestHttpBase to mockserver.
* Migrate TestSimpleHttpsBase to a separate mockserver.
* Rename TestHttpMockServerBase.
* Migrate TestHttpProxyBase to a separate mockserver.
* Refactor HTTP mockservers.
2025-07-25 12:42:27 +05:00
6b2997af90
Migrate to pytest-twisted ( #6938 )
...
* Migrate to pytest-twisted (WIP)
* Some typing fixes.
* Make --reactor=asyncio the default again.
* Try installing the correct event loop policy in tests on Windows.
* Make reactor_pytest a normal fixture.
* Fix test warnings.
* Fix FTPDownloadHandler teardown.
* Cleanups, typing.
* More cleanup.
* Update only_asyncio/only_not_asyncio mark messages.
2025-07-06 21:27:17 +05:00
df342eee6e
Bump tool versions. ( #6941 )
2025-07-05 10:30:30 +02:00
8b3c3ea4ae
Refactor test functions that were returning a Deferred directly. ( #6889 )
2025-06-30 09:19:23 +02:00
b4d11b8b25
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
8ae418df44
Rewrite download handler tests to coroutines. ( #6846 )
2025-06-02 16:02:08 +02:00
916fe50974
Fix and ban the top-level twisted.internet.reactor imports. ( #6835 )
2025-05-28 15:53:52 +05:00
b41aea4873
Restructure download handler tests. ( #6821 )
...
* Restructure download handler tests.
* Typo.
* Use mixins to reduce boilerplate.
2025-05-27 18:19:47 +05:00
Andrey Rakhmatullin
Laerte Pereira