08464e7ece
Merge 0ed5397c84 into ad43bf0c56
2026-08-15 11:31:49 -05:00
15885a8db4
Generalize the use of build_from_crawler() internally ( #7808 )
...
* Generalize the use of build_from_crawler() internally
* Do not use build_from_crawler() for spiders, since they are guaranteed having from_crawler()
2026-08-10 20:55:10 +05:00
d6dc238f82
Fix Windows support in tests
2026-08-06 11:32:27 +02:00
d00dc2b8b4
Add an item_processor feed option
2026-08-06 10:55:03 +02:00
b2d4eedea8
Fix immediate delivery of full feed export batches ( #7730 ) ( #7733 )
...
* Store full feed batches before spider closes (#7730 )
Start closing and storing each batch as soon as it reaches the configured item count. Track unfinished close tasks so spider shutdown still waits for all deliveries before emitting the exporter-closed signal.
Add an end-to-end regression test that verifies the first batch is stored while the crawl is still running.
* Remove the issue reference
---------
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2026-07-29 19:30:03 +05:00
cec86f216e
Remove remaining cross-imports in test modules. ( #7782 )
2026-07-27 12:19:12 +02:00
394c2797f3
Remove non-Twisted uses of zope.interface ( #7731 )
2026-07-21 13:30:21 +02:00
4ee3676464
Test restructuring ( #7736 )
2026-07-21 12:44:56 +02:00
61f99f2df1
More granular untyped defs checking in tests. ( #7712 )
2026-07-08 10:05:45 +02:00
3a36955261
Assorted test fixes ( #7616 )
2026-06-15 08:39:58 +02:00
55c17a8985
Disable mypy allow_any_generics. ( #7532 )
2026-05-18 15:54:06 +02:00
bfe34492fa
asyncio alternative to deferToThread() ( #7349 )
2026-03-24 15:41:15 +01:00
6fe27ba33e
Add more no-reactor tests ( #7259 )
...
* Generic changes and scrapy bench.
* scrapy check.
* scrapy crawl.
* scrapy fetch.
* scrapy parse.
* scrapy runspider.
* scrapy shell.
* Skip httpx tests on default-reactor.
* Review requires_reactor marks.
* Make test functions that require an event loop async def.
* Improve test_pending_asyncio_tasks().
* Add Mac OS DNS error.
* Refactor most of test_scheduler.py.
* Finish refactoring DownloaderAwareSchedulerTestMixin.
* Refactor test_engine_loop.py.
* Add the no-reactor-extra-deps tox env, run no-reactor on macos.
* Skip unhandled CancelledError when shutting down the engine.
* Fix typing and pre-commit checks.
* Fix typing problems in master.
---------
Co-authored-by: Adrian <adrian@zyte.com>
2026-03-24 15:29:22 +05:00
584d99af30
Split long test files ( #7329 )
2026-03-13 10:30:25 +01:00
Adrian
Fandu
Andrey Rakhmatullin