6fe27ba33e
Add more no-reactor tests ( #7259 )
...
* Generic changes and scrapy bench.
* scrapy check.
* scrapy crawl.
* scrapy fetch.
* scrapy parse.
* scrapy runspider.
* scrapy shell.
* Skip httpx tests on default-reactor.
* Review requires_reactor marks.
* Make test functions that require an event loop async def.
* Improve test_pending_asyncio_tasks().
* Add Mac OS DNS error.
* Refactor most of test_scheduler.py.
* Finish refactoring DownloaderAwareSchedulerTestMixin.
* Refactor test_engine_loop.py.
* Add the no-reactor-extra-deps tox env, run no-reactor on macos.
* Skip unhandled CancelledError when shutting down the engine.
* Fix typing and pre-commit checks.
* Fix typing problems in master.
---------
Co-authored-by: Adrian <adrian@zyte.com>
2026-03-24 15:29:22 +05:00
9f4651151d
Deprecate ScrapyCommand.set_crawler(), remove/fix some dead code. ( #7276 )
2026-03-24 10:36:48 +01:00
11977afba5
Rename async test decorators. ( #7234 )
2026-02-02 20:39:43 +01:00
c8aa429c9b
Reactorless tests 2: enable pytest-asyncio. ( #7233 )
...
* Move no-reactor to pytest-asyncio.
* Unify the style.
* Skip a test that installs a reactor in no-reactor.
* Fix pinned env deps.
* Strict pytest-asyncio mode.
2026-02-02 19:36:23 +01:00
3186ccf5d5
Reactorless tests 1: the test env ( #6952 )
...
* Reactorless tests.
* Updates after merging master.
* Update a new import.
* Updates after merging master.
* Update testenv:no-reactor-pinned.
* Foundations for the reactorless mode.
* Fixes after merging master.
* Updates after merging reactorless-base.
* Add simple subprocess tests for reactorless AsyncCrawler*.
* More reactorless tests.
* Refactor AsyncCrawlerProcess.start().
* More checks.
* Fix test_reactorless_import_hook.
* More tests.
* Call install_reactor() before asyncio.run().
* Cleanup.
* Rephrase.
* Fix a test.
* Fixes.
* Rephrasing.
* Set TELNETCONSOLE_ENABLED=False in the reactorless mode.
* Fix a new import.
* Clarify --reactor marks.
* Cleanup.
* More docs/comments.
* Update test_fallback_workflow().
* Update TestDeferredFFromCoroF.
* Typing improvements.
2026-02-02 19:25:30 +05:00
54d8562fb0
Assorted test improvements ( #7232 )
...
* Await some missed deferreds.
* Simplify test_engine_stop_download_*.
* Fix the header value.
* Extend TestHttpWithCrawlerBase.
* Make test_tls_logging more universal.
* Add more tests for header handling.
* Clarify certificate and ip_address integration tests.
* Make test_download_with_maxsize_very_large_file() more universal.
* Fix test_coroutine_asyncio().
* Typo.
2026-02-02 09:37:39 +01:00
ed63fa94d6
Deprecate spider args of middlewares and pipelines. ( #7006 )
...
* Deprecate spider args of spider middleware methods.
* Deprecate the spider arg of pipeline process_item().
* Fix a typing issue.
* Deprecate the spider arg of pipeline {open,close}_spider().
* Cleanup.
* Update docs.
* Add pragma: no cover to some deprecated code.
* More tests.
* More tests.
* More tests.
* Simplify _process_parallel().
2025-08-21 13:23:10 +05:00
d3e15a10cf
Add ItemPipelineManager.process_item_async() and ensure_awaitable(). ( #7005 )
...
* Add ItemPipelineManager.process_item_async() and ensure_awaitable().
* Wording.
2025-08-12 19:57:37 +05:00
6b2997af90
Migrate to pytest-twisted ( #6938 )
...
* Migrate to pytest-twisted (WIP)
* Some typing fixes.
* Make --reactor=asyncio the default again.
* Try installing the correct event loop policy in tests on Windows.
* Make reactor_pytest a normal fixture.
* Fix test warnings.
* Fix FTPDownloadHandler teardown.
* Cleanups, typing.
* More cleanup.
* Update only_asyncio/only_not_asyncio mark messages.
2025-07-06 21:27:17 +05:00
16f168b406
Deprecate mustbe_deferred() and defer_*(). ( #6937 )
2025-07-04 16:41:15 +05:00
8b3c3ea4ae
Refactor test functions that were returning a Deferred directly. ( #6889 )
2025-06-30 09:19:23 +02:00
b4d11b8b25
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
3aa5e75787
Use AsyncCrawlerProcess in commands. ( #6845 )
...
* Use AsyncCrawlerProcess in commands.
* Ignore coverage of abstract methods.
* Address feedback.
2025-06-03 17:47:52 +05:00
d400aa3e2d
Add _parallel_asyncio(). ( #6852 )
2025-06-03 11:19:15 +02:00
dceb85bf3e
Add is_asyncio_available(). ( #6827 )
...
* Add is_asyncio_available().
* Print unexpected warnings in test_install_asyncio_reactor().
* Fix printing warnings.
* Fix printing warnings - 2.
* Skip TestDeferredToFuture on non-asyncio.
* Test the is_asyncio_available() exception.
2025-05-28 14:46:39 +05:00
b9caaf8a63
Simplify deferred_from_coro(), add more tests.
2025-05-16 17:16:45 +05:00
d161d1d47d
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
a898331d14
Preparations for switching to direct pytest. ( #6678 )
2025-02-16 20:28:58 +01:00
7dd92e6e43
Add pylint rules to ruff, refresh the ignore list of pylint itself ( #6584 )
2024-12-16 09:44:36 +01:00
b042ad255d
Address some previously ignored pylint messages. ( #6531 )
2024-11-11 11:49:52 +01:00
ddc98fe91b
Deprecate scrapy.utils.defer.process_chain_both(). ( #6397 )
2024-06-10 10:16:26 +02:00
0d445a3224
refactor(yield): use `yield from` syntax
2024-01-05 12:30:10 +01:00
a5c1ef8276
sort imports with isort
2023-01-25 14:25:15 -06:00
e211ec0aa2
adding black formatter to all the code
2022-11-29 11:30:46 -03:00
f9a29f03d9
Address some issues reported by Pylint ( #5677 )
...
Co-authored-by: Felipe Boff Nunes <51033921+felipeboffnunes@users.noreply.github.com>
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
2022-11-27 11:00:13 +01:00
c105a7a85f
Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased
2021-10-22 21:52:47 +05:00
3a263280ba
Fix typos
2021-10-11 22:32:42 +08:00
a6034f942b
Add tests for _AsyncCooperatorAdapter.
2021-03-23 22:47:37 +05:00
40eab1d473
Drop a duplicate import.
2021-02-18 19:56:12 +05:00
58706c6598
Merge branch 'asyncdef-tests' into asyncio-parse-asyncgen-proper-rebased
2021-02-18 17:20:25 +05:00
abbbfbbb38
Add tests for deferred_f_from_coro_f.
2021-02-12 22:41:16 +05:00
acff1eb496
Add aiter_errback.
2021-02-05 16:21:56 +05:00
58ca8bbf6d
Use f-strings ( #4307 )
2020-08-26 07:11:05 +02:00
9aea1f0961
Remove backslash (tests)
2020-07-09 11:04:46 -03:00
6f8758624c
Flake8: remove F841
2020-05-11 13:50:34 -03:00
6fb85951ce
fix E22X flake8
2020-02-21 08:39:14 +01:00
91bbc70bc1
fix E30X flake8 ( #4355 )
2020-02-21 06:05:31 +01:00
b5c4c2cae8
Keep 2 spaces between code and inline comments ( #4195 )
2019-12-13 14:20:48 +01:00
714e0d3ac9
Merge remote-tracking branch 'upstream/master' into remove-six-code
2019-11-20 12:41:38 -03:00
f261cf65e9
Add missing blank lines between functions and classes
...
Also fixed 2 unrelated Flake8 issues
2019-11-18 17:16:09 +01:00
5ab0f189ce
Remove six.moves occurrences from tests
2019-11-03 00:01:09 -03:00
4ca61a2051
Update deprecated test aliases
...
- change ``failIf`` to ``assertFalse``
- change ``asertEquals`` to ``assertEqual``
- change ``assert_`` to ``assertTrue``
https://docs.python.org/2/library/unittest.html#deprecated-aliases
2017-08-09 12:52:10 +02:00
9aaa8f8125
Improve python3 support for test_utils_defer.
2015-07-25 12:58:39 +02:00
c2d716807a
Use LogCapture in testfixtures package for tests
...
This allows to remove `get_testlog` helper, `flushLoggedErrors` from
twisted.trial.unittest.TestCase and Twisted log observers created for
each test on conftest.py.
2015-04-22 17:24:40 -03:00
628a8d8a15
TST modernize some of the asserts
2014-08-01 23:44:23 +06:00
242c08553f
move tests under root dir
2014-07-30 19:45:08 -03:00
Andrey Rakhmatullin
Rotzbua
Jalil SA
Emmanuel Rondan
Mark Mayo
Kian-Meng, Ang
Ammar Najjar
Eugenio Lacuesta
Marc Hernandez Cabot
Adrián Chaves
Chomba Ng'ang'a
Zoltán Szeredi
Julia Medina
Mikhail Korobov
Daniel Graña