510f09a961
Deprecate ScrapyClientContextFactory. ( #7391 )
2026-04-02 09:29:47 +02:00
6fe27ba33e
Add more no-reactor tests ( #7259 )
...
* Generic changes and scrapy bench.
* scrapy check.
* scrapy crawl.
* scrapy fetch.
* scrapy parse.
* scrapy runspider.
* scrapy shell.
* Skip httpx tests on default-reactor.
* Review requires_reactor marks.
* Make test functions that require an event loop async def.
* Improve test_pending_asyncio_tasks().
* Add Mac OS DNS error.
* Refactor most of test_scheduler.py.
* Finish refactoring DownloaderAwareSchedulerTestMixin.
* Refactor test_engine_loop.py.
* Add the no-reactor-extra-deps tox env, run no-reactor on macos.
* Skip unhandled CancelledError when shutting down the engine.
* Fix typing and pre-commit checks.
* Fix typing problems in master.
---------
Co-authored-by: Adrian <adrian@zyte.com>
2026-03-24 15:29:22 +05:00
7010985e4f
Bump ruff, fix some of rules ( #7277 )
2026-02-23 11:48:38 +01:00
3186ccf5d5
Reactorless tests 1: the test env ( #6952 )
...
* Reactorless tests.
* Updates after merging master.
* Update a new import.
* Updates after merging master.
* Update testenv:no-reactor-pinned.
* Foundations for the reactorless mode.
* Fixes after merging master.
* Updates after merging reactorless-base.
* Add simple subprocess tests for reactorless AsyncCrawler*.
* More reactorless tests.
* Refactor AsyncCrawlerProcess.start().
* More checks.
* Fix test_reactorless_import_hook.
* More tests.
* Call install_reactor() before asyncio.run().
* Cleanup.
* Rephrase.
* Fix a test.
* Fixes.
* Rephrasing.
* Set TELNETCONSOLE_ENABLED=False in the reactorless mode.
* Fix a new import.
* Clarify --reactor marks.
* Cleanup.
* More docs/comments.
* Update test_fallback_workflow().
* Update TestDeferredFFromCoroF.
* Typing improvements.
2026-02-02 19:25:30 +05:00
2347138ba4
Unified download handler exceptions ( #7208 )
...
* Add DownloadTimeoutError.
* Don't expect ConnectionAborted in tests.
* Add DownloadCancelledError.
* Add ResponseDataLoss.
* Add more download handler tests.
* Add DownloadConnectionRefusedError.
* Add UnsupportedURLScheme.
* Add CannotResolveHostError.
* Add DownloadFailedError.
* Remove wrapped Twisted exceptions from lists.
* Update a test expectation.
* Extract wrap_twisted_exceptions().
* Wrap TxTimeoutError.
* Rename ResponseDataLoss and UnsupportedURLScheme.
2026-01-21 23:09:54 +05:00
8c8f4ff033
Extract in-process test HTTP servers into separate processes ( #6960 )
...
* Switch TestEngineBase to mockserver.
* Split mockserver.py.
* Stop using WrappingFactory.
* Remove keyfile/certfile fields from TestHttpBase.
* Move webclient-only resources to test_webclient.py.
* Refactor MockServer.url().
* Migrate TestHttpBase to mockserver.
* Migrate TestSimpleHttpsBase to a separate mockserver.
* Rename TestHttpMockServerBase.
* Migrate TestHttpProxyBase to a separate mockserver.
* Refactor HTTP mockservers.
2025-07-25 12:42:27 +05:00
6b2997af90
Migrate to pytest-twisted ( #6938 )
...
* Migrate to pytest-twisted (WIP)
* Some typing fixes.
* Make --reactor=asyncio the default again.
* Try installing the correct event loop policy in tests on Windows.
* Make reactor_pytest a normal fixture.
* Fix test warnings.
* Fix FTPDownloadHandler teardown.
* Cleanups, typing.
* More cleanup.
* Update only_asyncio/only_not_asyncio mark messages.
2025-07-06 21:27:17 +05:00
df342eee6e
Bump tool versions. ( #6941 )
2025-07-05 10:30:30 +02:00
b4d11b8b25
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
d825133284
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
916fe50974
Fix and ban the top-level twisted.internet.reactor imports. ( #6835 )
2025-05-28 15:53:52 +05:00
5a605969bd
Converting tests to plain asserts, part 2. ( #6699 )
2025-03-06 20:52:41 +01:00
0a80871c3a
Remove scrapy.core.downloader.webclient._parse().
2025-01-28 22:22:09 +05:00
16b998f9ca
Sort out webclient tests.
2025-01-28 01:35:04 +05:00
cde0845ab2
Ruff: migrate pyupgrade and bandit, enable some other rules ( #6577 )
2024-12-10 19:53:27 +01:00
4dcc04be48
Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code.
2024-11-24 12:44:48 +04:00
eda3a89b3f
Remove build_from_settings().
2024-11-11 16:44:47 +05:00
282767f23b
Bump black.
2024-02-28 14:30:38 +05:00
b0ef9a89a1
Remove usage of deprecated mktemp ( #5285 )
2024-02-26 19:53:06 +01:00
c67f730695
create_instance → build_from_crawler, build_from_settings ( #6169 )
2023-12-18 11:51:02 +01:00
a5c1ef8276
sort imports with isort
2023-01-25 14:25:15 -06:00
e211ec0aa2
adding black formatter to all the code
2022-11-29 11:30:46 -03:00
f9a29f03d9
Address some issues reported by Pylint ( #5677 )
...
Co-authored-by: Felipe Boff Nunes <51033921+felipeboffnunes@users.noreply.github.com>
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
2022-11-27 11:00:13 +01:00
40d9ca3bdd
use pathlib
2022-10-17 17:40:10 -04:00
6770d1ec62
chore(tests): Remove validations for unsupported modules versions
2022-06-09 09:08:09 -03:00
1e1cfc26db
Copy resource classes from twisted.web.test.test_webclient.
2022-02-08 21:01:16 +05:00
a71d6ef29d
2.5.0 release notes ( #5028 )
...
Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
2021-04-06 16:09:07 +02:00
b83a1a6fbf
Disable test under pypy
2020-12-21 17:21:08 +01:00
58ca8bbf6d
Use f-strings ( #4307 )
2020-08-26 07:11:05 +02:00
d29bec60d7
Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest) ( #4504 )
...
* Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest)
* Log a detailed error message to discover why MockServer is not working
* Go for all lines!
* Disable tests based on mitmproxy while running on PyPy
* Fix test_get_func_args for PyPy 3.6+
* Make testPayloadDefaultCiphers work regardless of OpenSSL default ciphers
* Crossing fingers…
* Rename: testPayloadDefaultCiphers → testPayloadDisabledCipher
* Test the PyPy version currently documented as the minimum required version
* Fix the PYPY_VERSION tag
* Update the documentation about supported PyPy versions
* Also test the latest 3.5 Python version with PyPy
* Fix the PYPY_VERSION value for the latest 3.5 version
* Use pinned dependencies for asyncio and PyPy tests against oldest supported Python versions
* Fix PyPy installation for the pypy3-pinned Tox environment
* Try installing Cython
* Maybe PyPy requires lxml 3.6.0?
* install.rst: minor clarification
* lxml 4.0.0 is required on PyPy
* Require setuptools 18.5+
* Revert "Require setuptools 18.5+"
This reverts commit 017ec33ac2
2020-07-17 02:19:24 +05:00
9aea1f0961
Remove backslash (tests)
2020-07-09 11:04:46 -03:00
881b4f417f
Flake8: Remove E128 (tests, part 3)
2020-05-06 19:33:22 -03:00
4c12a234ae
Flake8: Remove E126
2020-05-06 16:10:21 -03:00
d71804ef29
Flake8: Remove E122
2020-05-06 15:23:36 -03:00
8643e8d355
Flake8: remove E123 (Closing bracket does not match indentation of opening bracket's line)
2020-05-06 12:26:04 -03:00
94d7ad76cb
Fix pycodestyle E2XX (whitespace) ( #4468 )
2020-04-15 14:11:37 +02:00
915e363db5
Remove a 'twisted.test.proto_helpers' deprecation warning
2020-03-07 18:03:25 +00:00
6fb85951ce
fix E22X flake8
2020-02-21 08:39:14 +01:00
91bbc70bc1
fix E30X flake8 ( #4355 )
2020-02-21 06:05:31 +01:00
d72444b9c8
Remove more six imports
2019-11-03 01:11:28 -03:00
eaeaa40b99
Remove six.PY* checks
2019-11-03 01:08:08 -03:00
1aba513693
Remove six.iter* occurrences
2019-11-03 00:26:44 -03:00
aaa5229e5d
Fixes and improvements for DOWNLOADER_CLIENT_TLS_CIPHERS.
2019-08-13 16:56:26 +05:00
9a8edf2bf1
Tests for setting SSL ciphers.
2019-08-13 16:53:19 +05:00
fd27cde24d
Update asserts to use more generic ones
2017-08-09 12:52:10 +02:00
4ca61a2051
Update deprecated test aliases
...
- change ``failIf`` to ``assertFalse``
- change ``asertEquals`` to ``assertEqual``
- change ``assert_`` to ``assertTrue``
https://docs.python.org/2/library/unittest.html#deprecated-aliases
2017-08-09 12:52:10 +02:00
78cb46a079
TST fix a weird OS X testing issue
...
pytest intercepts temp file creation to provide readable file/folder
names; path is built from method name; in case of conflicts
pytests uses increasing numbers, but it seems it doesn’t account
for case-insensitive (but case preserving) OS X filesystem. There
are methods named test_encoding, pytest thinks test_Encoding is
different and fails to create a test folder
2017-02-16 18:22:04 +05:00
b9928558b2
TST remove temporary files and folders
2017-02-16 17:50:38 +05:00
4946434582
revert most changes to this test, and clarify - it is valid only on py2, because urls are strictly unicode on py3
2016-01-18 15:23:01 +03:00
a2efd389b0
clarify: rename r_transform to response_transform
2016-01-18 15:09:54 +03:00
Andrey Rakhmatullin
Jon
Swati Anshu
Jalil SA
Emmanuel Rondan
Mark Mayo
Matthew Donoughe
Laerte Pereira
Adrián Chaves
Eugenio Lacuesta
Ammar Najjar
Eugenio Lacuesta
nyov
Marc Hernandez Cabot
Chomba Ng'ang'a
Mikhail Korobov
Konstantin Lopuhin