* Remove support for sync process_spider_output().
* Slight check fix.
* Don't expect exceptions from calling process_spider_output().
* Remove dead code.
* Fix test_deprecated_mw_spider_arg() to run all methods.
* Fix typing.
* Typos.
* Remove references to the removed section, move the universal section to the spider middleware docs and write it from a different perspective
* Fix indentation issue
* Minor rewordings
* older versions → lower versions
* Update news.rst
---------
Co-authored-by: Adrian Chaves <adrian@zyte.com>
* Add Python 3.14 (alpha3) to CI.
* Disable mitmproxy on 3.14 for now.
* 3.14.0-alpha.4.
* 3.14.0-alpha.5
* 3.14.0-beta2.
* 3.14 release.
* Fix test_non_pickable_object.
* Fix handling of file:/path feed URIs.
* Better mocking of streams for TextTestResult.
* Do not use .php in test_file_path() as it's now a known extension.
* Fix the URL in TestFeedExporterSignals.
* Fix typing.
* Bump more envs to 3.14.
* Silence pylint.
* Fix another test for .php handling change.
* Remove test_install_asyncio_reactor.
* More bumps to 3.14.
* Revert docs-tests to use 3.13.
* Debug options for Windows.
* Re-enable xdist.
* Revert Windows PYTEST_ADDOPTS.
* Silence loop policy deprecation warnings.
* Restore a lost pylint suppression.
* Update asyncio_enabled_reactor_same_loop.py to new Twisted.
* Fix RobotFileParser tests for Python 3.14.5.
* Unpin Twisted, add a twisted-trunk tox env (without http2 for now).
* Hide the _setAcceptableProtocols import.
* Silence ScrapyClientContextFactory HTTP/1.0 warnings.
* Update ScrapyClientTLSOptions for unreleased Twisted.
* Add twisted-trunk to CI.
* Update for new changes in Twisted trunk.
* Xfail a test failing with newer Twisted.
* Test Twisted trunk with extra-deps, update the ALPN code.
* Cleanup.
* Fix typing.
* Update relevant type hints for Twisted trunk.
* Update test_no_context_sharing().
* Silence a weird mypy error.
* Update Twisted versions, unpin pyOpenSSL.
* Update a comment.
* Silence pylint.
* Make a factory fixture.
* Improve the _setAcceptableProtocols comment.
* Set OP_LEGACY_SERVER_CONNECT on new Twisted too.
* Add Twisted[http2] to extra-deps-pinned.
* Run mitmproxy-based tests for every handler and improve them.
* Fixes for H2DownloadHandler.
* Revert "Fixes for H2DownloadHandler."
This reverts commit bcdbd097cd.
* Add TestHttpsProxy for HTTP11DownloadHandler.
* Use the configured context factory in ScrapyProxyAgent.
* Reword.
* Reword.
* pragma: no cover for H2DownloadHandler proxy code.
* Raise an exception for HTTPS proxies instead of using HTTP.
* Remove non-working proxy support and explicitly forbid HTTP.
* pragma: no cover
* Add a docs note about HTTPS proxies.
* Rename ScrapyProxyAgent.
* Restore 2.14 getwithbase, add a new method for class key deduplication
* Use the original getwithbase code, not some equivalent
* Fix mocking
* Test key exceptions
* Popen cleanup.
* Improve the test for response headers.
* Fix default headers of Httpx and H2 handlers, add tests.
* Check Location in redirect tests.
* Simplify bindaddress tests.
* Add download_latency to HttpxDownloadHandler, add tests.
* Remove explicit method="GET".
* Don't expect a specific value in test_download_with_maxsize_very_large_file
* Use a non-existent scheme in test_unsupported_scheme.
* Remporarily rollback the test_download_latency test check.
* Better check in test_download_latency.
* Improve logic for StreamCloseReason.CANCELLED.
* Use AsyncCrawlerProcess if TWISTED_REACTOR_ENABLED=False.
* defer.Deferred -> Deferred.
* Allow Shell with TWISTED_REACTOR_ENABLED=False.
* Shell workflow notes.
* Remove an unused argument.
* Shell.fetch_available.
* Shell._use_reactor.
* Some more comments.
* Simplify _schedule() and fix the spider shell var.
* More async def.
* Reactorless shell support.
* Add pragma: no cover.
* More notes.
* Remove a TODO.
* overall prettyfication
- don't create empty mutable containers (lists, dicts) where it is appropriate
- removed from ignore section and applied some rules from ruff (but keep them ignored in tests)
- use `deque` in `_AsyncCooperatorAdapter` instead of `list.pop(0)`
- remove `f` prefix from strings without any formatting
* return `SIM300` to ignore
* apply ruff rules to all files
* another one
* remove extra space in pyproject.toml
* simplify `__getattr__` in `scrapy.utils.url`
* lazy `url_is_from_any_domain` and `url_is_from_spider`
* improve typing
* specify `arg_to_iter` with None as arg
* improve trackref
- use `NoneType` from `types` since python 3.9 is no longer supported
- use `monotonic` instead of `time` and fix flakiness of `get_oldest`
* tracing GC is no joke
* refine tests
* explain the tests
* add note for pypy in docs + ...
return empty tuple instead of list in `iter_all`
* Import some stuff from service_identity directly.
* Extract _log_tls().
* Sync validating IP addresses.
* Introduce _deps_compat.
* Deprecate getCertificateOptions().
* Deprecate ScrapyClientTLSOptions.
* Add pragma: no cover.
* verify=False and trustRoot=platformTrust() are the defaults.
* Improve docstrings for TLS classes.
* Deprecate AcceptableProtocolsContextFactory.
* Switch from DefaultOpenSSLContextFactory to CertificateOptions.
* Deprecate load_context_factory_from_settings().
* Remove the outdated warning about incompatible factories.
* Simplify _load_context_factory_from_settings().
* Make CertificateOptions and Context once per factory.
* pragma: no cover
Adrian Chaves
Ayush Singh
Ethan Kuo
Andrey Rakhmatullin
Fardin Alizadeh
Albert Eduardovich N.
Adrián Chaves
Hamzah Alshawwaf
Kailesh