* overall prettyfication
- don't create empty mutable containers (lists, dicts) where it is appropriate
- removed from ignore section and applied some rules from ruff (but keep them ignored in tests)
- use `deque` in `_AsyncCooperatorAdapter` instead of `list.pop(0)`
- remove `f` prefix from strings without any formatting
* return `SIM300` to ignore
* apply ruff rules to all files
* another one
* remove extra space in pyproject.toml
* simplify `__getattr__` in `scrapy.utils.url`
* lazy `url_is_from_any_domain` and `url_is_from_spider`
* improve typing
* specify `arg_to_iter` with None as arg
* improve trackref
- use `NoneType` from `types` since python 3.9 is no longer supported
- use `monotonic` instead of `time` and fix flakiness of `get_oldest`
* tracing GC is no joke
* refine tests
* explain the tests
* add note for pypy in docs + ...
return empty tuple instead of list in `iter_all`
* Add docs for HttpxDownloadHandler.
* Handle httpx not being installed.
* Revert test_not_configured_without_asyncio().
* Restore the comment.
* Cleanup.
* Import some stuff from service_identity directly.
* Extract _log_tls().
* Sync validating IP addresses.
* Introduce _deps_compat.
* Deprecate getCertificateOptions().
* Deprecate ScrapyClientTLSOptions.
* Add pragma: no cover.
* verify=False and trustRoot=platformTrust() are the defaults.
* Improve docstrings for TLS classes.
* Deprecate AcceptableProtocolsContextFactory.
* Switch from DefaultOpenSSLContextFactory to CertificateOptions.
* Deprecate load_context_factory_from_settings().
* Remove the outdated warning about incompatible factories.
* Simplify _load_context_factory_from_settings().
* Make CertificateOptions and Context once per factory.
* pragma: no cover
* Add security warning about pickle files in JOBDIR
The job directory contains files serialized with pickle (spider.state,
request queues), which can execute arbitrary code when loaded. Add a
warning so users know to treat JOBDIR with the same trust level as
their project code.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Make JOBDIR security warning more concise and accurate
Simplify the warning to focus on treating the job directory
as trusted, without mentioning specific serialization details.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Make test_start_deprecated_super() more robust.
* Work around the FTPFilesStore.FTP_* initialization.
* Enable xdist on CI.
* Add PYTEST_ADDOPTS to tox passenv.
Kailesh
Albert Eduardovich N.
Ahmed Arshad
Adrian
Andrey Rakhmatullin
Julian Hamze
Yuval Elbar
pierreeurope