* Add Python 3.14 (alpha3) to CI.
* Disable mitmproxy on 3.14 for now.
* 3.14.0-alpha.4.
* 3.14.0-alpha.5
* 3.14.0-beta2.
* 3.14 release.
* Fix test_non_pickable_object.
* Fix handling of file:/path feed URIs.
* Better mocking of streams for TextTestResult.
* Do not use .php in test_file_path() as it's now a known extension.
* Fix the URL in TestFeedExporterSignals.
* Fix typing.
* Bump more envs to 3.14.
* Silence pylint.
* Fix another test for .php handling change.
* Remove test_install_asyncio_reactor.
* More bumps to 3.14.
* Revert docs-tests to use 3.13.
* Debug options for Windows.
* Re-enable xdist.
* Revert Windows PYTEST_ADDOPTS.
* Silence loop policy deprecation warnings.
* Restore a lost pylint suppression.
* Update asyncio_enabled_reactor_same_loop.py to new Twisted.
* Fix RobotFileParser tests for Python 3.14.5.
* Run mitmproxy-based tests for every handler and improve them.
* Fixes for H2DownloadHandler.
* Revert "Fixes for H2DownloadHandler."
This reverts commit bcdbd097cd.
* Add TestHttpsProxy for HTTP11DownloadHandler.
* Use the configured context factory in ScrapyProxyAgent.
* Reword.
* Reword.
* pragma: no cover for H2DownloadHandler proxy code.
* Raise an exception for HTTPS proxies instead of using HTTP.
* Remove non-working proxy support and explicitly forbid HTTP.
* pragma: no cover
* Add a docs note about HTTPS proxies.
* Rename ScrapyProxyAgent.
* Cover 2.15.0 in the release notes (up to 31bf7c3).
* Replace VERSION.
* Document unified download handler exceptions.
* Updates.
* Cover 2.15.0 in the release notes (up to 0c6ccf5).
* Cover 2.15.0 in the release notes (up to b68f267).
* Run linkcheck and fix issues.
* Cleanup unused link targets.
* Cleanup.
* Use AsyncCrawlerProcess if TWISTED_REACTOR_ENABLED=False.
* defer.Deferred -> Deferred.
* Allow Shell with TWISTED_REACTOR_ENABLED=False.
* Shell workflow notes.
* Remove an unused argument.
* Shell.fetch_available.
* Shell._use_reactor.
* Some more comments.
* Simplify _schedule() and fix the spider shell var.
* More async def.
* Reactorless shell support.
* Add pragma: no cover.
* More notes.
* Remove a TODO.
* Add docs for HttpxDownloadHandler.
* Handle httpx not being installed.
* Revert test_not_configured_without_asyncio().
* Restore the comment.
* Cleanup.
* Add security warning about pickle files in JOBDIR
The job directory contains files serialized with pickle (spider.state,
request queues), which can execute arbitrary code when loaded. Add a
warning so users know to treat JOBDIR with the same trust level as
their project code.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Make JOBDIR security warning more concise and accurate
Simplify the warning to focus on treating the job directory
as trusted, without mentioning specific serialization details.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add Jobdir documentation
* Fix link format
* remove doubtful comments
* some more edits
* Keep the information in jobs.rst an example, and provide details in the reference docs of the corresponding components
* Minor edits
---------
Co-authored-by: Adrian Chaves <adrian@zyte.com>
* Add Pydantic support documentation
* Fix Sphinx references in Pydantic section for non-Sphinx docs
* Minor reformatting
---------
Co-authored-by: Adrian Chaves <adrian@zyte.com>
Andrey Rakhmatullin
Abhinav W
Adrian
Adrián Chaves
Sanjay M
NI283105
Albert Eduardovich N.
Kailesh
Ahmed Arshad
Julian Hamze
Yuval Elbar
Anant Murmu
Martin Schimandl
Michael