* Add security warning about pickle files in JOBDIR
The job directory contains files serialized with pickle (spider.state,
request queues), which can execute arbitrary code when loaded. Add a
warning so users know to treat JOBDIR with the same trust level as
their project code.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Make JOBDIR security warning more concise and accurate
Simplify the warning to focus on treating the job directory
as trusted, without mentioning specific serialization details.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Make test_start_deprecated_super() more robust.
* Work around the FTPFilesStore.FTP_* initialization.
* Enable xdist on CI.
* Add PYTEST_ADDOPTS to tox passenv.
* Fix test for getwithbase() to ensure class-keyed overrides with None are handled correctly
* Test actual import paths from default_settings
* Improvements
* Remove unused logger
* Run pre-commit
* Restore and improve duplicate key handling and warning
* type → object
---------
Co-authored-by: Adrian Chaves <adrian@zyte.com>
* Add Jobdir documentation
* Fix link format
* remove doubtful comments
* some more edits
* Keep the information in jobs.rst an example, and provide details in the reference docs of the corresponding components
* Minor edits
---------
Co-authored-by: Adrian Chaves <adrian@zyte.com>
* Add Pydantic support documentation
* Fix Sphinx references in Pydantic section for non-Sphinx docs
* Minor reformatting
---------
Co-authored-by: Adrian Chaves <adrian@zyte.com>
* Move no-reactor to pytest-asyncio.
* Unify the style.
* Skip a test that installs a reactor in no-reactor.
* Fix pinned env deps.
* Strict pytest-asyncio mode.
* Await some missed deferreds.
* Simplify test_engine_stop_download_*.
* Fix the header value.
* Extend TestHttpWithCrawlerBase.
* Make test_tls_logging more universal.
* Add more tests for header handling.
* Clarify certificate and ip_address integration tests.
* Make test_download_with_maxsize_very_large_file() more universal.
* Fix test_coroutine_asyncio().
* Typo.
* Foundations for the reactorless mode.
* Add simple subprocess tests for reactorless AsyncCrawler*.
* More reactorless tests.
* Refactor AsyncCrawlerProcess.start().
* More checks.
* Fix test_reactorless_import_hook.
* More tests.
* Call install_reactor() before asyncio.run().
* Cleanup.
* Rephrase.
* Rephrasing.
* Set TELNETCONSOLE_ENABLED=False in the reactorless mode.
* Initial release notes for 2.14.0, up to ed63fa9.
* Cover 2.14 in the release notes up to 393d715.
* Cover 2.14 in the release notes up to eb49647.
* Cover 2.14 in the release notes up to 426aafd.
* Cover 2.14 in the release notes up to 1e8de24.
* Cover 2.14 in the release notes up to 5a7e132.
* Bump sphinx-lint.
* Finalize the 2.14.0 release notes.
* Drop more of the old versionadded directives.
* Address feedback.
* Add a test for not having pending tasks.
* Refactor TestFeedExporterSignals.
* Refactor FeedExporter.close_spider().
* More engine start/stop robustness.
* Refactor send_catch_log_async(), deprecate send_catch_log_deferred().
* Add pragma: no cover.
* Warn on signal handlers returning a Deferred.
* Make _pending_close_coros an instance attribute.
* Remove an unused function.
* Remove the outdated TODO.
Lia Launtz
Andrey Rakhmatullin
Yuval Elbar
pierreeurope
Anant Murmu
Varun Chawla
Ryotaro
Martin Schimandl
Michael
Albert Eduardovich N.
Pádraic Slattery
Eugenio Lacuesta
Eugenio Lacuesta
Santiago Beltrán Ruiz
Adham 3llam
Thalison Fernandes
DenisC