Commit Graph

46 Commits

Author SHA1 Message Date
Andrey Rakhmatullin d8ba1571e7
Assorted docs fixes, part 2. (#7725)
* Assorted docs fixes, part 2.

* Second pass.

* Address feedback.
2026-07-14 00:49:44 +05:00
Andrey Rakhmatullin bdf3067935
Assorted docs fixes, part 1. (#7710) 2026-07-07 18:39:34 +05:00
Andrey Rakhmatullin 4cb049cb15
Small docs fixes. (#7598) 2026-06-11 10:51:03 +02:00
Andrey Rakhmatullin a8a8f20d9c
Remove support for sync process_spider_output(). (#7504)
* Remove support for sync process_spider_output().

* Slight check fix.

* Don't expect exceptions from calling process_spider_output().

* Remove dead code.

* Fix test_deprecated_mw_spider_arg() to run all methods.

* Fix typing.

* Typos.

* Remove references to the removed section, move the universal section to the spider middleware docs and write it from a different perspective

* Fix indentation issue

* Minor rewordings

* older versions → lower versions

* Update news.rst

---------

Co-authored-by: Adrian Chaves <adrian@zyte.com>
2026-05-15 16:36:05 +05:00
Kaileshwar R K 54a4c3af89
Deprecate the mail API (#7263) 2026-03-24 21:31:12 +01:00
Andrey Rakhmatullin 7bef98b4f1
Release notes for 2.14.0 (#7035)
* Initial release notes for 2.14.0, up to ed63fa9.

* Cover 2.14 in the release notes up to 393d715.

* Cover 2.14 in the release notes up to eb49647.

* Cover 2.14 in the release notes up to 426aafd.

* Cover 2.14 in the release notes up to 1e8de24.

* Cover 2.14 in the release notes up to 5a7e132.

* Bump sphinx-lint.

* Finalize the 2.14.0 release notes.

* Drop more of the old versionadded directives.

* Address feedback.
2026-01-05 14:51:05 +05:00
Andrey Rakhmatullin d1bd8eb49f
Async API for download handlers. (#7164) 2025-12-31 19:03:16 +01:00
Andrey Rakhmatullin a2463325db
Allow Crawler.crawl_async() without asyncio. (#7202) 2025-12-31 18:33:59 +01:00
Adham 3llam 9381ad893d
Remove outdated version marker directives (< 2.8.0) from documentation (#7198) 2025-12-22 10:00:39 +01:00
Andrey Rakhmatullin 9bfa58e36c
Deprecate send_catch_log_deferred(). (#7161)
* Add a test for not having pending tasks.

* Refactor TestFeedExporterSignals.

* Refactor FeedExporter.close_spider().

* More engine start/stop robustness.

* Refactor send_catch_log_async(), deprecate send_catch_log_deferred().

* Add pragma: no cover.

* Warn on signal handlers returning a Deferred.

* Make _pending_close_coros an instance attribute.

* Remove an unused function.

* Remove the outdated TODO.
2025-12-10 14:42:49 +05:00
Andrey Rakhmatullin 0adc561348
Add a list of remaining Deferred APIs. (#7146) 2025-11-27 20:13:04 +05:00
Andrey Rakhmatullin ed63fa94d6
Deprecate spider args of middlewares and pipelines. (#7006)
* Deprecate spider args of spider middleware methods.

* Deprecate the spider arg of pipeline process_item().

* Fix a typing issue.

* Deprecate the spider arg of pipeline {open,close}_spider().

* Cleanup.

* Update docs.

* Add pragma: no cover to some deprecated code.

* More tests.

* More tests.

* More tests.

* Simplify _process_parallel().
2025-08-21 13:23:10 +05:00
Felipe Benevolo 89f53f0555
Added download_async() to ExecutionEngine and test cases for download() and download_async() (#6842)
* added async_download()

* add case tests for ExecutionEngine download() and async_download()

* change async_download() to download_async() in Engine class and subsequent tests

* fixed reactor import in test_engine.py

* Simplify TestEngineDownloadAsync.

* replaced tearDown by self.engine.downloader.close in test_engine.py and replace @defer.inlineCallbacks by @inlineCallbacks

* Simplify ExecutionEngine.download().

* Remove an unneeded import.

* Refactor ExecutionEngine.download{,_async}() tests.

* Deprecate ExecutionEngine.download().

* Update doc examples to use download_async().

* Replace uses of download() with download_async().

* Fix test_downloadermiddleware_robotstxt.py.

* Remove a stray file.

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2025-07-27 23:17:51 +05:00
Andrey Rakhmatullin f2fc177f1f
Fix a wrong versionadded usage. (#6822) 2025-05-23 11:06:33 +02:00
Andrey Rakhmatullin 0ce693dfa9 Update VERSION strings. 2025-05-08 13:23:19 +05:00
Andrey Rakhmatullin b07e6d4ea8 Merge remote-tracking branch 'origin/master' into relnotes-2.13.0 2025-05-08 13:17:47 +05:00
Andrey Rakhmatullin e86b5051c2 Merge commit '373e501' into relnotes-2.13.0 2025-05-08 10:55:57 +05:00
Adrián Chaves 036f3e5627
Support asynchronous start requests (#6729) 2025-05-07 19:04:03 +02:00
Andrey Rakhmatullin acb5f895cd
Update docs/topics/coroutines.rst
Co-authored-by: Adrián Chaves <adrian@chaves.io>
2025-05-05 22:28:36 +05:00
Andrey Rakhmatullin b93290f28a Add a list of Deferred-only APIs. 2025-05-05 19:38:04 +05:00
Andrey Rakhmatullin 095140f134 Cover the current master in the release notes (up to daf9db7). 2025-04-25 19:43:29 +05:00
Andrey Rakhmatullin 9057bf4e1e
More docs about Deferred<->Future interoperability. (#6734) 2025-03-15 11:47:16 +01:00
Andrey Rakhmatullin c200458f24
Add more docs for updating sync spider middlewares. (#6688) 2025-02-25 00:16:23 +01:00
Adrián Chaves d362699fa3 Move inline request examples to the coroutines documentation 2023-05-16 13:39:02 +02:00
Andrey Rakhmatullin 76eba9977b
Skip or fix failing code block tests (#5826) 2023-02-14 09:40:38 +01:00
pankaj1707k c1bbb299d7
Add and run pre-commit hook 'blacken-docs'
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
2023-02-01 16:30:57 +05:30
Adrián Chaves 06c8f673af
2.7 release notes (#5680)
* Fix the display name of documented fingerprinter class methods

* Initial draft for the Scrapy 2.7 release notes

* Update VERSION and PREVIOUS_VERSION references

* Clarify the restrictions lifted for item field output names

* Fix the description of the BOM bug fix

* Fix the note about changes in MIME sniffing

* Fix typo

* Extend highlights

* Fyx typo
2022-10-17 18:04:29 +05:00
Andrey Rahmatullin a0681fb811
Merge branch 'master' into asyncio-parse-asyncgen-proper-rebased 2022-07-22 18:30:38 +05:00
Adrián Chaves b95c634b86 Document how to enforce Scrapy versions on Scrapy components 2022-03-17 22:23:25 +01:00
Adrián Chaves b78e6915c6 Clarify that without async-to-sync conversions items yielded before an exception are processed 2022-03-16 20:17:25 +01:00
Adrián Chaves fd08bb6cd9 Refactor the asynchronous process_spider_output documentation 2022-03-16 18:10:57 +01:00
Andrey Rakhmatullin f789547551 Implement spider middleware iterable upgrade/downgrade. 2022-01-11 18:28:32 +05:00
Burak Can Kahraman a9dfd85ea6 Document coroutines for signals. 2021-12-30 15:48:53 +03:00
Andrey Rakhmatullin c105a7a85f Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-10-22 21:52:47 +05:00
Andrey Rahmatullin 144d1eb834
Add Deferred-to-Future helpers (#5288) 2021-10-22 18:46:01 +02:00
Adrián Chaves 8a2768561f
Merge branch 'master' into asyncio-parse-asyncgen-proper-rebased 2021-07-16 14:32:15 +02:00
Marlena Chatzigrigoriou d7deba7e89
Document all import paths and use the shortest in examples (#5099) 2021-07-14 10:34:28 +02:00
Andrey Rakhmatullin de69d967f9 Fix async spider examples. 2021-06-11 16:12:25 +05:00
Andrey Rakhmatullin 849472535e Update docs. 2021-04-02 20:20:35 +05:00
Andrey Rakhmatullin a97dc55c71 Add/improve docs. 2021-03-25 20:49:12 +05:00
Andrey Rakhmatullin 58f8481301 Update docs. 2021-02-05 16:21:56 +05:00
Adrián Chaves 195f738bba
Update Python version references after dropping support for 3.5 (#4742)
* Update Python version references after dropping support for 3.5

* Remove outdated test

* Undo change affecting collect_asyncgen

* Undo change to be handled by #4743

* Remove unused import

* Remove unused import

* Update tests/requirements-py3.txt

Co-authored-by: Mikhail Korobov <kmike84@gmail.com>

Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
2020-08-27 15:43:43 +05:00
Eugenio Lacuesta ec98dabfab
Support for dataclass and attrs items (#3881) 2020-06-14 11:45:27 +02:00
Ashe c207dbf939
Remove the asyncio warning from coroutines page (#4513) 2020-04-27 19:45:19 +02:00
Aditya 80c69d68ad [docs] refactor python docs links using intersphinx 2020-03-24 18:35:42 +05:30
Adrián Chaves 6aa0ba4553
Write release notes for Scrapy 2.0.0 (#4329) 2020-03-03 09:11:11 +01:00