Commit Graph

26 Commits

Author SHA1 Message Date
Andrey Rakhmatullin d8ba1571e7
Assorted docs fixes, part 2. (#7725)
* Assorted docs fixes, part 2.

* Second pass.

* Address feedback.
2026-07-14 00:49:44 +05:00
Andrey Rakhmatullin bdf3067935
Assorted docs fixes, part 1. (#7710) 2026-07-07 18:39:34 +05:00
Adrian e7f229f5b2
Clarify the first-class-citizen nature of asyncio support (#7599) 2026-06-11 10:51:46 +02:00
Andrey Rakhmatullin b9be5ce053
Reactorless shell and other small shell improvements (#7395)
* Use AsyncCrawlerProcess if TWISTED_REACTOR_ENABLED=False.

* defer.Deferred -> Deferred.

* Allow Shell with TWISTED_REACTOR_ENABLED=False.

* Shell workflow notes.

* Remove an unused argument.

* Shell.fetch_available.

* Shell._use_reactor.

* Some more comments.

* Simplify _schedule() and fix the spider shell var.

* More async def.

* Reactorless shell support.

* Add pragma: no cover.

* More notes.

* Remove a TODO.
2026-04-08 18:24:51 +05:00
Andrey Rakhmatullin b2b2d0b015
Rename TWISTED_ENABLED to TWISTED_REACTOR_ENABLED. (#7394) 2026-04-02 10:20:54 +02:00
Andrey Rakhmatullin 0c6ccf50b3
Docs for the reactorless mode (#7385)
* Reactorless mode docs.

* Unset the ftp handler in the reactorless mode.

* Cleanup reactorless subprocess tests.

* Better handling of incompatible TWISTED_ENABLED cases.

* Improvements.

* Improve early error logging in test scripts.

* Address feedback.

* Address feedback.
2026-04-01 20:40:34 +05:00
Andrey Rakhmatullin 7bef98b4f1
Release notes for 2.14.0 (#7035)
* Initial release notes for 2.14.0, up to ed63fa9.

* Cover 2.14 in the release notes up to 393d715.

* Cover 2.14 in the release notes up to eb49647.

* Cover 2.14 in the release notes up to 426aafd.

* Cover 2.14 in the release notes up to 1e8de24.

* Cover 2.14 in the release notes up to 5a7e132.

* Bump sphinx-lint.

* Finalize the 2.14.0 release notes.

* Drop more of the old versionadded directives.

* Address feedback.
2026-01-05 14:51:05 +05:00
Adham 3llam 9381ad893d
Remove outdated version marker directives (< 2.8.0) from documentation (#7198) 2025-12-22 10:00:39 +01:00
Andrey Rakhmatullin 552f2fb91e
Generic improvements related to reactorless tests (#6968)
* Skip doctests that import the reactor.

* Extract FTPDownloadHandler tests.

* Skip TestHttps2ClientProtocol via pytestmark.

* Replace some explicit reactor.callLater() in tests.

* Simplify TestRequestSendOrder._test_request_order().
2025-07-25 13:35:00 +05:00
Andrey Rakhmatullin 05b3b205ce
Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` (#6817)
* Add a basic Crawler.crawl_async().

* Add custom loop tests for *CrawlerRunner.

* Add AsyncCrawlerProcess.

* Update related docs.

* Update practices.rst.

* Address test failures.

* Add a note about AsyncCrawler* to the docs about switching reactors.

* Address feedback.

* Update for TID253.

* Simplify test_crawler_crawl_async_twice_parallel_unsupported().
2025-05-28 18:55:44 +05:00
Andrey Rakhmatullin dceb85bf3e
Add is_asyncio_available(). (#6827)
* Add is_asyncio_available().

* Print unexpected warnings in test_install_asyncio_reactor().

* Fix printing warnings.

* Fix printing warnings - 2.

* Skip TestDeferredToFuture on non-asyncio.

* Test the is_asyncio_available() exception.
2025-05-28 14:46:39 +05:00
Andrey Rakhmatullin 9057bf4e1e
More docs about Deferred<->Future interoperability. (#6734) 2025-03-15 11:47:16 +01:00
Andrey Rakhmatullin fc566a7ff9
Don't install the reactor in is_asyncio_reactor_installed(). (#6732) 2025-03-14 20:46:17 +01:00
Andrey Rakhmatullin d0dabbc097
Enable AsyncioSelectorReactor by default. (#6713)
* Enable AsyncioSelectorReactor by default.

* Improve get_crawler(), switch more tests to it.

* Fix the remaining default-reactor test failures.

* Address documentation feedback.

* Make pinned envs more consistent.
2025-03-12 01:18:30 +05:00
Adrián Chaves 72de48be6d asyncio: cover accidental bad reactor installation, sort sections, reword the Windows section 2023-08-04 09:56:30 +02:00
pankaj1707k c1bbb299d7
Add and run pre-commit hook 'blacken-docs'
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
2023-02-01 16:30:57 +05:30
Adrián Chaves b95c634b86 Document how to enforce Scrapy versions on Scrapy components 2022-03-17 22:23:25 +01:00
Adrián Chaves aa0306a167
Cover 2.6.0 in the release notes (#5399) 2022-03-01 12:16:37 +01:00
Andrey Rahmatullin c5ab58056c
Set WindowsSelectorEventLoopPolicy on Windows (#5315) 2021-12-31 20:38:10 +01:00
Andrey Rahmatullin 57dc58123b
Remove the experimental note about asyncio (#5332) 2021-12-31 13:15:08 +01:00
Andrey Rahmatullin 144d1eb834
Add Deferred-to-Future helpers (#5288) 2021-10-22 18:46:01 +02:00
Eugenio Lacuesta d698b5147b
Merge branch 'master' into http2 2020-12-31 11:13:25 -03:00
Adrián Chaves 015c82b974
Scrapy 2.4 release notes (#4808) 2020-10-11 22:09:45 +02:00
Adrián Chaves 4d6359df2d Mark HTTP/2 as experimental 2020-09-11 13:51:05 +02:00
sakshamb2113 42383cc267
Add a setting to customize the asyncio event loop (#4414) 2020-08-19 09:18:14 +02:00
Adrián Chaves 6aa0ba4553
Write release notes for Scrapy 2.0.0 (#4329) 2020-03-03 09:11:11 +01:00