scrapy

Commit Graph

Author	SHA1	Message	Date
Andrey Rakhmatullin	05b3b205ce	Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` (#6817 ) * Add a basic Crawler.crawl_async(). * Add custom loop tests for CrawlerRunner. Add AsyncCrawlerProcess. * Update related docs. * Update practices.rst. * Address test failures. * Add a note about AsyncCrawler* to the docs about switching reactors. * Address feedback. * Update for TID253. * Simplify test_crawler_crawl_async_twice_parallel_unsupported().	2025-05-28 18:55:44 +05:00
Andrey Rakhmatullin	dceb85bf3e	Add is_asyncio_available(). (#6827 ) * Add is_asyncio_available(). * Print unexpected warnings in test_install_asyncio_reactor(). * Fix printing warnings. * Fix printing warnings - 2. * Skip TestDeferredToFuture on non-asyncio. * Test the is_asyncio_available() exception.	2025-05-28 14:46:39 +05:00
Andrey Rakhmatullin	9057bf4e1e	More docs about Deferred<->Future interoperability. (#6734 )	2025-03-15 11:47:16 +01:00
Andrey Rakhmatullin	fc566a7ff9	Don't install the reactor in is_asyncio_reactor_installed(). (#6732 )	2025-03-14 20:46:17 +01:00
Andrey Rakhmatullin	d0dabbc097	Enable AsyncioSelectorReactor by default. (#6713 ) * Enable AsyncioSelectorReactor by default. * Improve get_crawler(), switch more tests to it. * Fix the remaining default-reactor test failures. * Address documentation feedback. * Make pinned envs more consistent.	2025-03-12 01:18:30 +05:00
Adrián Chaves	72de48be6d	asyncio: cover accidental bad reactor installation, sort sections, reword the Windows section	2023-08-04 09:56:30 +02:00
pankaj1707k	c1bbb299d7	Add and run pre-commit hook 'blacken-docs' Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.	2023-02-01 16:30:57 +05:30
Adrián Chaves	b95c634b86	Document how to enforce Scrapy versions on Scrapy components	2022-03-17 22:23:25 +01:00
Adrián Chaves	aa0306a167	Cover 2.6.0 in the release notes (#5399 )	2022-03-01 12:16:37 +01:00
Andrey Rahmatullin	c5ab58056c	Set WindowsSelectorEventLoopPolicy on Windows (#5315 )	2021-12-31 20:38:10 +01:00
Andrey Rahmatullin	57dc58123b	Remove the experimental note about asyncio (#5332 )	2021-12-31 13:15:08 +01:00
Andrey Rahmatullin	144d1eb834	Add Deferred-to-Future helpers (#5288 )	2021-10-22 18:46:01 +02:00
Eugenio Lacuesta	d698b5147b	Merge branch 'master' into http2	2020-12-31 11:13:25 -03:00
Adrián Chaves	015c82b974	Scrapy 2.4 release notes (#4808 )	2020-10-11 22:09:45 +02:00
Adrián Chaves	4d6359df2d	Mark HTTP/2 as experimental	2020-09-11 13:51:05 +02:00
sakshamb2113	42383cc267	Add a setting to customize the asyncio event loop (#4414 )	2020-08-19 09:18:14 +02:00
Adrián Chaves	6aa0ba4553	Write release notes for Scrapy 2.0.0 (#4329 )	2020-03-03 09:11:11 +01:00

17 Commits