Commit Graph

7 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 05b3b205ce
Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` (#6817)
* Add a basic Crawler.crawl_async().

* Add custom loop tests for *CrawlerRunner.

* Add AsyncCrawlerProcess.

* Update related docs.

* Update practices.rst.

* Address test failures.

* Add a note about AsyncCrawler* to the docs about switching reactors.

* Address feedback.

* Update for TID253.

* Simplify test_crawler_crawl_async_twice_parallel_unsupported().
2025-05-28 18:55:44 +05:00
Adrián Chaves 036f3e5627
Support asynchronous start requests (#6729) 2025-05-07 19:04:03 +02:00
Andrey Rakhmatullin 7e07d48cc5 Small 3.7 and 3.8 cleanup. 2024-10-17 23:22:37 +05:00
Andrey Rakhmatullin 6428356584 Use twisted.python.log instead of failure.printTraceback(). 2023-09-13 18:20:42 +04:00
Andrey Rakhmatullin be0e33af92 Don't rely on unhandled exception logging in the expected to fail scripts. 2023-09-13 16:59:54 +04:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Adrián Chaves 960a7f68f6
Verify that the installed asyncio event loop matches ASYNCIO_EVENT_LOOP (#5529)
Co-authored-by: Laerte Pereira <5853172+Laerte@users.noreply.github.com>
2022-10-15 11:27:00 +02:00