From 8ea1ffe41cda2f5549c5aec688a23932ba04d2c9 Mon Sep 17 00:00:00 2001 From: Kelly Su Date: Sat, 8 Aug 2026 15:44:06 -0700 Subject: [PATCH] Document @inthread (#6552). --- docs/topics/coroutines.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/docs/topics/coroutines.rst b/docs/topics/coroutines.rst index b7ddb0a57..f6d238b33 100644 --- a/docs/topics/coroutines.rst +++ b/docs/topics/coroutines.rst @@ -203,6 +203,33 @@ This means you can use many useful Python libraries providing such code: .. note:: If you want to ``await`` on Deferreds while using the asyncio reactor, you need to :ref:`wrap them`. +Sometimes you need to call code that is not async-friendly at all, such as a +third-party client library with no asynchronous support. Calling such code +directly from a coroutine would block the underlying event loop, preventing +Scrapy from doing any other work while it runs. To avoid this, you can wrap +the blocking call with :func:`~scrapy.utils.decorators.inthread`, which runs +it in a separate thread and returns an awaitable with the result: + +.. code-block:: python + + from scrapy.utils.decorators import inthread + + + @inthread + def blocking_call(value): + # e.g. a call into a library with no async support + return some_sync_client.get(value) + + + class MySpider(Spider): + async def parse(self, response): + result = await blocking_call(response.url) + # ... use result to yield items and requests + +:func:`~scrapy.utils.decorators.inthread` uses :func:`asyncio.to_thread` when +asyncio support is enabled, and Twisted's thread pool otherwise, so it works +regardless of which reactor your project uses. + Common use cases for asynchronous code include: * requesting data from websites, databases and other services (in