diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index de665f42e..de74ad5cf 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -69,9 +69,10 @@ Example:: precedence and override the project ones. .. note:: :ref:`Pre-crawler settings ` cannot be defined - per spider, and :ref:`reactor settings ` should not have - a different value per spider when :ref:`running multiple spiders in the - same process `. + per spider, and :ref:`reactor settings ` and + :ref:`logging settings ` are subject to restrictions when + :ref:`running multiple spiders in the same process + `. One way to do so is by setting their :attr:`~scrapy.Spider.custom_settings` attribute: @@ -329,32 +330,41 @@ Reactor settings **Reactor settings** are settings tied to the :doc:`Twisted reactor `. -These settings can be defined from a spider. However, because only 1 reactor -can be used per process, these settings cannot use a different value per spider -when :ref:`running multiple spiders in the same process -`. +Because only 1 reactor can be used per process, these settings cannot use a +different value per spider when :ref:`running multiple spiders in the same +process `. -In general, if different spiders define different values, the first defined -value is used. However, if two spiders request a different reactor, an -exception is raised. - -These settings are: +These settings are used upon installing the reactor: - :setting:`ASYNCIO_EVENT_LOOP` (not possible to set per-spider when using :class:`~scrapy.crawler.AsyncCrawlerProcess`, see below) +- :setting:`TWISTED_REACTOR` (ignored when using + :class:`~scrapy.crawler.AsyncCrawlerProcess`, see below) + +They can be :ref:`set from a spider `, but only the values +from the first spider that runs are used, since that is when the reactor is +installed. If a later spider asks for a different reactor or a different event +loop, an exception is raised. With +:class:`~scrapy.crawler.CrawlerRunner` and +:class:`~scrapy.crawler.AsyncCrawlerRunner` the reactor must be installed +beforehand, so these settings are only used to check that the installed reactor +and event loop match them. + +These settings are applied when starting the reactor: + - :setting:`TWISTED_DNS_RESOLVER` and settings used by the corresponding component, e.g. :setting:`DNSCACHE_ENABLED`, :setting:`DNSCACHE_SIZE` and :setting:`DNS_TIMEOUT` for the default one. - :setting:`REACTOR_THREADPOOL_MAXSIZE` -- :setting:`TWISTED_REACTOR` (ignored when using - :class:`~scrapy.crawler.AsyncCrawlerProcess`, see below) - -:setting:`ASYNCIO_EVENT_LOOP` and :setting:`TWISTED_REACTOR` are used upon -installing the reactor. The rest of the settings are applied when starting -the reactor. +They are read from the settings of the +:class:`~scrapy.crawler.CrawlerProcess` or +:class:`~scrapy.crawler.AsyncCrawlerProcess` object, so setting them from a +spider or an :ref:`add-on ` has no effect. They are ignored +altogether when using :class:`~scrapy.crawler.CrawlerRunner` or +:class:`~scrapy.crawler.AsyncCrawlerRunner`, which do not start the reactor. There is an additional restriction for :setting:`TWISTED_REACTOR` and :setting:`ASYNCIO_EVENT_LOOP` when using