From 72462a53e2cfcac3ec6068fc94fec657c73e157c Mon Sep 17 00:00:00 2001 From: Andrey Rakhmatullin Date: Wed, 2 Aug 2023 12:32:53 +0400 Subject: [PATCH] Add more docs for update_settings(). --- docs/topics/settings.rst | 20 ++++++++++++++++++-- docs/topics/spiders.rst | 17 +++++++++++------ 2 files changed, 29 insertions(+), 8 deletions(-) diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index 219509c1e..d0f2acd89 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -66,8 +66,8 @@ Example:: ---------------------- Spiders (See the :ref:`topics-spiders` chapter for reference) can define their -own settings that will take precedence and override the project ones. They can -do so by setting their :attr:`~scrapy.Spider.custom_settings` attribute: +own settings that will take precedence and override the project ones. One way +to do so is by setting their :attr:`~scrapy.Spider.custom_settings` attribute: .. code-block:: python @@ -81,6 +81,22 @@ do so by setting their :attr:`~scrapy.Spider.custom_settings` attribute: "SOME_SETTING": "some value", } +It's often better to provide a :meth:`~scrapy.Spider.update_settings` instead, +and settings set there should use the "spider" priority explicitly: + +.. code-block:: python + + import scrapy + + + class MySpider(scrapy.Spider): + name = "myspider" + + @classmethod + def update_settings(cls, settings): + settings.set("SOME_SETTING", "some value", priority="spider") + super().update_settings(settings) + 3. Project settings module -------------------------- diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst index 0f6c2b1ba..97b525bd6 100644 --- a/docs/topics/spiders.rst +++ b/docs/topics/spiders.rst @@ -151,13 +151,18 @@ scrapy.Spider and is called during initialization of a spider instance. It takes a :class:`~scrapy.settings.Settings` object as a parameter and - can add or update the spider's configuration values. This method is a class method, - meaning that it is called on the :class:`~scrapy.Spider` class and allows all instances - of the spider to share the same configuration. + can add or update the spider's configuration values. This method is a + class method, meaning that it is called on the :class:`~scrapy.Spider` + class and allows all instances of the spider to share the same + configuration. - One of the main advantages of ``update_settings()`` is that it allows - you to dynamically add, remove or change settings based on other settings - or other external factors. + While per-spider settings can be set in + :attr:`~scrapy.Spider.custom_settings`, using ``update_settings()`` + allows you to dynamically add, remove or change settings based on other + settings, spider attributes or other factors and use setting priorities + other than ``'spider'``. Also, it's easy to extend ``update_settings()`` + in a subclass by overriding it, while doing the same with + :attr:`~scrapy.Spider.custom_settings` is hard or impossible. For example, suppose a spider needs to modify :setting:`FEEDS`: