diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index 602ab587d..e1936eb5b 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -67,8 +67,8 @@ Example:: ---------------------- Spiders (See the :ref:`topics-spiders` chapter for reference) can define their -own settings that will take precedence and override the project ones. They can -do so by setting their :attr:`~scrapy.Spider.custom_settings` attribute: +own settings that will take precedence and override the project ones. One way +to do so is by setting their :attr:`~scrapy.Spider.custom_settings` attribute: .. code-block:: python @@ -82,6 +82,22 @@ do so by setting their :attr:`~scrapy.Spider.custom_settings` attribute: "SOME_SETTING": "some value", } +It's often better to implement :meth:`~scrapy.Spider.update_settings` instead, +and settings set there should use the "spider" priority explicitly: + +.. code-block:: python + + import scrapy + + + class MySpider(scrapy.Spider): + name = "myspider" + + @classmethod + def update_settings(cls, settings): + super().update_settings(settings) + settings.set("SOME_SETTING", "some value", priority="spider") + 3. Project settings module -------------------------- diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst index 788bd7678..5c3bf6e72 100644 --- a/docs/topics/spiders.rst +++ b/docs/topics/spiders.rst @@ -145,6 +145,46 @@ scrapy.Spider :param kwargs: keyword arguments passed to the :meth:`__init__` method :type kwargs: dict + .. classmethod:: update_settings(settings) + + The ``update_settings()`` method is used to modify the spider's settings + and is called during initialization of a spider instance. + + It takes a :class:`~scrapy.settings.Settings` object as a parameter and + can add or update the spider's configuration values. This method is a + class method, meaning that it is called on the :class:`~scrapy.Spider` + class and allows all instances of the spider to share the same + configuration. + + While per-spider settings can be set in + :attr:`~scrapy.Spider.custom_settings`, using ``update_settings()`` + allows you to dynamically add, remove or change settings based on other + settings, spider attributes or other factors and use setting priorities + other than ``'spider'``. Also, it's easy to extend ``update_settings()`` + in a subclass by overriding it, while doing the same with + :attr:`~scrapy.Spider.custom_settings` can be hard. + + For example, suppose a spider needs to modify :setting:`FEEDS`: + + .. code-block:: python + + import scrapy + + + class MySpider(scrapy.Spider): + name = "myspider" + custom_feed = { + "/home/user/documents/items.json": { + "format": "json", + "indent": 4, + } + } + + @classmethod + def update_settings(cls, settings): + super().update_settings(settings) + settings.setdefault("FEEDS", {}).update(cls.custom_feed) + .. method:: start_requests() This method must return an iterable with the first Requests to crawl for