From 16e62e9c9bffc418af0b9534d5f05879281dcf9c Mon Sep 17 00:00:00 2001 From: Julia Medina Date: Wed, 13 Aug 2014 01:42:34 -0300 Subject: [PATCH] Per-spider settings documentation --- docs/topics/api.rst | 1 + docs/topics/settings.rst | 20 ++++++++++++++------ docs/topics/spiders.rst | 9 +++++++++ 3 files changed, 24 insertions(+), 6 deletions(-) diff --git a/docs/topics/api.rst b/docs/topics/api.rst index 03a0b4124..57b8ee0cf 100644 --- a/docs/topics/api.rst +++ b/docs/topics/api.rst @@ -172,6 +172,7 @@ Settings API 'default': 0, 'command': 10, 'project': 20, + 'spider': 30, 'cmdline': 40, } diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index e0c432cb3..89ee7605a 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -36,9 +36,10 @@ different precedence. Here is the list of them in decreasing order of precedence: 1. Command line options (most precedence) - 2. Project settings module - 3. Default settings per-command - 4. Default global settings (less precedence) + 2. Settings per-spider + 3. Project settings module + 4. Default settings per-command + 5. Default global settings (less precedence) The population of these settings sources is taken care of internally, but a manual handling is possible using API calls. See the @@ -59,14 +60,21 @@ Example:: scrapy crawl myspider -s LOG_FILE=scrapy.log -2. Project settings module +2. Settings per-spider +---------------------- + +Spiders (See the :ref:`topics-spiders` chapter for reference) can define their +own settings that will take precedence and override the project ones. They can +do so by setting their :attr:`scrapy.spider.Spider.custom_settings` attribute. + +3. Project settings module -------------------------- The project settings module is the standard configuration file for your Scrapy project. It's where most of your custom settings will be populated. For example:: ``myproject.settings``. -3. Default settings per-command +4. Default settings per-command ------------------------------- Each :doc:`Scrapy tool ` command can have its own default @@ -74,7 +82,7 @@ settings, which override the global default settings. Those custom command settings are specified in the ``default_settings`` attribute of the command class. -4. Default global settings +5. Default global settings -------------------------- The global defaults are located in the ``scrapy.settings.default_settings`` diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst index de8f988c0..188b51836 100644 --- a/docs/topics/spiders.rst +++ b/docs/topics/spiders.rst @@ -133,6 +133,15 @@ Spider listed here. The subsequent URLs will be generated successively from data contained in the start URLs. + .. attribute:: custom_settings + + A dictionary of settings that will be overridden from the project wide + configuration when running this spider. It must be defined as a class + attribute since the settings are updated before instantiation. + + For a list of available built-in settings see: + :ref:`topics-settings-ref`. + .. attribute:: crawler This attribute is set by the :meth:`from_crawler` class method after