diff --git a/docs/intro/tutorial.rst b/docs/intro/tutorial.rst index 64b9dca62..1df7fe366 100644 --- a/docs/intro/tutorial.rst +++ b/docs/intro/tutorial.rst @@ -262,7 +262,7 @@ This is what the shell looks like:: [s] item {} [s] request [s] response <200 http://www.dmoz.org/Computers/Programming/Languages/Python/Books/> - [s] settings + [s] settings [s] spider [s] Useful shortcuts: [s] shelp() Shell help (print this help) diff --git a/docs/topics/api.rst b/docs/topics/api.rst index 4f7eb19d2..341340c2a 100644 --- a/docs/topics/api.rst +++ b/docs/topics/api.rst @@ -103,25 +103,107 @@ how you :ref:`configure the downloader middlewares Start the crawler. This calls :meth:`configure` if it hasn't been called yet. Returns a deferred that is fired when the crawl is finished. +.. _topics-api-settings: + Settings API ============ .. module:: scrapy.settings :synopsis: Settings manager -.. class:: Settings() +.. attribute:: SETTINGS_PRIORITIES - This object that provides access to Scrapy settings. + Dictionary that sets the key name and priority level of the default + settings priorities used in Scrapy. - .. attribute:: overrides + Each item defines a settings entry point, giving it a code name for + identification and an integer priority. Greater priorities take more + precedence over lesser ones when setting and retrieving values in the + :class:`~scrapy.settings.Settings` class. - Global overrides are the ones that take most precedence, and are usually - populated by command-line options. + .. highlight:: python - Overrides should be populated *before* configuring the Crawler object + :: + + SETTINGS_PRIORITIES = { + 'default': 0, + 'command': 10, + 'project': 20, + 'cmdline': 40, + } + + For a detailed explanation on each settings sources, see: + :ref:`topics-settings`. + +.. class:: Settings(values={}, priority='project') + + This object stores Scrapy settings for the configuration of internal + components, and can be used for any further customization. + + After instantiation of this class, the new object will have the global + default settings described on :ref:`topics-settings-ref` already + populated. + + Additional values can be passed on initialization with the ``values`` + argument, and they would take the ``priority`` level. If the latter + argument is a string, the priority name will be looked up in + :attr:`~scrapy.settings.SETTINGS_PRIORITIES`. Otherwise, a expecific + integer should be provided. + + Once the object is created, new settings can be loaded or updated with the + :meth:`~scrapy.settings.Settings.set` method, and can be accessed with the + square bracket notation of dictionaries, or with the + :meth:`~scrapy.settings.Settings.get` method of the instance and its value + conversion variants. When requesting a stored key, the value with the + highest priority will be retrieved. + + .. method:: set(name, value, priority='project') + + Store a key/value attribute with a given priority. + + Settings should be populated *before* configuring the Crawler object (through the :meth:`~scrapy.crawler.Crawler.configure` method), - otherwise they won't have any effect. You don't typically need to worry - about overrides unless you are implementing your own Scrapy command. + otherwise they won't have any effect. + + :param name: the setting name + :type name: string + + :param value: the value to associate with the setting + :type value: any + + :param priority: the priority of the setting. Should be a key of + :attr:`~scrapy.settings.SETTINGS_PRIORITIES` or an integer + :type priority: string or int + + .. method:: setdict(values, priority='project') + + Store key/value pairs with a given priority. + + This is a helper function that calls + :meth:`~scrapy.settings.Settings.set` for every item of ``values`` + with the provided ``priority``. + + :param values: the settings names and values + :type values: dict + + :param priority: the priority of the settings. Should be a key of + :attr:`~scrapy.settings.SETTINGS_PRIORITIES` or an integer + :type priority: string or int + + .. method:: setmodule(module, priority='project') + + Store settings from a module with a given priority. + + This is a helper function that calls + :meth:`~scrapy.settings.Settings.set` for every globally declared + uppercase variable of ``module`` with the provided ``priority``. + + :param module: the module or the path of the module + :type module: module object or string + + :param priority: the priority of the settings. Should be a key of + :attr:`~scrapy.settings.SETTINGS_PRIORITIES` or an integer + :type priority: string or int .. method:: get(name, default=None) diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index 44d9128e9..629fac2dd 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -35,22 +35,23 @@ Settings can be populated using different mechanisms, each of which having a different precedence. Here is the list of them in decreasing order of precedence: - 1. Global overrides (most precedence) + 1. Command line options (most precedence) 2. Project settings module 3. Default settings per-command 4. Default global settings (less precedence) +The population of these settings sources is taken care of internally, but a +manual handling is possible using API calls. See the +:ref:`topics-api-settings` topic for reference. + These mechanisms are described in more detail below. -1. Global overrides -------------------- +1. Command line options +----------------------- -Global overrides are the ones that take most precedence, and are usually -populated by command-line options. You can also override one (or more) settings -from command line using the ``-s`` (or ``--set``) command line option. - -For more information see the :attr:`~scrapy.settings.Settings.overrides` -Settings attribute. +Arguments provided by the command line are the ones that take most precedence, +overriding any other options. You can explicitly override one (or more) +settings using the ``-s`` (or ``--set``) command line option. .. highlight:: sh diff --git a/docs/topics/shell.rst b/docs/topics/shell.rst index 8827f7fb6..f0c14dc76 100644 --- a/docs/topics/shell.rst +++ b/docs/topics/shell.rst @@ -118,7 +118,7 @@ all start with the ``[s]`` prefix):: [s] request [s] response <200 http://scrapy.org> [s] sel \n \n \n\n\n\n\n\n