5.2 KiB
Settings
System Message: ERROR/3 (<stdin>, line 7)
Unknown directive type "module".
.. module:: scrapy.conf :synopsis: Settings manager
The Scrapy settings allows you to customize the behaviour of all Scrapy components, including the core, extensions, pipelines and spiders themselves.
The settings infrastructure provides a global namespace of key-value mappings where the code can pull configuration values from. The settings can be populated through different mechanisms, which are described below.
Read :ref:`settings` for all supported entries.
System Message: ERROR/3 (<stdin>, line 17); backlink
Unknown interpreted text role "ref".How to populate settings
Settings can be populated using different mechanisms, each of which having a different precedence. Here is the list of them in decreasing order of precedence:
- Global overrides (most precedence)
- Environment variables
- scrapy_settings
- Default settings per-command
- Default global settings (less precedence)
This mechanisms are described with more detail below.
1. Global overrides
Global overrides are the ones that takes most precedence, and are usually populated by command line options.
- Example::
>>> from scrapy.conf import settings >>> settings.overrides['LOG_ENABLED'] = True
You can also override one (or more) settings from command line using the --set command line argument.
System Message: ERROR/3 (<stdin>, line 47)
Unknown directive type "highlight".
.. highlight:: sh
Example:
scrapy-ctl.py crawl domain.com --set LOG_FILE=scrapy.log
2. Environment variables
You can populate settings using environment variables prefixed with SCRAPY_. For example, to change the log file location un Unix systems:
$ export SCRAPY_LOG_FILE=scrapy.log $ scrapy-ctl.py crawl example.com
In Windows systems, you can change the environment variables from the Control Panel following these guidelines.
3. scrapy_settings
scrapy_settings is the standard configuration file for your Scrapy project. It's where most of your custom settings will be populated.
4. Default settings per-command
Each scrapy-ctl command can have its own default settings, which override the global default settings. Those custom command settings are located inside the scrapy.conf.commands module, or you can specify custom settings to override per-comand inside your project, by writing them in the module referenced by the :setting:`COMMANDS_SETTINGS_MODULE` setting. Those settings will take more
System Message: ERROR/3 (<stdin>, line 76); backlink
Unknown interpreted text role "setting".5. Default global settings
The global defaults are located in scrapy.conf.default_settings and documented in the :ref:`settings` page.
System Message: ERROR/3 (<stdin>, line 85); backlink
Unknown interpreted text role "ref".How to access settings
System Message: ERROR/3 (<stdin>, line 92)
Unknown directive type "highlight".
.. highlight:: python
Here's an example of the simplest way to access settings from Python code:
>>> from scrapy.conf import settings >>> print settings['LOG_ENABLED'] True
In other words, settings can be accesed like a dict, but it's usually preferred to extract the setting in the format you need it to avoid type errors. In order to do that you'll have to use one of the following methods:
The Settings object is automatically instantiated when the :mod:`scrapy.conf` module is loaded, and it's usually accessed like this:
System Message: ERROR/3 (<stdin>, line 106); backlink
Unknown interpreted text role "mod".>>> from scrapy.conf import settings
System Message: ERROR/3 (<stdin>, line 111)
Unknown directive type "method".
.. method:: Settings.get(name, default=None) Get a setting value without affecting its original type. ``name`` is a string with the setting name ``default`` is the value to return if no setting is found
System Message: ERROR/3 (<stdin>, line 119)
Unknown directive type "method".
.. method:: Settings.getbool(name, deafult=Flse) Get a setting value as a boolean. For example, both ``1`` and ``'1'``, and ``True`` return ``True``, while ``0``, ``'0'``, ``False`` and ``None`` return ``False```` For example, settings populated through environment variables set to ``'0'`` will return ``False`` when using this method. ``name`` is a string with the setting name ``default`` is the value to return if no setting is found
System Message: ERROR/3 (<stdin>, line 132)
Unknown directive type "method".
.. method:: Settings.getint(name, default=0) Get a setting value as an int ``name`` is a string with the setting name ``default`` is the value to return if no setting is found
System Message: ERROR/3 (<stdin>, line 140)
Unknown directive type "method".
.. method:: Settings.getfloat(name, default=0.0) Get a setting value as a float ``name`` is a string with the setting name ``default`` is the value to return if no setting is found
System Message: ERROR/3 (<stdin>, line 148)
Unknown directive type "method".
.. method:: Settings.getlist(name, default=None) Get a setting value as a list. If the setting original type is a list it will be returned verbatim. If it's a string it will be splitted by ",". For example, settings populated through environment variables set to ``'one,two'`` will return a list ['one', 'two'] when using this method. ``name`` is a string with the setting name ``default`` is the value to return if no setting is found
Available built-in settings
See :ref:`settings`.
System Message: ERROR/3 (<stdin>, line 163); backlink
Unknown interpreted text role "ref".Rationale for setting names
Setting names are usually prefixed with the component that they configure. For example, proper setting names for a fictional robots.txt extension would be ROBOTSTXT_ENABLED, ROBOTSTXT_OBEY, ROBOTSTXT_CACHEDIR, etc.