From 2259db46cb9503e9268a14cee73747687cb64982 Mon Sep 17 00:00:00 2001 From: Paul Tremberth Date: Thu, 6 Feb 2014 12:53:09 -0800 Subject: [PATCH] Copied some of SEP-019 --- GSoC-2014.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/GSoC-2014.md b/GSoC-2014.md index b7c7d4f..f5f7c04 100644 --- a/GSoC-2014.md +++ b/GSoC-2014.md @@ -36,7 +36,27 @@ which should encourage more writing more (and better) extensions **_Pablo to write a new SEP document._** ### Per-spider settings (SEP-19) -https://github.com/scrapy/scrapy/blob/master/sep/sep-019.rst + +This is a proposal to add support for overriding settings per-spiders in a consistent way. + +In short, you will be able to overwrite settings (on a per-spider basis) by implementing a class method in your spider: + + def MySpider(BaseSpider): + + @classmethod + def custom_settings(cls): + return { + "DOWNLOAD_DELAY": 5.0, + "RETRY_ENABLED": False, + } + +**What this solves** + +* support true overridable per-spider setting, from both command-line usage and library mode +* support for accessing settings from spiders (currently not supported without hacky code) +* avoids mistakenly believing you can change settings after they have been populated (you can, but they won't have any effect) + +More details and some implementation proposals can be found in https://github.com/scrapy/scrapy/blob/master/sep/sep-019.rst ### Better generator support Mikhail: