Merge pull request #634 from aspidites/sep-011.rst

sep 11 for #629

sep/sep-011.rst

@@ -0,0 +1,36 @@
+=======  ================================
+SEP      11
+Title    Process models for Scrapy
+Created  2009-11-16
+Status   Partially implemented - see #168
+=======  ================================
+
+==================================
+SEP-011: Process models for Scrapy
+==================================
+
+There is an interest of supporting different process models for Scrapy, mainly
+to help prevent memory leaks which affect running all spiders in the same
+process.
+
+By running each spider on a separate process (or pool of processes) we'll be
+able to "recycle" process when they exceed a maximum amount of memory.
+
+Supported process models
+========================
+
+The user could choose between different process models:
+
+1. in process (only method supported so far)
+2. pooled processes (a predefined pool of N processes, which could run more than one spider each)
+3. separate processes (one process per spider)
+
+Using different processes would increase reliability at the cost of performance.
+
+Another ideas to consider
+=========================
+
+- configuring pipeline process models - so that we can have a process exclusive
+  for running pipelines
+- support writing spidersr in different languages when we don't use an in
+  process model

sep/sep-011.trac

@@ -1,30 +0,0 @@
-= SEP-011: Process models for Scrapy =
-
-[[PageOutline(2-5,Contents)]]
-
-||'''SEP:'''||11||
-||'''Title:'''||Process models for Scrapy||
-||'''Author:'''||Pablo Hoffman||
-||'''Created:'''||2009-11-16||
-||'''Status'''||Partially implemented - see #168 ||
-
-== Introduction ==
-
-There is an interest of supporting different process models for Scrapy, mainly to help prevent memory leaks which affect running all spiders in the same process.
-
-By running each spider on a separate process (or pool of processes) we'll be able to "recycle" process when they exceed a maximum amount of memory.
-
-== Supported process models ==
-
-The user could choose between different process models:
-
- 1. in process (only method supported so far)
- 2. pooled processes (a predefined pool of N processes, which could run more than one spider each)
- 3. separate processes (one process per spider)
-
-Using different processes would increase reliability at the cost of performance.
-
-== Another ideas to consider ==
-
- * configuring pipeline process models - so that we can have a process exclusive for running pipelines
- * support writing spidersr in different languages when we don't use an in process model