RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

more cleanup to scheduler middelware doc, and permanentely moved to experimental doc

This commit is contained in:
Pablo Hoffman 2009-09-11 13:27:31 -03:00
parent 0af052b68f
commit be0f2beef0
2 changed files with 7 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
docs/experimental/scheduler-middleware.rst
View File

@ -4,8 +4,8 @@
Scheduler middleware Scheduler middleware
==================== ====================
The scheduler middleware is a framework of hooks in the Scrapy's scheduling The scheduler middleware is a framework of hooks in the Scrapy scheduling
mechanism where you can plug custom functionality to process request being mechanism where you can plug custom functionality to process requests being
enqueued. enqueued.
Activating a scheduler middleware Activating a scheduler middleware
@ -58,6 +58,9 @@ single Python class that defines one or more of the following methods:
.. method:: enqueue_request(domain, request) .. method:: enqueue_request(domain, request)
Process the given request which is being enqueued. This method can return
None to avoid the request from being scheduled.
:meth:`enqueue_request` should return either ``None``, a :meth:`enqueue_request` should return either ``None``, a
:class:`~scrapy.http.Response` object or a ``Deferred``. :class:`~scrapy.http.Response` object or a ``Deferred``.
@ -67,16 +70,6 @@ single Python class that defines one or more of the following methods:
:param requests: the request to be enqueued :param requests: the request to be enqueued
:type request: :class:`~scrapy.http.Request` object :type request: :class:`~scrapy.http.Request` object
.. method:: open_domain(domain)
:param domain: the domain being opened
:type domain: string
.. method:: close_domain(domain)
:param domain: the domain being closed
:type domain: string
.. _topics-scheduler-middleware-ref: .. _topics-scheduler-middleware-ref:
Built-in scheduler middleware reference Built-in scheduler middleware reference
@ -101,6 +94,6 @@ DuplicatesFilterMiddleware
The :class:`DuplicatesFilterMiddleware` can be configured through the following The :class:`DuplicatesFilterMiddleware` can be configured through the following
settings (see the settings documentation for more info): settings (see the settings documentation for more info):
* :setting:`DUPEFILTER_CLASS` - The class used to detect and filter * :setting:`DUPEFILTER_CLASS` - The class that implements the duplicate
duplicate requests. filtering policy.

3
docs/index.rst
View File

@ -149,9 +149,6 @@ Extending Scrapy
:doc:`topics/spider-middleware` :doc:`topics/spider-middleware`
Customize the input and output of your spiders. Customize the input and output of your spiders.
:doc:`experimental/scheduler-middleware`
Customize how pages are scheduled (warning: experimental doc).
:doc:`topics/extensions` :doc:`topics/extensions`
Add any custom functionality using :doc:`signals <topics/signals>` and the Add any custom functionality using :doc:`signals <topics/signals>` and the
Scrapy API Scrapy API