From be0f2beef034d2e0a5bd05ff7be9f4ab1e2cf03e Mon Sep 17 00:00:00 2001 From: Pablo Hoffman Date: Fri, 11 Sep 2009 13:27:31 -0300 Subject: [PATCH] more cleanup to scheduler middelware doc, and permanentely moved to experimental doc --- docs/experimental/scheduler-middleware.rst | 21 +++++++-------------- docs/index.rst | 3 --- 2 files changed, 7 insertions(+), 17 deletions(-) diff --git a/docs/experimental/scheduler-middleware.rst b/docs/experimental/scheduler-middleware.rst index 7dae54dc3..2956e84dd 100644 --- a/docs/experimental/scheduler-middleware.rst +++ b/docs/experimental/scheduler-middleware.rst @@ -4,8 +4,8 @@ Scheduler middleware ==================== -The scheduler middleware is a framework of hooks in the Scrapy's scheduling -mechanism where you can plug custom functionality to process request being +The scheduler middleware is a framework of hooks in the Scrapy scheduling +mechanism where you can plug custom functionality to process requests being enqueued. Activating a scheduler middleware @@ -58,6 +58,9 @@ single Python class that defines one or more of the following methods: .. method:: enqueue_request(domain, request) + Process the given request which is being enqueued. This method can return + None to avoid the request from being scheduled. + :meth:`enqueue_request` should return either ``None``, a :class:`~scrapy.http.Response` object or a ``Deferred``. @@ -67,16 +70,6 @@ single Python class that defines one or more of the following methods: :param requests: the request to be enqueued :type request: :class:`~scrapy.http.Request` object - .. method:: open_domain(domain) - - :param domain: the domain being opened - :type domain: string - - .. method:: close_domain(domain) - - :param domain: the domain being closed - :type domain: string - .. _topics-scheduler-middleware-ref: Built-in scheduler middleware reference @@ -101,6 +94,6 @@ DuplicatesFilterMiddleware The :class:`DuplicatesFilterMiddleware` can be configured through the following settings (see the settings documentation for more info): - * :setting:`DUPEFILTER_CLASS` - The class used to detect and filter - duplicate requests. + * :setting:`DUPEFILTER_CLASS` - The class that implements the duplicate + filtering policy. diff --git a/docs/index.rst b/docs/index.rst index ac062d7ce..628438480 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -149,9 +149,6 @@ Extending Scrapy :doc:`topics/spider-middleware` Customize the input and output of your spiders. -:doc:`experimental/scheduler-middleware` - Customize how pages are scheduled (warning: experimental doc). - :doc:`topics/extensions` Add any custom functionality using :doc:`signals ` and the Scrapy API