improve OffsiteMiddleware reference docs

--HG-- extra : rebase_source : 3ed3f23fc1ec63b521ead029c5749898f3ab05d7
2009-07-24 12:59:38 -03:00 · 2009-07-24 12:59:38 -03:00 · 96c3cdbec2
parent d21a22eab5
commit 96c3cdbec2
1 changed files with 6 additions and 0 deletions
--- a/docs/ref/spider-middleware.rst
+++ b/docs/ref/spider-middleware.rst
@ -54,6 +54,12 @@ OffsiteMiddleware

   Filters out Requests for URLs outside the domains covered by the spider.

+   This middleware filters out every request whose host names doesn't match
+   :attr:`~scrapy.spider.BaseSpider.domain_name`, or the spider
+   :attr:`~scrapy.spider.BaseSpider.domain_name` prefixed by "www.".  
+   Spider can add more domains to exclude using 
+   :attr:`~scrapy.spider.BaseSpider.extra_domain_names` attribute.
+
 RequestLimitMiddleware
 ----------------------