diff --git a/docs/faq.rst b/docs/faq.rst index c2cb0a986..e7da14364 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -54,12 +54,9 @@ to steal from us! Does Scrapy work with HTTP proxies? ----------------------------------- -No. Support for HTTP proxies is not currently implemented in Scrapy, but it -will be in the future. For more information about this, follow `this ticket -`_. Setting the ``http_proxy`` environment -variable won't work because Twisted (the library used by Scrapy to download -pages) doesn't support it. See `this Twisted ticket -`_ for more info. +Yes. Support for HTTP proxies is provided (since Scrapy 0.8) through the HTTP +Proxy downloader middleware. See +:class:`~scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware`. Scrapy crashes with: ImportError: No module named win32api ---------------------------------------------------------- diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index 64693f25b..c4cfde67d 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -253,11 +253,22 @@ HttpProxyMiddleware .. module:: scrapy.contrib.downloadermiddleware.httpproxy :synopsis: Http Proxy Middleware +.. versionadded:: 0.8 + .. class:: HttpProxyMiddleware - This middleware sets proxy to use for requests, it obeys enviroment - variables 'http_proxy', 'https_proxy', and 'no_proxy' + This middleware sets the HTTP proxy to use for requests, by setting the + ``proxy`` meta value to :class:`~scrapy.http.Request` objects. + Like the Python standard library modules `urllib`_ and `urllib2`_ it obeys + the following enviroment variables: + + * ``http_proxy`` + * ``https_proxy`` + * ``no_proxy`` + +.. _urllib: http://docs.python.org/library/urllib.html +.. _urllib2: http://docs.python.org/library/urllib2.html RedirectMiddleware ------------------- diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index 5cd6ff5b2..dc812ce49 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -402,6 +402,7 @@ Default:: 'scrapy.contrib.downloadermiddleware.defaultheaders.DefaultHeadersMiddleware': 550, 'scrapy.contrib.downloadermiddleware.redirect.RedirectMiddleware': 600, 'scrapy.contrib.downloadermiddleware.cookies.CookiesMiddleware': 700, + 'scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware': 750, 'scrapy.contrib.downloadermiddleware.httpcompression.HttpCompressionMiddleware': 800, 'scrapy.contrib.downloadermiddleware.stats.DownloaderStats': 850, 'scrapy.contrib.downloadermiddleware.httpcache.HttpCacheMiddleware': 900,