RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

improved documentation of http proxy middleware

This commit is contained in:
Pablo Hoffman 2009-10-07 21:00:34 -02:00
parent d03bf1f27f
commit 937acd91d1
3 changed files with 17 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
docs/faq.rst
View File

@ -54,12 +54,9 @@ to steal from us!
Does Scrapy work with HTTP proxies?
-----------------------------------
No. Support for HTTP proxies is not currently implemented in Scrapy, but it
will be in the future. For more information about this, follow `this ticket
<http://dev.scrapy.org/ticket/71>`_. Setting the ``http_proxy`` environment
variable won't work because Twisted (the library used by Scrapy to download
pages) doesn't support it. See `this Twisted ticket
<http://twistedmatrix.com/trac/ticket/2714>`_ for more info.
Yes. Support for HTTP proxies is provided (since Scrapy 0.8) through the HTTP
Proxy downloader middleware. See
:class:`~scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware`.
Scrapy crashes with: ImportError: No module named win32api
----------------------------------------------------------

15
docs/topics/downloader-middleware.rst
View File

@ -253,11 +253,22 @@ HttpProxyMiddleware
.. module:: scrapy.contrib.downloadermiddleware.httpproxy
:synopsis: Http Proxy Middleware
.. versionadded:: 0.8
.. class:: HttpProxyMiddleware
This middleware sets proxy to use for requests, it obeys enviroment
variables 'http_proxy', 'https_proxy', and 'no_proxy'
This middleware sets the HTTP proxy to use for requests, by setting the
``proxy`` meta value to :class:`~scrapy.http.Request` objects.
Like the Python standard library modules `urllib`_ and `urllib2`_ it obeys
the following enviroment variables:
* ``http_proxy``
* ``https_proxy``
* ``no_proxy``
.. _urllib: http://docs.python.org/library/urllib.html
.. _urllib2: http://docs.python.org/library/urllib2.html
RedirectMiddleware
-------------------

1
docs/topics/settings.rst
View File

@ -402,6 +402,7 @@ Default::
'scrapy.contrib.downloadermiddleware.defaultheaders.DefaultHeadersMiddleware': 550,
'scrapy.contrib.downloadermiddleware.redirect.RedirectMiddleware': 600,
'scrapy.contrib.downloadermiddleware.cookies.CookiesMiddleware': 700,
'scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware': 750,
'scrapy.contrib.downloadermiddleware.httpcompression.HttpCompressionMiddleware': 800,
'scrapy.contrib.downloadermiddleware.stats.DownloaderStats': 850,
'scrapy.contrib.downloadermiddleware.httpcache.HttpCacheMiddleware': 900,