Merge pull request #1071 from eliasdorneles/updating-request-meta-special-keys

updating list of Request.meta special keys

docs/topics/downloader-middleware.rst

@@ -603,10 +603,12 @@ HttpProxyMiddleware
 
 .. versionadded:: 0.8
 
+.. reqmeta:: proxy
+
 .. class:: HttpProxyMiddleware
 
    This middleware sets the HTTP proxy to use for requests, by setting the
-   ``proxy`` meta value to :class:`~scrapy.http.Request` objects.
+   ``proxy`` meta value for :class:`~scrapy.http.Request` objects.
 
    Like the Python standard library modules `urllib`_ and `urllib2`_, it obeys
    the following environment variables:
@@ -615,6 +617,9 @@ HttpProxyMiddleware
    * ``https_proxy``
    * ``no_proxy``
 
+   You can also set the meta key ``proxy`` per-request, to a value like
+   ``http://some_proxy_server:port``.
+
 .. _urllib: http://docs.python.org/library/urllib.html
 .. _urllib2: http://docs.python.org/library/urllib2.html
 

docs/topics/request-response.rst

@@ -224,12 +224,15 @@ Those are:
 * :reqmeta:`dont_redirect`
 * :reqmeta:`dont_retry`
 * :reqmeta:`handle_httpstatus_list`
+* :reqmeta:`handle_httpstatus_all`
 * ``dont_merge_cookies`` (see ``cookies`` parameter of :class:`Request` constructor)
 * :reqmeta:`cookiejar`
 * :reqmeta:`redirect_urls`
 * :reqmeta:`bindaddress`
 * :reqmeta:`dont_obey_robotstxt`
 * :reqmeta:`download_timeout`
+* :reqmeta:`download_maxsize`
+* :reqmeta:`proxy`
 
 .. reqmeta:: bindaddress
 

docs/topics/settings.rst

@@ -433,6 +433,8 @@ The maximum response size (in bytes) that downloader will download.
 
 If you want to disable it set to 0.
 
+.. reqmeta:: download_maxsize
+
 .. note::
 
     This size can be set per spider using :attr:`download_maxsize`

docs/topics/spider-middleware.rst

@@ -225,9 +225,12 @@ this::
 
 .. reqmeta:: handle_httpstatus_list
 
+.. reqmeta:: handle_httpstatus_all
+
 The ``handle_httpstatus_list`` key of :attr:`Request.meta
 <scrapy.http.Request.meta>` can also be used to specify which response codes to
-allow on a per-request basis.
+allow on a per-request basis. You can also set the meta key ``handle_httpstatus_all``
+to ``True`` if you want to allow any response code for a request.
 
 Keep in mind, however, that it's usually a bad idea to handle non-200
 responses, unless you really know what you're doing.