Add references to security.rst

docs/topics/download-handlers.rst

@@ -39,6 +39,10 @@ for additional schemes and to replace or disable default ones:
         "sftp": "my.download_handlers.SftpHandler",
     }
 
+.. seealso:: :ref:`security-unencrypted-protocols` and
+    :ref:`security-local-resources`, for the security implications of the
+    default ``http``, ``ftp``, ``file`` and ``data`` handlers.
+
 Replacing HTTP(S) download handlers
 -----------------------------------
 

docs/topics/downloader-middleware.rst

@@ -379,6 +379,8 @@ that this risks leaking credentials to unrelated domains.
 This setting must be explicitly configured whenever :setting:`HTTPAUTH_USER`
 or :setting:`HTTPAUTH_PASS` is set.
 
+.. seealso:: :ref:`security-credential-leakage`
+
 .. _Basic access authentication: https://en.wikipedia.org/wiki/Basic_access_authentication
 
 

docs/topics/security.rst

@@ -22,6 +22,8 @@ how to harden them along with the trade-offs involved.
     sources, whether the machine running Scrapy is exposed to a network you do
     not control, whether the data you handle is sensitive, and so on.
 
+.. _security-untrusted-responses:
+
 Treat responses as untrusted input
 ==================================
 
@@ -37,6 +39,8 @@ derived from the response itself.
 TLS connections
 ===============
 
+.. _security-certificate-verification:
+
 Certificate verification
 ------------------------
 
@@ -62,6 +66,8 @@ man-in-the-middle attacks), set:
 * **Con:** you can no longer scrape sites with misconfigured certificates
   without re-disabling verification for them.
 
+.. _security-tls-protocols-ciphers:
+
 Protocol versions and ciphers
 -----------------------------
 
@@ -85,6 +91,8 @@ ciphers:
 * **Con:** you can no longer connect to servers that only support the excluded
   ciphers.
 
+.. _security-unencrypted-protocols:
+
 Unencrypted protocols
 =====================
 
@@ -113,6 +121,8 @@ Note that disabling the ``http`` handler also prevents plain-HTTP requests that
 result from following an ``http://`` redirect or link, which is often the point
 of disabling it.
 
+.. _security-local-resources:
+
 Local and non-network resources
 ===============================
 
@@ -142,6 +152,8 @@ More generally, if you crawl URLs from untrusted sources, consider validating
 their schemes (and, where applicable, their hosts) before scheduling requests,
 to avoid server-side request forgery (SSRF) and similar issues.
 
+.. _security-telnet:
+
 Telnet console
 ==============
 
@@ -175,6 +187,8 @@ If you do not use the telnet console, disable it entirely:
 * **Con:** you can no longer :ref:`inspect and control a running crawler
   <topics-telnetconsole>` through it.
 
+.. _security-credential-leakage:
+
 Credential leakage across domains
 =================================
 

docs/topics/settings.rst

@@ -774,6 +774,8 @@ implementation.
     handler <topics-download-handlers>`, so it's not guaranteed to be supported
     by all 3rd-party handlers.
 
+.. seealso:: :ref:`security-tls-protocols-ciphers`
+
 .. setting:: DOWNLOAD_TLS_MAX_VERSION
 
 DOWNLOAD_TLS_MAX_VERSION
@@ -807,6 +809,8 @@ modern environments.
     by all 3rd-party handlers. Additionally, the set of supported TLS versions
     depends on the TLS implementation being used by the handler.
 
+.. seealso:: :ref:`security-tls-protocols-ciphers`
+
 .. setting:: DOWNLOAD_TLS_MIN_VERSION
 
 DOWNLOAD_TLS_MIN_VERSION
@@ -819,6 +823,8 @@ be used by Scrapy.
 
 See :setting:`DOWNLOAD_TLS_MAX_VERSION` for the details and limitations.
 
+.. seealso:: :ref:`security-tls-protocols-ciphers`
+
 .. setting:: DOWNLOADER_CLIENT_TLS_VERBOSE_LOGGING
 
 DOWNLOADER_CLIENT_TLS_VERBOSE_LOGGING
@@ -979,6 +985,9 @@ enabled in your project.
 
 See :setting:`DOWNLOAD_HANDLERS_BASE` for example format.
 
+.. seealso:: :ref:`security-unencrypted-protocols` and
+    :ref:`security-local-resources`
+
 .. setting:: DOWNLOAD_HANDLERS_BASE
 
 DOWNLOAD_HANDLERS_BASE
@@ -1026,6 +1035,9 @@ handler (without replacement), place this in your ``settings.py``:
         "ftp": None,
     }
 
+.. seealso:: :ref:`security-unencrypted-protocols` and
+    :ref:`security-local-resources`
+
 
 .. setting:: DOWNLOAD_SLOTS
 
@@ -1184,6 +1196,8 @@ when making a request and abort the request if the verification fails.
     certificate problems are logged when this setting is set to ``False``)
     depends on its implementation.
 
+.. seealso:: :ref:`security-certificate-verification`
+
 .. setting:: DUPEFILTER_CLASS
 
 DUPEFILTER_CLASS
@@ -2074,6 +2088,8 @@ Default: ``True`` (``False`` when :setting:`TWISTED_REACTOR_ENABLED` is ``False`
 A boolean which specifies if the :ref:`telnet console <topics-telnetconsole>`
 will be enabled (provided its extension is also enabled).
 
+.. seealso:: :ref:`security-telnet`
+
 .. setting:: TEMPLATES_DIR
 
 TEMPLATES_DIR

docs/topics/spider-middleware.rst

@@ -355,6 +355,8 @@ Default: ``'scrapy.spidermiddlewares.referer.DefaultReferrerPolicy'``
     using the special ``"referrer_policy"`` :ref:`Request.meta <topics-request-meta>` key,
     with the same acceptable values as for the ``REFERRER_POLICY`` setting.
 
+.. seealso:: :ref:`security-credential-leakage`
+
 Acceptable values for REFERRER_POLICY
 *************************************
 

docs/topics/telnetconsole.rst

@@ -29,6 +29,8 @@ disable it if you want. For more information about the extension itself see
 .. note::
     This feature is not supported when :setting:`TWISTED_REACTOR_ENABLED` is ``False``.
 
+.. seealso:: :ref:`security-telnet`
+
 .. highlight:: none
 
 How to access the telnet console
@@ -190,6 +192,8 @@ Default: ``'127.0.0.1'``
 
 The interface the telnet console should listen on
 
+.. seealso:: :ref:`security-telnet`
+
 
 .. setting:: TELNETCONSOLE_USERNAME