From ae4a8e39e10818e0c13b9e9c32173cee4a0fa00d Mon Sep 17 00:00:00 2001 From: Olivia Choi Date: Thu, 14 May 2026 04:27:43 -0700 Subject: [PATCH] Document TLS method setting in avoiding bans guide (#7518) * Document TLS method setting in avoiding bans guide Mention DOWNLOADER_CLIENT_TLS_METHOD in the avoiding getting banned section. * Update practices.rst --------- Co-authored-by: Andrey Rakhmatullin --- docs/topics/practices.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/topics/practices.rst b/docs/topics/practices.rst index 4f036db29..2a8f5b4c1 100644 --- a/docs/topics/practices.rst +++ b/docs/topics/practices.rst @@ -409,6 +409,10 @@ Here are some tips to keep in mind when dealing with these kinds of sites: * use a pool of rotating IPs. For example, the free `Tor project`_ or paid services like `ProxyMesh`_. An open source alternative is `scrapoxy`_, a super proxy that you can attach your own proxies to. +* for HTTPS websites, if blocking appears related to TLS behavior, consider + adjusting the :setting:`DOWNLOADER_CLIENT_TLS_METHOD` setting, since some + websites may respond differently depending on the TLS method used by the + client. * use a ban avoidance service, such as `Zyte API`_, which provides a `Scrapy plugin `__ and additional features, like `AI web scraping `__