diff --git a/docs/topics/practices.rst b/docs/topics/practices.rst index 4f036db29..2a8f5b4c1 100644 --- a/docs/topics/practices.rst +++ b/docs/topics/practices.rst @@ -409,6 +409,10 @@ Here are some tips to keep in mind when dealing with these kinds of sites: * use a pool of rotating IPs. For example, the free `Tor project`_ or paid services like `ProxyMesh`_. An open source alternative is `scrapoxy`_, a super proxy that you can attach your own proxies to. +* for HTTPS websites, if blocking appears related to TLS behavior, consider + adjusting the :setting:`DOWNLOADER_CLIENT_TLS_METHOD` setting, since some + websites may respond differently depending on the TLS method used by the + client. * use a ban avoidance service, such as `Zyte API`_, which provides a `Scrapy plugin `__ and additional features, like `AI web scraping `__