RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Document TLS method setting in avoiding bans guide (#7518) 

* Document TLS method setting in avoiding bans guide

Mention DOWNLOADER_CLIENT_TLS_METHOD in the avoiding getting banned section.

* Update practices.rst

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
This commit is contained in:
Olivia Choi 2026-05-14 04:27:43 -07:00 committed by GitHub
parent 4cfe7a08cd
commit ae4a8e39e1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/topics/practices.rst
View File

@ -409,6 +409,10 @@ Here are some tips to keep in mind when dealing with these kinds of sites:
* use a pool of rotating IPs. For example, the free `Tor project`_ or paid
services like `ProxyMesh`_. An open source alternative is `scrapoxy`_, a
super proxy that you can attach your own proxies to.
* for HTTPS websites, if blocking appears related to TLS behavior, consider
adjusting the :setting:`DOWNLOADER_CLIENT_TLS_METHOD` setting, since some
websites may respond differently depending on the TLS method used by the
client.
* use a ban avoidance service, such as `Zyte API`_, which provides a `Scrapy
plugin <https://github.com/scrapy-plugins/scrapy-zyte-api>`__ and additional
features, like `AI web scraping <https://www.zyte.com/ai-web-scraping/>`__