RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

SPM → Zyte API (#6163)

This commit is contained in:
Adrián Chaves 2023-11-30 11:01:22 +01:00 committed by GitHub
parent 731f749556
commit 70ba3a0868
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
docs/topics/practices.rst
View File

@ -288,9 +288,8 @@ Here are some tips to keep in mind when dealing with these kinds of sites:
* use a pool of rotating IPs. For example, the free `Tor project`_ or paid
services like `ProxyMesh`_. An open source alternative is `scrapoxy`_, a
super proxy that you can attach your own proxies to.
* use a highly distributed downloader that circumvents bans internally, so you
can just focus on parsing clean pages. One example of such downloaders is
`Zyte Smart Proxy Manager`_
* use a ban avoidance service, such as `Zyte API`_, which provides a `Scrapy
plugin <https://github.com/scrapy-plugins/scrapy-zyte-api>`__
If you are still unable to prevent your bot getting banned, consider contacting
`commercial support`_.
@ -301,4 +300,4 @@ If you are still unable to prevent your bot getting banned, consider contacting
.. _Common Crawl: https://commoncrawl.org/
.. _testspiders: https://github.com/scrapinghub/testspiders
.. _scrapoxy: https://scrapoxy.io/
.. _Zyte Smart Proxy Manager: https://www.zyte.com/smart-proxy-manager/
.. _Zyte API: https://docs.zyte.com/zyte-api/get-started.html