Scrapy, a fast high-level web crawling & scraping framework for Python.
Go to file
Adrian 0e324f3d4a
Let spiders change allowed_domains at run time (#7912)
2026-08-09 14:17:44 +05:00
.github
docs Let spiders change allowed_domains at run time (#7912) 2026-08-09 14:17:44 +05:00
extras
scrapy Let spiders change allowed_domains at run time (#7912) 2026-08-09 14:17:44 +05:00
sep
tests Let spiders change allowed_domains at run time (#7912) 2026-08-09 14:17:44 +05:00
tests_typing
.git-blame-ignore-revs
.gitattributes
.gitignore
.pre-commit-config.yaml
.readthedocs.yml
AUTHORS
CITATION.cff
CODE_OF_CONDUCT.md
CONTRIBUTING.md
INSTALL.md
LICENSE
NEWS
README.rst
SECURITY.md
codecov.yml
conftest.py
pyproject.toml
tox.ini

README.rst

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> </head>

Tests

Scrapy is a web scraping framework to extract structured data from websites. It is cross-platform, and requires Python 3.10+. It is maintained by Zyte (formerly Scrapinghub) and many other contributors.

Install with:

System Message: WARNING/2 (<stdin>, line 44)

Cannot analyze code. Pygments package not found.

.. code:: bash

    pip install scrapy

And follow the documentation to learn how to use it.

If you wish to contribute, see Contributing.

</html>