Scrapy, a fast high-level web crawling & scraping framework for Python.
Go to file
Adrian 629da8071e
Merge 2efa0b6b43 into ad43bf0c56
2026-08-15 11:16:53 -05:00
.github Make brotli a hard dependency (#7929) 2026-08-09 19:24:14 +02:00
docs Add a HTTP2_MAX_FRAME_SIZE setting (#7988) 2026-08-14 14:19:06 +05:00
extras Deprecate the download_delay spider attribute, and fix the suggested replacement for max_concurrent_requests (#7833) 2026-08-03 23:46:41 +05:00
scrapy Merge 2efa0b6b43 into ad43bf0c56 2026-08-15 11:16:53 -05:00
sep Add sphinx-lint. (#6920) 2025-06-28 01:37:20 +02:00
tests Merge 2efa0b6b43 into ad43bf0c56 2026-08-15 11:16:53 -05:00
tests_typing Next refactoring pass of test_utils_*. (#7797) 2026-07-28 14:24:34 +02:00
.git-blame-ignore-revs Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
.gitattributes
…
.gitignore Add llms.txt and llms-full.txt generation (#7380) 2026-04-06 10:24:21 +02:00
.pre-commit-config.yaml :issue: → :gh: (#7974) 2026-08-10 19:30:48 +02:00
.readthedocs.yml Add Python 3.14 to CI. (#6604) 2026-05-12 23:49:37 +05:00
AUTHORS
…
CITATION.cff Add CITATION.cff (#7519) 2026-05-14 13:07:13 +02:00
CODE_OF_CONDUCT.md
…
CONTRIBUTING.md
…
INSTALL.md
…
LICENSE
…
NEWS
…
README.rst Use a single badge for all tests (#7836) 2026-07-31 12:18:06 +05:00
SECURITY.md Bump version: 2.16.0 → 2.17.0 2026-07-07 12:26:52 +02:00
codecov.yml
…
conftest.py Rerun tests that need real Internet access when they fail (#7985) 2026-08-12 21:46:00 +02:00
pyproject.toml Address warnings from the VCS workflow (#7966) 2026-08-10 13:46:20 +05:00
tox.ini Rerun tests that need real Internet access when they fail (#7985) 2026-08-12 21:46:00 +02:00

README.rst

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> </head>

Tests

Scrapy is a web scraping framework to extract structured data from websites. It is cross-platform, and requires Python 3.10+. It is maintained by Zyte (formerly Scrapinghub) and many other contributors.

Install with:

System Message: WARNING/2 (<stdin>, line 44)

Cannot analyze code. Pygments package not found.

.. code:: bash

    pip install scrapy

And follow the documentation to learn how to use it.

If you wish to contribute, see Contributing.

</html>