Scrapy, a fast high-level web crawling & scraping framework for Python.
Go to file
aliowka b905a6b2a3 Revert PyPy envs back to plain pytest
PyPy environments explicitly disable coverage to avoid the significant
runtime overhead it adds. Without coverage data, delta has no accurate
mapping for these envs and falls back to the CPython mapping, causing:
- Unreliable test selection (CPython-mapped tests may fail on PyPy)
- Pre-existing PyPy/min-deps failures surfacing unexpectedly (exit code 1)

Affected envs: pypy3, pypy3-extra-deps, min-pypy3

This follows the same principle as the botocore/mitmproxy fix:
delta is only used where it has accurate coverage mapping.
2026-06-17 10:40:03 +03:00
.github Integrate Delta into all tox environments and clean up workflow 2026-06-17 10:29:07 +03:00
docs Implement HTTP Auth settings and request metadata keys (#7590) 2026-06-11 15:56:53 +05:00
extras chore: simplify some code, get rid of nested fns where it's makes sence + pylint (#7401) 2026-05-04 23:32:46 +05:00
scrapy Link.__eq__: return NotImplemented instead of raising NotImplementedError (#7611) 2026-06-12 14:56:15 +02:00
sep Add sphinx-lint. (#6920) 2025-06-28 01:37:20 +02:00
tests Complete coverage for addons.py (#7612) 2026-06-12 16:15:36 +02:00
tests_typing Enable mypy warn_return_any. (#7492) 2026-05-04 22:24:47 +02:00
.git-blame-ignore-revs Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
.gitattributes Maybe the problem is not in the code after all 2020-08-13 06:35:09 +02:00
.gitignore Add llms.txt and llms-full.txt generation (#7380) 2026-04-06 10:24:21 +02:00
.pre-commit-config.yaml sphinx-scrapy: 0.8.5 → 0.8.6 (#7507) 2026-05-12 12:49:23 +05:00
.readthedocs.yml Add Python 3.14 to CI. (#6604) 2026-05-12 23:49:37 +05:00
AUTHORS Scrapinghub → Zyte 2021-02-02 15:03:20 +01:00
CITATION.cff Add CITATION.cff (#7519) 2026-05-14 13:07:13 +02:00
CODE_OF_CONDUCT.md Update Code of Conduct to Contributor Covenant v2.1 2022-10-28 02:13:37 +02:00
CONTRIBUTING.md Be consistent with domain used for links to documentation website 2019-01-31 01:28:53 -03:00
INSTALL.md Update and rename INSTALL to INSTALL.md 2022-10-06 19:58:48 +02:00
LICENSE added oxford commas to LICENSE 2018-06-01 21:48:43 -03:00
NEWS added NEWS file pointing to docs/news.rst 2012-04-28 23:32:51 -03:00
README.rst Remove Python 3.9 support (#7121) 2025-10-27 12:37:49 +01:00
SECURITY.md Bump version: 2.15.2 → 2.16.0 2026-05-19 17:27:16 +05:00
codecov.yml codecov config: disable project check, tweak PR comments 2017-05-19 00:01:27 +05:00
conftest.py Assorted test fixes (#7585) 2026-06-10 11:01:06 +02:00
pyproject.toml Work around coverage slowness on Python 3.14. (#7574) 2026-06-05 15:34:02 +02:00
tox.ini Revert PyPy envs back to plain pytest 2026-06-17 10:40:03 +03:00

README.rst

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> </head>

Scrapy is a web scraping framework to extract structured data from websites. It is cross-platform, and requires Python 3.10+. It is maintained by Zyte (formerly Scrapinghub) and many other contributors.

Install with:

System Message: WARNING/2 (<stdin>, line 52)

Cannot analyze code. Pygments package not found.

.. code:: bash

    pip install scrapy

And follow the documentation to learn how to use it.

If you wish to contribute, see Contributing.

</html>