mirror of https://github.com/scrapy/scrapy.git
f7382561cf
`get_component_priority_dict_with_base()` normalizes every key of a
component priority dictionary by calling `load_object()` on it, including
keys whose value is `None`, i.e. components the user is disabling.
`normalize_key()` only guarded against `(NameError, TypeError, ValueError)`,
the three exceptions `load_object()` raises itself, but not against the
`ImportError` raised by the `import_module()` call it makes. A stale entry
disabling a component whose module no longer exists, e.g.
`{"scrapy.webservice.WebService": None}`, therefore aborted settings
resolution with `ModuleNotFoundError` instead of being ignored.
Add `ImportError` to the guard so that an unimportable key falls back to its
raw string form. Since its value is `None`, it is then dropped by the
existing `if v is not None` filter, and disabling by import path keeps
working. Enabled components with an unimportable path are unaffected:
`MiddlewareManager.from_crawler()` calls `load_object()` on them again and
still fails there.
Fixes #7820
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|---|---|---|
| .github | ||
| docs | ||
| extras | ||
| scrapy | ||
| sep | ||
| tests | ||
| tests_typing | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .pre-commit-config.yaml | ||
| .readthedocs.yml | ||
| AUTHORS | ||
| CITATION.cff | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| INSTALL.md | ||
| LICENSE | ||
| NEWS | ||
| README.rst | ||
| SECURITY.md | ||
| codecov.yml | ||
| conftest.py | ||
| pyproject.toml | ||
| tox.ini | ||
README.rst
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
</head>
</html>
Scrapy is a web scraping framework to extract structured data from websites. It is cross-platform, and requires Python 3.10+. It is maintained by Zyte (formerly Scrapinghub) and many other contributors.
Install with:
System Message: WARNING/2 (<stdin>, line 44)
Cannot analyze code. Pygments package not found.
.. code:: bash
pip install scrapy
And follow the documentation to learn how to use it.
If you wish to contribute, see Contributing.