ci(release): Pin metadata tooling to match publishing action
Pin `twine` and `packaging` versions in build job to match bundled versions in `gh-action-pypi-publish` v1.14.2. Enforce Core Metadata 2.4 in wheel and sdist targets with verification in validation scripts.
This commit is contained in:
parent
f89d3b1f20
commit
fd4953d772
|
|
@ -31,6 +31,11 @@ jobs:
|
||||||
name: Build package artifacts
|
name: Build package artifacts
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
|
|
||||||
|
# Match the validator versions bundled by the pinned publishing action.
|
||||||
|
env:
|
||||||
|
EXPECTED_TWINE_VERSION: '7.0.0'
|
||||||
|
EXPECTED_PACKAGING_VERSION: '26.2'
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
- name: Check out repository
|
- name: Check out repository
|
||||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||||
|
|
@ -44,11 +49,39 @@ jobs:
|
||||||
cache: pip
|
cache: pip
|
||||||
|
|
||||||
- name: Install build tooling
|
- name: Install build tooling
|
||||||
run: python -m pip install --upgrade build twine packaging
|
run: >-
|
||||||
|
python -m pip install --upgrade
|
||||||
|
build
|
||||||
|
"twine==$EXPECTED_TWINE_VERSION"
|
||||||
|
"packaging==$EXPECTED_PACKAGING_VERSION"
|
||||||
|
|
||||||
- name: Install documentation toolchain
|
- name: Install documentation toolchain
|
||||||
run: python -m pip install -r requirements.txt
|
run: python -m pip install -r requirements.txt
|
||||||
|
|
||||||
|
- name: Verify pre-publication tool versions
|
||||||
|
# Assert after all installation steps so twine check uses the expected
|
||||||
|
# validator, and reject any incompatible shared dependency constraints.
|
||||||
|
run: |
|
||||||
|
python - <<'PY'
|
||||||
|
import os
|
||||||
|
from importlib.metadata import version
|
||||||
|
|
||||||
|
expected = {
|
||||||
|
'twine': os.environ['EXPECTED_TWINE_VERSION'],
|
||||||
|
'packaging': os.environ['EXPECTED_PACKAGING_VERSION'],
|
||||||
|
}
|
||||||
|
|
||||||
|
for package, expected_version in expected.items():
|
||||||
|
installed_version = version(package)
|
||||||
|
print(f'{package}=={installed_version}')
|
||||||
|
if installed_version != expected_version:
|
||||||
|
raise SystemExit(f'{package}=={installed_version} is installed, expected {expected_version}')
|
||||||
|
|
||||||
|
print(f'build=={version("build")}')
|
||||||
|
PY
|
||||||
|
|
||||||
|
python -m pip check
|
||||||
|
|
||||||
- name: Render the documentation
|
- name: Render the documentation
|
||||||
# -c = clean cache, -s = strict (abort on warnings); verify_wheel_contents.py
|
# -c = clean cache, -s = strict (abort on warnings); verify_wheel_contents.py
|
||||||
# additionally guards against a partial render reaching the wheel.
|
# additionally guards against a partial render reaching the wheel.
|
||||||
|
|
@ -313,7 +346,9 @@ jobs:
|
||||||
path: dist/
|
path: dist/
|
||||||
|
|
||||||
- name: Publish package distributions to Test PyPI
|
- name: Publish package distributions to Test PyPI
|
||||||
uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
|
# Bundles twine 7.0.0 and packaging 26.2 (requirements/runtime.txt).
|
||||||
|
# Keep EXPECTED_TWINE_VERSION and EXPECTED_PACKAGING_VERSION aligned when updating this action.
|
||||||
|
uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
|
||||||
with:
|
with:
|
||||||
repository-url: https://test.pypi.org/legacy/
|
repository-url: https://test.pypi.org/legacy/
|
||||||
print-hash: true
|
print-hash: true
|
||||||
|
|
@ -345,6 +380,8 @@ jobs:
|
||||||
path: dist/
|
path: dist/
|
||||||
|
|
||||||
- name: Publish package distributions to PyPI
|
- name: Publish package distributions to PyPI
|
||||||
uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
|
# Bundles twine 7.0.0 and packaging 26.2 (requirements/runtime.txt).
|
||||||
|
# Keep EXPECTED_TWINE_VERSION and EXPECTED_PACKAGING_VERSION aligned when updating this action.
|
||||||
|
uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
|
||||||
with:
|
with:
|
||||||
print-hash: true
|
print-hash: true
|
||||||
|
|
|
||||||
|
|
@ -55,6 +55,12 @@ Check the built artifacts for valid package metadata and README rendering:
|
||||||
twine check dist/*
|
twine check dist/*
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The wheel and sdist deliberately use Core Metadata 2.4, the lowest version required by NetBox's current project metadata. Both build targets pin this format as `core-metadata-version` in `pyproject.toml`, and CI verifies the emitted `METADATA` and `PKG-INFO` values against those pins (`verify_wheel_metadata.py` and `verify_sdist_contents.py`).
|
||||||
|
|
||||||
|
The release workflow's build job pins `twine` and `packaging` to the versions bundled by the pinned `pypa/gh-action-pypi-publish` revision (its `requirements/runtime.txt`), so the pre-publication check uses the same Core Metadata validator as the publisher. Hatchling remains lower-bounded rather than pinned. The explicit Core Metadata setting prevents changes to its default from changing the artifact format.
|
||||||
|
|
||||||
|
Review these settings together when updating the packaging toolchain. Keep the `twine` and `packaging` pins aligned with the publishing action, but change the Core Metadata version only when NetBox needs a newer format and the complete publishing path supports it.
|
||||||
|
|
||||||
Confirm the wheel's version, dependency metadata, and extras match `netbox/release.yaml`, the pinned `requirements.txt`, and the declared optional-dependency groups:
|
Confirm the wheel's version, dependency metadata, and extras match `netbox/release.yaml`, the pinned `requirements.txt`, and the declared optional-dependency groups:
|
||||||
|
|
||||||
```no-highlight
|
```no-highlight
|
||||||
|
|
|
||||||
|
|
@ -91,6 +91,8 @@ omit = [
|
||||||
path = "scripts/packaging/hatch_metadata.py"
|
path = "scripts/packaging/hatch_metadata.py"
|
||||||
|
|
||||||
[tool.hatch.build.targets.wheel]
|
[tool.hatch.build.targets.wheel]
|
||||||
|
# Pinned artifact format. Change only when NetBox needs a newer format and the publishing path supports it.
|
||||||
|
core-metadata-version = "2.4"
|
||||||
sources = ["netbox"]
|
sources = ["netbox"]
|
||||||
packages = [
|
packages = [
|
||||||
"netbox/account",
|
"netbox/account",
|
||||||
|
|
@ -145,6 +147,8 @@ exclude = [
|
||||||
"netbox/netbox/configuration_testing.py" = "netbox/netbox/configuration_testing.py"
|
"netbox/netbox/configuration_testing.py" = "netbox/netbox/configuration_testing.py"
|
||||||
|
|
||||||
[tool.hatch.build.targets.sdist]
|
[tool.hatch.build.targets.sdist]
|
||||||
|
# Keep the sdist and wheel on the same Core Metadata contract.
|
||||||
|
core-metadata-version = "2.4"
|
||||||
include = [
|
include = [
|
||||||
"/.github/workflows/release.yml",
|
"/.github/workflows/release.yml",
|
||||||
"/CHANGELOG.md",
|
"/CHANGELOG.md",
|
||||||
|
|
|
||||||
|
|
@ -7,11 +7,16 @@ SECRET_KEY and database credentials), any other local configuration*.py variant,
|
||||||
any ldap_config*.py (which holds LDAP bind credentials). The wheel guard alone is not
|
any ldap_config*.py (which holds LDAP bind credentials). The wheel guard alone is not
|
||||||
enough: a wheel rebuilt from the sdist re-applies the wheel excludes, so it can come
|
enough: a wheel rebuilt from the sdist re-applies the wheel excludes, so it can come
|
||||||
out clean even when the sdist itself leaks a file.
|
out clean even when the sdist itself leaks a file.
|
||||||
|
|
||||||
|
The sdist must also declare the Core Metadata version pinned for the sdist target in
|
||||||
|
pyproject.toml, so a backend default change cannot silently alter the artifact format.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import sys
|
import sys
|
||||||
import tarfile
|
import tarfile
|
||||||
from pathlib import PurePosixPath
|
import tomllib
|
||||||
|
from email.parser import Parser
|
||||||
|
from pathlib import Path, PurePosixPath
|
||||||
|
|
||||||
# Allowed members, relative to the sdist's netbox-<version>/ root directory. The sdist
|
# Allowed members, relative to the sdist's netbox-<version>/ root directory. The sdist
|
||||||
# keeps the full repository layout (no `sources` strip), unlike the wheel.
|
# keeps the full repository layout (no `sources` strip), unlike the wheel.
|
||||||
|
|
@ -34,21 +39,46 @@ def configuration_members(sdist_path):
|
||||||
return members
|
return members
|
||||||
|
|
||||||
|
|
||||||
|
def expected_metadata_version():
|
||||||
|
"""Return the core-metadata-version pinned for the sdist target, or None."""
|
||||||
|
pyproject = tomllib.loads((Path(__file__).resolve().parent.parent / 'pyproject.toml').read_text())
|
||||||
|
return pyproject['tool']['hatch']['build']['targets'].get('sdist', {}).get('core-metadata-version')
|
||||||
|
|
||||||
|
|
||||||
|
def read_pkg_info(sdist_path):
|
||||||
|
"""Return the sdist's parsed top-level PKG-INFO, or None when the file is absent."""
|
||||||
|
with tarfile.open(sdist_path) as archive:
|
||||||
|
for member in archive.getmembers():
|
||||||
|
if PurePosixPath(member.name).parts[1:] == ('PKG-INFO',):
|
||||||
|
return Parser().parsestr(archive.extractfile(member).read().decode())
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def main(argv):
|
def main(argv):
|
||||||
if len(argv) != 2:
|
if len(argv) != 2:
|
||||||
print('usage: verify_sdist_contents.py <sdist>')
|
print('usage: verify_sdist_contents.py <sdist>')
|
||||||
return 2
|
return 2
|
||||||
|
errors = []
|
||||||
found = configuration_members(argv[1])
|
found = configuration_members(argv[1])
|
||||||
missing = sorted(ALLOWED - found)
|
if missing := sorted(ALLOWED - found):
|
||||||
unexpected = sorted(found - ALLOWED)
|
errors.append(f'missing templates: {missing}')
|
||||||
if missing or unexpected:
|
if unexpected := sorted(found - ALLOWED):
|
||||||
print('Sdist configuration files are not as expected:')
|
errors.append(f'unexpected (possible secret leak): {unexpected}')
|
||||||
if missing:
|
expected = expected_metadata_version()
|
||||||
print(f' - missing templates: {missing}')
|
if expected is None:
|
||||||
if unexpected:
|
errors.append('pyproject.toml does not pin core-metadata-version for the sdist target')
|
||||||
print(f' - unexpected (possible secret leak): {unexpected}')
|
elif (pkg_info := read_pkg_info(argv[1])) is None:
|
||||||
|
errors.append('sdist is missing its top-level PKG-INFO')
|
||||||
|
elif (actual := pkg_info['Metadata-Version']) is None:
|
||||||
|
errors.append('sdist PKG-INFO does not declare Metadata-Version')
|
||||||
|
elif actual != expected:
|
||||||
|
errors.append(f'metadata version mismatch: sdist PKG-INFO has {actual}, pyproject.toml pins {expected}')
|
||||||
|
if errors:
|
||||||
|
print('Sdist contents are not as expected:')
|
||||||
|
for error in errors:
|
||||||
|
print(f' - {error}')
|
||||||
return 1
|
return 1
|
||||||
print(f'OK: sdist ships only the {len(ALLOWED)} configuration templates')
|
print(f'OK: sdist ships only the {len(ALLOWED)} configuration templates and declares Metadata-Version {expected}')
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -12,11 +12,14 @@ Checks:
|
||||||
duplicates these requirement strings literally; this catches drift, for example a
|
duplicates these requirement strings literally; this catches drift, for example a
|
||||||
plugin pin bumped in one place only. Aggregates must not reference netbox itself,
|
plugin pin bumped in one place only. Aggregates must not reference netbox itself,
|
||||||
which would defeat this guard.
|
which would defeat this guard.
|
||||||
|
5. Metadata-Version equals the core-metadata-version pinned in pyproject.toml, which
|
||||||
|
must be pinned identically for the wheel and sdist targets.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import importlib.util
|
import importlib.util
|
||||||
import re
|
import re
|
||||||
import sys
|
import sys
|
||||||
|
import tomllib
|
||||||
import zipfile
|
import zipfile
|
||||||
from collections import defaultdict
|
from collections import defaultdict
|
||||||
from email.parser import Parser
|
from email.parser import Parser
|
||||||
|
|
@ -87,6 +90,33 @@ def split_requires(metadata):
|
||||||
return core, by_extra
|
return core, by_extra
|
||||||
|
|
||||||
|
|
||||||
|
def read_core_metadata_versions(root):
|
||||||
|
"""Return the per-target core-metadata-version pins from pyproject.toml."""
|
||||||
|
targets = tomllib.loads((root / 'pyproject.toml').read_text())['tool']['hatch']['build']['targets']
|
||||||
|
return {name: targets.get(name, {}).get('core-metadata-version') for name in ('wheel', 'sdist')}
|
||||||
|
|
||||||
|
|
||||||
|
def check_metadata_version(metadata, root):
|
||||||
|
configured = read_core_metadata_versions(root)
|
||||||
|
errors = [
|
||||||
|
f'pyproject.toml does not pin core-metadata-version for the {name} target'
|
||||||
|
for name, value in configured.items() if value is None
|
||||||
|
]
|
||||||
|
if errors:
|
||||||
|
return errors
|
||||||
|
if configured['wheel'] != configured['sdist']:
|
||||||
|
errors.append(
|
||||||
|
'core-metadata-version differs between targets: '
|
||||||
|
f'wheel {configured["wheel"]}, sdist {configured["sdist"]}'
|
||||||
|
)
|
||||||
|
if metadata['Metadata-Version'] != configured['wheel']:
|
||||||
|
errors.append(
|
||||||
|
f'metadata version mismatch: wheel has {metadata["Metadata-Version"]}, '
|
||||||
|
f'pyproject.toml pins {configured["wheel"]}'
|
||||||
|
)
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
def check_version(metadata, root, hatch_metadata):
|
def check_version(metadata, root, hatch_metadata):
|
||||||
release_text = (root / 'netbox' / 'release.yaml').read_text()
|
release_text = (root / 'netbox' / 'release.yaml').read_text()
|
||||||
version = hatch_metadata._read_release_field(release_text, 'version')
|
version = hatch_metadata._read_release_field(release_text, 'version')
|
||||||
|
|
@ -140,6 +170,7 @@ def main(argv):
|
||||||
metadata = read_metadata(argv[1])
|
metadata = read_metadata(argv[1])
|
||||||
core, by_extra = split_requires(metadata)
|
core, by_extra = split_requires(metadata)
|
||||||
errors = [
|
errors = [
|
||||||
|
*check_metadata_version(metadata, root),
|
||||||
*check_version(metadata, root, hatch_metadata),
|
*check_version(metadata, root, hatch_metadata),
|
||||||
*check_core_requires(core, root, hatch_metadata),
|
*check_core_requires(core, root, hatch_metadata),
|
||||||
*check_extras(metadata, by_extra),
|
*check_extras(metadata, by_extra),
|
||||||
|
|
@ -149,7 +180,10 @@ def main(argv):
|
||||||
for error in errors:
|
for error in errors:
|
||||||
print(f' - {error}')
|
print(f' - {error}')
|
||||||
return 1
|
return 1
|
||||||
print(f'OK: wheel {metadata["Version"]} matches release.yaml, requirements.txt, and expected extras')
|
print(
|
||||||
|
f'OK: wheel {metadata["Version"]} matches release.yaml, requirements.txt, expected extras, '
|
||||||
|
'and the core metadata pin'
|
||||||
|
)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue