Alpamys
|
6ddaeb30d1
|
polish(v0.60.0 strict-safetensors): tighten header cap + input guards
Follow-up to PR #198 (issue #189):
- _MAX_SAFETENSORS_HEADER_BYTES: 1 GiB -> 100 MiB. Real safetensors
headers are <10 MiB even for 70B-parameter models; 100 MiB is a
generous defence-in-depth ceiling that still rejects an adversary's
"header_len = 999 MiB" allocation attempt before fh.read() commits.
- is_safetensors_magic: input-shape guards (non-string / empty /
null-byte path return False, never raise). Matches project policy
for detection-style helpers (mirrors v0.30.0 Candidate, v0.41.0
lr_groups, v0.53.3 is_known_vlm_base).
- +2 regression tests in tests/test_v0600_part_c.py:
- test_is_safetensors_magic_rejects_invalid_input (5 bad inputs)
- test_max_safetensors_header_bytes_tightened (guards against
re-widening to 1 GiB in a future patch)
Module docstring updated to reference PR #198 / issue #189.
Verified locally:
- ruff check soup_cli/utils/strict_safetensors.py
tests/test_v0600_part_c.py -> clean
- pytest tests/test_v0600_part_c.py --no-cov -> 21 passed,
1 POSIX-skipped on Windows
Closes v0.60.0 Known Limitation (6) — full magic-byte + JSON-header
shape verification with hardened input surface.
|
2026-05-19 23:56:42 +05:00 |
|
Alpamys
|
f3c40e7753
|
feat(security): v0.60.0 — Supply Chain Security wedge (adapter scan/sign/verify, strict-safetensors, namespace-pin, license-matrix, airgap-bundle)
Six controls that hosted vendors structurally can't provide:
- soup adapters scan: spectral backdoor scanner (rank-1 dominance + energy
concentration + NaN/Inf + Frobenius outlier via robust median+MAD);
pure numpy, reuses v0.57.0 adapter_diff loader
- soup adapters sign / verify: Merkle-root manifest in .soup-signature.json;
recursive file enumeration (catches nested tokenizer/ tamper);
sigstore + ed25519 backends stub-then-live (v0.60.1)
- soup adapters check-safetensors: closed 8-entry unsafe-extension allowlist;
strict exit 3 for CI gating
- NamespacePinStore: TOFU SQLite anti-AI-Jacking; author + created_at
fingerprint compared via datetime.fromisoformat for offset-aware order;
bool opt-in rejected so --allow-namespace-shift cannot be a free-for-all
- License-conflict matrix: 33 SPDX-ish ids in MappingProxyType compat table;
soup adapters merge --license <id> --license-override <reason> gate
- soup airgap-bundle: signed tarball with deterministic dataset labeling
(sorted basename, NOT argv order); TOCTOU lstat+S_ISLNK on parent + output;
atomic os.replace; tarfile.data_filter for future extractall callers
Test count: 9294 -> 9446 (+152 net across 6 new test files).
Review-fix coverage across 5 waves (python / security / code / tdd / smoke):
0 CRITICAL + 12 HIGH + 11 MEDIUM + 6 LOW fixed before commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-19 13:05:14 +05:00 |
Vivaan Dhawan