a015ccc812
feat(v0.66.0): Post-train X-rays — SAE diff + live blame + sleeper probe + interference matrix + probe pack
...
Extends `soup diagnose` from 6 failure modes to 10. Closes v0.57 #171 —
live blame runner replaces the NotImplementedError stub.
- `soup probe sae-diff`: SAE feature attribution (pure-numpy; HF_HUB_ALLOWLIST)
- `soup adapters blame --top-k 50`: live DataInf influence runner
(closes #171 ; replaces v0.57 stub with cos(grad_row, grad_probe) × |grad_row|)
- `soup probe sleeper`: calibrated defection probe (6 bundled bases;
OK/MINOR/MAJOR at 1%/5%; exit 2 on MAJOR)
- `soup probe interference`: pairwise N×N matrix
(OK/MINOR/MAJOR at 5%/20%; exit 2 on MAJOR worst-pair)
- `soup probe pack`: per-base probe manifest assembler
Review-fix coverage across 3 sequential waves: 0 CRITICAL + 9 HIGH +
14 MEDIUM + 5 LOW. Notable hardening:
- TOCTOU O_NOFOLLOW probe-open in load_sae_weights + _count_dataset_rows
- hashlib.sha256 replaces process-salted hash() for CI reproducibility
- Rich-markup escape on adapter / verdict / description / layer
- TypeError on bool/non-str verdict before membership check
- Non-numeric loss rejection in `probe interference` CLI
- 10M-row hard reject (no silent truncate); 100k synthetic-probe cap
- _LOWER_INDEX MappingProxyType for O(1) case-insensitive lookup
- Mapping from collections.abc (PEP 585); frozenset[str] type params
- Frozen dataclasses + FrozenInstanceError regression tests
Note: Windows cp1251 print on stdout-capturing Python wrappers can crash
on Rich's '→' arrow output; the soup CLI itself uses force_utf8_stdio.
Test count: 10577 → 10836 (+259 net across test_v0660_part_{a-e}.py,
test_v0660_cli.py, test_v0660_followups.py). Full suite green
(10836 passed, 81 skipped); ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 14:08:12 +05:00
8577bc2800
test(adapters): strip ANSI before help-output substring asserts (v0.57.0 CI fix)
...
Rich-wrap-CI workaround — same fix pattern as v0.55.0 / v0.56.0:
CliRunner output contains ANSI color escapes that break literal
'--top-k' in output substring matches because Rich renders option
names as -\x1b[0m\x1b[1;36m-top-k.
Adds _ANSI_RE + _strip_ansi() helper to each of the 4 test files
(test_v0570_part_{a,b,c,d}.py) and routes every help-output
substring assertion through it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 18:50:52 +05:00
7da82d355a
feat(adapters): v0.57.0 — git for LoRA (diff, merge, blame, branch)
...
Ships "soup adapters {diff,merge,blame,branch,checkout,branches}" — git-shaped
UX for LoRA adapter management. Pure-numpy math (no torch); TOCTOU defence on
every path; atomic writes throughout.
Part A — adapters diff (utils/adapter_diff.py, ~270 LOC):
Per-layer ΔW Frobenius norm + relative drift; effective-rank delta via
SVD entropy; top-K changed projections; JSON/Markdown/table output.
Part B — adapters merge (utils/adapter_merge.py, ~280 LOC):
Four strategies — linear (weighted avg), ties (Yadav et al. — trim/
elect-sign/disjoint), dare (Yu et al. — drop+rescale, deterministic),
svd (low-rank reconstruction). MergeReport.verdict='UNKNOWN' is a stub;
live canary verdict via v0.55 eval gate ships in v0.57.1.
Part C — adapters blame (utils/blame.py, ~190 LOC):
Leave-one-out plan emitter + budget tracker (parse_budget mirrors v0.48.0
data_mix idiom). Per-shard work table + feasibility check. Live ablation
runner raises NotImplementedError v0.57.1 (mirrors v0.27.0 / v0.50.0 /
v0.56.0 stub-then-live pattern).
Part D — adapters branch / checkout / branches (utils/adapter_branch.py,
~230 LOC):
SHA-256 snapshot pointers under ~/.soup/branches/ (SOUP_BRANCHES_DIR
override, $HOME/$CWD/$TMPDIR-bounded). Drift detection on checkout —
refuses restore when source SHA != snapshot SHA. CRLF/null-byte
rejection on env override (mirrors v0.51.0 hub-endpoint policy).
5-agent review-fix wave (1 CRITICAL + 9 HIGH + 11 MEDIUM + 4 LOW):
- TIES tied-sign defaults to +1 (np.sign(0)==0 would silently zero all
tied parameters)
- load_branch / delete_branch reject symlinks via os.lstat + S_ISLNK
before read/unlink
- merge output safetensors + adapter_config.json atomic writes with
symlink target rejection at output path; source config size-capped
at 256 KB
- compute_adapter_diff weights-file path symlink-rejected via lstat
BEFORE is_file() (defends against .safetensors -> /etc/passwd escape)
- _count_dataset_rows opens via realpath captured at containment check
(closes TOCTOU window)
- diff --output write is atomic (tempfile + os.replace)
- SOUP_BRANCHES_DIR rejects every C0 control char, not just null
- SUPPORTED_STRATEGIES is now frozenset (matches v0.41.0+ allowlist
policy); STRATEGY_ORDER tuple preserved for canonical iteration
- 5× pytest.raises(Exception) tightened to FrozenInstanceError
- 2 zero-assertion Part D tests converted to real assertions
- Added: bool base_model rejection, top_k boundary 1/201, density=1.0
inclusive bound, inf weight rejection, tied-sign positive default,
bool False for num_shards/budget_seconds, POSIX symlink rejections
for diff weights / merge output / load_branch / delete_branch,
no-top-level-torch source-grep guards, traversal delete_branch.
Plus v0.56.0 follow-up: test_v0560.py version-floor tests widened from
exact-match to floor-check (matches v0.51.0 / v0.54.0 idiom — every
subsequent release would otherwise edit this one line).
Test count: 8849 → 8998 (+149 net in 4 new files; 4 POSIX-only symlink
tests skipped on Windows). Full suite green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 18:37:14 +05:00
Alpamys