- SECURITY.md: supported window 0.70.x -> 0.71.x
- CONTRIBUTING.md: dev-deps list now lists mypy + pre-commit and notes the
v0.71.0 deps-split ([dev] self-references [train], so torch & co are pulled in)
Heavy training stack (torch, transformers, peft, trl, datasets,
bitsandbytes, accelerate) moves out of the core install into a new
[train] optional-dependency extra. `pip install soup-cli` is now a
light CLI + data-tools install with no PyTorch; `pip install
'soup-cli[train]'` adds the training stack.
- pyproject: new [train] + [all] extras; [dev] self-references [train]
so CI (`pip install -e ".[dev]"`) still gets torch. Pins unchanged.
- errors.py: missing torch/transformers/peft/trl/datasets/bitsandbytes/
accelerate now surface a single 'install soup-cli[train]' fix.
- Dockerfile: install soup-cli[train,serve,data,eval] so the GPU image
can still fine-tune.
- README + docs/models.md: split install into light core vs [train].
- CHANGELOG: cut [0.71.0]; bump version 0.70.0 -> 0.71.0.
CLAUDE.md and .claude/plan.md are maintainer-local (gitignored), so any
committed file that links to or tells contributors to read/update them is
broken for anyone cloning the public repo. Repoint everything at the public
docs/ split (and the schema / CHANGELOG where relevant).
- CONTRIBUTING.md: "add a trainer/format" + "Version Bump Process" steps now
say README.md + the relevant docs/ page (was CLAUDE.md); removed the
"See CLAUDE.md for the complete release checklist" pointer (replaced with a
one-line public note: bump -> tag v* -> CI -> PyPI OIDC publish); reworded
the test-table note off CLAUDE.md
- AGENTS.md: removed the maintainer-local CLAUDE.md parenthetical
- advise.py / runs.py / quant_menu.py: reword docstring/comments that cited
.claude/plan.md or CLAUDE.md (comment-only; no behaviour change)
Acceptance: grep "CLAUDE.md" across *.md (excl .claude/) is now empty; no
committed file references .claude/CLAUDE.md or .claude/plan.md.
The repo moved to src-layout and trimmed README into a 238-line front door
with the feature reference under docs/, but several committed files still
referenced bare soup_cli/ paths or linked the gitignored .claude/CLAUDE.md
(which 404s for anyone cloning the public repo).
- docs/: `soup_cli/{plugins,templates,ui/plugins}/...` path refs -> `src/soup_cli/...`
(import statements `from soup_cli...` left unchanged — package name is still soup_cli)
- AGENTS.md: point external agents at public docs/, CONTRIBUTING.md, and the
config schema; note CLAUDE.md is a maintainer-local (gitignored) file
- CONTRIBUTING.md + .github/pull_request_template.md: PR checklist now says
"README.md and the matching page under docs/" (kept in sync); Questions
section links docs/ instead of the gitignored CLAUDE.md
- examples/README.md: fix two broken ../CLAUDE.md links -> config schema source
+ docs/ feature reference
- .gitignore: add root-anchored /_*.py temp-script guard + trailing newline
mypy findings are surfaced as a warning annotation + step log instead of
failing the job. The workflow was already green (job-level continue-on-error),
but the type-check job itself rendered as a red X in the commit checks. Move
the tolerance to the step and emit a :⚠️: so the check stays green while
type annotations are adopted incrementally.
The README had grown to 5046 lines (195 sections) — roughly one deep-dive per
feature accreted over 70 releases. Split it into a concise front door plus a
public docs/ tree:
- README (5046 -> 238 lines): hero, why, quickstart, config, a Documentation
map, data formats, common commands, models, Docker, requirements, dev.
- docs/*.md: all 185 feature sections preserved verbatim, grouped into 10 themed
guides + an index. Every original line is accounted for (content-conservation
checked); all 235 internal links + anchors verified to resolve.
- un-gitignore docs/ (it was empty); fix a pre-existing dangling
docs/QUANTIZATION.md link; correct the stale `ruff check soup_cli/` ->
`src/soup_cli/` reference in the Development section.
No version bump: docs-only — rides into the 0.71.0 deps-split release.
- add src/soup_cli/py.typed (PEP 561); verified it ships in the built wheel
- add .pre-commit-config.yaml (ruff lint+format + standard file-hygiene hooks)
- add mypy>=1.8.0 + pre-commit to the [dev] extra; lenient [tool.mypy] config
- add a non-blocking type-check CI job (mypy, continue-on-error: true)
- add CHANGELOG.md (Keep a Changelog; [Unreleased] + link to GitHub Releases)
- replace the ~221KB per-version security log in SECURITY.md with a concise policy
- raise the coverage gate 50% -> 77% (measured 79% on the suite; real-2 margin)
No version bump: hygiene/docs only — rides into the 0.71.0 deps-split release.
Move soup_cli/ -> src/soup_cli/ (history preserved via git mv). src-layout
forces the test suite to import the installed package instead of the
repo-root source tree, surfacing packaging bugs that flat-layout masks —
e.g. the v0.53.8 double-shipped-fixtures regression, invisible because
`pytest tests/` imports ./soup_cli directly and never from the wheel.
- pyproject: packages = ["src/soup_cli"]; artifacts globs -> src/soup_cli/...
The import name is unchanged, so the `soup` entry point, --cov=soup_cli,
and report_to/module-path strings stay `soup_cli`.
- CI / ownership: ruff lint path (ci.yml), recipe-validation `paths:` filters,
CODEOWNERS patterns, and the PR-template checklist all repointed to
src/soup_cli/.
- tests: source-grep regression tests that read package files by repo-relative
path repointed to src/soup_cli/ (64 files; 170 path literals). Lines pushed
over 100 chars by the prefix were wrapped to keep ruff E501 clean. Module
references (`import soup_cli`, `-m soup_cli`, mock.patch("soup_cli.x")) and
the `--cov=soup_cli` coverage target are deliberately unchanged.
- docs: AGENTS.md + CONTRIBUTING.md structure tree and lint commands.
Verified locally: ruff clean (src/soup_cli + tests); `import soup_cli`
resolves to src/soup_cli/__init__.py; built wheel ships
soup_cli/data/_fixtures/*.jsonl (10 files, no duplicates, no src/ prefix);
3174 tests across every touched test file pass. Packaging-only — no version bump.
Rich on narrow Windows columns splits `--lang` across colour-cycle ANSI
escapes (`\x1b[..m-\x1b[..m-lang`), so the literal substring check fails
on CI even though the rendered help renders correctly for humans. CI was
red on every commit landing after #234 hit a runner with that exact
column width + Python 3.9 + Rich combination.
Mirrors the `_ANSI_RE` strip pattern in tests/test_auto_tuning.py — flat
regex over the captured output before the `in` check.
v0.63.0 `score_uncertainty` raised on K>2 and `_row_uncertainty` fell back
to a monotone-broken `max(scores) - min(scores)`. Now generalises to K<=32
via population variance scaled by 4 — adding a fresh RM score equal to the
running mean strictly decreases uncertainty (the new contribution to the
sum-of-squares is zero while the denominator grows), so consensus on
redundant evidence can never spike the score.
K=1 max-entropy and K=2 disagreement formulas preserved verbatim (existing
operator dashboards depend on the |s1 - s2| value). Cap stays at K=32 for
DoS defence. _row_uncertainty K>2 path now routes through score_uncertainty
inside an isolated try/except — bad rows return 0.0 instead of crashing the
batch.
PEP 585 modernisation: collections.abc imports + list[...] annotations
(safe because `from __future__ import annotations` is in scope).
math.fsum used for the variance accumulation to keep rounding error
sub-ULP at K=32.
Tests: +32 net (25 in new tests/test_v0631_206.py + 7 TDD review-fix
followups). Full suite 11941 -> 11973 pass.
Closes#206.
Extends v0.69.0 Part E score_triviality + score_popularity_signal to
non-English corpora. New utils/brain_rot_lang.py ships a MappingProxyType
registry of frozen BrainRotLangBundle for en/es/fr/de/ru. Every public
scorer accepts an optional lang kwarg (default None preserves v0.69.0
English behaviour). The "auto" sentinel routes through the v0.53.10
[data-pro] langdetect helper with silent fallback to English on
missing-package / detector-exception / unsupported-code.
soup data brain-rot gains --lang en|es|fr|de|ru|auto, strictly validated
at the CLI boundary (exit 2 on typos). Per-row resolution backed by
eager _validate_lang_arg on dataset scorers so empty rows cannot
bypass shape checks.
Closes#234.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes#241.
Adds opt-in tokenizer-aware n-gram path for the v0.70.0 Part F echo-trap detector. The existing whitespace `score_echo_signal` is unchanged; callers opt in via the new `score_trajectory_repetition_tokenized` / `score_echo_signal_tokenized` helpers or the `--echo-trap-tokenizer-aware` train flag.
Acceptance criterion from #241 verified: synthetic case where decoded strings differ by punctuation but token-id sequence repeats — whitespace path returns OK, tokenizer-aware path returns TRAP.
Post-release audit caught two pre-existing doc drifts that v0.70.0 should
have closed in the initial commit:
1. README.md `## All Commands` table was missing the 6 new v0.70.0 surfaces
(`soup iterative-dpo`, `soup train --reward-hack-detector`,
`--uld-strategy`, `--minillm-enabled`, `--rl-checkpoint-save-every-steps`,
`--echo-trap-enabled`). Added 6 lines mirroring the v0.69.0 entry format.
2. CONTRIBUTING.md `utils/` directory tree had stale drift from v0.66.0 →
v0.70.0 (24 modules missing across 5 releases). Brought in sync:
v0.66 (sae_diff, sleeper_probe, interference, probe_pack), v0.67
(cmaes_merge, vector_bank, mole_routing, adapter_pr, soup_lock,
adapter_bisect), v0.68 (prompt_compile, prompt_distill, compile_tools,
apple_adapter, local_rl), v0.69 (build_dag, expectations, magpie,
persona_hub, brain_rot), v0.70 (reward_hacking, uld, minillm,
rl_checkpoint, iterative_dpo, echo_trap).
No code changes — docs-only hotfix per the checklist
"CI-only / docs-only hotfixes" rule. No version bump, no tag.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The v0.67.0 and v0.68.0 releases shipped 14 new commands but neither
release updated the README `## All Commands` reference table per Release
Checklist Step 9. Adds the missing rows:
- v0.67.0 (5): adapters merge --strategy cmaes / adapters pr / adapters
bisect / lock write / lock show + check.
- v0.68.0 (9): compile / distill-prompt / compile-tools / apple-adapter +
local-rl init / status / record / harvest / train.
Docs-only — no source code change, no test count delta, no version bump
per the checklist's CI-only / docs-only hotfix policy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 commands that hedge Soup against paradigm shifts. If 1M-context kills FT,
`soup compile` (DSPy + GEPA + TextGrad prompt-program compilation) takes its
place. If teams hit prompt-cost walls, `soup distill-prompt` bridges to small
FT. If only Apple Foundation Models win on-device, `soup apple-adapter` ships
the converter+signing surface. If personal-LLM flywheels become the shape,
`soup local-rl` captures thumbs into SQLite and emits DPO pairs.
- Part A: `soup compile <program.py> --eval <suite> [--optimizer mipro|gepa|...]`
- Part B: `soup distill-prompt --traces <jsonl> --teacher --student --strategy`
- Part C: `soup compile-tools <spec.json|yaml> --eval <jsonl>`
- Part D: `soup apple-adapter <source-dir> --direction hf-to-mlx|... --output`
- Part E: `soup local-rl init/status/record/harvest/train` (LIVE except train)
Schema + path containment + symlink rejection + atomic-write surface ship now;
live runners for Parts A/B/C/D + Part E nightly scheduler deferred to v0.68.1
(stub-then-live, mirrors v0.50.0 / v0.61.0 / v0.62.0 / v0.67.0 cadence).
Test count: 11021 -> 11225 (+204). Review-fix: 0 CRIT + 4 HIGH + 10 MED + 4 LOW.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- SECURITY.md: add v0.67.0 to supported-versions list
- CONTRIBUTING.md: bump test count 244 files / 10836 tests -> 251 / 11021
Docs-only follow-up to v0.67.0 release; no code change, no version bump.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Was `assert soup_cli.__version__ == "0.64.0"` — broke on v0.65.0 bump on CI
across all 9 platform×Python matrix jobs. The test intent is "v0.64.0 has
shipped at least once"; switch to a tuple floor check matching the v0.51 /
v0.54 / v0.57 / v0.60 idiom.
Caught by CI red on v0.65.0 push to main; local pytest passed because we
ran the v0.65 test files in isolation per the Release Checklist Step 4
``pytest --no-cov`` invocation. Lesson: include the full suite in step 4
or grep for ``soup_cli.__version__ ==`` exact-equality assertions in any
future version bump.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six new top-level commands close axis 1 + 11 of the roadmap: pick the
right base, lock the env, refuse OOMs before launch, and clear
license-clean deploys.
- soup tunability: probe-train 8 candidate bases (Qwen3-0.6/1.7B,
Llama-3.2-1/3B, Gemma-3-E2B, Phi-4-mini, SmolLM3, Qwen2.5-1.5B) ->
Pareto frontier over (delta x cost x license). Live LoRA probe -> v0.64.1.
- soup plan / soup apply: Terraform-shape lock-and-execute. `apply`
refuses on drift between soup.yaml and soup.tfstate (exit 3).
- soup env lock / status / check: hermetic env lockfile via
importlib.metadata across 15 ABI-sensitive packages + Python + CUDA.
`env check` exits 3 on drift.
- Hardware-fit calculator: static analytical 5-bucket VRAM predictor
with 10% safety margin + actionable hint on OOM.
- soup completions bash|zsh|fish: sourceable shell completion scripts;
recipe names auto-complete from the 115-recipe catalogue.
- soup license-advisor: per-deploy-target license matrix
(b2c/defense/embedded) + Llama community + 700M MAU gate (exit 3).
Composes with v0.60 license-conflict matrix.
Tests: 10035 -> 10306 (+271 net in 7 new files).
Review-fix coverage: 0 CRITICAL + 6 HIGH + 8 MEDIUM + 4 LOW across
consolidated code+security+TDD review wave. Every HIGH lands a regression
test in tests/test_v0640_followups.py (POSIX-skipped symlink rejection,
containment-before-existence ordering, drift-refusal exit-3 end-to-end).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Docs-only hotfix — adds the v0.63.0 commands (soup ingest /
prune-prompt / data active-sample / ab / drift-alarm) to the
"## All Commands" code-block reference card. The release v0.63.0 had
dedicated ## feature sections + the ## What's New block updated, but
the All Commands quick-reference was missed during the doc pass.
No source code changed — no version bump, no tag, no release.
Per CLAUDE.md hotfix policy: docs-only commits don't ship to PyPI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 new top-level commands close axis 7 of the roadmap. Every Part LIVE on
day one (no deferred stubs):
- soup ingest: universal trace importer (Langfuse / LangSmith / Helicone /
OpenPipe / OTel / OpenAI Stored Completions). 6 adapters
+ frozen TraceRecord with MappingProxyType-wrapped metadata.
Zero credential-handling threat surface — Soup parses the
JSONL export, never makes the SaaS network call.
- soup prune-prompt: detect + strip a shared system-prompt prefix so the
FT model internalises it (OpenPipe's signature trick,
OSS). Binary-search over up to 32 templates finds the
longest threshold-meeting prefix.
- soup data active-sample: surface top-uncertainty prod traces for human
review. Max-entropy on single rm_score or
pairwise disagreement on dual rm_scores.
- soup ab: Wald sequential SPRT for the point alternative. LLR is a
martingale under H0 so Type-I error is controlled at every
stopping time per the optional stopping theorem.
- soup drift-alarm: rolling KL on whitespace-tokenised output distribution
+ SSRF-hardened Slack/Discord webhook (full parity with
v0.51.0 validate_hub_endpoint). Exit 3 on drift for
cron-friendly automation.
Test count: 9816 -> 10035 (+219 net across 6 new test files).
Review-fix coverage (code-reviewer + tdd-guide returned actionable;
python-reviewer + security-reviewer agents context-thrashed on the large
CLAUDE.md release-notes history — matches the v0.58.0 / v0.59.0 / v0.60.0
/ v0.61.0 / v0.62.0 idiom; verified manually):
- 1 CRITICAL: mSPRT log-likelihood-ratio sign error drove Type-I error
to 1.0 as n grew. Replaced with Wald's classic point-
alternative SPRT (martingale under H0).
- 2 HIGH: detect_common_prefix early-exit on 100% match returned the
shortest qualifying prefix instead of the longest;
_MAX_SCAN_ROWS DoS cap used 'pass' instead of 'break'.
- 3 MEDIUM: TraceRecord.metadata now MappingProxyType-wrapped post-init
(frozen-dataclass mutation hazard); _AUTH_ENV table
deduplicated; drift_alarm precedence parens on SSRF gate.
- 2 LOW: pooled_se dead-branch refactor; mean_uncertainty NaN guard.
- 8 follow-up tests: msprt zero-variance, partial-majority binary-search
activation, score_uncertainty exact boundaries, rolling_kl identical
+ disjoint, validate_budget + validate_threshold exact endpoints,
_signal_from_thumbs boundaries, no-heavy-top-level-imports source-grep
guard across all 5 new util modules.
Step 6 smoke verified for all 5 commands + 6 failure-mode rejection
paths.
CRLF gotcha note for future maintainers: PowerShell wrote the smoke
fixtures with a UTF-8 BOM on Windows during Step 6 — switched to
inline Python for the fixture write. Production CLI input handling is
already BOM-tolerant (utf-8-sig in JSONL loaders via v0.40.1 Part E).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The list/info portions of 4f54179 follow the structure of @dreamer0129's
PR #175 draft. Adding Co-Authored-By trailer so GitHub contributors graph
reflects the original draft author.
Co-Authored-By: dreamer0129 <185807357+dreamer0129@users.noreply.github.com>
Closes#174. Picked up after PR #175 (dreamer0129) went quiet — the
list/info path was clean, but compare() escaped only inside the
highlight branch, leaving a shared crafted base_model = "[link=evil]
click[/]" un-escaped on equal-value rows.
Fix follows the "escape always at the value layer, decoration wraps
after" pattern mirroring v0.57.0 `adapters diff` / `info`:
- list_adapters: wrap base / lora_r / peft_type / rel_path with
rich.markup.escape() before table.add_row(); also escape
adapter_path in the JSONDecodeError fallback.
- info: wrap base_model / peft_type / task_type / lora_r / lora_alpha
/ lora_dropout / modules_str inside the Rich Panel f-string; also
escape adapter_path.name in the Panel title.
- compare: escape val1_str / val2_str unconditionally; [yellow]
highlight wraps already-escaped values when they differ. Equal-value
rows now also escape (was the v0.57.0 known-limitation gap).
+4 regression tests in tests/test_adapters.py::TestAdaptersMarkupEscape:
- test_list_escapes_crafted_base_model — asserts no ANSI hyperlink
sequence (\x1b]8;) leaks from a crafted [link=http://evil/...] payload.
- test_info_escapes_crafted_base_model — same assertion for Panel.
- test_compare_escapes_equal_crafted_values — the specific regression
for the PR #175 review gap (identical crafted values on both sides
must NOT smuggle live markup through the equal-branch).
- test_compare_escapes_differing_crafted_values — highlight branch
also escapes.
Closes v0.57.0 Known Limitation (9).
Verified locally:
- ruff check soup_cli/commands/adapters.py tests/test_adapters.py -> clean
- pytest tests/test_adapters.py --no-cov -> 20 passed
CI on ubuntu / macOS exposed a missed TOCTOU detail: `os.lstat(realpath(path))`
silently resolves symlinks before the lstat, so `S_ISLNK` never trips and a
symlinked input path passes the rejection. Windows CI was skipped (POSIX-only
symlink test), so the bug didn't surface in local smoke.
Three fixes, matching the v0.53.7 #106 project policy of "lstat the RAW path
before realpath":
- `edit_diff.load_probes` — lstat path first, then realpath after rejection.
- `unlearning_eval.load_evidence_file` — same.
- `unlearning_eval.get_fixture_path` — lstat raw candidate before realpath.
No new tests — the existing `test_symlink_rejected` covers it; it now passes
on POSIX where it previously failed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three failures on CI run 26084542388 — all are version-pin / Rich-wrap
artefacts, not real regressions in v0.60.0 functionality:
- test_v0560 test_pyproject_version: regex-based >=0.56 floor check
(was substring `version = "0.5`)
- test_v0590 test_version_is_0_59 -> test_version_is_at_least_0_59:
>=0.59 floor (matches v0.51/v0.54 floor-check idiom)
- test_v0600_part_e merge_help_lists_license_flags: strip ANSI codes
before substring check (Rich splits `--license` across `\x1b[1;36m`
escapes in the wrapped Typer table)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The v0.59.0 What's New block claimed +93 new tests, but the actual count
is +99 new tests in tests/test_v0590.py (9193 -> 9294). Numbers in
CLAUDE.md / SECURITY.md / CONTRIBUTING.md / the release notes already
have the correct count.
Docs-only hotfix; does not require a version bump (per release checklist).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous test_default_log_path_rejects_null_byte_env used monkeypatch.setenv
to inject a null byte into SOUP_AUDIT_LOG_PATH and expected default_log_path
to fall back gracefully. But the OS layer rejects null bytes in env vars on
every platform we ship on:
- POSIX (Linux/macOS): `ValueError: embedded null byte`
- Windows: `ValueError: embedded null character`
The setenv call itself raises, never reaching default_log_path. Split into two
tests that hit the actual validation surfaces:
1. test_default_log_path_rejects_null_byte_override — calls the private
_validate_log_path_override helper directly with a null-byte string and
asserts it returns None (so the caller falls back to the safe default).
2. test_default_log_path_handles_env_read_value_error — monkeypatches
os.environ.get to raise ValueError, exercising the defence-in-depth
try/except around the env read in default_log_path().
Both tests pass on Linux + macOS + Windows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three macOS-3.11 CI failures in test_v0590.py post-merge:
1+2. test_train_annex_xi_flag_present_in_help / test_train_repro_receipt_flag_present_in_help
— Typer's Rich-renderer wraps long lines and inserts ANSI colour codes
BETWEEN the two dashes of `--annex-xi` / `--repro-receipt`, so the
literal substring match fails. Strip ANSI escape codes via regex before
asserting; also accept the bare option name as a defence-in-depth
fallback against future Rich line-wrap quirks.
3. test_default_log_path_rejects_null_byte_env — POSIX `os.environ.get` raises
`ValueError("embedded null byte")` when the env value contains a NUL
character, while Windows allows the read. Wrap the env read in
`try/except ValueError` so the function falls back to the safe default
(~/.soup/audit.jsonl) on either platform.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six Parts ship the procurement-floor moat — every Soup run can now emit
the formats regulated orgs demand, with no SaaS structurally able to
follow. Pure orchestration on top of v0.26 Registry + v0.34 cost
tracker + v0.56 diagnose — schema + atomic-write surface only; live
Sigstore signing, CodeCarbon hook, PDF rendering deferred to v0.59.1.
Part A — `soup bom emit` CycloneDX 1.6 ML-BOM + SPDX 2.3 AI-profile
dual emitter from any RegistryEntry. SHA-256 validation on every sha
field, license-id chain, base-model component with hash, per-artifact
file components, energy properties under metadata.properties.
Part B — `soup attest emit` in-toto v1 Statement wrapping SLSA-3
provenance v1 predicate. Stage allowlist (extract/train/eval/export/
publish), subject SHA locked to 64-hex, builder_id capped, SignatureBackend
enum with UNSIGNED live + SIGSTORE/ED25519 stubs raising NotImplementedError
with explicit v0.59.1 marker.
Part C — `soup train --annex-xi` EU AI Act Annex XI Sections 1+2 +
Annex XII Article 53(1)(d) markdown auto-doc. Top-10 domain cap,
modality breakdown, FLOPs/kWh/CO2. `_md_escape` neutralises |[](){}!<>
plus newline/CR/tab in every operator-controlled field — defends
against forged-heading + Markdown-link injection in downstream PDF/HTML
renderers (mirrors v0.29.0 model-card v2 policy).
Part D — `soup audit-log tail/rotate` HIPAA/SOC2-shaped JSONL with
PII redaction across every string field via v0.40.3 _SECRET_RE policy.
POSIX O_NOFOLLOW on append + 0o600 perms + lstat-based symlink rejection
at rotation backup path (no lexists race). SOUP_AUDIT_LOG_PATH env
override containment-checked to $HOME / $CWD / $TMPDIR.
Part E — `soup train --repro-receipt` SR 11-7-style receipt: seeds
(torch/numpy/python), kernel versions (CUDA/cuDNN/NCCL via best-effort
torch probes), GPU model + driver, OS + arch, Python version. Atomic
write, cwd-contained.
Part F — CodeCarbon hook schema + electricityMap SSRF validator with
full parity to v0.51.0 hubs.validate_hub_endpoint (scheme allowlist,
loopback-only HTTP, RFC1918 / link-local / reserved / multicast IP
rejection via ipaddress.ip_address, control-char + null-byte
rejection). PUE math + attach_energy populating BomEntry.
Cross-cutting: new paths.atomic_write_text shared TOCTOU-safe helper
centralises the v0.33.0 #22 / v0.43.0 / v0.55.0 / v0.56.0 / v0.57.0
/ v0.58.0 atomic-write pattern from four separate copies into one
single-source-of-truth (mirrors v0.40.6 / v0.53.5 peft_wiring policy).
Four review waves (python-reviewer + general-purpose security/code/tdd):
0 CRITICAL + 8 HIGH + 12 MEDIUM + 4 LOW resolved before commit.
HIGH fixes: audit-log lstat-before-write TOCTOU, O_NOFOLLOW on
append, redaction extended to host_id/operator_id/command, audit-log
env override containment, bom artifact size_bytes validation,
BomEntry attach_energy type-hint fix, default_log_path public symbol,
duplicated seeds validation removed.
Test count 9193 → 9294 (+99 net in tests/test_v0590.py; 93 pass +
6 POSIX-skipped on Windows for symlink rejection branches). v0.58.0
floor-check assertions widened from exact-match in test_v0580.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #178 wired the live NCCL bandwidth measurement on top of the
v0.43.0 nccl_bandwidth_check reference table. Four polish items
on top of the merged version:
- Bump timing samples 5 -> 10 and report MEDIAN rather than mean.
Median is robust to one-off jitter (kernel preemption, page
faults, OS scheduler hiccups) that can scew a 5-sample mean by
10-30%. Median of 10 gives a much more honest GB/s number.
- Bump warmup 1 -> 3 iters. First all_reduce in a fresh process
pays for CUDA kernel JIT plus the initial NCCL collective
handshake; one warmup isn't always enough to amortise that out.
3 warmups is the standard NVIDIA recommendation for collective
benchmarks.
- Time each iteration separately (collect a per-iter list, then
median) instead of averaging total elapsed / iters. Same headline
cost, but exposes per-call variance for future percentile output.
- Snapshot + restore MASTER_ADDR / MASTER_PORT around the spawn.
The worker sets them unconditionally, so prior to this patch a
second doctor invocation in the same process (test harness,
notebook, future TUI auto-refresh) would inherit stale env. Now
we restore exactly the prior state (including absent vars).
Plus three module-level constants (_NCCL_BENCHMARK_TENSOR_BYTES
/ _WARMUP_ITERS / _TIMED_ITERS) with comments explaining each
number so future tuning is informed, not magic. Status line now
prints the actual params so users see what we measured.
Existing tests (mock mp.spawn at the outer boundary) keep passing
unchanged; no new tests needed for this polish layer.
* feat(doctor): add --nccl flag to measure and validate multi-GPU bandwidth
* test(doctor): add mocked CUDA tests to verify --nccl skip and success behaviors
* docs(readme): document the new --nccl bandwidth check flag for the doctor command
Step 9 follow-up: add dedicated `## Adapter Management (git for LoRA)`
section to the README body so the surface is self-contained when the
single-slot `## What's New` block is overwritten in v0.58.0.
Step 10 follow-up: add v0.57.0 to the SECURITY.md supported-versions
list + a detailed entry in the per-version fix notes covering all
9 HIGH fixes (TIES sign-tie default, 4× symlink TOCTOU rejections,
atomic writes, env CRLF rejection, allowlist policy migration, etc.)
and the 7 known limitations.
Tracked follow-ups filed as GitHub issues #171–#174:
#171 — live blame ablation runner
#172 — merge canary verdict via v0.55 eval gate
#173 — branch pointers → v0.26 Registry lineage
#174 — Rich-markup backfill for legacy adapters list/info/compare
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POSIX setenv (and Windows equivalent) reject null bytes + control chars
at the syscall boundary, so monkeypatch.setenv("SOUP_BRANCHES_DIR",
"/some\x00path") raises ValueError on every CI runner before our code
ever sees the env var.
Stub os.environ.get directly so the helper's rejection branch is
exercised exactly as it would be if the env var arrived through some
other channel (subprocess env inheritance, in-process programmatic
mutation, etc).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rich-wrap-CI workaround — same fix pattern as v0.55.0 / v0.56.0:
CliRunner output contains ANSI color escapes that break literal
'--top-k' in output substring matches because Rich renders option
names as -\x1b[0m\x1b[1;36m-top-k.
Adds _ANSI_RE + _strip_ansi() helper to each of the 4 test files
(test_v0570_part_{a,b,c,d}.py) and routes every help-output
substring assertion through it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #169 wired LOCAL_RANK==0 guard on _run_diagnose_gate so distributed
launches only run the gate on one worker per machine. Two minor polish
items on top of the merged version:
- Wrap the int() parse in try/except ValueError. A malformed LOCAL_RANK
(garbage value from a misconfigured launcher) would previously crash
the post-training gate. Falling back to True is safer than silently
skipping the gate -- over-running is recoverable, under-running hides
failures.
- Expand the docstring to explain why we use LOCAL_RANK (per-machine)
rather than RANK (global): the gate reads the local output_dir, so
one gate per machine is the right granularity for typical single-
machine multi-GPU runs. Documents the choice for future readers.
- Add a focused test (test_diagnose_gate_handles_malformed_local_rank)
asserting the safe fallback path.
Alpamys
Shivam
Vivaan Dhawan
Salil M