Typer/Rich injects ANSI codes under CI FORCE_COLOR that split flag names
(--mole, --citation-style) and wrap panel text, breaking raw substring
asserts. Normalise output (strip ANSI + collapse whitespace) before matching.
Same pattern as the v0.71.1 --record-thumbs fix.
Pre-existing v0.71.13 test flaked on windows-latest CI: record_thumb stamps
time.time() and count_new_thumbs_since uses strict `>`, so on Windows' ~15.6ms
clock resolution the two mid-train thumbs could land in the same tick as
run_started (ts == run_started, dropped). Sleep one clock tick at the start of
slow_train so the thumbs are strictly later. Prod semantics unchanged; tests-only.
CI caught a directory-rejection bug masked on Windows: on POSIX, os.open(dir,
O_RDONLY|O_NOFOLLOW) succeeds and os.fdopen() then raises IsADirectoryError
before the S_ISREG check ran (Windows os.open rejects the dir first). fstat the
raw fd before fdopen so a directory is rejected with a clean ValueError on every
platform. Local + CI-equivalent test_directory_rejected now green.
#261 iterative_dpo._default_train_fn rendered output as a {dir: ...} mapping
that SoupConfig rejected; render it flat so the spawned soup train succeeds.
#246 CMA-ES merge now loads the base model once and reuses it across the
candidate population (_CachedBaseScorer) instead of reloading per candidate.
#245 soup loop estimate_cost wires run_cost.estimate_run_cost_usd off the last
completed run instead of a 0.0 placeholder; never crashes the daemon.
#244 soup train --track-energy --energy-out persists the measurement JSON so
soup bom emit --energy can attach it to an ML-BOM.
#170 --diagnose-gate is RANK-aware: gate once per cluster (RANK==0), not per node.
Tests: 13476 -> 13511 (+35 in tests/test_v07115.py). Validated end-to-end on
SmolLM2-135M / RTX 3050.
PR #256 attached energy only to CycloneDX; #244's contract is "both
outputs". Add _energy_annotations() so the SPDX model package carries
energy as OTHER annotations (same soup:<field>=<value> naming). Strengthen
the happy-path test to assert energy actually lands in the BOM (not just
that a file is written) and add a --format both case asserting energy in
BOTH cdx + spdx.
Closes#244
Adds --energy flag to soup bom emit: validates the measurement JSON (cwd containment + symlink rejection + JSON + EnergyMeasurement shape) and attaches energy properties to both CycloneDX and SPDX outputs.
Co-authored-by: gittihub-jpg <gittihub-jpg@users.noreply.github.com>
Soup spans the full post-training stack — SFT + preference/RL
(DPO/GRPO/PPO/KTO/ORPO/SimPO/IPO/BCO) + distillation + unlearning +
knowledge-edit + steering + RAFT/RA-DIT — not just fine-tuning. "Fine-tune"
alone undersells the RL/alignment surface and the post-training category.
Update the canonical tagline to "Fine-tune and post-train LLMs in one
command. No SSH, no config hell." across the five places it appears:
README hero, PyPI description (pyproject), `soup --help` epilog + callback
docstring (cli.py), package docstring (__init__), and the auto-generated
HF model card (push.py). Searchable "fine-tune" keyword kept as the verb;
"post-train" added as the category claim.
Copy-only; no version bump (description ships with next release).
The topic pages still described both features as deferred stubs:
- FSDP consolidation showed the removed `--yes` flag and "lands in v0.44.1".
- KV-cache only documented the v0.53.0 schema ("once the runtime serve
path lands").
Both went live in v0.71.14. Update performance-and-quantization.md with
the live `soup merge-sharded-fsdp-weights` (streaming load, shape
validation, --plan-only) and `soup serve --kv-cache-type` (transformers
bf16/f16/q8_0/fp8, hqq advisory, Hopper gate, #140 vLLM/SGLang note), and
cross-link a short KV-cache subsection from serving-and-export.md.
Docs-only; no version bump (v0.71.14 already tagged).
quantized_cache_backend_available() called importlib.util.find_spec
("optimum.quanto"), which imports the parent `optimum` package to resolve
the submodule and raises ModuleNotFoundError when optimum is not installed
(instead of returning None). CI (no optimum) hit this on every test cell;
the live `soup serve --kv-cache-type q8_0` advisory path would crash the
same way on any box without optimum. Wrap submodule probes in a _spec_exists
helper that treats ModuleNotFoundError/ValueError as "not available".
CI failed on the HF-rate-limited runners: test_anchor_term_with_file did a
live from_pretrained that 429'd, so it failed AND its unique MiniLLM-anchor
lines went uncovered, tipping the 77% gate to 76.77% on exactly those jobs
(macos + 3.11 stayed green where the cache warmed).
- Skip test_anchor_term_with_file on OSError (offline / rate-limited) instead
of failing.
- Add test_anchor_term_with_fake_model: a fake tokenizer + tiny nn.Module
exercise the identical _load_anchor + anchor_term lines with no network, so
coverage no longer depends on HF availability.
- Add TestReachableInternals cushion (prompt_compile._resolve_metric,
prompt_distill._build_provider_fn + default-provider wiring) so the gate
sits comfortably above 77% (the DSPy/TextGrad/GEPA optimiser bodies are
uncoverable without the [compile] extra).
Tests 13424 -> 13430.
Lift the v0.68.0 deferred-stub family to live (closes#225, #226, #227, #229):
- #229 local-rl train --once: harvest thumbs -> DPO/KTO/ORPO train via a
soup train subprocess (argv list, no shell); state table tracks last_train_at
(skip-on-no-new-thumbs + skip-on-insufficient-pairs); no --once renders a
systemd/launchd nightly scheduler scaffold. New local_rl_scheduler.py.
- #226 distill-prompt: call the teacher once per trace (Ollama/Anthropic/vLLM)
and write a real dataset (sft/kl -> messages; preference -> chosen/rejected).
- #225 compile / #227 compile-tools: live DSPy/GEPA/TextGrad dispatch behind the
new [compile] extra with a friendly ImportError when absent; injectable seams.
Security: reject \n/\r in the model id + shell-quote ExecStart args (systemd
injection defence). Fix: render train output as a plain string (schema-valid),
with a regression test against SoupConfig.
Tests 13329 -> 13424. Smoked end-to-end: real DPO train on SmolLM2-135M (RTX 3050)
+ real Ollama teacher distillation.
The v0.71.12 release commit updated the test count but missed the two new
modules in the Project Structure tree: trainer/mole_routing.py (MoLE Routing)
and utils/mod.py (Mixture-of-Depths). Docs-only; no version bump.
The test job loads sshleifer/tiny-gpt2 and hf-internal-testing/tiny-random-gpt2
from HF Hub. Under transient HF rate-limiting (429) a single matrix cell would
fail the model download (test_v07111.py::TestMiniLLM::test_anchor_term_with_file)
and drop coverage under the 77% gate — observed reding 3 commits today (~2/7
cells per run).
Add a best-effort pre-test step that snapshot_downloads both tiny models with
6 retries + backoff so the tests read from the warmed cache (verified locally:
the test passes with HF_HUB_OFFLINE=1 once the cache is warm). The step never
fails the job — it emits a :⚠️: if warming can't complete.
CI-only; no version bump.
CI renders Typer help with ANSI colour codes under FORCE_COLOR that split the
leading `--` from the flag name, so raw-substring assertions on `--steer`
(serve) and `--output`/`--top-k` (steer train) passed locally but failed in CI.
Strip ANSI before the membership check (same fix as the v0.71.1 --record-thumbs
help assert). Local + FORCE_COLOR=1: 142/142 pass. tests-only, no version bump.
POSIX os.putenv forbids null bytes in env values, so
monkeypatch.setenv(SOUP_EDIT_GOVERNOR_DB, 'x\x00.db') raised
ValueError at setenv time on ubuntu/macos before the code under
test ran (windows tolerated it). Assert _validate_governor_db_override
rejects the null byte directly; the validated-None fallback branch is
already covered cross-platform by test_env_override_out_of_bounds_falls_back.
CI (FORCE_COLOR) makes Rich/Typer split flag tokens at colorized hyphens
(--auto-download -> -auto -download) and auto-highlight `=` in error text
(name=path), so contiguous-substring asserts fail. Add the _clean_help helper
(strip ANSI + all whitespace, matching the v0.71.1 / test_v0717 pattern) and
apply it to the sae-diff / train / sleeper / interference --help asserts plus
the bad-adapter-spec name=path error assert. Reproduced + verified with
FORCE_COLOR=1 locally. No source change; test count unchanged.
Rich splits `--pre-wired` / `--pack-cans` / `--push` with ANSI escapes under
CI FORCE_COLOR; _clean_help() strips them before the substring check (same
fix family as v0.71.1/v0.71.3). No src change.
CI (FORCE_COLOR) renders --track-energy / --no-audit-log as split ANSI colour
segments, and monkeypatch.setenv with a null byte raises at setup on POSIX
(Windows tolerated both). Strip ANSI via a shared `_plain()` helper for every
--help substring assert, and rewrite the never-raises audit test to monkeypatch
append_audit_event to throw instead of injecting a null-byte env path.
CI installs [dev] with FORCE_COLOR, so Rich colorizes Typer --help and
splits an option name like --key into ANSI-wrapped segments
(\x1b[1;36m-\x1b[0m\x1b[1;36m-key\x1b[0m). The 4 raw-substring help
asserts passed locally (no color) but failed on all 9 CI test jobs.
Add a module-level _strip_ansi() helper and route the sign/verify/merge/
attest-emit --help substring checks through it (mirrors the v0.71.1
test_serve --record-thumbs fix). Confirmed locally under FORCE_COLOR=1:
all 4 pass; ANSI-strip alone is sufficient (no flag line-wraps).
Test-only change on the unreleased v0.71.2 — no version bump.
The v0.71.1 release commit (514761c) went red on CI for two reasons:
- test_flag_in_help asserted a raw "--record-thumbs" substring, but Rich
splits an option name's dashes with ANSI codes under CI's FORCE_COLOR
(it passes locally without color). Strip ANSI before the substring check.
- Coverage fell to 76.96% (< 77% gate): CI installs [dev], which has no
FastAPI, so the new /v1/thumbs endpoint + record-thumbs startup block in
serve.py are uncovered there. Restore the gate honestly (no lowering, no
pragma) by adding 19 genuine no-FastAPI tests for previously-uncovered
pure-CLI paths: lock show / lock check (no-drift / drift exit 3 / missing),
env check (no-drift / missing / drift exit 3), env fix error branches,
env lock null-byte output, and load_evidence_file (the
`eval unlearning --evidence` loader).
CI-equivalent (no-fastapi) coverage: 76.96% -> 77.24%. Tests: 12134 -> 12153.
- SECURITY.md: supported window 0.70.x -> 0.71.x
- CONTRIBUTING.md: dev-deps list now lists mypy + pre-commit and notes the
v0.71.0 deps-split ([dev] self-references [train], so torch & co are pulled in)
Heavy training stack (torch, transformers, peft, trl, datasets,
bitsandbytes, accelerate) moves out of the core install into a new
[train] optional-dependency extra. `pip install soup-cli` is now a
light CLI + data-tools install with no PyTorch; `pip install
'soup-cli[train]'` adds the training stack.
- pyproject: new [train] + [all] extras; [dev] self-references [train]
so CI (`pip install -e ".[dev]"`) still gets torch. Pins unchanged.
- errors.py: missing torch/transformers/peft/trl/datasets/bitsandbytes/
accelerate now surface a single 'install soup-cli[train]' fix.
- Dockerfile: install soup-cli[train,serve,data,eval] so the GPU image
can still fine-tune.
- README + docs/models.md: split install into light core vs [train].
- CHANGELOG: cut [0.71.0]; bump version 0.70.0 -> 0.71.0.
CLAUDE.md and .claude/plan.md are maintainer-local (gitignored), so any
committed file that links to or tells contributors to read/update them is
broken for anyone cloning the public repo. Repoint everything at the public
docs/ split (and the schema / CHANGELOG where relevant).
- CONTRIBUTING.md: "add a trainer/format" + "Version Bump Process" steps now
say README.md + the relevant docs/ page (was CLAUDE.md); removed the
"See CLAUDE.md for the complete release checklist" pointer (replaced with a
one-line public note: bump -> tag v* -> CI -> PyPI OIDC publish); reworded
the test-table note off CLAUDE.md
- AGENTS.md: removed the maintainer-local CLAUDE.md parenthetical
- advise.py / runs.py / quant_menu.py: reword docstring/comments that cited
.claude/plan.md or CLAUDE.md (comment-only; no behaviour change)
Acceptance: grep "CLAUDE.md" across *.md (excl .claude/) is now empty; no
committed file references .claude/CLAUDE.md or .claude/plan.md.
The repo moved to src-layout and trimmed README into a 238-line front door
with the feature reference under docs/, but several committed files still
referenced bare soup_cli/ paths or linked the gitignored .claude/CLAUDE.md
(which 404s for anyone cloning the public repo).
- docs/: `soup_cli/{plugins,templates,ui/plugins}/...` path refs -> `src/soup_cli/...`
(import statements `from soup_cli...` left unchanged — package name is still soup_cli)
- AGENTS.md: point external agents at public docs/, CONTRIBUTING.md, and the
config schema; note CLAUDE.md is a maintainer-local (gitignored) file
- CONTRIBUTING.md + .github/pull_request_template.md: PR checklist now says
"README.md and the matching page under docs/" (kept in sync); Questions
section links docs/ instead of the gitignored CLAUDE.md
- examples/README.md: fix two broken ../CLAUDE.md links -> config schema source
+ docs/ feature reference
- .gitignore: add root-anchored /_*.py temp-script guard + trailing newline
mypy findings are surfaced as a warning annotation + step log instead of
failing the job. The workflow was already green (job-level continue-on-error),
but the type-check job itself rendered as a red X in the commit checks. Move
the tolerance to the step and emit a :⚠️: so the check stays green while
type annotations are adopted incrementally.
The README had grown to 5046 lines (195 sections) — roughly one deep-dive per
feature accreted over 70 releases. Split it into a concise front door plus a
public docs/ tree:
- README (5046 -> 238 lines): hero, why, quickstart, config, a Documentation
map, data formats, common commands, models, Docker, requirements, dev.
- docs/*.md: all 185 feature sections preserved verbatim, grouped into 10 themed
guides + an index. Every original line is accounted for (content-conservation
checked); all 235 internal links + anchors verified to resolve.
- un-gitignore docs/ (it was empty); fix a pre-existing dangling
docs/QUANTIZATION.md link; correct the stale `ruff check soup_cli/` ->
`src/soup_cli/` reference in the Development section.
No version bump: docs-only — rides into the 0.71.0 deps-split release.
Alpamys
gittihub-jpg