The two TestRewardHackMitigationCli help-text assertions grepped the raw
`soup train --help` output for the new flags. CI runners emit color codes
that split long option names into non-contiguous characters (same failure
mode documented in test_eval_gate.py for --gate), so the raw substring check
failed on every OS/Python combo while passing locally on a no-color terminal.
Fix follows the established repo pattern: render with COLUMNS=200 (no option
wrapping) and strip ANSI escapes before the membership check. Verified under
FORCE_COLOR=1: both assertions pass. Test-only change, no version bump.
Folds in the doc-vs-reality cleanup the plan flagged: the docs referenced
soup train --reward-hack-detector / --reward-hack-halt but they were config-only.
Add both as CLI passthroughs mirroring --reward-hack-mitigation (validate value,
set cfg.training field, accelerate re-exec passthrough) so the docs are true and
the reward-hack CLI is consistent. +4 tests (test_v07126: 180 -> 184; full suite
-> 14788).
Fixes from 5 sequential ECC reviews (python/code/security/tdd/verification).
python-review (2 CRITICAL + HIGH/MED/LOW):
- signal/vote coherence: schema now requires the active detector in
reward_hack_signals + rejects the inactive detector name (was silently
dropping the primary signal from the vote).
- integral_clamp is its own field (was wrongly hard-wired to beta_ceil).
- task/backend gate runs before controller-config checks; EMA convention
corrected; type hints; mutable-list default -> tuple + normalised compare.
code-review (4 HIGH + MED/LOW):
- _prune now trims _saved in sync with disk (rollback can't target a deleted
checkpoint); bang-bang release_count resets after each relaxation (hysteretic
descent); EMA formula uses standard convention; _escalate no longer burns a
recovery attempt on a None target; max_recovery_attempts>=1 required with
rollback; _action_history capped; on_step_end logs errors once; loud warning
when the mitigation callback can't attach (was a silent safety-off).
security-review (HIGH + MED):
- restore_checkpoint / save_checkpoint refuse a SYMLINKED optimizer.pt
(torch.load weights_only=False was an RCE via attacker-placed symlink);
bool-before-int/float guards on all new numeric fields; reward_hack_signals
max_length=4; empty-signals guard in the callback.
tdd-review: +13 coverage tests (dead-band hold, shim verbatim-on-error,
conservative boundary, read-only-beta dual-write, escalation postconditions,
both-restore, PID D exact, log cap + concurrency, no-top-level-import, fuzz
field-validity).
Test count 152 -> 180 (+2 POSIX-only symlink skips).
Recipe-count drift: the CLI help (commands.md) and Web UI pages
(serving-and-export.md) claimed 43 ready-made recipes and the catalog
docstring said ~30, while RECIPES actually holds 116 (test_recipes already
asserts len == 116). Aligned every user-facing count to 116.
Also refreshed 6 recipe descriptions still tagged "schema-only stub
(live in v0.52.1)": the TTS task (#131) and BitNet 1.58 SFT (#134) went
live in v0.71.20, so orpheus/sesame/llasa/spark/oute TTS and the Falcon-E
BitNet recipe now read "live (v0.71.20)".
Doc-drift only — description strings + one comment; no functional change,
no version bump.
The live-codec block claimed the entire data.format=audio path was "not
validated on the maintainer's box" and only surfaced a RuntimeError. v0.71.22
made the Orpheus SNAC encode live + validated; note that while the other four
families stay dependency-gated. Docs-only, no version bump.
- performance-and-quantization.md: the Quant Menu multi-trainer note said
"vision / audio modality is still SFT-only inline-BNB (wiring tracked as a
follow-up)" — stale after v0.71.19 #81 dropped the modality gate. Now states
vision/audio thread the unified loader (full gptq/awq/hqq/aqlm/eetq/mxfp4/fp8
menu), with the upstream class+kernel caveat.
- peft-and-efficiency.md: added the v0.71.19 #80 multi-GPU sharding paragraph to
the Multipack section (accelerator.prepare + BatchSamplerShard under
num_processes>1; identical bin seed across ranks; single-GPU unchanged).
Docs-only — no version bump / tag (the v0.71.19 code already shipped at f51331d).
#81 Quant Menu for vision/audio modality
- config/schema: drop the `modality != "text"` rejection in
_validate_quant_menu_supported_tasks (mlx-backend gate retained) so the full
Quant Menu (gptq/awq/hqq:Nbit/aqlm/eetq/mxfp4/fp8) applies to vision+audio.
- trainer/sft: _setup_vision_transformers + _setup_audio_transformers call
build_quantization_config_for_loader (strict superset of the inline 4bit/8bit
BNB blocks they replaced); drop BitsAndBytesConfig import; retain the
prepare_model_for_kbit_training gate on (4bit,8bit,mxfp4).
#80 multipack DataLoader sharding under FSDP/DeepSpeed/DDP
- utils/multipack_trainer: get_train_dataloader routes the multipack DataLoader
through accelerator.prepare when num_processes > 1 so accelerate's
BatchSamplerShard shards whole FFD bins across ranks (even_batches=True avoids
the epoch-boundary collective hang; seed identical across ranks, no `+ rank`).
Single-process path unchanged. Defence-in-depth guard against an unconfigured
MagicMock num_processes.
Tests: +37 in tests/test_v07119.py (13770 -> 13807). ruff clean.
Supersedes the v0.40.5 vision/audio Quant-Menu and v0.40.4 multipack-FSDP
known-limitations. Full multi-GPU validation remains an INFRA-BLOCKED QA item.
TestTrainCliMinillmOnPolicy::test_flag_in_help removed newlines + spaces but
not ANSI codes, so under CI FORCE_COLOR the Rich-rendered long flag (ANSI codes
between the dashes) failed the substring check. Strip ANSI + remove all
whitespace before the check, matching the cloud/sandbox help tests + the
v0.71.17 precedent. Verified under FORCE_COLOR=1 (114 passed).
Typer/Rich injects ANSI codes under CI FORCE_COLOR that split flag names
(--mole, --citation-style) and wrap panel text, breaking raw substring
asserts. Normalise output (strip ANSI + collapse whitespace) before matching.
Same pattern as the v0.71.1 --record-thumbs fix.
Pre-existing v0.71.13 test flaked on windows-latest CI: record_thumb stamps
time.time() and count_new_thumbs_since uses strict `>`, so on Windows' ~15.6ms
clock resolution the two mid-train thumbs could land in the same tick as
run_started (ts == run_started, dropped). Sleep one clock tick at the start of
slow_train so the thumbs are strictly later. Prod semantics unchanged; tests-only.
CI caught a directory-rejection bug masked on Windows: on POSIX, os.open(dir,
O_RDONLY|O_NOFOLLOW) succeeds and os.fdopen() then raises IsADirectoryError
before the S_ISREG check ran (Windows os.open rejects the dir first). fstat the
raw fd before fdopen so a directory is rejected with a clean ValueError on every
platform. Local + CI-equivalent test_directory_rejected now green.
#261 iterative_dpo._default_train_fn rendered output as a {dir: ...} mapping
that SoupConfig rejected; render it flat so the spawned soup train succeeds.
#246 CMA-ES merge now loads the base model once and reuses it across the
candidate population (_CachedBaseScorer) instead of reloading per candidate.
#245 soup loop estimate_cost wires run_cost.estimate_run_cost_usd off the last
completed run instead of a 0.0 placeholder; never crashes the daemon.
#244 soup train --track-energy --energy-out persists the measurement JSON so
soup bom emit --energy can attach it to an ML-BOM.
#170 --diagnose-gate is RANK-aware: gate once per cluster (RANK==0), not per node.
Tests: 13476 -> 13511 (+35 in tests/test_v07115.py). Validated end-to-end on
SmolLM2-135M / RTX 3050.
PR #256 attached energy only to CycloneDX; #244's contract is "both
outputs". Add _energy_annotations() so the SPDX model package carries
energy as OTHER annotations (same soup:<field>=<value> naming). Strengthen
the happy-path test to assert energy actually lands in the BOM (not just
that a file is written) and add a --format both case asserting energy in
BOTH cdx + spdx.
Closes#244
Adds --energy flag to soup bom emit: validates the measurement JSON (cwd containment + symlink rejection + JSON + EnergyMeasurement shape) and attaches energy properties to both CycloneDX and SPDX outputs.
Co-authored-by: gittihub-jpg <gittihub-jpg@users.noreply.github.com>
Soup spans the full post-training stack — SFT + preference/RL
(DPO/GRPO/PPO/KTO/ORPO/SimPO/IPO/BCO) + distillation + unlearning +
knowledge-edit + steering + RAFT/RA-DIT — not just fine-tuning. "Fine-tune"
alone undersells the RL/alignment surface and the post-training category.
Update the canonical tagline to "Fine-tune and post-train LLMs in one
command. No SSH, no config hell." across the five places it appears:
README hero, PyPI description (pyproject), `soup --help` epilog + callback
docstring (cli.py), package docstring (__init__), and the auto-generated
HF model card (push.py). Searchable "fine-tune" keyword kept as the verb;
"post-train" added as the category claim.
Copy-only; no version bump (description ships with next release).
The topic pages still described both features as deferred stubs:
- FSDP consolidation showed the removed `--yes` flag and "lands in v0.44.1".
- KV-cache only documented the v0.53.0 schema ("once the runtime serve
path lands").
Both went live in v0.71.14. Update performance-and-quantization.md with
the live `soup merge-sharded-fsdp-weights` (streaming load, shape
validation, --plan-only) and `soup serve --kv-cache-type` (transformers
bf16/f16/q8_0/fp8, hqq advisory, Hopper gate, #140 vLLM/SGLang note), and
cross-link a short KV-cache subsection from serving-and-export.md.
Docs-only; no version bump (v0.71.14 already tagged).
quantized_cache_backend_available() called importlib.util.find_spec
("optimum.quanto"), which imports the parent `optimum` package to resolve
the submodule and raises ModuleNotFoundError when optimum is not installed
(instead of returning None). CI (no optimum) hit this on every test cell;
the live `soup serve --kv-cache-type q8_0` advisory path would crash the
same way on any box without optimum. Wrap submodule probes in a _spec_exists
helper that treats ModuleNotFoundError/ValueError as "not available".
CI failed on the HF-rate-limited runners: test_anchor_term_with_file did a
live from_pretrained that 429'd, so it failed AND its unique MiniLLM-anchor
lines went uncovered, tipping the 77% gate to 76.77% on exactly those jobs
(macos + 3.11 stayed green where the cache warmed).
- Skip test_anchor_term_with_file on OSError (offline / rate-limited) instead
of failing.
- Add test_anchor_term_with_fake_model: a fake tokenizer + tiny nn.Module
exercise the identical _load_anchor + anchor_term lines with no network, so
coverage no longer depends on HF availability.
- Add TestReachableInternals cushion (prompt_compile._resolve_metric,
prompt_distill._build_provider_fn + default-provider wiring) so the gate
sits comfortably above 77% (the DSPy/TextGrad/GEPA optimiser bodies are
uncoverable without the [compile] extra).
Tests 13424 -> 13430.
Lift the v0.68.0 deferred-stub family to live (closes#225, #226, #227, #229):
- #229 local-rl train --once: harvest thumbs -> DPO/KTO/ORPO train via a
soup train subprocess (argv list, no shell); state table tracks last_train_at
(skip-on-no-new-thumbs + skip-on-insufficient-pairs); no --once renders a
systemd/launchd nightly scheduler scaffold. New local_rl_scheduler.py.
- #226 distill-prompt: call the teacher once per trace (Ollama/Anthropic/vLLM)
and write a real dataset (sft/kl -> messages; preference -> chosen/rejected).
- #225 compile / #227 compile-tools: live DSPy/GEPA/TextGrad dispatch behind the
new [compile] extra with a friendly ImportError when absent; injectable seams.
Security: reject \n/\r in the model id + shell-quote ExecStart args (systemd
injection defence). Fix: render train output as a plain string (schema-valid),
with a regression test against SoupConfig.
Tests 13329 -> 13424. Smoked end-to-end: real DPO train on SmolLM2-135M (RTX 3050)
+ real Ollama teacher distillation.
The v0.71.12 release commit updated the test count but missed the two new
modules in the Project Structure tree: trainer/mole_routing.py (MoLE Routing)
and utils/mod.py (Mixture-of-Depths). Docs-only; no version bump.
The test job loads sshleifer/tiny-gpt2 and hf-internal-testing/tiny-random-gpt2
from HF Hub. Under transient HF rate-limiting (429) a single matrix cell would
fail the model download (test_v07111.py::TestMiniLLM::test_anchor_term_with_file)
and drop coverage under the 77% gate — observed reding 3 commits today (~2/7
cells per run).
Add a best-effort pre-test step that snapshot_downloads both tiny models with
6 retries + backoff so the tests read from the warmed cache (verified locally:
the test passes with HF_HUB_OFFLINE=1 once the cache is warm). The step never
fails the job — it emits a :⚠️: if warming can't complete.
CI-only; no version bump.
Alpamys
Salil M
Kondamwar Akshaya Shrikant
Shatakshi Prasad
gittihub-jpg