- performance-and-quantization.md: the Quant Menu multi-trainer note said
"vision / audio modality is still SFT-only inline-BNB (wiring tracked as a
follow-up)" — stale after v0.71.19 #81 dropped the modality gate. Now states
vision/audio thread the unified loader (full gptq/awq/hqq/aqlm/eetq/mxfp4/fp8
menu), with the upstream class+kernel caveat.
- peft-and-efficiency.md: added the v0.71.19 #80 multi-GPU sharding paragraph to
the Multipack section (accelerator.prepare + BatchSamplerShard under
num_processes>1; identical bin seed across ranks; single-GPU unchanged).
Docs-only — no version bump / tag (the v0.71.19 code already shipped at f51331d).
#261 iterative_dpo._default_train_fn rendered output as a {dir: ...} mapping
that SoupConfig rejected; render it flat so the spawned soup train succeeds.
#246 CMA-ES merge now loads the base model once and reuses it across the
candidate population (_CachedBaseScorer) instead of reloading per candidate.
#245 soup loop estimate_cost wires run_cost.estimate_run_cost_usd off the last
completed run instead of a 0.0 placeholder; never crashes the daemon.
#244 soup train --track-energy --energy-out persists the measurement JSON so
soup bom emit --energy can attach it to an ML-BOM.
#170 --diagnose-gate is RANK-aware: gate once per cluster (RANK==0), not per node.
Tests: 13476 -> 13511 (+35 in tests/test_v07115.py). Validated end-to-end on
SmolLM2-135M / RTX 3050.
The topic pages still described both features as deferred stubs:
- FSDP consolidation showed the removed `--yes` flag and "lands in v0.44.1".
- KV-cache only documented the v0.53.0 schema ("once the runtime serve
path lands").
Both went live in v0.71.14. Update performance-and-quantization.md with
the live `soup merge-sharded-fsdp-weights` (streaming load, shape
validation, --plan-only) and `soup serve --kv-cache-type` (transformers
bf16/f16/q8_0/fp8, hqq advisory, Hopper gate, #140 vLLM/SGLang note), and
cross-link a short KV-cache subsection from serving-and-export.md.
Docs-only; no version bump (v0.71.14 already tagged).
Lift the v0.68.0 deferred-stub family to live (closes#225, #226, #227, #229):
- #229 local-rl train --once: harvest thumbs -> DPO/KTO/ORPO train via a
soup train subprocess (argv list, no shell); state table tracks last_train_at
(skip-on-no-new-thumbs + skip-on-insufficient-pairs); no --once renders a
systemd/launchd nightly scheduler scaffold. New local_rl_scheduler.py.
- #226 distill-prompt: call the teacher once per trace (Ollama/Anthropic/vLLM)
and write a real dataset (sft/kl -> messages; preference -> chosen/rejected).
- #225 compile / #227 compile-tools: live DSPy/GEPA/TextGrad dispatch behind the
new [compile] extra with a friendly ImportError when absent; injectable seams.
Security: reject \n/\r in the model id + shell-quote ExecStart args (systemd
injection defence). Fix: render train output as a plain string (schema-valid),
with a regression test against SoupConfig.
Tests 13329 -> 13424. Smoked end-to-end: real DPO train on SmolLM2-135M (RTX 3050)
+ real Ollama teacher distillation.
Heavy training stack (torch, transformers, peft, trl, datasets,
bitsandbytes, accelerate) moves out of the core install into a new
[train] optional-dependency extra. `pip install soup-cli` is now a
light CLI + data-tools install with no PyTorch; `pip install
'soup-cli[train]'` adds the training stack.
- pyproject: new [train] + [all] extras; [dev] self-references [train]
so CI (`pip install -e ".[dev]"`) still gets torch. Pins unchanged.
- errors.py: missing torch/transformers/peft/trl/datasets/bitsandbytes/
accelerate now surface a single 'install soup-cli[train]' fix.
- Dockerfile: install soup-cli[train,serve,data,eval] so the GPU image
can still fine-tune.
- README + docs/models.md: split install into light core vs [train].
- CHANGELOG: cut [0.71.0]; bump version 0.70.0 -> 0.71.0.
The repo moved to src-layout and trimmed README into a 238-line front door
with the feature reference under docs/, but several committed files still
referenced bare soup_cli/ paths or linked the gitignored .claude/CLAUDE.md
(which 404s for anyone cloning the public repo).
- docs/: `soup_cli/{plugins,templates,ui/plugins}/...` path refs -> `src/soup_cli/...`
(import statements `from soup_cli...` left unchanged — package name is still soup_cli)
- AGENTS.md: point external agents at public docs/, CONTRIBUTING.md, and the
config schema; note CLAUDE.md is a maintainer-local (gitignored) file
- CONTRIBUTING.md + .github/pull_request_template.md: PR checklist now says
"README.md and the matching page under docs/" (kept in sync); Questions
section links docs/ instead of the gitignored CLAUDE.md
- examples/README.md: fix two broken ../CLAUDE.md links -> config schema source
+ docs/ feature reference
- .gitignore: add root-anchored /_*.py temp-script guard + trailing newline
The README had grown to 5046 lines (195 sections) — roughly one deep-dive per
feature accreted over 70 releases. Split it into a concise front door plus a
public docs/ tree:
- README (5046 -> 238 lines): hero, why, quickstart, config, a Documentation
map, data formats, common commands, models, Docker, requirements, dev.
- docs/*.md: all 185 feature sections preserved verbatim, grouped into 10 themed
guides + an index. Every original line is accounted for (content-conservation
checked); all 235 internal links + anchors verified to resolve.
- un-gitignore docs/ (it was empty); fix a pre-existing dangling
docs/QUANTIZATION.md link; correct the stale `ruff check soup_cli/` ->
`src/soup_cli/` reference in the Development section.
No version bump: docs-only — rides into the 0.71.0 deps-split release.
Train-time support for 7 new quantization formats — close the width gap
with LlamaFactory. Wired into SFT trainer + transformers backend + text
modality; multi-trainer/modality expansion deferred to v0.38.1 (mirrors
v0.27.0 MII / v0.37.0 multipack stub-then-live pattern).
- Part A — GPTQ: quantization='gptq' + gptq_disable_exllama (PEFT triton)
- Part B — AWQ: quantization='awq' + GEMM/GEMV builder
- Part C — HQQ: hqq:1bit..hqq:8bit (no 7bit; not supported upstream)
- Part D — AQLM: locked-2-bit
- Part E — EETQ: locked-8-bit
- Part F — MXFP4 + FP8 dequantize-on-load
- Part G — bnb_4bit_quant_storage for FSDP+QLoRA
("crucial for fsdp+qlora" — LlamaFactory quantization.py:178)
- Part H — check_quant_distributed_compat matrix + docs/QUANTIZATION.md.
HQQ/EETQ/AQLM x {FSDP, ZeRO-3} hard-fail; BNB-4bit + FSDP without
quant_storage warns. Wired into commands/train.py startup.
Three new schema validators:
- _validate_prequantized_no_qat — pre-quantized + QAT incompatible
- _validate_bnb_quant_storage_only_with_4bit — silent no-op guard
- _validate_quant_menu_supported_tasks — sft + transformers + text gate
Net: +61 tests (4374 -> 4435). Four review-agent waves clean before tag
(python-review / code-review / security-review / tdd-guide);
verification-loop performed as manual equivalent per CLAUDE.md allowance.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Alpamys
gittihub-jpg