mirror of https://github.com/razor-ai/soup.git
4fb25404f9
DPO / ORPO / SimPO / KTO join task=sft on the layer-streaming engine. The reference model is the SAME streamed base with adapters disabled — one set of weights, one stream. Measured: streamed DPO peaks at 0.914x streamed SFT with a byte-identical store and pool, where forcing a real second instance costs +730.44 MB against 730.44 MB of weights. All four are bit-exact (0.0) against a resident run of the same loss. KTO is NOT reference-free: kto_trainer.py:466-476 is byte-for-byte DPO's three-branch reference selection, so it was gated separately. It also requires batch_size >= 2, refused at parse time rather than minutes into sharding. grpo/ppo stay excluded permanently — rollouts re-read every layer per generated token. The refusal deliberately names no release. The ~390-line streaming setup moved verbatim into trainer/stream_setup.py so five wrappers cannot drift. _STREAM_ROWS_PER_EXAMPLE is 2 for the concatenating losses and 1 for KTO, measured not assumed: the VRAM pre-flight would otherwise under-predict by half, and on Windows that is a silent spill, not an error. Honest cost: the reference is free in memory, not in time — DPO reads the layer stack 1.52x as often per step as SFT. Also closes five holes in the release checklist itself: - benchmarks/ was never in it, so gate records (which live under a gitignored .claude/) were never published. The public record behind the preprint DOI was about to fall a release behind; benchmarks/gate-v0.72.4-preference-losses.md and its index row are here, and the checklist now names the step. - tests/test_version_sync.py asserts pyproject.toml == __init__.py. Every other version test in the suite is a >= floor check, so bumping one and forgetting the other kept CI green. Verified red-green. - .claude/paper/ (the DOI preprint) had no "did this release change what it claims?" step. For v0.72.4 the answer is no: no measured number moves and its task: sft configs stay valid. - The Docs section header said steps 7-12 while containing 7-13, so plan.md sat outside its own section. - The README size anchor said ~238 lines against a real 426. Notes for whoever hits these next: - `pre-commit run --all-files` rewrites ~740 files here (ruff-format on pre-existing code). No pre-commit git hook is installed and CI runs only `ruff check`, so run it with --files on your own paths or the diff explodes. - Measuring streamed peak VRAM across setup() charges the pre-flight's own GEMM probe (three 4096^3 matrices, ~100 MB) to the step. Reset the peak counter after setup. - The buffer pool is freed by cycle collection, not by close(): back-to-back streamed runs in one process retain the previous pool (+47.65 MB measured) until a gc pass. Call gc.collect() between arms when measuring. - A resident model built from a float32 fixture vs a bf16 streamed one measures the dtype gap, not streaming — that cost an hour chasing a 9.96e-04 "failure". - Two concurrent pytest runs on a 4 GB card produce false CUDA failures; run the suite alone. Tests: 16977 -> 17051. |
||
|---|---|---|
| .. | ||
| README.md | ||
| adapters-and-governance.md | ||
| backends-and-ops.md | ||
| commands.md | ||
| compliance.md | ||
| data.md | ||
| evaluation.md | ||
| models.md | ||
| peft-and-efficiency.md | ||
| performance-and-quantization.md | ||
| serving-and-export.md | ||
| training.md | ||
README.md
Soup Documentation
The main README is the 5-minute front door. This directory holds the full
feature reference — every soup capability, grouped by area.
| Guide | Covers |
|---|---|
| Training tasks & methods | SFT, DPO/GRPO/PPO/KTO/ORPO/SimPO/IPO/BCO, tool-calling, PRM, pre-training, distillation, classification, vision/audio/TTS, unlearning, RAFT/RA-DIT, loop-hardening detectors, reward-verifier synthesis |
| PEFT, long context & efficiency | DoRA, LoRA+, rsLoRA, VeRA, OLoRA, NEFTune, PiSSA, ReLoRA, optimizer & PEFT zoo, LLaMA Pro, GaLore, YaRN/LongLoRA, packing, curriculum, auto-tuning, depth pruning + distill-heal (soup shrink) |
| Performance & quantization | QAT, FP8, Quant Menu (I + II), KV-cache, NVFP4, save formats, Cut Cross-Entropy, gradient checkpointing, kernels, activation offloading, layer streaming, multi-GPU / DeepSpeed / FSDP |
| Data engineering | Formats, the Axolotl/LF-parity pipeline, data tools, synthetic generation & forge, quality scorecards, trace tooling, remote datasets, mixing, recipe DAGs |
| Evaluation & probes | Eval design/gate, eval-gated training, benchmarks, NLG metrics, calibration, Elo arena, diagnose, soup ship verdict, post-train X-ray probes, A/B, drift, tunability, soup advise |
| Serving & export | OpenAI-compatible server, batch inference, benchmarking, merge/export, Anthropic Messages endpoint, speculative decoding (train + measure your own draft), deploy autopilot, Web UI, Agent Forge |
| Adapters, registry & governance | Adapter lifecycle/management, model registry, Soup Cans, the data flywheel (soup loop), knowledge editing, steering, supply-chain controls |
| Compliance & governance quickstart | HIPAA/SOC2/EU-AI-Act/SR-11-7 init templates, provenance (BOM/attest/repro-receipt), audit log, air-gap, model-card autogen (soup card), CI gate (soup ci init) |
| Backends, platform & ops | MLX/Unsloth backends, Modal cloud GPU training, alternative hubs, HF Hub integration, autopilot, experiment tracking, plan/apply, env lockfiles, hardware-fit, completions, plugins, utility commands |
| Command reference | The full soup command list |
| Supported models & extras | Recommended model families, the VRAM size guide, the pip extras matrix |
Per-release notes live on the GitHub Releases page; see also the repo-root CHANGELOG.md.