Visitors arrive from the Show HN headline and the first screen is all prose.
Adds a 14.5s GIF directly under the 8B-on-4GB claim, above "Why Soup?".
Segment 44.0-60.0s of the demo, chosen off the .vtt cue list and confirmed
frame by frame rather than from the brief's estimate: the scene cut sits
between 47 and 48s, so the clip opens on 3.5s of the static "Layer streaming
BETA" pre-flight panel (3.60 GB store across 32 layers, 2 x 113 MB VRAM
buffers, Training started!) and then runs the measurement card up to its
settled 3.32 GB / 119.6 tok/s. Both halves read without sound or context.
Encoding: two-pass palettegen/paletteuse so terminal colours survive, 960px
wide, 10 fps, dither=none, diff_mode=rectangle -> 4.63 MB. Readability was the
binding constraint, so the budget was met by cutting duration (16s -> 14.5s)
and fps (12 -> 10) rather than width; a bayer-dithered cut of the same clip
came to 5.08 MB and was dropped. Panel lines verified legible by opening the
generated GIF, not assumed from the source resolution.
Caption numbers are from benchmarks/gate-v0.72.2-nf4.md line 314 (the
re-measurement through shipped code), matching the claim line above it.
"What's New" untouched. README 440 -> 445 lines.
People arrive from the "Fine-tune an 8B model on a 4 GB laptop GPU" headline and
the phrase is nowhere on the first screen — badges, then "Why Soup?", then a
release-notes block. The reason for the click and the content did not match.
Adds a permanent 4-line claim between the pip install block and "Why Soup?":
119.6 tok/s at 3.32 GB peak on an RTX 3050 Laptop 4 GB (Llama-3.1-8B-Instruct
+ NF4), taken from benchmarks/gate-v0.72.2-nf4.md line 314 — the re-measurement
through the shipped code, not the earlier spike. Marked opt-in and BETA, with
links to the docs page, benchmarks/ and the DOI rather than inline detail.
"What's New" is deliberately untouched: per the release checklist it is a
single-slot block replaced every release, so the claim cannot live there.
README 433 -> 440 lines.
Reverts b060c99 on the funding side. The Stripe checkout is live again and is
the same one the site links from /support, so the two surfaces no longer
disagree about whether Soup accepts money.
The MePlay, Inc. disclosure comes back with it and is not optional: Stripe
charges under that name, not "Soup", so a donor who is not told beforehand sees
a company that appears nowhere else in the project and reasonably reads it as
the wrong checkout.
The hardware paragraph b060c99 added stays. It was the better half of that
commit and it is not in tension with the link - money and a run posted from a
bigger box move the same `help wanted` issues, so both are offered instead of
one replacing the other. Trimmed only where it repeated the sentence above it.
FUNDING.yml returns to `custom:` with `github:` commented out, since the
Sponsors account is still not enrolled and an enabled key there renders nothing.
The badge pointed at `zenodo.org/badge/DOI/....svg`, which intermittently
refuses automated fetchers (my first probe got HTTP 403; the same URL returns a
valid SVG with a browser User-Agent). GitHub renders badges through its camo
proxy, so one refused fetch is enough to leave a broken image cached.
The DOI itself is fine — verified against DataCite rather than Zenodo, so the
check is independent of the service that was failing:
state: findable | 2026 | "Exact Layer Streaming: LoRA Fine-Tuning of an 8B
Model on a 4 GB Laptop GPU"
Switched to shields.io, which every other badge in that row already uses, so
the row is now consistent in height and font as well as more reliable. Kept the
zenodo logo and made the alt text the actual DOI instead of just "DOI".
Docs-only: no version bump, no tag (per the release checklist's docs-only rule).
All nine badge URLs re-checked and returning 200.
DPO / ORPO / SimPO / KTO join task=sft on the layer-streaming engine.
The reference model is the SAME streamed base with adapters disabled — one set
of weights, one stream. Measured: streamed DPO peaks at 0.914x streamed SFT with
a byte-identical store and pool, where forcing a real second instance costs
+730.44 MB against 730.44 MB of weights. All four are bit-exact (0.0) against a
resident run of the same loss.
KTO is NOT reference-free: kto_trainer.py:466-476 is byte-for-byte DPO's
three-branch reference selection, so it was gated separately. It also requires
batch_size >= 2, refused at parse time rather than minutes into sharding.
grpo/ppo stay excluded permanently — rollouts re-read every layer per generated
token. The refusal deliberately names no release.
The ~390-line streaming setup moved verbatim into trainer/stream_setup.py so
five wrappers cannot drift. _STREAM_ROWS_PER_EXAMPLE is 2 for the concatenating
losses and 1 for KTO, measured not assumed: the VRAM pre-flight would otherwise
under-predict by half, and on Windows that is a silent spill, not an error.
Honest cost: the reference is free in memory, not in time — DPO reads the layer
stack 1.52x as often per step as SFT.
Also closes five holes in the release checklist itself:
- benchmarks/ was never in it, so gate records (which live under a gitignored
.claude/) were never published. The public record behind the preprint DOI was
about to fall a release behind; benchmarks/gate-v0.72.4-preference-losses.md
and its index row are here, and the checklist now names the step.
- tests/test_version_sync.py asserts pyproject.toml == __init__.py. Every other
version test in the suite is a >= floor check, so bumping one and forgetting
the other kept CI green. Verified red-green.
- .claude/paper/ (the DOI preprint) had no "did this release change what it
claims?" step. For v0.72.4 the answer is no: no measured number moves and its
task: sft configs stay valid.
- The Docs section header said steps 7-12 while containing 7-13, so plan.md sat
outside its own section.
- The README size anchor said ~238 lines against a real 426.
Notes for whoever hits these next:
- `pre-commit run --all-files` rewrites ~740 files here (ruff-format on
pre-existing code). No pre-commit git hook is installed and CI runs only
`ruff check`, so run it with --files on your own paths or the diff explodes.
- Measuring streamed peak VRAM across setup() charges the pre-flight's own GEMM
probe (three 4096^3 matrices, ~100 MB) to the step. Reset the peak counter
after setup.
- The buffer pool is freed by cycle collection, not by close(): back-to-back
streamed runs in one process retain the previous pool (+47.65 MB measured)
until a gc pass. Call gc.collect() between arms when measuring.
- A resident model built from a float32 fixture vs a bf16 streamed one measures
the dtype gap, not streaming — that cost an hour chasing a 9.96e-04 "failure".
- Two concurrent pytest runs on a 4 GB card produce false CUDA failures; run the
suite alone.
Tests: 16977 -> 17051.
The paper behind the layer-streaming feature is archived on Zenodo with a
DOI, so the README now carries a citation and the measurement records link
back to the work they are evidence for.
Makazhan, A. (2026). Exact Layer Streaming: LoRA Fine-Tuning of an 8B
Model on a 4 GB Laptop GPU. Zenodo. 10.5281/zenodo.21771064
The concept DOI is used throughout rather than the version DOI: it always
resolves to the latest version, so a future revision does not leave stale
citations behind.
The invite added in the previous commit was a default one, which Discord
expires after seven days. It is now in the README, CONTRIBUTING, the Code of
Conduct and `[project.urls]`, and the last of those ships in release metadata
where a dead link cannot be corrected after publish - so it has to be a
permanent invite, not a convenient one.
All six occurrences move together; a half-updated set is worse than the old
link, since the stale copies would be the ones a reader hits first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Discord is added as a fourth community channel, with the boundary stated
rather than left to guesswork: it is for live chat and setup help, while
anything that should still be findable in six months belongs in Issues or
Discussions. A Discord answer helps one person; an issue helps everyone who
hits the same thing, and the repository already routes public traffic that
way.
It lands in the four places a reader actually looks - the README header, the
badge row, the Contact section, and CONTRIBUTING's Community list - plus
`[project.urls]`, which is the one that matters most in practice: most users
arrive from PyPI, whose sidebar previously showed only Homepage, Repository
and Issues. That entry is metadata and takes effect on the next publish.
Both a Code of Conduct that does not name the server and a security policy
that does not exclude it are gaps a public chat channel creates, so the Code
of Conduct now states it applies there, and SECURITY.md says explicitly not to
report vulnerabilities in a public channel.
The single maintainer address becomes two with distinct roles, because one
address doing both jobs cannot be handed over: team@trysoup.dev is the project
address and survives a change of maintainer, while makazanalpamys@gmail.com
stays as the personal fallback. Both are listed everywhere a contact appears -
README, SECURITY.md, CODE_OF_CONDUCT.md - and pyproject's author email, which
PyPI renders as the package contact, moves to the project address.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The README had no contact route at all: the only address in the repository sat
in CODE_OF_CONDUCT.md, which nobody reads looking for one.
It routes public traffic to issues and Discussions first, since a question
answered there helps the next person with the same problem, and reserves the
email for what genuinely cannot be public - security reports, Code of Conduct
matters, press.
The Stripe checkout was registered under a business the project no longer has
any relationship with, so both the link and the paragraph naming that business
are removed rather than relabelled.
The Support section is not deleted, it is reaimed. Cash was never the binding
constraint: the `help wanted` issues are blocked on multi-GPU, 8B+ and Apple
Silicon hardware that a single 4 GB laptop cannot reach, and a run posted from
a bigger box is worth more to those issues than a donation. Stars stay the
cheapest way to help.
FUNDING.yml now points at GitHub Sponsors, which renders nothing until the
account is enrolled - so the button is absent today either way, and enabling
it later needs no further change here.
Lifts the v0.72.0-.2 scope freeze. Every capability was gated against a
streamed-vs-resident bit-exactness reference before it was written.
- Six more families (mistral/gemma/gemma2/gemma3_text/phi/phi3), each
bit-exact vs the same checkpoint loaded resident, under bf16 AND NF4.
Multimodal gemma3 stays refused — only gemma3_text.
- batch_size > 1 and gradient_accumulation_steps > 1 now work.
- A batch- and vocab-aware VRAM pre-flight that refuses a run predicted
not to fit. Fitted to 10 real runs: worst error 0.85%, never
under-predicts. On Windows an over-budget step does not OOM; WDDM
spills silently, so the estimator is the only guard.
- A throughput bracket from a GEMM ceiling measured on the user's own
card in the same session, printed with the SM clock.
- --resume / --hf-resume: load_state_dict narrows keys by child name, so
a canonical checkpoint matched 0 of N tensors and PEFT warned only.
Keys are now redirected at load time, mirroring the v0.72.1 save fix.
- An NVMe disk overflow tier (stream_source: auto|ram|disk), bit-exact
against the RAM tier. Its speed relative to RAM is UNMEASURED here and
no figure is claimed.
- soup doctor --disk reports the detected media type.
Fixes: estimate_logits_bytes charged 6 bytes/element where the measured
peak is 14; the NVMe tier guard was wired to a hardcoded constant;
streaming sources leaked handles when training raised; subprocess
helpers resolved tools by bare name (CWE-427 on Windows).
112 tests in tests/test_v07203.py; 16867 -> 16977.
Layer streaming (v0.72.0) was bf16-only, capping it near 3B on a small card.
Quantising the streamed base to NF4 makes the RAM store ~4x smaller, which is
what brings 8B within reach.
Measured on a 4 GB RTX 3050 Laptop through the shipped code (50 steps after 10
warm-up, batch 1, S=512, PagedAdamW8bit, GEMM ceiling taken in the same session):
Llama-3.1-8B-Instruct 119.6 tok/s 3.32 GB peak 3.60 GB pinned 100% 952 MHz
Qwen2.5-3B 264.2 tok/s 1.76 GB peak 1.43 GB pinned 100% 960 MHz
3B is 1.85x the bf16 path, but that is PINNING, not arithmetic: 1.43 GB
page-locks where 5.55 GB did not, restoring async copy_ (util 79.3% -> 100%).
14B was not run — its store exceeds this box's measured 7.12 GB pinned ceiling.
A streamed NF4 run is bit-exact against a RESIDENT NF4 run, now as CPU-runnable
CI tests rather than only a gate result.
Notable, because each fails silently:
- PEFT dispatches lora.bnb.Linear4bit only when is_loaded_in_4bit is stamped;
without it the generic lora.layer.Linear runs against a Linear4bit base and
casts differently (9.375e-01 logit divergence, no warning). Pinned by a test
with a control that deletes the marker.
- hf_quantizer must be stamped too, or Trainer.__init__ dies formatting its own
"cannot fine-tune" error. Found by the end-to-end test.
- The shard cache is keyed on quant/double_quant/quant_device as well as dtype
and source fingerprint; a bf16 cache reused for an NF4 request would feed
full-precision bytes to matmul_4bit.
- index.json is a trust boundary: its shape/blocksize reach bnb kernels that do
not bounds-check, so from_json validates and the runtime cross-checks the
claim against the bytes on disk.
- A streamed NF4 model over-reported parameters ~6.5x (878,154,048 vs
134,515,008 for SmolLM2-135M). Display-only; ~52 B at 8B.
Scope unchanged and still BETA: RAM tier, sft, Llama/Qwen, batch 1, no
accumulation, no resume. quantization values other than none/4bit are refused.
Tests: 16752 -> 16840 (+88 in tests/test_v07202.py).
Full suite: 16734 passed, 129 skipped.
.github/FUNDING.yml turns on the Sponsor button in the repo sidebar and on
every issue. It points at a Stripe payment link rather than GitHub Sponsors:
the account is not enrolled there (`hasSponsorsListing: false`), so the
previous `github: MakazhanAlpamys` entry rendered nothing. Left commented out
in case that changes.
The README section states the two things a reader needs and might otherwise
get wrong: the payment is one-off with an adjustable amount (the default shown
is not the required amount), and Stripe processes it under the maintainer's
registered business "MePlay, Inc." -- that name, not "Soup", is what appears at
checkout and on the card statement. Not disclosing that reads as a bait and
switch to anyone who checks.
Starring is offered first, since it is the ask that costs nothing and helps
most. The stated use of funds -- GPU time for the hardware-gated work -- links
to the `help wanted` issues, so the claim is checkable rather than a slogan.
Docs-only: no version bump, no PyPI publish.
v0.72.0's layer-streaming wrapper holds the real decoder layer as a child
named `inner`, so every saved LoRA adapter key carried an `.inner.` segment.
Such a file reloads as ZERO tensors into any normal model: soup merge,
soup serve, soup chat and PeftModel.from_pretrained all returned the untuned
base while PEFT emitted only a UserWarning. Training was correct; only the
artifact was inert.
StreamedDecoderLayer.state_dict() now delegates to the wrapped layer at the
wrapper's own prefix, so every artifact path -- the final trainer.save_model(),
each save_steps checkpoint, and therefore soup adapters, the Registry, merge
and serve -- becomes canonical at once. Serialisation-only by design: the
forward path is untouched, so v0.72.0's bit-exactness gates remain valid
without being re-earned.
Also fixes --hf-resume bypassing the streaming resume refusal. The guard
tested only --resume, while --hf-resume reaches resume_from through another
branch. Pre-fix that combination matched keys by accident; post-fix it would
have matched nothing and silently continued training with a freshly
initialised adapter -- i.e. the adapter-key fix alone would have made that
one path worse.
Roadmap renumbered (this release was inserted ahead of NF4): every
"lands in vX.Y.Z" refusal corrected -- NF4 v0.72.2, disk tier / more
architectures / larger batches / gradient accumulation / checkpoint-resume
v0.72.3, preference losses v0.72.4.
Found by the v0.72.2 NF4 gate, not by the 159 v0.72.0 tests -- none of them
saved an adapter and loaded it back. The new regression test does exactly
that, by count, by name and by value, with a negative control that re-mangles
the keys and asserts the reload yields zeros (0-of-N loading raises nothing,
so a green round-trip without the control proves nothing).
Tests: +17 in tests/test_v07201.py (16735 -> 16752).
Full suite: 16623 passed, 129 skipped, 4 deselected.
Note for maintainers: do not edit soup_cli/__init__.py while a suite is in
flight -- a mid-run version bump made test_cli_subprocess::test_version fail
spuriously (the subprocess and the imported constant disagreed).
Two spots the release missed: the README's own docs-index row for
Performance & quantization (docs/README.md's equivalent row was already
updated), and docs/commands.md, which lists config-driven training
features in the same style as LISA and Spectrum.
Docs-only — no version bump.
The frozen base lives in CPU RAM and is streamed into a small pool of
pre-allocated VRAM buffers one decoder layer at a time, so peak VRAM is
bounded by ONE layer instead of the whole model. Only the LoRA adapters,
their gradients and optimizer state stay resident.
Measured on an RTX 3050 Laptop 4 GB (Windows, 16.9 GB RAM), batch 1,
gradient checkpointing on, 50 steps after 10 warm-up:
Qwen2.5-0.5B S=512 978.6 tok/s 91.4% util 1.47 GB peak
Qwen2.5-1.5B S=512 525.0 tok/s 96.8% util 1.82 GB peak
Qwen2.5-1.5B S=1024 487.6 tok/s 96.7% util 2.96 GB peak
Qwen2.5-3B S=512 143.1 tok/s 79.3% util 2.15 GB peak
Qwen2.5-3B trains in 2.15 GB on a 4 GB card where a resident run OOMs.
Honest cost: 1.43x slower than resident, measured at 0.5B — the only
apples-to-apples comparison available on this box, because 1.5B and above
cannot run resident here at all.
Correctness was gated before any src/ code was written: streamed vs
resident logits are bit-exact (max abs diff 0.0), the layer-0 LoRA
gradient is non-zero on all layers, a 100-step loss curve matches
resident exactly, and same-seed runs are identical.
New:
- utils/layer_stream.py pure planner (no top-level torch)
- utils/layer_shard.py per-layer safetensors sharder
- utils/layer_stream_runtime.py buffer pool, RAM source, prefetch, wrapper
- training.stream_layers / stream_source / stream_buffers
Notes for future maintainers:
- transformers' Trainer.__init__ and accelerate's prepare_model BOTH call
model.to(), which raises NotImplementedError on meta parameters. The
streamed layer overrides _apply to pass meta tensors through, and the
model declares hf_device_map. Without either, every run dies at trainer
construction — no unit test that stops at model(input_ids=...) sees it.
- The shard cache is keyed to a fingerprint of the source checkpoint, not
just the model slug: a base retrained in place must re-shard rather than
silently stream stale weights.
- The pre-flight hardware-fit gate models a RESIDENT run, so it is skipped
for streaming — otherwise it refuses exactly the runs this enables.
- expandable_segments:True is silently ignored on Windows; probed, not
claimed.
Scope (every refusal names the release that lifts it): RAM tier, bf16,
task=sft, Llama/Qwen, batch 1, no gradient accumulation, no --resume.
NF4 is v0.72.1; disk tier / bigger batches / accumulation / resume are
v0.72.2. Proof-of-mechanism at 3B — nothing above 3B was measured.
Tests: 16576 -> 16735 (+159 in tests/test_v07200.py)
Turn the reward-hacking detector on the verifier itself: feed empty /
length-padded / repetition / sentinel-spam completions and flag any the
verifier accepts. Loads via the existing load_reward_fn (probes a synth .py
or a builtin); a gold-requiring target with no --references is a hard error,
never a false "robust". Exit 0=robust / 2=gameable / 1=error. Pure, offline,
no schema change, no new deps.
Also corrects the ops-docs Telemetry section (the sender exists but is wired
to nothing — no data is sent). Telemetry flywheel deferred pending a public
privacy policy.
Tests: 16490 -> 16529 (+39). 5 sequential ECC reviews, every finding fixed.
Point `soup reward synth <refs.jsonl> -o reward.py` at reference (gold) outputs and it
infers a deterministic verifier (numeric / json_schema / regex / tool_call), emits a
readable, committable .py reward_fn that rides load_reward_fn's existing .py path (no new
exec surface), and — the moat — REFUSES to emit one that can't discriminate its references
from auto-perturbed negatives via a mandatory calibration report (accept refs >=90% AND
reject negatives; hard floor at discrimination<=0). Nothing in TRL/Unsloth/Axolotl
synthesizes a reward.
Fixes#311: a comma-separated reward_fn ("accuracy,format") now loads as a reward ensemble
(GRPOTrainer reward_funcs=[...], unlocks the rm_ensemble detector), GRPO-only and validated
at config-parse; the deepseek-v3-reasoning recipe that shipped this previously crashed with
"Unknown reward function".
Riders: reward_fn field-validator (null-byte/blank/oversize/empty-comma-segment); comma-aware
verifiable-domain check; envs/calculator + guess_number docstrings corrected.
5 sequential ECC reviews, every finding fixed (python HIGH PPO gate; code 2xHIGH per-tool
arg binding + json_schema mixed-shape refuse; security HIGH rel_hint codegen injection; tdd
8xHIGH). Live smoke on RTX 3050: synth from envs/calculator agrees with math_verify;
degenerate refused (exit 2); real GRPO on SmolLM2-135M with reward_fn=accuracy,format
completed optimizer steps. +103 tests (tests/test_v07140.py); 16387 -> 16490.
soup ship's leg 2 — the catastrophic-forgetting / regression gate that carries
the whole SHIP / DON'T-SHIP claim — was 15 trivia prompts scored by raw
substring containment (it credited "B" for "Berlin", "3" for "13") with zero
coverage for tool-calling, safety, or JSON. This makes the gate real.
- forgetting.py: score_answer/extract_mcq_letter replace the substring scorer
with answer-extraction (cue -> paren -> clause-terminating bare letter) +
boundary-aware token match. MINI_BENCHMARKS expanded (mmlu 26 / common_sense
24 / instruction 24) + new mini_arithmetic (36) so a 1-item flip trips 0.05.
BREAKING: an existing run's verdict can change (the old gate under-reported).
- eval/gate_suites.py (new): bundled offline general-suite registry, no torch.
DEFAULT_GENERAL_SUITE = the 4 MCQ suites + 3 behavioural JSONL suites
(mini_tool_call / mini_format_json / mini_safety) scored per-model-absolute
by the pure custom/diagnose scorers. _fraction_passing isolates a per-item
scorer exception (deep-JSON RecursionError scores as a failed item).
- ship.py: leg-2 scores bundled suites offline (base+tuned) before routing any
non-bundled name to lm-eval; default general suite = the full bundled set.
Exit-code taxonomy: usage errors move 2 -> 3 so exit 2 means only DON'T-SHIP
(a typo'd flag was previously indistinguishable from a caught regression).
- diagnose/__init__: "Six" -> "Seven" probes + re-export all 7 score_* fns;
removed the dead SUPPORTED_TASK_MODES "pairwise reserved" gate.
- Bundled gate fixtures ship in the wheel via the pyproject artifacts glob.
Every bundled item is original, hand-authored (no MMLU/GSM8K rows copied).
Test count 16288 -> 16330 (+42 in tests/test_v07138.py).
Every printed and documented `pip install 'soup-cli[extra]'` was bash / zsh /
PowerShell syntax and failed on Windows cmd.exe:
ERROR: Invalid requirement: "'soup-cli[train]'": Expected package name at
the start of dependency specifier
cmd.exe has no single-quote quoting, so it passes the quotes to pip verbatim
and pip rejects the requirement. Nothing in Soup can fix that once the command
is typed -- pip and the shell own it, and Soup is not installed yet when the
README line runs -- so the fix is the spelling we print.
Migrated 147 sites across 67 files to `pip install "soup-cli[extra]"`:
- 64 in src/ (Rich console hints + plain ImportError text)
- 57 in README.md + docs/
- 22 in src/soup_cli/templates/*.yaml + examples/configs/*.yaml
- 3 in examples/README.md
Double quotes are the only spelling valid in every shell (cmd, PowerShell,
bash, zsh), which is why the repo already used `pip install -e ".[dev]"`.
Measured on Windows: single quotes fail ONLY on cmd; double quotes pass
everywhere; bare passes on Windows but zsh globs `[extra]` and fails.
Method note (the PR #247 class): the hints sit INSIDE double-quoted Python
string literals, so a blind ' -> " sed produces SyntaxError. A tokenize-based
rewriter escaped `\"` in DQUOTE tokens and left bare `"` in TRIPLE / COMMENT
tokens; every touched .py was compile-checked. The full suite (not ruff, not
compile-check) caught two rewriter blind spots: the real YAML templates under
src/soup_cli/templates/ (byte-identical drift test) and examples/README.md.
A regression test (tests/test_v07137.py) scans the package and every docs code
block for the single-quoted form; prose may still name it so a reader from an
older tutorial recognises the error.
Also bundles #315 (@Sanjays2402): eval-gate benchmark tasks now run via
ForgettingDetector instead of a helper that never existed. Closes#310.
Test count: 16283 -> 16288 (+4 in tests/test_v07137.py).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`pip install 'soup-cli[train]'` fails on Windows cmd.exe: cmd has no
single-quote quoting, so the quotes reach pip verbatim and it rejects the
requirement with
ERROR: Invalid requirement: "'soup-cli[train]'"
Nothing in Soup can fix that -- pip and cmd own the command, and Soup is not
installed yet when it runs. The only lever we have is what we tell people to
type, and existing videos/site copy already show the single-quoted form, so
document the escape hatch where someone who googles the error will land.
Verified on this box rather than assumed:
cmd.exe 'soup-cli[train]' -> ERROR "soup-cli[train]" -> ok bare -> ok
PowerShell 'soup-cli[train]' -> ok "soup-cli[train]" -> ok
Double quotes work in every shell, which is why the repo already uses
`pip install -e ".[dev]"`. Bare `soup-cli[train]` is not advised: zsh globs
the bracket and fails.
The 96 single-quoted hint sites are untouched here; making Soup's own
printed hints shell-aware is a follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Version 0.71.35 -> 0.71.36 in pyproject.toml + __init__.py.
Docs are written around what the live smoke MEASURED, not what the plan
assumed:
- The semantic-dedup headline is WITHDRAWN. docs/data.md leads with the
measured overlap: paraphrase cosines (0.49-0.76) overlap genuinely-distinct
rows (0.54-0.76), and "Add two numbers"/"Multiply two numbers" (0.759)
scores HIGHER than the true paraphrase "reverse a string"/"invert the order
of characters" (0.491). No threshold separates them, so the page says
plainly that lowering --threshold trades duplicates for silent data loss,
and explains why the 0.8 default is deliberately conservative. The claim it
does make -- catches REWORDINGS MinHash's shingling misses (0.88-0.91) --
is the one the numbers support.
- docs/data.md canary section states the real verdict rule (binomial tail
over the count, not any-single-canary) and why: at K=16 an any-canary rule
fires on a CLEAN model ~15% of the time.
- docs/training.md --replay reports the honest result: 7% better retention
than control, but forgetting without it was only +4% (mild), so it is
proof-of-mechanism at 135M + LoRA, not a production claim.
- CHANGELOG known-limitations carry the same numbers rather than hedging.
- README What's New leads with the two blocking bugs the smoke found (the
hardware-fit gate refusing locally-merged models; the [extra] hints
printing without the extra).
Counts 16001 -> 16254 tests, 313 -> 314 files (CONTRIBUTING).
Docs-only — no src/ changes, so no version bump (per the CI-only/docs-only rule).
Audit of the v0.71.35 release against the checklist found four misses:
* README's docs/ index table was missing the docs/compliance.md row — a new
topic page unreachable from the front door (step 9: "if the release adds a
whole new topic area, add/adjust the row in the README's docs/ index table").
It was only added to docs/README.md.
* The GGUF toolchain was never documented, despite the slot's own plan calling
it out ("Risk: MSVC build variance — document the exact toolchain; keep the
llama.cpp tag pinned"). Added the exact, verified build commands + the
VS2022 component actually required, the single- vs multi-config binary
layouts, and an explicit warning never to install llama.cpp's
requirements.txt (it pins torch~=2.2.1 CPU and downgrades a CUDA torch —
the bug fixed in v0.71.35).
* docs/peft-and-efficiency.md still said "16 built-in templates" — stale as a
direct result of the 4 compliance templates (now 21).
* Stale counts CONTRIBUTING "Test Files (307 files)" -> 313 and "existing 15
templates" -> 21 (that step also still pointed at schema.py rather than the
templates/ YAML + manifest.json registry).
The documented build command and binary path are the ones actually executed
during the release validation, not idealised.
Ship a regulated fine-tune with the paperwork it needs, plus GGUF export that
actually works on Windows.
Added:
* `soup init --template hipaa|soc2|eu-ai-act|sr-11-7` (templates 17 -> 21).
Design constraint verified in-repo, not assumed: Soup's compliance controls
are CLI flags/commands, NOT schema keys (audit_log/bom/attest/repro_receipt/
annex_xi/track_energy/pii/decontaminate have zero matches in config/schema.py),
so a template cannot "pre-wire audit-log on" as YAML. Each is a valid
SoupConfig + header comments naming that regime's exact commands.
* `soup card <registry-id> -o MODELCARD.md` — registry entry -> publishable HF
card (training config, eval scorecard, config/data hashes, lineage, artifacts).
Pure build_model_card() over dicts, reused by `soup push --card`.
* `soup ci init` — writes .github/workflows/soup-gate.yml chaining
data validate -> expect -> ship --evidence (exit 2 blocks the merge).
* docs/compliance.md quickstart.
Fixed (GGUF-on-Windows, validated end-to-end for the first time; closes the
CPU-validatable half of #70/#144). Four independently-fatal bugs:
* export cloned llama.cpp into the CURRENT directory: SOUP_DIR is the bare name
".soup" but was used relatively instead of anchored to home like tracker.py /
registry/store.py, so ~/.soup/llama.cpp was never found.
* the first GGUF export DOWNGRADED the user's torch and broke CUDA: the
auto-clone pip-installed llama.cpp's requirements.txt (pins torch~=2.2.1 from
the CPU index) into the user's interpreter. Observed live: torch 2.5.1+cu ->
2.2.2+cpu, transformers 4.57 -> 4.46. Now installs only gguf/sentencepiece/
protobuf, unpinned, non-fatally.
* a correctly-built llama.cpp was not found on Windows: MSVC (like Xcode) is a
multi-config generator emitting build/bin/Release/llama-quantize.exe.
* `soup deploy ollama` failed on a relative GGUF path ("pull model manifest:
file does not exist") — ollama resolves FROM against the Modelfile's dir and
Soup writes it to a temp dir. Modelfile now emits an absolute path.
Also fixed a pre-existing model-card injection hole (affects `soup push`'s own
auto-card): _render_training_section interpolated base/task/scheduler/recipe
unescaped, and SoupConfig.base/scheduler have no charset validator, so a
crafted-but-valid config could smuggle raw HTML or a code-span-breaking backtick
into a card published to the Hub.
Step-6 live smoke (real train -> registry push -> card) caught a bug 90 green
tests missed: is_adapter came only from registry artifacts, so a real LoRA run
with no artifacts rendered "Full model" + library_name: transformers — a false
claim in a provenance document.
5 sequential reviews, every finding fixed (code HIGH: the generated workflow ran
`pip install -e ".[dev]"`, which only works in the Soup source tree, breaking the
gate's first step for every downstream user; security HIGH/MEDIUM/3 LOW; tdd HIGH:
push --card was only --help-tested). All 5 new fixes mutation-verified as real
pins. Tests 15906 -> 16001 (+95); full suite 15872 passed / 126 skipped.
Version 0.71.32 -> 0.71.33. CHANGELOG (with a Known-limitations block), README
What's New, docs/commands.md, docs/serving-and-export.md, CLAUDE.md (arch + CLI
+ counts + history roll), CONTRIBUTING counts.
The live smoke changed the pitch. The plan promised '~1.5-2x faster to serve by
distilling its own draft'. Measured on SmolLM2-360M-Instruct <- 135M-Instruct:
the STOCK draft already scored 69.3% acceptance; distilling it gave 69.7% at 2
epochs and 69.3% at 10 -- no gain beyond noise. A small same-family draft is
already at its capacity ceiling vs the target, and logit KD cannot buy capacity
it does not have. Assisted decoding also measured 0.55-0.64x -- a net SLOWDOWN.
So the speedup claim is withdrawn, not shipped. The feature ships as the honest
gate: soup draft measure tells you whether speculative decoding is worth
enabling BEFORE you ship it, and on this pair it correctly says no. That
negative result is stated in the CHANGELOG, the README, the serving docs and
CLAUDE.md, and the pre-existing unverified '2-3x faster' line in the
speculative-decoding docs was tempered to match.
Full suite: 15680 passed / 124 skipped (15806 collected). ruff clean.
Add `soup data doctor` and `soup data lint`, killing the top *silent*
fine-tune failures before a single training step: EOS-missing-from-labels
(the #1 "model never stops generating" bug), BOS duplication, no-system-role
templates, and preference-data length bias (the #1 silent DPO degradation) —
none of which any competitor (Unsloth/Axolotl/LlamaFactory) checks for.
- utils/data_doctor.py: 8-check chat-template compat report over a
tokenizer + sampled rows, OK/MINOR/MAJOR taxonomy mirroring diagnose;
--show-mask N renders per-token trained/masked colouring through the
SAME masking dispatch (_build_row_labels) the report itself uses, so
the two can never disagree about what's actually trained.
- utils/data_lint.py: preference-data linter (dpo/orpo/simpo/ipo/bco/kto)
— length bias (Cohen's d), label imbalance, near-duplicates (MinHash),
identical chosen==rejected pairs, prompt leakage.
- commands/data_doctor.py: Typer layer for both commands; strips C0
control bytes before untrusted dataset content reaches the terminal.
- commands/diagnose.py: hardens the --evidence loader against a TOCTOU
symlink swap (O_NOFOLLOW + fstat-on-open-fd), backporting the pattern
soup ship shipped in v0.71.25 (closes v0.71.25 known-limitation (4)).
Live smoke against the real HuggingFaceTB/SmolLM2-135M-Instruct tokenizer
(Windows + RTX 3050) found and fixed two genuine bugs beyond the synthetic
fixtures: the EOS check needed to span-search the whole trained region
(not just the last token), and two apply_chat_template call sites needed
a broad except Exception for jinja2.exceptions.TemplateError.
+173 tests (14788 -> 15042). 5 sequential ECC reviews, every finding fixed.
#81 Quant Menu for vision/audio modality
- config/schema: drop the `modality != "text"` rejection in
_validate_quant_menu_supported_tasks (mlx-backend gate retained) so the full
Quant Menu (gptq/awq/hqq:Nbit/aqlm/eetq/mxfp4/fp8) applies to vision+audio.
- trainer/sft: _setup_vision_transformers + _setup_audio_transformers call
build_quantization_config_for_loader (strict superset of the inline 4bit/8bit
BNB blocks they replaced); drop BitsAndBytesConfig import; retain the
prepare_model_for_kbit_training gate on (4bit,8bit,mxfp4).
#80 multipack DataLoader sharding under FSDP/DeepSpeed/DDP
- utils/multipack_trainer: get_train_dataloader routes the multipack DataLoader
through accelerator.prepare when num_processes > 1 so accelerate's
BatchSamplerShard shards whole FFD bins across ranks (even_batches=True avoids
the epoch-boundary collective hang; seed identical across ranks, no `+ rank`).
Single-process path unchanged. Defence-in-depth guard against an unconfigured
MagicMock num_processes.
Tests: +37 in tests/test_v07119.py (13770 -> 13807). ruff clean.
Supersedes the v0.40.5 vision/audio Quant-Menu and v0.40.4 multipack-FSDP
known-limitations. Full multi-GPU validation remains an INFRA-BLOCKED QA item.
#261 iterative_dpo._default_train_fn rendered output as a {dir: ...} mapping
that SoupConfig rejected; render it flat so the spawned soup train succeeds.
#246 CMA-ES merge now loads the base model once and reuses it across the
candidate population (_CachedBaseScorer) instead of reloading per candidate.
#245 soup loop estimate_cost wires run_cost.estimate_run_cost_usd off the last
completed run instead of a 0.0 placeholder; never crashes the daemon.
#244 soup train --track-energy --energy-out persists the measurement JSON so
soup bom emit --energy can attach it to an ML-BOM.
#170 --diagnose-gate is RANK-aware: gate once per cluster (RANK==0), not per node.
Tests: 13476 -> 13511 (+35 in tests/test_v07115.py). Validated end-to-end on
SmolLM2-135M / RTX 3050.
Soup spans the full post-training stack — SFT + preference/RL
(DPO/GRPO/PPO/KTO/ORPO/SimPO/IPO/BCO) + distillation + unlearning +
knowledge-edit + steering + RAFT/RA-DIT — not just fine-tuning. "Fine-tune"
alone undersells the RL/alignment surface and the post-training category.
Update the canonical tagline to "Fine-tune and post-train LLMs in one
command. No SSH, no config hell." across the five places it appears:
README hero, PyPI description (pyproject), `soup --help` epilog + callback
docstring (cli.py), package docstring (__init__), and the auto-generated
HF model card (push.py). Searchable "fine-tune" keyword kept as the verb;
"post-train" added as the category claim.
Copy-only; no version bump (description ships with next release).
Lift the v0.68.0 deferred-stub family to live (closes#225, #226, #227, #229):
- #229 local-rl train --once: harvest thumbs -> DPO/KTO/ORPO train via a
soup train subprocess (argv list, no shell); state table tracks last_train_at
(skip-on-no-new-thumbs + skip-on-insufficient-pairs); no --once renders a
systemd/launchd nightly scheduler scaffold. New local_rl_scheduler.py.
- #226 distill-prompt: call the teacher once per trace (Ollama/Anthropic/vLLM)
and write a real dataset (sft/kl -> messages; preference -> chosen/rejected).
- #225 compile / #227 compile-tools: live DSPy/GEPA/TextGrad dispatch behind the
new [compile] extra with a friendly ImportError when absent; injectable seams.
Security: reject \n/\r in the model id + shell-quote ExecStart args (systemd
injection defence). Fix: render train output as a plain string (schema-valid),
with a regression test against SoupConfig.
Tests 13329 -> 13424. Smoked end-to-end: real DPO train on SmolLM2-135M (RTX 3050)
+ real Ollama teacher distillation.
Alpamys
Alpamys