mirror of https://github.com/razor-ai/soup.git
6cb84e7973
v0.72.4 capped `trl<0.25` from a staged-removal table that scored a MODULE
RELOCATION as a field removal. At 0.25 `BCOConfig` moved into
`trl/experimental/bco/`, and at 0.26 kto/orpo/cpo followed — but every one of
them stayed publicly re-exported from `trl` with `max_prompt_length` intact.
Seeing a config file vanish from `trl/trainer/` was read as the field going
away.
Re-derived by parsing each *Config class's own annotated fields across every
wheel 0.24.0 -> 0.29.1 (all five inherit TrainingArguments, so there is no
inherited-field escape hatch):
version dpo kto orpo cpo bco
0.24.0 - 0.26.2 yes yes yes yes yes
0.27.0 - 0.27.2 yes NO yes yes yes
0.28.0 yes NO NO NO NO
0.29.0 - 0.29.1 NO NO NO NO NO
Then settled the way reading source cannot — by CONSTRUCTING all six configs
with the exact kwargs the wrappers pass:
trl 0.26.2 OK dpo · OK ipo · OK kto · OK orpo · OK simpo · OK bco
trl 0.27.0 FAIL kto: KTOConfig.__init__() got an unexpected keyword
argument 'max_prompt_length'
OK dpo · OK orpo <- control: the boundary is KTO's alone
So the cap is <0.27, and 0.25.0/0.25.1/0.26.0/0.26.1/0.26.2 were excluded for
no reason. That is the third wrong answer this bound has had (after "trl 1.x
removed them" and "the break is 0.29.0"), and the first two were also produced
by reading rather than running. The rule this earns: a version bound derived
from source is a hypothesis; the experiment that settles it is constructing the
object.
Two more corrections from the same pass:
- The floor >=0.7.0 was impossible, and nobody checked it while carefully
fixing the ceiling. `setup()` imports GRPOTrainer unconditionally and trl
first exports it at 0.14.0 (OnlineDPO / KTO / BCO / BasePairwiseJudge at
0.11.0; 0.7.0 has none of them). Resolvers pick the newest allowed version,
so it only bit under a constraints file or anyone reading the metadata as a
support claim.
- ORPOConfig/CPOConfig are not "deleted" at 0.29 — the modules survive under
trl/experimental/. They, BCOConfig and their trainers are dropped from the
public `trl` namespace, which is what Soup imports, so the 0.29 break is an
ImportError rather than a rejected kwarg. Harder than described, not milder,
and it means going past 0.29 needs an import strategy (#326).
Closes the blind spot that let this ship, for all six trainers
(tests/test_trl_preference_config_contract.py)
The bug was invisible because the trl imports and the config construction live
inside `setup()`, and constructing a wrapper touches trl not at all. v0.72.4's
streaming suite closed that for four of the six; tests/test_bco.py:266 and
tests/test_ipo.py:191 still MOCK `.setup` out, so bco and ipo had no test that
executed it. The new file drives the real `setup()` for all six on the ordinary
non-streaming path, asserts the BUILT config carries max_prompt_length/
max_length (so a trl release that accepts the kwarg and stops storing it fails
too), and derives the covered set from the trainer sources so a seventh trainer
adopting the argument joins automatically.
Revert the KTO skip: it blamed the device, and the variable is the torch version
`skipif(not cuda)` on test_kto_streams_at_batch_two made the test dead in CI —
there are no GPU runners (ubuntu/windows/macos) — leaving it alive only on this
box, under the one torch where it passes, while torch carries no upper bound.
Three things say the rationale ("a streamed model on CPU is a test convenience")
does not hold:
- the same CI run had test_v07200.py::test_one_training_step_actually_runs, the
identical streamed train() for SFT, pass on that CPU runner;
- running this body here with CUDA masked passes (torch 2.5.1);
- the error comes from check_same_device, i.e. an op received a meta
placeholder next to a real tensor — a streaming property. Newer torch
decomposes more ops, which is why only the newer stack surfaces it.
Now it runs everywhere and xfails on exactly that signature on CPU; anything
else, and the same signature on CUDA, is a hard failure. Real defect filed as
#328. With CUDA hidden: 21 passed, 2 skipped (both genuinely CUDA-only).
Also fixed
- pyproject.toml: 14 double-encoded em-dashes, one of them the `unit` marker
description that `pytest --markers` prints to users. The existing mojibake
guard covered only the package docstring; widened to the file, which is not
importable Python and so was invisible to every source-level check.
- docs/commands.md called itself "the full soup command list" while missing
eight, three of them glued onto the end of a previous line by absent newlines
and therefore invisible on the rendered page. Now 77/77, asserted against the
live Typer app so the claim stays true by construction.
Verified: ruff clean; full suite 16935 passed / 129 skipped / 0 failed (13:15);
wheel builds with Requires-Dist: trl<0.27,>=0.14.0; `pytest --markers` renders
the em-dash. No version bump and no tag — the packaging metadata change wants
CI on the resolver stack before it rides a release.
|
||
|---|---|---|
| .. | ||
| README.md | ||
| gate-v0.72.0-layer-streaming.md | ||
| gate-v0.72.2-nf4.md | ||
| gate-v0.72.3-breadth.md | ||
| gate-v0.72.4-preference-losses.md | ||
README.md
Measurement records
Raw gate records for Soup's layer-streaming feature, published as written.
These are not a report assembled after the fact. They are the working records kept while each item was built and verified, so they contain the failures, the assumptions that turned out wrong, and the numbers that were measured and then discarded — in the order those things happened.
They are the evidence behind the preprint:
Makazhan, A. (2026). Exact Layer Streaming: LoRA Fine-Tuning of an 8B Model on a 4 GB Laptop GPU. Zenodo. 10.5281/zenodo.21771064
| File | What it gates | Headline |
|---|---|---|
gate-v0.72.0-layer-streaming.md |
The streaming path itself | Bit-exactness vs a resident reference; 3B bf16 trained on a 4 GB card |
gate-v0.72.2-nf4.md |
NF4 quantised streaming | Llama-3.1-8B at 119.6 tok/s in a 3.32 GB peak |
gate-v0.72.3-breadth.md |
Nine architectures, batching, accumulation, resume, disk tier | Peak-VRAM predictor at 0.85% worst-case error; accumulation is per-token I/O-neutral |
gate-v0.72.4-preference-losses.md |
DPO / ORPO / SimPO / KTO over the streaming engine | DPO's reference model costs no extra weights — 0.914x the SFT peak, against +730.44 MB for a real second instance |
Hardware
Every number was measured on one machine:
- GPU — RTX 3050 Laptop, 4 GB (4.29 GB usable)
- Host — 16.9 GB RAM, NVMe
- OS — Windows 11
Windows/WDDM matters for reading these: it spills into shared host memory rather
than raising CUDA out of memory, so a run completing is not evidence that its
configuration fits. That is why peak VRAM is reported alongside every throughput
figure, and why the fit decision refuses rather than warns.
Reading the numbers
- Throughput is quoted with the SM clock it was taken at. This card's boost clock varies about 13% between sessions, so a fraction-of-ceiling stated without its clock is not meaningful. Where a GEMM ceiling is compared against, it was measured in the same session.
- The correctness reference always matches the numerics under test — a streamed NF4 run is compared against a resident NF4 run, never against resident bf16, which would hide a real defect inside quantisation error.
- Derived figures are labelled as arithmetic. Where a line says "1M tokens = 2.3 h", that is division, not a measured wall-clock run.
Reproducing
The implementation ships in Soup under Apache-2.0. Reproduction commands are in Appendix A of the paper; the correctness protocol runs as part of the project's test suite, so a regression in bit-exactness fails CI rather than reaching a user.
pip install 'soup-cli[train]'