mirror of https://github.com/razor-ai/soup.git
4 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Alpamys
|
4fb25404f9 |
feat(streaming): preference losses over layer streaming (v0.72.4)
DPO / ORPO / SimPO / KTO join task=sft on the layer-streaming engine. The reference model is the SAME streamed base with adapters disabled — one set of weights, one stream. Measured: streamed DPO peaks at 0.914x streamed SFT with a byte-identical store and pool, where forcing a real second instance costs +730.44 MB against 730.44 MB of weights. All four are bit-exact (0.0) against a resident run of the same loss. KTO is NOT reference-free: kto_trainer.py:466-476 is byte-for-byte DPO's three-branch reference selection, so it was gated separately. It also requires batch_size >= 2, refused at parse time rather than minutes into sharding. grpo/ppo stay excluded permanently — rollouts re-read every layer per generated token. The refusal deliberately names no release. The ~390-line streaming setup moved verbatim into trainer/stream_setup.py so five wrappers cannot drift. _STREAM_ROWS_PER_EXAMPLE is 2 for the concatenating losses and 1 for KTO, measured not assumed: the VRAM pre-flight would otherwise under-predict by half, and on Windows that is a silent spill, not an error. Honest cost: the reference is free in memory, not in time — DPO reads the layer stack 1.52x as often per step as SFT. Also closes five holes in the release checklist itself: - benchmarks/ was never in it, so gate records (which live under a gitignored .claude/) were never published. The public record behind the preprint DOI was about to fall a release behind; benchmarks/gate-v0.72.4-preference-losses.md and its index row are here, and the checklist now names the step. - tests/test_version_sync.py asserts pyproject.toml == __init__.py. Every other version test in the suite is a >= floor check, so bumping one and forgetting the other kept CI green. Verified red-green. - .claude/paper/ (the DOI preprint) had no "did this release change what it claims?" step. For v0.72.4 the answer is no: no measured number moves and its task: sft configs stay valid. - The Docs section header said steps 7-12 while containing 7-13, so plan.md sat outside its own section. - The README size anchor said ~238 lines against a real 426. Notes for whoever hits these next: - `pre-commit run --all-files` rewrites ~740 files here (ruff-format on pre-existing code). No pre-commit git hook is installed and CI runs only `ruff check`, so run it with --files on your own paths or the diff explodes. - Measuring streamed peak VRAM across setup() charges the pre-flight's own GEMM probe (three 4096^3 matrices, ~100 MB) to the step. Reset the peak counter after setup. - The buffer pool is freed by cycle collection, not by close(): back-to-back streamed runs in one process retain the previous pool (+47.65 MB measured) until a gc pass. Call gc.collect() between arms when measuring. - A resident model built from a float32 fixture vs a bf16 streamed one measures the dtype gap, not streaming — that cost an hour chasing a 9.96e-04 "failure". - Two concurrent pytest runs on a 4 GB card produce false CUDA failures; run the suite alone. Tests: 16977 -> 17051. |
|
Alpamys
|
bcbf72e586 |
feat(train): layer streaming breadth — 6 more archs, bigger batches, resume, disk tier (v0.72.3)
Lifts the v0.72.0-.2 scope freeze. Every capability was gated against a streamed-vs-resident bit-exactness reference before it was written. - Six more families (mistral/gemma/gemma2/gemma3_text/phi/phi3), each bit-exact vs the same checkpoint loaded resident, under bf16 AND NF4. Multimodal gemma3 stays refused — only gemma3_text. - batch_size > 1 and gradient_accumulation_steps > 1 now work. - A batch- and vocab-aware VRAM pre-flight that refuses a run predicted not to fit. Fitted to 10 real runs: worst error 0.85%, never under-predicts. On Windows an over-budget step does not OOM; WDDM spills silently, so the estimator is the only guard. - A throughput bracket from a GEMM ceiling measured on the user's own card in the same session, printed with the SM clock. - --resume / --hf-resume: load_state_dict narrows keys by child name, so a canonical checkpoint matched 0 of N tensors and PEFT warned only. Keys are now redirected at load time, mirroring the v0.72.1 save fix. - An NVMe disk overflow tier (stream_source: auto|ram|disk), bit-exact against the RAM tier. Its speed relative to RAM is UNMEASURED here and no figure is claimed. - soup doctor --disk reports the detected media type. Fixes: estimate_logits_bytes charged 6 bytes/element where the measured peak is 14; the NVMe tier guard was wired to a hardcoded constant; streaming sources leaked handles when training raised; subprocess helpers resolved tools by bare name (CWE-427 on Windows). 112 tests in tests/test_v07203.py; 16867 -> 16977. |
|
|
Alpamys
|
08343f8f35 |
feat(train): NF4 layer streaming — fine-tune Llama-3.1-8B on a 4 GB card (v0.72.2)
Layer streaming (v0.72.0) was bf16-only, capping it near 3B on a small card. Quantising the streamed base to NF4 makes the RAM store ~4x smaller, which is what brings 8B within reach. Measured on a 4 GB RTX 3050 Laptop through the shipped code (50 steps after 10 warm-up, batch 1, S=512, PagedAdamW8bit, GEMM ceiling taken in the same session): Llama-3.1-8B-Instruct 119.6 tok/s 3.32 GB peak 3.60 GB pinned 100% 952 MHz Qwen2.5-3B 264.2 tok/s 1.76 GB peak 1.43 GB pinned 100% 960 MHz 3B is 1.85x the bf16 path, but that is PINNING, not arithmetic: 1.43 GB page-locks where 5.55 GB did not, restoring async copy_ (util 79.3% -> 100%). 14B was not run — its store exceeds this box's measured 7.12 GB pinned ceiling. A streamed NF4 run is bit-exact against a RESIDENT NF4 run, now as CPU-runnable CI tests rather than only a gate result. Notable, because each fails silently: - PEFT dispatches lora.bnb.Linear4bit only when is_loaded_in_4bit is stamped; without it the generic lora.layer.Linear runs against a Linear4bit base and casts differently (9.375e-01 logit divergence, no warning). Pinned by a test with a control that deletes the marker. - hf_quantizer must be stamped too, or Trainer.__init__ dies formatting its own "cannot fine-tune" error. Found by the end-to-end test. - The shard cache is keyed on quant/double_quant/quant_device as well as dtype and source fingerprint; a bf16 cache reused for an NF4 request would feed full-precision bytes to matmul_4bit. - index.json is a trust boundary: its shape/blocksize reach bnb kernels that do not bounds-check, so from_json validates and the runtime cross-checks the claim against the bytes on disk. - A streamed NF4 model over-reported parameters ~6.5x (878,154,048 vs 134,515,008 for SmolLM2-135M). Display-only; ~52 B at 8B. Scope unchanged and still BETA: RAM tier, sft, Llama/Qwen, batch 1, no accumulation, no resume. quantization values other than none/4bit are refused. Tests: 16752 -> 16840 (+88 in tests/test_v07202.py). Full suite: 16734 passed, 129 skipped. |
|
|
Alpamys
|
af8683b61d |
fix(train): streamed adapters were saved unloadable (v0.72.1)
v0.72.0's layer-streaming wrapper holds the real decoder layer as a child named `inner`, so every saved LoRA adapter key carried an `.inner.` segment. Such a file reloads as ZERO tensors into any normal model: soup merge, soup serve, soup chat and PeftModel.from_pretrained all returned the untuned base while PEFT emitted only a UserWarning. Training was correct; only the artifact was inert. StreamedDecoderLayer.state_dict() now delegates to the wrapped layer at the wrapper's own prefix, so every artifact path -- the final trainer.save_model(), each save_steps checkpoint, and therefore soup adapters, the Registry, merge and serve -- becomes canonical at once. Serialisation-only by design: the forward path is untouched, so v0.72.0's bit-exactness gates remain valid without being re-earned. Also fixes --hf-resume bypassing the streaming resume refusal. The guard tested only --resume, while --hf-resume reaches resume_from through another branch. Pre-fix that combination matched keys by accident; post-fix it would have matched nothing and silently continued training with a freshly initialised adapter -- i.e. the adapter-key fix alone would have made that one path worse. Roadmap renumbered (this release was inserted ahead of NF4): every "lands in vX.Y.Z" refusal corrected -- NF4 v0.72.2, disk tier / more architectures / larger batches / gradient accumulation / checkpoint-resume v0.72.3, preference losses v0.72.4. Found by the v0.72.2 NF4 gate, not by the 159 v0.72.0 tests -- none of them saved an adapter and loaded it back. The new regression test does exactly that, by count, by name and by value, with a negative control that re-mangles the keys and asserts the reload yields zeros (0-of-N loading raises nothing, so a green round-trip without the control proves nothing). Tests: +17 in tests/test_v07201.py (16735 -> 16752). Full suite: 16623 passed, 129 skipped, 4 deselected. Note for maintainers: do not edit soup_cli/__init__.py while a suite is in flight -- a mid-run version bump made test_cli_subprocess::test_version fail spuriously (the subprocess and the imported constant disagreed). |