mirror of https://github.com/razor-ai/soup.git
6 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Alpamys
|
4fb25404f9 |
feat(streaming): preference losses over layer streaming (v0.72.4)
DPO / ORPO / SimPO / KTO join task=sft on the layer-streaming engine. The reference model is the SAME streamed base with adapters disabled — one set of weights, one stream. Measured: streamed DPO peaks at 0.914x streamed SFT with a byte-identical store and pool, where forcing a real second instance costs +730.44 MB against 730.44 MB of weights. All four are bit-exact (0.0) against a resident run of the same loss. KTO is NOT reference-free: kto_trainer.py:466-476 is byte-for-byte DPO's three-branch reference selection, so it was gated separately. It also requires batch_size >= 2, refused at parse time rather than minutes into sharding. grpo/ppo stay excluded permanently — rollouts re-read every layer per generated token. The refusal deliberately names no release. The ~390-line streaming setup moved verbatim into trainer/stream_setup.py so five wrappers cannot drift. _STREAM_ROWS_PER_EXAMPLE is 2 for the concatenating losses and 1 for KTO, measured not assumed: the VRAM pre-flight would otherwise under-predict by half, and on Windows that is a silent spill, not an error. Honest cost: the reference is free in memory, not in time — DPO reads the layer stack 1.52x as often per step as SFT. Also closes five holes in the release checklist itself: - benchmarks/ was never in it, so gate records (which live under a gitignored .claude/) were never published. The public record behind the preprint DOI was about to fall a release behind; benchmarks/gate-v0.72.4-preference-losses.md and its index row are here, and the checklist now names the step. - tests/test_version_sync.py asserts pyproject.toml == __init__.py. Every other version test in the suite is a >= floor check, so bumping one and forgetting the other kept CI green. Verified red-green. - .claude/paper/ (the DOI preprint) had no "did this release change what it claims?" step. For v0.72.4 the answer is no: no measured number moves and its task: sft configs stay valid. - The Docs section header said steps 7-12 while containing 7-13, so plan.md sat outside its own section. - The README size anchor said ~238 lines against a real 426. Notes for whoever hits these next: - `pre-commit run --all-files` rewrites ~740 files here (ruff-format on pre-existing code). No pre-commit git hook is installed and CI runs only `ruff check`, so run it with --files on your own paths or the diff explodes. - Measuring streamed peak VRAM across setup() charges the pre-flight's own GEMM probe (three 4096^3 matrices, ~100 MB) to the step. Reset the peak counter after setup. - The buffer pool is freed by cycle collection, not by close(): back-to-back streamed runs in one process retain the previous pool (+47.65 MB measured) until a gc pass. Call gc.collect() between arms when measuring. - A resident model built from a float32 fixture vs a bf16 streamed one measures the dtype gap, not streaming — that cost an hour chasing a 9.96e-04 "failure". - Two concurrent pytest runs on a 4 GB card produce false CUDA failures; run the suite alone. Tests: 16977 -> 17051. |
|
Alpamys
|
bcbf72e586 |
feat(train): layer streaming breadth — 6 more archs, bigger batches, resume, disk tier (v0.72.3)
Lifts the v0.72.0-.2 scope freeze. Every capability was gated against a streamed-vs-resident bit-exactness reference before it was written. - Six more families (mistral/gemma/gemma2/gemma3_text/phi/phi3), each bit-exact vs the same checkpoint loaded resident, under bf16 AND NF4. Multimodal gemma3 stays refused — only gemma3_text. - batch_size > 1 and gradient_accumulation_steps > 1 now work. - A batch- and vocab-aware VRAM pre-flight that refuses a run predicted not to fit. Fitted to 10 real runs: worst error 0.85%, never under-predicts. On Windows an over-budget step does not OOM; WDDM spills silently, so the estimator is the only guard. - A throughput bracket from a GEMM ceiling measured on the user's own card in the same session, printed with the SM clock. - --resume / --hf-resume: load_state_dict narrows keys by child name, so a canonical checkpoint matched 0 of N tensors and PEFT warned only. Keys are now redirected at load time, mirroring the v0.72.1 save fix. - An NVMe disk overflow tier (stream_source: auto|ram|disk), bit-exact against the RAM tier. Its speed relative to RAM is UNMEASURED here and no figure is claimed. - soup doctor --disk reports the detected media type. Fixes: estimate_logits_bytes charged 6 bytes/element where the measured peak is 14; the NVMe tier guard was wired to a hardcoded constant; streaming sources leaked handles when training raised; subprocess helpers resolved tools by bare name (CWE-427 on Windows). 112 tests in tests/test_v07203.py; 16867 -> 16977. |
|
|
Alpamys
|
08343f8f35 |
feat(train): NF4 layer streaming — fine-tune Llama-3.1-8B on a 4 GB card (v0.72.2)
Layer streaming (v0.72.0) was bf16-only, capping it near 3B on a small card. Quantising the streamed base to NF4 makes the RAM store ~4x smaller, which is what brings 8B within reach. Measured on a 4 GB RTX 3050 Laptop through the shipped code (50 steps after 10 warm-up, batch 1, S=512, PagedAdamW8bit, GEMM ceiling taken in the same session): Llama-3.1-8B-Instruct 119.6 tok/s 3.32 GB peak 3.60 GB pinned 100% 952 MHz Qwen2.5-3B 264.2 tok/s 1.76 GB peak 1.43 GB pinned 100% 960 MHz 3B is 1.85x the bf16 path, but that is PINNING, not arithmetic: 1.43 GB page-locks where 5.55 GB did not, restoring async copy_ (util 79.3% -> 100%). 14B was not run — its store exceeds this box's measured 7.12 GB pinned ceiling. A streamed NF4 run is bit-exact against a RESIDENT NF4 run, now as CPU-runnable CI tests rather than only a gate result. Notable, because each fails silently: - PEFT dispatches lora.bnb.Linear4bit only when is_loaded_in_4bit is stamped; without it the generic lora.layer.Linear runs against a Linear4bit base and casts differently (9.375e-01 logit divergence, no warning). Pinned by a test with a control that deletes the marker. - hf_quantizer must be stamped too, or Trainer.__init__ dies formatting its own "cannot fine-tune" error. Found by the end-to-end test. - The shard cache is keyed on quant/double_quant/quant_device as well as dtype and source fingerprint; a bf16 cache reused for an NF4 request would feed full-precision bytes to matmul_4bit. - index.json is a trust boundary: its shape/blocksize reach bnb kernels that do not bounds-check, so from_json validates and the runtime cross-checks the claim against the bytes on disk. - A streamed NF4 model over-reported parameters ~6.5x (878,154,048 vs 134,515,008 for SmolLM2-135M). Display-only; ~52 B at 8B. Scope unchanged and still BETA: RAM tier, sft, Llama/Qwen, batch 1, no accumulation, no resume. quantization values other than none/4bit are refused. Tests: 16752 -> 16840 (+88 in tests/test_v07202.py). Full suite: 16734 passed, 129 skipped. |
|
|
Alpamys
|
af8683b61d |
fix(train): streamed adapters were saved unloadable (v0.72.1)
v0.72.0's layer-streaming wrapper holds the real decoder layer as a child named `inner`, so every saved LoRA adapter key carried an `.inner.` segment. Such a file reloads as ZERO tensors into any normal model: soup merge, soup serve, soup chat and PeftModel.from_pretrained all returned the untuned base while PEFT emitted only a UserWarning. Training was correct; only the artifact was inert. StreamedDecoderLayer.state_dict() now delegates to the wrapped layer at the wrapper's own prefix, so every artifact path -- the final trainer.save_model(), each save_steps checkpoint, and therefore soup adapters, the Registry, merge and serve -- becomes canonical at once. Serialisation-only by design: the forward path is untouched, so v0.72.0's bit-exactness gates remain valid without being re-earned. Also fixes --hf-resume bypassing the streaming resume refusal. The guard tested only --resume, while --hf-resume reaches resume_from through another branch. Pre-fix that combination matched keys by accident; post-fix it would have matched nothing and silently continued training with a freshly initialised adapter -- i.e. the adapter-key fix alone would have made that one path worse. Roadmap renumbered (this release was inserted ahead of NF4): every "lands in vX.Y.Z" refusal corrected -- NF4 v0.72.2, disk tier / more architectures / larger batches / gradient accumulation / checkpoint-resume v0.72.3, preference losses v0.72.4. Found by the v0.72.2 NF4 gate, not by the 159 v0.72.0 tests -- none of them saved an adapter and loaded it back. The new regression test does exactly that, by count, by name and by value, with a negative control that re-mangles the keys and asserts the reload yields zeros (0-of-N loading raises nothing, so a green round-trip without the control proves nothing). Tests: +17 in tests/test_v07201.py (16735 -> 16752). Full suite: 16623 passed, 129 skipped, 4 deselected. Note for maintainers: do not edit soup_cli/__init__.py while a suite is in flight -- a mid-run version bump made test_cli_subprocess::test_version fail spuriously (the subprocess and the imported constant disagreed). |
|
|
Alpamys
|
72be45db85 |
fix(tests): two v0.72.0 tests were platform-dependent, not portable
CI went red on 8 of 9 test jobs. Both causes were test bugs, not product bugs — the streaming code is unchanged. 1. `test_control_chars_in_override_rejected` set an env var containing a NUL byte. A NUL cannot live in an environment variable at all: POSIX putenv rejects it, and CPython >= 3.11 rejects it on Windows too. The only combination that ever allowed it was windows + 3.10, which is exactly the box it was written on — so it passed locally and failed on the other eight jobs with "embedded null byte". Now uses ESC (0x1b), which is still < 0x20 and still must be refused. The override is otherwise a VALID path under $TMPDIR, so the control character is the only reason it can be rejected — verified by backing the `ord(ch) < 0x20` guard out and watching the test go red. 2. `test_one_training_step_actually_runs` failed on macOS with "found at least two devices, mps:0 and cpu". The suite builds the streamed model on cpu when CUDA is absent, but transformers picks `mps` as its default device on Apple Silicon, so the trainer moved batches to a device the model was not on. Skipped when MPS is the accelerator: v0.72.0 measured CUDA and CPU only, and claiming MPS support that has never been run would be worse than skipping. The test still executes on every CUDA and pure-CPU runner. No src/ changes. Test count unchanged at 16735. |
|
|
Alpamys
|
00833ac789 |
feat(train): layer streaming — fine-tune models larger than VRAM (v0.72.0 BETA)
The frozen base lives in CPU RAM and is streamed into a small pool of pre-allocated VRAM buffers one decoder layer at a time, so peak VRAM is bounded by ONE layer instead of the whole model. Only the LoRA adapters, their gradients and optimizer state stay resident. Measured on an RTX 3050 Laptop 4 GB (Windows, 16.9 GB RAM), batch 1, gradient checkpointing on, 50 steps after 10 warm-up: Qwen2.5-0.5B S=512 978.6 tok/s 91.4% util 1.47 GB peak Qwen2.5-1.5B S=512 525.0 tok/s 96.8% util 1.82 GB peak Qwen2.5-1.5B S=1024 487.6 tok/s 96.7% util 2.96 GB peak Qwen2.5-3B S=512 143.1 tok/s 79.3% util 2.15 GB peak Qwen2.5-3B trains in 2.15 GB on a 4 GB card where a resident run OOMs. Honest cost: 1.43x slower than resident, measured at 0.5B — the only apples-to-apples comparison available on this box, because 1.5B and above cannot run resident here at all. Correctness was gated before any src/ code was written: streamed vs resident logits are bit-exact (max abs diff 0.0), the layer-0 LoRA gradient is non-zero on all layers, a 100-step loss curve matches resident exactly, and same-seed runs are identical. New: - utils/layer_stream.py pure planner (no top-level torch) - utils/layer_shard.py per-layer safetensors sharder - utils/layer_stream_runtime.py buffer pool, RAM source, prefetch, wrapper - training.stream_layers / stream_source / stream_buffers Notes for future maintainers: - transformers' Trainer.__init__ and accelerate's prepare_model BOTH call model.to(), which raises NotImplementedError on meta parameters. The streamed layer overrides _apply to pass meta tensors through, and the model declares hf_device_map. Without either, every run dies at trainer construction — no unit test that stops at model(input_ids=...) sees it. - The shard cache is keyed to a fingerprint of the source checkpoint, not just the model slug: a base retrained in place must re-shard rather than silently stream stale weights. - The pre-flight hardware-fit gate models a RESIDENT run, so it is skipped for streaming — otherwise it refuses exactly the runs this enables. - expandable_segments:True is silently ignored on Windows; probed, not claimed. Scope (every refusal names the release that lifts it): RAM tier, bf16, task=sft, Llama/Qwen, batch 1, no gradient accumulation, no --resume. NF4 is v0.72.1; disk tier / bigger batches / accumulation / resume are v0.72.2. Proof-of-mechanism at 3B — nothing above 3B was measured. Tests: 16576 -> 16735 (+159 in tests/test_v07200.py) |