mirror of https://github.com/razor-ai/soup.git
fix(train): streamed adapters were saved unloadable (v0.72.1)
v0.72.0's layer-streaming wrapper holds the real decoder layer as a child named `inner`, so every saved LoRA adapter key carried an `.inner.` segment. Such a file reloads as ZERO tensors into any normal model: soup merge, soup serve, soup chat and PeftModel.from_pretrained all returned the untuned base while PEFT emitted only a UserWarning. Training was correct; only the artifact was inert. StreamedDecoderLayer.state_dict() now delegates to the wrapped layer at the wrapper's own prefix, so every artifact path -- the final trainer.save_model(), each save_steps checkpoint, and therefore soup adapters, the Registry, merge and serve -- becomes canonical at once. Serialisation-only by design: the forward path is untouched, so v0.72.0's bit-exactness gates remain valid without being re-earned. Also fixes --hf-resume bypassing the streaming resume refusal. The guard tested only --resume, while --hf-resume reaches resume_from through another branch. Pre-fix that combination matched keys by accident; post-fix it would have matched nothing and silently continued training with a freshly initialised adapter -- i.e. the adapter-key fix alone would have made that one path worse. Roadmap renumbered (this release was inserted ahead of NF4): every "lands in vX.Y.Z" refusal corrected -- NF4 v0.72.2, disk tier / more architectures / larger batches / gradient accumulation / checkpoint-resume v0.72.3, preference losses v0.72.4. Found by the v0.72.2 NF4 gate, not by the 159 v0.72.0 tests -- none of them saved an adapter and loaded it back. The new regression test does exactly that, by count, by name and by value, with a negative control that re-mangles the keys and asserts the reload yields zeros (0-of-N loading raises nothing, so a green round-trip without the control proves nothing). Tests: +17 in tests/test_v07201.py (16735 -> 16752). Full suite: 16623 passed, 129 skipped, 4 deselected. Note for maintainers: do not edit soup_cli/__init__.py while a suite is in flight -- a mid-run version bump made test_cli_subprocess::test_version fail spuriously (the subprocess and the imported constant disagreed).
This commit is contained in:
Alpamys
parent
4dae34a6a4
commit
af8683b61d
50
CHANGELOG.md
50
CHANGELOG.md
|
|
@ -12,8 +12,58 @@ reproducing 70+ versions of notes.
|
|||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.72.1] - 2026-07-27
|
||||
|
||||
**Fixed — layer-streaming adapters were saved in an unloadable form.** If you
|
||||
trained with `stream_layers: true` on v0.72.0, the adapter that run wrote is
|
||||
**inert**: every tensor was saved under a key containing an extra `.inner.`
|
||||
segment, so `soup merge`, `soup serve`, `soup chat` and
|
||||
`PeftModel.from_pretrained` all loaded **zero** adapter tensors and silently
|
||||
returned the untuned base model. PEFT emitted only a `UserWarning`, so nothing
|
||||
failed and nothing looked wrong.
|
||||
|
||||
The training itself was correct — the streamed run's numerics are unaffected,
|
||||
and v0.72.0's bit-exactness results still stand. Only the saved file was
|
||||
affected.
|
||||
|
||||
**If you have a v0.72.0 streamed adapter: re-save or re-run it on v0.72.1.**
|
||||
There is no way to recover the original file's association with the base model
|
||||
beyond renaming its keys; re-running is the reliable path. A quick check —
|
||||
if `adapter_model.safetensors` contains keys with `.inner.` in them, it is
|
||||
affected:
|
||||
|
||||
```bash
|
||||
python -c "from safetensors.torch import load_file; \
|
||||
print([k for k in load_file('adapter_model.safetensors') if '.inner.' in k][:3])"
|
||||
```
|
||||
|
||||
Streamed adapters now save byte-for-byte in the same layout as an ordinary LoRA
|
||||
run, and are portable to any tool that has never heard of layer streaming.
|
||||
|
||||
**Also fixed — `--hf-resume` bypassed the streaming resume refusal.** The guard
|
||||
only tested `--resume`, but `--hf-resume` reaches `resume_from` through a
|
||||
different branch. That combination previously appeared to work by accident
|
||||
(checkpoint and live model shared the same key shape); once adapters are saved
|
||||
canonically it would instead have matched *nothing* and continued with a
|
||||
freshly initialised adapter, silently. Both flags are now refused for streaming
|
||||
runs, naming v0.72.3.
|
||||
|
||||
Also in this release: every "this lands in vX.Y.Z" refusal message was corrected
|
||||
after the v0.72.x roadmap was renumbered (NF4 streaming is now v0.72.2; the disk
|
||||
tier, wider architectures, larger batches, gradient accumulation and
|
||||
checkpoint/resume are v0.72.3; preference losses are v0.72.4).
|
||||
|
||||
**Known limitation:** in memory the streamed model's `named_parameters()` still
|
||||
carries the wrapper segment, so loading *into* a streaming run (`--resume`)
|
||||
remains unsupported and is refused with a message naming v0.72.3.
|
||||
|
||||
## [0.72.0] - 2026-07-26
|
||||
|
||||
> **Superseded by v0.72.1 — adapters saved by this version load as zero
|
||||
> tensors.** The entry below is left as published; the defect and the fix are
|
||||
> described under [0.72.1]. Version numbers named as "upcoming" below were also
|
||||
> renumbered there (NF4 is v0.72.2, not v0.72.1).
|
||||
|
||||
**Layer streaming (BETA) — fine-tune models that don't fit in your card.** The
|
||||
frozen base lives in CPU RAM and is streamed into two pre-allocated VRAM buffers
|
||||
one decoder layer at a time, so peak VRAM is bounded by the size of *one layer*
|
||||
|
|
|
|||
|
|
@ -120,7 +120,7 @@ src/soup_cli/
|
|||
templates/ - 21 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0, +4 compliance v0.71.35)
|
||||
ui/ - Web UI (FastAPI + HTML/JS SPA)
|
||||
|
||||
tests/ - Test suite (324 files, 16735 tests)
|
||||
tests/ - Test suite (325 files, 16752 tests)
|
||||
examples/ - Real-world config examples and datasets
|
||||
```
|
||||
|
||||
|
|
|
|||
26
README.md
26
README.md
|
|
@ -49,27 +49,39 @@ infrastructure instead of improving models. Soup fixes that.
|
|||
|
||||
## What's New
|
||||
|
||||
**v0.72.0 — Layer streaming (BETA).** Fine-tune models that don't fit in your card. The frozen
|
||||
base streams from CPU RAM one decoder layer at a time into a small pool of VRAM buffers while the
|
||||
LoRA adapters stay resident, so peak VRAM is bounded by one layer rather than the whole model.
|
||||
**Qwen2.5-3B trains in 2.15 GB on a 4 GB card, where a resident run OOMs.**
|
||||
**v0.72.1 — Fix: layer-streaming adapters were saved unloadable.** If you trained with
|
||||
`stream_layers: true` on v0.72.0, that adapter is **inert** — every tensor was written under a
|
||||
key with an extra `.inner.` segment, so `soup merge`, `soup serve`, `soup chat` and
|
||||
`PeftModel.from_pretrained` loaded **zero** tensors and silently returned the untuned base.
|
||||
Only a `UserWarning` was emitted.
|
||||
|
||||
- **The training was correct; only the saved file was affected.** v0.72.0's bit-exactness
|
||||
results still stand.
|
||||
- **Check an existing adapter:** if `adapter_model.safetensors` has keys containing `.inner.`,
|
||||
it is affected — re-run or re-save it on v0.72.1.
|
||||
- Streamed adapters now save in exactly the same layout as an ordinary LoRA run.
|
||||
|
||||
**Layer streaming (BETA), from v0.72.0.** Fine-tune models that don't fit in your card. The
|
||||
frozen base streams from CPU RAM one decoder layer at a time into a small pool of VRAM buffers
|
||||
while the LoRA adapters stay resident, so peak VRAM is bounded by one layer rather than the
|
||||
whole model. **Qwen2.5-3B trains in 2.15 GB on a 4 GB card, where a resident run OOMs.**
|
||||
|
||||
- **`training.stream_layers: true`** — a config key, not a CLI flag. Tune with
|
||||
`stream_source` (`ram` in v0.72.0; `disk` is v0.72.2) and `stream_buffers` (2–8, default 2).
|
||||
`stream_source` (`ram` today; `disk` is v0.72.3) and `stream_buffers` (2–8, default 2).
|
||||
- **Measured on a 4 GB RTX 3050 Laptop** (batch 1, gradient checkpointing on): 0.5B at
|
||||
978.6 tok/s / 1.47 GB, 1.5B at 525.0 tok/s / 1.82 GB, 3B at 143.1 tok/s / 2.15 GB.
|
||||
- **Honest cost: 1.43× slower than resident**, measured at 0.5B — the only apples-to-apples
|
||||
comparison available on that box, because 1.5B and above cannot run resident there at all.
|
||||
- **Proof-of-mechanism at 3B.** Nothing above 3B was measured; no 8B/14B claim is supported.
|
||||
Scope: RAM tier, bf16, `task: sft`, Llama/Qwen, batch size 1, no gradient accumulation, no
|
||||
`--resume`. 4-bit (NF4) streaming is **v0.72.1** and is refused with a clear message today.
|
||||
`--resume`. 4-bit (NF4) streaming is **v0.72.2** and is refused with a clear message today.
|
||||
|
||||
```yaml
|
||||
# soup.yaml — then just `soup train --config soup.yaml`
|
||||
training:
|
||||
stream_layers: true # base streams from RAM; only the adapter trains
|
||||
batch_size: 1
|
||||
quantization: none # NF4 streaming lands in v0.72.1
|
||||
quantization: none # NF4 streaming lands in v0.72.2
|
||||
```
|
||||
|
||||
<details>
|
||||
|
|
|
|||
|
|
@ -233,7 +233,7 @@ Stream frozen base-model decoder layers ONE at a time from CPU RAM into small VR
|
|||
```yaml
|
||||
training:
|
||||
stream_layers: true # Enable layer streaming
|
||||
stream_source: auto # 'auto' (same-host RAM), 'ram', 'disk' (v0.72.2)
|
||||
stream_source: auto # 'auto' (same-host RAM), 'ram', 'disk' (v0.72.3)
|
||||
stream_buffers: 2 # Double-buffering; range [2, 8]
|
||||
```
|
||||
|
||||
|
|
@ -259,20 +259,20 @@ The tradeoff: **1.43× slower than resident training**, measured at 0.5B — the
|
|||
|
||||
**Honest scope (v0.72.0 = proof-of-mechanism):**
|
||||
- **Models measured:** Qwen2.5-0.5B, 1.5B and 3B, plus a live `soup train` on SmolLM2-135M. **Nothing above 3B was measured, and no 8B / 14B / 70B claim is supported.**
|
||||
- **RAM tier only.** `stream_source: ram` (`auto` resolves to it). The disk overflow tier ships in **v0.72.2**.
|
||||
- **RAM tier only.** `stream_source: ram` (`auto` resolves to it). The disk overflow tier ships in **v0.72.3**.
|
||||
- **Llama / Qwen only**, `task: sft`, `backend: transformers`, `modality: text`.
|
||||
- **`batch_size: 1`**, no gradient accumulation, no `--resume` — all ship in **v0.72.2**.
|
||||
- **bf16 base; `quantization: none` is required.** 4-bit (NF4) weights carry a quantisation state and cannot be byte-copied into a plain buffer, so NF4 streaming is **v0.72.1**. A `4bit` config is refused today with a message saying exactly that.
|
||||
- **`batch_size: 1`**, no gradient accumulation, no `--resume` — all ship in **v0.72.3**.
|
||||
- **bf16 base; `quantization: none` is required.** 4-bit (NF4) weights carry a quantisation state and cannot be byte-copied into a plain buffer, so NF4 streaming is **v0.72.2**. A `4bit` config is refused today with a message saying exactly that.
|
||||
- **The 3B throughput is a LOWER BOUND.** The reference box could not page-lock the 5.55 GB base (its measured page-locked ceiling is 7.65 GB, and a CUDA context plus the model skeleton did not leave room), so that run fell back to a pageable store. Pageable memory makes the host-to-device copy synchronous, which costs overlap — visible as the GPU-utilisation drop from 96.8% (1.5B, pinned) to 79.3% (3B, pageable). Soup does this fallback automatically **and prints the cost** rather than absorbing it silently.
|
||||
- Numbers are Windows/WDDM and therefore systematically pessimistic versus Linux. `expandable_segments:True` is silently ignored on Windows; Soup detects that and does not claim it is active.
|
||||
|
||||
**Rejected at config load (each names the release that lifts it):**
|
||||
- `stream_source: disk` → the disk tier is v0.72.2
|
||||
- `quantization` other than `none` → NF4 streaming is v0.72.1
|
||||
- `stream_source: disk` → the disk tier is v0.72.3
|
||||
- `quantization` other than `none` → NF4 streaming is v0.72.2
|
||||
- `backend: unsloth` / `backend: mlx` → streaming replaces the model-load path those backends own
|
||||
- `task` other than `sft` → preference losses are v0.72.3
|
||||
- `gradient_accumulation_steps > 1` → every micro-batch re-reads the whole base, so accumulation multiplies streaming IO linearly (v0.72.2)
|
||||
- `batch_size` other than `1` → v0.72.2
|
||||
- `task` other than `sft` → preference losses are v0.72.4
|
||||
- `gradient_accumulation_steps > 1` → every micro-batch re-reads the whole base, so accumulation multiplies streaming IO linearly (v0.72.3)
|
||||
- `batch_size` other than `1` → v0.72.3
|
||||
- `lora.use_dora` / `lora.use_vera` / `lora.init_strategy` other than `random` → these initialise from the real base weight, which is on the meta device under streaming
|
||||
- `unfrozen_parameters`, `lisa_enabled`, `packing`, `multipack`, `use_fsdp2_compile`, `train_router_only`, `expand_layers` → each independently rewrites or re-freezes the same layers
|
||||
- `stream_source` / `stream_buffers` set while `stream_layers: false` → a footgun, refused
|
||||
|
|
@ -294,12 +294,12 @@ data:
|
|||
training:
|
||||
epochs: 3
|
||||
lr: 2e-5
|
||||
batch_size: 1 # required; larger batches in v0.72.2
|
||||
gradient_accumulation_steps: 1 # required; accumulation in v0.72.2
|
||||
quantization: none # required; NF4 streaming is v0.72.1
|
||||
batch_size: 1 # required; larger batches in v0.72.3
|
||||
gradient_accumulation_steps: 1 # required; accumulation in v0.72.3
|
||||
quantization: none # required; NF4 streaming is v0.72.2
|
||||
gradient_checkpointing: true # handled per-layer by the streamer
|
||||
stream_layers: true # Enable layer streaming
|
||||
stream_source: auto # RAM-based (disk in v0.72.2)
|
||||
stream_source: auto # RAM-based (disk in v0.72.3)
|
||||
stream_buffers: 2 # double-buffering
|
||||
lora:
|
||||
r: 64
|
||||
|
|
@ -313,16 +313,25 @@ output: ./output
|
|||
- The 1.5B runs sit at ~97% GPU utilisation, i.e. compute-bound: with a page-locked store the layer loads hide almost completely behind compute. The 3B run's 79.3% is **not** a model-size effect — it is the cost of the pageable-store fallback on that particular box.
|
||||
- Correctness is not a tradeoff: streamed and resident forward passes were verified **bit-exact**, and a 100-step streamed loss curve matched resident exactly. Streaming substitutes the same weight bytes into the same kernels.
|
||||
|
||||
> **v0.72.0 adapters are unloadable — re-run them on v0.72.1.** In v0.72.0 a streamed run saved every adapter tensor under a key carrying an extra `.inner.` segment, so `soup merge`, `soup serve`, `soup chat` and `PeftModel.from_pretrained` loaded **zero** tensors and silently returned the untuned base (PEFT emitted only a `UserWarning`). The training itself was correct — only the saved file was affected. Check with:
|
||||
>
|
||||
> ```bash
|
||||
> python -c "from safetensors.torch import load_file; \
|
||||
> print([k for k in load_file('adapter_model.safetensors') if '.inner.' in k][:3])"
|
||||
> ```
|
||||
>
|
||||
> If that prints anything, the adapter is affected. From v0.72.1 a streamed adapter is byte-for-byte in the same layout as an ordinary LoRA run.
|
||||
|
||||
**Troubleshooting:**
|
||||
- **"layer streaming needs the base to fit in RAM"** — the base is larger than free RAM. Free RAM or pick a smaller base; the disk overflow tier is v0.72.2.
|
||||
- **"layer streaming needs the base to fit in RAM"** — the base is larger than free RAM. Free RAM or pick a smaller base; the disk overflow tier is v0.72.3.
|
||||
- **"could not page-lock the base … falling back to a PAGEABLE RAM store"** — expected on a busy machine. Training continues, more slowly. Close other applications to keep the pinned store.
|
||||
- **"layer streaming does not support model_type=…"** — v0.72.0 covers Llama and Qwen only.
|
||||
- **Slower than you expected** — layer streaming trades time for memory. If the model already fits resident on your card, do not enable it.
|
||||
|
||||
**Roadmap (each refusal names its release):**
|
||||
- 4-bit (NF4) streaming — **v0.72.1**, the first genuinely useful capability jump
|
||||
- Disk overflow tier, batch size > 1, gradient accumulation, checkpoint/resume, more architectures (Mistral / Gemma / Phi) — **v0.72.2**
|
||||
- Preference losses (DPO / ORPO / SimPO / KTO) — **v0.72.3**. GRPO and PPO are explicitly **not** planned: rollouts need generation, which re-reads the model per token
|
||||
- 4-bit (NF4) streaming — **v0.72.2**, the first genuinely useful capability jump
|
||||
- Disk overflow tier, batch size > 1, gradient accumulation, checkpoint/resume, more architectures (Mistral / Gemma / Phi) — **v0.72.3**
|
||||
- Preference losses (DPO / ORPO / SimPO / KTO) — **v0.72.4**. GRPO and PPO are explicitly **not** planned: rollouts need generation, which re-reads the model per token
|
||||
- A published 14B-on-8 GB reference benchmark — hardware-blocked; it needs an 8 GB card and 32 GB of RAM, which the development box does not have
|
||||
|
||||
**Shard cache.** The first streaming run rewrites the checkpoint into one safetensors shard per decoder layer under `~/.soup/layer-stream/` (override with `SOUP_LAYER_STREAM_CACHE_DIR`). That costs disk space roughly equal to the base. The cache is keyed to a fingerprint of the source checkpoint, so a base retrained in place re-shards instead of silently training against stale weights.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "soup-cli"
|
||||
version = "0.72.0"
|
||||
version = "0.72.1"
|
||||
description = "Fine-tune and post-train LLMs in one command. No SSH, no config hell."
|
||||
readme = "README.md"
|
||||
license = "Apache-2.0"
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
"""Soup CLI — Fine-tune and post-train LLMs in one command."""
|
||||
|
||||
__version__ = "0.72.0"
|
||||
__version__ = "0.72.1"
|
||||
|
|
|
|||
|
|
@ -531,13 +531,20 @@ def train(
|
|||
raise typer.Exit(code=2) from exc
|
||||
|
||||
# --- v0.72.0 BETA — layer streaming does not support resume yet ---
|
||||
# Silently ignoring --resume would restart from scratch and look like it
|
||||
# Silently ignoring a resume would restart from scratch and look like it
|
||||
# worked, so refuse before any heavy work happens.
|
||||
if resume and cfg.training.stream_layers:
|
||||
#
|
||||
# v0.72.1: --hf-resume must be covered too. It sets resume_from through a
|
||||
# different branch (see prepare_hf_resume below), so a guard on --resume
|
||||
# alone let it through. That mattered more after the adapter-key fix: the
|
||||
# pushed checkpoint now carries canonical keys while a live streamed model
|
||||
# is still `.inner.`-shaped, so PEFT's strict=False load would match
|
||||
# NOTHING and silently continue with a freshly initialised adapter.
|
||||
if (resume or hf_resume) and cfg.training.stream_layers:
|
||||
console.print(
|
||||
"[red]--resume is not supported with training.stream_layers in "
|
||||
"v0.72.0[/] — checkpoint/resume for layer streaming lands in "
|
||||
"v0.72.2. Drop --resume, or set stream_layers: false."
|
||||
"[red]--resume / --hf-resume are not supported with "
|
||||
"training.stream_layers[/] — checkpoint/resume for layer streaming "
|
||||
"lands in v0.72.3. Drop the flag, or set stream_layers: false."
|
||||
)
|
||||
raise typer.Exit(code=2)
|
||||
|
||||
|
|
|
|||
|
|
@ -2930,7 +2930,7 @@ class TrainingConfig(BaseModel):
|
|||
default="auto",
|
||||
description=(
|
||||
"Where the streamed base lives. 'ram' (the only tier implemented in "
|
||||
"v0.72.0) pins the base in CPU RAM; 'disk' is the v0.72.2 overflow "
|
||||
"v0.72.0) pins the base in CPU RAM; 'disk' is the v0.72.3 overflow "
|
||||
"tier; 'auto' picks per free RAM."
|
||||
),
|
||||
)
|
||||
|
|
@ -4662,7 +4662,7 @@ class SoupConfig(BaseModel):
|
|||
raise ValueError(
|
||||
f"training.stream_layers requires task='sft' in v0.72.0; got "
|
||||
f"task={self.task!r}. Preference losses (DPO/ORPO/SimPO/KTO) "
|
||||
f"land in v0.72.3."
|
||||
f"land in v0.72.4."
|
||||
)
|
||||
if self.backend != "transformers":
|
||||
raise ValueError(
|
||||
|
|
@ -4680,24 +4680,24 @@ class SoupConfig(BaseModel):
|
|||
f"training.stream_layers requires quantization='none' in "
|
||||
f"v0.72.0 (got {tcfg.quantization!r}). NF4 weights carry a "
|
||||
f"quant_state and cannot be byte-copied into a plain buffer; "
|
||||
f"NF4 streaming lands in v0.72.1."
|
||||
f"NF4 streaming lands in v0.72.2."
|
||||
)
|
||||
if tcfg.stream_source == "disk":
|
||||
raise ValueError(
|
||||
"training.stream_source='disk' is not implemented in v0.72.0 — "
|
||||
"the disk overflow tier lands in v0.72.2. Use 'ram' or 'auto'."
|
||||
"the disk overflow tier lands in v0.72.3. Use 'ram' or 'auto'."
|
||||
)
|
||||
if tcfg.batch_size != 1:
|
||||
raise ValueError(
|
||||
f"training.stream_layers requires batch_size=1 in v0.72.0 (got "
|
||||
f"{tcfg.batch_size!r}); larger batches land in v0.72.2."
|
||||
f"{tcfg.batch_size!r}); larger batches land in v0.72.3."
|
||||
)
|
||||
if tcfg.gradient_accumulation_steps != 1:
|
||||
raise ValueError(
|
||||
f"training.stream_layers requires gradient_accumulation_steps=1 "
|
||||
f"(got {tcfg.gradient_accumulation_steps}); every micro-batch "
|
||||
f"re-reads the entire base, so accumulation multiplies streaming "
|
||||
f"IO linearly. Accumulation lands in v0.72.2."
|
||||
f"IO linearly. Accumulation lands in v0.72.3."
|
||||
)
|
||||
if tcfg.lora.r < 1:
|
||||
raise ValueError(
|
||||
|
|
|
|||
|
|
@ -1019,7 +1019,7 @@ class SFTTrainerWrapper:
|
|||
f"layer streaming needs the base to fit in RAM: {cfg.base} is "
|
||||
f"{source_bytes / 1e9:.1f} GB on disk and only "
|
||||
f"{early_free_ram / 1e9:.1f} GB of RAM is free. The disk "
|
||||
f"overflow tier lands in v0.72.2 — free RAM or pick a "
|
||||
f"overflow tier lands in v0.72.3 — free RAM or pick a "
|
||||
f"smaller base."
|
||||
)
|
||||
|
||||
|
|
@ -1062,7 +1062,7 @@ class SFTTrainerWrapper:
|
|||
f"layer streaming needs the base to fit in RAM: it is "
|
||||
f"{(layer_bytes * index.n_layers + embed_bytes) / 1e9:.1f} GB and only "
|
||||
f"{free_ram / 1e9:.1f} GB is free. The disk overflow tier lands "
|
||||
f"in v0.72.2 — free RAM or pick a smaller base."
|
||||
f"in v0.72.3 — free RAM or pick a smaller base."
|
||||
)
|
||||
console.print(render_stream_panel(plan))
|
||||
console.print(
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ DEFAULT_STREAM_BUFFERS = 2
|
|||
#: the base is frozen). Streaming always checkpoints, so this is never 4.
|
||||
FLOPS_PER_PARAM_PER_TOKEN = 6
|
||||
|
||||
#: v0.72.0 scope. Breadth (Mistral / Gemma / Phi) is v0.72.2.
|
||||
#: v0.72.0 scope. Breadth (Mistral / Gemma / Phi) is v0.72.3.
|
||||
SUPPORTED_STREAM_ARCHS = ("llama", "qwen2", "qwen3")
|
||||
|
||||
_DTYPE_BYTES = {"bfloat16": 2, "float16": 2, "float32": 4}
|
||||
|
|
@ -84,7 +84,7 @@ def stream_arch_of(config: Any) -> str:
|
|||
raise ValueError(
|
||||
f"layer streaming does not support model_type={family!r} in "
|
||||
f"v0.72.0. Supported: {', '.join(SUPPORTED_STREAM_ARCHS)}. "
|
||||
f"More architectures land in v0.72.2."
|
||||
f"More architectures land in v0.72.3."
|
||||
)
|
||||
return family
|
||||
|
||||
|
|
@ -313,7 +313,7 @@ def build_stream_plan(
|
|||
notes = []
|
||||
if tier == TIER_DISK:
|
||||
notes.append(
|
||||
"base does not fit in RAM — the disk tier lands in v0.72.2; "
|
||||
"base does not fit in RAM — the disk tier lands in v0.72.3; "
|
||||
"v0.72.0 supports stream_source='ram' only"
|
||||
)
|
||||
decision = decide_pinning(store_bytes, pinned_limit_bytes)
|
||||
|
|
|
|||
|
|
@ -278,6 +278,50 @@ def _build_streamed_layer_class():
|
|||
|
||||
return super()._apply(_skip_meta, recurse=recurse)
|
||||
|
||||
def state_dict(
|
||||
self,
|
||||
*args: Any,
|
||||
destination: Any = None,
|
||||
prefix: str = "",
|
||||
keep_vars: bool = False,
|
||||
) -> Any:
|
||||
# v0.72.1 — serialise as though this wrapper were not in the tree.
|
||||
#
|
||||
# The wrapper holds the real layer as a child named `inner`, so
|
||||
# every adapter parameter would otherwise be written as
|
||||
# `...layers.0.inner.self_attn.q_proj.lora_A.weight`. That file
|
||||
# loads as ZERO tensors into any normal model — PEFT reports the
|
||||
# keys as missing and returns the untuned base, with no exception.
|
||||
# Every adapter artifact (the final `trainer.save_model()`, each
|
||||
# `save_steps` checkpoint, and therefore everything downstream:
|
||||
# `soup merge` / `serve` / `chat` / `adapters *` / the Registry)
|
||||
# reaches disk through this method, so delegating at OUR prefix is
|
||||
# what makes a streamed adapter indistinguishable from a normal
|
||||
# LoRA run.
|
||||
#
|
||||
# Serialisation-only, deliberately: the forward path is untouched,
|
||||
# so v0.72.0's bit-exactness gates remain valid. The cost is that
|
||||
# `named_parameters()` still shows `.inner.`, i.e. loading INTO a
|
||||
# streamed model stays unsupported (`--resume` is refused; the
|
||||
# checkpoint/resume slot is v0.72.3).
|
||||
#
|
||||
# The wrapper owns no parameters or buffers of its own — they all
|
||||
# live on `inner` — so nothing is lost by not serialising it. It
|
||||
# also means bypassing nn.Module.state_dict skips only hooks
|
||||
# registered on the WRAPPER itself, of which there are none (the
|
||||
# prefetch hook lives on the decoder container, not here).
|
||||
if args:
|
||||
# torch's legacy positional form: (destination, prefix, keep_vars)
|
||||
if destination is None:
|
||||
destination = args[0]
|
||||
if len(args) > 1 and prefix == "":
|
||||
prefix = args[1]
|
||||
if len(args) > 2 and keep_vars is False:
|
||||
keep_vars = args[2]
|
||||
return self.inner.state_dict(
|
||||
destination=destination, prefix=prefix, keep_vars=keep_vars
|
||||
)
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
# transformers reads contract attributes straight off the layer
|
||||
# object (this version reads `decoder_layer.attention_type`). The
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ class TestStreamArchAllowlist:
|
|||
stream_arch_of(_Cfg("gpt2"))
|
||||
|
||||
def test_mistral_is_rejected_in_v0720(self):
|
||||
"""Mistral is v0.72.2 breadth — it must not silently half-work."""
|
||||
"""Mistral is v0.72.3 breadth — it must not silently half-work."""
|
||||
from soup_cli.utils.layer_stream import stream_arch_of
|
||||
|
||||
with pytest.raises(ValueError, match="mistral"):
|
||||
|
|
@ -776,13 +776,13 @@ class TestStreamTaskAndBackendGates:
|
|||
|
||||
|
||||
class TestStreamScopeGates:
|
||||
def test_quantization_rejected_and_names_v0721(self):
|
||||
"""NF4 streaming is the v0.72.1 slot — say so instead of half-working."""
|
||||
with pytest.raises(ValueError, match="v0.72.1"):
|
||||
def test_quantization_rejected_and_names_nf4_slot(self):
|
||||
"""NF4 streaming is the v0.72.2 slot — say so instead of half-working."""
|
||||
with pytest.raises(ValueError, match="v0.72.2"):
|
||||
_load(_stream_yaml(training={"quantization": "4bit"}))
|
||||
|
||||
def test_disk_source_rejected_and_names_v0722(self):
|
||||
with pytest.raises(ValueError, match="v0.72.2"):
|
||||
def test_disk_source_rejected_and_names_breadth_slot(self):
|
||||
with pytest.raises(ValueError, match="v0.72.3"):
|
||||
_load(_stream_yaml(training={"stream_source": "disk"}))
|
||||
|
||||
def test_batch_size_above_one_rejected(self):
|
||||
|
|
|
|||
|
|
@ -0,0 +1,510 @@
|
|||
"""v0.72.1 — a streamed adapter must be a NORMAL adapter on disk.
|
||||
|
||||
v0.72.0's ``install_streaming`` replaces ``layers[i]`` with a wrapper that holds
|
||||
the real layer as a child named ``inner``. Every adapter parameter therefore
|
||||
serialised as ``...layers.0.inner.self_attn.q_proj.lora_A.weight``, and loading
|
||||
that file into any normal model dropped **every** tensor — PEFT warns about
|
||||
missing keys and hands back the untuned base. Training was correct; the artifact
|
||||
it wrote was inert in ``soup merge`` / ``serve`` / ``chat`` and in
|
||||
``PeftModel.from_pretrained``.
|
||||
|
||||
The assertion that matters is NOT "no exception raised": 0-of-N tensors loading
|
||||
raises nothing at all. These tests assert by count, by name, and by value.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# fixtures (standalone, mirroring tests/test_v07200.py)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _tiny_llama_dir(tmp_path, n_layers=2, tie=True):
|
||||
"""A real (tiny) Llama checkpoint on disk: config.json + model.safetensors."""
|
||||
import torch
|
||||
from safetensors.torch import save_file
|
||||
from transformers import LlamaConfig, LlamaForCausalLM
|
||||
|
||||
torch.manual_seed(7)
|
||||
config = LlamaConfig(
|
||||
vocab_size=64,
|
||||
hidden_size=32,
|
||||
intermediate_size=64,
|
||||
num_hidden_layers=n_layers,
|
||||
num_attention_heads=4,
|
||||
num_key_value_heads=2,
|
||||
tie_word_embeddings=tie,
|
||||
max_position_embeddings=128,
|
||||
)
|
||||
model = LlamaForCausalLM(config).to(torch.float32).eval()
|
||||
weights = tmp_path / "model"
|
||||
weights.mkdir(parents=True, exist_ok=True)
|
||||
state = {k: v.contiguous() for k, v in model.state_dict().items()}
|
||||
if tie:
|
||||
state.pop("lm_head.weight", None)
|
||||
save_file(state, str(weights / "model.safetensors"))
|
||||
config.save_pretrained(str(weights))
|
||||
return str(weights)
|
||||
|
||||
|
||||
def _tiny_lora():
|
||||
from peft import LoraConfig, TaskType
|
||||
|
||||
return LoraConfig(
|
||||
r=4, lora_alpha=8, lora_dropout=0.0, bias="none",
|
||||
target_modules=["q_proj", "v_proj"], task_type=TaskType.CAUSAL_LM,
|
||||
)
|
||||
|
||||
|
||||
def _build_streamed_cpu(tmp_path, n_layers=2):
|
||||
from soup_cli.utils.layer_shard import shard_checkpoint
|
||||
from soup_cli.utils.layer_stream_runtime import build_streamed_model
|
||||
|
||||
weights = _tiny_llama_dir(tmp_path, n_layers=n_layers)
|
||||
shards = str(tmp_path / "shards")
|
||||
index = shard_checkpoint(weights, shards, dtype="float32", arch="llama")
|
||||
model, runtime = build_streamed_model(
|
||||
model_id=weights, shard_dir=shards, index=index,
|
||||
lora_config=_tiny_lora(), device="cpu", dtype="float32",
|
||||
buffers=2, pin=False, seed=3,
|
||||
)
|
||||
return model, runtime, weights
|
||||
|
||||
|
||||
def _build_plain_peft(weights_dir):
|
||||
"""The reference: an ordinary, non-streaming LoRA model on the same base."""
|
||||
import torch
|
||||
from peft import get_peft_model
|
||||
from transformers import AutoModelForCausalLM
|
||||
|
||||
base = AutoModelForCausalLM.from_pretrained(weights_dir, dtype=torch.float32)
|
||||
return get_peft_model(base, _tiny_lora())
|
||||
|
||||
|
||||
def _make_adapters_detectable(model, value=0.05):
|
||||
"""PEFT initialises lora_B to ZERO.
|
||||
|
||||
An adapter that fails to load is byte-identical to a freshly-initialised one
|
||||
while B is zero, so every "did it load?" assertion would pass vacuously.
|
||||
Writing a non-zero B is what makes a dropped tensor detectable at all.
|
||||
"""
|
||||
import torch
|
||||
|
||||
with torch.no_grad():
|
||||
for name, param in model.named_parameters():
|
||||
if "lora_B" in name:
|
||||
param.copy_(torch.full_like(param, value))
|
||||
elif "lora_A" in name:
|
||||
param.copy_(torch.full_like(param, value / 2))
|
||||
|
||||
|
||||
def _adapter_keys(directory):
|
||||
from safetensors.torch import load_file
|
||||
|
||||
return load_file(os.path.join(directory, "adapter_model.safetensors"))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# the saved artifact
|
||||
# --------------------------------------------------------------------------
|
||||
class TestSavedAdapterKeysAreCanonical:
|
||||
def test_saved_keys_carry_no_inner_prefix(self, tmp_path):
|
||||
model, _, _ = _build_streamed_cpu(tmp_path)
|
||||
_make_adapters_detectable(model)
|
||||
out = tmp_path / "adapter"
|
||||
model.save_pretrained(str(out))
|
||||
|
||||
saved = _adapter_keys(str(out))
|
||||
leaked = [key for key in saved if ".inner." in key]
|
||||
assert saved, "no adapter tensors were written at all"
|
||||
assert leaked == [], (
|
||||
f"{len(leaked)} of {len(saved)} saved keys carry the streaming "
|
||||
f"wrapper's '.inner.' segment, e.g. {leaked[:1]}"
|
||||
)
|
||||
|
||||
def test_saved_key_set_is_identical_to_a_normal_lora_run(self, tmp_path):
|
||||
"""The artifact must be indistinguishable from a non-streamed run.
|
||||
|
||||
Portability is the point: tooling that has never heard of layer
|
||||
streaming has to load this file.
|
||||
"""
|
||||
model, _, weights = _build_streamed_cpu(tmp_path)
|
||||
streamed_out = tmp_path / "streamed_adapter"
|
||||
model.save_pretrained(str(streamed_out))
|
||||
|
||||
reference = _build_plain_peft(weights)
|
||||
plain_out = tmp_path / "plain_adapter"
|
||||
reference.save_pretrained(str(plain_out))
|
||||
|
||||
assert set(_adapter_keys(str(streamed_out))) == set(_adapter_keys(str(plain_out)))
|
||||
|
||||
def test_in_memory_state_dict_is_canonical(self, tmp_path):
|
||||
"""The checkpoint path (Trainer._save) serialises via state_dict()."""
|
||||
from peft import get_peft_model_state_dict
|
||||
|
||||
model, _, _ = _build_streamed_cpu(tmp_path)
|
||||
keys = list(get_peft_model_state_dict(model))
|
||||
assert keys
|
||||
assert [k for k in keys if ".inner." in k] == []
|
||||
|
||||
def test_adapter_config_is_unchanged(self, tmp_path):
|
||||
"""The fix must not perturb adapter_config.json."""
|
||||
model, _, weights = _build_streamed_cpu(tmp_path)
|
||||
out = tmp_path / "adapter"
|
||||
model.save_pretrained(str(out))
|
||||
with open(out / "adapter_config.json", encoding="utf-8") as handle:
|
||||
cfg = json.load(handle)
|
||||
assert cfg["r"] == 4
|
||||
assert sorted(cfg["target_modules"]) == ["q_proj", "v_proj"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# THE regression test
|
||||
# --------------------------------------------------------------------------
|
||||
class TestAdapterReloadsIntoNormalModel:
|
||||
def test_every_tensor_lands_by_count_and_by_name_and_by_value(self, tmp_path):
|
||||
import torch
|
||||
from peft import PeftModel
|
||||
from transformers import AutoModelForCausalLM
|
||||
|
||||
model, _, weights = _build_streamed_cpu(tmp_path)
|
||||
_make_adapters_detectable(model)
|
||||
out = tmp_path / "adapter"
|
||||
model.save_pretrained(str(out))
|
||||
saved = _adapter_keys(str(out))
|
||||
|
||||
base = AutoModelForCausalLM.from_pretrained(weights, dtype=torch.float32)
|
||||
reloaded = PeftModel.from_pretrained(base, str(out))
|
||||
landed = {
|
||||
name: param
|
||||
for name, param in reloaded.named_parameters()
|
||||
if "lora_" in name
|
||||
}
|
||||
|
||||
# by count
|
||||
assert len(landed) == len(saved), (
|
||||
f"saved {len(saved)} adapter tensors but the reloaded model exposes "
|
||||
f"{len(landed)}"
|
||||
)
|
||||
# by name + by value
|
||||
for key, tensor in saved.items():
|
||||
target = key.replace(".weight", ".default.weight")
|
||||
assert target in landed, f"saved key {key!r} did not land as {target!r}"
|
||||
assert torch.equal(landed[target].detach().cpu(), tensor.cpu()), (
|
||||
f"{target} loaded with different values"
|
||||
)
|
||||
# and it is not the untuned base: B must be non-zero somewhere
|
||||
assert any(
|
||||
param.abs().sum().item() > 0
|
||||
for name, param in landed.items()
|
||||
if "lora_B" in name
|
||||
), "every lora_B is zero — the adapter loaded as a no-op"
|
||||
|
||||
def test_the_assertion_would_catch_a_dropped_adapter(self, tmp_path):
|
||||
"""Control: prove the check above is not vacuous.
|
||||
|
||||
Rewriting the saved keys back to the broken v0.72.0 form must make the
|
||||
reload fail the same assertions — otherwise a green test says nothing.
|
||||
"""
|
||||
import shutil
|
||||
|
||||
import torch
|
||||
from peft import PeftModel
|
||||
from safetensors.torch import save_file
|
||||
from transformers import AutoModelForCausalLM
|
||||
|
||||
model, _, weights = _build_streamed_cpu(tmp_path)
|
||||
_make_adapters_detectable(model)
|
||||
out = tmp_path / "adapter"
|
||||
model.save_pretrained(str(out))
|
||||
|
||||
# Write the v0.72.0-shaped copy into its OWN directory: safetensors
|
||||
# mmaps the file it just read, and overwriting it in place fails on
|
||||
# Windows with error 1224 (the same trap utils/adapter_fuse.py documents).
|
||||
broken_dir = tmp_path / "adapter_v0720_shaped"
|
||||
broken_dir.mkdir()
|
||||
shutil.copy(out / "adapter_config.json", broken_dir / "adapter_config.json")
|
||||
mangled = {
|
||||
key.replace(".self_attn.", ".inner.self_attn."): value
|
||||
for key, value in _adapter_keys(str(out)).items()
|
||||
}
|
||||
save_file(mangled, str(broken_dir / "adapter_model.safetensors"))
|
||||
|
||||
base = AutoModelForCausalLM.from_pretrained(weights, dtype=torch.float32)
|
||||
reloaded = PeftModel.from_pretrained(base, str(broken_dir))
|
||||
b_sum = sum(
|
||||
param.abs().sum().item()
|
||||
for name, param in reloaded.named_parameters()
|
||||
if "lora_B" in name
|
||||
)
|
||||
assert b_sum == 0.0, (
|
||||
"the v0.72.0-shaped keys were expected to load as nothing; if this "
|
||||
"fails the round-trip assertion above proves nothing"
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# the fix must not touch numerics
|
||||
# --------------------------------------------------------------------------
|
||||
class TestFixIsNumericsNeutral:
|
||||
def test_streamed_forward_still_matches_resident_bit_exactly(self, tmp_path):
|
||||
"""v0.72.0's correctness gates must remain valid after this change.
|
||||
|
||||
The fix is serialisation-only precisely so that the bit-exactness
|
||||
result does not have to be re-earned; this pins that claim.
|
||||
"""
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM
|
||||
|
||||
model, _, weights = _build_streamed_cpu(tmp_path)
|
||||
resident = AutoModelForCausalLM.from_pretrained(weights, dtype=torch.float32).eval()
|
||||
model.eval()
|
||||
|
||||
ids = torch.arange(8, dtype=torch.long).unsqueeze(0) % 64
|
||||
with torch.no_grad():
|
||||
streamed_logits = model(input_ids=ids).logits
|
||||
resident_logits = resident(input_ids=ids).logits
|
||||
# lora_B is still zero here, so the adapter is a no-op and the streamed
|
||||
# base must reproduce the resident base exactly
|
||||
assert torch.equal(streamed_logits, resident_logits)
|
||||
|
||||
def test_layer_zero_adapter_still_receives_gradient(self, tmp_path):
|
||||
"""plan P2: a severed graph still lowers the loss, so check the grad."""
|
||||
import torch
|
||||
|
||||
model, _, _ = _build_streamed_cpu(tmp_path)
|
||||
_make_adapters_detectable(model)
|
||||
ids = torch.arange(8, dtype=torch.long).unsqueeze(0) % 64
|
||||
model(input_ids=ids, labels=ids.clone()).loss.backward()
|
||||
|
||||
grads = {
|
||||
name: param.grad
|
||||
for name, param in model.named_parameters()
|
||||
if param.requires_grad and "lora_" in name and ".layers.0." in name
|
||||
}
|
||||
assert grads, "no layer-0 adapter parameters found"
|
||||
assert any(g is not None and g.abs().max().item() > 0 for g in grads.values())
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# the known limitation, pinned so it cannot silently half-work
|
||||
# --------------------------------------------------------------------------
|
||||
class TestLoadingIntoAStreamedModelStaysUnsupported:
|
||||
def test_named_parameters_still_carry_the_wrapper_segment(self, tmp_path):
|
||||
"""Fix A is serialisation-only, by design.
|
||||
|
||||
In memory the wrapper is still a real module, so ``named_parameters()``
|
||||
and ``state_dict()`` disagree. That is why loading INTO a streamed model
|
||||
(i.e. ``--resume``) stays refused; this documents the asymmetry instead
|
||||
of leaving it to be discovered.
|
||||
"""
|
||||
model, _, _ = _build_streamed_cpu(tmp_path)
|
||||
live = [n for n, _ in model.named_parameters() if "lora_" in n]
|
||||
assert live
|
||||
assert any(".inner." in n for n in live)
|
||||
|
||||
def test_hf_resume_with_streaming_exits_before_training(self, tmp_path, monkeypatch):
|
||||
"""--hf-resume takes a DIFFERENT branch to resume_from.
|
||||
|
||||
A guard on ``--resume`` alone let ``--hf-resume`` through. Before the
|
||||
adapter-key fix that combination happened to work (checkpoint and live
|
||||
model were both ``.inner.``-shaped, so the keys matched by accident);
|
||||
after it, the pushed checkpoint is canonical while the live streamed
|
||||
model is not, so PEFT's ``strict=False`` load matches nothing and
|
||||
training silently continues with a freshly initialised adapter. The
|
||||
guard has to cover both flags or the fix makes that path worse.
|
||||
|
||||
Asserted behaviourally — a source grep would break on a harmless
|
||||
refactor and pass on a guard that had been moved into dead code.
|
||||
"""
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from soup_cli.cli import app
|
||||
|
||||
weights = _tiny_llama_dir(tmp_path)
|
||||
data = tmp_path / "data.jsonl"
|
||||
data.write_text('{"text": "hello world"}\n', encoding="utf-8")
|
||||
config = tmp_path / "soup.yaml"
|
||||
config.write_text(
|
||||
f"base: {weights}\n"
|
||||
"task: sft\n"
|
||||
f"data:\n train: {json.dumps(str(data))}\n format: plaintext\n"
|
||||
"training:\n"
|
||||
" stream_layers: true\n batch_size: 1\n quantization: none\n"
|
||||
# both default to values streaming refuses, and those gates fire
|
||||
# before the resume guard — without them this asserts the wrong refusal
|
||||
" gradient_accumulation_steps: 1\n epochs: 1\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
monkeypatch.chdir(tmp_path)
|
||||
result = CliRunner().invoke(
|
||||
app,
|
||||
["train", "--config", str(config), "--yes",
|
||||
"--push-as", "someone/somewhere", "--hf-resume"],
|
||||
)
|
||||
assert result.exit_code == 2, (result.output, repr(result.exception))
|
||||
assert "--hf-resume" in result.output
|
||||
assert "v0.72.3" in result.output, "the refusal must name the slot that lifts it"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# the PRODUCTION save path, not just the helper
|
||||
# --------------------------------------------------------------------------
|
||||
def _write_tiny_tokenizer(directory):
|
||||
"""A real, offline PreTrainedTokenizerFast so `setup()` can tokenize."""
|
||||
from tokenizers import Tokenizer, models, pre_tokenizers
|
||||
|
||||
vocab = {"<unk>": 0, "<s>": 1, "</s>": 2, "<pad>": 3}
|
||||
for word in ("hello", "world", "hi", "yo", "the", "cat", "sat", "on", "mat"):
|
||||
vocab[word] = len(vocab)
|
||||
tokenizer = Tokenizer(models.WordLevel(vocab=vocab, unk_token="<unk>"))
|
||||
tokenizer.pre_tokenizer = pre_tokenizers.Whitespace()
|
||||
tokenizer.save(os.path.join(directory, "tokenizer.json"))
|
||||
with open(os.path.join(directory, "tokenizer_config.json"), "w", encoding="utf-8") as fh:
|
||||
json.dump(
|
||||
{
|
||||
"tokenizer_class": "PreTrainedTokenizerFast",
|
||||
"unk_token": "<unk>",
|
||||
"bos_token": "<s>",
|
||||
"eos_token": "</s>",
|
||||
"pad_token": "<pad>",
|
||||
"model_max_length": 128,
|
||||
"clean_up_tokenization_spaces": False,
|
||||
},
|
||||
fh,
|
||||
)
|
||||
|
||||
|
||||
class TestProductionSavePathWritesCanonicalKeys:
|
||||
"""The tests above save via `model.save_pretrained()` directly.
|
||||
|
||||
Real runs save through `SFTTrainerWrapper` -> TRL `SFTTrainer` ->
|
||||
`Trainer.save_model()` -> `PeftModel.save_pretrained()`, which is also the
|
||||
`save_steps` checkpoint path. Asserting only on the helper leaves exactly
|
||||
the integration gap that hid the original v0.72.0 defect, so this drives
|
||||
the production path and then inspects the bytes on disk.
|
||||
"""
|
||||
|
||||
def test_trainer_save_model_writes_a_loadable_adapter(self, tmp_path, monkeypatch):
|
||||
import torch
|
||||
import yaml
|
||||
from peft import PeftModel
|
||||
from transformers import AutoModelForCausalLM
|
||||
|
||||
from soup_cli.config.loader import load_config_from_string
|
||||
from soup_cli.trainer.sft import SFTTrainerWrapper
|
||||
|
||||
weights = _tiny_llama_dir(tmp_path, n_layers=2)
|
||||
_write_tiny_tokenizer(weights)
|
||||
monkeypatch.setenv("SOUP_LAYER_STREAM_CACHE_DIR", str(tmp_path / "cache"))
|
||||
monkeypatch.chdir(tmp_path)
|
||||
cfg = load_config_from_string(
|
||||
yaml.safe_dump(
|
||||
{
|
||||
"base": weights,
|
||||
"task": "sft",
|
||||
"backend": "transformers",
|
||||
"modality": "text",
|
||||
"data": {"train": "train.jsonl", "max_length": 64,
|
||||
"chat_template": "chatml"},
|
||||
"training": {
|
||||
"batch_size": 1, "gradient_accumulation_steps": 1,
|
||||
"quantization": "none", "stream_layers": True,
|
||||
"epochs": 1, "logging_steps": 1, "save_steps": 1000,
|
||||
"gradient_checkpointing": True,
|
||||
"lora": {"r": 4, "alpha": 8,
|
||||
"target_modules": ["q_proj", "v_proj"]},
|
||||
},
|
||||
"output": str(tmp_path / "out"),
|
||||
}
|
||||
)
|
||||
)
|
||||
dataset = {
|
||||
"train": [
|
||||
{"messages": [{"role": "user", "content": "hi"},
|
||||
{"role": "assistant", "content": "hello world"}]}
|
||||
for _ in range(4)
|
||||
]
|
||||
}
|
||||
wrapper = SFTTrainerWrapper(cfg, device="cpu")
|
||||
wrapper.setup(dataset)
|
||||
|
||||
# make a dropped adapter detectable before the production save
|
||||
_make_adapters_detectable(wrapper.model)
|
||||
|
||||
saved_dir = tmp_path / "saved"
|
||||
wrapper.trainer.save_model(str(saved_dir))
|
||||
|
||||
saved = _adapter_keys(str(saved_dir))
|
||||
assert saved, "the production save path wrote no adapter tensors"
|
||||
assert [k for k in saved if ".inner." in k] == []
|
||||
|
||||
base = AutoModelForCausalLM.from_pretrained(weights, dtype=torch.float32)
|
||||
reloaded = PeftModel.from_pretrained(base, str(saved_dir))
|
||||
b_tensors = [p for n, p in reloaded.named_parameters() if "lora_B" in n]
|
||||
assert b_tensors
|
||||
assert all(p.abs().sum().item() > 0 for p in b_tensors), (
|
||||
"adapter saved by the production path reloaded as a no-op"
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# the renumber: every refusal must name the slot that actually lifts it
|
||||
# --------------------------------------------------------------------------
|
||||
def _stream_yaml(**training):
|
||||
# quantization defaults to 4bit repo-wide, which trips the NF4 gate before
|
||||
# any other streaming gate is reached — every case below would otherwise
|
||||
# assert the wrong refusal.
|
||||
fields = {"stream_layers": "true", "batch_size": 1, "quantization": "none"}
|
||||
fields.update(training)
|
||||
lines = [
|
||||
"base: hf-internal-testing/tiny-random-LlamaForCausalLM",
|
||||
"task: sft",
|
||||
"data:",
|
||||
" train: data.jsonl",
|
||||
"training:",
|
||||
]
|
||||
lines += [f" {key}: {value}" for key, value in fields.items()]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
class TestRefusalsNameThePostRenumberSlot:
|
||||
@pytest.mark.parametrize(
|
||||
"field,value,slot",
|
||||
[
|
||||
("quantization", "4bit", "v0.72.2"), # NF4
|
||||
("stream_source", "disk", "v0.72.3"), # disk tier
|
||||
("batch_size", "2", "v0.72.3"), # larger batches
|
||||
("gradient_accumulation_steps", "4", "v0.72.3"),
|
||||
],
|
||||
)
|
||||
def test_gate_names_the_right_release(self, field, value, slot):
|
||||
from soup_cli.config.loader import load_config_from_string
|
||||
|
||||
yaml_text = _stream_yaml(**{field: value})
|
||||
with pytest.raises(ValueError, match=slot):
|
||||
load_config_from_string(yaml_text)
|
||||
|
||||
def test_preference_losses_name_v0724(self):
|
||||
from soup_cli.config.loader import load_config_from_string
|
||||
|
||||
yaml_text = (
|
||||
"base: hf-internal-testing/tiny-random-LlamaForCausalLM\n"
|
||||
"task: dpo\n"
|
||||
"data:\n train: data.jsonl\n"
|
||||
"training:\n stream_layers: true\n batch_size: 1\n"
|
||||
)
|
||||
with pytest.raises(ValueError, match="v0.72.4"):
|
||||
load_config_from_string(yaml_text)
|
||||
|
||||
def test_arch_allowlist_names_v0723(self):
|
||||
from soup_cli.utils.layer_stream import stream_arch_of
|
||||
|
||||
class _Cfg:
|
||||
model_type = "gpt2"
|
||||
|
||||
with pytest.raises(ValueError, match="v0.72.3"):
|
||||
stream_arch_of(_Cfg())
|
||||
Loading…
Reference in New Issue