diff --git a/CHANGELOG.md b/CHANGELOG.md index f57506e..2b43234 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,27 @@ reproducing 70+ versions of notes. ## [Unreleased] +## [0.71.34] - 2026-07-15 + +### Added +- **`soup adapters arithmetic` — task-vector algebra over LoRA adapters (add / scale / negate).** + Apply task arithmetic (arXiv:2212.04089) to LoRA deltas via an expression such as + `"coder + 0.5*math - toxic"`, mapping names to adapter dirs with repeatable + `--adapter name=path`. Produces one merged adapter you can serve or merge. + - Signed, un-normalized element-wise combine over same-rank adapters; the effective + delta `ΔW = B @ A` scales **linearly** with each coefficient (negation flips the + delta, `0.5·` halves it) via a √|c| factor split — not the `c²` a naive sum gives. + Mixed-rank inputs are refused with a clear "harmonize rank" message. + - Reuses the backdoor-scan gate (refuses a FAIL-scanned input unless `--allow-unscanned`) + and a same-base-model check (`--allow-cross-base` to override). Hand-written expression + parser (no `eval`), cwd-contained/symlink-rejecting paths, exit 0 = ok / 1 = refusal. +- **LISA — Layerwise Importance Sampled AdamW (arXiv:2403.17919).** Full-fine-tuning + quality at LoRA-like memory: every N steps LISA freezes all decoder layers except a + small random set (embeddings + head always trainable). Enable with + `training.lisa_enabled: true` (+ `lisa_num_layers`, `lisa_interval_steps`) on a + `task: sft`, transformers, text, `quantization: none` run; mutually exclusive with + LoRA features and the other freeze mechanisms. Live on a 4 GB GPU for small models. + ## [0.71.33] - 2026-07-13 ### Added diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6a9357b..24d0093 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -120,7 +120,7 @@ src/soup_cli/ templates/ - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0) ui/ - Web UI (FastAPI + HTML/JS SPA) -tests/ - Test suite (311 files, 15806 tests) +tests/ - Test suite (312 files, 15906 tests) examples/ - Real-world config examples and datasets ``` diff --git a/README.md b/README.md index 36e36ca..2bcabef 100644 --- a/README.md +++ b/README.md @@ -49,47 +49,40 @@ infrastructure instead of improving models. Soup fixes that. ## What's New -**v0.71.33 — `soup draft`: know whether speculative decoding is actually worth it.** Everyone tells you to bolt a draft model onto your server for a free speedup. Nobody tells you to *measure* it first. Now you can. +**v0.71.34 — adapter algebra + LISA.** Compose fine-tunes like vectors, and train big-model quality on a small-model memory budget. -- **`soup draft measure`.** Reports a draft's **acceptance rate** — the fraction of your target's - own greedy tokens the draft would have proposed correctly — plus **real plain-vs-assisted - tok/s**. Exit 0 / 2 (below `--min-acceptance`) / 1, so CI can gate on it. -- **`soup draft distill`.** Distils your tuned target into a tiny draft base (logit KD over the - existing `task: distill` trainer) and emits a **dense** model, loadable straight as an - `assistant_model`. -- **Auto-wired into serving.** Drafts land in a local registry that `soup serve --auto-spec` - consults *before* the built-in pairing table — so a draft you trained yourself just gets used. -- **What the measurement actually told us (honestly).** On `SmolLM2-360M-Instruct` ← - `SmolLM2-135M-Instruct`: the stock draft already scored **69.3%**, and distilling it changed - nothing (69.7% at 2 epochs, 69.3% at 10). Assisted decoding was a **net slowdown** (0.55–0.64×). - A small same-family draft is already at its ceiling. **That negative result is the feature - working** — it's the number you want *before* you ship speculative decoding, not after. - Whether distillation pays off on a larger or genuinely diverged pair is unproven on a 4 GB box. +- **`soup adapters arithmetic "coder + 0.5*math - toxic"`.** Task-vector algebra over LoRA + adapters (arXiv:2212.04089) — **add, scale, and NEGATE** trained behaviours into one merged + adapter. The math is done right: a LoRA's effective delta `ΔW = B·A` scales *linearly* with + each coefficient (subtracting an adapter actually removes its behaviour), not quadratically. + Same-base + backdoor-scan gated; mixed ranks refused with a clear message. +- **LISA — layerwise importance sampling (`training.lisa_enabled`).** Full-fine-tuning quality + at LoRA-like memory (arXiv:2403.17919): every N steps LISA re-activates a small random set of + decoder layers (embeddings + head always on) and freezes the rest. Live on a 4 GB GPU. +- **Verified on real models (RTX 3050).** LISA trains SmolLM2-135M end-to-end; `2·a` produces an + adapter whose ΔW is *exactly* 2× the original's — the merged adapter loads and serves. + +```bash +soup adapters arithmetic "coder - toxic" \ + --adapter coder=./coder-lora --adapter toxic=./toxic-lora -o ./cleaned + +soup train --config sft.yaml # with training.lisa_enabled: true, lisa_num_layers: 2 +``` + +
+Previous release — v0.71.33, soup draft (measure speculative decoding) + +`soup draft measure` reports a draft model's **acceptance rate** + real plain-vs-assisted tok/s +(exit 0/2/1 for CI); `soup draft distill` distils your target into a dense tiny draft, auto-wired +into `soup serve --auto-spec`. The honest result on a small same-family pair: distillation didn't +move acceptance (69.3% → 69.3%) and assisted decoding was a net slowdown — which is exactly the +number you want *before* shipping speculative decoding. ```bash soup draft measure --target ./my-tuned-model --draft HuggingFaceTB/SmolLM2-135M-Instruct \ --prompts prod-prompts.jsonl # -> acceptance %, real tok/s, ship-or-not ``` -
-Previous release — v0.71.32, ASR fine-tuning (Whisper) - -Fine-tune Whisper on your accent or domain, locally: `task='asr'` (`AsrTrainerWrapper` over HF -`Seq2SeqTrainer` + `WhisperProcessor`), `soup infer --task asr` with per-row + corpus **WER/CER**, -pure-python metrics in `soup_cli.utils.asr_metrics`, and 4 new recipes (catalog 138 → 142). -whisper-tiny (39M) / base (74M) train on a 4 GB GPU. - -```yaml -base: openai/whisper-tiny -task: asr -data: - format: asr # rows: {"audio": "clip.wav", "text": "hello world"} - audio_dir: ./data/audio -training: - asr_language: en - asr_lora: true # optional; default = full fine-tune -``` -
Full history: [CHANGELOG.md](CHANGELOG.md) · [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases). diff --git a/docs/adapters-and-governance.md b/docs/adapters-and-governance.md index 8b34051..7d790dc 100644 --- a/docs/adapters-and-governance.md +++ b/docs/adapters-and-governance.md @@ -270,6 +270,12 @@ soup adapters merge ./run-v17 ./run-v18 ./run-v19 -o ./merged --strategy ties \ soup adapters merge ./run-v17 ./run-v18 -o ./merged --strategy dare \ --density 0.5 --seed 42 +# Task-vector arithmetic (v0.71.34) — add / scale / NEGATE trained behaviours +# into one adapter (arXiv:2212.04089). Names map to dirs via --adapter name=path. +soup adapters arithmetic "coder + 0.5*math - toxic" \ + --adapter coder=./coder-lora --adapter math=./math-lora \ + --adapter toxic=./toxic-lora -o ./blended + # Leave-one-out ablation plan against a 4-hour wall-clock budget soup adapters blame ./run-v18 --dataset train.jsonl --layer q_proj.7 \ --budget 4h --shards 10 --plan-only @@ -314,6 +320,8 @@ soup adapters branches **v0.71.4:** the merge verdict is now LIVE — `soup adapters merge … --canary suite.json` lifts the `MergeReport.verdict` `UNKNOWN` stub. A pre-scored `{"baseline_scores","candidate_scores"}` suite classifies the blend OK / MINOR / MAJOR with no model load; a `{"tasks":[...]}` suite uses an injectable scorer. `--strict-verdict` exits 2 on MAJOR. The backdoor-scan and license-conflict gates now run for **every** strategy, including `cmaes`. `soup adapters branch --from-registry ` / `--attach-to-registry ` link training-env snapshots into the Registry lineage DAG (shown as a `branches` node in `soup history`). +**v0.71.34 — task-vector arithmetic.** `soup adapters arithmetic ""` applies task arithmetic (arXiv:2212.04089) to LoRA deltas: **add** (blend two skills), **scale** (`2*coder`), and — the differentiator — **negate** (`- toxic` removes a behaviour). Names in the expression map to adapter dirs via repeatable `--adapter name=path`. The math is done at the effective-delta level: a LoRA contributes `ΔW = B·A`, so the coefficient is applied so that `ΔW` scales **linearly** (subtracting an adapter actually negates its delta — a naïve element-wise sum would scale by `c²`, making negation a no-op). Adapters must share the base model (`--allow-cross-base` to override) and rank (mixed ranks are refused, not silently approximated). Each input passes the v0.71.2 backdoor-scan gate (`--allow-unscanned` to skip). Output is a single loadable adapter; exit 0 = ok, 1 = refusal. + ## Soup Cans (Shareable Recipes) diff --git a/docs/commands.md b/docs/commands.md index 49548ec..c42cc34 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -27,6 +27,7 @@ soup merge --adapter ./output Merge LoRA with base model soup merge-sharded-fsdp-weights ./shards -o merged.safetensors Consolidate FSDP shards into one safetensors (v0.71.14; --plan-only previews) soup delinearize-llama4 ./src --target ./out [--num-experts N] [--plan-only] Live Llama-4 fused-expert reshape [E*din,dout] -> [E,din,dout] + sidecar copy (v0.71.21) soup spectrum scan --model --top-percent 50 [--modules mlp,attn] [-o patch.yaml] Spectrum SNR scan (no model load) -> training.unfrozen_parameters YAML patch (v0.71.23) +soup train --config sft.yaml # training.lisa_enabled: true [lisa_num_layers lisa_interval_steps] LISA layerwise importance sampling — full-FT quality at LoRA-like memory (sft/transformers/text/quantization=none) (v0.71.34) soup export --model ./output --format gguf Export to GGUF (Ollama) soup export --model ./output --deploy ollama Export GGUF + auto-deploy to Ollama soup export --model ./output --format onnx Export to ONNX @@ -184,6 +185,7 @@ soup adapters sign [--backend unsigned|ed25519] [--key |--generat soup adapters verify [--strict] [--public-key ] Verify manifest + ed25519 signature soup adapters check-safetensors [--strict] Refuse pickle / PyTorch-classic weights soup adapters merge ... [--license ] [--license-override ] [--allow-unscanned] License + backdoor-scan gates (auto-detect license; scan FAIL refused) +soup adapters arithmetic "coder + 0.5*math - toxic" --adapter coder=

--adapter math=

--adapter toxic=

-o [--allow-unscanned --allow-cross-base] Task-vector algebra over LoRA adapters (add/scale/negate; same-rank; scan + same-base gated) (v0.71.34) soup attest emit ... [--sign ed25519 --key ] [-o att.json] in-toto/SLSA-3 attestation (+ .sig sidecar) soup attest verify --signature [--public-key ] Verify ed25519 attestation signature soup airgap-bundle --model --output [--repro-receipt ] Signed tarball for data-diode transfer (embeds repro-receipt) diff --git a/docs/peft-and-efficiency.md b/docs/peft-and-efficiency.md index 580431c..ca929c7 100644 --- a/docs/peft-and-efficiency.md +++ b/docs/peft-and-efficiency.md @@ -380,6 +380,25 @@ training: Works with and without LoRA. When used with LoRA, LoRA is applied only to unfrozen layers. +## LISA — Layerwise Importance Sampling (v0.71.34) + +LISA (Layerwise Importance Sampled AdamW, [arXiv:2403.17919](https://arxiv.org/abs/2403.17919)) targets full-fine-tuning quality at LoRA-like memory. Instead of picking layers once (that's Spectrum's static `unfrozen_parameters`), LISA re-samples a small random set of decoder layers **every N steps** and freezes the rest; the input embeddings, the LM head, and the final norm stay trainable throughout. + +```yaml +task: sft +backend: transformers +modality: text +training: + quantization: none # LISA is full-FT of the active layers + lisa_enabled: true + lisa_num_layers: 2 # decoder layers active per interval (clamped to model depth) + lisa_interval_steps: 20 # re-sample cadence, in global steps +``` + +Because only a handful of layers train at any moment (and their optimizer state is cleared when they're re-frozen), peak optimizer memory is roughly `embeddings + head + lisa_num_layers` — far below a full fine-tune, while every layer still gets updated over the course of training. LISA is `sft` + `transformers` + `text` + `quantization: none` only, and is mutually exclusive with LoRA features, `freeze_layers`/`freeze_ratio`, and Spectrum's `unfrozen_parameters` (each independently decides what trains). + +Implementation note: the model is left fully trainable at trainer-setup time so HF's optimizer (built before the first callback fires) contains every decoder parameter; the LISA callback then toggles `requires_grad` per interval — frozen parameters produce no gradient and the optimizer skips them. + ## Loss Watchdog diff --git a/pyproject.toml b/pyproject.toml index fbeb880..57493de 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "soup-cli" -version = "0.71.33" +version = "0.71.34" description = "Fine-tune and post-train LLMs in one command. No SSH, no config hell." readme = "README.md" license = "Apache-2.0" diff --git a/src/soup_cli/__init__.py b/src/soup_cli/__init__.py index 7968a48..f2e9d14 100644 --- a/src/soup_cli/__init__.py +++ b/src/soup_cli/__init__.py @@ -1,3 +1,3 @@ """Soup CLI — Fine-tune and post-train LLMs in one command.""" -__version__ = "0.71.33" +__version__ = "0.71.34"