RomanNum3ral
/
soup
mirror of https://github.com/razor-ai/soup.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

feat(v0.53.0): Quant Menu II — UD GGUFs + KV cache + NVFP4 + LF parity + save formats 

Schema-only release. Live wiring deferred to v0.53.1 (mirrors v0.50.0 /
v0.51.0 / v0.52.0 stub-then-live pattern).

- Part A — Unsloth Dynamic 2.0 GGUF ladder (14 entries: UD-Q8_K_XL ... UD-IQ1_M)
  + validate_calibration_data_path shape validator.
- Part B — IQ (12) + Apple/ARM (10) GGUF flavours in utils/gguf_quant.py;
  O(1) _LOWER_INDEX MappingProxyType for case-insensitive lookup.
- Part C — training.kv_cache_type: q8_0 | bf16 | f16 | fp8 (fp8 Hopper-only;
  MLX rejected). requires_hopper reads from spec metadata (single source).
- Part D — fp8_attention (requires quantization_aware='fp8') + nvfp4 (Blackwell)
  + native unsloth_bnb_4bit bool flags with cross-validators.
- Part E — bnb_4bit_use_double_quant + llm_int8 (explicit 8bit assertion,
  distinct from v0.41.0 load_in_8bit aliasing) + quantize_ref_model
  (extends ref-task set with grpo/kto/ppo) + quantize_reward_model.
- Part F — soup merge --save-format {fp16, 4bit, 4bit_forced} + soup export
  --format torchao with closed PTQ scheme allowlist (Int4WeightOnly,
  Int8DynActInt4, Float8DynActFloat8, NVFP4 — case-sensitive PyTorch names).

Test count: 7453 → 7610 (+157 net new across 154 tests in test_v0530.py).
ruff check soup_cli/ tests/ — clean.

5 review agents ran (python / code / security / tdd / verification);
every CRITICAL / HIGH / MEDIUM / LOW finding fixed or documented:
- O(N) gguf walk → O(1) _LOWER_INDEX MappingProxyType
- ref_tasks extended with grpo + kto + ppo (silent-no-op footgun)
- _validate_v053_bool_fields no longer coerces None → False
- requires_hopper delegates to _KV_CACHE_METADATA spec
- fp8_attention validator order: quantization_aware before MLX
- validate_calibration_data_path + validate_quant_config_path docstrings
  name the exact controls v0.53.1 CLI dispatch MUST add (TOCTOU contract)
- validate_torchao_scheme case-sensitivity documented at validator
- tautological `result == result` test replaced with allowlist invariant
- bool guards added on backend/modality/quantization across all Part D
  validators
- exact 4096/4097 boundary tests for path shape validators

Docs updated: CLAUDE.md (test counts + utils list + changelog + test-table),
README.md (What's New replaced + 5 new dedicated sections), SECURITY.md
(support window + v0.53.0 hardening entry), CONTRIBUTING.md (test count
+ utils list + test-table row), .claude/plan.md (heading + boxes + banner —
gitignored, local only).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alpamys 2026-05-12 14:31:26 +05:00
parent df7f49feda
commit 07e7214ed3
11 changed files with 2085 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
CONTRIBUTING.md
View File

@ -107,11 +107,11 @@ soup_cli/
cans/ - Shareable .can artifact format + run/publish orchestrator (v0.26.0 + v0.33.0)
data/traces/ - Trace-to-Preference harvester (v0.26.0)
data/collators.py - CrossDocCollator for sample packing (v0.33.0)
utils/ - GPU, errors, MoE, GaLore, QAT, Unsloth, vLLM, SGLang, Liger, FlashAttn, FSDP, Ring Attention, long-context, quality, curriculum, freeze, dataset-registry, mlx, peft_builder, paths, topology, launcher, mii, pipeline, cut_ce, fp8, gradient_ckpt, kernel_picker, cross_doc_attn, activation_offload, hf, spec_pairing, structured_output, metrics, tracing, auto_quant, lr_finder, grad_accum, mixed_precision, warmup, spike_recovery, convergence, v028_features, multipack_sampler, multipack, neat_packing, jinja_analyzer, quant_menu, relora, peft_patches, peft_wiring, dpo_variants, optimizer_zoo, lr_groups, loftq_init, block_expansion
utils/ - GPU, errors, MoE, GaLore, QAT, Unsloth, vLLM, SGLang, Liger, FlashAttn, FSDP, Ring Attention, long-context, quality, curriculum, freeze, dataset-registry, mlx, peft_builder, paths, topology, launcher, mii, pipeline, cut_ce, fp8, gradient_ckpt, kernel_picker, cross_doc_attn, activation_offload, hf, spec_pairing, structured_output, metrics, tracing, auto_quant, lr_finder, grad_accum, mixed_precision, warmup, spike_recovery, convergence, v028_features, multipack_sampler, multipack, neat_packing, jinja_analyzer, quant_menu, relora, peft_patches, peft_wiring, dpo_variants, optimizer_zoo, lr_groups, loftq_init, block_expansion, tts, classifier, distill, bitnet, ebft_gdpo, moe_quant, reasoning_effort, gguf_quant, kv_cache, advanced_precision, save_formats
templates/ - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
ui/ - Web UI (FastAPI + HTML/JS SPA)
tests/ - Test suite (179 files, 7456 tests)
tests/ - Test suite (180 files, 7610 tests)
examples/ - Real-world config examples and datasets
```
@ -263,6 +263,7 @@ pytest tests/ --cov=soup_cli --cov-report=html
| test_v0500_part_d.py | v0.50.0 Part D — 7 stability/efficiency knobs (`ref_model_ema_alpha` / `replay_buffer_size` / `async_grpo_prefetch` / `tis_threshold` / `mask_truncated_completions` / `defer_rerolling` / `skip_zero_advantage` / `off_policy_mask_threshold`); explicit bool-rejection field_validator across all numeric fields (tdd-guide HIGH fix); `mask_truncated_completions` requires `tis_threshold` cross-validator; SoupConfig task-gate naming every offending field; `grpo_fp16` task-gate (code-review HIGH fix) (v0.50.0 Part D) |
| test_v0500_part_e.py | v0.50.0 Part E — `task='prm'` (Process Reward Model) + `vision_grpo` flag; `validate_prm_compat` (data.format / modality / mlx gates); `validate_vision_grpo_compat` (task ∈ {grpo, ppo} / modality='vision' / non-mlx); `build_prm_trainer` deferred stub; SoupConfig integration with all rejection paths exercised (v0.50.0 Part E) |
| test_v0520.py | v0.52.0 Modality II — TTS / classifier / distill / BitNet / EBFT-GDPO / MoE quant / reasoning_effort: TTS family allowlist + per-family emotion allowlists (Orpheus + Oute) + validate_tts_compat; classifier / reranker / cross_encoder tasks + num_labels (with field_validator bool guard) + label_names dedup + classifier-only field gates; distill divergence (kl alias canonicalised, Literal excludes alias) + teacher_model + distill_temperature bounds; BitNet 1.58 quant + bitnet/tq1_0 export-format stubs + Falcon-E recipe + is_bitnet_model org-prefix detect; EBFT (structured/strided) + GDPO (standard/length_normalized/margin) variant allowlists + task gates; MoE expert quant (nf4/int8_rowwise) + train_router_only requiring moe_lora=true; reasoning_effort + train_on_eot with SFT-family task gate; 6 new recipes (5 TTS + Falcon-E BitNet); review-fix coverage (num_labels bool guard, Oute emotion allowlist, lazy-import in classifier validator, task gates, oversize / NaN / Inf matrices). Test count: 272 (v0.52.0) |
| test_v0530.py | v0.53.0 Quant Menu II — UD GGUFs + KV cache + NVFP4 + LF parity + save formats: Parts A+B GGUF (UD ladder 14 entries + IQ 12 + Apple/ARM 10 frozensets + non-overlap invariant + `validate_*` case-insensitive + rejection matrix + `is_advanced_gguf_format` union + `_LOWER_INDEX` O(1) lookup + MappingProxyType immutability + `validate_calibration_data_path` shape rejection + 4096-boundary + `export_advanced_gguf` v0.53.1 deferred stub); Part C KV cache (`KV_CACHE_TYPES` frozenset + `validate_kv_cache_type` case + bool/null/oversize/non-string rejection + `requires_hopper` delegates to spec + `get_kv_cache_spec` frozen + schema fp8-on-mlx rejected with specific message + q8_0-on-mlx allowed); Part D advanced precision (`fp8_attention` requires `quantization_aware='fp8'` BEFORE mlx-gate ordering + bool guards on every string param + schema rejects-without-fp8-qat; `nvfp4` mlx + vision rejection + bool guards; `unsloth_bnb_4bit` backend='unsloth' + quantization='4bit' rejection matrix; `apply_*` deferred); Part E LF parity (`bnb_4bit_use_double_quant` rejects none/8bit/gptq parametrize; `llm_int8` rejects default-none + 4bit; `quantize_ref_model` happy on dpo/grpo/kto + rejects sft/pretrain; `quantize_reward_model` happy on ppo/reward_model + rejects dpo; explicit `TypeError("v0.53.0 flag must be bool")` from `_validate_v053_bool_fields`; explicit-null surfaces as `valid boolean` ValidationError); Part F save formats (`MERGE_SAVE_FORMATS` lowercase normalisation + rejection matrix; `TORCHAO_PTQ_SCHEMES` CASE-SENSITIVE — `int4weightonly` rejected; `validate_quant_config_path` 4096-boundary; `MergeSaveSpec` + `TorchAOPTQSpec` frozen + MappingProxyType immutability; `merge_4bit` + `export_torchao` deferred); Cross-cutting (full 5-field YAML round-trip + cardinality invariant + tautological-assert replaced with allowlist + idempotent re-validate + `get_gguf_spec` unknown raises + bool guards on backend/modality/quantization across every Part D validator). Test count: 154 (v0.53.0) |
| test_v0510.py | v0.51.0 Model Catalog Expansion + Alternative Model Hubs: Part E hubs.py (`SUPPORTED_HUBS` + `validate_hub_name` + `validate_hub_endpoint` SSRF parity / CRLF rejection / IPv6 mapped private rejected / IPv6 loopback ok / control chars; `resolve_endpoint` env-var override; `default_endpoint` + `endpoint_env_var` + `required_hub_package` + `is_hf` with bool guards; MappingProxyType immutability); TrainingConfig `hub` field (default + Literal accept + None reject + case-insensitive normalisation + YAML round-trip) + SoupConfig `_validate_hub_supported` (mlx + non-hf rejected; mlx + hf accepted; modelers + transformers accepted); Part D MULTIPACK_ARCHITECTURES extension (20 new arches parametrize + legacy preserved + exact count=38 + frozenset immutability); Parts A/B/C 26 new recipes (parametrize over every name × {get_recipe / RecipeMeta / SoupConfig load / yaml.safe_load / model id no null/whitespace/empty parts / max_length bounds / GRPO required fields}); baichuan-sft uses `hub: modelscope`; total recipe count >= 105 (v0.51.0) |
(Note: the test-file table above covers v0.25.0–v0.35.0 + v0.47.0 + v0.48.0 + v0.49.0 + v0.50.0 only; full per-release table lives in `.claude/CLAUDE.md`.)

47
README.md
View File

@ -43,16 +43,15 @@ soup train
Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
**v0.52.0 — Modality II (TTS + Distillation + BitNet + EBFT/GDPO + MoE quant + reasoning_effort)**: 5 TTS model families, classifier / reranker / cross-encoder training, knowledge distillation, BitNet 1.58-bit, Energy-Based FT, Generalized DPO, per-expert MoE quantization, and gpt-oss reasoning-effort schema — schema-only release; live trainer / loss / export wiring lands in v0.52.1.
**v0.53.0 — Quant Menu II (UD GGUFs + KV cache + NVFP4 + LF parity + save formats)**: Unsloth Dynamic 2.0 GGUF ladder (UD-Q8_K_XL … UD-IQ1_M), IQ + Apple/ARM GGUF flavours, `kv_cache_type` (q8_0 / bf16 / f16 / fp8), FP8 attention, NVFP4 (Blackwell), explicit `unsloth_bnb_4bit`, BNB double-quant, ref/reward model quantization, merge-4bit save, TorchAO PTQ export schema — schema-only release; live llama.cpp imatrix + serve / merge / export wiring lands in v0.53.1.
- **TTS fine-tuning.** New `task='tts'` + `modality='audio_out'` with a closed allowlist of five model families (Orpheus, Sesame-CSM, Llasa, Spark, Oute) and per-family emotion-tag allowlists for Orpheus + Oute. Five new recipes: `orpheus-tts-sft`, `sesame-csm-tts`, `llasa-tts`, `spark-tts`, `oute-tts`.
- **Classifier / reranker / cross-encoder tasks.** New `task` values `classifier`, `reranker`, `cross_encoder` with `num_labels`, `classifier_kind` (single_label / multi_label), `label_names` (1024-cap + dedup + null-byte rejection). Cross-validator requires `num_labels` and matches `len(label_names) == num_labels`.
- **Knowledge distillation.** New `task='distill'` + `teacher_model` (HF id or local path) + `distill_divergence` (`kl` / `forward_kl` / `reverse_kl` / `js` — `kl` canonicalises to `forward_kl`) + `distill_temperature` (math.isfinite + [0.05, 100.0] bounds).
- **BitNet 1.58-bit + GGUF export schema.** `quantization='bitnet_1.58'` accepted for task ∈ {sft, pretrain, dpo} on transformers/unsloth backends; new BitNet / TQ1_0 export-format allowlist; `falcon-e-bitnet-sft` recipe shipped.
- **EBFT + GDPO.** Energy-Based FT variants (`structured` / `strided`) gated to `task='sft'`; Generalized DPO variants (`standard` / `length_normalized` / `margin`) gated to DPO-family tasks.
- **MoE expert quant + router-only training.** `moe_expert_quant: nf4 | int8_rowwise` and `train_router_only: true` — both require `moe_lora=true` (silent-no-op rejection at config-load).
- **gpt-oss `reasoning_effort: low | medium | high`** + `train_on_eot: bool`. Both gated to the SFT-family task set (sft / pretrain / distill / classifier / reranker / cross_encoder) — non-SFT tasks reject loudly.
- **+272 net new tests** (7184 → 7456). 4 review agents (python-reviewer, security-reviewer, code-reviewer, tdd-guide) ran; every finding fixed: `num_labels` bool-before-int guard, `reasoning_effort` / `train_on_eot` task-gate, Oute emotion allowlist, `_validate_classifier_compat` lazy-import guard, `_MAX_LEN` → `_MAX_REASONING_EFFORT_LEN`, `DIVERGENCES` derived from alias map, sister-function bool guards on every compat helper, expanded TDD coverage (oversize on EBFT variant, full GDPO rejection matrix, explicit-exc on temperature bounds, TTS compat input guards, recipe model-id null/whitespace check).
- **Unsloth Dynamic 2.0 GGUF ladder.** 14-entry closed allowlist (UD-Q{8..2}_K_XL + UD-IQ{4_XS, 3_M, 3_XXS, 2_M, 2_XS, 2_XXS, 1_M, 1_S}) with `validate_ud_gguf_format` case-insensitive canonical normalisation. `--calibration-data <jsonl>` flag shape-validates now; cwd-containment + TOCTOU symlink rejection land at CLI dispatch in v0.53.1.
- **IQ + Apple/ARM GGUF.** 12-entry IQ family (IQ1/2/3/4 — including IQ4_NL non-linear) + 10-entry Apple/ARM-friendly set (Q4_0_4_4 / Q4_NL / Q5_K_M / etc.) wrapped in `MappingProxyType` metadata.
- **KV cache types.** New `training.kv_cache_type: q8_0 | bf16 | f16 | fp8`. FP8 is Hopper-only — cross-validator rejects `fp8` on the MLX backend; the SM-capability check fires at serve construction.
- **FP8 attention + NVFP4 + native `unsloth_bnb_4bit`.** Three new bool flags. `fp8_attention=true` requires `quantization_aware='fp8'` and a non-MLX backend; `nvfp4=true` is gated to CUDA + text modality (Blackwell SM ≥ 12 check is runtime-only); `unsloth_bnb_4bit=true` requires `backend='unsloth'` + `quantization='4bit'`.
- **LF / Axolotl parity.** `bnb_4bit_use_double_quant` (requires `quantization='4bit'`), `llm_int8` (asserts `quantization='8bit'` — distinct from v0.41.0 `load_in_8bit` aliasing), `quantize_ref_model` (extends v0.40.5 Quant Menu to the ref model on DPO/IPO/SimPO/ORPO/BCO/KTO/GRPO/PPO/preference), `quantize_reward_model` (PPO + reward_model tasks).
- **Advanced save formats.** `soup merge --save-format 4bit | 4bit_forced` (single BNB-4bit merged checkpoint without dequant/merge/requant cycle) and `soup export --format torchao --quant-config <yaml>` (closed allowlist Int4WeightOnly / Int8DynActInt4 / Float8DynActFloat8 / NVFP4) — schema lands now; live writers land in v0.53.1.
- **+157 net new tests** (7453 → 7610) across 154 tests in `test_v0530.py`. Five review agents (python / code / security / tdd / verification) ran in parallel; every CRITICAL / HIGH / MEDIUM / LOW finding fixed or documented: O(1) `_LOWER_INDEX` for GGUF lookup, ref-task set extended with GRPO + KTO, `_validate_v053_bool_fields` no longer silently coerces `None`, `requires_hopper` reads from spec metadata, `fp8_attention` validator order swapped so the `quantization_aware='fp8'` error fires first, `validate_calibration_data_path` / `validate_quant_config_path` docstrings name the exact controls v0.53.1 CLI dispatch must add.
## Why Soup?
@ -3612,6 +3611,36 @@ Both reject silently-no-op combinations: setting either flag without `moe_lora=t
`training.reasoning_effort: low | medium | high` injects a system-prefix token at training time for gpt-oss models; `training.train_on_eot: true` includes explicit EOT/EOS control tokens in the SFT loss (axolotl `train_on_eot`). Both are gated to the SFT-family task set (`sft` / `pretrain` / `distill` / `classifier` / `reranker` / `cross_encoder`) — setting them on DPO / GRPO / PPO / etc. fails at config load. Live formatter wiring in v0.52.1.
## Unsloth Dynamic 2.0 GGUF Ladder (v0.53.0)
`soup export --format gguf-ud --calibration-data <calib.jsonl>` is the planned dispatch surface for the 14-entry UD ladder (`UD-Q8_K_XL` … `UD-IQ1_M`). v0.53.0 ships the closed-allowlist validators, `MappingProxyType`-wrapped metadata, and a calibration-data path shape check; live llama.cpp `imatrix` invocation lands in v0.53.1. The IQ + Apple/ARM-friendly GGUF flavours (`IQ4_NL`, `Q4_0_4_4`, `Q5_K_M`, etc.) ship as separate frozensets so future export-CLI dispatch can pick by family.
## KV Cache Types (v0.53.0)
`training.kv_cache_type: q8_0 | bf16 | f16 | fp8` controls the inference-time KV cache element type. `fp8` is Hopper-only; the MLX backend is rejected at config load. The other three types pass through every backend in v0.53.0; v0.53.1 may narrow MLX further once the runtime serve path lands. The Hopper SM-capability check (compute capability ≥ 9.0) is intentionally runtime-only — `pip install -U vllm` users on a Hopper box won't trip it unless they ship a Hopper-incompatible GPU into the runtime.
## FP8 Attention + NVFP4 + Native `unsloth_bnb_4bit` (v0.53.0)
Three new TrainingConfig bools extend the v0.28.0 FP8 menu:
- `fp8_attention: true` — requires `quantization_aware: fp8` AND a non-MLX backend. Targets axolotl parity for FP8 attention on Hopper+ GPUs.
- `nvfp4: true` — Blackwell-only FP4 training. Gated to non-MLX + `modality: text`; the SM ≥ 12.0 runtime check fires at trainer construction.
- `unsloth_bnb_4bit: true` — promotes "Unsloth Dynamic 4-bit" from an implicit `backend=unsloth + quantization=4bit` combo to a named flag. Mutual rejection of inconsistent combos at config load.
Cross-validator ordering picks the most actionable error: `quantization_aware='fp8'` prerequisite fires before the MLX rejection on `fp8_attention`, so a YAML missing both surfaces the deeper issue first.
## LF / Axolotl Quant Parity (v0.53.0)
- `bnb_4bit_use_double_quant: true` — requires `quantization: 4bit`. Activates BNB's double-quantization. Combinations with the Quant Menu formats (gptq / awq / hqq:Nbit / aqlm / eetq / mxfp4 / fp8) are rejected at config load.
- `llm_int8: true` — an explicit 8-bit assertion. Unlike v0.41.0 `load_in_8bit` (which **rewrites** `quantization` to `8bit`), `llm_int8` enforces that the user has ALSO set `quantization: 8bit`. Mismatch raises with an actionable message.
- `quantize_ref_model: true` / `quantize_reward_model: true` — extend the v0.40.5 Quant Menu wiring to the reference / reward models inside preference and RLHF training. `quantize_ref_model` accepts any task with a reference policy (`dpo / ipo / simpo / orpo / bco / kto / preference / grpo / ppo`); `quantize_reward_model` accepts `ppo / reward_model`.
## Advanced Save Formats (v0.53.0)
`soup merge --save-format 4bit` and `--save-format 4bit_forced` will write a single BNB-4bit-quantized merged checkpoint without the wasteful dequant → merge → requant cycle (unsloth `merged_4bit` recipe). v0.53.0 ships the closed allowlist + spec metadata; the live writer lands in v0.53.1.
`soup export --format torchao --quant-config <yaml>` is the planned PTQ export surface for `torchao.quantize_` + `save_pretrained`. Four schemes are allowlisted: `Int4WeightOnly`, `Int8DynActInt4`, `Float8DynActFloat8`, `NVFP4`. CASE-SENSITIVE — these are PyTorch class names and `torchao.quantize_` looks them up by exact name. Diverges from `--save-format` (lowercase-normalised) on purpose; documented at both validators.
## Changelog
See [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases) for version history.

11
SECURITY.md
View File

@ -9,14 +9,14 @@ We provide security updates for the following versions:
- **Versions older than 3 minor versions:** No support
Example:
- v0.52.0 -- Full support (latest)
- v0.53.0 -- Full support (latest)
- v0.52.0 -- Full support
- v0.51.0 -- Full support
- v0.50.0 -- Full support
- v0.49.0 -- Full support
- v0.48.0 -- Full support
- v0.47.0 -- Bug-fix support only
- v0.46.0-v0.46.x -- Bug-fix support only
- v0.45.x and below -- No support
- v0.48.0 -- Bug-fix support only
- v0.47.0-v0.47.x -- Bug-fix support only
- v0.46.x and below -- No support
## Reporting a Vulnerability
@ -147,6 +147,7 @@ No known critical vulnerabilities in current releases.
- **v0.32.0 — Training Stability & Auto-Tuning**: `--find-lr-output` containment via shared `utils/paths.is_under_cwd` (prevents writes outside cwd); `save_lr_finder_report` rejects NaN / Infinity floats in `lrs` / `losses` and serialises with `allow_nan=False` (keeps the report parser-safe); `compute_lr_schedule` rejects non-positive `start_lr`, inverted ranges, and `num_steps` outside `[2, 10_000]`; `pick_mixed_precision` rejects empty / null-byte / >200-char model names and resolves multi-version quirks (`qwen2.5` vs `qwen2`, `phi-3.5` vs `phi-3`) by longest-substring-first iteration so an added family can never accidentally make a more-specific entry dead code; `compute_warmup_steps` clamps to `[10, 1000]` with a `ratio==0.0` short-circuit matching HF Trainer's "no warmup" convention; `SpikeRecoveryStrategy` is `@dataclass(frozen=True)` (post-construction mutation cannot bypass validation), `max_attempts ∈ [1, 10]`, `lr_decay ∈ (0, 1)`, `min_lr > 0`; cross-validator `_validate_spike_recovery_requires_watchdog` rejects `loss_spike_recovery=true, loss_watchdog=false` at config-load (fails fast instead of never triggering); `convergence_window ∈ [5, 10_000]`, `convergence_rel_tol ∈ (0, 1]`, `recommend_action` reuses `detect_plateau` so plateau heuristic stays single-source-of-truth; `GradAccumMonitor.recommend()` caps doubled `accum` at `MAX_ACCUM=1024` so a runaway advisory loop cannot blow up DataLoader prefetch; `generate_config` validates BOTH the YAML output path AND the embedded `decisions["output"]` field via `is_under_cwd` (closes the gap where a crafted `decisions["output"]="../../etc"` would have silently propagated into the rendered YAML)
- **v0.34.0 — Observability & Dev UX**: `.crash` bundle generator (`utils/crash.py`) recursively redacts `hf_*` / `sk-*` / `Bearer …` token-shaped strings in any captured `config` and metric tail before serialisation, so a `.crash` file shared on a public GitHub issue cannot leak credentials; `output_dir` is reduced to `os.path.basename` so `$HOME` doesn't leak; `write_crash_bundle` uses `os.path.realpath + commonpath` for cwd containment (Windows-safe; raises `ValueError` not `PermissionError` so callers cannot silently swallow with `except OSError`); filename appends `secrets.token_hex(4)` so two crashes in the same UTC second don't collide; bundle truncated to `MAX_BUNDLE_BYTES=1_000_000`. `train.py` crash-write surfaces failures to the user (no silent missing-bundle). `profiling.py` `resolve_trace_path` rejects empty / `.` / `..` / `/` / `\\` / null-byte `run_id` (closes the `output_dir/profiles/../trace.json` escape) and uses `os.path.realpath + is_under_cwd`; profiles dir is created only on successful torch import (no stale empty dirs on torch-less CI). `tracker.get_run` LIKE-prefix match escapes `%` / `_` / `\\` and uses `ESCAPE '\\'` so a crafted `run_id` cannot widen the match (mirrors v0.26.0 registry policy). Lazy schema migration (`_ensure_schema`) tolerates the "duplicate column" race when two CLI processes start simultaneously on a fresh DB (fork-based multi-GPU training, TUI auto-refresh). `runs.py show/replay/clean` switched user `run_id` rendering to `markup_escape` and switched `clean` containment from broken `Path.resolve() + relative_to()` to project-standard `os.path.realpath + is_under_cwd`. `tui_app.py` lazy-imports `ExperimentTracker` and `markup_escape`s every DB-sourced string before passing into Textual widgets so a crafted base_model / experiment_name cannot inject `[bold red]…[/]` markup. `run_cost.estimate_run_cost_usd` rejects `bool` in `num_gpus` (bool is a subclass of int — same defence as v0.30.0 `Candidate.__post_init__`); duration clamped to `[0, 1 year]`; unknown GPU returns `None` so callers render `—` instead of fabricating `$0.00`. `log_level.parse_log_level` rejects non-string + null-byte input.
- **v0.33.0 — Live Wire**: RLVR `code_exec_reward` adds OS-level isolation (Linux best-effort `os.unshare(CLONE_NEWUSER|CLONE_NEWNET|CLONE_NEWPID)`, macOS `sandbox-exec` with default-deny `MACOS_SANDBOX_PROFILE` narrowed to a 3-name `mach-lookup` allowlist to prevent DNS / NSURLSession bypass of `(deny network*)`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat + S_ISLNK` + `shutil.rmtree(onerror=_abort_on_symlink)` so a symlink encountered mid-walk aborts rather than escapes; `run_gate` wraps each task scorer in a typed `try/except` so backend failures produce `score=None, error=str(exc)` (never silent `score=1.0`); `_parse_judge_url` removes the bare `http://` catch-all (defence-in-depth after the Pydantic GateTask validator); `soup can run` requires `--yes` or explicit consent callback and raises `ValueError` (not `PermissionError`, which is an `OSError` subclass that broad `except` blocks would swallow); GGUF `rglob` result for ollama deploy is `realpath+commonpath` checked against extract_dir (prevents symlink escape from a crafted can); `DeployTarget.path` validator normalises mixed `\\`/`/` separators before splitting (closes a Windows `..` bypass); `CAN_FORMAT_VERSION` 1→2 (additive — v1 still loads); `soup can publish` validates `repo_id` via `utils/hf.validate_repo_id`, resolves token via `resolve_token`, sanitises commit messages (first-line, 200-char cap), uses HTTPS-only HfApi; `_write_spike_recovery_hint` adds `is_under_cwd` containment check on `args.output_dir` from raw HF `TrainingArguments`; `lookup_entry_by_output_dir` emits `ResourceWarning` when 1000-row scan limit is hit (no silent miss); `CrossDocCollator` no longer mutates input feature dicts (HF Dataset rows are cached and reused — mutation broke subsequent batches); `Candidate` rejects `bool` in `score`/`latency_ms` (was sneaking past `int` isinstance check); `evaluate_candidate` latency mean now divides by *completed* prompts (excludes crashed) so a broken candidate isn't artificially fast; `auto_quant.run_auto_quant_picker` soft-falls-back to highest-scored candidate when no candidate clears `min_score` (server still binds); `build_logits_processors` returns `[]` when neither `outlines` nor `lm-format-enforcer` is installed (server degrades to free-form rather than 500); MII server uses loopback-only CORS, max_tokens cap [1, 16384], stream rejection, generic 500 with no stack-trace leak; `os.execvp` auto-reexec uses list args (no shell), all forwarded flags pre-validated; `cleanup_extract_dir` uses `os.path.commonpath` (Windows-safe) instead of `startswith`; `_run_subprocess` catches `TimeoutExpired` and returns rc=124 (coreutils convention) instead of an unhandled traceback; new `eval_results` and `tensorrt` artifact kinds in `RegistryStore._VALID_KINDS`
- **v0.53.0 — Quant Menu II (UD GGUFs + KV cache + NVFP4 + LF parity + save formats)**: 6 schema-only Parts; live wiring deferred to v0.53.1. Every new validator follows the project's established hardening policy: closed allowlists (`UD_GGUF_FORMATS`, `IQ_GGUF_FORMATS`, `APPLE_ARM_GGUF_FORMATS`, `KV_CACHE_TYPES`, `MERGE_SAVE_FORMATS`, `TORCHAO_PTQ_SCHEMES`) as `frozenset` so registries cannot be mutated; `_GGUF_METADATA` / `_KV_CACHE_METADATA` / `_MERGE_METADATA` / `_TORCHAO_METADATA` wrapped in `MappingProxyType`; `_LOWER_INDEX` for GGUF lookup is also `MappingProxyType`-wrapped (replaces O(N) walk with O(1) lookup — code-review MEDIUM fix). All string validators reject non-string / bool / empty / null-byte / oversize with case-insensitive normalisation (matches v0.41.0 `validate_optimizer_name` / v0.51.0 `validate_hub_name` policy); `validate_torchao_scheme` is INTENTIONALLY case-sensitive (PyTorch class names — `torchao.quantize_` looks them up by exact name) with the asymmetry documented at both validators (security-review LOW fix). `validate_calibration_data_path` + `validate_quant_config_path` are shape-only at this release; their docstrings name the exact controls a v0.53.1 CLI dispatch contributor MUST add (`os.path.realpath` + `os.path.commonpath` cwd containment, `os.lstat` + `stat.S_ISLNK` symlink rejection before `open()`, existence check, `yaml.safe_load`-only for quant configs) — closes the security-review MEDIUM "documentation gap at trust boundary" finding. SoupConfig cross-validators: `_validate_fp8_attention_compat` (requires `quantization_aware='fp8'` BEFORE the MLX gate so the more actionable error fires first — code-review MEDIUM fix); `_validate_nvfp4_compat` (non-MLX + `modality='text'`; Blackwell SM ≥ 12.0 runtime check fires at trainer construction); `_validate_unsloth_bnb_4bit_compat` (requires `backend='unsloth'` + `quantization='4bit'`); `_validate_bnb_4bit_double_quant` (requires `quantization='4bit'` — rejects `none`/`8bit`/Quant-Menu); `_validate_llm_int8_alias` (asserts `quantization='8bit'`, deliberately disjoint from v0.41.0 `load_in_8bit` aliasing); `_validate_quantize_ref_reward` (extended ref-task allowlist `{dpo, ipo, simpo, orpo, bco, kto, preference, grpo, ppo}` per code-review HIGH fix — first-cut omitted grpo + kto + ppo which all have reference policies); `_validate_kv_cache_type_supported` (only `fp8` gated to non-MLX in v0.53.0; q8_0/bf16/f16 pass-through documented at validator site so v0.53.1 contributor sees the gate immediately). `requires_hopper` reads from `_KV_CACHE_METADATA` spec — single source of truth so adding a Hopper-only type means flipping the spec field only (code-review MEDIUM fix). All 7 new bool fields share `_validate_v053_bool_fields` `field_validator(mode='before')` that rejects bool-as-int with explicit `TypeError("v0.53.0 flag must be bool")` and passes `None` through to Pydantic's `default=False` rather than silently coercing it (python-review MEDIUM fix — `fp8_attention: null` in YAML now surfaces as a "valid boolean" ValidationError instead of masquerading as `False`). Known limitations: (1) Every live wiring is deferred to v0.53.1 — `export_advanced_gguf`, `apply_kv_cache_type`, `apply_fp8_attention`, `apply_nvfp4`, `merge_4bit`, `export_torchao` all raise `NotImplementedError` with explicit `v0.53.1` markers. (2) `validate_calibration_data_path` + `validate_quant_config_path` are shape-only this release; CLI dispatch in v0.53.1 MUST add cwd-containment + TOCTOU symlink rejection. (3) `kv_cache_type` MLX permissive policy: only `fp8` is rejected, the other three pass-through; v0.53.1 may narrow further. (4) Hopper SM-capability check is runtime-only — schema accepts `kv_cache_type='fp8'` + `fp8_attention=true` without GPU probe. (5) NVFP4 + Blackwell (SM ≥ 12.0) check is runtime-only. (6) `bnb_4bit_use_double_quant` only gated against `quantization`, not against `quantization_aware` — the latter combination is already rejected by v0.28.0 Quant-Menu + QAT cross-validator. (7) `llm_int8` is an assertion not an aliaser — diverges from v0.41.0 `load_in_8bit` design on purpose. (v0.53.0)
- **v0.52.0 — Modality II (TTS + Distillation + BitNet + EBFT-GDPO + MoE quant + reasoning_effort)**: 7 schema-only Parts; live trainer / loss / export wiring deferred to v0.52.1. Every new validator follows the project's established hardening policy: closed allowlist (`SUPPORTED_TTS_FAMILIES`, `CLASSIFIER_TASKS`, `DIVERGENCES`, `BITNET_QUANT_FORMATS`, `BITNET_EXPORT_FORMATS`, `EBFT_VARIANTS`, `GDPO_VARIANTS`, `MOE_EXPERT_QUANT_FORMATS`, `REASONING_EFFORT_LEVELS`, per-family `_FAMILY_EMOTIONS`) wrapped in `frozenset` / `MappingProxyType` so registries cannot be mutated at runtime; `validate_*` helpers reject non-string / bool / empty / null-byte / oversize / unknown inputs with case-insensitive normalisation (matches v0.41.0 `validate_optimizer_name` / v0.50.0 `grpo_variant` / v0.51.0 `hub` policy); float validators (`validate_distill_temperature`, `validate_ebft_temperature`) gate on `math.isfinite` to reject NaN AND `±inf` (matches v0.32.0 `save_lr_finder_report` policy). `field_validator(mode="before")` on `num_labels` (security-review HIGH fix) rejects `bool` before Pydantic's `ge=1` coercion silently treats `True` as `1`. Field validator on `reasoning_effort` routes through the shared `validate_reasoning_effort` helper so the schema and runtime validator agree on what's accepted (security-review MEDIUM fix). SoupConfig cross-validators: `_validate_tts_compat` (requires `task='tts'` + `modality='audio_out'` + non-MLX backend; per-family emotion allowlist via `_FAMILY_EMOTIONS`), `_validate_classifier_compat` (with lazy-import early-return — code-review HIGH fix — so SFT hot path doesn't pay import cost; requires `num_labels` on classifier tasks; rejects classifier-only fields outside the task family with named offenders), `_validate_distill_compat` (requires `teacher_model` when `task='distill'`; rejects distill-only fields outside the task), `_validate_bitnet_compat` (gates to non-MLX + text-modality + task ∈ {sft, pretrain, dpo}), `_validate_ebft_compat` + `_validate_gdpo_compat` (task-family gates), `_validate_moe_expert_quant_compat` (requires `moe_lora=true` to prevent silent no-op), `_validate_reasoning_effort_task_gate` (code-review HIGH fix — rejects `reasoning_effort` + `train_on_eot` outside the SFT-family task set with named offenders; mirrors v0.50.0 GRPO stability task-gate policy). Public `DIVERGENCES` frozenset is derived from `_DIVERGENCE_ALIASES` so adding a new alias updates both the accepted-input set and the error message in lockstep (review fix LOW). `validate_bitnet_export` enforces a closed-allowlist canonical form for `soup export --format <bitnet|tq1_0>`, both of which are CLI-registered with a yellow advisory panel + `Exit(0)` stub (no artifact written until v0.52.1 — the format flag is accepted so existing scripts pinned to v0.52.0 will not break). 6 new YAML recipes appended (5 TTS + Falcon-E BitNet) — every entry is exercised by `tests/test_v0520.py` for `load_config_from_string` round-trip + `_no_null_or_whitespace` model-id check (mirrors v0.51.0 review-fix LOW). Known limitations: (1) Every live trainer / loss / export path is deferred to v0.52.1 — `build_tts_trainer`, `build_classifier_trainer`, `build_distill_trainer`, `build_bitnet_trainer`, `export_bitnet_gguf`, `apply_ebft_loss`, `apply_gdpo_loss`, `apply_moe_expert_quant` all raise `NotImplementedError` with explicit `v0.52.1` markers; schema accepts every new task / quant / variant + the CLI stub for `soup export --format bitnet/tq1_0` prints a deferred-advisory panel and exits 0. (2) `modality='audio_out'` accepted on non-TTS tasks — design choice this release so future audio-output tasks (ASR / V2A) can reuse it; today's runtime trainer dispatch must check `task == 'tts'` to avoid silent routing into the deferred TTS path. (3) Oute emotion allowlist is a tight 6-entry subset (neutral / happy / sad / angry / calm / excited); operators wanting custom emotions will need a v0.52.1 patch to extend `OUTE_EMOTIONS`. (4) `is_bitnet_model` is best-effort heuristic over name prefixes (`bitnet`, `falcon-e`, `1bitllm`, `onebit`); a BitNet checkpoint published under an org without any of those prefixes returns False. This is detection, not gating — the trainer wrapper (v0.52.1) loads the model regardless of the heuristic. (5) `quantization='bitnet_1.58'` gated to task ∈ {sft, pretrain, dpo} — extending to GRPO / PPO / RewardModel requires upstream onebitllms RL kernels not yet shipped. (v0.52.0)
- **v0.51.0 — Model Catalog Expansion + Alternative Model Hubs**: 5 release Parts. New `soup_cli/utils/hubs.py` ships closed allowlist `SUPPORTED_HUBS = frozenset({hf, modelscope, modelers})` + three `MappingProxyType`-wrapped registries (`_HUB_DEFAULT_ENDPOINTS` / `_HUB_ENDPOINT_ENV` / `_HUB_PACKAGE`) so the registry cannot be mutated at runtime (matches v0.36.0 `_REGISTRY` policy). `validate_hub_name` rejects non-string / bool / empty / null-byte / >32-char / unknown with case-insensitive normalisation (matches v0.41.0 `validate_optimizer_name` policy). `validate_hub_endpoint` is the SSRF kernel — full parity with v0.29.0 `utils/hf.resolve_endpoint`: scheme allowlist (`http`/`https` only), null-byte rejection, **control-character / CRLF rejection** added in v0.51.0 as a defence-in-depth review fix (defends against URL-as-HTTP-header injection if the URL ever flows into a raw HTTP client), `0.0.0.0` explicitly rejected, plain HTTP only for loopback `{localhost, 127.0.0.1, ::1}`, RFC1918 / link-local / cloud-metadata IPs (169.254.x) rejected via `ipaddress.ip_address` for plain HTTP. `resolve_endpoint(hub, *, env=None)` looks up the per-hub env var (`HF_ENDPOINT` / `MODELSCOPE_ENDPOINT` / `MODELERS_ENDPOINT`) and runs the override through `validate_hub_endpoint`; default endpoints are baked-in HTTPS URLs. `is_hf` rejects `bool` explicitly (review fix HIGH — bool is a subclass of int and would have silently fallen through `hub.lower() == "hf"` → `False`, which happens to be correct by accident but violates the contract; matches v0.30.0 `Candidate` / v0.34.0 `estimate_run_cost_usd` policy). `TrainingConfig.hub: Literal["hf","modelscope","modelers"]` field gets a `field_validator(mode="before")` `_normalize_hub` that delegates to `validate_hub_name` so `hub: HF` in YAML normalises to `"hf"` (review fix HIGH — first-cut had Pydantic Literal exact-match while `validate_hub_name` was case-insensitive, breaking the v0.41.0 `validate_optimizer_name` / v0.50.0 `grpo_variant` / `rollout_backend` policy of agreement between schema and shared validator). SoupConfig `_validate_hub_supported` cross-validator rejects `hub != 'hf'` on `backend == 'mlx'` with a distinct error message (review fix HIGH — `mlx-lm` only downloads from HF Hub; without this gate a `backend: mlx` + `hub: modelscope` config would silently pass schema load and fail at runtime with a confusing `mlx-lm` error). 26 new YAML recipes appended to `soup_cli/recipes/catalog.py` — every entry is exercised by `tests/test_v0510.py` via `load_config_from_string` round-trip + `yaml.safe_load` (no Python tags / no template injection / no credential leak in the YAML strings) + a `_no_null_or_whitespace` model-id check that rejects empty path components (review fix LOW — first-cut allowed `"/name"` leading-slash IDs to pass). Two non-`<N>B` `size` strings (`"image"` / `"ocr"` / `"moe"` / `"medium"`) were normalised to `"N/A"` (review fix MEDIUM — `search_recipes(size=…)` would silently miss those entries, and the autopilot VRAM estimator could not parse them). Known limitations: (1) Live downloader / uploader / push integration deferred to v0.51.1 — `TrainingConfig.hub` schema lock-in ships now (Literal accept + MLX cross-validator + case-normalisation), but `soup data download --hub modelscope` and `soup push --hub modelers` still route through the existing HF Hub code path; the actual `modelscope-sdk` / `openmind-hub` adapters are the v0.51.1 deliverable. Same stub-then-live pattern as v0.27.0 MII / v0.37.0 multipack / v0.50.0 GRPO Plus. (2) Speculative / aspirational `base` model IDs in some Part A/C recipes — the catalog ships entries for `openai/gpt-oss-{20,120}b`, `THUDM/glm-5`, `Qwen/Qwen-Image`, `deepseek-ai/DeepSeek-OCR`, `PaddlePaddle/PaddleOCR-VL`, `google/embeddinggemma-300m` so users have ready-made recipes the moment those repos go live (matches the plan's "match Unsloth's day-zero coverage" directive). Recipes for not-yet-published repos will surface a clear HF Hub 404 when the user runs `soup train --recipe <name>`. (3) DNS-resolved private hostnames not blocked — `validate_hub_endpoint` only rejects literal RFC1918 / link-local IP addresses; a hostname like `corp-proxy.internal` that DNS-resolves to a private IP is accepted at validation time (mirrors the v0.29.0 `HF_ENDPOINT` policy — DNS resolution is intentionally not performed in this local-tool threat model). (v0.51.0)
- **v0.50.0 — GRPO Plus (RL parity)**: 22 features across 5 Parts shipped as schema-only (closed allowlists + Pydantic validators + NotImplementedError stubs for live wiring deferred to v0.50.1). All new validators follow the project's bool-rejection-before-int policy (matches v0.30.0 `Candidate`); closed-allowlist `validate_grpo_variant` / `validate_rollout_backend` reject non-string / bool / empty / null-byte / oversize / unknown inputs with actionable error messages and case-insensitive normalisation. `validate_grpo_delta` is bool-first / `math.isfinite` / `(0, 1]` bounded (matches v0.32.0 `save_lr_finder_report` / v0.41.0 Part B `lr_groups` policy). New `_VARIANT_METADATA` (Part A) and `_BACKEND_METADATA` (Part C) are `MappingProxyType`-wrapped frozen-dataclass registries (matches v0.36.0 `_REGISTRY` / v0.41.0 `_OPTIMIZER_PACKAGES` policy). Security-review fixes: (1) `grpo_delta` schema gets an explicit `field_validator(mode='after')` calling `math.isfinite` — Pydantic's `gt=0, le=1` bounds only incidentally reject NaN (since `NaN > 0` is False); the explicit validator prevents a future Pydantic change from regressing the guard. (2) `validate_long_context_grpo_compat` adds null-byte rejection on `task` AND `backend` strings + a `bool` guard on `use_ring_attention` (parity with `validate_grpo_variant` / `validate_rollout_backend`). (3) `validate_vllm_sleep_mode_compat` adds null-byte rejection on `backend`. Code-review HIGH fixes: (4) `_validate_grpo_stability_task_gate` now includes `grpo_fp16` in the GRPO-only-fields list — previously a user could silently set `grpo_fp16: true` on `task='sft'` and have it no-op. (5) `_validate_vllm_sleep_mode` now requires `task='grpo'` (sleep mode is a between-rollouts feature, meaningless on SFT) and rejects with a `task='grpo'` message. TDD-review HIGH fixes: (6) new `_reject_bool_on_grpo_numerics` field_validator on every Part D numeric field + `grpo_delta` explicitly rejects `bool` before Pydantic's `True→1` coercion (matches v0.30.0 / v0.41.0 Part B / v0.43.0 Part B policy). Known limitations: (1) Every live loss kernel / launcher (`apply_variant_loss`, `apply_vllm_sleep_mode`, `launch_rollout`, `build_prm_trainer`) raises `NotImplementedError` with explicit `v0.50.1` markers — same stub-then-live pattern as v0.27.0 MII / v0.37.0 multipack / v0.41.0 LLaMA Pro / v0.45.0 plugins / v0.48.0 curriculum / v0.49.0 LongLoRA. (2) `long_context_grpo` requires Tiled MLP (v0.56.0 Part A) to actually run; the schema gate ships now so v0.50.0 configs are stable. (3) `vision_grpo=true` does not check whether the base model is actually a VLM — upstream trainer surfaces that error loudly. (4) The 7 stability knobs schema-validate but none are wired into a live callback in this release; `replay_buffer_size`, `defer_rerolling`, and `skip_zero_advantage` are pure schema lock-ins. (v0.50.0)

2
pyproject.toml
View File

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "soup-cli"
version = "0.52.0"
version = "0.53.0"
description = "Fine-tune LLMs in one command. No SSH, no config hell."
readme = "README.md"
license = "Apache-2.0"

2
soup_cli/__init__.py
View File

@ -1,3 +1,3 @@
"""Soup CLI — Fine-tune LLMs in one command."""
__version__ = "0.52.0"
__version__ = "0.53.0"

254
soup_cli/config/schema.py
View File

@ -1184,6 +1184,106 @@ class TrainingConfig(BaseModel):
"convention. (v0.52.0)"
),
)
# ---- v0.53.0 Quant Menu II — UD GGUFs + KV cache + NVFP4 ---------------
# Part C — KV cache types (serve-side hint, captured here for round-trip).
kv_cache_type: Optional[Literal["q8_0", "bf16", "f16", "fp8"]] = Field(
default=None,
description=(
"KV-cache element type for inference (q8_0 / bf16 / f16 / fp8). "
"Schema-only in v0.53.0; live wiring deferred to v0.53.1."
),
)
# Part D — Train-time advanced precision.
fp8_attention: bool = Field(
default=False,
description=(
"Extend the v0.28.0 FP8 menu to FP8 attention "
"(axolotl-parity flag). Requires quantization_aware='fp8'. "
"Schema-only in v0.53.0; live wiring deferred to v0.53.1."
),
)
nvfp4: bool = Field(
default=False,
description=(
"Blackwell-only NVFP4 training (unsloth + axolotl). "
"Schema-only in v0.53.0; live wiring deferred to v0.53.1."
),
)
unsloth_bnb_4bit: bool = Field(
default=False,
description=(
"Promote Unsloth Dynamic 4-bit from 'inferable' to a native flag. "
"Requires backend='unsloth' and quantization='4bit'. (v0.53.0)"
),
)
# Part E — LF / Axolotl parity.
bnb_4bit_use_double_quant: bool = Field(
default=False,
description=(
"Apply BNB 4-bit double-quantization (LF / Axolotl parity). "
"Only meaningful when quantization='4bit'. (v0.53.0)"
),
)
llm_int8: bool = Field(
default=False,
description=(
"Explicit 8-bit LLM.int8 alias for quantization='8bit'. "
"When True, requires quantization='8bit'. (v0.53.0)"
),
)
quantize_ref_model: bool = Field(
default=False,
description=(
"Apply the same Quant Menu config to the reference model "
"(DPO/IPO/SimPO/ORPO/BCO ref model) — extends v0.40.5. (v0.53.0)"
),
)
quantize_reward_model: bool = Field(
default=False,
description=(
"Apply the same Quant Menu config to the reward model "
"(PPO/reward_model task) — extends v0.40.5. (v0.53.0)"
),
)
@field_validator(
"fp8_attention",
"nvfp4",
"unsloth_bnb_4bit",
"bnb_4bit_use_double_quant",
"llm_int8",
"quantize_ref_model",
"quantize_reward_model",
mode="before",
)
@classmethod
def _validate_v053_bool_fields(cls, v):
"""v0.53.0 — explicit bool guard so YAML ``yes`` / ``1`` integers
cannot silently coerce. Matches project bool-before-int policy.
``None`` falls through to Pydantic so the field's ``default=False``
applies (review-fix — avoids silent ``None → False`` coercion that
would mask YAML typos like ``fp8_attention: ~``).
"""
if v is None:
return v
if isinstance(v, bool):
return v
raise TypeError(
f"v0.53.0 flag must be bool, got {type(v).__name__}"
)
@field_validator("kv_cache_type", mode="before")
@classmethod
def _validate_kv_cache_type(cls, v):
"""v0.53.0 Part C — bool / null-byte / oversize / case-insensitive
normalisation via the shared helper.
"""
if v is None:
return None
from soup_cli.utils.kv_cache import validate_kv_cache_type
return validate_kv_cache_type(v)
@field_validator("teacher_model")
@classmethod
@ -2761,6 +2861,160 @@ class SoupConfig(BaseModel):
raise ValueError(str(exc)) from exc
return self
# ---- v0.53.0 Quant Menu II cross-validators ----------------------------
@model_validator(mode="after")
def _validate_fp8_attention_compat(self) -> "SoupConfig":
"""v0.53.0 Part D — ``fp8_attention=True`` requires
``quantization_aware='fp8'`` and a non-mlx backend. Silent-no-op
footgun rejection (mirrors v0.32.0 spike-recovery policy).
"""
tcfg = self.training
if not tcfg.fp8_attention:
return self
from soup_cli.utils.advanced_precision import (
validate_fp8_attention_compat,
)
try:
validate_fp8_attention_compat(
fp8_attention=tcfg.fp8_attention,
quantization_aware=tcfg.quantization_aware,
backend=self.backend,
)
except ValueError as exc:
raise ValueError(str(exc)) from exc
return self
@model_validator(mode="after")
def _validate_nvfp4_compat(self) -> "SoupConfig":
"""v0.53.0 Part D — ``nvfp4=True`` requires non-mlx + text-modality.
Blackwell SM-capability check is runtime-only (live wiring v0.53.1).
"""
tcfg = self.training
if not tcfg.nvfp4:
return self
from soup_cli.utils.advanced_precision import validate_nvfp4_compat
try:
validate_nvfp4_compat(
nvfp4=tcfg.nvfp4, backend=self.backend, modality=self.modality,
)
except ValueError as exc:
raise ValueError(str(exc)) from exc
return self
@model_validator(mode="after")
def _validate_unsloth_bnb_4bit_compat(self) -> "SoupConfig":
"""v0.53.0 Part D — ``unsloth_bnb_4bit=True`` requires
``backend='unsloth'`` and ``quantization='4bit'``.
"""
tcfg = self.training
if not tcfg.unsloth_bnb_4bit:
return self
from soup_cli.utils.advanced_precision import (
validate_unsloth_bnb_4bit_compat,
)
try:
validate_unsloth_bnb_4bit_compat(
unsloth_bnb_4bit=tcfg.unsloth_bnb_4bit,
backend=self.backend,
quantization=tcfg.quantization,
)
except ValueError as exc:
raise ValueError(str(exc)) from exc
return self
@model_validator(mode="after")
def _validate_bnb_4bit_double_quant(self) -> "SoupConfig":
"""v0.53.0 Part E — ``bnb_4bit_use_double_quant=True`` requires
``quantization='4bit'`` (silent-no-op footgun otherwise).
"""
tcfg = self.training
if not tcfg.bnb_4bit_use_double_quant:
return self
if tcfg.quantization != "4bit":
raise ValueError(
"training.bnb_4bit_use_double_quant=true requires "
f"training.quantization='4bit'; got "
f"quantization={tcfg.quantization!r}"
)
return self
@model_validator(mode="after")
def _validate_llm_int8_alias(self) -> "SoupConfig":
"""v0.53.0 Part E — ``llm_int8=True`` requires ``quantization='8bit'``.
Unlike v0.41.0 ``load_in_8bit`` (which rewrites quantization), the
``llm_int8`` flag is a pure assertion: the user explicitly says
"this is an LLM.int8 run" and we enforce the matching quantization
rather than silently rewriting it.
"""
tcfg = self.training
if not tcfg.llm_int8:
return self
if tcfg.quantization != "8bit":
raise ValueError(
"training.llm_int8=true requires training.quantization='8bit'; "
f"got quantization={tcfg.quantization!r}"
)
return self
@model_validator(mode="after")
def _validate_quantize_ref_reward(self) -> "SoupConfig":
"""v0.53.0 Part E — ``quantize_ref_model`` requires a ref-model task
and ``quantize_reward_model`` requires a reward-model task.
Silent-no-op footgun rejection.
Ref-model tasks (review fix): all preference-family trainers PLUS
``grpo`` (KL to ref policy) and ``kto`` (unpaired preference, also
keeps a frozen ref). ``ppo`` also has a ref but uses a separately
named ``policy_ref`` checkpoint; covered by the reward path too.
"""
tcfg = self.training
ref_tasks = {
"dpo", "ipo", "simpo", "orpo", "bco", "kto",
"preference", "grpo", "ppo",
}
reward_tasks = {"ppo", "reward_model"}
if tcfg.quantize_ref_model and self.task not in ref_tasks:
raise ValueError(
"training.quantize_ref_model=true requires a task with a "
"reference model "
f"(one of {sorted(ref_tasks)}); got task={self.task!r}"
)
if tcfg.quantize_reward_model and self.task not in reward_tasks:
raise ValueError(
"training.quantize_reward_model=true requires task in "
f"{sorted(reward_tasks)}; got task={self.task!r}"
)
return self
@model_validator(mode="after")
def _validate_kv_cache_type_supported(self) -> "SoupConfig":
"""v0.53.0 Part C — ``kv_cache_type`` schema gate.
Currently only ``fp8`` is gated (Hopper-only — MLX rejected). The
three remaining types (``q8_0`` / ``bf16`` / ``f16``) pass through
for every backend; v0.53.1 live wiring MAY need to narrow this
further (e.g. MLX serve may not support ``q8_0``). The schema-only
permissive policy is deliberate this release — kept here so the
v0.53.1 contributor sees the gate site immediately.
Hopper SM-capability check (compute_cap >= 9.0) is runtime-only.
"""
kv = self.training.kv_cache_type
if kv is None:
return self
if self.backend == "mlx" and kv == "fp8":
raise ValueError(
"training.kv_cache_type='fp8' is not supported on the mlx "
"backend (Hopper-only). Use kv_cache_type in {q8_0,bf16,f16} "
"or switch backend."
)
return self
@model_validator(mode="after")
def _validate_relora_supported_tasks(self) -> "SoupConfig":
"""v0.40.6 (#67) — ReLoRA callback wired in every transformer-backend

143
soup_cli/utils/advanced_precision.py Normal file
View File

@ -0,0 +1,143 @@
"""v0.53.0 Part D — Train-time advanced precision schema helpers.
Three new TrainingConfig surfaces ship this release (schema-only):
* ``fp8_attention: bool`` — extend the v0.28.0 FP8 menu to apply FP8 to
attention (axolotl-parity flag). Requires ``quantization_aware='fp8'``.
* ``nvfp4: bool`` — Blackwell-only NVFP4 training (unsloth + axolotl). Gated
to non-mlx text-modality training.
* ``unsloth_bnb_4bit: bool`` — promote Unsloth Dynamic 4-bit to a native
TrainingConfig flag (previously inferable only from ``backend='unsloth'``
+ ``quantization='4bit'``). When True, requires ``backend='unsloth'`` and
``quantization='4bit'``.
Live wiring lands in v0.53.1 (mirrors v0.50.0 / v0.52.0 stub-then-live).
"""
from __future__ import annotations
def validate_fp8_attention_compat(
*,
fp8_attention: bool,
quantization_aware: object,
backend: str,
) -> None:
"""Schema-time gate for ``fp8_attention=True``.
Rejects:
- non-bool ``fp8_attention`` (defence-in-depth).
- ``fp8_attention=True`` without ``quantization_aware='fp8'`` (silent
no-op footgun — mirrors v0.32.0 ``loss_spike_recovery`` policy).
- non-string / empty ``backend``.
- ``backend == 'mlx'`` (MLX path has no FP8 attention kernel).
"""
if not isinstance(fp8_attention, bool):
raise TypeError(
f"fp8_attention must be bool, got {type(fp8_attention).__name__}"
)
if not fp8_attention:
return
if isinstance(backend, bool):
raise TypeError(f"backend must not be bool, got {backend!r}")
if not isinstance(backend, str) or not backend:
raise ValueError("backend must be a non-empty string")
# Check quantization_aware prerequisite BEFORE backend gate so a YAML
# missing both gets the more actionable error (matches v0.52.0
# validate_bitnet_compat ordering).
if quantization_aware != "fp8":
raise ValueError(
"fp8_attention=true requires training.quantization_aware='fp8' "
f"(got quantization_aware={quantization_aware!r})"
)
if backend == "mlx":
raise ValueError(
"fp8_attention=true is not supported on backend=mlx"
)
def validate_nvfp4_compat(
*,
nvfp4: bool,
backend: str,
modality: str,
) -> None:
"""Schema-time gate for ``nvfp4=True``.
NVFP4 is Blackwell-only and CUDA-only; the *runtime* SM-capability
check fires at trainer-construction time. This schema gate is the
cheap defence-in-depth layer.
"""
if not isinstance(nvfp4, bool):
raise TypeError(f"nvfp4 must be bool, got {type(nvfp4).__name__}")
if not nvfp4:
return
for name, value in (("backend", backend), ("modality", modality)):
if isinstance(value, bool):
raise TypeError(f"{name} must not be bool, got {value!r}")
if not isinstance(value, str) or not value:
raise ValueError(f"{name} must be a non-empty string")
if backend == "mlx":
raise ValueError(
"nvfp4=true is not supported on backend=mlx "
"(NVFP4 is CUDA-only — requires Blackwell)"
)
if modality != "text":
raise ValueError(
f"nvfp4=true is wired for modality='text' only; "
f"got modality={modality!r}"
)
def validate_unsloth_bnb_4bit_compat(
*,
unsloth_bnb_4bit: bool,
backend: str,
quantization: str,
) -> None:
"""Schema-time gate for ``unsloth_bnb_4bit=True``.
Promotes "Unsloth Dynamic 4-bit" from "inferable from backend+quant"
to a native flag. The flag requires:
- ``backend == 'unsloth'`` (otherwise silently no-op).
- ``quantization == '4bit'`` (the BNB Dynamic 4-bit path; conflicts
with the v0.38.0 Quant Menu formats which raise loudly at runtime).
"""
if not isinstance(unsloth_bnb_4bit, bool):
raise TypeError(
f"unsloth_bnb_4bit must be bool, "
f"got {type(unsloth_bnb_4bit).__name__}"
)
if not unsloth_bnb_4bit:
return
for name, value in (("backend", backend), ("quantization", quantization)):
if isinstance(value, bool):
raise TypeError(f"{name} must not be bool, got {value!r}")
if not isinstance(value, str) or not value:
raise ValueError(f"{name} must be a non-empty string")
if backend != "unsloth":
raise ValueError(
f"unsloth_bnb_4bit=true requires backend='unsloth'; "
f"got backend={backend!r}"
)
if quantization != "4bit":
raise ValueError(
f"unsloth_bnb_4bit=true requires quantization='4bit'; "
f"got quantization={quantization!r}"
)
def apply_fp8_attention() -> None:
"""Live FP8-attention wiring — deferred to v0.53.1."""
raise NotImplementedError(
"fp8_attention live wiring deferred to v0.53.1. Schema accepts the "
"flag but no torchao FP8 attention swap is registered yet."
)
def apply_nvfp4() -> None:
"""Live NVFP4 wiring — deferred to v0.53.1."""
raise NotImplementedError(
"NVFP4 live wiring deferred to v0.53.1. Schema accepts the flag "
"but no Blackwell-FP4 quant prep is registered yet."
)

292
soup_cli/utils/gguf_quant.py Normal file
View File

@ -0,0 +1,292 @@
"""v0.53.0 Parts A+B — UD GGUF + IQ / Apple-ARM quant schema helpers.
Schema-only support for Unsloth Dynamic 2.0 GGUF ladder (``UD-Q8_K_XL`` …
``UD-IQ1_M``), the IQ1/IQ2/IQ3 family, the Apple/ARM-friendly Q4_NL /
Q5.x / Q4.x variants, and the existing TQ1_0 1.58-bit GGUF flavour (from
v0.52.0 Part D, re-exposed here for ``soup export --format gguf-iq``).
Live llama.cpp ``imatrix`` calibration + actual GGUF write are deferred
to v0.53.1 (mirrors v0.50.0 stub-then-live pattern).
"""
from __future__ import annotations
from dataclasses import dataclass
from types import MappingProxyType
from typing import Mapping
# --- Part A: Unsloth Dynamic 2.0 GGUF ladder ---------------------------------
UD_GGUF_FORMATS: frozenset[str] = frozenset({
"UD-Q8_K_XL",
"UD-Q6_K_XL",
"UD-Q5_K_XL",
"UD-Q4_K_XL",
"UD-Q3_K_XL",
"UD-Q2_K_XL",
"UD-IQ4_XS",
"UD-IQ3_M",
"UD-IQ3_XXS",
"UD-IQ2_M",
"UD-IQ2_XS",
"UD-IQ2_XXS",
"UD-IQ1_M",
"UD-IQ1_S",
})
# --- Part B: IQ + Apple/ARM quant flavours -----------------------------------
IQ_GGUF_FORMATS: frozenset[str] = frozenset({
"IQ1_S",
"IQ1_M",
"IQ2_XXS",
"IQ2_XS",
"IQ2_S",
"IQ2_M",
"IQ3_XXS",
"IQ3_XS",
"IQ3_S",
"IQ3_M",
"IQ4_XS",
"IQ4_NL",
})
APPLE_ARM_GGUF_FORMATS: frozenset[str] = frozenset({
"Q4_0_4_4",
"Q4_0_4_8",
"Q4_0_8_8",
"Q4_NL",
"Q5_0",
"Q5_1",
"Q5_K_S",
"Q5_K_M",
"Q4_K_S",
"Q4_K_M",
})
# Union of all v0.53.0 schema-only GGUF flavours (excludes TQ1_0 which is owned
# by v0.52.0 Part D ``utils/bitnet.py``; ``is_advanced_gguf_format`` returns
# True for the BitNet family too via the helper below for export-CLI parity).
ALL_ADVANCED_GGUF_FORMATS: frozenset[str] = (
UD_GGUF_FORMATS | IQ_GGUF_FORMATS | APPLE_ARM_GGUF_FORMATS
)
_MAX_FORMAT_LEN: int = 32
@dataclass(frozen=True)
class GGUFQuantSpec:
"""Frozen metadata for a v0.53.0 advanced GGUF flavour."""
name: str
family: str # "ud" | "iq" | "apple_arm"
bits: float
description: str
live_wired: bool
def _spec(name: str, family: str, bits: float, description: str) -> GGUFQuantSpec:
return GGUFQuantSpec(
name=name, family=family, bits=bits,
description=description, live_wired=False,
)
_GGUF_METADATA: Mapping[str, GGUFQuantSpec] = MappingProxyType({
# Unsloth Dynamic 2.0 ladder
"UD-Q8_K_XL": _spec("UD-Q8_K_XL", "ud", 8.0, "UD Q8_K_XL (Unsloth Dynamic 2.0)"),
"UD-Q6_K_XL": _spec("UD-Q6_K_XL", "ud", 6.0, "UD Q6_K_XL"),
"UD-Q5_K_XL": _spec("UD-Q5_K_XL", "ud", 5.0, "UD Q5_K_XL"),
"UD-Q4_K_XL": _spec("UD-Q4_K_XL", "ud", 4.0, "UD Q4_K_XL"),
"UD-Q3_K_XL": _spec("UD-Q3_K_XL", "ud", 3.0, "UD Q3_K_XL"),
"UD-Q2_K_XL": _spec("UD-Q2_K_XL", "ud", 2.0, "UD Q2_K_XL"),
"UD-IQ4_XS": _spec("UD-IQ4_XS", "ud", 4.0, "UD IQ4_XS"),
"UD-IQ3_M": _spec("UD-IQ3_M", "ud", 3.0, "UD IQ3_M"),
"UD-IQ3_XXS": _spec("UD-IQ3_XXS", "ud", 3.0, "UD IQ3_XXS"),
"UD-IQ2_M": _spec("UD-IQ2_M", "ud", 2.0, "UD IQ2_M"),
"UD-IQ2_XS": _spec("UD-IQ2_XS", "ud", 2.0, "UD IQ2_XS"),
"UD-IQ2_XXS": _spec("UD-IQ2_XXS", "ud", 2.0, "UD IQ2_XXS"),
"UD-IQ1_M": _spec("UD-IQ1_M", "ud", 1.0, "UD IQ1_M (smallest UD)"),
"UD-IQ1_S": _spec("UD-IQ1_S", "ud", 1.0, "UD IQ1_S"),
# IQ family (non-UD)
"IQ1_S": _spec("IQ1_S", "iq", 1.0, "IQ1_S 1-bit"),
"IQ1_M": _spec("IQ1_M", "iq", 1.0, "IQ1_M 1-bit"),
"IQ2_XXS": _spec("IQ2_XXS", "iq", 2.0, "IQ2_XXS 2-bit"),
"IQ2_XS": _spec("IQ2_XS", "iq", 2.0, "IQ2_XS 2-bit"),
"IQ2_S": _spec("IQ2_S", "iq", 2.0, "IQ2_S 2-bit"),
"IQ2_M": _spec("IQ2_M", "iq", 2.0, "IQ2_M 2-bit"),
"IQ3_XXS": _spec("IQ3_XXS", "iq", 3.0, "IQ3_XXS 3-bit"),
"IQ3_XS": _spec("IQ3_XS", "iq", 3.0, "IQ3_XS 3-bit"),
"IQ3_S": _spec("IQ3_S", "iq", 3.0, "IQ3_S 3-bit"),
"IQ3_M": _spec("IQ3_M", "iq", 3.0, "IQ3_M 3-bit"),
"IQ4_XS": _spec("IQ4_XS", "iq", 4.0, "IQ4_XS 4-bit"),
"IQ4_NL": _spec("IQ4_NL", "iq", 4.0, "IQ4_NL 4-bit (non-linear)"),
# Apple/ARM neural-engine-friendly
"Q4_0_4_4": _spec("Q4_0_4_4", "apple_arm", 4.0, "Apple/ARM Q4_0_4_4"),
"Q4_0_4_8": _spec("Q4_0_4_8", "apple_arm", 4.0, "Apple/ARM Q4_0_4_8"),
"Q4_0_8_8": _spec("Q4_0_8_8", "apple_arm", 4.0, "Apple/ARM Q4_0_8_8"),
"Q4_NL": _spec("Q4_NL", "apple_arm", 4.0, "Apple/ARM Q4_NL"),
"Q5_0": _spec("Q5_0", "apple_arm", 5.0, "Apple/ARM Q5_0"),
"Q5_1": _spec("Q5_1", "apple_arm", 5.0, "Apple/ARM Q5_1"),
"Q5_K_S": _spec("Q5_K_S", "apple_arm", 5.0, "Apple/ARM Q5_K_S"),
"Q5_K_M": _spec("Q5_K_M", "apple_arm", 5.0, "Apple/ARM Q5_K_M"),
"Q4_K_S": _spec("Q4_K_S", "apple_arm", 4.0, "Apple/ARM Q4_K_S"),
"Q4_K_M": _spec("Q4_K_M", "apple_arm", 4.0, "Apple/ARM Q4_K_M"),
})
def _basic_validate(value: object, field: str) -> str:
if isinstance(value, bool):
raise TypeError(f"{field} must not be bool, got {value!r}")
if not isinstance(value, str):
raise TypeError(f"{field} must be str, got {type(value).__name__}")
if not value:
raise ValueError(f"{field} must be non-empty")
if "\x00" in value:
raise ValueError(f"{field} must not contain null bytes")
if len(value) > _MAX_FORMAT_LEN:
raise ValueError(f"{field} too long (max {_MAX_FORMAT_LEN} chars)")
return value
# Lowercase index built once at module load — O(1) lookup vs O(N) walk
# (code-review MEDIUM fix). Mirrors v0.32.0 ``pick_mixed_precision`` quirk
# ordering policy where sorting / indexing is precomputed.
_LOWER_INDEX: Mapping[str, str] = MappingProxyType({
name.lower(): name for name in ALL_ADVANCED_GGUF_FORMATS
})
def _resolve_canonical(value: str) -> str | None:
"""Match ``value`` against the union allowlist case-insensitively.
Returns the canonical (original-case) entry from the allowlist or
``None`` if no match.
"""
return _LOWER_INDEX.get(value.lower())
def validate_ud_gguf_format(value: object) -> str:
"""Validate ``value`` is a UD GGUF format string. Returns canonical form."""
_basic_validate(value, "ud_gguf_format")
canonical = _resolve_canonical(value) # type: ignore[arg-type]
if canonical is None or canonical not in UD_GGUF_FORMATS:
supported = ", ".join(sorted(UD_GGUF_FORMATS))
raise ValueError(
f"ud_gguf_format {value!r} not supported. Supported: {supported}"
)
return canonical
def validate_iq_gguf_format(value: object) -> str:
"""Validate ``value`` is an IQ GGUF format string. Returns canonical form."""
_basic_validate(value, "iq_gguf_format")
canonical = _resolve_canonical(value) # type: ignore[arg-type]
if canonical is None or canonical not in IQ_GGUF_FORMATS:
supported = ", ".join(sorted(IQ_GGUF_FORMATS))
raise ValueError(
f"iq_gguf_format {value!r} not supported. Supported: {supported}"
)
return canonical
def validate_apple_arm_gguf_format(value: object) -> str:
"""Validate ``value`` is an Apple/ARM GGUF format string."""
_basic_validate(value, "apple_arm_gguf_format")
canonical = _resolve_canonical(value) # type: ignore[arg-type]
if canonical is None or canonical not in APPLE_ARM_GGUF_FORMATS:
supported = ", ".join(sorted(APPLE_ARM_GGUF_FORMATS))
raise ValueError(
f"apple_arm_gguf_format {value!r} not supported. "
f"Supported: {supported}"
)
return canonical
def is_ud_gguf_format(value: object) -> bool:
"""Return True iff ``value`` is one of the UD GGUF ladder entries."""
if isinstance(value, bool) or not isinstance(value, str):
return False
canonical = _resolve_canonical(value)
return canonical is not None and canonical in UD_GGUF_FORMATS
def is_iq_gguf_format(value: object) -> bool:
"""Return True iff ``value`` is one of the IQ GGUF flavours."""
if isinstance(value, bool) or not isinstance(value, str):
return False
canonical = _resolve_canonical(value)
return canonical is not None and canonical in IQ_GGUF_FORMATS
def is_apple_arm_gguf_format(value: object) -> bool:
"""Return True iff ``value`` is one of the Apple/ARM GGUF flavours."""
if isinstance(value, bool) or not isinstance(value, str):
return False
canonical = _resolve_canonical(value)
return canonical is not None and canonical in APPLE_ARM_GGUF_FORMATS
def is_advanced_gguf_format(value: object) -> bool:
"""Return True iff ``value`` is any v0.53.0 advanced GGUF format."""
return (
is_ud_gguf_format(value)
or is_iq_gguf_format(value)
or is_apple_arm_gguf_format(value)
)
def get_gguf_spec(name: str) -> GGUFQuantSpec:
"""Return the frozen :class:`GGUFQuantSpec` for ``name`` (case-insensitive)."""
if isinstance(name, bool) or not isinstance(name, str):
raise TypeError(f"name must be str, got {type(name).__name__}")
canonical = _resolve_canonical(name)
if canonical is None:
supported_n = len(ALL_ADVANCED_GGUF_FORMATS)
raise ValueError(
f"GGUF format {name!r} not in v0.53.0 catalog "
f"({supported_n} known)"
)
return _GGUF_METADATA[canonical]
def validate_calibration_data_path(path: object) -> str:
"""Validate ``--calibration-data <jsonl>`` argument shape.
Boundary contract — what's enforced HERE vs at CLI dispatch (v0.53.1):
THIS helper enforces:
* non-empty ``str`` (bool / None / other types rejected with ``TypeError``)
* no null bytes
* length <= 4096 chars
CLI dispatch in v0.53.1 MUST additionally apply (mirrors v0.43.0 /
v0.46.0 / v0.47.0 TOCTOU policy):
* ``os.path.realpath`` + ``os.path.commonpath`` cwd containment
* ``os.lstat`` + ``stat.S_ISLNK`` rejection BEFORE any ``open()``
* existence check via ``os.path.isfile``
Do NOT skip the dispatch-time controls — this helper is shape-only.
"""
if isinstance(path, bool):
raise TypeError(f"calibration_data must not be bool, got {path!r}")
if not isinstance(path, str):
raise TypeError(
f"calibration_data must be str, got {type(path).__name__}"
)
if not path:
raise ValueError("calibration_data must be non-empty")
if "\x00" in path:
raise ValueError("calibration_data must not contain null bytes")
if len(path) > 4096:
raise ValueError("calibration_data path too long (max 4096 chars)")
return path
def export_advanced_gguf() -> None:
"""Live UD/IQ/Apple-ARM GGUF export — deferred to v0.53.1.
Mirrors v0.52.0 ``export_bitnet_gguf`` stub-then-live pattern.
"""
raise NotImplementedError(
"UD / IQ / Apple-ARM GGUF export live wiring deferred to v0.53.1. "
"Schema accepts every format in ALL_ADVANCED_GGUF_FORMATS but no "
"llama.cpp imatrix invocation is registered yet."
)

121
soup_cli/utils/kv_cache.py Normal file
View File

@ -0,0 +1,121 @@
"""v0.53.0 Part C — KV cache types schema helpers.
Closed allowlist of ``kv_cache_type`` strings exposed to
``soup serve --kv-cache-type <type>`` and YAML ``training.kv_cache_type`` (the
field is reused by serve / chat runtime in v0.53.1). Mirrors the unsloth
serve recipe.
* ``q8_0`` — 8-bit (default for the unsloth runtime)
* ``bf16`` — bfloat16
* ``f16`` — float16
* ``fp8`` — FP8 on Hopper+ (gated by a separate runtime check)
Live wiring into the vLLM / SGLang / transformers serve loops is deferred
to v0.53.1.
"""
from __future__ import annotations
from dataclasses import dataclass
from types import MappingProxyType
from typing import Mapping
KV_CACHE_TYPES: frozenset[str] = frozenset({"q8_0", "bf16", "f16", "fp8"})
_MAX_KV_CACHE_LEN: int = 16
@dataclass(frozen=True)
class KVCacheSpec:
"""Frozen metadata for a KV-cache type. Immutable by construction."""
name: str
bits: int
requires_hopper: bool
description: str
live_wired: bool
_KV_CACHE_METADATA: Mapping[str, KVCacheSpec] = MappingProxyType({
"q8_0": KVCacheSpec(
name="q8_0", bits=8, requires_hopper=False,
description="8-bit KV cache (default for unsloth runtime)",
live_wired=False,
),
"bf16": KVCacheSpec(
name="bf16", bits=16, requires_hopper=False,
description="bfloat16 KV cache",
live_wired=False,
),
"f16": KVCacheSpec(
name="f16", bits=16, requires_hopper=False,
description="float16 KV cache",
live_wired=False,
),
"fp8": KVCacheSpec(
name="fp8", bits=8, requires_hopper=True,
description="FP8 KV cache (Hopper+ only)",
live_wired=False,
),
})
def validate_kv_cache_type(value: object) -> str:
"""Validate a ``kv_cache_type`` string and return the canonical form.
Mirrors v0.52.0 ``validate_reasoning_effort`` policy: bool-first /
null-byte / oversize / case-insensitive normalisation.
"""
if isinstance(value, bool):
raise TypeError(f"kv_cache_type must not be bool, got {value!r}")
if not isinstance(value, str):
raise TypeError(
f"kv_cache_type must be str, got {type(value).__name__}"
)
if not value:
raise ValueError("kv_cache_type must be non-empty")
if "\x00" in value:
raise ValueError("kv_cache_type must not contain null bytes")
if len(value) > _MAX_KV_CACHE_LEN:
raise ValueError(
f"kv_cache_type too long (max {_MAX_KV_CACHE_LEN} chars)"
)
canonical = value.lower()
if canonical not in KV_CACHE_TYPES:
supported = ", ".join(sorted(KV_CACHE_TYPES))
raise ValueError(
f"kv_cache_type {value!r} not supported. Supported: {supported}"
)
return canonical
def get_kv_cache_spec(name: str) -> KVCacheSpec:
"""Return the frozen :class:`KVCacheSpec` for ``name`` (canonical)."""
canonical = validate_kv_cache_type(name)
return _KV_CACHE_METADATA[canonical]
def requires_hopper(name: object) -> bool:
"""Return True iff ``name`` needs a Hopper+ GPU.
Single source of truth: reads ``requires_hopper`` from the
``_KV_CACHE_METADATA`` spec. Adding a Hopper-only type means flipping
the spec field only — no separate update here (code-review MEDIUM fix).
"""
if isinstance(name, bool) or not isinstance(name, str):
return False
canonical = name.lower()
spec = _KV_CACHE_METADATA.get(canonical)
return spec is not None and spec.requires_hopper
def apply_kv_cache_type() -> None:
"""Live KV-cache-type wiring — deferred to v0.53.1.
Mirrors v0.50.0 ``apply_vllm_sleep_mode`` and v0.52.0
``apply_moe_expert_quant`` stub-then-live pattern.
"""
raise NotImplementedError(
"kv_cache_type live wiring deferred to v0.53.1. Schema accepts "
"q8_0 / bf16 / f16 / fp8 but no serve backend is routing the flag yet."
)

208
soup_cli/utils/save_formats.py Normal file
View File

@ -0,0 +1,208 @@
"""v0.53.0 Part F — Advanced save / merge formats schema helpers.
Two new save-format surfaces ship this release (schema-only):
* ``soup merge --save-format <fmt>`` where fmt ∈ {fp16, 4bit, 4bit_forced}.
``4bit`` writes a single BNB-4bit-quantized merged checkpoint without
the dequant → merge → requant cycle (unsloth ``merged_4bit`` recipe).
``4bit_forced`` is the unsloth ``4bit_forced`` shortcut.
* ``soup export --format torchao --quant-config <yaml>`` — invokes
``torchao.quantize_`` then ``save_pretrained`` for the
``Int4WeightOnly`` / ``Int8DynActInt4`` / ``Float8DynActFloat8`` /
``NVFP4`` PTQ schemes (unsloth + axolotl parity).
Live wiring deferred to v0.53.1.
"""
from __future__ import annotations
from dataclasses import dataclass
from types import MappingProxyType
from typing import Mapping
# --- Merge save formats ------------------------------------------------------
MERGE_SAVE_FORMATS: frozenset[str] = frozenset({
"fp16", "4bit", "4bit_forced",
})
# --- TorchAO PTQ schemes (a closed allowlist, mirrors the v0.38.0 Quant Menu
# string convention so YAML round-trips cleanly).
TORCHAO_PTQ_SCHEMES: frozenset[str] = frozenset({
"Int4WeightOnly",
"Int8DynActInt4",
"Float8DynActFloat8",
"NVFP4",
})
_MAX_SAVE_FORMAT_LEN: int = 32
_MAX_TORCHAO_SCHEME_LEN: int = 48
@dataclass(frozen=True)
class MergeSaveSpec:
"""Frozen metadata for a merge-save format."""
name: str
bits: int
description: str
live_wired: bool
_MERGE_METADATA: Mapping[str, MergeSaveSpec] = MappingProxyType({
"fp16": MergeSaveSpec(
name="fp16", bits=16,
description="Standard FP16 merged checkpoint (default — pre-v0.53.0)",
live_wired=True,
),
"4bit": MergeSaveSpec(
name="4bit", bits=4,
description="Single BNB-4bit-quantized merged checkpoint",
live_wired=False,
),
"4bit_forced": MergeSaveSpec(
name="4bit_forced", bits=4,
description="Forced BNB-4bit merge (unsloth 4bit_forced)",
live_wired=False,
),
})
@dataclass(frozen=True)
class TorchAOPTQSpec:
"""Frozen metadata for a TorchAO PTQ scheme."""
name: str
bits: int
description: str
live_wired: bool
_TORCHAO_METADATA: Mapping[str, TorchAOPTQSpec] = MappingProxyType({
"Int4WeightOnly": TorchAOPTQSpec(
name="Int4WeightOnly", bits=4,
description="TorchAO Int4WeightOnly (weight-only int4)",
live_wired=False,
),
"Int8DynActInt4": TorchAOPTQSpec(
name="Int8DynActInt4", bits=4,
description="TorchAO Int8DynActInt4 (dynamic int8 act + int4 weight)",
live_wired=False,
),
"Float8DynActFloat8": TorchAOPTQSpec(
name="Float8DynActFloat8", bits=8,
description="TorchAO FP8 dynamic activations + FP8 weight",
live_wired=False,
),
"NVFP4": TorchAOPTQSpec(
name="NVFP4", bits=4,
description="TorchAO NVFP4 (Blackwell FP4 PTQ)",
live_wired=False,
),
})
def _validate_string_field(value: object, field: str, max_len: int) -> str:
if isinstance(value, bool):
raise TypeError(f"{field} must not be bool, got {value!r}")
if not isinstance(value, str):
raise TypeError(f"{field} must be str, got {type(value).__name__}")
if not value:
raise ValueError(f"{field} must be non-empty")
if "\x00" in value:
raise ValueError(f"{field} must not contain null bytes")
if len(value) > max_len:
raise ValueError(f"{field} too long (max {max_len} chars)")
return value
def validate_merge_save_format(value: object) -> str:
"""Validate ``--save-format`` arg. Case-insensitive normalisation.
Mirrors v0.41.0 ``optimizer`` policy. Returns the lowercase canonical
form so ``_MERGE_METADATA`` (all-lowercase keys) lookups are O(1).
"""
validated = _validate_string_field(value, "save_format", _MAX_SAVE_FORMAT_LEN)
canonical = validated.lower()
if canonical not in MERGE_SAVE_FORMATS:
supported = ", ".join(sorted(MERGE_SAVE_FORMATS))
raise ValueError(
f"save_format {value!r} not supported. Supported: {supported}"
)
return canonical
def validate_torchao_scheme(value: object) -> str:
"""Validate a TorchAO PTQ scheme name.
INTENTIONALLY CASE-SENSITIVE — these are PyTorch class names
(``Int4WeightOnly`` / ``NVFP4`` etc.) that ``torchao.quantize_`` looks up
by exact name. Diverges from ``validate_merge_save_format`` (which
lowercase-normalises) and ``validate_kv_cache_type`` (also lowercase)
on purpose: TorchAO uses CapWords, the others are operator-facing flags.
"""
validated = _validate_string_field(
value, "torchao_scheme", _MAX_TORCHAO_SCHEME_LEN,
)
if validated not in TORCHAO_PTQ_SCHEMES:
supported = ", ".join(sorted(TORCHAO_PTQ_SCHEMES))
raise ValueError(
f"torchao_scheme {value!r} not supported. Supported: {supported}"
)
return validated
def get_merge_save_spec(name: str) -> MergeSaveSpec:
"""Return the frozen :class:`MergeSaveSpec` for ``name`` (case-insensitive)."""
canonical = validate_merge_save_format(name)
return _MERGE_METADATA[canonical]
def get_torchao_spec(name: str) -> TorchAOPTQSpec:
"""Return the frozen :class:`TorchAOPTQSpec` for ``name`` (case-sensitive)."""
canonical = validate_torchao_scheme(name)
return _TORCHAO_METADATA[canonical]
def validate_quant_config_path(path: object) -> str:
"""Validate ``--quant-config <yaml>`` argument shape.
Boundary contract — what's enforced HERE vs at CLI dispatch (v0.53.1):
THIS helper enforces non-empty ``str``, no null bytes, length <= 4096.
CLI dispatch in v0.53.1 MUST additionally apply (mirrors v0.43.0 /
v0.46.0 / v0.47.0 TOCTOU policy):
* ``os.path.realpath`` + ``os.path.commonpath`` cwd containment
* ``os.lstat`` + ``stat.S_ISLNK`` rejection before any ``open()``
* existence + extension check (``.yaml`` / ``.yml``)
* ``yaml.safe_load`` only.
"""
if isinstance(path, bool):
raise TypeError(f"quant_config must not be bool, got {path!r}")
if not isinstance(path, str):
raise TypeError(
f"quant_config must be str, got {type(path).__name__}"
)
if not path:
raise ValueError("quant_config must be non-empty")
if "\x00" in path:
raise ValueError("quant_config must not contain null bytes")
if len(path) > 4096:
raise ValueError("quant_config path too long (max 4096 chars)")
return path
def merge_4bit() -> None:
"""Live 4bit-merge wiring — deferred to v0.53.1."""
raise NotImplementedError(
"soup merge --save-format 4bit live wiring deferred to v0.53.1. "
"Schema accepts the flag but no merged-4bit writer is registered yet."
)
def export_torchao() -> None:
"""Live TorchAO PTQ export — deferred to v0.53.1."""
raise NotImplementedError(
"soup export --format torchao live wiring deferred to v0.53.1. "
"Schema accepts the format but no torchao.quantize_ call is wired."
)

1018
tests/test_v0530.py Normal file
View File

File diff suppressed because it is too large Load Diff