mirror of https://github.com/razor-ai/soup.git
feat(v0.52.0): Modality II — TTS + Distillation + BitNet + EBFT-GDPO + MoE quant + reasoning_effort
7 schema-only Parts; live trainer / loss / export wiring deferred to v0.52.1
(mirrors v0.27.0 MII / v0.50.0 GRPO Plus / v0.51.0 hubs stub-then-live pattern).
- Part A: task='tts' + modality='audio_out' + 5 families (Orpheus/Sesame-CSM/
Llasa/Spark/Oute) + per-family emotion allowlist (Orpheus 8 / Oute 6) +
5 recipes (orpheus-tts-sft, sesame-csm-tts, llasa-tts, spark-tts, oute-tts).
- Part B: classifier / reranker / cross_encoder tasks + num_labels (bool-
before-int guard) + classifier_kind + label_names (dedup + null-byte + cap).
- Part C: task='distill' + teacher_model + distill_divergence (kl alias
canonicalises to forward_kl; Literal excludes alias) + distill_temperature
(math.isfinite + [0.05, 100.0] bounds).
- Part D: quantization='bitnet_1.58' (gated to non-MLX + text + task in
{sft, pretrain, dpo}) + Falcon-E BitNet recipe + soup export
--format bitnet/tq1_0 CLI stubs (yellow deferred-advisory panel, exit 0).
- Part E: EBFT (structured/strided + bounded ebft_temperature; SFT-only)
+ GDPO (standard/length_normalized/margin; DPO-family-only).
- Part F: moe_expert_quant (nf4/int8_rowwise) + train_router_only — both
require moe_lora=true (silent-no-op footgun rejection).
- Part G: reasoning_effort (low/medium/high) + train_on_eot — both gated to
the SFT-family task set (sft/pretrain/distill/classifier/reranker/
cross_encoder); rejected on DPO/GRPO/PPO/etc. with named offenders.
Review fixes (5 agents: python, security, code, tdd, verification-loop-manual):
- num_labels bool-before-int field_validator (security HIGH)
- reasoning_effort + train_on_eot SoupConfig task-gate (code HIGH)
- _validate_classifier_compat lazy-import early-return (code HIGH)
- Oute emotion allowlist via data-driven _FAMILY_EMOTIONS (python+code MED)
- _MAX_LEN -> _MAX_REASONING_EFFORT_LEN (python MED)
- validate_reasoning_effort wired via field_validator (security MED)
- distill_divergence Literal excludes "kl" alias (code MED)
- DIVERGENCES derived from _DIVERGENCE_ALIASES drift guard (python LOW)
- is_bitnet_model comment fixed to match impl (python LOW)
- sister-fn bool guards on every compat helper (python+security MED)
- TDD coverage gaps closed: EBFT variant oversize, GDPO full rejection matrix,
ebft_temperature explicit-exc table, TTS compat input guards, recipe
model-id null/whitespace, full reasoning_effort task-matrix.
Drift fixes:
- tests/test_onnx_tensorrt_export.py + tests/test_awq_gptq_export.py
SUPPORTED_FORMATS count bumped 5 -> 7 (bitnet + tq1_0 stubs).
- tests/test_recipes.py catalog_size assertion 106 -> 112 (5 TTS + Falcon-E).
Test count: 7184 -> 7456 (+272). 230 new tests in tests/test_v0520.py;
remainder from drift-fix parametrize expansions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alpamys
parent
352f8d4bfd
commit
df7f49feda
|
|
@ -111,7 +111,7 @@ soup_cli/
|
|||
templates/ - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
|
||||
ui/ - Web UI (FastAPI + HTML/JS SPA)
|
||||
|
||||
tests/ - Test suite (178 files, 7184 tests)
|
||||
tests/ - Test suite (179 files, 7456 tests)
|
||||
examples/ - Real-world config examples and datasets
|
||||
```
|
||||
|
||||
|
|
@ -262,6 +262,7 @@ pytest tests/ --cov=soup_cli --cov-report=html
|
|||
| test_v0500_part_c.py | v0.50.0 Part C — Multi-turn agent rollout backend allowlist (art / ruler / nemo_gym / openenv); frozen `RolloutBackendSpec` + `MappingProxyType` immutability; `validate_rollout_backend` (bool rejected); `required_rollout_package` per-entry mapping; `launch_rollout` deferred stub; SoupConfig task-gate + mlx rejection (v0.50.0 Part C) |
|
||||
| test_v0500_part_d.py | v0.50.0 Part D — 7 stability/efficiency knobs (`ref_model_ema_alpha` / `replay_buffer_size` / `async_grpo_prefetch` / `tis_threshold` / `mask_truncated_completions` / `defer_rerolling` / `skip_zero_advantage` / `off_policy_mask_threshold`); explicit bool-rejection field_validator across all numeric fields (tdd-guide HIGH fix); `mask_truncated_completions` requires `tis_threshold` cross-validator; SoupConfig task-gate naming every offending field; `grpo_fp16` task-gate (code-review HIGH fix) (v0.50.0 Part D) |
|
||||
| test_v0500_part_e.py | v0.50.0 Part E — `task='prm'` (Process Reward Model) + `vision_grpo` flag; `validate_prm_compat` (data.format / modality / mlx gates); `validate_vision_grpo_compat` (task ∈ {grpo, ppo} / modality='vision' / non-mlx); `build_prm_trainer` deferred stub; SoupConfig integration with all rejection paths exercised (v0.50.0 Part E) |
|
||||
| test_v0520.py | v0.52.0 Modality II — TTS / classifier / distill / BitNet / EBFT-GDPO / MoE quant / reasoning_effort: TTS family allowlist + per-family emotion allowlists (Orpheus + Oute) + validate_tts_compat; classifier / reranker / cross_encoder tasks + num_labels (with field_validator bool guard) + label_names dedup + classifier-only field gates; distill divergence (kl alias canonicalised, Literal excludes alias) + teacher_model + distill_temperature bounds; BitNet 1.58 quant + bitnet/tq1_0 export-format stubs + Falcon-E recipe + is_bitnet_model org-prefix detect; EBFT (structured/strided) + GDPO (standard/length_normalized/margin) variant allowlists + task gates; MoE expert quant (nf4/int8_rowwise) + train_router_only requiring moe_lora=true; reasoning_effort + train_on_eot with SFT-family task gate; 6 new recipes (5 TTS + Falcon-E BitNet); review-fix coverage (num_labels bool guard, Oute emotion allowlist, lazy-import in classifier validator, task gates, oversize / NaN / Inf matrices). Test count: 272 (v0.52.0) |
|
||||
| test_v0510.py | v0.51.0 Model Catalog Expansion + Alternative Model Hubs: Part E hubs.py (`SUPPORTED_HUBS` + `validate_hub_name` + `validate_hub_endpoint` SSRF parity / CRLF rejection / IPv6 mapped private rejected / IPv6 loopback ok / control chars; `resolve_endpoint` env-var override; `default_endpoint` + `endpoint_env_var` + `required_hub_package` + `is_hf` with bool guards; MappingProxyType immutability); TrainingConfig `hub` field (default + Literal accept + None reject + case-insensitive normalisation + YAML round-trip) + SoupConfig `_validate_hub_supported` (mlx + non-hf rejected; mlx + hf accepted; modelers + transformers accepted); Part D MULTIPACK_ARCHITECTURES extension (20 new arches parametrize + legacy preserved + exact count=38 + frozenset immutability); Parts A/B/C 26 new recipes (parametrize over every name × {get_recipe / RecipeMeta / SoupConfig load / yaml.safe_load / model id no null/whitespace/empty parts / max_length bounds / GRPO required fields}); baichuan-sft uses `hub: modelscope`; total recipe count >= 105 (v0.51.0) |
|
||||
|
||||
(Note: the test-file table above covers v0.25.0–v0.35.0 + v0.47.0 + v0.48.0 + v0.49.0 + v0.50.0 only; full per-release table lives in `.claude/CLAUDE.md`.)
|
||||
|
|
|
|||
110
README.md
110
README.md
|
|
@ -43,13 +43,16 @@ soup train
|
|||
|
||||
Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
|
||||
|
||||
**v0.51.0 — Model Catalog Expansion**: 26 new ready-made recipes covering 25 model families, plus alternative-hub support (ModelScope + Modelers / Openmind) for users in regions where HF Hub is unreachable. Closes the day-zero coverage gap with Unsloth.
|
||||
**v0.52.0 — Modality II (TTS + Distillation + BitNet + EBFT/GDPO + MoE quant + reasoning_effort)**: 5 TTS model families, classifier / reranker / cross-encoder training, knowledge distillation, BitNet 1.58-bit, Energy-Based FT, Generalized DPO, per-expert MoE quantization, and gpt-oss reasoning-effort schema — schema-only release; live trainer / loss / export wiring lands in v0.52.1.
|
||||
|
||||
- **26 new recipes** across reasoning + agent (GPT-OSS 20B/120B, GLM 4.6 / 5, Kimi K2 / K2-Thinking GRPO, MiniMax-M2, QwQ-32B GRPO, QVQ-72B), small / specialist (Granite 4, Liquid LFM2, Cogito v2, Mistral Small 3 / Medium 3.5, Magistral / Devstral / Ministral, MedGemma, EmbeddingGemma), and vision / multimodal (LLaVA-Next, InternVL 3.5, Voxtral, Baichuan 2, Qwen-Image, DeepSeek-OCR, Paddle-OCR-VL). Browse them via `soup recipes list` / `soup recipes search <keyword>`. Catalog grows 80 → 106.
|
||||
- **Alternative model hubs.** New `training.hub: hf | modelscope | modelers` with full SSRF-hardened endpoint validators (parity with v0.29.0 `HF_ENDPOINT` policy — scheme allowlist, loopback-only HTTP, RFC1918 / link-local / cloud-metadata IP rejection, control-character rejection). `MODELSCOPE_ENDPOINT` / `MODELERS_ENDPOINT` env vars override the default hub URL the same way `HF_ENDPOINT` already does. Schema-only this release; live downloader + uploader wiring lands in v0.51.1.
|
||||
- **20 new architectures in the multipack allowlist.** `MULTIPACK_ARCHITECTURES` grows 18 → 38 to enable FFD bin-packing on Granite, GLM, Kimi, MiniMax, QwQ, QVQ, GPT-OSS, Magistral, Devstral, Ministral, MedGemma, LFM2, Cogito, Hunyuan, Ernie, Yi, Baichuan, ChatGLM.
|
||||
- **MLX backend cross-validator.** `backend: mlx` + `hub: modelscope` is now rejected at config-load with a distinct error message (`mlx-lm` only downloads from HF Hub) — prevents a silent runtime confusion.
|
||||
- **+449 net new tests** (6729 → 7178). 4 sequential review agents (python-review, security-review, code-review, tdd-guide) each fed back HIGH/MEDIUM/LOW findings; every finding was fixed before commit (case-insensitive `hub` field-validator, MLX cross-validator, control-char rejection in endpoint validator, `is_hf` bool guard, exact-count multipack arch invariant, empty-component model-id check).
|
||||
- **TTS fine-tuning.** New `task='tts'` + `modality='audio_out'` with a closed allowlist of five model families (Orpheus, Sesame-CSM, Llasa, Spark, Oute) and per-family emotion-tag allowlists for Orpheus + Oute. Five new recipes: `orpheus-tts-sft`, `sesame-csm-tts`, `llasa-tts`, `spark-tts`, `oute-tts`.
|
||||
- **Classifier / reranker / cross-encoder tasks.** New `task` values `classifier`, `reranker`, `cross_encoder` with `num_labels`, `classifier_kind` (single_label / multi_label), `label_names` (1024-cap + dedup + null-byte rejection). Cross-validator requires `num_labels` and matches `len(label_names) == num_labels`.
|
||||
- **Knowledge distillation.** New `task='distill'` + `teacher_model` (HF id or local path) + `distill_divergence` (`kl` / `forward_kl` / `reverse_kl` / `js` — `kl` canonicalises to `forward_kl`) + `distill_temperature` (math.isfinite + [0.05, 100.0] bounds).
|
||||
- **BitNet 1.58-bit + GGUF export schema.** `quantization='bitnet_1.58'` accepted for task ∈ {sft, pretrain, dpo} on transformers/unsloth backends; new BitNet / TQ1_0 export-format allowlist; `falcon-e-bitnet-sft` recipe shipped.
|
||||
- **EBFT + GDPO.** Energy-Based FT variants (`structured` / `strided`) gated to `task='sft'`; Generalized DPO variants (`standard` / `length_normalized` / `margin`) gated to DPO-family tasks.
|
||||
- **MoE expert quant + router-only training.** `moe_expert_quant: nf4 | int8_rowwise` and `train_router_only: true` — both require `moe_lora=true` (silent-no-op rejection at config-load).
|
||||
- **gpt-oss `reasoning_effort: low | medium | high`** + `train_on_eot: bool`. Both gated to the SFT-family task set (sft / pretrain / distill / classifier / reranker / cross_encoder) — non-SFT tasks reject loudly.
|
||||
- **+272 net new tests** (7184 → 7456). 4 review agents (python-reviewer, security-reviewer, code-reviewer, tdd-guide) ran; every finding fixed: `num_labels` bool-before-int guard, `reasoning_effort` / `train_on_eot` task-gate, Oute emotion allowlist, `_validate_classifier_compat` lazy-import guard, `_MAX_LEN` → `_MAX_REASONING_EFFORT_LEN`, `DIVERGENCES` derived from alias map, sister-function bool guards on every compat helper, expanded TDD coverage (oversize on EBFT variant, full GDPO rejection matrix, explicit-exc on temperature bounds, TTS compat input guards, recipe model-id null/whitespace check).
|
||||
|
||||
## Why Soup?
|
||||
|
||||
|
|
@ -137,6 +140,10 @@ soup export --model ./output --format awq --bits 4 --group-size 128
|
|||
|
||||
# Export to GPTQ quantized model (pip install 'soup-cli[gptq]')
|
||||
soup export --model ./output --format gptq --bits 4 --group-size 128
|
||||
|
||||
# BitNet 1.58-bit + TQ1_0 GGUF (schema-locked in v0.52.0; live conversion in v0.52.1)
|
||||
soup export --model ./output --format bitnet
|
||||
soup export --model ./output --format tq1_0
|
||||
```
|
||||
|
||||
## Config Example
|
||||
|
|
@ -3514,6 +3521,97 @@ soup data mix --apply mix_recipe.yaml
|
|||
|
||||
Live wiring of the proxy training loop into a short `soup train` run is the v0.48.1 deliverable; v0.48.0 ships a synthetic offline proxy (quadratic penalty around the uniform simplex) so the budget tracker, optimiser surface, and recipe writer can be exercised without GPUs. `scikit-optimize` is opt-in via `OptimizerProtocol`; the default fallback is a deterministic Dirichlet sampler.
|
||||
|
||||
## TTS Fine-Tuning (BETA, v0.52.0)
|
||||
|
||||
Schema-only this release; live trainer wiring lands in v0.52.1.
|
||||
|
||||
Five upstream model families are recognised: `orpheus`, `sesame_csm`, `llasa`, `spark`, `oute`. Pair `task: tts` with `modality: audio_out` and set `training.tts_family`. Orpheus + Oute support emotion conditioning via `training.tts_emotion` from a per-family allowlist (Orpheus: neutral / happy / sad / angry / excited / calm / whisper / laugh; Oute: neutral / happy / sad / angry / calm / excited).
|
||||
|
||||
```yaml
|
||||
base: canopylabs/orpheus-3b-0.1-ft
|
||||
task: tts
|
||||
modality: audio_out
|
||||
data:
|
||||
train: ./data/tts_train.jsonl
|
||||
format: audio
|
||||
audio_dir: ./data/audio
|
||||
training:
|
||||
tts_family: orpheus
|
||||
tts_emotion: neutral
|
||||
```
|
||||
|
||||
Five ready-made recipes ship in v0.52.0: `orpheus-tts-sft`, `sesame-csm-tts`, `llasa-tts`, `spark-tts`, `oute-tts` — copy with `soup recipes use <name>`. Cross-validators reject the `mlx` backend, `modality != audio_out`, and emotion tags outside the per-family allowlist.
|
||||
|
||||
## Classifier / Reranker / Cross-Encoder Training (BETA, v0.52.0)
|
||||
|
||||
Three new task types build on the existing embedding trainer: `task: classifier` (single-label or multi-label sequence classification), `task: reranker` (pointwise retrieval scoring), `task: cross_encoder` (paired-input scoring). Schema-only; live trainer wrapper in v0.52.1.
|
||||
|
||||
```yaml
|
||||
base: BAAI/bge-base-en-v1.5
|
||||
task: classifier
|
||||
data:
|
||||
train: ./data/classification.jsonl
|
||||
training:
|
||||
num_labels: 3
|
||||
classifier_kind: single_label
|
||||
label_names: [negative, neutral, positive]
|
||||
```
|
||||
|
||||
`num_labels` is bounded `[1, 1024]` with explicit bool-before-int rejection; `label_names` (optional) must be unique, ≤128 chars each, and match `num_labels` in length when set.
|
||||
|
||||
## Knowledge Distillation (BETA, v0.52.0)
|
||||
|
||||
New `task: distill` with `training.teacher_model` (HF id or local path), `training.distill_divergence` (`kl` / `forward_kl` / `reverse_kl` / `js` — `kl` canonicalises to `forward_kl`), and `training.distill_temperature` (bounded `[0.05, 100.0]`, finite-only). Schema-only; live loop in v0.52.1.
|
||||
|
||||
```yaml
|
||||
base: meta-llama/Llama-3.2-1B
|
||||
task: distill
|
||||
data:
|
||||
train: ./data/distill.jsonl
|
||||
training:
|
||||
teacher_model: meta-llama/Llama-3.1-8B
|
||||
distill_divergence: forward_kl
|
||||
distill_temperature: 2.0
|
||||
```
|
||||
|
||||
The cross-validator rejects `task='distill'` without `teacher_model`, and rejects `teacher_model` / `distill_*` fields when `task` is anything other than `distill`.
|
||||
|
||||
## BitNet 1.58-Bit Fine-Tuning (BETA, v0.52.0)
|
||||
|
||||
New `training.quantization: bitnet_1.58` for ternary-weight training (axolotl + onebitllms wrapping). Schema-only on the trainer side; the new export targets are wired as CLI stubs:
|
||||
|
||||
```bash
|
||||
# Schema-locked; live export lands in v0.52.1.
|
||||
soup export --model ./output --format bitnet
|
||||
soup export --model ./output --format tq1_0
|
||||
```
|
||||
|
||||
A ready-made `falcon-e-bitnet-sft` recipe is shipped:
|
||||
|
||||
```bash
|
||||
soup recipes use falcon-e-bitnet-sft
|
||||
soup train --config soup.yaml
|
||||
```
|
||||
|
||||
Restricted to `task ∈ {sft, pretrain, dpo}` on `backend ∈ {transformers, unsloth}` with text modality; the cross-validator rejects MLX and vision/audio configurations loudly at config load.
|
||||
|
||||
## EBFT + GDPO (BETA, v0.52.0)
|
||||
|
||||
Energy-Based Fine-Tuning (axolotl) lands as `training.ebft_variant ∈ {structured, strided}` + `training.ebft_temperature` (bounded `[1e-4, 100.0]`). Gated to `task: sft`. Generalized DPO lands as `training.gdpo_variant ∈ {standard, length_normalized, margin}` — gated to `task ∈ {dpo, preference}`. Live loss kernels in v0.52.1.
|
||||
|
||||
## MoE Expert Quantization + Router-Only Training (v0.52.0)
|
||||
|
||||
For fused-MoE models trained with `moe_lora: true`, two new toggles ship:
|
||||
|
||||
- `training.moe_expert_quant: nf4 | int8_rowwise` — per-expert weight quantization (axolotl).
|
||||
- `training.train_router_only: true` — freeze every expert and train only the gating router (unsloth pattern).
|
||||
|
||||
Both reject silently-no-op combinations: setting either flag without `moe_lora=true` fails at config load with an actionable message.
|
||||
|
||||
## gpt-oss `reasoning_effort` + `train_on_eot` (v0.52.0)
|
||||
|
||||
`training.reasoning_effort: low | medium | high` injects a system-prefix token at training time for gpt-oss models; `training.train_on_eot: true` includes explicit EOT/EOS control tokens in the SFT loss (axolotl `train_on_eot`). Both are gated to the SFT-family task set (`sft` / `pretrain` / `distill` / `classifier` / `reranker` / `cross_encoder`) — setting them on DPO / GRPO / PPO / etc. fails at config load. Live formatter wiring in v0.52.1.
|
||||
|
||||
## Changelog
|
||||
|
||||
See [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases) for version history.
|
||||
|
|
|
|||
|
|
@ -9,7 +9,8 @@ We provide security updates for the following versions:
|
|||
- **Versions older than 3 minor versions:** No support
|
||||
|
||||
Example:
|
||||
- v0.51.0 -- Full support (latest)
|
||||
- v0.52.0 -- Full support (latest)
|
||||
- v0.51.0 -- Full support
|
||||
- v0.50.0 -- Full support
|
||||
- v0.49.0 -- Full support
|
||||
- v0.48.0 -- Full support
|
||||
|
|
@ -146,6 +147,7 @@ No known critical vulnerabilities in current releases.
|
|||
- **v0.32.0 — Training Stability & Auto-Tuning**: `--find-lr-output` containment via shared `utils/paths.is_under_cwd` (prevents writes outside cwd); `save_lr_finder_report` rejects NaN / Infinity floats in `lrs` / `losses` and serialises with `allow_nan=False` (keeps the report parser-safe); `compute_lr_schedule` rejects non-positive `start_lr`, inverted ranges, and `num_steps` outside `[2, 10_000]`; `pick_mixed_precision` rejects empty / null-byte / >200-char model names and resolves multi-version quirks (`qwen2.5` vs `qwen2`, `phi-3.5` vs `phi-3`) by longest-substring-first iteration so an added family can never accidentally make a more-specific entry dead code; `compute_warmup_steps` clamps to `[10, 1000]` with a `ratio==0.0` short-circuit matching HF Trainer's "no warmup" convention; `SpikeRecoveryStrategy` is `@dataclass(frozen=True)` (post-construction mutation cannot bypass validation), `max_attempts ∈ [1, 10]`, `lr_decay ∈ (0, 1)`, `min_lr > 0`; cross-validator `_validate_spike_recovery_requires_watchdog` rejects `loss_spike_recovery=true, loss_watchdog=false` at config-load (fails fast instead of never triggering); `convergence_window ∈ [5, 10_000]`, `convergence_rel_tol ∈ (0, 1]`, `recommend_action` reuses `detect_plateau` so plateau heuristic stays single-source-of-truth; `GradAccumMonitor.recommend()` caps doubled `accum` at `MAX_ACCUM=1024` so a runaway advisory loop cannot blow up DataLoader prefetch; `generate_config` validates BOTH the YAML output path AND the embedded `decisions["output"]` field via `is_under_cwd` (closes the gap where a crafted `decisions["output"]="../../etc"` would have silently propagated into the rendered YAML)
|
||||
- **v0.34.0 — Observability & Dev UX**: `.crash` bundle generator (`utils/crash.py`) recursively redacts `hf_*` / `sk-*` / `Bearer …` token-shaped strings in any captured `config` and metric tail before serialisation, so a `.crash` file shared on a public GitHub issue cannot leak credentials; `output_dir` is reduced to `os.path.basename` so `$HOME` doesn't leak; `write_crash_bundle` uses `os.path.realpath + commonpath` for cwd containment (Windows-safe; raises `ValueError` not `PermissionError` so callers cannot silently swallow with `except OSError`); filename appends `secrets.token_hex(4)` so two crashes in the same UTC second don't collide; bundle truncated to `MAX_BUNDLE_BYTES=1_000_000`. `train.py` crash-write surfaces failures to the user (no silent missing-bundle). `profiling.py` `resolve_trace_path` rejects empty / `.` / `..` / `/` / `\\` / null-byte `run_id` (closes the `output_dir/profiles/../trace.json` escape) and uses `os.path.realpath + is_under_cwd`; profiles dir is created only on successful torch import (no stale empty dirs on torch-less CI). `tracker.get_run` LIKE-prefix match escapes `%` / `_` / `\\` and uses `ESCAPE '\\'` so a crafted `run_id` cannot widen the match (mirrors v0.26.0 registry policy). Lazy schema migration (`_ensure_schema`) tolerates the "duplicate column" race when two CLI processes start simultaneously on a fresh DB (fork-based multi-GPU training, TUI auto-refresh). `runs.py show/replay/clean` switched user `run_id` rendering to `markup_escape` and switched `clean` containment from broken `Path.resolve() + relative_to()` to project-standard `os.path.realpath + is_under_cwd`. `tui_app.py` lazy-imports `ExperimentTracker` and `markup_escape`s every DB-sourced string before passing into Textual widgets so a crafted base_model / experiment_name cannot inject `[bold red]…[/]` markup. `run_cost.estimate_run_cost_usd` rejects `bool` in `num_gpus` (bool is a subclass of int — same defence as v0.30.0 `Candidate.__post_init__`); duration clamped to `[0, 1 year]`; unknown GPU returns `None` so callers render `—` instead of fabricating `$0.00`. `log_level.parse_log_level` rejects non-string + null-byte input.
|
||||
- **v0.33.0 — Live Wire**: RLVR `code_exec_reward` adds OS-level isolation (Linux best-effort `os.unshare(CLONE_NEWUSER|CLONE_NEWNET|CLONE_NEWPID)`, macOS `sandbox-exec` with default-deny `MACOS_SANDBOX_PROFILE` narrowed to a 3-name `mach-lookup` allowlist to prevent DNS / NSURLSession bypass of `(deny network*)`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat + S_ISLNK` + `shutil.rmtree(onerror=_abort_on_symlink)` so a symlink encountered mid-walk aborts rather than escapes; `run_gate` wraps each task scorer in a typed `try/except` so backend failures produce `score=None, error=str(exc)` (never silent `score=1.0`); `_parse_judge_url` removes the bare `http://` catch-all (defence-in-depth after the Pydantic GateTask validator); `soup can run` requires `--yes` or explicit consent callback and raises `ValueError` (not `PermissionError`, which is an `OSError` subclass that broad `except` blocks would swallow); GGUF `rglob` result for ollama deploy is `realpath+commonpath` checked against extract_dir (prevents symlink escape from a crafted can); `DeployTarget.path` validator normalises mixed `\\`/`/` separators before splitting (closes a Windows `..` bypass); `CAN_FORMAT_VERSION` 1→2 (additive — v1 still loads); `soup can publish` validates `repo_id` via `utils/hf.validate_repo_id`, resolves token via `resolve_token`, sanitises commit messages (first-line, 200-char cap), uses HTTPS-only HfApi; `_write_spike_recovery_hint` adds `is_under_cwd` containment check on `args.output_dir` from raw HF `TrainingArguments`; `lookup_entry_by_output_dir` emits `ResourceWarning` when 1000-row scan limit is hit (no silent miss); `CrossDocCollator` no longer mutates input feature dicts (HF Dataset rows are cached and reused — mutation broke subsequent batches); `Candidate` rejects `bool` in `score`/`latency_ms` (was sneaking past `int` isinstance check); `evaluate_candidate` latency mean now divides by *completed* prompts (excludes crashed) so a broken candidate isn't artificially fast; `auto_quant.run_auto_quant_picker` soft-falls-back to highest-scored candidate when no candidate clears `min_score` (server still binds); `build_logits_processors` returns `[]` when neither `outlines` nor `lm-format-enforcer` is installed (server degrades to free-form rather than 500); MII server uses loopback-only CORS, max_tokens cap [1, 16384], stream rejection, generic 500 with no stack-trace leak; `os.execvp` auto-reexec uses list args (no shell), all forwarded flags pre-validated; `cleanup_extract_dir` uses `os.path.commonpath` (Windows-safe) instead of `startswith`; `_run_subprocess` catches `TimeoutExpired` and returns rc=124 (coreutils convention) instead of an unhandled traceback; new `eval_results` and `tensorrt` artifact kinds in `RegistryStore._VALID_KINDS`
|
||||
- **v0.52.0 — Modality II (TTS + Distillation + BitNet + EBFT-GDPO + MoE quant + reasoning_effort)**: 7 schema-only Parts; live trainer / loss / export wiring deferred to v0.52.1. Every new validator follows the project's established hardening policy: closed allowlist (`SUPPORTED_TTS_FAMILIES`, `CLASSIFIER_TASKS`, `DIVERGENCES`, `BITNET_QUANT_FORMATS`, `BITNET_EXPORT_FORMATS`, `EBFT_VARIANTS`, `GDPO_VARIANTS`, `MOE_EXPERT_QUANT_FORMATS`, `REASONING_EFFORT_LEVELS`, per-family `_FAMILY_EMOTIONS`) wrapped in `frozenset` / `MappingProxyType` so registries cannot be mutated at runtime; `validate_*` helpers reject non-string / bool / empty / null-byte / oversize / unknown inputs with case-insensitive normalisation (matches v0.41.0 `validate_optimizer_name` / v0.50.0 `grpo_variant` / v0.51.0 `hub` policy); float validators (`validate_distill_temperature`, `validate_ebft_temperature`) gate on `math.isfinite` to reject NaN AND `±inf` (matches v0.32.0 `save_lr_finder_report` policy). `field_validator(mode="before")` on `num_labels` (security-review HIGH fix) rejects `bool` before Pydantic's `ge=1` coercion silently treats `True` as `1`. Field validator on `reasoning_effort` routes through the shared `validate_reasoning_effort` helper so the schema and runtime validator agree on what's accepted (security-review MEDIUM fix). SoupConfig cross-validators: `_validate_tts_compat` (requires `task='tts'` + `modality='audio_out'` + non-MLX backend; per-family emotion allowlist via `_FAMILY_EMOTIONS`), `_validate_classifier_compat` (with lazy-import early-return — code-review HIGH fix — so SFT hot path doesn't pay import cost; requires `num_labels` on classifier tasks; rejects classifier-only fields outside the task family with named offenders), `_validate_distill_compat` (requires `teacher_model` when `task='distill'`; rejects distill-only fields outside the task), `_validate_bitnet_compat` (gates to non-MLX + text-modality + task ∈ {sft, pretrain, dpo}), `_validate_ebft_compat` + `_validate_gdpo_compat` (task-family gates), `_validate_moe_expert_quant_compat` (requires `moe_lora=true` to prevent silent no-op), `_validate_reasoning_effort_task_gate` (code-review HIGH fix — rejects `reasoning_effort` + `train_on_eot` outside the SFT-family task set with named offenders; mirrors v0.50.0 GRPO stability task-gate policy). Public `DIVERGENCES` frozenset is derived from `_DIVERGENCE_ALIASES` so adding a new alias updates both the accepted-input set and the error message in lockstep (review fix LOW). `validate_bitnet_export` enforces a closed-allowlist canonical form for `soup export --format <bitnet|tq1_0>`, both of which are CLI-registered with a yellow advisory panel + `Exit(0)` stub (no artifact written until v0.52.1 — the format flag is accepted so existing scripts pinned to v0.52.0 will not break). 6 new YAML recipes appended (5 TTS + Falcon-E BitNet) — every entry is exercised by `tests/test_v0520.py` for `load_config_from_string` round-trip + `_no_null_or_whitespace` model-id check (mirrors v0.51.0 review-fix LOW). Known limitations: (1) Every live trainer / loss / export path is deferred to v0.52.1 — `build_tts_trainer`, `build_classifier_trainer`, `build_distill_trainer`, `build_bitnet_trainer`, `export_bitnet_gguf`, `apply_ebft_loss`, `apply_gdpo_loss`, `apply_moe_expert_quant` all raise `NotImplementedError` with explicit `v0.52.1` markers; schema accepts every new task / quant / variant + the CLI stub for `soup export --format bitnet/tq1_0` prints a deferred-advisory panel and exits 0. (2) `modality='audio_out'` accepted on non-TTS tasks — design choice this release so future audio-output tasks (ASR / V2A) can reuse it; today's runtime trainer dispatch must check `task == 'tts'` to avoid silent routing into the deferred TTS path. (3) Oute emotion allowlist is a tight 6-entry subset (neutral / happy / sad / angry / calm / excited); operators wanting custom emotions will need a v0.52.1 patch to extend `OUTE_EMOTIONS`. (4) `is_bitnet_model` is best-effort heuristic over name prefixes (`bitnet`, `falcon-e`, `1bitllm`, `onebit`); a BitNet checkpoint published under an org without any of those prefixes returns False. This is detection, not gating — the trainer wrapper (v0.52.1) loads the model regardless of the heuristic. (5) `quantization='bitnet_1.58'` gated to task ∈ {sft, pretrain, dpo} — extending to GRPO / PPO / RewardModel requires upstream onebitllms RL kernels not yet shipped. (v0.52.0)
|
||||
- **v0.51.0 — Model Catalog Expansion + Alternative Model Hubs**: 5 release Parts. New `soup_cli/utils/hubs.py` ships closed allowlist `SUPPORTED_HUBS = frozenset({hf, modelscope, modelers})` + three `MappingProxyType`-wrapped registries (`_HUB_DEFAULT_ENDPOINTS` / `_HUB_ENDPOINT_ENV` / `_HUB_PACKAGE`) so the registry cannot be mutated at runtime (matches v0.36.0 `_REGISTRY` policy). `validate_hub_name` rejects non-string / bool / empty / null-byte / >32-char / unknown with case-insensitive normalisation (matches v0.41.0 `validate_optimizer_name` policy). `validate_hub_endpoint` is the SSRF kernel — full parity with v0.29.0 `utils/hf.resolve_endpoint`: scheme allowlist (`http`/`https` only), null-byte rejection, **control-character / CRLF rejection** added in v0.51.0 as a defence-in-depth review fix (defends against URL-as-HTTP-header injection if the URL ever flows into a raw HTTP client), `0.0.0.0` explicitly rejected, plain HTTP only for loopback `{localhost, 127.0.0.1, ::1}`, RFC1918 / link-local / cloud-metadata IPs (169.254.x) rejected via `ipaddress.ip_address` for plain HTTP. `resolve_endpoint(hub, *, env=None)` looks up the per-hub env var (`HF_ENDPOINT` / `MODELSCOPE_ENDPOINT` / `MODELERS_ENDPOINT`) and runs the override through `validate_hub_endpoint`; default endpoints are baked-in HTTPS URLs. `is_hf` rejects `bool` explicitly (review fix HIGH — bool is a subclass of int and would have silently fallen through `hub.lower() == "hf"` → `False`, which happens to be correct by accident but violates the contract; matches v0.30.0 `Candidate` / v0.34.0 `estimate_run_cost_usd` policy). `TrainingConfig.hub: Literal["hf","modelscope","modelers"]` field gets a `field_validator(mode="before")` `_normalize_hub` that delegates to `validate_hub_name` so `hub: HF` in YAML normalises to `"hf"` (review fix HIGH — first-cut had Pydantic Literal exact-match while `validate_hub_name` was case-insensitive, breaking the v0.41.0 `validate_optimizer_name` / v0.50.0 `grpo_variant` / `rollout_backend` policy of agreement between schema and shared validator). SoupConfig `_validate_hub_supported` cross-validator rejects `hub != 'hf'` on `backend == 'mlx'` with a distinct error message (review fix HIGH — `mlx-lm` only downloads from HF Hub; without this gate a `backend: mlx` + `hub: modelscope` config would silently pass schema load and fail at runtime with a confusing `mlx-lm` error). 26 new YAML recipes appended to `soup_cli/recipes/catalog.py` — every entry is exercised by `tests/test_v0510.py` via `load_config_from_string` round-trip + `yaml.safe_load` (no Python tags / no template injection / no credential leak in the YAML strings) + a `_no_null_or_whitespace` model-id check that rejects empty path components (review fix LOW — first-cut allowed `"/name"` leading-slash IDs to pass). Two non-`<N>B` `size` strings (`"image"` / `"ocr"` / `"moe"` / `"medium"`) were normalised to `"N/A"` (review fix MEDIUM — `search_recipes(size=…)` would silently miss those entries, and the autopilot VRAM estimator could not parse them). Known limitations: (1) Live downloader / uploader / push integration deferred to v0.51.1 — `TrainingConfig.hub` schema lock-in ships now (Literal accept + MLX cross-validator + case-normalisation), but `soup data download --hub modelscope` and `soup push --hub modelers` still route through the existing HF Hub code path; the actual `modelscope-sdk` / `openmind-hub` adapters are the v0.51.1 deliverable. Same stub-then-live pattern as v0.27.0 MII / v0.37.0 multipack / v0.50.0 GRPO Plus. (2) Speculative / aspirational `base` model IDs in some Part A/C recipes — the catalog ships entries for `openai/gpt-oss-{20,120}b`, `THUDM/glm-5`, `Qwen/Qwen-Image`, `deepseek-ai/DeepSeek-OCR`, `PaddlePaddle/PaddleOCR-VL`, `google/embeddinggemma-300m` so users have ready-made recipes the moment those repos go live (matches the plan's "match Unsloth's day-zero coverage" directive). Recipes for not-yet-published repos will surface a clear HF Hub 404 when the user runs `soup train --recipe <name>`. (3) DNS-resolved private hostnames not blocked — `validate_hub_endpoint` only rejects literal RFC1918 / link-local IP addresses; a hostname like `corp-proxy.internal` that DNS-resolves to a private IP is accepted at validation time (mirrors the v0.29.0 `HF_ENDPOINT` policy — DNS resolution is intentionally not performed in this local-tool threat model). (v0.51.0)
|
||||
- **v0.50.0 — GRPO Plus (RL parity)**: 22 features across 5 Parts shipped as schema-only (closed allowlists + Pydantic validators + NotImplementedError stubs for live wiring deferred to v0.50.1). All new validators follow the project's bool-rejection-before-int policy (matches v0.30.0 `Candidate`); closed-allowlist `validate_grpo_variant` / `validate_rollout_backend` reject non-string / bool / empty / null-byte / oversize / unknown inputs with actionable error messages and case-insensitive normalisation. `validate_grpo_delta` is bool-first / `math.isfinite` / `(0, 1]` bounded (matches v0.32.0 `save_lr_finder_report` / v0.41.0 Part B `lr_groups` policy). New `_VARIANT_METADATA` (Part A) and `_BACKEND_METADATA` (Part C) are `MappingProxyType`-wrapped frozen-dataclass registries (matches v0.36.0 `_REGISTRY` / v0.41.0 `_OPTIMIZER_PACKAGES` policy). Security-review fixes: (1) `grpo_delta` schema gets an explicit `field_validator(mode='after')` calling `math.isfinite` — Pydantic's `gt=0, le=1` bounds only incidentally reject NaN (since `NaN > 0` is False); the explicit validator prevents a future Pydantic change from regressing the guard. (2) `validate_long_context_grpo_compat` adds null-byte rejection on `task` AND `backend` strings + a `bool` guard on `use_ring_attention` (parity with `validate_grpo_variant` / `validate_rollout_backend`). (3) `validate_vllm_sleep_mode_compat` adds null-byte rejection on `backend`. Code-review HIGH fixes: (4) `_validate_grpo_stability_task_gate` now includes `grpo_fp16` in the GRPO-only-fields list — previously a user could silently set `grpo_fp16: true` on `task='sft'` and have it no-op. (5) `_validate_vllm_sleep_mode` now requires `task='grpo'` (sleep mode is a between-rollouts feature, meaningless on SFT) and rejects with a `task='grpo'` message. TDD-review HIGH fixes: (6) new `_reject_bool_on_grpo_numerics` field_validator on every Part D numeric field + `grpo_delta` explicitly rejects `bool` before Pydantic's `True→1` coercion (matches v0.30.0 / v0.41.0 Part B / v0.43.0 Part B policy). Known limitations: (1) Every live loss kernel / launcher (`apply_variant_loss`, `apply_vllm_sleep_mode`, `launch_rollout`, `build_prm_trainer`) raises `NotImplementedError` with explicit `v0.50.1` markers — same stub-then-live pattern as v0.27.0 MII / v0.37.0 multipack / v0.41.0 LLaMA Pro / v0.45.0 plugins / v0.48.0 curriculum / v0.49.0 LongLoRA. (2) `long_context_grpo` requires Tiled MLP (v0.56.0 Part A) to actually run; the schema gate ships now so v0.50.0 configs are stable. (3) `vision_grpo=true` does not check whether the base model is actually a VLM — upstream trainer surfaces that error loudly. (4) The 7 stability knobs schema-validate but none are wired into a live callback in this release; `replay_buffer_size`, `defer_rerolling`, and `skip_zero_advantage` are pure schema lock-ins. (v0.50.0)
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "soup-cli"
|
||||
version = "0.51.0"
|
||||
version = "0.52.0"
|
||||
description = "Fine-tune LLMs in one command. No SSH, no config hell."
|
||||
readme = "README.md"
|
||||
license = "Apache-2.0"
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
"""Soup CLI — Fine-tune LLMs in one command."""
|
||||
|
||||
__version__ = "0.51.0"
|
||||
__version__ = "0.52.0"
|
||||
|
|
|
|||
|
|
@ -13,7 +13,12 @@ from rich.panel import Panel
|
|||
|
||||
console = Console()
|
||||
|
||||
SUPPORTED_FORMATS = ("gguf", "onnx", "tensorrt", "awq", "gptq")
|
||||
SUPPORTED_FORMATS = (
|
||||
"gguf", "onnx", "tensorrt", "awq", "gptq",
|
||||
# v0.52.0 Part D — BitNet 1.58-bit + TQ1_0 GGUF.
|
||||
# Schema-only stubs in v0.52.0; live conversion lands in v0.52.1.
|
||||
"bitnet", "tq1_0",
|
||||
)
|
||||
GGUF_QUANT_TYPES = ("q4_0", "q4_k_m", "q5_k_m", "q8_0", "f16", "f32")
|
||||
LLAMA_CPP_DIR_NAME = "llama.cpp"
|
||||
# Pin to a known release tag for supply-chain safety
|
||||
|
|
@ -31,7 +36,7 @@ def export(
|
|||
"gguf",
|
||||
"--format",
|
||||
"-f",
|
||||
help="Export format: gguf, onnx, tensorrt",
|
||||
help="Export format: gguf, onnx, tensorrt, awq, gptq, bitnet, tq1_0",
|
||||
),
|
||||
quant: str = typer.Option(
|
||||
"q4_k_m",
|
||||
|
|
@ -147,6 +152,29 @@ def export(
|
|||
)
|
||||
return
|
||||
|
||||
# --- BitNet 1.58-bit / TQ1_0 GGUF — schema-only stubs (v0.52.0) ---
|
||||
# Live conversion via onebitllms + llama.cpp TQ1_0 lands in v0.52.1.
|
||||
if fmt in ("bitnet", "tq1_0"):
|
||||
from soup_cli.utils.bitnet import validate_bitnet_export
|
||||
try:
|
||||
canonical = validate_bitnet_export(fmt)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(2)
|
||||
console.print(Panel.fit(
|
||||
(
|
||||
f"[yellow]{canonical} export is schema-locked in v0.52.0; "
|
||||
"live conversion lands in v0.52.1.[/]\n\n"
|
||||
"The format flag is accepted so existing scripts pinned to "
|
||||
"v0.52.0 will not break, but no artifact is written yet. "
|
||||
"Watch the v0.52.1 release notes for the live "
|
||||
"onebitllms / llama.cpp wiring."
|
||||
),
|
||||
title=f"export --format {canonical} (deferred)",
|
||||
border_style="yellow",
|
||||
))
|
||||
raise typer.Exit(0)
|
||||
|
||||
if quant not in GGUF_QUANT_TYPES:
|
||||
console.print(
|
||||
f"[red]Unsupported quantization: {quant}[/]\n"
|
||||
|
|
|
|||
|
|
@ -705,6 +705,8 @@ class TrainingConfig(BaseModel):
|
|||
"eetq",
|
||||
"mxfp4",
|
||||
"fp8",
|
||||
# v0.52.0 Part D — BitNet 1.58-bit (axolotl + onebitllms).
|
||||
"bitnet_1.58",
|
||||
] = Field(
|
||||
default="4bit",
|
||||
description=(
|
||||
|
|
@ -714,7 +716,8 @@ class TrainingConfig(BaseModel):
|
|||
"hqq:Nbit (HQQ 1-8 bit, N in {1..6, 8}), "
|
||||
"aqlm (extreme 2-bit), eetq (8-bit fast), "
|
||||
"mxfp4 (BNB 4-bit MXFP4 quant_type), "
|
||||
"fp8 (load FP8 checkpoint with dequantize-on-load)."
|
||||
"fp8 (load FP8 checkpoint with dequantize-on-load), "
|
||||
"bitnet_1.58 (BitNet ternary, v0.52.0 schema-only)."
|
||||
),
|
||||
)
|
||||
gptq_disable_exllama: bool = Field(
|
||||
|
|
@ -1059,6 +1062,243 @@ class TrainingConfig(BaseModel):
|
|||
),
|
||||
)
|
||||
|
||||
# ---- v0.52.0 — Modality II (schema-only; live wiring in v0.52.1) ----
|
||||
# Part A — TTS
|
||||
tts_family: Optional[Literal[
|
||||
"orpheus", "sesame_csm", "llasa", "spark", "oute"
|
||||
]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"TTS model family — required when task='tts'. One of: orpheus, "
|
||||
"sesame_csm, llasa, spark, oute. Schema-only in v0.52.0; live "
|
||||
"trainer wrapper deferred to v0.52.1."
|
||||
),
|
||||
)
|
||||
tts_emotion: Optional[str] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Optional emotion tag for emotion-conditioned families "
|
||||
"(Orpheus / Oute). Allowlisted per-family. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
# Part B — classifier / reranker / cross_encoder
|
||||
num_labels: Optional[int] = Field(
|
||||
default=None, ge=1, le=1024,
|
||||
description=(
|
||||
"Number of output labels for task in (classifier, reranker, "
|
||||
"cross_encoder). Required when task is one of those. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
classifier_kind: Optional[Literal["single_label", "multi_label"]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Sequence-classification head kind: single_label (default for "
|
||||
"task='classifier') or multi_label. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
label_names: Optional[List[str]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Optional human-readable label names. Length must match "
|
||||
"num_labels. Capped at 1024 entries. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
# Part C — knowledge distillation
|
||||
teacher_model: Optional[str] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Teacher model HF id or local path — required when task='distill'. "
|
||||
"Null-byte rejected, capped at 512 chars. Schema-only in v0.52.0; "
|
||||
"live distill trainer deferred to v0.52.1."
|
||||
),
|
||||
)
|
||||
distill_divergence: Optional[Literal[
|
||||
"forward_kl", "reverse_kl", "js"
|
||||
]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Divergence used for distillation loss. 'kl' is an alias for "
|
||||
"'forward_kl' (canonical form). (v0.52.0)"
|
||||
),
|
||||
)
|
||||
distill_temperature: Optional[float] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Softmax temperature applied to teacher and student logits "
|
||||
"before the divergence. Bounded [0.05, 100.0]. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
# Part E — EBFT + GDPO
|
||||
ebft_variant: Optional[Literal["structured", "strided"]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Energy-Based FT variant. SFT-task-only; live loss kernel "
|
||||
"deferred to v0.52.1. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
ebft_temperature: Optional[float] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Sampling temperature for EBFT energy proxy. Bounded "
|
||||
"[1e-4, 100.0]. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
gdpo_variant: Optional[Literal[
|
||||
"standard", "length_normalized", "margin"
|
||||
]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Generalized DPO variant. DPO-family-task-only; live loss kernel "
|
||||
"deferred to v0.52.1. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
# Part F — MoE expert quantization + router-only training
|
||||
moe_expert_quant: Optional[Literal["nf4", "int8_rowwise"]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Per-expert quantization for fused-MoE Linear blocks. "
|
||||
"Requires moe_lora=true. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
train_router_only: bool = Field(
|
||||
default=False,
|
||||
description=(
|
||||
"Freeze every expert + train only the gating router (unsloth "
|
||||
"MoE recipe). Requires moe_lora=true. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
# Part G — gpt-oss reasoning effort + EOT control
|
||||
reasoning_effort: Optional[Literal["low", "medium", "high"]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"gpt-oss train-time reasoning effort level. Routes through a "
|
||||
"prompt prefix at training time; live formatter wiring deferred "
|
||||
"to v0.52.1. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
train_on_eot: bool = Field(
|
||||
default=False,
|
||||
description=(
|
||||
"Include explicit EOT / EOS control tokens in the SFT loss "
|
||||
"(axolotl ``train_on_eot``). Default False matches HF Trainer "
|
||||
"convention. (v0.52.0)"
|
||||
),
|
||||
)
|
||||
|
||||
@field_validator("teacher_model")
|
||||
@classmethod
|
||||
def _validate_teacher_model(cls, v: Optional[str]) -> Optional[str]:
|
||||
"""v0.52.0 Part C — null-byte rejection + 512-char cap.
|
||||
|
||||
Mirrors v0.40.5 ``reward_model`` field-validator policy. The bool
|
||||
rejection happens inside ``validate_teacher_model`` which lives in
|
||||
``utils/distill.py`` so the runtime validator and schema agree on
|
||||
what's accepted.
|
||||
"""
|
||||
if v is None:
|
||||
return v
|
||||
from soup_cli.utils.distill import validate_teacher_model
|
||||
|
||||
return validate_teacher_model(v)
|
||||
|
||||
@field_validator("distill_divergence", mode="before")
|
||||
@classmethod
|
||||
def _normalize_distill_divergence(cls, v):
|
||||
"""v0.52.0 Part C — canonicalise ``kl`` → ``forward_kl``.
|
||||
|
||||
Mirrors v0.51.0 ``_normalize_hub`` policy: the field validator runs
|
||||
the shared ``validate_*`` helper at ``mode='before'`` so the public
|
||||
schema and runtime validator agree on what's accepted.
|
||||
"""
|
||||
if v is None:
|
||||
return None
|
||||
from soup_cli.utils.distill import validate_divergence
|
||||
|
||||
return validate_divergence(v)
|
||||
|
||||
@field_validator("distill_temperature", mode="before")
|
||||
@classmethod
|
||||
def _validate_distill_temperature(cls, v):
|
||||
"""v0.52.0 Part C — bool/NaN-rejected float in [0.05, 100]."""
|
||||
if v is None:
|
||||
return None
|
||||
from soup_cli.utils.distill import validate_distill_temperature
|
||||
|
||||
return validate_distill_temperature(v)
|
||||
|
||||
@field_validator("ebft_temperature", mode="before")
|
||||
@classmethod
|
||||
def _validate_ebft_temperature(cls, v):
|
||||
"""v0.52.0 Part E — bool/NaN-rejected float in [1e-4, 100]."""
|
||||
if v is None:
|
||||
return None
|
||||
from soup_cli.utils.ebft_gdpo import validate_ebft_temperature
|
||||
|
||||
return validate_ebft_temperature(v)
|
||||
|
||||
@field_validator("label_names")
|
||||
@classmethod
|
||||
def _validate_label_names(cls, v):
|
||||
"""v0.52.0 Part B — dedup + per-entry validation."""
|
||||
if v is None:
|
||||
return None
|
||||
from soup_cli.utils.classifier import validate_label_names
|
||||
|
||||
return validate_label_names(v)
|
||||
|
||||
@field_validator("num_labels", mode="before")
|
||||
@classmethod
|
||||
def _validate_num_labels(cls, v):
|
||||
"""v0.52.0 Part B (security review fix) — bool-before-int guard.
|
||||
|
||||
Pydantic v2's ``Field(ge=1, le=1024)`` accepts ``True`` because bool
|
||||
subclasses int; explicit guard matches the project policy
|
||||
established in v0.30.0 ``Candidate`` / v0.36.0 ``make_cache_key`` /
|
||||
v0.41.0 ``expand_layers`` / v0.50.0 GRPO numeric fields.
|
||||
"""
|
||||
if v is None:
|
||||
return None
|
||||
from soup_cli.utils.classifier import validate_num_labels
|
||||
|
||||
return validate_num_labels(v)
|
||||
|
||||
@field_validator("reasoning_effort", mode="before")
|
||||
@classmethod
|
||||
def _validate_reasoning_effort(cls, v):
|
||||
"""v0.52.0 Part G (security review fix) — canonicalise case +
|
||||
bool/null-byte/oversize rejection via the shared helper.
|
||||
|
||||
Mirrors v0.51.0 ``_normalize_hub`` and v0.41.0 ``optimizer``
|
||||
policy of routing through the public ``validate_*`` helper at
|
||||
``mode='before'`` so the schema and runtime helper agree on what's
|
||||
accepted.
|
||||
"""
|
||||
if v is None:
|
||||
return None
|
||||
from soup_cli.utils.reasoning_effort import validate_reasoning_effort
|
||||
|
||||
return validate_reasoning_effort(v)
|
||||
|
||||
@field_validator("tts_emotion")
|
||||
@classmethod
|
||||
def _validate_tts_emotion_field(cls, v: Optional[str]) -> Optional[str]:
|
||||
"""v0.52.0 Part A — bool / null-byte / oversize rejection (without
|
||||
family-specific allowlist; that fires in the cross-validator).
|
||||
"""
|
||||
if v is None:
|
||||
return None
|
||||
if isinstance(v, bool):
|
||||
raise ValueError("tts_emotion must not be bool")
|
||||
if not isinstance(v, str):
|
||||
raise ValueError("tts_emotion must be str")
|
||||
if not v:
|
||||
raise ValueError("tts_emotion must be non-empty")
|
||||
if "\x00" in v:
|
||||
raise ValueError("tts_emotion must not contain null bytes")
|
||||
if len(v) > 32:
|
||||
raise ValueError("tts_emotion too long (max 32 chars)")
|
||||
return v
|
||||
|
||||
@field_validator("hub", mode="before")
|
||||
@classmethod
|
||||
def _normalize_hub(cls, v):
|
||||
|
|
@ -1896,16 +2136,24 @@ class SoupConfig(BaseModel):
|
|||
task: Literal[
|
||||
"sft", "dpo", "grpo", "ppo", "reward_model", "kto", "orpo", "simpo", "ipo",
|
||||
"bco", "preference", "pretrain", "embedding", "prm",
|
||||
# v0.52.0 Modality II — TTS / classifier-family / distillation.
|
||||
"tts", "classifier", "reranker", "cross_encoder", "distill",
|
||||
] = Field(
|
||||
default="sft",
|
||||
description=(
|
||||
"Training task type. v0.50.0 Part E adds 'prm' (Process Reward "
|
||||
"Model / stepwise supervised — paired with data.format='prm')."
|
||||
"Training task type. v0.50.0 Part E added 'prm'; v0.52.0 adds "
|
||||
"'tts' (TTS fine-tuning), 'classifier' / 'reranker' / "
|
||||
"'cross_encoder' (classification heads), and 'distill' "
|
||||
"(knowledge distillation)."
|
||||
),
|
||||
)
|
||||
modality: Literal["text", "vision", "audio"] = Field(
|
||||
modality: Literal["text", "vision", "audio", "audio_out"] = Field(
|
||||
default="text",
|
||||
description="Training modality: text (default), vision (multimodal), or audio",
|
||||
description=(
|
||||
"Training modality: text (default), vision (multimodal), audio "
|
||||
"(audio-input), or audio_out (audio-output — paired with task='tts', "
|
||||
"v0.52.0)."
|
||||
),
|
||||
)
|
||||
backend: Literal["transformers", "unsloth", "mlx"] = Field(
|
||||
default="transformers",
|
||||
|
|
@ -2230,6 +2478,249 @@ class SoupConfig(BaseModel):
|
|||
)
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_tts_compat(self) -> "SoupConfig":
|
||||
"""v0.52.0 Part A — ``task='tts'`` gate."""
|
||||
tcfg = self.training
|
||||
if self.task != "tts" and tcfg.tts_family is None and tcfg.tts_emotion is None:
|
||||
return self
|
||||
if self.task == "tts":
|
||||
from soup_cli.utils.tts import (
|
||||
validate_emotion_tag,
|
||||
validate_tts_compat,
|
||||
)
|
||||
|
||||
try:
|
||||
validate_tts_compat(
|
||||
task=self.task,
|
||||
modality=self.modality,
|
||||
backend=self.backend,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
if tcfg.tts_family is None:
|
||||
raise ValueError(
|
||||
"task='tts' requires training.tts_family in "
|
||||
"(orpheus, sesame_csm, llasa, spark, oute)"
|
||||
)
|
||||
if tcfg.tts_emotion is not None:
|
||||
try:
|
||||
validate_emotion_tag(tcfg.tts_emotion, family=tcfg.tts_family)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
return self
|
||||
# tts_family / tts_emotion outside task='tts' is a silent-no-op
|
||||
# footgun; reject loudly (mirrors v0.50.0 GRPO stability policy).
|
||||
if tcfg.tts_family is not None:
|
||||
raise ValueError(
|
||||
f"training.tts_family={tcfg.tts_family!r} requires task='tts'; "
|
||||
f"got task={self.task!r}"
|
||||
)
|
||||
if tcfg.tts_emotion is not None:
|
||||
raise ValueError(
|
||||
f"training.tts_emotion={tcfg.tts_emotion!r} requires task='tts'; "
|
||||
f"got task={self.task!r}"
|
||||
)
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_classifier_compat(self) -> "SoupConfig":
|
||||
"""v0.52.0 Part B — classifier / reranker / cross_encoder gate.
|
||||
|
||||
Lazy-import policy (code-review fix): guard the import behind the
|
||||
cheap str-only ``self.task`` check so the common ``task='sft'`` hot
|
||||
path does not pay an import cost on every config load.
|
||||
"""
|
||||
tcfg = self.training
|
||||
classifier_tasks = {"classifier", "reranker", "cross_encoder"}
|
||||
classifier_fields_set = (
|
||||
tcfg.num_labels is not None
|
||||
or tcfg.classifier_kind is not None
|
||||
or tcfg.label_names is not None
|
||||
)
|
||||
if self.task not in classifier_tasks and not classifier_fields_set:
|
||||
return self
|
||||
from soup_cli.utils.classifier import (
|
||||
is_classifier_task,
|
||||
validate_classifier_compat,
|
||||
)
|
||||
|
||||
if is_classifier_task(self.task):
|
||||
try:
|
||||
validate_classifier_compat(
|
||||
task=self.task,
|
||||
backend=self.backend,
|
||||
modality=self.modality,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
if tcfg.num_labels is None:
|
||||
raise ValueError(
|
||||
f"task={self.task!r} requires training.num_labels "
|
||||
"(positive int <= 1024)"
|
||||
)
|
||||
if (
|
||||
tcfg.label_names is not None
|
||||
and len(tcfg.label_names) != tcfg.num_labels
|
||||
):
|
||||
raise ValueError(
|
||||
f"len(label_names)={len(tcfg.label_names)} does not "
|
||||
f"match num_labels={tcfg.num_labels}"
|
||||
)
|
||||
return self
|
||||
# Reject classifier-only fields when task is not a classifier task.
|
||||
for field in ("num_labels", "classifier_kind", "label_names"):
|
||||
value = getattr(tcfg, field)
|
||||
if value is not None:
|
||||
raise ValueError(
|
||||
f"training.{field} requires task in "
|
||||
"(classifier, reranker, cross_encoder); "
|
||||
f"got task={self.task!r}"
|
||||
)
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_distill_compat(self) -> "SoupConfig":
|
||||
"""v0.52.0 Part C — ``task='distill'`` gate."""
|
||||
tcfg = self.training
|
||||
distill_fields_set = (
|
||||
tcfg.teacher_model is not None
|
||||
or tcfg.distill_divergence is not None
|
||||
or tcfg.distill_temperature is not None
|
||||
)
|
||||
if self.task == "distill":
|
||||
from soup_cli.utils.distill import validate_distill_compat
|
||||
|
||||
try:
|
||||
validate_distill_compat(
|
||||
task=self.task,
|
||||
backend=self.backend,
|
||||
teacher_model=tcfg.teacher_model,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
return self
|
||||
if distill_fields_set:
|
||||
offenders = [
|
||||
name for name, value in (
|
||||
("teacher_model", tcfg.teacher_model),
|
||||
("distill_divergence", tcfg.distill_divergence),
|
||||
("distill_temperature", tcfg.distill_temperature),
|
||||
) if value is not None
|
||||
]
|
||||
raise ValueError(
|
||||
f"Distillation fields {offenders} require task='distill'; "
|
||||
f"got task={self.task!r}"
|
||||
)
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_bitnet_compat(self) -> "SoupConfig":
|
||||
"""v0.52.0 Part D — ``quantization='bitnet_1.58'`` gate."""
|
||||
if self.training.quantization != "bitnet_1.58":
|
||||
return self
|
||||
from soup_cli.utils.bitnet import validate_bitnet_compat
|
||||
|
||||
try:
|
||||
validate_bitnet_compat(
|
||||
task=self.task,
|
||||
backend=self.backend,
|
||||
modality=self.modality,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_ebft_compat(self) -> "SoupConfig":
|
||||
"""v0.52.0 Part E — ``ebft_variant`` requires SFT, non-MLX."""
|
||||
tcfg = self.training
|
||||
if tcfg.ebft_variant is None and tcfg.ebft_temperature is None:
|
||||
return self
|
||||
if tcfg.ebft_variant is None and tcfg.ebft_temperature is not None:
|
||||
raise ValueError(
|
||||
"training.ebft_temperature requires training.ebft_variant "
|
||||
"to be set"
|
||||
)
|
||||
from soup_cli.utils.ebft_gdpo import validate_ebft_compat
|
||||
|
||||
try:
|
||||
validate_ebft_compat(task=self.task, backend=self.backend)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_gdpo_compat(self) -> "SoupConfig":
|
||||
"""v0.52.0 Part E — ``gdpo_variant`` requires DPO/preference, non-MLX."""
|
||||
if self.training.gdpo_variant is None:
|
||||
return self
|
||||
from soup_cli.utils.ebft_gdpo import validate_gdpo_compat
|
||||
|
||||
try:
|
||||
validate_gdpo_compat(task=self.task, backend=self.backend)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_reasoning_effort_task_gate(self) -> "SoupConfig":
|
||||
"""v0.52.0 Part G (code-review fix) — surface the silent-no-op
|
||||
footgun when ``reasoning_effort`` / ``train_on_eot`` is set on a
|
||||
task they cannot influence.
|
||||
|
||||
Mirrors v0.50.0 ``_validate_grpo_stability_task_gate`` policy.
|
||||
``reasoning_effort`` only makes sense on SFT-family training
|
||||
(sft / pretrain / distill / classifier-family) because the live
|
||||
formatter (v0.52.1) will inject a system-prefix token. The other
|
||||
tasks (DPO / GRPO / KTO / ORPO / SimPO / IPO / BCO / preference /
|
||||
PPO / reward_model / embedding / prm / tts) do not consume it.
|
||||
|
||||
``train_on_eot`` is an SFT loss-mask flag; setting it on
|
||||
DPO/GRPO/etc. is a silent no-op.
|
||||
"""
|
||||
tcfg = self.training
|
||||
sft_family_tasks = {
|
||||
"sft", "pretrain", "distill",
|
||||
"classifier", "reranker", "cross_encoder",
|
||||
}
|
||||
if tcfg.reasoning_effort is not None and self.task not in sft_family_tasks:
|
||||
raise ValueError(
|
||||
f"training.reasoning_effort={tcfg.reasoning_effort!r} requires "
|
||||
f"task in {sorted(sft_family_tasks)}; got task={self.task!r}"
|
||||
)
|
||||
if tcfg.train_on_eot and self.task not in sft_family_tasks:
|
||||
raise ValueError(
|
||||
f"training.train_on_eot=true requires task in "
|
||||
f"{sorted(sft_family_tasks)}; got task={self.task!r}"
|
||||
)
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_moe_expert_quant_compat(self) -> "SoupConfig":
|
||||
"""v0.52.0 Part F — ``moe_expert_quant`` + ``train_router_only`` gates."""
|
||||
tcfg = self.training
|
||||
from soup_cli.utils.moe_quant import (
|
||||
validate_moe_expert_quant_compat,
|
||||
validate_train_router_only_compat,
|
||||
)
|
||||
|
||||
if tcfg.moe_expert_quant is not None:
|
||||
try:
|
||||
validate_moe_expert_quant_compat(
|
||||
backend=self.backend, moe_lora=tcfg.moe_lora,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
if tcfg.train_router_only:
|
||||
try:
|
||||
validate_train_router_only_compat(
|
||||
backend=self.backend, moe_lora=tcfg.moe_lora,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_rollout_backend(self) -> "SoupConfig":
|
||||
"""v0.50.0 Part C — ``rollout_backend`` requires task='grpo' and a
|
||||
|
|
|
|||
|
|
@ -3241,6 +3241,170 @@ training:
|
|||
quantization: 4bit
|
||||
gradient_checkpointing: true
|
||||
|
||||
output: ./output
|
||||
""",
|
||||
),
|
||||
# ---- v0.52.0 Modality II — TTS / BitNet / classifier / distill ----
|
||||
"orpheus-tts-sft": RecipeMeta(
|
||||
model="canopylabs/orpheus-3b-0.1-ft",
|
||||
task="tts",
|
||||
size="3B",
|
||||
tags=("tts", "orpheus", "audio_out", "v0.52.0"),
|
||||
description="Orpheus emotional TTS — schema-only stub (live in v0.52.1)",
|
||||
yaml_str="""\
|
||||
base: canopylabs/orpheus-3b-0.1-ft
|
||||
task: tts
|
||||
modality: audio_out
|
||||
|
||||
data:
|
||||
train: ./data/tts_train.jsonl
|
||||
format: audio
|
||||
audio_dir: ./data/audio
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 5e-5
|
||||
batch_size: auto
|
||||
tts_family: orpheus
|
||||
tts_emotion: neutral
|
||||
lora:
|
||||
r: 16
|
||||
alpha: 32
|
||||
target_modules: auto
|
||||
|
||||
output: ./output
|
||||
""",
|
||||
),
|
||||
"sesame-csm-tts": RecipeMeta(
|
||||
model="sesame/csm-1b",
|
||||
task="tts",
|
||||
size="1B",
|
||||
tags=("tts", "sesame", "audio_out", "v0.52.0"),
|
||||
description="Sesame CSM conversational speech — schema-only stub",
|
||||
yaml_str="""\
|
||||
base: sesame/csm-1b
|
||||
task: tts
|
||||
modality: audio_out
|
||||
|
||||
data:
|
||||
train: ./data/tts_train.jsonl
|
||||
format: audio
|
||||
audio_dir: ./data/audio
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 5e-5
|
||||
batch_size: auto
|
||||
tts_family: sesame_csm
|
||||
|
||||
output: ./output
|
||||
""",
|
||||
),
|
||||
"llasa-tts": RecipeMeta(
|
||||
model="HKUSTAudio/Llasa-1B",
|
||||
task="tts",
|
||||
size="1B",
|
||||
tags=("tts", "llasa", "audio_out", "v0.52.0"),
|
||||
description="Llasa-TTS — schema-only stub",
|
||||
yaml_str="""\
|
||||
base: HKUSTAudio/Llasa-1B
|
||||
task: tts
|
||||
modality: audio_out
|
||||
|
||||
data:
|
||||
train: ./data/tts_train.jsonl
|
||||
format: audio
|
||||
audio_dir: ./data/audio
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 5e-5
|
||||
batch_size: auto
|
||||
tts_family: llasa
|
||||
|
||||
output: ./output
|
||||
""",
|
||||
),
|
||||
"spark-tts": RecipeMeta(
|
||||
model="SparkAudio/Spark-TTS-0.5B",
|
||||
task="tts",
|
||||
size="0.5B",
|
||||
tags=("tts", "spark", "audio_out", "v0.52.0"),
|
||||
description="Spark-TTS — schema-only stub",
|
||||
yaml_str="""\
|
||||
base: SparkAudio/Spark-TTS-0.5B
|
||||
task: tts
|
||||
modality: audio_out
|
||||
|
||||
data:
|
||||
train: ./data/tts_train.jsonl
|
||||
format: audio
|
||||
audio_dir: ./data/audio
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 5e-5
|
||||
batch_size: auto
|
||||
tts_family: spark
|
||||
|
||||
output: ./output
|
||||
""",
|
||||
),
|
||||
"oute-tts": RecipeMeta(
|
||||
model="OuteAI/OuteTTS-0.3-500M",
|
||||
task="tts",
|
||||
size="0.5B",
|
||||
tags=("tts", "oute", "audio_out", "emotion", "v0.52.0"),
|
||||
description="Oute-TTS with emotion conditioning — schema-only stub",
|
||||
yaml_str="""\
|
||||
base: OuteAI/OuteTTS-0.3-500M
|
||||
task: tts
|
||||
modality: audio_out
|
||||
|
||||
data:
|
||||
train: ./data/tts_train.jsonl
|
||||
format: audio
|
||||
audio_dir: ./data/audio
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 5e-5
|
||||
batch_size: auto
|
||||
tts_family: oute
|
||||
|
||||
output: ./output
|
||||
""",
|
||||
),
|
||||
"falcon-e-bitnet-sft": RecipeMeta(
|
||||
model="tiiuae/Falcon-E-1B-Instruct",
|
||||
task="sft",
|
||||
size="1B",
|
||||
tags=("bitnet", "1.58bit", "falcon-e", "ternary", "v0.52.0"),
|
||||
description="Falcon-E BitNet 1.58-bit SFT — schema-only stub",
|
||||
yaml_str="""\
|
||||
base: tiiuae/Falcon-E-1B-Instruct
|
||||
task: sft
|
||||
|
||||
data:
|
||||
train: ./data/train.jsonl
|
||||
format: auto
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-4
|
||||
batch_size: auto
|
||||
quantization: bitnet_1.58
|
||||
lora:
|
||||
r: 16
|
||||
alpha: 32
|
||||
target_modules: auto
|
||||
|
||||
output: ./output
|
||||
""",
|
||||
),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,173 @@
|
|||
"""v0.52.0 Part D — BitNet 1.58-bit fine-tuning + export schema helpers.
|
||||
|
||||
Schema-only support for ``quantization='bitnet_1.58'`` and the new
|
||||
``soup export --format bitnet`` / ``--format tq1_0`` GGUF flavours.
|
||||
|
||||
Live ``onebitllms`` wrapping + llama.cpp ``TQ1_0`` export wiring are
|
||||
deferred to v0.52.1 (mirrors v0.50.0 stub-then-live pattern).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from types import MappingProxyType
|
||||
from typing import Mapping
|
||||
|
||||
# Closed allowlist of BitNet-flavoured quant strings exposed to YAML.
|
||||
BITNET_QUANT_FORMATS: frozenset[str] = frozenset({"bitnet_1.58"})
|
||||
|
||||
# Closed allowlist of BitNet export targets (the actual export formats).
|
||||
BITNET_EXPORT_FORMATS: frozenset[str] = frozenset({"bitnet", "tq1_0"})
|
||||
|
||||
_BITNET_FAMILY_RE_PREFIXES: tuple[str, ...] = (
|
||||
"bitnet", "falcon-e", "falcone", "1bitllm", "onebit",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BitNetSpec:
|
||||
"""Metadata for the BitNet quant path. Frozen — immutable."""
|
||||
|
||||
name: str
|
||||
description: str
|
||||
bits: float
|
||||
live_wired: bool
|
||||
|
||||
|
||||
_BITNET_METADATA: Mapping[str, BitNetSpec] = MappingProxyType({
|
||||
"bitnet_1.58": BitNetSpec(
|
||||
name="bitnet_1.58",
|
||||
description="BitNet 1.58-bit ternary weights (axolotl + onebitllms)",
|
||||
bits=1.58,
|
||||
live_wired=False,
|
||||
),
|
||||
})
|
||||
|
||||
|
||||
def is_bitnet_quant(value: object) -> bool:
|
||||
"""Return True iff ``value`` is a BitNet quant string."""
|
||||
if isinstance(value, bool):
|
||||
return False
|
||||
if not isinstance(value, str):
|
||||
return False
|
||||
return value in BITNET_QUANT_FORMATS
|
||||
|
||||
|
||||
def is_bitnet_export_format(value: object) -> bool:
|
||||
"""Return True iff ``value`` is a BitNet export-format string."""
|
||||
if isinstance(value, bool):
|
||||
return False
|
||||
if not isinstance(value, str):
|
||||
return False
|
||||
return value in BITNET_EXPORT_FORMATS
|
||||
|
||||
|
||||
def get_bitnet_spec(name: str) -> BitNetSpec:
|
||||
"""Return the frozen :class:`BitNetSpec` for ``name`` or raise."""
|
||||
if not is_bitnet_quant(name):
|
||||
supported = ", ".join(sorted(BITNET_QUANT_FORMATS))
|
||||
raise ValueError(
|
||||
f"BitNet quant {name!r} not supported. Supported: {supported}"
|
||||
)
|
||||
return _BITNET_METADATA[name]
|
||||
|
||||
|
||||
def is_bitnet_model(model_name: object) -> bool:
|
||||
"""Best-effort detect whether ``model_name`` references a BitNet family.
|
||||
|
||||
Checks every slash-delimited component (lowercased) against the
|
||||
``_BITNET_FAMILY_RE_PREFIXES`` prefix list. This is intentionally more
|
||||
permissive than v0.39.0 ``is_gemma4_model`` because BitNet families
|
||||
typically live under namespaced orgs (``1bitllm/...``, ``OneBitLLM/...``)
|
||||
rather than being identifiable by repo name alone.
|
||||
"""
|
||||
if isinstance(model_name, bool):
|
||||
return False
|
||||
if not isinstance(model_name, str):
|
||||
return False
|
||||
if not model_name or "\x00" in model_name:
|
||||
return False
|
||||
# Check each path component so an org name like "1bitllm/foo" matches
|
||||
# while still rejecting unrelated substrings (e.g. an SFT model that
|
||||
# happens to embed "bitnet" inside a description path).
|
||||
for part in model_name.lower().split("/"):
|
||||
if any(part.startswith(prefix) for prefix in _BITNET_FAMILY_RE_PREFIXES):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def validate_bitnet_compat(*, task: str, backend: str, modality: str) -> None:
|
||||
"""Schema-time gate for ``quantization='bitnet_1.58'``.
|
||||
|
||||
Rejects:
|
||||
- non-string / bool args (defence-in-depth).
|
||||
- ``backend == 'mlx'`` — onebitllms is CUDA-only in v0.52.0.
|
||||
- ``modality != 'text'`` — vision/audio BitNet not modelled.
|
||||
- ``task`` outside {sft, pretrain, dpo} — BitNet wiring is text-LM
|
||||
training only this release.
|
||||
"""
|
||||
for name, value in (("task", task), ("backend", backend), ("modality", modality)):
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"{name} must not be bool, got {value!r}")
|
||||
if not isinstance(value, str) or not value:
|
||||
raise ValueError(f"{name} must be a non-empty string")
|
||||
if backend == "mlx":
|
||||
raise ValueError(
|
||||
"quantization='bitnet_1.58' is not supported on backend=mlx "
|
||||
"(onebitllms is CUDA-only). Use backend='transformers'."
|
||||
)
|
||||
if modality != "text":
|
||||
raise ValueError(
|
||||
f"quantization='bitnet_1.58' is wired for modality='text' only; "
|
||||
f"got modality={modality!r}"
|
||||
)
|
||||
if task not in ("sft", "pretrain", "dpo"):
|
||||
raise ValueError(
|
||||
f"quantization='bitnet_1.58' is only wired for "
|
||||
f"task in (sft, pretrain, dpo); got task={task!r}"
|
||||
)
|
||||
|
||||
|
||||
def validate_bitnet_export(format_name: object) -> str:
|
||||
"""Validate a BitNet export-format string. Returns canonical form."""
|
||||
if isinstance(format_name, bool):
|
||||
raise TypeError(
|
||||
f"bitnet export format must not be bool, got {format_name!r}"
|
||||
)
|
||||
if not isinstance(format_name, str):
|
||||
raise TypeError(
|
||||
f"bitnet export format must be str, "
|
||||
f"got {type(format_name).__name__}"
|
||||
)
|
||||
if not format_name:
|
||||
raise ValueError("bitnet export format must be non-empty")
|
||||
if "\x00" in format_name:
|
||||
raise ValueError(
|
||||
"bitnet export format must not contain null bytes"
|
||||
)
|
||||
canonical = format_name.lower()
|
||||
if canonical not in BITNET_EXPORT_FORMATS:
|
||||
supported = ", ".join(sorted(BITNET_EXPORT_FORMATS))
|
||||
raise ValueError(
|
||||
f"bitnet export format {format_name!r} not supported. "
|
||||
f"Supported: {supported}"
|
||||
)
|
||||
return canonical
|
||||
|
||||
|
||||
def build_bitnet_trainer() -> None:
|
||||
"""Live BitNet trainer factory — deferred to v0.52.1."""
|
||||
raise NotImplementedError(
|
||||
"BitNet 1.58-bit fine-tuning live wiring deferred to v0.52.1. "
|
||||
"Schema accepts quantization='bitnet_1.58' but no trainer integration "
|
||||
"is registered yet."
|
||||
)
|
||||
|
||||
|
||||
def export_bitnet_gguf() -> None:
|
||||
"""Live BitNet GGUF export — deferred to v0.52.1."""
|
||||
raise NotImplementedError(
|
||||
"BitNet / TQ1_0 GGUF export live wiring deferred to v0.52.1. "
|
||||
"Schema accepts --format bitnet / --format tq1_0 but no export "
|
||||
"pipeline is registered yet."
|
||||
)
|
||||
|
|
@ -0,0 +1,168 @@
|
|||
"""v0.52.0 Part B — Classifier / reranker / cross_encoder task helpers.
|
||||
|
||||
Three new task strings build on the existing v0.16.0 embedding trainer:
|
||||
|
||||
* ``classifier`` — sequence classification head (single-label / multi-label).
|
||||
* ``reranker`` — pointwise scoring head for retrieval reranking.
|
||||
* ``cross_encoder`` — paired-input scoring (e.g. MS-MARCO-style).
|
||||
|
||||
Schema-only release: validators here are reused by the SoupConfig
|
||||
cross-validator, while the live trainer wrappers ship in v0.52.1
|
||||
(mirrors v0.50.0 stub-then-live pattern).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from types import MappingProxyType
|
||||
from typing import Mapping
|
||||
|
||||
CLASSIFIER_TASKS: frozenset[str] = frozenset(
|
||||
{"classifier", "reranker", "cross_encoder"}
|
||||
)
|
||||
|
||||
_CLASSIFIER_KIND: frozenset[str] = frozenset({"single_label", "multi_label"})
|
||||
|
||||
_MAX_LABELS: int = 1024
|
||||
_MAX_LABEL_LEN: int = 128
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ClassifierKindSpec:
|
||||
"""Metadata for a classifier task. Frozen so callers cannot mutate."""
|
||||
|
||||
name: str
|
||||
description: str
|
||||
paired_input: bool
|
||||
live_wired: bool
|
||||
|
||||
|
||||
_CLASSIFIER_METADATA: Mapping[str, ClassifierKindSpec] = MappingProxyType({
|
||||
"classifier": ClassifierKindSpec(
|
||||
name="classifier",
|
||||
description="Sequence classification head (single/multi-label)",
|
||||
paired_input=False,
|
||||
live_wired=False,
|
||||
),
|
||||
"reranker": ClassifierKindSpec(
|
||||
name="reranker",
|
||||
description="Pointwise scoring head for retrieval reranking",
|
||||
paired_input=False,
|
||||
live_wired=False,
|
||||
),
|
||||
"cross_encoder": ClassifierKindSpec(
|
||||
name="cross_encoder",
|
||||
description="Paired-input scoring head (query/document)",
|
||||
paired_input=True,
|
||||
live_wired=False,
|
||||
),
|
||||
})
|
||||
|
||||
|
||||
def is_classifier_task(task: object) -> bool:
|
||||
"""Return True iff ``task`` is one of classifier/reranker/cross_encoder."""
|
||||
if isinstance(task, bool):
|
||||
return False
|
||||
if not isinstance(task, str):
|
||||
return False
|
||||
return task in CLASSIFIER_TASKS
|
||||
|
||||
|
||||
def get_classifier_spec(task: str) -> ClassifierKindSpec:
|
||||
"""Return the frozen :class:`ClassifierKindSpec` for ``task`` or raise."""
|
||||
if not is_classifier_task(task):
|
||||
supported = ", ".join(sorted(CLASSIFIER_TASKS))
|
||||
raise ValueError(
|
||||
f"task {task!r} is not a classifier task. Supported: {supported}"
|
||||
)
|
||||
return _CLASSIFIER_METADATA[task]
|
||||
|
||||
|
||||
def validate_num_labels(value: object) -> int:
|
||||
"""Validate a ``num_labels`` integer (1..1024). Rejects bool."""
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"num_labels must not be bool, got {value!r}")
|
||||
if not isinstance(value, int):
|
||||
raise TypeError(
|
||||
f"num_labels must be int, got {type(value).__name__}"
|
||||
)
|
||||
if value < 1:
|
||||
raise ValueError(f"num_labels must be >= 1, got {value}")
|
||||
if value > _MAX_LABELS:
|
||||
raise ValueError(
|
||||
f"num_labels must be <= {_MAX_LABELS}, got {value}"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def validate_label_names(value: object) -> list[str]:
|
||||
"""Validate an optional label-name list. Returns a defensive copy."""
|
||||
if not isinstance(value, list):
|
||||
raise TypeError(
|
||||
f"label_names must be a list, got {type(value).__name__}"
|
||||
)
|
||||
if len(value) > _MAX_LABELS:
|
||||
raise ValueError(
|
||||
f"label_names too long (max {_MAX_LABELS} entries)"
|
||||
)
|
||||
seen: set[str] = set()
|
||||
result: list[str] = []
|
||||
for entry in value:
|
||||
if isinstance(entry, bool):
|
||||
raise TypeError("label_names entries must not be bool")
|
||||
if not isinstance(entry, str):
|
||||
raise TypeError(
|
||||
f"label_names entries must be str, got {type(entry).__name__}"
|
||||
)
|
||||
if not entry:
|
||||
raise ValueError("label_names entries must be non-empty")
|
||||
if "\x00" in entry:
|
||||
raise ValueError("label_names entries must not contain null bytes")
|
||||
if len(entry) > _MAX_LABEL_LEN:
|
||||
raise ValueError(
|
||||
f"label_names entry too long (max {_MAX_LABEL_LEN} chars)"
|
||||
)
|
||||
if entry in seen:
|
||||
raise ValueError(f"label_names entries must be unique: {entry!r}")
|
||||
seen.add(entry)
|
||||
result.append(entry)
|
||||
return result
|
||||
|
||||
|
||||
def validate_classifier_compat(*, task: str, backend: str, modality: str) -> None:
|
||||
"""Schema-time gate for the three classifier tasks.
|
||||
|
||||
Rejects:
|
||||
- non-string / bool args (defence-in-depth).
|
||||
- non-classifier task.
|
||||
- ``backend == 'mlx'`` (no MLX classifier path in v0.52.0).
|
||||
- ``modality != 'text'`` (vision/audio classifier deferred).
|
||||
"""
|
||||
for name, value in (("task", task), ("backend", backend), ("modality", modality)):
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"{name} must not be bool, got {value!r}")
|
||||
if not isinstance(value, str) or not value:
|
||||
raise ValueError(f"{name} must be a non-empty string")
|
||||
if task not in CLASSIFIER_TASKS:
|
||||
supported = ", ".join(sorted(CLASSIFIER_TASKS))
|
||||
raise ValueError(
|
||||
f"validate_classifier_compat called with task={task!r}; "
|
||||
f"expected one of {supported}"
|
||||
)
|
||||
if backend == "mlx":
|
||||
raise ValueError(
|
||||
f"task={task!r} is not supported on backend=mlx in v0.52.0"
|
||||
)
|
||||
if modality != "text":
|
||||
raise ValueError(
|
||||
f"task={task!r} requires modality='text'; got modality={modality!r}"
|
||||
)
|
||||
|
||||
|
||||
def build_classifier_trainer() -> None:
|
||||
"""Live classifier trainer factory — deferred to v0.52.1."""
|
||||
raise NotImplementedError(
|
||||
"Classifier / reranker / cross_encoder trainer live wiring deferred "
|
||||
"to v0.52.1. Schema accepts the task but no trainer wrapper is "
|
||||
"registered yet."
|
||||
)
|
||||
|
|
@ -0,0 +1,200 @@
|
|||
"""v0.52.0 Part C — Knowledge Distillation schema helpers.
|
||||
|
||||
Schema-only support for ``task='distill'`` — teacher/student training.
|
||||
Four divergence options are recognised, mirroring axolotl's distillation
|
||||
plugin:
|
||||
|
||||
* ``kl`` (forward KL — student KL teacher, standard distillation)
|
||||
* ``forward_kl`` (alias for ``kl``)
|
||||
* ``reverse_kl`` (teacher KL student)
|
||||
* ``js`` (Jensen-Shannon, symmetric)
|
||||
|
||||
The live distillation trainer lands in v0.52.1; this module exposes pure
|
||||
validators so the schema gate can fail fast on misconfiguration.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from types import MappingProxyType
|
||||
from typing import Mapping
|
||||
|
||||
_DIVERGENCE_ALIASES: Mapping[str, str] = MappingProxyType({
|
||||
"kl": "forward_kl",
|
||||
"forward_kl": "forward_kl",
|
||||
"reverse_kl": "reverse_kl",
|
||||
"js": "js",
|
||||
})
|
||||
|
||||
# Public, derived from the alias map so adding a new alias updates both the
|
||||
# accepted-input set and the error message in lockstep.
|
||||
DIVERGENCES: frozenset[str] = frozenset(_DIVERGENCE_ALIASES)
|
||||
|
||||
_MAX_TEACHER_LEN: int = 512
|
||||
_MAX_DIVERGENCE_LEN: int = 16
|
||||
_MIN_TEMPERATURE: float = 0.05
|
||||
_MAX_TEMPERATURE: float = 100.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DivergenceSpec:
|
||||
"""Metadata for a divergence kernel. Frozen so callers cannot mutate."""
|
||||
|
||||
name: str
|
||||
description: str
|
||||
symmetric: bool
|
||||
live_wired: bool
|
||||
|
||||
|
||||
_DIVERGENCE_METADATA: Mapping[str, DivergenceSpec] = MappingProxyType({
|
||||
"forward_kl": DivergenceSpec(
|
||||
name="forward_kl",
|
||||
description="Forward KL (standard distillation)",
|
||||
symmetric=False,
|
||||
live_wired=False,
|
||||
),
|
||||
"reverse_kl": DivergenceSpec(
|
||||
name="reverse_kl",
|
||||
description="Reverse KL (mode-seeking)",
|
||||
symmetric=False,
|
||||
live_wired=False,
|
||||
),
|
||||
"js": DivergenceSpec(
|
||||
name="js",
|
||||
description="Jensen-Shannon (symmetric KL)",
|
||||
symmetric=True,
|
||||
live_wired=False,
|
||||
),
|
||||
})
|
||||
|
||||
|
||||
def validate_divergence(name: object) -> str:
|
||||
"""Validate a divergence name and return the canonical form.
|
||||
|
||||
Accepts ``kl`` as an alias for ``forward_kl``. Mirrors v0.41.0
|
||||
``validate_optimizer_name`` policy.
|
||||
"""
|
||||
if isinstance(name, bool):
|
||||
raise TypeError(f"distill_divergence must not be bool, got {name!r}")
|
||||
if not isinstance(name, str):
|
||||
raise TypeError(
|
||||
f"distill_divergence must be str, got {type(name).__name__}"
|
||||
)
|
||||
if not name:
|
||||
raise ValueError("distill_divergence must be non-empty")
|
||||
if "\x00" in name:
|
||||
raise ValueError("distill_divergence must not contain null bytes")
|
||||
if len(name) > _MAX_DIVERGENCE_LEN:
|
||||
raise ValueError(
|
||||
f"distill_divergence too long (max {_MAX_DIVERGENCE_LEN} chars)"
|
||||
)
|
||||
canonical = name.lower()
|
||||
if canonical not in _DIVERGENCE_ALIASES:
|
||||
supported = ", ".join(sorted(DIVERGENCES))
|
||||
raise ValueError(
|
||||
f"distill_divergence {name!r} not supported. Supported: {supported}"
|
||||
)
|
||||
return _DIVERGENCE_ALIASES[canonical]
|
||||
|
||||
|
||||
def get_divergence_spec(name: str) -> DivergenceSpec:
|
||||
"""Return the frozen :class:`DivergenceSpec` for ``name`` or raise."""
|
||||
canonical = validate_divergence(name)
|
||||
return _DIVERGENCE_METADATA[canonical]
|
||||
|
||||
|
||||
def validate_distill_temperature(value: object) -> float:
|
||||
"""Validate a distillation temperature scalar.
|
||||
|
||||
Bounds [0.05, 100.0]. Rejects bool, NaN, ±inf.
|
||||
"""
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(
|
||||
f"distill_temperature must not be bool, got {value!r}"
|
||||
)
|
||||
if not isinstance(value, (int, float)):
|
||||
raise TypeError(
|
||||
f"distill_temperature must be float, got {type(value).__name__}"
|
||||
)
|
||||
fval = float(value)
|
||||
if not math.isfinite(fval):
|
||||
raise ValueError(
|
||||
f"distill_temperature must be finite, got {value!r}"
|
||||
)
|
||||
if fval < _MIN_TEMPERATURE:
|
||||
raise ValueError(
|
||||
f"distill_temperature must be >= {_MIN_TEMPERATURE}, got {fval}"
|
||||
)
|
||||
if fval > _MAX_TEMPERATURE:
|
||||
raise ValueError(
|
||||
f"distill_temperature must be <= {_MAX_TEMPERATURE}, got {fval}"
|
||||
)
|
||||
return fval
|
||||
|
||||
|
||||
def validate_teacher_model(value: object) -> str:
|
||||
"""Validate a teacher model string (HF repo id or local path).
|
||||
|
||||
Mirrors the v0.40.5 ``reward_model`` field validator: null-byte
|
||||
rejection + 512-char cap.
|
||||
"""
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"teacher_model must not be bool, got {value!r}")
|
||||
if not isinstance(value, str):
|
||||
raise TypeError(
|
||||
f"teacher_model must be str, got {type(value).__name__}"
|
||||
)
|
||||
if not value:
|
||||
raise ValueError("teacher_model must be non-empty")
|
||||
if "\x00" in value:
|
||||
raise ValueError("teacher_model must not contain null bytes")
|
||||
if len(value) > _MAX_TEACHER_LEN:
|
||||
raise ValueError(
|
||||
f"teacher_model too long (max {_MAX_TEACHER_LEN} chars)"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def validate_distill_compat(
|
||||
*,
|
||||
task: str,
|
||||
backend: str,
|
||||
teacher_model: object,
|
||||
) -> None:
|
||||
"""Schema-time gate for ``task='distill'``.
|
||||
|
||||
Rejects:
|
||||
- non-distill task.
|
||||
- ``backend == 'mlx'`` (no MLX teacher-load path yet).
|
||||
- missing teacher_model — distillation is meaningless without one.
|
||||
"""
|
||||
for name, value in (("task", task), ("backend", backend)):
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"{name} must not be bool, got {value!r}")
|
||||
if not isinstance(value, str) or not value:
|
||||
raise ValueError(f"{name} must be a non-empty string")
|
||||
if task != "distill":
|
||||
raise ValueError(
|
||||
f"validate_distill_compat called with task={task!r} "
|
||||
"(expected 'distill')"
|
||||
)
|
||||
if backend == "mlx":
|
||||
raise ValueError(
|
||||
"task='distill' is not supported on backend=mlx in v0.52.0"
|
||||
)
|
||||
if teacher_model is None:
|
||||
raise ValueError(
|
||||
"task='distill' requires training.teacher_model to be set"
|
||||
)
|
||||
# Reuse the standard validator — null-byte / oversize / type check.
|
||||
validate_teacher_model(teacher_model)
|
||||
|
||||
|
||||
def build_distill_trainer() -> None:
|
||||
"""Live distillation trainer factory — deferred to v0.52.1."""
|
||||
raise NotImplementedError(
|
||||
"Distillation trainer (task='distill') live wiring deferred to "
|
||||
"v0.52.1. Schema accepts the value but no trainer wrapper is "
|
||||
"registered yet."
|
||||
)
|
||||
|
|
@ -0,0 +1,191 @@
|
|||
"""v0.52.0 Part E — Energy-Based FT (EBFT) + Generalized DPO (GDPO) helpers.
|
||||
|
||||
Schema-only release: each algorithm has a closed allowlist of variant names
|
||||
plus pure validators. Live loss kernels land in v0.52.1.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from types import MappingProxyType
|
||||
from typing import Mapping
|
||||
|
||||
# Closed allowlists.
|
||||
EBFT_VARIANTS: frozenset[str] = frozenset({"structured", "strided"})
|
||||
GDPO_VARIANTS: frozenset[str] = frozenset({"standard", "length_normalized", "margin"})
|
||||
|
||||
_MAX_VARIANT_LEN: int = 32
|
||||
|
||||
_MIN_EBFT_TEMP: float = 1e-4
|
||||
_MAX_EBFT_TEMP: float = 100.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EBFTSpec:
|
||||
"""Metadata for an EBFT variant. Frozen — immutable."""
|
||||
|
||||
name: str
|
||||
description: str
|
||||
live_wired: bool
|
||||
|
||||
|
||||
_EBFT_METADATA: Mapping[str, EBFTSpec] = MappingProxyType({
|
||||
"structured": EBFTSpec(
|
||||
name="structured",
|
||||
description="Structured Energy-Based FT (per-token energies)",
|
||||
live_wired=False,
|
||||
),
|
||||
"strided": EBFTSpec(
|
||||
name="strided",
|
||||
description="Strided Energy-Based FT (block-sampled energies)",
|
||||
live_wired=False,
|
||||
),
|
||||
})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GDPOSpec:
|
||||
"""Metadata for a GDPO variant. Frozen — immutable."""
|
||||
|
||||
name: str
|
||||
description: str
|
||||
live_wired: bool
|
||||
|
||||
|
||||
_GDPO_METADATA: Mapping[str, GDPOSpec] = MappingProxyType({
|
||||
"standard": GDPOSpec(
|
||||
name="standard",
|
||||
description="Standard GDPO (general preference objective)",
|
||||
live_wired=False,
|
||||
),
|
||||
"length_normalized": GDPOSpec(
|
||||
name="length_normalized",
|
||||
description="Length-normalized GDPO (SimPO-style normalisation)",
|
||||
live_wired=False,
|
||||
),
|
||||
"margin": GDPOSpec(
|
||||
name="margin",
|
||||
description="Margin-augmented GDPO (DPO + margin term)",
|
||||
live_wired=False,
|
||||
),
|
||||
})
|
||||
|
||||
|
||||
def _validate_variant(name: object, allowed: frozenset[str], label: str) -> str:
|
||||
"""Shared variant-name validator."""
|
||||
if isinstance(name, bool):
|
||||
raise TypeError(f"{label} must not be bool, got {name!r}")
|
||||
if not isinstance(name, str):
|
||||
raise TypeError(f"{label} must be str, got {type(name).__name__}")
|
||||
if not name:
|
||||
raise ValueError(f"{label} must be non-empty")
|
||||
if "\x00" in name:
|
||||
raise ValueError(f"{label} must not contain null bytes")
|
||||
if len(name) > _MAX_VARIANT_LEN:
|
||||
raise ValueError(
|
||||
f"{label} too long (max {_MAX_VARIANT_LEN} chars)"
|
||||
)
|
||||
canonical = name.lower()
|
||||
if canonical not in allowed:
|
||||
supported = ", ".join(sorted(allowed))
|
||||
raise ValueError(
|
||||
f"{label} {name!r} not supported. Supported: {supported}"
|
||||
)
|
||||
return canonical
|
||||
|
||||
|
||||
def validate_ebft_variant(name: object) -> str:
|
||||
"""Validate an EBFT variant and return the canonical form."""
|
||||
return _validate_variant(name, EBFT_VARIANTS, "ebft_variant")
|
||||
|
||||
|
||||
def validate_gdpo_variant(name: object) -> str:
|
||||
"""Validate a GDPO variant and return the canonical form."""
|
||||
return _validate_variant(name, GDPO_VARIANTS, "gdpo_variant")
|
||||
|
||||
|
||||
def get_ebft_spec(name: str) -> EBFTSpec:
|
||||
"""Return the frozen :class:`EBFTSpec` for ``name`` or raise."""
|
||||
return _EBFT_METADATA[validate_ebft_variant(name)]
|
||||
|
||||
|
||||
def get_gdpo_spec(name: str) -> GDPOSpec:
|
||||
"""Return the frozen :class:`GDPOSpec` for ``name`` or raise."""
|
||||
return _GDPO_METADATA[validate_gdpo_variant(name)]
|
||||
|
||||
|
||||
def validate_ebft_temperature(value: object) -> float:
|
||||
"""Validate an EBFT temperature scalar in [1e-4, 100]. Rejects bool/NaN."""
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"ebft_temperature must not be bool, got {value!r}")
|
||||
if not isinstance(value, (int, float)):
|
||||
raise TypeError(
|
||||
f"ebft_temperature must be float, got {type(value).__name__}"
|
||||
)
|
||||
fval = float(value)
|
||||
if not math.isfinite(fval):
|
||||
raise ValueError(
|
||||
f"ebft_temperature must be finite, got {value!r}"
|
||||
)
|
||||
if fval < _MIN_EBFT_TEMP:
|
||||
raise ValueError(
|
||||
f"ebft_temperature must be >= {_MIN_EBFT_TEMP}, got {fval}"
|
||||
)
|
||||
if fval > _MAX_EBFT_TEMP:
|
||||
raise ValueError(
|
||||
f"ebft_temperature must be <= {_MAX_EBFT_TEMP}, got {fval}"
|
||||
)
|
||||
return fval
|
||||
|
||||
|
||||
def _check_task_backend(task: object, backend: object) -> None:
|
||||
"""Shared bool/str guard for cross-compat helpers."""
|
||||
for name, value in (("task", task), ("backend", backend)):
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"{name} must not be bool, got {value!r}")
|
||||
if not isinstance(value, str) or not value:
|
||||
raise ValueError(f"{name} must be a non-empty string")
|
||||
|
||||
|
||||
def validate_ebft_compat(*, task: str, backend: str) -> None:
|
||||
"""Schema-time gate for ``ebft_variant`` — SFT-only, non-MLX."""
|
||||
_check_task_backend(task, backend)
|
||||
if backend == "mlx":
|
||||
raise ValueError(
|
||||
"ebft_variant is not supported on backend=mlx in v0.52.0"
|
||||
)
|
||||
if task != "sft":
|
||||
raise ValueError(
|
||||
f"ebft_variant requires task='sft'; got task={task!r}"
|
||||
)
|
||||
|
||||
|
||||
def validate_gdpo_compat(*, task: str, backend: str) -> None:
|
||||
"""Schema-time gate for ``gdpo_variant`` — DPO-family-only, non-MLX."""
|
||||
_check_task_backend(task, backend)
|
||||
if backend == "mlx":
|
||||
raise ValueError(
|
||||
"gdpo_variant is not supported on backend=mlx in v0.52.0"
|
||||
)
|
||||
if task not in ("dpo", "preference"):
|
||||
raise ValueError(
|
||||
f"gdpo_variant requires task in ('dpo', 'preference'); "
|
||||
f"got task={task!r}"
|
||||
)
|
||||
|
||||
|
||||
def apply_ebft_loss() -> None:
|
||||
"""Live EBFT loss kernel — deferred to v0.52.1."""
|
||||
raise NotImplementedError(
|
||||
"EBFT (Energy-Based FT) live loss kernel deferred to v0.52.1. "
|
||||
"Schema accepts the variant but no loss is wired yet."
|
||||
)
|
||||
|
||||
|
||||
def apply_gdpo_loss() -> None:
|
||||
"""Live GDPO loss kernel — deferred to v0.52.1."""
|
||||
raise NotImplementedError(
|
||||
"GDPO (Generalized DPO) live loss kernel deferred to v0.52.1. "
|
||||
"Schema accepts the variant but no loss is wired yet."
|
||||
)
|
||||
|
|
@ -0,0 +1,141 @@
|
|||
"""v0.52.0 Part F — MoE expert quantization + router-only training schema.
|
||||
|
||||
Two new TrainingConfig fields are introduced this release:
|
||||
|
||||
* ``moe_expert_quant: Optional[Literal["nf4", "int8_rowwise"]]`` — per-expert
|
||||
weight quantization for fused-MoE Linear blocks. Wraps axolotl's MoE
|
||||
expert quant path.
|
||||
* ``train_router_only: bool`` — freeze every expert + train only the
|
||||
gating router (unsloth MoE recipe). Useful for router calibration.
|
||||
|
||||
Schema-only this release; live wiring lands in v0.52.1.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from types import MappingProxyType
|
||||
from typing import Mapping
|
||||
|
||||
MOE_EXPERT_QUANT_FORMATS: frozenset[str] = frozenset({"nf4", "int8_rowwise"})
|
||||
|
||||
_MAX_QUANT_LEN: int = 32
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MoEExpertQuantSpec:
|
||||
"""Metadata for a MoE-expert quant format. Frozen."""
|
||||
|
||||
name: str
|
||||
description: str
|
||||
bits: int
|
||||
live_wired: bool
|
||||
|
||||
|
||||
_MOE_EXPERT_QUANT_METADATA: Mapping[str, MoEExpertQuantSpec] = MappingProxyType({
|
||||
"nf4": MoEExpertQuantSpec(
|
||||
name="nf4",
|
||||
description="NF4 per-expert (BNB 4-bit Normal-Float)",
|
||||
bits=4,
|
||||
live_wired=False,
|
||||
),
|
||||
"int8_rowwise": MoEExpertQuantSpec(
|
||||
name="int8_rowwise",
|
||||
description="INT8 row-wise per-expert (LLM.int8 row-wise)",
|
||||
bits=8,
|
||||
live_wired=False,
|
||||
),
|
||||
})
|
||||
|
||||
|
||||
def validate_moe_expert_quant(name: object) -> str:
|
||||
"""Validate a MoE expert-quant name. Returns canonical form."""
|
||||
if isinstance(name, bool):
|
||||
raise TypeError(f"moe_expert_quant must not be bool, got {name!r}")
|
||||
if not isinstance(name, str):
|
||||
raise TypeError(
|
||||
f"moe_expert_quant must be str, got {type(name).__name__}"
|
||||
)
|
||||
if not name:
|
||||
raise ValueError("moe_expert_quant must be non-empty")
|
||||
if "\x00" in name:
|
||||
raise ValueError("moe_expert_quant must not contain null bytes")
|
||||
if len(name) > _MAX_QUANT_LEN:
|
||||
raise ValueError(
|
||||
f"moe_expert_quant too long (max {_MAX_QUANT_LEN} chars)"
|
||||
)
|
||||
canonical = name.lower()
|
||||
if canonical not in MOE_EXPERT_QUANT_FORMATS:
|
||||
supported = ", ".join(sorted(MOE_EXPERT_QUANT_FORMATS))
|
||||
raise ValueError(
|
||||
f"moe_expert_quant {name!r} not supported. "
|
||||
f"Supported: {supported}"
|
||||
)
|
||||
return canonical
|
||||
|
||||
|
||||
def get_moe_expert_quant_spec(name: str) -> MoEExpertQuantSpec:
|
||||
"""Return the frozen spec for ``name`` or raise."""
|
||||
canonical = validate_moe_expert_quant(name)
|
||||
return _MOE_EXPERT_QUANT_METADATA[canonical]
|
||||
|
||||
|
||||
def validate_moe_expert_quant_compat(*, backend: str, moe_lora: bool) -> None:
|
||||
"""Schema-time gate for ``moe_expert_quant``.
|
||||
|
||||
Rejects:
|
||||
- non-string ``backend`` / non-bool ``moe_lora`` (defence-in-depth).
|
||||
- ``backend == 'mlx'`` (no MLX MoE expert quant).
|
||||
- ``moe_lora == False`` (MoE expert quant only meaningful when training
|
||||
LoRA adapters that sit on top of fused-MoE experts — otherwise the
|
||||
operator is asking for full-precision MoE training with quantized
|
||||
experts, which silently no-ops).
|
||||
"""
|
||||
if isinstance(backend, bool):
|
||||
raise TypeError(f"backend must not be bool, got {backend!r}")
|
||||
if not isinstance(backend, str) or not backend:
|
||||
raise ValueError("backend must be a non-empty string")
|
||||
if not isinstance(moe_lora, bool):
|
||||
raise TypeError(f"moe_lora must be bool, got {type(moe_lora).__name__}")
|
||||
if backend == "mlx":
|
||||
raise ValueError(
|
||||
"moe_expert_quant is not supported on backend=mlx in v0.52.0"
|
||||
)
|
||||
if not moe_lora:
|
||||
raise ValueError(
|
||||
"moe_expert_quant requires moe_lora=true "
|
||||
"(per-expert quant is only meaningful with MoE-aware LoRA wiring)"
|
||||
)
|
||||
|
||||
|
||||
def validate_train_router_only_compat(*, backend: str, moe_lora: bool) -> None:
|
||||
"""Schema-time gate for ``train_router_only=True``.
|
||||
|
||||
Requires ``moe_lora=true`` (without it, every expert would still be
|
||||
trained and the flag would silently no-op). Defence-in-depth bool /
|
||||
str rejection on the args.
|
||||
"""
|
||||
if isinstance(backend, bool):
|
||||
raise TypeError(f"backend must not be bool, got {backend!r}")
|
||||
if not isinstance(backend, str) or not backend:
|
||||
raise ValueError("backend must be a non-empty string")
|
||||
if not isinstance(moe_lora, bool):
|
||||
raise TypeError(f"moe_lora must be bool, got {type(moe_lora).__name__}")
|
||||
if backend == "mlx":
|
||||
raise ValueError(
|
||||
"train_router_only is not supported on backend=mlx in v0.52.0"
|
||||
)
|
||||
if not moe_lora:
|
||||
raise ValueError(
|
||||
"train_router_only requires moe_lora=true "
|
||||
"(router-only training freezes the experts, which is only "
|
||||
"meaningful with MoE-aware LoRA wiring)"
|
||||
)
|
||||
|
||||
|
||||
def apply_moe_expert_quant() -> None:
|
||||
"""Live MoE expert-quant wiring — deferred to v0.52.1."""
|
||||
raise NotImplementedError(
|
||||
"MoE expert quantization live wiring deferred to v0.52.1. "
|
||||
"Schema accepts the format but no per-expert quant path is wired."
|
||||
)
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
"""v0.52.0 Part G — gpt-oss reasoning_effort schema helper.
|
||||
|
||||
Schema-only support for ``training.reasoning_effort: Literal["low","medium","high"]``
|
||||
mirroring the unsloth gpt-oss training recipe. Routes through the prompt
|
||||
prefix at training time; live formatter wiring lands in v0.52.1.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
REASONING_EFFORT_LEVELS: frozenset[str] = frozenset({"low", "medium", "high"})
|
||||
|
||||
_MAX_REASONING_EFFORT_LEN: int = 16
|
||||
|
||||
|
||||
def validate_reasoning_effort(value: object) -> str:
|
||||
"""Validate a reasoning_effort string and return the canonical form."""
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"reasoning_effort must not be bool, got {value!r}")
|
||||
if not isinstance(value, str):
|
||||
raise TypeError(
|
||||
f"reasoning_effort must be str, got {type(value).__name__}"
|
||||
)
|
||||
if not value:
|
||||
raise ValueError("reasoning_effort must be non-empty")
|
||||
if "\x00" in value:
|
||||
raise ValueError("reasoning_effort must not contain null bytes")
|
||||
if len(value) > _MAX_REASONING_EFFORT_LEN:
|
||||
raise ValueError(
|
||||
f"reasoning_effort too long (max {_MAX_REASONING_EFFORT_LEN} chars)"
|
||||
)
|
||||
canonical = value.lower()
|
||||
if canonical not in REASONING_EFFORT_LEVELS:
|
||||
supported = ", ".join(sorted(REASONING_EFFORT_LEVELS))
|
||||
raise ValueError(
|
||||
f"reasoning_effort {value!r} not supported. Supported: {supported}"
|
||||
)
|
||||
return canonical
|
||||
|
|
@ -0,0 +1,214 @@
|
|||
"""v0.52.0 Part A — TTS (text-to-speech) fine-tuning schema helpers.
|
||||
|
||||
Schema-only support for ``task='tts'`` paired with ``modality='audio_out'``.
|
||||
Five upstream TTS model families are recognised: orpheus / sesame_csm /
|
||||
llasa / spark / oute. Each has a stable name string + per-family ``emotion``
|
||||
allowlist so trainer wiring (deferred to v0.52.1) can route correctly.
|
||||
|
||||
Mirrors v0.50.0 stub-then-live pattern: this module exposes pure validators
|
||||
and a frozen ``TTSFamilySpec`` dataclass; the live ``TTSTrainerWrapper``
|
||||
lands in v0.52.1.
|
||||
|
||||
Security:
|
||||
- Pure schema-time validation; no filesystem touch.
|
||||
- All validators raise ``ValueError`` / ``TypeError`` with actionable
|
||||
messages.
|
||||
- Bool rejected before int / str checks (project bool-as-int policy —
|
||||
matches v0.30.0 ``Candidate``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from types import MappingProxyType
|
||||
from typing import Mapping
|
||||
|
||||
# Closed allowlist — wrapped via MappingProxyType so callers cannot mutate
|
||||
# the registry at runtime (mirrors v0.36.0 ``_REGISTRY`` / v0.51.0 ``hubs``
|
||||
# policy).
|
||||
SUPPORTED_TTS_FAMILIES: frozenset[str] = frozenset(
|
||||
{"orpheus", "sesame_csm", "llasa", "spark", "oute"}
|
||||
)
|
||||
|
||||
_MAX_TTS_FAMILY_LEN: int = 32
|
||||
_MAX_EMOTION_LEN: int = 32
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TTSFamilySpec:
|
||||
"""Metadata for a TTS family. Frozen so callers cannot mutate."""
|
||||
|
||||
name: str
|
||||
description: str
|
||||
supports_emotion: bool
|
||||
live_wired: bool
|
||||
|
||||
|
||||
_TTS_FAMILY_METADATA: Mapping[str, TTSFamilySpec] = MappingProxyType({
|
||||
"orpheus": TTSFamilySpec(
|
||||
name="orpheus",
|
||||
description="Orpheus emotional TTS (canopylabs)",
|
||||
supports_emotion=True,
|
||||
live_wired=False,
|
||||
),
|
||||
"sesame_csm": TTSFamilySpec(
|
||||
name="sesame_csm",
|
||||
description="Sesame CSM conversational speech",
|
||||
supports_emotion=False,
|
||||
live_wired=False,
|
||||
),
|
||||
"llasa": TTSFamilySpec(
|
||||
name="llasa",
|
||||
description="Llasa-TTS (HKUSTAudio)",
|
||||
supports_emotion=False,
|
||||
live_wired=False,
|
||||
),
|
||||
"spark": TTSFamilySpec(
|
||||
name="spark",
|
||||
description="Spark-TTS (SparkAudio)",
|
||||
supports_emotion=False,
|
||||
live_wired=False,
|
||||
),
|
||||
"oute": TTSFamilySpec(
|
||||
name="oute",
|
||||
description="Oute-TTS (outeai)",
|
||||
supports_emotion=True,
|
||||
live_wired=False,
|
||||
),
|
||||
})
|
||||
|
||||
# Per-family emotion allowlists. Orpheus + Oute support emotion conditioning;
|
||||
# the others ignore the tag. Closed allowlist keeps trainer dispatch
|
||||
# deterministic.
|
||||
ORPHEUS_EMOTIONS: frozenset[str] = frozenset({
|
||||
"neutral", "happy", "sad", "angry", "excited", "calm", "whisper", "laugh",
|
||||
})
|
||||
|
||||
# Oute supports a tighter set focused on prosody / register.
|
||||
OUTE_EMOTIONS: frozenset[str] = frozenset({
|
||||
"neutral", "happy", "sad", "angry", "calm", "excited",
|
||||
})
|
||||
|
||||
_FAMILY_EMOTIONS: Mapping[str, frozenset[str]] = MappingProxyType({
|
||||
"orpheus": ORPHEUS_EMOTIONS,
|
||||
"oute": OUTE_EMOTIONS,
|
||||
})
|
||||
|
||||
|
||||
def validate_tts_family(name: object) -> str:
|
||||
"""Validate a TTS family name and return the canonical (lowercase) form."""
|
||||
if isinstance(name, bool):
|
||||
raise TypeError(f"tts_family must not be bool, got {name!r}")
|
||||
if not isinstance(name, str):
|
||||
raise TypeError(
|
||||
f"tts_family must be str, got {type(name).__name__}"
|
||||
)
|
||||
if not name:
|
||||
raise ValueError("tts_family must be non-empty")
|
||||
if "\x00" in name:
|
||||
raise ValueError("tts_family must not contain null bytes")
|
||||
if len(name) > _MAX_TTS_FAMILY_LEN:
|
||||
raise ValueError(
|
||||
f"tts_family too long (max {_MAX_TTS_FAMILY_LEN} chars)"
|
||||
)
|
||||
canonical = name.lower()
|
||||
if canonical not in SUPPORTED_TTS_FAMILIES:
|
||||
supported = ", ".join(sorted(SUPPORTED_TTS_FAMILIES))
|
||||
raise ValueError(
|
||||
f"tts_family {name!r} not supported. Supported: {supported}"
|
||||
)
|
||||
return canonical
|
||||
|
||||
|
||||
def get_tts_family_spec(name: str) -> TTSFamilySpec:
|
||||
"""Return the frozen :class:`TTSFamilySpec` for ``name`` or raise."""
|
||||
canonical = validate_tts_family(name)
|
||||
return _TTS_FAMILY_METADATA[canonical]
|
||||
|
||||
|
||||
def family_supports_emotion(name: str) -> bool:
|
||||
"""Whether the named family supports an ``emotion`` tag."""
|
||||
canonical = validate_tts_family(name)
|
||||
return _TTS_FAMILY_METADATA[canonical].supports_emotion
|
||||
|
||||
|
||||
def validate_emotion_tag(emotion: object, *, family: str) -> str:
|
||||
"""Validate an ``emotion`` tag for ``family`` (currently Orpheus only).
|
||||
|
||||
Raises if the family does not support emotion conditioning or the tag
|
||||
is not in the per-family allowlist. ``family`` is canonicalised first
|
||||
so callers don't have to.
|
||||
"""
|
||||
canonical_family = validate_tts_family(family)
|
||||
if isinstance(emotion, bool):
|
||||
raise TypeError(f"emotion must not be bool, got {emotion!r}")
|
||||
if not isinstance(emotion, str):
|
||||
raise TypeError(
|
||||
f"emotion must be str, got {type(emotion).__name__}"
|
||||
)
|
||||
if not emotion:
|
||||
raise ValueError("emotion must be non-empty")
|
||||
if "\x00" in emotion:
|
||||
raise ValueError("emotion must not contain null bytes")
|
||||
if len(emotion) > _MAX_EMOTION_LEN:
|
||||
raise ValueError(
|
||||
f"emotion too long (max {_MAX_EMOTION_LEN} chars)"
|
||||
)
|
||||
spec = _TTS_FAMILY_METADATA[canonical_family]
|
||||
if not spec.supports_emotion:
|
||||
raise ValueError(
|
||||
f"tts_family={canonical_family!r} does not support emotion "
|
||||
"conditioning"
|
||||
)
|
||||
canonical = emotion.lower()
|
||||
# Per-family allowlist — data-driven so future emotion-supporting
|
||||
# families cannot silently bypass the check.
|
||||
family_allowlist = _FAMILY_EMOTIONS.get(canonical_family)
|
||||
if family_allowlist is not None and canonical not in family_allowlist:
|
||||
allowed = ", ".join(sorted(family_allowlist))
|
||||
raise ValueError(
|
||||
f"emotion {emotion!r} not in {canonical_family} allowlist. "
|
||||
f"Allowed: {allowed}"
|
||||
)
|
||||
return canonical
|
||||
|
||||
|
||||
def validate_tts_compat(*, task: str, modality: str, backend: str) -> None:
|
||||
"""Schema-time gate for ``task='tts'``.
|
||||
|
||||
Rejects:
|
||||
- non-string / bool args (defence-in-depth — sister-function bool
|
||||
guards align with v0.30.0 ``Candidate`` policy).
|
||||
- non-TTS task (intended for ``task == 'tts'``).
|
||||
- ``modality != 'audio_out'`` (TTS is audio-output by definition).
|
||||
- ``backend == 'mlx'`` (no MLX TTS path in v0.52.0).
|
||||
"""
|
||||
for name, value in (("task", task), ("modality", modality), ("backend", backend)):
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"{name} must not be bool, got {value!r}")
|
||||
if not isinstance(value, str) or not value:
|
||||
raise ValueError(f"{name} must be a non-empty string")
|
||||
if task != "tts":
|
||||
raise ValueError(
|
||||
f"validate_tts_compat called with task={task!r} (expected 'tts')"
|
||||
)
|
||||
if modality != "audio_out":
|
||||
raise ValueError(
|
||||
f"task='tts' requires modality='audio_out'; got modality={modality!r}"
|
||||
)
|
||||
if backend == "mlx":
|
||||
raise ValueError(
|
||||
"task='tts' is not supported on backend=mlx in v0.52.0"
|
||||
)
|
||||
|
||||
|
||||
def build_tts_trainer() -> None:
|
||||
"""Live TTS trainer factory — deferred to v0.52.1.
|
||||
|
||||
Mirrors v0.50.0 ``build_prm_trainer`` / v0.49.0 ``apply_longlora_forward_override``
|
||||
stub-then-live pattern.
|
||||
"""
|
||||
raise NotImplementedError(
|
||||
"TTS trainer (task='tts') live wiring deferred to v0.52.1. "
|
||||
"Schema accepts the value but no trainer wrapper is registered yet."
|
||||
)
|
||||
|
|
@ -39,12 +39,12 @@ class TestExportFormatsExtended:
|
|||
assert "gptq" in SUPPORTED_FORMATS
|
||||
|
||||
def test_format_count(self):
|
||||
"""Should support exactly 5 export formats."""
|
||||
assert len(SUPPORTED_FORMATS) == 5
|
||||
"""v0.52.0 — 5 live formats + 2 BitNet stubs (bitnet, tq1_0)."""
|
||||
assert len(SUPPORTED_FORMATS) == 7
|
||||
|
||||
def test_all_formats_present(self):
|
||||
"""All five formats should be present."""
|
||||
expected = {"gguf", "onnx", "tensorrt", "awq", "gptq"}
|
||||
"""All seven formats should be present (v0.52.0 + bitnet/tq1_0 stubs)."""
|
||||
expected = {"gguf", "onnx", "tensorrt", "awq", "gptq", "bitnet", "tq1_0"}
|
||||
assert set(SUPPORTED_FORMATS) == expected
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -21,8 +21,10 @@ class TestExportFormats:
|
|||
assert "tensorrt" in SUPPORTED_FORMATS
|
||||
|
||||
def test_format_count(self):
|
||||
"""Should support exactly 5 export formats (gguf, onnx, tensorrt, awq, gptq)."""
|
||||
assert len(SUPPORTED_FORMATS) == 5
|
||||
"""v0.52.0 — 5 live formats (gguf/onnx/tensorrt/awq/gptq) + 2 BitNet stubs."""
|
||||
assert len(SUPPORTED_FORMATS) == 7
|
||||
assert "bitnet" in SUPPORTED_FORMATS
|
||||
assert "tq1_0" in SUPPORTED_FORMATS
|
||||
|
||||
|
||||
# ─── ONNX Export CLI Tests ──────────────────────────────────────────────
|
||||
|
|
|
|||
|
|
@ -260,17 +260,18 @@ class TestV025NewRecipes:
|
|||
assert cfg.base == recipe.model
|
||||
assert cfg.task == recipe.task
|
||||
|
||||
def test_catalog_size_is_106(self):
|
||||
def test_catalog_size_is_112(self):
|
||||
"""Total catalog size — grew with each release.
|
||||
|
||||
v0.25.0 shipped 43 recipes (29 + 9 Part A + 2 Part B tools + 3 Part E MLX).
|
||||
v0.27.0 added 3 multi-GPU recipes -> 46.
|
||||
v0.31.0 added 34 (vision/audio/reasoning/edge/domain/multimodal) -> 80.
|
||||
v0.51.0 added 26 (model catalog expansion) -> 106.
|
||||
v0.52.0 added 6 (5 TTS + Falcon-E BitNet) -> 112.
|
||||
"""
|
||||
from soup_cli.recipes.catalog import RECIPES
|
||||
|
||||
assert len(RECIPES) == 106
|
||||
assert len(RECIPES) == 112
|
||||
|
||||
def test_new_recipes_searchable(self):
|
||||
"""Search returns the new recipes via keyword/task filter."""
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue