mirror of https://github.com/razor-ai/soup.git
feat(lora): v0.39.0 — LoRA Quality (PiSSA + ReLoRA + per-pattern rank + surgical patches + templates registry)
Five PEFT-surface improvements that LlamaFactory and Axolotl maintain: - LoraConfig.init_strategy Literal["random","pissa","olora"]; PiSSA SVD init via PEFT init_lora_weights="pissa". Back-compat: use_olora=True aligns to init_strategy="olora" via dict-copy model_validator(mode="before"); explicit conflict (use_olora=True + init_strategy="pissa"/"random") rejected. Mutual-exclusion vs DoRA / VeRA. - ReLoRA callback (utils/relora.py): frozen ReLoRAPolicy with bounds-checked steps [1, 1e7] / warmup_ratio [0,1] / prune_ratio (0,1) (strict — prevents zero-everything footgun); magnitude_prune_tensor (in-place torch.kthvalue, rejects non-Tensor / single-element short-circuit); duck-typed ReLoRACallback (no transformers import at module load). TrainingConfig fields relora_steps / relora_warmup_ratio / relora_reset_optimizer / relora_prune_ratio. SoupConfig _validate_relora_supported_tasks gates to task=sft + transformers backend with distinct MLX-backend error message; multi-trainer expansion deferred to v0.39.1 (mirrors v0.27.0 MII / v0.37.0 multipack / v0.38.0 quant menu stub-then-live pattern). - LoraConfig.rank_pattern / alpha_pattern Optional[Dict[str,int]]; field validator caps at 256 keys × value (0, 1024], rejects bool / null-byte / empty key. Cross-validator rejects with use_vera=True (VeRA shares one rank). peft_builder propagates into LoraConfig init_kwargs. - utils/peft_patches.py: is_gemma4_model uses regex word boundary (?:^|[^a-z0-9])gemma-?4(?:[^a-z0-9]|$) so "ungemma4ed" no longer matches. apply_gemma4_clippable_patch swaps ClippableLinear → nn.Linear by class name (weight-copy fallback logs at DEBUG). strip_lora_dropout_for_3d_experts zeroes lora_dropout.p on 3-D weights (handles ModuleDict variant for PEFT >=0.10). apply_surgical_patches orchestrator validates model_name. Wired into sft.py _setup_transformers with is_gemma4_model gate before the pre-LoRA swap; post-LoRA 3-D dropout strip runs unconditionally (architecture-detected internally). - 16 inline templates migrated to soup_cli/templates/*.yaml + manifest.json + load_template loader (path-traversal-rejecting name validator; os.path.realpath + commonpath containment so a tampered manifest cannot read files outside the package directory; 256 KB file-size cap with inline fallback). Inline TEMPLATES kept with deprecation comment (planned removal v0.41.0+); test_templates_yaml asserts byte-equality of all 16 inline ↔ YAML pairs to prevent silent drift. Net +164 tests (4374 → 4538). All 5 review-agent waves clean before tag. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alpamys
parent
6361f7cca4
commit
e6a9c087c3
|
|
@ -107,10 +107,11 @@ soup_cli/
|
|||
cans/ - Shareable .can artifact format + run/publish orchestrator (v0.26.0 + v0.33.0)
|
||||
data/traces/ - Trace-to-Preference harvester (v0.26.0)
|
||||
data/collators.py - CrossDocCollator for sample packing (v0.33.0)
|
||||
utils/ - GPU, errors, MoE, GaLore, QAT, Unsloth, vLLM, SGLang, Liger, FlashAttn, FSDP, Ring Attention, long-context, quality, curriculum, freeze, dataset-registry, mlx, peft_builder, paths, topology, launcher, mii, pipeline, cut_ce, fp8, gradient_ckpt, kernel_picker, cross_doc_attn, activation_offload, hf, spec_pairing, structured_output, metrics, tracing, auto_quant, lr_finder, grad_accum, mixed_precision, warmup, spike_recovery, convergence, v028_features, multipack_sampler, multipack, neat_packing, jinja_analyzer, quant_menu
|
||||
utils/ - GPU, errors, MoE, GaLore, QAT, Unsloth, vLLM, SGLang, Liger, FlashAttn, FSDP, Ring Attention, long-context, quality, curriculum, freeze, dataset-registry, mlx, peft_builder, paths, topology, launcher, mii, pipeline, cut_ce, fp8, gradient_ckpt, kernel_picker, cross_doc_attn, activation_offload, hf, spec_pairing, structured_output, metrics, tracing, auto_quant, lr_finder, grad_accum, mixed_precision, warmup, spike_recovery, convergence, v028_features, multipack_sampler, multipack, neat_packing, jinja_analyzer, quant_menu, relora, peft_patches
|
||||
templates/ - 16 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0)
|
||||
ui/ - Web UI (FastAPI + HTML/JS SPA)
|
||||
|
||||
tests/ - Test suite (122 files, 4435 tests)
|
||||
tests/ - Test suite (132 files, 4538 tests)
|
||||
examples/ - Real-world config examples and datasets
|
||||
```
|
||||
|
||||
|
|
|
|||
62
README.md
62
README.md
|
|
@ -40,16 +40,14 @@ soup train
|
|||
|
||||
Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
|
||||
|
||||
**v0.38.0 — Quant Menu**: 7 new train-time quantization formats. Train LoRA on top of any pre-quantized base model — close the width gap with LlamaFactory.
|
||||
**v0.39.0 — LoRA Quality**: PEFT surface improvements that LlamaFactory and Axolotl maintain. Faster init, periodic adapter refresh, per-module rank, surgical architecture patches, and a cleaner template registry.
|
||||
|
||||
- **GPTQ / AWQ / HQQ / AQLM / EETQ / MXFP4 / FP8** — set `training.quantization` to one of `gptq` / `awq` / `hqq:1bit`..`hqq:8bit` / `aqlm` / `eetq` / `mxfp4` / `fp8`. Loaded as `quantization_config` on the underlying `from_pretrained` call; LoRA trains on top.
|
||||
- **HQQ wide bit range** — 1, 2, 3, 4, 5, 6, or 8 bits via `hqq:Nbit` syntax. `hqq:7bit` is intentionally rejected (HQQ does not support it).
|
||||
- **Compatibility matrix enforced at startup** — `check_quant_distributed_compat` hard-fails HQQ/EETQ/AQLM × {FSDP, ZeRO-3} (sourced from LlamaFactory `quantization.py:199/211`). BNB-4bit + FSDP without `bnb_4bit_quant_storage` emits a yellow warning before the silent 2-3x perf cliff hits.
|
||||
- **`bnb_4bit_quant_storage` field** — set to `bfloat16` / `float16` for the canonical FSDP+QLoRA combo ("crucial for fsdp+qlora" — LlamaFactory `quantization.py:178`). Schema rejects the setting on any non-BNB-4bit format.
|
||||
- **`docs/QUANTIZATION.md`** — full compatibility matrix (format × {DDP / FSDP / ZeRO-1 / ZeRO-2 / ZeRO-3}) with per-format optional dep + use case.
|
||||
- **Pre-quantized + QAT mutually exclusive** — every pre-quantized format combined with `quantization_aware` (int8 QAT or `'fp8'`) is rejected at config-load with the actual reason.
|
||||
- **v0.38.0 scope** — wired into the SFT trainer + transformers backend. Multi-trainer expansion (DPO/GRPO/KTO/...) tracked for v0.38.1, mirroring v0.27.0 MII / v0.37.0 multipack stub-then-live pattern.
|
||||
- **Net +61 tests** (4374 → 4435) covering schema acceptance, builder shape, checkpoint validators, cross-validators, the 7-row × 5-column compat matrix, modality/task/backend gate coverage, adversarial inputs (oversized HQQ suffix, bool group_size, hyphenated DeepSpeed presets), and the central loader entry point.
|
||||
- **PiSSA init** — set `training.lora.init_strategy: pissa` for SVD-initialized LoRA pairs. Faster early convergence vs random init at the cost of one extra SVD pass on the first epoch. `init_strategy: olora` is also accepted (equivalent to legacy `use_olora=True`, which auto-aligns for back-compat).
|
||||
- **ReLoRA callback** — set `training.relora_steps: 500` to magnitude-prune LoRA adapter weights every 500 steps and clear optimizer state. Useful for very long training runs where the LoRA capacity saturates. Bounds: `relora_warmup_ratio` [0,1], `relora_prune_ratio` (0,1), `relora_reset_optimizer: true`. Wired into the SFT trainer; multi-trainer expansion deferred to v0.39.1.
|
||||
- **Per-pattern LoRA rank** — `training.lora.rank_pattern: {q_proj: 8, v_proj: 16}` and `alpha_pattern` give different ranks per target module pattern. Useful in MoE configs where expert FFNs need lower rank than attention. Caps: 256 keys × value 1024.
|
||||
- **Surgical PEFT patches** — Gemma 4 `ClippableLinear` is auto-swapped to plain `nn.Linear` so PEFT's matcher recognises it; fused-MoE 3-D expert weights have `lora_dropout` zeroed to dodge `ParamWrapper` crashes. Both are gated by architecture detection (regex word-boundary on the model name) and never run on unrelated models.
|
||||
- **Template registry** — the 16 built-in templates now live as `soup_cli/templates/*.yaml` with a `manifest.json` index. `soup init --template <name>` reads the YAML; the inline copies in `schema.py` stay as a back-compat fallback (deprecation pointing at v0.41.0+).
|
||||
- **Net +164 tests** (4374 → 4538) across PiSSA mutual-exclusion, ReLoRA frozen-policy + real-optimizer-state reset, rank_pattern bounds + null-byte rejection, regex word-boundary Gemma 4 detection, and template-registry path-traversal + manifest-tampering containment.
|
||||
|
||||
## Why Soup?
|
||||
|
||||
|
|
@ -632,6 +630,52 @@ training:
|
|||
|
||||
Replaces the static memory formula with a real try-halve-then-double-to-ceiling loop. Picked size is cached at `~/.soup/batch_cache.json` keyed on `(model, max_length, quantization, lora_r, gpu_name, gpu_memory_gb)` so repeat runs short-circuit.
|
||||
|
||||
## LoRA Quality — PiSSA, ReLoRA, Per-Pattern Rank, Surgical Patches
|
||||
|
||||
Five PEFT-surface improvements that LlamaFactory and Axolotl maintain:
|
||||
|
||||
```yaml
|
||||
training:
|
||||
lora:
|
||||
init_strategy: pissa # 'random' (default), 'pissa', 'olora'
|
||||
rank_pattern: # per-target-module rank override
|
||||
q_proj: 8
|
||||
v_proj: 16
|
||||
alpha_pattern: # per-target-module alpha override
|
||||
q_proj: 16
|
||||
relora_steps: 500 # magnitude-prune LoRA every 500 steps
|
||||
relora_warmup_ratio: 0.1 # skip first 10% of training
|
||||
relora_prune_ratio: 0.9 # zero out smallest 90% by magnitude
|
||||
relora_reset_optimizer: true # clear optimizer state on each fire
|
||||
```
|
||||
|
||||
**PiSSA** initializes the LoRA pair from the SVD of the base weight, giving faster
|
||||
early convergence than random init at the cost of one extra SVD pass on the first
|
||||
epoch. `init_strategy: olora` is also accepted; setting the legacy `use_olora: true`
|
||||
auto-aligns for back-compat.
|
||||
|
||||
**ReLoRA** fires every N global steps, magnitude-prunes the LoRA adapter weights
|
||||
(keeping the top `1 - relora_prune_ratio` by absolute value), and optionally clears
|
||||
optimizer state for the pruned parameters so momentum doesn't fight the new sparse
|
||||
weights. Useful for very long training runs where the LoRA capacity saturates.
|
||||
|
||||
**Per-pattern rank/alpha** map module name patterns to integer ranks. Useful in MoE
|
||||
configs where expert FFNs need lower rank than attention. Caps: 256 keys × value 1024.
|
||||
|
||||
**Surgical patches** (Gemma 4 `ClippableLinear` swap, fused-MoE 3-D expert
|
||||
`lora_dropout` strip) auto-fire when the model name and architecture match. Both are
|
||||
gated and silent on unrelated models.
|
||||
|
||||
**Template registry** — the 16 built-in templates now live as
|
||||
`soup_cli/templates/*.yaml` with a `manifest.json` index. `soup init --template <name>`
|
||||
reads the YAML; the inline copies in `schema.py` stay as a back-compat fallback,
|
||||
deprecated in favour of the YAML registry.
|
||||
|
||||
**v0.39.0 scope** — wired into the SFT trainer + transformers backend.
|
||||
Multi-trainer expansion of ReLoRA (DPO/GRPO/KTO/...) is tracked for v0.39.1
|
||||
(mirrors v0.27.0 MII / v0.37.0 multipack / v0.38.0 quant menu stub-then-live
|
||||
pattern). MLX backend gets a distinct error message.
|
||||
|
||||
## DPO Training
|
||||
|
||||
Train with preference data using Direct Preference Optimization:
|
||||
|
|
|
|||
|
|
@ -9,9 +9,9 @@ We provide security updates for the following versions:
|
|||
- **Versions older than 3 minor versions:** No support
|
||||
|
||||
Example:
|
||||
- v0.38.0-0.38.x -- Full support (latest)
|
||||
- v0.37.0-0.37.x -- Bug-fix support only
|
||||
- v0.36.x and below -- No support
|
||||
- v0.39.0-0.39.x -- Full support (latest)
|
||||
- v0.38.0-0.38.x -- Bug-fix support only
|
||||
- v0.37.x and below -- No support
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
|
|
@ -142,6 +142,7 @@ No known critical vulnerabilities in current releases.
|
|||
- **v0.32.0 — Training Stability & Auto-Tuning**: `--find-lr-output` containment via shared `utils/paths.is_under_cwd` (prevents writes outside cwd); `save_lr_finder_report` rejects NaN / Infinity floats in `lrs` / `losses` and serialises with `allow_nan=False` (keeps the report parser-safe); `compute_lr_schedule` rejects non-positive `start_lr`, inverted ranges, and `num_steps` outside `[2, 10_000]`; `pick_mixed_precision` rejects empty / null-byte / >200-char model names and resolves multi-version quirks (`qwen2.5` vs `qwen2`, `phi-3.5` vs `phi-3`) by longest-substring-first iteration so an added family can never accidentally make a more-specific entry dead code; `compute_warmup_steps` clamps to `[10, 1000]` with a `ratio==0.0` short-circuit matching HF Trainer's "no warmup" convention; `SpikeRecoveryStrategy` is `@dataclass(frozen=True)` (post-construction mutation cannot bypass validation), `max_attempts ∈ [1, 10]`, `lr_decay ∈ (0, 1)`, `min_lr > 0`; cross-validator `_validate_spike_recovery_requires_watchdog` rejects `loss_spike_recovery=true, loss_watchdog=false` at config-load (fails fast instead of never triggering); `convergence_window ∈ [5, 10_000]`, `convergence_rel_tol ∈ (0, 1]`, `recommend_action` reuses `detect_plateau` so plateau heuristic stays single-source-of-truth; `GradAccumMonitor.recommend()` caps doubled `accum` at `MAX_ACCUM=1024` so a runaway advisory loop cannot blow up DataLoader prefetch; `generate_config` validates BOTH the YAML output path AND the embedded `decisions["output"]` field via `is_under_cwd` (closes the gap where a crafted `decisions["output"]="../../etc"` would have silently propagated into the rendered YAML)
|
||||
- **v0.34.0 — Observability & Dev UX**: `.crash` bundle generator (`utils/crash.py`) recursively redacts `hf_*` / `sk-*` / `Bearer …` token-shaped strings in any captured `config` and metric tail before serialisation, so a `.crash` file shared on a public GitHub issue cannot leak credentials; `output_dir` is reduced to `os.path.basename` so `$HOME` doesn't leak; `write_crash_bundle` uses `os.path.realpath + commonpath` for cwd containment (Windows-safe; raises `ValueError` not `PermissionError` so callers cannot silently swallow with `except OSError`); filename appends `secrets.token_hex(4)` so two crashes in the same UTC second don't collide; bundle truncated to `MAX_BUNDLE_BYTES=1_000_000`. `train.py` crash-write surfaces failures to the user (no silent missing-bundle). `profiling.py` `resolve_trace_path` rejects empty / `.` / `..` / `/` / `\\` / null-byte `run_id` (closes the `output_dir/profiles/../trace.json` escape) and uses `os.path.realpath + is_under_cwd`; profiles dir is created only on successful torch import (no stale empty dirs on torch-less CI). `tracker.get_run` LIKE-prefix match escapes `%` / `_` / `\\` and uses `ESCAPE '\\'` so a crafted `run_id` cannot widen the match (mirrors v0.26.0 registry policy). Lazy schema migration (`_ensure_schema`) tolerates the "duplicate column" race when two CLI processes start simultaneously on a fresh DB (fork-based multi-GPU training, TUI auto-refresh). `runs.py show/replay/clean` switched user `run_id` rendering to `markup_escape` and switched `clean` containment from broken `Path.resolve() + relative_to()` to project-standard `os.path.realpath + is_under_cwd`. `tui_app.py` lazy-imports `ExperimentTracker` and `markup_escape`s every DB-sourced string before passing into Textual widgets so a crafted base_model / experiment_name cannot inject `[bold red]…[/]` markup. `run_cost.estimate_run_cost_usd` rejects `bool` in `num_gpus` (bool is a subclass of int — same defence as v0.30.0 `Candidate.__post_init__`); duration clamped to `[0, 1 year]`; unknown GPU returns `None` so callers render `—` instead of fabricating `$0.00`. `log_level.parse_log_level` rejects non-string + null-byte input.
|
||||
- **v0.33.0 — Live Wire**: RLVR `code_exec_reward` adds OS-level isolation (Linux best-effort `os.unshare(CLONE_NEWUSER|CLONE_NEWNET|CLONE_NEWPID)`, macOS `sandbox-exec` with default-deny `MACOS_SANDBOX_PROFILE` narrowed to a 3-name `mach-lookup` allowlist to prevent DNS / NSURLSession bypass of `(deny network*)`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat + S_ISLNK` + `shutil.rmtree(onerror=_abort_on_symlink)` so a symlink encountered mid-walk aborts rather than escapes; `run_gate` wraps each task scorer in a typed `try/except` so backend failures produce `score=None, error=str(exc)` (never silent `score=1.0`); `_parse_judge_url` removes the bare `http://` catch-all (defence-in-depth after the Pydantic GateTask validator); `soup can run` requires `--yes` or explicit consent callback and raises `ValueError` (not `PermissionError`, which is an `OSError` subclass that broad `except` blocks would swallow); GGUF `rglob` result for ollama deploy is `realpath+commonpath` checked against extract_dir (prevents symlink escape from a crafted can); `DeployTarget.path` validator normalises mixed `\\`/`/` separators before splitting (closes a Windows `..` bypass); `CAN_FORMAT_VERSION` 1→2 (additive — v1 still loads); `soup can publish` validates `repo_id` via `utils/hf.validate_repo_id`, resolves token via `resolve_token`, sanitises commit messages (first-line, 200-char cap), uses HTTPS-only HfApi; `_write_spike_recovery_hint` adds `is_under_cwd` containment check on `args.output_dir` from raw HF `TrainingArguments`; `lookup_entry_by_output_dir` emits `ResourceWarning` when 1000-row scan limit is hit (no silent miss); `CrossDocCollator` no longer mutates input feature dicts (HF Dataset rows are cached and reused — mutation broke subsequent batches); `Candidate` rejects `bool` in `score`/`latency_ms` (was sneaking past `int` isinstance check); `evaluate_candidate` latency mean now divides by *completed* prompts (excludes crashed) so a broken candidate isn't artificially fast; `auto_quant.run_auto_quant_picker` soft-falls-back to highest-scored candidate when no candidate clears `min_score` (server still binds); `build_logits_processors` returns `[]` when neither `outlines` nor `lm-format-enforcer` is installed (server degrades to free-form rather than 500); MII server uses loopback-only CORS, max_tokens cap [1, 16384], stream rejection, generic 500 with no stack-trace leak; `os.execvp` auto-reexec uses list args (no shell), all forwarded flags pre-validated; `cleanup_extract_dir` uses `os.path.commonpath` (Windows-safe) instead of `startswith`; `_run_subprocess` catches `TimeoutExpired` and returns rc=124 (coreutils convention) instead of an unhandled traceback; new `eval_results` and `tensorrt` artifact kinds in `RegistryStore._VALID_KINDS`
|
||||
- **v0.39.0 — LoRA Quality**: `LoraConfig.init_strategy: Literal["random","pissa","olora"]` rejects unknown strategies; PiSSA + DoRA / VeRA combinations rejected at config-load. `model_validator(mode="before")` aligns `use_olora=True` → `init_strategy="olora"` via dict-copy (no caller mutation; matches v0.33.0 #47 immutability policy). `rank_pattern`/`alpha_pattern: Optional[Dict[str, int]]` capped at 256 keys × value (0, 1024], rejects `bool` (subclass of `int` — matches v0.30.0 `Candidate` policy), null bytes in keys, empty keys; cross-validator rejects with `use_vera=True`. `ReLoRAPolicy` is `@dataclass(frozen=True)` (post-construction mutation raises `FrozenInstanceError`); bounds: `steps ∈ [1, 1e7]`, `warmup_ratio ∈ [0, 1]`, `prune_ratio ∈ (0, 1)` (strict — prevents zero-everything footgun). `magnitude_prune_tensor` strict `0 < prune_ratio < 1` rejection, non-Tensor input raises `TypeError`, empty / single-element tensor short-circuits (avoids `kthvalue(_, 0)` runtime crash). `_validate_relora_supported_tasks` cross-validator rejects `relora_steps` with `task != "sft"` and `backend=mlx` with distinct error messages (matches v0.34.0 distinct-reason policy); multi-trainer expansion deferred to v0.39.1. `is_gemma4_model` uses a word-boundary regex (`(?:^|[^a-z0-9])gemma-?4(?:[^a-z0-9]|$)`) so `"ungemma4ed"` / `"my-gemma4ish"` no longer over-match; null-byte rejection on `model_name`. `apply_gemma4_clippable_patch` weight-copy fallback logs at DEBUG instead of silent random-init; the patch is gated by `is_gemma4_model(cfg.base)` in `sft.py` before invocation so non-Gemma4 trainings never traverse the module tree. `apply_surgical_patches` rejects empty / null-byte `model_name` with `ValueError`. `templates/load_template` containment: filename re-validated via `_validate_name` (rejects `..`/`/`/`\\`/null/empty); `os.path.realpath + os.path.commonpath` containment check on the resolved path against `_templates_dir()` so a tampered `manifest.json` cannot read files outside the package directory (mirrors v0.26.0 registry policy). Tampered-manifest `ValueError` from `_validate_name` caught and falls back to inline (no propagating exception). 256 KB file-size cap. Inline `TEMPLATES` carries an explicit deprecation comment pointing at the canonical YAML registry; `tests/test_templates_yaml.py` asserts byte-equality of all 16 inline ↔ YAML pairs to prevent silent drift. Planned removal: v0.41.0+.
|
||||
- **v0.38.0 — Quant Menu**: `TrainingConfig.quantization` Literal extended with `gptq` / `awq` / `hqq:1bit`..`hqq:8bit` (no `hqq:7bit` — HQQ doesn't support it) / `aqlm` / `eetq` / `mxfp4` / `fp8`; Pydantic rejects every other string at config-load. `validate_gptq_checkpoint` and `validate_awq_checkpoint` probe local paths for `quantize_config.json` / `quant_config.json`; HF repo IDs fall through; null-byte rejection + non-string `TypeError` on the ref. `_validate_prequantized_no_qat` rejects every pre-quantized format combined with `quantization_aware` (int8 QAT or `'fp8'`) — pre-quantized weights carry their own scale and QAT/FP8 prepare would silently corrupt them (mirrors LlamaFactory `quantization.py:117/199/211`). `_validate_bnb_quant_storage_only_with_4bit` rejects `bnb_4bit_quant_storage` on every non-BNB-4bit format (silent no-op otherwise); allowed dtypes: `Literal["uint8", "float16", "bfloat16", "float32"]`. `_validate_quant_menu_supported_tasks` restricts the new formats to `task='sft'` on `backend='transformers'` in v0.38.0 with distinct MLX-backend vs unsupported-task error messages (matches v0.34.0 distinct-reason policy). `check_quant_distributed_compat` hard-fails HQQ/EETQ/AQLM × {FSDP, ZeRO-3} (sourced from LlamaFactory `quantization.py:199/211` plus AQLM dequant constraints); warning-tier (not error) for BNB-4bit + FSDP without `bnb_4bit_quant_storage` so users see the silent perf cliff; unknown `quantization` raises `ValueError` (no silent pass) and the check is wired into `commands/train.py` startup. `parse_hqq_bits` rejects unsupported bit-rates and malformed `hqq:` strings before any kernel build.
|
||||
- **v0.37.0 — Multipack**: `validate_multipack_architecture` raises `ValueError` on unknown arch (loud-fail vs Axolotl's silent-miss footgun); 18-arch frozen allowlist (Llama 3.x / Qwen 2/3 / Mistral / Gemma 2/3 / Phi 3/4 / DeepSeek V2/V3 / Mixtral / Falcon / StableLM / SmolLM2). FFD packer caps at `_MAX_FFD_ITEMS=1_000_000` (algorithm is O(N²) worst-case — defence against adversarial dataset DoS); `bool` rejection on every numeric input (`max_len`, per-element `lengths`, `batch_max_len`, `batch_size`, `seed`, `max_seq_length`) matches v0.30.0+ `Candidate` policy; generator-input materialisation prevents silent empty-bin output when validation exhausts the iterator. `MultipackBatchSampler` rejects empty `lengths`, non-positive `batch_max_len`/`batch_size`, items larger than `batch_max_len`. `build_multipack_sampler_for_lengths` rejects `tcfg.batch_size="auto"` with actionable message (must be resolved upstream). `_validate_multipack_packing_exclusive` cross-validator on `TrainingConfig` rejects both `multipack` and `packing` set; `_validate_multipack_supported_tasks` on `SoupConfig` restricts multipack to `sft`/`pretrain` on `transformers` backend with distinct error messages for MLX backend vs unsupported task (matches v0.34.0 distinct-reason policy). `build_4d_attention_mask` caps allocations at `_MAX_MASK_ELEMENTS=2³¹` cells (~8GB float32) — defence against `max_length=1M` × `batch_size=8` OOM; `tag_sub_sequences` capped at `_MAX_BOUNDARY_SEGMENTS=1_000_000`. Mask builder rejects non-floating dtypes (was silent `np.finfo` ValueError), non-2D `seq_pos_ids`, negative segment IDs; padding (id=0) tokens are fully masked, including diagonal, so softmax is well-defined. `select_packing_strategy` rejects non-bool `flash_attn_available`. `JinjaTemplateAnalyzer` parses chat templates via `Environment.parse` only — never renders, so a crafted soup.yaml cannot trigger SSRF / filesystem reads; 128KB template cap, null-byte rejection, `TemplateSyntaxError` re-raised as `ValueError`. `DEFAULT_MESSAGE_FIELDS` is a `frozenset` (runtime-immutable); `JinjaTemplateAnalyzer.message_fields` returns a defensive copy.
|
||||
- **v0.36.0 — Correctness First**: `--trust-remote-code` opt-in replaces 9 unconditional `trust_remote_code=True` call sites across `soup train` / `chat` / `serve` / `data download` / `eval auto`; `KNOWN_SAFE_PREFIXES` allowlist (15 first-party orgs) suppresses warning panel for trusted repos; `model_requires_trust_remote_code` probes local `config.json` for `auto_map` (HF Hub repo IDs return `None`/unknown — HF still raises loudly when custom code is actually needed); `resolve_trust_remote_code` raises `ValueError` with actionable message when model needs custom code but the user did not opt in. Chat-template hardening: `DataConfig.chat_template` validator rejects null bytes, oversize (>64KB), AND filesystem-touching Jinja directives (`{% include %}`, `{% import %}`, `{% from %}`, `{% macro %}`, `{% extends %}` — both whitespace-control variants); empty string normalised to `None`; `_REGISTRY` wrapped in `MappingProxyType` (callers cannot mutate); `apply_chat_template_override` emits yellow advisory when active so users know `soup push` will persist the override into `tokenizer_config.json`. SFT silent f-string fallback (`f"{role}: {content}"`) replaced with hard `ValueError` — produced wrong loss labels for years on tokenizers without a chat template. Loss-mask fallback passes `add_special_tokens=False` to incremental tokenize calls so HF cannot double-prepend BOS at front of each render (consistent prefix-delta walk); narrowed exception catch in `_apply_template_with_mask` from `(TypeError, ValueError)` to `TypeError` only so a malformed messages list propagates instead of falling through to the loose path. `SOUP_BATCH_CACHE_PATH` env override containment-checked via `os.path.realpath + commonpath` against `~`/cwd/`tempfile.gettempdir()`; out-of-bounds values fall through to safe default; cache file gets best-effort `0o600` perms after atomic rename (matches v0.26.0 registry.db policy). `make_cache_key` rejects `bool` in numeric inputs (matches v0.30.0 `Candidate` policy). Documented limitation: non-SFT trainers (DPO/GRPO/KTO/ORPO/SimPO/IPO/PPO/RewardModel/Pretrain/Embedding) and `commands/{diff,export,merge,infer,generate}.py` still hardcode `trust_remote_code=True` — v0.36.x patch follow-up.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "soup-cli"
|
||||
version = "0.38.0"
|
||||
version = "0.39.0"
|
||||
description = "Fine-tune LLMs in one command. No SSH, no config hell."
|
||||
readme = "README.md"
|
||||
license = "Apache-2.0"
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
"""Soup CLI — Fine-tune LLMs in one command."""
|
||||
|
||||
__version__ = "0.38.0"
|
||||
__version__ = "0.39.0"
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ from rich.console import Console
|
|||
from rich.panel import Panel
|
||||
from rich.prompt import Prompt
|
||||
|
||||
from soup_cli.config.schema import TEMPLATES
|
||||
from soup_cli.templates import list_templates, load_template
|
||||
|
||||
console = Console()
|
||||
|
||||
|
|
@ -36,11 +36,11 @@ def init(
|
|||
raise typer.Exit()
|
||||
|
||||
if template:
|
||||
if template not in TEMPLATES:
|
||||
config_text = load_template(template)
|
||||
if config_text is None:
|
||||
console.print(f"[red]Unknown template: {template}[/]")
|
||||
console.print(f"Available: {', '.join(TEMPLATES.keys())}")
|
||||
console.print(f"Available: {', '.join(list_templates())}")
|
||||
raise typer.Exit(1)
|
||||
config_text = TEMPLATES[template]
|
||||
console.print(f"[green]Using template:[/] {template}")
|
||||
else:
|
||||
config_text = _interactive_wizard()
|
||||
|
|
|
|||
|
|
@ -1,10 +1,14 @@
|
|||
"""Pydantic schemas for soup.yaml config — single source of truth."""
|
||||
|
||||
import re
|
||||
from typing import List, Literal, Optional, Union
|
||||
from typing import Dict, List, Literal, Optional, Union
|
||||
|
||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||
|
||||
# v0.39.0 Part C — per-pattern LoRA rank/alpha bounds
|
||||
_MAX_LORA_RANK_PATTERN_KEYS = 256
|
||||
_MAX_LORA_RANK_PATTERN_VALUE = 1024
|
||||
|
||||
|
||||
class LoraConfig(BaseModel):
|
||||
r: int = Field(default=64, description="LoRA rank")
|
||||
|
|
@ -35,7 +39,35 @@ class LoraConfig(BaseModel):
|
|||
description=(
|
||||
"Enable OLoRA (Orthogonal LoRA init via QR decomposition). "
|
||||
"Passes init_lora_weights='olora' to peft. "
|
||||
"Mutually exclusive with use_dora and use_vera."
|
||||
"Mutually exclusive with use_dora and use_vera. "
|
||||
"Equivalent to init_strategy='olora'."
|
||||
),
|
||||
)
|
||||
rank_pattern: Optional[Dict[str, int]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Per-target-module-pattern LoRA rank override. Maps module name "
|
||||
"patterns (e.g. 'q_proj', 'experts.*.w1') to integer rank values. "
|
||||
"Useful for MoE configs where expert FFNs need lower rank than attn. "
|
||||
"Incompatible with use_vera (VeRA shares one rank across modules)."
|
||||
),
|
||||
)
|
||||
alpha_pattern: Optional[Dict[str, int]] = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Per-target-module-pattern LoRA alpha override. Maps module name "
|
||||
"patterns to integer alpha values. Pairs with rank_pattern. "
|
||||
"Incompatible with use_vera."
|
||||
),
|
||||
)
|
||||
init_strategy: Literal["random", "pissa", "olora"] = Field(
|
||||
default="random",
|
||||
description=(
|
||||
"LoRA init strategy. 'random' (default) is standard Kaiming init. "
|
||||
"'pissa' (PiSSA) initializes A/B from the SVD of the base weight — "
|
||||
"faster early convergence but adds an SVD pass on the first epoch. "
|
||||
"'olora' is equivalent to use_olora=True (orthogonal QR init). "
|
||||
"Cannot be combined with use_dora or use_vera."
|
||||
),
|
||||
)
|
||||
|
||||
|
|
@ -57,6 +89,85 @@ class LoraConfig(BaseModel):
|
|||
)
|
||||
return self
|
||||
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
def _backcompat_align_olora(cls, values):
|
||||
"""Back-compat: pre-validation, align init_strategy='olora' when only use_olora was set."""
|
||||
if not isinstance(values, dict):
|
||||
return values
|
||||
# Copy to avoid mutating the caller's dict (matches v0.33.0 #47
|
||||
# CrossDocCollator immutability fix).
|
||||
if values.get("use_olora") and "init_strategy" not in values:
|
||||
values = dict(values)
|
||||
values["init_strategy"] = "olora"
|
||||
return values
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_init_strategy(self) -> "LoraConfig":
|
||||
# use_olora=True must agree with init_strategy when both are explicit
|
||||
if self.use_olora and self.init_strategy != "olora":
|
||||
raise ValueError(
|
||||
f"use_olora=True conflicts with init_strategy={self.init_strategy!r}. "
|
||||
f"Either set init_strategy='olora' (or omit it), or set use_olora=False."
|
||||
)
|
||||
# init_strategy='pissa' is incompatible with DoRA / VeRA
|
||||
if self.init_strategy == "pissa" and (self.use_dora or self.use_vera):
|
||||
other = "use_dora" if self.use_dora else "use_vera"
|
||||
raise ValueError(
|
||||
f"init_strategy='pissa' is incompatible with {other}=True. "
|
||||
f"PiSSA initializes the LoRA pair via SVD; combine with plain LoRA "
|
||||
f"(or rsLoRA) only."
|
||||
)
|
||||
return self
|
||||
|
||||
@field_validator("rank_pattern", "alpha_pattern", mode="before")
|
||||
@classmethod
|
||||
def _validate_pattern_dict(cls, value) -> Optional[Dict[str, int]]:
|
||||
if value is None:
|
||||
return None
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("rank_pattern/alpha_pattern must be a dict[str, int]")
|
||||
if len(value) > _MAX_LORA_RANK_PATTERN_KEYS:
|
||||
raise ValueError(
|
||||
f"rank_pattern/alpha_pattern caps at {_MAX_LORA_RANK_PATTERN_KEYS} keys, "
|
||||
f"got {len(value)}"
|
||||
)
|
||||
cleaned: Dict[str, int] = {}
|
||||
for key, val in value.items():
|
||||
if not isinstance(key, str) or not key:
|
||||
raise ValueError(
|
||||
"rank_pattern/alpha_pattern keys must be non-empty strings"
|
||||
)
|
||||
if "\x00" in key:
|
||||
raise ValueError("rank_pattern/alpha_pattern keys cannot contain null bytes")
|
||||
if isinstance(val, bool) or not isinstance(val, int):
|
||||
raise ValueError(
|
||||
f"rank_pattern/alpha_pattern values must be int, "
|
||||
f"got {type(val).__name__} for {key!r}"
|
||||
)
|
||||
if val <= 0 or val > _MAX_LORA_RANK_PATTERN_VALUE:
|
||||
raise ValueError(
|
||||
f"rank_pattern/alpha_pattern values must be in (0, "
|
||||
f"{_MAX_LORA_RANK_PATTERN_VALUE}], got {val} for {key!r}"
|
||||
)
|
||||
cleaned[key] = val
|
||||
return cleaned
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_pattern_vera_exclusivity(self) -> "LoraConfig":
|
||||
if self.use_vera and self.rank_pattern:
|
||||
raise ValueError(
|
||||
"rank_pattern is incompatible with use_vera=True (VeRA shares "
|
||||
"a single rank across all target modules). Disable use_vera or "
|
||||
"remove rank_pattern."
|
||||
)
|
||||
if self.use_vera and self.alpha_pattern:
|
||||
raise ValueError(
|
||||
"alpha_pattern is incompatible with use_vera=True. Disable "
|
||||
"use_vera or remove alpha_pattern."
|
||||
)
|
||||
return self
|
||||
|
||||
|
||||
class DataConfig(BaseModel):
|
||||
train: str = Field(..., description="Path to training data or HF dataset name")
|
||||
|
|
@ -486,6 +597,29 @@ class TrainingConfig(BaseModel):
|
|||
default=0.5, gt=0.0, lt=1.0,
|
||||
description="Multiply LR by this factor on each spike recovery (0.5 = halve)",
|
||||
)
|
||||
# ReLoRA (v0.39.0 Part B)
|
||||
relora_steps: Optional[int] = Field(
|
||||
default=None, ge=1, le=10**7,
|
||||
description=(
|
||||
"Fire ReLoRA magnitude-prune + optimizer reset every N global steps. "
|
||||
"None disables. Requires a LoRA-style PEFT (not VeRA)."
|
||||
),
|
||||
)
|
||||
relora_warmup_ratio: float = Field(
|
||||
default=0.1, ge=0.0, le=1.0,
|
||||
description="Skip ReLoRA firings during the first warmup_ratio fraction of training",
|
||||
)
|
||||
relora_reset_optimizer: bool = Field(
|
||||
default=True,
|
||||
description="Clear optimizer state for pruned LoRA params on each ReLoRA fire",
|
||||
)
|
||||
relora_prune_ratio: float = Field(
|
||||
default=0.9, gt=0.0, lt=1.0,
|
||||
description=(
|
||||
"Fraction of LoRA weights to zero out by magnitude on each fire "
|
||||
"(0.9 keeps the top 10%). Must be < 1.0."
|
||||
),
|
||||
)
|
||||
# Convergence detection (v0.32.0 Part F)
|
||||
convergence_detection: bool = Field(
|
||||
default=False,
|
||||
|
|
@ -865,6 +999,31 @@ class SoupConfig(BaseModel):
|
|||
)
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_relora_supported_tasks(self) -> "SoupConfig":
|
||||
"""v0.39.0 — ReLoRA callback is wired in the SFT trainer only.
|
||||
|
||||
Multi-trainer expansion (DPO / GRPO / KTO / ORPO / SimPO / IPO /
|
||||
PPO / RewardModel / Pretrain / Embedding) deferred to v0.39.1
|
||||
(mirrors v0.27.0 MII / v0.37.0 multipack / v0.38.0 quant menu
|
||||
stub-then-live pattern).
|
||||
"""
|
||||
if self.training.relora_steps is None:
|
||||
return self
|
||||
if self.backend == "mlx":
|
||||
raise ValueError(
|
||||
"relora_steps is not supported on the mlx backend "
|
||||
"in v0.39.0 (callback is HF Trainer-specific). "
|
||||
"Use backend='transformers' or remove relora_steps."
|
||||
)
|
||||
if self.task != "sft":
|
||||
raise ValueError(
|
||||
f"relora_steps is wired only for task='sft' in v0.39.0, "
|
||||
f"got task={self.task!r}. Multi-trainer expansion is tracked "
|
||||
"as a known limitation; remove relora_steps or set task='sft'."
|
||||
)
|
||||
return self
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_quant_menu_supported_tasks(self) -> "SoupConfig":
|
||||
"""v0.38.0 — Quant Menu (gptq/awq/hqq:Nbit/aqlm/eetq/mxfp4/fp8) is wired
|
||||
|
|
@ -919,6 +1078,11 @@ class SoupConfig(BaseModel):
|
|||
|
||||
# --- Built-in templates ---
|
||||
|
||||
# DEPRECATED (v0.39.0 Part E) — these inline templates are kept for back-compat.
|
||||
# The canonical source is `soup_cli/templates/*.yaml` with `manifest.json`.
|
||||
# Both sources are asserted equal in tests/test_templates_yaml.py — when editing
|
||||
# a template, update both. Planned removal: v0.41.0+ once external consumers
|
||||
# have migrated to the YAML registry.
|
||||
TEMPLATES: dict[str, str] = {
|
||||
"chat": """# Soup template: Chat Assistant
|
||||
# Fine-tune a model for conversational chat
|
||||
|
|
|
|||
|
|
@ -0,0 +1,110 @@
|
|||
"""Soup template registry (v0.39.0 Part E).
|
||||
|
||||
Templates live as YAML files alongside this module. ``manifest.json``
|
||||
declares which files exist; the loader resolves a template name to its
|
||||
file content. Falls back to the inline ``TEMPLATES`` dict in
|
||||
``soup_cli.config.schema`` for back-compat.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
# Cap to prevent symlink-based escapes or pathologically huge templates.
|
||||
_MAX_TEMPLATE_BYTES = 256 * 1024
|
||||
|
||||
|
||||
def _templates_dir() -> Path:
|
||||
return Path(__file__).resolve().parent
|
||||
|
||||
|
||||
def _load_manifest() -> dict:
|
||||
path = _templates_dir() / "manifest.json"
|
||||
if not path.is_file():
|
||||
return {"templates": {}, "version": 1}
|
||||
try:
|
||||
with path.open("r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return {"templates": {}, "version": 1}
|
||||
if not isinstance(data, dict):
|
||||
return {"templates": {}, "version": 1}
|
||||
templates = data.get("templates")
|
||||
if not isinstance(templates, dict):
|
||||
templates = {}
|
||||
return {"templates": templates, "version": int(data.get("version", 1))}
|
||||
|
||||
|
||||
def _validate_name(name: str) -> None:
|
||||
if not isinstance(name, str) or not name:
|
||||
raise ValueError("Template name must be a non-empty string")
|
||||
if "\x00" in name or "/" in name or "\\" in name or ".." in name:
|
||||
raise ValueError(f"Invalid template name: {name!r}")
|
||||
|
||||
|
||||
def list_templates() -> list[str]:
|
||||
"""Return all known template names (YAML manifest + inline fallback)."""
|
||||
names: set[str] = set()
|
||||
manifest = _load_manifest()
|
||||
names.update(manifest["templates"].keys())
|
||||
# Fold in inline templates for back-compat.
|
||||
try:
|
||||
from soup_cli.config.schema import TEMPLATES as _INLINE
|
||||
names.update(_INLINE.keys())
|
||||
except ImportError:
|
||||
pass
|
||||
return sorted(names)
|
||||
|
||||
|
||||
def load_template(name: str) -> Optional[str]:
|
||||
"""Resolve a template name to its YAML body. Returns None if unknown.
|
||||
|
||||
Order:
|
||||
1. YAML file in this directory (per ``manifest.json``).
|
||||
2. Inline ``TEMPLATES`` dict in ``soup_cli.config.schema``.
|
||||
"""
|
||||
_validate_name(name)
|
||||
manifest = _load_manifest()
|
||||
filename = manifest["templates"].get(name)
|
||||
if isinstance(filename, str) and filename:
|
||||
# Re-validate filename to defeat manifest tampering. A tampered
|
||||
# manifest pointing at "../secret.yaml" must not crash the loader —
|
||||
# silently fall back to inline so callers see a degraded but safe
|
||||
# result instead of a propagating ValueError.
|
||||
try:
|
||||
_validate_name(filename.replace(".yaml", ""))
|
||||
except ValueError:
|
||||
return _fallback_inline(name)
|
||||
if not filename.endswith(".yaml"):
|
||||
return _fallback_inline(name)
|
||||
templates_dir_str = os.path.realpath(str(_templates_dir()))
|
||||
path = _templates_dir() / filename
|
||||
# Containment: defeat crafted manifest entries with extra dots / mixed
|
||||
# separators that slip past _validate_name on Windows.
|
||||
try:
|
||||
real_path = os.path.realpath(str(path))
|
||||
if os.path.commonpath([real_path, templates_dir_str]) != templates_dir_str:
|
||||
return _fallback_inline(name)
|
||||
except (OSError, ValueError):
|
||||
return _fallback_inline(name)
|
||||
try:
|
||||
if path.is_file():
|
||||
size = path.stat().st_size
|
||||
if size > _MAX_TEMPLATE_BYTES:
|
||||
return _fallback_inline(name)
|
||||
with path.open("r", encoding="utf-8") as f:
|
||||
return f.read()
|
||||
except OSError:
|
||||
pass
|
||||
return _fallback_inline(name)
|
||||
|
||||
|
||||
def _fallback_inline(name: str) -> Optional[str]:
|
||||
try:
|
||||
from soup_cli.config.schema import TEMPLATES as _INLINE
|
||||
except ImportError:
|
||||
return None
|
||||
return _INLINE.get(name)
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
# Soup template: Audio / Speech
|
||||
# Fine-tune an audio-language model for speech understanding
|
||||
#
|
||||
# Supported models: Qwen2-Audio, Whisper (via transformers)
|
||||
#
|
||||
# Data format (JSONL):
|
||||
# {"audio": "path/to/audio.wav", "messages": [
|
||||
# {"role": "user", "content": "Transcribe."},
|
||||
# {"role": "assistant", "content": "Hello world."}]}
|
||||
|
||||
base: Qwen/Qwen2-Audio-7B-Instruct
|
||||
task: sft
|
||||
modality: audio
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/audio_train.jsonl
|
||||
format: audio
|
||||
audio_dir: ./data/audio
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 8
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
|
||||
output: ./output_audio
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
# Soup template: Chat Assistant
|
||||
# Fine-tune a model for conversational chat
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: sft
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/train.jsonl
|
||||
format: alpaca
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 2e-5
|
||||
batch_size: auto
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
# Soup template: Code Model
|
||||
# Fine-tune a model for code generation / completion
|
||||
|
||||
base: codellama/CodeLlama-7b-Instruct-hf
|
||||
task: sft
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/code_train.jsonl
|
||||
format: alpaca
|
||||
val_split: 0.1
|
||||
max_length: 4096
|
||||
|
||||
training:
|
||||
epochs: 2
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
lora:
|
||||
r: 128
|
||||
alpha: 32
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
# Soup template: Embedding Model Fine-tuning
|
||||
# Fine-tune a sentence embedding model (BGE, E5, GTE, etc.)
|
||||
#
|
||||
# Data format (JSONL) — contrastive pairs:
|
||||
# {"anchor": "What is Python?", "positive": "Python is a programming language."}
|
||||
#
|
||||
# Data format (JSONL) — triplets:
|
||||
# {"anchor": "query", "positive": "relevant doc", "negative": "unrelated doc"}
|
||||
|
||||
base: BAAI/bge-base-en-v1.5
|
||||
task: embedding
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/embedding_train.jsonl
|
||||
format: embedding
|
||||
val_split: 0.1
|
||||
max_length: 512
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 2e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 4
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: none
|
||||
embedding_loss: contrastive
|
||||
embedding_margin: 0.5
|
||||
embedding_pooling: mean
|
||||
|
||||
output: ./output_embedding
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# Soup template: IPO (Identity Preference Optimization)
|
||||
# A theoretically grounded variant of DPO with stronger regularization
|
||||
#
|
||||
# Data format (JSONL):
|
||||
# {"prompt": "What is 2+2?", "chosen": "4", "rejected": "Fish"}
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: ipo
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/preference_train.jsonl
|
||||
format: dpo
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 4
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
ipo_tau: 0.1
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
# Soup template: KTO (Kahneman-Tversky Optimization)
|
||||
# Align a model using unpaired preference data (no need for chosen+rejected pairs)
|
||||
#
|
||||
# Data format (JSONL):
|
||||
# {"prompt": "What is 2+2?", "completion": "4", "label": true}
|
||||
# {"prompt": "What is 2+2?", "completion": "Fish", "label": false}
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: kto
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/kto_train.jsonl
|
||||
format: kto
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 4
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
kto_beta: 0.1
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
# Soup template: Long-Context Fine-tuning (128k+)
|
||||
# Extend model context window for long-document understanding
|
||||
#
|
||||
# Uses RoPE scaling + gradient checkpointing + FlashAttention for 128k tokens.
|
||||
# Optionally enable Liger Kernel for additional memory savings.
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: sft
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/long_context_train.jsonl
|
||||
format: alpaca
|
||||
val_split: 0.05
|
||||
max_length: 131072
|
||||
|
||||
training:
|
||||
epochs: 1
|
||||
lr: 5e-6
|
||||
batch_size: 1
|
||||
gradient_accumulation_steps: 16
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
gradient_checkpointing: true
|
||||
rope_scaling_type: dynamic
|
||||
use_flash_attn: true
|
||||
# use_liger: true # pip install 'soup-cli[liger]' for fused ops
|
||||
# use_ring_attention: true # Multi-GPU sequence parallelism
|
||||
|
||||
output: ./output_longctx
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
{
|
||||
"templates": {
|
||||
"audio": "audio.yaml",
|
||||
"chat": "chat.yaml",
|
||||
"code": "code.yaml",
|
||||
"embedding": "embedding.yaml",
|
||||
"ipo": "ipo.yaml",
|
||||
"kto": "kto.yaml",
|
||||
"longcontext": "longcontext.yaml",
|
||||
"medical": "medical.yaml",
|
||||
"moe": "moe.yaml",
|
||||
"orpo": "orpo.yaml",
|
||||
"pretrain": "pretrain.yaml",
|
||||
"reasoning": "reasoning.yaml",
|
||||
"rlhf": "rlhf.yaml",
|
||||
"simpo": "simpo.yaml",
|
||||
"tool-calling": "tool-calling.yaml",
|
||||
"vision": "vision.yaml"
|
||||
},
|
||||
"version": 1
|
||||
}
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
# Soup template: Medical / Domain Expert
|
||||
# Fine-tune a model with domain-specific knowledge
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: sft
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/medical_train.jsonl
|
||||
format: alpaca
|
||||
val_split: 0.15
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 5
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 8
|
||||
lora:
|
||||
r: 128
|
||||
alpha: 32
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# Soup template: MoE (Mixture of Experts) Fine-tuning
|
||||
# Fine-tune a Mixture of Experts model with ScatterMoE LoRA
|
||||
#
|
||||
# Supported MoE models: Qwen3-30B-A3B, Mixtral-8x7B, DeepSeek-V3, etc.
|
||||
|
||||
base: Qwen/Qwen3-30B-A3B
|
||||
task: sft
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/train.jsonl
|
||||
format: alpaca
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 8
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
moe_lora: true
|
||||
moe_aux_loss_coeff: 0.01
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# Soup template: ORPO (Odds Ratio Preference Optimization)
|
||||
# Align a model without a reference model — simpler than DPO
|
||||
#
|
||||
# Data format (JSONL):
|
||||
# {"prompt": "What is 2+2?", "chosen": "4", "rejected": "Fish"}
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: orpo
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/preference_train.jsonl
|
||||
format: dpo
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 4
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
orpo_beta: 0.1
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
# Soup template: Continued Pre-training
|
||||
# Continue pre-training a model on raw text data (domain adaptation)
|
||||
#
|
||||
# Data format (JSONL):
|
||||
# {"text": "Your raw text document here..."}
|
||||
#
|
||||
# Or plain .txt files (one document per line or entire file as one document).
|
||||
|
||||
base: meta-llama/Llama-3.1-8B
|
||||
task: pretrain
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/corpus.jsonl
|
||||
format: plaintext
|
||||
val_split: 0.05
|
||||
max_length: 4096
|
||||
|
||||
training:
|
||||
epochs: 1
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 8
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
|
||||
output: ./output_pretrain
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
# Soup template: Reasoning / GRPO
|
||||
# Fine-tune a model for chain-of-thought reasoning with GRPO
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: grpo
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/reasoning_train.jsonl
|
||||
format: sharegpt
|
||||
val_split: 0.1
|
||||
max_length: 4096
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 8
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
grpo_beta: 0.1
|
||||
num_generations: 4
|
||||
reward_fn: accuracy
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
# Soup template: Full RLHF Pipeline (SFT + Reward Model + PPO)
|
||||
# Three-stage training: 1) SFT warmup, 2) Reward model, 3) PPO alignment
|
||||
#
|
||||
# Usage:
|
||||
# Step 1: soup train --config soup_sft.yaml # SFT warmup
|
||||
# Step 2: soup train --config soup_rm.yaml # Train reward model
|
||||
# Step 3: soup train --config soup_ppo.yaml # PPO with reward model
|
||||
#
|
||||
# This template generates the PPO config (step 3).
|
||||
# For steps 1-2, use: soup init --template chat (SFT) and edit task to reward_model.
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: ppo
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/prompts.jsonl
|
||||
format: chatml
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 1
|
||||
lr: 1e-6
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 4
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
reward_model: ./output_rm
|
||||
ppo_epochs: 4
|
||||
ppo_clip_ratio: 0.2
|
||||
ppo_kl_penalty: 0.05
|
||||
|
||||
output: ./output_ppo
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
# Soup template: SimPO (Simple Preference Optimization)
|
||||
# Reference-free preference alignment with length-normalized rewards
|
||||
#
|
||||
# Data format (JSONL):
|
||||
# {"prompt": "What is 2+2?", "chosen": "4", "rejected": "Fish"}
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: simpo
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/preference_train.jsonl
|
||||
format: dpo
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 4
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
simpo_gamma: 0.5
|
||||
cpo_alpha: 1.0
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
# Soup template: Tool-Calling / Agentic Fine-tuning
|
||||
# Fine-tune a model to call tools / functions correctly
|
||||
#
|
||||
# Data format (JSONL):
|
||||
# {
|
||||
# "messages": [{"role": "user", "content": "What's the weather in Tokyo?"}],
|
||||
# "tools": [{"type": "function", "function": {
|
||||
# "name": "get_weather",
|
||||
# "description": "Get current weather for a city",
|
||||
# "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}
|
||||
# }}],
|
||||
# "tool_calls": [{"function": {
|
||||
# "name": "get_weather",
|
||||
# "arguments": "{\"city\": \"Tokyo\"}"
|
||||
# }}]
|
||||
# }
|
||||
|
||||
base: meta-llama/Llama-3.1-8B-Instruct
|
||||
task: sft
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/tool_calling_train.jsonl
|
||||
format: tool-calling
|
||||
val_split: 0.1
|
||||
max_length: 4096
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 2e-4
|
||||
batch_size: auto
|
||||
gradient_accumulation_steps: 4
|
||||
lora:
|
||||
r: 16
|
||||
alpha: 32
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
|
||||
output: ./output
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
# Soup template: Vision / Multimodal
|
||||
# Fine-tune a vision-language model for image understanding
|
||||
|
||||
base: meta-llama/Llama-3.2-11B-Vision-Instruct
|
||||
task: sft
|
||||
modality: vision
|
||||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||||
|
||||
data:
|
||||
train: ./data/vision_train.jsonl
|
||||
format: llava
|
||||
image_dir: ./data/images
|
||||
val_split: 0.1
|
||||
max_length: 2048
|
||||
|
||||
training:
|
||||
epochs: 3
|
||||
lr: 1e-5
|
||||
batch_size: auto
|
||||
lora:
|
||||
r: 64
|
||||
alpha: 16
|
||||
target_modules: auto
|
||||
quantization: 4bit
|
||||
|
||||
output: ./output
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
"""SFT (Supervised Fine-Tuning) trainer — wraps HuggingFace transformers + peft + trl."""
|
||||
|
||||
import logging
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
|
@ -9,6 +10,8 @@ from rich.console import Console
|
|||
from soup_cli.config.schema import SoupConfig
|
||||
from soup_cli.utils.gpu import estimate_batch_size, model_size_from_name
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
|
|
@ -470,7 +473,27 @@ class SFTTrainerWrapper:
|
|||
use_dora=tcfg.lora.use_dora,
|
||||
use_rslora=tcfg.lora.use_rslora,
|
||||
)
|
||||
# v0.39.0 Part D — surgical PEFT patches (Gemma4 ClippableLinear,
|
||||
# MoE 3D expert dropout-strip). Pre-LoRA pass for ClippableLinear so
|
||||
# PEFT's matcher sees the swapped nn.Linear; the model-name gate
|
||||
# inside is_gemma4_model keeps the swap from running on non-Gemma4.
|
||||
from soup_cli.utils.peft_patches import (
|
||||
apply_gemma4_clippable_patch,
|
||||
is_gemma4_model,
|
||||
strip_lora_dropout_for_3d_experts,
|
||||
)
|
||||
if is_gemma4_model(cfg.base):
|
||||
try:
|
||||
apply_gemma4_clippable_patch(self.model)
|
||||
except Exception as exc: # noqa: BLE001 — best-effort patch, log + continue
|
||||
logger.debug("apply_gemma4_clippable_patch skipped: %s", exc)
|
||||
self.model = get_peft_model(self.model, lora_config)
|
||||
# Post-LoRA pass for 3-D expert dropout strip (architecture-detected
|
||||
# via weight.ndim==3 inside the helper; safe to call unconditionally).
|
||||
try:
|
||||
strip_lora_dropout_for_3d_experts(self.model)
|
||||
except Exception as exc: # noqa: BLE001 — best-effort patch, log + continue
|
||||
logger.debug("strip_lora_dropout_for_3d_experts skipped: %s", exc)
|
||||
|
||||
self._apply_quantization_aware(tcfg)
|
||||
|
||||
|
|
@ -789,6 +812,18 @@ class SFTTrainerWrapper:
|
|||
)
|
||||
)
|
||||
|
||||
# ReLoRA callback (v0.39.0 Part B) — magnitude-prune LoRA weights every N steps
|
||||
relora_steps = getattr(self.config.training, "relora_steps", None)
|
||||
if relora_steps:
|
||||
from soup_cli.utils.relora import ReLoRACallback, ReLoRAPolicy
|
||||
policy = ReLoRAPolicy(
|
||||
steps=int(relora_steps),
|
||||
warmup_ratio=float(self.config.training.relora_warmup_ratio),
|
||||
reset_optimizer=bool(self.config.training.relora_reset_optimizer),
|
||||
prune_ratio=float(self.config.training.relora_prune_ratio),
|
||||
)
|
||||
self.trainer.add_callback(ReLoRACallback(policy=policy))
|
||||
|
||||
# Activation offloading (v0.28.0) — wrap train() so saved-tensor hooks
|
||||
# are active only during training (and removed afterwards).
|
||||
from soup_cli.utils.activation_offload import offload_context
|
||||
|
|
|
|||
|
|
@ -50,7 +50,18 @@ def build_peft_config(
|
|||
"use_rslora": lora_cfg.use_rslora,
|
||||
}
|
||||
|
||||
if lora_cfg.use_olora:
|
||||
# v0.39.0 Part C — per-pattern rank/alpha (peft natively supports these)
|
||||
if lora_cfg.rank_pattern:
|
||||
init_kwargs["rank_pattern"] = dict(lora_cfg.rank_pattern)
|
||||
if lora_cfg.alpha_pattern:
|
||||
init_kwargs["alpha_pattern"] = dict(lora_cfg.alpha_pattern)
|
||||
|
||||
# init_strategy is the canonical source; use_olora is back-compat (validator
|
||||
# aligns init_strategy='olora' when use_olora=True).
|
||||
if lora_cfg.init_strategy in ("pissa", "olora"):
|
||||
init_kwargs["init_lora_weights"] = lora_cfg.init_strategy
|
||||
elif lora_cfg.use_olora:
|
||||
# Defensive fallback if init_strategy alignment was bypassed.
|
||||
init_kwargs["init_lora_weights"] = "olora"
|
||||
|
||||
return {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,151 @@
|
|||
"""Surgical PEFT/architecture patches (v0.39.0 Part D).
|
||||
|
||||
Logging is via the standard ``logging`` module so failures are inspectable
|
||||
without raising. Best-effort patches never crash training.
|
||||
|
||||
|
||||
Two narrow patches that PEFT upstream doesn't (yet) handle:
|
||||
|
||||
1. **Gemma4 ``ClippableLinear``** — Gemma 4 uses a ``ClippableLinear`` subclass
|
||||
that PEFT's ``LoraConfig.target_modules`` matcher doesn't recognise. We
|
||||
detect it by class name and swap to plain ``nn.Linear`` so PEFT's normal
|
||||
matcher takes over. We don't try to preserve the clipping semantics
|
||||
because Gemma4 only invokes them at inference; training is unaffected.
|
||||
|
||||
2. **Fused-MoE 3-D expert weights** — ``ParamWrapper`` in PEFT crashes when a
|
||||
LoRA layer wraps a 3-D weight tensor (``[num_experts, in, out]``). We
|
||||
detect 3-D LoRA target modules and silently strip ``lora_dropout`` since
|
||||
the dropout layer is what triggers the crash.
|
||||
|
||||
Both patches are version-gated and architecture-gated. Apply via the
|
||||
public entry point :func:`apply_surgical_patches` which inspects the
|
||||
model name and runs only the patches that match.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Word-boundary match against ``model_name`` (case-insensitive). Bare substring
|
||||
# would over-match on names like "ungemma4ed" or "my-gemma4ish". Boundaries:
|
||||
# start/end-of-string or a non-alnum/underscore char on each side.
|
||||
_GEMMA4_RE = re.compile(r"(?:^|[^a-z0-9])gemma-?4(?:[^a-z0-9]|$)", re.IGNORECASE)
|
||||
|
||||
|
||||
def is_gemma4_model(model_name: Any) -> bool:
|
||||
if not isinstance(model_name, str) or not model_name:
|
||||
return False
|
||||
if "\x00" in model_name:
|
||||
return False
|
||||
return _GEMMA4_RE.search(model_name) is not None
|
||||
|
||||
|
||||
def _looks_like_clippable_linear(module: Any) -> bool:
|
||||
"""Match by class name, not isinstance, so we don't import gemma internals."""
|
||||
return type(module).__name__ == "ClippableLinear"
|
||||
|
||||
|
||||
def apply_gemma4_clippable_patch(model: Any) -> int:
|
||||
"""Swap any ``ClippableLinear`` submodules with plain ``nn.Linear``.
|
||||
|
||||
Returns the number of modules patched. Safe to call when no
|
||||
``ClippableLinear`` is present (returns 0).
|
||||
"""
|
||||
import torch.nn as nn # lazy
|
||||
|
||||
if model is None:
|
||||
return 0
|
||||
|
||||
swapped = 0
|
||||
# Walk parents → swap by attribute. Avoid mutation during iteration:
|
||||
# collect first, then patch.
|
||||
targets: list[tuple[Any, str, Any]] = []
|
||||
for parent in model.modules():
|
||||
for child_name, child in list(parent.named_children()):
|
||||
if _looks_like_clippable_linear(child):
|
||||
targets.append((parent, child_name, child))
|
||||
|
||||
for parent, child_name, child in targets:
|
||||
# Build a plain Linear with the same shape + dtype + device.
|
||||
in_features = getattr(child, "in_features", None)
|
||||
out_features = getattr(child, "out_features", None)
|
||||
bias = getattr(child, "bias", None) is not None
|
||||
if in_features is None or out_features is None:
|
||||
continue
|
||||
replacement = nn.Linear(in_features, out_features, bias=bias)
|
||||
# Copy weights over (best-effort).
|
||||
try:
|
||||
replacement.weight.data.copy_(child.weight.data)
|
||||
if bias and replacement.bias is not None and child.bias is not None:
|
||||
replacement.bias.data.copy_(child.bias.data)
|
||||
replacement = replacement.to(
|
||||
dtype=child.weight.dtype, device=child.weight.device
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 — fall back to random init w/ log
|
||||
logger.debug(
|
||||
"Failed to copy ClippableLinear weights at %s: %s; using fresh init",
|
||||
child_name, exc,
|
||||
)
|
||||
setattr(parent, child_name, replacement)
|
||||
swapped += 1
|
||||
return swapped
|
||||
|
||||
|
||||
def strip_lora_dropout_for_3d_experts(peft_model: Any) -> int:
|
||||
"""Zero out ``lora_dropout`` on any LoRA target whose base weight is 3-D.
|
||||
|
||||
PEFT's ``ParamWrapper`` for fused-MoE experts cannot wrap an
|
||||
``nn.Dropout`` instance whose forward expects a 2-D tensor when the
|
||||
expert weight is shaped ``[num_experts, in, out]``. Setting ``p=0.0``
|
||||
is the documented workaround upstream (Axolotl ``adapter.py``).
|
||||
|
||||
Returns the number of modules whose dropout was disabled.
|
||||
"""
|
||||
if peft_model is None:
|
||||
return 0
|
||||
count = 0
|
||||
for _name, module in peft_model.named_modules():
|
||||
weight = getattr(module, "weight", None)
|
||||
if weight is None:
|
||||
continue
|
||||
ndim = getattr(weight, "ndim", None)
|
||||
if ndim != 3:
|
||||
continue
|
||||
dropout = getattr(module, "lora_dropout", None)
|
||||
if dropout is None:
|
||||
continue
|
||||
# dropout may be an nn.Dropout, an nn.ModuleDict (PEFT >=0.10), or a
|
||||
# plain attribute with a ``p`` field. Cover the common shapes.
|
||||
try:
|
||||
if hasattr(dropout, "p"):
|
||||
dropout.p = 0.0
|
||||
count += 1
|
||||
elif hasattr(dropout, "values"):
|
||||
for sub in dropout.values():
|
||||
if hasattr(sub, "p"):
|
||||
sub.p = 0.0
|
||||
count += 1
|
||||
except Exception:
|
||||
continue
|
||||
return count
|
||||
|
||||
|
||||
def apply_surgical_patches(model: Any, model_name: str) -> dict[str, int]:
|
||||
"""Run all gated PEFT/architecture patches for ``model``.
|
||||
|
||||
Returns a dict ``{"gemma4_clippable": int, "moe_3d_dropout": int}`` —
|
||||
counts of modules patched.
|
||||
"""
|
||||
if not isinstance(model_name, str) or not model_name:
|
||||
raise ValueError("model_name must be a non-empty string")
|
||||
if "\x00" in model_name:
|
||||
raise ValueError("model_name cannot contain null bytes")
|
||||
counts = {"gemma4_clippable": 0, "moe_3d_dropout": 0}
|
||||
if is_gemma4_model(model_name):
|
||||
counts["gemma4_clippable"] = apply_gemma4_clippable_patch(model)
|
||||
counts["moe_3d_dropout"] = strip_lora_dropout_for_3d_experts(model)
|
||||
return counts
|
||||
|
|
@ -0,0 +1,177 @@
|
|||
"""ReLoRA — periodic LoRA adapter magnitude-prune + optimizer reset.
|
||||
|
||||
Mirrors the technique described in the ReLoRA paper / Axolotl
|
||||
``monkeypatch/relora.py``: every N steps, magnitude-prune the LoRA
|
||||
adapter weights and reset the optimizer state. Useful for very long
|
||||
training runs where the LoRA capacity saturates.
|
||||
|
||||
The callback is a no-op when ``policy`` is ``None``. Pass a
|
||||
:class:`ReLoRAPolicy` to enable it. Lazy imports keep the module CLI-fast.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Optional
|
||||
|
||||
# Cap relora_steps to a sane upper bound.
|
||||
MAX_RELORA_STEPS = 10**7
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReLoRAPolicy:
|
||||
"""Frozen policy for the ReLoRA callback.
|
||||
|
||||
Attributes:
|
||||
steps: fire every N global steps (must be > 0).
|
||||
warmup_ratio: fraction of total steps to skip at the start ([0, 1]).
|
||||
reset_optimizer: if True, clears optimizer state for the pruned
|
||||
LoRA parameters after pruning so momentum doesn't fight the
|
||||
new sparse weights.
|
||||
prune_ratio: fraction of LoRA weights to zero out, by magnitude
|
||||
(0 < x <= 1; e.g. 0.9 keeps the top 10%).
|
||||
"""
|
||||
|
||||
steps: int
|
||||
warmup_ratio: float = 0.1
|
||||
reset_optimizer: bool = True
|
||||
prune_ratio: float = 0.9
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if not isinstance(self.steps, int) or isinstance(self.steps, bool):
|
||||
raise ValueError("ReLoRAPolicy.steps must be int")
|
||||
if self.steps <= 0 or self.steps > MAX_RELORA_STEPS:
|
||||
raise ValueError(
|
||||
f"ReLoRAPolicy.steps must be in (0, {MAX_RELORA_STEPS}], got {self.steps}"
|
||||
)
|
||||
if not (0.0 <= self.warmup_ratio <= 1.0):
|
||||
raise ValueError(
|
||||
f"ReLoRAPolicy.warmup_ratio must be in [0, 1], got {self.warmup_ratio}"
|
||||
)
|
||||
# Mirror magnitude_prune_tensor's strict (0, 1) bound. prune_ratio=1.0
|
||||
# would zero every weight on first fire — that's a footgun, not a feature.
|
||||
if not (0.0 < self.prune_ratio < 1.0):
|
||||
raise ValueError(
|
||||
f"ReLoRAPolicy.prune_ratio must be in (0, 1), got {self.prune_ratio}"
|
||||
)
|
||||
|
||||
def should_fire(self, global_step: int, total_steps: Optional[int] = None) -> bool:
|
||||
if global_step <= 0:
|
||||
return False
|
||||
if global_step % self.steps != 0:
|
||||
return False
|
||||
if total_steps is not None and total_steps > 0:
|
||||
warmup_cutoff = int(total_steps * self.warmup_ratio)
|
||||
if global_step < warmup_cutoff:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def magnitude_prune_tensor(tensor: Any, prune_ratio: float) -> Any:
|
||||
"""Zero out the smallest-magnitude entries of ``tensor`` in place.
|
||||
|
||||
``prune_ratio=0.9`` keeps the top 10% of weights by absolute value.
|
||||
``prune_ratio`` of exactly 0.0 or 1.0 is rejected to avoid silent
|
||||
no-ops or wholesale zeroing.
|
||||
"""
|
||||
if not (0.0 < prune_ratio < 1.0):
|
||||
raise ValueError(
|
||||
f"magnitude_prune_tensor prune_ratio must be in (0, 1), got {prune_ratio}"
|
||||
)
|
||||
import torch # lazy
|
||||
|
||||
if not isinstance(tensor, torch.Tensor):
|
||||
raise TypeError(f"magnitude_prune_tensor expects torch.Tensor, got {type(tensor)}")
|
||||
|
||||
flat = tensor.detach().abs().reshape(-1)
|
||||
# Empty / 1-element tensor: nothing meaningful to prune.
|
||||
if flat.numel() <= 1:
|
||||
return tensor
|
||||
k = max(1, int(flat.numel() * prune_ratio))
|
||||
if k >= flat.numel():
|
||||
# keep the single largest
|
||||
k = flat.numel() - 1
|
||||
# The k-th smallest absolute value: anything <= it gets zeroed
|
||||
threshold = torch.kthvalue(flat, k).values
|
||||
mask = tensor.detach().abs() > threshold
|
||||
tensor.detach().mul_(mask.to(tensor.dtype))
|
||||
return tensor
|
||||
|
||||
|
||||
def _is_lora_param_name(name: str) -> bool:
|
||||
"""Match PEFT's lora_A / lora_B parameter naming."""
|
||||
return ("lora_A" in name) or ("lora_B" in name)
|
||||
|
||||
|
||||
class ReLoRACallback:
|
||||
"""HF TrainerCallback that magnitude-prunes LoRA weights every N steps.
|
||||
|
||||
We don't subclass ``transformers.TrainerCallback`` here so importing
|
||||
this module never loads transformers. The Trainer's callback dispatch
|
||||
works structurally — any object with the right method names is fine.
|
||||
"""
|
||||
|
||||
def __init__(self, policy: Optional[ReLoRAPolicy], console: Any = None) -> None:
|
||||
self.policy = policy
|
||||
self.console = console
|
||||
self.fire_count = 0
|
||||
|
||||
def on_step_end(
|
||||
self,
|
||||
args: Any,
|
||||
state: Any,
|
||||
control: Any,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
if self.policy is None:
|
||||
return control
|
||||
global_step = int(getattr(state, "global_step", 0) or 0)
|
||||
total_steps = getattr(state, "max_steps", None)
|
||||
try:
|
||||
total_steps_int = int(total_steps) if total_steps else None
|
||||
except (TypeError, ValueError):
|
||||
total_steps_int = None
|
||||
if not self.policy.should_fire(global_step, total_steps_int):
|
||||
return control
|
||||
model = kwargs.get("model")
|
||||
optimizer = kwargs.get("optimizer")
|
||||
if model is None:
|
||||
return control
|
||||
self._prune_and_reset(model, optimizer)
|
||||
self.fire_count += 1
|
||||
if self.console is not None:
|
||||
try:
|
||||
self.console.print(
|
||||
f"[yellow]ReLoRA[/yellow] fired at step {global_step} "
|
||||
f"(prune_ratio={self.policy.prune_ratio})"
|
||||
)
|
||||
except Exception: # noqa: BLE001 — console.print is best-effort in callback
|
||||
pass
|
||||
return control
|
||||
|
||||
def _prune_and_reset(self, model: Any, optimizer: Any) -> None:
|
||||
import torch # lazy
|
||||
|
||||
pruned_params = []
|
||||
for name, param in model.named_parameters():
|
||||
if not _is_lora_param_name(name):
|
||||
continue
|
||||
if param.requires_grad and param.numel() > 0:
|
||||
with torch.no_grad():
|
||||
magnitude_prune_tensor(param.data, self.policy.prune_ratio)
|
||||
pruned_params.append(param)
|
||||
|
||||
if optimizer is None or not self.policy.reset_optimizer:
|
||||
return
|
||||
# Reset optimizer state only for the pruned parameters.
|
||||
try:
|
||||
state = getattr(optimizer, "state", None)
|
||||
if state is None:
|
||||
return
|
||||
for param in pruned_params:
|
||||
if param in state:
|
||||
state[param] = type(state[param])() if state[param] else {}
|
||||
except Exception:
|
||||
# Optimizer state structure varies (DeepSpeed / FSDP wrap it);
|
||||
# silent best-effort is the documented Axolotl behaviour too.
|
||||
return
|
||||
|
|
@ -0,0 +1,197 @@
|
|||
"""Tests for v0.39.0 Part D — surgical PEFT patches.
|
||||
|
||||
Covers detection helpers + gated patch entry points for:
|
||||
- Gemma4 ``ClippableLinear`` (PEFT's LoRA layer registry doesn't know about it)
|
||||
- Fused-MoE 3-D expert weights (PEFT's ParamWrapper crashes on dropout)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
# --- Gemma4 ClippableLinear detection ---------------------------------------
|
||||
|
||||
|
||||
class TestGemma4Detection:
|
||||
def test_is_gemma4_positive_lower(self):
|
||||
from soup_cli.utils.peft_patches import is_gemma4_model
|
||||
assert is_gemma4_model("google/gemma-4-9b") is True
|
||||
assert is_gemma4_model("google/gemma-4-it") is True
|
||||
assert is_gemma4_model("Gemma-4-2B") is True
|
||||
|
||||
def test_is_gemma4_negative(self):
|
||||
from soup_cli.utils.peft_patches import is_gemma4_model
|
||||
assert is_gemma4_model("google/gemma-2-9b") is False
|
||||
assert is_gemma4_model("meta-llama/Meta-Llama-3.1-8B") is False
|
||||
assert is_gemma4_model("") is False
|
||||
assert is_gemma4_model(None) is False # type: ignore
|
||||
|
||||
def test_is_gemma4_word_boundary(self):
|
||||
"""v0.39.0 security fix — substring match would over-match."""
|
||||
from soup_cli.utils.peft_patches import is_gemma4_model
|
||||
# NOT Gemma 4
|
||||
assert is_gemma4_model("ungemma4ed") is False
|
||||
assert is_gemma4_model("megagemma40-experiment") is False
|
||||
# IS Gemma 4 — word-boundary cases
|
||||
assert is_gemma4_model("my-org/finetuned-gemma-4-style") is True
|
||||
assert is_gemma4_model("google/gemma4_instruct") is True
|
||||
|
||||
def test_is_gemma4_rejects_null_byte(self):
|
||||
from soup_cli.utils.peft_patches import is_gemma4_model
|
||||
# crafted name with null byte should not match
|
||||
assert is_gemma4_model("gemma-4\x00malicious") is False
|
||||
|
||||
|
||||
class TestClippableLinearPatch:
|
||||
def test_no_clippable_linear_returns_zero(self):
|
||||
try:
|
||||
import torch.nn as nn
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.peft_patches import apply_gemma4_clippable_patch
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.fc = nn.Linear(4, 4)
|
||||
|
||||
model = Model()
|
||||
count = apply_gemma4_clippable_patch(model)
|
||||
assert count == 0
|
||||
|
||||
def test_clippable_linear_replaced(self):
|
||||
try:
|
||||
import torch.nn as nn
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.peft_patches import apply_gemma4_clippable_patch
|
||||
|
||||
# Simulate Gemma4's ClippableLinear by name
|
||||
class ClippableLinear(nn.Linear):
|
||||
pass
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.fc1 = ClippableLinear(4, 4)
|
||||
self.fc2 = nn.Linear(4, 4)
|
||||
|
||||
model = Model()
|
||||
count = apply_gemma4_clippable_patch(model)
|
||||
assert count == 1
|
||||
# After patch: fc1 is plain nn.Linear (or PEFT-recognised)
|
||||
assert type(model.fc1) is nn.Linear
|
||||
|
||||
|
||||
# --- MoE 3D expert dropout strip --------------------------------------------
|
||||
|
||||
|
||||
class TestMoE3DDropoutStrip:
|
||||
def test_strip_when_no_3d_experts(self):
|
||||
from soup_cli.utils.peft_patches import strip_lora_dropout_for_3d_experts
|
||||
# peft model with only 2-D weights — strip is no-op
|
||||
peft_model = MagicMock()
|
||||
peft_model.named_modules.return_value = [
|
||||
("base.layer1", MagicMock(weight=MagicMock(ndim=2))),
|
||||
]
|
||||
count = strip_lora_dropout_for_3d_experts(peft_model)
|
||||
assert count == 0
|
||||
|
||||
def test_strip_zeroes_dropout_on_3d_module(self):
|
||||
from soup_cli.utils.peft_patches import strip_lora_dropout_for_3d_experts
|
||||
# Build a fake module tree: experts.0.gate_proj has 3-D weight + lora_dropout
|
||||
expert = MagicMock()
|
||||
expert.weight = MagicMock(ndim=3)
|
||||
expert.lora_dropout = MagicMock()
|
||||
expert.lora_dropout.p = 0.1
|
||||
|
||||
peft_model = MagicMock()
|
||||
peft_model.named_modules.return_value = [
|
||||
("base.experts.0.gate_proj", expert),
|
||||
]
|
||||
count = strip_lora_dropout_for_3d_experts(peft_model)
|
||||
assert count == 1
|
||||
assert expert.lora_dropout.p == 0.0
|
||||
|
||||
def test_strip_handles_module_dict_dropout(self):
|
||||
"""PEFT >=0.10 wraps lora_dropout in a ModuleDict — exercise the values() branch."""
|
||||
from soup_cli.utils.peft_patches import strip_lora_dropout_for_3d_experts
|
||||
|
||||
sub_a = MagicMock(spec=["p"])
|
||||
sub_a.p = 0.1
|
||||
sub_b = MagicMock(spec=["p"])
|
||||
sub_b.p = 0.2
|
||||
|
||||
class FakeModuleDict:
|
||||
def values(self):
|
||||
return [sub_a, sub_b]
|
||||
|
||||
expert = MagicMock(spec=["weight", "lora_dropout"])
|
||||
expert.weight = MagicMock(ndim=3)
|
||||
# FakeModuleDict has no `p` attribute → hasattr() is False → elif fires.
|
||||
expert.lora_dropout = FakeModuleDict()
|
||||
|
||||
peft_model = MagicMock()
|
||||
peft_model.named_modules.return_value = [("base.experts.0", expert)]
|
||||
count = strip_lora_dropout_for_3d_experts(peft_model)
|
||||
assert count == 2
|
||||
assert sub_a.p == 0.0
|
||||
assert sub_b.p == 0.0
|
||||
|
||||
|
||||
class TestApplySurgicalPatches:
|
||||
def test_returns_dict_with_counts(self):
|
||||
try:
|
||||
import torch.nn as nn
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.peft_patches import apply_surgical_patches
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.fc = nn.Linear(4, 4)
|
||||
|
||||
result = apply_surgical_patches(Model(), model_name="meta-llama/Meta-Llama-3.1-8B")
|
||||
assert isinstance(result, dict)
|
||||
assert "gemma4_clippable" in result
|
||||
assert "moe_3d_dropout" in result
|
||||
assert result["gemma4_clippable"] == 0
|
||||
|
||||
def test_gemma4_only_runs_for_gemma4_models(self):
|
||||
try:
|
||||
import torch.nn as nn
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.peft_patches import apply_surgical_patches
|
||||
|
||||
class ClippableLinear(nn.Linear):
|
||||
pass
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.fc1 = ClippableLinear(4, 4)
|
||||
|
||||
# non-Gemma4: skip
|
||||
m = Model()
|
||||
result = apply_surgical_patches(m, model_name="meta-llama/Llama-3-8B")
|
||||
assert result["gemma4_clippable"] == 0
|
||||
assert type(m.fc1) is ClippableLinear
|
||||
|
||||
# Gemma4: apply
|
||||
m2 = Model()
|
||||
result2 = apply_surgical_patches(m2, model_name="google/gemma-4-9b")
|
||||
assert result2["gemma4_clippable"] == 1
|
||||
|
||||
def test_rejects_empty_model_name(self):
|
||||
from soup_cli.utils.peft_patches import apply_surgical_patches
|
||||
with pytest.raises(ValueError):
|
||||
apply_surgical_patches(MagicMock(), model_name="")
|
||||
|
||||
def test_rejects_null_byte_model_name(self):
|
||||
from soup_cli.utils.peft_patches import apply_surgical_patches
|
||||
with pytest.raises(ValueError):
|
||||
apply_surgical_patches(MagicMock(), model_name="gemma-4\x00x")
|
||||
|
|
@ -0,0 +1,133 @@
|
|||
"""Tests for v0.39.0 Part A — PiSSA init + init_strategy field on LoraConfig."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from soup_cli.config.schema import LoraConfig
|
||||
from soup_cli.utils.peft_builder import build_peft_config
|
||||
|
||||
|
||||
class TestInitStrategyField:
|
||||
def test_default_is_random(self):
|
||||
cfg = LoraConfig()
|
||||
assert cfg.init_strategy == "random"
|
||||
|
||||
def test_accepts_pissa(self):
|
||||
cfg = LoraConfig(init_strategy="pissa")
|
||||
assert cfg.init_strategy == "pissa"
|
||||
|
||||
def test_accepts_olora(self):
|
||||
cfg = LoraConfig(init_strategy="olora")
|
||||
assert cfg.init_strategy == "olora"
|
||||
|
||||
def test_rejects_unknown(self):
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(init_strategy="loftq")
|
||||
|
||||
def test_rejects_non_string(self):
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(init_strategy=42)
|
||||
|
||||
|
||||
class TestBackcompatNoMutation:
|
||||
def test_backcompat_does_not_mutate_input_dict(self):
|
||||
"""v0.39.0 security fix — _backcompat_align_olora must copy."""
|
||||
original = {"use_olora": True}
|
||||
LoraConfig(**original)
|
||||
assert "init_strategy" not in original, (
|
||||
"Validator mutated caller's dict in-place"
|
||||
)
|
||||
|
||||
|
||||
class TestInitStrategyOloraBackcompat:
|
||||
def test_use_olora_true_alone_still_works(self):
|
||||
cfg = LoraConfig(use_olora=True)
|
||||
assert cfg.use_olora is True
|
||||
assert cfg.init_strategy == "olora" # auto-aligned for back-compat
|
||||
|
||||
def test_init_strategy_olora_alone_works(self):
|
||||
cfg = LoraConfig(init_strategy="olora")
|
||||
assert cfg.init_strategy == "olora"
|
||||
# use_olora may be True or False; effect is via init_strategy
|
||||
|
||||
def test_both_set_consistent_ok(self):
|
||||
cfg = LoraConfig(use_olora=True, init_strategy="olora")
|
||||
assert cfg.init_strategy == "olora"
|
||||
|
||||
def test_use_olora_true_with_pissa_rejected(self):
|
||||
with pytest.raises(ValidationError, match="init_strategy"):
|
||||
LoraConfig(use_olora=True, init_strategy="pissa")
|
||||
|
||||
def test_use_olora_true_with_random_rejected(self):
|
||||
# explicit conflict — user said both random and olora; loud-fail
|
||||
with pytest.raises(ValidationError, match="init_strategy"):
|
||||
LoraConfig(use_olora=True, init_strategy="random")
|
||||
|
||||
|
||||
class TestInitStrategyMutualExclusion:
|
||||
def test_pissa_with_dora_rejected(self):
|
||||
# PiSSA + DoRA isn't supported (PEFT init_lora_weights conflicts with use_dora init)
|
||||
with pytest.raises(ValidationError, match="init_strategy"):
|
||||
LoraConfig(use_dora=True, init_strategy="pissa")
|
||||
|
||||
def test_pissa_with_vera_rejected(self):
|
||||
# VeRA doesn't have init_lora_weights — PiSSA meaningless
|
||||
with pytest.raises(ValidationError, match="init_strategy"):
|
||||
LoraConfig(use_vera=True, init_strategy="pissa")
|
||||
|
||||
def test_pissa_with_rslora_ok(self):
|
||||
# rsLoRA only changes scaling factor — orthogonal to init
|
||||
cfg = LoraConfig(use_rslora=True, init_strategy="pissa")
|
||||
assert cfg.init_strategy == "pissa"
|
||||
assert cfg.use_rslora is True
|
||||
|
||||
|
||||
class TestPeftBuilderInitStrategy:
|
||||
def test_random_does_not_set_init_lora_weights(self):
|
||||
cfg = LoraConfig(init_strategy="random")
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert spec["peft_cls"] == "LoraConfig"
|
||||
assert "init_lora_weights" not in spec["init_kwargs"]
|
||||
|
||||
def test_pissa_sets_init_lora_weights(self):
|
||||
cfg = LoraConfig(init_strategy="pissa")
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert spec["init_kwargs"]["init_lora_weights"] == "pissa"
|
||||
|
||||
def test_olora_via_init_strategy(self):
|
||||
cfg = LoraConfig(init_strategy="olora")
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert spec["init_kwargs"]["init_lora_weights"] == "olora"
|
||||
|
||||
def test_olora_via_use_olora_legacy(self):
|
||||
cfg = LoraConfig(use_olora=True)
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert spec["init_kwargs"]["init_lora_weights"] == "olora"
|
||||
|
||||
def test_vera_ignores_init_strategy_random(self):
|
||||
cfg = LoraConfig(use_vera=True, init_strategy="random")
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert spec["peft_cls"] == "VeraConfig"
|
||||
assert "init_lora_weights" not in spec["init_kwargs"]
|
||||
|
||||
|
||||
class TestInstantiatePeftConfig:
|
||||
def test_instantiate_lora_config(self):
|
||||
from soup_cli.utils.peft_builder import instantiate_peft_config
|
||||
cfg = LoraConfig(init_strategy="pissa")
|
||||
spec = build_peft_config(cfg, target_modules=["q_proj", "v_proj"], task_type="CAUSAL_LM")
|
||||
result = instantiate_peft_config(spec)
|
||||
# Verify it really is a peft.LoraConfig with the right kwargs
|
||||
import peft
|
||||
assert isinstance(result, peft.LoraConfig)
|
||||
assert result.r == 64
|
||||
assert result.init_lora_weights == "pissa"
|
||||
|
||||
def test_instantiate_with_rank_pattern(self):
|
||||
from soup_cli.utils.peft_builder import instantiate_peft_config
|
||||
cfg = LoraConfig(rank_pattern={"q_proj": 8})
|
||||
spec = build_peft_config(cfg, target_modules=["q_proj"], task_type="CAUSAL_LM")
|
||||
result = instantiate_peft_config(spec)
|
||||
assert result.rank_pattern == {"q_proj": 8}
|
||||
|
|
@ -0,0 +1,98 @@
|
|||
"""Tests for v0.39.0 Part C — per-pattern LoRA rank/alpha."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from soup_cli.config.schema import LoraConfig
|
||||
from soup_cli.utils.peft_builder import build_peft_config
|
||||
|
||||
|
||||
class TestRankPatternSchema:
|
||||
def test_default_none(self):
|
||||
cfg = LoraConfig()
|
||||
assert cfg.rank_pattern is None
|
||||
assert cfg.alpha_pattern is None
|
||||
|
||||
def test_rank_pattern_dict_accepted(self):
|
||||
cfg = LoraConfig(rank_pattern={"q_proj": 8, "v_proj": 16})
|
||||
assert cfg.rank_pattern == {"q_proj": 8, "v_proj": 16}
|
||||
|
||||
def test_alpha_pattern_dict_accepted(self):
|
||||
cfg = LoraConfig(alpha_pattern={"q_proj": 16, "v_proj": 32})
|
||||
assert cfg.alpha_pattern == {"q_proj": 16, "v_proj": 32}
|
||||
|
||||
def test_rank_pattern_rejects_non_int_value(self):
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(rank_pattern={"q_proj": "high"})
|
||||
|
||||
def test_rank_pattern_rejects_negative(self):
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(rank_pattern={"q_proj": -1})
|
||||
|
||||
def test_rank_pattern_rejects_zero(self):
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(rank_pattern={"q_proj": 0})
|
||||
|
||||
def test_rank_pattern_rejects_too_large(self):
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(rank_pattern={"q_proj": 10_000})
|
||||
|
||||
def test_rank_pattern_rejects_empty_key(self):
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(rank_pattern={"": 8})
|
||||
|
||||
def test_rank_pattern_rejects_null_byte_key(self):
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(rank_pattern={"q\x00proj": 8})
|
||||
|
||||
def test_rank_pattern_rejects_too_many_keys(self):
|
||||
# Cap at 256 patterns to prevent absurd configs
|
||||
big = {f"k{i}": 8 for i in range(257)}
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(rank_pattern=big)
|
||||
|
||||
def test_rank_pattern_rejects_bool_value(self):
|
||||
# bool is subclass of int in Python — exclude explicitly
|
||||
with pytest.raises(ValidationError):
|
||||
LoraConfig(rank_pattern={"q_proj": True})
|
||||
|
||||
|
||||
class TestRankPatternMutualExclusion:
|
||||
def test_rank_pattern_with_vera_rejected(self):
|
||||
with pytest.raises(ValidationError, match="rank_pattern"):
|
||||
LoraConfig(use_vera=True, rank_pattern={"q_proj": 8})
|
||||
|
||||
def test_alpha_pattern_with_vera_rejected(self):
|
||||
with pytest.raises(ValidationError, match="alpha_pattern"):
|
||||
LoraConfig(use_vera=True, alpha_pattern={"q_proj": 16})
|
||||
|
||||
def test_rank_pattern_with_dora_ok(self):
|
||||
# DoRA still uses standard LoraConfig; rank_pattern works
|
||||
cfg = LoraConfig(use_dora=True, rank_pattern={"q_proj": 8})
|
||||
assert cfg.rank_pattern == {"q_proj": 8}
|
||||
|
||||
|
||||
class TestPeftBuilderRankPattern:
|
||||
def test_rank_pattern_propagated(self):
|
||||
cfg = LoraConfig(rank_pattern={"q_proj": 8, "v_proj": 16})
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert spec["init_kwargs"]["rank_pattern"] == {"q_proj": 8, "v_proj": 16}
|
||||
|
||||
def test_alpha_pattern_propagated(self):
|
||||
cfg = LoraConfig(alpha_pattern={"q_proj": 16})
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert spec["init_kwargs"]["alpha_pattern"] == {"q_proj": 16}
|
||||
|
||||
def test_neither_pattern_omitted_when_none(self):
|
||||
cfg = LoraConfig()
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert "rank_pattern" not in spec["init_kwargs"]
|
||||
assert "alpha_pattern" not in spec["init_kwargs"]
|
||||
|
||||
def test_vera_path_ignores_patterns_when_unset(self):
|
||||
cfg = LoraConfig(use_vera=True)
|
||||
spec = build_peft_config(cfg, target_modules="auto", task_type="CAUSAL_LM")
|
||||
assert spec["peft_cls"] == "VeraConfig"
|
||||
assert "rank_pattern" not in spec["init_kwargs"]
|
||||
|
|
@ -0,0 +1,359 @@
|
|||
"""Tests for v0.39.0 Part B — ReLoRA callback."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from soup_cli.config.schema import TrainingConfig
|
||||
|
||||
|
||||
class TestReLoRASchema:
|
||||
def test_default_disabled(self):
|
||||
cfg = TrainingConfig()
|
||||
assert cfg.relora_steps is None
|
||||
assert cfg.relora_warmup_ratio == 0.1
|
||||
assert cfg.relora_reset_optimizer is True
|
||||
assert 0.0 < cfg.relora_prune_ratio <= 1.0
|
||||
|
||||
def test_relora_steps_positive(self):
|
||||
cfg = TrainingConfig(relora_steps=500)
|
||||
assert cfg.relora_steps == 500
|
||||
|
||||
def test_relora_steps_rejects_zero(self):
|
||||
with pytest.raises(ValidationError):
|
||||
TrainingConfig(relora_steps=0)
|
||||
|
||||
def test_relora_steps_rejects_negative(self):
|
||||
with pytest.raises(ValidationError):
|
||||
TrainingConfig(relora_steps=-1)
|
||||
|
||||
def test_relora_steps_upper_bound(self):
|
||||
# Cap at 10**7 to prevent overflow / nonsensical values
|
||||
with pytest.raises(ValidationError):
|
||||
TrainingConfig(relora_steps=10**8)
|
||||
|
||||
def test_relora_warmup_ratio_bounds(self):
|
||||
TrainingConfig(relora_warmup_ratio=0.0)
|
||||
TrainingConfig(relora_warmup_ratio=1.0)
|
||||
with pytest.raises(ValidationError):
|
||||
TrainingConfig(relora_warmup_ratio=-0.01)
|
||||
with pytest.raises(ValidationError):
|
||||
TrainingConfig(relora_warmup_ratio=1.01)
|
||||
|
||||
def test_relora_prune_ratio_bounds(self):
|
||||
TrainingConfig(relora_prune_ratio=0.5)
|
||||
TrainingConfig(relora_prune_ratio=0.99)
|
||||
with pytest.raises(ValidationError):
|
||||
TrainingConfig(relora_prune_ratio=0.0)
|
||||
with pytest.raises(ValidationError):
|
||||
TrainingConfig(relora_prune_ratio=1.0)
|
||||
with pytest.raises(ValidationError):
|
||||
TrainingConfig(relora_prune_ratio=1.01)
|
||||
|
||||
|
||||
class TestReLoRAPolicy:
|
||||
def test_policy_frozen(self):
|
||||
from dataclasses import FrozenInstanceError
|
||||
|
||||
from soup_cli.utils.relora import ReLoRAPolicy
|
||||
p = ReLoRAPolicy(steps=500, warmup_ratio=0.1, reset_optimizer=True, prune_ratio=0.9)
|
||||
with pytest.raises(FrozenInstanceError):
|
||||
p.steps = 999 # type: ignore
|
||||
|
||||
def test_policy_should_fire_step_zero_no(self):
|
||||
from soup_cli.utils.relora import ReLoRAPolicy
|
||||
p = ReLoRAPolicy(steps=500)
|
||||
assert p.should_fire(global_step=0) is False
|
||||
|
||||
def test_policy_should_fire_at_multiple(self):
|
||||
from soup_cli.utils.relora import ReLoRAPolicy
|
||||
p = ReLoRAPolicy(steps=500)
|
||||
assert p.should_fire(global_step=500) is True
|
||||
assert p.should_fire(global_step=1000) is True
|
||||
|
||||
def test_policy_should_fire_skips_warmup(self):
|
||||
from soup_cli.utils.relora import ReLoRAPolicy
|
||||
# warmup_ratio=0.5 over total 1000 = first 500 skipped
|
||||
p = ReLoRAPolicy(steps=200, warmup_ratio=0.5)
|
||||
assert p.should_fire(global_step=200, total_steps=1000) is False
|
||||
assert p.should_fire(global_step=600, total_steps=1000) is True
|
||||
|
||||
def test_policy_rejects_invalid_steps(self):
|
||||
from soup_cli.utils.relora import ReLoRAPolicy
|
||||
with pytest.raises(ValueError):
|
||||
ReLoRAPolicy(steps=0)
|
||||
with pytest.raises(ValueError):
|
||||
ReLoRAPolicy(steps=-1)
|
||||
|
||||
def test_policy_rejects_invalid_prune_ratio(self):
|
||||
from soup_cli.utils.relora import ReLoRAPolicy
|
||||
with pytest.raises(ValueError):
|
||||
ReLoRAPolicy(steps=500, prune_ratio=0.0)
|
||||
with pytest.raises(ValueError):
|
||||
ReLoRAPolicy(steps=500, prune_ratio=1.0)
|
||||
with pytest.raises(ValueError):
|
||||
ReLoRAPolicy(steps=500, prune_ratio=1.5)
|
||||
|
||||
|
||||
class TestMagnitudePrune:
|
||||
def test_magnitude_prune_zeroes_low_magnitude(self):
|
||||
try:
|
||||
import torch
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.relora import magnitude_prune_tensor
|
||||
|
||||
x = torch.tensor([0.01, 0.02, 0.5, 1.0, 2.0])
|
||||
# prune_ratio=0.6 → keep top 40% (2 of 5) → smallest 3 zeroed
|
||||
out = magnitude_prune_tensor(x.clone(), prune_ratio=0.6)
|
||||
nonzero = (out != 0).sum().item()
|
||||
assert nonzero == 2
|
||||
# the two largest must survive
|
||||
assert (out.abs() == 2.0).any()
|
||||
assert (out.abs() == 1.0).any()
|
||||
|
||||
def test_magnitude_prune_single_element_no_crash(self):
|
||||
try:
|
||||
import torch
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.relora import magnitude_prune_tensor
|
||||
|
||||
# 1-element tensor — kthvalue(_, 0) would raise; helper must short-circuit.
|
||||
x = torch.tensor([3.14])
|
||||
out = magnitude_prune_tensor(x.clone(), prune_ratio=0.5)
|
||||
# untouched (use approx for float32 storage)
|
||||
assert out.item() == pytest.approx(3.14, abs=1e-5)
|
||||
|
||||
def test_magnitude_prune_keep_all(self):
|
||||
try:
|
||||
import torch
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.relora import magnitude_prune_tensor
|
||||
|
||||
x = torch.tensor([1.0, 2.0, 3.0])
|
||||
out = magnitude_prune_tensor(x.clone(), prune_ratio=0.001)
|
||||
# near-zero prune ratio → at least one element kept
|
||||
assert (out != 0).any()
|
||||
|
||||
def test_magnitude_prune_rejects_invalid_ratio(self):
|
||||
try:
|
||||
import torch
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.relora import magnitude_prune_tensor
|
||||
x = torch.zeros(3)
|
||||
with pytest.raises(ValueError):
|
||||
magnitude_prune_tensor(x, prune_ratio=0.0)
|
||||
with pytest.raises(ValueError):
|
||||
magnitude_prune_tensor(x, prune_ratio=1.0)
|
||||
|
||||
def test_magnitude_prune_rejects_non_tensor_input(self):
|
||||
try:
|
||||
import torch # noqa: F401
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.relora import magnitude_prune_tensor
|
||||
with pytest.raises(TypeError):
|
||||
magnitude_prune_tensor([1.0, 2.0, 3.0], prune_ratio=0.5)
|
||||
with pytest.raises(TypeError):
|
||||
magnitude_prune_tensor("not a tensor", prune_ratio=0.5)
|
||||
|
||||
|
||||
class TestReLoRACallback:
|
||||
def test_callback_disabled_no_op(self):
|
||||
from soup_cli.utils.relora import ReLoRACallback
|
||||
cb = ReLoRACallback(policy=None)
|
||||
# disabled callback never fires
|
||||
state = MagicMock(global_step=500, max_steps=1000)
|
||||
ctrl = MagicMock()
|
||||
args = MagicMock()
|
||||
cb.on_step_end(args, state, ctrl)
|
||||
assert cb.fire_count == 0
|
||||
|
||||
def test_callback_fires_on_relora_step(self):
|
||||
from soup_cli.utils.relora import ReLoRACallback, ReLoRAPolicy
|
||||
cb = ReLoRACallback(policy=ReLoRAPolicy(steps=100))
|
||||
# mock model with PEFT-style lora_A / lora_B parameters
|
||||
cb._prune_and_reset = MagicMock() # type: ignore[method-assign]
|
||||
state = MagicMock(global_step=100, max_steps=1000)
|
||||
ctrl = MagicMock()
|
||||
args = MagicMock()
|
||||
cb.on_step_end(args, state, ctrl, model=MagicMock(), optimizer=MagicMock())
|
||||
assert cb.fire_count == 1
|
||||
cb._prune_and_reset.assert_called_once()
|
||||
|
||||
def test_callback_does_not_fire_off_step(self):
|
||||
from soup_cli.utils.relora import ReLoRACallback, ReLoRAPolicy
|
||||
cb = ReLoRACallback(policy=ReLoRAPolicy(steps=100))
|
||||
cb._prune_and_reset = MagicMock() # type: ignore[method-assign]
|
||||
state = MagicMock(global_step=99, max_steps=1000)
|
||||
cb.on_step_end(MagicMock(), state, MagicMock())
|
||||
assert cb.fire_count == 0
|
||||
cb._prune_and_reset.assert_not_called()
|
||||
|
||||
def test_callback_skips_warmup(self):
|
||||
from soup_cli.utils.relora import ReLoRACallback, ReLoRAPolicy
|
||||
cb = ReLoRACallback(policy=ReLoRAPolicy(steps=100, warmup_ratio=0.5))
|
||||
cb._prune_and_reset = MagicMock() # type: ignore[method-assign]
|
||||
# at step 100 with total 1000 → warmup is 500 → skip
|
||||
state = MagicMock(global_step=100, max_steps=1000)
|
||||
cb.on_step_end(MagicMock(), state, MagicMock(), model=MagicMock(), optimizer=MagicMock())
|
||||
assert cb.fire_count == 0
|
||||
|
||||
|
||||
class TestReLoRATaskGate:
|
||||
def _base_cfg(self, task: str = "sft", backend: str = "transformers") -> dict:
|
||||
return {
|
||||
"base": "meta-llama/Llama-3.1-8B",
|
||||
"task": task,
|
||||
"backend": backend,
|
||||
"data": {"train": "./data.jsonl"},
|
||||
"training": {"relora_steps": 100},
|
||||
}
|
||||
|
||||
def test_sft_accepted(self):
|
||||
import yaml
|
||||
|
||||
from soup_cli.config.loader import load_config_from_string
|
||||
cfg = load_config_from_string(yaml.safe_dump(self._base_cfg("sft")))
|
||||
assert cfg.training.relora_steps == 100
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"task",
|
||||
["dpo", "grpo", "kto", "orpo", "simpo", "ipo",
|
||||
"ppo", "reward_model", "pretrain", "embedding"],
|
||||
)
|
||||
def test_other_tasks_rejected(self, task):
|
||||
import yaml
|
||||
|
||||
from soup_cli.config.loader import load_config_from_string
|
||||
# load_config_from_string surfaces a ValueError (Pydantic validation
|
||||
# error) that wraps the cross-validator's message.
|
||||
with pytest.raises(ValueError, match="relora_steps"):
|
||||
load_config_from_string(yaml.safe_dump(self._base_cfg(task)))
|
||||
|
||||
def test_mlx_backend_rejected(self):
|
||||
import yaml
|
||||
|
||||
from soup_cli.config.loader import load_config_from_string
|
||||
with pytest.raises(ValueError, match="mlx"):
|
||||
load_config_from_string(yaml.safe_dump(self._base_cfg("sft", "mlx")))
|
||||
|
||||
def test_no_relora_no_gate(self):
|
||||
# multi-task without relora_steps stays valid
|
||||
import yaml
|
||||
|
||||
from soup_cli.config.loader import load_config_from_string
|
||||
d = self._base_cfg("dpo")
|
||||
d["training"].pop("relora_steps")
|
||||
cfg = load_config_from_string(yaml.safe_dump(d))
|
||||
assert cfg.training.relora_steps is None
|
||||
|
||||
|
||||
class TestPruneAndReset:
|
||||
def test_prune_and_reset_targets_lora_modules(self):
|
||||
try:
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.relora import ReLoRACallback, ReLoRAPolicy
|
||||
|
||||
class FakeLoraModule(nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.lora_A = nn.Linear(4, 2, bias=False)
|
||||
self.lora_B = nn.Linear(2, 4, bias=False)
|
||||
self.base = nn.Linear(4, 4, bias=False)
|
||||
|
||||
def forward(self, x):
|
||||
return self.base(x) + self.lora_B(self.lora_A(x))
|
||||
|
||||
model = FakeLoraModule()
|
||||
# set known weight values
|
||||
with torch.no_grad():
|
||||
model.lora_A.weight.fill_(1.0)
|
||||
model.lora_A.weight[0, 0] = 100.0
|
||||
model.lora_B.weight.fill_(0.5)
|
||||
model.base.weight.fill_(7.0)
|
||||
before_base = model.base.weight.clone()
|
||||
|
||||
cb = ReLoRACallback(policy=ReLoRAPolicy(steps=10, prune_ratio=0.9))
|
||||
opt = MagicMock()
|
||||
opt.state = {}
|
||||
cb._prune_and_reset(model, opt)
|
||||
|
||||
# base must be untouched
|
||||
assert torch.equal(model.base.weight, before_base)
|
||||
# lora_A retains highest-magnitude entry
|
||||
assert (model.lora_A.weight.abs() >= 100.0).any()
|
||||
# lora_A overall has many zeros now (prune_ratio=0.9)
|
||||
zero_frac = (model.lora_A.weight == 0).float().mean().item()
|
||||
assert zero_frac > 0.5
|
||||
|
||||
def test_prune_and_reset_clears_real_optimizer_state(self):
|
||||
try:
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.relora import ReLoRACallback, ReLoRAPolicy
|
||||
|
||||
class FakeLoraModule(nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.lora_A = nn.Linear(4, 2, bias=False)
|
||||
|
||||
def forward(self, x):
|
||||
return self.lora_A(x)
|
||||
|
||||
model = FakeLoraModule()
|
||||
opt = torch.optim.AdamW(model.parameters(), lr=1e-3)
|
||||
# Run a step so opt.state is populated for the param.
|
||||
loss = model(torch.randn(2, 4)).sum()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
param = model.lora_A.weight
|
||||
assert param in opt.state
|
||||
assert len(opt.state[param]) > 0 # exp_avg, exp_avg_sq, step
|
||||
|
||||
cb = ReLoRACallback(policy=ReLoRAPolicy(steps=10, prune_ratio=0.9))
|
||||
cb._prune_and_reset(model, opt)
|
||||
# Optimizer state for the pruned param must have been reset to empty
|
||||
assert len(opt.state[param]) == 0
|
||||
|
||||
def test_prune_and_reset_respects_reset_optimizer_false(self):
|
||||
try:
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
except ImportError:
|
||||
pytest.skip("torch not available")
|
||||
from soup_cli.utils.relora import ReLoRACallback, ReLoRAPolicy
|
||||
|
||||
class FakeLoraModule(nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.lora_A = nn.Linear(4, 2, bias=False)
|
||||
|
||||
def forward(self, x):
|
||||
return self.lora_A(x)
|
||||
|
||||
model = FakeLoraModule()
|
||||
opt = torch.optim.AdamW(model.parameters(), lr=1e-3)
|
||||
loss = model(torch.randn(2, 4)).sum()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
param = model.lora_A.weight
|
||||
before = len(opt.state[param])
|
||||
|
||||
cb = ReLoRACallback(
|
||||
policy=ReLoRAPolicy(steps=10, prune_ratio=0.9, reset_optimizer=False)
|
||||
)
|
||||
cb._prune_and_reset(model, opt)
|
||||
# State preserved when reset_optimizer=False
|
||||
assert len(opt.state[param]) == before
|
||||
|
|
@ -0,0 +1,140 @@
|
|||
"""Tests for v0.39.0 Part E — template registry YAML migration."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
class TestTemplateRegistry:
|
||||
def test_list_templates_includes_core_set(self):
|
||||
from soup_cli.templates import list_templates
|
||||
names = list_templates()
|
||||
for required in (
|
||||
"chat", "code", "reasoning", "vision", "kto", "orpo",
|
||||
"simpo", "ipo", "pretrain", "moe", "longcontext", "embedding",
|
||||
"audio", "tool-calling", "rlhf", "medical",
|
||||
):
|
||||
assert required in names, f"{required} missing from {names}"
|
||||
|
||||
def test_list_templates_returns_sorted(self):
|
||||
from soup_cli.templates import list_templates
|
||||
names = list_templates()
|
||||
assert names == sorted(names)
|
||||
|
||||
def test_load_template_chat_from_yaml(self):
|
||||
from soup_cli.templates import load_template
|
||||
body = load_template("chat")
|
||||
assert body is not None
|
||||
assert "Soup template: Chat Assistant" in body
|
||||
assert "task: sft" in body
|
||||
|
||||
def test_load_template_unknown_returns_none(self):
|
||||
from soup_cli.templates import load_template
|
||||
assert load_template("does-not-exist") is None
|
||||
|
||||
def test_load_template_rejects_path_traversal(self):
|
||||
from soup_cli.templates import load_template
|
||||
with pytest.raises(ValueError):
|
||||
load_template("../../etc/passwd")
|
||||
with pytest.raises(ValueError):
|
||||
load_template("foo/bar")
|
||||
with pytest.raises(ValueError):
|
||||
load_template("foo\\bar")
|
||||
|
||||
def test_load_template_rejects_null_byte(self):
|
||||
from soup_cli.templates import load_template
|
||||
with pytest.raises(ValueError):
|
||||
load_template("chat\x00malicious")
|
||||
|
||||
def test_load_template_rejects_empty_name(self):
|
||||
from soup_cli.templates import load_template
|
||||
with pytest.raises(ValueError):
|
||||
load_template("")
|
||||
|
||||
def test_yaml_files_exist_for_all_inline(self):
|
||||
from soup_cli.config.schema import TEMPLATES
|
||||
templates_dir = Path(__file__).resolve().parent.parent / "soup_cli" / "templates"
|
||||
for name in TEMPLATES:
|
||||
yaml_path = templates_dir / f"{name}.yaml"
|
||||
assert yaml_path.is_file(), f"Missing YAML for inline template {name}"
|
||||
|
||||
def test_manifest_well_formed(self):
|
||||
templates_dir = Path(__file__).resolve().parent.parent / "soup_cli" / "templates"
|
||||
manifest_path = templates_dir / "manifest.json"
|
||||
assert manifest_path.is_file()
|
||||
with manifest_path.open() as f:
|
||||
data = json.load(f)
|
||||
assert "templates" in data
|
||||
assert "version" in data
|
||||
assert isinstance(data["templates"], dict)
|
||||
|
||||
def test_yaml_content_matches_inline_for_all_templates(self):
|
||||
"""All 16 inline templates must match their YAML siblings exactly.
|
||||
|
||||
v0.39.0 Part E ships both sources for back-compat; drift between
|
||||
them is a contributor footgun. This guards against silent edits
|
||||
of one source without the other.
|
||||
"""
|
||||
from soup_cli.config.schema import TEMPLATES
|
||||
from soup_cli.templates import load_template
|
||||
for name in TEMPLATES:
|
||||
assert load_template(name) == TEMPLATES[name], (
|
||||
f"YAML / inline drift for template {name!r}"
|
||||
)
|
||||
|
||||
|
||||
class TestSecurityFallbacks:
|
||||
def test_oversized_file_falls_back_to_inline(self, tmp_path, monkeypatch):
|
||||
"""v0.39.0 — _MAX_TEMPLATE_BYTES guard."""
|
||||
import soup_cli.templates as tpl_mod
|
||||
|
||||
fake_dir = tmp_path / "templates"
|
||||
fake_dir.mkdir()
|
||||
(fake_dir / "manifest.json").write_text(
|
||||
json.dumps({"templates": {"chat": "chat.yaml"}, "version": 1})
|
||||
)
|
||||
# Write a 300 KB file (> 256 KB cap)
|
||||
(fake_dir / "chat.yaml").write_text("x" * (300 * 1024))
|
||||
|
||||
monkeypatch.setattr(tpl_mod, "_templates_dir", lambda: fake_dir)
|
||||
# Should fall back to inline TEMPLATES["chat"], not return the giant blob.
|
||||
body = tpl_mod.load_template("chat")
|
||||
from soup_cli.config.schema import TEMPLATES
|
||||
assert body == TEMPLATES["chat"]
|
||||
assert len(body) < 256 * 1024
|
||||
|
||||
def test_crafted_manifest_outside_dir_falls_back(self, tmp_path, monkeypatch):
|
||||
"""A manifest entry with a relative path that escapes must be ignored."""
|
||||
import soup_cli.templates as tpl_mod
|
||||
|
||||
fake_dir = tmp_path / "templates"
|
||||
fake_dir.mkdir()
|
||||
# Create a file OUTSIDE the templates dir
|
||||
(tmp_path / "secret.yaml").write_text("LEAKED CONTENT")
|
||||
# Crafted manifest pointing to it via filename traversal-like component
|
||||
# _validate_name will reject any name with `..`/`/`/`\` so this also
|
||||
# exercises the manifest-tampering rejection chain.
|
||||
(fake_dir / "manifest.json").write_text(
|
||||
json.dumps({"templates": {"chat": "../secret.yaml"}, "version": 1})
|
||||
)
|
||||
|
||||
monkeypatch.setattr(tpl_mod, "_templates_dir", lambda: fake_dir)
|
||||
body = tpl_mod.load_template("chat")
|
||||
from soup_cli.config.schema import TEMPLATES
|
||||
# Must NOT contain the leaked content; falls back to inline.
|
||||
assert "LEAKED CONTENT" not in (body or "")
|
||||
assert body == TEMPLATES["chat"]
|
||||
|
||||
|
||||
class TestInitUsesRegistry:
|
||||
def test_init_command_lists_template_options(self):
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from soup_cli.cli import app
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(app, ["init", "--help"])
|
||||
# init --help should still succeed after the migration
|
||||
assert result.exit_code == 0, (result.output, repr(result.exception))
|
||||
Loading…
Reference in New Issue