Alpamys
|
5e0872b9ea
|
feat(trainer): ReLoRA + surgical PEFT non-SFT (v0.40.6 #67)
Extends the v0.39.0 ReLoRA callback (Part B) and surgical PEFT patches
(Part D — Gemma4 ClippableLinear swap + 3-D fused-MoE expert dropout
strip) from SFT-only to all 11 non-SFT transformer-backend trainers
(DPO, GRPO, KTO, ORPO, SimPO, IPO, PPO, RewardModel, Pretrain,
Embedding, BCO).
- New shared helper soup_cli/utils/peft_wiring.py exposes
apply_pre_lora_patches, apply_post_lora_patches, attach_relora_callback.
- SFT migrated to the same helpers in the same release (centralisation
invariant; no drift between SFT and non-SFT wiring).
- SoupConfig._validate_relora_supported_tasks: task != "sft" rejection
removed; MLX backend still rejected with distinct message.
Review fixes:
- attach_relora_callback uses `if relora_steps is None:` (project
policy) so a schema-bypassing relora_steps=0 surfaces as a loud
ReLoRAPolicy ValueError rather than a silent skip.
- Direct attribute access on tcfg.relora_warmup_ratio / _reset_optimizer
/ _prune_ratio (Pydantic schema guarantees them); no getattr defaults.
- 11 behavioural helper tests (Gemma4 happy path + exception swallow,
post-LoRA strip happy + exception swallow, ReLoRA policy field
forwarding, schema-bypass loud-fail).
- Schema-gate matrix covers `task='preference'` dispatcher.
Tests: 5061 -> 5122 (+61).
Closes #67.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-09 15:48:17 +05:00 |
|
Alpamys
|
e6a9c087c3
|
feat(lora): v0.39.0 — LoRA Quality (PiSSA + ReLoRA + per-pattern rank + surgical patches + templates registry)
Five PEFT-surface improvements that LlamaFactory and Axolotl maintain:
- LoraConfig.init_strategy Literal["random","pissa","olora"]; PiSSA SVD init
via PEFT init_lora_weights="pissa". Back-compat: use_olora=True aligns to
init_strategy="olora" via dict-copy model_validator(mode="before"); explicit
conflict (use_olora=True + init_strategy="pissa"/"random") rejected.
Mutual-exclusion vs DoRA / VeRA.
- ReLoRA callback (utils/relora.py): frozen ReLoRAPolicy with bounds-checked
steps [1, 1e7] / warmup_ratio [0,1] / prune_ratio (0,1) (strict — prevents
zero-everything footgun); magnitude_prune_tensor (in-place torch.kthvalue,
rejects non-Tensor / single-element short-circuit); duck-typed
ReLoRACallback (no transformers import at module load). TrainingConfig
fields relora_steps / relora_warmup_ratio / relora_reset_optimizer /
relora_prune_ratio. SoupConfig _validate_relora_supported_tasks gates to
task=sft + transformers backend with distinct MLX-backend error message;
multi-trainer expansion deferred to v0.39.1 (mirrors v0.27.0 MII /
v0.37.0 multipack / v0.38.0 quant menu stub-then-live pattern).
- LoraConfig.rank_pattern / alpha_pattern Optional[Dict[str,int]]; field
validator caps at 256 keys × value (0, 1024], rejects bool / null-byte /
empty key. Cross-validator rejects with use_vera=True (VeRA shares one
rank). peft_builder propagates into LoraConfig init_kwargs.
- utils/peft_patches.py: is_gemma4_model uses regex word boundary
(?:^|[^a-z0-9])gemma-?4(?:[^a-z0-9]|$) so "ungemma4ed" no longer matches.
apply_gemma4_clippable_patch swaps ClippableLinear → nn.Linear by class
name (weight-copy fallback logs at DEBUG). strip_lora_dropout_for_3d_experts
zeroes lora_dropout.p on 3-D weights (handles ModuleDict variant for
PEFT >=0.10). apply_surgical_patches orchestrator validates model_name.
Wired into sft.py _setup_transformers with is_gemma4_model gate before
the pre-LoRA swap; post-LoRA 3-D dropout strip runs unconditionally
(architecture-detected internally).
- 16 inline templates migrated to soup_cli/templates/*.yaml + manifest.json
+ load_template loader (path-traversal-rejecting name validator;
os.path.realpath + commonpath containment so a tampered manifest cannot
read files outside the package directory; 256 KB file-size cap with
inline fallback). Inline TEMPLATES kept with deprecation comment
(planned removal v0.41.0+); test_templates_yaml asserts byte-equality
of all 16 inline ↔ YAML pairs to prevent silent drift.
Net +164 tests (4374 → 4538). All 5 review-agent waves clean before tag.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-01 16:18:07 +05:00 |