Part A — BCO Trainer (Binary Classifier Optimization): new task='bco',
training.bco_beta, bco.yaml template, train+sweep routing. Internal
_split_dpo_rows_to_bco adapts paired DPO input to TRL's BCO unpaired
schema; skipped rows logged at DEBUG (mirrors v0.33.0 #47 policy).
Part B — Unified preference dispatcher: additive task='preference' +
training.preference_loss Literal {dpo,simpo,orpo,ipo,bco}. Legacy
task='dpo' / 'simpo' / 'orpo' / 'ipo' / 'bco' remain first-class —
the new surface is purely additive, not a breaking collapse.
_make_inner_cfg uses model_copy so re-validation never sees an
intermediate inconsistent state and the caller's cfg is never mutated.
Part C — KL-controlled DPO variants: dpo_beta_schedule (linear /
cosine / exponential) + dpo_beta_end + dpo_ref_regen_epochs [1, 1000].
BetaScheduleCallback resolves total_steps lazily in on_train_begin
(closes a first-cut bug where total_steps=0 silently emitted beta_end
for every step). RefModelRegenCallback uses load_state_dict(strict=True)
with WARNING-on-mismatch (closes a first-cut silent partial-copy
hazard). Gated to DPO-family tasks only; rejected on mlx backend with
distinct error message.
Part D — Multi-objective preference_loss_weights (2-5 entries, key
allowlist + null-byte rejection, sum-to-1 ±1e-6). Schema-level surface
only; live runtime weighted-loss combination deferred to v0.40.1 with
NotImplementedError stub-then-live (mirrors v0.27.0 MII / v0.37.0
multipack / v0.38.0 quant menu / v0.39.0 ReLoRA pattern).
Net +118 tests (4538 → 4656). All four review-agent waves clean
(Python / Code / Security / TDD).
Known limitation: BCOTrainerWrapper still hardcodes
trust_remote_code=True (carry-over of the v0.36.0 #63 family across
non-SFT trainers).
Also: add docs/ to .gitignore (internal-only docs going forward;
existing docs/QUANTIZATION.md from v0.38.0 stays tracked).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships v0.25.0 with eight new capabilities (Parts A–H) that close every
competitive gap vs LLaMA-Factory/Axolotl/Unsloth and add unique differentiators:
Part A — 9 new model recipes: Llama 4 Scout (sft/dpo/grpo), Qwen 3 14B/32B/8B-grpo,
Gemma 3 12B/27B-dpo, DeepSeek V3 (MoE LoRA).
Part B — Tool-calling / agentic fine-tuning: new "tool-calling" data format with
detection + normalization, synth data template, init template, eval scoring
(tool_call_match / tool_call_name_match / tool_call_args_subset), plus
qwen3-8b-tools and llama4-scout-tools recipes.
Part C — RLVR (RL from Verifiable Rewards): reward_fn=verifiable routing to
math_verify_reward (regex-only, no eval), code_exec_reward (subprocess sandbox
with RLIMIT_AS/RLIMIT_CPU on POSIX, ephemeral tempdir cwd, concurrency cap,
one-time warning panel), and json_schema_reward. verifiable_domain Literal
validated via model_validator.
Part D — VeRA + OLoRA PEFT methods: LoraConfig.use_vera / use_olora with
mutual-exclusion validator and a unified peft_builder helper that returns
either LoraConfig or VeraConfig with the right init kwargs.
Part E — Apple Silicon MLX backend: detection + hardware profiling in utils/mlx,
MLXSFTTrainerWrapper via mlx-lm, scaffolding DPO/GRPO wrappers rejected at
config load time by SoupConfig._validate_mlx_task_support, lazy trainer
registry, doctor integration, 3 MLX SFT recipes, [mlx] extra in pyproject.
Part F — Data augmentation: soup data augment with rephrase / translate / style
strategies, path-traversal-protected input/output, count capped 1-10, lang/styles
lists bounded (10 entries × 32 chars), rate limiting, and optional --dedup.
Part G — Training intelligence: forgetting detection (ForgettingDetector with
3 built-in mini benchmarks and warning levels) and checkpoint intelligence
(CheckpointTracker with composite metric, early-stop on regression, safe
top-N pruning refusing symlinks and non-checkpoint dirs). SQLite schema
extended with checkpoint_quality + forgetting_eval tables.
Part H — Autopilot: soup autopilot command with dataset/model/hardware
profilers, decision engine (task/quant/peft/batch/lr/epochs/max_length/perf
flags), YAML generator, and full CLI with dry-run + --yes + path-traversal
protection + goal whitelist + gpu_budget bounds [1GB, 1TB]. Bakes forgetting
detection + checkpoint intelligence + early-stop into the generated config.
Totals:
- 2313 tests passing (183 new, up from 2130)
- 86 test files (8 new)
- 43 ready-made recipes (14 new)
- 16 built-in templates (tool-calling added)
- Review findings: all CRITICAL/HIGH/MEDIUM/LOW addressed (3 documented
design limitations: code_exec best-effort sandbox, prune_checkpoints TOCTOU,
MLX training integration test requires real hardware)
Docs: CLAUDE.md, README.md, SECURITY.md, CONTRIBUTING.md updated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Alpamys