Alpamys
|
a66e4b9ebe
|
feat: architecture + distill + adapter-train live wiring (v0.71.12)
Lift seven schema-only stubs to live, validated on tiny models:
- #145 distill_mode token|sequence — sequence-level teacher-continuation KD
- #146 classifier LoRA — frozen encoder + adapter (classifier/reranker/cross_encoder)
- #148 LLaMA Pro block expansion — per-arch (Llama/Qwen/Mistral) zero-init blocks
- #158 LongLoRA S2 — shifted-sparse attention on Q/K projections (Llama/Mistral/Qwen/Phi)
- #84 Mixture-of-Depths — per-layer top-k token router (use_mod; Llama/Qwen/Mistral)
- #221 VeRA/VB-LoRA serving — soup serve --bank, per-user delta via X-User-Id header
- #222 MoLE — task=moe_lora_routing, per-token gate over N frozen task LoRAs (gate-only)
Tests 13203 -> 13329 (+126; tests/test_v07112.py). ruff clean, coverage 78.49%.
|
2026-06-04 19:36:04 +05:00 |