Alpamys
|
06fbd15ec4
|
feat(multipack): v0.37.0 — Multipack (5 Parts A/B/C/D/E)
FFD bin-packing sampler closes the throughput gap with Axolotl on
uneven-length chat data. Five focused parts:
- Part A: MultipackBatchSampler (FFD + 18-arch allowlist + loud-fail
vs Axolotl silent miss + _MAX_FFD_ITEMS=1M DoS cap)
- Part B: schema gate (sft/pretrain only on transformers backend,
multipack/packing mutually exclusive, distinct mlx error),
build_multipack_sampler_for_lengths helper
- Part C: neat_packing 4D attention mask + FA-vs-4D strategy picker,
_MAX_MASK_ELEMENTS=2**31 / _MAX_BOUNDARY_SEGMENTS=1M caps
- Part D: JinjaTemplateAnalyzer (parse-only AST walker, 128KB cap)
- Part E: cross-module property tests (4-seed x 200 samples, 5k stress,
FFD-to-4D-mask coherence)
All five review-agent waves clean before tag (python / code /
security / tdd / verification-loop).
Net +125 tests (4249 -> 4374), 121 test files (+5).
Live HF Trainer sampler-swap wiring deferred to v0.37.1 (mirrors v0.27.0
MII stub-then-live pattern). Schema gate + helper ship now.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-30 13:48:16 +05:00 |