docs: refresh quant-menu modality + multipack sharding notes (v0.71.19)

- performance-and-quantization.md: the Quant Menu multi-trainer note said
  "vision / audio modality is still SFT-only inline-BNB (wiring tracked as a
  follow-up)" — stale after v0.71.19 #81 dropped the modality gate. Now states
  vision/audio thread the unified loader (full gptq/awq/hqq/aqlm/eetq/mxfp4/fp8
  menu), with the upstream class+kernel caveat.
- peft-and-efficiency.md: added the v0.71.19 #80 multi-GPU sharding paragraph to
  the Multipack section (accelerator.prepare + BatchSamplerShard under
  num_processes>1; identical bin seed across ranks; single-GPU unchanged).

Docs-only — no version bump / tag (the v0.71.19 code already shipped at f51331d).
This commit is contained in:
Alpamys 2026-06-09 13:02:30 +05:00
parent f51331d637
commit 853b348898
2 changed files with 7 additions and 2 deletions

View File

@ -60,6 +60,8 @@ training:
**Live wiring** — landed. SFT and Pretrain trainer wrappers actually instantiate the multipack subclass when `multipack: true` is set. The factory's `get_train_dataloader` override installs `MultipackBatchSampler(real_batches=False)` (yields a flat `list[int]` per packed sequence — DataLoader-compatible) as the DataLoader's `batch_sampler=`, forwarding `dataloader_drop_last`/`num_workers`/`pin_memory` from `TrainingArguments`. The `_get_train_sampler` override stays as a defensive no-op fallback that always delegates to super, so any HF eval / prediction loop bypassing `get_train_dataloader` still gets the correct `Sampler[int]` shape (no nested-list shape mismatch). Multipack is **sft / pretrain only** on the `transformers` backend; preference / RLHF trainers and MLX backend get distinct error messages naming the actual reason. Datasets must expose `input_ids` (preferred) or `length` per row; raw text triggers an all-zeros warning.
**Multi-GPU sharding (v0.71.19).** Under FSDP / DeepSpeed ZeRO / DDP (`num_processes > 1`) the `get_train_dataloader` override routes the multipack DataLoader through `accelerator.prepare`, so accelerate's `BatchSamplerShard` round-robins whole FFD-packed bins to each rank (preserving the packing; the bin seed is identical across ranks so every rank agrees on the global order before sharding). The single-GPU path returns the raw DataLoader unchanged. Multi-GPU correctness is mocked-tested — a real 2+-GPU validation run is tracked QA.
**DoS hardening** — the FFD packer caps at 1M items (algorithm is O(N²) worst-case); the 4D mask builder caps allocations at 2³¹ cells; the chat-template Jinja analyzer caps at 128KB. Every numeric input rejects `bool` explicitly (matches v0.30.0+ project policy).
The `JinjaTemplateAnalyzer` (also v0.37.0) walks chat-template ASTs to discover non-standard `message.<field>` references (`tool_calls`, `name`, `weight`, `train`) — used by the v0.36.0 `train_on_messages_with_train_field` path so per-message training masks are aware of fields beyond `role` / `content`. The analyzer parses templates without rendering them, so a crafted `soup.yaml` cannot trigger SSRF.

View File

@ -204,8 +204,11 @@ trainers (SFT / DPO / GRPO / KTO / ORPO / SimPO / IPO / PPO / RewardModel /
Pretrain / Embedding / BCO). PPO's reward model also loads with the same Quant
Menu config as the policy when `tcfg` is passed in, so a GPTQ-policy + GPTQ-reward
run does not silently OOM in fp16. MLX backend is rejected with a distinct error
message; vision / audio modality is still SFT-only inline-BNB (multi-modal
Quant Menu wiring tracked as a follow-up).
message; vision and audio modality now thread the same unified Quant Menu loader
(the `modality: text` gate was dropped in v0.71.19), so the full menu —
`gptq` / `awq` / `hqq:*` / `aqlm` / `eetq` / `mxfp4` / `fp8` — applies to
multi-modal SFT too (a given vision/audio checkpoint still needs a class + kernel
that supports the chosen format, e.g. `autoawq` for awq).
## Activation Offloading (Small-VRAM Large-Batch)