diff --git a/docs/peft-and-efficiency.md b/docs/peft-and-efficiency.md index e7839bd..cb5c17e 100644 --- a/docs/peft-and-efficiency.md +++ b/docs/peft-and-efficiency.md @@ -60,6 +60,8 @@ training: **Live wiring** — landed. SFT and Pretrain trainer wrappers actually instantiate the multipack subclass when `multipack: true` is set. The factory's `get_train_dataloader` override installs `MultipackBatchSampler(real_batches=False)` (yields a flat `list[int]` per packed sequence — DataLoader-compatible) as the DataLoader's `batch_sampler=`, forwarding `dataloader_drop_last`/`num_workers`/`pin_memory` from `TrainingArguments`. The `_get_train_sampler` override stays as a defensive no-op fallback that always delegates to super, so any HF eval / prediction loop bypassing `get_train_dataloader` still gets the correct `Sampler[int]` shape (no nested-list shape mismatch). Multipack is **sft / pretrain only** on the `transformers` backend; preference / RLHF trainers and MLX backend get distinct error messages naming the actual reason. Datasets must expose `input_ids` (preferred) or `length` per row; raw text triggers an all-zeros warning. +**Multi-GPU sharding (v0.71.19).** Under FSDP / DeepSpeed ZeRO / DDP (`num_processes > 1`) the `get_train_dataloader` override routes the multipack DataLoader through `accelerator.prepare`, so accelerate's `BatchSamplerShard` round-robins whole FFD-packed bins to each rank (preserving the packing; the bin seed is identical across ranks so every rank agrees on the global order before sharding). The single-GPU path returns the raw DataLoader unchanged. Multi-GPU correctness is mocked-tested — a real 2+-GPU validation run is tracked QA. + **DoS hardening** — the FFD packer caps at 1M items (algorithm is O(N²) worst-case); the 4D mask builder caps allocations at 2³¹ cells; the chat-template Jinja analyzer caps at 128KB. Every numeric input rejects `bool` explicitly (matches v0.30.0+ project policy). The `JinjaTemplateAnalyzer` (also v0.37.0) walks chat-template ASTs to discover non-standard `message.` references (`tool_calls`, `name`, `weight`, `train`) — used by the v0.36.0 `train_on_messages_with_train_field` path so per-message training masks are aware of fields beyond `role` / `content`. The analyzer parses templates without rendering them, so a crafted `soup.yaml` cannot trigger SSRF. diff --git a/docs/performance-and-quantization.md b/docs/performance-and-quantization.md index 1f1c105..a84259c 100644 --- a/docs/performance-and-quantization.md +++ b/docs/performance-and-quantization.md @@ -204,8 +204,11 @@ trainers (SFT / DPO / GRPO / KTO / ORPO / SimPO / IPO / PPO / RewardModel / Pretrain / Embedding / BCO). PPO's reward model also loads with the same Quant Menu config as the policy when `tcfg` is passed in, so a GPTQ-policy + GPTQ-reward run does not silently OOM in fp16. MLX backend is rejected with a distinct error -message; vision / audio modality is still SFT-only inline-BNB (multi-modal -Quant Menu wiring tracked as a follow-up). +message; vision and audio modality now thread the same unified Quant Menu loader +(the `modality: text` gate was dropped in v0.71.19), so the full menu — +`gptq` / `awq` / `hqq:*` / `aqlm` / `eetq` / `mxfp4` / `fp8` — applies to +multi-modal SFT too (a given vision/audio checkpoint still needs a class + kernel +that supports the chosen format, e.g. `autoawq` for awq). ## Activation Offloading (Small-VRAM Large-Batch)