diff --git a/docs/performance-and-quantization.md b/docs/performance-and-quantization.md index 548c205..fcaf9dc 100644 --- a/docs/performance-and-quantization.md +++ b/docs/performance-and-quantization.md @@ -24,7 +24,7 @@ - [MoE Expert Quantization + Router-Only Training (live in v0.71.20)](#moe-expert-quantization--router-only-training-live-in-v07120) - [Unsloth Dynamic 2.0 GGUF Ladder (v0.53.0)](#unsloth-dynamic-20-gguf-ladder-v0530) - [KV Cache Types (v0.53.0)](#kv-cache-types-v0530) -- [FP8 Attention + NVFP4 + Native `unsloth_bnb_4bit` (v0.53.0)](#fp8-attention--nvfp4--native-unsloth_bnb_4bit-v0530) +- [FP8 Attention + NVFP4 + Native `unsloth_bnb_4bit` (v0.53.0)](#fp8-attention--nvfp4--native-unsloth_bnb_4bit) - [LF / Axolotl Quant Parity (v0.53.0)](#lf--axolotl-quant-parity-v0530) - [Advanced Save Formats (v0.53.0)](#advanced-save-formats-v0530) - [Quant Menu II + Export Pipeline (v0.53.1)](#quant-menu-ii--export-pipeline-v0531) diff --git a/docs/training.md b/docs/training.md index 6b5c364..24e876e 100644 --- a/docs/training.md +++ b/docs/training.md @@ -6,8 +6,9 @@ > **Training a model bigger than your GPU?** `training.stream_layers: true` streams the > frozen base from CPU RAM one decoder layer at a time, so peak VRAM is bounded by one -> layer instead of the whole model — -> see [Layer Streaming](performance-and-quantization.md#layer-streaming-beta-v0720). +> layer instead of the whole model. Add `quantization: 4bit` and an 8B base fits a 4 GB +> card — see +> [Layer Streaming](performance-and-quantization.md#layer-streaming-beta-v0720-nf4-v0722). **Contents:**