diff --git a/README.md b/README.md index de70ab1..3113545 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,11 @@ resident run. Opt-in (`stream_layers: true`) and still BETA — [how it works](docs/performance-and-quantization.md#layer-streaming-beta-v0720-nf4-v0722-disk--wider-archs-v0723-preference-losses-v0724) · [all measurements](benchmarks/) · [paper](https://doi.org/10.5281/zenodo.21771064) +

+ soup train pre-flight for Llama-3.1-8B on a 4 GB card: a 3.60 GB base store pinned in RAM across 32 layers and two 113 MB VRAM buffers, then a measured peak of 3.32 GB at 119.6 tok/s, stopping short of the 4 GB line
+ Llama-3.1-8B-Instruct + NF4, LoRA, batch 1, seq 512 on an RTX 3050 Laptop 4 GB — 3.32 GB peak, 119.6 tok/s. Full video (90s) +

+ ## Why Soup? Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting diff --git a/docs/assets/layer-streaming.gif b/docs/assets/layer-streaming.gif new file mode 100644 index 0000000..5ab4356 Binary files /dev/null and b/docs/assets/layer-streaming.gif differ