mirror of https://github.com/razor-ai/soup.git
1045 lines
28 KiB
Markdown
1045 lines
28 KiB
Markdown
<p align="center">
|
||
<img src="soup.png" alt="Soup" width="280">
|
||
</p>
|
||
|
||
<h1 align="center">Soup</h1>
|
||
|
||
<p align="center">
|
||
<strong>Fine-tune LLMs in one command. No SSH, no config hell.</strong>
|
||
</p>
|
||
|
||
<p align="center">
|
||
<a href="#quick-start">Quick Start</a> ·
|
||
<a href="#features">Features</a> ·
|
||
<a href="#data-tools">Data Tools</a> ·
|
||
<a href="#experiment-tracking">Tracking</a> ·
|
||
<a href="#model-evaluation">Eval</a> ·
|
||
<a href="#all-commands">Commands</a>
|
||
</p>
|
||
|
||
<p align="center">
|
||
<a href="https://pypi.org/project/soup-cli/"><img src="https://img.shields.io/pypi/v/soup-cli?color=blue" alt="PyPI"></a>
|
||
<a href="https://pepy.tech/project/soup-cli"><img src="https://img.shields.io/pepy/dt/soup-cli?color=blue" alt="Downloads"></a>
|
||
<img src="https://img.shields.io/badge/python-3.9%2B-blue" alt="Python 3.9+">
|
||
<img src="https://img.shields.io/badge/license-MIT-green" alt="MIT License">
|
||
<a href="https://github.com/MakazhanAlpamys/Soup/actions"><img src="https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/MakazhanAlpamys/65fdc943f85f3b2c46ecddb415c2b779/raw/soup_tests.json" alt="Tests"></a>
|
||
<a href="https://github.com/MakazhanAlpamys/Soup/actions"><img src="https://github.com/MakazhanAlpamys/Soup/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
|
||
</p>
|
||
|
||
---
|
||
|
||
Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.
|
||
|
||
```bash
|
||
pip install soup-cli
|
||
soup init --template chat
|
||
soup train
|
||
```
|
||
|
||
## Why Soup?
|
||
|
||
Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that.
|
||
|
||
- **Zero SSH.** Never SSH into a broken GPU box again.
|
||
- **One config.** A simple YAML file is all you need.
|
||
- **Auto everything.** Batch size, GPU detection, quantization — handled.
|
||
- **Works locally.** Train on your own GPU with QLoRA. No cloud required.
|
||
|
||
## Quick Start
|
||
|
||
### 1. Install
|
||
|
||
```bash
|
||
# From PyPI (recommended):
|
||
pip install soup-cli
|
||
|
||
# Or from GitHub (latest dev):
|
||
pip install git+https://github.com/MakazhanAlpamys/Soup.git
|
||
```
|
||
|
||
### 2. Create config
|
||
|
||
```bash
|
||
# Interactive wizard
|
||
soup init
|
||
|
||
# Or use a template
|
||
soup init --template chat # conversational fine-tune
|
||
soup init --template code # code generation
|
||
soup init --template medical # domain expert
|
||
soup init --template reasoning # GRPO reasoning training
|
||
soup init --template vision # vision/multimodal fine-tune
|
||
soup init --template kto # KTO unpaired preference alignment
|
||
soup init --template orpo # ORPO (no reference model needed)
|
||
soup init --template simpo # SimPO length-normalized preference
|
||
soup init --template ipo # IPO regularized preference
|
||
soup init --template rlhf # full RLHF pipeline (SFT→RM→PPO)
|
||
```
|
||
|
||
### 3. Train
|
||
|
||
```bash
|
||
soup train --config soup.yaml
|
||
```
|
||
|
||
That's it. Soup handles LoRA setup, quantization, batch size, monitoring, and checkpoints.
|
||
|
||
### 4. Test your model
|
||
|
||
```bash
|
||
soup chat --model ./output
|
||
```
|
||
|
||
### 5. Push to HuggingFace
|
||
|
||
```bash
|
||
soup push --model ./output --repo your-username/my-model
|
||
```
|
||
|
||
### 6. Merge & Export
|
||
|
||
```bash
|
||
# Merge LoRA adapter with base model
|
||
soup merge --adapter ./output
|
||
|
||
# Export to GGUF for Ollama / llama.cpp
|
||
soup export --model ./output --format gguf --quant q4_k_m
|
||
```
|
||
|
||
## Config Example
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: sft
|
||
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
|
||
|
||
data:
|
||
train: ./data/train.jsonl
|
||
format: alpaca
|
||
val_split: 0.1
|
||
|
||
training:
|
||
epochs: 3
|
||
lr: 2e-5
|
||
batch_size: auto
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
quantization: 4bit
|
||
|
||
output: ./output
|
||
```
|
||
|
||
## Unsloth Backend (2-5x Faster Training)
|
||
|
||
Use the [Unsloth](https://github.com/unslothai/unsloth) backend for significantly faster training and up to 80% less VRAM:
|
||
|
||
```bash
|
||
# Install unsloth support
|
||
pip install 'soup-cli[fast]'
|
||
```
|
||
|
||
Then add one line to your config:
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: sft
|
||
backend: unsloth # 2-5x faster, -80% VRAM
|
||
|
||
data:
|
||
train: ./data/train.jsonl
|
||
format: alpaca
|
||
|
||
training:
|
||
epochs: 3
|
||
lr: 2e-5
|
||
quantization: 4bit
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
```
|
||
|
||
Works with all training tasks: SFT, DPO, GRPO, PPO, KTO, ORPO, SimPO, and IPO. If unsloth is installed but not enabled, Soup will suggest it automatically.
|
||
|
||
> **Tip:** Soup auto-detects unsloth. When installed, you'll see a hint during `soup train` if you haven't enabled it yet.
|
||
|
||
## Vision / Multimodal Fine-tuning
|
||
|
||
Fine-tune vision-language models (LLaMA-3.2-Vision, Qwen2-VL, Pixtral) on image+text data:
|
||
|
||
```bash
|
||
# Install vision support
|
||
pip install 'soup-cli[vision]'
|
||
|
||
# Create a vision config
|
||
soup init --template vision
|
||
|
||
# Train
|
||
soup train --config soup.yaml
|
||
```
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.2-11B-Vision-Instruct
|
||
task: sft
|
||
modality: vision
|
||
|
||
data:
|
||
train: ./data/vision_train.jsonl
|
||
format: llava
|
||
image_dir: ./data/images
|
||
val_split: 0.1
|
||
|
||
training:
|
||
epochs: 3
|
||
lr: 1e-5
|
||
quantization: 4bit
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
```
|
||
|
||
**Supported vision data formats:**
|
||
|
||
**LLaVA:**
|
||
```json
|
||
{"image": "photo.jpg", "conversations": [{"from": "human", "value": "<image>\nDescribe this image."}, {"from": "gpt", "value": "A cat on a mat."}]}
|
||
```
|
||
|
||
**ShareGPT4V:**
|
||
```json
|
||
{"image": "chart.png", "conversations": [{"from": "human", "value": "<image>\nWhat does this show?"}, {"from": "gpt", "value": "Quarterly revenue."}]}
|
||
```
|
||
|
||
`soup data inspect` automatically shows image statistics (count, formats, missing files) for vision datasets.
|
||
|
||
## Quantization-Aware Training (QAT)
|
||
|
||
Train with simulated quantization for significantly better post-quantization quality compared to standard QLoRA:
|
||
|
||
```bash
|
||
# Install QAT support
|
||
pip install 'soup-cli[qat]'
|
||
```
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: sft
|
||
|
||
data:
|
||
train: ./data/train.jsonl
|
||
format: alpaca
|
||
|
||
training:
|
||
epochs: 3
|
||
lr: 2e-5
|
||
quantization: 4bit
|
||
quantization_aware: true # Enable QAT
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
|
||
output: ./output
|
||
```
|
||
|
||
**When to use QAT vs post-training quantization:**
|
||
- **QAT** (`quantization_aware: true`): Better quality when you plan to deploy with aggressive quantization (int8/int4). ~5-10% slower training, but the model learns to compensate for quantization noise.
|
||
- **Post-training quantization** (default): Faster training, good enough for most use cases. Quantize after training with `soup export --quant q4_k_m`.
|
||
|
||
QAT works with all training tasks (SFT, DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO) and vision modality. Not compatible with the unsloth backend. After QAT training, export to GGUF normally with `soup export`.
|
||
|
||
## DPO Training
|
||
|
||
Train with preference data using Direct Preference Optimization:
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: dpo
|
||
|
||
data:
|
||
train: ./data/preferences.jsonl
|
||
format: dpo
|
||
|
||
training:
|
||
epochs: 3
|
||
dpo_beta: 0.1
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
quantization: 4bit
|
||
```
|
||
|
||
## GRPO Training (Reasoning)
|
||
|
||
Train reasoning models with Group Relative Policy Optimization (DeepSeek-R1 style):
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: grpo
|
||
|
||
data:
|
||
train: ./data/reasoning_train.jsonl
|
||
format: sharegpt
|
||
max_length: 4096
|
||
|
||
training:
|
||
epochs: 3
|
||
lr: 1e-5
|
||
grpo_beta: 0.1
|
||
num_generations: 4
|
||
reward_fn: accuracy # or 'format', or path to custom .py
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
quantization: 4bit
|
||
```
|
||
|
||
```bash
|
||
# Create a reasoning config
|
||
soup init --template reasoning
|
||
|
||
# Train
|
||
soup train --config soup.yaml
|
||
```
|
||
|
||
**Built-in reward functions:**
|
||
- `accuracy` — checks if the final answer matches expected (supports `####` and `\boxed{}` formats)
|
||
- `format` — checks for structured `<think>...</think>` reasoning blocks
|
||
|
||
**Custom reward functions** — point to a Python file:
|
||
```python
|
||
# my_reward.py
|
||
def reward_fn(completions, **kwargs):
|
||
"""Score each completion. Return list of floats."""
|
||
return [1.0 if "correct" in c[-1]["content"] else 0.0 for c in completions]
|
||
```
|
||
```yaml
|
||
training:
|
||
reward_fn: ./my_reward.py
|
||
```
|
||
|
||
## PPO / Full RLHF Pipeline
|
||
|
||
Train models with the full RLHF pipeline: SFT warmup → Reward Model → PPO alignment.
|
||
|
||
```bash
|
||
# Create an RLHF config
|
||
soup init --template rlhf
|
||
```
|
||
|
||
**Step 1: SFT warmup** — fine-tune a base model on your data:
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: sft
|
||
data:
|
||
train: ./data/train.jsonl
|
||
format: alpaca
|
||
output: ./output_sft
|
||
```
|
||
|
||
**Step 2: Train reward model** — learn preferences from human feedback:
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: reward_model
|
||
data:
|
||
train: ./data/preferences.jsonl
|
||
format: dpo
|
||
output: ./output_rm
|
||
```
|
||
|
||
**Step 3: PPO alignment** — optimize the policy using the reward model:
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: ppo
|
||
data:
|
||
train: ./data/prompts.jsonl
|
||
format: chatml
|
||
training:
|
||
reward_model: ./output_rm
|
||
ppo_epochs: 4
|
||
ppo_clip_ratio: 0.2
|
||
ppo_kl_penalty: 0.05
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
quantization: 4bit
|
||
output: ./output_ppo
|
||
```
|
||
|
||
PPO supports two reward sources:
|
||
- **Reward model** (`reward_model`): pre-trained reward model (from step 2)
|
||
- **Reward function** (`reward_fn`): callable function (same as GRPO — `accuracy`, `format`, or custom `.py`)
|
||
|
||
## KTO Training (Unpaired Preferences)
|
||
|
||
Train with unpaired preference data — no need for chosen+rejected pairs:
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: kto
|
||
|
||
data:
|
||
train: ./data/kto_train.jsonl
|
||
format: kto
|
||
|
||
training:
|
||
epochs: 3
|
||
kto_beta: 0.1
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
quantization: 4bit
|
||
```
|
||
|
||
**KTO data format:**
|
||
```json
|
||
{"prompt": "What is 2+2?", "completion": "4", "label": true}
|
||
{"prompt": "What is 2+2?", "completion": "Fish", "label": false}
|
||
```
|
||
|
||
## ORPO Training (No Reference Model)
|
||
|
||
ORPO combines SFT and alignment in one step — no reference model needed:
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: orpo
|
||
|
||
data:
|
||
train: ./data/preferences.jsonl
|
||
format: dpo
|
||
|
||
training:
|
||
epochs: 3
|
||
orpo_beta: 0.1
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
quantization: 4bit
|
||
```
|
||
|
||
## SimPO Training (Simple Preference)
|
||
|
||
SimPO uses length-normalized log probabilities as implicit rewards — reference-free:
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: simpo
|
||
|
||
data:
|
||
train: ./data/preferences.jsonl
|
||
format: dpo
|
||
|
||
training:
|
||
epochs: 3
|
||
simpo_gamma: 0.5
|
||
cpo_alpha: 1.0
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
quantization: 4bit
|
||
```
|
||
|
||
## IPO Training (Regularized Preference)
|
||
|
||
IPO is a theoretically grounded DPO variant with stronger regularization:
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: ipo
|
||
|
||
data:
|
||
train: ./data/preferences.jsonl
|
||
format: dpo
|
||
|
||
training:
|
||
epochs: 3
|
||
ipo_tau: 0.1
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
quantization: 4bit
|
||
```
|
||
|
||
## DoRA (Weight-Decomposed LoRA)
|
||
|
||
Enable DoRA for improved LoRA quality with magnitude decomposition:
|
||
|
||
```yaml
|
||
training:
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
use_dora: true # Enable DoRA
|
||
```
|
||
|
||
Works with all training tasks and backends.
|
||
|
||
## LoRA+ (Differentiated Learning Rates)
|
||
|
||
Use different learning rates for LoRA A and B matrices:
|
||
|
||
```yaml
|
||
training:
|
||
lr: 2e-5
|
||
loraplus_lr_ratio: 16.0 # lr_B = lr × 16
|
||
lora:
|
||
r: 64
|
||
alpha: 16
|
||
```
|
||
|
||
## GaLore (Memory-Efficient Full-Parameter Training)
|
||
|
||
Train without LoRA using gradient low-rank projection — saves optimizer memory:
|
||
|
||
```yaml
|
||
base: meta-llama/Llama-3.1-8B-Instruct
|
||
task: sft
|
||
|
||
data:
|
||
train: ./data/train.jsonl
|
||
format: alpaca
|
||
|
||
training:
|
||
epochs: 3
|
||
lr: 2e-5
|
||
quantization: none # Required: GaLore is incompatible with quantization
|
||
use_galore: true
|
||
galore_rank: 128
|
||
galore_update_proj_gap: 200
|
||
galore_scale: 0.25
|
||
```
|
||
|
||
> **Note:** GaLore requires `quantization: none` and `backend: transformers` (not unsloth).
|
||
|
||
## Chat with your model
|
||
|
||
```bash
|
||
# Chat with a LoRA adapter (auto-detects base model)
|
||
soup chat --model ./output
|
||
|
||
# Specify base model explicitly
|
||
soup chat --model ./output --base meta-llama/Llama-3.1-8B-Instruct
|
||
|
||
# Adjust generation
|
||
soup chat --model ./output --temperature 0.3 --max-tokens 256
|
||
```
|
||
|
||
## Push to HuggingFace
|
||
|
||
```bash
|
||
# Upload model to HF Hub
|
||
soup push --model ./output --repo your-username/my-model
|
||
|
||
# Make it private
|
||
soup push --model ./output --repo your-username/my-model --private
|
||
```
|
||
|
||
## Merge LoRA Adapter
|
||
|
||
Merge a LoRA adapter with its base model into a standalone model:
|
||
|
||
```bash
|
||
# Auto-detect base model from adapter_config.json
|
||
soup merge --adapter ./output --output ./merged
|
||
|
||
# Specify base model and dtype
|
||
soup merge --adapter ./output --base meta-llama/Llama-3.1-8B --dtype bfloat16
|
||
```
|
||
|
||
## Export to GGUF
|
||
|
||
Export models to GGUF format for use with [Ollama](https://ollama.com/) and [llama.cpp](https://github.com/ggerganov/llama.cpp):
|
||
|
||
```bash
|
||
# Export LoRA adapter (auto-merges with base, then converts)
|
||
soup export --model ./output --format gguf --quant q4_k_m
|
||
|
||
# Export with different quantizations
|
||
soup export --model ./output --format gguf --quant q8_0
|
||
soup export --model ./output --format gguf --quant f16
|
||
|
||
# Export a full (already merged) model
|
||
soup export --model ./merged --format gguf
|
||
|
||
# Specify llama.cpp path manually
|
||
soup export --model ./output --format gguf --llama-cpp /path/to/llama.cpp
|
||
```
|
||
|
||
Supported quantizations: `q4_0`, `q4_k_m`, `q5_k_m`, `q8_0`, `f16`, `f32`
|
||
|
||
After export, use with Ollama:
|
||
```bash
|
||
echo 'FROM ./my-model.q4_k_m.gguf' > Modelfile
|
||
ollama create my-model -f Modelfile
|
||
ollama run my-model
|
||
```
|
||
|
||
## Resume Training
|
||
|
||
Resume a training run from a checkpoint:
|
||
|
||
```bash
|
||
# Auto-detect latest checkpoint in output directory
|
||
soup train --config soup.yaml --resume auto
|
||
|
||
# Resume from a specific checkpoint
|
||
soup train --config soup.yaml --resume ./output/checkpoint-500
|
||
```
|
||
|
||
## Batch Inference
|
||
|
||
Run a model on a list of prompts and save results:
|
||
|
||
```bash
|
||
# JSONL input (each line: {"prompt": "..."})
|
||
soup infer --model ./output --input prompts.jsonl --output results.jsonl
|
||
|
||
# Plain text input (one prompt per line)
|
||
soup infer --model ./output --input prompts.txt --output results.jsonl
|
||
|
||
# Custom generation settings
|
||
soup infer --model ./output --input prompts.jsonl --output results.jsonl \
|
||
--max-tokens 512 --temperature 0.3
|
||
```
|
||
|
||
Output is JSONL with `prompt`, `response`, and `tokens_generated` fields. Shows a progress bar and throughput summary.
|
||
|
||
## TensorBoard Integration
|
||
|
||
Log training metrics to TensorBoard for local visualization:
|
||
|
||
```bash
|
||
# Enable TensorBoard logging (requires: pip install tensorboard)
|
||
soup train --config soup.yaml --tensorboard
|
||
|
||
# View logs
|
||
tensorboard --logdir ./output/runs/
|
||
```
|
||
|
||
> **Note:** `--tensorboard` and `--wandb` cannot be used together. Pick one.
|
||
|
||
## Weights & Biases Integration
|
||
|
||
Send training metrics to [W&B](https://wandb.ai/) for cloud-based experiment tracking:
|
||
|
||
```bash
|
||
# Enable W&B logging (requires: pip install wandb)
|
||
soup train --config soup.yaml --wandb
|
||
```
|
||
|
||
Make sure `WANDB_API_KEY` is set or run `wandb login` first.
|
||
|
||
## Inference Server
|
||
|
||
Start a local OpenAI-compatible inference server:
|
||
|
||
```bash
|
||
# Install server dependencies
|
||
pip install 'soup-cli[serve]'
|
||
|
||
# Start server
|
||
soup serve --model ./output --port 8000
|
||
|
||
# With custom settings
|
||
soup serve --model ./output --port 8080 --host 127.0.0.1 --max-tokens 1024
|
||
```
|
||
|
||
Endpoints:
|
||
- `POST /v1/chat/completions` — chat completions (streaming supported)
|
||
- `GET /v1/models` — list available models
|
||
- `GET /health` — health check
|
||
|
||
Compatible with OpenAI SDK:
|
||
```python
|
||
from openai import OpenAI
|
||
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
|
||
response = client.chat.completions.create(
|
||
model="output",
|
||
messages=[{"role": "user", "content": "Hello!"}],
|
||
)
|
||
```
|
||
|
||
### vLLM Backend (2-4x Faster Inference)
|
||
|
||
Use [vLLM](https://github.com/vllm-project/vllm) for significantly better throughput in production:
|
||
|
||
```bash
|
||
# Install vLLM support
|
||
pip install 'soup-cli[serve-fast]'
|
||
|
||
# Start with vLLM backend
|
||
soup serve --model ./output --backend vllm
|
||
|
||
# Multi-GPU with tensor parallelism
|
||
soup serve --model ./output --backend vllm --tensor-parallel 2
|
||
|
||
# Control GPU memory usage
|
||
soup serve --model ./output --backend vllm --gpu-memory 0.8
|
||
```
|
||
|
||
> **Tip:** Soup auto-detects vLLM. When installed, you'll see a hint during `soup serve` if you haven't enabled it yet.
|
||
|
||
> **Note (v0.10.10+):** `max_tokens` is capped at 16,384 per request. Error details are never exposed in HTTP responses.
|
||
|
||
## Synthetic Data Generation
|
||
|
||
Generate training data using LLMs:
|
||
|
||
```bash
|
||
# Generate using OpenAI API
|
||
soup data generate --prompt "Create math word problems" --count 100 --format alpaca
|
||
|
||
# Use a different model
|
||
soup data generate --prompt "Medical Q&A pairs" --model gpt-4o --count 500
|
||
|
||
# Deduplicate against existing data
|
||
soup data generate --prompt "..." --count 200 --dedup-with existing.jsonl
|
||
|
||
# Use seed examples to guide style
|
||
soup data generate --prompt "..." --seed examples.jsonl --count 100
|
||
```
|
||
|
||
## Hyperparameter Sweep
|
||
|
||
Search for the best hyperparameters:
|
||
|
||
```bash
|
||
# Grid search over learning rate and LoRA rank
|
||
soup sweep --config soup.yaml --param lr=1e-5,2e-5,5e-5 --param lora_r=8,16,32
|
||
|
||
# Random search with max runs
|
||
soup sweep --config soup.yaml --param lr=1e-5,2e-5,5e-5 --strategy random --max-runs 5
|
||
|
||
# Preview without running
|
||
soup sweep --config soup.yaml --param lr=1e-5,2e-5 --param epochs=2,3 --dry-run
|
||
|
||
# Early stopping: skip remaining runs if loss exceeds 1.5x best
|
||
soup sweep --config soup.yaml --param lr=1e-5,2e-5,5e-5 --early-stop 1.5
|
||
```
|
||
|
||
## Model Comparison
|
||
|
||
Compare outputs of two models side-by-side:
|
||
|
||
```bash
|
||
# Compare with inline prompts
|
||
soup diff --model-a ./model_v1 --model-b ./model_v2 --prompt "Explain gravity"
|
||
|
||
# Compare with a prompts file
|
||
soup diff --model-a ./base --model-b ./finetuned --prompts test_prompts.jsonl
|
||
|
||
# Save results
|
||
soup diff --model-a ./a --model-b ./b --prompts prompts.txt --output results.jsonl
|
||
```
|
||
|
||
## Multi-GPU / DeepSpeed
|
||
|
||
Train on multiple GPUs with DeepSpeed:
|
||
|
||
```bash
|
||
# ZeRO Stage 2 (recommended for most cases)
|
||
soup train --config soup.yaml --deepspeed zero2
|
||
|
||
# ZeRO Stage 3 (for very large models)
|
||
soup train --config soup.yaml --deepspeed zero3
|
||
|
||
# ZeRO Stage 2 with CPU offload (memory-constrained)
|
||
soup train --config soup.yaml --deepspeed zero2_offload
|
||
|
||
# Custom DeepSpeed config
|
||
soup train --config soup.yaml --deepspeed ./my_ds_config.json
|
||
```
|
||
|
||
## Quickstart Demo
|
||
|
||
Run a complete demo in one command — creates sample data, config, and trains a tiny model:
|
||
|
||
```bash
|
||
# Full demo (creates data + config + trains TinyLlama)
|
||
soup quickstart
|
||
|
||
# Just create files without training
|
||
soup quickstart --dry-run
|
||
|
||
# Skip confirmation
|
||
soup quickstart --yes
|
||
```
|
||
|
||
## Health Check
|
||
|
||
Check your environment for compatibility issues:
|
||
|
||
```bash
|
||
soup doctor
|
||
```
|
||
|
||
Shows: Python version, GPU availability, all dependency versions, and fix suggestions.
|
||
|
||
## Version Info
|
||
|
||
```bash
|
||
# Basic version
|
||
soup version
|
||
|
||
# Full system info (useful for bug reports)
|
||
soup version --full
|
||
# -> soup v0.13.2 | Python 3.11.5 | CUDA 12.1 | extras: serve, data
|
||
```
|
||
|
||
## Web UI
|
||
|
||
Launch a local web interface to manage experiments, start training, explore data, and chat with models — all from your browser.
|
||
|
||
```bash
|
||
pip install 'soup-cli[ui]'
|
||
soup ui
|
||
# -> opens http://127.0.0.1:7860 in your browser
|
||
# -> prints auth token to console
|
||
```
|
||
|
||
**Pages:**
|
||
- **Dashboard** — view all experiment runs, loss charts, system info
|
||
- **New Training** — create configs from templates, validate, and start training
|
||
- **Data Explorer** — browse and inspect datasets (JSONL, JSON, CSV, Parquet)
|
||
- **Model Chat** — chat with a running `soup serve` inference server
|
||
|
||
**Security (v0.10.10+):** The Web UI generates a random auth token at startup (printed to console). All mutating endpoints (start/stop training, delete runs, inspect data, validate config) require `Authorization: Bearer <token>` header. CORS is restricted to the served origin. Data inspection is sandboxed to the working directory.
|
||
|
||
```bash
|
||
# Custom port, don't auto-open browser
|
||
soup ui --port 8080 --no-browser
|
||
```
|
||
|
||
## Error Handling
|
||
|
||
Soup shows friendly error messages by default (2-3 lines with a fix suggestion). For full tracebacks:
|
||
|
||
```bash
|
||
# Global flag goes BEFORE the command
|
||
soup --verbose train --config soup.yaml
|
||
|
||
# Works with any command
|
||
soup --verbose eval --model ./output --benchmarks mmlu
|
||
```
|
||
|
||
> **Note:** `--verbose` is a global flag — it must go **before** the command name, not after.
|
||
|
||
## Data Formats
|
||
|
||
Soup supports these formats (auto-detected). Files can be JSONL, JSON, CSV, or Parquet.
|
||
|
||
**Alpaca:**
|
||
```json
|
||
{"instruction": "Explain gravity", "input": "", "output": "Gravity is..."}
|
||
```
|
||
|
||
**ShareGPT:**
|
||
```json
|
||
{"conversations": [{"from": "human", "value": "Hi"}, {"from": "gpt", "value": "Hello!"}]}
|
||
```
|
||
|
||
**ChatML:**
|
||
```json
|
||
{"messages": [{"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello!"}]}
|
||
```
|
||
|
||
**DPO / ORPO / SimPO / IPO (preference pairs):**
|
||
```json
|
||
{"prompt": "Explain gravity", "chosen": "Gravity is a force...", "rejected": "I don't know"}
|
||
```
|
||
|
||
**KTO (unpaired preferences):**
|
||
```json
|
||
{"prompt": "Explain gravity", "completion": "Gravity is a force...", "label": true}
|
||
```
|
||
|
||
**LLaVA (vision):**
|
||
```json
|
||
{"image": "photo.jpg", "conversations": [{"from": "human", "value": "<image>\nDescribe this."}, {"from": "gpt", "value": "A cat."}]}
|
||
```
|
||
|
||
**ShareGPT4V (vision):**
|
||
```json
|
||
{"image": "chart.png", "conversations": [{"from": "human", "value": "<image>\nExplain this chart."}, {"from": "gpt", "value": "Revenue growth."}]}
|
||
```
|
||
|
||
## Data Tools
|
||
|
||
```bash
|
||
# Inspect a dataset
|
||
soup data inspect ./data/train.jsonl
|
||
|
||
# Validate format
|
||
soup data validate ./data/train.jsonl --format alpaca
|
||
|
||
# Convert between formats
|
||
soup data convert ./data/train.jsonl --to sharegpt --output converted.jsonl
|
||
|
||
# Merge multiple datasets
|
||
soup data merge data1.jsonl data2.jsonl --output merged.jsonl --shuffle
|
||
|
||
# Remove near-duplicates (requires: pip install 'soup-cli[data]')
|
||
soup data dedup ./data/train.jsonl --threshold 0.8
|
||
|
||
# Extended statistics (length distribution, token counts, languages)
|
||
soup data stats ./data/train.jsonl
|
||
```
|
||
|
||
## Experiment Tracking
|
||
|
||
Every `soup train` run is automatically tracked in a local SQLite database (`~/.soup/experiments.db`).
|
||
|
||
```bash
|
||
# List all training runs
|
||
soup runs
|
||
|
||
# Show detailed info + loss curve for a run
|
||
soup runs show run_20260223_143052_a1b2
|
||
|
||
# Compare two runs side by side
|
||
soup runs compare run_1 run_2
|
||
|
||
# Delete a run
|
||
soup runs delete run_1
|
||
```
|
||
|
||
## Model Evaluation
|
||
|
||
Evaluate models on standard benchmarks using [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness):
|
||
|
||
```bash
|
||
# Install eval dependencies
|
||
pip install 'soup-cli[eval]'
|
||
|
||
# Evaluate on benchmarks
|
||
soup eval --model ./output --benchmarks mmlu,gsm8k,hellaswag
|
||
|
||
# Link results to a training run
|
||
soup eval --model ./output --benchmarks mmlu --run-id run_20260223_143052_a1b2
|
||
```
|
||
|
||
## All Commands
|
||
|
||
```
|
||
soup init [--template chat|code|medical|reasoning|vision|kto|orpo|simpo|ipo|rlhf] Create config
|
||
soup train --config soup.yaml Start training
|
||
soup train --config soup.yaml --tensorboard Train with TensorBoard logging
|
||
soup infer --model ./output --input p.jsonl Batch inference
|
||
soup chat --model ./output Interactive chat
|
||
soup push --model ./output --repo user/name Upload to HuggingFace
|
||
soup merge --adapter ./output Merge LoRA with base model
|
||
soup export --model ./output --format gguf Export to GGUF (Ollama)
|
||
soup eval --model ./output --benchmarks mmlu Evaluate on benchmarks
|
||
soup serve --model ./output --port 8000 OpenAI-compatible API server
|
||
soup serve --model ./output --backend vllm vLLM backend (2-4x throughput)
|
||
soup sweep --config soup.yaml --param lr=... Hyperparameter search
|
||
soup diff --model-a ./a --model-b ./b Compare two models
|
||
soup data inspect <path> View dataset stats
|
||
soup data validate <path> --format alpaca Check format
|
||
soup data convert <path> --to chatml Convert between formats
|
||
soup data merge data1.jsonl data2.jsonl Combine datasets
|
||
soup data dedup <path> --threshold 0.8 Remove duplicates (MinHash)
|
||
soup data stats <path> Extended statistics
|
||
soup data generate --prompt "..." --count 100 Generate synthetic data
|
||
soup runs List training runs
|
||
soup runs show <run_id> Run details + loss graph
|
||
soup runs compare <run_1> <run_2> Compare two runs
|
||
soup ui [--port 7860] Web UI (experiments, training, data)
|
||
soup doctor Check environment
|
||
soup quickstart [--dry-run] Full demo
|
||
soup version [--full] Show version (--full: system info)
|
||
soup --verbose <command> Full traceback on errors
|
||
```
|
||
|
||
## Supported Models
|
||
|
||
Soup works with **any** of the **340,000+** text-generation models on [HuggingFace Hub](https://huggingface.co/models?pipeline_tag=text-generation). If a model supports `AutoModelForCausalLM`, it works with Soup — zero config changes needed.
|
||
|
||
### Recommended Models
|
||
|
||
| Model Family | Models | Sizes | Best For |
|
||
|---|---|---|---|
|
||
| **Llama 4** | Llama-4-Scout-17B, Llama-4-Maverick-17B | 17B | General, multilingual |
|
||
| **Llama 3.x** | Llama-3.1-8B-Instruct, Llama-3.3-70B-Instruct | 1B–70B | Chat, instruction following |
|
||
| **Llama 3.2 Vision** | Llama-3.2-11B-Vision-Instruct, Llama-3.2-90B-Vision | 11B–90B | Image understanding |
|
||
| **Gemma 3** | Gemma-3-4B-IT, Gemma-3-9B-IT, Gemma-3-27B-IT | 4B–27B | Efficient, multilingual |
|
||
| **Qwen 3** | Qwen3-8B, Qwen3-14B, Qwen3-32B, Qwen3-235B-A22B | 0.6B–235B | Reasoning, code, MoE |
|
||
| **Qwen 2.5** | Qwen2.5-7B-Instruct, Qwen2.5-Coder-32B-Instruct | 0.5B–72B | Code, math |
|
||
| **DeepSeek** | DeepSeek-R1-Distill-Llama-8B, DeepSeek-V3-0324 | 1.5B–671B | Reasoning (GRPO), code |
|
||
| **Phi-4** | Phi-4-14B, Phi-4-mini-reasoning | 3.8B–14B | Compact reasoning |
|
||
| **Mistral** | Mistral-7B-Instruct-v0.3, Mistral-Small-24B-Instruct | 7B–24B | Fast, efficient |
|
||
| **Mixtral** | Mixtral-8x7B-Instruct-v0.1, Mixtral-8x22B | 47B–141B | MoE architecture |
|
||
| **CodeLlama** | CodeLlama-7b-Instruct-hf, CodeLlama-34b-Instruct | 7B–34B | Code generation |
|
||
| **StarCoder 2** | StarCoder2-15B, StarCoder2-7B | 3B–15B | Code completion |
|
||
| **Yi** | Yi-1.5-34B-Chat, Yi-1.5-9B-Chat | 6B–34B | Multilingual chat |
|
||
| **InternLM 3** | InternLM3-8B-Instruct | 8B | Chinese + English |
|
||
| **Falcon** | Falcon-11B, Falcon-40B-Instruct | 7B–180B | Open-weight |
|
||
|
||
### Vision Models (with `modality: vision`)
|
||
|
||
| Model | Size | Supported Formats |
|
||
|---|---|---|
|
||
| LLaMA-3.2-11B-Vision-Instruct | 11B | LLaVA, ShareGPT4V |
|
||
| Qwen2-VL-7B-Instruct | 7B | LLaVA, ShareGPT4V |
|
||
| Pixtral-12B-2409 | 12B | LLaVA, ShareGPT4V |
|
||
|
||
### Quick Size Guide
|
||
|
||
| VRAM | Max Model (QLoRA 4-bit) | Example |
|
||
|---|---|---|
|
||
| 8 GB | ~7B | Llama-3.1-8B, Mistral-7B |
|
||
| 16 GB | ~14B | Phi-4-14B, Qwen2.5-14B |
|
||
| 24 GB | ~34B | CodeLlama-34B, Yi-1.5-34B |
|
||
| 48 GB | ~70B | Llama-3.3-70B |
|
||
| 80 GB+ | 70B+ (full) or MoE | Mixtral-8x22B, DeepSeek-V3 |
|
||
|
||
> **Note:** Soup auto-detects your GPU and estimates the optimal batch size. Use `soup doctor` to check your setup.
|
||
|
||
## Requirements
|
||
|
||
- Python 3.9+
|
||
- GPU with CUDA (recommended) or Apple Silicon (MPS) or CPU (experimental)
|
||
- 8 GB+ VRAM for 7B models with QLoRA
|
||
|
||
> **CPU note:** All training tasks (SFT, DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO) work on CPU but will be very slow. Quantization (`4bit`/`8bit`) is auto-disabled on CPU. GRPO on CPU uses `min_new_tokens=1` to prevent empty generation errors. A default chat template is set automatically if the tokenizer lacks one. PPO datasets are tokenized before training to ensure compatibility with trl's experimental API.
|
||
|
||
### Optional Extras
|
||
|
||
| Extra | Install | What it adds |
|
||
|---|---|---|
|
||
| `vision` | `pip install 'soup-cli[vision]'` | Vision/multimodal fine-tuning (Pillow) |
|
||
| `qat` | `pip install 'soup-cli[qat]'` | Quantization-Aware Training (torchao) |
|
||
| `fast` | `pip install 'soup-cli[fast]'` | Unsloth backend (2-5x faster, -80% VRAM) |
|
||
| `ui` | `pip install 'soup-cli[ui]'` | Web UI + inference server (FastAPI + uvicorn) |
|
||
| `serve` | `pip install 'soup-cli[serve]'` | Inference server (FastAPI + uvicorn) |
|
||
| `serve-fast` | `pip install 'soup-cli[serve-fast]'` | vLLM inference backend (2-4x throughput) |
|
||
| `data` | `pip install 'soup-cli[data]'` | Deduplication (MinHash via datasketch) |
|
||
| `eval` | `pip install 'soup-cli[eval]'` | Benchmark evaluation (lm-evaluation-harness) |
|
||
| `deepspeed` | `pip install 'soup-cli[deepspeed]'` | Multi-GPU training (DeepSpeed ZeRO) |
|
||
| `dev` | `pip install 'soup-cli[dev]'` | Tests + linting (pytest, ruff) |
|
||
|
||
## Development
|
||
|
||
```bash
|
||
git clone https://github.com/MakazhanAlpamys/Soup.git
|
||
cd Soup
|
||
pip install -e ".[dev]"
|
||
|
||
# Lint
|
||
ruff check soup_cli/ tests/
|
||
|
||
# Run unit tests (fast, no GPU needed)
|
||
pytest tests/ -v
|
||
|
||
# Run smoke tests (downloads tiny model, runs real training)
|
||
pytest tests/ -m smoke -v
|
||
```
|
||
|
||
## Changelog
|
||
|
||
See [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases) for version history.
|
||
|
||
## License
|
||
|
||
MIT
|