Soup
Fine-tune LLMs in one command. No SSH, no config hell.
Quick Start ·
Features ·
Data Tools ·
Tracking ·
Eval ·
Commands
---
Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.
```bash
pip install soup-cli
soup init --template chat
soup train
```
## Why Soup?
Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that.
- **Zero SSH.** Never SSH into a broken GPU box again.
- **One config.** A simple YAML file is all you need.
- **Auto everything.** Batch size, GPU detection, quantization — handled.
- **Works locally.** Train on your own GPU with QLoRA. No cloud required.
## Quick Start
### 1. Install
```bash
# From PyPI (recommended):
pip install soup-cli
# Or from GitHub (latest dev):
pip install git+https://github.com/MakazhanAlpamys/Soup.git
```
### 2. Create config
```bash
# Interactive wizard
soup init
# Or use a template
soup init --template chat # conversational fine-tune
soup init --template code # code generation
soup init --template medical # domain expert
soup init --template reasoning # GRPO reasoning training
soup init --template vision # vision/multimodal fine-tune
soup init --template rlhf # full RLHF pipeline (SFT→RM→PPO)
```
### 3. Train
```bash
soup train --config soup.yaml
```
That's it. Soup handles LoRA setup, quantization, batch size, monitoring, and checkpoints.
### 4. Test your model
```bash
soup chat --model ./output
```
### 5. Push to HuggingFace
```bash
soup push --model ./output --repo your-username/my-model
```
### 6. Merge & Export
```bash
# Merge LoRA adapter with base model
soup merge --adapter ./output
# Export to GGUF for Ollama / llama.cpp
soup export --model ./output --format gguf --quant q4_k_m
```
## Config Example
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
# backend: unsloth # 2-5x faster, pip install 'soup-cli[fast]'
data:
train: ./data/train.jsonl
format: alpaca
val_split: 0.1
training:
epochs: 3
lr: 2e-5
batch_size: auto
lora:
r: 64
alpha: 16
quantization: 4bit
output: ./output
```
## Unsloth Backend (2-5x Faster Training)
Use the [Unsloth](https://github.com/unslothai/unsloth) backend for significantly faster training and up to 80% less VRAM:
```bash
# Install unsloth support
pip install 'soup-cli[fast]'
```
Then add one line to your config:
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
backend: unsloth # 2-5x faster, -80% VRAM
data:
train: ./data/train.jsonl
format: alpaca
training:
epochs: 3
lr: 2e-5
quantization: 4bit
lora:
r: 64
alpha: 16
```
Works with all training tasks: SFT, DPO, GRPO, and PPO. If unsloth is installed but not enabled, Soup will suggest it automatically.
> **Tip:** Soup auto-detects unsloth. When installed, you'll see a hint during `soup train` if you haven't enabled it yet.
## Vision / Multimodal Fine-tuning
Fine-tune vision-language models (LLaMA-3.2-Vision, Qwen2-VL, Pixtral) on image+text data:
```bash
# Install vision support
pip install 'soup-cli[vision]'
# Create a vision config
soup init --template vision
# Train
soup train --config soup.yaml
```
```yaml
base: meta-llama/Llama-3.2-11B-Vision-Instruct
task: sft
modality: vision
data:
train: ./data/vision_train.jsonl
format: llava
image_dir: ./data/images
val_split: 0.1
training:
epochs: 3
lr: 1e-5
quantization: 4bit
lora:
r: 64
alpha: 16
```
**Supported vision data formats:**
**LLaVA:**
```json
{"image": "photo.jpg", "conversations": [{"from": "human", "value": "\nDescribe this image."}, {"from": "gpt", "value": "A cat on a mat."}]}
```
**ShareGPT4V:**
```json
{"image": "chart.png", "conversations": [{"from": "human", "value": "\nWhat does this show?"}, {"from": "gpt", "value": "Quarterly revenue."}]}
```
`soup data inspect` automatically shows image statistics (count, formats, missing files) for vision datasets.
## Quantization-Aware Training (QAT)
Train with simulated quantization for significantly better post-quantization quality compared to standard QLoRA:
```bash
# Install QAT support
pip install 'soup-cli[qat]'
```
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
data:
train: ./data/train.jsonl
format: alpaca
training:
epochs: 3
lr: 2e-5
quantization: 4bit
quantization_aware: true # Enable QAT
lora:
r: 64
alpha: 16
output: ./output
```
**When to use QAT vs post-training quantization:**
- **QAT** (`quantization_aware: true`): Better quality when you plan to deploy with aggressive quantization (int8/int4). ~5-10% slower training, but the model learns to compensate for quantization noise.
- **Post-training quantization** (default): Faster training, good enough for most use cases. Quantize after training with `soup export --quant q4_k_m`.
QAT works with all training tasks (SFT, DPO, GRPO, PPO) and vision modality. Not compatible with the unsloth backend. After QAT training, export to GGUF normally with `soup export`.
## DPO Training
Train with preference data using Direct Preference Optimization:
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: dpo
data:
train: ./data/preferences.jsonl
format: dpo
training:
epochs: 3
dpo_beta: 0.1
lora:
r: 64
alpha: 16
quantization: 4bit
```
## GRPO Training (Reasoning)
Train reasoning models with Group Relative Policy Optimization (DeepSeek-R1 style):
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: grpo
data:
train: ./data/reasoning_train.jsonl
format: sharegpt
max_length: 4096
training:
epochs: 3
lr: 1e-5
grpo_beta: 0.1
num_generations: 4
reward_fn: accuracy # or 'format', or path to custom .py
lora:
r: 64
alpha: 16
quantization: 4bit
```
```bash
# Create a reasoning config
soup init --template reasoning
# Train
soup train --config soup.yaml
```
**Built-in reward functions:**
- `accuracy` — checks if the final answer matches expected (supports `####` and `\boxed{}` formats)
- `format` — checks for structured `...` reasoning blocks
**Custom reward functions** — point to a Python file:
```python
# my_reward.py
def reward_fn(completions, **kwargs):
"""Score each completion. Return list of floats."""
return [1.0 if "correct" in c[-1]["content"] else 0.0 for c in completions]
```
```yaml
training:
reward_fn: ./my_reward.py
```
## PPO / Full RLHF Pipeline
Train models with the full RLHF pipeline: SFT warmup → Reward Model → PPO alignment.
```bash
# Create an RLHF config
soup init --template rlhf
```
**Step 1: SFT warmup** — fine-tune a base model on your data:
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
data:
train: ./data/train.jsonl
format: alpaca
output: ./output_sft
```
**Step 2: Train reward model** — learn preferences from human feedback:
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: reward_model
data:
train: ./data/preferences.jsonl
format: dpo
output: ./output_rm
```
**Step 3: PPO alignment** — optimize the policy using the reward model:
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: ppo
data:
train: ./data/prompts.jsonl
format: chatml
training:
reward_model: ./output_rm
ppo_epochs: 4
ppo_clip_ratio: 0.2
ppo_kl_penalty: 0.05
lora:
r: 64
alpha: 16
quantization: 4bit
output: ./output_ppo
```
PPO supports two reward sources:
- **Reward model** (`reward_model`): pre-trained reward model (from step 2)
- **Reward function** (`reward_fn`): callable function (same as GRPO — `accuracy`, `format`, or custom `.py`)
## Chat with your model
```bash
# Chat with a LoRA adapter (auto-detects base model)
soup chat --model ./output
# Specify base model explicitly
soup chat --model ./output --base meta-llama/Llama-3.1-8B-Instruct
# Adjust generation
soup chat --model ./output --temperature 0.3 --max-tokens 256
```
## Push to HuggingFace
```bash
# Upload model to HF Hub
soup push --model ./output --repo your-username/my-model
# Make it private
soup push --model ./output --repo your-username/my-model --private
```
## Merge LoRA Adapter
Merge a LoRA adapter with its base model into a standalone model:
```bash
# Auto-detect base model from adapter_config.json
soup merge --adapter ./output --output ./merged
# Specify base model and dtype
soup merge --adapter ./output --base meta-llama/Llama-3.1-8B --dtype bfloat16
```
## Export to GGUF
Export models to GGUF format for use with [Ollama](https://ollama.com/) and [llama.cpp](https://github.com/ggerganov/llama.cpp):
```bash
# Export LoRA adapter (auto-merges with base, then converts)
soup export --model ./output --format gguf --quant q4_k_m
# Export with different quantizations
soup export --model ./output --format gguf --quant q8_0
soup export --model ./output --format gguf --quant f16
# Export a full (already merged) model
soup export --model ./merged --format gguf
# Specify llama.cpp path manually
soup export --model ./output --format gguf --llama-cpp /path/to/llama.cpp
```
Supported quantizations: `q4_0`, `q4_k_m`, `q5_k_m`, `q8_0`, `f16`, `f32`
After export, use with Ollama:
```bash
echo 'FROM ./my-model.q4_k_m.gguf' > Modelfile
ollama create my-model -f Modelfile
ollama run my-model
```
## Resume Training
Resume a training run from a checkpoint:
```bash
# Auto-detect latest checkpoint in output directory
soup train --config soup.yaml --resume auto
# Resume from a specific checkpoint
soup train --config soup.yaml --resume ./output/checkpoint-500
```
## Weights & Biases Integration
Send training metrics to [W&B](https://wandb.ai/) for cloud-based experiment tracking:
```bash
# Enable W&B logging (requires: pip install wandb)
soup train --config soup.yaml --wandb
```
Make sure `WANDB_API_KEY` is set or run `wandb login` first.
## Inference Server
Start a local OpenAI-compatible inference server:
```bash
# Install server dependencies
pip install 'soup-cli[serve]'
# Start server
soup serve --model ./output --port 8000
# With custom settings
soup serve --model ./output --port 8080 --host 127.0.0.1 --max-tokens 1024
```
Endpoints:
- `POST /v1/chat/completions` — chat completions (streaming supported)
- `GET /v1/models` — list available models
- `GET /health` — health check
Compatible with OpenAI SDK:
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
response = client.chat.completions.create(
model="output",
messages=[{"role": "user", "content": "Hello!"}],
)
```
### vLLM Backend (2-4x Faster Inference)
Use [vLLM](https://github.com/vllm-project/vllm) for significantly better throughput in production:
```bash
# Install vLLM support
pip install 'soup-cli[serve-fast]'
# Start with vLLM backend
soup serve --model ./output --backend vllm
# Multi-GPU with tensor parallelism
soup serve --model ./output --backend vllm --tensor-parallel 2
# Control GPU memory usage
soup serve --model ./output --backend vllm --gpu-memory 0.8
```
> **Tip:** Soup auto-detects vLLM. When installed, you'll see a hint during `soup serve` if you haven't enabled it yet.
> **Note (v0.10.10+):** `max_tokens` is capped at 16,384 per request. Error details are never exposed in HTTP responses.
## Synthetic Data Generation
Generate training data using LLMs:
```bash
# Generate using OpenAI API
soup data generate --prompt "Create math word problems" --count 100 --format alpaca
# Use a different model
soup data generate --prompt "Medical Q&A pairs" --model gpt-4o --count 500
# Deduplicate against existing data
soup data generate --prompt "..." --count 200 --dedup-with existing.jsonl
# Use seed examples to guide style
soup data generate --prompt "..." --seed examples.jsonl --count 100
```
## Hyperparameter Sweep
Search for the best hyperparameters:
```bash
# Grid search over learning rate and LoRA rank
soup sweep --config soup.yaml --param lr=1e-5,2e-5,5e-5 --param lora_r=8,16,32
# Random search with max runs
soup sweep --config soup.yaml --param lr=1e-5,2e-5,5e-5 --strategy random --max-runs 5
# Preview without running
soup sweep --config soup.yaml --param lr=1e-5,2e-5 --param epochs=2,3 --dry-run
# Early stopping: skip remaining runs if loss exceeds 1.5x best
soup sweep --config soup.yaml --param lr=1e-5,2e-5,5e-5 --early-stop 1.5
```
## Model Comparison
Compare outputs of two models side-by-side:
```bash
# Compare with inline prompts
soup diff --model-a ./model_v1 --model-b ./model_v2 --prompt "Explain gravity"
# Compare with a prompts file
soup diff --model-a ./base --model-b ./finetuned --prompts test_prompts.jsonl
# Save results
soup diff --model-a ./a --model-b ./b --prompts prompts.txt --output results.jsonl
```
## Multi-GPU / DeepSpeed
Train on multiple GPUs with DeepSpeed:
```bash
# ZeRO Stage 2 (recommended for most cases)
soup train --config soup.yaml --deepspeed zero2
# ZeRO Stage 3 (for very large models)
soup train --config soup.yaml --deepspeed zero3
# ZeRO Stage 2 with CPU offload (memory-constrained)
soup train --config soup.yaml --deepspeed zero2_offload
# Custom DeepSpeed config
soup train --config soup.yaml --deepspeed ./my_ds_config.json
```
## Quickstart Demo
Run a complete demo in one command — creates sample data, config, and trains a tiny model:
```bash
# Full demo (creates data + config + trains TinyLlama)
soup quickstart
# Just create files without training
soup quickstart --dry-run
# Skip confirmation
soup quickstart --yes
```
## Health Check
Check your environment for compatibility issues:
```bash
soup doctor
```
Shows: Python version, GPU availability, all dependency versions, and fix suggestions.
## Version Info
```bash
# Basic version
soup version
# Full system info (useful for bug reports)
soup version --full
# -> soup v0.10.9 | Python 3.11.5 | CUDA 12.1 | extras: serve, data
```
## Web UI
Launch a local web interface to manage experiments, start training, explore data, and chat with models — all from your browser.
```bash
pip install 'soup-cli[ui]'
soup ui
# -> opens http://127.0.0.1:7860 in your browser
# -> prints auth token to console
```
**Pages:**
- **Dashboard** — view all experiment runs, loss charts, system info
- **New Training** — create configs from templates, validate, and start training
- **Data Explorer** — browse and inspect datasets (JSONL, JSON, CSV, Parquet)
- **Model Chat** — chat with a running `soup serve` inference server
**Security (v0.10.10+):** The Web UI generates a random auth token at startup (printed to console). All mutating endpoints (start/stop training, delete runs, inspect data, validate config) require `Authorization: Bearer ` header. CORS is restricted to the served origin. Data inspection is sandboxed to the working directory.
```bash
# Custom port, don't auto-open browser
soup ui --port 8080 --no-browser
```
## Error Handling
Soup shows friendly error messages by default (2-3 lines with a fix suggestion). For full tracebacks:
```bash
# Global flag goes BEFORE the command
soup --verbose train --config soup.yaml
# Works with any command
soup --verbose eval --model ./output --benchmarks mmlu
```
> **Note:** `--verbose` is a global flag — it must go **before** the command name, not after.
## Data Formats
Soup supports these formats (auto-detected). Files can be JSONL, JSON, CSV, or Parquet.
**Alpaca:**
```json
{"instruction": "Explain gravity", "input": "", "output": "Gravity is..."}
```
**ShareGPT:**
```json
{"conversations": [{"from": "human", "value": "Hi"}, {"from": "gpt", "value": "Hello!"}]}
```
**ChatML:**
```json
{"messages": [{"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello!"}]}
```
**DPO (preference pairs):**
```json
{"prompt": "Explain gravity", "chosen": "Gravity is a force...", "rejected": "I don't know"}
```
**LLaVA (vision):**
```json
{"image": "photo.jpg", "conversations": [{"from": "human", "value": "\nDescribe this."}, {"from": "gpt", "value": "A cat."}]}
```
**ShareGPT4V (vision):**
```json
{"image": "chart.png", "conversations": [{"from": "human", "value": "\nExplain this chart."}, {"from": "gpt", "value": "Revenue growth."}]}
```
## Data Tools
```bash
# Inspect a dataset
soup data inspect ./data/train.jsonl
# Validate format
soup data validate ./data/train.jsonl --format alpaca
# Convert between formats
soup data convert ./data/train.jsonl --to sharegpt --output converted.jsonl
# Merge multiple datasets
soup data merge data1.jsonl data2.jsonl --output merged.jsonl --shuffle
# Remove near-duplicates (requires: pip install 'soup-cli[data]')
soup data dedup ./data/train.jsonl --threshold 0.8
# Extended statistics (length distribution, token counts, languages)
soup data stats ./data/train.jsonl
```
## Experiment Tracking
Every `soup train` run is automatically tracked in a local SQLite database (`~/.soup/experiments.db`).
```bash
# List all training runs
soup runs
# Show detailed info + loss curve for a run
soup runs show run_20260223_143052_a1b2
# Compare two runs side by side
soup runs compare run_1 run_2
# Delete a run
soup runs delete run_1
```
## Model Evaluation
Evaluate models on standard benchmarks using [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness):
```bash
# Install eval dependencies
pip install 'soup-cli[eval]'
# Evaluate on benchmarks
soup eval --model ./output --benchmarks mmlu,gsm8k,hellaswag
# Link results to a training run
soup eval --model ./output --benchmarks mmlu --run-id run_20260223_143052_a1b2
```
## All Commands
```
soup init [--template chat|code|medical|reasoning|vision|rlhf] Create config
soup train --config soup.yaml Start training
soup chat --model ./output Interactive chat
soup push --model ./output --repo user/name Upload to HuggingFace
soup merge --adapter ./output Merge LoRA with base model
soup export --model ./output --format gguf Export to GGUF (Ollama)
soup eval --model ./output --benchmarks mmlu Evaluate on benchmarks
soup serve --model ./output --port 8000 OpenAI-compatible API server
soup serve --model ./output --backend vllm vLLM backend (2-4x throughput)
soup sweep --config soup.yaml --param lr=... Hyperparameter search
soup diff --model-a ./a --model-b ./b Compare two models
soup data inspect View dataset stats
soup data validate --format alpaca Check format
soup data convert --to chatml Convert between formats
soup data merge data1.jsonl data2.jsonl Combine datasets
soup data dedup --threshold 0.8 Remove duplicates (MinHash)
soup data stats Extended statistics
soup data generate --prompt "..." --count 100 Generate synthetic data
soup runs List training runs
soup runs show Run details + loss graph
soup runs compare Compare two runs
soup ui [--port 7860] Web UI (experiments, training, data)
soup doctor Check environment
soup quickstart [--dry-run] Full demo
soup version [--full] Show version (--full: system info)
soup --verbose Full traceback on errors
```
## Requirements
- Python 3.9+
- GPU with CUDA (recommended) or Apple Silicon (MPS) or CPU (experimental)
- 8 GB+ VRAM for 7B models with QLoRA
> **CPU note:** All training tasks (SFT, DPO, GRPO, PPO) work on CPU but will be very slow. Quantization (`4bit`/`8bit`) is auto-disabled on CPU. GRPO on CPU uses `min_new_tokens=1` to prevent empty generation errors. A default chat template is set automatically if the tokenizer lacks one. PPO datasets are tokenized before training to ensure compatibility with trl's experimental API.
### Optional Extras
| Extra | Install | What it adds |
|---|---|---|
| `vision` | `pip install 'soup-cli[vision]'` | Vision/multimodal fine-tuning (Pillow) |
| `qat` | `pip install 'soup-cli[qat]'` | Quantization-Aware Training (torchao) |
| `fast` | `pip install 'soup-cli[fast]'` | Unsloth backend (2-5x faster, -80% VRAM) |
| `ui` | `pip install 'soup-cli[ui]'` | Web UI + inference server (FastAPI + uvicorn) |
| `serve` | `pip install 'soup-cli[serve]'` | Inference server (FastAPI + uvicorn) |
| `serve-fast` | `pip install 'soup-cli[serve-fast]'` | vLLM inference backend (2-4x throughput) |
| `data` | `pip install 'soup-cli[data]'` | Deduplication (MinHash via datasketch) |
| `eval` | `pip install 'soup-cli[eval]'` | Benchmark evaluation (lm-evaluation-harness) |
| `deepspeed` | `pip install 'soup-cli[deepspeed]'` | Multi-GPU training (DeepSpeed ZeRO) |
| `dev` | `pip install 'soup-cli[dev]'` | Tests + linting (pytest, ruff) |
## Development
```bash
git clone https://github.com/MakazhanAlpamys/Soup.git
cd Soup
pip install -e ".[dev]"
# Lint
ruff check soup_cli/ tests/
# Run unit tests (fast, no GPU needed)
pytest tests/ -v
# Run smoke tests (downloads tiny model, runs real training)
pytest tests/ -m smoke -v
```
## Changelog
See [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases) for version history.
## License
MIT