mirror of https://github.com/razor-ai/soup.git
v0.10.2: Fix remaining Windows/CPU issues from re-test
- ASCII progress bar in display.py (replace Unicode blocks with #/-) - Catch UnicodeEncodeError from plotext in data stats (Windows fallback) - Auto-disable 4bit/8bit quantization on CPU (bitsandbytes doesn't support it) - Friendly error messages for CPU tensor/dtype/bf16/torchvision issues - Fix diff.py: torch_dtype= -> dtype= (deprecation warning) - Add torchvision version compatibility check to soup doctor 7 new tests (631 total), ruff clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
f460ad3470
commit
4fca7a9664
|
|
@ -182,4 +182,4 @@ Test suite lives in `tests/`:
|
|||
| `test_ui.py` | Web UI command, FastAPI endpoints, static files, config validation |
|
||||
| `test_vllm_serve.py` | vLLM backend detection, engine creation, serve --backend flag, FastAPI app |
|
||||
| `test_ppo.py` | PPO config, reward model config, data prep, RLHF template, routing, sweep |
|
||||
| `test_bugfixes.py` | v0.10.1 bugfixes: Windows Unicode, PPO params, compute dtype, diff dtype, wandb pin |
|
||||
| `test_bugfixes.py` | v0.10.1-v0.10.2 bugfixes: Unicode, PPO params, compute dtype, diff, CPU compat, doctor |
|
||||
|
|
|
|||
|
|
@ -603,7 +603,7 @@ soup version
|
|||
|
||||
# Full system info (useful for bug reports)
|
||||
soup version --full
|
||||
# -> soup v0.10.1 | Python 3.11.5 | CUDA 12.1 | extras: serve, data
|
||||
# -> soup v0.10.2 | Python 3.11.5 | CUDA 12.1 | extras: serve, data
|
||||
```
|
||||
|
||||
## Web UI
|
||||
|
|
|
|||
120
plan.md
120
plan.md
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
**Repo:** https://github.com/MakazhanAlpamys/Soup
|
||||
**PyPI:** https://pypi.org/project/soup-cli/ (`pip install soup-cli`)
|
||||
**Version:** v0.10.1 | 624 tests | CI green
|
||||
**Version:** v0.10.2 | 631 tests | CI green
|
||||
|
||||
### How to publish
|
||||
|
||||
|
|
@ -16,7 +16,7 @@ git push --tags
|
|||
|
||||
---
|
||||
|
||||
## Completed (v0.1.0 – v0.10.1)
|
||||
## Completed (v0.0.1 – v0.10.2)
|
||||
|
||||
- **CLI:** init, train, chat, push, merge, export, eval, serve, sweep, diff, doctor, quickstart, ui, version
|
||||
- **Data:** inspect, validate, convert, merge, dedup, stats, generate
|
||||
|
|
@ -30,100 +30,74 @@ git push --tags
|
|||
- **Community:** CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md, examples/, FUNDING.yml
|
||||
- **Tests:** 624 tests, 41 files, ruff lint, CI on Python 3.9/3.11/3.12
|
||||
- **v0.10.1 bugfixes:** Windows UnicodeEncodeError, PPO trl compat, compute dtype for CPU, diff torch_dtype, wandb version pin
|
||||
- **v0.10.2 bugfixes:** ASCII progress bar, plotext fallback, auto-disable 4bit on CPU, friendly CPU error messages, torchvision compat check in doctor
|
||||
|
||||
---
|
||||
|
||||
## Roadmap
|
||||
|
||||
### P0 — Critical (v1.0–v1.1)
|
||||
### v0.11.0 — Alignment methods
|
||||
|
||||
#### Community & Marketing
|
||||
- [ ] Discord server
|
||||
- [ ] First Reddit post (r/LocalLLaMA, r/MachineLearning)
|
||||
- [ ] HuggingFace models showcase ("trained-with-soup" tag)
|
||||
- [ ] 3-5 YouTube tutorials / blog posts with benchmarks (Soup vs LLaMA-Factory vs Axolotl)
|
||||
- [ ] Supported models page in README (Llama 4, Gemma 3, Qwen 2.5/3, Phi-4, DeepSeek R1/V3, Mistral, Mixtral)
|
||||
- [ ] KTO (`task: kto`) — unpaired preference data, wraps trl.KTOTrainer
|
||||
- [ ] ORPO (`task: orpo`) — no reference model needed, wraps trl.ORPOTrainer
|
||||
- [ ] SimPO (`task: simpo`) — simple preference optimization
|
||||
- [ ] IPO (`task: ipo`) — identity preference optimization
|
||||
- [ ] Templates: `soup init --template kto`, `--template orpo`
|
||||
|
||||
#### Advanced PEFT methods
|
||||
- [ ] DoRA (`peft_type: dora`) — improved LoRA with magnitude decomposition
|
||||
### v0.12.0 — Advanced PEFT
|
||||
|
||||
- [ ] DoRA (`use_dora: true`) — magnitude decomposition, already in PEFT
|
||||
- [ ] LoRA+ (`loraplus_lr_ratio`) — different lr for A and B matrices
|
||||
- [ ] GaLore — memory-efficient full-parameter training on consumer GPUs
|
||||
- [ ] LoRA+ — different learning rates for A and B matrices
|
||||
|
||||
#### More alignment methods
|
||||
- [ ] KTO (Kahneman-Tversky Optimization) — `task: kto`, doesn't need paired data
|
||||
- [ ] ORPO (Odds Ratio Preference Optimization) — `task: orpo`, no reference model needed
|
||||
- [ ] SimPO — `task: simpo`, simple preference optimization
|
||||
- [ ] IPO (Identity Preference Optimization) — `task: ipo`
|
||||
### v0.13.0 — Batch inference + TensorBoard
|
||||
|
||||
- [ ] `soup infer --input prompts.jsonl --output results.jsonl` — batch inference (reuse diff.py code)
|
||||
- [ ] `--tensorboard` flag in train — `report_to="tensorboard"`, HF Trainer handles it
|
||||
- [ ] Supported models page in README (Llama 4, Gemma 3, Qwen 2.5/3, Phi-4, DeepSeek R1/V3)
|
||||
|
||||
### v0.14.0 — Pre-training + MoE
|
||||
|
||||
#### Pre-training / Continued pre-training
|
||||
- [ ] `task: pretrain` — continued pre-training on raw text
|
||||
- [ ] Support for plain text / tokenized datasets
|
||||
- [ ] Plain text / tokenized datasets support
|
||||
- [ ] MoE model support (Qwen3 30B-A3B, Mixtral, DeepSeek V3)
|
||||
- [ ] ScatterMoE LoRA
|
||||
|
||||
#### Cloud GPU providers
|
||||
- [ ] `soup cloud run --provider runpod --gpu a100 --config soup.yaml`
|
||||
- [ ] RunPod, Vast.ai, Lambda Labs, Modal integration
|
||||
- [ ] Auto-setup: upload data → rent GPU → train → return adapter → teardown
|
||||
- [ ] Cost estimator and budget auto-stop
|
||||
### v0.15.0 — Performance + Long-context
|
||||
|
||||
---
|
||||
|
||||
### P1 — Important (v1.2–v1.3)
|
||||
|
||||
#### MoE support
|
||||
- [ ] Explicit MoE model support (Qwen3 30B-A3B, Mixtral, DeepSeek V3)
|
||||
- [ ] ScatterMoE LoRA for efficient MoE fine-tuning
|
||||
- [ ] Documentation and examples for MoE training
|
||||
|
||||
#### Long-context training
|
||||
- [ ] Sequence parallelism via Ring FlashAttention
|
||||
- [ ] 128k+ context fine-tuning across multiple GPUs
|
||||
- [ ] Neat packing (contamination-free) for long sequences
|
||||
|
||||
#### Embedding models
|
||||
- [ ] `task: embedding` — fine-tune sentence transformers, BGE, E5
|
||||
- [ ] `soup init --template embedding`
|
||||
- [ ] Contrastive loss, triplet loss support
|
||||
|
||||
#### Advanced distributed training
|
||||
- [ ] FSDP2 support alongside DeepSpeed
|
||||
- [ ] Multi-node training via torchrun / Ray
|
||||
- [ ] Tensor Parallelism + Context Parallelism combined
|
||||
|
||||
#### Performance optimizations
|
||||
- [ ] Liger Kernel integration (fused operations)
|
||||
- [ ] FlashAttention-3/4 auto-detection
|
||||
- [ ] SageAttention support
|
||||
- [ ] FSDP2 support alongside DeepSpeed
|
||||
- [ ] Sequence parallelism via Ring FlashAttention
|
||||
- [ ] 128k+ context fine-tuning
|
||||
|
||||
---
|
||||
### v0.16.0 — Embedding models + Export
|
||||
|
||||
### P2 — Nice to have (v1.4+)
|
||||
|
||||
#### Export formats
|
||||
- [ ] `task: embedding` — sentence transformers, BGE, E5
|
||||
- [ ] Contrastive loss, triplet loss
|
||||
- [ ] ONNX export (`soup export --format onnx`)
|
||||
- [ ] TensorRT-LLM export
|
||||
- [ ] Speculative decoding (`soup serve --speculative-decoding`)
|
||||
|
||||
### v0.17.0 — Data + Audio
|
||||
|
||||
- [ ] Local model as data generation provider
|
||||
- [ ] Quality filters (perplexity, coherence scoring)
|
||||
- [ ] `modality: audio` — Qwen2-Audio, Whisper fine-tuning
|
||||
- [ ] SGLang backend for serving
|
||||
|
||||
#### Integrations
|
||||
- [ ] MLflow tracking (enterprise alternative to W&B)
|
||||
- [ ] TensorBoard integration
|
||||
- [ ] `uv` package manager support
|
||||
### v0.18.0 — Cloud GPU (monetization)
|
||||
|
||||
#### Serving improvements
|
||||
- [ ] Speculative decoding (`soup serve --speculative-decoding`)
|
||||
- [ ] Batch inference mode (`soup infer --input prompts.jsonl --output results.jsonl`)
|
||||
Last step — all features are built, product is ready to sell.
|
||||
|
||||
#### Data improvements
|
||||
- [ ] Multi-agent synthetic data generation (GraphGen-style)
|
||||
- [ ] Quality filters for generated data (perplexity, coherence scoring)
|
||||
- [ ] Local model as data generation provider (not just OpenAI API)
|
||||
|
||||
#### Audio modality
|
||||
- [ ] `modality: audio` — Qwen2-Audio, Whisper fine-tuning
|
||||
- [ ] Audio dataset formats and inspection
|
||||
|
||||
#### Smart suggestions
|
||||
- [ ] Auto-detect hardware and recommend optimal config
|
||||
- [ ] "You have 2×H100 — recommend FSDP2 + Unsloth" style hints
|
||||
- [ ] `soup login` — user registration, Stripe card linking
|
||||
- [ ] `soup cloud run --config soup.yaml --gpu a100` — cloud training
|
||||
- [ ] Backend API (FastAPI on VPS) — pod management, billing
|
||||
- [ ] RunPod API integration (our account, 20-30% markup)
|
||||
- [ ] `soup cloud status` — usage, balance, history
|
||||
- [ ] Landing page soup.cloud
|
||||
- [ ] Cost estimator and budget auto-stop
|
||||
- [ ] Vast.ai, Lambda Labs, Modal — additional providers
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "soup-cli"
|
||||
version = "0.10.1"
|
||||
version = "0.10.2"
|
||||
description = "Fine-tune LLMs in one command. No SSH, no config hell."
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
"""Soup CLI — Fine-tune LLMs in one command."""
|
||||
|
||||
__version__ = "0.10.1"
|
||||
__version__ = "0.10.2"
|
||||
|
|
|
|||
|
|
@ -361,6 +361,11 @@ def stats(
|
|||
plt.ylabel("Count")
|
||||
plt.theme("dark")
|
||||
plt.show()
|
||||
except UnicodeEncodeError:
|
||||
console.print(
|
||||
"\n[dim]Histogram skipped (Windows encoding).[/] "
|
||||
"Set PYTHONIOENCODING=utf-8 to enable."
|
||||
)
|
||||
except ImportError:
|
||||
console.print(
|
||||
"\n[dim]Install plotext for histograms:[/] [bold]pip install plotext[/]"
|
||||
|
|
|
|||
|
|
@ -231,7 +231,7 @@ def _load_model(model_path: str, base_model: Optional[str], device: str):
|
|||
base_model,
|
||||
trust_remote_code=True,
|
||||
device_map="auto",
|
||||
torch_dtype=torch.float16,
|
||||
dtype=torch.float16,
|
||||
)
|
||||
model_obj = PeftModel.from_pretrained(base, model_path)
|
||||
else:
|
||||
|
|
@ -239,7 +239,7 @@ def _load_model(model_path: str, base_model: Optional[str], device: str):
|
|||
model_path,
|
||||
trust_remote_code=True,
|
||||
device_map="auto",
|
||||
torch_dtype=torch.float16,
|
||||
dtype=torch.float16,
|
||||
)
|
||||
|
||||
model_obj.eval()
|
||||
|
|
|
|||
|
|
@ -100,6 +100,9 @@ def doctor():
|
|||
|
||||
console.print(table)
|
||||
|
||||
# Check torchvision + torch compatibility
|
||||
_check_torchvision_compat(issues)
|
||||
|
||||
# Summary
|
||||
if issues:
|
||||
console.print(f"\n[yellow]Found {len(issues)} issue(s):[/]")
|
||||
|
|
@ -162,6 +165,34 @@ def _check_gpu():
|
|||
)
|
||||
|
||||
|
||||
def _check_torchvision_compat(issues: list):
|
||||
"""Check that torchvision version is compatible with torch."""
|
||||
try:
|
||||
import torch
|
||||
import torchvision
|
||||
|
||||
torch_ver = torch.__version__.split("+")[0]
|
||||
tv_ver = torchvision.__version__.split("+")[0]
|
||||
torch_minor = ".".join(torch_ver.split(".")[:2])
|
||||
tv_minor = ".".join(tv_ver.split(".")[:2])
|
||||
|
||||
# Known compatible pairs (torch minor -> torchvision minor)
|
||||
compat = {
|
||||
"2.6": "0.21", "2.5": "0.20", "2.4": "0.19",
|
||||
"2.3": "0.18", "2.2": "0.17", "2.1": "0.16", "2.0": "0.15",
|
||||
}
|
||||
expected_tv = compat.get(torch_minor)
|
||||
if expected_tv and not tv_minor.startswith(expected_tv):
|
||||
msg = (
|
||||
f"torchvision {tv_ver} may be incompatible with torch {torch_ver}. "
|
||||
f"Expected torchvision {expected_tv}.x"
|
||||
)
|
||||
console.print(f" [yellow]Warning:[/] {msg}")
|
||||
issues.append(msg)
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
|
||||
def _version_ok(installed: str, minimum: str) -> bool:
|
||||
"""Check if installed version meets minimum requirement."""
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -107,12 +107,13 @@ def train(
|
|||
device, device_name = detect_device()
|
||||
gpu_info = get_gpu_info()
|
||||
|
||||
# Warn about quantization on CPU
|
||||
# Auto-disable quantization on CPU (bitsandbytes doesn't support CPU)
|
||||
if device == "cpu" and cfg.training.quantization in ("4bit", "8bit"):
|
||||
console.print(
|
||||
f"[yellow]Warning: {cfg.training.quantization} quantization on CPU "
|
||||
"may cause errors. Consider using quantization: none for CPU.[/]"
|
||||
f"[yellow]Warning: {cfg.training.quantization} quantization is not "
|
||||
"supported on CPU. Switching to quantization: none.[/]"
|
||||
)
|
||||
cfg.training.quantization = "none"
|
||||
|
||||
backend_label = cfg.backend
|
||||
if cfg.backend == "unsloth":
|
||||
|
|
|
|||
|
|
@ -59,7 +59,7 @@ class TrainingDisplay:
|
|||
progress_pct = 0
|
||||
bar_width = 30
|
||||
filled = int(bar_width * progress_pct / 100)
|
||||
bar = "\u2588" * filled + "\u2591" * (bar_width - filled)
|
||||
bar = "#" * filled + "-" * (bar_width - filled)
|
||||
|
||||
epochs = self.config.training.epochs
|
||||
epoch_str = f"Epoch {self.current_epoch:.1f}/{epochs}"
|
||||
|
|
|
|||
|
|
@ -72,6 +72,27 @@ ERROR_MAP = [
|
|||
"BitsAndBytes is not installed (needed for quantization).",
|
||||
"Run: pip install bitsandbytes>=0.41.0",
|
||||
),
|
||||
# CPU / quantization issues
|
||||
(
|
||||
"expanded size of the tensor",
|
||||
"Model generation failed (likely 4bit quantization on CPU).",
|
||||
"Use a GPU, or set quantization: none in your config for CPU training.",
|
||||
),
|
||||
(
|
||||
"expected m1 and m2 to have the same dtype",
|
||||
"Dtype mismatch (likely 4bit quantization on CPU).",
|
||||
"Use a GPU, or set quantization: none in your config for CPU training.",
|
||||
),
|
||||
(
|
||||
"Your setup doesn't support bf16",
|
||||
"This training task requires GPU with bf16 support.",
|
||||
"Use a CUDA GPU, or try a simpler task (SFT/DPO work on CPU).",
|
||||
),
|
||||
(
|
||||
"nms does not exist",
|
||||
"torchvision version is incompatible with torch.",
|
||||
"Run: pip install torchvision --force-reinstall (or check soup doctor).",
|
||||
),
|
||||
# Connection errors
|
||||
(
|
||||
"ConnectionError",
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
"""Tests for v0.10.1 bug fixes - Windows Unicode, PPO params, dtype, diff."""
|
||||
"""Tests for v0.10.1/v0.10.2 bug fixes - Unicode, PPO, dtype, CPU compat."""
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
|
@ -135,15 +135,15 @@ class TestComputeDtype:
|
|||
class TestDiffModelLoading:
|
||||
"""Test diff command uses correct parameter names."""
|
||||
|
||||
def test_load_model_uses_torch_dtype(self):
|
||||
"""_load_model should pass torch_dtype, not dtype."""
|
||||
def test_load_model_uses_dtype(self):
|
||||
"""_load_model should pass dtype= (not the old torch_dtype=)."""
|
||||
import inspect
|
||||
|
||||
from soup_cli.commands.diff import _load_model
|
||||
|
||||
source = inspect.getsource(_load_model)
|
||||
assert "torch_dtype=" in source
|
||||
assert "dtype=" not in source or "torch_dtype=" in source
|
||||
assert "dtype=torch.float16" in source
|
||||
assert "torch_dtype=" not in source
|
||||
|
||||
|
||||
# --- BUG-006: wandb version pin ---
|
||||
|
|
@ -165,11 +165,98 @@ class TestWandbVersionPin:
|
|||
class TestCPUQuantWarning:
|
||||
"""Test that CPU + quantization produces a warning."""
|
||||
|
||||
def test_train_source_has_cpu_quant_warning(self):
|
||||
"""train.py should warn about quantization on CPU."""
|
||||
def test_train_auto_disables_quant_on_cpu(self):
|
||||
"""train.py should auto-disable quantization on CPU."""
|
||||
import inspect
|
||||
|
||||
from soup_cli.commands import train
|
||||
|
||||
source = inspect.getsource(train)
|
||||
assert "quantization on CPU" in source
|
||||
assert "quantization is not" in source
|
||||
assert 'cfg.training.quantization = "none"' in source
|
||||
|
||||
|
||||
# --- v0.10.2: Display progress bar uses ASCII ---
|
||||
|
||||
|
||||
class TestDisplayASCII:
|
||||
"""Test that training display uses ASCII-safe progress bars."""
|
||||
|
||||
def test_progress_bar_uses_ascii_chars(self):
|
||||
"""Progress bar should use # and - instead of Unicode blocks."""
|
||||
import inspect
|
||||
|
||||
from soup_cli.monitoring.display import TrainingDisplay
|
||||
|
||||
source = inspect.getsource(TrainingDisplay)
|
||||
assert '"#"' in source
|
||||
assert '"-"' in source
|
||||
assert "\\u2588" not in source
|
||||
assert "\\u2591" not in source
|
||||
|
||||
|
||||
# --- v0.10.2: Plotext UnicodeEncodeError handling ---
|
||||
|
||||
|
||||
class TestPlotextFallback:
|
||||
"""Test that plotext errors are caught gracefully."""
|
||||
|
||||
def test_stats_catches_unicode_error(self):
|
||||
"""data stats should catch UnicodeEncodeError from plotext."""
|
||||
import inspect
|
||||
|
||||
from soup_cli.commands import data
|
||||
|
||||
source = inspect.getsource(data)
|
||||
assert "UnicodeEncodeError" in source
|
||||
|
||||
|
||||
# --- v0.10.2: Error messages for CPU issues ---
|
||||
|
||||
|
||||
class TestCPUErrorMessages:
|
||||
"""Test friendly error messages for CPU-specific failures."""
|
||||
|
||||
def test_tensor_size_error_mapped(self):
|
||||
"""Tensor expansion error should have a friendly message."""
|
||||
from soup_cli.utils.errors import ERROR_MAP
|
||||
|
||||
patterns = [pattern for pattern, _, _ in ERROR_MAP]
|
||||
assert any("expanded size" in p for p in patterns)
|
||||
|
||||
def test_dtype_mismatch_error_mapped(self):
|
||||
"""Dtype mismatch error should have a friendly message."""
|
||||
from soup_cli.utils.errors import ERROR_MAP
|
||||
|
||||
patterns = [pattern for pattern, _, _ in ERROR_MAP]
|
||||
assert any("same dtype" in p for p in patterns)
|
||||
|
||||
def test_bf16_error_mapped(self):
|
||||
"""bf16 GPU error should have a friendly message."""
|
||||
from soup_cli.utils.errors import ERROR_MAP
|
||||
|
||||
patterns = [pattern for pattern, _, _ in ERROR_MAP]
|
||||
assert any("bf16" in p for p in patterns)
|
||||
|
||||
def test_torchvision_error_mapped(self):
|
||||
"""torchvision nms error should have a friendly message."""
|
||||
from soup_cli.utils.errors import ERROR_MAP
|
||||
|
||||
patterns = [pattern for pattern, _, _ in ERROR_MAP]
|
||||
assert any("nms" in p for p in patterns)
|
||||
|
||||
|
||||
# --- v0.10.2: Doctor torchvision check ---
|
||||
|
||||
|
||||
class TestDoctorTorchvisionCheck:
|
||||
"""Test that soup doctor checks torchvision compatibility."""
|
||||
|
||||
def test_doctor_has_torchvision_check(self):
|
||||
"""doctor.py should have torchvision compatibility check."""
|
||||
import inspect
|
||||
|
||||
from soup_cli.commands import doctor
|
||||
|
||||
source = inspect.getsource(doctor)
|
||||
assert "_check_torchvision_compat" in source
|
||||
|
|
|
|||
Loading…
Reference in New Issue