v0.10.2: Fix remaining Windows/CPU issues from re-test

- ASCII progress bar in display.py (replace Unicode blocks with #/-)
- Catch UnicodeEncodeError from plotext in data stats (Windows fallback)
- Auto-disable 4bit/8bit quantization on CPU (bitsandbytes doesn't support it)
- Friendly error messages for CPU tensor/dtype/bf16/torchvision issues
- Fix diff.py: torch_dtype= -> dtype= (deprecation warning)
- Add torchvision version compatibility check to soup doctor

7 new tests (631 total), ruff clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alpamys 2026-03-24 13:20:27 +05:00
parent f460ad3470
commit 4fca7a9664
12 changed files with 210 additions and 91 deletions

View File

@ -182,4 +182,4 @@ Test suite lives in `tests/`:
| `test_ui.py` | Web UI command, FastAPI endpoints, static files, config validation |
| `test_vllm_serve.py` | vLLM backend detection, engine creation, serve --backend flag, FastAPI app |
| `test_ppo.py` | PPO config, reward model config, data prep, RLHF template, routing, sweep |
| `test_bugfixes.py` | v0.10.1 bugfixes: Windows Unicode, PPO params, compute dtype, diff dtype, wandb pin |
| `test_bugfixes.py` | v0.10.1-v0.10.2 bugfixes: Unicode, PPO params, compute dtype, diff, CPU compat, doctor |

View File

@ -603,7 +603,7 @@ soup version
# Full system info (useful for bug reports)
soup version --full
# -> soup v0.10.1 | Python 3.11.5 | CUDA 12.1 | extras: serve, data
# -> soup v0.10.2 | Python 3.11.5 | CUDA 12.1 | extras: serve, data
```
## Web UI

120
plan.md
View File

@ -2,7 +2,7 @@
**Repo:** https://github.com/MakazhanAlpamys/Soup
**PyPI:** https://pypi.org/project/soup-cli/ (`pip install soup-cli`)
**Version:** v0.10.1 | 624 tests | CI green
**Version:** v0.10.2 | 631 tests | CI green
### How to publish
@ -16,7 +16,7 @@ git push --tags
---
## Completed (v0.1.0 v0.10.1)
## Completed (v0.0.1 v0.10.2)
- **CLI:** init, train, chat, push, merge, export, eval, serve, sweep, diff, doctor, quickstart, ui, version
- **Data:** inspect, validate, convert, merge, dedup, stats, generate
@ -30,100 +30,74 @@ git push --tags
- **Community:** CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md, examples/, FUNDING.yml
- **Tests:** 624 tests, 41 files, ruff lint, CI on Python 3.9/3.11/3.12
- **v0.10.1 bugfixes:** Windows UnicodeEncodeError, PPO trl compat, compute dtype for CPU, diff torch_dtype, wandb version pin
- **v0.10.2 bugfixes:** ASCII progress bar, plotext fallback, auto-disable 4bit on CPU, friendly CPU error messages, torchvision compat check in doctor
---
## Roadmap
### P0 — Critical (v1.0v1.1)
### v0.11.0 — Alignment methods
#### Community & Marketing
- [ ] Discord server
- [ ] First Reddit post (r/LocalLLaMA, r/MachineLearning)
- [ ] HuggingFace models showcase ("trained-with-soup" tag)
- [ ] 3-5 YouTube tutorials / blog posts with benchmarks (Soup vs LLaMA-Factory vs Axolotl)
- [ ] Supported models page in README (Llama 4, Gemma 3, Qwen 2.5/3, Phi-4, DeepSeek R1/V3, Mistral, Mixtral)
- [ ] KTO (`task: kto`) — unpaired preference data, wraps trl.KTOTrainer
- [ ] ORPO (`task: orpo`) — no reference model needed, wraps trl.ORPOTrainer
- [ ] SimPO (`task: simpo`) — simple preference optimization
- [ ] IPO (`task: ipo`) — identity preference optimization
- [ ] Templates: `soup init --template kto`, `--template orpo`
#### Advanced PEFT methods
- [ ] DoRA (`peft_type: dora`) — improved LoRA with magnitude decomposition
### v0.12.0 — Advanced PEFT
- [ ] DoRA (`use_dora: true`) — magnitude decomposition, already in PEFT
- [ ] LoRA+ (`loraplus_lr_ratio`) — different lr for A and B matrices
- [ ] GaLore — memory-efficient full-parameter training on consumer GPUs
- [ ] LoRA+ — different learning rates for A and B matrices
#### More alignment methods
- [ ] KTO (Kahneman-Tversky Optimization) — `task: kto`, doesn't need paired data
- [ ] ORPO (Odds Ratio Preference Optimization) — `task: orpo`, no reference model needed
- [ ] SimPO — `task: simpo`, simple preference optimization
- [ ] IPO (Identity Preference Optimization) — `task: ipo`
### v0.13.0 — Batch inference + TensorBoard
- [ ] `soup infer --input prompts.jsonl --output results.jsonl` — batch inference (reuse diff.py code)
- [ ] `--tensorboard` flag in train — `report_to="tensorboard"`, HF Trainer handles it
- [ ] Supported models page in README (Llama 4, Gemma 3, Qwen 2.5/3, Phi-4, DeepSeek R1/V3)
### v0.14.0 — Pre-training + MoE
#### Pre-training / Continued pre-training
- [ ] `task: pretrain` — continued pre-training on raw text
- [ ] Support for plain text / tokenized datasets
- [ ] Plain text / tokenized datasets support
- [ ] MoE model support (Qwen3 30B-A3B, Mixtral, DeepSeek V3)
- [ ] ScatterMoE LoRA
#### Cloud GPU providers
- [ ] `soup cloud run --provider runpod --gpu a100 --config soup.yaml`
- [ ] RunPod, Vast.ai, Lambda Labs, Modal integration
- [ ] Auto-setup: upload data → rent GPU → train → return adapter → teardown
- [ ] Cost estimator and budget auto-stop
### v0.15.0 — Performance + Long-context
---
### P1 — Important (v1.2v1.3)
#### MoE support
- [ ] Explicit MoE model support (Qwen3 30B-A3B, Mixtral, DeepSeek V3)
- [ ] ScatterMoE LoRA for efficient MoE fine-tuning
- [ ] Documentation and examples for MoE training
#### Long-context training
- [ ] Sequence parallelism via Ring FlashAttention
- [ ] 128k+ context fine-tuning across multiple GPUs
- [ ] Neat packing (contamination-free) for long sequences
#### Embedding models
- [ ] `task: embedding` — fine-tune sentence transformers, BGE, E5
- [ ] `soup init --template embedding`
- [ ] Contrastive loss, triplet loss support
#### Advanced distributed training
- [ ] FSDP2 support alongside DeepSpeed
- [ ] Multi-node training via torchrun / Ray
- [ ] Tensor Parallelism + Context Parallelism combined
#### Performance optimizations
- [ ] Liger Kernel integration (fused operations)
- [ ] FlashAttention-3/4 auto-detection
- [ ] SageAttention support
- [ ] FSDP2 support alongside DeepSpeed
- [ ] Sequence parallelism via Ring FlashAttention
- [ ] 128k+ context fine-tuning
---
### v0.16.0 — Embedding models + Export
### P2 — Nice to have (v1.4+)
#### Export formats
- [ ] `task: embedding` — sentence transformers, BGE, E5
- [ ] Contrastive loss, triplet loss
- [ ] ONNX export (`soup export --format onnx`)
- [ ] TensorRT-LLM export
- [ ] Speculative decoding (`soup serve --speculative-decoding`)
### v0.17.0 — Data + Audio
- [ ] Local model as data generation provider
- [ ] Quality filters (perplexity, coherence scoring)
- [ ] `modality: audio` — Qwen2-Audio, Whisper fine-tuning
- [ ] SGLang backend for serving
#### Integrations
- [ ] MLflow tracking (enterprise alternative to W&B)
- [ ] TensorBoard integration
- [ ] `uv` package manager support
### v0.18.0 — Cloud GPU (monetization)
#### Serving improvements
- [ ] Speculative decoding (`soup serve --speculative-decoding`)
- [ ] Batch inference mode (`soup infer --input prompts.jsonl --output results.jsonl`)
Last step — all features are built, product is ready to sell.
#### Data improvements
- [ ] Multi-agent synthetic data generation (GraphGen-style)
- [ ] Quality filters for generated data (perplexity, coherence scoring)
- [ ] Local model as data generation provider (not just OpenAI API)
#### Audio modality
- [ ] `modality: audio` — Qwen2-Audio, Whisper fine-tuning
- [ ] Audio dataset formats and inspection
#### Smart suggestions
- [ ] Auto-detect hardware and recommend optimal config
- [ ] "You have 2×H100 — recommend FSDP2 + Unsloth" style hints
- [ ] `soup login` — user registration, Stripe card linking
- [ ] `soup cloud run --config soup.yaml --gpu a100` — cloud training
- [ ] Backend API (FastAPI on VPS) — pod management, billing
- [ ] RunPod API integration (our account, 20-30% markup)
- [ ] `soup cloud status` — usage, balance, history
- [ ] Landing page soup.cloud
- [ ] Cost estimator and budget auto-stop
- [ ] Vast.ai, Lambda Labs, Modal — additional providers
---

View File

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "soup-cli"
version = "0.10.1"
version = "0.10.2"
description = "Fine-tune LLMs in one command. No SSH, no config hell."
readme = "README.md"
license = "MIT"

View File

@ -1,3 +1,3 @@
"""Soup CLI — Fine-tune LLMs in one command."""
__version__ = "0.10.1"
__version__ = "0.10.2"

View File

@ -361,6 +361,11 @@ def stats(
plt.ylabel("Count")
plt.theme("dark")
plt.show()
except UnicodeEncodeError:
console.print(
"\n[dim]Histogram skipped (Windows encoding).[/] "
"Set PYTHONIOENCODING=utf-8 to enable."
)
except ImportError:
console.print(
"\n[dim]Install plotext for histograms:[/] [bold]pip install plotext[/]"

View File

@ -231,7 +231,7 @@ def _load_model(model_path: str, base_model: Optional[str], device: str):
base_model,
trust_remote_code=True,
device_map="auto",
torch_dtype=torch.float16,
dtype=torch.float16,
)
model_obj = PeftModel.from_pretrained(base, model_path)
else:
@ -239,7 +239,7 @@ def _load_model(model_path: str, base_model: Optional[str], device: str):
model_path,
trust_remote_code=True,
device_map="auto",
torch_dtype=torch.float16,
dtype=torch.float16,
)
model_obj.eval()

View File

@ -100,6 +100,9 @@ def doctor():
console.print(table)
# Check torchvision + torch compatibility
_check_torchvision_compat(issues)
# Summary
if issues:
console.print(f"\n[yellow]Found {len(issues)} issue(s):[/]")
@ -162,6 +165,34 @@ def _check_gpu():
)
def _check_torchvision_compat(issues: list):
"""Check that torchvision version is compatible with torch."""
try:
import torch
import torchvision
torch_ver = torch.__version__.split("+")[0]
tv_ver = torchvision.__version__.split("+")[0]
torch_minor = ".".join(torch_ver.split(".")[:2])
tv_minor = ".".join(tv_ver.split(".")[:2])
# Known compatible pairs (torch minor -> torchvision minor)
compat = {
"2.6": "0.21", "2.5": "0.20", "2.4": "0.19",
"2.3": "0.18", "2.2": "0.17", "2.1": "0.16", "2.0": "0.15",
}
expected_tv = compat.get(torch_minor)
if expected_tv and not tv_minor.startswith(expected_tv):
msg = (
f"torchvision {tv_ver} may be incompatible with torch {torch_ver}. "
f"Expected torchvision {expected_tv}.x"
)
console.print(f" [yellow]Warning:[/] {msg}")
issues.append(msg)
except ImportError:
pass
def _version_ok(installed: str, minimum: str) -> bool:
"""Check if installed version meets minimum requirement."""
try:

View File

@ -107,12 +107,13 @@ def train(
device, device_name = detect_device()
gpu_info = get_gpu_info()
# Warn about quantization on CPU
# Auto-disable quantization on CPU (bitsandbytes doesn't support CPU)
if device == "cpu" and cfg.training.quantization in ("4bit", "8bit"):
console.print(
f"[yellow]Warning: {cfg.training.quantization} quantization on CPU "
"may cause errors. Consider using quantization: none for CPU.[/]"
f"[yellow]Warning: {cfg.training.quantization} quantization is not "
"supported on CPU. Switching to quantization: none.[/]"
)
cfg.training.quantization = "none"
backend_label = cfg.backend
if cfg.backend == "unsloth":

View File

@ -59,7 +59,7 @@ class TrainingDisplay:
progress_pct = 0
bar_width = 30
filled = int(bar_width * progress_pct / 100)
bar = "\u2588" * filled + "\u2591" * (bar_width - filled)
bar = "#" * filled + "-" * (bar_width - filled)
epochs = self.config.training.epochs
epoch_str = f"Epoch {self.current_epoch:.1f}/{epochs}"

View File

@ -72,6 +72,27 @@ ERROR_MAP = [
"BitsAndBytes is not installed (needed for quantization).",
"Run: pip install bitsandbytes>=0.41.0",
),
# CPU / quantization issues
(
"expanded size of the tensor",
"Model generation failed (likely 4bit quantization on CPU).",
"Use a GPU, or set quantization: none in your config for CPU training.",
),
(
"expected m1 and m2 to have the same dtype",
"Dtype mismatch (likely 4bit quantization on CPU).",
"Use a GPU, or set quantization: none in your config for CPU training.",
),
(
"Your setup doesn't support bf16",
"This training task requires GPU with bf16 support.",
"Use a CUDA GPU, or try a simpler task (SFT/DPO work on CPU).",
),
(
"nms does not exist",
"torchvision version is incompatible with torch.",
"Run: pip install torchvision --force-reinstall (or check soup doctor).",
),
# Connection errors
(
"ConnectionError",

View File

@ -1,4 +1,4 @@
"""Tests for v0.10.1 bug fixes - Windows Unicode, PPO params, dtype, diff."""
"""Tests for v0.10.1/v0.10.2 bug fixes - Unicode, PPO, dtype, CPU compat."""
from pathlib import Path
from unittest.mock import patch
@ -135,15 +135,15 @@ class TestComputeDtype:
class TestDiffModelLoading:
"""Test diff command uses correct parameter names."""
def test_load_model_uses_torch_dtype(self):
"""_load_model should pass torch_dtype, not dtype."""
def test_load_model_uses_dtype(self):
"""_load_model should pass dtype= (not the old torch_dtype=)."""
import inspect
from soup_cli.commands.diff import _load_model
source = inspect.getsource(_load_model)
assert "torch_dtype=" in source
assert "dtype=" not in source or "torch_dtype=" in source
assert "dtype=torch.float16" in source
assert "torch_dtype=" not in source
# --- BUG-006: wandb version pin ---
@ -165,11 +165,98 @@ class TestWandbVersionPin:
class TestCPUQuantWarning:
"""Test that CPU + quantization produces a warning."""
def test_train_source_has_cpu_quant_warning(self):
"""train.py should warn about quantization on CPU."""
def test_train_auto_disables_quant_on_cpu(self):
"""train.py should auto-disable quantization on CPU."""
import inspect
from soup_cli.commands import train
source = inspect.getsource(train)
assert "quantization on CPU" in source
assert "quantization is not" in source
assert 'cfg.training.quantization = "none"' in source
# --- v0.10.2: Display progress bar uses ASCII ---
class TestDisplayASCII:
"""Test that training display uses ASCII-safe progress bars."""
def test_progress_bar_uses_ascii_chars(self):
"""Progress bar should use # and - instead of Unicode blocks."""
import inspect
from soup_cli.monitoring.display import TrainingDisplay
source = inspect.getsource(TrainingDisplay)
assert '"#"' in source
assert '"-"' in source
assert "\\u2588" not in source
assert "\\u2591" not in source
# --- v0.10.2: Plotext UnicodeEncodeError handling ---
class TestPlotextFallback:
"""Test that plotext errors are caught gracefully."""
def test_stats_catches_unicode_error(self):
"""data stats should catch UnicodeEncodeError from plotext."""
import inspect
from soup_cli.commands import data
source = inspect.getsource(data)
assert "UnicodeEncodeError" in source
# --- v0.10.2: Error messages for CPU issues ---
class TestCPUErrorMessages:
"""Test friendly error messages for CPU-specific failures."""
def test_tensor_size_error_mapped(self):
"""Tensor expansion error should have a friendly message."""
from soup_cli.utils.errors import ERROR_MAP
patterns = [pattern for pattern, _, _ in ERROR_MAP]
assert any("expanded size" in p for p in patterns)
def test_dtype_mismatch_error_mapped(self):
"""Dtype mismatch error should have a friendly message."""
from soup_cli.utils.errors import ERROR_MAP
patterns = [pattern for pattern, _, _ in ERROR_MAP]
assert any("same dtype" in p for p in patterns)
def test_bf16_error_mapped(self):
"""bf16 GPU error should have a friendly message."""
from soup_cli.utils.errors import ERROR_MAP
patterns = [pattern for pattern, _, _ in ERROR_MAP]
assert any("bf16" in p for p in patterns)
def test_torchvision_error_mapped(self):
"""torchvision nms error should have a friendly message."""
from soup_cli.utils.errors import ERROR_MAP
patterns = [pattern for pattern, _, _ in ERROR_MAP]
assert any("nms" in p for p in patterns)
# --- v0.10.2: Doctor torchvision check ---
class TestDoctorTorchvisionCheck:
"""Test that soup doctor checks torchvision compatibility."""
def test_doctor_has_torchvision_check(self):
"""doctor.py should have torchvision compatibility check."""
import inspect
from soup_cli.commands import doctor
source = inspect.getsource(doctor)
assert "_check_torchvision_compat" in source