diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index f12b07a..f0cae2c 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -1,12 +1,12 @@ # Soup CLI — Project CLAUDE.md -Soup is a CLI-first LLM fine-tuning tool (v0.13.0). Python 3.9+, MIT license. +Soup is a CLI-first LLM fine-tuning tool (v0.13.2). Python 3.9+, MIT license. ## Build & Development ```bash pip install -e ".[dev]" # Install editable + test deps -pytest tests/ -v --tb=short # Run all tests (906 tests) +pytest tests/ -v --tb=short # Run all tests (917 tests) ruff check soup_cli/ tests/ # Lint (must pass before commit) ruff check --fix soup_cli/ tests/ # Auto-fix lint issues ``` @@ -70,7 +70,7 @@ soup_cli/ vllm.py # AsyncLLMEngine backend (2-4x inference throughput) galore.py # GaLore optimizer config + validation constants.py # APP_NAME, paths, default chat template -tests/ # 44 test files, 906 tests +tests/ # 44 test files, 917 tests examples/ configs/ # 7 production-ready YAML examples data/ # Sample datasets @@ -155,8 +155,9 @@ soup version # Show version (--full for details) - **Deprecated CLI secrets**: `--api-key` and `--token` flags read from env vars, marked deprecated - **Custom reward warning**: Prominent warning before executing arbitrary .py reward files - **max_tokens bound**: Capped at 16384 on inference endpoints -- **experiment_name validation**: Path separators and null bytes blocked (v0.13.0) -- **GaLore params**: Type-enforced before string interpolation (v0.13.0) +- **experiment_name validation**: Path separators and null bytes blocked (v0.12.0) +- **GaLore params**: Type-enforced before string interpolation (v0.12.0) +- **Batch inference**: max_tokens capped at 16384, trust_remote_code warning (v0.13.0) ## Code Conventions @@ -217,7 +218,7 @@ soup version # Show version (--full for details) 12. **Tag**: `git tag v0.X.Y && git push origin v0.X.Y` 13. **Release**: `gh release create v0.X.Y` with changelog (What's New, Install/Upgrade) -## Tests (44 test files, 906 tests) +## Tests (44 test files, 917 tests) | File | Covers | |------|--------| diff --git a/README.md b/README.md index 8a626dd..7d7a65e 100644 --- a/README.md +++ b/README.md @@ -783,7 +783,7 @@ soup version # Full system info (useful for bug reports) soup version --full -# -> soup v0.13.0 | Python 3.11.5 | CUDA 12.1 | extras: serve, data +# -> soup v0.13.2 | Python 3.11.5 | CUDA 12.1 | extras: serve, data ``` ## Web UI @@ -953,20 +953,47 @@ soup --verbose Full traceback on errors ## Supported Models -Soup works with any HuggingFace-compatible causal language model. Tested and recommended: +Soup works with **any** of the **340,000+** text-generation models on [HuggingFace Hub](https://huggingface.co/models?pipeline_tag=text-generation). If a model supports `AutoModelForCausalLM`, it works with Soup — zero config changes needed. -| Model Family | Example Models | Notes | +### Recommended Models + +| Model Family | Models | Sizes | Best For | +|---|---|---|---| +| **Llama 4** | Llama-4-Scout-17B, Llama-4-Maverick-17B | 17B | General, multilingual | +| **Llama 3.x** | Llama-3.1-8B-Instruct, Llama-3.3-70B-Instruct | 1B–70B | Chat, instruction following | +| **Llama 3.2 Vision** | Llama-3.2-11B-Vision-Instruct, Llama-3.2-90B-Vision | 11B–90B | Image understanding | +| **Gemma 3** | Gemma-3-4B-IT, Gemma-3-9B-IT, Gemma-3-27B-IT | 4B–27B | Efficient, multilingual | +| **Qwen 3** | Qwen3-8B, Qwen3-14B, Qwen3-32B, Qwen3-235B-A22B | 0.6B–235B | Reasoning, code, MoE | +| **Qwen 2.5** | Qwen2.5-7B-Instruct, Qwen2.5-Coder-32B-Instruct | 0.5B–72B | Code, math | +| **DeepSeek** | DeepSeek-R1-Distill-Llama-8B, DeepSeek-V3-0324 | 1.5B–671B | Reasoning (GRPO), code | +| **Phi-4** | Phi-4-14B, Phi-4-mini-reasoning | 3.8B–14B | Compact reasoning | +| **Mistral** | Mistral-7B-Instruct-v0.3, Mistral-Small-24B-Instruct | 7B–24B | Fast, efficient | +| **Mixtral** | Mixtral-8x7B-Instruct-v0.1, Mixtral-8x22B | 47B–141B | MoE architecture | +| **CodeLlama** | CodeLlama-7b-Instruct-hf, CodeLlama-34b-Instruct | 7B–34B | Code generation | +| **StarCoder 2** | StarCoder2-15B, StarCoder2-7B | 3B–15B | Code completion | +| **Yi** | Yi-1.5-34B-Chat, Yi-1.5-9B-Chat | 6B–34B | Multilingual chat | +| **InternLM 3** | InternLM3-8B-Instruct | 8B | Chinese + English | +| **Falcon** | Falcon-11B, Falcon-40B-Instruct | 7B–180B | Open-weight | + +### Vision Models (with `modality: vision`) + +| Model | Size | Supported Formats | |---|---|---| -| **Llama 4** | Llama-4-Scout-17B, Llama-4-Maverick-17B | Latest Meta models | -| **Llama 3.x** | Llama-3.1-8B-Instruct, Llama-3.2-11B-Vision | Text + vision | -| **Gemma 3** | Gemma-3-9B-IT, Gemma-3-27B-IT | Google's latest | -| **Qwen 2.5/3** | Qwen2.5-7B-Instruct, Qwen3-8B | Alibaba series | -| **Phi-4** | Phi-4-14B | Microsoft | -| **DeepSeek** | DeepSeek-R1-Distill-Llama-8B, DeepSeek-V3 | Reasoning models (GRPO) | -| **Mistral** | Mistral-7B-Instruct-v0.3, Mixtral-8x7B | Including MoE | -| **CodeLlama** | CodeLlama-7b-Instruct-hf | Code generation | +| LLaMA-3.2-11B-Vision-Instruct | 11B | LLaVA, ShareGPT4V | +| Qwen2-VL-7B-Instruct | 7B | LLaVA, ShareGPT4V | +| Pixtral-12B-2409 | 12B | LLaVA, ShareGPT4V | -Any model on HuggingFace Hub that supports `AutoModelForCausalLM` will work. For vision tasks, use models with vision capabilities (LLaMA-3.2-Vision, Qwen2-VL, Pixtral). +### Quick Size Guide + +| VRAM | Max Model (QLoRA 4-bit) | Example | +|---|---|---| +| 8 GB | ~7B | Llama-3.1-8B, Mistral-7B | +| 16 GB | ~14B | Phi-4-14B, Qwen2.5-14B | +| 24 GB | ~34B | CodeLlama-34B, Yi-1.5-34B | +| 48 GB | ~70B | Llama-3.3-70B | +| 80 GB+ | 70B+ (full) or MoE | Mixtral-8x22B, DeepSeek-V3 | + +> **Note:** Soup auto-detects your GPU and estimates the optimal batch size. Use `soup doctor` to check your setup. ## Requirements