mirror of https://github.com/razor-ai/soup.git
docs: expand Supported Models to 340k+ HF models with VRAM guide
- Lead with 340,000+ text-generation models count from HuggingFace Hub - Expand recommended models table: 15 families with sizes and use cases - Add Vision Models table (LLaMA-Vision, Qwen2-VL, Pixtral) - Add Quick Size Guide: VRAM -> max model size with examples - Fix version references to v0.13.2 across CLAUDE.md and README
This commit is contained in:
Alpamys
parent
9451b980d4
commit
d0be93af68
|
|
@ -1,12 +1,12 @@
|
|||
# Soup CLI — Project CLAUDE.md
|
||||
|
||||
Soup is a CLI-first LLM fine-tuning tool (v0.13.0). Python 3.9+, MIT license.
|
||||
Soup is a CLI-first LLM fine-tuning tool (v0.13.2). Python 3.9+, MIT license.
|
||||
|
||||
## Build & Development
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev]" # Install editable + test deps
|
||||
pytest tests/ -v --tb=short # Run all tests (906 tests)
|
||||
pytest tests/ -v --tb=short # Run all tests (917 tests)
|
||||
ruff check soup_cli/ tests/ # Lint (must pass before commit)
|
||||
ruff check --fix soup_cli/ tests/ # Auto-fix lint issues
|
||||
```
|
||||
|
|
@ -70,7 +70,7 @@ soup_cli/
|
|||
vllm.py # AsyncLLMEngine backend (2-4x inference throughput)
|
||||
galore.py # GaLore optimizer config + validation
|
||||
constants.py # APP_NAME, paths, default chat template
|
||||
tests/ # 44 test files, 906 tests
|
||||
tests/ # 44 test files, 917 tests
|
||||
examples/
|
||||
configs/ # 7 production-ready YAML examples
|
||||
data/ # Sample datasets
|
||||
|
|
@ -155,8 +155,9 @@ soup version # Show version (--full for details)
|
|||
- **Deprecated CLI secrets**: `--api-key` and `--token` flags read from env vars, marked deprecated
|
||||
- **Custom reward warning**: Prominent warning before executing arbitrary .py reward files
|
||||
- **max_tokens bound**: Capped at 16384 on inference endpoints
|
||||
- **experiment_name validation**: Path separators and null bytes blocked (v0.13.0)
|
||||
- **GaLore params**: Type-enforced before string interpolation (v0.13.0)
|
||||
- **experiment_name validation**: Path separators and null bytes blocked (v0.12.0)
|
||||
- **GaLore params**: Type-enforced before string interpolation (v0.12.0)
|
||||
- **Batch inference**: max_tokens capped at 16384, trust_remote_code warning (v0.13.0)
|
||||
|
||||
## Code Conventions
|
||||
|
||||
|
|
@ -217,7 +218,7 @@ soup version # Show version (--full for details)
|
|||
12. **Tag**: `git tag v0.X.Y && git push origin v0.X.Y`
|
||||
13. **Release**: `gh release create v0.X.Y` with changelog (What's New, Install/Upgrade)
|
||||
|
||||
## Tests (44 test files, 906 tests)
|
||||
## Tests (44 test files, 917 tests)
|
||||
|
||||
| File | Covers |
|
||||
|------|--------|
|
||||
|
|
|
|||
51
README.md
51
README.md
|
|
@ -783,7 +783,7 @@ soup version
|
|||
|
||||
# Full system info (useful for bug reports)
|
||||
soup version --full
|
||||
# -> soup v0.13.0 | Python 3.11.5 | CUDA 12.1 | extras: serve, data
|
||||
# -> soup v0.13.2 | Python 3.11.5 | CUDA 12.1 | extras: serve, data
|
||||
```
|
||||
|
||||
## Web UI
|
||||
|
|
@ -953,20 +953,47 @@ soup --verbose <command> Full traceback on errors
|
|||
|
||||
## Supported Models
|
||||
|
||||
Soup works with any HuggingFace-compatible causal language model. Tested and recommended:
|
||||
Soup works with **any** of the **340,000+** text-generation models on [HuggingFace Hub](https://huggingface.co/models?pipeline_tag=text-generation). If a model supports `AutoModelForCausalLM`, it works with Soup — zero config changes needed.
|
||||
|
||||
| Model Family | Example Models | Notes |
|
||||
### Recommended Models
|
||||
|
||||
| Model Family | Models | Sizes | Best For |
|
||||
|---|---|---|---|
|
||||
| **Llama 4** | Llama-4-Scout-17B, Llama-4-Maverick-17B | 17B | General, multilingual |
|
||||
| **Llama 3.x** | Llama-3.1-8B-Instruct, Llama-3.3-70B-Instruct | 1B–70B | Chat, instruction following |
|
||||
| **Llama 3.2 Vision** | Llama-3.2-11B-Vision-Instruct, Llama-3.2-90B-Vision | 11B–90B | Image understanding |
|
||||
| **Gemma 3** | Gemma-3-4B-IT, Gemma-3-9B-IT, Gemma-3-27B-IT | 4B–27B | Efficient, multilingual |
|
||||
| **Qwen 3** | Qwen3-8B, Qwen3-14B, Qwen3-32B, Qwen3-235B-A22B | 0.6B–235B | Reasoning, code, MoE |
|
||||
| **Qwen 2.5** | Qwen2.5-7B-Instruct, Qwen2.5-Coder-32B-Instruct | 0.5B–72B | Code, math |
|
||||
| **DeepSeek** | DeepSeek-R1-Distill-Llama-8B, DeepSeek-V3-0324 | 1.5B–671B | Reasoning (GRPO), code |
|
||||
| **Phi-4** | Phi-4-14B, Phi-4-mini-reasoning | 3.8B–14B | Compact reasoning |
|
||||
| **Mistral** | Mistral-7B-Instruct-v0.3, Mistral-Small-24B-Instruct | 7B–24B | Fast, efficient |
|
||||
| **Mixtral** | Mixtral-8x7B-Instruct-v0.1, Mixtral-8x22B | 47B–141B | MoE architecture |
|
||||
| **CodeLlama** | CodeLlama-7b-Instruct-hf, CodeLlama-34b-Instruct | 7B–34B | Code generation |
|
||||
| **StarCoder 2** | StarCoder2-15B, StarCoder2-7B | 3B–15B | Code completion |
|
||||
| **Yi** | Yi-1.5-34B-Chat, Yi-1.5-9B-Chat | 6B–34B | Multilingual chat |
|
||||
| **InternLM 3** | InternLM3-8B-Instruct | 8B | Chinese + English |
|
||||
| **Falcon** | Falcon-11B, Falcon-40B-Instruct | 7B–180B | Open-weight |
|
||||
|
||||
### Vision Models (with `modality: vision`)
|
||||
|
||||
| Model | Size | Supported Formats |
|
||||
|---|---|---|
|
||||
| **Llama 4** | Llama-4-Scout-17B, Llama-4-Maverick-17B | Latest Meta models |
|
||||
| **Llama 3.x** | Llama-3.1-8B-Instruct, Llama-3.2-11B-Vision | Text + vision |
|
||||
| **Gemma 3** | Gemma-3-9B-IT, Gemma-3-27B-IT | Google's latest |
|
||||
| **Qwen 2.5/3** | Qwen2.5-7B-Instruct, Qwen3-8B | Alibaba series |
|
||||
| **Phi-4** | Phi-4-14B | Microsoft |
|
||||
| **DeepSeek** | DeepSeek-R1-Distill-Llama-8B, DeepSeek-V3 | Reasoning models (GRPO) |
|
||||
| **Mistral** | Mistral-7B-Instruct-v0.3, Mixtral-8x7B | Including MoE |
|
||||
| **CodeLlama** | CodeLlama-7b-Instruct-hf | Code generation |
|
||||
| LLaMA-3.2-11B-Vision-Instruct | 11B | LLaVA, ShareGPT4V |
|
||||
| Qwen2-VL-7B-Instruct | 7B | LLaVA, ShareGPT4V |
|
||||
| Pixtral-12B-2409 | 12B | LLaVA, ShareGPT4V |
|
||||
|
||||
Any model on HuggingFace Hub that supports `AutoModelForCausalLM` will work. For vision tasks, use models with vision capabilities (LLaMA-3.2-Vision, Qwen2-VL, Pixtral).
|
||||
### Quick Size Guide
|
||||
|
||||
| VRAM | Max Model (QLoRA 4-bit) | Example |
|
||||
|---|---|---|
|
||||
| 8 GB | ~7B | Llama-3.1-8B, Mistral-7B |
|
||||
| 16 GB | ~14B | Phi-4-14B, Qwen2.5-14B |
|
||||
| 24 GB | ~34B | CodeLlama-34B, Yi-1.5-34B |
|
||||
| 48 GB | ~70B | Llama-3.3-70B |
|
||||
| 80 GB+ | 70B+ (full) or MoE | Mixtral-8x22B, DeepSeek-V3 |
|
||||
|
||||
> **Note:** Soup auto-detects your GPU and estimates the optimal batch size. Use `soup doctor` to check your setup.
|
||||
|
||||
## Requirements
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue