mirror of https://github.com/razor-ai/soup.git
feat: add Ollama integration — deploy GGUF models in one command (v0.18.0)
New commands: - `soup deploy ollama` — deploy GGUF to local Ollama with auto-template detection - `soup deploy ollama --list` / `--remove` — manage Soup-deployed models - `soup export --deploy ollama` — export + auto-deploy in one step New files: - soup_cli/utils/ollama.py — detect, deploy, list, remove, Modelfile generation - soup_cli/commands/deploy.py — Typer command group with Rich panels - tests/test_deploy_ollama.py — 78 tests covering all paths Security hardening: - GGUF path traversal protection + .gguf extension validation - Model name validation (no path separators, null bytes) - Modelfile parameter key allowlist prevents directive injection - Parameter value newline/null sanitization - Subprocess calls use list args (no shell injection) - Warning panel before overwriting existing Ollama models 1449 tests, 57 test files, all passing.
This commit is contained in:
Alpamys
parent
986f8cb26c
commit
f98519ef87
|
|
@ -1,12 +1,12 @@
|
|||
# Soup CLI — Project CLAUDE.md
|
||||
|
||||
Soup is a CLI-first LLM fine-tuning tool (v0.17.3). Python 3.9+, MIT license.
|
||||
Soup is a CLI-first LLM fine-tuning tool (v0.18.0). Python 3.9+, MIT license.
|
||||
|
||||
## Build & Development
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev]" # Install editable + test deps
|
||||
pytest tests/ -v --tb=short # Run all tests (1348 tests)
|
||||
pytest tests/ -v --tb=short # Run all tests (1449 tests)
|
||||
ruff check soup_cli/ tests/ # Lint (must pass before commit)
|
||||
ruff check --fix soup_cli/ tests/ # Auto-fix lint issues
|
||||
```
|
||||
|
|
@ -16,7 +16,7 @@ ruff check --fix soup_cli/ tests/ # Auto-fix lint issues
|
|||
```
|
||||
soup_cli/
|
||||
cli.py # Entry point, Typer app, all command registration
|
||||
__init__.py # __version__ = "0.17.3"
|
||||
__init__.py # __version__ = "0.18.0"
|
||||
config/
|
||||
schema.py # Pydantic models (SoupConfig, DataConfig, TrainingConfig, LoraConfig)
|
||||
loader.py # YAML -> SoupConfig, load_config_from_string()
|
||||
|
|
@ -47,6 +47,7 @@ soup_cli/
|
|||
tracker.py # SQLite at ~/.soup/experiments.db (runs, metrics, eval_results)
|
||||
commands/
|
||||
train.py # soup train (routes to SFT/DPO/GRPO/PPO/Reward/KTO/ORPO/SimPO/IPO)
|
||||
deploy.py # soup deploy ollama (deploy GGUF to Ollama)
|
||||
init.py # soup init (interactive wizard + 10 templates)
|
||||
chat.py # soup chat (terminal REPL)
|
||||
serve.py # soup serve (OpenAI-compatible API, transformers/vllm backends)
|
||||
|
|
@ -82,6 +83,7 @@ soup_cli/
|
|||
long_context.py # RoPE scaling for 128k+ context fine-tuning
|
||||
quality.py # Perplexity + coherence scoring for data quality filters
|
||||
sglang.py # SGLang runtime backend (high-throughput serving)
|
||||
ollama.py # Ollama integration (detect, deploy, list, remove, Modelfile gen)
|
||||
constants.py # APP_NAME, paths, default chat template
|
||||
tests/ # 56 test files, 1348 tests
|
||||
examples/
|
||||
|
|
@ -98,6 +100,10 @@ soup infer # Batch inference (--model, --input, --output)
|
|||
soup chat # Terminal chat with model
|
||||
soup serve # OpenAI-compatible inference server (--backend transformers|vllm|sglang, --speculative-decoding)
|
||||
soup export # Convert to GGUF/ONNX/TensorRT for deployment
|
||||
soup export --deploy ollama # Export GGUF + auto-deploy to Ollama
|
||||
soup deploy ollama # Deploy GGUF model to local Ollama instance
|
||||
soup deploy ollama --list # List Soup-deployed models in Ollama
|
||||
soup deploy ollama --remove # Remove model from Ollama
|
||||
soup merge # Merge LoRA adapter with base model
|
||||
soup push # Upload to HuggingFace Hub
|
||||
soup eval # Run benchmarks (mmlu, gsm8k, etc.)
|
||||
|
|
@ -192,6 +198,12 @@ soup version # Show version (--full for details)
|
|||
- **Audio model**: trust_remote_code warning panel before loading audio models (v0.17.0)
|
||||
- **Audio paths**: path traversal protection — resolved paths confined to audio_dir (v0.17.0)
|
||||
- **SGLang**: trust_remote_code warning panel before runtime creation (v0.17.0)
|
||||
- **Ollama deploy**: GGUF path traversal protection + `.gguf` extension validation (v0.18.0)
|
||||
- **Ollama deploy**: model name validation — no path separators or null bytes (v0.18.0)
|
||||
- **Ollama deploy**: subprocess calls use list args (no shell injection) (v0.18.0)
|
||||
- **Ollama deploy**: Modelfile parameter key allowlist prevents directive injection (v0.18.0)
|
||||
- **Ollama deploy**: parameter value newline/null sanitization prevents Modelfile injection (v0.18.0)
|
||||
- **Ollama deploy**: warning panel before `ollama create` (overwrites existing model) (v0.18.0)
|
||||
|
||||
## Code Conventions
|
||||
|
||||
|
|
@ -261,7 +273,7 @@ soup version # Show version (--full for details)
|
|||
15. **Tag**: `git tag v0.X.Y && git push origin v0.X.Y`
|
||||
16. **Release**: `gh release create v0.X.Y` with changelog (What's New, Install/Upgrade)
|
||||
|
||||
## Tests (56 test files, 1348 tests)
|
||||
## Tests (57 test files, 1449 tests)
|
||||
|
||||
| File | Covers |
|
||||
|------|--------|
|
||||
|
|
@ -320,3 +332,4 @@ soup version # Show version (--full for details)
|
|||
| test_quality_filter.py | Perplexity + coherence scoring, `soup data filter` |
|
||||
| test_audio.py | Audio modality config, format, template, routing, loader |
|
||||
| test_sglang_serve.py | SGLang backend detection, runtime creation, serve --backend |
|
||||
| test_deploy_ollama.py | Ollama deploy, Modelfile gen, template mapping, security validation |
|
||||
|
|
|
|||
37
README.md
37
README.md
|
|
@ -683,13 +683,44 @@ soup export --model ./output --format tensorrt
|
|||
soup export --model ./output --format tensorrt --output ./model_trt
|
||||
```
|
||||
|
||||
After export, use with Ollama:
|
||||
After export, use with Ollama manually or auto-deploy:
|
||||
```bash
|
||||
# Manual (3-step)
|
||||
echo 'FROM ./my-model.q4_k_m.gguf' > Modelfile
|
||||
ollama create my-model -f Modelfile
|
||||
ollama run my-model
|
||||
|
||||
# Auto-deploy (1-step)
|
||||
soup export --model ./output --format gguf --deploy ollama --deploy-name my-model
|
||||
```
|
||||
|
||||
### Deploy to Ollama
|
||||
|
||||
Deploy a GGUF model directly to your local [Ollama](https://ollama.com/) instance:
|
||||
|
||||
```bash
|
||||
# Deploy a GGUF model
|
||||
soup deploy ollama --model ./output/model.q4_k_m.gguf --name soup-my-model
|
||||
|
||||
# Deploy with system prompt and parameters
|
||||
soup deploy ollama --model ./model.gguf --name soup-chat \
|
||||
--system "You are a helpful assistant." \
|
||||
--template chatml \
|
||||
--parameter temperature=0.7 \
|
||||
--parameter top_p=0.9
|
||||
|
||||
# Export + deploy in one command
|
||||
soup export --model ./output --format gguf --deploy ollama
|
||||
|
||||
# List Soup-deployed models
|
||||
soup deploy ollama --list
|
||||
|
||||
# Remove a model
|
||||
soup deploy ollama --remove soup-my-model
|
||||
```
|
||||
|
||||
Auto-detected chat templates: `chatml`, `llama`, `mistral`, `vicuna`, `zephyr` (or `auto` to infer from soup.yaml).
|
||||
|
||||
## Resume Training
|
||||
|
||||
Resume a training run from a checkpoint:
|
||||
|
|
@ -1132,8 +1163,12 @@ soup chat --model ./output Interactive chat
|
|||
soup push --model ./output --repo user/name Upload to HuggingFace
|
||||
soup merge --adapter ./output Merge LoRA with base model
|
||||
soup export --model ./output --format gguf Export to GGUF (Ollama)
|
||||
soup export --model ./output --deploy ollama Export GGUF + auto-deploy to Ollama
|
||||
soup export --model ./output --format onnx Export to ONNX
|
||||
soup export --model ./output --format tensorrt Export to TensorRT-LLM
|
||||
soup deploy ollama --model m.gguf --name x Deploy GGUF to Ollama
|
||||
soup deploy ollama --list List Soup-deployed models
|
||||
soup deploy ollama --remove <name> Remove model from Ollama
|
||||
soup eval --model ./output --benchmarks mmlu Evaluate on benchmarks
|
||||
soup serve --model ./output --port 8000 OpenAI-compatible API server
|
||||
soup serve --model ./output --backend vllm vLLM backend (2-4x throughput)
|
||||
|
|
|
|||
11
SECURITY.md
11
SECURITY.md
|
|
@ -4,14 +4,14 @@
|
|||
|
||||
We provide security updates for the following versions:
|
||||
|
||||
- **Latest minor version:** Active support (e.g., v0.17.x)
|
||||
- **Latest minor version:** Active support (e.g., v0.18.x)
|
||||
- **Previous minor versions:** Bug-fix support only
|
||||
- **Versions older than 3 minor versions:** No support
|
||||
|
||||
Example:
|
||||
- v0.17.0-0.17.x → Full support (latest)
|
||||
- v0.16.0-0.16.x → Bug-fix support only
|
||||
- v0.15.x and below → No support
|
||||
- v0.18.0-0.18.x → Full support (latest)
|
||||
- v0.17.0-0.17.x → Bug-fix support only
|
||||
- v0.16.x and below → No support
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
|
|
@ -120,6 +120,7 @@ No known critical vulnerabilities in current releases.
|
|||
- **v0.15.0**: `rope_scaling_type` Literal constraint, `max_length` bounds (ge=64, le=1048576), FSDP config key allowlist, Liger Kernel exception handling narrowed
|
||||
- **v0.16.0**: `embedding_loss` Literal constraint, `embedding_margin` gt=0 validation, ONNX export without trust_remote_code (with warning), TensorRT export subprocess list args (no shell injection), speculative decoding SSRF-protected (URL blocked) with warning panel, vLLM speculative model URL validation
|
||||
- **v0.17.0**: Server data generation provider SSRF validation (scheme whitelist + localhost-only HTTP), audio model trust_remote_code warning panel, audio file path traversal protection (resolved paths confined to audio_dir), SGLang backend trust_remote_code warning panel
|
||||
- **v0.18.0**: Ollama deploy GGUF path traversal protection + `.gguf` extension validation, model name validation (no path separators/null bytes), subprocess list args (no shell injection), Modelfile parameter key allowlist + value newline/null sanitization, overwrite warning panel
|
||||
|
||||
## Security Scanning
|
||||
|
||||
|
|
@ -156,6 +157,6 @@ This Security Policy is provided under the MIT license, same as the Soup project
|
|||
|
||||
---
|
||||
|
||||
**Last Updated**: March 2026
|
||||
**Last Updated**: April 2026
|
||||
|
||||
For the latest version of this policy, visit: https://github.com/MakazhanAlpamys/Soup/blob/main/SECURITY.md
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "soup-cli"
|
||||
version = "0.17.3"
|
||||
version = "0.18.0"
|
||||
description = "Fine-tune LLMs in one command. No SSH, no config hell."
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
"""Soup CLI — Fine-tune LLMs in one command."""
|
||||
|
||||
__version__ = "0.17.3"
|
||||
__version__ = "0.18.0"
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ from soup_cli import __version__
|
|||
from soup_cli.commands import (
|
||||
chat,
|
||||
data,
|
||||
deploy,
|
||||
diff,
|
||||
eval,
|
||||
export,
|
||||
|
|
@ -53,6 +54,10 @@ app.add_typer(
|
|||
data.app, name="data",
|
||||
help="Dataset tools: inspect, convert, merge, dedup, validate, stats.",
|
||||
)
|
||||
app.add_typer(
|
||||
deploy.app, name="deploy",
|
||||
help="Deploy models: Ollama integration (deploy, list, remove).",
|
||||
)
|
||||
app.add_typer(runs.app, name="runs", help="Experiment tracking: list, show, compare runs.")
|
||||
app.command(name="eval")(eval.eval_model)
|
||||
app.command()(serve.serve)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,253 @@
|
|||
"""soup deploy — deploy models to inference runtimes (Ollama)."""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
|
||||
console = Console()
|
||||
|
||||
app = typer.Typer(no_args_is_help=True)
|
||||
|
||||
|
||||
@app.command()
|
||||
def ollama(
|
||||
model: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--model",
|
||||
"-m",
|
||||
help="Path to GGUF model file",
|
||||
),
|
||||
name: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--name",
|
||||
"-n",
|
||||
help="Ollama model name (e.g. soup-my-model)",
|
||||
),
|
||||
system: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--system",
|
||||
"-s",
|
||||
help="System prompt for the model",
|
||||
),
|
||||
template: str = typer.Option(
|
||||
"auto",
|
||||
"--template",
|
||||
"-t",
|
||||
help="Chat template: auto, chatml, llama, mistral, vicuna, zephyr",
|
||||
),
|
||||
parameter: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--parameter",
|
||||
"-p",
|
||||
help="Ollama parameter (repeatable): temperature=0.7, top_p=0.9, etc.",
|
||||
),
|
||||
list_models: bool = typer.Option(
|
||||
False,
|
||||
"--list",
|
||||
"-l",
|
||||
help="List Soup-deployed models in Ollama",
|
||||
),
|
||||
remove: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--remove",
|
||||
"-r",
|
||||
help="Remove a model from Ollama by name",
|
||||
),
|
||||
yes: bool = typer.Option(
|
||||
False,
|
||||
"--yes",
|
||||
"-y",
|
||||
help="Skip confirmation prompt",
|
||||
),
|
||||
):
|
||||
"""Deploy a GGUF model to local Ollama instance."""
|
||||
from soup_cli.utils.ollama import (
|
||||
OLLAMA_TEMPLATES,
|
||||
create_modelfile,
|
||||
deploy_to_ollama,
|
||||
detect_ollama,
|
||||
list_soup_models,
|
||||
remove_model,
|
||||
validate_gguf_path,
|
||||
validate_model_name,
|
||||
)
|
||||
|
||||
# --- List mode ---
|
||||
if list_models:
|
||||
version = detect_ollama()
|
||||
if not version:
|
||||
console.print("[red]Ollama not found.[/] Install from https://ollama.com")
|
||||
raise typer.Exit(1)
|
||||
|
||||
models = list_soup_models()
|
||||
if not models:
|
||||
console.print("[yellow]No Soup-deployed models found in Ollama.[/]")
|
||||
console.print("[dim]Deploy a model with: soup deploy ollama --model <gguf>[/]")
|
||||
raise typer.Exit(0)
|
||||
|
||||
table = Table(title="Soup Models in Ollama")
|
||||
table.add_column("Name", style="bold cyan")
|
||||
table.add_column("Size", style="green")
|
||||
for entry in models:
|
||||
table.add_row(entry["name"], entry["size"])
|
||||
console.print(table)
|
||||
raise typer.Exit(0)
|
||||
|
||||
# --- Remove mode ---
|
||||
if remove:
|
||||
version = detect_ollama()
|
||||
if not version:
|
||||
console.print("[red]Ollama not found.[/] Install from https://ollama.com")
|
||||
raise typer.Exit(1)
|
||||
|
||||
if not yes:
|
||||
confirm = typer.confirm(f"Remove model '{remove}' from Ollama?")
|
||||
if not confirm:
|
||||
raise typer.Exit(0)
|
||||
|
||||
success, message = remove_model(remove)
|
||||
if success:
|
||||
console.print(f"[green]{message}[/]")
|
||||
else:
|
||||
console.print(f"[red]{message}[/]")
|
||||
raise typer.Exit(1)
|
||||
raise typer.Exit(0)
|
||||
|
||||
# --- Deploy mode: require --model and --name ---
|
||||
if not model:
|
||||
console.print("[red]--model is required for deploy.[/]")
|
||||
console.print("[dim]Usage: soup deploy ollama --model <gguf> --name <name>[/]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
if not name:
|
||||
console.print("[red]--name is required for deploy.[/]")
|
||||
console.print("[dim]Usage: soup deploy ollama --model <gguf> --name <name>[/]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Validate model name
|
||||
valid_name, name_err = validate_model_name(name)
|
||||
if not valid_name:
|
||||
console.print(f"[red]Invalid model name:[/] {name_err}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Validate GGUF path
|
||||
gguf_path = Path(model)
|
||||
valid_path, path_err = validate_gguf_path(gguf_path)
|
||||
if not valid_path:
|
||||
console.print(f"[red]{path_err}[/]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Check Ollama is installed
|
||||
version = detect_ollama()
|
||||
if not version:
|
||||
console.print(
|
||||
"[red]Ollama not found.[/]\n"
|
||||
"Install from: [bold]https://ollama.com[/]"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Resolve template
|
||||
resolved_template = None
|
||||
if template == "auto":
|
||||
# Try to infer from soup.yaml in cwd
|
||||
resolved_template = _auto_detect_template()
|
||||
if not resolved_template:
|
||||
resolved_template = "chatml" # Default fallback
|
||||
elif template in OLLAMA_TEMPLATES:
|
||||
resolved_template = template
|
||||
else:
|
||||
console.print(
|
||||
f"[red]Unknown template: {template}[/]\n"
|
||||
f"Available: auto, {', '.join(OLLAMA_TEMPLATES.keys())}"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Parse parameters
|
||||
params = {}
|
||||
if parameter:
|
||||
for param_str in parameter:
|
||||
if "=" not in param_str:
|
||||
console.print(f"[red]Invalid parameter format: {param_str}[/]")
|
||||
console.print("[dim]Expected format: key=value (e.g. temperature=0.7)[/]")
|
||||
raise typer.Exit(1)
|
||||
key, value = param_str.split("=", 1)
|
||||
params[key.strip()] = value.strip()
|
||||
|
||||
# Show deploy plan
|
||||
console.print(
|
||||
Panel(
|
||||
f"Model: [bold]{name}[/]\n"
|
||||
f"GGUF: [bold]{gguf_path}[/]\n"
|
||||
f"Template: [bold]{resolved_template}[/]"
|
||||
+ (f"\nSystem: [bold]{system}[/]" if system else "")
|
||||
+ (f"\nParams: [bold]{params}[/]" if params else ""),
|
||||
title="Deploy to Ollama",
|
||||
)
|
||||
)
|
||||
|
||||
# Confirmation — warn that this overwrites an existing model
|
||||
if not yes:
|
||||
console.print(
|
||||
"[yellow]Warning:[/] This will overwrite any existing Ollama model "
|
||||
f"named '{name}'."
|
||||
)
|
||||
confirm = typer.confirm("Proceed?")
|
||||
if not confirm:
|
||||
raise typer.Exit(0)
|
||||
|
||||
# Generate Modelfile
|
||||
console.print(f"[green]\u2713[/] Ollama v{version} detected")
|
||||
try:
|
||||
modelfile = create_modelfile(
|
||||
gguf_path=gguf_path,
|
||||
template=resolved_template,
|
||||
system_prompt=system,
|
||||
parameters=params,
|
||||
)
|
||||
except ValueError as exc:
|
||||
console.print(f"[red]Invalid parameter:[/] {exc}")
|
||||
raise typer.Exit(1)
|
||||
console.print("[green]\u2713[/] Modelfile generated")
|
||||
|
||||
# Deploy
|
||||
console.print("[dim]Creating model in Ollama...[/]")
|
||||
success, message = deploy_to_ollama(name, modelfile)
|
||||
if not success:
|
||||
console.print(f"[red]Deploy failed:[/] {message}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
console.print(f"[green]\u2713[/] Model created: [bold]{name}[/]")
|
||||
console.print(
|
||||
Panel(
|
||||
f"Run: [bold]ollama run {name}[/]",
|
||||
title="[bold green]Deploy Complete![/]",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _auto_detect_template() -> Optional[str]:
|
||||
"""Try to infer chat template from soup.yaml in cwd."""
|
||||
from soup_cli.utils.ollama import infer_chat_template
|
||||
|
||||
config_path = Path("soup.yaml")
|
||||
if not config_path.exists():
|
||||
return None
|
||||
|
||||
try:
|
||||
import yaml
|
||||
|
||||
with open(config_path, encoding="utf-8") as fh:
|
||||
config = yaml.safe_load(fh)
|
||||
if not isinstance(config, dict):
|
||||
return None
|
||||
data_section = config.get("data", {})
|
||||
if isinstance(data_section, dict):
|
||||
fmt = data_section.get("format")
|
||||
return infer_chat_template(fmt)
|
||||
except (yaml.YAMLError, OSError, KeyError, ImportError):
|
||||
return None
|
||||
return None
|
||||
|
|
@ -61,6 +61,16 @@ def export(
|
|||
"--onnx-task",
|
||||
help="ONNX export task: text-generation (causal LM) or feature-extraction (embedding)",
|
||||
),
|
||||
deploy: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--deploy",
|
||||
help="Auto-deploy after export. Currently supported: ollama",
|
||||
),
|
||||
deploy_name: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--deploy-name",
|
||||
help="Model name for deployment (used with --deploy)",
|
||||
),
|
||||
):
|
||||
"""Export a model to GGUF, ONNX, or TensorRT-LLM format."""
|
||||
model_path = Path(model)
|
||||
|
|
@ -191,6 +201,10 @@ def export(
|
|||
)
|
||||
)
|
||||
|
||||
# --- Auto-deploy to Ollama if requested ---
|
||||
if deploy:
|
||||
_auto_deploy_ollama(output_path, model_name, deploy, deploy_name)
|
||||
|
||||
|
||||
def _detect_base_model(adapter_config_path: Path) -> Optional[str]:
|
||||
"""Read base_model_name_or_path from adapter_config.json."""
|
||||
|
|
@ -552,6 +566,55 @@ def _export_tensorrt(model_path: Path, output: Optional[str], base: Optional[str
|
|||
)
|
||||
|
||||
|
||||
def _auto_deploy_ollama(
|
||||
output_path: Path, model_name: str, deploy_target: str, deploy_name: Optional[str]
|
||||
):
|
||||
"""Auto-deploy a GGUF file to Ollama after export."""
|
||||
if deploy_target != "ollama":
|
||||
console.print(
|
||||
f"[red]Unsupported deploy target: {deploy_target}[/]\n"
|
||||
"Supported: ollama"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
from soup_cli.utils.ollama import (
|
||||
create_modelfile,
|
||||
deploy_to_ollama,
|
||||
detect_ollama,
|
||||
validate_model_name,
|
||||
)
|
||||
|
||||
ollama_name = deploy_name or f"soup-{model_name}"
|
||||
|
||||
valid, err = validate_model_name(ollama_name)
|
||||
if not valid:
|
||||
console.print(f"[red]Invalid deploy name:[/] {err}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
version = detect_ollama()
|
||||
if not version:
|
||||
console.print(
|
||||
"[red]Ollama not found — skipping deploy.[/]\n"
|
||||
"Install from: [bold]https://ollama.com[/]"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
console.print(f"\n[green]✓[/] Ollama v{version} detected — deploying as [bold]{ollama_name}[/]")
|
||||
console.print(
|
||||
"[yellow]Warning:[/] This will overwrite any existing Ollama model "
|
||||
f"named '{ollama_name}'."
|
||||
)
|
||||
|
||||
modelfile = create_modelfile(gguf_path=output_path, template="chatml")
|
||||
success, message = deploy_to_ollama(ollama_name, modelfile)
|
||||
if not success:
|
||||
console.print(f"[red]Deploy failed:[/] {message}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
console.print(f"[green]✓[/] Deployed to Ollama: [bold]{ollama_name}[/]")
|
||||
console.print(f"Run: [bold]ollama run {ollama_name}[/]")
|
||||
|
||||
|
||||
def _format_size(size_bytes: int) -> str:
|
||||
"""Format bytes into human-readable string."""
|
||||
value: float = float(size_bytes)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,292 @@
|
|||
"""Ollama integration utilities — detect, deploy, list, remove models."""
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
# Ollama TEMPLATE blocks for common chat formats
|
||||
OLLAMA_TEMPLATES: Dict[str, str] = {
|
||||
"chatml": (
|
||||
'{{ if .System }}<|im_start|>system\n'
|
||||
'{{ .System }}<|im_end|>\n'
|
||||
'{{ end }}{{ range .Messages }}'
|
||||
'{{ if eq .Role "user" }}<|im_start|>user\n'
|
||||
'{{ .Content }}<|im_end|>\n'
|
||||
'{{ else if eq .Role "assistant" }}<|im_start|>assistant\n'
|
||||
'{{ .Content }}<|im_end|>\n'
|
||||
'{{ end }}{{ end }}<|im_start|>assistant\n'
|
||||
),
|
||||
"llama": (
|
||||
'<|begin_of_text|>'
|
||||
'{{ if .System }}<|start_header_id|>system<|end_header_id|>\n\n'
|
||||
'{{ .System }}<|eot_id|>{{ end }}'
|
||||
'{{ range .Messages }}'
|
||||
'{{ if eq .Role "user" }}<|start_header_id|>user<|end_header_id|>\n\n'
|
||||
'{{ .Content }}<|eot_id|>'
|
||||
'{{ else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|>\n\n'
|
||||
'{{ .Content }}<|eot_id|>'
|
||||
'{{ end }}{{ end }}'
|
||||
'<|start_header_id|>assistant<|end_header_id|>\n\n'
|
||||
),
|
||||
"mistral": (
|
||||
'{{ if .System }}[INST] {{ .System }} [/INST]\n{{ end }}'
|
||||
'{{ range .Messages }}'
|
||||
'{{ if eq .Role "user" }}[INST] {{ .Content }} [/INST]\n'
|
||||
'{{ else if eq .Role "assistant" }}{{ .Content }}</s>\n'
|
||||
'{{ end }}{{ end }}'
|
||||
),
|
||||
"vicuna": (
|
||||
'{{ if .System }}{{ .System }}\n\n{{ end }}'
|
||||
'{{ range .Messages }}'
|
||||
'{{ if eq .Role "user" }}USER: {{ .Content }}\n'
|
||||
'{{ else if eq .Role "assistant" }}ASSISTANT: {{ .Content }}</s>\n'
|
||||
'{{ end }}{{ end }}ASSISTANT:'
|
||||
),
|
||||
"zephyr": (
|
||||
'{{ if .System }}<|system|>\n{{ .System }}</s>\n{{ end }}'
|
||||
'{{ range .Messages }}'
|
||||
'{{ if eq .Role "user" }}<|user|>\n{{ .Content }}</s>\n'
|
||||
'{{ else if eq .Role "assistant" }}<|assistant|>\n{{ .Content }}</s>\n'
|
||||
'{{ end }}{{ end }}<|assistant|>\n'
|
||||
),
|
||||
}
|
||||
|
||||
# Map Soup data formats to Ollama template names
|
||||
FORMAT_TO_TEMPLATE: Dict[str, str] = {
|
||||
"chatml": "chatml",
|
||||
"alpaca": "llama",
|
||||
"llama": "llama",
|
||||
"mistral": "mistral",
|
||||
"vicuna": "vicuna",
|
||||
"zephyr": "zephyr",
|
||||
"sharegpt": "chatml",
|
||||
}
|
||||
|
||||
# Valid model name pattern: alphanumeric, hyphens, underscores, colons (for tags)
|
||||
_MODEL_NAME_RE = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9._:-]*$")
|
||||
|
||||
# Soup-deployed model prefix
|
||||
SOUP_MODEL_PREFIX = "soup-"
|
||||
|
||||
# Allowed Ollama PARAMETER keys (prevents injection of arbitrary directives)
|
||||
ALLOWED_OLLAMA_PARAMS = frozenset({
|
||||
"temperature", "top_p", "top_k", "num_ctx", "num_predict",
|
||||
"stop", "repeat_penalty", "repeat_last_n", "seed", "mirostat",
|
||||
"mirostat_tau", "mirostat_eta", "num_gpu", "num_thread",
|
||||
"tfs_z", "typical_p", "penalize_newline",
|
||||
})
|
||||
|
||||
|
||||
def validate_model_name(name: str) -> Tuple[bool, str]:
|
||||
"""Validate Ollama model name.
|
||||
|
||||
Returns (is_valid, error_message).
|
||||
"""
|
||||
if not name:
|
||||
return False, "Model name cannot be empty"
|
||||
if len(name) > 128:
|
||||
return False, "Model name too long (max 128 characters)"
|
||||
# Block path separators and null bytes
|
||||
if "/" in name or "\\" in name or "\0" in name:
|
||||
return False, "Model name must not contain path separators or null bytes"
|
||||
if not _MODEL_NAME_RE.match(name):
|
||||
return False, (
|
||||
"Model name must start with alphanumeric and contain only "
|
||||
"alphanumeric, hyphens, underscores, dots, or colons"
|
||||
)
|
||||
return True, ""
|
||||
|
||||
|
||||
def validate_gguf_path(gguf_path: Path) -> Tuple[bool, str]:
|
||||
"""Validate GGUF file path — must exist and stay under cwd.
|
||||
|
||||
Returns (is_valid, error_message).
|
||||
"""
|
||||
if not gguf_path.exists():
|
||||
return False, f"GGUF file not found: {gguf_path}"
|
||||
if not gguf_path.is_file():
|
||||
return False, f"GGUF path is not a file: {gguf_path}"
|
||||
if gguf_path.suffix.lower() != ".gguf":
|
||||
return False, "File must have a .gguf extension"
|
||||
# Path traversal protection: resolve and check it's under cwd
|
||||
resolved = gguf_path.resolve()
|
||||
cwd = Path.cwd().resolve()
|
||||
try:
|
||||
resolved.relative_to(cwd)
|
||||
except ValueError:
|
||||
return False, "GGUF path must be under the current working directory"
|
||||
return True, ""
|
||||
|
||||
|
||||
def detect_ollama() -> Optional[str]:
|
||||
"""Check if Ollama is installed, return version string or None."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ollama", "--version"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
output = result.stdout.strip() or result.stderr.strip()
|
||||
# Extract version from output like "ollama version is 0.6.2"
|
||||
match = re.search(r"(\d+\.\d+\.\d+)", output)
|
||||
if match:
|
||||
return match.group(1)
|
||||
return output if output else "unknown"
|
||||
return None
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired, OSError):
|
||||
return None
|
||||
|
||||
|
||||
def infer_chat_template(config_format: Optional[str]) -> Optional[str]:
|
||||
"""Map a Soup config format to an Ollama template name.
|
||||
|
||||
Returns template name (key into OLLAMA_TEMPLATES) or None if unknown.
|
||||
"""
|
||||
if not config_format:
|
||||
return None
|
||||
return FORMAT_TO_TEMPLATE.get(config_format.lower())
|
||||
|
||||
|
||||
def create_modelfile(
|
||||
gguf_path: Path,
|
||||
template: Optional[str] = None,
|
||||
system_prompt: Optional[str] = None,
|
||||
parameters: Optional[Dict[str, str]] = None,
|
||||
) -> str:
|
||||
"""Generate Ollama Modelfile content.
|
||||
|
||||
Args:
|
||||
gguf_path: Path to the GGUF model file.
|
||||
template: Template name (key in OLLAMA_TEMPLATES) or raw template string.
|
||||
system_prompt: Optional system prompt.
|
||||
parameters: Optional dict of PARAMETER key=value pairs.
|
||||
|
||||
Returns:
|
||||
Modelfile content as string.
|
||||
"""
|
||||
lines = [f"FROM {gguf_path}"]
|
||||
|
||||
# Template
|
||||
if template:
|
||||
template_content = OLLAMA_TEMPLATES.get(template, template)
|
||||
lines.append(f'TEMPLATE """{template_content}"""')
|
||||
|
||||
# System prompt
|
||||
if system_prompt:
|
||||
# Escape double quotes and strip newlines in system prompt
|
||||
escaped = system_prompt.replace('"', '\\"')
|
||||
escaped = escaped.replace("\n", " ").replace("\r", "")
|
||||
lines.append(f'SYSTEM "{escaped}"')
|
||||
|
||||
# Parameters — validated against allowlist, no control chars
|
||||
if parameters:
|
||||
for key, value in sorted(parameters.items()):
|
||||
if key not in ALLOWED_OLLAMA_PARAMS:
|
||||
raise ValueError(
|
||||
f"Unknown Ollama parameter: {key!r}. "
|
||||
f"Allowed: {', '.join(sorted(ALLOWED_OLLAMA_PARAMS))}"
|
||||
)
|
||||
if "\n" in value or "\r" in value or "\0" in value:
|
||||
raise ValueError(
|
||||
f"Parameter value for {key!r} contains "
|
||||
"illegal characters (newline or null)"
|
||||
)
|
||||
lines.append(f"PARAMETER {key} {value}")
|
||||
|
||||
return "\n\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def deploy_to_ollama(name: str, modelfile_content: str) -> Tuple[bool, str]:
|
||||
"""Deploy a model to Ollama via `ollama create`.
|
||||
|
||||
Args:
|
||||
name: Model name for Ollama.
|
||||
modelfile_content: Full Modelfile content.
|
||||
|
||||
Returns:
|
||||
(success, message) tuple.
|
||||
"""
|
||||
import tempfile
|
||||
|
||||
with tempfile.NamedTemporaryFile(
|
||||
mode="w", suffix=".Modelfile", delete=False, encoding="utf-8"
|
||||
) as tmp:
|
||||
tmp.write(modelfile_content)
|
||||
tmp_path = tmp.name
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ollama", "create", name, "-f", tmp_path],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=300,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return True, result.stdout.strip() or "Model created successfully"
|
||||
return False, result.stderr.strip() or "ollama create failed"
|
||||
except FileNotFoundError:
|
||||
return False, "Ollama not found. Install from https://ollama.com"
|
||||
except subprocess.TimeoutExpired:
|
||||
return False, "ollama create timed out (5 minutes)"
|
||||
except OSError as exc:
|
||||
return False, f"Failed to run ollama: {exc}"
|
||||
finally:
|
||||
Path(tmp_path).unlink(missing_ok=True)
|
||||
|
||||
|
||||
def list_soup_models() -> List[Dict[str, str]]:
|
||||
"""List Ollama models deployed by Soup (prefixed with soup-).
|
||||
|
||||
Returns list of dicts with 'name' and 'size' keys.
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ollama", "list"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired, OSError):
|
||||
return []
|
||||
|
||||
models = []
|
||||
for line in result.stdout.strip().splitlines()[1:]: # Skip header
|
||||
parts = line.split()
|
||||
if not parts:
|
||||
continue
|
||||
model_name = parts[0]
|
||||
# Filter to soup- prefixed models
|
||||
if model_name.startswith(SOUP_MODEL_PREFIX) or (
|
||||
":" in model_name and model_name.split(":")[0].startswith(SOUP_MODEL_PREFIX)
|
||||
):
|
||||
size = parts[2] + " " + parts[3] if len(parts) >= 4 else "unknown"
|
||||
models.append({"name": model_name, "size": size})
|
||||
return models
|
||||
|
||||
|
||||
def remove_model(name: str) -> Tuple[bool, str]:
|
||||
"""Remove a model from Ollama.
|
||||
|
||||
Returns (success, message) tuple.
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ollama", "rm", name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return True, result.stdout.strip() or f"Model '{name}' removed"
|
||||
return False, result.stderr.strip() or f"Failed to remove '{name}'"
|
||||
except FileNotFoundError:
|
||||
return False, "Ollama not found. Install from https://ollama.com"
|
||||
except subprocess.TimeoutExpired:
|
||||
return False, "ollama rm timed out"
|
||||
except OSError as exc:
|
||||
return False, f"Failed to run ollama: {exc}"
|
||||
|
|
@ -0,0 +1,728 @@
|
|||
"""Tests for soup deploy ollama command and Ollama utilities."""
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import click
|
||||
import pytest
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from soup_cli.cli import app
|
||||
from soup_cli.utils.ollama import (
|
||||
ALLOWED_OLLAMA_PARAMS,
|
||||
FORMAT_TO_TEMPLATE,
|
||||
OLLAMA_TEMPLATES,
|
||||
SOUP_MODEL_PREFIX,
|
||||
create_modelfile,
|
||||
deploy_to_ollama,
|
||||
detect_ollama,
|
||||
infer_chat_template,
|
||||
list_soup_models,
|
||||
remove_model,
|
||||
validate_gguf_path,
|
||||
validate_model_name,
|
||||
)
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
# Patch targets — lazy imports in deploy.py resolve to soup_cli.utils.ollama
|
||||
_OLLAMA = "soup_cli.utils.ollama"
|
||||
|
||||
|
||||
# ─── validate_model_name ───
|
||||
|
||||
|
||||
def test_validate_model_name_valid():
|
||||
valid, err = validate_model_name("my-model")
|
||||
assert valid is True
|
||||
assert err == ""
|
||||
|
||||
|
||||
def test_validate_model_name_with_colon():
|
||||
valid, err = validate_model_name("soup-model:latest")
|
||||
assert valid is True
|
||||
|
||||
|
||||
def test_validate_model_name_with_dots():
|
||||
valid, err = validate_model_name("my.model.v2")
|
||||
assert valid is True
|
||||
|
||||
|
||||
def test_validate_model_name_empty():
|
||||
valid, err = validate_model_name("")
|
||||
assert valid is False
|
||||
assert "empty" in err.lower()
|
||||
|
||||
|
||||
def test_validate_model_name_too_long():
|
||||
valid, err = validate_model_name("a" * 129)
|
||||
assert valid is False
|
||||
assert "long" in err.lower()
|
||||
|
||||
|
||||
def test_validate_model_name_with_slash():
|
||||
valid, err = validate_model_name("bad/name")
|
||||
assert valid is False
|
||||
assert "path separator" in err.lower()
|
||||
|
||||
|
||||
def test_validate_model_name_with_backslash():
|
||||
valid, err = validate_model_name("bad\\name")
|
||||
assert valid is False
|
||||
assert "path separator" in err.lower()
|
||||
|
||||
|
||||
def test_validate_model_name_with_null():
|
||||
valid, err = validate_model_name("bad\0name")
|
||||
assert valid is False
|
||||
assert "null" in err.lower()
|
||||
|
||||
|
||||
def test_validate_model_name_starts_with_hyphen():
|
||||
valid, err = validate_model_name("-bad")
|
||||
assert valid is False
|
||||
assert "alphanumeric" in err.lower()
|
||||
|
||||
|
||||
# ─── validate_gguf_path ───
|
||||
|
||||
|
||||
def test_validate_gguf_path_exists(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake gguf")
|
||||
valid, err = validate_gguf_path(gguf)
|
||||
assert valid is True
|
||||
assert err == ""
|
||||
|
||||
|
||||
def test_validate_gguf_path_not_exists(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
valid, err = validate_gguf_path(tmp_path / "missing.gguf")
|
||||
assert valid is False
|
||||
assert "not found" in err.lower()
|
||||
|
||||
|
||||
def test_validate_gguf_path_is_directory(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
valid, err = validate_gguf_path(tmp_path)
|
||||
assert valid is False
|
||||
assert "not a file" in err.lower()
|
||||
|
||||
|
||||
def test_validate_gguf_path_bad_extension(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
bad = tmp_path / "model.bin"
|
||||
bad.write_bytes(b"fake")
|
||||
valid, err = validate_gguf_path(bad)
|
||||
assert valid is False
|
||||
assert ".gguf" in err.lower()
|
||||
|
||||
|
||||
def test_validate_gguf_path_traversal(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
outside = tmp_path.parent / "outside.gguf"
|
||||
outside.write_bytes(b"fake")
|
||||
try:
|
||||
valid, err = validate_gguf_path(outside)
|
||||
assert valid is False
|
||||
assert "current working directory" in err.lower()
|
||||
finally:
|
||||
outside.unlink(missing_ok=True)
|
||||
|
||||
|
||||
# ─── detect_ollama ───
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_detect_ollama_installed(mock_run):
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0, stdout="ollama version is 0.6.2", stderr=""
|
||||
)
|
||||
version = detect_ollama()
|
||||
assert version == "0.6.2"
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_detect_ollama_not_installed(mock_run):
|
||||
mock_run.side_effect = FileNotFoundError
|
||||
assert detect_ollama() is None
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_detect_ollama_timeout(mock_run):
|
||||
mock_run.side_effect = subprocess.TimeoutExpired(cmd="ollama", timeout=10)
|
||||
assert detect_ollama() is None
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_detect_ollama_nonzero(mock_run):
|
||||
mock_run.return_value = MagicMock(returncode=1, stdout="", stderr="")
|
||||
assert detect_ollama() is None
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_detect_ollama_no_version_match(mock_run):
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0, stdout="ollama unknown", stderr=""
|
||||
)
|
||||
version = detect_ollama()
|
||||
assert version == "ollama unknown"
|
||||
|
||||
|
||||
# ─── infer_chat_template ───
|
||||
|
||||
|
||||
def test_infer_chatml():
|
||||
assert infer_chat_template("chatml") == "chatml"
|
||||
|
||||
|
||||
def test_infer_alpaca():
|
||||
assert infer_chat_template("alpaca") == "llama"
|
||||
|
||||
|
||||
def test_infer_sharegpt():
|
||||
assert infer_chat_template("sharegpt") == "chatml"
|
||||
|
||||
|
||||
def test_infer_mistral():
|
||||
assert infer_chat_template("mistral") == "mistral"
|
||||
|
||||
|
||||
def test_infer_unknown():
|
||||
assert infer_chat_template("xyz") is None
|
||||
|
||||
|
||||
def test_infer_none():
|
||||
assert infer_chat_template(None) is None
|
||||
|
||||
|
||||
def test_infer_case_insensitive():
|
||||
assert infer_chat_template("ChatML") == "chatml"
|
||||
|
||||
|
||||
# ─── create_modelfile ───
|
||||
|
||||
|
||||
def test_create_modelfile_basic():
|
||||
result = create_modelfile(Path("model.gguf"))
|
||||
assert "FROM" in result
|
||||
assert "model.gguf" in result
|
||||
assert "TEMPLATE" not in result
|
||||
|
||||
|
||||
def test_create_modelfile_with_template():
|
||||
result = create_modelfile(Path("model.gguf"), template="chatml")
|
||||
assert "model.gguf" in result
|
||||
assert "TEMPLATE" in result
|
||||
assert "im_start" in result
|
||||
|
||||
|
||||
def test_create_modelfile_with_system():
|
||||
result = create_modelfile(Path("m.gguf"), system_prompt="You are helpful.")
|
||||
assert 'SYSTEM "You are helpful."' in result
|
||||
|
||||
|
||||
def test_create_modelfile_with_system_quotes():
|
||||
result = create_modelfile(Path("m.gguf"), system_prompt='Say "hello"')
|
||||
assert 'SYSTEM "Say \\"hello\\""' in result
|
||||
|
||||
|
||||
def test_create_modelfile_with_params():
|
||||
result = create_modelfile(
|
||||
Path("m.gguf"), parameters={"temperature": "0.7", "top_p": "0.9"}
|
||||
)
|
||||
assert "PARAMETER temperature 0.7" in result
|
||||
assert "PARAMETER top_p 0.9" in result
|
||||
|
||||
|
||||
def test_create_modelfile_full():
|
||||
result = create_modelfile(
|
||||
Path("out") / "model.gguf",
|
||||
template="llama",
|
||||
system_prompt="Be concise.",
|
||||
parameters={"temperature": "0.5"},
|
||||
)
|
||||
assert "FROM" in result
|
||||
assert "model.gguf" in result
|
||||
assert "TEMPLATE" in result
|
||||
assert "begin_of_text" in result
|
||||
assert 'SYSTEM "Be concise."' in result
|
||||
assert "PARAMETER temperature 0.5" in result
|
||||
|
||||
|
||||
def test_create_modelfile_custom_template():
|
||||
custom = "{{ .Prompt }}"
|
||||
result = create_modelfile(Path("m.gguf"), template=custom)
|
||||
assert custom in result
|
||||
|
||||
|
||||
def test_create_modelfile_rejects_unknown_param():
|
||||
with pytest.raises(ValueError, match="Unknown Ollama parameter"):
|
||||
create_modelfile(Path("m.gguf"), parameters={"evil_key": "value"})
|
||||
|
||||
|
||||
def test_create_modelfile_rejects_newline_in_param_value():
|
||||
with pytest.raises(ValueError, match="illegal characters"):
|
||||
create_modelfile(
|
||||
Path("m.gguf"), parameters={"temperature": "0.7\nSYSTEM injected"}
|
||||
)
|
||||
|
||||
|
||||
def test_create_modelfile_allows_valid_params():
|
||||
result = create_modelfile(
|
||||
Path("m.gguf"),
|
||||
parameters={"temperature": "0.7", "top_k": "40", "seed": "42"},
|
||||
)
|
||||
assert "PARAMETER temperature 0.7" in result
|
||||
assert "PARAMETER top_k 40" in result
|
||||
assert "PARAMETER seed 42" in result
|
||||
|
||||
|
||||
# ─── deploy_to_ollama ───
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_deploy_to_ollama_success(mock_run):
|
||||
mock_run.return_value = MagicMock(returncode=0, stdout="success")
|
||||
success, msg = deploy_to_ollama("test-model", "FROM m.gguf\n")
|
||||
assert success is True
|
||||
assert "success" in msg
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_deploy_to_ollama_failure(mock_run):
|
||||
mock_run.return_value = MagicMock(returncode=1, stderr="error creating model")
|
||||
success, msg = deploy_to_ollama("test-model", "FROM m.gguf\n")
|
||||
assert success is False
|
||||
assert "error" in msg.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_deploy_to_ollama_not_found(mock_run):
|
||||
mock_run.side_effect = FileNotFoundError
|
||||
success, msg = deploy_to_ollama("test-model", "FROM m.gguf\n")
|
||||
assert success is False
|
||||
assert "not found" in msg.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_deploy_to_ollama_timeout(mock_run):
|
||||
mock_run.side_effect = subprocess.TimeoutExpired(cmd="ollama", timeout=300)
|
||||
success, msg = deploy_to_ollama("test-model", "FROM m.gguf\n")
|
||||
assert success is False
|
||||
assert "timed out" in msg.lower()
|
||||
|
||||
|
||||
# ─── list_soup_models ───
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_list_soup_models_found(mock_run):
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout=(
|
||||
"NAME ID SIZE MODIFIED\n"
|
||||
"soup-my-model:latest abc123def456 4.1 GB 2 hours ago\n"
|
||||
"llama3.1:latest xyz789abc123 8.0 GB 3 days ago\n"
|
||||
"soup-code:latest def456ghi789 3.2 GB 1 day ago\n"
|
||||
),
|
||||
)
|
||||
models = list_soup_models()
|
||||
assert len(models) == 2
|
||||
assert models[0]["name"] == "soup-my-model:latest"
|
||||
assert models[1]["name"] == "soup-code:latest"
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_list_soup_models_empty(mock_run):
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout="NAME ID SIZE MODIFIED\n",
|
||||
)
|
||||
assert list_soup_models() == []
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_list_soup_models_ollama_not_found(mock_run):
|
||||
mock_run.side_effect = FileNotFoundError
|
||||
assert list_soup_models() == []
|
||||
|
||||
|
||||
# ─── remove_model ───
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_remove_model_success(mock_run):
|
||||
mock_run.return_value = MagicMock(returncode=0, stdout="deleted 'soup-test'")
|
||||
success, msg = remove_model("soup-test")
|
||||
assert success is True
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_remove_model_failure(mock_run):
|
||||
mock_run.return_value = MagicMock(returncode=1, stderr="model not found")
|
||||
success, msg = remove_model("soup-test")
|
||||
assert success is False
|
||||
assert "not found" in msg.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.subprocess.run")
|
||||
def test_remove_model_not_installed(mock_run):
|
||||
mock_run.side_effect = FileNotFoundError
|
||||
success, msg = remove_model("soup-test")
|
||||
assert success is False
|
||||
assert "not found" in msg.lower()
|
||||
|
||||
|
||||
# ─── Constants ───
|
||||
|
||||
|
||||
def test_ollama_templates_keys():
|
||||
assert "chatml" in OLLAMA_TEMPLATES
|
||||
assert "llama" in OLLAMA_TEMPLATES
|
||||
assert "mistral" in OLLAMA_TEMPLATES
|
||||
assert "vicuna" in OLLAMA_TEMPLATES
|
||||
assert "zephyr" in OLLAMA_TEMPLATES
|
||||
|
||||
|
||||
def test_format_to_template_mapping():
|
||||
assert FORMAT_TO_TEMPLATE["chatml"] == "chatml"
|
||||
assert FORMAT_TO_TEMPLATE["alpaca"] == "llama"
|
||||
assert FORMAT_TO_TEMPLATE["sharegpt"] == "chatml"
|
||||
|
||||
|
||||
def test_soup_model_prefix():
|
||||
assert SOUP_MODEL_PREFIX == "soup-"
|
||||
|
||||
|
||||
def test_allowed_ollama_params():
|
||||
assert "temperature" in ALLOWED_OLLAMA_PARAMS
|
||||
assert "top_p" in ALLOWED_OLLAMA_PARAMS
|
||||
assert "seed" in ALLOWED_OLLAMA_PARAMS
|
||||
assert len(ALLOWED_OLLAMA_PARAMS) >= 10
|
||||
|
||||
|
||||
# ─── CLI: soup deploy ollama --help ───
|
||||
|
||||
|
||||
def test_deploy_ollama_help():
|
||||
result = runner.invoke(app, ["deploy", "ollama", "--help"])
|
||||
assert result.exit_code == 0
|
||||
assert "ollama" in result.output.lower()
|
||||
assert "--model" in result.output
|
||||
assert "--name" in result.output
|
||||
|
||||
|
||||
def test_deploy_help():
|
||||
result = runner.invoke(app, ["deploy", "--help"])
|
||||
assert result.exit_code == 0
|
||||
assert "ollama" in result.output.lower()
|
||||
|
||||
|
||||
# ─── CLI: soup deploy ollama --list ───
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
@patch(f"{_OLLAMA}.list_soup_models", return_value=[])
|
||||
def test_deploy_list_empty(mock_list, mock_detect):
|
||||
result = runner.invoke(app, ["deploy", "ollama", "--list"])
|
||||
assert result.exit_code == 0
|
||||
assert "no soup-deployed" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
@patch(
|
||||
f"{_OLLAMA}.list_soup_models",
|
||||
return_value=[{"name": "soup-test:latest", "size": "4.1 GB"}],
|
||||
)
|
||||
def test_deploy_list_with_models(mock_list, mock_detect):
|
||||
result = runner.invoke(app, ["deploy", "ollama", "--list"])
|
||||
assert result.exit_code == 0
|
||||
assert "soup-test" in result.output
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value=None)
|
||||
def test_deploy_list_no_ollama(mock_detect):
|
||||
result = runner.invoke(app, ["deploy", "ollama", "--list"])
|
||||
assert result.exit_code == 1
|
||||
assert "not found" in result.output.lower()
|
||||
|
||||
|
||||
# ─── CLI: soup deploy ollama --remove ───
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
@patch(f"{_OLLAMA}.remove_model", return_value=(True, "deleted"))
|
||||
def test_deploy_remove_success(mock_rm, mock_detect):
|
||||
result = runner.invoke(app, ["deploy", "ollama", "--remove", "soup-test", "--yes"])
|
||||
assert result.exit_code == 0
|
||||
assert "deleted" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
@patch(f"{_OLLAMA}.remove_model", return_value=(False, "not found"))
|
||||
def test_deploy_remove_failure(mock_rm, mock_detect):
|
||||
result = runner.invoke(app, ["deploy", "ollama", "--remove", "soup-bad", "--yes"])
|
||||
assert result.exit_code == 1
|
||||
assert "not found" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value=None)
|
||||
def test_deploy_remove_no_ollama(mock_detect):
|
||||
result = runner.invoke(app, ["deploy", "ollama", "--remove", "soup-test", "--yes"])
|
||||
assert result.exit_code == 1
|
||||
|
||||
|
||||
# ─── CLI: soup deploy ollama (deploy mode) ───
|
||||
|
||||
|
||||
def test_deploy_missing_model():
|
||||
result = runner.invoke(app, ["deploy", "ollama"])
|
||||
assert result.exit_code == 1
|
||||
assert "--model" in result.output
|
||||
|
||||
|
||||
def test_deploy_missing_name(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake")
|
||||
result = runner.invoke(app, ["deploy", "ollama", "--model", str(gguf)])
|
||||
assert result.exit_code == 1
|
||||
assert "--name" in result.output
|
||||
|
||||
|
||||
def test_deploy_invalid_name(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake")
|
||||
result = runner.invoke(
|
||||
app, ["deploy", "ollama", "--model", str(gguf), "--name", "bad/name"]
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "invalid" in result.output.lower()
|
||||
|
||||
|
||||
def test_deploy_gguf_not_found(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
result = runner.invoke(
|
||||
app, ["deploy", "ollama", "--model", "missing.gguf", "--name", "soup-test"]
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "not found" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value=None)
|
||||
def test_deploy_ollama_not_installed(mock_detect, tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake")
|
||||
result = runner.invoke(
|
||||
app,
|
||||
["deploy", "ollama", "--model", str(gguf), "--name", "soup-test"],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "not found" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
def test_deploy_invalid_template(mock_detect, tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake")
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"deploy", "ollama",
|
||||
"--model", str(gguf),
|
||||
"--name", "soup-test",
|
||||
"--template", "nonexistent",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "unknown template" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
def test_deploy_bad_parameter_format(mock_detect, tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake")
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"deploy", "ollama",
|
||||
"--model", str(gguf),
|
||||
"--name", "soup-test",
|
||||
"--template", "chatml",
|
||||
"--parameter", "bad_no_equals",
|
||||
"--yes",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "invalid parameter" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
def test_deploy_unknown_parameter_key(mock_detect, tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake")
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"deploy", "ollama",
|
||||
"--model", str(gguf),
|
||||
"--name", "soup-test",
|
||||
"--template", "chatml",
|
||||
"--parameter", "evil_key=value",
|
||||
"--yes",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "invalid parameter" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.deploy_to_ollama", return_value=(True, "created"))
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
def test_deploy_full_success(mock_detect, mock_deploy, tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake gguf data")
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"deploy", "ollama",
|
||||
"--model", str(gguf),
|
||||
"--name", "soup-test",
|
||||
"--template", "chatml",
|
||||
"--system", "You are helpful.",
|
||||
"--parameter", "temperature=0.7",
|
||||
"--yes",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "soup-test" in result.output
|
||||
assert "deploy complete" in result.output.lower()
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.deploy_to_ollama", return_value=(False, "disk full"))
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
def test_deploy_create_fails(mock_detect, mock_deploy, tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
gguf = tmp_path / "model.gguf"
|
||||
gguf.write_bytes(b"fake gguf data")
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"deploy", "ollama",
|
||||
"--model", str(gguf),
|
||||
"--name", "soup-test",
|
||||
"--template", "chatml",
|
||||
"--yes",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "disk full" in result.output.lower()
|
||||
|
||||
|
||||
# ─── CLI: soup export --deploy ───
|
||||
|
||||
|
||||
def test_export_deploy_flag_in_help():
|
||||
result = runner.invoke(app, ["export", "--help"])
|
||||
assert result.exit_code == 0
|
||||
assert "--deploy" in result.output
|
||||
|
||||
|
||||
# ─── _auto_detect_template ───
|
||||
|
||||
|
||||
def test_auto_detect_from_soup_yaml(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
(tmp_path / "soup.yaml").write_text(
|
||||
"base: test\ndata:\n train: data.jsonl\n format: chatml\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
from soup_cli.commands.deploy import _auto_detect_template
|
||||
|
||||
result = _auto_detect_template()
|
||||
assert result == "chatml"
|
||||
|
||||
|
||||
def test_auto_detect_no_soup_yaml(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
from soup_cli.commands.deploy import _auto_detect_template
|
||||
|
||||
result = _auto_detect_template()
|
||||
assert result is None
|
||||
|
||||
|
||||
def test_auto_detect_invalid_yaml(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
(tmp_path / "soup.yaml").write_text(":::invalid", encoding="utf-8")
|
||||
from soup_cli.commands.deploy import _auto_detect_template
|
||||
|
||||
result = _auto_detect_template()
|
||||
assert result is None
|
||||
|
||||
|
||||
def test_auto_detect_no_format(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
(tmp_path / "soup.yaml").write_text(
|
||||
"base: test\ndata:\n train: data.jsonl\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
from soup_cli.commands.deploy import _auto_detect_template
|
||||
|
||||
result = _auto_detect_template()
|
||||
assert result is None
|
||||
|
||||
|
||||
# ─── Export _auto_deploy_ollama ───
|
||||
|
||||
|
||||
def test_export_deploy_unsupported_target():
|
||||
"""_auto_deploy_ollama rejects non-ollama targets."""
|
||||
from soup_cli.commands.export import _auto_deploy_ollama
|
||||
|
||||
with pytest.raises((SystemExit, click.exceptions.Exit)):
|
||||
_auto_deploy_ollama(Path("m.gguf"), "model", "kubernetes", None)
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value=None)
|
||||
def test_export_deploy_ollama_not_found(mock_detect):
|
||||
"""_auto_deploy_ollama exits if Ollama not installed."""
|
||||
from soup_cli.commands.export import _auto_deploy_ollama
|
||||
|
||||
with pytest.raises((SystemExit, click.exceptions.Exit)):
|
||||
_auto_deploy_ollama(Path("m.gguf"), "model", "ollama", None)
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.deploy_to_ollama", return_value=(True, "ok"))
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
def test_export_deploy_ollama_success(mock_detect, mock_deploy):
|
||||
"""_auto_deploy_ollama succeeds with valid inputs."""
|
||||
from soup_cli.commands.export import _auto_deploy_ollama
|
||||
|
||||
# Should not raise
|
||||
_auto_deploy_ollama(Path("model.gguf"), "mymodel", "ollama", "soup-mymodel")
|
||||
|
||||
|
||||
@patch(f"{_OLLAMA}.deploy_to_ollama", return_value=(False, "fail"))
|
||||
@patch(f"{_OLLAMA}.detect_ollama", return_value="0.6.2")
|
||||
def test_export_deploy_ollama_create_fails(mock_detect, mock_deploy):
|
||||
"""_auto_deploy_ollama exits on deploy failure."""
|
||||
from soup_cli.commands.export import _auto_deploy_ollama
|
||||
|
||||
with pytest.raises((SystemExit, click.exceptions.Exit)):
|
||||
_auto_deploy_ollama(Path("m.gguf"), "model", "ollama", "soup-model")
|
||||
|
||||
|
||||
def test_export_deploy_invalid_name():
|
||||
"""_auto_deploy_ollama rejects invalid model name."""
|
||||
from soup_cli.commands.export import _auto_deploy_ollama
|
||||
|
||||
with pytest.raises((SystemExit, click.exceptions.Exit)):
|
||||
_auto_deploy_ollama(Path("m.gguf"), "model", "ollama", "bad/name")
|
||||
Loading…
Reference in New Issue