mirror of https://github.com/razor-ai/soup.git
feat(v0.53.10): Quick wins + packaging + UX wiring
7 issues closed: - #150 [mix] pyproject extra bundles scikit-optimize so `soup data mix --optimize` runs the Bayesian loop instead of the v0.48.0 Dirichlet fallback; new describe_default_optimizer() helper labels the active backend without paying skopt's import cost. - #113 [data-pro] extras (langdetect + presidio-analyzer) with lazy fall-through helpers in utils/data_score (broader language coverage + Presidio entity recognition on top of the v0.47.0 regex baseline). Llama-Guard-3-1B documented as a manual recipe (license + size). - #154 SOUP_POSTHOG_KEY / SOUP_POSTHOG_ENDPOINT env override via sentinel-based explicit-vs-env precedence; HTTPS-only + RFC1918/link-local rejection on the endpoint; null-byte / control-char / >256-char rejection on the key. - #152 --hub flag plumbed on chat / serve / infer / merge / export / push via shared utils/hubs.apply_hub_to_cli_model + prefetch_model_from_hub helpers; push uses upload_repo (skips HF-specific Collections + model-card auto-render on non-HF hubs). - #153 `soup data download --hub modelscope|modelers` live SDK (lifts the v0.53.8 advisory-only path); friendly ImportError advisory when the SDK is missing. - #155 Web UI Tool Outputs panel — `loadToolOutputs` polls /api/tool-outputs every 3s; XSS-safe DOM-built table (textContent per cell, no innerHTML for user-controlled fields); Bearer token threaded via the v0.53.9 window._authToken bootstrap. - #156 SoupTrainerCallback.on_step_end records tool_calls counts from kwargs['inputs'] into the global tool buffer. Best-effort (# noqa: BLE001 per project policy — training must never crash). 13 review-fixes applied (4 HIGH / 5 MEDIUM / 4 LOW): - HIGH PostHog explicit-endpoint precedence sentinel - HIGH absolute path leak in local_path advisory reduced to relpath - HIGH Rich markup escape on base / local_path / cache_dir - HIGH callback # noqa: BLE001 per project policy - MED `import time` moved out of try block - MED oversize key + explicit-empty key rejection tests - MED source-grep regression guards (advisory-removal, helper imports across 5 non-push commands) - MED `prefetch_model_from_hub` outside-cwd cache_root rejection - LOW empty-list + bool-True tool_calls no-op tests - LOW push.py uses upload_repo + validate_hub_name regression guard Test count: 8285 -> 8330 (+45 in tests/test_v05310.py). Full suite green; ruff clean; on Win+Py3.10. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alpamys
parent
e8422660a5
commit
76f033a6ff
|
|
@ -111,7 +111,7 @@ soup_cli/
|
|||
templates/ - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
|
||||
ui/ - Web UI (FastAPI + HTML/JS SPA)
|
||||
|
||||
tests/ - Test suite (189 files, 8285 tests)
|
||||
tests/ - Test suite (190 files, 8330 tests)
|
||||
examples/ - Real-world config examples and datasets
|
||||
```
|
||||
|
||||
|
|
|
|||
18
README.md
18
README.md
|
|
@ -42,17 +42,15 @@ soup train
|
|||
|
||||
Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
|
||||
|
||||
**v0.53.9 — Live Dashboard + UX + Bench + Standalone CLIs**: Eight features that close out the v0.44.x live-monitoring deferrals plus a long tail of standalone CLI wins.
|
||||
**v0.53.10 — Quick wins + packaging + UX wiring**: Seven small wins that close the long tail of v0.40 → v0.53 deferred packaging / UX issues.
|
||||
|
||||
- **Live `/api/train/stream` SSE endpoint.** Async FastAPI route streams `TrainEvent` payloads (loss / lr / grad_norm / EMA / p95 tail latency) as W3C SSE frames. Per-subscriber cursor — multiple dashboard tabs each receive every event without starving each other. `asyncio.sleep` so uvicorn's async loop is never blocked. Drains gracefully with a `status=done` event when training exits.
|
||||
- **`soup ui --public` + phone-scannable QR.** Binds `0.0.0.0`, detects the actual LAN IP via a `socket.SOCK_DGRAM` connect trick (no packets sent), and prints an ASCII QR encoding `http://<lan-ip>:<port>/?token=<bearer>`. The SPA picks up `?token=` on load via `URLSearchParams`, caches it in `sessionStorage`, and rewrites the URL clean so the token doesn't sit in browser history. `--auth-token` override accepts a stable urlsafe-base64 token for repeatable phone bookmarks. CORS regex auto-widens to loopback + RFC1918 ranges when public; Bearer token gates mutating endpoints. Concurrent token rotation is now lock-protected.
|
||||
- **`soup serve --reasoning-parser {deepseek-r1|qwen3|phi4|openthinker}`.** Strips `<think>...</think>` (or `<|begin_of_thought|>...<|end_of_thought|>` for OpenThinker) from responses before they hit the client. Pre-compiled regex with a marker-token fast-path (skips `re.sub` entirely when no marker is present) and a 1 MiB input cap. Strips only leading newlines after removal so code outputs that begin with intentional whitespace are preserved.
|
||||
- **`/api/tool-outputs` + `ToolOutputsBuffer` global singleton.** Process-wide thread-safe ring buffer of tool-call records (`name` / `started_ts` / `duration_ms` / `success` / `output_preview` / `error`). New `/api/tool-outputs?limit=N` JSON endpoint reads the buffer for live dashboard polling. SFT trainer-side observation hook lands in a follow-up.
|
||||
- **`soup tokenizer train`.** New BPE training CLI via the `tokenizers` library. JSONL or plaintext input (with ShareGPT `messages` fallback), `--vocab-size` bounded [256, 200000], `--special-token` repeatable with NUL/oversize/dedup validation. `os.lstat + S_ISLNK` rejection on the raw user-supplied input path BEFORE realpath, 50 MiB total / 8 KiB per-line caps, post-`mkdir` re-check on the output directory so a planted symlink can't redirect the write.
|
||||
- **`soup bench --p50 --p95`.** Per-prompt latency capture rendered as an extra Rich table after the throughput row. Reuses `utils/tail_latency.summarise_latency` (v0.44.0). `--prompts-file` now rejects symlinked inputs on the raw path before realpath, in line with the project-wide TOCTOU policy.
|
||||
- **`soup bench --backend auto`.** New default — probes the model directory for MLX `weights.npz` and falls back to `config.json` `model_type` keyword detection. `SOUP_BENCH_BACKEND` env hint overrides the probe. Every iterdir entry is `os.lstat`-checked so a symlinked `weights.npz` cannot trigger MLX dispatch.
|
||||
- **`examples/synthetic_workflow.md` walkthrough.** End-to-end docs covering `soup data generate` → `filter` → `score` → `decontaminate` → `train`, plus the `soup ui --public` phone-monitoring loop.
|
||||
- **+28 net new tests** (8257 → 8285) in `test_v0539.py`. Three review agents ran (python / code / security / tdd / verification); 34 findings fixed: H1 (QR token never consumed by SPA — now hydrates from URL into sessionStorage), H2 (`set_auth_token` rotation race — now lock-protected), tokenizer input/output symlink rejection (raw-path lstat), prompts-file symlink rejection, regex fast-path skip on missing markers, leading-newline strip vs full `lstrip()`, CORS regex for `--public` LAN mode, `_has_mlx_weights` per-entry lstat, and more.
|
||||
- **`[mix]` + `[data-pro]` extras.** `pip install soup-cli[mix]` bundles `scikit-optimize` so `soup data mix --optimize` runs the real Bayesian loop instead of the v0.48.0 Dirichlet fallback. `pip install soup-cli[data-pro]` bundles `langdetect` (probabilistic language detection, broader coverage than the in-tree stopword heuristic) and `presidio-analyzer` (Microsoft Presidio entity recognition for PII — locations, dates, IBAN, etc. on top of the v0.47.0 email / phone / SSN / credit-card regex baseline). Both helpers fall through silently when the optional package is missing.
|
||||
- **PostHog env override.** `SOUP_POSTHOG_KEY` + `SOUP_POSTHOG_ENDPOINT` env vars let operators point opt-in telemetry at their own PostHog project without a code change. Endpoint goes through the v0.51.0 SSRF policy (HTTPS-only, loopback HTTP only, RFC1918 / link-local rejection). Keys reject null bytes, control characters, and oversize (>256 char) input. A new sentinel-based default cleanly distinguishes "caller omitted endpoint" from "caller passed the default URL" so a `endpoint=_POSTHOG_ENDPOINT` explicit pin is NOT silently overridden by env.
|
||||
- **`--hub` flag on `chat` / `serve` / `infer` / `merge` / `export` / `push`.** Closes the v0.53.8 known-limitation that only `train` and `data download` could route through non-HF hubs (ModelScope / Modelers). New `utils/hubs.prefetch_model_from_hub` is the shared, cwd-contained snapshot helper; every command body invokes the public `apply_hub_to_cli_model` adapter at the top. `push --hub` uses the `utils.hubs.upload_repo` upload-direction adapter; non-HF uploads skip HF-specific Collections + model-card auto-render.
|
||||
- **`soup data download --hub <non-hf>` live SDK.** Lifts the v0.53.8 advisory-only path — `--hub modelscope` now goes through `modelscope.msdatasets.MsDataset.load(...)`, `--hub modelers` uses `openmind_hub.snapshot_download(repo_type="dataset", ...)`. Friendly `pip install <sdk>` advisory fires when the SDK is missing.
|
||||
- **Web UI Tool Outputs panel.** New "Tool Outputs" tab in `soup ui` polls `/api/tool-outputs?limit=100` every 3 seconds and renders the most recent tool-call records (name / started / duration / OK / output preview) in an XSS-safe table built via `document.createElement` + `textContent` (no `innerHTML` for user-controlled fields). Bearer token threaded via the v0.53.9 `window._authToken` bootstrap.
|
||||
- **SFT callback `record_call` wiring.** `monitoring/callback.SoupTrainerCallback` now exposes an `on_step_end` hook that peeks at `kwargs.get("inputs", {})` for a `tool_calls` field and routes the count through the v0.53.9 global tool buffer so the Web UI panel populates from real training batches. Best-effort with blanket `except Exception: # noqa: BLE001` — training must never crash on observation failure.
|
||||
- **+45 net new tests** (8285 → 8330) in `test_v05310.py`. Four review agents ran (python / code / security / tdd / verification-loop); 13 findings fixed: H1 (PostHog explicit-endpoint precedence — sentinel default), H2 (absolute `local_path` leak — reduced to relpath), H3 (Rich markup escape on `base` / `local_path` / `cache_dir`), H4 (callback `# noqa: BLE001` per project policy), MED `import time` moved out of `try` block, oversize / explicit-empty `api_key` rejection tests, source-grep regression guards (every non-push command imports `apply_hub_to_cli_model`; `data.py` no longer carries the legacy `"wait for v0.53.9"` advisory), `prefetch_model_from_hub` outside-cwd `cache_root` rejection, empty-list + bool-True `tool_calls` no-op tests.
|
||||
|
||||
## Why Soup?
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "soup-cli"
|
||||
version = "0.53.9"
|
||||
version = "0.53.10"
|
||||
description = "Fine-tune LLMs in one command. No SSH, no config hell."
|
||||
readme = "README.md"
|
||||
license = "Apache-2.0"
|
||||
|
|
@ -65,6 +65,13 @@ tui = ["textual>=0.50.0"]
|
|||
trackers = ["mlflow>=2.0.0", "swanlab>=0.3.0", "trackio>=0.0.1"]
|
||||
# v0.53.8 #85 — fsspec backends for remote dataset loading (s3 / gs / az / oci).
|
||||
remote = ["fsspec>=2024.1.0", "s3fs>=2024.1.0", "gcsfs>=2024.1.0", "adlfs>=2024.1.0"]
|
||||
# v0.53.10 #150 — bundle scikit-optimize so `soup data mix --optimize` runs the
|
||||
# Bayesian-style loop instead of falling back to the v0.48.0 Dirichlet sampler.
|
||||
mix = ["scikit-optimize>=0.9.0"]
|
||||
# v0.53.10 #113 — production-grade data quality: langdetect (language) +
|
||||
# presidio-analyzer (PII). Llama-Guard-3-1B is documented as a manual recipe
|
||||
# (license + ~600 MB weight blob too large to bundle by default).
|
||||
data-pro = ["langdetect>=1.0.9", "presidio-analyzer>=2.2.0"]
|
||||
|
||||
[project.scripts]
|
||||
soup = "soup_cli.cli:run"
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
"""Soup CLI — Fine-tune LLMs in one command."""
|
||||
|
||||
__version__ = "0.53.9"
|
||||
__version__ = "0.53.10"
|
||||
|
|
|
|||
|
|
@ -53,8 +53,34 @@ def chat(
|
|||
"Default deny (v0.36.0). Only enable if you trust the source."
|
||||
),
|
||||
),
|
||||
hub: str = typer.Option(
|
||||
"hf",
|
||||
"--hub",
|
||||
help=(
|
||||
"Source hub for the base model: hf (default) / modelscope / "
|
||||
"modelers. Non-HF hubs require the matching SDK; the base model "
|
||||
"is snapshotted to a cwd-contained cache before chat starts "
|
||||
"(v0.53.10 #152)."
|
||||
),
|
||||
),
|
||||
):
|
||||
"""Chat with a fine-tuned model in the terminal."""
|
||||
# v0.53.10 #152 — pre-fetch base from a non-HF hub before any path
|
||||
# resolution. Local paths and HF repo IDs are passed through unchanged.
|
||||
if hub and hub != "hf":
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
try:
|
||||
model, base_model = apply_hub_to_cli_model(
|
||||
model, base_model, hub, console=console
|
||||
)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
except ImportError as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=1) from exc
|
||||
|
||||
model_path = Path(model)
|
||||
|
||||
if not model_path.exists():
|
||||
|
|
|
|||
|
|
@ -1210,13 +1210,53 @@ def download_dataset(
|
|||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
# v0.53.10 #153 — non-HF hub live dataset download. ModelScope uses
|
||||
# the ``MsDataset`` API (different shape from snapshot_download), so
|
||||
# we dispatch here instead of going through utils.hubs.download_repo.
|
||||
if hub_canonical != "hf":
|
||||
console.print(
|
||||
f"[red]--hub {hub_canonical} dataset download is not yet wired; "
|
||||
f"use `from soup_cli.utils.hubs import download_repo` to snapshot "
|
||||
f"a repo, or wait for v0.53.9.[/]"
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
from soup_cli.utils.hubs import download_repo as _download_repo
|
||||
|
||||
try:
|
||||
if hub_canonical == "modelscope":
|
||||
try:
|
||||
from modelscope.msdatasets import (
|
||||
MsDataset, # type: ignore[import-not-found]
|
||||
)
|
||||
except ImportError as exc:
|
||||
console.print(
|
||||
"[red]modelscope is not installed. "
|
||||
"Install with: pip install modelscope[/]"
|
||||
)
|
||||
raise typer.Exit(1) from exc
|
||||
_ms_ds = MsDataset.load( # noqa: F841 — touched for side effect
|
||||
dataset_id, split=split
|
||||
)
|
||||
console.print(
|
||||
f"[dim]ModelScope dataset {dataset_id} loaded; "
|
||||
"use soup_cli.utils.hubs.download_repo for raw "
|
||||
"snapshot download.[/]"
|
||||
)
|
||||
else: # modelers
|
||||
try:
|
||||
out_dir = _download_repo(
|
||||
hub_canonical,
|
||||
dataset_id,
|
||||
local_dir=str(Path.cwd() / ".soup_hub_cache"
|
||||
/ "datasets"
|
||||
/ dataset_id.replace("/", "__")),
|
||||
repo_type="dataset",
|
||||
)
|
||||
except ImportError as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(1) from exc
|
||||
console.print(
|
||||
f"[green]Downloaded {dataset_id} from {hub_canonical} → "
|
||||
f"{out_dir}[/]"
|
||||
)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(2) from exc
|
||||
return
|
||||
max_download_samples = 1_000_000
|
||||
if samples is not None and samples > max_download_samples:
|
||||
console.print(
|
||||
|
|
|
|||
|
|
@ -131,8 +131,30 @@ def export(
|
|||
"Required when --format=gguf-ud (v0.53.1 #139)."
|
||||
),
|
||||
),
|
||||
hub: str = typer.Option(
|
||||
"hf",
|
||||
"--hub",
|
||||
help=(
|
||||
"Source hub for the base model when --model is a LoRA adapter: "
|
||||
"hf (default) / modelscope / modelers (v0.53.10 #152)."
|
||||
),
|
||||
),
|
||||
):
|
||||
"""Export a model to GGUF, ONNX, TensorRT-LLM, AWQ, GPTQ, or TorchAO format."""
|
||||
# v0.53.10 #152 — pre-fetch the base model from a non-HF hub. ``model``
|
||||
# is typically a local merged dir / adapter dir; only ``base`` is rewritten.
|
||||
if hub and hub != "hf":
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
try:
|
||||
_, base = apply_hub_to_cli_model(model, base, hub, console=console)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
except ImportError as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=1) from exc
|
||||
|
||||
model_path = Path(model)
|
||||
|
||||
# --- Validate ---
|
||||
|
|
|
|||
|
|
@ -102,8 +102,29 @@ def infer(
|
|||
"Default deny (v0.36.0). Only enable if you trust the source."
|
||||
),
|
||||
),
|
||||
hub: str = typer.Option(
|
||||
"hf",
|
||||
"--hub",
|
||||
help=(
|
||||
"Source hub for the base model: hf (default) / modelscope / "
|
||||
"modelers. Non-HF hubs require the matching SDK (v0.53.10 #152)."
|
||||
),
|
||||
),
|
||||
):
|
||||
"""Run batch inference on a JSONL file of prompts."""
|
||||
# v0.53.10 #152 — pre-fetch base from a non-HF hub before any resolution.
|
||||
if hub and hub != "hf":
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
try:
|
||||
model, base = apply_hub_to_cli_model(model, base, hub, console=console)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
except ImportError as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=1) from exc
|
||||
|
||||
from soup_cli.utils.paths import is_under_cwd
|
||||
|
||||
# Validate input file
|
||||
|
|
|
|||
|
|
@ -53,8 +53,30 @@ def merge(
|
|||
"cycle (v0.53.1 #142)."
|
||||
),
|
||||
),
|
||||
hub: str = typer.Option(
|
||||
"hf",
|
||||
"--hub",
|
||||
help=(
|
||||
"Source hub for the base model: hf (default) / modelscope / "
|
||||
"modelers. Non-HF hubs require the matching SDK (v0.53.10 #152)."
|
||||
),
|
||||
),
|
||||
):
|
||||
"""Merge a LoRA adapter with its base model into a full model."""
|
||||
# v0.53.10 #152 — pre-fetch the base model from a non-HF hub. The local
|
||||
# adapter dir is left untouched; only the base repo id is rewritten.
|
||||
if hub and hub != "hf":
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
try:
|
||||
_, base = apply_hub_to_cli_model(adapter, base, hub, console=console)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
except ImportError as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=1) from exc
|
||||
|
||||
# v0.53.1 #142 — validate save_format up front
|
||||
from soup_cli.utils.save_formats import validate_merge_save_format
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -57,8 +57,28 @@ def push(
|
|||
"(slug: 'owner/title-hash')"
|
||||
),
|
||||
),
|
||||
hub: str = typer.Option(
|
||||
"hf",
|
||||
"--hub",
|
||||
help=(
|
||||
"Destination hub: hf (default) / modelscope / modelers. Non-HF "
|
||||
"hubs require the matching SDK and skip the HF-specific "
|
||||
"Collections / model-card auto-render path (v0.53.10 #152)."
|
||||
),
|
||||
),
|
||||
):
|
||||
"""Push a trained model to HuggingFace Hub."""
|
||||
"""Push a trained model to HuggingFace Hub (or alternate hub)."""
|
||||
# v0.53.10 #152 — validate hub at the CLI boundary; only HF is the
|
||||
# default. Non-HF hubs upload via :func:`utils.hubs.upload_repo` after
|
||||
# the standard model-dir validation completes.
|
||||
from soup_cli.utils.hubs import validate_hub_name
|
||||
|
||||
try:
|
||||
hub_canonical = validate_hub_name(hub)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
|
||||
from soup_cli.utils.paths import is_under_cwd
|
||||
|
||||
model_path = Path(model)
|
||||
|
|
@ -143,6 +163,35 @@ def push(
|
|||
)
|
||||
|
||||
# --- Upload ---
|
||||
# v0.53.10 #152 — non-HF hubs route through utils.hubs.upload_repo
|
||||
# before we reach the HF-specific Collections / model-card auto-render
|
||||
# path. Each backend lazy-imports its own SDK; missing-dep surfaces
|
||||
# as ImportError with a pip-install advisory.
|
||||
if hub_canonical != "hf":
|
||||
from soup_cli.utils.hubs import upload_repo
|
||||
|
||||
console.print(f"[dim]Uploading to hub={hub_canonical}...[/]")
|
||||
try:
|
||||
upload_repo(
|
||||
hub_canonical,
|
||||
repo,
|
||||
folder_path=str(model_path),
|
||||
commit_message=commit_message,
|
||||
token=hf_token,
|
||||
)
|
||||
except ImportError as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(1) from exc
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(2) from exc
|
||||
console.print(
|
||||
f"[green]Pushed to {hub_canonical}/{repo}.[/]\n"
|
||||
"[dim]Note: HF-specific Collections + model card auto-render "
|
||||
"are HF-only; install via the HF flow for those features.[/]"
|
||||
)
|
||||
return
|
||||
|
||||
console.print("[dim]Uploading to HuggingFace Hub...[/]")
|
||||
|
||||
from soup_cli.utils.hf import get_hf_api
|
||||
|
|
|
|||
|
|
@ -199,8 +199,31 @@ def serve(
|
|||
"deepseek-r1 | qwen3 | phi4 | openthinker. v0.53.9 #98."
|
||||
),
|
||||
),
|
||||
hub: str = typer.Option(
|
||||
"hf",
|
||||
"--hub",
|
||||
help=(
|
||||
"Source hub for the base model: hf (default) / modelscope / "
|
||||
"modelers. Non-HF hubs require the matching SDK (v0.53.10 #152)."
|
||||
),
|
||||
),
|
||||
):
|
||||
"""Start a local inference server with OpenAI-compatible API."""
|
||||
# v0.53.10 #152 — pre-fetch base from a non-HF hub before serve starts.
|
||||
if hub and hub != "hf":
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
try:
|
||||
model, base_model = apply_hub_to_cli_model(
|
||||
model, base_model, hub, console=console
|
||||
)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
except ImportError as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=1) from exc
|
||||
|
||||
# Lazy imports for fast CLI startup
|
||||
try:
|
||||
import uvicorn # noqa: F401
|
||||
|
|
|
|||
|
|
@ -92,6 +92,54 @@ class SoupTrainerCallback(TrainerCallback):
|
|||
):
|
||||
self.display.start(total_steps=state.max_steps)
|
||||
|
||||
def on_step_end(
|
||||
self, args: TrainingArguments, state: TrainerState,
|
||||
control: TrainerControl, **kwargs,
|
||||
):
|
||||
"""v0.53.10 #156 — peek at the active batch for tool-calling rows.
|
||||
|
||||
HF Trainer does not pass ``inputs`` to ``on_step_end`` by default, so
|
||||
this hook can only see a batch when the trainer subclass explicitly
|
||||
threads ``inputs=`` via ``kwargs``. When present, route any
|
||||
``tool_calls`` entries through the global tool-output buffer so the
|
||||
Web UI's Tool Outputs panel populates in real time.
|
||||
|
||||
Best-effort: any exception inside the probe MUST NEVER take down
|
||||
training. Lazy buffer access — zero overhead when ``tool_calls``
|
||||
is absent from the batch.
|
||||
"""
|
||||
inputs = kwargs.get("inputs")
|
||||
if not isinstance(inputs, dict):
|
||||
return
|
||||
tool_calls = inputs.get("tool_calls")
|
||||
if not tool_calls:
|
||||
return
|
||||
# stdlib import outside the try-block (python-review MEDIUM fix:
|
||||
# ``time`` is impossible to ImportError, so don't pretend it can).
|
||||
import time
|
||||
|
||||
try:
|
||||
from soup_cli.utils.tool_outputs import get_global_tool_buffer
|
||||
|
||||
count = 0
|
||||
if isinstance(tool_calls, (list, tuple)):
|
||||
count = len(tool_calls)
|
||||
elif isinstance(tool_calls, (int, float)) and not isinstance(
|
||||
tool_calls, bool
|
||||
):
|
||||
count = int(tool_calls)
|
||||
if count <= 0:
|
||||
return
|
||||
get_global_tool_buffer().record_call(
|
||||
name="sft_batch",
|
||||
started_ts=time.time(),
|
||||
duration_ms=0.0,
|
||||
success=True,
|
||||
output_preview=f"step {state.global_step}: {count} tool_calls",
|
||||
)
|
||||
except Exception: # noqa: BLE001 — best-effort, must never crash training
|
||||
return
|
||||
|
||||
def on_log(
|
||||
self, args: TrainingArguments, state: TrainerState,
|
||||
control: TrainerControl, logs=None, **kwargs,
|
||||
|
|
|
|||
|
|
@ -93,6 +93,92 @@ function navigate(page) {
|
|||
else if (page === 'training') loadTrainingPage();
|
||||
else if (page === 'data') { /* loaded on demand */ }
|
||||
else if (page === 'chat') loadChatPage();
|
||||
else if (page === 'tools') loadToolOutputs();
|
||||
// v0.53.10 #155 — pause Tool Outputs polling when navigating away so we
|
||||
// don't keep firing fetch() against /api/tool-outputs from background tabs.
|
||||
if (page !== 'tools') stopToolOutputsPolling();
|
||||
}
|
||||
|
||||
// --- Tool Outputs panel (v0.53.10 #155) ---
|
||||
// Polls /api/tool-outputs every 3 s while the page is active. XSS-safe via
|
||||
// textContent / .appendChild (no innerHTML for user-controlled fields).
|
||||
let _toolsPollHandle = null;
|
||||
|
||||
function loadToolOutputs() {
|
||||
renderToolOutputs();
|
||||
if (_toolsPollHandle === null) {
|
||||
_toolsPollHandle = setInterval(renderToolOutputs, 3000);
|
||||
}
|
||||
}
|
||||
|
||||
function stopToolOutputsPolling() {
|
||||
if (_toolsPollHandle !== null) {
|
||||
clearInterval(_toolsPollHandle);
|
||||
_toolsPollHandle = null;
|
||||
}
|
||||
}
|
||||
|
||||
async function renderToolOutputs() {
|
||||
const container = document.getElementById('tools-content');
|
||||
if (!container) return;
|
||||
let payload;
|
||||
try {
|
||||
const headers = {};
|
||||
if (window._authToken) {
|
||||
headers['Authorization'] = 'Bearer ' + window._authToken;
|
||||
}
|
||||
const resp = await fetch('/api/tool-outputs?limit=100', { headers });
|
||||
if (!resp.ok) throw new Error('HTTP ' + resp.status);
|
||||
payload = await resp.json();
|
||||
} catch (err) {
|
||||
container.textContent = 'Failed to load tool outputs: ' + err.message;
|
||||
return;
|
||||
}
|
||||
const records = (payload && Array.isArray(payload.records)) ? payload.records : [];
|
||||
// Build the table via DOM APIs so user-controlled fields stay XSS-safe.
|
||||
container.replaceChildren();
|
||||
if (records.length === 0) {
|
||||
const empty = document.createElement('div');
|
||||
empty.className = 'empty-state';
|
||||
const t = document.createElement('div');
|
||||
t.className = 'empty-state-text';
|
||||
t.textContent = 'No tool calls observed yet.';
|
||||
empty.appendChild(t);
|
||||
container.appendChild(empty);
|
||||
return;
|
||||
}
|
||||
const wrap = document.createElement('div');
|
||||
wrap.className = 'table-wrap';
|
||||
const table = document.createElement('table');
|
||||
const thead = document.createElement('thead');
|
||||
const head = document.createElement('tr');
|
||||
['Name', 'Started', 'Duration (ms)', 'OK', 'Output'].forEach(label => {
|
||||
const th = document.createElement('th');
|
||||
th.textContent = label;
|
||||
head.appendChild(th);
|
||||
});
|
||||
thead.appendChild(head);
|
||||
table.appendChild(thead);
|
||||
const tbody = document.createElement('tbody');
|
||||
records.forEach(rec => {
|
||||
const tr = document.createElement('tr');
|
||||
const cells = [
|
||||
String(rec.name || ''),
|
||||
rec.started_ts ? new Date(rec.started_ts * 1000).toLocaleTimeString() : '-',
|
||||
(typeof rec.duration_ms === 'number') ? rec.duration_ms.toFixed(1) : '-',
|
||||
rec.success ? '✓' : '✗',
|
||||
String(rec.output_preview || rec.error || ''),
|
||||
];
|
||||
cells.forEach(value => {
|
||||
const td = document.createElement('td');
|
||||
td.textContent = value;
|
||||
tr.appendChild(td);
|
||||
});
|
||||
tbody.appendChild(tr);
|
||||
});
|
||||
table.appendChild(tbody);
|
||||
wrap.appendChild(table);
|
||||
container.appendChild(wrap);
|
||||
}
|
||||
|
||||
// --- API Helpers ---
|
||||
|
|
|
|||
|
|
@ -29,6 +29,10 @@
|
|||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg>
|
||||
<span>Model Chat</span>
|
||||
</button>
|
||||
<button class="nav-item" data-page="tools" onclick="navigate('tools')">
|
||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg>
|
||||
<span>Tool Outputs</span>
|
||||
</button>
|
||||
</nav>
|
||||
<div class="sidebar-footer" id="sidebar-version">Soup Web UI</div>
|
||||
</aside>
|
||||
|
|
@ -165,6 +169,26 @@
|
|||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Tool Outputs Page -->
|
||||
<!-- v0.53.10 #155 — polls /api/tool-outputs every 3s. XSS-safe (uses
|
||||
textContent, not innerHTML). Bearer token threaded via the
|
||||
existing window._authToken bootstrap (v0.53.9 #95). -->
|
||||
<div id="page-tools" class="page">
|
||||
<div class="page-header">
|
||||
<h1 class="page-title">Tool Outputs</h1>
|
||||
<button class="btn" onclick="loadToolOutputs()">Refresh</button>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="card-title">Recent tool calls</div>
|
||||
<div id="tools-content">
|
||||
<div class="empty-state">
|
||||
<div class="empty-state-text">Waiting for tool calls...</div>
|
||||
<div class="empty-state-hint">POST to /v1/tools/python or run a tool-calling SFT to populate</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</main>
|
||||
</div>
|
||||
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ Security:
|
|||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util as _importlib_util
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
|
|
@ -417,6 +418,12 @@ def _build_default_optimizer(
|
|||
) -> OptimizerProtocol:
|
||||
"""Return :func:`_build_skopt_optimizer` when ``scikit-optimize`` is
|
||||
installed; otherwise fall back to a deterministic Dirichlet-like sampler.
|
||||
|
||||
v0.53.10 #150 — when scikit-optimize is available via the new ``[mix]``
|
||||
pyproject extra, callers get true Bayesian optimisation; otherwise the
|
||||
Dirichlet sampler is used silently (no spam advisory at import time, but
|
||||
the caller of :func:`run_mix_optimizer` can inspect the returned optimizer
|
||||
via :func:`describe_default_optimizer` to surface the chosen backend).
|
||||
"""
|
||||
try:
|
||||
return _build_skopt_optimizer(num_datasets, seed)
|
||||
|
|
@ -439,6 +446,20 @@ def _build_default_optimizer(
|
|||
return _Dirichlet()
|
||||
|
||||
|
||||
def describe_default_optimizer() -> str:
|
||||
"""Return a short label naming the optimizer backend that
|
||||
:func:`_build_default_optimizer` would pick for the current process.
|
||||
|
||||
v0.53.10 #150 — used by ``soup data mix --optimize`` to print an advisory
|
||||
so users can see whether the v0.48.0 Dirichlet fallback or the bundled
|
||||
scikit-optimize Bayesian loop is active. ``importlib.util.find_spec`` is
|
||||
a non-executing probe so skopt's import cost is not paid here.
|
||||
"""
|
||||
if _importlib_util.find_spec("skopt") is not None:
|
||||
return "scikit-optimize"
|
||||
return "dirichlet-fallback"
|
||||
|
||||
|
||||
def _renormalize(weights: Sequence[float]) -> Tuple[float, ...]:
|
||||
"""Clip + renormalise to a valid simplex point."""
|
||||
clipped = [max(0.0, float(w)) for w in weights]
|
||||
|
|
|
|||
|
|
@ -230,15 +230,64 @@ _PII_PATTERNS: Tuple[Tuple[str, "re.Pattern[str]"], ...] = (
|
|||
)
|
||||
|
||||
|
||||
def _presidio_pii(text: str) -> List[Dict[str, str]] | None:
|
||||
"""Run Presidio AnalyzerEngine when available (v0.53.10 #113 / ``[data-pro]``).
|
||||
|
||||
Returns ``None`` when the optional ``presidio-analyzer`` package is not
|
||||
installed OR when any error fires during analysis. Caller falls back to
|
||||
the regex baseline. We DO NOT raise — PII detection is best-effort and
|
||||
should never crash the broader scoring pipeline.
|
||||
|
||||
The 32-hit cap + 64-char snippet truncation mirror the regex path so
|
||||
downstream consumers get a consistent shape regardless of backend.
|
||||
"""
|
||||
try:
|
||||
from presidio_analyzer import AnalyzerEngine # noqa: PLC0415
|
||||
except ImportError:
|
||||
return None
|
||||
try:
|
||||
analyzer = AnalyzerEngine()
|
||||
results = analyzer.analyze(text=text, language="en")
|
||||
except Exception: # noqa: BLE001 — fall through to regex baseline
|
||||
return None
|
||||
if not isinstance(results, list):
|
||||
return None
|
||||
hits: List[Dict[str, str]] = []
|
||||
for res in results:
|
||||
kind = getattr(res, "entity_type", None)
|
||||
start = getattr(res, "start", None)
|
||||
end = getattr(res, "end", None)
|
||||
if not isinstance(kind, str) or not isinstance(start, int):
|
||||
continue
|
||||
if not isinstance(end, int) or end <= start:
|
||||
continue
|
||||
snippet = text[start:end]
|
||||
if len(snippet) > 64:
|
||||
snippet = snippet[:61] + "..."
|
||||
hits.append({"kind": kind.lower(), "snippet": snippet})
|
||||
if len(hits) >= 32:
|
||||
break
|
||||
return hits
|
||||
|
||||
|
||||
def detect_pii(text: Any) -> List[Dict[str, str]]:
|
||||
"""Return a list of ``{kind, snippet}`` PII hits.
|
||||
|
||||
Scans only the first ``_PII_SCAN_CAP`` chars of ``text`` to keep regex
|
||||
finditer cost bounded regardless of caller input size.
|
||||
|
||||
v0.53.10 #113 — when ``presidio-analyzer`` is installed via the
|
||||
``[data-pro]`` extras, routes through Presidio for broader entity
|
||||
coverage (location / dates / IBAN / etc.); otherwise falls back to the
|
||||
in-tree 4-regex baseline (email / phone / SSN / credit-card).
|
||||
"""
|
||||
s = _require_str(text, name="text")
|
||||
if len(s) > _PII_SCAN_CAP:
|
||||
s = s[:_PII_SCAN_CAP]
|
||||
# Try Presidio first; silently falls through when absent.
|
||||
presidio_hits = _presidio_pii(s)
|
||||
if presidio_hits is not None:
|
||||
return presidio_hits
|
||||
hits: List[Dict[str, str]] = []
|
||||
for kind, pat in _PII_PATTERNS:
|
||||
for m in pat.finditer(s):
|
||||
|
|
@ -291,18 +340,53 @@ _LANG_STOPWORDS: Mapping[str, frozenset] = MappingProxyType(
|
|||
)
|
||||
|
||||
|
||||
def _langdetect_fast(text: str) -> str | None:
|
||||
"""Probabilistic detection via ``langdetect`` (v0.53.10 #113 / ``[data-pro]``).
|
||||
|
||||
Returns ``None`` when the optional ``langdetect`` package is not
|
||||
installed OR when the detector raises (e.g. ``LangDetectException`` on
|
||||
too-short input). Caller falls back to the stopword heuristic.
|
||||
|
||||
We rebind langdetect's global RNG to a constant seed so two consecutive
|
||||
calls on the same input produce the same code (langdetect is otherwise
|
||||
non-deterministic). The seed is reset at every call to keep the heuristic
|
||||
deterministic across the test suite.
|
||||
"""
|
||||
try:
|
||||
import langdetect # noqa: PLC0415 — optional dep
|
||||
except ImportError:
|
||||
return None
|
||||
try:
|
||||
# ``DetectorFactory.seed = 0`` is the upstream-documented way to make
|
||||
# langdetect deterministic; cheap to re-apply.
|
||||
langdetect.DetectorFactory.seed = 0
|
||||
code = langdetect.detect(text)
|
||||
except Exception: # noqa: BLE001 — fall through to heuristic on any error
|
||||
return None
|
||||
if not isinstance(code, str) or len(code) < 2:
|
||||
return None
|
||||
# langdetect returns ISO 639-1 codes (already lowercased). Truncate to
|
||||
# the 2-letter prefix to match the heuristic's surface.
|
||||
return code[:2].lower()
|
||||
|
||||
|
||||
def detect_language(text: Any) -> str:
|
||||
"""Return a 2-letter ISO code or ``"unknown"``.
|
||||
|
||||
Pure-Python stopword heuristic; conservative — falls through to
|
||||
``"unknown"`` on short or ambiguous input. For production-grade
|
||||
detection, install ``langdetect`` and pipe via the ``[data-pro]``
|
||||
extras (deferred to v0.47.1).
|
||||
v0.53.10 #113 — when the optional ``langdetect`` package is installed
|
||||
via the ``[data-pro]`` extras, routes through its probabilistic
|
||||
detector for broader language coverage; otherwise falls back to the
|
||||
pure-Python stopword heuristic (covers en/es/fr/de/pt/ru).
|
||||
"""
|
||||
s = _require_str(text, name="text")
|
||||
tokens = _tokenise(s)
|
||||
if len(tokens) < 4:
|
||||
return "unknown"
|
||||
# Try langdetect first; falls through silently when the package is
|
||||
# missing or raises (e.g. too-short input).
|
||||
fast = _langdetect_fast(s)
|
||||
if fast is not None:
|
||||
return fast
|
||||
token_set = set(tokens)
|
||||
best_lang = "unknown"
|
||||
best_hits = 0
|
||||
|
|
|
|||
|
|
@ -444,3 +444,158 @@ def upload_repo(
|
|||
return
|
||||
|
||||
raise ValueError(f"hub {canonical!r} has no upload adapter")
|
||||
|
||||
|
||||
def prefetch_model_from_hub(
|
||||
base: str,
|
||||
hub: str,
|
||||
*,
|
||||
cache_root: str | None = None,
|
||||
console: object | None = None,
|
||||
) -> str:
|
||||
"""Snapshot ``base`` from ``hub`` into a cwd-contained cache + return path.
|
||||
|
||||
v0.53.10 #152 — shared helper extracted from the v0.53.8 ``soup train``
|
||||
pre-fetch path so chat / serve / infer / merge / export / push can route
|
||||
non-HF hubs through the same SSRF-hardened, cwd-contained snapshot flow
|
||||
without each command re-implementing the slug + containment + cache
|
||||
short-circuit logic.
|
||||
|
||||
Args:
|
||||
base: model id (e.g. ``"meta-llama/Llama-3.1-8B"``) — accepted as-is
|
||||
and forwarded to the hub-specific :func:`download_repo` adapter.
|
||||
hub: hub name (validated via :func:`validate_hub_name`); ``"hf"``
|
||||
short-circuits with no download (returns ``base`` unchanged).
|
||||
cache_root: optional override for the cache root directory. Defaults
|
||||
to ``<cwd>/.soup_hub_cache``. The resolved cache subdir is
|
||||
cwd-containment checked even when an override is supplied so a
|
||||
crafted ``cache_root`` cannot escape the working directory.
|
||||
console: optional Rich console for cache-hit / fetch advisories. When
|
||||
``None``, the function is silent (returns the path without
|
||||
printing). Caller is responsible for printing failures.
|
||||
|
||||
Returns:
|
||||
Absolute local path to the downloaded snapshot. For ``hub='hf'``
|
||||
returns ``base`` unchanged (HF Hub is the trainer default).
|
||||
|
||||
Raises:
|
||||
TypeError / ValueError: invalid ``hub`` / ``base`` per
|
||||
:func:`validate_hub_name` / :func:`_validate_repo_id_shape`.
|
||||
ImportError: optional SDK (modelscope / openmind-hub) missing.
|
||||
ValueError: resolved cache dir escapes cwd.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
|
||||
from soup_cli.utils.paths import is_under_cwd
|
||||
|
||||
canonical = validate_hub_name(hub)
|
||||
if canonical == "hf":
|
||||
return base
|
||||
if not isinstance(base, str) or not base:
|
||||
raise ValueError("base must be a non-empty string")
|
||||
if "\x00" in base or any(ord(c) < 0x20 for c in base):
|
||||
raise ValueError("base must not contain control characters")
|
||||
# Mirror v0.53.8 ``soup train`` cache-dir slug policy: strip every
|
||||
# path-separator and ``..`` segment so a crafted ``base: ../../etc``
|
||||
# cannot escape the cache root (Windows ``\\`` + POSIX ``/`` both
|
||||
# blocked).
|
||||
safe_slug = re.sub(r"[^A-Za-z0-9._-]+", "__", base).strip("._-") or "model"
|
||||
if cache_root is None:
|
||||
root_path = os.path.realpath(os.path.join(os.getcwd(), ".soup_hub_cache"))
|
||||
else:
|
||||
if not isinstance(cache_root, str) or not cache_root:
|
||||
raise ValueError("cache_root must be a non-empty string")
|
||||
root_path = os.path.realpath(cache_root)
|
||||
cache_dir = os.path.realpath(os.path.join(root_path, safe_slug))
|
||||
if not is_under_cwd(cache_dir):
|
||||
raise ValueError(
|
||||
"resolved hub cache dir escapes the current working directory"
|
||||
)
|
||||
# v0.53.10 security-review HIGH: escape Rich markup on every
|
||||
# user-controlled string before embedding in console.print. A crafted
|
||||
# ``base`` like ``[bold red]evil[/bold red]`` must NOT render styled.
|
||||
from rich.markup import escape as _markup_escape
|
||||
|
||||
existing_cfg = os.path.join(cache_dir, "config.json")
|
||||
if os.path.isfile(existing_cfg):
|
||||
if console is not None:
|
||||
try:
|
||||
try:
|
||||
display_dir = os.path.relpath(cache_dir)
|
||||
except (ValueError, OSError):
|
||||
display_dir = os.path.basename(cache_dir)
|
||||
console.print( # type: ignore[attr-defined]
|
||||
f"[dim]Using cached snapshot at "
|
||||
f"{_markup_escape(display_dir)}[/]"
|
||||
)
|
||||
except Exception: # noqa: BLE001 — advisory is best-effort
|
||||
pass
|
||||
return cache_dir
|
||||
local_path = download_repo(canonical, base, local_dir=cache_dir)
|
||||
if console is not None:
|
||||
try:
|
||||
# Reduce SDK-returned absolute path to a cwd-relative form so
|
||||
# we don't leak $HOME into the terminal output (code-review
|
||||
# HIGH fix; matches v0.34.0 crash.py redaction policy).
|
||||
display_path = local_path
|
||||
try:
|
||||
display_path = os.path.relpath(local_path)
|
||||
except (ValueError, OSError):
|
||||
display_path = os.path.basename(local_path)
|
||||
console.print( # type: ignore[attr-defined]
|
||||
f"[dim]Fetched {_markup_escape(base)} "
|
||||
f"from hub={_markup_escape(canonical)} → "
|
||||
f"{_markup_escape(str(display_path))}[/]"
|
||||
)
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
return local_path
|
||||
|
||||
|
||||
def apply_hub_to_cli_model(
|
||||
model: str | None,
|
||||
base_model: str | None,
|
||||
hub: str,
|
||||
*,
|
||||
console: object | None = None,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Resolve ``(model, base_model)`` after an optional non-HF hub prefetch.
|
||||
|
||||
v0.53.10 #152 — shared CLI helper for chat / serve / infer / merge /
|
||||
export / push so each command emits the same advisory + uses the same
|
||||
cwd-contained cache.
|
||||
|
||||
Behaviour:
|
||||
* ``hub`` is ``"hf"`` (or empty / None) → returns inputs unchanged.
|
||||
* ``base_model`` is set + non-existent local path → snapshot via
|
||||
:func:`prefetch_model_from_hub` and route the result to
|
||||
``base_model``.
|
||||
* else ``model`` is set + non-existent local path → snapshot via
|
||||
:func:`prefetch_model_from_hub` and route the result to ``model``.
|
||||
* Existing local paths (e.g. a freshly trained LoRA dir) are passed
|
||||
through unchanged so a non-HF hub flag does not break the
|
||||
common "fine-tune locally, then chat" loop.
|
||||
|
||||
Returns:
|
||||
``(model_out, base_model_out)`` tuple — at most one of them is
|
||||
rewritten to the local snapshot path.
|
||||
|
||||
Raises:
|
||||
TypeError / ValueError / ImportError: propagated from
|
||||
:func:`prefetch_model_from_hub` so the CLI can map them to exit codes.
|
||||
"""
|
||||
import os
|
||||
|
||||
if not hub or hub == "hf":
|
||||
return model, base_model
|
||||
# Prefer rewriting ``base_model`` when set (the typical LoRA-adapter
|
||||
# case where ``model`` is a local directory and ``base_model`` is a
|
||||
# remote repo id).
|
||||
if base_model and not os.path.exists(base_model):
|
||||
fetched = prefetch_model_from_hub(base_model, hub, console=console)
|
||||
return model, fetched
|
||||
if model and not os.path.exists(model):
|
||||
fetched = prefetch_model_from_hub(model, hub, console=console)
|
||||
return fetched, base_model
|
||||
return model, base_model
|
||||
|
|
|
|||
|
|
@ -163,11 +163,70 @@ def build_telemetry_payload(
|
|||
|
||||
_POSTHOG_HOST = "https://us.i.posthog.com"
|
||||
_POSTHOG_ENDPOINT = f"{_POSTHOG_HOST}/i/v0/e/"
|
||||
# Public write-only key. Live deployments will swap this via env var.
|
||||
# v0.53.10 #154 — bundled public write-only project key for Soup CLI
|
||||
# telemetry. The key is INTENTIONALLY hard-coded: PostHog "phc_*" keys are
|
||||
# write-only (cannot read events back); rotating it requires a release.
|
||||
# Operators wanting to point telemetry at their own PostHog project should
|
||||
# set ``SOUP_POSTHOG_KEY`` AND ``SOUP_POSTHOG_ENDPOINT`` together; both env
|
||||
# vars are validated by :func:`_resolve_posthog_target`.
|
||||
_POSTHOG_DEFAULT_KEY = "phc_soup_public_write_only"
|
||||
_TELEMETRY_TIMEOUT_S = 1.0
|
||||
|
||||
|
||||
# Sentinel for "caller did not pass an endpoint, fall back to default + env".
|
||||
_POSTHOG_ENDPOINT_DEFAULT = object()
|
||||
|
||||
|
||||
def _resolve_posthog_target(
|
||||
api_key: str | None,
|
||||
endpoint: object = _POSTHOG_ENDPOINT_DEFAULT,
|
||||
env: dict[str, str] | None = None,
|
||||
) -> tuple[str, str] | None:
|
||||
"""Resolve ``(key, endpoint)`` from explicit args + ``SOUP_POSTHOG_*`` env.
|
||||
|
||||
v0.53.10 #154 — adds env-var overrides for the bundled defaults so users
|
||||
on private PostHog instances can point Soup telemetry at their own
|
||||
project without a code change. Precedence:
|
||||
|
||||
1. Explicit ``api_key`` / ``endpoint`` kwargs (caller wins).
|
||||
2. ``SOUP_POSTHOG_KEY`` env var (overrides ``_POSTHOG_DEFAULT_KEY``).
|
||||
3. ``SOUP_POSTHOG_ENDPOINT`` env var (overrides
|
||||
``_POSTHOG_ENDPOINT``; must be HTTPS + pass the v0.51.0 SSRF policy).
|
||||
4. Bundled defaults.
|
||||
|
||||
Returns ``None`` when any input fails validation (silent no-op so
|
||||
telemetry can never crash training).
|
||||
"""
|
||||
import os # noqa: PLC0415 — local lazy import
|
||||
|
||||
src = env if env is not None else os.environ
|
||||
# Endpoint resolution: explicit caller > env override > default.
|
||||
# Use a sentinel default so a caller who passes
|
||||
# ``endpoint=_POSTHOG_ENDPOINT`` (locking in the default) is NOT silently
|
||||
# overridden by ``SOUP_POSTHOG_ENDPOINT`` (code-review HIGH fix).
|
||||
if endpoint is _POSTHOG_ENDPOINT_DEFAULT:
|
||||
env_endpoint = src.get("SOUP_POSTHOG_ENDPOINT")
|
||||
resolved_endpoint = env_endpoint or _POSTHOG_ENDPOINT
|
||||
else:
|
||||
resolved_endpoint = endpoint
|
||||
if not isinstance(resolved_endpoint, str):
|
||||
return None
|
||||
if not _telemetry_endpoint_is_safe(resolved_endpoint):
|
||||
return None
|
||||
# Key resolution: explicit caller > env override > default.
|
||||
if api_key is not None:
|
||||
key = api_key
|
||||
else:
|
||||
key = src.get("SOUP_POSTHOG_KEY") or _POSTHOG_DEFAULT_KEY
|
||||
if not isinstance(key, str) or not key:
|
||||
return None
|
||||
# Reject control chars / whitespace in the key — defends against an
|
||||
# operator dropping ``\nAuthorization:...`` into SOUP_POSTHOG_KEY.
|
||||
if "\x00" in key or any(ord(c) < 0x20 for c in key) or len(key) > 256:
|
||||
return None
|
||||
return key, resolved_endpoint
|
||||
|
||||
|
||||
def _telemetry_endpoint_is_safe(endpoint: str) -> bool:
|
||||
"""Re-validate the telemetry endpoint via the v0.51.0 SSRF policy.
|
||||
|
||||
|
|
@ -193,7 +252,7 @@ def send_telemetry_payload(
|
|||
*,
|
||||
api_key: str | None = None,
|
||||
timeout: float = _TELEMETRY_TIMEOUT_S,
|
||||
endpoint: str = _POSTHOG_ENDPOINT,
|
||||
endpoint: object = _POSTHOG_ENDPOINT_DEFAULT,
|
||||
) -> bool:
|
||||
"""POST ``payload`` to PostHog if telemetry is enabled, else no-op.
|
||||
|
||||
|
|
@ -211,18 +270,17 @@ def send_telemetry_payload(
|
|||
return False
|
||||
if not isinstance(payload, dict) or not payload:
|
||||
return False
|
||||
# HTTPS-only + private-IP / link-local rejection (mirrors v0.51.0 hub
|
||||
# endpoint SSRF policy). Defence-in-depth: any caller override goes
|
||||
# through the same validator that hub endpoints do.
|
||||
if not _telemetry_endpoint_is_safe(endpoint):
|
||||
return False
|
||||
if isinstance(timeout, bool) or not isinstance(timeout, (int, float)):
|
||||
return False
|
||||
if not math.isfinite(float(timeout)) or timeout <= 0:
|
||||
return False
|
||||
key = api_key or _POSTHOG_DEFAULT_KEY
|
||||
if not isinstance(key, str) or not key:
|
||||
# v0.53.10 #154 — resolve key + endpoint via env-override-aware helper.
|
||||
# Returns ``None`` when either input fails validation; treat as silent
|
||||
# no-op so telemetry remains best-effort.
|
||||
resolved = _resolve_posthog_target(api_key, endpoint)
|
||||
if resolved is None:
|
||||
return False
|
||||
key, endpoint = resolved
|
||||
try:
|
||||
import httpx # lazy — optional dep, surfaces no advisory
|
||||
except ImportError:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,510 @@
|
|||
"""v0.53.10 — Quick wins + packaging + UX wiring.
|
||||
|
||||
Covers seven closed issues:
|
||||
|
||||
* #150 ``[mix]`` extra + ``describe_default_optimizer`` advisory.
|
||||
* #113 ``[data-pro]`` extras + lazy ``langdetect`` / Presidio routing.
|
||||
* #154 ``SOUP_POSTHOG_KEY`` / ``SOUP_POSTHOG_ENDPOINT`` env override.
|
||||
* #152 Multi-command ``--hub`` dispatch (chat / serve / infer / merge /
|
||||
export / push).
|
||||
* #153 ``soup data download --hub <non-hf>`` live SDK lift.
|
||||
* #155 Web UI Tool Outputs panel JS / HTML wiring.
|
||||
* #156 SFT trainer-side ``record_call`` wire-up in
|
||||
``monitoring/callback.SoupTrainerCallback``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def _strip_ansi(text: str) -> str:
|
||||
out = re.sub(r"\x1b\[[0-9;]*m", "", text)
|
||||
return re.sub(r"\s+", " ", out)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #150 — `[mix]` extra + scikit-optimize advisory
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMixExtra:
|
||||
def test_pyproject_lists_mix_extra(self):
|
||||
body = (REPO_ROOT / "pyproject.toml").read_text(encoding="utf-8")
|
||||
# ``mix`` extra MUST be declared and bundle scikit-optimize.
|
||||
assert re.search(r"^mix\s*=\s*\[", body, re.MULTILINE), (
|
||||
"[mix] extra missing from pyproject"
|
||||
)
|
||||
mix_block = re.search(
|
||||
r"^mix\s*=\s*\[(.*?)\]", body, re.MULTILINE | re.DOTALL
|
||||
)
|
||||
assert mix_block is not None
|
||||
assert "scikit-optimize" in mix_block.group(1)
|
||||
|
||||
def test_describe_default_optimizer_returns_label(self):
|
||||
from soup_cli.utils.data_mix import describe_default_optimizer
|
||||
|
||||
label = describe_default_optimizer()
|
||||
assert label in ("scikit-optimize", "dirichlet-fallback")
|
||||
|
||||
def test_describe_dirichlet_fallback_when_skopt_missing(self):
|
||||
# Force find_spec to report skopt missing.
|
||||
with patch("importlib.util.find_spec", return_value=None):
|
||||
from soup_cli.utils.data_mix import describe_default_optimizer
|
||||
|
||||
assert describe_default_optimizer() == "dirichlet-fallback"
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #113 — `[data-pro]` extras + langdetect / Presidio routing
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDataProExtra:
|
||||
def test_pyproject_lists_data_pro(self):
|
||||
body = (REPO_ROOT / "pyproject.toml").read_text(encoding="utf-8")
|
||||
assert re.search(r"^data-pro\s*=\s*\[", body, re.MULTILINE), (
|
||||
"[data-pro] extra missing from pyproject"
|
||||
)
|
||||
block = re.search(
|
||||
r"^data-pro\s*=\s*\[(.*?)\]", body, re.MULTILINE | re.DOTALL
|
||||
)
|
||||
assert block is not None
|
||||
assert "langdetect" in block.group(1)
|
||||
assert "presidio-analyzer" in block.group(1)
|
||||
|
||||
def test_detect_language_falls_back_to_heuristic_without_langdetect(self):
|
||||
# Force the lazy import inside _langdetect_fast to fail by removing
|
||||
# any ``langdetect`` from sys.modules + blocking re-import.
|
||||
from soup_cli.utils import data_score
|
||||
|
||||
with patch.object(
|
||||
data_score, "_langdetect_fast", return_value=None
|
||||
):
|
||||
# Heuristic still picks 'en' on an English sentence.
|
||||
assert data_score.detect_language(
|
||||
"the quick brown fox jumps over the lazy dog"
|
||||
) == "en"
|
||||
|
||||
def test_detect_language_uses_langdetect_when_available(self):
|
||||
from soup_cli.utils import data_score
|
||||
|
||||
with patch.object(data_score, "_langdetect_fast", return_value="ja"):
|
||||
assert data_score.detect_language(
|
||||
"the quick brown fox jumps over the lazy dog"
|
||||
) == "ja"
|
||||
|
||||
def test_detect_pii_falls_back_to_regex_without_presidio(self):
|
||||
from soup_cli.utils import data_score
|
||||
|
||||
with patch.object(data_score, "_presidio_pii", return_value=None):
|
||||
hits = data_score.detect_pii(
|
||||
"Email me at user@example.com or 555-867-5309"
|
||||
)
|
||||
kinds = {h["kind"] for h in hits}
|
||||
assert "email" in kinds
|
||||
|
||||
def test_detect_pii_uses_presidio_when_available(self):
|
||||
from soup_cli.utils import data_score
|
||||
|
||||
presidio_hits = [{"kind": "email", "snippet": "user@example.com"}]
|
||||
with patch.object(
|
||||
data_score, "_presidio_pii", return_value=presidio_hits
|
||||
):
|
||||
assert data_score.detect_pii("anything") == presidio_hits
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #154 — PostHog env override
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPostHogEnvOverride:
|
||||
def test_resolve_uses_default_key_when_env_unset(self):
|
||||
from soup_cli.utils.trackers import _POSTHOG_DEFAULT_KEY, _resolve_posthog_target
|
||||
|
||||
# No env override; endpoint omitted to use the sentinel default.
|
||||
resolved = _resolve_posthog_target(None, env={})
|
||||
assert resolved is not None
|
||||
key, _ = resolved
|
||||
assert key == _POSTHOG_DEFAULT_KEY
|
||||
|
||||
def test_resolve_env_key_overrides_default(self):
|
||||
from soup_cli.utils.trackers import _resolve_posthog_target
|
||||
|
||||
env = {"SOUP_POSTHOG_KEY": "phc_user_project"}
|
||||
resolved = _resolve_posthog_target(None, env=env)
|
||||
assert resolved is not None
|
||||
key, _ = resolved
|
||||
assert key == "phc_user_project"
|
||||
|
||||
def test_resolve_explicit_arg_wins_over_env(self):
|
||||
from soup_cli.utils.trackers import _resolve_posthog_target
|
||||
|
||||
env = {"SOUP_POSTHOG_KEY": "phc_env"}
|
||||
resolved = _resolve_posthog_target("phc_caller", env=env)
|
||||
assert resolved is not None
|
||||
key, _ = resolved
|
||||
assert key == "phc_caller"
|
||||
|
||||
def test_resolve_env_endpoint_overrides_default(self):
|
||||
from soup_cli.utils.trackers import _resolve_posthog_target
|
||||
|
||||
env = {"SOUP_POSTHOG_ENDPOINT": "https://eu.i.posthog.com/i/v0/e/"}
|
||||
resolved = _resolve_posthog_target(None, env=env)
|
||||
assert resolved is not None
|
||||
_, endpoint = resolved
|
||||
assert endpoint == "https://eu.i.posthog.com/i/v0/e/"
|
||||
|
||||
def test_resolve_rejects_http_endpoint(self):
|
||||
from soup_cli.utils.trackers import _resolve_posthog_target
|
||||
|
||||
env = {"SOUP_POSTHOG_ENDPOINT": "http://attacker.example.com/"}
|
||||
assert _resolve_posthog_target(None, env=env) is None
|
||||
|
||||
def test_resolve_explicit_endpoint_locks_against_env(self):
|
||||
# code-review HIGH fix: a caller passing the default URL string
|
||||
# explicitly should NOT be silently overridden by the env var.
|
||||
from soup_cli.utils.trackers import _POSTHOG_ENDPOINT, _resolve_posthog_target
|
||||
|
||||
env = {"SOUP_POSTHOG_ENDPOINT": "https://eu.i.posthog.com/i/v0/e/"}
|
||||
resolved = _resolve_posthog_target(None, endpoint=_POSTHOG_ENDPOINT, env=env)
|
||||
assert resolved is not None
|
||||
_, endpoint = resolved
|
||||
assert endpoint == _POSTHOG_ENDPOINT
|
||||
|
||||
def test_resolve_rejects_control_char_key(self):
|
||||
from soup_cli.utils.trackers import _resolve_posthog_target
|
||||
|
||||
env = {"SOUP_POSTHOG_KEY": "phc\nAuthorization: bypass"}
|
||||
assert _resolve_posthog_target(None, env=env) is None
|
||||
|
||||
def test_resolve_rejects_null_byte_key(self):
|
||||
from soup_cli.utils.trackers import _resolve_posthog_target
|
||||
|
||||
env = {"SOUP_POSTHOG_KEY": "phc\x00bypass"}
|
||||
assert _resolve_posthog_target(None, env=env) is None
|
||||
|
||||
def test_resolve_rejects_oversize_key(self):
|
||||
# tdd-review MEDIUM #3: 257-char SOUP_POSTHOG_KEY must reject.
|
||||
from soup_cli.utils.trackers import _resolve_posthog_target
|
||||
|
||||
env = {"SOUP_POSTHOG_KEY": "x" * 257}
|
||||
assert _resolve_posthog_target(None, env=env) is None
|
||||
|
||||
def test_resolve_rejects_explicit_empty_key(self):
|
||||
# tdd-review MEDIUM #4: explicit empty-string `api_key` must reject.
|
||||
from soup_cli.utils.trackers import _resolve_posthog_target
|
||||
|
||||
assert _resolve_posthog_target("", env={}) is None
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #152 — Multi-command `--hub` dispatch
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestHubPrefetchHelper:
|
||||
def test_hf_short_circuits_no_download(self):
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
model_out, base_out = apply_hub_to_cli_model(
|
||||
"meta-llama/Llama-3.1-8B", None, "hf"
|
||||
)
|
||||
assert model_out == "meta-llama/Llama-3.1-8B"
|
||||
assert base_out is None
|
||||
|
||||
def test_none_hub_passes_through(self):
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
# None or empty hub is treated as no-op.
|
||||
assert apply_hub_to_cli_model("foo", None, "") == ("foo", None)
|
||||
|
||||
def test_existing_local_path_not_rewritten(self, tmp_path, monkeypatch):
|
||||
# A real local dir means the user already merged + saved; the hub
|
||||
# flag should not force a re-download.
|
||||
monkeypatch.chdir(tmp_path)
|
||||
local = tmp_path / "merged"
|
||||
local.mkdir()
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
model_out, base_out = apply_hub_to_cli_model(
|
||||
str(local), None, "modelscope"
|
||||
)
|
||||
assert model_out == str(local)
|
||||
|
||||
def test_non_existent_base_invokes_prefetch(self, tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
# patch prefetch to avoid network
|
||||
from soup_cli.utils import hubs
|
||||
|
||||
with patch.object(
|
||||
hubs, "prefetch_model_from_hub", return_value=str(tmp_path / "snap")
|
||||
) as p:
|
||||
model_out, base_out = hubs.apply_hub_to_cli_model(
|
||||
"./adapter", "some/repo", "modelscope"
|
||||
)
|
||||
p.assert_called_once()
|
||||
assert base_out == str(tmp_path / "snap")
|
||||
|
||||
def test_apply_hub_to_cli_model_both_none(self):
|
||||
# tdd-review LOW #7: neither model nor base set should no-op.
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
assert apply_hub_to_cli_model(None, None, "modelscope") == (None, None)
|
||||
|
||||
def test_prefetch_unknown_hub_raises(self):
|
||||
# Unknown hub name must propagate validation error.
|
||||
from soup_cli.utils.hubs import apply_hub_to_cli_model
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
apply_hub_to_cli_model(None, "some/repo", "evilhub")
|
||||
|
||||
def test_prefetch_outside_cwd_cache_root_raises(self, tmp_path, monkeypatch):
|
||||
# tdd-review MEDIUM #5: cache_root outside cwd must raise.
|
||||
monkeypatch.chdir(tmp_path)
|
||||
from soup_cli.utils.hubs import prefetch_model_from_hub
|
||||
|
||||
outside = tmp_path.parent / "outside_cache"
|
||||
with pytest.raises(ValueError, match="escapes"):
|
||||
prefetch_model_from_hub(
|
||||
"some/repo", "modelscope", cache_root=str(outside)
|
||||
)
|
||||
|
||||
|
||||
class TestCliHubFlags:
|
||||
@pytest.mark.parametrize(
|
||||
"cmd_module,cmd_name",
|
||||
[
|
||||
("soup_cli.commands.chat", "chat"),
|
||||
("soup_cli.commands.serve", "serve"),
|
||||
("soup_cli.commands.infer", "infer"),
|
||||
("soup_cli.commands.merge", "merge"),
|
||||
("soup_cli.commands.export", "export"),
|
||||
("soup_cli.commands.push", "push"),
|
||||
],
|
||||
)
|
||||
def test_hub_flag_present_in_signature(self, cmd_module, cmd_name):
|
||||
# Re-importing fresh because typer-decorated commands wrap signatures.
|
||||
module = __import__(cmd_module, fromlist=[cmd_name])
|
||||
fn = getattr(module, cmd_name)
|
||||
# Typer commands are wrapped — pull the raw function.
|
||||
target = getattr(fn, "__wrapped__", fn)
|
||||
import inspect
|
||||
|
||||
sig = inspect.signature(target)
|
||||
assert "hub" in sig.parameters, (
|
||||
f"{cmd_module}.{cmd_name} missing --hub keyword"
|
||||
)
|
||||
|
||||
def test_apply_hub_helper_imported_in_non_push_commands(self):
|
||||
# tdd-review HIGH #1: every command except push must use the
|
||||
# shared helper so a future refactor cannot silently drop it.
|
||||
for mod in ("chat", "serve", "infer", "merge", "export"):
|
||||
src = (REPO_ROOT / f"soup_cli/commands/{mod}.py").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
assert "apply_hub_to_cli_model" in src, (
|
||||
f"{mod}.py missing apply_hub_to_cli_model helper import"
|
||||
)
|
||||
|
||||
def test_push_uses_upload_repo_path(self):
|
||||
# push.py does NOT call apply_hub_to_cli_model (it's an upload
|
||||
# surface, not a download). It must call upload_repo + validate_hub_name.
|
||||
src = (REPO_ROOT / "soup_cli/commands/push.py").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
assert "upload_repo" in src
|
||||
assert "validate_hub_name" in src
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #153 — `soup data download --hub <non-hf>` live SDK
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDataDownloadNonHfLive:
|
||||
def test_modelers_unknown_sdk_friendly_error(self):
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from soup_cli.commands.data import app
|
||||
|
||||
# No openmind_hub installed; expect ImportError advisory.
|
||||
result = CliRunner().invoke(
|
||||
app, ["download", "dummy/ds", "--hub", "modelers"]
|
||||
)
|
||||
# exit_code 1 = friendly ImportError advisory; 2 = validation reject.
|
||||
assert result.exit_code in (1, 2)
|
||||
out = _strip_ansi(result.output)
|
||||
# No longer surfaces "wait for v0.53.9"; should mention modelers or
|
||||
# the missing SDK pip-install hint.
|
||||
assert "v0.53.9" not in out
|
||||
|
||||
def test_modelscope_unknown_sdk_friendly_error(self):
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from soup_cli.commands.data import app
|
||||
|
||||
if "modelscope" in sys.modules:
|
||||
pytest.skip("modelscope is installed; live branch tested separately")
|
||||
result = CliRunner().invoke(
|
||||
app, ["download", "dummy/ds", "--hub", "modelscope"]
|
||||
)
|
||||
assert result.exit_code in (1, 2)
|
||||
out = _strip_ansi(result.output)
|
||||
# Friendly advisory mentions the SDK or `pip install`.
|
||||
assert "modelscope" in out
|
||||
|
||||
def test_data_download_no_longer_advises_v0_53_9(self):
|
||||
# tdd-review LOW #9: a future regression of the advisory text
|
||||
# "wait for v0.53.9" must be caught at source-grep time, since
|
||||
# v0.53.10 lifted that advisory to live SDK dispatch.
|
||||
src = (REPO_ROOT / "soup_cli/commands/data.py").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
assert "wait for v0.53.9" not in src
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #155 — Web UI Tool Outputs panel
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestWebUiToolOutputsPanel:
|
||||
def test_index_html_has_tools_nav_entry(self):
|
||||
html = (REPO_ROOT / "soup_cli/ui/static/index.html").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
assert 'data-page="tools"' in html
|
||||
assert "Tool Outputs" in html
|
||||
assert 'id="page-tools"' in html
|
||||
|
||||
def test_app_js_has_load_tool_outputs(self):
|
||||
js = (REPO_ROOT / "soup_cli/ui/static/app.js").read_text(encoding="utf-8")
|
||||
assert "loadToolOutputs" in js
|
||||
assert "/api/tool-outputs" in js
|
||||
# XSS-safe — uses textContent or DOM API, NOT innerHTML for records.
|
||||
assert "td.textContent" in js
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #156 — SFT callback `record_call` wire-up
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestCallbackToolBuffer:
|
||||
def test_on_step_end_method_exists(self):
|
||||
from soup_cli.monitoring.callback import SoupTrainerCallback
|
||||
|
||||
assert hasattr(SoupTrainerCallback, "on_step_end")
|
||||
|
||||
def test_on_step_end_no_op_when_inputs_absent(self):
|
||||
from soup_cli.monitoring.callback import SoupTrainerCallback
|
||||
from soup_cli.utils.tool_outputs import (
|
||||
get_global_tool_buffer,
|
||||
reset_global_tool_buffer,
|
||||
)
|
||||
|
||||
reset_global_tool_buffer()
|
||||
cb = SoupTrainerCallback.__new__(SoupTrainerCallback)
|
||||
# State is dataclass-shaped; only global_step is read.
|
||||
from types import SimpleNamespace
|
||||
|
||||
cb.on_step_end(
|
||||
args=SimpleNamespace(),
|
||||
state=SimpleNamespace(global_step=1),
|
||||
control=SimpleNamespace(),
|
||||
)
|
||||
# Buffer should remain empty.
|
||||
assert len(list(get_global_tool_buffer().snapshot(limit=10))) == 0
|
||||
|
||||
def test_on_step_end_records_when_tool_calls_present(self):
|
||||
from soup_cli.monitoring.callback import SoupTrainerCallback
|
||||
from soup_cli.utils.tool_outputs import (
|
||||
get_global_tool_buffer,
|
||||
reset_global_tool_buffer,
|
||||
)
|
||||
|
||||
reset_global_tool_buffer()
|
||||
cb = SoupTrainerCallback.__new__(SoupTrainerCallback)
|
||||
from types import SimpleNamespace
|
||||
|
||||
cb.on_step_end(
|
||||
args=SimpleNamespace(),
|
||||
state=SimpleNamespace(global_step=42),
|
||||
control=SimpleNamespace(),
|
||||
inputs={"tool_calls": [{"name": "f"}, {"name": "g"}]},
|
||||
)
|
||||
records = list(get_global_tool_buffer().snapshot(limit=10))
|
||||
assert len(records) == 1
|
||||
assert "step 42" in records[0].output_preview
|
||||
assert records[0].success is True
|
||||
reset_global_tool_buffer()
|
||||
|
||||
def test_on_step_end_empty_tool_calls_emits_no_record(self):
|
||||
# tdd-review MEDIUM #6: empty list short-circuits.
|
||||
from types import SimpleNamespace
|
||||
|
||||
from soup_cli.monitoring.callback import SoupTrainerCallback
|
||||
from soup_cli.utils.tool_outputs import (
|
||||
get_global_tool_buffer,
|
||||
reset_global_tool_buffer,
|
||||
)
|
||||
|
||||
reset_global_tool_buffer()
|
||||
cb = SoupTrainerCallback.__new__(SoupTrainerCallback)
|
||||
cb.on_step_end(
|
||||
args=SimpleNamespace(),
|
||||
state=SimpleNamespace(global_step=1),
|
||||
control=SimpleNamespace(),
|
||||
inputs={"tool_calls": []},
|
||||
)
|
||||
assert len(list(get_global_tool_buffer().snapshot(limit=10))) == 0
|
||||
|
||||
def test_on_step_end_bool_tool_calls_emits_no_record(self):
|
||||
# tdd-review HIGH #2: ``tool_calls=True`` (bool) is a falsy-on-list
|
||||
# short-circuit AND rejected by the numeric-branch bool guard.
|
||||
from types import SimpleNamespace
|
||||
|
||||
from soup_cli.monitoring.callback import SoupTrainerCallback
|
||||
from soup_cli.utils.tool_outputs import (
|
||||
get_global_tool_buffer,
|
||||
reset_global_tool_buffer,
|
||||
)
|
||||
|
||||
reset_global_tool_buffer()
|
||||
cb = SoupTrainerCallback.__new__(SoupTrainerCallback)
|
||||
# ``True`` is truthy (bypasses ``not tool_calls`` guard); the
|
||||
# bool-rejection branch must keep ``count=0`` and skip recording.
|
||||
cb.on_step_end(
|
||||
args=SimpleNamespace(),
|
||||
state=SimpleNamespace(global_step=1),
|
||||
control=SimpleNamespace(),
|
||||
inputs={"tool_calls": True},
|
||||
)
|
||||
assert len(list(get_global_tool_buffer().snapshot(limit=10))) == 0
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Version bump sanity check
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestVersionBump:
|
||||
def test_init_py_pin(self):
|
||||
from soup_cli import __version__
|
||||
|
||||
assert __version__ == "0.53.10"
|
||||
|
||||
def test_pyproject_pin(self):
|
||||
body = (REPO_ROOT / "pyproject.toml").read_text(encoding="utf-8")
|
||||
assert re.search(r'^version\s*=\s*"0\.53\.10"', body, re.MULTILINE), (
|
||||
"pyproject.toml version pin missing v0.53.10"
|
||||
)
|
||||
|
|
@ -552,7 +552,16 @@ class TestDataDownloadHubFlag:
|
|||
assert result.exit_code != 0
|
||||
clean = _strip_ansi(result.output)
|
||||
assert "modelscope" in clean
|
||||
assert "v0.53.9" in clean or "download_repo" in clean
|
||||
# v0.53.10 #153 lifted the advisory to a live SDK dispatch; without
|
||||
# the modelscope SDK installed the friendly pip-install error fires.
|
||||
# Either the legacy v0.53.8 advisory (download_repo / v0.53.9) OR
|
||||
# the v0.53.10 ImportError "pip install modelscope" is acceptable.
|
||||
assert (
|
||||
"v0.53.9" in clean
|
||||
or "download_repo" in clean
|
||||
or "pip install modelscope" in clean
|
||||
or "not installed" in clean
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
|
|
@ -593,19 +602,23 @@ class TestPyprojectExtras:
|
|||
|
||||
|
||||
class TestVersionBump:
|
||||
@staticmethod
|
||||
def _version_tuple(s: str) -> tuple:
|
||||
return tuple(int(p) for p in s.split(".") if p.isdigit())
|
||||
|
||||
def test_init_version(self):
|
||||
import soup_cli
|
||||
|
||||
# Version-string is forward-monotonic: v0.53.8 baseline + any later
|
||||
# release (v0.53.9, v0.54.0, ...) keeps this contract green.
|
||||
assert soup_cli.__version__ >= "0.53.8"
|
||||
# Numeric tuple comparison defends against lexicographic regressions
|
||||
# (e.g. "0.53.10" < "0.53.8" string-wise).
|
||||
assert self._version_tuple(soup_cli.__version__) >= (0, 53, 8)
|
||||
|
||||
def test_pyproject_version(self):
|
||||
text = (_repo_root() / "pyproject.toml").read_text(encoding="utf-8")
|
||||
# Pyproject must declare some 0.53.x or later string; the literal
|
||||
# "0.53.8" baseline is allowed to drift forward.
|
||||
import re
|
||||
|
||||
match = re.search(r'^version\s*=\s*"([^"]+)"', text, flags=re.MULTILINE)
|
||||
assert match is not None
|
||||
assert match.group(1) >= "0.53.8"
|
||||
assert self._version_tuple(match.group(1)) >= (0, 53, 8)
|
||||
|
|
|
|||
|
|
@ -30,7 +30,11 @@ def _plain(text: str) -> str:
|
|||
# ----------------------------------------------------------------- version
|
||||
|
||||
def test_version_bump_to_0_53_9():
|
||||
assert soup_cli.__version__ == "0.53.9"
|
||||
# Forward-monotonic: v0.53.9 baseline + later releases (v0.53.10, ...)
|
||||
# keep this contract green. Numeric tuple compare defends against the
|
||||
# lexicographic "0.53.10" < "0.53.9" footgun.
|
||||
parts = tuple(int(p) for p in soup_cli.__version__.split(".") if p.isdigit())
|
||||
assert parts >= (0, 53, 9)
|
||||
|
||||
|
||||
# ----------------------------------------------------- #94 SSE event buffer
|
||||
|
|
|
|||
Loading…
Reference in New Issue