mirror of https://github.com/razor-ai/soup.git
feat(v0.53.8): Remote data + Hubs + Trackers (wave 2) — 6 features
- #85 fsspec live loaders — data/loader.py routes the v0.42.0 fsspec scheme allowlist (s3:// / gs:// / gcs:// / az:// / abfs:// / abfss:// / oci://) through fsspec.open with validate_remote_uri containment BEFORE connection. Friendly Rich panel names the pip install advisory when the backend SDK is missing. Threads data.streaming + data.buffer_size. Row count capped at 1M. - #130 Hub dispatcher live — utils/hubs.download_repo() and upload_repo() lazy-import per backend (huggingface_hub / modelscope / openmind_hub). Shared _validate_repo_id_shape (bool / null-byte / leading-slash / .. / control-char / oversize) + cwd containment on local_dir / folder_path. commands/train.py pre-fetches non-HF base into .soup_hub_cache/ (sanitised slug, idempotent on resume, cfg.base updated via model_copy). soup data download --hub flag plumbed. Multi-command rollout for chat / serve / infer / merge / export / push tracked for v0.53.9. - #89 [trackers] pyproject extra bundles mlflow / swanlab / trackio; tracker_missing_dep_message surfaces a friendly pip install advisory via importlib.util.find_spec (non-executing probe). - #90 utils/trackers.send_telemetry_payload — opt-IN via SOUP_TELEMETRY=1; lazy httpx; 1s hard timeout; HTTPS-only with SSRF re-validation (mirrors v0.51.0 hub endpoint policy); silent-fail on every exception. - #93 Fixtures migrated to soup_cli/data/_fixtures/ — zipapp / namespace-package safe via [tool.hatch.build.targets.wheel.force-include]; _bundle_source_path falls back to examples/data/ for editable installs. - #69 utils/hf_space.detect_space_sdk(requirements_text) — picks "streamlit" / "gradio" from the rendered requirements.txt; closes the v0.40.2 known limitation that custom Spaces always defaulted to gradio. Wired into commands/deploy.py. Review pass: python-review + code-review + security-review ran in parallel; 16 findings fixed (3 HIGH + 8 MEDIUM + 5 LOW). Highlights: cwd-containment on local_dir/folder_path, Windows ..\ traversal defence on .soup_hub_cache slug, Pydantic model_copy(update=...) instead of attribute mutation, idempotent pre-fetch via cache probe, 1M-row cap on remote materialisation, SSRF re-validation on telemetry endpoint override, 256 KB cap on detect_space_sdk input, modelscope.push_model commit_message kwarg removed (would TypeError at runtime), find_spec instead of __import__ to avoid swanlab side-effects. Test count: 8162 -> 8257 (+66 in tests/test_v0538.py + 29 net adjustments). Lint clean. CPU smoke: version, --help, load_config_from_string with hub: modelscope passes; mlx + non-HF rejected; data download --hub modelscope advisory rendered; detect_space_sdk live on real requirements.txt bodies; package-data fixtures resolve from soup_cli/data/_fixtures/. v0.53.7 known limitation #1 (bash 501 marker) bumped to v0.53.9. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alpamys
parent
0ddefc5c6f
commit
6d2170c4f3
|
|
@ -111,7 +111,7 @@ soup_cli/
|
|||
templates/ - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
|
||||
ui/ - Web UI (FastAPI + HTML/JS SPA)
|
||||
|
||||
tests/ - Test suite (190 files, 8162 tests)
|
||||
tests/ - Test suite (188 files, 8257 tests)
|
||||
examples/ - Real-world config examples and datasets
|
||||
```
|
||||
|
||||
|
|
|
|||
104
README.md
104
README.md
|
|
@ -42,19 +42,15 @@ soup train
|
|||
|
||||
Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
|
||||
|
||||
**v0.53.7 — Data Forge + Pipeline live (wave 1)**: Eleven features wired end-to-end across synthetic data generation, preprocessing, and inference.
|
||||
**v0.53.8 — Remote data + Hubs + Trackers (wave 2)**: Six features wired live across cloud storage, alternative model hubs, experiment trackers, and HF Spaces.
|
||||
|
||||
- **Data Forge live judge providers.** `soup data forge --judge-provider {ollama,anthropic,vllm}` now routes judge calls through real backends (not just offline stubs). Ollama enforces localhost-only SSRF; Anthropic uses env-only API key; vLLM validates scheme allowlist. Per-call exceptions logged at DEBUG so one bad provider never crashes the forge pipeline.
|
||||
- **AOT tokenization with `soup data preprocess`.** New command tokenizes your dataset once and caches Arrow shards keyed by `(dataset, tokenizer, max_length, format)`. SFT and Pretrain trainers short-circuit at the schema level when `format='pre_tokenized'` + `tokenized_path` is set, eliminating the per-epoch tokenization tax. Resume-safe via atomic writes.
|
||||
- **Data Recipe DAGs live (`soup data recipe --execute`).** Six node kinds now execute end-to-end: seed dataset, LLM text generation, code execution (RLVR sandbox), judge binary scoring, regex/JSON validation, and deterministic sampling. Checkpoint written per node; resume rehydrates from per-node sidecars. `failed_reason` fields redacted to prevent path leakage.
|
||||
- **Anthropic-style `/v1/messages` + streaming SSE on both backends.** The transformers-only route from v0.53.6 now ships on vLLM too. Both backends convert Anthropic ↔ OpenAI messages, serve non-stream requests with Anthropic envelope, and stream with Anthropic event-shape SSE (`message_start` → `content_block_delta` → `message_delta` + `message_stop`). Loopback-only CORS on both.
|
||||
- **Server-side `/v1/tools/python` + `/v1/tools/web_search` HTTP endpoints.** Python sandbox wraps the v0.25.0 RLVR `code_exec` with Bearer auth gate (5s timeout, 64KB code cap). Web search backend is httpx with hard 5s timeout, 5-result cap, and domain allowlist (deny-by-default via `WebSearchConfig`). (`/v1/tools/bash` reverted to 501 — child-process isolation insufficient; tracked for v0.53.8 with container/namespace work.)
|
||||
- **Live trainer-plugin instantiation.** `instantiate_trainer_plugins` now lazy-imports upstream plugins (grokfast, spectrum, llmcompressor, sonicmoe, cce_plugin, math_verify) and friendlily advises on missing packages.
|
||||
- **Markdown heading-aware ingest + operator-supplied decontamination corpus + custom prompt strategies.**
|
||||
- `soup data ingest` now splits Markdown by headings (one JSONL row per section with `section` + `level` + `text` fields).
|
||||
- `soup data decontaminate --benchmark-file <path>` accepts operator-supplied JSONL corpus instead of only bundled benchmarks.
|
||||
- `soup data forge` and other data tools now support custom prompt strategies via `prompt_strategy: module:function` in the config (importlib resolver + per-row exception handling).
|
||||
- **+111 net new tests** (8051 → 8162) in the new `test_v0537.py`. All four review agents ran; 38 findings fixed. Key hardening: symlink checks via `os.lstat` on raw paths (TOCTOU policy), atomic writes via tempfile + `os.replace`, error message redaction, narrowed exception handlers, tool endpoint Bearer auth opt-in, and comprehensive coverage for every live pathway.
|
||||
- **fsspec live loaders for remote datasets.** `data.train: s3://bucket/data.jsonl` (plus `gs://` / `gcs://` / `az://` / `abfs://` / `abfss://` / `oci://`) now loads through `fsspec` with the v0.42.0 SSRF-hardened URI validator (bucket regex + userinfo/query/fragment rejection). Threads `data.streaming` (HF `load_dataset(streaming=True)`) and `data.buffer_size` shuffle. Friendly Rich panel names the `pip install <driver>` advisory when the backend SDK is missing. Row-count capped at 1M to defend against pathological remote objects.
|
||||
- **Live ModelScope + Modelers hub dispatcher.** New `utils/hubs.download_repo()` and `upload_repo()` lazy-import the matching SDK per backend (`huggingface_hub`, `modelscope`, `openmind-hub`). Shared `_validate_repo_id_shape` rejects bool / null-byte / leading-slash / `..` segments / control characters / oversize; `local_dir` and `folder_path` containment-checked under cwd. `soup train` now pre-fetches non-HF `base` models into `.soup_hub_cache/<sanitized-slug>/` and reuses cached snapshots on resumed runs.
|
||||
- **`[trackers]` extra + friendly missing-dep advisory.** New `pip install soup-cli[trackers]` bundles MLflow + SwanLab + Trackio. When you pass `--tracker mlflow` without the package installed, `soup train` now surfaces a clear `pip install` advisory before construction instead of HF Trainer's generic ImportError. The probe uses `importlib.util.find_spec` (non-executing) so swanlab can't initialise network threads during the check.
|
||||
- **PostHog telemetry network (opt-IN, silent-fail).** `utils/trackers.send_telemetry_payload()` lazy-imports `httpx` and POSTs hardware-info-only payloads with a 1-second hard timeout. HTTPS-only endpoint check goes through the same SSRF validator as hub endpoints (private-IP / link-local / RFC1918 rejected). Disabled unless `SOUP_TELEMETRY=1`; every exception is swallowed so telemetry can never crash training.
|
||||
- **Bundled demo fixtures as package data.** `soup data demo` fixtures (4 JSONLs) migrated from `examples/data/` to `soup_cli/data/_fixtures/` so they ship inside the wheel — zipapp / namespace-package safe. `_bundle_source_path` falls back to the legacy `examples/data/` location for editable installs.
|
||||
- **HF Space SDK auto-pick from `requirements.txt`.** `soup deploy hf-space --template-dir <path>` now reads the rendered `requirements.txt` and picks `space_sdk="streamlit"` when the file lists `streamlit`, `"gradio"` otherwise. Closes the v0.40.2 known limitation that custom templates always created Spaces with `space_sdk="gradio"`.
|
||||
- **+95 net new tests** (8162 → 8257) in `test_v0538.py`. Three review agents ran (python / code / security); 16 findings fixed: cwd-containment on `local_dir`/`folder_path`, Pydantic `model_copy(update=...)` instead of attribute mutation, idempotent train pre-fetch via cache probe, row-count cap on remote materialisation, `modelscope.push_model` kwarg fix, SSRF re-validation on telemetry endpoint override, `detect_space_sdk` size cap, and more.
|
||||
|
||||
## Why Soup?
|
||||
|
||||
|
|
@ -3481,6 +3477,90 @@ soup data decontaminate --input training.jsonl --benchmarks mmlu,gsm8k,humaneval
|
|||
|
||||
The scorecard reports PII flagged, toxic flagged, language distribution, mean educational value, and decontamination removed. PII detection uses a narrow ReDoS-hardened regex set (email / phone / SSN / credit-card) with a 50 KB pre-cap on every input. Language detection is a stopword heuristic across six languages. Toxicity is a keyword baseline; the Llama-Guard-3-1B variant + FineWeb-Edu classifier ship behind `[data-pro]` extras. Decontamination uses n-gram containment against benchmark corpora: use `--benchmarks mmlu,gsm8k` for built-in allowlist, or `--benchmark-file custom_benchmark.jsonl` for your own corpus.
|
||||
|
||||
## Remote Datasets (S3 / GCS / Azure / OCI)
|
||||
|
||||
Point `data.train` at any object in the v0.42.0 fsspec allowlist and `soup train` will stream it through `fsspec.open` after running the URI through the same SSRF-hardened validator used everywhere else in Soup (bucket regex, no userinfo / query / fragment):
|
||||
|
||||
```yaml
|
||||
data:
|
||||
train: s3://my-bucket/datasets/train.jsonl
|
||||
format: alpaca
|
||||
streaming: true # opt-in HF datasets streaming with shuffle
|
||||
buffer_size: 10000 # shuffle buffer (requires streaming=true)
|
||||
```
|
||||
|
||||
Recognised schemes: `s3://`, `gs://`, `gcs://`, `az://`, `abfs://`, `abfss://`, `oci://`. The matching backend SDK (`s3fs` / `gcsfs` / `adlfs` / `ocifs`) is lazy-imported — install only what you need or grab the convenience extra:
|
||||
|
||||
```bash
|
||||
pip install soup-cli[remote] # fsspec + s3fs + gcsfs + adlfs
|
||||
```
|
||||
|
||||
Materialised rows are capped at 1M to defend against pathological remote objects; use a local split for larger jobs.
|
||||
|
||||
## Alternative Model Hubs (ModelScope / Modelers)
|
||||
|
||||
Set `training.hub` to fetch the base model from a non-HF Hub:
|
||||
|
||||
```yaml
|
||||
base: baichuan-inc/Baichuan2-7B
|
||||
task: sft
|
||||
training:
|
||||
hub: modelscope # or "modelers"
|
||||
```
|
||||
|
||||
`soup train` pre-fetches the model into `./.soup_hub_cache/<sanitized-slug>/` via the matching SDK (`modelscope.snapshot_download` / `openmind_hub.snapshot_download`) and rewrites `cfg.base` to the local snapshot. Re-runs reuse the cached snapshot. Both `huggingface-hub`, `modelscope`, and `openmind-hub` are lazy-imported — install only what you need.
|
||||
|
||||
Programmatic API:
|
||||
|
||||
```python
|
||||
from soup_cli.utils.hubs import download_repo, upload_repo
|
||||
|
||||
local_path = download_repo("modelscope", "baichuan-inc/Baichuan2-7B", local_dir="./snap")
|
||||
upload_repo("modelers", "my-org/my-model", folder_path="./output", commit_message="Soup v0.53.8")
|
||||
```
|
||||
|
||||
The dispatcher enforces shape validation on every input (bool / null-byte / leading-slash / `..` segments / control characters / oversize all rejected) and runs cwd-containment on `local_dir` / `folder_path`.
|
||||
|
||||
## Experiment Trackers (MLflow / SwanLab / Trackio)
|
||||
|
||||
Pick a tracker on the CLI; Soup threads it into HF Trainer's `report_to`:
|
||||
|
||||
```bash
|
||||
soup train --tracker mlflow
|
||||
soup train --tracker swanlab
|
||||
soup train --tracker trackio
|
||||
```
|
||||
|
||||
If the package is not installed, Soup now surfaces a friendly advisory before training starts instead of a mid-run ImportError:
|
||||
|
||||
```
|
||||
--tracker mlflow requires the 'mlflow' package. Install with: pip install soup-cli[trackers] (or pip install mlflow)
|
||||
```
|
||||
|
||||
```bash
|
||||
pip install soup-cli[trackers] # mlflow + swanlab + trackio
|
||||
```
|
||||
|
||||
## Telemetry (opt-IN, hardware-info-only)
|
||||
|
||||
Soup ships an opt-IN telemetry sender that POSTs hardware-info-only payloads (`soup_version` / `command` / `python` major.minor / `os` / `arch` / optional `duration_seconds`) — no dataset paths, model names, or config contents. Enable per-shell:
|
||||
|
||||
```bash
|
||||
SOUP_TELEMETRY=1 soup train --config soup.yaml
|
||||
```
|
||||
|
||||
The sender uses a 1-second hard timeout, HTTPS-only with private-IP / link-local rejection (same SSRF policy as hub endpoints), and swallows every exception silently — telemetry can never crash training. Disabled by default until a public privacy policy ships.
|
||||
|
||||
## HF Space SDK Auto-Pick
|
||||
|
||||
When you deploy a custom Space template directory, Soup now picks `space_sdk="streamlit"` / `"gradio"` from the rendered `requirements.txt`:
|
||||
|
||||
```bash
|
||||
soup deploy hf-space --space my-org/my-app --model my-org/my-model --template-dir ./my-template
|
||||
```
|
||||
|
||||
If `requirements.txt` lists `streamlit`, the Space is created with the Streamlit SDK. Otherwise (no requirements, gradio listed, etc.), Soup falls back to the Gradio default. The HF Hub allows `docker` and `static` SDKs too, but those cannot be inferred from `requirements.txt` alone — use the built-in templates or supply a custom one with an explicit `--sdk` override.
|
||||
|
||||
## Plugin System
|
||||
|
||||
Drop a Python module under `soup_cli/plugins/` (or any package importable by Soup) and register at import time:
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "soup-cli"
|
||||
version = "0.53.7"
|
||||
version = "0.53.8"
|
||||
description = "Fine-tune LLMs in one command. No SSH, no config hell."
|
||||
readme = "README.md"
|
||||
license = "Apache-2.0"
|
||||
|
|
@ -61,6 +61,10 @@ sglang = ["sglang>=0.2.0", "fastapi>=0.104.0", "uvicorn>=0.24.0"]
|
|||
mlx = ["mlx>=0.20.0", "mlx-lm>=0.20.0"]
|
||||
cce = ["cut-cross-entropy>=24.10.0"]
|
||||
tui = ["textual>=0.50.0"]
|
||||
# v0.53.8 #89 — bundle MLflow / SwanLab / Trackio for `--tracker` users.
|
||||
trackers = ["mlflow>=2.0.0", "swanlab>=0.3.0", "trackio>=0.0.1"]
|
||||
# v0.53.8 #85 — fsspec backends for remote dataset loading (s3 / gs / az / oci).
|
||||
remote = ["fsspec>=2024.1.0", "s3fs>=2024.1.0", "gcsfs>=2024.1.0", "adlfs>=2024.1.0"]
|
||||
|
||||
[project.scripts]
|
||||
soup = "soup_cli.cli:run"
|
||||
|
|
@ -72,6 +76,10 @@ Issues = "https://github.com/MakazhanAlpamys/Soup/issues"
|
|||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["soup_cli"]
|
||||
# v0.53.8 #93 — include bundled fixture JSONLs as package data so
|
||||
# `soup data demo` works in zipapp / namespace-package installs.
|
||||
[tool.hatch.build.targets.wheel.force-include]
|
||||
"soup_cli/data/_fixtures" = "soup_cli/data/_fixtures"
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py39"
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
"""Soup CLI — Fine-tune LLMs in one command."""
|
||||
|
||||
__version__ = "0.53.7"
|
||||
__version__ = "0.53.8"
|
||||
|
|
|
|||
|
|
@ -1189,8 +1189,34 @@ def download_dataset(
|
|||
"(v0.36.0). Only enable if you trust the source."
|
||||
),
|
||||
),
|
||||
hub: str = typer.Option(
|
||||
"hf",
|
||||
"--hub",
|
||||
help=(
|
||||
"Source hub: hf (default) / modelscope / modelers. "
|
||||
"Non-HF hubs require the matching SDK (v0.53.8 #130)."
|
||||
),
|
||||
),
|
||||
):
|
||||
"""Download a HuggingFace dataset and save as JSONL."""
|
||||
# v0.53.8 #130 — validate --hub at the CLI boundary; only `hf` is wired
|
||||
# for live dataset download in this release (modelscope / modelers
|
||||
# dataset SDKs differ from snapshot_download; live wiring tracked for
|
||||
# v0.53.9). Non-HF hubs surface a friendly error.
|
||||
from soup_cli.utils.hubs import validate_hub_name
|
||||
|
||||
try:
|
||||
hub_canonical = validate_hub_name(hub)
|
||||
except (TypeError, ValueError) as exc:
|
||||
console.print(f"[red]{exc}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
if hub_canonical != "hf":
|
||||
console.print(
|
||||
f"[red]--hub {hub_canonical} dataset download is not yet wired; "
|
||||
f"use `from soup_cli.utils.hubs import download_repo` to snapshot "
|
||||
f"a repo, or wait for v0.53.9.[/]"
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
max_download_samples = 1_000_000
|
||||
if samples is not None and samples > max_download_samples:
|
||||
console.print(
|
||||
|
|
|
|||
|
|
@ -499,11 +499,15 @@ def hf_space(
|
|||
# --- Render template files ---
|
||||
try:
|
||||
if template_dir is not None:
|
||||
from soup_cli.utils.hf_space import render_custom_template_dir
|
||||
from soup_cli.utils.hf_space import (
|
||||
detect_space_sdk,
|
||||
render_custom_template_dir,
|
||||
)
|
||||
files = render_custom_template_dir(template_dir, model_repo=model)
|
||||
# Custom templates default to gradio SDK unless requirements
|
||||
# imply otherwise; we record gradio for create_repo space_sdk.
|
||||
sdk = "gradio"
|
||||
# v0.53.8 #69 — auto-pick space_sdk from requirements.txt
|
||||
# (closes the v0.40.2 known limitation that custom templates
|
||||
# always created the Space with space_sdk="gradio").
|
||||
sdk = detect_space_sdk(files.get("requirements.txt"))
|
||||
else:
|
||||
files = render_space_template(template, model_repo=model)
|
||||
sdk = HF_SPACE_TEMPLATES[template]["sdk"]
|
||||
|
|
|
|||
|
|
@ -1286,12 +1286,12 @@ def _create_app(
|
|||
# the RLVR sandbox's OS-level isolation (``unshare(CLONE_NEWNET)``
|
||||
# / macOS ``sandbox-exec``); a caller can reach
|
||||
# ``http://169.254.169.254/...`` from the child shell. Reverted to
|
||||
# 501 until container/namespace work lands in v0.53.8.
|
||||
# 501 until container/namespace work lands in v0.53.9.
|
||||
raise HTTPException(
|
||||
status_code=501,
|
||||
detail=(
|
||||
"Server-side tool 'bash' live execution deferred to "
|
||||
"v0.53.8 — sandbox isolation requires container/namespace "
|
||||
"v0.53.9 — sandbox isolation requires container/namespace "
|
||||
"work."
|
||||
),
|
||||
)
|
||||
|
|
|
|||
|
|
@ -692,6 +692,65 @@ def train(
|
|||
console.print(f"[red]{_esc(str(exc))}[/]")
|
||||
raise typer.Exit(code=2) from exc
|
||||
console.print("[dim]Setting up model + trainer...[/]")
|
||||
# v0.53.8 #130 — pre-fetch model from non-HF hub into a local cache and
|
||||
# rewrite cfg.base to point at the local snapshot. The trainer wrappers
|
||||
# still use transformers.from_pretrained, which reads HF Hub by default;
|
||||
# by snapshotting first we keep every wrapper unchanged.
|
||||
hub_name = getattr(cfg.training, "hub", "hf") or "hf"
|
||||
if hub_name != "hf":
|
||||
import re as _re
|
||||
|
||||
from rich.markup import escape as _markup_escape
|
||||
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
from soup_cli.utils.paths import is_under_cwd
|
||||
|
||||
# Sanitise cache subdir name — strip every path-separator and
|
||||
# `..` segment so a crafted ``base: ../../etc`` cannot escape the
|
||||
# cache root (Windows ``\\`` and POSIX ``/`` both blocked).
|
||||
safe_slug = _re.sub(r"[^A-Za-z0-9._-]+", "__", cfg.base).strip("._-") or "model"
|
||||
cache_dir = (Path.cwd() / ".soup_hub_cache" / safe_slug).resolve()
|
||||
if not is_under_cwd(str(cache_dir)):
|
||||
console.print(
|
||||
"[red]Resolved hub cache dir escaped the current working "
|
||||
"directory; refusing to download.[/]"
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
try:
|
||||
# Idempotency: if the cache dir already has a config.json, skip
|
||||
# the re-download (modelscope/openmind-hub also short-circuit on
|
||||
# match but having an explicit probe lets us print a clear hint).
|
||||
existing_cfg = cache_dir / "config.json"
|
||||
if existing_cfg.is_file():
|
||||
local_path = str(cache_dir)
|
||||
console.print(
|
||||
f"[dim]Using cached snapshot at {local_path}[/]"
|
||||
)
|
||||
else:
|
||||
local_path = download_repo(
|
||||
hub_name,
|
||||
cfg.base,
|
||||
local_dir=str(cache_dir),
|
||||
)
|
||||
console.print(
|
||||
f"[dim]Fetched {cfg.base} from hub={hub_name} → "
|
||||
f"{local_path}[/]"
|
||||
)
|
||||
# Use ``model_copy(update=...)`` so the Pydantic field
|
||||
# validators on ``base`` rerun (matches v0.33.0 #47 / v0.40.0
|
||||
# Part B immutability policy).
|
||||
cfg = cfg.model_copy(update={"base": local_path})
|
||||
except ImportError as exc:
|
||||
console.print(f"[red]{_markup_escape(str(exc))}[/]")
|
||||
raise typer.Exit(code=1) from exc
|
||||
|
||||
# v0.53.8 #89 — friendly missing-dep advisory for `--tracker <name>`.
|
||||
if report_to and report_to not in ("none", "wandb", "tensorboard"):
|
||||
from soup_cli.utils.trackers import tracker_missing_dep_message
|
||||
|
||||
msg = tracker_missing_dep_message(report_to)
|
||||
if msg:
|
||||
console.print(f"[yellow]{msg}[/]")
|
||||
trainer_kwargs = {
|
||||
"device": device,
|
||||
"report_to": report_to,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,10 @@
|
|||
{"instruction": "Classify the sentiment of this sentence.", "input": "I love this product, it's amazing!", "output": "Positive"}
|
||||
{"instruction": "Translate to Spanish.", "input": "Hello, how are you?", "output": "Hola, ¿cómo estás?"}
|
||||
{"instruction": "Write a haiku about nature.", "input": "", "output": "Green leaves dance in wind\nWhispering ancient forest\nPeace in silence dwells"}
|
||||
{"instruction": "Explain quantum computing in simple terms.", "input": "", "output": "Quantum computers use quantum bits (qubits) instead of normal bits. Unlike regular bits which are 0 or 1, qubits can be both at once (superposition). This allows quantum computers to solve certain problems much faster than regular computers."}
|
||||
{"instruction": "Fix the grammar in this sentence.", "input": "She go to the store yesterday.", "output": "She went to the store yesterday."}
|
||||
{"instruction": "List 3 benefits of exercise.", "input": "", "output": "1. Improves cardiovascular health and strengthens the heart.\n2. Helps maintain a healthy weight and builds muscle.\n3. Reduces stress and improves mental health and mood."}
|
||||
{"instruction": "What is the capital of France?", "input": "", "output": "The capital of France is Paris."}
|
||||
{"instruction": "Summarize this text in one sentence.", "input": "Machine learning is a subset of artificial intelligence that enables computers to learn from data without being explicitly programmed. It uses algorithms to identify patterns in data and improve through experience.", "output": "Machine learning allows computers to learn patterns from data without explicit programming."}
|
||||
{"instruction": "Create a Python function to add two numbers.", "input": "", "output": "def add_numbers(a, b):\n \"\"\"Add two numbers and return the result.\"\"\"\n return a + b\n\n# Example usage\nresult = add_numbers(5, 3)\nprint(result) # Output: 8"}
|
||||
{"instruction": "Identify the main idea of this paragraph.", "input": "Climate change is caused by increased greenhouse gases in the atmosphere, primarily from human activities like burning fossil fuels. This leads to global warming, which affects weather patterns, sea levels, and ecosystems.", "output": "Greenhouse gases from human activities cause climate change and have significant environmental impacts."}
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
{"prompt": "What is machine learning?", "chosen": [{"role": "user", "content": "What is machine learning?"}, {"role": "assistant", "content": "Machine learning is a field of artificial intelligence where systems learn from data automatically. It uses algorithms to identify patterns and improve performance through experience, without being explicitly programmed for every task."}], "rejected": [{"role": "user", "content": "What is machine learning?"}, {"role": "assistant", "content": "It's computers doing stuff with data I guess."}]}
|
||||
{"prompt": "How do I learn Python?", "chosen": [{"role": "user", "content": "How do I learn Python?"}, {"role": "assistant", "content": "Here are effective ways to learn Python:\n1. Start with fundamentals: variables, loops, functions\n2. Practice with small projects\n3. Use resources like Python.org docs and tutorials\n4. Build projects to apply what you learn\n5. Join communities for support"}], "rejected": [{"role": "user", "content": "How do I learn Python?"}, {"role": "assistant", "content": "Just google it."}]}
|
||||
{"prompt": "What's the difference between AI and ML?", "chosen": [{"role": "user", "content": "What's the difference between AI and ML?"}, {"role": "assistant", "content": "AI (Artificial Intelligence) is the broader field of creating intelligent systems. ML (Machine Learning) is a subset of AI focused on learning from data. All machine learning is AI, but not all AI uses machine learning."}], "rejected": [{"role": "user", "content": "What's the difference between AI and ML?"}, {"role": "assistant", "content": "They're basically the same thing, just different names."}]}
|
||||
{"prompt": "Explain neural networks simply.", "chosen": [{"role": "user", "content": "Explain neural networks simply."}, {"role": "assistant", "content": "Neural networks are inspired by the human brain. They consist of layers of connected nodes (neurons) that process information. Each connection has a weight that adjusts during training. This allows the network to learn complex patterns in data."}], "rejected": [{"role": "user", "content": "Explain neural networks simply."}, {"role": "assistant", "content": "Neural networks are very complex mathematical structures that use backpropagation through computational graphs with gradient descent optimization."}]}
|
||||
{"prompt": "How should I structure a machine learning project?", "chosen": [{"role": "user", "content": "How should I structure a machine learning project?"}, {"role": "assistant", "content": "Follow this structure:\n1. Define the problem and gather data\n2. Explore and clean the data\n3. Choose and train a model\n4. Evaluate performance\n5. Iterate and improve\n6. Deploy to production\n7. Monitor and maintain"}], "rejected": [{"role": "user", "content": "How should I structure a machine learning project?"}, {"role": "assistant", "content": "Just throw all your code in one file and train the model."}]}
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
{"prompt": "What is photosynthesis?", "chosen": [{"role": "user", "content": "What is photosynthesis?"}, {"role": "assistant", "content": "Photosynthesis is the process by which green plants, algae, and some bacteria convert light energy into chemical energy. Using sunlight, water, and carbon dioxide, they produce glucose and oxygen. The overall equation is: 6CO2 + 6H2O + light energy → C6H12O6 + 6O2. This process occurs primarily in the chloroplasts of plant cells."}], "rejected": [{"role": "user", "content": "What is photosynthesis?"}, {"role": "assistant", "content": "It's when plants eat sunlight."}]}
|
||||
{"prompt": "How do I write a good commit message?", "chosen": [{"role": "user", "content": "How do I write a good commit message?"}, {"role": "assistant", "content": "A good commit message should:\n1. Use the imperative mood in the subject line (e.g., 'Add feature' not 'Added feature')\n2. Keep the subject line under 50 characters\n3. Separate the subject from the body with a blank line\n4. Use the body to explain what and why, not how\n5. Reference relevant issue numbers when applicable"}], "rejected": [{"role": "user", "content": "How do I write a good commit message?"}, {"role": "assistant", "content": "Just write 'fix' or 'update' and push it."}]}
|
||||
{"prompt": "What are the benefits of exercise?", "chosen": [{"role": "user", "content": "What are the benefits of exercise?"}, {"role": "assistant", "content": "Regular exercise provides numerous benefits:\n- Cardiovascular health: strengthens the heart and improves circulation\n- Mental health: reduces anxiety and depression through endorphin release\n- Weight management: helps maintain a healthy body composition\n- Bone and muscle strength: prevents osteoporosis and sarcopenia\n- Improved sleep quality and cognitive function\n- Reduced risk of chronic diseases like type 2 diabetes"}], "rejected": [{"role": "user", "content": "What are the benefits of exercise?"}, {"role": "assistant", "content": "Exercise is good for you. You should do it."}]}
|
||||
{"prompt": "Explain the concept of recursion in programming.", "chosen": [{"role": "user", "content": "Explain the concept of recursion in programming."}, {"role": "assistant", "content": "Recursion is when a function calls itself to solve a problem by breaking it into smaller subproblems. Every recursive function needs:\n1. A base case that stops the recursion\n2. A recursive case that reduces the problem\n\nFor example, calculating factorial:\n```python\ndef factorial(n):\n if n <= 1: # base case\n return 1\n return n * factorial(n - 1) # recursive case\n```\nfactorial(4) calls factorial(3), which calls factorial(2), and so on until reaching the base case."}], "rejected": [{"role": "user", "content": "Explain the concept of recursion in programming."}, {"role": "assistant", "content": "To understand recursion, you must first understand recursion. It's just a function that calls itself forever."}]}
|
||||
{"prompt": "What is the difference between HTTP and HTTPS?", "chosen": [{"role": "user", "content": "What is the difference between HTTP and HTTPS?"}, {"role": "assistant", "content": "HTTP (HyperText Transfer Protocol) transmits data in plain text, making it vulnerable to interception. HTTPS (HTTP Secure) adds TLS/SSL encryption, which:\n- Encrypts data in transit between client and server\n- Authenticates the server's identity via certificates\n- Ensures data integrity (no tampering)\nHTTPS is now the standard for all websites and is required for handling sensitive data like passwords and payments."}], "rejected": [{"role": "user", "content": "What is the difference between HTTP and HTTPS?"}, {"role": "assistant", "content": "HTTPS has an S at the end. It's more secure I think."}]}
|
||||
{"prompt": "How does a hash table work?", "chosen": [{"role": "user", "content": "How does a hash table work?"}, {"role": "assistant", "content": "A hash table stores key-value pairs for fast lookup. Here's how it works:\n1. A hash function converts the key into an array index\n2. The value is stored at that index\n3. On lookup, the same hash function computes the index to retrieve the value\n\nCollisions (two keys mapping to the same index) are handled via chaining (linked lists at each slot) or open addressing (probing for the next empty slot). Average time complexity is O(1) for insert, lookup, and delete."}], "rejected": [{"role": "user", "content": "How does a hash table work?"}, {"role": "assistant", "content": "It's like a dictionary. You put stuff in and get stuff out."}]}
|
||||
{"prompt": "What should I consider when choosing a database?", "chosen": [{"role": "user", "content": "What should I consider when choosing a database?"}, {"role": "assistant", "content": "Key factors when choosing a database:\n- Data model: relational (SQL) for structured data with relationships, document (NoSQL) for flexible schemas, key-value for simple lookups, graph for relationship-heavy data\n- Scale requirements: read-heavy vs write-heavy workloads, expected data volume\n- Consistency needs: strong consistency (ACID) vs eventual consistency (BASE)\n- Query patterns: complex joins favor SQL; simple lookups favor NoSQL\n- Operational concerns: hosting, backups, team expertise, licensing costs"}], "rejected": [{"role": "user", "content": "What should I consider when choosing a database?"}, {"role": "assistant", "content": "Just use MongoDB for everything, it's web scale."}]}
|
||||
{"prompt": "How do vaccines work?", "chosen": [{"role": "user", "content": "How do vaccines work?"}, {"role": "assistant", "content": "Vaccines train the immune system to recognize and fight specific pathogens without causing the disease. They work by introducing a harmless component of a pathogen (such as an inactivated virus, a protein subunit, or mRNA instructions) to the body. The immune system responds by producing antibodies and memory cells. If the real pathogen is encountered later, the immune system can mount a rapid, effective response, preventing or reducing the severity of illness."}], "rejected": [{"role": "user", "content": "How do vaccines work?"}, {"role": "assistant", "content": "They inject you with stuff to make you immune. That's pretty much it."}]}
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
{"instruction": "Solve this math problem: 7 + 5 * 2 =", "input": "", "output": "To solve 7 + 5 * 2, I need to follow the order of operations (PEMDAS).\n\nStep 1: Multiply first (5 * 2)\n5 * 2 = 10\n\nStep 2: Add (7 + 10)\n7 + 10 = 17\n\nAnswer: 17"}
|
||||
{"instruction": "Explain the Pythagorean theorem and solve an example.", "input": "", "output": "The Pythagorean theorem states that in a right triangle, a² + b² = c², where c is the hypotenuse (longest side).\n\nExample: If a = 3 and b = 4, find c\n\nStep 1: Substitute into formula\n3² + 4² = c²\n\nStep 2: Calculate squares\n9 + 16 = c²\n\nStep 3: Add\n25 = c²\n\nStep 4: Take square root\nc = √25 = 5\n\nAnswer: c = 5"}
|
||||
{"instruction": "Find the GCD of 48 and 18.", "input": "", "output": "To find the GCD (Greatest Common Divisor) of 48 and 18:\n\nMethod: Euclidean algorithm\n\nStep 1: 48 = 18 × 2 + 12\nStep 2: 18 = 12 × 1 + 6\nStep 3: 12 = 6 × 2 + 0\n\nWhen remainder is 0, the GCD is the last non-zero remainder.\n\nGCD(48, 18) = 6\n\nAnswer: 6"}
|
||||
{"instruction": "Solve: x + 5 = 12", "input": "", "output": "This is a simple linear equation.\n\nGiven: x + 5 = 12\n\nStep 1: Subtract 5 from both sides\nx + 5 - 5 = 12 - 5\n\nStep 2: Simplify\nx = 7\n\nVerification: 7 + 5 = 12 ✓\n\nAnswer: x = 7"}
|
||||
{"instruction": "What is 15% of 200?", "input": "", "output": "To find 15% of 200:\n\nMethod: Multiply by the percentage as a decimal\n15% = 15/100 = 0.15\n\n0.15 × 200 = 30\n\nAnswer: 30"}
|
||||
|
|
@ -116,9 +116,18 @@ def load_dataset(data_config: DataConfig) -> dict:
|
|||
Supports:
|
||||
- Local files (.jsonl, .json, .csv, .parquet, .txt)
|
||||
- HuggingFace dataset names (auto-detected if no file extension)
|
||||
- Remote fsspec URIs (s3://, gs://, gcs://, az://, abfs://, abfss://, oci://) — v0.53.8 #85
|
||||
"""
|
||||
train_path = data_config.train
|
||||
|
||||
# v0.53.8 #85 — fsspec live remote loader. Schema accepts these URIs
|
||||
# since v0.42.0; live loader lands here. Lazy-imports fsspec + the
|
||||
# backend driver (s3fs / gcsfs / adlfs / ocifs) and surfaces a
|
||||
# friendly Rich panel naming the pip install when the driver is
|
||||
# missing.
|
||||
if _looks_like_remote_uri(train_path):
|
||||
return _load_remote_dataset(train_path, data_config)
|
||||
|
||||
# Check if it's a HuggingFace dataset
|
||||
if not Path(train_path).suffix:
|
||||
return _load_hf_dataset(train_path, data_config)
|
||||
|
|
@ -214,6 +223,128 @@ def _validate_audio_files(data: list[dict], audio_dir: Path) -> list[dict]:
|
|||
return valid
|
||||
|
||||
|
||||
def _looks_like_remote_uri(value: str) -> bool:
|
||||
"""Quick sniff for the fsspec scheme allowlist (v0.42.0 Part B)."""
|
||||
if not isinstance(value, str) or "://" not in value:
|
||||
return False
|
||||
from soup_cli.utils.data_pipeline import is_remote_uri
|
||||
|
||||
return is_remote_uri(value)
|
||||
|
||||
|
||||
def _load_remote_dataset(train_path: str, data_config: DataConfig) -> dict:
|
||||
"""Load JSONL from a remote fsspec URI (s3 / gs / az / oci / etc.).
|
||||
|
||||
Validates the URI via the v0.42.0 ``validate_remote_uri`` allowlist
|
||||
(bucket regex, no userinfo/query/fragment) BEFORE opening any
|
||||
connection — defends against URL injection into the fsspec backend.
|
||||
|
||||
Streaming knobs (``data_config.streaming`` + ``buffer_size`` + ``shards``)
|
||||
are honoured via :func:`datasets.load_dataset` when present; otherwise
|
||||
the file is streamed as JSONL through :func:`fsspec.open`.
|
||||
"""
|
||||
from soup_cli.utils.data_pipeline import (
|
||||
required_remote_package,
|
||||
validate_remote_uri,
|
||||
)
|
||||
|
||||
canonical = validate_remote_uri(train_path)
|
||||
scheme = canonical.split("://", 1)[0]
|
||||
|
||||
try:
|
||||
import fsspec # type: ignore[import-not-found]
|
||||
except ImportError:
|
||||
from rich.panel import Panel
|
||||
|
||||
pkg = required_remote_package(scheme) or scheme
|
||||
console.print(
|
||||
Panel(
|
||||
f"[bold yellow]Missing dependency:[/] reading from "
|
||||
f"[bold]{scheme}://[/] requires the [bold]{pkg}[/] package.\n\n"
|
||||
f"Install with:\n [bold]pip install {pkg}[/]",
|
||||
title="Remote loader",
|
||||
border_style="yellow",
|
||||
)
|
||||
)
|
||||
raise
|
||||
|
||||
# Cap on rows materialised from a remote URI — matches v0.24.0
|
||||
# ``soup data download --samples`` ceiling. Defends against OOM when a
|
||||
# crafted / oversized bucket object is pointed at via streaming +
|
||||
# eager-materialise.
|
||||
max_remote_rows = 1_000_000
|
||||
|
||||
# Try the HF datasets streaming path first when the user opted in via
|
||||
# ``data.streaming=true`` — gives us free interleaving, shuffling, and
|
||||
# caching. Falls back to direct fsspec.open when datasets is missing or
|
||||
# rejects the URI.
|
||||
if data_config.streaming:
|
||||
try:
|
||||
from datasets import load_dataset as hf_load
|
||||
except ImportError as exc:
|
||||
raise ImportError(
|
||||
"data.streaming=true requires the 'datasets' package: "
|
||||
"pip install datasets"
|
||||
) from exc
|
||||
ds = hf_load(
|
||||
"json",
|
||||
data_files=canonical,
|
||||
split="train",
|
||||
streaming=True,
|
||||
)
|
||||
buf = data_config.buffer_size
|
||||
if buf:
|
||||
ds = ds.shuffle(buffer_size=buf)
|
||||
# Eager materialise capped at max_remote_rows — emit a clear advisory
|
||||
# if the cap trips.
|
||||
raw_data: list[dict] = []
|
||||
for i, row in enumerate(ds):
|
||||
if i >= max_remote_rows:
|
||||
console.print(
|
||||
f"[yellow]Remote dataset truncated at {max_remote_rows:,} "
|
||||
f"rows (use a local split for larger jobs).[/]"
|
||||
)
|
||||
break
|
||||
raw_data.append(row)
|
||||
else:
|
||||
# Non-streaming: open once, read lines, decode JSON.
|
||||
raw_data = []
|
||||
with fsspec.open(canonical, mode="rt", encoding="utf-8-sig") as fh:
|
||||
for i, raw_line in enumerate(fh):
|
||||
if i >= max_remote_rows:
|
||||
console.print(
|
||||
f"[yellow]Remote dataset truncated at "
|
||||
f"{max_remote_rows:,} rows.[/]"
|
||||
)
|
||||
break
|
||||
stripped = raw_line.strip()
|
||||
if not stripped:
|
||||
continue
|
||||
try:
|
||||
raw_data.append(json.loads(stripped))
|
||||
except json.JSONDecodeError as exc:
|
||||
console.print(
|
||||
f"[yellow]Warning: invalid JSON on line "
|
||||
f"{i + 1}: {exc}[/]"
|
||||
)
|
||||
|
||||
fmt = data_config.format
|
||||
if fmt == "auto":
|
||||
fmt = detect_format(raw_data)
|
||||
console.print(f"[dim]Auto-detected format: {fmt}[/]")
|
||||
|
||||
formatted = [format_to_messages(row, fmt) for row in raw_data]
|
||||
formatted = [r for r in formatted if r is not None]
|
||||
|
||||
if data_config.val_split > 0:
|
||||
split_idx = int(len(formatted) * (1 - data_config.val_split))
|
||||
return {
|
||||
"train": formatted[:split_idx],
|
||||
"val": formatted[split_idx:],
|
||||
}
|
||||
return {"train": formatted}
|
||||
|
||||
|
||||
def _load_hf_dataset(name: str, data_config: DataConfig) -> dict:
|
||||
"""Load a dataset from HuggingFace Hub."""
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -89,7 +89,12 @@ def get_bundle(name: str) -> DemoBundle:
|
|||
def _bundle_source_path(bundle: DemoBundle) -> str:
|
||||
"""Resolve the on-disk path for a bundle's fixture.
|
||||
|
||||
Uses importlib.resources to handle both editable + wheel installs.
|
||||
v0.53.8 #93 — fixtures live under ``soup_cli/data/_fixtures/`` (package
|
||||
data), with a back-compat fallback to ``examples/data/`` for editable
|
||||
installs / repo-root invocations. The package-data location is
|
||||
zipapp / namespace-package safe; the legacy location is kept so
|
||||
contributors editing fixtures via ``examples/data/`` still see their
|
||||
changes.
|
||||
"""
|
||||
# Filenames are baked-in constants (no user input), so direct join
|
||||
# is safe; we still defensively reject path separators.
|
||||
|
|
@ -97,17 +102,23 @@ def _bundle_source_path(bundle: DemoBundle) -> str:
|
|||
raise ValueError(
|
||||
f"bundle fixture name has separator: {bundle.fixture!r}"
|
||||
)
|
||||
# examples/ lives at the repo root, alongside the soup_cli/ package.
|
||||
# 1) Preferred — package data at soup_cli/data/_fixtures/.
|
||||
pkg_root = files("soup_cli")
|
||||
pkg_candidate = os.path.realpath(
|
||||
os.path.join(str(pkg_root), "data", "_fixtures", bundle.fixture)
|
||||
)
|
||||
if os.path.isfile(pkg_candidate):
|
||||
return pkg_candidate
|
||||
# 2) Fallback — legacy examples/data/ at repo root.
|
||||
repo_root = os.path.dirname(str(pkg_root))
|
||||
candidate = os.path.realpath(
|
||||
legacy = os.path.realpath(
|
||||
os.path.join(repo_root, "examples", "data", bundle.fixture)
|
||||
)
|
||||
if not os.path.isfile(candidate):
|
||||
raise FileNotFoundError(
|
||||
f"bundle fixture missing: {bundle.fixture}"
|
||||
)
|
||||
return candidate
|
||||
if os.path.isfile(legacy):
|
||||
return legacy
|
||||
raise FileNotFoundError(
|
||||
f"bundle fixture missing: {bundle.fixture}"
|
||||
)
|
||||
|
||||
|
||||
def copy_bundle_to(name: str, output_path: str) -> str:
|
||||
|
|
|
|||
|
|
@ -28,6 +28,66 @@ _MAX_TEMPLATE_FILE_BYTES = 256 * 1024 # 256 KB
|
|||
_KNOWN_FILES = ("app.py", "README.md", "requirements.txt")
|
||||
_REQUIRED_FILES = ("app.py", "README.md")
|
||||
|
||||
# v0.53.8 #69 — Auto-pick Space SDK from requirements.txt.
|
||||
# Reads the rendered requirements.txt content and returns one of the HF
|
||||
# Hub-supported `space_sdk` strings. Closes the v0.40.2 known limitation
|
||||
# that all custom templates defaulted to `space_sdk="gradio"`.
|
||||
|
||||
_SUPPORTED_SDKS = ("gradio", "streamlit", "docker", "static")
|
||||
|
||||
|
||||
def detect_space_sdk(requirements_text: str | None) -> str:
|
||||
"""Detect the Space SDK from a requirements.txt body.
|
||||
|
||||
Returns ``"streamlit"`` when the rendered ``requirements.txt`` lists
|
||||
``streamlit``, ``"gradio"`` when it lists ``gradio`` (or in any other
|
||||
case — the project default for chat templates).
|
||||
|
||||
The HF Hub Space-SDK allowlist also recognises ``"docker"`` and
|
||||
``"static"``, but those cannot be inferred from ``requirements.txt``
|
||||
alone — operators wanting those SDKs must use the built-in templates
|
||||
or extend this helper.
|
||||
|
||||
Rules (first-match-wins on a case-insensitive scan of dependency names,
|
||||
after stripping ``#`` comments / extras ``[all]`` / version specifiers
|
||||
``>=1.0``):
|
||||
- any line whose base name is ``streamlit`` → ``"streamlit"``
|
||||
- any line whose base name is ``gradio`` → ``"gradio"``
|
||||
- else → ``"gradio"``
|
||||
|
||||
Input is capped at 256 KB to defend against pathological requirements
|
||||
files (matches the v0.39.0 Part E template size policy).
|
||||
"""
|
||||
if requirements_text is None:
|
||||
return "gradio"
|
||||
if not isinstance(requirements_text, str):
|
||||
# Defence-in-depth — caller shouldn't pass non-str but be safe.
|
||||
return "gradio"
|
||||
if len(requirements_text) > _MAX_TEMPLATE_FILE_BYTES:
|
||||
# Refuse to scan oversized text — degrade to the default.
|
||||
return "gradio"
|
||||
for raw_line in requirements_text.splitlines():
|
||||
line = raw_line.strip().lower()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
# Strip version specifiers / extras: "streamlit>=1.0" → "streamlit"
|
||||
token = line
|
||||
for sep in ("=", ">", "<", "!", "~", "[", " ", ";"):
|
||||
idx = token.find(sep)
|
||||
if idx >= 0:
|
||||
token = token[:idx]
|
||||
token = token.strip()
|
||||
if token == "streamlit":
|
||||
return "streamlit"
|
||||
if token == "gradio":
|
||||
return "gradio"
|
||||
return "gradio"
|
||||
|
||||
|
||||
def is_supported_space_sdk(sdk: str) -> bool:
|
||||
"""Whether ``sdk`` is in the HF Hub Space-SDK allowlist."""
|
||||
return isinstance(sdk, str) and sdk in _SUPPORTED_SDKS
|
||||
|
||||
|
||||
def render_custom_template_dir(template_dir: str, model_repo: str) -> dict[str, str]:
|
||||
"""Render a custom Space template directory.
|
||||
|
|
|
|||
|
|
@ -204,3 +204,243 @@ def is_hf(hub: str) -> bool:
|
|||
if not isinstance(hub, str):
|
||||
return False
|
||||
return hub.lower() == "hf"
|
||||
|
||||
|
||||
# v0.53.8 #130 — Live download / upload dispatcher.
|
||||
# Each backend lazy-imports its SDK so a missing optional dep only surfaces
|
||||
# when the user actually selects that hub. Mirrors v0.51.0 stub-then-live
|
||||
# pattern: schema (TrainingConfig.hub Literal) shipped v0.51.0; live wiring
|
||||
# ships now.
|
||||
|
||||
_REPO_ID_MAX = 200
|
||||
|
||||
|
||||
def _validate_repo_id_shape(repo_id: str) -> str:
|
||||
"""Cheap shape-only repo-id validator shared by all hub adapters.
|
||||
|
||||
Does NOT mirror the full v0.29.0 HF ``validate_repo_id`` regex (which is
|
||||
HF-specific). Each hub's SDK applies its own canonicalisation; we just
|
||||
reject obviously dangerous shapes (null bytes, leading slash, ``..``,
|
||||
oversize) before forwarding.
|
||||
"""
|
||||
if isinstance(repo_id, bool):
|
||||
raise TypeError(f"repo_id must not be bool, got {repo_id!r}")
|
||||
if not isinstance(repo_id, str):
|
||||
raise TypeError(
|
||||
f"repo_id must be str, got {type(repo_id).__name__}"
|
||||
)
|
||||
if not repo_id:
|
||||
raise ValueError("repo_id must be non-empty")
|
||||
if "\x00" in repo_id:
|
||||
raise ValueError("repo_id must not contain null bytes")
|
||||
if len(repo_id) > _REPO_ID_MAX:
|
||||
raise ValueError(
|
||||
f"repo_id too long (max {_REPO_ID_MAX} chars)"
|
||||
)
|
||||
if repo_id.startswith("/") or repo_id.startswith("\\"):
|
||||
raise ValueError("repo_id must not start with a path separator")
|
||||
if ".." in repo_id.split("/"):
|
||||
raise ValueError("repo_id must not contain '..' segments")
|
||||
# Defence-in-depth: control chars (incl. CR / LF) would be a header
|
||||
# injection hazard if the id ever flowed into an HTTP request line.
|
||||
if any(ord(c) < 0x20 for c in repo_id):
|
||||
raise ValueError("repo_id must not contain control characters")
|
||||
return repo_id
|
||||
|
||||
|
||||
def _missing_dep_message(hub: str) -> str:
|
||||
"""Friendly ImportError message naming the pip install command."""
|
||||
pkg = required_hub_package(hub) or hub
|
||||
return (
|
||||
f"hub={hub!r} requires the '{pkg}' package. "
|
||||
f"Install with: pip install {pkg}"
|
||||
)
|
||||
|
||||
|
||||
def _validate_local_path(value: str, *, field: str) -> str:
|
||||
"""Cwd-containment + shape check for ``local_dir`` / ``folder_path``.
|
||||
|
||||
Mirrors the project-standard ``utils.paths.is_under_cwd`` policy used by
|
||||
every other path-accepting helper since v0.26.0. Rejects bool BEFORE
|
||||
`isinstance(str)` (matches v0.30.0 ``Candidate`` policy).
|
||||
"""
|
||||
from soup_cli.utils.paths import is_under_cwd
|
||||
|
||||
if isinstance(value, bool):
|
||||
raise TypeError(f"{field} must not be bool, got {value!r}")
|
||||
if not isinstance(value, str):
|
||||
raise TypeError(
|
||||
f"{field} must be str, got {type(value).__name__}"
|
||||
)
|
||||
if not value:
|
||||
raise ValueError(f"{field} must be a non-empty string")
|
||||
if "\x00" in value:
|
||||
raise ValueError(f"{field} must not contain null bytes")
|
||||
if not is_under_cwd(value):
|
||||
raise ValueError(
|
||||
f"{field} must stay under the current working directory"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def download_repo(
|
||||
hub: str,
|
||||
repo_id: str,
|
||||
*,
|
||||
local_dir: str,
|
||||
revision: str | None = None,
|
||||
allow_patterns: list[str] | None = None,
|
||||
repo_type: str = "model",
|
||||
) -> str:
|
||||
"""Snapshot-download ``repo_id`` from ``hub`` into ``local_dir``.
|
||||
|
||||
Returns the absolute local path to the downloaded snapshot. Lazy-imports
|
||||
the appropriate SDK per ``hub``:
|
||||
|
||||
* ``hf`` → :func:`huggingface_hub.snapshot_download`
|
||||
* ``modelscope`` → :func:`modelscope.snapshot_download`
|
||||
* ``modelers`` → :func:`openmind_hub.snapshot_download`
|
||||
|
||||
Raises ``ImportError`` (with pip-install hint) when the SDK is missing,
|
||||
``ValueError`` for invalid args, ``TypeError`` for wrong types.
|
||||
"""
|
||||
canonical = validate_hub_name(hub)
|
||||
_validate_repo_id_shape(repo_id)
|
||||
_validate_local_path(local_dir, field="local_dir")
|
||||
if revision is not None:
|
||||
if not isinstance(revision, str):
|
||||
raise TypeError("revision must be str or None")
|
||||
if "\x00" in revision or any(ord(c) < 0x20 for c in revision):
|
||||
raise ValueError("revision must not contain control characters")
|
||||
if repo_type not in ("model", "dataset", "space"):
|
||||
raise ValueError(
|
||||
"repo_type must be one of 'model' / 'dataset' / 'space'"
|
||||
)
|
||||
|
||||
if canonical == "hf":
|
||||
try:
|
||||
from huggingface_hub import snapshot_download
|
||||
except ImportError as exc:
|
||||
raise ImportError(_missing_dep_message("hf")) from exc
|
||||
return snapshot_download(
|
||||
repo_id=repo_id,
|
||||
repo_type=repo_type,
|
||||
revision=revision,
|
||||
local_dir=local_dir,
|
||||
allow_patterns=allow_patterns,
|
||||
)
|
||||
|
||||
if canonical == "modelscope":
|
||||
try:
|
||||
from modelscope import (
|
||||
snapshot_download as ms_download, # type: ignore[import-not-found]
|
||||
)
|
||||
except ImportError as exc:
|
||||
raise ImportError(_missing_dep_message("modelscope")) from exc
|
||||
# modelscope's snapshot_download uses a different kwarg set; we map
|
||||
# the canonical args here so callers see one consistent API.
|
||||
ms_kwargs: dict[str, object] = {
|
||||
"model_id": repo_id,
|
||||
"cache_dir": local_dir,
|
||||
}
|
||||
if revision is not None:
|
||||
ms_kwargs["revision"] = revision
|
||||
if allow_patterns is not None:
|
||||
ms_kwargs["allow_file_pattern"] = allow_patterns
|
||||
return ms_download(**ms_kwargs)
|
||||
|
||||
if canonical == "modelers":
|
||||
try:
|
||||
from openmind_hub import (
|
||||
snapshot_download as om_download, # type: ignore[import-not-found]
|
||||
)
|
||||
except ImportError as exc:
|
||||
raise ImportError(_missing_dep_message("modelers")) from exc
|
||||
om_kwargs: dict[str, object] = {
|
||||
"repo_id": repo_id,
|
||||
"local_dir": local_dir,
|
||||
}
|
||||
if revision is not None:
|
||||
om_kwargs["revision"] = revision
|
||||
if allow_patterns is not None:
|
||||
om_kwargs["allow_patterns"] = allow_patterns
|
||||
return om_download(**om_kwargs)
|
||||
|
||||
# Unreachable — validate_hub_name has already rejected unknown hubs.
|
||||
raise ValueError(f"hub {canonical!r} has no download adapter")
|
||||
|
||||
|
||||
def upload_repo(
|
||||
hub: str,
|
||||
repo_id: str,
|
||||
*,
|
||||
folder_path: str,
|
||||
commit_message: str = "Upload via Soup",
|
||||
token: str | None = None,
|
||||
repo_type: str = "model",
|
||||
) -> None:
|
||||
"""Upload ``folder_path`` to ``repo_id`` on ``hub``.
|
||||
|
||||
Same lazy-import policy as :func:`download_repo`. Token resolution is
|
||||
left to the caller (each backend has its own conventions); pass
|
||||
``token`` explicitly or rely on the SDK's env-var defaults.
|
||||
"""
|
||||
canonical = validate_hub_name(hub)
|
||||
_validate_repo_id_shape(repo_id)
|
||||
_validate_local_path(folder_path, field="folder_path")
|
||||
if not isinstance(commit_message, str) or not commit_message:
|
||||
raise ValueError("commit_message must be a non-empty string")
|
||||
# Mirror v0.29.0 push policy: first line only, ≤200 chars (prevents
|
||||
# multi-line injection into public commit history).
|
||||
commit_message = commit_message.splitlines()[0][:200]
|
||||
if repo_type not in ("model", "dataset", "space"):
|
||||
raise ValueError(
|
||||
"repo_type must be one of 'model' / 'dataset' / 'space'"
|
||||
)
|
||||
|
||||
if canonical == "hf":
|
||||
try:
|
||||
from huggingface_hub import HfApi
|
||||
except ImportError as exc:
|
||||
raise ImportError(_missing_dep_message("hf")) from exc
|
||||
api = HfApi(token=token)
|
||||
api.upload_folder(
|
||||
repo_id=repo_id,
|
||||
folder_path=folder_path,
|
||||
repo_type=repo_type,
|
||||
commit_message=commit_message,
|
||||
)
|
||||
return
|
||||
|
||||
if canonical == "modelscope":
|
||||
try:
|
||||
from modelscope.hub.api import HubApi # type: ignore[import-not-found]
|
||||
except ImportError as exc:
|
||||
raise ImportError(_missing_dep_message("modelscope")) from exc
|
||||
api = HubApi()
|
||||
if token:
|
||||
api.login(token)
|
||||
# ModelScope's `push_model` does not accept `commit_message` — pass
|
||||
# only the model id + dir. The sanitised commit_message is recorded
|
||||
# in the operator's local git log via the HF/Modelers backends.
|
||||
api.push_model(
|
||||
model_id=repo_id,
|
||||
model_dir=folder_path,
|
||||
)
|
||||
return
|
||||
|
||||
if canonical == "modelers":
|
||||
try:
|
||||
from openmind_hub import HubApi # type: ignore[import-not-found]
|
||||
except ImportError as exc:
|
||||
raise ImportError(_missing_dep_message("modelers")) from exc
|
||||
api = HubApi(token=token)
|
||||
api.upload_folder(
|
||||
repo_id=repo_id,
|
||||
folder_path=folder_path,
|
||||
repo_type=repo_type,
|
||||
commit_message=commit_message,
|
||||
)
|
||||
return
|
||||
|
||||
raise ValueError(f"hub {canonical!r} has no upload adapter")
|
||||
|
|
|
|||
|
|
@ -157,6 +157,139 @@ def build_telemetry_payload(
|
|||
}
|
||||
|
||||
|
||||
# v0.53.8 #90 — PostHog telemetry live wiring.
|
||||
# Opt-IN via SOUP_TELEMETRY=1; silent-fail on any network/transport error
|
||||
# so telemetry can NEVER crash training. 1s hard timeout, HTTPS-only.
|
||||
|
||||
_POSTHOG_HOST = "https://us.i.posthog.com"
|
||||
_POSTHOG_ENDPOINT = f"{_POSTHOG_HOST}/i/v0/e/"
|
||||
# Public write-only key. Live deployments will swap this via env var.
|
||||
_POSTHOG_DEFAULT_KEY = "phc_soup_public_write_only"
|
||||
_TELEMETRY_TIMEOUT_S = 1.0
|
||||
|
||||
|
||||
def _telemetry_endpoint_is_safe(endpoint: str) -> bool:
|
||||
"""Re-validate the telemetry endpoint via the v0.51.0 SSRF policy.
|
||||
|
||||
Even though :func:`send_telemetry_payload` only POSTs to a static
|
||||
PostHog URL by default, callers can override ``endpoint``. Re-run the
|
||||
same private-IP / link-local rejection used for hub endpoints so a
|
||||
crafted ``endpoint='https://10.0.0.1/'`` cannot reach an internal
|
||||
network from a misconfigured caller.
|
||||
"""
|
||||
if not isinstance(endpoint, str) or not endpoint.startswith("https://"):
|
||||
return False
|
||||
try:
|
||||
from soup_cli.utils.hubs import validate_hub_endpoint
|
||||
|
||||
validate_hub_endpoint(endpoint, hub="telemetry")
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def send_telemetry_payload(
|
||||
payload: dict[str, object],
|
||||
*,
|
||||
api_key: str | None = None,
|
||||
timeout: float = _TELEMETRY_TIMEOUT_S,
|
||||
endpoint: str = _POSTHOG_ENDPOINT,
|
||||
) -> bool:
|
||||
"""POST ``payload`` to PostHog if telemetry is enabled, else no-op.
|
||||
|
||||
Returns ``True`` on a 2xx response, ``False`` on any failure or skip.
|
||||
NEVER raises — telemetry is best-effort and must never crash training.
|
||||
|
||||
Args:
|
||||
payload: dict built by :func:`build_telemetry_payload`. Required keys
|
||||
are validated upstream by the builder.
|
||||
api_key: PostHog project key. Defaults to the bundled write-only key.
|
||||
timeout: hard wall-clock cap (default 1 s).
|
||||
endpoint: full PostHog capture URL (must be HTTPS).
|
||||
"""
|
||||
if not is_telemetry_enabled():
|
||||
return False
|
||||
if not isinstance(payload, dict) or not payload:
|
||||
return False
|
||||
# HTTPS-only + private-IP / link-local rejection (mirrors v0.51.0 hub
|
||||
# endpoint SSRF policy). Defence-in-depth: any caller override goes
|
||||
# through the same validator that hub endpoints do.
|
||||
if not _telemetry_endpoint_is_safe(endpoint):
|
||||
return False
|
||||
if isinstance(timeout, bool) or not isinstance(timeout, (int, float)):
|
||||
return False
|
||||
if not math.isfinite(float(timeout)) or timeout <= 0:
|
||||
return False
|
||||
key = api_key or _POSTHOG_DEFAULT_KEY
|
||||
if not isinstance(key, str) or not key:
|
||||
return False
|
||||
try:
|
||||
import httpx # lazy — optional dep, surfaces no advisory
|
||||
except ImportError:
|
||||
return False
|
||||
body = {
|
||||
"api_key": key,
|
||||
"event": payload.get("command", "soup_event"),
|
||||
"properties": {k: v for k, v in payload.items() if k != "command"},
|
||||
}
|
||||
try:
|
||||
resp = httpx.post(endpoint, json=body, timeout=timeout)
|
||||
return 200 <= resp.status_code < 300
|
||||
except Exception: # noqa: BLE001 — telemetry must never crash training
|
||||
return False
|
||||
|
||||
|
||||
# v0.53.8 #89 — Friendly missing-dep panel for HF Trainer `--tracker`.
|
||||
# When user passes `--tracker mlflow` without mlflow installed, HF raises
|
||||
# a generic ImportError mid-training; this helper lets the CLI surface a
|
||||
# pip-install advisory BEFORE construction.
|
||||
|
||||
|
||||
def tracker_missing_dep_message(name: str) -> str | None:
|
||||
"""Return a friendly install advisory for ``name`` if the package is
|
||||
missing, else None.
|
||||
|
||||
Always returns ``None`` for `wandb` / `tensorboard` / `none` (the
|
||||
legacy backends), since those are part of the standard HF Trainer
|
||||
extra and not part of v0.43.0's additive set.
|
||||
"""
|
||||
if not isinstance(name, str):
|
||||
return None
|
||||
canonical = name.lower()
|
||||
if canonical not in NEW_TRACKERS_V0_43:
|
||||
return None
|
||||
pkg = required_tracker_package(canonical)
|
||||
if not pkg:
|
||||
return None
|
||||
# Use ``importlib.util.find_spec`` (non-executing probe) so we don't
|
||||
# incur side effects from the tracker's top-level module (e.g. swanlab
|
||||
# initialises network threads on import). ``sys.modules[pkg] = None``
|
||||
# raises ``ValueError`` on find_spec — treat that as missing too so
|
||||
# tests can simulate the absent-package path without subprocess.
|
||||
import importlib.util
|
||||
import sys
|
||||
|
||||
sentinel = object()
|
||||
cached = sys.modules.get(pkg, sentinel)
|
||||
if cached is None:
|
||||
missing = True
|
||||
elif cached is not sentinel:
|
||||
# Module is already imported (or test injected a real-shaped mock).
|
||||
missing = False
|
||||
else:
|
||||
try:
|
||||
missing = importlib.util.find_spec(pkg) is None
|
||||
except (ImportError, ValueError):
|
||||
missing = True
|
||||
if missing:
|
||||
return (
|
||||
f"--tracker {canonical} requires the '{pkg}' package. "
|
||||
f"Install with: pip install soup-cli[trackers] "
|
||||
f"(or pip install {pkg})"
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def resolve_report_to(
|
||||
*,
|
||||
wandb: bool = False,
|
||||
|
|
|
|||
|
|
@ -952,7 +952,7 @@ class TestToolEndpointsLive:
|
|||
json={"command": "echo hello"},
|
||||
)
|
||||
assert resp.status_code == 501
|
||||
assert "v0.53.8" in resp.text
|
||||
assert "v0.53.9" in resp.text
|
||||
|
||||
def test_bash_tool_returns_501_with_empty_body(self):
|
||||
"""The 501 stub does not parse the body — it always returns 501."""
|
||||
|
|
@ -1609,7 +1609,7 @@ class TestReviewFixesC1BashStub:
|
|||
client = TestClient(app)
|
||||
resp = client.post("/v1/tools/bash", json={"command": "ls"})
|
||||
assert resp.status_code == 501
|
||||
assert "v0.53.8" in resp.text
|
||||
assert "v0.53.9" in resp.text
|
||||
|
||||
|
||||
class TestReviewFixesAuthToken:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,583 @@
|
|||
"""v0.53.8 — Remote data + Hubs + Trackers (wave 2).
|
||||
|
||||
Tests cover:
|
||||
* #85 — fsspec live loaders (data/loader.py)
|
||||
* #130 — Live hub download/upload dispatcher (utils/hubs.py)
|
||||
* #89 — `[trackers]` extra + missing-dep advisory
|
||||
* #90 — PostHog telemetry network (best-effort, silent-fail)
|
||||
* #93 — package-data migration (soup_cli/data/_fixtures/)
|
||||
* #69 — HF Space SDK auto-pick from requirements.txt
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #69 — HF Space SDK auto-pick
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDetectSpaceSdk:
|
||||
def _import(self):
|
||||
from soup_cli.utils.hf_space import detect_space_sdk
|
||||
|
||||
return detect_space_sdk
|
||||
|
||||
def test_none_defaults_to_gradio(self):
|
||||
assert self._import()(None) == "gradio"
|
||||
|
||||
def test_empty_defaults_to_gradio(self):
|
||||
assert self._import()("") == "gradio"
|
||||
|
||||
def test_streamlit_picked(self):
|
||||
assert self._import()("streamlit>=1.0\nrequests") == "streamlit"
|
||||
|
||||
def test_gradio_picked(self):
|
||||
assert self._import()("gradio==4.0") == "gradio"
|
||||
|
||||
def test_streamlit_wins_when_first(self):
|
||||
# First match wins
|
||||
assert self._import()("streamlit\ngradio") == "streamlit"
|
||||
|
||||
def test_comment_ignored(self):
|
||||
assert self._import()("# streamlit\ngradio") == "gradio"
|
||||
|
||||
def test_no_match_defaults_gradio(self):
|
||||
assert self._import()("torch\nnumpy") == "gradio"
|
||||
|
||||
def test_non_string_defaults_gradio(self):
|
||||
assert self._import()(123) == "gradio" # type: ignore[arg-type]
|
||||
|
||||
def test_with_extras(self):
|
||||
assert self._import()("streamlit[all]>=1.0") == "streamlit"
|
||||
|
||||
def test_uppercase_normalised(self):
|
||||
assert self._import()("STREAMLIT==1.0") == "streamlit"
|
||||
|
||||
def test_oversize_input_degrades_to_default(self):
|
||||
# security-review LOW — size cap defends against pathological input
|
||||
huge = "streamlit\n" + ("x" * (260 * 1024))
|
||||
assert self._import()(huge) == "gradio"
|
||||
|
||||
|
||||
class TestIsSupportedSpaceSdk:
|
||||
def test_known(self):
|
||||
from soup_cli.utils.hf_space import is_supported_space_sdk
|
||||
|
||||
assert is_supported_space_sdk("gradio")
|
||||
assert is_supported_space_sdk("streamlit")
|
||||
assert is_supported_space_sdk("docker")
|
||||
assert is_supported_space_sdk("static")
|
||||
|
||||
def test_unknown(self):
|
||||
from soup_cli.utils.hf_space import is_supported_space_sdk
|
||||
|
||||
assert not is_supported_space_sdk("flask")
|
||||
assert not is_supported_space_sdk("")
|
||||
assert not is_supported_space_sdk(None) # type: ignore[arg-type]
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #130 — Hub download / upload dispatcher
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDownloadRepoValidation:
|
||||
def test_unknown_hub_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match="not supported"):
|
||||
download_repo("evil_hub", "owner/repo", local_dir="./snap_v0538")
|
||||
|
||||
def test_empty_repo_id_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match="non-empty"):
|
||||
download_repo("hf", "", local_dir="./snap_v0538")
|
||||
|
||||
def test_null_byte_repo_id_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match="null bytes"):
|
||||
download_repo("hf", "owner\x00/repo", local_dir="./snap_v0538")
|
||||
|
||||
def test_bool_repo_id_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(TypeError, match="bool"):
|
||||
download_repo("hf", True, local_dir="./snap_v0538") # type: ignore[arg-type]
|
||||
|
||||
def test_leading_slash_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match="path separator"):
|
||||
download_repo("hf", "/etc/passwd", local_dir="./snap_v0538")
|
||||
|
||||
def test_dotdot_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match=".."):
|
||||
download_repo("hf", "../escape", local_dir="./snap_v0538")
|
||||
|
||||
def test_control_char_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match="control"):
|
||||
download_repo("hf", "owner/repo\n", local_dir="./snap_v0538")
|
||||
|
||||
def test_oversize_repo_id_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match="too long"):
|
||||
download_repo("hf", "a" * 250, local_dir="./snap_v0538")
|
||||
|
||||
def test_empty_local_dir_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match="local_dir"):
|
||||
download_repo("hf", "owner/repo", local_dir="")
|
||||
|
||||
def test_local_dir_outside_cwd_raises(self, tmp_path):
|
||||
# security-review HIGH — containment check on local_dir
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
# tmp_path is a sibling of cwd, not under it
|
||||
out = str(tmp_path / "snap")
|
||||
with pytest.raises(ValueError, match="under the current working"):
|
||||
download_repo("hf", "owner/repo", local_dir=out)
|
||||
|
||||
def test_local_dir_bool_rejected(self):
|
||||
# security-review LOW — bool before str check
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(TypeError, match="bool"):
|
||||
download_repo("hf", "owner/repo", local_dir=True) # type: ignore[arg-type]
|
||||
|
||||
def test_bad_repo_type_raises(self):
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
with pytest.raises(ValueError, match="repo_type"):
|
||||
download_repo("hf", "owner/repo", local_dir="./snap_v0538", repo_type="evil")
|
||||
|
||||
|
||||
class TestDownloadRepoLazyImport:
|
||||
def test_modelscope_missing_friendly_error(self, monkeypatch):
|
||||
from soup_cli.utils import hubs
|
||||
|
||||
# Block modelscope import even if installed
|
||||
monkeypatch.setitem(sys.modules, "modelscope", None)
|
||||
with pytest.raises(ImportError, match="modelscope"):
|
||||
hubs.download_repo("modelscope", "owner/repo", local_dir="./snap_v0538")
|
||||
|
||||
def test_modelers_missing_friendly_error(self, monkeypatch):
|
||||
from soup_cli.utils import hubs
|
||||
|
||||
monkeypatch.setitem(sys.modules, "openmind_hub", None)
|
||||
with pytest.raises(ImportError, match="openmind-hub"):
|
||||
hubs.download_repo("modelers", "owner/repo", local_dir="./snap_v0538")
|
||||
|
||||
def test_hf_dispatch_calls_snapshot_download(self):
|
||||
with patch(
|
||||
"huggingface_hub.snapshot_download", return_value="/local/snap"
|
||||
) as mocked:
|
||||
from soup_cli.utils.hubs import download_repo
|
||||
|
||||
result = download_repo("hf", "owner/repo", local_dir="./snap_v0538")
|
||||
assert result == "/local/snap"
|
||||
mocked.assert_called_once()
|
||||
kwargs = mocked.call_args.kwargs
|
||||
assert kwargs["repo_id"] == "owner/repo"
|
||||
assert kwargs["local_dir"] == "./snap_v0538"
|
||||
|
||||
|
||||
class TestUploadRepoValidation:
|
||||
def test_unknown_hub_raises(self):
|
||||
from soup_cli.utils.hubs import upload_repo
|
||||
|
||||
with pytest.raises(ValueError, match="not supported"):
|
||||
upload_repo("evil", "o/r", folder_path="./folder_v0538")
|
||||
|
||||
def test_empty_folder_raises(self):
|
||||
from soup_cli.utils.hubs import upload_repo
|
||||
|
||||
with pytest.raises(ValueError, match="folder_path"):
|
||||
upload_repo("hf", "o/r", folder_path="")
|
||||
|
||||
def test_folder_path_outside_cwd_raises(self, tmp_path):
|
||||
# security-review HIGH — containment check on folder_path
|
||||
from soup_cli.utils.hubs import upload_repo
|
||||
|
||||
out = str(tmp_path / "out")
|
||||
with pytest.raises(ValueError, match="under the current working"):
|
||||
upload_repo("hf", "o/r", folder_path=out)
|
||||
|
||||
def test_empty_commit_raises(self):
|
||||
from soup_cli.utils.hubs import upload_repo
|
||||
|
||||
with pytest.raises(ValueError, match="commit_message"):
|
||||
upload_repo("hf", "o/r", folder_path="./folder_v0538", commit_message="")
|
||||
|
||||
def test_commit_message_truncated(self):
|
||||
# First line + 200 char cap (mirrors v0.29.0 push policy)
|
||||
with patch("huggingface_hub.HfApi") as mock_api_cls:
|
||||
mock_api = MagicMock()
|
||||
mock_api_cls.return_value = mock_api
|
||||
from soup_cli.utils.hubs import upload_repo
|
||||
|
||||
long_msg = "line1\nline2" + "x" * 500
|
||||
upload_repo("hf", "o/r", folder_path="./folder_v0538", commit_message=long_msg)
|
||||
sent = mock_api.upload_folder.call_args.kwargs["commit_message"]
|
||||
assert "\n" not in sent
|
||||
assert len(sent) <= 200
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #90 — PostHog telemetry network
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSendTelemetryPayload:
|
||||
def test_disabled_short_circuits(self, monkeypatch):
|
||||
monkeypatch.delenv("SOUP_TELEMETRY", raising=False)
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert send_telemetry_payload({"command": "train"}) is False
|
||||
|
||||
def test_empty_payload_returns_false(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert send_telemetry_payload({}) is False
|
||||
|
||||
def test_non_dict_returns_false(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert send_telemetry_payload("not a dict") is False # type: ignore[arg-type]
|
||||
|
||||
def test_http_endpoint_rejected(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert (
|
||||
send_telemetry_payload(
|
||||
{"command": "train"}, endpoint="http://evil.example/i/"
|
||||
)
|
||||
is False
|
||||
)
|
||||
|
||||
def test_private_ip_endpoint_rejected(self, monkeypatch):
|
||||
# security-review MEDIUM — SSRF via private IP override
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert (
|
||||
send_telemetry_payload(
|
||||
{"command": "train"}, endpoint="https://10.0.0.1/i/"
|
||||
)
|
||||
is False
|
||||
)
|
||||
|
||||
def test_link_local_endpoint_rejected(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert (
|
||||
send_telemetry_payload(
|
||||
{"command": "train"}, endpoint="https://169.254.169.254/i/"
|
||||
)
|
||||
is False
|
||||
)
|
||||
|
||||
def test_bool_timeout_rejected(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert send_telemetry_payload({"command": "train"}, timeout=True) is False # type: ignore[arg-type]
|
||||
|
||||
def test_negative_timeout_rejected(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert send_telemetry_payload({"command": "train"}, timeout=-1) is False
|
||||
|
||||
def test_httpx_missing_returns_false(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
# Force ImportError on the lazy import
|
||||
monkeypatch.setitem(sys.modules, "httpx", None)
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
assert send_telemetry_payload({"command": "train"}) is False
|
||||
|
||||
def test_happy_path_2xx(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
|
||||
fake_httpx = MagicMock()
|
||||
fake_resp = MagicMock()
|
||||
fake_resp.status_code = 200
|
||||
fake_httpx.post.return_value = fake_resp
|
||||
monkeypatch.setitem(sys.modules, "httpx", fake_httpx)
|
||||
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
result = send_telemetry_payload({"command": "train", "soup_version": "x"})
|
||||
assert result is True
|
||||
# HTTPS-only + 1s timeout
|
||||
call = fake_httpx.post.call_args
|
||||
assert call.args[0].startswith("https://")
|
||||
assert call.kwargs["timeout"] == 1.0
|
||||
|
||||
def test_network_exception_swallowed(self, monkeypatch):
|
||||
monkeypatch.setenv("SOUP_TELEMETRY", "1")
|
||||
|
||||
fake_httpx = MagicMock()
|
||||
fake_httpx.post.side_effect = RuntimeError("network down")
|
||||
monkeypatch.setitem(sys.modules, "httpx", fake_httpx)
|
||||
|
||||
from soup_cli.utils.trackers import send_telemetry_payload
|
||||
|
||||
# Must never raise
|
||||
assert send_telemetry_payload({"command": "train"}) is False
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #89 — Tracker missing-dep advisory
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestTrackerMissingDepMessage:
|
||||
def test_wandb_returns_none(self):
|
||||
from soup_cli.utils.trackers import tracker_missing_dep_message
|
||||
|
||||
# Legacy backend — never advise
|
||||
assert tracker_missing_dep_message("wandb") is None
|
||||
|
||||
def test_tensorboard_returns_none(self):
|
||||
from soup_cli.utils.trackers import tracker_missing_dep_message
|
||||
|
||||
assert tracker_missing_dep_message("tensorboard") is None
|
||||
|
||||
def test_unknown_returns_none(self):
|
||||
from soup_cli.utils.trackers import tracker_missing_dep_message
|
||||
|
||||
assert tracker_missing_dep_message("evil") is None
|
||||
|
||||
def test_non_string_returns_none(self):
|
||||
from soup_cli.utils.trackers import tracker_missing_dep_message
|
||||
|
||||
assert tracker_missing_dep_message(123) is None # type: ignore[arg-type]
|
||||
|
||||
def test_missing_mlflow_advisory(self, monkeypatch):
|
||||
monkeypatch.setitem(sys.modules, "mlflow", None)
|
||||
from soup_cli.utils.trackers import tracker_missing_dep_message
|
||||
|
||||
msg = tracker_missing_dep_message("mlflow")
|
||||
assert msg is not None
|
||||
assert "mlflow" in msg
|
||||
assert "soup-cli[trackers]" in msg
|
||||
|
||||
def test_present_mlflow_returns_none(self, monkeypatch):
|
||||
# Pretend mlflow is installed
|
||||
fake = MagicMock()
|
||||
monkeypatch.setitem(sys.modules, "mlflow", fake)
|
||||
from soup_cli.utils.trackers import tracker_missing_dep_message
|
||||
|
||||
assert tracker_missing_dep_message("mlflow") is None
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #85 — fsspec live remote loader
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRemoteLoader:
|
||||
def test_looks_like_remote_uri(self):
|
||||
from soup_cli.data.loader import _looks_like_remote_uri
|
||||
|
||||
assert _looks_like_remote_uri("s3://bucket/path")
|
||||
assert _looks_like_remote_uri("gs://bucket/file.jsonl")
|
||||
assert _looks_like_remote_uri("oci://bkt/x")
|
||||
assert not _looks_like_remote_uri("local.jsonl")
|
||||
assert not _looks_like_remote_uri("owner/dataset")
|
||||
assert not _looks_like_remote_uri("")
|
||||
assert not _looks_like_remote_uri(None) # type: ignore[arg-type]
|
||||
|
||||
def test_fsspec_missing_raises_friendly(self, monkeypatch, tmp_path):
|
||||
# Force fsspec ImportError
|
||||
monkeypatch.setitem(sys.modules, "fsspec", None)
|
||||
from soup_cli.config.schema import DataConfig
|
||||
from soup_cli.data.loader import _load_remote_dataset
|
||||
|
||||
cfg = DataConfig(train="s3://my-bucket/data.jsonl", format="alpaca", val_split=0)
|
||||
with pytest.raises(ImportError):
|
||||
_load_remote_dataset("s3://my-bucket/data.jsonl", cfg)
|
||||
|
||||
def test_invalid_remote_uri_rejected(self, monkeypatch):
|
||||
from soup_cli.config.schema import DataConfig
|
||||
from soup_cli.data.loader import _load_remote_dataset
|
||||
|
||||
cfg = DataConfig(train="s3://bucket/x", format="alpaca", val_split=0)
|
||||
# Userinfo embedded URI should be rejected by validate_remote_uri
|
||||
# BEFORE fsspec is even imported.
|
||||
with pytest.raises(ValueError):
|
||||
_load_remote_dataset("s3://user:pw@bucket/x", cfg)
|
||||
|
||||
def test_non_streaming_reads_jsonl(self, monkeypatch, tmp_path):
|
||||
from soup_cli.config.schema import DataConfig
|
||||
|
||||
# Build a fake fsspec that yields two JSONL rows
|
||||
rows = [
|
||||
'{"instruction": "hi", "output": "hello"}',
|
||||
'{"instruction": "bye", "output": "later"}',
|
||||
]
|
||||
|
||||
class FakeFile:
|
||||
def __enter__(self):
|
||||
return iter([r + "\n" for r in rows])
|
||||
|
||||
def __exit__(self, *a):
|
||||
return False
|
||||
|
||||
fake_fsspec = MagicMock()
|
||||
fake_fsspec.open.return_value = FakeFile()
|
||||
monkeypatch.setitem(sys.modules, "fsspec", fake_fsspec)
|
||||
|
||||
from soup_cli.data.loader import _load_remote_dataset
|
||||
|
||||
cfg = DataConfig(
|
||||
train="s3://bucket/data.jsonl", format="alpaca", val_split=0
|
||||
)
|
||||
result = _load_remote_dataset("s3://bucket/data.jsonl", cfg)
|
||||
assert "train" in result
|
||||
assert len(result["train"]) == 2
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #93 — Package-data fixture migration
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPackageDataFixtures:
|
||||
def test_fixtures_dir_exists(self):
|
||||
import soup_cli
|
||||
|
||||
pkg = Path(soup_cli.__file__).parent
|
||||
fixtures = pkg / "data" / "_fixtures"
|
||||
assert fixtures.is_dir(), f"missing package-data fixtures dir: {fixtures}"
|
||||
|
||||
def test_all_known_bundles_present(self):
|
||||
import soup_cli
|
||||
|
||||
pkg = Path(soup_cli.__file__).parent
|
||||
fixtures = pkg / "data" / "_fixtures"
|
||||
expected = {
|
||||
"alpaca_tiny.jsonl",
|
||||
"chat_preferences.jsonl",
|
||||
"dpo_sample.jsonl",
|
||||
"reasoning_math.jsonl",
|
||||
}
|
||||
present = {p.name for p in fixtures.glob("*.jsonl")}
|
||||
assert expected.issubset(present)
|
||||
|
||||
def test_bundle_source_prefers_package_data(self):
|
||||
from soup_cli.utils.demo_bundles import _bundle_source_path, get_bundle
|
||||
|
||||
bundle = get_bundle("alpaca_demo")
|
||||
src = _bundle_source_path(bundle)
|
||||
# Should resolve under soup_cli/data/_fixtures (not examples/data/).
|
||||
assert os.sep + "_fixtures" + os.sep in src, src
|
||||
|
||||
def test_bundle_content_valid_jsonl(self):
|
||||
from soup_cli.utils.demo_bundles import _bundle_source_path, get_bundle
|
||||
|
||||
for name in ("alpaca_demo", "sharegpt_demo", "dpo_demo", "grpo_demo"):
|
||||
bundle = get_bundle(name)
|
||||
src = _bundle_source_path(bundle)
|
||||
with open(src, encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
if line.strip():
|
||||
json.loads(line)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# #130 wiring smoke (CLI flag plumbing)
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDataDownloadHubFlag:
|
||||
def test_help_lists_hub_flag(self):
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from soup_cli.commands.data import app
|
||||
|
||||
result = CliRunner().invoke(app, ["download", "--help"])
|
||||
assert result.exit_code == 0
|
||||
assert "--hub" in result.output
|
||||
|
||||
def test_unknown_hub_rejected(self):
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from soup_cli.commands.data import app
|
||||
|
||||
result = CliRunner().invoke(
|
||||
app, ["download", "ds", "--hub", "evilcorp"]
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
assert "evilcorp" in result.output or "not supported" in result.output
|
||||
|
||||
def test_modelscope_hub_advisory(self):
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from soup_cli.commands.data import app
|
||||
|
||||
result = CliRunner().invoke(
|
||||
app, ["download", "ds", "--hub", "modelscope"]
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
assert "modelscope" in result.output
|
||||
assert "v0.53.9" in result.output or "download_repo" in result.output
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# pyproject extras
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPyprojectExtras:
|
||||
def test_trackers_extra_present(self):
|
||||
text = Path("pyproject.toml").read_text(encoding="utf-8")
|
||||
assert "trackers = [" in text
|
||||
assert "mlflow" in text
|
||||
assert "swanlab" in text
|
||||
assert "trackio" in text
|
||||
|
||||
def test_remote_extra_present(self):
|
||||
text = Path("pyproject.toml").read_text(encoding="utf-8")
|
||||
assert "remote = [" in text
|
||||
assert "fsspec" in text
|
||||
|
||||
def test_force_include_package_data(self):
|
||||
text = Path("pyproject.toml").read_text(encoding="utf-8")
|
||||
assert "_fixtures" in text
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Version bump
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestVersionBump:
|
||||
def test_init_version(self):
|
||||
import soup_cli
|
||||
|
||||
assert soup_cli.__version__ == "0.53.8"
|
||||
|
||||
def test_pyproject_version(self):
|
||||
text = Path("pyproject.toml").read_text(encoding="utf-8")
|
||||
assert 'version = "0.53.8"' in text
|
||||
Loading…
Reference in New Issue