docs(v0.33.0): version bump + What's New + security entries

- pyproject.toml + soup_cli/__init__.py: 0.32.0 → 0.33.0
- README.md: replace What's New block with v0.33.0 highlights (10
  bullets covering all 17 closed issues), add `soup can run` /
  `soup can publish` / multi-GPU one-command body sections, update
  Soup Cans security note to reflect format-version 1+2 + new
  consent gate + token resolution.
- SECURITY.md: shift supported-version window (v0.33.x full,
  v0.32.x bug-fix only); add v0.33.0 "Live Wire" entry covering
  every CRITICAL+HIGH+MEDIUM security fix from this release wave.

CLAUDE.md / plan.md updates land in the next commit (those are
gitignored / project-internal so kept separate).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alpamys 2026-04-27 20:07:14 +05:00
parent ff55e751ab
commit c4a27c1f9e
4 changed files with 29 additions and 13 deletions

View File

@ -40,12 +40,18 @@ soup train
Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
- **LR Range Finder**`soup train --find-lr` runs a fast.ai-style geometric LR sweep and writes a JSON report with the recommended learning rate. Pre-flight tuning before the real run.
- **Auto warmup schedule** — set `training.warmup_auto: true` and Soup picks `warmup_steps` from your dataset size × epochs × `warmup_ratio`, clamped to a sane range.
- **Auto mixed-precision** — set `training.auto_mixed_precision: true` and Soup picks `bf16` (Ampere+) or `fp16` (Turing or known fp16-stable models like Qwen2 / Phi-3.5) based on your GPU and base model.
- **Loss spike auto-recovery** — extends the watchdog: when loss spikes, decay LR and resume instead of dying. `loss_spike_recovery: true` on top of `loss_watchdog: true`.
- **Convergence detector** — surfaces "loss has plateaued — early-stop or cut LR" advice via `convergence_detection: true`. Catches stuck training before you waste GPU hours.
- **VRAM-pressure advisory**`grad_accum_auto_tune: true` records peak memory per step and recommends a new (batch, accum) pair when pressure crosses your threshold.
**v0.33.0 — Live Wire**: closes 17 issues across 6 internal Parts. Every "API + flag (v0.30.0)" / "wired live in v0.x.1" deferral from the last six minor releases is now actually live.
- **`soup can run` + `soup can publish`** — end-to-end: extract a `.can`, run the embedded config via `soup train`, optionally deploy. `soup can publish` uploads a `.can` to HF Hub as a dataset (`repo_type='dataset'`, can-format-v1 tag). Manifest schema bumped to v2 with optional `deploy_targets` (ollama / gguf / vllm).
- **Multi-GPU one command**`soup train --gpus 2` now auto-reexecs under `accelerate launch` instead of just printing the command. `--no-reexec` opt-out preserved. Critical flags (`--fsdp`, `--deepspeed`, `--resume`, `--wandb`, `--tensorboard`, `--yes`) forwarded to the reexec'd run.
- **Live eval gate scoring**`soup eval gate` and `soup eval quant-check` no longer return `score=1.0` stubs for judge / benchmark task types. Backend failures surface as `score=None, error=str(exc)` so a broken eval never silently passes the gate.
- **Structured output + auto-quant live on `soup serve`**`--structured-output json --json-schema s.json` builds a real `LogitsProcessor` (via outlines or lm-format-enforcer). `--auto-quant` runs a tiny eval over candidate quantisations and picks (score, -latency).
- **DeepSpeed-MII live serve**`soup serve --backend mii` now starts a real OpenAI-compatible HTTP server (was a stub-warning + exit-1 in v0.27.0).
- **Multi-trainer v0.28.0 features**`use_cut_ce`, `quantization_aware="fp8"`, `kernel_auto_compose`, `activation_offloading` now wired for SFT / DPO / Pretrain trainers (previously SFT-only).
- **Live `--find-lr`** — replaces the synthetic stub curve with a real in-process LR-sweep training loop.
- **Spike recovery + grad-accum advisory**`loss_spike_recovery: true` writes a `spike_recovery.json` hint with the decayed LR for re-launch. `grad_accum_auto_tune: true` prints a (batch, accum) recommendation when VRAM pressure crosses the threshold.
- **`soup eval custom --attach-to-registry <id>` + `soup export --registry-id <id>`** — auto-attach eval results / exported artifacts to a Local Model Registry entry as `eval_results` / `gguf` / `awq` / `gptq` / `onnx` / `tensorrt` artifacts.
- **RLVR sandbox + checkpoint prune hardening**`code_exec_reward` adds OS-level isolation (Linux `unshare`, macOS `sandbox-exec`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat` + `S_ISLNK` + `onerror`-abort `rmtree`.
## Why Soup?
@ -1313,9 +1319,16 @@ soup can verify ./llama31-chat.can
# Fork with modifications (dotted-path overrides) and re-pack
soup can fork ./llama31-chat.can --out ./llama31-chat-hot.can \
--modify training.lr=5e-5 --modify training.epochs=5
# Run a .can end-to-end: extract → train (→ optional deploy)
soup can run ./llama31-chat.can --yes
soup can run ./llama31-chat.can --yes --deploy --env-capture ./env.txt
# Publish a .can to HF Hub as a dataset
soup can publish ./llama31-chat.can --hf-hub me/llama31-chat-recipe
```
**Security** — tar extraction uses `filter="data"` on Python 3.12+ with symlink/hardlink rejection fallback for older runtimes. Size cap: 100 MB. `DataRef.url` must be HTTPS. Fork overrides reject dunder keys (`__class__`, `__init__`) and null bytes. Manifest format version is pinned to `1`.
**Security** — tar extraction uses `filter="data"` on Python 3.12+ with symlink/hardlink rejection fallback for older runtimes. Size cap: 100 MB. `DataRef.url` must be HTTPS. Fork overrides reject dunder keys (`__class__`, `__init__`) and null bytes. Manifest format version supports `1` and `2` (additive bump in v0.33.0 added `deploy_targets`). `soup can run` requires `--yes` (mandatory consent — auto-downloads data + auto-trains). `soup can publish` validates `repo_id` and resolves the HF token via env / cache files; commit messages are first-line + 200-char capped.
## Batch Inference
@ -2155,7 +2168,7 @@ soup eval human --input p.jsonl Human A/B evaluation
soup eval gate --suite gate.yaml Run eval-gate suite standalonesoup eval quant-check --before X --after Y --tasks t.jsonl Before/after quantsoup serve --model ./output --port 8000 OpenAI-compatible API server
soup serve --model ./output --backend vllm vLLM backend (2-4x throughput)
soup serve --model ./output --backend sglang SGLang backend
soup serve --model ./output --backend mii DeepSpeed-MII backend (registered; live in v0.27.1)
soup serve --model ./output --backend mii DeepSpeed-MII backend (live)
soup serve --model ./output --speculative-decoding draft-model Speculative decoding
soup serve --model <m> --auto-spec Auto-pair draft model for speculative decoding
soup serve --model <m> --backend vllm --prefix-cache vLLM prefix caching (RAG/agent)
@ -2220,6 +2233,8 @@ soup registry delete <ref> --yes Remove entry (cascades)
soup history <name> Lineage DAG tree for a namesoup can pack --entry-id <id> --out r.can Pack registry entry as .cansoup can inspect r.can Preview manifest without extracting
soup can verify r.can Verify schema + config parseability
soup can fork r.can --out fork.can --modify training.lr=5e-5 Fork + re-pack
soup can run r.can --yes [--deploy] [--env-capture env.txt] Run a .can end-to-end
soup can publish r.can --hf-hub user/name Publish .can to HF Hub as dataset
soup runs List training runs
soup runs show <run_id> Run details + loss graph
soup runs compare <run_1> <run_2> Compare two runs

View File

@ -9,9 +9,9 @@ We provide security updates for the following versions:
- **Versions older than 3 minor versions:** No support
Example:
- v0.32.0-0.32.x -- Full support (latest)
- v0.31.0-0.31.x -- Bug-fix support only
- v0.30.x and below -- No support
- v0.33.0-0.33.x -- Full support (latest)
- v0.32.0-0.32.x -- Bug-fix support only
- v0.31.x and below -- No support
## Reporting a Vulnerability
@ -140,6 +140,7 @@ No known critical vulnerabilities in current releases.
- **v0.29.0 — HF Hub Deep Integration**: `HF_ENDPOINT` SSRF-hardened — scheme allowlist (http/https), null-byte rejection, `0.0.0.0` explicitly rejected, plain-HTTP only permitted for loopback (`localhost`/`127.0.0.1`/`::1`), RFC1918 / link-local / cloud-metadata (169.254.x) IPs rejected via `ipaddress.ip_address`; repo ID regex `[A-Za-z0-9][A-Za-z0-9._-]{0,95}` per component, ≤200 chars total, null-byte / whitespace / `..` / leading-`/` rejection (applied to `push --repo`, `train --push-as`, `data push --hf-dataset`, `deploy hf-space --model/--space`); collection slug `owner/slug-hash` regex-validated, ≤256 chars; HF token resolution single-sourced in `utils/hf.resolve_token` (env > cached login), explicit non-printable tokens rejected, `push --token` flag deprecated with yellow warning; `soup push --model` confined to cwd via `is_under_cwd` (prevents crafted `soup.yaml output:` from uploading system files); auto-push checkpoint `allow_patterns` restricts uploaded files to `*.safetensors`/`*.bin`/`*.pt`/`*.json`/`tokenizer*`/`trainer_state.json`/`training_args.bin`/`README.md` (keeps `.env` and source files out of auto-pushed branches); `prepare_hf_resume` enforces cwd containment and passes `local_dir_use_symlinks=False` (defeats symlink-based FS escape on older `huggingface_hub`); commit messages stripped to first line and capped at 200 chars (prevents multi-line injection into public HF commit history); `_render_eval_scorecard` neutralises `|`/`[`/`]`/`(`/`)`/`!`/newlines/tabs/`<`/`>` in task names and non-numeric scores; `data_lineage` HTML-escaped (defeats XSS on HF Hub README viewer); `render_space_template` validates `model_repo` via `validate_repo_id` before substitution into rendered `app.py` (crafted repo id cannot inject Python code); `HFPushCallback` uses sticky `_repo_failed` flag to short-circuit retries after hard failure (no log spam, no wasted API calls); `add_to_collection` prefers HfHubHTTPError 409 detection over string-match for duplicate handling
- **v0.30.0 — Inference Excellence**: OTLP endpoint SSRF-hardened matching v0.29.0 `HF_ENDPOINT` (scheme allowlist, `0.0.0.0` rejected, RFC1918 / link-local / cloud-metadata via `ipaddress.ip_address`, plain HTTP loopback-only); `pick_draft_model` rejects URL-scheme target names (`http://`/`https://`/`file://`), null bytes, names >200 chars; `validate_regex_pattern` length-capped at 2048 + null-byte rejection + must compile; `validate_json_schema` dict shape + 64KB serialised cap + required `type` field; `--json-schema` file path confined to cwd via shared `utils/paths.is_under_cwd`; `--structured-output json` requires `--json-schema` (fail-fast prevents silent no-op); FastAPI `/v1/adapters/activate/{name}` pattern `^[a-zA-Z0-9][a-zA-Z0-9\-]*$` enforced before handler runs; activate/deactivate state protected by `threading.Lock` (no race on concurrent hot-swap); `/v1/adapters` response omits filesystem paths (names + active flag only); CORS on transformers backend restricted to loopback origins (`^https?://(localhost|127\.0\.0\.1)(:\d+)?$`) since hot-swap endpoints mutate state without auth; `auto_quant.Candidate` name regex + score `[0.0, 1.0]` + finite-float check + non-negative latency; `pick_best` generator-safe (materialises to list) so error-message count is accurate; OTel span context uses `contextlib.ExitStack` so `__exit__` sees real exception info (spans correctly marked error on HTTPException); `record_latency` always runs in `finally` so tail-latency percentiles include failure paths; `build_tracer` idempotent — only installs provider when current is `ProxyTracerProvider`/`NoOpTracerProvider` (preserves operator-supplied instrumentation)
- **v0.32.0 — Training Stability & Auto-Tuning**: `--find-lr-output` containment via shared `utils/paths.is_under_cwd` (prevents writes outside cwd); `save_lr_finder_report` rejects NaN / Infinity floats in `lrs` / `losses` and serialises with `allow_nan=False` (keeps the report parser-safe); `compute_lr_schedule` rejects non-positive `start_lr`, inverted ranges, and `num_steps` outside `[2, 10_000]`; `pick_mixed_precision` rejects empty / null-byte / >200-char model names and resolves multi-version quirks (`qwen2.5` vs `qwen2`, `phi-3.5` vs `phi-3`) by longest-substring-first iteration so an added family can never accidentally make a more-specific entry dead code; `compute_warmup_steps` clamps to `[10, 1000]` with a `ratio==0.0` short-circuit matching HF Trainer's "no warmup" convention; `SpikeRecoveryStrategy` is `@dataclass(frozen=True)` (post-construction mutation cannot bypass validation), `max_attempts ∈ [1, 10]`, `lr_decay ∈ (0, 1)`, `min_lr > 0`; cross-validator `_validate_spike_recovery_requires_watchdog` rejects `loss_spike_recovery=true, loss_watchdog=false` at config-load (fails fast instead of never triggering); `convergence_window ∈ [5, 10_000]`, `convergence_rel_tol ∈ (0, 1]`, `recommend_action` reuses `detect_plateau` so plateau heuristic stays single-source-of-truth; `GradAccumMonitor.recommend()` caps doubled `accum` at `MAX_ACCUM=1024` so a runaway advisory loop cannot blow up DataLoader prefetch; `generate_config` validates BOTH the YAML output path AND the embedded `decisions["output"]` field via `is_under_cwd` (closes the gap where a crafted `decisions["output"]="../../etc"` would have silently propagated into the rendered YAML)
- **v0.33.0 — Live Wire**: RLVR `code_exec_reward` adds OS-level isolation (Linux best-effort `os.unshare(CLONE_NEWUSER|CLONE_NEWNET|CLONE_NEWPID)`, macOS `sandbox-exec` with default-deny `MACOS_SANDBOX_PROFILE` narrowed to a 3-name `mach-lookup` allowlist to prevent DNS / NSURLSession bypass of `(deny network*)`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat + S_ISLNK` + `shutil.rmtree(onerror=_abort_on_symlink)` so a symlink encountered mid-walk aborts rather than escapes; `run_gate` wraps each task scorer in a typed `try/except` so backend failures produce `score=None, error=str(exc)` (never silent `score=1.0`); `_parse_judge_url` removes the bare `http://` catch-all (defence-in-depth after the Pydantic GateTask validator); `soup can run` requires `--yes` or explicit consent callback and raises `ValueError` (not `PermissionError`, which is an `OSError` subclass that broad `except` blocks would swallow); GGUF `rglob` result for ollama deploy is `realpath+commonpath` checked against extract_dir (prevents symlink escape from a crafted can); `DeployTarget.path` validator normalises mixed `\\`/`/` separators before splitting (closes a Windows `..` bypass); `CAN_FORMAT_VERSION` 1→2 (additive — v1 still loads); `soup can publish` validates `repo_id` via `utils/hf.validate_repo_id`, resolves token via `resolve_token`, sanitises commit messages (first-line, 200-char cap), uses HTTPS-only HfApi; `_write_spike_recovery_hint` adds `is_under_cwd` containment check on `args.output_dir` from raw HF `TrainingArguments`; `lookup_entry_by_output_dir` emits `ResourceWarning` when 1000-row scan limit is hit (no silent miss); `CrossDocCollator` no longer mutates input feature dicts (HF Dataset rows are cached and reused — mutation broke subsequent batches); `Candidate` rejects `bool` in `score`/`latency_ms` (was sneaking past `int` isinstance check); `evaluate_candidate` latency mean now divides by *completed* prompts (excludes crashed) so a broken candidate isn't artificially fast; `auto_quant.run_auto_quant_picker` soft-falls-back to highest-scored candidate when no candidate clears `min_score` (server still binds); `build_logits_processors` returns `[]` when neither `outlines` nor `lm-format-enforcer` is installed (server degrades to free-form rather than 500); MII server uses loopback-only CORS, max_tokens cap [1, 16384], stream rejection, generic 500 with no stack-trace leak; `os.execvp` auto-reexec uses list args (no shell), all forwarded flags pre-validated; `cleanup_extract_dir` uses `os.path.commonpath` (Windows-safe) instead of `startswith`; `_run_subprocess` catches `TimeoutExpired` and returns rc=124 (coreutils convention) instead of an unhandled traceback; new `eval_results` and `tensorrt` artifact kinds in `RegistryStore._VALID_KINDS`
## Security Scanning

View File

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "soup-cli"
version = "0.32.0"
version = "0.33.0"
description = "Fine-tune LLMs in one command. No SSH, no config hell."
readme = "README.md"
license = "Apache-2.0"

View File

@ -1,3 +1,3 @@
"""Soup CLI — Fine-tune LLMs in one command."""
__version__ = "0.32.0"
__version__ = "0.33.0"