From c4a27c1f9e3136c479510a4a1142c5cd75f65b4d Mon Sep 17 00:00:00 2001 From: Alpamys Date: Mon, 27 Apr 2026 20:07:14 +0500 Subject: [PATCH] docs(v0.33.0): version bump + What's New + security entries MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - pyproject.toml + soup_cli/__init__.py: 0.32.0 → 0.33.0 - README.md: replace What's New block with v0.33.0 highlights (10 bullets covering all 17 closed issues), add `soup can run` / `soup can publish` / multi-GPU one-command body sections, update Soup Cans security note to reflect format-version 1+2 + new consent gate + token resolution. - SECURITY.md: shift supported-version window (v0.33.x full, v0.32.x bug-fix only); add v0.33.0 "Live Wire" entry covering every CRITICAL+HIGH+MEDIUM security fix from this release wave. CLAUDE.md / plan.md updates land in the next commit (those are gitignored / project-internal so kept separate). Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 31 +++++++++++++++++++++++-------- SECURITY.md | 7 ++++--- pyproject.toml | 2 +- soup_cli/__init__.py | 2 +- 4 files changed, 29 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index a5284fb..040a3cc 100644 --- a/README.md +++ b/README.md @@ -40,12 +40,18 @@ soup train Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases). -- **LR Range Finder** — `soup train --find-lr` runs a fast.ai-style geometric LR sweep and writes a JSON report with the recommended learning rate. Pre-flight tuning before the real run. -- **Auto warmup schedule** — set `training.warmup_auto: true` and Soup picks `warmup_steps` from your dataset size × epochs × `warmup_ratio`, clamped to a sane range. -- **Auto mixed-precision** — set `training.auto_mixed_precision: true` and Soup picks `bf16` (Ampere+) or `fp16` (Turing or known fp16-stable models like Qwen2 / Phi-3.5) based on your GPU and base model. -- **Loss spike auto-recovery** — extends the watchdog: when loss spikes, decay LR and resume instead of dying. `loss_spike_recovery: true` on top of `loss_watchdog: true`. -- **Convergence detector** — surfaces "loss has plateaued — early-stop or cut LR" advice via `convergence_detection: true`. Catches stuck training before you waste GPU hours. -- **VRAM-pressure advisory** — `grad_accum_auto_tune: true` records peak memory per step and recommends a new (batch, accum) pair when pressure crosses your threshold. +**v0.33.0 — Live Wire**: closes 17 issues across 6 internal Parts. Every "API + flag (v0.30.0)" / "wired live in v0.x.1" deferral from the last six minor releases is now actually live. + +- **`soup can run` + `soup can publish`** — end-to-end: extract a `.can`, run the embedded config via `soup train`, optionally deploy. `soup can publish` uploads a `.can` to HF Hub as a dataset (`repo_type='dataset'`, can-format-v1 tag). Manifest schema bumped to v2 with optional `deploy_targets` (ollama / gguf / vllm). +- **Multi-GPU one command** — `soup train --gpus 2` now auto-reexecs under `accelerate launch` instead of just printing the command. `--no-reexec` opt-out preserved. Critical flags (`--fsdp`, `--deepspeed`, `--resume`, `--wandb`, `--tensorboard`, `--yes`) forwarded to the reexec'd run. +- **Live eval gate scoring** — `soup eval gate` and `soup eval quant-check` no longer return `score=1.0` stubs for judge / benchmark task types. Backend failures surface as `score=None, error=str(exc)` so a broken eval never silently passes the gate. +- **Structured output + auto-quant live on `soup serve`** — `--structured-output json --json-schema s.json` builds a real `LogitsProcessor` (via outlines or lm-format-enforcer). `--auto-quant` runs a tiny eval over candidate quantisations and picks (score, -latency). +- **DeepSpeed-MII live serve** — `soup serve --backend mii` now starts a real OpenAI-compatible HTTP server (was a stub-warning + exit-1 in v0.27.0). +- **Multi-trainer v0.28.0 features** — `use_cut_ce`, `quantization_aware="fp8"`, `kernel_auto_compose`, `activation_offloading` now wired for SFT / DPO / Pretrain trainers (previously SFT-only). +- **Live `--find-lr`** — replaces the synthetic stub curve with a real in-process LR-sweep training loop. +- **Spike recovery + grad-accum advisory** — `loss_spike_recovery: true` writes a `spike_recovery.json` hint with the decayed LR for re-launch. `grad_accum_auto_tune: true` prints a (batch, accum) recommendation when VRAM pressure crosses the threshold. +- **`soup eval custom --attach-to-registry ` + `soup export --registry-id `** — auto-attach eval results / exported artifacts to a Local Model Registry entry as `eval_results` / `gguf` / `awq` / `gptq` / `onnx` / `tensorrt` artifacts. +- **RLVR sandbox + checkpoint prune hardening** — `code_exec_reward` adds OS-level isolation (Linux `unshare`, macOS `sandbox-exec`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat` + `S_ISLNK` + `onerror`-abort `rmtree`. ## Why Soup? @@ -1313,9 +1319,16 @@ soup can verify ./llama31-chat.can # Fork with modifications (dotted-path overrides) and re-pack soup can fork ./llama31-chat.can --out ./llama31-chat-hot.can \ --modify training.lr=5e-5 --modify training.epochs=5 + +# Run a .can end-to-end: extract → train (→ optional deploy) +soup can run ./llama31-chat.can --yes +soup can run ./llama31-chat.can --yes --deploy --env-capture ./env.txt + +# Publish a .can to HF Hub as a dataset +soup can publish ./llama31-chat.can --hf-hub me/llama31-chat-recipe ``` -**Security** — tar extraction uses `filter="data"` on Python 3.12+ with symlink/hardlink rejection fallback for older runtimes. Size cap: 100 MB. `DataRef.url` must be HTTPS. Fork overrides reject dunder keys (`__class__`, `__init__`) and null bytes. Manifest format version is pinned to `1`. +**Security** — tar extraction uses `filter="data"` on Python 3.12+ with symlink/hardlink rejection fallback for older runtimes. Size cap: 100 MB. `DataRef.url` must be HTTPS. Fork overrides reject dunder keys (`__class__`, `__init__`) and null bytes. Manifest format version supports `1` and `2` (additive bump in v0.33.0 added `deploy_targets`). `soup can run` requires `--yes` (mandatory consent — auto-downloads data + auto-trains). `soup can publish` validates `repo_id` and resolves the HF token via env / cache files; commit messages are first-line + 200-char capped. ## Batch Inference @@ -2155,7 +2168,7 @@ soup eval human --input p.jsonl Human A/B evaluation soup eval gate --suite gate.yaml Run eval-gate suite standalonesoup eval quant-check --before X --after Y --tasks t.jsonl Before/after quantsoup serve --model ./output --port 8000 OpenAI-compatible API server soup serve --model ./output --backend vllm vLLM backend (2-4x throughput) soup serve --model ./output --backend sglang SGLang backend -soup serve --model ./output --backend mii DeepSpeed-MII backend (registered; live in v0.27.1) +soup serve --model ./output --backend mii DeepSpeed-MII backend (live) soup serve --model ./output --speculative-decoding draft-model Speculative decoding soup serve --model --auto-spec Auto-pair draft model for speculative decoding soup serve --model --backend vllm --prefix-cache vLLM prefix caching (RAG/agent) @@ -2220,6 +2233,8 @@ soup registry delete --yes Remove entry (cascades) soup history Lineage DAG tree for a namesoup can pack --entry-id --out r.can Pack registry entry as .cansoup can inspect r.can Preview manifest without extracting soup can verify r.can Verify schema + config parseability soup can fork r.can --out fork.can --modify training.lr=5e-5 Fork + re-pack +soup can run r.can --yes [--deploy] [--env-capture env.txt] Run a .can end-to-end +soup can publish r.can --hf-hub user/name Publish .can to HF Hub as dataset soup runs List training runs soup runs show Run details + loss graph soup runs compare Compare two runs diff --git a/SECURITY.md b/SECURITY.md index 2cdcbcf..75933a8 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -9,9 +9,9 @@ We provide security updates for the following versions: - **Versions older than 3 minor versions:** No support Example: -- v0.32.0-0.32.x -- Full support (latest) -- v0.31.0-0.31.x -- Bug-fix support only -- v0.30.x and below -- No support +- v0.33.0-0.33.x -- Full support (latest) +- v0.32.0-0.32.x -- Bug-fix support only +- v0.31.x and below -- No support ## Reporting a Vulnerability @@ -140,6 +140,7 @@ No known critical vulnerabilities in current releases. - **v0.29.0 — HF Hub Deep Integration**: `HF_ENDPOINT` SSRF-hardened — scheme allowlist (http/https), null-byte rejection, `0.0.0.0` explicitly rejected, plain-HTTP only permitted for loopback (`localhost`/`127.0.0.1`/`::1`), RFC1918 / link-local / cloud-metadata (169.254.x) IPs rejected via `ipaddress.ip_address`; repo ID regex `[A-Za-z0-9][A-Za-z0-9._-]{0,95}` per component, ≤200 chars total, null-byte / whitespace / `..` / leading-`/` rejection (applied to `push --repo`, `train --push-as`, `data push --hf-dataset`, `deploy hf-space --model/--space`); collection slug `owner/slug-hash` regex-validated, ≤256 chars; HF token resolution single-sourced in `utils/hf.resolve_token` (env > cached login), explicit non-printable tokens rejected, `push --token` flag deprecated with yellow warning; `soup push --model` confined to cwd via `is_under_cwd` (prevents crafted `soup.yaml output:` from uploading system files); auto-push checkpoint `allow_patterns` restricts uploaded files to `*.safetensors`/`*.bin`/`*.pt`/`*.json`/`tokenizer*`/`trainer_state.json`/`training_args.bin`/`README.md` (keeps `.env` and source files out of auto-pushed branches); `prepare_hf_resume` enforces cwd containment and passes `local_dir_use_symlinks=False` (defeats symlink-based FS escape on older `huggingface_hub`); commit messages stripped to first line and capped at 200 chars (prevents multi-line injection into public HF commit history); `_render_eval_scorecard` neutralises `|`/`[`/`]`/`(`/`)`/`!`/newlines/tabs/`<`/`>` in task names and non-numeric scores; `data_lineage` HTML-escaped (defeats XSS on HF Hub README viewer); `render_space_template` validates `model_repo` via `validate_repo_id` before substitution into rendered `app.py` (crafted repo id cannot inject Python code); `HFPushCallback` uses sticky `_repo_failed` flag to short-circuit retries after hard failure (no log spam, no wasted API calls); `add_to_collection` prefers HfHubHTTPError 409 detection over string-match for duplicate handling - **v0.30.0 — Inference Excellence**: OTLP endpoint SSRF-hardened matching v0.29.0 `HF_ENDPOINT` (scheme allowlist, `0.0.0.0` rejected, RFC1918 / link-local / cloud-metadata via `ipaddress.ip_address`, plain HTTP loopback-only); `pick_draft_model` rejects URL-scheme target names (`http://`/`https://`/`file://`), null bytes, names >200 chars; `validate_regex_pattern` length-capped at 2048 + null-byte rejection + must compile; `validate_json_schema` dict shape + 64KB serialised cap + required `type` field; `--json-schema` file path confined to cwd via shared `utils/paths.is_under_cwd`; `--structured-output json` requires `--json-schema` (fail-fast prevents silent no-op); FastAPI `/v1/adapters/activate/{name}` pattern `^[a-zA-Z0-9][a-zA-Z0-9\-]*$` enforced before handler runs; activate/deactivate state protected by `threading.Lock` (no race on concurrent hot-swap); `/v1/adapters` response omits filesystem paths (names + active flag only); CORS on transformers backend restricted to loopback origins (`^https?://(localhost|127\.0\.0\.1)(:\d+)?$`) since hot-swap endpoints mutate state without auth; `auto_quant.Candidate` name regex + score `[0.0, 1.0]` + finite-float check + non-negative latency; `pick_best` generator-safe (materialises to list) so error-message count is accurate; OTel span context uses `contextlib.ExitStack` so `__exit__` sees real exception info (spans correctly marked error on HTTPException); `record_latency` always runs in `finally` so tail-latency percentiles include failure paths; `build_tracer` idempotent — only installs provider when current is `ProxyTracerProvider`/`NoOpTracerProvider` (preserves operator-supplied instrumentation) - **v0.32.0 — Training Stability & Auto-Tuning**: `--find-lr-output` containment via shared `utils/paths.is_under_cwd` (prevents writes outside cwd); `save_lr_finder_report` rejects NaN / Infinity floats in `lrs` / `losses` and serialises with `allow_nan=False` (keeps the report parser-safe); `compute_lr_schedule` rejects non-positive `start_lr`, inverted ranges, and `num_steps` outside `[2, 10_000]`; `pick_mixed_precision` rejects empty / null-byte / >200-char model names and resolves multi-version quirks (`qwen2.5` vs `qwen2`, `phi-3.5` vs `phi-3`) by longest-substring-first iteration so an added family can never accidentally make a more-specific entry dead code; `compute_warmup_steps` clamps to `[10, 1000]` with a `ratio==0.0` short-circuit matching HF Trainer's "no warmup" convention; `SpikeRecoveryStrategy` is `@dataclass(frozen=True)` (post-construction mutation cannot bypass validation), `max_attempts ∈ [1, 10]`, `lr_decay ∈ (0, 1)`, `min_lr > 0`; cross-validator `_validate_spike_recovery_requires_watchdog` rejects `loss_spike_recovery=true, loss_watchdog=false` at config-load (fails fast instead of never triggering); `convergence_window ∈ [5, 10_000]`, `convergence_rel_tol ∈ (0, 1]`, `recommend_action` reuses `detect_plateau` so plateau heuristic stays single-source-of-truth; `GradAccumMonitor.recommend()` caps doubled `accum` at `MAX_ACCUM=1024` so a runaway advisory loop cannot blow up DataLoader prefetch; `generate_config` validates BOTH the YAML output path AND the embedded `decisions["output"]` field via `is_under_cwd` (closes the gap where a crafted `decisions["output"]="../../etc"` would have silently propagated into the rendered YAML) +- **v0.33.0 — Live Wire**: RLVR `code_exec_reward` adds OS-level isolation (Linux best-effort `os.unshare(CLONE_NEWUSER|CLONE_NEWNET|CLONE_NEWPID)`, macOS `sandbox-exec` with default-deny `MACOS_SANDBOX_PROFILE` narrowed to a 3-name `mach-lookup` allowlist to prevent DNS / NSURLSession bypass of `(deny network*)`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat + S_ISLNK` + `shutil.rmtree(onerror=_abort_on_symlink)` so a symlink encountered mid-walk aborts rather than escapes; `run_gate` wraps each task scorer in a typed `try/except` so backend failures produce `score=None, error=str(exc)` (never silent `score=1.0`); `_parse_judge_url` removes the bare `http://` catch-all (defence-in-depth after the Pydantic GateTask validator); `soup can run` requires `--yes` or explicit consent callback and raises `ValueError` (not `PermissionError`, which is an `OSError` subclass that broad `except` blocks would swallow); GGUF `rglob` result for ollama deploy is `realpath+commonpath` checked against extract_dir (prevents symlink escape from a crafted can); `DeployTarget.path` validator normalises mixed `\\`/`/` separators before splitting (closes a Windows `..` bypass); `CAN_FORMAT_VERSION` 1→2 (additive — v1 still loads); `soup can publish` validates `repo_id` via `utils/hf.validate_repo_id`, resolves token via `resolve_token`, sanitises commit messages (first-line, 200-char cap), uses HTTPS-only HfApi; `_write_spike_recovery_hint` adds `is_under_cwd` containment check on `args.output_dir` from raw HF `TrainingArguments`; `lookup_entry_by_output_dir` emits `ResourceWarning` when 1000-row scan limit is hit (no silent miss); `CrossDocCollator` no longer mutates input feature dicts (HF Dataset rows are cached and reused — mutation broke subsequent batches); `Candidate` rejects `bool` in `score`/`latency_ms` (was sneaking past `int` isinstance check); `evaluate_candidate` latency mean now divides by *completed* prompts (excludes crashed) so a broken candidate isn't artificially fast; `auto_quant.run_auto_quant_picker` soft-falls-back to highest-scored candidate when no candidate clears `min_score` (server still binds); `build_logits_processors` returns `[]` when neither `outlines` nor `lm-format-enforcer` is installed (server degrades to free-form rather than 500); MII server uses loopback-only CORS, max_tokens cap [1, 16384], stream rejection, generic 500 with no stack-trace leak; `os.execvp` auto-reexec uses list args (no shell), all forwarded flags pre-validated; `cleanup_extract_dir` uses `os.path.commonpath` (Windows-safe) instead of `startswith`; `_run_subprocess` catches `TimeoutExpired` and returns rc=124 (coreutils convention) instead of an unhandled traceback; new `eval_results` and `tensorrt` artifact kinds in `RegistryStore._VALID_KINDS` ## Security Scanning diff --git a/pyproject.toml b/pyproject.toml index a811354..1a36b45 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "soup-cli" -version = "0.32.0" +version = "0.33.0" description = "Fine-tune LLMs in one command. No SSH, no config hell." readme = "README.md" license = "Apache-2.0" diff --git a/soup_cli/__init__.py b/soup_cli/__init__.py index 84cfd36..c771edf 100644 --- a/soup_cli/__init__.py +++ b/soup_cli/__init__.py @@ -1,3 +1,3 @@ """Soup CLI — Fine-tune LLMs in one command.""" -__version__ = "0.32.0" +__version__ = "0.33.0"