From 0ddefc5c6f4a34851801a668fc0f845e1ad1d408 Mon Sep 17 00:00:00 2001
From: Alpamys <vpn.alpamys@gmail.com>
Date: Wed, 13 May 2026 20:11:18 +0500
Subject: [PATCH] docs: SECURITY.md v0.53.7 per-version fix note
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Audit follow-up — v0.53.7 was added to the supported-version list but
the per-version security fix paragraph was missing. Add the dense
hardening summary matching the v0.53.6 / v0.53.5 style: bash 501 revert
rationale, atomic checkpoint write, _node_seed lstat-on-raw-path,
failed_reason redaction, Bearer auth gate on tool endpoints, SSE header
injection defence, vLLM /v1/messages loopback CORS, atomic Arrow save,
cache-hash gate on pre_tokenized short-circuit, prompt_strategy
trusted-input limitation. Docs-only — no version bump.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
 SECURITY.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/SECURITY.md b/SECURITY.md
index bf936b9..aad66d3 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -153,6 +153,7 @@ No known critical vulnerabilities in current releases.
 - **v0.32.0 — Training Stability & Auto-Tuning**: `--find-lr-output` containment via shared `utils/paths.is_under_cwd` (prevents writes outside cwd); `save_lr_finder_report` rejects NaN / Infinity floats in `lrs` / `losses` and serialises with `allow_nan=False` (keeps the report parser-safe); `compute_lr_schedule` rejects non-positive `start_lr`, inverted ranges, and `num_steps` outside `[2, 10_000]`; `pick_mixed_precision` rejects empty / null-byte / >200-char model names and resolves multi-version quirks (`qwen2.5` vs `qwen2`, `phi-3.5` vs `phi-3`) by longest-substring-first iteration so an added family can never accidentally make a more-specific entry dead code; `compute_warmup_steps` clamps to `[10, 1000]` with a `ratio==0.0` short-circuit matching HF Trainer's "no warmup" convention; `SpikeRecoveryStrategy` is `@dataclass(frozen=True)` (post-construction mutation cannot bypass validation), `max_attempts ∈ [1, 10]`, `lr_decay ∈ (0, 1)`, `min_lr > 0`; cross-validator `_validate_spike_recovery_requires_watchdog` rejects `loss_spike_recovery=true, loss_watchdog=false` at config-load (fails fast instead of never triggering); `convergence_window ∈ [5, 10_000]`, `convergence_rel_tol ∈ (0, 1]`, `recommend_action` reuses `detect_plateau` so plateau heuristic stays single-source-of-truth; `GradAccumMonitor.recommend()` caps doubled `accum` at `MAX_ACCUM=1024` so a runaway advisory loop cannot blow up DataLoader prefetch; `generate_config` validates BOTH the YAML output path AND the embedded `decisions["output"]` field via `is_under_cwd` (closes the gap where a crafted `decisions["output"]="../../etc"` would have silently propagated into the rendered YAML)
 - **v0.34.0 — Observability & Dev UX**: `.crash` bundle generator (`utils/crash.py`) recursively redacts `hf_*` / `sk-*` / `Bearer …` token-shaped strings in any captured `config` and metric tail before serialisation, so a `.crash` file shared on a public GitHub issue cannot leak credentials; `output_dir` is reduced to `os.path.basename` so `$HOME` doesn't leak; `write_crash_bundle` uses `os.path.realpath + commonpath` for cwd containment (Windows-safe; raises `ValueError` not `PermissionError` so callers cannot silently swallow with `except OSError`); filename appends `secrets.token_hex(4)` so two crashes in the same UTC second don't collide; bundle truncated to `MAX_BUNDLE_BYTES=1_000_000`. `train.py` crash-write surfaces failures to the user (no silent missing-bundle). `profiling.py` `resolve_trace_path` rejects empty / `.` / `..` / `/` / `\\` / null-byte `run_id` (closes the `output_dir/profiles/../trace.json` escape) and uses `os.path.realpath + is_under_cwd`; profiles dir is created only on successful torch import (no stale empty dirs on torch-less CI). `tracker.get_run` LIKE-prefix match escapes `%` / `_` / `\\` and uses `ESCAPE '\\'` so a crafted `run_id` cannot widen the match (mirrors v0.26.0 registry policy). Lazy schema migration (`_ensure_schema`) tolerates the "duplicate column" race when two CLI processes start simultaneously on a fresh DB (fork-based multi-GPU training, TUI auto-refresh). `runs.py show/replay/clean` switched user `run_id` rendering to `markup_escape` and switched `clean` containment from broken `Path.resolve() + relative_to()` to project-standard `os.path.realpath + is_under_cwd`. `tui_app.py` lazy-imports `ExperimentTracker` and `markup_escape`s every DB-sourced string before passing into Textual widgets so a crafted base_model / experiment_name cannot inject `[bold red]…[/]` markup. `run_cost.estimate_run_cost_usd` rejects `bool` in `num_gpus` (bool is a subclass of int — same defence as v0.30.0 `Candidate.__post_init__`); duration clamped to `[0, 1 year]`; unknown GPU returns `None` so callers render `—` instead of fabricating `$0.00`. `log_level.parse_log_level` rejects non-string + null-byte input.
 - **v0.33.0 — Live Wire**: RLVR `code_exec_reward` adds OS-level isolation (Linux best-effort `os.unshare(CLONE_NEWUSER|CLONE_NEWNET|CLONE_NEWPID)`, macOS `sandbox-exec` with default-deny `MACOS_SANDBOX_PROFILE` narrowed to a 3-name `mach-lookup` allowlist to prevent DNS / NSURLSession bypass of `(deny network*)`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat + S_ISLNK` + `shutil.rmtree(onerror=_abort_on_symlink)` so a symlink encountered mid-walk aborts rather than escapes; `run_gate` wraps each task scorer in a typed `try/except` so backend failures produce `score=None, error=str(exc)` (never silent `score=1.0`); `_parse_judge_url` removes the bare `http://` catch-all (defence-in-depth after the Pydantic GateTask validator); `soup can run` requires `--yes` or explicit consent callback and raises `ValueError` (not `PermissionError`, which is an `OSError` subclass that broad `except` blocks would swallow); GGUF `rglob` result for ollama deploy is `realpath+commonpath` checked against extract_dir (prevents symlink escape from a crafted can); `DeployTarget.path` validator normalises mixed `\\`/`/` separators before splitting (closes a Windows `..` bypass); `CAN_FORMAT_VERSION` 1→2 (additive — v1 still loads); `soup can publish` validates `repo_id` via `utils/hf.validate_repo_id`, resolves token via `resolve_token`, sanitises commit messages (first-line, 200-char cap), uses HTTPS-only HfApi; `_write_spike_recovery_hint` adds `is_under_cwd` containment check on `args.output_dir` from raw HF `TrainingArguments`; `lookup_entry_by_output_dir` emits `ResourceWarning` when 1000-row scan limit is hit (no silent miss); `CrossDocCollator` no longer mutates input feature dicts (HF Dataset rows are cached and reused — mutation broke subsequent batches); `Candidate` rejects `bool` in `score`/`latency_ms` (was sneaking past `int` isinstance check); `evaluate_candidate` latency mean now divides by *completed* prompts (excludes crashed) so a broken candidate isn't artificially fast; `auto_quant.run_auto_quant_picker` soft-falls-back to highest-scored candidate when no candidate clears `min_score` (server still binds); `build_logits_processors` returns `[]` when neither `outlines` nor `lm-format-enforcer` is installed (server degrades to free-form rather than 500); MII server uses loopback-only CORS, max_tokens cap [1, 16384], stream rejection, generic 500 with no stack-trace leak; `os.execvp` auto-reexec uses list args (no shell), all forwarded flags pre-validated; `cleanup_extract_dir` uses `os.path.commonpath` (Windows-safe) instead of `startswith`; `_run_subprocess` catches `TimeoutExpired` and returns rc=124 (coreutils convention) instead of an unhandled traceback; new `eval_results` and `tensorrt` artifact kinds in `RegistryStore._VALID_KINDS`
+- **v0.53.7 — Data Forge + Pipeline live (wave 1)**: 11 features (6 new + 5 v0.53.6 stub-to-live), 1 CRITICAL + 11 HIGH + 17 MEDIUM + 10 LOW review findings fixed. (#103 `/v1/tools/bash` REVERTED to HTTP 501) security review caught that `/bin/sh -c` spawns a child process outside the RLVR sandbox's OS-level isolation (`unshare(CLONE_NEWNET)` on Linux, `sandbox-exec` on macOS, socket monkey-patch); a caller could reach the cloud-metadata service from the child shell. Endpoint returns 501 with v0.53.8 marker until container/namespace work lands. Python + web_search remain LIVE. (#103 `/v1/tools/{python,web_search}` Bearer auth gate) optional `auth_token` kwarg on `_create_app` + `Bearer` gate on both endpoints; default `None` preserves loopback-only back-compat per v0.30.0 Part C CORS policy. (#103 `/v1/tools/web_search` deny-by-default) `WebSearchConfig.domain_allowlist` defaults to empty = deny-all (returns 403 `"web_search disabled"`); only operators who supply a domain list via constructor enable the endpoint. 5s `httpx` timeout, 5-result cap, snippet null-byte stripped. (#102 `/v1/messages` redaction) malformed Anthropic payloads (incl. validator-shape errors) mapped to generic `"Invalid request"` 400 body; detailed exception text logged server-side at DEBUG (matches v0.40.3 #33 trace-log policy). (#102 vLLM `/v1/messages` CORS) restricted to loopback origins on both transformers AND vLLM backends since it's a mutation route (matches v0.30.0 Part C transformers-backend policy that was previously vLLM-wildcard). (#102 SSE header injection) `model` + `msg_id` strings stripped of CRLF / NUL / oversize before SSE `data:` embedding — defends against `model="foo\ndata:{injected}\n\n"` injecting spurious SSE frames; `Cache-Control: no-store` + `X-Accel-Buffering: no` headers on `StreamingResponse` prevent intermediary caching of model outputs. (#106 `_node_seed` symlink TOCTOU) `os.lstat` runs on the RAW path BEFORE `realpath` resolves the symlink (matches v0.33.0 #22 `prune_checkpoints` policy); `realpath`-first ordering would have been a no-op since `S_ISLNK` on a resolved path is always False. (#106 `_save_checkpoint` atomic) switched from `path + ".tmp"` + plain `open()` to `tempfile.mkstemp(dir=path.parent, suffix=".tmp")` + `os.fdopen` + `os.replace` (matches v0.43.0 Part D `copy_bundle_to` / v0.48.0 `write_mix_recipe` policy); eliminates predictable `.tmp` symlink hazard. (#106 `failed_reason` redaction) raw exception messages run through `_redact_exc_message` which strips POSIX `/abs/path` and Windows `C:\abs\path` runs to `os.path.basename` and caps at 256 chars (matches v0.34.0 `crash.py` redaction policy); a `FileNotFoundError` no longer leaks the operator's `$HOME` into the world-readable `.checkpoint.json`. (#106 `_node_code` row JSON double-encode) row payloads serialised via `json.dumps(json.dumps(row))` before f-string embedding into the RLVR sandbox wrapper — defends against Python `repr()` escape edge cases on Unicode that could break sandbox invocation. (#106 resume rehydration) completed-node outputs rehydrated from per-node `.node-<name>.jsonl` sidecar files (NOT empty list — a v0.53.7 in-progress draft had a correctness defect that silently fed downstream consumers an empty list, causing zero-row output on resume). (#86 atomic Arrow write) `Dataset.save_to_disk` writes to a `.tmp_<name>` sibling directory then `os.replace`-ed (mirrors v0.48.0 `write_mix_recipe` pattern); mid-write Ctrl-C / OOM no longer leaves a partial Arrow directory that looks complete to subsequent runs. (#86 cache-hash gate) SFT + Pretrain wrappers verify `metadata.json::cache_key` matches the freshly-computed `make_preprocess_cache_key(base, max_length, format, train)` BEFORE consuming the Arrow shards; mismatch raises `ValueError("cache hash mismatch: re-run `soup data preprocess`")` so a user who pre-tokenized with one config and changed `max_length` cannot silently train on stale shards. Missing metadata.json falls back to yellow advisory + trusted mode. (#87 `resolve_prompt_strategy`) operator-supplied module imported via `importlib`; signature-validated `Mapping → Mapping`; `except` narrowed to `TypeError` only so the explicit ValueError from the signature-shape check propagates (a v0.53.7 in-progress draft had `except (TypeError, ValueError): pass` which swallowed its own validator); `assert isinstance(spec, str)` replaced with `raise TypeError` so the validator survives `python -O`. Per-row invocations swallow exceptions at DEBUG and fall through to the raw row (matches v0.33.0 #47 `CrossDocCollator` immutability policy). LIMITATION: this is a trusted-input surface — do NOT use `prompt_strategy` with operator-controlled YAML on shared CI hosts. (#88 markdown ingest) heading-section text capped per regex `^#{1,6}\s` so a crafted MD with a 1-byte heading followed by 1 GB body still respects the existing `data/ingest` MD branch size cap from v0.42.0 Part F. (#111 forge `--judge-provider`) lazy v0.20.0 provider instantiation reuses existing SSRF hardening per backend: Ollama localhost-only (rejects remote IPs), Anthropic env-only API key (`ANTHROPIC_API_KEY` — never CLI arg, matches v0.20.0 policy), vLLM scheme-allowlist + loopback HTTP. Per-call exceptions counted + logged at DEBUG (no silent silent-degrade). (#112 `--benchmark-file`) operator JSONL path runs through `is_under_cwd` + `os.lstat + S_ISLNK` rejection BEFORE the corpus is loaded (matches v0.46.0 `agent_forge.load_spec_file` policy). (#105 `instantiate_trainer_plugins`) `_instantiate_simple_plugin` returns None + WARNING when no `attr_candidates` match the imported module (matches v0.45.0 `register_plugin` policy — surface bad shapes loudly, not silently); `except TypeError` narrowed (a v0.53.7 in-progress draft swallowed all `TypeError` from the wrong layer). Cross-cutting: SSE generator return types annotated (`Generator[str, None, None]`); `JUDGE_PROVIDERS: frozenset[str]` PEP 585 element type; `extract_row_text` exposed publicly in `utils/data_score.__all__` (cross-module import of private helper eliminated). Test surface: `tests/test_v0537.py` carries 111 new tests; CI green across Ubuntu/macOS/Windows × Python 3.9/3.11/3.12. (v0.53.7)
 - **v0.53.6 — Plugin + Agent + Anthropic API**: 6 features — 3 live, 3 deferred-live stubs. (#101 SoupPluginCallback live) New `soup_cli/monitoring/plugin_callback.py` ships `SoupPluginCallback(transformers.TrainerCallback)` that dispatches the four canonical hook events to every enabled plugin in the v0.45.0 registry. Per-hook invocation runs through `_safe_invoke` which swallows ALL exceptions at WARNING with `exc_info=True` — defends against one misbehaving plugin crashing a multi-hour training run (mirrors v0.44.0 / v0.45.0 plugin-loader policy). Hook snapshot is collected ONCE in `build_plugin_callback` and passed to the constructor so a plugin registered between the "is empty?" check and ctor cannot silently slip into the active hook list (code-review HIGH fix — race-window closed). `attach_plugin_callback(trainer, console=None) -> bool` short-circuits to False when no plugins are enabled (zero overhead on the hot path); inner `add_callback` failure is swallowed at DEBUG so a plugin-infrastructure bug never crashes training. Wired into all 13 transformer-backend trainers; source-level grep regression test parametrized over every trainer file. (#102 Anthropic `/v1/messages` live) New POST `/v1/messages` route reuses the v0.45.0 SSRF-hardened `from_anthropic` converter + the existing chat handler. Streaming requests return 501 BEFORE schema validation (defence-in-depth — defends against stream-only attackers leaking validator-error detail). All validation paths (`validate_anthropic_payload` / `from_anthropic` / `ChatCompletionRequest(**openai_payload)`) are wrapped to map to a generic `"Invalid request"` 400 body; detailed exception text is logged server-side at DEBUG (security-review MEDIUM fix — matches `utils/errors.py` policy "HTTP error responses return generic messages, details logged server-side"). The existing loopback-CORS restriction from v0.30.0 Part C and the trace-log-redaction from v0.40.3 #33 both apply to the new route. (#104 n-gram speculative decoding live) `_generate_response` gains `ngram_config: Any = None` kwarg; when set, emits `prompt_lookup_num_tokens=int(ngram_config.num_draft_tokens)` into `model.generate(...)`. The int cast is wrapped in `try/except (TypeError, AttributeError)` as defence-in-depth; primary validation lives upstream in v0.45.0 `validate_ngram_config` (bool-rejected, bounded `[1, 32]`). Mutually exclusive with `assistant_model` — real draft model wins. (#103 server-side tool endpoints STUB) Three new POST routes `/v1/tools/python` / `/v1/tools/bash` / `/v1/tools/web_search` return 501 with v0.53.7 marker. Tool names are hardcoded string literals (not user-controlled); no payload inspection. Live RLVR `code_exec` sandbox HTTP wrapper (python / bash) + `WebSearchConfig.domain_allowlist` enforcement (web_search) land in v0.53.7. (#106 `run_recipe` STUB + `soup data recipe --execute`) New `utils/recipe_run.run_recipe(dag, *, output_dir, ...)` type-checks every parameter (TypeError on non-`RecipeDAG` / non-string / empty `output_dir` / non-bool `resume` / non-string `judge_*`) BEFORE the `NotImplementedError("v0.53.7")`. CLI `--execute --output <dir>` enforces `is None` guard on `--output` (project policy since v0.40.6 — empty-string is a distinct operator error, NOT silent missing) + `is_under_cwd` containment at the CLI boundary BEFORE the `run_recipe` call (security-review MEDIUM fix — defends against future v0.53.7 live runner bypass; never want the live runner to be the first/only enforcement point per project policy). Regression test asserts the v0.53.7 marker does NOT surface on outside-cwd paths — guards against future live runner accidentally bypassing CLI containment. (#105 `instantiate_trainer_plugins` STUB) Validates name list through existing `validate_trainer_plugin_list` (closed allowlist, dedup, ≤8/run, canonicalisation — all v0.45.0 hardening) BEFORE raising `NotImplementedError`. Canonical names echoed in the error message are allowlist-validated lowercase strings (no injection surface). Test surface: 1 new test file (`tests/test_v0536.py`) carrying 53 new tests covering: per-hook exception swallow with caplog isolation, source-level grep parametrized over 13 trainer files (canonical import + invocation), generic `"Invalid request"` body redaction with detail logged at DEBUG, oversize `max_tokens` cap, n-gram kwarg omitted when config is None (regression guard against future "always emit" refactor), `run_recipe` type-rejection matrix (non-DAG / empty output_dir / non-str checkpoint_dir / non-str judge_provider / non-str judge_model / bool resume), `--execute --output ""` rejected with `"must be a non-empty path"`, outside-cwd `--output` rejected with `v0.53.7` marker NEVER surfacing on rejected paths. Known limitations: (1) vLLM `/v1/messages` parity deferred to v0.53.7 — Anthropic route is transformers-only. (2) Streaming `/v1/messages` returns 501 — true SSE event-shape ships in v0.53.7. (3) Plugin hook snapshot is at callback-construction time — a plugin registered mid-run does not retroactively receive hooks (by design — prevents partial-run inconsistencies). (4) n-gram is transformers-only — vLLM ships its own prompt-lookup path that needs a different kwarg. (5) Tool endpoints + recipe runner + trainer-plugin instantiation are stubs — live wiring in v0.53.7. (v0.53.6)
 
 - **v0.53.5 — Adaptive Training (BETA → stable)**: six closes lifting the v0.48.0 BETA deferrals to live wiring. (#114 DynamicCurriculumCallback) new `soup_cli/monitoring/curriculum_callback.py` accepts `output_dir` through `is_under_cwd` containment + null-byte / oversize / non-string rejection (matches v0.40.5 `reward_model` policy) BEFORE any filesystem touch; per-step bucket-stats appended to `curriculum_history.jsonl` via `tempfile.mkstemp` + `os.replace` atomic write (mirrors v0.48.0 `write_mix_recipe` policy) so a crash mid-write cannot leave a half-row at the target. Rank-0 guard via `_is_rank_zero` helper — defends against multi-rank double-write on shared filesystems. Multi-rank coordination via `torch.distributed.all_reduce(SUM)` of per-bucket stats BEFORE `compute_bucket_weights` invocation — defends against per-rank divergent samplers (the DDP footgun the v0.48.0 schema gate explicitly warned about). (#115 multi-trainer expansion) `_validate_curriculum_dynamic_supported` cross-validator widened from `{sft, pretrain}` to every transformer-backend trainer; MLX backend still rejected with a distinct error message (matches v0.34.0 review-fix policy). New `attach_curriculum_callback(trainer, tcfg, output_dir, console=None) -> bool` shared helper follows the v0.40.6 `attach_relora_callback` pattern with `is None` guard on `tcfg.curriculum_dynamic` — defends against the schema-bypass footgun where `curriculum_dynamic=0` (falsy but explicit) would silently no-op a deliberate disable. (#116 `soup data mix --live`) new `soup_cli/utils/mix_proxy.py::proxy_run_for_weights` — argv-list `subprocess.run([sys.executable, "-m", "soup_cli.cli", "train", ...], timeout=…)` with NO shell (defends against shell injection via crafted dataset path / base-yaml path), tmp YAML staged under `tempfile.mkdtemp(prefix=".soup_mix_proxy.")` then cleaned up in `try/finally`. Weights validated through simplex (sum-to-1 ± 1e-6), finite (`math.isfinite`), `bool`-rejected, per-element `[0, 1]` bounds (matches v0.30.0 `Candidate` / v0.48.0 `MixCandidate` policy). `base_yaml_path` runs through `is_under_cwd` containment + null-byte rejection BEFORE the tmp YAML is rendered. `timeout_seconds ∈ [60, 30*60]` Pydantic-style bounds. Tracker-SQLite read uses existing v0.34.0 `ExperimentTracker.get_run` policy (LIKE escape, parameterised SQL). `SOUP_DB_PATH` env-override propagated via subprocess `env=` so per-candidate DB isolation is possible. (#117 skopt OptimizerProtocol) lazy `import skopt` — when the optional dep is absent, `_build_default_optimizer` falls back silently to the v0.48.0 Dirichlet sampler (mirrors v0.43.0 Part A tracker-package policy). The skopt path drives `skopt.Optimizer(GP)` through the existing v0.48.0 `OptimizerProtocol` — no API surface change. (#118 `MixOptimizationReport.elapsed_seconds`) headline elapsed now sums only successful-candidate wall-clock; failed-proxy time excluded. No security-surface change — observability fix only. Test surface: 1 new test file (`tests/test_v0535.py`) carrying 57 new tests covering cwd containment, symlink rejection, simplex + finite + bool validation, argv-list subprocess shape, atomic JSONL append, idempotent attach helper, MLX-rejection cross-validator, multi-trainer task gate. Known limitations: (1) `_pick_bucket` is step-mod round-robin (BETA); loss-percentile / curriculum-metric routing tracked for a follow-up patch. (2) `validate_distributed_curriculum` helper still requires callers to attest `rank_coordinated=True` for external invocations; the new callback wires `all_reduce` internally so the schema invocation passes. (3) `proxy_run_for_weights` is single-shot — concurrent proxy runs would race on `~/.soup/experiments.db`; the `SOUP_DB_PATH` env override is the per-candidate isolation hatch. (4) `scikit-optimize` is an optional dep, not bundled — silent fallback to Dirichlet sampler when absent. (5) `MixOptimizationReport.elapsed_seconds` is a behaviour change — operators printing the report's headline elapsed must note it now excludes failed-candidate time. (v0.53.5)