BETA. Adds `training.curriculum_dynamic: true` schema flag with online
uncertainty estimation: every N steps, aggregate per-sample loss + grad-norm
into per-bucket softmax weights, water-filled to enforce a minimum
`curriculum_dynamic_floor`. DDP/grad-accum safety via
`validate_distributed_curriculum` cross-validator that rejects un-coordinated
multi-rank runs upfront — the well-known footgun where divergent per-rank
stats silently desynchronise the sampler.
New `soup runs curriculum-curve <run_id>` visualiser with TOCTOU
(`os.lstat + S_ISLNK`) + 50 MB file-size cap + 100k-line streaming cap on
the history file. Schema gated to sft/pretrain on transformers backend;
mlx + non-SFT rejected with distinct messages.
Live HF Trainer callback wiring deferred to v0.48.1 (stub-then-live
pattern; mirrors v0.27.0 MII / v0.37.0 multipack / v0.41.0 LLaMA Pro).
Review fixes:
- water-fill design fix (code-review HIGH): removed trailing renorm that
could push elements below `floor` when accumulated float error left
sum slightly > 1.0. Softmax already sums to 1.0, so water-fill output
also sums to 1.0 (drift bounded by nb*eps).
- DoS caps on `render_curve` + `parse_history_jsonl`
(`_MAX_HISTORY_ROWS=100_000`) — without these an attacker-controlled
JSONL with 10M rows would OOM the process.
- `curriculum-curve` CLI: symlink rejection + 50 MB + 100k-line caps,
null-byte rejection on tracker-supplied `output_dir`.
+74 tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Alpamys