The repo moved to src-layout and trimmed README into a 238-line front door
with the feature reference under docs/, but several committed files still
referenced bare soup_cli/ paths or linked the gitignored .claude/CLAUDE.md
(which 404s for anyone cloning the public repo).
- docs/: `soup_cli/{plugins,templates,ui/plugins}/...` path refs -> `src/soup_cli/...`
(import statements `from soup_cli...` left unchanged — package name is still soup_cli)
- AGENTS.md: point external agents at public docs/, CONTRIBUTING.md, and the
config schema; note CLAUDE.md is a maintainer-local (gitignored) file
- CONTRIBUTING.md + .github/pull_request_template.md: PR checklist now says
"README.md and the matching page under docs/" (kept in sync); Questions
section links docs/ instead of the gitignored CLAUDE.md
- examples/README.md: fix two broken ../CLAUDE.md links -> config schema source
+ docs/ feature reference
- .gitignore: add root-anchored /_*.py temp-script guard + trailing newline
Eight features that close out the v0.44.x live-monitoring deferrals
plus a long tail of standalone CLI wins:
- #94 /api/train/stream async SSE with per-subscriber cursor + JS
EventSource consumer; SoupTrainerCallback pushes TrainEvent on
each on_log.
- #95 soup ui --public derives LAN IP via SOCK_DGRAM connect-trick,
prints scannable QR; --auth-token override; SPA bootstrap
hydrates window._authToken from ?token= + sessionStorage and
cleans the URL via history.replaceState; CORS regex auto-widens
to loopback + RFC1918 in public mode; set_auth_token rotation
race fixed via threading.Lock.
- #98 soup serve --reasoning-parser strips <think>...</think> (and
OpenThinker tags); pre-compiled regex with marker-token
fast-path + 1 MiB cap + leading-newline-only strip.
- #100 ToolOutputsBuffer global singleton + /api/tool-outputs JSON
endpoint; best-effort observation hook in callback.on_log.
- #15 soup tokenizer train: BPE training CLI with raw-path lstat
symlink rejection, 50 MiB total / 8 KiB per-line caps,
post-mkdir output-dir re-check, --special-token NUL/oversize
dedup, vocab bounds [256, 200000].
- #26 soup bench --p50 --p95 renders extra per-prompt tail-latency
Rich table; --prompts-file gains symlink rejection.
- #28 soup bench --backend auto: MLX weights.npz probe (per-entry
lstat) -> config.json model_type keyword -> transformers
fallback; SOUP_BENCH_BACKEND env hint.
- #12 examples/synthetic_workflow.{md,yaml} end-to-end walkthrough.
Review fixes: 0 CRITICAL + 11 HIGH + 14 MEDIUM + 9 LOW across the
python / code / security / tdd review agents. Notable HIGH:
- QR token now consumed by SPA (was unreachable previously).
- set_auth_token rotation lock-protected, 8-thread stress tested.
- Tokenizer input + output symlink TOCTOU defence on raw path.
- SSE generator switched to async (asyncio.sleep) for non-blocking
multi-subscriber operation.
- CORS regex for --public LAN mode (the old fixed allowlist of
http://0.0.0.0:port never matched a real Origin header).
- _has_mlx_weights per-entry lstat so a symlinked weights.npz can't
trigger MLX dispatch.
Test count: 8257 -> 8285 (+57 in tests/test_v0539.py, minus the
relaxed v0.53.8 version-pin asserts in tests/test_v0538.py).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a working DPO (Direct Preference Optimization) example using the
current Pydantic config schema with Llama 3.1 8B Instruct and QLoRA.
- examples/configs/dpo_example.yaml: DPO config with all core training
and LoRA parameters, plus commented-out advanced options
- examples/data/dpo_sample.jsonl: 8 preference pairs in DPO format
with ShareGPT-style message lists for chosen/rejected
- tests/test_dpo_example.py: 7 tests validating config loading, field
values, data format detection, and data validation
- examples/README.md: document the new DPO with QLoRA example
Alpamys
Chinmaya Sahu