mirror of https://github.com/razor-ai/soup.git
docs: soup train has no --task flag
`soup train --help` contains no `--task`; the task is a `soup.yaml` key. Two places said otherwise, and both read as an instruction rather than as history: - `docs/commands.md` listed `soup train --task unlearn` in the command cheat-sheet, where lines are copied straight into a terminal. Rewritten to `soup train # task: unlearn`, the form the RAFT line directly below it already uses. - `docs/training.md` said "`soup train --task unlearn` is live", two lines under a code block that correctly runs `soup train --config unlearn.yaml`. Now `task: unlearn`. Checked every other `--task` in docs/: all 25 remaining are real flags on other commands (`infer`, `data forge`, `bom emit`, `recipes search`, `eval coverage`, `ship --task-eval/--task-mode`). Verified against the registered options, not by reading a diff. Left alone deliberately: CHANGELOG.md and benchmarks/gate-v0.72.4 also carry the phrase, but they describe which code path broke rather than telling anyone what to run, and the gate records are published verbatim on purpose. Four docstrings in src/ and two in tests/ are narrative, not emitted to users. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
8f9c8435b1
commit
9117da1054
|
|
@ -232,7 +232,7 @@ soup --no-audit-log <cmd> / SOUP_NO_AUDIT_LOG=1 Opt out of the per-command audi
|
|||
soup eval unlearning <run-id> --benchmark tofu|muse|wmdp Forget Quality + Model Utility + PrivLeak verdict
|
||||
soup edit set --base <m> --method rome|memit|alphaedit|grace --subject "..." --target "..." [--output <dir>] [--device cpu] [--governor/--no-governor] [--registry-id <id>] [--cov-corpus <jsonl|txt>] Live surgical knowledge edit (GPT-2 Conv1D + Llama; --cov-corpus = covariance-preconditioned ROME, rome-only; --plan-only available)
|
||||
soup edit diff <before-run> <after-run> --probes p.jsonl [--before-model <m> --after-model <m>] Knowledge-injection diff (live before/after generation when both models given)
|
||||
soup train --task unlearn NPO/SimNPO/RMU unlearning from data.forget_set (+ optional data.retain_set)
|
||||
soup train # task: unlearn NPO/SimNPO/RMU unlearning from data.forget_set (+ optional data.retain_set)
|
||||
soup train # data.format='raft' Answer-only span-mask RAFT training (golden+distractor docs, [doc-N] citations); generator-stage configs auto-link the latest RA-DIT retriever
|
||||
soup ra-dit --retriever-config <r.yaml> --generator-config <g.yaml> [--retriever-model <m>] [--plan-only] One-shot two-stage RA-DIT: train retriever → record pairing → train generator
|
||||
soup eval citation <data> [--style bracket|inline|footnote] [--shuffle-seed N] [--output o.json] Citation precision/recall/F1 over predictions or RAFT rows
|
||||
|
|
|
|||
|
|
@ -210,7 +210,7 @@ soup train --config unlearn.yaml --yes
|
|||
soup eval unlearning <run-id> --benchmark tofu --evidence evidence.json --output report.json
|
||||
```
|
||||
|
||||
`soup train --task unlearn` is live (v0.71.9): it loads a LoRA-wrapped policy, a frozen reference copy (NPO / RMU), and the forget / retain JSONL sets, then optimises the per-method loss — NPO's `(2/β)·mean(-logσ(-β·(π_logp − ref_logp)))` drives the policy's forget-set log-prob below the reference (= forgetting), while the retain set anchors capability. Run NPO/SimNPO **with** a `retain_set` — without one the policy has no utility anchor and Soup warns loudly.
|
||||
`task: unlearn` is live (v0.71.9): it loads a LoRA-wrapped policy, a frozen reference copy (NPO / RMU), and the forget / retain JSONL sets, then optimises the per-method loss — NPO's `(2/β)·mean(-logσ(-β·(π_logp − ref_logp)))` drives the policy's forget-set log-prob below the reference (= forgetting), while the retain set anchors capability. Run NPO/SimNPO **with** a `retain_set` — without one the policy has no utility anchor and Soup warns loudly.
|
||||
|
||||
Three orthogonal axes: **Forget Quality** (pre/post forget-loss delta), **Model Utility** (retain-accuracy preserved), **PrivLeak** (membership-inference AUC distance from 0.5). Bundled mini-fixtures for all three benchmarks ship in the box (v0.71.1 added MUSE + WMDP alongside the existing TOFU set), so `--benchmark muse|wmdp` runs without supplying evidence. The WMDP forget-set probes ship **redacted** (placeholder prompts + `REFUSED` responses) — Soup never bundles verbatim hazardous-knowledge content.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue