feat(reward): soup reward synth — synthesize a deterministic reward verifier (v0.71.40)

Point `soup reward synth <refs.jsonl> -o reward.py` at reference (gold) outputs and it
infers a deterministic verifier (numeric / json_schema / regex / tool_call), emits a
readable, committable .py reward_fn that rides load_reward_fn's existing .py path (no new
exec surface), and — the moat — REFUSES to emit one that can't discriminate its references
from auto-perturbed negatives via a mandatory calibration report (accept refs >=90% AND
reject negatives; hard floor at discrimination<=0). Nothing in TRL/Unsloth/Axolotl
synthesizes a reward.

Fixes #311: a comma-separated reward_fn ("accuracy,format") now loads as a reward ensemble
(GRPOTrainer reward_funcs=[...], unlocks the rm_ensemble detector), GRPO-only and validated
at config-parse; the deepseek-v3-reasoning recipe that shipped this previously crashed with
"Unknown reward function".

Riders: reward_fn field-validator (null-byte/blank/oversize/empty-comma-segment); comma-aware
verifiable-domain check; envs/calculator + guess_number docstrings corrected.

5 sequential ECC reviews, every finding fixed (python HIGH PPO gate; code 2xHIGH per-tool
arg binding + json_schema mixed-shape refuse; security HIGH rel_hint codegen injection; tdd
8xHIGH). Live smoke on RTX 3050: synth from envs/calculator agrees with math_verify;
degenerate refused (exit 2); real GRPO on SmolLM2-135M with reward_fn=accuracy,format
completed optimizer steps. +103 tests (tests/test_v07140.py); 16387 -> 16490.

CHANGELOG.md

@@ -12,6 +12,45 @@ reproducing 70+ versions of notes.
 
 ## [Unreleased]
 
+## [0.71.40] - 2026-07-19
+
+**`soup reward synth`: auto-generate a deterministic reward verifier from your data.**
+Point it at a JSONL of reference (gold) outputs and it infers a verifier, emits a
+readable / committable `.py` reward function, and — the moat — *refuses* to emit one
+that can't tell your references from auto-generated bad answers. Nothing in
+TRL / Unsloth / Axolotl / OpenRLHF synthesizes a reward; every reward today is
+hand-written, hand-picked, or a trained-weights artifact.
+
+### Added
+
+- **`soup reward synth <references.jsonl> -o reward.py`** — deterministic verifier
+  synthesis. Four families, auto-detected (or pick with `--kind`): `numeric`
+  (last-number / `\boxed{}` / `####` extraction, exact or `--tolerance`), `json_schema`
+  (induced keys + types + required), `regex` (positional char-classes over
+  equal-length golds), `tool_call` (per-tool `required`/`allowed` argument binding).
+  The emitted file is self-contained and rides `load_reward_fn`'s existing `.py`
+  path — no new trusted-exec surface; you read, edit, commit, and diff it.
+- **Mandatory calibration report** — the synthesized verifier is loaded back and run
+  against its own references (must accept ≥90%) and auto-perturbed negatives (must
+  reject). A degenerate always-accept verifier is **refused** (`exit 2`), never
+  silently emitted. `--plan-only` reports the induced spec without writing;
+  `--output-report` persists the calibration JSON.
+- **Comma-separated `reward_fn` (`"accuracy,format"`)** now trains — it resolves to a
+  reward *ensemble* (`GRPOTrainer(reward_funcs=[...])`, and unlocks the `rm_ensemble`
+  reward-hack detector which needs ≥2 rewards). GRPO-only, validated at config-parse
+  time. Fixes a recipe (`deepseek-v3-reasoning`) that shipped exactly this and
+  previously crashed with `Unknown reward function` (#311).
+
+### Changed / Fixed
+
+- `training.reward_fn` gains a field validator (null-byte / blank / oversize /
+  empty-comma-segment rejection) — the oldest arbitrary-code field was the least
+  guarded. Comma + `verifiable` without a `verifiable_domain` now fails at parse
+  time like the bare `verifiable` form.
+- `envs/calculator.py` / `envs/guess_number.py` docstrings corrected: the reward is
+  `reward_fn: verifiable` + `verifiable_domain: math` (the bare `reward_fn='math'`
+  they showed was never valid).
+
 ## [0.71.39] - 2026-07-19
 
 **"CI for weights, not prompts": close the evidence loop.** `soup ship`'s verdict is

CONTRIBUTING.md

@@ -120,7 +120,7 @@ src/soup_cli/
   templates/          - 21 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0, +4 compliance v0.71.35)
   ui/                 - Web UI (FastAPI + HTML/JS SPA)
 
-tests/                - Test suite (317 files, 16387 tests)
+tests/                - Test suite (318 files, 16490 tests)
 examples/             - Real-world config examples and datasets
 ```
 

README.md

@@ -49,29 +49,37 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.39 — CI for weights, not prompts.** `soup ship`'s verdict is now something Soup can emit, commit, review, and bind to the exact model that produced it — so a fine-tuning gate renders on every PR instead of asking you to hand-edit a JSON file.
+**v0.71.40 — `soup reward synth`: generate a reward verifier from your data.** Point it at a JSONL of reference (gold) outputs and it infers a deterministic verifier, writes a readable / committable `.py` reward function, and — the part nobody else does — *refuses* to emit one that can't tell your references from bad answers. Nothing in TRL / Unsloth / Axolotl synthesizes a reward.
 
-- **Output is finally input.** `soup ship --emit-evidence ev.json` writes the scores in the
-  `--evidence` schema, so a run replays offline into an identical verdict — the round-trip the
-  old `--output` couldn't do.
-- **Commit the gate policy.** Put `eval.ship` in your `soup.yaml` (`forgetting_threshold`,
-  `general_suite`, `task_mode`, …) and `soup ship --config soup.yaml` reads it — the gate is now
-  reviewable in a PR diff. Explicit flags still win.
-- **Provenance-bound evidence.** `--config --emit-evidence` STAMPS a `config_sha` onto the
-  evidence; `--config --evidence` GATES on it — a PR that changed the recipe but forgot to
-  recompute its evidence is refused (exit 3). Tuning the threshold never falsely invalidates it.
-- **The verdict on your PR.** `soup ship --push owner/repo#N` posts the SHIP / DON'T-SHIP card as
-  a GitHub comment; `soup ci init --config soup.yaml` wires the whole thing into CI.
+- **Four families, auto-detected** (or pick with `--kind`): `numeric` (`\boxed{}` / `####` / last-number,
+  exact or `--tolerance`), `json_schema` (induced keys + types + required), `regex`, and `tool_call`
+  (per-tool `required`/`allowed` argument binding — a call can't borrow another tool's keys).
+- **A calibration report is the moat.** The verifier is loaded back and run against its own references
+  (must accept ≥ 90%) and auto-generated bad answers (must reject). A degenerate always-accept verifier
+  is **refused** (exit 2), never silently shipped. `--plan-only` previews; `--output-report` saves the JSON.
+- **No new exec surface.** The emitted file rides the existing `reward_fn: reward.py` path — you read,
+  edit, commit, and diff it like any other reward.
+- **Reward ensembles now train.** `reward_fn: "accuracy,format"` loads as multiple rewards (and unlocks
+  the `rm_ensemble` reward-hack detector) — a recipe that shipped exactly this used to crash. (#311)
 
 ```bash
-# producer (train job): stamp the scores with the config that produced them
-soup ship --evidence scores.json --config soup.yaml --emit-evidence ship_evidence.json
+# synth a verifier from reference answers, calibrate it, keep it only if it discriminates
+soup reward synth references.jsonl -o reward.py --output-report calib.json
 
-# gate (PR CI): refuse evidence that doesn't match the committed config
-soup ship --evidence ship_evidence.json --config soup.yaml --push owner/repo#42
-soup ci init --config soup.yaml   # writes .github/workflows/soup-gate.yml
+# then train against the reward you just generated
+# training: { reward_fn: reward.py }   (or an ensemble: reward_fn: "accuracy,format")
 ```
 
+<details>
+<summary>Previous release — v0.71.39, CI for weights not prompts (emit + provenance-bind the ship verdict)</summary>
+
+`soup ship`'s verdict became emittable, committable, and provenance-bound: `--emit-evidence` makes a
+run replay into an identical verdict, `eval.ship` in `soup.yaml` + `--config` makes the gate policy
+reviewable, and `--config` binds evidence to the exact recipe that produced it (stale evidence → exit 3).
+`soup ship --push owner/repo#N` posts the SHIP / DON'T-SHIP card on the PR.
+
+</details>
+
 <details>
 <summary>Previous release — v0.71.38, The gate grows teeth (real leg-2 regression gate)</summary>
 

docs/commands.md

@@ -182,6 +182,10 @@ soup draft measure ... --min-acceptance 0.6 -o report.json  Exit 2 below the flo
 soup draft distill --target <tuned> --draft-base <tiny> --data d.jsonl -o draft/  Distil a DENSE speculative-decoding draft + register it (v0.71.33)
 soup draft distill ... --steps N --device cpu --force --plan-only  Training budget / device / overwrite -o / render the config only
 soup draft list                               List local drafts that `soup serve --auto-spec` will pick up (v0.71.33)
+soup reward synth refs.jsonl -o reward.py     Synthesize a deterministic reward verifier from gold outputs (v0.71.40)
+soup reward synth ... --kind numeric|json_schema|regex|tool_call  Force a verifier family (default: auto-detect)
+soup reward synth ... --plan-only             Report the induced spec + calibration plan; write nothing
+soup reward synth ... --output-report r.json --min-discrimination 0.5  Save the calibration JSON / set the refusal threshold (exit 0 emit / 2 refuse / 1 error)
 soup tui                                      Full-screen Textual dashboard (requires [tui] extra)
 soup train --config soup.yaml --profile       Record torch.profiler trace to <output>/profiles/
 soup --log-level quiet|normal|verbose|debug   Global logging tier (Rich-formatted)

docs/training.md

@@ -866,6 +866,37 @@ training:
   reward_fn: ./my_reward.py
 ```
 
+**Reward ensembles** — list several rewards, comma-separated, and they combine (GRPO only).
+This also unlocks the `rm_ensemble` reward-hack detector, which needs ≥ 2 rewards:
+```yaml
+training:
+  reward_fn: "accuracy,format"   # both are scored every step
+```
+
+### Synthesize a verifier from your data (`soup reward synth`)
+
+Don't hand-write a verifier — generate one from reference (gold) outputs. Soup infers a
+*deterministic* verifier (numeric / JSON-schema / regex / tool-call), writes a readable, editable
+`.py`, and **refuses to emit** one that can't tell your references from auto-generated bad answers
+(the mandatory calibration report). The emitted file is a normal `reward_fn: reward.py`.
+
+```bash
+# infer + calibrate + emit (exit 0 kept, 2 refused, 1 error)
+soup reward synth references.jsonl -o reward.py --output-report calib.json
+
+# preview the induced spec without writing anything
+soup reward synth references.jsonl --plan-only
+
+# force a family instead of auto-detecting
+soup reward synth answers.jsonl -o reward.py --kind numeric --tolerance 1e-6
+```
+
+References are a JSONL where each row's gold answer is in an `answer` field (override with
+`--field`) or the last assistant turn of a `messages` list. `--min-discrimination` sets how
+strongly the verifier must separate references from perturbed negatives before it's emitted.
+v1 is deterministic families only — a `\boxed{}`/`####` marker helps the numeric verifier, and
+completions are prompted to mark their answer (standard RLVR practice).
+
 ### Verifiable Rewards (RLVR)
 
 Use `reward_fn: verifiable` with a `verifiable_domain` for deterministic, math-checkable rewards — no judge model, no heuristics. Great for GRPO on math, code, or structured-output tasks.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "soup-cli"
-version = "0.71.39"
+version = "0.71.40"
 description = "Fine-tune and post-train LLMs in one command. No SSH, no config hell."
 readme = "README.md"
 license = "Apache-2.0"

src/soup_cli/__init__.py

@@ -1,3 +1,3 @@
 """Soup CLI — Fine-tune and post-train LLMs in one command."""
 
-__version__ = "0.71.39"
+__version__ = "0.71.40"

src/soup_cli/cli.py

@@ -563,6 +563,18 @@ app.add_typer(
     ),
 )
 
+# v0.71.40 — `soup reward synth` auto-generate a deterministic verifier.
+from soup_cli.commands import reward as _reward_cmd  # noqa: E402
+
+app.add_typer(
+    _reward_cmd.app,
+    name="reward",
+    help=(
+        "Synthesize a deterministic reward verifier from reference outputs, with "
+        "a calibration report that refuses degenerate verifiers (v0.71.40)."
+    ),
+)
+
 
 def _rewrite_advise_argv(argv: list) -> list:
     """Inject `run` between `advise` and a non-subcommand first argument.

src/soup_cli/commands/reward.py

@@ -0,0 +1,225 @@
+"""soup reward — synthesize a deterministic reward verifier (v0.71.40).
+
+    soup reward synth <references.jsonl> -o reward.py
+        [--kind auto|numeric|json_schema|regex|tool_call]
+
+Infers a deterministic verifier from a dataset of reference (gold) outputs, emits a
+readable / committable ``.py`` that rides ``load_reward_fn``'s existing ``.py`` path,
+and REFUSES to emit a degenerate verifier via a mandatory calibration report.
+
+Exit codes mirror ``soup ship`` / ``soup shrink``: 0 = emitted, 2 = refused (the
+verifier could not discriminate references from perturbed negatives), 1 = usage /
+runtime error.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+import os
+from typing import NoReturn, Optional
+
+import typer
+from rich.console import Console
+from rich.markup import escape
+from rich.panel import Panel
+from rich.table import Table
+
+from soup_cli.utils import reward_synth as rs
+from soup_cli.utils.paths import atomic_write_text, enforce_under_cwd_and_no_symlink
+
+app = typer.Typer(help="Synthesize a deterministic reward verifier from reference outputs.")
+console = Console()
+
+_MAX_INPUT_BYTES = 64 * 1024 * 1024
+_MAX_ROWS = 1_000_000
+_ALLOWED_KINDS = ("auto",) + rs.KINDS
+
+
+@app.callback()
+def _reward() -> None:
+    """Reward-verifier tooling. Forces ``synth`` to be a named subcommand (a
+    single-command Typer app otherwise collapses and eats the subcommand token)."""
+
+
+def _fail(message: str, code: int = 1) -> NoReturn:
+    """Print a red error and exit (raises internally — a forgotten ``raise`` at a
+    call site can never silently become a no-op; mirrors ``commands/ship.py``)."""
+    console.print(f"[red]{escape(message)}[/]")
+    raise typer.Exit(code)
+
+
+def _read_jsonl(path: str, label: str) -> list[dict]:
+    """Read a JSONL file: cwd-contained, O_NOFOLLOW, size- and row-capped."""
+    enforce_under_cwd_and_no_symlink(path, label)
+    flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0)
+    try:
+        fd = os.open(path, flags)
+    except OSError as exc:
+        raise ValueError(f"{label} unreadable: {exc}") from exc
+    rows: list[dict] = []
+    with os.fdopen(fd, "r", encoding="utf-8") as handle:
+        if os.fstat(handle.fileno()).st_size > _MAX_INPUT_BYTES:
+            raise ValueError(f"{label} exceeds {_MAX_INPUT_BYTES} bytes")
+        for line in handle:
+            line = line.strip()
+            if not line:
+                continue
+            try:
+                row = json.loads(line)
+            except json.JSONDecodeError:
+                continue
+            if isinstance(row, dict):
+                rows.append(row)
+            if len(rows) >= _MAX_ROWS:
+                break
+    return rows
+
+
+def _spec_summary(kind: str, spec: object) -> str:
+    """One-line human summary of the induced spec (for --plan-only / the panel)."""
+    if kind == "numeric" and isinstance(spec, rs.NumericSpec):
+        return f"numeric (float={spec.is_float}, tolerance={spec.tolerance})"
+    if kind == "json_schema" and isinstance(spec, dict):
+        props = ", ".join(sorted(spec.get("properties", {}))) or "(none)"
+        return f"json_schema (type={spec.get('type')}, keys={props})"
+    if kind == "tool_call" and isinstance(spec, rs.ToolCallSpec):
+        return f"tool_call (names={list(spec.names)}, arg_keys={list(spec.arg_keys)})"
+    if kind == "regex":
+        return f"regex (pattern={spec})"
+    return kind
+
+
+def _render_report_panel(report: rs.CalibrationReport, kind: str, out_path: str) -> Panel:
+    table = Table(show_header=False, box=None, pad_edge=False)
+    table.add_column(justify="right", style="dim")
+    table.add_column()
+    table.add_row("verifier kind", escape(kind))
+    table.add_row("references (accepted)", f"{report.positives} ({report.pos_accept:.0%})")
+    table.add_row("negatives (accepted)", f"{report.negatives} ({report.neg_accept:.0%})")
+    table.add_row("discrimination", f"{report.discrimination:.2f}")
+    table.add_row("precision", f"{report.precision:.2f}")
+    table.add_row("emitted", escape(out_path))
+    return Panel(table, title="[bold green]reward verifier synthesized[/]",
+                 border_style="green")
+
+
+@app.command()
+def synth(
+    references: str = typer.Argument(
+        ..., help="JSONL of reference (gold) outputs — a gold field or chat messages."
+    ),
+    output: Optional[str] = typer.Option(
+        None, "-o", "--output", help="Where to write the verifier .py."
+    ),
+    kind: str = typer.Option(
+        "auto", "--kind",
+        help="Verifier family: auto | numeric | json_schema | regex | tool_call.",
+    ),
+    field: str = typer.Option(
+        "answer", "--field", help="Gold-output field (default: answer)."
+    ),
+    tolerance: Optional[float] = typer.Option(
+        None, "--tolerance", help="Numeric match tolerance (numeric kind only)."
+    ),
+    min_discrimination: float = typer.Option(
+        rs.DEFAULT_MIN_DISCRIMINATION, "--min-discrimination",
+        help="Refuse to emit unless (accept_rate refs - accept_rate negatives) >= this.",
+    ),
+    force: bool = typer.Option(False, "--force", help="Overwrite an existing output."),
+    plan_only: bool = typer.Option(
+        False, "--plan-only", help="Detect + report the induced spec; write nothing."
+    ),
+    output_report: Optional[str] = typer.Option(
+        None, "--output-report", help="Also write the calibration report as JSON."
+    ),
+) -> None:
+    """Synthesize a deterministic reward verifier from reference outputs."""
+    if kind not in _ALLOWED_KINDS:
+        _fail(f"unknown --kind {kind!r}; options: {', '.join(_ALLOWED_KINDS)}")
+    if not 0.0 <= min_discrimination <= 1.0:
+        _fail("--min-discrimination must be in [0.0, 1.0]")
+    if tolerance is not None and (not math.isfinite(tolerance) or tolerance < 0):
+        _fail("--tolerance must be a finite, non-negative number")
+
+    # Read references.
+    try:
+        rows = _read_jsonl(references, "references path")
+    except (ValueError, OSError) as exc:
+        _fail(str(exc))
+    if not rows:
+        _fail("references file has no JSON-object rows")
+
+    # Detect + induce + render (no file written yet).
+    try:
+        result = rs.synthesize(rows, field=field, kind=kind, tolerance=tolerance,
+                               rel_hint=(output or "reward.py"))
+    except (ValueError, TypeError) as exc:
+        _fail(str(exc))
+
+    if plan_only:
+        console.print(Panel(
+            escape(_spec_summary(result.kind, result.spec)),
+            title=f"[bold]plan: {escape(result.kind)} verifier[/]", border_style="cyan"))
+        raise typer.Exit(0)
+
+    # Output guards.
+    if not output:
+        _fail("must pass -o/--output to emit a verifier (or use --plan-only)")
+    if not output.endswith(".py"):
+        _fail(f"output must be a .py file, got {output!r}")
+    try:
+        enforce_under_cwd_and_no_symlink(output, "output path")
+    except (ValueError, OSError) as exc:
+        _fail(str(exc))
+    if os.path.exists(output) and not force:
+        _fail(f"{output!r} already exists — pass --force to overwrite")
+    # Validate the optional report path UP FRONT so a bad report path can never
+    # cause us to destroy an already-written, already-accepted verifier later.
+    if output_report:
+        try:
+            enforce_under_cwd_and_no_symlink(output_report, "report path")
+        except (ValueError, OSError) as exc:
+            _fail(str(exc))
+
+    # Write, then LOAD it back through the real reward-loader path (round-trip
+    # validation) and calibrate the loaded callable.
+    atomic_write_text(result.source, output)
+    try:
+        from soup_cli.trainer.rewards import load_reward_fn
+        reward_fn = load_reward_fn(output)
+        golds = rs.extract_golds(rows, field=field)
+        negatives = rs.perturb_negatives(golds, result.kind)
+        report = rs.calibrate(reward_fn, golds, negatives, kind=result.kind,
+                              min_discrimination=min_discrimination)
+    except Exception as exc:  # noqa: BLE001 — clean up the partial artifact
+        _cleanup(output)
+        _fail(f"calibration failed: {exc}")
+
+    # Write the diagnostic report (path already validated up front) for BOTH the
+    # accepted and refused cases. Best-effort: a write failure here warns but must
+    # not delete an otherwise-valid verifier.
+    if output_report:
+        try:
+            from dataclasses import asdict
+            atomic_write_text(json.dumps(asdict(report), indent=2), output_report)
+        except OSError as exc:
+            console.print(f"[yellow]Warning: could not write report: {escape(str(exc))}[/]")
+
+    if report.refused:
+        _cleanup(output)
+        console.print(Panel(
+            escape(report.reason),
+            title="[bold red]verifier refused (not emitted)[/]", border_style="red"))
+        raise typer.Exit(2)
+
+    console.print(_render_report_panel(report, result.kind, output))
+    raise typer.Exit(0)
+
+
+def _cleanup(path: str) -> None:
+    """Remove a just-written artifact; best-effort (never masks the real error)."""
+    try:
+        os.remove(path)
+    except OSError:
+        pass

src/soup_cli/config/schema.py

@@ -1107,10 +1107,38 @@ class TrainingConfig(BaseModel):
     reward_fn: Optional[str] = Field(
         default="accuracy",
         description=(
-            "Reward function: 'accuracy', 'format', 'verifiable', "
-            "or path to custom .py file"
+            "Reward function: 'accuracy', 'format', 'verifiable', a path to a "
+            "custom .py file, or a comma-separated ensemble of the above "
+            "(e.g. 'accuracy,format') — the comma form is GRPO-only (v0.71.40)."
         ),
     )
+    @field_validator("reward_fn", mode="before")
+    @classmethod
+    def _validate_reward_fn_field(cls, value: Any) -> Optional[str]:
+        """v0.71.40 #311 — shape-only validation (containment enforced at load).
+
+        Accepts a comma-separated spec ("accuracy,format"); rejects null bytes,
+        oversize, and empty comma segments so a stray comma fails loud rather
+        than silently dropping a reward.
+        """
+        if value is None:
+            return None
+        if isinstance(value, bool) or not isinstance(value, str):
+            raise ValueError(
+                f"reward_fn must be a string, got {type(value).__name__}"
+            )
+        if "\x00" in value:
+            raise ValueError("reward_fn must not contain null bytes")
+        if len(value) > 512:
+            raise ValueError("reward_fn must be <= 512 chars")
+        if not value.strip():
+            raise ValueError("reward_fn must not be blank")
+        if any(not seg.strip() for seg in value.split(",")):
+            raise ValueError(
+                "reward_fn has an empty comma segment — remove the stray comma"
+            )
+        return value
+
     # RLVR — verifiable reward domain (Part C of v0.25.0)
     verifiable_domain: Optional[Literal["math", "code", "json_schema"]] = Field(
         default=None,
@@ -2978,8 +3006,18 @@ class TrainingConfig(BaseModel):
 
     @model_validator(mode="after")
     def _validate_verifiable_reward(self) -> "TrainingConfig":
-        """RLVR: reward_fn='verifiable' requires verifiable_domain."""
-        if self.reward_fn == "verifiable" and self.verifiable_domain is None:
+        """RLVR: reward_fn='verifiable' requires verifiable_domain.
+
+        Comma-aware (v0.71.40 #311): ``"accuracy,verifiable"`` must fail at
+        config-parse time exactly like the bare ``"verifiable"`` form, not only
+        at trainer construction.
+        """
+        segments = (
+            [s.strip() for s in self.reward_fn.split(",")]
+            if isinstance(self.reward_fn, str)
+            else []
+        )
+        if "verifiable" in segments and self.verifiable_domain is None:
             raise ValueError(
                 "reward_fn='verifiable' requires verifiable_domain "
                 "(one of: math, code, json_schema)"
@@ -4662,6 +4700,25 @@ class SoupConfig(BaseModel):
             )
         return self
 
+    @model_validator(mode="after")
+    def _validate_reward_fn_multi_compat(self) -> "SoupConfig":
+        """v0.71.40 #311 — a comma-separated reward_fn is GRPO-only.
+
+        Only ``trainer/grpo.py::_select_reward_fn`` resolves a multi-reward spec
+        into a ``reward_funcs=[...]`` list; ``trainer/ppo.py`` (and every other
+        consumer) calls the single-name loader, so a comma there would silently
+        pass config validation and then crash at training start with
+        ``Unknown reward function``. Footgun-reject it up front (mirrors the
+        online_dpo / asr / lisa task gates).
+        """
+        rf = self.training.reward_fn
+        if isinstance(rf, str) and "," in rf and self.task != "grpo":
+            raise ValueError(
+                f"a comma-separated training.reward_fn (an ensemble, {rf!r}) is "
+                f"only supported for task='grpo'; got task={self.task!r}"
+            )
+        return self
+
     @model_validator(mode="after")
     def _validate_asr_compat(self) -> "SoupConfig":
         """v0.71.32 — ASR (Whisper) gate.

src/soup_cli/envs/calculator.py

@@ -1,7 +1,8 @@
 """Calculator tool-use rollout env — v0.71.30.
 
 Generates a deterministic set of single-operation arithmetic problems as GRPO
-prompt+answer rows. Score with ``reward_fn='math'`` (or ``'accuracy'``).
+prompt+answer rows. Score with ``reward_fn: verifiable`` +
+``verifiable_domain: math`` (or ``reward_fn: accuracy``).
 
 Usage: ``training.rollout_backend='openenv'`` +
 ``training.rollout_func='soup_cli.envs.calculator:rollout'``.

src/soup_cli/envs/guess_number.py

@@ -2,7 +2,7 @@
 
 Generates deterministic single-shot number-deduction puzzles (constraints that
 pin a unique integer) as GRPO prompt+answer rows. Score with
-``reward_fn='math'`` (or ``'accuracy'``).
+``reward_fn: verifiable`` + ``verifiable_domain: math`` (or ``reward_fn: accuracy``).
 
 Honesty: this is single-shot *deduction*, not interactive guess-and-feedback —
 the live openenv contract does not pass the model, so a true guessing loop is

src/soup_cli/trainer/grpo.py

@@ -129,7 +129,9 @@ def _read_attr(obj: Any, name: str) -> Any:
     return getattr(obj, name, None)
 
 
-def _select_reward_fn(tcfg: TrainingConfig, device: str, trust_remote_code: bool) -> Any:
+def _select_reward_fn(
+    tcfg: TrainingConfig, device: str, trust_remote_code: bool
+) -> "Any":  # Callable | list[Callable] (a single reward or a comma-split ensemble)
     """Choose the GRPO reward function (v0.71.30).
 
     When ``tcfg.prm_reward`` is set, a trained Soup PRM scores each completion's
@@ -142,9 +144,13 @@ def _select_reward_fn(tcfg: TrainingConfig, device: str, trust_remote_code: bool
         from soup_cli.utils.prm_reward import build_prm_reward_fn
 
         return build_prm_reward_fn(tcfg, device, trust_remote_code)
-    from soup_cli.trainer.rewards import load_reward_fn
+    # v0.71.40 #311 — ``reward_fn`` may be comma-separated ("accuracy,format").
+    # Return a single callable for one reward (back-compat) or a list for several
+    # (TRL's reward_funcs=[...] + the rm_ensemble detector both accept the list).
+    from soup_cli.trainer.rewards import load_reward_fns
 
-    return load_reward_fn(tcfg.reward_fn, verifiable_domain=tcfg.verifiable_domain)
+    fns = load_reward_fns(tcfg.reward_fn, verifiable_domain=tcfg.verifiable_domain)
+    return fns[0] if len(fns) == 1 else fns
 
 
 class GRPOTrainerWrapper:

src/soup_cli/trainer/rewards.py

@@ -557,3 +557,44 @@ def load_reward_fn(
         f"Options: {', '.join(BUILTIN_REWARDS.keys())}, 'verifiable', "
         f"or path to a .py file"
     )
+
+
+def load_reward_fns(
+    reward_fn_spec: str, verifiable_domain: "str | None" = None,
+) -> list[Callable]:
+    """Load one OR MORE reward functions from a comma-separated spec (v0.71.40 #311).
+
+    ``reward_fn`` may name several rewards, e.g. ``"accuracy,format"`` — each is
+    resolved via :func:`load_reward_fn` and returned as a list, which TRL's
+    ``GRPOTrainer(reward_funcs=[...])`` accepts and which the ``rm_ensemble``
+    reward-hack detector (needs >= 2 reward fns) unlocks. A single name still
+    returns a one-element list, so callers can always treat the result uniformly.
+
+    Splitting is on ``,``; blank / empty segments raise (``"accuracy,"`` is a typo,
+    not "accuracy plus nothing"), and duplicate segments raise (they would collide
+    by ``__name__`` in the ``rm_ensemble`` capture buffer, silently shrinking the
+    ensemble). A ``.py`` path containing a literal comma is not supported — rename
+    the file.
+    """
+    if not isinstance(reward_fn_spec, str):
+        raise ValueError(
+            "reward_fn must be a string (a name, a .py path, or a comma-separated "
+            f"list), got {type(reward_fn_spec).__name__}"
+        )
+    if not reward_fn_spec.strip():
+        raise ValueError("reward_fn must not be blank")
+    segments = [seg.strip() for seg in reward_fn_spec.split(",")]
+    if any(not seg for seg in segments):
+        raise ValueError(
+            f"reward_fn {reward_fn_spec!r} has an empty comma segment — "
+            "remove the stray comma"
+        )
+    seen: set[str] = set()
+    for seg in segments:
+        if seg in seen:
+            raise ValueError(
+                f"reward_fn {reward_fn_spec!r} lists {seg!r} twice — "
+                "duplicate rewards collide in the ensemble"
+            )
+        seen.add(seg)
+    return [load_reward_fn(seg, verifiable_domain=verifiable_domain) for seg in segments]

src/soup_cli/utils/reward_synth.py

@@ -0,0 +1,716 @@
+"""Reward-verifier synthesis — `soup reward synth` (v0.71.40).
+
+Auto-generate a *deterministic* reward verifier from a dataset of reference (gold)
+outputs. The emitted artifact is a readable, editable, committable ``.py`` that rides
+:func:`soup_cli.trainer.rewards.load_reward_fn`'s existing ``.py`` path — NO new
+trusted-exec hot path.
+
+Pipeline (all pure, NO top-level torch):
+  detect_kind → induce_* → render_verifier_py → (load) → perturb_negatives + calibrate.
+
+v1 verifier families (deterministic only): ``numeric`` · ``json_schema`` · ``regex`` ·
+``tool_call``. Judge-based and LLM-codegen are explicitly out of scope.
+
+The calibration report is the moat: a synthesized verifier is only emitted if it
+accepts its own references AND rejects auto-perturbed negatives — a degenerate
+always-1.0 verifier is refused.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+import re
+from collections.abc import Callable, Mapping, Sequence
+from dataclasses import dataclass
+from typing import Any, Optional, Union
+
+KINDS: tuple[str, ...] = ("numeric", "json_schema", "regex", "tool_call")
+# Auto-detect precedence: most-specific shape wins.
+_DETECT_ORDER: tuple[str, ...] = ("tool_call", "json_schema", "numeric", "regex")
+
+# Fraction of golds that must fit a kind for auto-detection / regex confidence.
+_MIN_CONFIDENCE = 0.9
+DEFAULT_NUMERIC_TOLERANCE = 1e-6
+DEFAULT_MIN_DISCRIMINATION = 0.5
+# A verifier must accept at least this fraction of its OWN references, else the
+# induced spec plainly does not fit the golds (mirrors _MIN_CONFIDENCE).
+_MIN_SELF_ACCEPT = 0.9
+_MAX_ROWS = 1_000_000
+
+_NUMBER_RE = re.compile(r"[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?")
+_WHOLE_NUMBER_RE = re.compile(r"^[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?$")
+
+
+# ---------------------------------------------------------------------------
+# Frozen result types
+# ---------------------------------------------------------------------------
+@dataclass(frozen=True)
+class NumericSpec:
+    is_float: bool
+    tolerance: float
+
+
+@dataclass(frozen=True)
+class ToolCallSpec:
+    """Per-tool argument signatures.
+
+    ``tools`` maps each tool name to ``{"required": (...), "allowed": (...)}`` —
+    ``required`` = keys present in EVERY observed call of that name (intersection),
+    ``allowed`` = union of keys seen for that name. The verifier accepts a call iff
+    ``required <= call_args <= allowed`` for the matching name, so a call cannot
+    borrow another tool's argument keys or omit required ones.
+    """
+
+    tools: dict  # name -> {"required": tuple[str, ...], "allowed": tuple[str, ...]}
+
+    @property
+    def names(self) -> tuple[str, ...]:
+        return tuple(sorted(self.tools))
+
+    @property
+    def arg_keys(self) -> tuple[str, ...]:
+        keys: set[str] = set()
+        for sig in self.tools.values():
+            keys.update(sig.get("allowed", ()))
+        return tuple(sorted(keys))
+
+
+@dataclass(frozen=True)
+class CalibrationReport:
+    kind: str
+    positives: int
+    negatives: int
+    pos_accept: float
+    neg_accept: float
+    discrimination: float
+    precision: float
+    recall: float
+    refused: bool
+    reason: str
+
+
+SpecType = Union[NumericSpec, dict, ToolCallSpec, str]
+
+
+@dataclass(frozen=True)
+class SynthResult:
+    kind: str
+    source: str
+    spec: SpecType  # NumericSpec | dict | ToolCallSpec | str (pattern)
+    report: Optional[CalibrationReport] = None
+
+
+# ---------------------------------------------------------------------------
+# Gold extraction
+# ---------------------------------------------------------------------------
+def _row_gold(row: Mapping[str, Any], field: str) -> Optional[str]:
+    if not isinstance(row, Mapping):
+        return None
+    if field in row and row[field] is not None:
+        value = row[field]
+        if isinstance(value, str):
+            return value
+        if isinstance(value, (dict, list)):
+            return json.dumps(value)
+        return str(value)  # numbers, bools -> their str form
+    messages = row.get("messages")
+    if isinstance(messages, list):
+        for msg in reversed(messages):
+            if isinstance(msg, Mapping) and msg.get("role") == "assistant":
+                content = msg.get("content")
+                if isinstance(content, str):
+                    return content
+    return None
+
+
+def extract_golds(rows: Sequence[Mapping[str, Any]], *, field: str = "answer") -> list[str]:
+    """Pull the gold output from each row (``field`` else chat last-assistant).
+
+    Raises ``TypeError`` if ``rows`` is not a sequence of mappings; ``ValueError``
+    if no row yields a gold.
+    """
+    if not isinstance(rows, Sequence) or isinstance(rows, (str, bytes)):
+        raise TypeError("rows must be a sequence of mapping rows")
+    if len(rows) > _MAX_ROWS:
+        raise ValueError(f"rows exceed cap of {_MAX_ROWS}")
+    golds: list[str] = []
+    for row in rows:
+        gold = _row_gold(row, field)
+        if gold is not None:
+            golds.append(gold)
+    if not golds:
+        raise ValueError(
+            f"no gold outputs found (field={field!r}; rows need that key or a "
+            "chat 'messages' list with an assistant turn)"
+        )
+    return golds
+
+
+# ---------------------------------------------------------------------------
+# Kind primitives
+# ---------------------------------------------------------------------------
+def _is_number(text: str) -> bool:
+    return bool(_WHOLE_NUMBER_RE.match(text.strip()))
+
+
+def _json_or_none(text: str) -> Any:
+    try:
+        return json.loads(text.strip())
+    except (json.JSONDecodeError, ValueError):
+        return None
+
+
+def _is_tool_call(obj: Any) -> bool:
+    return (
+        isinstance(obj, dict)
+        and isinstance(obj.get("name"), str)
+        and isinstance(obj.get("arguments"), dict)
+    )
+
+
+def _fraction(golds: Sequence[str], predicate: Callable[[str], bool]) -> float:
+    if not golds:
+        return 0.0
+    return sum(1 for g in golds if predicate(g)) / len(golds)
+
+
+def detect_kind(golds: Sequence[str]) -> Optional[str]:
+    """Infer the verifier family from the golds, or ``None`` if un-inferrable.
+
+    Precedence: tool_call → json_schema → numeric → regex.
+    """
+    if not golds:
+        return None
+    for kind in _DETECT_ORDER:
+        if kind == "tool_call":
+            frac = _fraction(golds, lambda g: _is_tool_call(_json_or_none(g)))
+        elif kind == "json_schema":
+            frac = _fraction(
+                golds, lambda g: isinstance(_json_or_none(g), (dict, list))
+            )
+        elif kind == "numeric":
+            frac = _fraction(golds, _is_number)
+        else:  # regex
+            frac = 1.0 if induce_regex(golds) is not None else 0.0
+        if frac >= _MIN_CONFIDENCE:
+            return kind
+    return None
+
+
+# ---------------------------------------------------------------------------
+# Inducers
+# ---------------------------------------------------------------------------
+def induce_numeric(
+    golds: Sequence[str], *, tolerance: Optional[float] = None
+) -> NumericSpec:
+    if _fraction(golds, _is_number) < _MIN_CONFIDENCE:
+        raise ValueError(
+            "references are not numeric — cannot induce a numeric verifier "
+            "(each gold must be a bare number); pass a different --kind"
+        )
+    is_float = any(("." in g) or ("e" in g.lower()) for g in golds)
+    if tolerance is None:
+        tol = DEFAULT_NUMERIC_TOLERANCE if is_float else 0.0
+    else:
+        tol = float(tolerance)
+    if not math.isfinite(tol) or tol < 0:
+        raise ValueError(f"tolerance must be a finite, non-negative number, got {tol!r}")
+    return NumericSpec(is_float=is_float, tolerance=tol)
+
+
+def _json_type(value: Any) -> str:
+    if isinstance(value, bool):
+        return "boolean"
+    if isinstance(value, int):
+        return "integer"
+    if isinstance(value, float):
+        return "number"
+    if isinstance(value, str):
+        return "string"
+    if isinstance(value, list):
+        return "array"
+    if isinstance(value, dict):
+        return "object"
+    return "null"
+
+
+def induce_json_schema(golds: Sequence[str]) -> dict:
+    """Induce a top-level JSON schema (keys + types + required) from the golds.
+
+    Refuses (``ValueError``) rather than emitting a wrong schema when the golds
+    mix top-level shapes (some object, some array) or parse to no JSON container
+    at all — mirroring :func:`induce_regex`'s refuse-don't-guess policy. A key
+    whose type is inconsistent across rows is left un-typed (presence only), so
+    an over-narrow ``type`` can never reject a valid completion.
+    """
+    parsed = [_json_or_none(g) for g in golds]
+    containers = [o for o in parsed if isinstance(o, (dict, list))]
+    if not containers:
+        raise ValueError(
+            "no JSON object/array references found — cannot induce a json_schema"
+        )
+    dicts = [o for o in containers if isinstance(o, dict)]
+    lists = [o for o in containers if isinstance(o, list)]
+    if dicts and lists:
+        raise ValueError(
+            "references mix JSON objects and arrays — split them or pass an "
+            "explicit --kind; refusing to induce an ambiguous schema"
+        )
+    if lists:
+        return {"type": "array"}
+    key_types: dict[str, set[str]] = {}
+    key_counts: dict[str, int] = {}
+    for obj in dicts:
+        for key, value in obj.items():
+            key_counts[key] = key_counts.get(key, 0) + 1
+            key_types.setdefault(key, set()).add(_json_type(value))
+    properties: dict[str, dict] = {}
+    for key, types in key_types.items():
+        # Only constrain the type when EVERY row agreed on it; else presence-only.
+        properties[key] = {"type": next(iter(types))} if len(types) == 1 else {}
+    required = sorted(k for k, c in key_counts.items() if c == len(dicts))
+    return {"type": "object", "properties": properties, "required": required}
+
+
+def induce_tool_call(golds: Sequence[str]) -> ToolCallSpec:
+    """Induce per-tool argument signatures (name -> required/allowed keys)."""
+    per_name: dict[str, list[set]] = {}
+    for gold in golds:
+        obj = _json_or_none(gold)
+        if _is_tool_call(obj):
+            per_name.setdefault(obj["name"], []).append(
+                {str(k) for k in obj["arguments"].keys()}
+            )
+    if not per_name:
+        raise ValueError(
+            "no tool-call references found (each needs a JSON object with string "
+            "'name' + object 'arguments') — cannot induce a tool_call verifier"
+        )
+    tools: dict = {}
+    for name, calls in per_name.items():
+        allowed = set().union(*calls)
+        required = set.intersection(*calls)
+        tools[name] = {
+            "required": tuple(sorted(required)),
+            "allowed": tuple(sorted(allowed)),
+        }
+    return ToolCallSpec(tools=tools)
+
+
+def _char_class(chars: set[str]) -> Optional[str]:
+    if len(chars) == 1:
+        return re.escape(next(iter(chars)))
+    if all(c.isdigit() for c in chars):
+        return r"\d"
+    if all(c.isalpha() for c in chars):
+        return r"[A-Za-z]"
+    if all(c.isalnum() for c in chars):
+        return r"[A-Za-z0-9]"
+    return None
+
+
+def induce_regex(golds: Sequence[str]) -> Optional[str]:
+    """Conservative positional pattern induction over equal-length golds.
+
+    Returns ``None`` unless the golds share a length and every position
+    generalizes to a confident char class (the safe direction: refuse rather
+    than emit a loose pattern).
+    """
+    golds = [g.strip() for g in golds if g.strip()]
+    if len(golds) < 2:
+        return None
+    length = len(golds[0])
+    if any(len(g) != length for g in golds) or length == 0:
+        return None
+    parts: list[str] = []
+    for pos in range(length):
+        cls = _char_class({g[pos] for g in golds})
+        if cls is None:
+            return None
+        parts.append(cls)
+    # By construction every gold matches (each position's class is a superset of
+    # the observed chars), so no post-hoc confidence check is needed.
+    return "^" + "".join(parts) + "$"
+
+
+# ---------------------------------------------------------------------------
+# Source rendering
+# ---------------------------------------------------------------------------
+_HEADER = '''"""Deterministic reward verifier — generated by `soup reward synth` (v0.71.40).
+
+Kind: {kind}. Induced from {n_refs} reference output(s). EDIT FREELY — this is a
+self-contained, committable reward function. Wire it via:
+
+    training:
+      reward_fn: {rel_hint}
+
+Signature: reward_fn(completions, **kwargs) -> list[float]; the per-row gold answer
+arrives as kwargs["answer"] (a list aligned with completions), matching TRL/GRPO.
+"""
+
+import json
+import re
+
+
+def _last_content(completion):
+    return completion[-1]["content"] if completion else ""
+'''
+
+_NUMERIC_BODY = '''
+_NUMBER_RE = re.compile(r"[+-]?(?:\\d+\\.?\\d*|\\.\\d+)(?:[eE][+-]?\\d+)?")
+_BOXED_RE = re.compile(r"\\\\boxed\\{([^}]*)\\}")
+
+
+def _extract_number(text):
+    # Prefer an explicit answer marker (\\boxed{...} then ####) over "last number
+    # in the whole string", so trailing chit-chat numbers do not hijack the score.
+    boxed = _BOXED_RE.search(text)
+    if boxed:
+        nums = _NUMBER_RE.findall(boxed.group(1))
+        if nums:
+            return nums[-1]
+    if "####" in text:
+        nums = _NUMBER_RE.findall(text.split("####")[-1])
+        if nums:
+            return nums[-1]
+    nums = _NUMBER_RE.findall(text)
+    return nums[-1] if nums else None
+
+
+def _numbers_match(pred, gold, tol):
+    if pred is None:
+        return False
+    try:
+        if tol == 0:
+            # Exact: integer compare first so large ints dodge float rounding.
+            try:
+                return int(pred) == int(gold)
+            except (ValueError, TypeError):
+                pass
+        return abs(float(pred) - float(gold)) <= tol
+    except (ValueError, TypeError):
+        return False
+
+
+def reward_fn(completions, **kwargs):
+    answers = kwargs.get("answer", [])
+    out = []
+    for completion, expected in zip(completions, answers):
+        predicted = _extract_number(_last_content(completion))
+        gold = str(expected).strip()
+        out.append(1.0 if _numbers_match(predicted, gold, _TOLERANCE) else 0.0)
+    return out
+'''
+
+_JSON_SCHEMA_BODY = '''
+
+def _json_type(value):
+    if isinstance(value, bool):
+        return "boolean"
+    if isinstance(value, int):
+        return "integer"
+    if isinstance(value, float):
+        return "number"
+    if isinstance(value, str):
+        return "string"
+    if isinstance(value, list):
+        return "array"
+    if isinstance(value, dict):
+        return "object"
+    return "null"
+
+
+def _matches_schema(data):
+    expected = _SCHEMA.get("type")
+    if expected == "array":
+        return isinstance(data, list)
+    if expected == "object":
+        if not isinstance(data, dict):
+            return False
+        for key in _SCHEMA.get("required", []):
+            if key not in data:
+                return False
+        props = _SCHEMA.get("properties", {})
+        _NUMERIC = {"integer", "number"}
+        for key, spec in props.items():
+            if key in data:
+                want = spec.get("type")
+                got = _json_type(data[key])
+                if not want:
+                    continue  # presence-only (type varied across references)
+                if want in _NUMERIC and got in _NUMERIC:
+                    continue  # int/float interchangeable
+                if want != got:
+                    return False
+        return True
+    return True
+
+
+def reward_fn(completions, **kwargs):
+    out = []
+    for completion in completions:
+        content = _last_content(completion)
+        fenced = re.search(r"```(?:json)?\\s*(.*?)```", content, re.DOTALL)
+        if fenced:
+            content = fenced.group(1)
+        try:
+            data = json.loads(content.strip())
+        except (json.JSONDecodeError, ValueError):
+            out.append(0.0)
+            continue
+        out.append(1.0 if _matches_schema(data) else 0.0)
+    return out
+'''
+
+_REGEX_BODY = '''
+
+def reward_fn(completions, **kwargs):
+    out = []
+    for completion in completions:
+        content = _last_content(completion).strip()
+        out.append(1.0 if _PATTERN.fullmatch(content) else 0.0)
+    return out
+'''
+
+_TOOL_CALL_BODY = '''
+
+def reward_fn(completions, **kwargs):
+    out = []
+    for completion in completions:
+        content = _last_content(completion)
+        fenced = re.search(r"```(?:json)?\\s*(.*?)```", content, re.DOTALL)
+        if fenced:
+            content = fenced.group(1)
+        try:
+            data = json.loads(content.strip())
+        except (json.JSONDecodeError, ValueError):
+            out.append(0.0)
+            continue
+        if not (isinstance(data, dict) and isinstance(data.get("arguments"), dict)):
+            out.append(0.0)
+            continue
+        sig = _TOOLS.get(data.get("name"))
+        if sig is None:
+            out.append(0.0)
+            continue
+        keys = set(data["arguments"].keys())
+        required = set(sig["required"])
+        allowed = set(sig["allowed"])
+        out.append(1.0 if required <= keys <= allowed else 0.0)
+    return out
+'''
+
+
+def _safe_hint(value: object) -> str:
+    """One-line, quote/backslash-free display string for the header docstring.
+
+    ``rel_hint`` (the ``-o`` path) is the only value ``.format``'d RAW into the
+    generated source (everything else is baked via ``repr``). The file is later
+    exec'd, so a path containing ``\"\"\"`` + code must not be able to break out
+    of the docstring — strip control chars, quotes and backslashes and cap it.
+    """
+    text = str(value)
+    text = "".join(ch for ch in text if ch >= " " and ch not in '"\\')
+    return text[:200] or "reward.py"
+
+
+def render_verifier_py(kind: str, spec: Any, *, meta: Mapping[str, Any]) -> str:
+    """Render a self-contained reward-verifier ``.py`` source string.
+
+    The induced spec is baked in as a top-level constant via ``repr`` (so the
+    code bodies stay literal — no ``str.format`` over code containing braces).
+    """
+    n_refs = int(meta.get("n_refs", 0))
+    rel_hint = _safe_hint(meta.get("rel_hint", "reward.py"))
+    header = _HEADER.format(kind=kind, n_refs=n_refs, rel_hint=rel_hint)
+    if kind == "numeric":
+        if not isinstance(spec, NumericSpec):
+            raise TypeError("numeric verifier requires a NumericSpec")
+        const = f"\n_TOLERANCE = {spec.tolerance!r}\n"
+        body = const + _NUMERIC_BODY
+    elif kind == "json_schema":
+        if not isinstance(spec, dict):
+            raise TypeError("json_schema verifier requires a schema dict")
+        const = f"\n_SCHEMA = {spec!r}\n"
+        body = const + _JSON_SCHEMA_BODY
+    elif kind == "regex":
+        if not isinstance(spec, str):
+            raise TypeError("regex verifier requires a pattern string")
+        const = f"\n_PATTERN = re.compile({spec!r})\n"
+        body = const + _REGEX_BODY
+    elif kind == "tool_call":
+        if not isinstance(spec, ToolCallSpec):
+            raise TypeError("tool_call verifier requires a ToolCallSpec")
+        const = f"\n_TOOLS = {spec.tools!r}\n"
+        body = const + _TOOL_CALL_BODY
+    else:
+        raise ValueError(f"unknown verifier kind: {kind!r} (options: {', '.join(KINDS)})")
+    return header + body
+
+
+# ---------------------------------------------------------------------------
+# Negatives + calibration
+# ---------------------------------------------------------------------------
+def perturb_negatives(golds: Sequence[str], kind: str) -> list[str]:
+    """Deterministically corrupt golds into known-bad outputs for calibration."""
+    negatives: list[str] = []
+    for gold in golds:
+        gold = gold.strip()
+        if kind == "numeric":
+            num = _NUMBER_RE.findall(gold)
+            if num:
+                try:
+                    negatives.append(str(float(num[-1]) + 9999.0))
+                except ValueError:
+                    negatives.append("not-a-number")
+            else:
+                negatives.append("not-a-number")
+        elif kind == "json_schema":
+            negatives.append("this is definitely not json {")
+        elif kind == "tool_call":
+            negatives.append("this is definitely not json {")
+            obj = _json_or_none(gold)
+            if _is_tool_call(obj):
+                # Valid JSON, WRONG tool name — a "just parse JSON" verifier would
+                # accept this; a name-bound verifier must reject it.
+                negatives.append(
+                    json.dumps({"name": "__nonexistent_tool__",
+                                "arguments": obj["arguments"]})
+                )
+                # Valid JSON, right name, a foreign arg key outside `allowed`.
+                negatives.append(
+                    json.dumps({"name": obj["name"],
+                                "arguments": {"__foreign_arg__": 1}})
+                )
+        elif kind == "regex":
+            negatives.append(gold + "ZZZ_definitely_wrong")
+        else:
+            raise ValueError(f"unknown kind: {kind!r} (options: {', '.join(KINDS)})")
+    # Always include a couple of universal degenerate cases.
+    negatives.extend(["", "the model rambled without answering at all"])
+    return negatives
+
+
+def calibrate(
+    reward_fn: Callable[..., list[float]],
+    positives: Sequence[str],
+    negatives: Sequence[str],
+    *,
+    kind: str = "",
+    min_discrimination: float = DEFAULT_MIN_DISCRIMINATION,
+) -> CalibrationReport:
+    """Run ``reward_fn`` over positives (should accept) + negatives (should reject).
+
+    ``discrimination = pos_accept - neg_accept``. Refuses when: (a) the verifier
+    fails to accept ``>= _MIN_SELF_ACCEPT`` of its own references (the induced
+    spec does not fit the golds), (b) ``discrimination <= 0`` (no separation at
+    all — a hard floor the user cannot disable via ``--min-discrimination 0``),
+    or (c) ``discrimination < min_discrimination``.
+    """
+    def _accept_rate(
+        items: Sequence[str], answers: Sequence[str]
+    ) -> tuple[float, int]:
+        if not items:
+            return 0.0, 0
+        completions = [[{"role": "assistant", "content": s}] for s in items]
+        scores = reward_fn(completions, answer=list(answers))
+        accepted = sum(1 for s in scores if float(s) >= 0.5)
+        return accepted / len(items), accepted
+
+    # Positives are scored as content==gold. Negatives are corrupted completions
+    # but must be scored against the REAL gold (cycled), else a gold-comparison
+    # verifier would accept a corrupted answer matched against itself.
+    pos_accept, tp = _accept_rate(positives, positives)
+    if positives:
+        neg_answers = [positives[j % len(positives)] for j in range(len(negatives))]
+    else:
+        neg_answers = list(negatives)
+    neg_accept, fp = _accept_rate(negatives, neg_answers)
+    discrimination = pos_accept - neg_accept
+    precision = (tp / (tp + fp)) if (tp + fp) > 0 else 0.0
+    recall = pos_accept
+    refused = False
+    reason = "verifier discriminates references from perturbed negatives"
+    if pos_accept < _MIN_SELF_ACCEPT:
+        refused = True
+        reason = (
+            f"verifier rejects its own references (accepts only "
+            f"{pos_accept:.0%}, need >= {_MIN_SELF_ACCEPT:.0%}) — the induced "
+            "spec does not fit the golds"
+        )
+    elif discrimination <= 0.0:
+        refused = True
+        reason = (
+            f"verifier accepts perturbed negatives as readily as references "
+            f"(accepts {pos_accept:.0%} of refs and {neg_accept:.0%} of negatives, "
+            "discrimination <= 0) — degenerate, refusing regardless of "
+            "--min-discrimination"
+        )
+    elif discrimination < min_discrimination:
+        refused = True
+        reason = (
+            f"discrimination {discrimination:.2f} < required {min_discrimination:.2f} "
+            f"(accepts {pos_accept:.0%} of references but also {neg_accept:.0%} of "
+            "perturbed negatives) — the verifier is too permissive to reward-train on"
+        )
+    return CalibrationReport(
+        kind=kind,
+        positives=len(positives),
+        negatives=len(negatives),
+        pos_accept=pos_accept,
+        neg_accept=neg_accept,
+        discrimination=discrimination,
+        precision=precision,
+        recall=recall,
+        refused=refused,
+        reason=reason,
+    )
+
+
+# ---------------------------------------------------------------------------
+# Orchestration (pure — the CLI does write→load→calibrate)
+# ---------------------------------------------------------------------------
+def _induce(kind: str, golds: Sequence[str], tolerance: Optional[float]) -> Any:
+    if kind == "numeric":
+        return induce_numeric(golds, tolerance=tolerance)
+    if kind == "json_schema":
+        return induce_json_schema(golds)
+    if kind == "tool_call":
+        return induce_tool_call(golds)
+    if kind == "regex":
+        pattern = induce_regex(golds)
+        if pattern is None:
+            raise ValueError(
+                "could not induce a confident regex from the references "
+                "(golds must share a length + positional char classes)"
+            )
+        return pattern
+    raise ValueError(f"unknown kind: {kind!r} (options: {', '.join(KINDS)})")
+
+
+def synthesize(
+    rows: Sequence[Mapping[str, Any]],
+    *,
+    field: str = "answer",
+    kind: str = "auto",
+    tolerance: Optional[float] = None,
+    rel_hint: str = "reward.py",
+) -> SynthResult:
+    """Detect + induce + render (no calibration — the CLI loads then calibrates)."""
+    if kind != "auto" and kind not in KINDS:
+        raise ValueError(f"unknown kind: {kind!r} (options: auto, {', '.join(KINDS)})")
+    golds = extract_golds(rows, field=field)
+    resolved = kind
+    if kind == "auto":
+        resolved = detect_kind(golds)
+        if resolved is None:
+            raise ValueError(
+                "could not auto-detect a deterministic verifier kind from the "
+                "references — pass --kind numeric|json_schema|regex|tool_call"
+            )
+    spec = _induce(resolved, golds, tolerance)
+    source = render_verifier_py(
+        resolved, spec, meta={"n_refs": len(golds), "rel_hint": rel_hint}
+    )
+    return SynthResult(kind=resolved, source=source, spec=spec, report=None)

tests/test_v07140.py

@@ -0,0 +1,855 @@
+"""Tests for v0.71.40 — `soup reward synth` + comma-split reward loader (#311).
+
+Covers:
+- utils/reward_synth.py: extract_golds / detect_kind / inducers / render / calibrate / synthesize.
+- commands/reward.py: `soup reward synth` CLI (happy / refuse / plan-only / guards).
+- trainer/rewards.load_reward_fns + grpo._select_reward_fn comma-split (#311).
+- config reward_fn validator; envs docstring fix.
+"""
+
+from __future__ import annotations
+
+import ast
+import importlib.util
+from pathlib import Path
+
+import pytest
+
+from soup_cli.utils import reward_synth as rs
+
+
+# ---------------------------------------------------------------------------
+# extract_golds
+# ---------------------------------------------------------------------------
+class TestExtractGolds:
+    def test_answer_field(self):
+        rows = [{"prompt": "q", "answer": "5"}, {"prompt": "q2", "answer": 7}]
+        assert rs.extract_golds(rows, field="answer") == ["5", "7"]
+
+    def test_chat_last_assistant(self):
+        rows = [
+            {"messages": [{"role": "user", "content": "hi"},
+                          {"role": "assistant", "content": "gold-out"}]}
+        ]
+        assert rs.extract_golds(rows, field="answer") == ["gold-out"]
+
+    def test_dict_gold_is_json_serialised(self):
+        rows = [{"answer": {"k": 1}}]
+        assert rs.extract_golds(rows, field="answer") == ['{"k": 1}']
+
+    def test_no_resolvable_gold_raises(self):
+        with pytest.raises(ValueError, match="no gold"):
+            rs.extract_golds([{"prompt": "q"}], field="answer")
+
+    def test_rows_must_be_sequence(self):
+        with pytest.raises(TypeError):
+            rs.extract_golds("not-a-list", field="answer")
+
+
+# ---------------------------------------------------------------------------
+# detect_kind
+# ---------------------------------------------------------------------------
+class TestDetectKind:
+    def test_numeric(self):
+        assert rs.detect_kind(["5", "12", "-3"]) == "numeric"
+
+    def test_json_schema(self):
+        assert rs.detect_kind(['{"a": 1}', '{"a": 2, "b": 3}']) == "json_schema"
+
+    def test_tool_call_precedes_json_schema(self):
+        golds = ['{"name": "search", "arguments": {"q": "x"}}',
+                 '{"name": "lookup", "arguments": {"id": 3}}']
+        assert rs.detect_kind(golds) == "tool_call"
+
+    def test_regex_last_resort(self):
+        # Same-length structured strings, not numbers, not JSON.
+        assert rs.detect_kind(["2031-01-02", "1999-12-31"]) == "regex"
+
+    def test_uninferrable_returns_none(self):
+        assert rs.detect_kind(["hello world", "a completely different sentence here"]) is None
+
+
+# ---------------------------------------------------------------------------
+# inducers
+# ---------------------------------------------------------------------------
+class TestInduceNumeric:
+    def test_int(self):
+        spec = rs.induce_numeric(["1", "2", "3"])
+        assert spec.is_float is False and spec.tolerance == 0.0
+
+    def test_float(self):
+        spec = rs.induce_numeric(["1.5", "2"])
+        assert spec.is_float is True and spec.tolerance == rs.DEFAULT_NUMERIC_TOLERANCE
+
+    def test_negative_tolerance_raises(self):
+        with pytest.raises(ValueError, match="finite"):
+            rs.induce_numeric(["1.0"], tolerance=-1)
+
+
+class TestInduceJsonSchema:
+    def test_keys_types_required(self):
+        schema = rs.induce_json_schema(['{"a": 1, "b": "x"}', '{"a": 2}'])
+        assert schema["type"] == "object"
+        assert set(schema["properties"]) == {"a", "b"}
+        assert schema["properties"]["a"]["type"] == "integer"
+        assert schema["required"] == ["a"]  # b missing from row 2
+
+    def test_array(self):
+        schema = rs.induce_json_schema(["[1, 2]", "[3]"])
+        assert schema["type"] == "array"
+
+
+class TestInduceToolCall:
+    def test_names_and_arg_keys(self):
+        golds = ['{"name": "search", "arguments": {"q": "x"}}',
+                 '{"name": "lookup", "arguments": {"id": 3}}']
+        spec = rs.induce_tool_call(golds)
+        assert set(spec.names) == {"search", "lookup"}
+        assert set(spec.arg_keys) == {"q", "id"}
+
+    def test_per_tool_required_and_allowed(self):
+        # search always has q (required); lookup varies id/limit (id required only if
+        # in every lookup call). allowed = union per name.
+        golds = ['{"name": "search", "arguments": {"q": "a"}}',
+                 '{"name": "search", "arguments": {"q": "b", "page": 2}}',
+                 '{"name": "lookup", "arguments": {"id": 1}}']
+        spec = rs.induce_tool_call(golds)
+        assert spec.tools["search"]["required"] == ("q",)
+        assert set(spec.tools["search"]["allowed"]) == {"q", "page"}
+        assert spec.tools["lookup"]["required"] == ("id",)
+
+    def test_no_tool_calls_raises(self):
+        with pytest.raises(ValueError, match="no tool-call"):
+            rs.induce_tool_call(["just some text", "5"])
+
+
+class TestInduceRegex:
+    def test_confident(self):
+        pat = rs.induce_regex(["2031-01-02", "1999-12-31"])
+        import re
+        assert pat is not None and re.fullmatch(pat, "2000-05-06")
+
+    def test_not_confident_returns_none(self):
+        assert rs.induce_regex(["ab", "abcde", "x"]) is None
+
+
+# ---------------------------------------------------------------------------
+# render + round-trip
+# ---------------------------------------------------------------------------
+def _load_reward_fn(source: str, tmp_path: Path):
+    p = tmp_path / "gen_reward.py"
+    p.write_text(source, encoding="utf-8")
+    spec = importlib.util.spec_from_file_location("gen_reward", p)
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    return mod.reward_fn
+
+
+def _completions(*contents):
+    return [[{"role": "assistant", "content": c}] for c in contents]
+
+
+class TestRenderVerifier:
+    def test_compiles_and_defines_reward_fn(self, tmp_path):
+        src = rs.render_verifier_py("numeric", rs.NumericSpec(False, 0.0), meta={"n_refs": 3})
+        compile(src, "<gen>", "exec")  # syntactically valid
+        fn = _load_reward_fn(src, tmp_path)
+        assert callable(fn)
+
+    def test_numeric_roundtrip(self, tmp_path):
+        src = rs.render_verifier_py("numeric", rs.NumericSpec(False, 0.0), meta={"n_refs": 2})
+        fn = _load_reward_fn(src, tmp_path)
+        out = fn(_completions("The answer is 5", "42"), answer=["5", "7"])
+        assert out == [1.0, 0.0]
+
+    def test_json_schema_roundtrip(self, tmp_path):
+        schema = {"type": "object", "properties": {"a": {"type": "integer"}}, "required": ["a"]}
+        src = rs.render_verifier_py("json_schema", schema, meta={"n_refs": 2})
+        fn = _load_reward_fn(src, tmp_path)
+        out = fn(_completions('{"a": 1}', "not json"), answer=["", ""])
+        assert out == [1.0, 0.0]
+
+    def test_tool_call_roundtrip(self, tmp_path):
+        spec = rs.ToolCallSpec(tools={"search": {"required": ("q",), "allowed": ("q",)}})
+        src = rs.render_verifier_py("tool_call", spec, meta={"n_refs": 1})
+        fn = _load_reward_fn(src, tmp_path)
+        out = fn(_completions('{"name": "search", "arguments": {"q": "x"}}',
+                              '{"name": "evil", "arguments": {}}'), answer=["", ""])
+        assert out == [1.0, 0.0]
+
+    def test_tool_call_binding_rejects_hacks(self, tmp_path):
+        # The code-review H1: two tools; a call must not borrow the other's arg key
+        # or drop a required one. Union-of-all-keys + subset-check would accept both.
+        spec = rs.induce_tool_call(['{"name": "search", "arguments": {"q": "x"}}',
+                                    '{"name": "lookup", "arguments": {"id": 3}}'])
+        fn = _load_reward_fn(rs.render_verifier_py("tool_call", spec, meta={"n_refs": 2}),
+                             tmp_path)
+        out = fn(_completions(
+            '{"name": "search", "arguments": {"q": "y"}}',   # ok
+            '{"name": "search", "arguments": {}}',           # missing required q
+            '{"name": "search", "arguments": {"id": 3}}',    # borrowed lookup's key
+        ), answer=["", "", ""])
+        assert out == [1.0, 0.0, 0.0]
+
+    def test_numeric_boxed_beats_trailing_number(self, tmp_path):
+        # M1: a \boxed answer wins over an incidental trailing number.
+        src = rs.render_verifier_py("numeric", rs.NumericSpec(False, 0.0), meta={"n_refs": 1})
+        fn = _load_reward_fn(src, tmp_path)
+        out = fn(_completions(r"So \boxed{4}. This took 12 seconds."), answer=["4"])
+        assert out == [1.0]
+
+    def test_numeric_bigint_exact(self, tmp_path):
+        # M1: two 17-digit ints that collide under float() must NOT match at tol 0.
+        src = rs.render_verifier_py("numeric", rs.NumericSpec(False, 0.0), meta={"n_refs": 1})
+        fn = _load_reward_fn(src, tmp_path)
+        out = fn(_completions("90071992547409915"), answer=["90071992547409916"])
+        assert out == [0.0]
+
+    def test_json_schema_int_float_symmetric(self, tmp_path):
+        # H2: gold key induced as integer; a completion with 5.0 must still match.
+        schema = {"type": "object", "properties": {"x": {"type": "integer"}}, "required": ["x"]}
+        fn = _load_reward_fn(rs.render_verifier_py("json_schema", schema, meta={"n_refs": 1}),
+                             tmp_path)
+        out = fn(_completions('{"x": 5.0}'), answer=[""])
+        assert out == [1.0]
+
+    def test_json_schema_typeless_key_presence_only(self, tmp_path):
+        # H2: a key with no agreed type (varied across refs) is presence-only.
+        schema = {"type": "object", "properties": {"a": {}}, "required": ["a"]}
+        fn = _load_reward_fn(rs.render_verifier_py("json_schema", schema, meta={"n_refs": 1}),
+                             tmp_path)
+        assert fn(_completions('{"a": "text"}', '{"a": 9}'), answer=["", ""]) == [1.0, 1.0]
+
+    def test_baked_constant_reflects_spec(self):
+        # Mutation-guard: a different tolerance must change the emitted source.
+        a = rs.render_verifier_py("numeric", rs.NumericSpec(True, 0.001), meta={"n_refs": 1})
+        b = rs.render_verifier_py("numeric", rs.NumericSpec(True, 0.999), meta={"n_refs": 1})
+        assert "0.001" in a and "0.001" not in b
+
+    def test_rel_hint_cannot_inject_code(self):
+        # security: rel_hint (the -o path) is .format'd raw into the header
+        # docstring, and the file is later exec'd — a triple-quote break must not
+        # survive into the emitted source.
+        evil = 'x\n"""\nimport os\nopen("PWNED", "w").close()\n"""'
+        src = rs.render_verifier_py("numeric", rs.NumericSpec(False, 0.0),
+                                    meta={"n_refs": 1, "rel_hint": evil})
+        benign = rs.render_verifier_py("numeric", rs.NumericSpec(False, 0.0),
+                                       meta={"n_refs": 1, "rel_hint": "reward.py"})
+        compile(src, "<g>", "exec")  # still valid Python (no docstring break-out)
+        # No EXTRA triple-quote injected (the hint can't close the docstring early)
+        # and the quotes needed to make the payload executable are stripped.
+        assert src.count('"""') == benign.count('"""')
+        assert 'open("PWNED"' not in src
+
+    def test_hostile_gold_stays_repr_safe(self):
+        # Untrusted tool names/keys are baked via repr(); they cannot break out.
+        spec = rs.ToolCallSpec(tools={
+            'ev"l\n"""x': {"required": (), "allowed": ('a"\nb',)}})
+        src = rs.render_verifier_py("tool_call", spec, meta={"n_refs": 1})
+        compile(src, "<g>", "exec")
+
+
+class TestInduceJsonSchemaEdges:
+    def test_mixed_shapes_refused(self):
+        with pytest.raises(ValueError, match="mix"):
+            rs.induce_json_schema(['{"a": 1}', "[1, 2]"])
+
+    def test_no_containers_refused(self):
+        with pytest.raises(ValueError, match="no JSON"):
+            rs.induce_json_schema(["hello", "world"])
+
+    def test_inconsistent_key_type_is_typeless(self):
+        schema = rs.induce_json_schema(['{"a": 1}', '{"a": "x"}'])
+        assert schema["properties"]["a"] == {}  # no type constraint
+
+
+class TestInduceNumericGuard:
+    def test_non_numeric_refused(self):
+        with pytest.raises(ValueError, match="not numeric"):
+            rs.induce_numeric(["hello", "world"])
+
+
+class TestCalibrateFloors:
+    def test_hard_floor_refuses_at_zero_threshold(self):
+        # M2: an always-accept verifier is refused even with min_discrimination=0.
+        def always_one(completions, **kw):
+            return [1.0] * len(completions)
+        rep = rs.calibrate(always_one, ["a", "b"], ["x", "y"], min_discrimination=0.0)
+        assert rep.refused is True and rep.discrimination <= 0.0
+
+    def test_self_accept_floor(self):
+        # H2: a verifier accepting only half its own refs is refused (floor 0.9).
+        state = {"i": 0}
+        def half(completions, **kw):
+            out = []
+            for _ in completions:
+                out.append(1.0 if state["i"] % 2 == 0 else 0.0)
+                state["i"] += 1
+            return out
+        rep = rs.calibrate(half, ["a", "b", "c", "d"], ["x"], min_discrimination=0.1)
+        assert rep.refused is True and "references" in rep.reason
+
+
+class TestLoadRewardFnsGuards:
+    def test_none_friendly(self):
+        from soup_cli.trainer.rewards import load_reward_fns
+        with pytest.raises(ValueError, match="must be a string"):
+            load_reward_fns(None)
+
+    def test_blank_rejected(self):
+        from soup_cli.trainer.rewards import load_reward_fns
+        with pytest.raises(ValueError, match="blank"):
+            load_reward_fns("   ")
+
+    def test_duplicate_rejected(self):
+        from soup_cli.trainer.rewards import load_reward_fns
+        with pytest.raises(ValueError, match="twice"):
+            load_reward_fns("accuracy,accuracy")
+
+
+# ---------------------------------------------------------------------------
+# perturb + calibrate
+# ---------------------------------------------------------------------------
+class TestPerturbNegatives:
+    def test_negatives_differ_from_golds(self):
+        golds = ["5", "12"]
+        negs = rs.perturb_negatives(golds, "numeric")
+        assert negs and all(n not in golds for n in negs)
+
+
+class TestCalibrate:
+    def test_always_one_verifier_refused(self):
+        def always_one(completions, **kw):
+            return [1.0] * len(completions)
+        rep = rs.calibrate(always_one, ["a", "b"], ["x", "y"], min_discrimination=0.5)
+        assert rep.refused is True and rep.discrimination == 0.0
+
+    def test_good_verifier_passes(self):
+        def exact(completions, **kw):
+            answers = kw.get("answer", [])
+            return [1.0 if c[-1]["content"] == a else 0.0
+                    for c, a in zip(completions, answers)]
+        rep = rs.calibrate(exact, ["a", "b"], ["neg1", "neg2"], min_discrimination=0.5)
+        assert rep.refused is False and rep.pos_accept == 1.0 and rep.neg_accept == 0.0
+
+    def test_discrimination_boundary_exact(self):
+        # Accept ALL positives (clears the 0.9 self-accept floor) but half the
+        # negatives → discrimination 0.5 == threshold → pass (not < threshold).
+        phase = {"neg": False}
+
+        def v(completions, **kw):
+            if not phase["neg"]:
+                phase["neg"] = True
+                return [1.0] * len(completions)  # positives: all accepted
+            return [1.0 if i % 2 == 0 else 0.0 for i in range(len(completions))]
+        rep = rs.calibrate(v, ["a", "b"], ["x", "y"], min_discrimination=0.5)
+        assert rep.pos_accept == 1.0 and rep.neg_accept == 0.5
+        assert rep.discrimination == 0.5 and rep.refused is False
+
+
+# ---------------------------------------------------------------------------
+# synthesize (end-to-end pure)
+# ---------------------------------------------------------------------------
+class TestSynthesize:
+    def test_numeric_from_calculator_shape(self):
+        rows = [{"prompt": "2+2", "answer": "4"}, {"prompt": "3+3", "answer": "6"}]
+        res = rs.synthesize(rows, field="answer", kind="auto")
+        assert res.kind == "numeric"
+        assert "def reward_fn" in res.source
+
+    def test_bad_kind_raises(self):
+        with pytest.raises(ValueError, match="kind"):
+            rs.synthesize([{"answer": "1"}], field="answer", kind="bogus")
+
+
+# ---------------------------------------------------------------------------
+# CLI — soup reward synth
+# ---------------------------------------------------------------------------
+def _write_jsonl(path: Path, rows):
+    import json as _json
+    path.write_text("\n".join(_json.dumps(r) for r in rows), encoding="utf-8")
+
+
+class TestRewardSynthCli:
+    def _runner(self):
+        from typer.testing import CliRunner
+        return CliRunner()
+
+    def test_help(self):
+        from soup_cli.commands.reward import app
+        res = self._runner().invoke(app, ["synth", "--help"])
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        assert "reward" in res.output.lower()
+
+    def test_numeric_happy(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"),
+                     [{"prompt": "2+2", "answer": "4"}, {"prompt": "3+3", "answer": "6"}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py"])
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        assert Path("reward.py").exists()
+        # The emitted verifier agrees with the golds.
+        fn = _load_reward_fn(Path("reward.py").read_text(encoding="utf-8"), tmp_path)
+        assert fn(_completions("4", "99"), answer=["4", "6"]) == [1.0, 0.0]
+
+    def test_cant_induce_is_error_exit_1(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        # All golds empty → regex can't be induced → INDUCTION ERROR (exit 1),
+        # distinct from a calibration refusal (exit 2). No file written.
+        _write_jsonl(Path("refs.jsonl"),
+                     [{"answer": ""}, {"answer": ""}, {"answer": ""}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py",
+                                          "--kind", "regex"])
+        assert res.exit_code == 1, (res.output, repr(res.exception))
+        assert "regex" in res.output.lower()
+        assert not Path("reward.py").exists()
+
+    def test_refusal_deletes_file(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        # A numeric verifier with a huge tolerance accepts the perturbed (+9999)
+        # negatives too → discrimination below the default 0.5 → REFUSED (exit 2),
+        # and the emitted file is removed (no partial artifact left behind).
+        _write_jsonl(Path("refs.jsonl"), [{"answer": str(i)} for i in range(8)])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py",
+                                          "--kind", "numeric", "--tolerance", "1e9"])
+        assert res.exit_code == 2, (res.output, repr(res.exception))
+        assert "discrimination" in res.output.lower() and "refus" in res.output.lower()
+        assert not Path("reward.py").exists()
+
+    def test_plan_only_writes_nothing(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}, {"answer": "6"}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py",
+                                          "--plan-only"])
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        assert not Path("reward.py").exists()
+        # Spec-specific fields, not just the kind name in the panel title.
+        assert "numeric" in res.output.lower() and "tolerance" in res.output.lower()
+        assert "float=false" in res.output.lower().replace(" ", "")
+
+    def test_output_must_be_py(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.txt"])
+        assert res.exit_code == 1
+        assert ".py" in res.output
+
+    def test_overwrite_without_force_rejected(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}, {"answer": "6"}])
+        Path("reward.py").write_text("# existing\n", encoding="utf-8")
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py"])
+        assert res.exit_code == 1
+        assert "force" in res.output.lower()
+        assert Path("reward.py").read_text(encoding="utf-8") == "# existing\n"
+
+    def test_bad_kind_rejected(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py",
+                                          "--kind", "bogus"])
+        assert res.exit_code == 1
+        assert "kind" in res.output.lower()
+
+
+# ---------------------------------------------------------------------------
+# Blocking riders — #311 comma-split loader + reward_fn validator + envs docstring
+# ---------------------------------------------------------------------------
+class TestCommaSplitLoader:
+    def test_two_names_two_callables(self):
+        from soup_cli.trainer.rewards import load_reward_fns
+        fns = load_reward_fns("accuracy,format")
+        assert isinstance(fns, list) and len(fns) == 2
+        assert {f.__name__ for f in fns} == {"accuracy_reward", "format_reward"}
+
+    def test_single_name_single_element_list(self):
+        from soup_cli.trainer.rewards import load_reward_fns
+        fns = load_reward_fns("accuracy")
+        assert isinstance(fns, list) and len(fns) == 1
+        assert fns[0].__name__ == "accuracy_reward"
+
+    def test_whitespace_around_segments(self):
+        from soup_cli.trainer.rewards import load_reward_fns
+        fns = load_reward_fns(" accuracy , format ")
+        assert len(fns) == 2
+
+    def test_empty_segment_raises(self):
+        from soup_cli.trainer.rewards import load_reward_fns
+        with pytest.raises(ValueError, match="empty"):
+            load_reward_fns("accuracy,")
+
+    def test_verifiable_with_domain(self):
+        from soup_cli.trainer.rewards import load_reward_fns
+        fns = load_reward_fns("verifiable,format", verifiable_domain="math")
+        assert len(fns) == 2
+
+
+class TestSelectRewardFn:
+    def test_comma_split_returns_list(self):
+        from soup_cli.config.schema import TrainingConfig
+        from soup_cli.trainer.grpo import _select_reward_fn
+        tcfg = TrainingConfig(reward_fn="accuracy,format")
+        out = _select_reward_fn(tcfg, "cpu", False)
+        assert isinstance(out, list) and len(out) == 2
+
+    def test_single_returns_callable(self):
+        from soup_cli.config.schema import TrainingConfig
+        from soup_cli.trainer.grpo import _select_reward_fn
+        tcfg = TrainingConfig(reward_fn="accuracy")
+        out = _select_reward_fn(tcfg, "cpu", False)
+        assert callable(out) and not isinstance(out, list)
+
+
+class TestRewardFnValidator:
+    def test_null_byte_rejected(self):
+        from pydantic import ValidationError
+
+        from soup_cli.config.schema import TrainingConfig
+        with pytest.raises(ValidationError, match="null"):
+            TrainingConfig(reward_fn="acc\x00uracy")
+
+    def test_empty_comma_segment_rejected(self):
+        from pydantic import ValidationError
+
+        from soup_cli.config.schema import TrainingConfig
+        with pytest.raises(ValidationError, match="empty"):
+            TrainingConfig(reward_fn="accuracy,,format")
+
+    def test_overlong_rejected(self):
+        from pydantic import ValidationError
+
+        from soup_cli.config.schema import TrainingConfig
+        with pytest.raises(ValidationError, match="512"):
+            TrainingConfig(reward_fn="a" * 600)
+
+    def test_comma_combo_accepted(self):
+        from soup_cli.config.schema import TrainingConfig
+        assert TrainingConfig(reward_fn="accuracy,format").reward_fn == "accuracy,format"
+
+
+class TestRewardFnTaskGate:
+    def _cfg(self, task, reward_fn):
+        from soup_cli.config.loader import load_config_from_string
+        return load_config_from_string(
+            f"base: sshleifer/tiny-gpt2\ntask: {task}\n"
+            f"data:\n  train: d.jsonl\ntraining:\n  reward_fn: {reward_fn}\n"
+        )
+
+    def test_grpo_allows_comma(self):
+        cfg = self._cfg("grpo", "accuracy,format")
+        assert cfg.training.reward_fn == "accuracy,format"
+
+    def test_ppo_rejects_comma(self):
+        # load_config_from_string re-wraps pydantic's ValidationError as ValueError.
+        with pytest.raises(ValueError, match="grpo"):
+            self._cfg("ppo", "accuracy,format")
+
+    def test_sft_rejects_comma(self):
+        with pytest.raises(ValueError, match="grpo"):
+            self._cfg("sft", "accuracy,format")
+
+    def test_ppo_single_name_ok(self):
+        cfg = self._cfg("ppo", "accuracy")
+        assert cfg.training.reward_fn == "accuracy"
+
+
+class TestReviewFixes:
+    def test_null_gold_falls_through_to_messages(self):
+        rows = [{"answer": None,
+                 "messages": [{"role": "assistant", "content": "gold"}]}]
+        assert rs.extract_golds(rows, field="answer") == ["gold"]
+
+    def test_null_gold_only_skipped(self):
+        with pytest.raises(ValueError, match="no gold"):
+            rs.extract_golds([{"answer": None}], field="answer")
+
+    def test_non_finite_tolerance_raises(self):
+        with pytest.raises(ValueError, match="finite"):
+            rs.induce_numeric(["1.0"], tolerance=float("nan"))
+        with pytest.raises(ValueError, match="finite"):
+            rs.induce_numeric(["1.0"], tolerance=float("inf"))
+
+    def test_cli_bad_min_discrimination(self, tmp_path, monkeypatch):
+        from typer.testing import CliRunner
+
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}])
+        res = CliRunner().invoke(app, ["synth", "refs.jsonl", "-o", "r.py",
+                                       "--min-discrimination", "2.0"])
+        assert res.exit_code == 1 and "min-discrimination" in res.output
+
+    def test_cli_bad_tolerance(self, tmp_path, monkeypatch):
+        from typer.testing import CliRunner
+
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}])
+        res = CliRunner().invoke(app, ["synth", "refs.jsonl", "-o", "r.py",
+                                       "--tolerance", "-1"])
+        assert res.exit_code == 1 and "tolerance" in res.output
+
+
+class TestPerKindPipeline:
+    """Every kind runs the FULL write->load->calibrate pipeline (not just numeric)."""
+
+    def _runner(self):
+        from typer.testing import CliRunner
+        return CliRunner()
+
+    def _synth(self, tmp_path, monkeypatch, rows, extra=()):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), rows)
+        return self._runner().invoke(
+            app, ["synth", "refs.jsonl", "-o", "reward.py", *extra])
+
+    def test_json_schema_end_to_end(self, tmp_path, monkeypatch):
+        rows = [{"answer": '{"city": "Paris", "pop": 2}'},
+                {"answer": '{"city": "Rome", "pop": 3}'}]
+        res = self._synth(tmp_path, monkeypatch, rows)
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        fn = _load_reward_fn(Path("reward.py").read_text(encoding="utf-8"), tmp_path)
+        assert fn(_completions('{"city": "X", "pop": 9}', "nope"),
+                  answer=["", ""]) == [1.0, 0.0]
+
+    def test_tool_call_end_to_end(self, tmp_path, monkeypatch):
+        rows = [{"answer": '{"name": "search", "arguments": {"q": "a"}}'},
+                {"answer": '{"name": "search", "arguments": {"q": "b"}}'}]
+        res = self._synth(tmp_path, monkeypatch, rows)
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        fn = _load_reward_fn(Path("reward.py").read_text(encoding="utf-8"), tmp_path)
+        assert fn(_completions('{"name": "search", "arguments": {"q": "z"}}',
+                               '{"name": "search", "arguments": {}}'),
+                  answer=["", ""]) == [1.0, 0.0]
+
+    def test_regex_end_to_end(self, tmp_path, monkeypatch):
+        rows = [{"answer": "2031-01-02"}, {"answer": "1999-12-31"},
+                {"answer": "2000-05-06"}]
+        res = self._synth(tmp_path, monkeypatch, rows, extra=["--kind", "regex"])
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        fn = _load_reward_fn(Path("reward.py").read_text(encoding="utf-8"), tmp_path)
+        assert fn(_completions("2020-11-11", "not a date"), answer=["", ""]) == [1.0, 0.0]
+
+
+class TestPerturbNegativesPerKind:
+    def test_tool_call_generates_valid_json_hacks(self):
+        golds = ['{"name": "search", "arguments": {"q": "x"}}']
+        negs = rs.perturb_negatives(golds, "tool_call")
+        # Must include a VALID-JSON wrong-name negative and a foreign-arg negative,
+        # not just the syntactically-invalid string.
+        assert any('"__nonexistent_tool__"' in n for n in negs)
+        assert any("__foreign_arg__" in n for n in negs)
+
+    def test_json_schema_negatives(self):
+        negs = rs.perturb_negatives(['{"a": 1}'], "json_schema")
+        assert negs and all(n for n in negs[:1])  # non-empty invalid-json negative
+
+    def test_unknown_kind_raises(self):
+        with pytest.raises(ValueError, match="unknown kind"):
+            rs.perturb_negatives(["x"], "bogus")
+
+
+class TestOutputReport:
+    def _runner(self):
+        from typer.testing import CliRunner
+        return CliRunner()
+
+    def test_report_written_and_parseable(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}, {"answer": "6"}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py",
+                                          "--output-report", "rep.json"])
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        import json as _json
+        rep = _json.loads(Path("rep.json").read_text(encoding="utf-8"))
+        assert rep["kind"] == "numeric" and rep["refused"] is False
+        assert set(rep) >= {"pos_accept", "neg_accept", "discrimination", "precision"}
+
+    def test_bad_report_path_fails_before_writing_verifier(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}, {"answer": "6"}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py",
+                                          "--output-report", "../escape.json"])
+        assert res.exit_code == 1, (res.output, repr(res.exception))
+        # The verifier is never written when the report path is rejected up front.
+        assert not Path("reward.py").exists()
+
+
+class TestCliRobustness:
+    def _runner(self):
+        from typer.testing import CliRunner
+        return CliRunner()
+
+    def test_missing_references_file(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        res = self._runner().invoke(app, ["synth", "nope.jsonl", "-o", "reward.py"])
+        assert res.exit_code == 1, (res.output, repr(res.exception))
+
+    def test_empty_references_file(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        Path("refs.jsonl").write_text("\n\n", encoding="utf-8")
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py"])
+        assert res.exit_code == 1 and "no JSON-object rows" in res.output
+
+    def test_force_overwrites(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}, {"answer": "6"}])
+        Path("reward.py").write_text("# stale\n", encoding="utf-8")
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py",
+                                          "--force"])
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        assert "# stale" not in Path("reward.py").read_text(encoding="utf-8")
+
+    def test_custom_field(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"gold": "4"}, {"gold": "6"}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "reward.py",
+                                          "--field", "gold"])
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+        assert Path("reward.py").exists()
+
+    def test_output_outside_cwd_rejected(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        _write_jsonl(Path("refs.jsonl"), [{"answer": "4"}])
+        res = self._runner().invoke(app, ["synth", "refs.jsonl", "-o", "../evil.py"])
+        assert res.exit_code == 1 and "cwd" in res.output.lower()
+
+    def test_references_outside_cwd_rejected(self, tmp_path, monkeypatch):
+        from soup_cli.commands.reward import app
+        monkeypatch.chdir(tmp_path)
+        res = self._runner().invoke(app, ["synth", "../secret.jsonl", "-o", "reward.py"])
+        assert res.exit_code == 1 and "cwd" in res.output.lower()
+
+    def test_registered_on_top_level_cli(self):
+        from soup_cli.cli import app as root
+        res = self._runner().invoke(root, ["reward", "synth", "--help"])
+        assert res.exit_code == 0, (res.output, repr(res.exception))
+
+
+class TestBoundaries:
+    def test_self_accept_boundary_9_of_10(self):
+        # 9/10 == 0.9 self-accept floor → NOT refused; 8/9 < 0.9 → refused.
+        pos9 = ["p"] * 10
+        seq = {"i": 0}
+
+        def accept_first_9(completions, **kw):
+            out = []
+            for _ in completions:
+                out.append(1.0 if seq["i"] < 9 else 0.0)
+                seq["i"] += 1
+            return out
+        rep = rs.calibrate(accept_first_9, pos9, ["n"], min_discrimination=0.1)
+        assert abs(rep.pos_accept - 0.9) < 1e-9 and rep.refused is False
+
+    def test_self_accept_below_floor_refused(self):
+        seq = {"i": 0}
+
+        def accept_first_7(completions, **kw):  # 7/9 ~= 0.78 < 0.9
+            out = []
+            for _ in completions:
+                out.append(1.0 if seq["i"] < 7 else 0.0)
+                seq["i"] += 1
+            return out
+        rep = rs.calibrate(accept_first_7, ["p"] * 9, ["n"], min_discrimination=0.1)
+        assert rep.pos_accept < rs._MIN_SELF_ACCEPT and rep.refused is True
+        assert "own references" in rep.reason
+
+    def test_numeric_tolerance_inclusive(self, tmp_path):
+        src = rs.render_verifier_py("numeric", rs.NumericSpec(True, 0.1), meta={"n_refs": 1})
+        fn = _load_reward_fn(src, tmp_path)
+        # |5.1 - 5.0| == 0.1 == tol → inclusive match.
+        assert fn(_completions("5.1"), answer=["5.0"]) == [1.0]
+        assert fn(_completions("5.2"), answer=["5.0"]) == [0.0]
+
+    def test_precision_value(self):
+        def accept_all(completions, **kw):
+            return [1.0] * len(completions)
+        # 2 refs accepted (tp=2), 3 negatives all accepted (fp=3) → precision 2/5.
+        rep = rs.calibrate(accept_all, ["a", "b"], ["x", "y", "z"])
+        assert abs(rep.precision - 0.4) < 1e-9
+
+    def test_detect_kind_numeric_confidence(self):
+        # 9/10 numeric → numeric; 5/10 → not numeric (falls to regex/None).
+        assert rs.detect_kind(["1", "2", "3", "4", "5", "6", "7", "8", "9", "x"]) == "numeric"
+
+    def test_reward_fn_512_boundary(self):
+        from soup_cli.config.schema import TrainingConfig
+        assert TrainingConfig(reward_fn="a" * 512).reward_fn == "a" * 512
+        from pydantic import ValidationError
+        with pytest.raises(ValidationError, match="512"):
+            TrainingConfig(reward_fn="a" * 513)
+
+
+class TestVerifiableComboGate:
+    def test_comma_verifiable_without_domain_rejected(self):
+        # #8: "accuracy,verifiable" must fail at config parse (like bare
+        # "verifiable"), not silently defer to a runtime crash.
+        from soup_cli.config.loader import load_config_from_string
+        with pytest.raises(ValueError, match="verifiable_domain"):
+            load_config_from_string(
+                "base: sshleifer/tiny-gpt2\ntask: grpo\ndata:\n  train: d.jsonl\n"
+                "training:\n  reward_fn: accuracy,verifiable\n"
+            )
+
+    def test_comma_verifiable_with_domain_ok(self):
+        from soup_cli.config.loader import load_config_from_string
+        cfg = load_config_from_string(
+            "base: sshleifer/tiny-gpt2\ntask: grpo\ndata:\n  train: d.jsonl\n"
+            "training:\n  reward_fn: accuracy,verifiable\n  verifiable_domain: math\n"
+        )
+        assert cfg.training.reward_fn == "accuracy,verifiable"
+
+
+class TestRewardFnValidatorBlank:
+    def test_blank_rejected(self):
+        from pydantic import ValidationError
+
+        from soup_cli.config.schema import TrainingConfig
+        with pytest.raises(ValidationError, match="blank"):
+            TrainingConfig(reward_fn="   ")
+
+    def test_bool_rejected(self):
+        from pydantic import ValidationError
+
+        from soup_cli.config.schema import TrainingConfig
+        with pytest.raises(ValidationError, match="string"):
+            TrainingConfig(reward_fn=True)
+
+
+class TestEnvsDocstringFix:
+    def test_calculator_docstring(self):
+        from soup_cli.envs import calculator
+        assert "reward_fn='math'" not in (calculator.__doc__ or "")
+        assert "verifiable" in (calculator.__doc__ or "")
+
+    def test_guess_number_docstring(self):
+        from soup_cli.envs import guess_number
+        assert "reward_fn='math'" not in (guess_number.__doc__ or "")
+        assert "verifiable" in (guess_number.__doc__ or "")
+
+
+class TestNoTopLevelTorch:
+    def test_reward_synth_has_no_top_level_torch(self):
+        src = Path(rs.__file__).read_text(encoding="utf-8")
+        tree = ast.parse(src)
+        for node in tree.body:  # module-level only
+            if isinstance(node, ast.Import):
+                assert all(not n.name.startswith(("torch", "transformers", "peft"))
+                           for n in node.names)
+            if isinstance(node, ast.ImportFrom):
+                assert not (node.module or "").startswith(("torch", "transformers", "peft"))