soup-cli 0.71.39__tar.gz → 0.71.40__tar.gz

{soup_cli-0.71.39 → soup_cli-0.71.40}/CHANGELOG.md

@@ -12,6 +12,45 @@ reproducing 70+ versions of notes.
 
 ## [Unreleased]
 
+## [0.71.40] - 2026-07-19
+
+**`soup reward synth`: auto-generate a deterministic reward verifier from your data.**
+Point it at a JSONL of reference (gold) outputs and it infers a verifier, emits a
+readable / committable `.py` reward function, and — the moat — *refuses* to emit one
+that can't tell your references from auto-generated bad answers. Nothing in
+TRL / Unsloth / Axolotl / OpenRLHF synthesizes a reward; every reward today is
+hand-written, hand-picked, or a trained-weights artifact.
+
+### Added
+
+- **`soup reward synth <references.jsonl> -o reward.py`** — deterministic verifier
+  synthesis. Four families, auto-detected (or pick with `--kind`): `numeric`
+  (last-number / `\boxed{}` / `####` extraction, exact or `--tolerance`), `json_schema`
+  (induced keys + types + required), `regex` (positional char-classes over
+  equal-length golds), `tool_call` (per-tool `required`/`allowed` argument binding).
+  The emitted file is self-contained and rides `load_reward_fn`'s existing `.py`
+  path — no new trusted-exec surface; you read, edit, commit, and diff it.
+- **Mandatory calibration report** — the synthesized verifier is loaded back and run
+  against its own references (must accept ≥90%) and auto-perturbed negatives (must
+  reject). A degenerate always-accept verifier is **refused** (`exit 2`), never
+  silently emitted. `--plan-only` reports the induced spec without writing;
+  `--output-report` persists the calibration JSON.
+- **Comma-separated `reward_fn` (`"accuracy,format"`)** now trains — it resolves to a
+  reward *ensemble* (`GRPOTrainer(reward_funcs=[...])`, and unlocks the `rm_ensemble`
+  reward-hack detector which needs ≥2 rewards). GRPO-only, validated at config-parse
+  time. Fixes a recipe (`deepseek-v3-reasoning`) that shipped exactly this and
+  previously crashed with `Unknown reward function` (#311).
+
+### Changed / Fixed
+
+- `training.reward_fn` gains a field validator (null-byte / blank / oversize /
+  empty-comma-segment rejection) — the oldest arbitrary-code field was the least
+  guarded. Comma + `verifiable` without a `verifiable_domain` now fails at parse
+  time like the bare `verifiable` form.
+- `envs/calculator.py` / `envs/guess_number.py` docstrings corrected: the reward is
+  `reward_fn: verifiable` + `verifiable_domain: math` (the bare `reward_fn='math'`
+  they showed was never valid).
+
 ## [0.71.39] - 2026-07-19
 
 **"CI for weights, not prompts": close the evidence loop.** `soup ship`'s verdict is

{soup_cli-0.71.39 → soup_cli-0.71.40}/CONTRIBUTING.md

@@ -120,7 +120,7 @@ src/soup_cli/
   templates/          - 21 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0, +4 compliance v0.71.35)
   ui/                 - Web UI (FastAPI + HTML/JS SPA)
 
-tests/                - Test suite (317 files, 16387 tests)
+tests/                - Test suite (318 files, 16490 tests)
 examples/             - Real-world config examples and datasets
 ```
 

{soup_cli-0.71.39 → soup_cli-0.71.40}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: soup-cli
-Version: 0.71.39
+Version: 0.71.40
 Summary: Fine-tune and post-train LLMs in one command. No SSH, no config hell.
 Project-URL: Homepage, https://github.com/MakazhanAlpamys/Soup
 Project-URL: Repository, https://github.com/MakazhanAlpamys/Soup
@@ -193,29 +193,37 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.39 — CI for weights, not prompts.** `soup ship`'s verdict is now something Soup can emit, commit, review, and bind to the exact model that produced it — so a fine-tuning gate renders on every PR instead of asking you to hand-edit a JSON file.
+**v0.71.40 — `soup reward synth`: generate a reward verifier from your data.** Point it at a JSONL of reference (gold) outputs and it infers a deterministic verifier, writes a readable / committable `.py` reward function, and — the part nobody else does — *refuses* to emit one that can't tell your references from bad answers. Nothing in TRL / Unsloth / Axolotl synthesizes a reward.
 
-- **Output is finally input.** `soup ship --emit-evidence ev.json` writes the scores in the
-  `--evidence` schema, so a run replays offline into an identical verdict — the round-trip the
-  old `--output` couldn't do.
-- **Commit the gate policy.** Put `eval.ship` in your `soup.yaml` (`forgetting_threshold`,
-  `general_suite`, `task_mode`, …) and `soup ship --config soup.yaml` reads it — the gate is now
-  reviewable in a PR diff. Explicit flags still win.
-- **Provenance-bound evidence.** `--config --emit-evidence` STAMPS a `config_sha` onto the
-  evidence; `--config --evidence` GATES on it — a PR that changed the recipe but forgot to
-  recompute its evidence is refused (exit 3). Tuning the threshold never falsely invalidates it.
-- **The verdict on your PR.** `soup ship --push owner/repo#N` posts the SHIP / DON'T-SHIP card as
-  a GitHub comment; `soup ci init --config soup.yaml` wires the whole thing into CI.
+- **Four families, auto-detected** (or pick with `--kind`): `numeric` (`\boxed{}` / `####` / last-number,
+  exact or `--tolerance`), `json_schema` (induced keys + types + required), `regex`, and `tool_call`
+  (per-tool `required`/`allowed` argument binding — a call can't borrow another tool's keys).
+- **A calibration report is the moat.** The verifier is loaded back and run against its own references
+  (must accept ≥ 90%) and auto-generated bad answers (must reject). A degenerate always-accept verifier
+  is **refused** (exit 2), never silently shipped. `--plan-only` previews; `--output-report` saves the JSON.
+- **No new exec surface.** The emitted file rides the existing `reward_fn: reward.py` path — you read,
+  edit, commit, and diff it like any other reward.
+- **Reward ensembles now train.** `reward_fn: "accuracy,format"` loads as multiple rewards (and unlocks
+  the `rm_ensemble` reward-hack detector) — a recipe that shipped exactly this used to crash. (#311)
 
 ```bash
-# producer (train job): stamp the scores with the config that produced them
-soup ship --evidence scores.json --config soup.yaml --emit-evidence ship_evidence.json
+# synth a verifier from reference answers, calibrate it, keep it only if it discriminates
+soup reward synth references.jsonl -o reward.py --output-report calib.json
 
-# gate (PR CI): refuse evidence that doesn't match the committed config
-soup ship --evidence ship_evidence.json --config soup.yaml --push owner/repo#42
-soup ci init --config soup.yaml   # writes .github/workflows/soup-gate.yml
+# then train against the reward you just generated
+# training: { reward_fn: reward.py }   (or an ensemble: reward_fn: "accuracy,format")
 ```
 
+<details>
+<summary>Previous release — v0.71.39, CI for weights not prompts (emit + provenance-bind the ship verdict)</summary>
+
+`soup ship`'s verdict became emittable, committable, and provenance-bound: `--emit-evidence` makes a
+run replay into an identical verdict, `eval.ship` in `soup.yaml` + `--config` makes the gate policy
+reviewable, and `--config` binds evidence to the exact recipe that produced it (stale evidence → exit 3).
+`soup ship --push owner/repo#N` posts the SHIP / DON'T-SHIP card on the PR.
+
+</details>
+
 <details>
 <summary>Previous release — v0.71.38, The gate grows teeth (real leg-2 regression gate)</summary>
 

{soup_cli-0.71.39 → soup_cli-0.71.40}/README.md

@@ -49,29 +49,37 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.39 — CI for weights, not prompts.** `soup ship`'s verdict is now something Soup can emit, commit, review, and bind to the exact model that produced it — so a fine-tuning gate renders on every PR instead of asking you to hand-edit a JSON file.
-
-- **Output is finally input.** `soup ship --emit-evidence ev.json` writes the scores in the
-  `--evidence` schema, so a run replays offline into an identical verdict — the round-trip the
-  old `--output` couldn't do.
-- **Commit the gate policy.** Put `eval.ship` in your `soup.yaml` (`forgetting_threshold`,
-  `general_suite`, `task_mode`, …) and `soup ship --config soup.yaml` reads it — the gate is now
-  reviewable in a PR diff. Explicit flags still win.
-- **Provenance-bound evidence.** `--config --emit-evidence` STAMPS a `config_sha` onto the
-  evidence; `--config --evidence` GATES on it — a PR that changed the recipe but forgot to
-  recompute its evidence is refused (exit 3). Tuning the threshold never falsely invalidates it.
-- **The verdict on your PR.** `soup ship --push owner/repo#N` posts the SHIP / DON'T-SHIP card as
-  a GitHub comment; `soup ci init --config soup.yaml` wires the whole thing into CI.
+**v0.71.40 — `soup reward synth`: generate a reward verifier from your data.** Point it at a JSONL of reference (gold) outputs and it infers a deterministic verifier, writes a readable / committable `.py` reward function, and — the part nobody else does — *refuses* to emit one that can't tell your references from bad answers. Nothing in TRL / Unsloth / Axolotl synthesizes a reward.
+
+- **Four families, auto-detected** (or pick with `--kind`): `numeric` (`\boxed{}` / `####` / last-number,
+  exact or `--tolerance`), `json_schema` (induced keys + types + required), `regex`, and `tool_call`
+  (per-tool `required`/`allowed` argument binding — a call can't borrow another tool's keys).
+- **A calibration report is the moat.** The verifier is loaded back and run against its own references
+  (must accept ≥ 90%) and auto-generated bad answers (must reject). A degenerate always-accept verifier
+  is **refused** (exit 2), never silently shipped. `--plan-only` previews; `--output-report` saves the JSON.
+- **No new exec surface.** The emitted file rides the existing `reward_fn: reward.py` path — you read,
+  edit, commit, and diff it like any other reward.
+- **Reward ensembles now train.** `reward_fn: "accuracy,format"` loads as multiple rewards (and unlocks
+  the `rm_ensemble` reward-hack detector) — a recipe that shipped exactly this used to crash. (#311)
 
 ```bash
-# producer (train job): stamp the scores with the config that produced them
-soup ship --evidence scores.json --config soup.yaml --emit-evidence ship_evidence.json
+# synth a verifier from reference answers, calibrate it, keep it only if it discriminates
+soup reward synth references.jsonl -o reward.py --output-report calib.json
 
-# gate (PR CI): refuse evidence that doesn't match the committed config
-soup ship --evidence ship_evidence.json --config soup.yaml --push owner/repo#42
-soup ci init --config soup.yaml   # writes .github/workflows/soup-gate.yml
+# then train against the reward you just generated
+# training: { reward_fn: reward.py }   (or an ensemble: reward_fn: "accuracy,format")
 ```
 
+<details>
+<summary>Previous release — v0.71.39, CI for weights not prompts (emit + provenance-bind the ship verdict)</summary>
+
+`soup ship`'s verdict became emittable, committable, and provenance-bound: `--emit-evidence` makes a
+run replay into an identical verdict, `eval.ship` in `soup.yaml` + `--config` makes the gate policy
+reviewable, and `--config` binds evidence to the exact recipe that produced it (stale evidence → exit 3).
+`soup ship --push owner/repo#N` posts the SHIP / DON'T-SHIP card on the PR.
+
+</details>
+
 <details>
 <summary>Previous release — v0.71.38, The gate grows teeth (real leg-2 regression gate)</summary>
 

{soup_cli-0.71.39 → soup_cli-0.71.40}/docs/commands.md

@@ -182,6 +182,10 @@ soup draft measure ... --min-acceptance 0.6 -o report.json  Exit 2 below the flo
 soup draft distill --target <tuned> --draft-base <tiny> --data d.jsonl -o draft/  Distil a DENSE speculative-decoding draft + register it (v0.71.33)
 soup draft distill ... --steps N --device cpu --force --plan-only  Training budget / device / overwrite -o / render the config only
 soup draft list                               List local drafts that `soup serve --auto-spec` will pick up (v0.71.33)
+soup reward synth refs.jsonl -o reward.py     Synthesize a deterministic reward verifier from gold outputs (v0.71.40)
+soup reward synth ... --kind numeric|json_schema|regex|tool_call  Force a verifier family (default: auto-detect)
+soup reward synth ... --plan-only             Report the induced spec + calibration plan; write nothing
+soup reward synth ... --output-report r.json --min-discrimination 0.5  Save the calibration JSON / set the refusal threshold (exit 0 emit / 2 refuse / 1 error)
 soup tui                                      Full-screen Textual dashboard (requires [tui] extra)
 soup train --config soup.yaml --profile       Record torch.profiler trace to <output>/profiles/
 soup --log-level quiet|normal|verbose|debug   Global logging tier (Rich-formatted)

{soup_cli-0.71.39 → soup_cli-0.71.40}/docs/training.md

@@ -866,6 +866,37 @@ training:
   reward_fn: ./my_reward.py
 ```
 
+**Reward ensembles** — list several rewards, comma-separated, and they combine (GRPO only).
+This also unlocks the `rm_ensemble` reward-hack detector, which needs ≥ 2 rewards:
+```yaml
+training:
+  reward_fn: "accuracy,format"   # both are scored every step
+```
+
+### Synthesize a verifier from your data (`soup reward synth`)
+
+Don't hand-write a verifier — generate one from reference (gold) outputs. Soup infers a
+*deterministic* verifier (numeric / JSON-schema / regex / tool-call), writes a readable, editable
+`.py`, and **refuses to emit** one that can't tell your references from auto-generated bad answers
+(the mandatory calibration report). The emitted file is a normal `reward_fn: reward.py`.
+
+```bash
+# infer + calibrate + emit (exit 0 kept, 2 refused, 1 error)
+soup reward synth references.jsonl -o reward.py --output-report calib.json
+
+# preview the induced spec without writing anything
+soup reward synth references.jsonl --plan-only
+
+# force a family instead of auto-detecting
+soup reward synth answers.jsonl -o reward.py --kind numeric --tolerance 1e-6
+```
+
+References are a JSONL where each row's gold answer is in an `answer` field (override with
+`--field`) or the last assistant turn of a `messages` list. `--min-discrimination` sets how
+strongly the verifier must separate references from perturbed negatives before it's emitted.
+v1 is deterministic families only — a `\boxed{}`/`####` marker helps the numeric verifier, and
+completions are prompted to mark their answer (standard RLVR practice).
+
 ### Verifiable Rewards (RLVR)
 
 Use `reward_fn: verifiable` with a `verifiable_domain` for deterministic, math-checkable rewards — no judge model, no heuristics. Great for GRPO on math, code, or structured-output tasks.

{soup_cli-0.71.39 → soup_cli-0.71.40}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "soup-cli"
-version = "0.71.39"
+version = "0.71.40"
 description = "Fine-tune and post-train LLMs in one command. No SSH, no config hell."
 readme = "README.md"
 license = "Apache-2.0"

{soup_cli-0.71.39 → soup_cli-0.71.40}/src/soup_cli/__init__.py

@@ -1,3 +1,3 @@
 """Soup CLI — Fine-tune and post-train LLMs in one command."""
 
-__version__ = "0.71.39"
+__version__ = "0.71.40"

{soup_cli-0.71.39 → soup_cli-0.71.40}/src/soup_cli/cli.py

@@ -563,6 +563,18 @@ app.add_typer(
     ),
 )
 
+# v0.71.40 — `soup reward synth` auto-generate a deterministic verifier.
+from soup_cli.commands import reward as _reward_cmd  # noqa: E402
+
+app.add_typer(
+    _reward_cmd.app,
+    name="reward",
+    help=(
+        "Synthesize a deterministic reward verifier from reference outputs, with "
+        "a calibration report that refuses degenerate verifiers (v0.71.40)."
+    ),
+)
+
 
 def _rewrite_advise_argv(argv: list) -> list:
     """Inject `run` between `advise` and a non-subcommand first argument.

soup_cli-0.71.40/src/soup_cli/commands/reward.py

@@ -0,0 +1,225 @@
+"""soup reward — synthesize a deterministic reward verifier (v0.71.40).
+
+    soup reward synth <references.jsonl> -o reward.py
+        [--kind auto|numeric|json_schema|regex|tool_call]
+
+Infers a deterministic verifier from a dataset of reference (gold) outputs, emits a
+readable / committable ``.py`` that rides ``load_reward_fn``'s existing ``.py`` path,
+and REFUSES to emit a degenerate verifier via a mandatory calibration report.
+
+Exit codes mirror ``soup ship`` / ``soup shrink``: 0 = emitted, 2 = refused (the
+verifier could not discriminate references from perturbed negatives), 1 = usage /
+runtime error.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+import os
+from typing import NoReturn, Optional
+
+import typer
+from rich.console import Console
+from rich.markup import escape
+from rich.panel import Panel
+from rich.table import Table
+
+from soup_cli.utils import reward_synth as rs
+from soup_cli.utils.paths import atomic_write_text, enforce_under_cwd_and_no_symlink
+
+app = typer.Typer(help="Synthesize a deterministic reward verifier from reference outputs.")
+console = Console()
+
+_MAX_INPUT_BYTES = 64 * 1024 * 1024
+_MAX_ROWS = 1_000_000
+_ALLOWED_KINDS = ("auto",) + rs.KINDS
+
+
+@app.callback()
+def _reward() -> None:
+    """Reward-verifier tooling. Forces ``synth`` to be a named subcommand (a
+    single-command Typer app otherwise collapses and eats the subcommand token)."""
+
+
+def _fail(message: str, code: int = 1) -> NoReturn:
+    """Print a red error and exit (raises internally — a forgotten ``raise`` at a
+    call site can never silently become a no-op; mirrors ``commands/ship.py``)."""
+    console.print(f"[red]{escape(message)}[/]")
+    raise typer.Exit(code)
+
+
+def _read_jsonl(path: str, label: str) -> list[dict]:
+    """Read a JSONL file: cwd-contained, O_NOFOLLOW, size- and row-capped."""
+    enforce_under_cwd_and_no_symlink(path, label)
+    flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0)
+    try:
+        fd = os.open(path, flags)
+    except OSError as exc:
+        raise ValueError(f"{label} unreadable: {exc}") from exc
+    rows: list[dict] = []
+    with os.fdopen(fd, "r", encoding="utf-8") as handle:
+        if os.fstat(handle.fileno()).st_size > _MAX_INPUT_BYTES:
+            raise ValueError(f"{label} exceeds {_MAX_INPUT_BYTES} bytes")
+        for line in handle:
+            line = line.strip()
+            if not line:
+                continue
+            try:
+                row = json.loads(line)
+            except json.JSONDecodeError:
+                continue
+            if isinstance(row, dict):
+                rows.append(row)
+            if len(rows) >= _MAX_ROWS:
+                break
+    return rows
+
+
+def _spec_summary(kind: str, spec: object) -> str:
+    """One-line human summary of the induced spec (for --plan-only / the panel)."""
+    if kind == "numeric" and isinstance(spec, rs.NumericSpec):
+        return f"numeric (float={spec.is_float}, tolerance={spec.tolerance})"
+    if kind == "json_schema" and isinstance(spec, dict):
+        props = ", ".join(sorted(spec.get("properties", {}))) or "(none)"
+        return f"json_schema (type={spec.get('type')}, keys={props})"
+    if kind == "tool_call" and isinstance(spec, rs.ToolCallSpec):
+        return f"tool_call (names={list(spec.names)}, arg_keys={list(spec.arg_keys)})"
+    if kind == "regex":
+        return f"regex (pattern={spec})"
+    return kind
+
+
+def _render_report_panel(report: rs.CalibrationReport, kind: str, out_path: str) -> Panel:
+    table = Table(show_header=False, box=None, pad_edge=False)
+    table.add_column(justify="right", style="dim")
+    table.add_column()
+    table.add_row("verifier kind", escape(kind))
+    table.add_row("references (accepted)", f"{report.positives} ({report.pos_accept:.0%})")
+    table.add_row("negatives (accepted)", f"{report.negatives} ({report.neg_accept:.0%})")
+    table.add_row("discrimination", f"{report.discrimination:.2f}")
+    table.add_row("precision", f"{report.precision:.2f}")
+    table.add_row("emitted", escape(out_path))
+    return Panel(table, title="[bold green]reward verifier synthesized[/]",
+                 border_style="green")
+
+
+@app.command()
+def synth(
+    references: str = typer.Argument(
+        ..., help="JSONL of reference (gold) outputs — a gold field or chat messages."
+    ),
+    output: Optional[str] = typer.Option(
+        None, "-o", "--output", help="Where to write the verifier .py."
+    ),
+    kind: str = typer.Option(
+        "auto", "--kind",
+        help="Verifier family: auto | numeric | json_schema | regex | tool_call.",
+    ),
+    field: str = typer.Option(
+        "answer", "--field", help="Gold-output field (default: answer)."
+    ),
+    tolerance: Optional[float] = typer.Option(
+        None, "--tolerance", help="Numeric match tolerance (numeric kind only)."
+    ),
+    min_discrimination: float = typer.Option(
+        rs.DEFAULT_MIN_DISCRIMINATION, "--min-discrimination",
+        help="Refuse to emit unless (accept_rate refs - accept_rate negatives) >= this.",
+    ),
+    force: bool = typer.Option(False, "--force", help="Overwrite an existing output."),
+    plan_only: bool = typer.Option(
+        False, "--plan-only", help="Detect + report the induced spec; write nothing."
+    ),
+    output_report: Optional[str] = typer.Option(
+        None, "--output-report", help="Also write the calibration report as JSON."
+    ),
+) -> None:
+    """Synthesize a deterministic reward verifier from reference outputs."""
+    if kind not in _ALLOWED_KINDS:
+        _fail(f"unknown --kind {kind!r}; options: {', '.join(_ALLOWED_KINDS)}")
+    if not 0.0 <= min_discrimination <= 1.0:
+        _fail("--min-discrimination must be in [0.0, 1.0]")
+    if tolerance is not None and (not math.isfinite(tolerance) or tolerance < 0):
+        _fail("--tolerance must be a finite, non-negative number")
+
+    # Read references.
+    try:
+        rows = _read_jsonl(references, "references path")
+    except (ValueError, OSError) as exc:
+        _fail(str(exc))
+    if not rows:
+        _fail("references file has no JSON-object rows")
+
+    # Detect + induce + render (no file written yet).
+    try:
+        result = rs.synthesize(rows, field=field, kind=kind, tolerance=tolerance,
+                               rel_hint=(output or "reward.py"))
+    except (ValueError, TypeError) as exc:
+        _fail(str(exc))
+
+    if plan_only:
+        console.print(Panel(
+            escape(_spec_summary(result.kind, result.spec)),
+            title=f"[bold]plan: {escape(result.kind)} verifier[/]", border_style="cyan"))
+        raise typer.Exit(0)
+
+    # Output guards.
+    if not output:
+        _fail("must pass -o/--output to emit a verifier (or use --plan-only)")
+    if not output.endswith(".py"):
+        _fail(f"output must be a .py file, got {output!r}")
+    try:
+        enforce_under_cwd_and_no_symlink(output, "output path")
+    except (ValueError, OSError) as exc:
+        _fail(str(exc))
+    if os.path.exists(output) and not force:
+        _fail(f"{output!r} already exists — pass --force to overwrite")
+    # Validate the optional report path UP FRONT so a bad report path can never
+    # cause us to destroy an already-written, already-accepted verifier later.
+    if output_report:
+        try:
+            enforce_under_cwd_and_no_symlink(output_report, "report path")
+        except (ValueError, OSError) as exc:
+            _fail(str(exc))
+
+    # Write, then LOAD it back through the real reward-loader path (round-trip
+    # validation) and calibrate the loaded callable.
+    atomic_write_text(result.source, output)
+    try:
+        from soup_cli.trainer.rewards import load_reward_fn
+        reward_fn = load_reward_fn(output)
+        golds = rs.extract_golds(rows, field=field)
+        negatives = rs.perturb_negatives(golds, result.kind)
+        report = rs.calibrate(reward_fn, golds, negatives, kind=result.kind,
+                              min_discrimination=min_discrimination)
+    except Exception as exc:  # noqa: BLE001 — clean up the partial artifact
+        _cleanup(output)
+        _fail(f"calibration failed: {exc}")
+
+    # Write the diagnostic report (path already validated up front) for BOTH the
+    # accepted and refused cases. Best-effort: a write failure here warns but must
+    # not delete an otherwise-valid verifier.
+    if output_report:
+        try:
+            from dataclasses import asdict
+            atomic_write_text(json.dumps(asdict(report), indent=2), output_report)
+        except OSError as exc:
+            console.print(f"[yellow]Warning: could not write report: {escape(str(exc))}[/]")
+
+    if report.refused:
+        _cleanup(output)
+        console.print(Panel(
+            escape(report.reason),
+            title="[bold red]verifier refused (not emitted)[/]", border_style="red"))
+        raise typer.Exit(2)
+
+    console.print(_render_report_panel(report, result.kind, output))
+    raise typer.Exit(0)
+
+
+def _cleanup(path: str) -> None:
+    """Remove a just-written artifact; best-effort (never masks the real error)."""
+    try:
+        os.remove(path)
+    except OSError:
+        pass

{soup_cli-0.71.39 → soup_cli-0.71.40}/src/soup_cli/config/schema.py

@@ -1107,10 +1107,38 @@ class TrainingConfig(BaseModel):
     reward_fn: Optional[str] = Field(
         default="accuracy",
         description=(
-            "Reward function: 'accuracy', 'format', 'verifiable', "
-            "or path to custom .py file"
+            "Reward function: 'accuracy', 'format', 'verifiable', a path to a "
+            "custom .py file, or a comma-separated ensemble of the above "
+            "(e.g. 'accuracy,format') — the comma form is GRPO-only (v0.71.40)."
         ),
     )
+    @field_validator("reward_fn", mode="before")
+    @classmethod
+    def _validate_reward_fn_field(cls, value: Any) -> Optional[str]:
+        """v0.71.40 #311 — shape-only validation (containment enforced at load).
+
+        Accepts a comma-separated spec ("accuracy,format"); rejects null bytes,
+        oversize, and empty comma segments so a stray comma fails loud rather
+        than silently dropping a reward.
+        """
+        if value is None:
+            return None
+        if isinstance(value, bool) or not isinstance(value, str):
+            raise ValueError(
+                f"reward_fn must be a string, got {type(value).__name__}"
+            )
+        if "\x00" in value:
+            raise ValueError("reward_fn must not contain null bytes")
+        if len(value) > 512:
+            raise ValueError("reward_fn must be <= 512 chars")
+        if not value.strip():
+            raise ValueError("reward_fn must not be blank")
+        if any(not seg.strip() for seg in value.split(",")):
+            raise ValueError(
+                "reward_fn has an empty comma segment — remove the stray comma"
+            )
+        return value
+
     # RLVR — verifiable reward domain (Part C of v0.25.0)
     verifiable_domain: Optional[Literal["math", "code", "json_schema"]] = Field(
         default=None,
@@ -2978,8 +3006,18 @@ class TrainingConfig(BaseModel):
 
     @model_validator(mode="after")
     def _validate_verifiable_reward(self) -> "TrainingConfig":
-        """RLVR: reward_fn='verifiable' requires verifiable_domain."""
-        if self.reward_fn == "verifiable" and self.verifiable_domain is None:
+        """RLVR: reward_fn='verifiable' requires verifiable_domain.
+
+        Comma-aware (v0.71.40 #311): ``"accuracy,verifiable"`` must fail at
+        config-parse time exactly like the bare ``"verifiable"`` form, not only
+        at trainer construction.
+        """
+        segments = (
+            [s.strip() for s in self.reward_fn.split(",")]
+            if isinstance(self.reward_fn, str)
+            else []
+        )
+        if "verifiable" in segments and self.verifiable_domain is None:
             raise ValueError(
                 "reward_fn='verifiable' requires verifiable_domain "
                 "(one of: math, code, json_schema)"
@@ -4662,6 +4700,25 @@ class SoupConfig(BaseModel):
             )
         return self
 
+    @model_validator(mode="after")
+    def _validate_reward_fn_multi_compat(self) -> "SoupConfig":
+        """v0.71.40 #311 — a comma-separated reward_fn is GRPO-only.
+
+        Only ``trainer/grpo.py::_select_reward_fn`` resolves a multi-reward spec
+        into a ``reward_funcs=[...]`` list; ``trainer/ppo.py`` (and every other
+        consumer) calls the single-name loader, so a comma there would silently
+        pass config validation and then crash at training start with
+        ``Unknown reward function``. Footgun-reject it up front (mirrors the
+        online_dpo / asr / lisa task gates).
+        """
+        rf = self.training.reward_fn
+        if isinstance(rf, str) and "," in rf and self.task != "grpo":
+            raise ValueError(
+                f"a comma-separated training.reward_fn (an ensemble, {rf!r}) is "
+                f"only supported for task='grpo'; got task={self.task!r}"
+            )
+        return self
+
     @model_validator(mode="after")
     def _validate_asr_compat(self) -> "SoupConfig":
         """v0.71.32 — ASR (Whisper) gate.

{soup_cli-0.71.39 → soup_cli-0.71.40}/src/soup_cli/envs/calculator.py

@@ -1,7 +1,8 @@
 """Calculator tool-use rollout env — v0.71.30.
 
 Generates a deterministic set of single-operation arithmetic problems as GRPO
-prompt+answer rows. Score with ``reward_fn='math'`` (or ``'accuracy'``).
+prompt+answer rows. Score with ``reward_fn: verifiable`` +
+``verifiable_domain: math`` (or ``reward_fn: accuracy``).
 
 Usage: ``training.rollout_backend='openenv'`` +
 ``training.rollout_func='soup_cli.envs.calculator:rollout'``.

{soup_cli-0.71.39 → soup_cli-0.71.40}/src/soup_cli/envs/guess_number.py

@@ -2,7 +2,7 @@
 
 Generates deterministic single-shot number-deduction puzzles (constraints that
 pin a unique integer) as GRPO prompt+answer rows. Score with
-``reward_fn='math'`` (or ``'accuracy'``).
+``reward_fn: verifiable`` + ``verifiable_domain: math`` (or ``reward_fn: accuracy``).
 
 Honesty: this is single-shot *deduction*, not interactive guess-and-feedback —
 the live openenv contract does not pass the model, so a true guessing loop is

{soup_cli-0.71.39 → soup_cli-0.71.40}/src/soup_cli/trainer/grpo.py

@@ -129,7 +129,9 @@ def _read_attr(obj: Any, name: str) -> Any:
     return getattr(obj, name, None)
 
 
-def _select_reward_fn(tcfg: TrainingConfig, device: str, trust_remote_code: bool) -> Any:
+def _select_reward_fn(
+    tcfg: TrainingConfig, device: str, trust_remote_code: bool
+) -> "Any":  # Callable | list[Callable] (a single reward or a comma-split ensemble)
     """Choose the GRPO reward function (v0.71.30).
 
     When ``tcfg.prm_reward`` is set, a trained Soup PRM scores each completion's
@@ -142,9 +144,13 @@ def _select_reward_fn(tcfg: TrainingConfig, device: str, trust_remote_code: bool
         from soup_cli.utils.prm_reward import build_prm_reward_fn
 
         return build_prm_reward_fn(tcfg, device, trust_remote_code)
-    from soup_cli.trainer.rewards import load_reward_fn
+    # v0.71.40 #311 — ``reward_fn`` may be comma-separated ("accuracy,format").
+    # Return a single callable for one reward (back-compat) or a list for several
+    # (TRL's reward_funcs=[...] + the rm_ensemble detector both accept the list).
+    from soup_cli.trainer.rewards import load_reward_fns
 
-    return load_reward_fn(tcfg.reward_fn, verifiable_domain=tcfg.verifiable_domain)
+    fns = load_reward_fns(tcfg.reward_fn, verifiable_domain=tcfg.verifiable_domain)
+    return fns[0] if len(fns) == 1 else fns
 
 
 class GRPOTrainerWrapper: