agent-ultramode 0.1.4 → 0.2.0

package/README.md

@@ -6,6 +6,18 @@
 
 Took the [LLM-as-a-Verifier](https://github.com/llm-as-a-verifier/llm-as-a-verifier) paper (Kwok et al., 2026) and turned it into a simple `/ultra` command for coding agents.
 
+> ### New in v2: repair climbs past the best-of-N ceiling, putting a mid-range flash model in the frontier's coding tier
+>
+> Picking the best of N attempts can never beat **oracle@N**. If none of the attempts solved the task, selection cannot invent a fix. That ceiling is where every best-of-N method stops.
+>
+> v2 adds a **verifier-guided repair pass**: the verifier critiques the winner, a repair runs against that critique, and the result is kept **only if it verifies better** (your tests when they exist, the verifier otherwise). Because repair can synthesize a fix no attempt produced, it goes past the ceiling. On a 24-task SWE-bench Lite slice it reaches **91.7%, above the 87.5% oracle@5** of its own candidate pool.
+>
+> Plus **adaptive early-exit**: N is a budget, not a quota. The moment an attempt passes your tests, ultra takes it and abandons the rest.
+>
+> Stack that on the verify step and a small, non-vision flash model reaches **90.4%** on Terminal-Bench 2.1's coding subset, inside the band of GPT-5.6 Sol (89.5%), Claude Opus 5 (89.1%) and Grok 4.6 (88.4%), at a fraction of the per-token cost. Ours is best-of-5 against their pass@1, so read it as reaching the tier, not a like-for-like beat.
+>
+> [How it works](#repair-and-early-exit-v2)
+
 Instead of shipping the agent's first attempt, `ultra` runs your task **N times in isolated git worktrees**, in parallel, then uses the **same model** as a verifier to pick the best result. When it is confident (or when most attempts succeeded) it applies the winning diff to your working tree; when the attempts diverge and it is not sure, it hands you the top candidates. No cross-model dependency, no extra services, no first-attempt lottery.
 
 And unlike most "add a verifier" posts, I benchmarked it before believing it. The receipts are below, good and bad.
@@ -18,14 +30,52 @@ And unlike most "add a verifier" posts, I benchmarked it before believing it. Th
 
 ## The receipts
 
-Benchmarked on **Terminal-Bench** with **DeepSeek V4 Flash 0731** as both the agent and the verifier, running inside **opencode** (N=5 attempts, same-model verifier). Verified end to end with **OpenCode**, **Claude Code**, and **cline** as the agent (the numbers above are the OpenCode run; a Claude Code number will be added once benchmarked). Works with any agent, not just opencode.
+Benchmarked with **a small open model (DeepSeek V4 Flash 0731)** as both the agent and the verifier, best-of-5. Every number below is from a real run, good and bad. Full method, per-experiment setup and the honest scope notes are in the technical report: **[PAPER.md](./PAPER.md)**.
+
+![On Terminal-Bench 2.1, ultra ties the paper's verifier; on SWE-bench Lite, repair beats the oracle ceiling](viz/chart.png)
+
+![Cost versus score on Terminal-Bench 2.1: a small non-vision flash model reaches 90.4% on the coding subset, inside the frontier's band, at a fraction of the per-token cost. Inset: verifier-guided repair reaches 91.7% on SWE-bench Lite, above the 87.5% oracle@5 ceiling.](viz/chart2.png)
+
+### Terminal-Bench 2.1: reasoned verify ties the paper, with far fewer calls
+
+| Stage | Terminal-Bench 2.1 (89 tasks) |
+|---|:---:|
+| base@1 (single shot) | 78.7% |
+| **ultra (reasoned verify)** | **87.6%** (78/89) |
+| oracle@5 (selection ceiling) | 96.6% |
+
+The reasoned verifier **ties the LLM-as-a-Verifier paper's logprob PPT** (88.0% +/- 0.6%) while using **7.5x fewer verifier calls** (576 vs 4,320) and **no logprobs** at all. Same accuracy, a fraction of the verifier budget, and it runs on any OpenAI-compatible endpoint.
+
+### The non-vision handicap (and the fair arena)
+
+DeepSeek V4 Flash has **no vision**. Terminal-Bench 2.1 mixes image tasks in with the coding ones, so the blended score carries a penalty a text-only model can never pay down:
+
+| Terminal-Bench 2.1 split | tasks | ultra |
+|---|:---:|:---:|
+| Vision tasks | 12 | 58% |
+| **Coding tasks** | **77** | **90.4%** |
+| Blended (headline) | 89 | 87.6% |
+
+On the **coding subset**, the fair arena for a small, non-vision flash model, ultra reaches **90.4%**.
 
-| Slice | base@1 (single shot) | ultra (best-of-N + verify) | oracle@5 (ceiling) |
-|---|:---:|:---:|:---:|
-| All 15 tasks | 24% | **33%** | 40% |
-| 4 recoverable tasks | 40% | **75%** | rescued 3 of 4 |
+For context on the full 89-task set, as reported by Artificial Analysis (pass@1, avg of 3): GPT-5.6 Sol 89.5%, Claude Opus 5 89.1%, Grok 4.6 88.4%. Their coding-subset numbers are unpublished. So this is not a beat: ours is best-of-5 and theirs is pass@1. The honest framing is that a small, non-vision flash model reaches the **frontier's coding tier at a fraction of the cost**.
 
-Overall the verifier captured about **56% of the pass@1 to pass@5 headroom**. On the tasks with real variance, where best-of-N can actually help, it took the passing attempt on `chess-best-move`, `new-encrypt-command`, and `decommissioning-service` (missing only `jupyter-notebook-server`). A 40 to 75 jump on recoverable tasks is not a rounding error.
+### SWE-bench Lite: repair climbs past the selection ceiling
+
+Pure code repair, no vision, 24 django/pytest tasks, N=5:
+
+| Stage | SWE-bench Lite (24 tasks) |
+|---|:---:|
+| base@1 (single shot) | 70.8% |
+| ultra (reasoned verify) | 75.0% |
+| oracle@5 (selection ceiling) | 87.5% |
+| **ultra + repair** | **91.7%** (22/24) |
+
+This is the clean "past the ceiling" result. Selection alone can never beat the best attempt it was handed (oracle@5 = 87.5%). Best-of-N **repair** rescued 2 tasks none of the five attempts solved, landing **above** the oracle ceiling at 91.7%.
+
+### The trace-reading ceiling
+
+The verifier is strong on code diffs and weak on **self-reported terminal outcomes**. It saturates on convincing-but-wrong self-reports: on `extract-elf`, a failing run reported "4102 entries, zero mismatches" when 698 was the correct answer, and the verifier believed it. Repair plus a real test overcomes this; a verifier reading a trace alone cannot. This is why SWE-bench (pure code diffs) is the clean signal and Terminal-Bench is muddied by both vision and self-report.
 
 ## How the competition works
 
@@ -60,13 +110,38 @@ Overall the verifier captured about **56% of the pass@1 to pass@5 headroom**. On
 
 Progress streams live: phase titles in the web UI tool card, and toasts in the terminal.
 
+## Repair and early-exit (v2)
+
+Two additions in v2, both designed so they can only help.
+
+**Verifier-guided best-of-N repair (beyond the best-of-N ceiling).** Selection alone can never do better than the best attempt it was handed (its ceiling is oracle@N). After the tournament picks a winner, `ultra` critiques it and runs up to `--repair-n` guided repair passes (default 2), keeping the **first pass that passes your tests**; if none pass, it keeps the tournament winner unchanged. The keep-check uses your repo's own tests if there are any, otherwise the reasoned verifier. It is regression-safe and **never worse than the winner**: a repair is adopted only when it clears the bar, so repair can solve tasks no single attempt did while a bad repair is simply discarded. In a SWE-bench Lite run this reached 91.7%, above the 87.5% oracle@5 ceiling, by rescuing two tasks none of the five attempts solved.
+
+**Adaptive early-exit (N as an upper bound).** N is a budget, not a quota. The moment an attempt's diff **passes your tests**, `ultra` takes it as the verified winner, abandons the still-running attempts, and skips the tournament and repair. Only this hard signal stops early; low verifier confidence never does, so you never trade quality for speed. With no detectable test command it runs all N, exactly as before. Turn repair off with `--no-repair` (or `ULTRA_REPAIR=0`); turn early-exit off with `--no-early-exit` (or `ULTRA_NO_EARLY_EXIT=1`).
+
 ## Install
 
+Pick your agent. Each is one command, and each gives you the same native `/ultra <task>`.
+
+**Claude Code**
+
+```sh
+mkdir -p ~/.claude/commands
+curl -fsSL https://raw.githubusercontent.com/maverick-tr/agent-ultramode/main/install/ultra.md -o ~/.claude/commands/ultra.md
+```
+
+**Grok**
+
+```sh
+grok plugin install maverick-tr/agent-ultramode --trust
+```
+
+**opencode**
+
 ```sh
 opencode plugin agent-ultramode
 ```
 
-That is the whole setup. With no options it drafts and verifies with your **current session model**, so `/ultra <task>` just works:
+Then run it in any session:
 
 ```text
 /ultra fix the failing test in foo/bar
@@ -74,7 +149,17 @@ That is the whole setup. With no options it drafts and verifies with your **curr
 
 `ultra` branches attempts off `HEAD`, so run it inside a git repo with at least one commit.
 
-**Pin a specific model (optional).** To draft and verify with a model other than the session one:
+On **Claude Code** and **Grok**, attempts run as parallel subagents in the host itself (Grok shows them live in the Tasks pane, Ctrl+G). On **opencode** they run as isolated git worktrees and the winning diff is applied to your working tree. Same loop either way: fan out N, verify, repair the winner, keep it only if it is better.
+
+### More install options
+
+**Claude Code, as a full plugin** (instead of the single file above):
+
+```sh
+/plugin marketplace add maverick-tr/agent-ultramode
+```
+
+**opencode, pinning a specific model (optional).** To draft and verify with a model other than the session one:
 
 ```jsonc
 { "plugin": [ ["agent-ultramode", { "model": "myprovider/my-model" }] ] }
@@ -105,6 +190,10 @@ Every option also reads from an `ULTRA_*` env var.
 | `effort` | `"none"` | `reasoning_effort` for the verifier calls (kept low so judging stays fast) |
 | `concurrency` | `6` | how many attempts run at once |
 | `agentTimeout` | `600000` | per-attempt timeout in ms |
+| `repair` | `true` | run verifier-guided repair on the winner, kept only if it verifies better |
+| `repairN` | `2` | repair passes to run (1 to 5); keeps the first that passes your tests, else the winner |
+| `test` | auto-detected | test command for the repair keep-check and early-exit (npm / pytest / cargo / go if unset) |
+| `earlyExit` | `true` | stop as soon as an attempt passes the test command, and apply it |
 
 ## Use without opencode (the CLI)
 
@@ -140,9 +229,17 @@ The 6 attempts round-robin across the three agents, and the verifier picks the b
 
 The verifier needs an OpenAI-compatible endpoint: by default it reads `OPENAI_API_KEY` and `OPENAI_BASE_URL`, or pass `--api-key` / `--base-url` / `--verify-model`. Run `agent-ultramode --help` for all options.
 
+## How the hosts differ
+
+One plugin serves all three, each using its host's own subagent primitive: Grok's `spawn_subagent` with worktree isolation, Claude Code's `Agent` tool, and opencode's own worktree runner. The best-of-N, verify and repair loop is identical; only the fan-out mechanism and the progress UI change. Both the Claude Code and Grok paths are verified end to end.
+
+## DeepSeek Harness (experimental)
+
+A native DeepSeek Harness `/ultra` plugin exists but is not recommended yet: the harness surfaces each attempt as a separate child session, which is noisy in the web UI. Use the Grok or Claude Code plugins above for a clean native `/ultra` today.
+
 ## Why I built this, and the honest story
 
-I wanted to know if best-of-N verification could squeeze real quality out of a cheap-but-capable model without reaching for a bigger, pricier one. So, ran it on real Terminal-Bench tasks and let the numbers decide. What I found:
+I wanted to know if best-of-N verification could squeeze real quality out of a small-but-capable model without reaching for a bigger, pricier one. So, ran it on real Terminal-Bench tasks and let the numbers decide. What I found:
 
 - **Planning-first best-of-N does nothing for execution tasks.** Drafting N plans, picking the best, then executing once: **0 out of 5** outcomes changed on Terminal-Bench, at 3 to 10 times the cost. It amplifies effort, not capability. I dropped it.
 - **Best-of-N over full trajectories plus a same-model verifier does work.** This is the version that ships.
@@ -153,19 +250,22 @@ It significantly helps on the tasks that matter, and I can point at the per-task
 
 ## The honest caveats (because benchmarks lie by omission)
 
-1. **Confidence is a noisy signal at this scale.** The one miss (`jupyter`) had confidence 0.25, higher than two tasks it got right (0.17, 0.21). Some beyond-capability tasks even show high confidence while being pure failures. "Trust when confident" is directionally real, not a clean gate.
-2. **It cannot help beyond-capability tasks.** 9 of the 15 never passed in 5 tries, and no verifier manufactures a solution that was never there.
-3. **n=15 (4 with real variance) is a credible but modest sample.** Enough to headline honestly, not enough to over-claim precision.
+1. **The model has no vision, and image tasks are out of fair reach.** DeepSeek V4 Flash is text-only, so the 12 Terminal-Bench 2.1 vision tasks (58%) drag the blended score down in a way no amount of verifying can fix. The **coding subset (90.4%)** is the fair arena for a non-vision model; read the blended 87.6% with that in mind.
+2. **The frontier comparison is not apples to apples.** Our headline numbers are **best-of-5**; the GPT-5.6 / Opus 5 / Grok 4.6 numbers are **pass@1**. So "reaches the frontier's coding tier" is a cost-and-capability framing, not a claim that this small model beats them head to head.
+3. **The verifier hits a trace-reading ceiling.** It reasons well over code diffs but is fooled by convincing-but-wrong self-reports (the `extract-elf` run that claimed "4102 entries, zero mismatches" when 698 was correct). Selection over self-reported terminal outcomes saturates; only repair plus a real test breaks past it.
+4. **SWE-bench is the clean signal; Terminal-Bench is muddied.** SWE-bench Lite is pure code repair with real tests, which is why repair cleanly beats the oracle@5 ceiling there (91.7% > 87.5%). Terminal-Bench mixes in vision and self-reported outcomes, so its blended number carries penalties that have nothing to do with the verifier.
 
 ## What this likely means at scale
 
-The 15 tasks were deliberately failure-skewed, so the +9 points is not what you would see on the whole benchmark. On the full set the model already passes about 83%, so most tasks have no headroom. My honest estimate for a full run is a **modest +2 to +5 points on average**, because the dramatic gains only land on the minority of tasks the model *sometimes* solves. Average lift small, per-recoverable-task lift large. Same fact, two views.
+On any full benchmark most tasks have no headroom: the model either always solves them or never does, and a verifier changes neither. The lift concentrates on the minority of tasks the model *sometimes* solves, where selection can grab the passing attempt and repair can climb one rung higher. So on Terminal-Bench 2.1 the base-to-verify jump is a real **+8.9 points blended** (78.7% to 87.6%), and on the coding subset it lands at **90.4%**, but the honest reading is that the dramatic per-task gains average out into a moderate headline lift, larger on repair-friendly pure-code sets like SWE-bench than on mixed sets diluted by vision. Average lift moderate, per-recoverable-task lift large. Same fact, two views.
 
 ## Roadmap
 
 - [x] **Standalone CLI** (`npx agent-ultramode`) so the loop runs anywhere, with any agent, no opencode required.
 - [x] **Multiple models in one pass** (repeatable `--agent`): spread attempts across different models, one neutral verifier picks the best.
 - [x] **OpenCode, Claude Code, and cline** verified end to end.
+- [x] **Verifier-guided best-of-N repair** (v2): up to `--repair-n` critique-guided passes on the winner, keeping the first that passes your tests. Beats the oracle@N ceiling on SWE-bench Lite (91.7% vs 87.5%).
+- [x] **Adaptive early-exit** (v2): N becomes an upper bound; a test-passing attempt ends the run early with no quality trade-off.
 - [ ] **First-class agent integrations** (tuned defaults and a benchmark number) for Grok, Pi, and Codex.
 - [ ] Native slash-command or MCP packaging per agent. Contributions welcome.
 

package/cli.mjs

@@ -9,7 +9,7 @@
 // Node 18+ (uses global fetch and node:util parseArgs). Zero dependencies.
 import { execFile } from "node:child_process"
 import { promisify } from "node:util"
-import { mkdtemp, rm, writeFile, mkdir } from "node:fs/promises"
+import { mkdtemp, rm, writeFile, mkdir, readFile } from "node:fs/promises"
 import { tmpdir } from "node:os"
 import { join } from "node:path"
 import { parseArgs } from "node:util"
@@ -39,8 +39,32 @@ OPTIONS
       --repo <path>        repo to run in (default: current directory)
       --concurrency <int>  attempts to run at once (default 6)
       --effort <level>     verifier reasoning_effort (default none)
+      --test <cmd>         test command for the repair keep-check (else auto-detected:
+                           npm test / pytest / cargo / go). Passing exit code = ok.
+      --repair-agent <cmd> agent for the repair pass ("{task}" substituted; default: --agent)
+      --repair-n <int>     repair attempts to run, 1 to 5 (default 2; env ULTRA_REPAIR_N).
+                           Keeps the best that verifies; never worse than the winner.
+      --no-repair          disable the verifier-guided repair pass (on by default)
+      --no-early-exit      disable adaptive early-exit (on by default when a test
+                           command is available)
   -h, --help               show this help
 
+REPAIR (on by default; ULTRA_REPAIR=0 or --no-repair to disable)
+  After the tournament picks a winner, ultra critiques it and runs --repair-n
+  guided repair passes (default 2), each in its own worktree seeded with the
+  winner's diff and the same critique, then keeps the best that verifies:
+  with a test command, the first repair that PASSES the tests; otherwise the
+  first repair the reasoned verifier clearly prefers over the winner. If none
+  qualify, the winner is kept. So repair can lift the result past the best-of-N
+  ceiling, and the applied result is never worse than the tournament winner.
+
+ADAPTIVE EARLY-EXIT (on by default; --no-early-exit or ULTRA_NO_EARLY_EXIT=1 to disable)
+  N is an upper bound. If an attempt PASSES the repo's tests (--test or an
+  auto-detected npm/pytest/cargo/go command), ultra takes it as a verified
+  winner, abandons the still-running attempts, and skips the tournament and
+  repair. Only this hard signal stops early; low verifier confidence never does,
+  so the result can never be worse than running all N.
+
 EXAMPLES
   # default: opencode as the per-attempt agent, verify with your OpenAI key
   agent-ultramode "fix the failing test in foo/bar"
@@ -131,6 +155,71 @@ async function tournament(C, task, summaries) {
   return { ranked, conf: ratio(ranked[0]) - ratio(ranked[1]) }
 }
 
+// --- verifier-guided repair (generic: never benchmark-specific) ------------
+// After the tournament picks a winner, critique it and run one guided repair
+// pass, then KEEP the repair only when it verifies as better (repo's own tests
+// if available, else the reasoned verifier). So repair can only help, not hurt.
+
+async function critique(C, task, diff, log) {
+  const prompt =
+    `A coding task and the patch a verifier selected as the best of several attempts. You are a strict ` +
+    `reviewer. In 3-5 sentences name the MOST LIKELY remaining problems: missed edge cases, incomplete ` +
+    `coverage, wrong root cause, or regressions it could introduce. If it looks fully correct, say so and ` +
+    `name the single thing most worth double-checking. Be concrete and actionable.\n\n` +
+    `TASK:\n${task}\n\nPATCH:\n${diff.slice(0, 6000)}\n\nAGENT LOG (tail):\n${(log || "").slice(-1200)}\n\nReview:`
+  try { return (await chat(C, prompt)).trim().slice(0, 1500) } catch { return "" }
+}
+
+const XDG_ENV = (d) => ({
+  ...process.env, XDG_DATA_HOME: d, XDG_STATE_HOME: d, XDG_CACHE_HOME: d,
+  OPENCODE_DISABLE_DEFAULT_PLUGINS: "1", OPENCODE_DISABLE_AUTOUPDATE: "1", OPENCODE_DISABLE_MODELS_FETCH: "1",
+})
+
+// run an agent command inside a fresh worktree (optionally pre-seeded with a
+// base diff), return the full diff off `base` + the log tail.
+async function agentInWorktree(repo, base, work, tag, agentCmd, taskText, seedDiff, timeout) {
+  const wt = join(work, tag)
+  await git(repo, "worktree", "add", "--detach", wt, base)
+  if (seedDiff && seedDiff.trim()) {
+    const p = join(work, `${tag}.seed.patch`); await writeFile(p, seedDiff)
+    await git(wt, "apply", "--3way", p).catch(() => {})
+  }
+  const cmd = agentCmd.replace("{task}", taskText.replace(/"/g, '\\"'))
+  const d = join(work, `${tag}-xdg`); await mkdir(d, { recursive: true }).catch(() => {})
+  let log = ""
+  try {
+    const { stdout, stderr } = await execFileP("bash", ["-lc", `exec </dev/null; ${cmd}`], { cwd: wt, env: XDG_ENV(d), timeout, maxBuffer: 32 * 1024 * 1024 })
+    log = (stdout || "") + (stderr || "")
+  } catch (e) { log = `agent error: ${e?.message || e}` }
+  await git(wt, "add", "-A").catch(() => {})
+  const diff = await git(wt, "diff", "--cached").catch(() => "")
+  await git(repo, "worktree", "remove", "--force", wt).catch(() => {})
+  return { diff, log }
+}
+
+// true iff `testCmd` exits 0 in a fresh worktree with `diff` applied.
+async function runTests(repo, base, work, tag, diff, testCmd, timeout) {
+  const wt = join(work, `t-${tag}`)
+  await git(repo, "worktree", "add", "--detach", wt, base)
+  try {
+    if (diff.trim()) { const p = join(work, `t-${tag}.patch`); await writeFile(p, diff); await git(wt, "apply", "--3way", p).catch(() => {}) }
+    await execFileP("bash", ["-lc", `exec </dev/null; ${testCmd}`], { cwd: wt, env: process.env, timeout, maxBuffer: 32 * 1024 * 1024 })
+    return true
+  } catch { return false }
+  finally { await git(repo, "worktree", "remove", "--force", wt).catch(() => {}) }
+}
+
+// auto-detect a repo test command if the user didn't pass --test.
+async function detectTestCmd(repo) {
+  const has = async (f) => !!(await readFile(join(repo, f), "utf8").catch(() => ""))
+  const pkg = await readFile(join(repo, "package.json"), "utf8").catch(() => "")
+  if (pkg && /"test"\s*:/.test(pkg) && !/no test specified/.test(pkg)) return "npm test --silent"
+  if (await has("pytest.ini") || await has("pyproject.toml") || await has("setup.cfg") || await has("tox.ini")) return "python -m pytest -q"
+  if (await has("Cargo.toml")) return "cargo test -q"
+  if (await has("go.mod")) return "go test ./..."
+  return ""
+}
+
 async function main() {
   const { values, positionals } = parseArgs({
     allowPositionals: true,
@@ -146,6 +235,11 @@ async function main() {
       repo: { type: "string" },
       concurrency: { type: "string" },
       effort: { type: "string" },
+      "no-repair": { type: "boolean" },
+      "no-early-exit": { type: "boolean" },
+      "repair-agent": { type: "string" },
+      "repair-n": { type: "string" },
+      test: { type: "string" },
       help: { type: "boolean", short: "h" },
     },
   })
@@ -160,6 +254,11 @@ async function main() {
   const conf = Number(values.conf ?? 0.34)
   const cc = int(values.concurrency, 6, 1, 12)
   const agentTimeout = 600000
+  const doRepair = !values["no-repair"] && process.env.ULTRA_REPAIR !== "0"
+  const repairN = int(values["repair-n"] ?? process.env.ULTRA_REPAIR_N, 2, 1, 5)
+  const earlyExit = !values["no-early-exit"] && process.env.ULTRA_NO_EARLY_EXIT !== "1"
+  const repairAgent = values["repair-agent"] || (agents[0] || 'opencode run "{task}"')
+  let testCmd = values.test ?? process.env.ULTRA_TEST ?? ""
   const C = {
     verifyModel: values["verify-model"] || process.env.ULTRA_VERIFY_MODEL || process.env.OPENAI_MODEL || "gpt-4o-mini",
     baseURL: values["base-url"] || process.env.OPENAI_BASE_URL || process.env.ULTRA_BASE_URL || "https://api.openai.com/v1",
@@ -204,26 +303,63 @@ async function main() {
     }
     const limit = pLimit(Math.max(1, Math.min(cc, n)))
     const esc = task.replace(/"/g, '\\"')
-    const outs = await Promise.all(worktrees.map((wt, i) => limit(async () => {
+
+    // Adaptive early-exit (#141): N is an upper bound. If a completed attempt
+    // PASSES the repo's tests (a hard, non-regressible signal), take it as the
+    // verified winner, abandon the still-running/queued attempts, and skip the
+    // tournament + repair. Low verifier confidence never stops early, so the
+    // result can never be worse than running all N.
+    if (earlyExit && testCmd === "") testCmd = await detectTestCmd(repo)
+    const exitTestCmd = earlyExit ? testCmd : ""
+    if (exitTestCmd) err(`>>> early-exit armed: stop as soon as an attempt passes '${exitTestCmd}'`)
+    const ac = new AbortController()
+    let verified = null
+    const outs = new Array(n)
+
+    const runAgent = (wt, i) => new Promise((resolve) => {
       const cmd = agents[i % agents.length].replace("{task}", esc)
       const d = join(work, `xdg-${i}`)
-      await mkdir(d, { recursive: true }).catch(() => {})
-      const env = {
-        ...process.env,
-        XDG_DATA_HOME: d, XDG_STATE_HOME: d, XDG_CACHE_HOME: d,
-        OPENCODE_DISABLE_DEFAULT_PLUGINS: "1", OPENCODE_DISABLE_AUTOUPDATE: "1", OPENCODE_DISABLE_MODELS_FETCH: "1",
-      }
-      let log = ""
-      try {
-        const { stdout, stderr } = await execFileP("bash", ["-lc", `exec </dev/null; ${cmd}`], { cwd: wt, env, timeout: agentTimeout, maxBuffer: 32 * 1024 * 1024 })
-        log = (stdout || "") + (stderr || "")
-      } catch (e) { log = `agent error: ${e?.message || e}` }
+      mkdir(d, { recursive: true }).catch(() => {}).finally(() => {
+        const env = {
+          ...process.env,
+          XDG_DATA_HOME: d, XDG_STATE_HOME: d, XDG_CACHE_HOME: d,
+          OPENCODE_DISABLE_DEFAULT_PLUGINS: "1", OPENCODE_DISABLE_AUTOUPDATE: "1", OPENCODE_DISABLE_MODELS_FETCH: "1",
+        }
+        const child = execFile("bash", ["-lc", `exec </dev/null; ${cmd}`],
+          { cwd: wt, env, timeout: agentTimeout, maxBuffer: 32 * 1024 * 1024, detached: true },
+          (e, stdout, stderr) => resolve(((stdout || "") + (stderr || "")) || (e ? `agent error: ${e?.message || e}` : "")))
+        // if a verified winner is found elsewhere, kill this attempt's process group
+        ac.signal.addEventListener("abort", () => { try { process.kill(-child.pid, "SIGTERM") } catch {} }, { once: true })
+      })
+    })
+
+    await Promise.all(worktrees.map((wt, i) => limit(async () => {
+      if (ac.signal.aborted) return            // verified winner already found; don't start queued attempts
+      const log = await runAgent(wt, i)
       await git(wt, "add", "-A").catch(() => {})
       const diff = await git(wt, "diff", "--cached").catch(() => "")
+      outs[i] = { diff, log, summary: `AGENT LOG (tail):\n${log.slice(-1500)}\n\nDIFF:\n${diff.slice(0, 6000) || "(no changes)"}` }
       err(`    attempt ${i} [${binOf(agents[i % agents.length])}]: ${diff ? diff.length + " diff chars" : "no changes"}`)
-      return { diff, summary: `AGENT LOG (tail):\n${log.slice(-1500)}\n\nDIFF:\n${diff.slice(0, 6000) || "(no changes)"}` }
+      if (exitTestCmd && !verified && diff.trim()) {
+        const pass = await runTests(repo, base, work, `ee-${i}`, diff, exitTestCmd, agentTimeout)
+        if (pass && !verified) { verified = { i, ...outs[i] }; err(`    ✔ attempt ${i} PASSED tests; abandoning the rest`); ac.abort() }
+      }
     })))
 
+    // verified fast path: a test-passing attempt is correct by the repo's own
+    // criterion, so apply it directly (no tournament, no repair, no regression).
+    if (verified && verified.diff.trim()) {
+      const patch = join(work, "winner.patch")
+      await writeFile(patch, verified.diff)
+      try {
+        await git(repo, "apply", "--3way", patch)
+        console.log(`\n🏆 attempt ${verified.i} passed the repo's tests (${exitTestCmd}); applied it and abandoned the rest. Review, then commit.`)
+      } catch (e) {
+        console.log(`\nattempt ${verified.i} passed tests but the patch did not apply cleanly (${e?.message || e}). The diff:\n\n${verified.diff.slice(0, 8000)}`)
+      }
+      return
+    }
+
     const diffs = outs.map((o) => o.diff)
     const summaries = outs.map((o) => o.summary)
     if (diffs.every((d) => !d.trim())) {
@@ -234,6 +370,59 @@ async function main() {
     err(`>>> verify: probabilistic pivot tournament (${C.verifyModel}) ...`)
     const { ranked, conf: margin } = await tournament(C, task, summaries)
     const best = ranked[0]
+
+    // --- repair: critique the winner, run repairN guided passes, keep the best that verifies.
+    // Best-of-N repair (regression-safe): fan out repairN attempts off `base`, each seeded with
+    // the winner's diff and the SAME critique injected (one critique, reused). Keep-policy:
+    //   * with tests: keep the FIRST repair that PASSES the repo's tests; else keep the winner.
+    //     (a passing repair is correct by the repo's own criterion, so it cannot regress.)
+    //   * without tests: keep the FIRST repair the reasoned verifier CLEARLY prefers over the
+    //     current winner (same >k/2 threshold as before); else keep the winner.
+    // INVARIANT: the final applied result is never worse than the tournament winner.
+    if (doRepair && diffs[best].trim()) {
+      err(`>>> repair: critique + ${repairN} guided pass(es) on the winner (keep the best that verifies) ...`)
+      const crit = await critique(C, task, diffs[best], outs[best].log)
+      const rtask =
+        `${task}\n\n--- A previous attempt (already applied to your working tree) produced a partial ` +
+        `solution. A reviewer flagged the issues below. Improve and COMPLETE it, keeping what is correct; ` +
+        `verify before finishing. ---\nREVIEWER NOTES:\n${crit}`
+      if (testCmd === "") testCmd = await detectTestCmd(repo)
+      // run the repairN attempts in parallel, honoring the concurrency limiter (cc).
+      const rlimit = pLimit(C.cc)
+      const reps = await Promise.all(Array.from({ length: repairN }, (_, r) => rlimit(async () => {
+        const rr = await agentInWorktree(repo, base, work, `repair-${r}`, repairAgent, rtask, diffs[best], agentTimeout)
+        const rsum = `AGENT LOG (tail):\n${rr.log.slice(-1500)}\n\nDIFF:\n${rr.diff.slice(0, 6000) || "(no changes)"}`
+        err(`    repair ${r} [${binOf(repairAgent)}]: ${rr.diff ? rr.diff.length + " diff chars" : "no changes"}`)
+        return { diff: rr.diff, summary: rsum }
+      })))
+      // only real, changed candidates are eligible.
+      const cands = reps.filter((x) => x.diff.trim() && x.diff !== diffs[best])
+      let kept = false, why = ""
+      if (!cands.length) { err(`    repair produced no new change; winner kept`) }
+      else if (testCmd) {
+        // regression-safe: keep the FIRST repair that PASSES the repo's own tests.
+        err(`    keep-check: running tests (${testCmd}) on ${cands.length} repair candidate(s) ...`)
+        for (let r = 0; r < cands.length; r++) {
+          const pass = await runTests(repo, base, work, `rep-${r}`, cands[r].diff, testCmd, agentTimeout)
+          if (pass) { diffs[best] = cands[r].diff; summaries[best] = cands[r].summary; kept = true; why = `repair ${r} passes the tests`; break }
+        }
+        if (!kept) why = "no repair passed the tests"
+      } else {
+        // no tests: keep the FIRST repair the verifier CLEARLY prefers over the current winner.
+        const preferRepair = async (rsum) => {   // reasoned verifier: keep repair only if it clearly wins
+          const limit = pLimit(C.cc)
+          const votes = await Promise.all(Array.from({ length: C.k }, () => judge(C, limit, task, summaries[best], rsum)))
+          return votes.filter((v) => v === "B").length > C.k / 2   // B = repaired
+        }
+        for (let r = 0; r < cands.length; r++) {
+          if (await preferRepair(cands[r].summary)) { diffs[best] = cands[r].diff; summaries[best] = cands[r].summary; kept = true; why = `verifier prefers repair ${r}`; break }
+        }
+        if (!kept) why = "verifier keeps winner"
+      }
+      if (kept) err(`    ✔ repair KEPT (${why})`)
+      else if (cands.length) err(`    all repairs discarded (${why})`)
+    }
+
     const nonEmpty = diffs.filter((d) => d.trim()).length
     const majority = nonEmpty >= Math.max(2, Math.ceil(n / 2))
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "agent-ultramode",
-  "version": "0.1.4",
-  "description": "Best-of-N for coding agents with a same-model verifier: run your task N times in isolated git worktrees and apply the verified winner. A /ultra command for opencode.",
+  "version": "0.2.0",
+  "description": "Best-of-N for coding agents with a same-model verifier, verifier-guided repair, and adaptive early-exit: run your task N times in isolated git worktrees, verify, repair the winner past the best-of-N ceiling, and apply it. A /ultra command for opencode.",
   "module": "ultra.ts",
   "main": "ultra.ts",
   "exports": {
@@ -28,6 +28,8 @@
     "verifier",
     "llm-as-a-verifier",
     "coding-agent",
+    "repair",
+    "self-repair",
     "ultra"
   ],
   "license": "MIT",
@@ -37,10 +39,6 @@
     "url": "git+https://github.com/maverick-tr/agent-ultramode.git"
   },
   "devDependencies": {
-    "@opencode-ai/plugin": "^1.0.153",
-    "@types/bun": "latest"
-  },
-  "peerDependencies": {
-    "typescript": "^5"
+    "@opencode-ai/plugin": "^1.0.153"
   }
 }

package/ultra.ts

@@ -8,9 +8,11 @@
  *   low  confidence -> shows you the top candidates and applies nothing
  * (a low-confidence pick is a coin flip and should not be applied silently).
  *
- * This is the version validated on Terminal-Bench: plan-first best-of-N gave no edge,
- * but best-of-N over full trajectories + this verifier lifted the recoverable tasks
- * from 40% to 75% (n=15, same model, no cross-model dependency).
+ * Validated on Terminal-Bench 2.1 (89 tasks, same model as the solver, no cross-model
+ * dependency): base@1 78.7% -> 87.6% with best-of-5 and this verifier, against an
+ * oracle@5 ceiling of 96.6%. Plan-first best-of-N gave no edge; best-of-N over full
+ * trajectories is what moved the number. On 24 SWE-bench-style django/pytest tasks,
+ * verifier-guided repair reached 91.7%, above the 87.5% oracle@5 of the attempts.
  *
  * Install (opencode.json):
  *   { "plugin": [ ["file:///abs/path/ultra.ts", { "model": "provider/model", "n": 5 }] ] }
@@ -22,14 +24,14 @@ import type { Plugin } from "@opencode-ai/plugin"
 import { tool } from "@opencode-ai/plugin"
 import { execFile } from "node:child_process"
 import { promisify } from "node:util"
-import { mkdtemp, rm, writeFile, mkdir } from "node:fs/promises"
+import { mkdtemp, rm, writeFile, mkdir, readFile } from "node:fs/promises"
 import { existsSync } from "node:fs"
 import { tmpdir, homedir } from "node:os"
 import { join } from "node:path"
 
 const execFileP = promisify(execFile)
 type Opts = Record<string, any>
-const VERSION = "0821g"
+const VERSION = "v2-0824r"
 
 interface Cfg {
   url: string
@@ -45,6 +47,10 @@ interface Cfg {
   judgeTokens: number
   cc: number
   agentTimeoutMs: number
+  repair: boolean
+  repairN: number
+  repairAgent: string
+  testCmd: string
 }
 
 const num = (v: any, def: number, lo: number, hi: number): number => {
@@ -75,6 +81,12 @@ function buildCfg(opts: Opts): Cfg {
     judgeTokens: num(opts.judgeTokens ?? process.env.ULTRA_JUDGE_TOKENS, 12000, 500, 200000),
     cc: num(opts.concurrency ?? process.env.ULTRA_CC, 6, 1, 12),
     agentTimeoutMs: num(opts.agentTimeout ?? process.env.ULTRA_AGENT_TIMEOUT, 600000, 30000, 1800000),
+    // verifier-guided repair: on by default; a repaired winner is kept only when it verifies better.
+    repair: String(opts.repair ?? process.env.ULTRA_REPAIR ?? "1") !== "0" && opts.repair !== false,
+    // best-of-N repair: run this many repair attempts, keep the best that verifies (never worse than the winner).
+    repairN: num(opts.repairN ?? process.env.ULTRA_REPAIR_N, 2, 1, 5),
+    repairAgent: String(opts.repairAgent ?? process.env.ULTRA_REPAIR_AGENT ?? opts.agent ?? process.env.ULTRA_AGENT ?? 'opencode run "{task}"'),
+    testCmd: String(opts.test ?? process.env.ULTRA_TEST ?? ""),
   }
 }
 
@@ -163,6 +175,44 @@ async function tournament(C: Cfg, task: string, summaries: string[]): Promise<{
   return { ranked, conf: ratio(ranked[0]) - ratio(ranked[1]) }
 }
 
+// ---- verifier-guided repair (generic; never benchmark-specific) -------------------------
+// After the tournament, critique the winner and run ONE guided repair pass, then keep the
+// repair ONLY when it verifies as better (the repo's own tests if available, else the
+// reasoned verifier). Repair can lift past the best-of-N ceiling, and never makes it worse.
+
+async function critique(C: Cfg, task: string, diff: string, log: string): Promise<string> {
+  const prompt =
+    `A coding task and the patch a verifier selected as the best of several attempts. You are a strict ` +
+    `reviewer. In 3-5 sentences name the MOST LIKELY remaining problems: missed edge cases, incomplete ` +
+    `coverage, wrong root cause, or regressions it could introduce. If it looks fully correct, say so and ` +
+    `name the single thing most worth double-checking. Be concrete and actionable.\n\n` +
+    `TASK:\n${task}\n\nPATCH:\n${diff.slice(0, 6000)}\n\nAGENT LOG (tail):\n${(log || "").slice(-1200)}\n\nReview:`
+  try { return (await chat(C, prompt)).trim().slice(0, 1500) } catch { return "" }
+}
+
+// auto-detect a repo test command when the user did not pass one.
+async function detectTestCmd(dir: string): Promise<string> {
+  const read = (f: string) => readFile(join(dir, f), "utf8").catch(() => "")
+  const pkg = await read("package.json")
+  if (pkg && /"test"\s*:/.test(pkg) && !/no test specified/.test(pkg)) return "npm test --silent"
+  if (await read("pytest.ini") || await read("pyproject.toml") || await read("setup.cfg") || await read("tox.ini")) return "python -m pytest -q"
+  if (await read("Cargo.toml")) return "cargo test -q"
+  if (await read("go.mod")) return "go test ./..."
+  return ""
+}
+
+// true iff `testCmd` exits 0 in a fresh worktree with `diff` applied.
+async function runTests(dir: string, base: string, work: string, tag: string, diff: string, testCmd: string, timeout: number, abort?: AbortSignal): Promise<boolean> {
+  const wt = join(work, `t-${tag}`)
+  await git(dir, "worktree", "add", "--detach", wt, base)
+  try {
+    if (diff.trim()) { const p = join(work, `t-${tag}.patch`); await writeFile(p, diff); await git(wt, "apply", "--3way", p).catch(() => {}) }
+    await execFileP("bash", ["-lc", `exec </dev/null; ${testCmd}`], { cwd: wt, env: process.env, timeout, maxBuffer: 32 * 1024 * 1024, signal: abort })
+    return true
+  } catch { return false }
+  finally { await git(dir, "worktree", "remove", "--force", wt).catch(() => {}) }
+}
+
 // ---- lean sandbox for sub-agent attempts ------------------------------------------------
 // Sub-agents run under XDG_CONFIG_HOME -> a provider-only config so they skip the user's MCP
 // servers and plugins (the slow part). Deps install once into the sandbox and cache forever.
@@ -204,9 +254,10 @@ export const Ultra: Plugin = async (_input, options) => {
 
   const ultra = tool({
     description:
-      "Best-of-N with a verifier. Runs a coding/terminal task N times in isolated git worktrees, ranks the " +
-      "attempts with the same model as a verifier, and applies the winning diff when confident (otherwise " +
-      "reports the top candidates). Use for a task worth spending compute to get right the first time.",
+      "v2: Best-of-N with a verifier + repair. Runs a coding/terminal task N times in isolated git worktrees, " +
+      "ranks the attempts with the same model as a verifier, then runs one critique-guided REPAIR pass on the " +
+      "winner and keeps it only if it verifies better (the repo's tests, else the verifier). Applies the " +
+      "result when confident (otherwise reports the top candidates). For a task worth getting right the first time.",
     args: {
       task: tool.schema.string().describe("The task to solve N times and verify. Be specific."),
     },
@@ -277,25 +328,59 @@ export const Ultra: Plugin = async (_input, options) => {
           worktrees.push(wt)
         }
 
-        // Run the agent in each worktree IN PARALLEL (lean sandbox keeps this cheap).
+        // Adaptive early-exit (#141): N is an upper bound. If an attempt PASSES the
+        // repo's tests (a hard, non-regressible signal), take it as the verified winner,
+        // abandon the still-running/queued attempts, and skip the tournament + repair.
+        // Low verifier confidence never stops early, so the result can never be worse
+        // than running all N. Off with earlyExit:false or ULTRA_NO_EARLY_EXIT=1.
+        const earlyExit = (eff as any).earlyExit !== false && process.env.ULTRA_NO_EARLY_EXIT !== "1"
+        let testCmd = eff.testCmd || (await detectTestCmd(dir))
+        const exitTestCmd = earlyExit ? testCmd : ""
+        // one combined abort: fires on host cancel (ctx.abort) OR our verified-winner signal.
+        const ee = new AbortController()
+        try { (ctx.abort as AbortSignal | undefined)?.addEventListener?.("abort", () => ee.abort(), { once: true }) } catch {}
+        let verified: { i: number; diff: string; log: string; summary: string } | null = null
+
+        // Run the agent in each worktree IN PARALLEL (lean sandbox keeps this light).
         const limit = pLimit(Math.max(1, Math.min(eff.cc, eff.n)))
         let done = 0
         status(`ultra: running ${eff.n} attempts in parallel...`)
-        toast(`running ${eff.n} attempts in parallel`)
+        toast(`running ${eff.n} attempts in parallel${exitTestCmd ? `, early-exit on '${exitTestCmd}'` : ""}`)
         const outs = await Promise.all(worktrees.map((wt, i) => limit(async () => {
+          if (ee.signal.aborted) return null   // verified winner already found; skip queued attempts
           const env = await isolatedEnv(String(i))
           let log = ""
           try {
-            const { stdout, stderr } = await execFileP("bash", ["-lc", `exec </dev/null; ${cmd}`], { cwd: wt, env, timeout: eff.agentTimeoutMs, maxBuffer: 32 * 1024 * 1024, signal: ctx.abort })
+            const { stdout, stderr } = await execFileP("bash", ["-lc", `exec </dev/null; ${cmd}`], { cwd: wt, env, timeout: eff.agentTimeoutMs, maxBuffer: 32 * 1024 * 1024, signal: ee.signal })
             log = (stdout || "") + (stderr || "")
           } catch (e: any) { log = `agent error: ${e?.message || e}` }
           await git(wt, "add", "-A").catch(() => {})
           const diff = await git(wt, "diff", "--cached").catch(() => "")
           done++
           status(`ultra: ${done}/${eff.n} attempts done`, `attempt ${i}: ${diff ? diff.length + " diff chars" : "no changes"}`)
-          return { diff, summary: `AGENT LOG (tail):\n${log.slice(-1500)}\n\nDIFF:\n${diff.slice(0, 6000) || "(no changes)"}` }
+          const out = { diff, log, summary: `AGENT LOG (tail):\n${log.slice(-1500)}\n\nDIFF:\n${diff.slice(0, 6000) || "(no changes)"}` }
+          if (exitTestCmd && !verified && diff.trim()) {
+            const pass = await runTests(dir, base, work, `ee-${i}`, diff, exitTestCmd, eff.agentTimeoutMs, ee.signal)
+            if (pass && !verified) { verified = { i, ...out }; toast(`attempt ${i} passed tests; abandoning the rest`, "success"); ee.abort() }
+          }
+          return out
         })))
-        for (const o of outs) { diffs.push(o.diff); summaries.push(o.summary) }
+        const completed = outs.filter((o): o is { diff: string; log: string; summary: string } => !!o)
+        for (const o of completed) { diffs.push(o.diff); summaries.push(o.summary) }
+
+        // verified fast path: a test-passing attempt is correct by the repo's own
+        // criterion, so apply it directly (no tournament, no repair, no regression).
+        if (verified && verified.diff.trim()) {
+          const patch = join(work, "winner.patch")
+          await writeFile(patch, verified.diff)
+          try {
+            await git(dir, "apply", "--3way", patch)
+            toast("applied the verified winner", "success")
+            return `🏆 ultra: attempt ${verified.i} passed the repo's tests (${exitTestCmd}); applied it and abandoned the rest. Review it before committing.`
+          } catch (e: any) {
+            return `ultra: attempt ${verified.i} passed tests but the patch did not apply cleanly (${e?.message || e}). The diff:\n\n\`\`\`diff\n${verified.diff.slice(0, 6000)}\n\`\`\``
+          }
+        }
 
         if (diffs.every((d) => !d.trim())) {
           return `ultra ran ${eff.n} attempts but none made any changes (all diffs empty). The sub-agent could not act: the task may be too ambiguous, or the sandbox model could not reach its endpoint. Try a more specific task, or check the model endpoint.`
@@ -306,6 +391,73 @@ export const Ultra: Plugin = async (_input, options) => {
         const { ranked, conf } = await tournament(eff, task, summaries)
         const best = ranked[0]
 
+        // --- repair: critique the winner, run repairN guided passes, keep the best that verifies -----
+        // Best-of-N repair (regression-safe): fan out eff.repairN attempts off `base`, each seeded
+        // with the winner's diff and the SAME critique injected (one critique, reused). Keep-policy:
+        //   * with tests: keep the FIRST repair that PASSES the repo's tests; else keep the winner.
+        //     (a passing repair is correct by the repo's own criterion, so it cannot regress.)
+        //   * without tests: keep the FIRST repair the reasoned verifier CLEARLY prefers over the
+        //     current winner (same >k/2 threshold as before); else keep the winner.
+        // INVARIANT: the final applied result is never worse than the tournament winner.
+        if (eff.repair && diffs[best].trim()) {
+          status(`ultra: repair, ${eff.repairN} critique-guided pass(es) on the winner...`)
+          toast(`repair: ${eff.repairN} pass(es) on the winner`)
+          const crit = await critique(eff, task, diffs[best], completed[best].log)
+          const rtask =
+            `${task}\n\n--- A previous attempt (already applied to your working tree) produced a partial ` +
+            `solution. A reviewer flagged the issues below. Improve and COMPLETE it, keeping what is correct; ` +
+            `verify before finishing. ---\nREVIEWER NOTES:\n${crit}`
+          const rcmd = eff.repairAgent.replace("{task}", rtask.replace(/"/g, '\\"'))
+          // run one seeded repair attempt in its own worktree; returns its diff + summary.
+          const runRepair = async (r: number): Promise<{ diff: string; summary: string }> => {
+            const rwt = join(work, `repair-${r}`)
+            await git(dir, "worktree", "add", "--detach", rwt, base)
+            worktrees.push(rwt)
+            const wpatch = join(work, `winner-for-repair-${r}.patch`)
+            await writeFile(wpatch, diffs[best])
+            await git(rwt, "apply", "--3way", wpatch).catch(() => {})
+            const renv = await isolatedEnv(`repair-${r}`)
+            let rlog = ""
+            try {
+              const { stdout, stderr } = await execFileP("bash", ["-lc", `exec </dev/null; ${rcmd}`], { cwd: rwt, env: renv, timeout: eff.agentTimeoutMs, maxBuffer: 32 * 1024 * 1024, signal: ctx.abort })
+              rlog = (stdout || "") + (stderr || "")
+            } catch (e: any) { rlog = `agent error: ${e?.message || e}` }
+            await git(rwt, "add", "-A").catch(() => {})
+            const rdiff = await git(rwt, "diff", "--cached").catch(() => "")
+            const rsum = `AGENT LOG (tail):\n${rlog.slice(-1500)}\n\nDIFF:\n${rdiff.slice(0, 6000) || "(no changes)"}`
+            return { diff: rdiff, summary: rsum }
+          }
+          // run the repairN attempts in parallel, honoring the concurrency limiter (cc).
+          const rlimit = pLimit(eff.cc)
+          const reps = await Promise.all(Array.from({ length: eff.repairN }, (_, r) => rlimit(() => runRepair(r))))
+          // only real, changed candidates are eligible.
+          const cands = reps.filter((x) => x.diff.trim() && x.diff !== diffs[best])
+          let kept = false, why = ""
+          if (!cands.length) { status("ultra: repair produced no new change; winner kept") }
+          else if (testCmd) {
+            // regression-safe: keep the FIRST repair that PASSES the repo's own tests.
+            status(`ultra: repair keep-check (tests: ${testCmd}) on ${cands.length} candidate(s)...`)
+            for (let r = 0; r < cands.length; r++) {
+              const pass = await runTests(dir, base, work, `rep-${r}`, cands[r].diff, testCmd, eff.agentTimeoutMs, ctx.abort)
+              if (pass) { diffs[best] = cands[r].diff; summaries[best] = cands[r].summary; kept = true; why = `repair ${r} passes the tests`; break }
+            }
+            if (!kept) why = "no repair passed the tests"
+          } else {
+            // no tests: keep the FIRST repair the verifier CLEARLY prefers over the current winner.
+            const preferRepair = async (rsum: string) => {
+              const lim = pLimit(eff.cc)
+              const votes = await Promise.all(Array.from({ length: eff.k }, () => judge(eff, lim, task, summaries[best], rsum)))
+              return votes.filter((v) => v === "B").length > eff.k / 2  // B = repaired
+            }
+            for (let r = 0; r < cands.length; r++) {
+              if (await preferRepair(cands[r].summary)) { diffs[best] = cands[r].diff; summaries[best] = cands[r].summary; kept = true; why = `verifier prefers repair ${r}`; break }
+            }
+            if (!kept) why = "verifier keeps winner"
+          }
+          if (kept) { toast(`repair kept (${why})`, "success"); status(`ultra: repair kept (${why})`) }
+          else if (cands.length) status(`ultra: all repairs discarded (${why})`)
+        }
+
         // Normalised added-lines of a diff, so we can tell when attempts AGREE on the same change.
         const nonEmpty = diffs.filter((d) => d.trim()).length
         // Apply the verifier's TOP pick when it is confident, OR when a majority of attempts
@@ -345,7 +497,7 @@ export const Ultra: Plugin = async (_input, options) => {
     config: async (cfg: any) => {
       cfg.command = {
         ultra: {
-          description: "Best-of-N + verifier: run the task N times in isolated worktrees, apply the best. /ultra <task>",
+          description: "[v2] Best-of-N + verifier + repair: runs the task N times in isolated worktrees, verifies, then repairs the winner toward best-of-N. /ultra <task>",
           template:
             "Call the `ultra` tool exactly once, with `task` set to the request below. It runs the task several " +
             "times in isolated git worktrees and verifies the results. When it returns, report to the user exactly " +
@@ -357,7 +509,7 @@ export const Ultra: Plugin = async (_input, options) => {
       // routes each request through best-of-N. Sub-attempts run the default agent, so no recursion.
       cfg.agent = {
         ultra: {
-          description: "Best-of-N mode: run the request N times in isolated worktrees and apply the verified winner.",
+          description: "[v2] Best-of-N + repair mode: run the request N times in isolated worktrees, verify, repair, then apply the verified winner.",
           mode: "primary",
           color: "#A855F7",
           prompt: