pi-harness-delegate 0.2.2 → 0.4.0

package/README.md

@@ -39,18 +39,45 @@ Only the prompt is required. A **harness as first word** and/or **mode as next w
 Some modes have **default tasks** when the prompt is omitted:
 `/delegate review` reviews the current git diff (`scope: diff`), `/delegate security-audit` audits the repo. Modes without a default (`plan`, `implement`, `docs`, `general`) print a hint asking for a prompt.
 
-The `delegate` tool takes: `harness`, `task`, `mode`, `scope` (`diff` = git diff, `pr` = PR diff, path list, or whole repo), `model`, `maxBudgetUsd`, `allowDangerous`, `sessionId`, `pr`.
+The `delegate` tool takes: `harness`, `task`, `mode`, `scope` (`diff` = git diff, `pr` = PR diff, path list, or whole repo), `model`, `maxBudgetUsd`, `allowDangerous`, `sessionId`, `pr`. (`verify` is deliberately *not* a tool parameter — see below.)
 
 `claude_delegate` remains as a deprecated alias for `delegate{harness:claude}`.
 
+### Fan out to multiple harnesses
+
+`harness` also accepts `all` or a comma-separated list — the same task runs on every harness **concurrently**, up to `maxConcurrent`, and comes back as one comparison report instead of one report per harness:
+
+```bash
+/delegate all review the auth flow                 # every *detected* harness
+/delegate claude,codex plan the migration          # just these two
+delegate({ harness: "all", mode: "review", scope: "diff" })   # tool call form
+```
+
+- `all` resolves to whatever's actually installed (`detectAll()`) — an uninstalled harness is skipped and named in the report, it doesn't fail the run. An explicit list is validated the same way; an unknown name is also reported rather than aborting the rest.
+- Each harness's run goes through the same `delegate()` engine as a single-harness call and writes its own transcript to its own `~/.pi/agent/delegate/outputs/<harness>/`. Runs are launched together and execute in parallel, bounded by `maxConcurrent` (default `4`, one slot per supported harness) — a run beyond the cap queues for a free slot instead of failing, and the cap is enforced across pi processes, not just this one. **This means fan-out spend is genuinely simultaneous**: with the default cap, a 4-harness fan-out can bill all four at once instead of one after another — budget accordingly (`maxBudgetUsd` still applies per run).
+- The synthesized report is always ordered by the resolved harness list (e.g. `claude, codex, opencode`), regardless of which harness actually finishes first — it groups each harness's metrics + output and a total spend line (unknown-cost runs called out separately, same as `/delegate status`), assembled mechanically, not by asking a model to summarize.
+- A single-harness call (`harness: "claude"`, or omitted) behaves exactly as before, including the concurrency guard: it still fails fast with "another delegate run is already in progress" at capacity rather than queueing. Fan-out is opt-in by typing `all`/a list.
+- `/delegate all …` batches successful completions into one notification instead of one per harness; a failure is never delayed or folded into the batch — it surfaces immediately.
+- In the TUI, a fan-out shows **one overlay for the whole run** — a compact row per harness (spinner/✓/✗, elapsed, current tool activity) — rather than one popup per harness or an interleaved feed you can't attribute to a harness:
+  ```
+  ╭─ ⠋ delegate all · review · 1/4 · ⏱ 0:42──────────────────╮
+  │ ✓ claude   0:38  done                                    │
+  │ ⠹ codex    0:41  ▶ Bash: bun test                        │
+  │ ⠹ opencode 0:12  ✍ Looking at the auth middleware next…  │
+  │ … amp      queued                                        │
+  │ esc cancel all · m minimize                               │
+  ╰────────────────────────────────────────────────────────────╯
+  ```
+  Double-ESC cancels every in-flight (and still-queued) run at once; `m` minimizes; the status bar chip shows aggregate state across every status (e.g. `● 1✓ 1✗ 1▶ 1…` — done, failed, running, queued; zero counts are omitted, so it reads `● 4▶` while all four are in flight). A harness that fails keeps its failure reason on its row rather than blanking, so the overlay still says *why*. Single-harness runs keep the original one-run overlay unchanged.
+
 ## Harnesses
 
 | Harness | Binary | Permission mapping | Notes |
 | --- | --- | --- | --- |
-| `claude` | `claude` | `readonly→plan`, `edit→acceptEdits`, `danger→bypassPermissions` | Full stream-json, cost + context% |
-| `codex` | `codex` | `readonly→read-only`, `edit→workspace-write`, `danger→danger-full-access` | `codex exec --json`, best-effort JSONL |
-| `opencode` | `opencode` | `readonly→read-only`, `edit→allow-edit`, `danger→danger` | `opencode run --format json` |
-| `amp` | `amp` (`omp` alias) | `readonly→read-only`, `edit→workspace`, `danger→danger` | `amp --output jsonl` |
+| `claude` | `claude` | `readonly→plan`, `edit→acceptEdits`, `danger→bypassPermissions` | Full stream-json, cost + context%. Schema-verified against Claude Code 2.1.247. |
+| `codex` | `codex` | `readonly→read-only`, `edit→workspace-write`, `danger→danger-full-access` | `codex exec --json`. Schema-verified against codex-cli 0.149.1; cost is always unmeasured (`null`) on ChatGPT-plan auth. |
+| `opencode` | `opencode` | `readonly→read-only`, `edit→allow-edit`, `danger→danger` | `opencode run --format json`. Schema-verified against opencode 1.18.16. |
+| `amp` | `amp` (`omp` alias) | `readonly→read-only`, `edit→workspace`, `danger→danger` | `<binary> -p --mode json`, resolves whichever of `amp`/`omp` is actually on `PATH`. Schema-verified against omp 17.2.9 (Sourcegraph's real Amp CLI is unverified). |
 
 Detect availability: `delegate` checks `harness --version` at startup; missing harnesses hint install instructions.
 
@@ -69,19 +96,24 @@ Each mode is a markdown template with frontmatter:
 
 ```yaml
 ---
-name: review
-description: Code review of a scope. Read-only.
-permission: readonly      # normalized: readonly | edit | danger
+name: implement
+description: Implement a task with file edits. Runs checks.
+permission: edit          # normalized: readonly | edit | danger
 model: sonnet             # or gpt-5 for codex, etc. Aliases resolved via modelAliases
-defaultTask: Review the current git diff
-defaultScope: diff
+verify: bun test          # optional — host-run check after the harness exits
 ---
-You are a senior code reviewer delegated by the pi coding agent.
+You are a senior engineer delegated by the pi coding agent.
 ...
 ```
 
 **Native escape hatch:** if you need a harness-specific permission not covered by the normalized set, use the native key (`permissionMode: dontAsk`, `sandbox: ...`) — it overrides `permission` for that harness.
 
+**Verify:** `verify` is a shell command run **on the host** (never handed to the harness) right after it exits — e.g. `verify: bun test` on an `implement`/`docs`/`general` template turns "the harness says it's done" into an actual pass/fail. It's report-only: a failing verify is appended as its own section in the transcript and injected report, and surfaced in the tool result's `details.verify` (`{command, exitCode, ok}`), but it never changes whether the run itself is reported as an error — that stays whatever the harness reported. No template ships one by default — there's no universally-correct check command, so nothing is invented for you.
+
+- **Sources, deliberately limited:** a verify command can only come from a template's `verify:` frontmatter, or a human typing `/delegate --verify="<cmd>"` (quotes needed for multi-word commands) — the call-level value wins over the template's. **It is not a parameter on the `delegate` tool** — that's on purpose, not an oversight: a tool param is set by the model, and the model's context includes repo content and delegated-harness output, both of which an attacker could influence, so a model-settable verify command would be a prompt-injection → arbitrary-host-command path. A model that wants verification simply picks a template that declares one.
+- **Never runs on a `readonly` template.** `readonly` (`review`/`plan`/`security-audit`) guarantees no execution or modification — a verify command riding along on one would quietly break that guarantee. If a `readonly` template (or override) has a `verify` configured, it's recorded as skipped (`### Verify: \`cmd\`` / `⊘ skipped (readonly run)`) rather than run, and never silently dropped.
+- A project-local template's `verify` command is gated by the same trust check (`PI_TRUSTED=1` / `.pi/trusted`) as the rest of the template.
+
 **Template sources (later wins):**
 
 - `templates/shared/*.md` — portable prompt bodies
@@ -127,7 +159,7 @@ In `~/.pi/agent/settings.json`:
     "maxBudgetUsd": 3,
     "autoDelegateHints": false,
     "modelAliases": { "economy": "haiku", "balanced": "sonnet", "max": "opus" },
-    "maxConcurrent": 1,
+    "maxConcurrent": 4,
     "maxTranscripts": 100,
     "harnesses": {
       "claude": { "model": "sonnet" },
@@ -141,7 +173,7 @@ In `~/.pi/agent/settings.json`:
 Legacy `claudeDelegate` is auto-migrated into `delegate.harnesses.claude` (deprecated).
 
 - `modelAliases` — templates may use `economy|balanced|max` or any alias; resolution: call → template → harness → global.
-- `maxConcurrent` — cap overlapping runs (default 1 global; may be `{global:1, perHarness:{claude:1}}`).
+- `maxConcurrent` — cap overlapping runs (default **`4`**, one slot per supported harness; may be `{global:4, perHarness:{claude:1}}`). Enforced across pi processes, not just the current one — a file-based registry under `~/.pi/agent/delegate/runs/` tracks active runs, so the slots available to you also depend on any other pi session running `delegate`. This is a **genuinely parallel** spend cap now, not just a "don't overlap" guard: a single-harness `/delegate` call still fails fast (`another delegate run is already in progress`) the moment it's at capacity, but `/delegate all …` fan-out queues for a free slot instead and can run up to `maxConcurrent` harnesses at once — meaning up to that many harnesses billing simultaneously. Lower it if you want fan-out to stay sequential/cheaper (`"maxConcurrent": 1` restores the old one-at-a-time behavior for everything, single runs included).
 - `maxTranscripts` — oldest transcripts pruned beyond this count per harness (`0` disables).
 
 `autoDelegateHints` is off by default — no system-prompt bias. When `true`, explicit markers (`@harness`, `with codex`, `delegate … to claude`) and imperative review/plan phrasing append a hint.
@@ -150,6 +182,8 @@ Legacy `claudeDelegate` is auto-migrated into `delegate.harnesses.claude` (depre
 
 Every run records in details + transcript: harness, mode, permission (normalized + native), cost, tokens (input/output/cache), context% (prompt ÷ window), model, turns, duration, TTFT, stop reason, session id. Token + cost feed pi's `Usage`.
 
+Claude reports turns and cost on every run; Codex/OpenCode/Amp don't always. An unmeasured turn count or cost renders as `—`/`n/a` (never `0`/`$0.000`) everywhere it's shown — the transcript header, `formatMetrics`, tool results, and `/delegate history` — so an unmeasured run is never mistaken for a free one. `/delegate status` shows a per-harness spend rollup (e.g. `$1.234 over 12 run(s) (3 unknown)`); runs with unknown cost are counted separately rather than folded into the total as `$0`.
+
 ## Security model
 
 - `readonly` — no edits (e.g. Claude `plan`, Codex `read-only`).

package/extensions/activity.ts

@@ -1,6 +1,6 @@
 import { readdirSync, rmSync, statSync } from 'node:fs';
 import { join } from 'node:path';
-import type { ActivityEvent } from './harnesses/types.ts';
+import type { ActivityEvent, NormalizedPermission } from './harnesses/types.ts';
 
 function truncate(s: string, max: number): string {
   return s.length > max ? `${s.slice(0, max - 1)}…` : s;
@@ -13,18 +13,18 @@ export function safeSegmentName(name: string): string {
 }
 
 export interface MetricsInput {
-  numTurns: number;
-  totalCostUsd: number;
+  numTurns: number | null;
+  totalCostUsd: number | null;
   promptTokens: number;
   contextPercent: number | null;
   durationMs: number | null;
 }
 
-/** Compact run summary: `3 turn(s) · $0.54 · 62k tok · 6.2% ctx · 12s`. */
+/** Compact run summary: `3 turn(s) · $0.54 · 62k tok · 6.2% ctx · 12s`. Unknown numTurns/cost render as `—`. */
 export function formatMetrics(m: MetricsInput): string {
   const parts: Array<string | null> = [
-    `${m.numTurns} turn(s)`,
-    `$${m.totalCostUsd.toFixed(3)}`,
+    m.numTurns !== null ? `${m.numTurns} turn(s)` : '— turn(s)',
+    m.totalCostUsd !== null ? `$${m.totalCostUsd.toFixed(3)}` : '$—',
     m.promptTokens > 0 ? `${Math.round(m.promptTokens / 1000)}k tok` : null,
     typeof m.contextPercent === 'number' ? `${m.contextPercent.toFixed(1)}% ctx` : null,
     typeof m.durationMs === 'number' && m.durationMs !== null ? `${(m.durationMs / 1000).toFixed(0)}s` : null,
@@ -32,15 +32,15 @@ export function formatMetrics(m: MetricsInput): string {
   return parts.filter((p): p is string => Boolean(p)).join(' · ');
 }
 
-/** Parse the metadata header of a transcript file (without loading the whole body). */
+/** Parse the metadata header of a transcript file (without loading the whole body). `cost` is null when unknown. */
 export function parseTranscriptMeta(head: string): {
   mode: string;
-  cost: number;
+  cost: number | null;
   sessionId: string | null;
   harness: string | null;
 } {
   let mode = 'delegate';
-  let cost = 0;
+  let cost: number | null = null;
   let sessionId: string | null = null;
   let harness: string | null = null;
   const mm = /^# Delegated (?:Claude|Harness) run — (.+)$/m.exec(head);
@@ -58,6 +58,117 @@ export function parseTranscriptMeta(head: string): {
   return { mode, cost, sessionId, harness };
 }
 
+/** One history entry's harness + cost, for spend aggregation. */
+export interface SpendEntry {
+  harness: string;
+  cost: number | null;
+}
+
+/** Total cost, run count and unknown-cost run count — either per harness or overall. */
+export interface HarnessSpend {
+  totalCostUsd: number;
+  runs: number;
+  unknownRuns: number;
+}
+
+/** Roll up cost across history entries, per harness and overall. Runs with unknown cost are counted
+ *  separately rather than silently treated as $0. Pure — testable without the TUI. */
+export function aggregateSpend(entries: SpendEntry[]): {
+  byHarness: Record<string, HarnessSpend>;
+  total: HarnessSpend;
+} {
+  const byHarness: Record<string, HarnessSpend> = {};
+  const total: HarnessSpend = { totalCostUsd: 0, runs: 0, unknownRuns: 0 };
+  for (const e of entries) {
+    if (!byHarness[e.harness]) byHarness[e.harness] = { totalCostUsd: 0, runs: 0, unknownRuns: 0 };
+    const h = byHarness[e.harness];
+    h.runs++;
+    total.runs++;
+    if (e.cost === null) {
+      h.unknownRuns++;
+      total.unknownRuns++;
+    } else {
+      h.totalCostUsd += e.cost;
+      total.totalCostUsd += e.cost;
+    }
+  }
+  return { byHarness, total };
+}
+
+/** Format a `HarnessSpend` as `$1.234 over 12 run(s) (3 unknown)`. */
+export function formatSpend(s: HarnessSpend): string {
+  const unknown = s.unknownRuns > 0 ? ` (${s.unknownRuns} unknown)` : '';
+  return `$${s.totalCostUsd.toFixed(3)} over ${s.runs} run(s)${unknown}`;
+}
+
+/**
+ * Host-run verification (e.g. `bun test`) result — evidence only, never flips a run's `isError`.
+ * `skipped` is set (with a human-readable reason) when a verify command was configured but
+ * deliberately not executed — e.g. a `readonly` permission tier — instead of `exitCode`/`ok`
+ * describing a real run. Never silently dropped: a skip is still recorded and reported.
+ */
+export interface VerifyResult {
+  command: string;
+  /** `null` when the command was never run (see `skipped`). */
+  exitCode: number | null;
+  ok: boolean;
+  /** Combined stdout+stderr, capped to the last `maxTail` chars. Empty when skipped. */
+  outputTail: string;
+  /** Reason the command was not run, e.g. `"readonly run"`. Absent when it actually ran. */
+  skipped?: string;
+}
+
+const VERIFY_TAIL_MAX = 2000;
+
+/** Which verify command applies for a run: an explicit per-call override wins over the template's. */
+export function resolveVerifyCommand(
+  callOverride: string | undefined,
+  templateVerify: string | undefined,
+): string | undefined {
+  return callOverride || templateVerify || undefined;
+}
+
+/**
+ * Resolve the verify command AND decide whether it may actually run, given the run's effective
+ * permission. `readonly` templates guarantee no execution/modification — a verify command must
+ * never ride along on one (see AGENTS.md's Conventions), so `skip` is true whenever `permission`
+ * is `readonly`, regardless of who configured the command. Pure — testable without `pi.exec`.
+ */
+export function resolveVerifyPlan(
+  callOverride: string | undefined,
+  templateVerify: string | undefined,
+  permission: NormalizedPermission,
+): { command: string; skip: boolean } | undefined {
+  const command = resolveVerifyCommand(callOverride, templateVerify);
+  if (!command) return undefined;
+  return { command, skip: permission === 'readonly' };
+}
+
+/** Build a `VerifyResult` from a raw exit code + combined output, capping the tail. Pure. */
+export function buildVerifyResult(
+  command: string,
+  exitCode: number,
+  output: string,
+  maxTail = VERIFY_TAIL_MAX,
+): VerifyResult {
+  const outputTail = output.length > maxTail ? output.slice(-maxTail) : output;
+  return { command, exitCode, ok: exitCode === 0, outputTail };
+}
+
+/** A `VerifyResult` recording that a configured verify command was deliberately not run. */
+export function skipVerifyResult(command: string, reason: string): VerifyResult {
+  return { command, exitCode: null, ok: false, outputTail: '', skipped: reason };
+}
+
+/** Markdown section for a verify result — report-only evidence, distinct from the harness's own isError. */
+export function formatVerifySection(v: VerifyResult): string {
+  if (v.skipped) return `### Verify: \`${v.command}\`\n⊘ skipped (${v.skipped})`;
+  const lines = [`### Verify: \`${v.command}\``, v.ok ? `✓ exit ${v.exitCode}` : `✗ exit ${v.exitCode}`];
+  const tail = v.outputTail.trim();
+  if (tail) lines.push(...tail.split('\n').map(l => `  ${l}`));
+  return lines.join('\n');
+}
+
 /** Build the markdown report content injected into the session on the next turn. */
 export function buildReportContent(opts: {
   harness?: string;
@@ -66,6 +177,7 @@ export function buildReportContent(opts: {
   body: string;
   file?: string;
   sessionId?: string;
+  verify?: VerifyResult;
 }): string {
   const harness = opts.harness ?? 'claude';
   const header = `## ${harness} ${opts.mode} (${opts.metrics})`;
@@ -75,7 +187,61 @@ export function buildReportContent(opts: {
     foot.push(
       `resume: \`/delegate --harness=${opts.harness} --resume=${opts.sessionId} <prompt>\` (or /${opts.harness} --resume=${opts.sessionId})`,
     );
-  return [header, '', opts.body, foot.length > 0 ? `\n_${foot.join(' · ')}_` : ''].join('\n');
+  const verifySection = opts.verify ? `\n${formatVerifySection(opts.verify)}\n` : '';
+  return [header, '', opts.body, verifySection, foot.length > 0 ? `\n_${foot.join(' · ')}_` : ''].join('\n');
+}
+
+/** One harness's outcome within a fan-out, for the synthesized comparison report. */
+export interface FanoutRunSummary {
+  harness: string;
+  ok: boolean;
+  /** `formatMetrics` output — omitted (and `error` used instead) when the run failed to complete. */
+  metrics?: string;
+  cost: number | null;
+  body?: string;
+  file?: string;
+  sessionId?: string;
+  error?: string;
+  verify?: VerifyResult;
+}
+
+/**
+ * Order fan-out results by the originally resolved harness list rather than completion order.
+ * Concurrent fan-out runs finish in whatever order their harnesses happen to complete; this keeps
+ * `buildFanoutReport`'s output deterministic regardless of which one lands first. Entries with a
+ * harness not present in `order` are dropped (shouldn't happen — every result comes from `order`).
+ */
+export function orderFanoutResults<T extends { harness: string }>(order: readonly string[], results: T[]): T[] {
+  const byHarness = new Map(results.map(r => [r.harness, r]));
+  return order.map(h => byHarness.get(h)).filter((r): r is T => r !== undefined);
+}
+
+/**
+ * Mechanically assemble one comparison report across all fan-out runs — no second model call.
+ * Groups per-harness metrics/output and rolls up total spend via `aggregateSpend`/`formatSpend`.
+ * Header-free — callers wrap this body with their own `## <label> (...)` header.
+ */
+export function buildFanoutReport(opts: { runs: FanoutRunSummary[]; skipped: string[]; unknown: string[] }): string {
+  const lines: string[] = [];
+  if (opts.unknown.length > 0) lines.push(`_unknown harness(es), skipped: ${opts.unknown.join(', ')}_`);
+  if (opts.skipped.length > 0) lines.push(`_not installed, skipped: ${opts.skipped.join(', ')}_`);
+  if (opts.unknown.length > 0 || opts.skipped.length > 0) lines.push('');
+
+  const spend = aggregateSpend(opts.runs.map(r => ({ harness: r.harness, cost: r.cost })));
+  lines.push(`**Total spend:** ${formatSpend(spend.total)}`, '');
+
+  for (const r of opts.runs) {
+    lines.push(`### ${r.harness}${r.ok ? '' : ' — failed'}`);
+    if (r.ok) {
+      lines.push(r.metrics ?? '', '', r.body ?? '(empty)');
+    } else {
+      lines.push(`error: ${r.error ?? 'unknown error'}`);
+    }
+    if (r.verify) lines.push('', formatVerifySection(r.verify));
+    if (r.file) lines.push('', `_transcript: ${r.file}_`);
+    lines.push('');
+  }
+  return lines.join('\n').trimEnd();
 }
 
 /** Legacy wrapper for compat */
@@ -124,16 +290,54 @@ export function formatToolUse(name: string, input: Record<string, unknown>): str
   return first ? `${name}: ${truncate(first, 90)}` : name;
 }
 
+/**
+ * Tracks pending `tool_input` array indices by id, so a later `tool_result` can be matched to
+ * the row it actually belongs to instead of always landing on the last row.
+ *
+ * A harness emits N `tool_input` events followed by N `tool_result` events for a parallel
+ * tool-call batch, so "attach the result to the last entry" stamps every mark on the last row.
+ * Harnesses that don't carry an id fall back to that last-entry behavior via `resolve`.
+ * Shared by the transcript log builder and the live-feed builders (which also splice old
+ * entries off the front — `shift` keeps pending indices correct after that).
+ */
+export class ToolCallIndex {
+  private pending = new Map<string, number>();
+
+  set(id: string | undefined, index: number): void {
+    if (id) this.pending.set(id, index);
+  }
+
+  /** Index to mark, or -1 when an id was given but has no matching pending entry (don't mis-attribute). */
+  resolve(id: string | undefined, fallbackIndex: number): number {
+    if (id === undefined) return fallbackIndex;
+    const idx = this.pending.get(id);
+    if (idx === undefined) return -1;
+    this.pending.delete(id);
+    return idx;
+  }
+
+  /** Adjust pending indices after removing `count` entries from the front of the backing array. */
+  shift(count: number): void {
+    for (const [id, idx] of this.pending) {
+      const next = idx - count;
+      if (next < 0) this.pending.delete(id);
+      else this.pending.set(id, next);
+    }
+  }
+}
+
 /** Compact per-line activity log for the transcript (tool_input + results only). */
 export function collectActivityLog(events: ActivityEvent[]): string[] {
   const log: string[] = [];
+  const index = new ToolCallIndex();
   for (const ev of events) {
     if (ev.kind === 'tool_input') {
       log.push(`▶ ${formatToolUse(ev.name, ev.input)}`);
+      index.set(ev.id, log.length - 1);
     } else if (ev.kind === 'tool_result') {
-      const last = log.length - 1;
-      if (last >= 0 && log[last].startsWith('▶')) {
-        log[last] += ev.isError ? '  ✗ error' : '  ✓';
+      const idx = index.resolve(ev.id, log.length - 1);
+      if (idx >= 0 && log[idx].startsWith('▶')) {
+        log[idx] += ev.isError ? '  ✗ error' : '  ✓';
       }
     }
   }
@@ -152,8 +356,8 @@ export function buildTranscript(
     cwd: string;
     sessionId: string | null;
     resumed: boolean;
-    numTurns: number;
-    totalCostUsd: number;
+    numTurns: number | null;
+    totalCostUsd: number | null;
     isError: boolean;
     stopReason: string | null;
     durationMs: number | null;
@@ -167,6 +371,8 @@ export function buildTranscript(
     contextWindow: number | null;
     activityLog: string[];
     output: string;
+    /** Host-run verification result, when a `verify` command was configured for this run. */
+    verify?: VerifyResult;
   } & Record<string, unknown>,
 ): string {
   const harness = (opts.harness as string | undefined) ?? 'claude';
@@ -215,7 +421,7 @@ export function buildTranscript(
     `- model: ${opts.model ?? 'default'}`,
     `- cwd: ${opts.cwd}`,
     `- session: ${opts.sessionId ?? 'n/a'}${opts.resumed ? ' (resumed)' : ''}`,
-    `- turns: ${opts.numTurns} · cost: $${opts.totalCostUsd.toFixed(4)} · isError: ${opts.isError}`,
+    `- turns: ${opts.numTurns ?? 'n/a'} · cost: ${opts.totalCostUsd !== null ? `$${opts.totalCostUsd.toFixed(4)}` : 'n/a'} · isError: ${opts.isError}`,
     `- tokens: ${tokens ?? 'n/a'}`,
     `- context: ${context ?? 'n/a'}`,
     `- duration: ${duration ?? 'n/a'}`,
@@ -227,6 +433,7 @@ export function buildTranscript(
     '## Output',
     opts.output || '(empty)',
     '',
+    ...(opts.verify ? [formatVerifySection(opts.verify), ''] : []),
   ].join('\n');
 }
 

package/extensions/command.ts

@@ -16,32 +16,47 @@ export interface DelegateCommandArgs {
   sessionId?: string;
   /** GitHub PR number/URL to review (--pr=). */
   pr?: string;
+  /** Host-run verification command override (--verify=); takes precedence over the template's. */
+  verify?: string;
 }
 
 export type ClaudeCommandArgs = DelegateCommandArgs;
 
 const KNOWN_HARNESSES = new Set(['claude', 'codex', 'opencode', 'amp', 'omp']);
 
+/** True when `word` is `all`, a single known harness/alias, or a comma-separated list of them. */
+function looksLikeHarnessSpec(word: string, knownHarnesses: ReadonlySet<string>): boolean {
+  const lower = word.toLowerCase();
+  if (lower === 'all' || knownHarnesses.has(lower)) return true;
+  const parts = lower.split(',').filter(Boolean);
+  return parts.length > 1 && parts.every(p => knownHarnesses.has(p));
+}
+
 export function parseDelegateCommand(
   raw: string,
   knownModes: ReadonlySet<string>,
   knownHarnesses: ReadonlySet<string> = KNOWN_HARNESSES,
 ): DelegateCommandArgs {
   const flags: Record<string, string> = {};
-  const rest = raw.replace(/--([a-zA-Z-]+)=(\S+)/g, (_m, k: string, v: string) => {
-    flags[k] = v;
-    return '';
-  });
+  // supports quoted values ("…"/'…') so multi-word flags like --verify="bun test" survive intact
+  const rest = raw.replace(
+    /--([a-zA-Z-]+)=(?:"([^"]*)"|'([^']*)'|(\S+))/g,
+    (_m, k: string, dq: string | undefined, sq: string | undefined, bare: string | undefined) => {
+      flags[k] = dq ?? sq ?? bare ?? '';
+      return '';
+    },
+  );
 
   let harness = flags.harness?.toLowerCase();
   let mode = flags.mode;
   let task = rest.trim();
 
-  // First word handling: harness, mode, or both
+  // First word handling: harness (single, `all`, or comma list), mode, or both
   const words = task.split(/\s+/).filter(Boolean);
   let idx = 0;
-  if (!harness && words[idx] && knownHarnesses.has(words[idx].toLowerCase())) {
+  if (!harness && words[idx] && looksLikeHarnessSpec(words[idx], knownHarnesses)) {
     harness = words[idx].toLowerCase();
+    // single-name alias normalization only — a list/`all` is resolved later by resolveHarnessList
     if (harness === 'omp') harness = 'amp';
     idx++;
   }
@@ -62,6 +77,7 @@ export function parseDelegateCommand(
   }
   if (flags.resume) out.sessionId = flags.resume;
   if (flags.pr) out.pr = flags.pr;
+  if (flags.verify) out.verify = flags.verify;
   return out;
 }
 
@@ -69,6 +85,67 @@ export function parseClaudeCommand(raw: string, knownModes: ReadonlySet<string>)
   return parseDelegateCommand(raw, knownModes);
 }
 
+/** True when a `harness` field selects more than one harness: `all` or a comma-separated list. */
+export function isFanoutSpec(harness: string | undefined): boolean {
+  if (!harness) return false;
+  const lower = harness.trim().toLowerCase();
+  return lower === 'all' || lower.includes(',');
+}
+
+export interface HarnessListResolution {
+  /** Canonical harness names to run, in request order, deduped. */
+  resolved: string[];
+  /** Requested names that don't match any known harness or alias. */
+  unknown: string[];
+  /** Known harnesses that were requested/selected by `all` but aren't detected as installed. */
+  skipped: string[];
+}
+
+/**
+ * Resolve a `harness` field (`all` or a comma-separated list of names/aliases) into the
+ * canonical harness names a fan-out should actually run. Pure — detection results and the
+ * known-harness/alias lookups are passed in, no I/O happens here.
+ *
+ * `all` resolves to every *detected* harness (skipping uninstalled ones). An explicit list is
+ * validated against `isKnown`/`aliasOf` and also filtered by detection, so a named-but-uninstalled
+ * harness is reported (via `skipped`) instead of failing the whole run.
+ */
+export function resolveHarnessList(
+  spec: string,
+  opts: {
+    knownHarnesses: readonly string[];
+    aliasOf: (name: string) => string;
+    isKnown: (name: string) => boolean;
+    detection: Readonly<Record<string, { ok: boolean }>>;
+  },
+): HarnessListResolution {
+  const lower = spec.trim().toLowerCase();
+  const isAll = lower === 'all';
+  const requested = isAll
+    ? opts.knownHarnesses
+    : lower
+        .split(',')
+        .map(s => s.trim())
+        .filter(Boolean);
+
+  const resolved: string[] = [];
+  const unknown: string[] = [];
+  const skipped: string[] = [];
+  const seen = new Set<string>();
+  for (const raw of requested) {
+    if (!isAll && !opts.isKnown(raw)) {
+      unknown.push(raw);
+      continue;
+    }
+    const canon = opts.aliasOf(raw);
+    if (seen.has(canon)) continue;
+    seen.add(canon);
+    if (opts.detection[canon]?.ok) resolved.push(canon);
+    else skipped.push(canon);
+  }
+  return { resolved, unknown, skipped };
+}
+
 /**
  * Apply template defaults when the prompt is empty.
  */