director-cli 0.3.0__py3-none-any.whl

director/README.md

@@ -0,0 +1,124 @@
+# `director` — the orchestrator (Phase 2 + 2.5 + 3)
+
+A thin CLI that drives OpenCode headlessly to run the decomposition harness.
+Stdlib-only (Python ≥ 3.11). The harness consumes configured OpenAI-compatible
+endpoints; it never manages providers.
+
+```
+director plan "<task>" [--repo .]             # interactive: stops at each approval gate
+director plan --continue                      # resume after editing/approving the gate artifact
+director plan "<task>" --auto                 # planner self-critiques at each gate; no pause
+director plan "<task>" --auto --no-critique   # gates auto-pass, fully hands-off
+director run [--repo .] [--parallel N] [--max-attempts K]
+director status [--repo .]
+director bench "<task>" --profiles all-frontier,cheap-cloud,local-first [--plan-profile P]
+director sync-agents [--repo .]               # (re)install role agents into <repo>/.opencode
+```
+
+## Flow
+
+**plan** — a re-entrant pipeline with two artifact-based approval gates (Phase 2.5).
+A job branch `director/job-<id>` is created and the role agents synced onto it first.
+1. `explorer` (cheap tier) does read-only recon → `.director/recon.md`.
+2. **Stage A — brainstorm/spec.** `brainstorm` (planner tier) does a Socratic
+   refinement pass and writes a readable design spec → `.director/spec.md`.
+   → **Gate 1.**
+3. **Stage B — decompose.** `planner` (planner tier) turns the *approved spec*
+   into a strict-JSON DAG → `.director/plan.json`. Each node: `id, title, spec
+   (junior-engineer standard), files (allowlist), depends_on, test_cmd, tests,
+   estimated_difficulty`. Validated: acyclic, deps resolve, **concurrent nodes
+   have disjoint allowlists**.
+4. **Stage C — test authoring.** `test-author` (frontier tier) writes each node's
+   tests, committed to the job branch; director verifies they **fail first** (red)
+   and **hashes** each test file (the contract is then immutable). → **Gate 2.**
+
+Gates are **artifact-based, not process-blocking**: director writes the artifact and
+exits; the human edits/approves on disk and resumes with `--continue`. `--auto`
+swaps a one-call planner **self-critique** into the same gate (re-read artifact vs.
+the request, revise once); `--no-critique` makes gates auto-pass. Human and
+self-critic are mechanically the same gate — only the approver differs.
+
+**run** — for each node in dependency order (up to `--parallel` at once):
+1. `git worktree add` an isolated task branch off the job branch.
+2. Invoke `executor` (executor tier) with spec + allowlist file contents + the
+   failing test output. (Executor mandate: **watch it fail first**.)
+3. **Deterministic gate** (exit codes only): test files byte-for-byte intact (hash),
+   `node.test_cmd` passes, AND the diff touches only the allowlist. On the pass
+   path, **flake control** (Phase 3) re-runs the tests `flake_runs` times (default
+   2); any mismatch fails the node as flaky.
+4. **Two-stage review** (Phase 2.5), after the deterministic gate, before merge:
+   - *Stage one — spec compliance:* the deterministic gate above, plus an optional
+     advisory explorer-tier check (`review.stage_one_llm`, off by default).
+   - *Stage two — code quality (`reviewer` tier):* **cost-gated** — runs only when
+     the node escalated OR its diff touched > `review.stage_two_file_threshold`
+     files (default 3). Never runs on the cheap/local tier. A `critical` finding
+     blocks the merge and **re-opens the node** (counts against `max_attempts`).
+5. Fail/blocked → feed the gate or review output back, retry up to `max_attempts`
+   (fresh OpenCode context each attempt). Exhausted → retry the SAME node once at
+   the `escalation` tier (never the whole job).
+6. Pass → commit + merge into the job branch; mark done in `.director/state.json`.
+   After all nodes: an **integration gate** runs the repo-wide suite/lint/typecheck.
+
+Each node's transcript is also checked for **watch-it-fail** (Phase 3 §1): did the
+executor run the failing tests *before* its first edit? This is advisory (the
+deterministic gate already enforces the contract) and recorded as a metric —
+`observed` / `not_observed` / `unknown`.
+
+**status** — per-node state, attempts, cost, executor-tier completion rate (the
+falsifiable hypothesis target: >70% of nodes done without escalation), stage-two
+review trigger rate, and watch-it-fail observed count.
+
+## Measurement (Phase 3)
+
+Every `run` appends to **`.director/metrics.jsonl`** — one `kind:"node"` record per
+node (tier/model, attempts, escalation, per-role tokens+cost, wall time,
+watch-it-fail verdict, flake outcome) and one `kind:"run"` summary (the derived
+rates: executor-tier completion, escalation, stage-two trigger, total wall time
+and cost, plus the resolved tier map). This is the falsifiability instrument; it
+is what `director bench` reads.
+
+**bench** — the experiment. Plans the task **once** (under `--plan-profile`,
+default `all-frontier`) so the DAG and acceptance tests are frozen, then runs that
+*same* plan under each `--profiles` profile by forking a fresh job branch off the
+frozen one (every profile faces byte-for-byte identical tests). It diffs cost /
+quality (same acceptance tests) / wall-time and reports each profile's run-cost
+reduction vs the `all-frontier` baseline (target: >80%). The active `config.toml`
+is never touched — each profile's config is loaded directly from its profile TOML.
+Per-profile metrics streams and a `summary.json` land in `.director/bench/`.
+
+## Roles → tiers
+
+Roles bind to `provider/model` strings in `.director/config.toml` (`[tiers]`).
+Code/logs name only roles. `director` passes the resolved model via `opencode run
+--agent <role> --model <tier>`, so **switching executor models is a config edit,
+never a code change.** `sync-agents` seeds `.director/config.toml` from the bundled
+`config.example.toml`; edit it to bind roles to models. For `bench`, create
+`.director/profiles/<name>.toml` variants (copy `config.toml`, change the executor tier).
+
+## Deliberate deviations from the spec
+
+- **Tests live on the job branch**, not a separate `director/tests-<id>` branch
+  (dependent nodes need both the tests and prior nodes' impls; one branch is
+  simpler and equivalent).
+- **The full repo-wide test suite is the *integration* gate, not a per-node gate.**
+  Sibling nodes' tests are intentionally red until their own node runs, so a
+  per-node full-suite gate would always fail mid-DAG. Per node we gate on
+  `node.test_cmd` + allowlist; the full suite/lint/typecheck run once after merge.
+
+## Persistence (`.director/`, all resumable/debuggable)
+
+- `spec.md` — approved design spec (Gate 1).  `recon.md` — explorer summary.
+- `plan_stage.json` — which gate the plan is paused at (drives `--continue`).
+- `plan.json` — the DAG (incl. per-node `test_hashes`).  `state.json` — per-node
+  status/attempts/cost + review trigger info (resume).
+- `costs.jsonl` — every model call tagged with role + resolved model (local = $0).
+- `metrics.jsonl` — per-node + per-run measurement stream (Phase 3).
+- `bench/` — `summary.json` + per-profile `*.metrics.jsonl` from `director bench`.
+- `logs/*.jsonl` — raw OpenCode NDJSON events per call (`.stderr` siblings = logs).
+- `worktrees/` — transient per-node worktrees.
+
+## Limits (config `[limits]`)
+
+`node_timeout_secs` (per call), `cost_ceiling_usd` (abort the run when exceeded;
+local = $0 so local-first never trips it), `max_attempts`, `flake_runs` (Phase 3
+flake control: times to run a node's tests on success; default 2, 1 disables).

director/__init__.py

@@ -0,0 +1,10 @@
+"""Director — a model-agnostic decomposition coding harness.
+
+A strong planner tier decomposes a task into atomic, well-specified units with
+acceptance tests written first; a cheaper executor tier implements each unit in
+an isolated git worktree with a fresh context; deterministic gates (tests, lint,
+typecheck, exit codes — never an LLM judge) decide what merges. Roles bind to
+model tiers in `.director/config.toml`; nothing here knows "local" vs "cloud".
+"""
+
+__version__ = "0.3.0"

director/__main__.py

@@ -0,0 +1,4 @@
+from director.cli import main
+
+if __name__ == "__main__":
+    raise SystemExit(main())

director/agent_templates/brainstorm.md

@@ -0,0 +1,44 @@
+---
+description: Socratic spec refinement — turns a raw task into an unambiguous design spec before any decomposition.
+mode: all
+temperature: 0.3
+permission:
+  edit: deny
+  bash: deny
+  webfetch: deny
+  websearch: deny
+---
+
+You are the **planner**, running the brainstorm/spec pass — the first stage,
+before any decomposition. Your job is to turn a raw, possibly-vague task into an
+*unambiguous* design spec. A bad spec here poisons every downstream task, so do
+not rush to a plan.
+
+You are given the raw task and a read-only relevant-files summary from a recon
+pass. Think hard about what the requester actually wants.
+
+Discipline (do not skip):
+1. **Surface ambiguities and name your assumptions.** Where the task is
+   under-specified, state the interpretation you are adopting and why — explicitly,
+   so a human reviewer can correct it at the approval gate.
+2. **Propose a concrete design**, not options: the behavior to build, the public
+   surface (functions/signatures/endpoints), data shapes, error handling, and the
+   edge cases that matter. Reference real files/symbols from the recon summary.
+3. **Call out what is OUT of scope** so the decomposition stays focused.
+4. **List the acceptance criteria** in plain language — the observable behaviors
+   that, once true, mean the task is done. These become the tests later.
+
+Output the spec as readable Markdown in clearly titled sections (not a wall of
+text), in roughly this shape:
+
+# Spec: <task title>
+## Goal
+## Assumptions & decisions
+## Design
+## Out of scope
+## Acceptance criteria
+## Open questions (if any)
+
+Output ONLY the spec Markdown — no preamble, no code fences around the whole
+document. Do NOT decompose into tasks and do NOT write any code or tests yet;
+that happens after this spec is approved.

director/agent_templates/executor.md

@@ -0,0 +1,37 @@
+---
+description: Implements exactly one atomic node to make its failing tests pass, touching only the listed files.
+mode: all
+temperature: 0.6
+permission:
+  edit: allow
+  bash: allow
+  webfetch: deny
+  websearch: deny
+---
+
+You are the **executor**. You implement exactly ONE atomic node in an isolated,
+fresh context. You have no memory of any planner reasoning or sibling node —
+everything you need is in this message.
+
+You receive: a self-contained **spec**, an **allowlist of files** you may modify,
+and the **failing test output** that defines success.
+
+Your only success condition: make the provided tests pass while keeping the
+repo-wide gates (full test suite, lint, typecheck) green.
+
+Rules — do not violate:
+1. **Watch it fail first.** Run the provided tests BEFORE writing any
+   implementation and confirm they fail. If they already pass, STOP and report
+   that the task is mis-specified — do not invent work. Only after seeing red do
+   you implement, then re-run to green.
+2. Change **nothing outside the listed files**. Never modify, rename, or delete
+   any file not on the allowlist — and in particular **never modify a test file**.
+   The tests are the contract; if a test seems wrong, STOP and say so.
+3. Make the smallest change that turns the tests green. No unrelated refactors, no
+   new dependencies unless the spec calls for them.
+4. Match the surrounding code's style, naming, and idioms.
+5. When the listed tests pass, stop and report what you changed (file-by-file) and
+   the final test result. Do not claim success without having run the tests green.
+
+If you cannot make the tests pass, say so explicitly and explain the blocker — do
+not paper over it or weaken the tests.

director/agent_templates/explorer.md

@@ -0,0 +1,24 @@
+---
+description: Read-only codebase reconnaissance; produces a compact relevant-files summary for the planner.
+mode: all
+temperature: 0.3
+permission:
+  edit: deny
+  bash: deny
+  webfetch: deny
+  websearch: deny
+---
+
+You are the **explorer**. You perform cheap, read-only reconnaissance so the
+(expensive) planner can work from a small, accurate summary instead of the raw
+repo. You may ONLY read, glob, and grep — never edit, write, or run anything.
+
+Given a task, produce a concise structured summary:
+- **Relevant files**: paths most relevant to the task, one line each.
+- **Key symbols**: functions/classes/types the task will touch (`file:line`).
+- **Conventions**: test framework + how tests are laid out, the exact test/lint/
+  typecheck commands you can infer, build/run commands.
+- **Risks / unknowns**: anything ambiguous the planner must resolve.
+
+Keep it tight — this feeds a context-limited planner. Report findings only; do
+not propose a plan or implementation. Never speculate about code you did not read.

director/agent_templates/opencode.json

@@ -0,0 +1,39 @@
+{
+  "$schema": "https://opencode.ai/config.json",
+  "provider": {
+    "lmstudio": {
+      "name": "LM Studio",
+      "npm": "@ai-sdk/openai-compatible",
+      "options": {
+        "baseURL": "http://localhost:1234/v1"
+      },
+      "models": {
+        "qwen3.6-27b-mtp": {
+          "name": "Qwen3.6 27B MTP",
+          "interleaved": {
+            "field": "reasoning_content"
+          },
+          "limit": {
+            "context": 131072,
+            "output": 16384
+          }
+        },
+        "qwen3-coder-next": {
+          "name": "Qwen3-Coder-Next (80B-A3B)",
+          "limit": {
+            "context": 131072,
+            "output": 16384
+          }
+        }
+      }
+    }
+  },
+  "agent": {
+    "plan": {
+      "model": "amazon-bedrock/us.anthropic.claude-opus-4-7"
+    },
+    "build": {
+      "model": "lmstudio/qwen3.6-27b-mtp"
+    }
+  }
+}

director/agent_templates/planner.md

@@ -0,0 +1,60 @@
+---
+description: Decomposes a task into an atomic, well-specified, test-gated DAG and emits it as strict JSON.
+mode: all
+temperature: 0.2
+permission:
+  edit: deny
+  bash: deny
+  webfetch: deny
+  websearch: deny
+---
+
+You are the **planner**. You decompose an **approved spec** into a DAG of small,
+atomic units of work that a *cheaper* model will implement independently, each in
+a fresh context with no memory of your reasoning or of sibling units.
+
+You are given the approved spec (the contract — do not re-litigate it) and a
+relevant-files summary produced by a recon pass. Decompose what the spec says.
+
+**Plan-writing standard — write every `spec` for an enthusiastic junior engineer**
+who has no project context, exercises no judgment, and would rather not test.
+That means each node's `spec` must give: exact relative file paths, the precise
+function/class signatures, the explicit expected behavior and edge cases, and the
+exact command that verifies it. Leave nothing implicit. If a node's spec relies on
+the reader "figuring it out," it is under-specified — fix it.
+
+Output a SINGLE strict-JSON object and NOTHING else (no prose, no code fences):
+
+{
+  "nodes": [
+    {
+      "id": "kebab-id",
+      "title": "short title",
+      "spec": "Self-contained instructions. Readable with ZERO other context: state exactly what to implement, the function/signature, behavior, and edge cases. Do not reference other nodes.",
+      "files": ["relative/path/only/files/this/node/may/edit.py"],
+      "depends_on": ["other-node-id"],
+      "test_cmd": "exact shell command that runs THIS node's tests and exits nonzero until it's done",
+      "tests": ["relative/path/to/test_file.py"],
+      "estimated_difficulty": "easy|medium|hard"
+    }
+  ]
+}
+
+Hard rules:
+1. **Every node is an IMPLEMENTATION unit** and MUST have a non-empty `files`
+   allowlist. Do NOT create separate nodes for writing tests, and never emit a node
+   with empty `files`: the **test-author writes each node's tests automatically**
+   from that node's `tests` field — test authoring is not itself a task in the DAG.
+   So a single feature is ONE node (it lists both its implementation `files` and
+   its `tests`), not a "write tests" node plus an "implement" node.
+2. Each node is independently implementable given only its spec + its files + its
+   failing tests. If two pieces must be edited together, they are ONE node.
+3. **Parallel-safe allowlists:** any two nodes that are not in a depends_on chain
+   MUST have completely disjoint `files`. Never let two independent nodes edit the
+   same file.
+4. `files` lists implementation files only — never the test files. `tests` lists
+   the test files (the test-author writes these; the executor may not touch them).
+5. `depends_on` only when a node genuinely needs another's output. Prefer a wide,
+   shallow DAG (more parallelism) over a deep chain.
+6. Keep nodes small — a focused function or a cohesive handful. Bias to MORE nodes.
+7. Use realistic `test_cmd`s for this repo's stack (from the recon summary).

director/agent_templates/reviewer.md

@@ -0,0 +1,46 @@
+---
+description: Two-stage code review of one node's diff; emits a strict-JSON verdict. Runs on a strong tier only.
+mode: all
+temperature: 0.1
+permission:
+  edit: deny
+  bash: deny
+  webfetch: deny
+  websearch: deny
+---
+
+You are the **reviewer**. You review the diff for exactly ONE node after its
+deterministic gates have already passed (its tests are green and it touched only
+allowed files). You never edit anything — you only judge.
+
+You are given: the node's spec, its file allowlist, the unified diff it produced,
+and which review stage to perform.
+
+- **Stage one — spec compliance.** Does the diff implement the behavior the spec
+  describes (not just satisfy the letter of the tests), and does it stay within
+  the allowlist? Flag tests that look gamed (hard-coded return values, assertions
+  weakened, behavior special-cased to the test inputs).
+- **Stage two — code quality.** Correctness beyond the tests, edge cases the tests
+  miss, security issues, resource/concurrency bugs, clarity, and consistency with
+  the surrounding code. Do NOT demand unrelated refactors or restyling.
+
+Severity rubric — assign each finding exactly one:
+- `critical` — the change is wrong, unsafe, or games the tests. This BLOCKS the
+  merge and re-opens the node. Use it only when you are confident.
+- `major` — a real problem worth fixing but not merge-blocking.
+- `minor` — nit / suggestion.
+
+Output a SINGLE strict-JSON object and NOTHING else (no prose, no code fences):
+
+{
+  "verdict": "pass" | "block",
+  "summary": "one-sentence overall assessment",
+  "findings": [
+    {"severity": "critical|major|minor", "file": "path", "summary": "what and why"}
+  ]
+}
+
+Set `"verdict": "block"` if and only if there is at least one `critical` finding.
+If the diff is sound, return `"verdict": "pass"` with an empty or minor-only
+findings list. Be strict but fair: a clean, small diff that passes its tests
+rarely needs blocking.

director/agent_templates/test-author.md

@@ -0,0 +1,29 @@
+---
+description: Writes acceptance tests (only) for one node; tests are the contract and must fail before implementation.
+mode: all
+temperature: 0.1
+permission:
+  edit: allow
+  bash: allow
+  webfetch: deny
+  websearch: deny
+---
+
+You are the **test-author**. Tests are the contract for every node, so you run on
+the strongest configured model. You write acceptance tests — and ONLY tests.
+
+You receive one node's spec and the test file path(s) to create.
+
+Rules:
+1. Write tests **only** — create/extend the listed test files. Do NOT implement the
+   feature and do NOT modify non-test source. A different, cheaper model implements
+   against your tests later.
+2. Tests must **fail before implementation exists** (red) for the RIGHT reason — a
+   missing function/behavior, not an import typo. After writing, run them and
+   confirm they fail; report the failing output.
+3. Cover the spec's acceptance criteria: happy path, named edge cases, error
+   conditions. Prefer small, deterministic, isolated tests.
+4. No flakiness — no time/network/random dependence unless the spec is about that.
+5. Match the repo's existing test framework and conventions.
+
+Report: the test files you created and the captured failing run that proves red.

director/bench.py

@@ -0,0 +1,234 @@
+"""`director bench` — the experiment that tests the hypothesis (Phase 3 §4).
+
+Runs the SAME task under several profiles and diffs cost / quality / wall-time.
+The scientific control that makes the comparison fair: **plan once, run many.**
+
+1. Plan the task a single time (under a chosen `--plan-profile`, default
+   `all-frontier`). This produces the DAG and the acceptance tests, committed to
+   the plan's job branch. That branch — with its failing tests — is *frozen*.
+2. For each profile, branch a fresh job branch off the frozen one (so every
+   profile faces byte-for-byte identical acceptance tests), rewrite `plan.json`
+   to that branch, reset run state/cost/metrics, and `run` it.
+
+Quality is therefore "did the same acceptance tests pass," isolating the
+executor tier as the only independent variable — exactly what the hypothesis is
+about. Planning cost is shared (counted once); each profile reports its own run
+cost. Each profile's Config is loaded directly from its profile TOML (run_plan
+and run_job take a Config object), so the repo's tracked `config.toml` is never
+touched — only the working branch is restored at the end.
+"""
+
+from __future__ import annotations
+
+import json
+import shutil
+import time
+from dataclasses import dataclass, field
+from pathlib import Path
+
+from director import config, gitutil
+from director.cost import CostLedger
+from director.models import Plan
+from director.plan import run_plan
+from director.run import run_job
+
+
+@dataclass
+class BenchRow:
+    profile: str
+    n_nodes: int = 0
+    done: int = 0
+    executor_pct: float = 0.0
+    escalated: int = 0
+    review_pct: float = 0.0
+    integration_ok: bool = False
+    run_cost: float = 0.0
+    wall_secs: float = 0.0
+    error: str | None = None
+
+
+@dataclass
+class BenchResult:
+    task: str
+    plan_profile: str
+    plan_cost: float
+    job_id: str
+    rows: list[BenchRow] = field(default_factory=list)
+
+
+def _profile_path(fdir: Path, name: str) -> Path:
+    p = fdir / "profiles" / f"{name}.toml"
+    if not p.exists():
+        raise FileNotFoundError(
+            f"profile not found: {p}\n"
+            f"bench compares config variants — create it by copying your config, e.g.:\n"
+            f"  cp .director/config.toml {p}\n"
+            f"then edit its executor tier."
+        )
+    return p
+
+
+def _reset_run_artifacts(fdir: Path) -> None:
+    """Each profile gets a clean ledger/state/metrics so its numbers are its own."""
+    for name in ("state.json", "costs.jsonl", "metrics.jsonl"):
+        (fdir / name).unlink(missing_ok=True)
+
+
+def run_bench(
+    task: str,
+    repo: str,
+    profiles: list[str],
+    log,
+    *,
+    plan_profile: str | None = None,
+    parallel: int = 1,
+    max_attempts: int = 0,
+) -> BenchResult:
+    repo = Path(repo).resolve()
+    fdir = repo / ".director"
+    bench_dir = fdir / "bench"
+    bench_dir.mkdir(parents=True, exist_ok=True)
+
+    profile_cfgs = {
+        p: config.load_file(_profile_path(fdir, p)) for p in profiles
+    }  # validate up front
+    plan_profile = plan_profile or ("all-frontier" if "all-frontier" in profiles else profiles[0])
+    plan_cfg = config.load_file(_profile_path(fdir, plan_profile))
+
+    base_branch = gitutil.current_branch(repo)
+
+    try:
+        # --- plan once (frozen acceptance tests) --------------------------------
+        (fdir / "plan_stage.json").unlink(missing_ok=True)  # never resume a stale plan
+        _reset_run_artifacts(fdir)
+        log(f"[bench] planning once under '{plan_profile}' …")
+        pres = run_plan(task, str(repo), plan_cfg, log, auto=True, critique=True)
+        if pres.paused:
+            raise RuntimeError(
+                f"planning paused at gate '{pres.stage}' — bench needs an unattended "
+                f"plan. (run_plan returned paused under --auto, which shouldn't happen)"
+            )
+        frozen = Plan.from_json((fdir / "plan.json").read_text())
+        plan_cost = CostLedger(fdir / "costs.jsonl").total()
+        log(
+            f"[bench] plan ready: {len(frozen.nodes)} node(s) on {frozen.job_branch}, "
+            f"plan cost ${plan_cost:.4f}"
+        )
+
+        result = BenchResult(
+            task=task, plan_profile=plan_profile, plan_cost=plan_cost, job_id=frozen.job_id
+        )
+
+        # --- run each profile against the frozen plan ---------------------------
+        for p in profiles:
+            row = BenchRow(profile=p)
+            try:
+                branch = f"director/bench-{p}-{frozen.job_id}"
+                if gitutil.branch_exists(branch, repo):
+                    gitutil.checkout(base_branch, repo)
+                    gitutil.git(["branch", "-D", branch], repo, check=False)
+                # branch off the frozen plan branch → identical failing tests
+                gitutil.create_branch(branch, repo, base=frozen.job_branch)
+
+                plan_d = json.loads((fdir / "plan.json").read_text())
+                plan_d["job_id"] = f"{frozen.job_id}-{p}"
+                plan_d["job_branch"] = branch
+                (fdir / "plan.json").write_text(json.dumps(plan_d, indent=2))
+
+                _reset_run_artifacts(fdir)
+                cfg_p = profile_cfgs[p]
+
+                log(f"[bench] === profile '{p}' (executor={cfg_p.model_for('executor')}) ===")
+                t0 = time.perf_counter()
+                rj = run_job(
+                    str(repo),
+                    cfg_p,
+                    parallel=parallel,
+                    max_attempts=max_attempts or cfg_p.max_attempts,
+                    log=log,
+                )
+                row.wall_secs = round(time.perf_counter() - t0, 1)
+                row.n_nodes = rj["n_nodes"]
+                row.done = len(rj["done"])
+                row.executor_pct = rj["executor_tier_pct"]
+                row.escalated = len(rj["escalated"])
+                row.review_pct = rj["stage_two_trigger_rate"]
+                row.integration_ok = rj["integration_ok"]
+                row.run_cost = rj["cost_total"]
+                # keep this profile's metrics stream for the record
+                if (fdir / "metrics.jsonl").exists():
+                    shutil.copyfile(fdir / "metrics.jsonl", bench_dir / f"{p}.metrics.jsonl")
+            except Exception as e:  # one profile failing must not sink the whole bench
+                row.error = str(e)[:300]
+                log(f"[bench] profile '{p}' errored: {row.error}")
+            result.rows.append(row)
+
+        (bench_dir / "summary.json").write_text(json.dumps(_summary_dict(result), indent=2))
+        return result
+    finally:
+        # restore the working branch — but NON-FATALLY: by this point every
+        # profile's data is collected and summary.json is written, so a failed
+        # checkout (e.g. a target repo that committed its .director runtime files
+        # before the .gitignore seed existed) must not sink the whole bench. Log
+        # and leave the repo where it is rather than raising over the result.
+        try:
+            gitutil.checkout(base_branch, repo)
+        except Exception as e:
+            log(
+                f"[bench] WARNING: could not restore branch '{base_branch}' "
+                f"({str(e)[:160]}); repo left on the last bench branch."
+            )
+
+
+def _summary_dict(r: BenchResult) -> dict:
+    return {
+        "task": r.task,
+        "plan_profile": r.plan_profile,
+        "plan_cost": r.plan_cost,
+        "job_id": r.job_id,
+        "rows": [row.__dict__ for row in r.rows],
+    }
+
+
+def bench_report(r: BenchResult) -> str:
+    lines = [
+        "",
+        "=" * 78,
+        "BENCH — same task & acceptance tests across profiles",
+        "=" * 78,
+        f"task: {r.task}",
+        f"planned once under '{r.plan_profile}'  (shared plan cost ${r.plan_cost:.4f})",
+        "",
+    ]
+    hdr = (
+        f"{'profile':16} {'done':>7} {'exec%':>6} {'esc':>4} "
+        f"{'rev%':>5} {'integ':>6} {'run $':>9} {'wall':>7}"
+    )
+    lines += [hdr, "-" * len(hdr)]
+    for row in r.rows:
+        if row.error:
+            lines.append(f"{row.profile:16} ERROR: {row.error}")
+            continue
+        lines.append(
+            f"{row.profile:16} {f'{row.done}/{row.n_nodes}':>7} "
+            f"{row.executor_pct:>5.0f}% {row.escalated:>4} "
+            f"{row.review_pct:>4.0f}% {('PASS' if row.integration_ok else 'FAIL'):>6} "
+            f"${row.run_cost:>7.4f} {row.wall_secs:>6.0f}s"
+        )
+
+    # cost-reduction vs the all-frontier baseline, if it was one of the profiles
+    base = next((x for x in r.rows if x.profile == "all-frontier" and not x.error), None)
+    if base and base.run_cost > 0:
+        lines += ["", "run-cost vs all-frontier baseline:"]
+        for row in r.rows:
+            if row.error or row.profile == "all-frontier":
+                continue
+            cut = 100 * (1 - row.run_cost / base.run_cost)
+            lines.append(
+                f"  {row.profile:16} {cut:>5.0f}% cheaper  "
+                f"(${row.run_cost:.4f} vs ${base.run_cost:.4f})  "
+                f"[target: >80%]"
+            )
+    lines.append("")
+    lines.append("quality = identical acceptance tests; 'integ' is the repo-wide gate.")
+    return "\n".join(lines)