@basein/runner 0.2.11 → 0.3.0

package/docs/mcpmark.md

@@ -86,8 +86,8 @@ identical prompt the recorded values *are* the right ones.
 ```
 claude -p  (cwd = the run dir)
  ├── mcp__bir__run_scenario ─────────────▶ bir-scenario ──┐   direct-mode delivery
- │                                                        │   (§4.4 — needs a trusted
- └── mcp__postgres__*  ──▶ bir-proxy ──▶ postgres-mcp     │    workspace, or it is denied)
+ │                                                        │   (§4.4 — allowed by bir's
+ └── mcp__postgres__*  ──▶ bir-proxy ──▶ postgres-mcp     │    hook once a plan is announced)
                                │            (uvx,         │
                       reports every call     mcp<2)       │
                                ▼              │           ▼
@@ -200,13 +200,17 @@ no `pipx`; both are equivalent here.
 Then:
 
 ```bash
-bir install --replay --local
+bir install --replay --local --server postgres
 ```
 
-That wraps `postgres` with `bir-proxy`, installs the hooks into `.claude/settings.json`, and
-adds the `bir` scenario server that direct mode delivers through. Note the port it reports —
+That writes a copy of `postgres` wrapped with `bir-proxy` into this project's local scope in
+`~/.claude.json` (`.mcp.json` stays as written), installs the hooks into
+`.claude/settings.local.json`, and adds the `bir` scenario server that direct mode delivers
+through. `--server postgres` is needed in a fresh directory: Claude Code never asks about a
+local-scope server, so `bir install` copies a `.mcp.json` server only once Claude Code has
+approved it, and otherwise prints `postgres — left unwrapped`. Note the port it reports —
 it picks a free one (**53411** here, not the 53455 the incident demo uses), and the hook
-URLs in `settings.json` are written for that port.
+URLs in `settings.local.json` are written for that port.
 
 ### 4.4 Deny the built-ins — this is the whole ballgame
 
@@ -235,13 +239,28 @@ Ignoring 2 permissions.allow entries from .claude/settings.json:
 this workspace has not been trusted.
 ```
 
-This is not cosmetic, and it fails *asymmetrically* in the worst possible way:
+This is not cosmetic, and it fails *asymmetrically*. Which half breaks depends on the
+runner version.
 
-- `mcp__postgres__*` keeps working, because bir's own `PreToolUse` answers
+**Current runner.** bir no longer approves wrapped MCP calls, other than a step a steered
+replay pins. For an ordinary
+`mcp__postgres__*` call its `PreToolUse` answers with no permission decision and joins the
+proxy's report by argument fingerprint ([server.ts](../src/control/server.ts),
+`injectionDecision`). With the allow list gone those calls fall to Claude Code's own
+permission check, and in `-p` mode they are denied, so trial 1 cannot even record the task.
+`mcp__bir__run_scenario` is now the tool the hook allows itself, and only while this
+session's direct plan has been announced (never in plan mode; your own deny and ask rules
+still apply). If `run_scenario` is refused anyway, or the model's first call is some other
+tool, the plan is retired without running a step (`replay.vetoed`, booked `not_steered`)
+and the model does the task.
+
+**0.2.12 and earlier, which §11 measured.** The other way round:
+
+- `mcp__postgres__*` kept working, because bir's own `PreToolUse` answered
   `permissionDecision: "allow"` on the correlation path ([server.ts:859](../src/control/server.ts#L859)).
-- `mcp__bir__run_scenario` is the **one** tool bir deliberately passes through
-  ([controller.ts:301](../src/replay/controller.ts#L301) → `{kind:"passthrough"}`), so it gets
-  no hook-granted allow. With the allow list gone it is denied, and in `-p` mode a denial is
+- `mcp__bir__run_scenario` was the **one** tool bir deliberately passed through
+  ([controller.ts:301](../src/replay/controller.ts#L301) → `{kind:"passthrough"}`), so it got
+  no hook-granted allow. With the allow list gone it was denied, and in `-p` mode a denial is
   final:
 
   ```
@@ -249,18 +268,22 @@ This is not cosmetic, and it fails *asymmetrically* in the worst possible way:
   but you haven't granted it yet.
   ```
 
-The plan then arms in `direct` mode, the model cannot reach the delivery tool, it does the
-task by hand, and the first manual call triggers divergence. Everything *looks* healthy —
-the task passes and the audit log says `mode=direct` — while the saving is negative. §11 is
+The plan then armed in `direct` mode, the model could not reach the delivery tool, it did the
+task by hand, and the first manual call triggered divergence. Everything *looked* healthy —
+the task passed and the audit log said `mode=direct` — while the saving was negative. §11 is
 the measurement.
 
-Fix before the first replay, either by running `claude` interactively in the directory once
+Fix before the first trial, either by running `claude` interactively in the directory once
 and accepting the trust dialog, or by setting in `~/.claude.json`:
 
 ```json
 "projects": { "C:/Users/Admin/Desktop/BasIns/mcpmark-hys": { "hasTrustDialogAccepted": true } }
 ```
 
+Other ways to let the postgres calls through: `--allowedTools` on the `claude -p` line, or
+`BIR_CORRELATION_DECISION=allow` in the shell that starts `bir-hooks` (§4.5), which brings
+back 0.2.12's auto-approval of every wrapped call.
+
 ### 4.5 Start the control server, in this directory
 
 ```bash
@@ -505,6 +528,11 @@ roughly the cost of just doing the task, plus overhead.
 **The mechanism is sound; the delivery channel was blocked.** Nothing in these six trials
 measures direct mode's actual saving, because direct delivery never once happened.
 
+*Superseded by later runners (§4.4):* bir now allows `run_scenario` itself for an announced
+direct plan, and a direct plan whose delivery never happens is retired without running a
+step. An untrusted directory no longer produces these `diverged` rows; it shows up as denied
+`mcp__postgres__*` calls instead.
+
 ### Next
 
 1. Trust the workspace (§4.4). Without it every replay measures the divergence path.
@@ -744,9 +772,14 @@ and read every `sampleValue` as though it were about to be pasted into a shell.
 | §13 (2) | a step whose tool returned `Error:` logged `ok=true` | [tool-error.ts](../src/replay/tool-error.ts): an MCP result with `isError: true` is a `failed` step at stage `tool_call`, in direct, divergence and steer mode alike. Since 2026-09-19 a leading `Error` text with `isError: false` no longer counts: replay judges a step by the same rule the recorder does |
 | §13 (3) | `steered_full` booked a saving on errored steps | [controller.ts](../src/replay/controller.ts): any errored step forces `outcome: "failed"`, which books a baseline sample |
 | §12 header | bundle said "previously recorded" for live work | [bundle.ts](../src/replay/bundle.ts): a bundle with no recorded entry says the work was just executed and its side effects are in place |
-| §12 session | `/scenario/run` picked the first unfinished session | [server.ts](../src/control/server.ts): it picks the session whose plan is armed |
+| §12 session | `/scenario/run` picked the first unfinished session | [server.ts](../src/control/server.ts): it runs only the plan of the session whose `run_scenario` call is waiting for it, matched on Claude Code's tool-use id where the host sends one; with none waiting it answers `no_plan` and runs nothing |
 | §12 latent | direct-mode bundle uncapped | [plan.ts](../src/replay/plan.ts): `runToCompletion` caps at `MAX_REPLAY_REASON` |
 | §6 | `steps_pinned` reported 1 for sequential pins | [controller.ts](../src/replay/controller.ts): counted as each pin threads, not from `pinned.size` |
 
 Not yet re-measured. Trials 9+ need a re-`calc` (the scenario must be regenerated for the
 validation pass to run) and then the §5 protocol again from a fresh baseline.
+
+Before re-running on a runner newer than 0.2.12, read §4.3–4.4 again: `bir install` needs
+`--server postgres` to wrap a fresh directory's server, and wrapped `mcp__postgres__*` calls
+need the trusted `allow` list (or `--allowedTools`) even to record trial 1, while
+`run_scenario` is now allowed by bir's hook.