@basein/runner 0.3.6 → 0.3.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -303,9 +303,14 @@ does a plain `bir install` where replay was never set up. `bir install --replay`
303
303
  never turns back on a `bir replay off` of yours. The switches are kept per
304
304
  project in `~/.baseinstrunner/installed.json`, so a recorder the hook starts — with Claude
305
305
  Code's environment, not your terminal's — keeps them. A recorder reads them when
306
- it starts: `bir up --restart` applies a change. `BIR_REPLAY`,
307
- `BIR_REPLAY_ALLOW_SERVERS` and `BIR_MIN_STEER_SIMILARITY` in the environment
308
- still override them, and `bir doctor` says which source is in effect.
306
+ it starts: `bir up --restart` applies a change. `BIR_REPLAY` and
307
+ `BIR_REPLAY_ALLOW_SERVERS` in the environment still override them, and
308
+ `bir doctor` says which source is in effect.
309
+
310
+ Every match is used. When the service matches a prompt to a recording with a
311
+ ready scenario, bir runs the scenario; there is no second, stricter score on
312
+ this side. (Up to 0.3.6 bir declined a match under 0.92, and
313
+ `BIR_MIN_STEER_SIMILARITY` set that bar; it is ignored now.)
309
314
 
310
315
  Two switches, not one, because of what replay is: **a scenario is a pre-approved
311
316
  list of tool calls with computed arguments.** A steered step is answered
@@ -314,15 +319,19 @@ prompt you would have seen — only a call that plausibly is the step, though (t
314
319
  same command, the same file, the same kind of arguments); any other call runs as
315
320
  written and is asked about as usual. A *directly executed* step never reaches the
316
321
  permission system at all — it is dispatched to a `bir-proxy` and called on a
317
- connection that is already open — and `run_scenario`, the call that starts the
322
+ connection that is already open, or, for Claude Code's own `Read`, `Glob`,
323
+ `Grep`, `Edit`, `Write` and `Bash`, run by the recorder itself in the project
324
+ folder — and `run_scenario`, the call that starts the
318
325
  run, is answered `allow` while the recorder has announced the plan. In plan mode
319
326
  nothing is announced and nothing is approved. Set `BIR_REPLAY_ALLOW_SERVERS` to
320
327
  the servers you are content to have called unattended (it also covers a steered
321
- plan's recovery), and read [the guide's §5.1](docs/calculatedReplayGuide.md)
328
+ plan's recovery), `BIR_BUILTIN_TOOLS` and `BIR_BASH_ALLOW` to the built-ins and
329
+ commands, and read [the guide's §5.1](docs/calculatedReplayGuide.md)
322
330
  before enabling it anywhere that matters.
323
331
 
324
- What it buys: on a scenario whose every step is a wrapped MCP tool, the whole
325
- sequence runs through connections the proxies already hold — **zero model tokens**
332
+ What it buys: on a scenario whose every step is a wrapped MCP tool or one of
333
+ those built-ins, the whole sequence runs through connections the proxies already
334
+ hold, or on the recorder itself — **zero model tokens**
326
335
  — and the model's only job is one tool call that reads the results. Such a plan
327
336
  is announced only while a `bir` server has told the recorder it is connected,
328
337
  and runs only through `run_scenario`: if the model calls anything else first,
@@ -433,9 +442,8 @@ what to do when they register, so there is one switch and no second copy of it:
433
442
  |---|---|
434
443
  | `BIR_REPLAY` | `0` turns replay off and any other value on, over the project's stored switch (`bir replay on` / `off`; a first plain `bir install` stores off). Nothing below matters while it is off. |
435
444
  | `BIR_REPLAY_ALLOW_SERVERS` | Server keys bir may call unattended: **direct** execution, and a steered plan's recovery. Unset means the list `bir replay allow` stored for the project, else every wrapped server; narrowing it is the recommendation. |
436
- | `BIR_BUILTIN_TOOLS` | Built-in tools bir may execute itself during a replay — in the session's own project directory, and confined to it. Default `Read,Glob,Grep` (read-only). Add `Edit,Write,Bash` only deliberately: a runner-side step never reaches Claude Code's permission prompts. `none` turns it off, and a built-in step hands the turn back as before. |
437
- | `BIR_BASH_ALLOW` | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset, `Bash` never executes on the runner, even when `BIR_BUILTIN_TOOLS` lists it. A plan's Bash steps share a working directory, as Claude Code's do (variables and functions do not persist, also as in Claude Code). |
438
- | `BIR_MIN_STEER_SIMILARITY` | Minimum match similarity to replay (default `0.92`, above the service's own `0.9` detection threshold). |
445
+ | `BIR_BUILTIN_TOOLS` | Built-in tools bir may execute itself during a replay — in the session's own project directory, and confined to it. Default all six: `Read,Glob,Grep,Edit,Write,Bash` (up to 0.3.7 only the read-only `Read,Glob,Grep`). A runner-side step never reaches Claude Code's permission prompts, so leave off the list any tool you want the model to call itself. `none` turns it off, and built-in steps are steered in the session. |
446
+ | `BIR_BASH_ALLOW` | Command prefixes a runner-side `Bash` step may run. Unset means any command (`*`); a list such as `git,npm test` narrows it; `none` allows no command, and `Bash` steps are then steered in the session. A plan's Bash steps share a working directory, as Claude Code's do (variables and functions do not persist, also as in Claude Code). |
439
447
  | `BIR_DERIVE_API_KEY` | **Optional.** Working out what a new request is about — which fleet, which file, which date — is done for you by the service on its own key, as long as you are signed in. Set this (an Anthropic key) only to keep that reading on your machine, on your key. Signed out *and* unset, a scenario whose values change between requests is declined rather than replayed on stale ones. |
440
448
  | `ANTHROPIC_API_KEY` | Not used by the background recorder, which has Claude Code's environment and would bill your own key — unless `BIR_USE_ANTHROPIC_API_KEY=1`. A `bir-hooks` run by hand in a terminal uses that terminal's key. |
441
449
  | `BIR_DERIVE_MODEL` | Derivation model, when you set a key of your own (default `claude-haiku-4-5-20251001`). |
@@ -64,16 +64,15 @@
64
64
  * BIR_BUILTIN_TOOLS comma-separated built-ins the runner may execute
65
65
  * itself during a replay, in the session's own
66
66
  * project directory and confined to it
67
- * (replay/builtins.ts). Default Read,Glob,Grep —
68
- * the read-only set. Add Edit,Write,Bash only
69
- * deliberately: a runner-side step never reaches
70
- * the host's permission prompts. `none` turns
71
- * built-ins off, and then a built-in step hands
72
- * the turn back, as before
67
+ * (replay/builtins.ts). Default all six:
68
+ * Read,Glob,Grep,Edit,Write,Bash. A runner-side
69
+ * step never reaches the host's permission
70
+ * prompts; Read,Glob,Grep is the read-only set
71
+ * (the default up to 0.3.7). `none` turns
72
+ * built-ins off, and their steps are steered
73
73
  * BIR_BASH_ALLOW command prefixes a runner-side Bash step may run
74
- * (`*` for all). Unset, Bash never executes on the
75
- * runner even when BIR_BUILTIN_TOOLS lists it
76
- * BIR_MIN_STEER_SIMILARITY minimum match similarity to replay (default 0.92)
74
+ * (`*` for all). Unset means `*`; `none` means no
75
+ * command, and Bash steps are steered
77
76
  * BIR_DERIVE_API_KEY OPTIONAL. Derivation — reading what this turn acts
78
77
  * on — is done by the service for a signed-in
79
78
  * runner. Set this (an Anthropic key) only to keep
@@ -64,16 +64,15 @@
64
64
  * BIR_BUILTIN_TOOLS comma-separated built-ins the runner may execute
65
65
  * itself during a replay, in the session's own
66
66
  * project directory and confined to it
67
- * (replay/builtins.ts). Default Read,Glob,Grep —
68
- * the read-only set. Add Edit,Write,Bash only
69
- * deliberately: a runner-side step never reaches
70
- * the host's permission prompts. `none` turns
71
- * built-ins off, and then a built-in step hands
72
- * the turn back, as before
67
+ * (replay/builtins.ts). Default all six:
68
+ * Read,Glob,Grep,Edit,Write,Bash. A runner-side
69
+ * step never reaches the host's permission
70
+ * prompts; Read,Glob,Grep is the read-only set
71
+ * (the default up to 0.3.7). `none` turns
72
+ * built-ins off, and their steps are steered
73
73
  * BIR_BASH_ALLOW command prefixes a runner-side Bash step may run
74
- * (`*` for all). Unset, Bash never executes on the
75
- * runner even when BIR_BUILTIN_TOOLS lists it
76
- * BIR_MIN_STEER_SIMILARITY minimum match similarity to replay (default 0.92)
74
+ * (`*` for all). Unset means `*`; `none` means no
75
+ * command, and Bash steps are steered
77
76
  * BIR_DERIVE_API_KEY OPTIONAL. Derivation — reading what this turn acts
78
77
  * on — is done by the service for a signed-in
79
78
  * runner. Set this (an Anthropic key) only to keep
package/dist/bin/bir.js CHANGED
@@ -3292,8 +3292,7 @@ async function doctor(args) {
3292
3292
  // reading a log file.
3293
3293
  const replay = health?.replay;
3294
3294
  if (replay?.enabled) {
3295
- out(`Replay : ON servers=${replay.allowServers?.join(",") || "(all wrapped)"} ` +
3296
- `minSimilarity=${replay.minSimilarity ?? "?"}` +
3295
+ out(`Replay : ON servers=${replay.allowServers?.join(",") || "(all wrapped)"}` +
3297
3296
  (replay.source ? ` source=${replay.source}` : ""));
3298
3297
  // An older control server sends only `deriveKey`; read it as the two states
3299
3298
  // it could describe then.
@@ -3527,8 +3526,7 @@ function replayPolicyCommand(args) {
3527
3526
  ? "off (not asked for at install: --no-replay, `bir uninstall --replay`, or no --replay) — `bir replay on` turns it on"
3528
3527
  : "off"}`);
3529
3528
  out(` direct servers ${policy?.allowServers?.length ? policy.allowServers.join(", ") : "(all wrapped)"}`);
3530
- out(` minSimilarity ${policy?.minSimilarity ?? 0.92}`);
3531
- const overrides = ["BIR_REPLAY", "BIR_REPLAY_ALLOW_SERVERS", "BIR_MIN_STEER_SIMILARITY"].filter((k) => process.env[k] !== undefined);
3529
+ const overrides = ["BIR_REPLAY", "BIR_REPLAY_ALLOW_SERVERS"].filter((k) => process.env[k] !== undefined);
3532
3530
  if (overrides.length) {
3533
3531
  out(` (set in this shell's environment, which a recorder started from it would prefer: ${overrides.join(", ")})`);
3534
3532
  }
@@ -17,6 +17,7 @@
17
17
  * declined and what mode armed; the service knows the baseline, the ledger and
18
18
  * whether a run ever became a recording. Both lists are printed, worst first.
19
19
  */
20
+ import { EXECUTABLE_BUILTINS } from "../replay/builtins.js";
20
21
  import { ReadClock, jsonText, readWithin, serviceProblem, underMcp, unreachable } from "./service-read.js";
21
22
  const num = (v) => {
22
23
  if (v === undefined || v === null || v === "")
@@ -170,9 +171,11 @@ export const DECLINES = {
170
171
  cause: "the matched recording had no ready scenario at that moment.",
171
172
  fix: "wait for the calculation, or start one with `bir scenario calc <runId>`, then run the prompt again.",
172
173
  },
174
+ // Written only by runners up to 0.3.6, which re-scored a match against a
175
+ // bar of their own; kept so their log lines still read.
173
176
  similarity: {
174
- cause: "the prompt was a hit on the recording but below the steering threshold (BIR_MIN_STEER_SIMILARITY, default 0.92).",
175
- fix: "phrase the prompt closer to the recorded one, or lower BIR_MIN_STEER_SIMILARITY.",
177
+ cause: "an older runner used a scenario only when the prompt scored 0.92 or more, above the service's own bar for a match (0.9, or a checked match under it).",
178
+ fix: "update the runner (`npm install -g @basein/runner`) and restart it (`bir up --restart`): every match is used now.",
176
179
  },
177
180
  coverage: {
178
181
  cause: "no step of the scenario could run anywhere: its tools are neither wrapped MCP servers registered in this session nor built-ins.",
@@ -265,12 +268,25 @@ export function localFindings(turn) {
265
268
  .filter((t) => Boolean(t));
266
269
  if (turn.plan.mode === "steer" && live.length > 0) {
267
270
  const lowSaving = savedPct(turn) != null && savedPct(turn) < LOW_SAVING_PCT;
271
+ const names = [...new Set(live)];
272
+ // One of the runner's own built-ins steered only because it was off the
273
+ // list (or the turn ran on a runner older than 0.3.8); anything else can
274
+ // run directly only behind a wrapped MCP server.
275
+ const offList = names.filter((n) => EXECUTABLE_BUILTINS.has(n));
276
+ const elsewhere = names.filter((n) => !EXECUTABLE_BUILTINS.has(n));
277
+ const fixes = [];
278
+ if (offList.length > 0) {
279
+ fixes.push(`The runner runs ${offList.join(", ")} itself when ${offList.length === 1 ? "it is" : "they are"} on \`BIR_BUILTIN_TOOLS\` (unset means all six, from runner 0.3.8) and, for Bash, \`BIR_BASH_ALLOW\` is not \`none\`: fix those where the recorder starts, then \`bir up --restart\`.`);
280
+ }
281
+ if (elsewhere.length > 0) {
282
+ fixes.push(`Put the data ${elsewhere.join(", ")} ${elsewhere.length === 1 ? "reaches" : "reach"} behind an MCP server, wrap it (\`bir install --server <name>\`), and record the prompt again so its steps are MCP calls.`);
283
+ }
268
284
  out.push({
269
285
  code: "steer_mode",
270
286
  severity: lowSaving ? "warn" : "info",
271
- title: `The plan armed in steer mode: ${live.length} of ${coverage.length} steps are reachable only inside the session (${[...new Set(live)].join(", ")}).`,
272
- cause: "In steer mode the runner pins each call's inputs and auto-approves it, but the model still emits every tool call, reads every result and writes the answer — so the whole live turn is still paid for. Direct mode, where the model makes no tool call, needs every step to be a wrapped MCP server tool.",
273
- fix: "Put the data those tools reach behind an MCP server, wrap it (`bir install --server <name>`), and record the prompt again so its steps are MCP calls.",
287
+ title: `The plan armed in steer mode: ${live.length} of ${coverage.length} steps are reachable only inside the session (${names.join(", ")}).`,
288
+ cause: "In steer mode the runner pins each call's inputs and auto-approves it, but the model still emits every tool call, reads every result and writes the answer — so the whole live turn is still paid for. Direct mode, where the model makes no tool call, needs every step to be a wrapped MCP server tool or a built-in the runner runs itself (Read, Glob, Grep, Edit, Write, Bash).",
289
+ fix: fixes.join(" "),
274
290
  evidence: { coverage, tools },
275
291
  });
276
292
  }
@@ -278,8 +294,8 @@ export function localFindings(turn) {
278
294
  out.push({
279
295
  code: "direct_mode",
280
296
  severity: "info",
281
- title: `The plan armed in direct mode: ${turn.plan.steps ?? coverage.length} steps ran on the proxies, no model in the loop.`,
282
- cause: "Every step is a wrapped MCP tool this session has a proxy for.",
297
+ title: `The plan armed in direct mode: ${turn.plan.steps ?? coverage.length} steps ran on the proxies and the runner, no model in the loop.`,
298
+ cause: "Every step is a wrapped MCP tool this session has a proxy for, or a built-in the runner runs itself.",
283
299
  fix: "",
284
300
  });
285
301
  }
@@ -330,7 +346,7 @@ export function localFindings(turn) {
330
346
  title: `Saved ${pct(p)}: ${usd(r.savedUsd ?? 0)} of a ${usd((r.savedUsd ?? 0) + cost)} baseline.`,
331
347
  cause: `The live session cost ${usd(r.session)} of that baseline${turn.plan?.mode === "steer" ? " — steer mode pays for the whole turn." : "."}`,
332
348
  fix: turn.plan?.mode === "steer"
333
- ? "Direct mode is the fix: wrap the servers so every step is an MCP call (see steer_mode)."
349
+ ? "Direct mode is the fix (see steer_mode for what kept this plan out of it)."
334
350
  : "Compare the baseline with a few unsteered runs (BIR_REPLAY=0); a single cheap source run makes any replay look poor.",
335
351
  evidence: { ...r, costUsd: cost, savedPct: p },
336
352
  });
@@ -412,8 +412,6 @@ export interface ReplayPolicy {
412
412
  disabledBy?: "install";
413
413
  /** Server keys eligible for direct execution. Absent means every wrapped one. */
414
414
  allowServers?: string[];
415
- /** `BIR_MIN_STEER_SIMILARITY`'s stored twin. */
416
- minSimilarity?: number;
417
415
  }
418
416
  /** What `bir install` decided for one project directory. */
419
417
  export interface ProjectRecord {
@@ -52,7 +52,7 @@ export declare function derivationKey(daemon?: boolean): {
52
52
  apiKey?: string;
53
53
  source?: string;
54
54
  };
55
- /** Where the three replay switches came from, for `/health` and `bir doctor`. */
55
+ /** Where the two replay switches came from, for `/health` and `bir doctor`. */
56
56
  export type ReplaySource = "env" | "sidecar" | "default";
57
57
  /** What a recorder still serves (K4, `ControlServer.activity()`). */
58
58
  export interface RecorderActivity {
@@ -155,11 +155,9 @@ export function derivationKey(daemon = process.env[DAEMON_ENV] === "1") {
155
155
  /**
156
156
  * Assemble the replay configuration (docs/calculatedReplay.md §5.3 of the guide).
157
157
  *
158
- * The steer threshold defaults **above** the service's own detection threshold
159
- * (0.9). Detection decides "do not record this again", which is cheap — the worst
160
- * case is a lost recording. Steering decides "do not think about this again",
161
- * which is a much stronger claim, and a 0.90-similar prompt usually covers only
162
- * part of the live task.
158
+ * There is no steer threshold: a match the service hands out is steered
159
+ * (controller.ts, gate 4). A recorder started with `BIR_MIN_STEER_SIMILARITY`
160
+ * or `BIR_MIN_SEGMENT_STEER_SIMILARITY` set ignores them.
163
161
  */
164
162
  function buildReplayOptions(keeper, policy, key) {
165
163
  // The environment still wins — an operator at a terminal is the most
@@ -168,8 +166,7 @@ function buildReplayOptions(keeper, policy, key) {
168
166
  // switches the project stored instead of forgetting them (generate.ts).
169
167
  const envReplay = process.env.BIR_REPLAY;
170
168
  const envAllow = process.env.BIR_REPLAY_ALLOW_SERVERS;
171
- const envMin = Number(process.env.BIR_MIN_STEER_SIMILARITY);
172
- const fromEnv = envReplay !== undefined || envAllow !== undefined || Number.isFinite(envMin);
169
+ const fromEnv = envReplay !== undefined || envAllow !== undefined;
173
170
  const source = fromEnv ? "env" : policy ? "sidecar" : "default";
174
171
  const enabled = envReplay !== undefined ? envReplay !== "0" : policy?.enabled !== false;
175
172
  const allowServers = envAllow !== undefined
@@ -177,16 +174,14 @@ function buildReplayOptions(keeper, policy, key) {
177
174
  : policy?.allowServers
178
175
  ? parseAllowList(policy.allowServers.join(","))
179
176
  : undefined;
180
- const minSimilarity = Number.isFinite(envMin) ? envMin : (policy?.minSimilarity ?? 0.92);
181
177
  // Which built-ins the runner may execute itself (replay/builtins.ts):
182
- // read-only by default, `BIR_BUILTIN_TOOLS` to change, `BIR_BASH_ALLOW`
183
- // before a `Bash` step may run at all. Environment-only knobs, like
184
- // `BIR_REPLAY_RECORDED_OUTPUTS`.
178
+ // all six by default, `BIR_BUILTIN_TOOLS` to narrow them, `BIR_BASH_ALLOW`
179
+ // to narrow the commands a `Bash` step may run. Environment-only knobs,
180
+ // like `BIR_REPLAY_RECORDED_OUTPUTS`.
185
181
  const builtinTools = parseBuiltinTools(process.env.BIR_BUILTIN_TOOLS);
186
182
  const bashAllow = parseBashAllow(process.env.BIR_BASH_ALLOW);
187
183
  const opts = {
188
184
  enabled,
189
- minSimilarity,
190
185
  allowServers,
191
186
  builtinTools,
192
187
  bashAllow,
@@ -222,10 +217,6 @@ function buildReplayOptions(keeper, policy, key) {
222
217
  // Observe-only until an operator has read the logs and turned it on
223
218
  // (segmented.md R-OUT-10, R-LIFE-7). Arming is never the way to find out.
224
219
  segmentArm: process.env.BIR_SEGMENT_ARM === "1",
225
- minSegmentSimilarity: (() => {
226
- const n = Number(process.env.BIR_MIN_SEGMENT_STEER_SIMILARITY);
227
- return Number.isFinite(n) ? n : 0.95;
228
- })(),
229
220
  // Both read live, never captured (C5). The recorder signs in after it
230
221
  // listens, refreshes the token as the session outlives it, and may sign
231
222
  // in or out while it runs; replay reads whatever holds *now*. The old
@@ -244,12 +235,11 @@ function buildReplayOptions(keeper, policy, key) {
244
235
  // and nothing else: an ordinary wrapped call keeps its permission prompt
245
236
  // (see the correlation line under `control.listening`).
246
237
  logLine("replay.enabled", {
247
- minSimilarity: opts.minSimilarity,
248
238
  allowServers: allowServers ? [...allowServers].join(",") : "(all wrapped)",
249
239
  // Which built-ins may run on the runner itself, and whether Bash may at
250
240
  // all — loud for the same reason as the rest of this line.
251
241
  builtins: builtinTools.size > 0 ? [...builtinTools].join(",") : "(none)",
252
- bashAllow: builtinTools.has("Bash") ? (bashAllow ? bashAllow.join(",") : "(unset — Bash never runs here)") : undefined,
242
+ bashAllow: builtinTools.has("Bash") ? (bashAllow.length > 0 ? bashAllow.join(",") : "(none — Bash steps are steered)") : undefined,
253
243
  // Who reads what this turn acts on (segmented.md R-PARAM-5). Named at
254
244
  // startup because the third state is a replay that silently never runs.
255
245
  derive: key.source ??
@@ -31,7 +31,7 @@ import { BIR_CALL_ID, fingerprint, newCallId, parseQualifiedName, qualifyToolNam
31
31
  import { StepIndexAllocator } from "./ordering.js";
32
32
  import { pidAlive } from "./discovery.js";
33
33
  import { pinCompatible } from "./pin-guard.js";
34
- import { contextForToolUse, intentForToolUse, markTranscriptUsage, recentToolResults, settledLastAssistantText, usageSince, } from "./transcript.js";
34
+ import { contextForToolUse, intentForToolUse, markTranscriptUsage, recentToolResults, settledLastAssistantText, thinkingForToolUse, usageSince, } from "./transcript.js";
35
35
  import { isHousekeepingCall } from "../record/housekeeping.js";
36
36
  import { redact } from "../record/redact.js";
37
37
  import { DEFAULT_MAX_STRING_BYTES, serializeCapped, truncateStrings } from "../record/truncate.js";
@@ -404,7 +404,7 @@ export class ControlServer {
404
404
  this.token = opts.token ?? randomBytes(32).toString("hex");
405
405
  for (const name of opts.wrappedServers ?? [])
406
406
  this.wrapped.add(name);
407
- this.replay = new ReplayController(opts.replay ?? { enabled: false, minSimilarity: 1 });
407
+ this.replay = new ReplayController(opts.replay ?? { enabled: false });
408
408
  this.deriveRecentResults = opts.deriveRecentResults ?? 5;
409
409
  this.intentSources = opts.intentSources ?? new Set(["text", "thinking", "tool"]);
410
410
  this.timings = { ...DEFAULT_SESSION_TIMINGS, ...(opts.sessionTimings ?? {}) };
@@ -579,7 +579,6 @@ export class ControlServer {
579
579
  replay: {
580
580
  enabled: this.replay.enabled,
581
581
  source: this.opts.replaySource ?? null,
582
- minSimilarity: this.opts.replay?.minSimilarity ?? null,
583
582
  allowServers: this.opts.replay?.allowServers ? [...this.opts.replay.allowServers] : null,
584
583
  deriveKey: Boolean(this.opts.replay?.apiKey ?? process.env.ANTHROPIC_API_KEY),
585
584
  // Who reads the turn for its parameters (segmented.md R-PARAM-5). The
@@ -2347,15 +2346,18 @@ export class ControlServer {
2347
2346
  const agentId = payload.agent_id ?? session.agentId;
2348
2347
  // The real intent, read from every line sharing this message's id
2349
2348
  // (segmented.md R-INTENT-3). `context` stays the *written* text only, with
2350
- // its agent prefix: thinking is embedded but never stored as text
2351
- // (R-INTENT-2), so it rides separately in `intent`.
2349
+ // its agent prefix, so thinking rides separately in `intent`.
2352
2350
  const intent = intentForToolUse(session.transcriptPath, toolUseId, this.intentSources);
2353
2351
  const reasoning = intent?.source === "text" ? intent.text : "";
2354
2352
  // The prefix rides on whether there is a subagent, not on whether it said
2355
2353
  // anything: a subagent's steps must be tellable apart even when it narrated
2356
2354
  // nothing. What it never carries is thinking (R-INTENT-2).
2357
2355
  const context = agentId ? `[agent ${agentId}] ${reasoning}`.trim() : reasoning;
2358
- const thinkingIntent = intent?.source === "thinking" ? { text: intent.text, source: "thinking" } : undefined;
2356
+ // Sent whether or not there was written text: the service keeps it for the
2357
+ // recording page (R-INTENT-2, amended 2026-09-29), and still embeds the
2358
+ // written text first. The probe below reads `intent`, unchanged.
2359
+ const thinking = thinkingForToolUse(session.transcriptPath, toolUseId, this.intentSources);
2360
+ const thinkingIntent = thinking ? { text: thinking, source: "thinking" } : undefined;
2359
2361
  // ── replay steering (docs/calculatedReplay.md §7) ────────────────────────
2360
2362
  // Runs before anything else, because two of its four answers end the call.
2361
2363
  // Not in plan mode (nothing is approved on the user's behalf there), not
@@ -78,6 +78,15 @@ export interface ToolUseIntent {
78
78
  * next rung tried. It governs only what may be *sent*; it never stops a probe.
79
79
  */
80
80
  export declare function intentForToolUse(transcriptPath: string | undefined, toolUseId: string, sources?: ReadonlySet<IntentSource>): ToolUseIntent | undefined;
81
+ /**
82
+ * The thinking `toolUseId` was chosen in, whether or not the agent also wrote
83
+ * text. The service keeps it with the step so the recording page can show it
84
+ * (segmented.md R-INTENT-2, amended 2026-09-29); {@link intentForToolUse}
85
+ * returns thinking only in place of missing text, which is what matching wants.
86
+ *
87
+ * `sources` is `BIR_INTENT_SOURCES`: without `thinking` in it, none is sent.
88
+ */
89
+ export declare function thinkingForToolUse(transcriptPath: string | undefined, toolUseId: string, sources?: ReadonlySet<IntentSource>): string | undefined;
81
90
  /** The assistant text that accompanied `toolUseId`, or "" when unavailable. */
82
91
  export declare function contextForToolUse(transcriptPath: string | undefined, toolUseId: string): string;
83
92
  /** The last assistant text in the transcript — the turn's final answer. */
@@ -92,8 +92,9 @@ const MAX_INDEXES = 16;
92
92
  const MAX_MESSAGES = 128;
93
93
  /**
94
94
  * Characters of one message's text, and separately of its thinking, kept.
95
- * Text becomes a step's `context`; thinking is only embedded, and the intent
96
- * probe sends 4,000 characters of either.
95
+ * Text becomes a step's `context`; thinking goes with the step as its
96
+ * `intent`, capped again at 8 KB on the way out, and the intent probe sends
97
+ * 4,000 characters of either.
97
98
  */
98
99
  const MAX_MESSAGE_CHARS = 16 * 1024;
99
100
  /** `tool_use_id`s remembered, newest last. */
@@ -443,11 +444,7 @@ export function recentToolResults(transcriptPath, n = 5, maxBytes = 2048) {
443
444
  * next rung tried. It governs only what may be *sent*; it never stops a probe.
444
445
  */
445
446
  export function intentForToolUse(transcriptPath, toolUseId, sources = new Set(["text", "thinking", "tool"])) {
446
- if (!transcriptPath || !toolUseId)
447
- return undefined;
448
- const idx = indexFor(transcriptPath);
449
- const holder = idx?.toolUses.get(toolUseId);
450
- const message = holder ? idx?.messages.get(holder.message) : undefined;
447
+ const message = messageFor(transcriptPath, toolUseId);
451
448
  if (!message)
452
449
  return undefined;
453
450
  if (sources.has("text")) {
@@ -462,6 +459,28 @@ export function intentForToolUse(transcriptPath, toolUseId, sources = new Set(["
462
459
  }
463
460
  return undefined;
464
461
  }
462
+ /**
463
+ * The thinking `toolUseId` was chosen in, whether or not the agent also wrote
464
+ * text. The service keeps it with the step so the recording page can show it
465
+ * (segmented.md R-INTENT-2, amended 2026-09-29); {@link intentForToolUse}
466
+ * returns thinking only in place of missing text, which is what matching wants.
467
+ *
468
+ * `sources` is `BIR_INTENT_SOURCES`: without `thinking` in it, none is sent.
469
+ */
470
+ export function thinkingForToolUse(transcriptPath, toolUseId, sources = new Set(["text", "thinking", "tool"])) {
471
+ if (!sources.has("thinking"))
472
+ return undefined;
473
+ const thinking = messageFor(transcriptPath, toolUseId)?.thinkings.join("\n").trim();
474
+ return thinking || undefined;
475
+ }
476
+ /** The message that emitted `toolUseId`: its text and thinking, gathered across its lines. */
477
+ function messageFor(transcriptPath, toolUseId) {
478
+ if (!transcriptPath || !toolUseId)
479
+ return undefined;
480
+ const idx = indexFor(transcriptPath);
481
+ const holder = idx?.toolUses.get(toolUseId);
482
+ return holder ? idx?.messages.get(holder.message) : undefined;
483
+ }
465
484
  /** The assistant text that accompanied `toolUseId`, or "" when unavailable. */
466
485
  export function contextForToolUse(transcriptPath, toolUseId) {
467
486
  if (!transcriptPath || !toolUseId)
@@ -62,7 +62,7 @@ export declare const TEXT_CAPS: {
62
62
  readonly answer: number;
63
63
  /** A step's written reasoning. */
64
64
  readonly context: number;
65
- /** A step's thinking text, sent only to be embedded. */
65
+ /** A step's thinking text: embedded, and kept for the recording page. */
66
66
  readonly intent: number;
67
67
  /** A tool's error text when it is not JSON. */
68
68
  readonly toolError: number;
@@ -66,7 +66,7 @@ export const TEXT_CAPS = {
66
66
  answer: 64 * 1024,
67
67
  /** A step's written reasoning. */
68
68
  context: 8 * 1024,
69
- /** A step's thinking text, sent only to be embedded. */
69
+ /** A step's thinking text: embedded, and kept for the recording page. */
70
70
  intent: 8 * 1024,
71
71
  /** A tool's error text when it is not JSON. */
72
72
  toolError: 8 * 1024,
@@ -10,18 +10,21 @@
10
10
  *
11
11
  * THE RULES, because this bypasses the host's permission prompts entirely:
12
12
  *
13
- * - **Default deny beyond read-only.** Which built-ins may run at all is
14
- * `BIR_BUILTIN_TOOLS` ({@link parseBuiltinTools}), and its default is the
15
- * read-only set (`Read,Glob,Grep`). `Edit`, `Write` and `Bash` are an
16
- * explicit, per-environment opt-in — a recorded `rm` must never run
17
- * because a prompt looked similar.
13
+ * - **All six by default, each a switch.** Which built-ins may run at all is
14
+ * `BIR_BUILTIN_TOOLS` ({@link parseBuiltinTools}); unset, it is every one
15
+ * this module implements (up to 0.3.7 it was the read-only set,
16
+ * `Read,Glob,Grep`). The permission story is the one a steered step
17
+ * already has — its pinned call is answered `allow` too. What running it
18
+ * here changes is that no model sees the call before it runs, so an
19
+ * operator who wants one in the loop takes the tool off the list.
18
20
  * - **Path jail.** Every path is resolved against, and confined to, the
19
21
  * session's project directory ({@link jailed}), symlinks included. The
20
22
  * daemon's own cwd is never used: it is `~/.baseinstrunner`, not the
21
23
  * project (§5.1 of the change plan).
22
- * - **Bash runs nothing unless `BIR_BASH_ALLOW` says so.** A command must
23
- * match one of its prefixes ({@link bashAllowed}); unset means `Bash`
24
- * never executes here, even when allow-listed.
24
+ * - **Bash runs any command unless `BIR_BASH_ALLOW` narrows it.** Set, a
25
+ * command must match one of its prefixes ({@link bashAllowed}); `none`
26
+ * allows no command, and then `Bash` counts as off the list, so its steps
27
+ * are steered in the session (controller.ts).
25
28
  * - **Same failure contract as an MCP step.** A step that *cannot* run here
26
29
  * throws {@link BuiltinNotRunnable} (→ `not_run`: the session can still do
27
30
  * it, or a recorded output may stand in under
@@ -36,14 +39,15 @@
36
39
  * best effort and may drift from a future host's, the same caveat the
37
40
  * hook-view threading fallback already carries ("shapes may differ").
38
41
  */
42
+ /** The built-ins {@link runBuiltin} actually implements. */
43
+ export declare const EXECUTABLE_BUILTINS: ReadonlySet<string>;
39
44
  /**
40
45
  * The built-ins bir may stand in for or execute when `BIR_BUILTIN_TOOLS` is
41
- * unset: the read-only set. Reading a file the session itself would read
42
- * without a prompt is the risk floor; anything that writes or runs is not.
46
+ * unset: every one it implements, so a chain of built-ins arms `direct` just
47
+ * as a chain of wrapped MCP calls does. Up to 0.3.7 this was the read-only
48
+ * set (`Read,Glob,Grep`).
43
49
  */
44
50
  export declare const DEFAULT_BUILTIN_TOOLS: readonly string[];
45
- /** The built-ins {@link runBuiltin} actually implements. */
46
- export declare const EXECUTABLE_BUILTINS: ReadonlySet<string>;
47
51
  /**
48
52
  * Parse `BIR_BUILTIN_TOOLS`. Unset means {@link DEFAULT_BUILTIN_TOOLS}; set,
49
53
  * it is the whole list — `none` (or an empty value) turns built-ins off
@@ -54,10 +58,11 @@ export declare const EXECUTABLE_BUILTINS: ReadonlySet<string>;
54
58
  export declare function parseBuiltinTools(value: string | undefined): ReadonlySet<string>;
55
59
  /**
56
60
  * Parse `BIR_BASH_ALLOW`: comma-separated command prefixes, `*` for
57
- * everything. Undefined (unset or blank) means `Bash` never executes on the
58
- * runner — the flag is the opt-in, exactly like the tool list itself.
61
+ * everything. Unset means everything, because `Bash` is on the default list.
62
+ * Set, it is the whole list, as `BIR_BUILTIN_TOOLS` is: `none` (or an empty
63
+ * value) allows no command.
59
64
  */
60
- export declare function parseBashAllow(value: string | undefined): readonly string[] | undefined;
65
+ export declare function parseBashAllow(value: string | undefined): readonly string[];
61
66
  /**
62
67
  * Whether `command` matches the `BIR_BASH_ALLOW` prefixes. A prefix that ends
63
68
  * mid-word must end the command's word too: `git` allows `git status` and
@@ -92,7 +97,7 @@ export interface BuiltinContext {
92
97
  cwd: string;
93
98
  /** The step's clock, the same budget an MCP dispatch would get. */
94
99
  timeoutMs?: number;
95
- /** Parsed `BIR_BASH_ALLOW` ({@link parseBashAllow}). */
100
+ /** Parsed `BIR_BASH_ALLOW` ({@link parseBashAllow}). Absent or empty, no command runs. */
96
101
  bashAllow?: readonly string[];
97
102
  /**
98
103
  * Where this plan's *previous* `Bash` step left its shell — the host's own
@@ -10,18 +10,21 @@
10
10
  *
11
11
  * THE RULES, because this bypasses the host's permission prompts entirely:
12
12
  *
13
- * - **Default deny beyond read-only.** Which built-ins may run at all is
14
- * `BIR_BUILTIN_TOOLS` ({@link parseBuiltinTools}), and its default is the
15
- * read-only set (`Read,Glob,Grep`). `Edit`, `Write` and `Bash` are an
16
- * explicit, per-environment opt-in — a recorded `rm` must never run
17
- * because a prompt looked similar.
13
+ * - **All six by default, each a switch.** Which built-ins may run at all is
14
+ * `BIR_BUILTIN_TOOLS` ({@link parseBuiltinTools}); unset, it is every one
15
+ * this module implements (up to 0.3.7 it was the read-only set,
16
+ * `Read,Glob,Grep`). The permission story is the one a steered step
17
+ * already has — its pinned call is answered `allow` too. What running it
18
+ * here changes is that no model sees the call before it runs, so an
19
+ * operator who wants one in the loop takes the tool off the list.
18
20
  * - **Path jail.** Every path is resolved against, and confined to, the
19
21
  * session's project directory ({@link jailed}), symlinks included. The
20
22
  * daemon's own cwd is never used: it is `~/.baseinstrunner`, not the
21
23
  * project (§5.1 of the change plan).
22
- * - **Bash runs nothing unless `BIR_BASH_ALLOW` says so.** A command must
23
- * match one of its prefixes ({@link bashAllowed}); unset means `Bash`
24
- * never executes here, even when allow-listed.
24
+ * - **Bash runs any command unless `BIR_BASH_ALLOW` narrows it.** Set, a
25
+ * command must match one of its prefixes ({@link bashAllowed}); `none`
26
+ * allows no command, and then `Bash` counts as off the list, so its steps
27
+ * are steered in the session (controller.ts).
25
28
  * - **Same failure contract as an MCP step.** A step that *cannot* run here
26
29
  * throws {@link BuiltinNotRunnable} (→ `not_run`: the session can still do
27
30
  * it, or a recorded output may stand in under
@@ -42,12 +45,6 @@ import { existsSync } from "node:fs";
42
45
  import { readdir, readFile, realpath, stat, writeFile } from "node:fs/promises";
43
46
  import { dirname, join, resolve, sep } from "node:path";
44
47
  // ── configuration ────────────────────────────────────────────────────────────
45
- /**
46
- * The built-ins bir may stand in for or execute when `BIR_BUILTIN_TOOLS` is
47
- * unset: the read-only set. Reading a file the session itself would read
48
- * without a prompt is the risk floor; anything that writes or runs is not.
49
- */
50
- export const DEFAULT_BUILTIN_TOOLS = ["Read", "Glob", "Grep"];
51
48
  /** The built-ins {@link runBuiltin} actually implements. */
52
49
  export const EXECUTABLE_BUILTINS = new Set([
53
50
  "Read",
@@ -57,6 +54,13 @@ export const EXECUTABLE_BUILTINS = new Set([
57
54
  "Write",
58
55
  "Bash",
59
56
  ]);
57
+ /**
58
+ * The built-ins bir may stand in for or execute when `BIR_BUILTIN_TOOLS` is
59
+ * unset: every one it implements, so a chain of built-ins arms `direct` just
60
+ * as a chain of wrapped MCP calls does. Up to 0.3.7 this was the read-only
61
+ * set (`Read,Glob,Grep`).
62
+ */
63
+ export const DEFAULT_BUILTIN_TOOLS = [...EXECUTABLE_BUILTINS];
60
64
  /** `read` → `Read`: the vocabulary is small enough to forgive casing. */
61
65
  const CANONICAL_BUILTIN = new Map([...EXECUTABLE_BUILTINS].map((name) => [name.toLowerCase(), name]));
62
66
  /**
@@ -78,15 +82,17 @@ export function parseBuiltinTools(value) {
78
82
  }
79
83
  /**
80
84
  * Parse `BIR_BASH_ALLOW`: comma-separated command prefixes, `*` for
81
- * everything. Undefined (unset or blank) means `Bash` never executes on the
82
- * runner — the flag is the opt-in, exactly like the tool list itself.
85
+ * everything. Unset means everything, because `Bash` is on the default list.
86
+ * Set, it is the whole list, as `BIR_BUILTIN_TOOLS` is: `none` (or an empty
87
+ * value) allows no command.
83
88
  */
84
89
  export function parseBashAllow(value) {
85
- const prefixes = (value ?? "")
90
+ if (value === undefined)
91
+ return ["*"];
92
+ return value
86
93
  .split(",")
87
94
  .map((s) => s.trim())
88
- .filter(Boolean);
89
- return prefixes.length > 0 ? prefixes : undefined;
95
+ .filter((s) => s.length > 0 && !["none", "off", "0"].includes(s.toLowerCase()));
90
96
  }
91
97
  /**
92
98
  * Whether `command` matches the `BIR_BASH_ALLOW` prefixes. A prefix that ends
@@ -485,13 +491,24 @@ async function runBash(input, ctx) {
485
491
  if (typeof command !== "string" || command.trim() === "") {
486
492
  throw new BuiltinNotRunnable("command is required");
487
493
  }
488
- if (!ctx.bashAllow) {
489
- throw new BuiltinNotRunnable("BIR_BASH_ALLOW is not set, so bir runs no Bash command unattended");
494
+ if (!ctx.bashAllow?.length) {
495
+ throw new BuiltinNotRunnable("BIR_BASH_ALLOW allows no command, so bir runs no Bash command unattended");
490
496
  }
491
497
  if (!bashAllowed(command, ctx.bashAllow)) {
492
498
  throw new BuiltinNotRunnable("the command does not match BIR_BASH_ALLOW, so bir does not run it unattended");
493
499
  }
500
+ // What the host would run differently stays the session's, never started
501
+ // here: a command it leaves in the background (there is no shell here to
502
+ // leave it in, so it would be stopped with the step), and one that asks for
503
+ // more time than this step has. Cut off part-way, an `npm i -g` or a
504
+ // migration can leave things half done, which is worse than a hand-over.
505
+ if (input.run_in_background === true) {
506
+ throw new BuiltinNotRunnable("the command runs in the background, which only the session can do");
507
+ }
494
508
  const timeoutMs = ctx.timeoutMs ?? 120_000;
509
+ if (typeof input.timeout === "number" && input.timeout > timeoutMs) {
510
+ throw new BuiltinNotRunnable(`the command asks for up to ${Math.ceil(input.timeout / 1000)} s and this step has ${Math.floor(timeoutMs / 1000)} s, so the session runs it`);
511
+ }
495
512
  // The host's Bash semantics, which the recorded chain was written against:
496
513
  // the working directory carries over from the plan's previous Bash step,
497
514
  // variables and functions do not (a fresh `-l` shell each call).
@@ -180,15 +180,6 @@ export declare const RUN_NO_PLAN = "no_plan";
180
180
  export interface ReplayOptions {
181
181
  /** On unless `BIR_REPLAY=0`. Everything here is inert when false. */
182
182
  enabled: boolean;
183
- /** Minimum similarity to steer — deliberately above the service's detection threshold. */
184
- minSimilarity: number;
185
- /**
186
- * Minimum similarity to steer a **segment** (`BIR_MIN_SEGMENT_STEER_SIMILARITY`,
187
- * segmented.md R-OUT-11). The same number as the server's clear band, so a
188
- * clear hand-out is always accepted and no score falls between "clear" and
189
- * "accepted".
190
- */
191
- minSegmentSimilarity?: number;
192
183
  /**
193
184
  * `BIR_SEGMENT_ARM`. Off means observe-only: the runner asks for no segment
194
185
  * and arms none, and only logs what would have armed (R-OUT-10).
@@ -200,17 +191,17 @@ export interface ReplayOptions {
200
191
  * `BIR_BUILTIN_TOOLS` — the built-ins the runner may execute itself, from
201
192
  * the session's own project directory (replay/builtins.ts), or stand in
202
193
  * for with a recorded output under `BIR_REPLAY_RECORDED_OUTPUTS=1`.
203
- * Omitted, the environment decides; the default is the read-only set
204
- * (`Read,Glob,Grep`). `Edit`, `Write` and `Bash` run only when listed
205
- * explicitly — and `Bash` additionally only for commands matching
206
- * {@link ReplayOptions.bashAllow}.
194
+ * Omitted, the environment decides; the default is all six (`Read`,
195
+ * `Glob`, `Grep`, `Edit`, `Write`, `Bash`), and `Bash` runs only the
196
+ * commands {@link ReplayOptions.bashAllow} allows.
207
197
  */
208
198
  builtinTools?: ReadonlySet<string>;
209
199
  /**
210
200
  * `BIR_BASH_ALLOW` — command prefixes a runner-side `Bash` step may run
211
- * (`*` for all). Undefined means `Bash` never executes here, even when
212
- * allow-listed: runner-side execution bypasses the host's permission
213
- * prompts, so the shell needs its own explicit opt-in.
201
+ * (`*` for all). Omitted, the environment decides; the default is `*`.
202
+ * Empty allows no command, and then `Bash` counts as off
203
+ * {@link ReplayOptions.builtinTools}: its steps are steered in the session,
204
+ * rather than armed `direct` and stopped at the first one.
214
205
  */
215
206
  bashAllow?: readonly string[];
216
207
  budgets?: Partial<ReplayBudgets>;
@@ -439,7 +430,7 @@ export interface ReplayState {
439
430
  * Why a match did not arm. One code per gate of the ladder, so the caller can
440
431
  * act on a decline without parsing the sentence that explains it.
441
432
  */
442
- export type DeclineCode = "replay_disabled" | "not_ready" | "flatten_failed" | "similarity" | "known_bad_first_step" | "nondeterministic_first_step" | "unusable_first_step" | "coverage" | "missing_target" | "no_derive_key";
433
+ export type DeclineCode = "replay_disabled" | "not_ready" | "flatten_failed" | "known_bad_first_step" | "nondeterministic_first_step" | "unusable_first_step" | "coverage" | "missing_target" | "no_derive_key";
443
434
  /** What `PreToolUse` should do about this call. */
444
435
  export type PreToolAction =
445
436
  /** Pin the arguments and let the real tool run. */
@@ -484,7 +475,7 @@ export declare class ReplayController {
484
475
  /** See {@link ReplayOptions.builtinTools}. */
485
476
  private readonly builtinTools;
486
477
  /** See {@link ReplayOptions.bashAllow}. */
487
- private readonly bashAllow?;
478
+ private readonly bashAllow;
488
479
  /** Calls waiting for a plan's pins to change (see `awaitPins`). */
489
480
  private readonly pinWaiters;
490
481
  /**
@@ -250,8 +250,14 @@ export class ReplayController {
250
250
  this.intentMatch = { ...DEFAULT_INTENT_MATCH, ...(opts.intentMatch ?? {}) };
251
251
  this.recordedFallback =
252
252
  opts.recordedFallback ?? process.env.BIR_REPLAY_RECORDED_OUTPUTS === "1";
253
- this.builtinTools = opts.builtinTools ?? parseBuiltinTools(process.env.BIR_BUILTIN_TOOLS);
254
253
  this.bashAllow = opts.bashAllow ?? parseBashAllow(process.env.BIR_BASH_ALLOW);
254
+ const builtins = opts.builtinTools ?? parseBuiltinTools(process.env.BIR_BUILTIN_TOOLS);
255
+ // A `Bash` that may run no command is not reachable here. Left on the
256
+ // list, a chain of Bash steps would arm `direct` and stop at its first.
257
+ this.builtinTools =
258
+ this.bashAllow.length > 0 || !builtins.has("Bash")
259
+ ? builtins
260
+ : new Set([...builtins].filter((name) => name !== "Bash"));
255
261
  }
256
262
  /**
257
263
  * Whether a handed-out segment may actually arm (segmented.md R-OUT-10).
@@ -333,14 +339,7 @@ export class ReplayController {
333
339
  verified: match.verified,
334
340
  sharedWith: [],
335
341
  };
336
- // A segment is judged against its own gate, and a `same` verdict stands in
337
- // for the score (segmented.md R-OUT-11): the question was asked *because*
338
- // the score sat below the threshold, so a numeric gate would decline every
339
- // verified hand-out.
340
342
  const isSegment = state.kind === "segment";
341
- const threshold = isSegment
342
- ? (this.opts.minSegmentSimilarity ?? 0.95)
343
- : this.opts.minSimilarity;
344
343
  const decline = (why, code) => {
345
344
  state.declined = code;
346
345
  logLine("replay.decision", {
@@ -349,7 +348,6 @@ export class ReplayController {
349
348
  run: match.runId,
350
349
  scenario: match.scenarioId ?? undefined,
351
350
  similarity: match.similarity.toFixed(3),
352
- threshold,
353
351
  kind: state.kind,
354
352
  key: state.key,
355
353
  verified: state.verified?.verdict,
@@ -371,18 +369,14 @@ export class ReplayController {
371
369
  if (!match.scenarioId || !isReadyScenario(scenario)) {
372
370
  return decline(match.scenarioId ? "scenario is not ready" : "no scenario for the matched run", "not_ready");
373
371
  }
374
- // Gate 4 — similarity. Detection decides "don't record this again"; steering
375
- // decides "don't think about this again", which is a much stronger claim and
376
- // deserves a stronger threshold.
377
- //
378
- // For a segment, a `same` verdict from the live question stands in for the
379
- // score (R-OUT-11). Only `same` ever reaches a runner with a ticket, so it
380
- // is the only verdict this sees.
381
- const scoreOk = match.similarity >= threshold;
382
- const verifiedOk = isSegment && match.verified?.verdict === "same";
383
- if (!scoreOk && !verifiedOk) {
384
- return decline("similarity below threshold", "similarity");
385
- }
372
+ // Gate 4 — similarity — is gone. A hand-out is the service's decision that
373
+ // this scenario does what the prompt asks: its match threshold, the
374
+ // same-request question under it, the intent pass, a segment's clear band
375
+ // or `same` verdict. Scoring it again here declined matches the service
376
+ // had made (a checked 0.85, a task line at 0.91), so every match now
377
+ // steers. What a plan cannot do — a step that keeps failing, a tool
378
+ // nobody wrapped, a target the prompt never named — the gates below
379
+ // still stop.
386
380
  // Flatten the chain before the remaining gates, so every one of them judges
387
381
  // the steps that will *really* run — the caller's own and, in order, those
388
382
  // of every segment it calls (segmented.md R-CALL-29, R-CALL-30).
@@ -2079,10 +2073,10 @@ export class ReplayController {
2079
2073
  // An allow-listed built-in runs *here*, against the session's own
2080
2074
  // project directory (replay/builtins.ts). Everything else about it —
2081
2075
  // no session cwd (`bir replay`, an older server), off the list, no
2082
- // runner-side implementation, a path that leaves the project, `Bash`
2083
- // without `BIR_BASH_ALLOW` — stays the old contract: `not_run`, so
2084
- // the session can do the step itself or its recorded output can
2085
- // stand in (`BIR_REPLAY_RECORDED_OUTPUTS=1`).
2076
+ // runner-side implementation, a path that leaves the project, a `Bash`
2077
+ // command `BIR_BASH_ALLOW` does not allow — stays the old contract:
2078
+ // `not_run`, so the session can do the step itself or its recorded
2079
+ // output can stand in (`BIR_REPLAY_RECORDED_OUTPUTS=1`).
2086
2080
  if (this.builtinTools.has(name) && state.sessionCwd) {
2087
2081
  return this.executeBuiltin(name, input, state, opts.deadline);
2088
2082
  }
@@ -8,14 +8,15 @@
8
8
  * only happen inside the live session.
9
9
  *
10
10
  * `live` covers three genuinely different cases and treats them identically,
11
- * correctly: a built-in (`Bash`, `Read`, `Edit`, …) outside the built-in
12
- * allow-list, an MCP server the user chose not to wrap, and `claude-in-chrome`
13
- * — which is `scope: "dynamic"`, appears in no config file, and therefore can
14
- * never be wrapped at all.
11
+ * correctly: a built-in outside the built-in allow-list (`WebFetch`, `Task`,
12
+ * or one the operator took off it), an MCP server the user chose not to wrap,
13
+ * and `claude-in-chrome` — which is `scope: "dynamic"`, appears in no config
14
+ * file, and therefore can never be wrapped at all.
15
15
  *
16
- * A built-in *on* the allow-list (`BIR_BUILTIN_TOOLS`, default `Read,Glob,
17
- * Grep`) is `direct`: the runner executes it itself (replay/builtins.ts), or —
18
- * under `BIR_REPLAY_RECORDED_OUTPUTS=1` — serves its recorded output. The
16
+ * A built-in *on* the allow-list (`BIR_BUILTIN_TOOLS`, by default all six the
17
+ * runner implements) is `direct`: the runner executes it itself
18
+ * (replay/builtins.ts), or — under `BIR_REPLAY_RECORDED_OUTPUTS=1` — serves
19
+ * its recorded output. The
19
20
  * callers that ask a *different* question, "will a proxy report this call"
20
21
  * (`ReplayController.reachFor`), deliberately pass no allow-list: a built-in
21
22
  * is never proxied, whoever executes it.
@@ -8,14 +8,15 @@
8
8
  * only happen inside the live session.
9
9
  *
10
10
  * `live` covers three genuinely different cases and treats them identically,
11
- * correctly: a built-in (`Bash`, `Read`, `Edit`, …) outside the built-in
12
- * allow-list, an MCP server the user chose not to wrap, and `claude-in-chrome`
13
- * — which is `scope: "dynamic"`, appears in no config file, and therefore can
14
- * never be wrapped at all.
11
+ * correctly: a built-in outside the built-in allow-list (`WebFetch`, `Task`,
12
+ * or one the operator took off it), an MCP server the user chose not to wrap,
13
+ * and `claude-in-chrome` — which is `scope: "dynamic"`, appears in no config
14
+ * file, and therefore can never be wrapped at all.
15
15
  *
16
- * A built-in *on* the allow-list (`BIR_BUILTIN_TOOLS`, default `Read,Glob,
17
- * Grep`) is `direct`: the runner executes it itself (replay/builtins.ts), or —
18
- * under `BIR_REPLAY_RECORDED_OUTPUTS=1` — serves its recorded output. The
16
+ * A built-in *on* the allow-list (`BIR_BUILTIN_TOOLS`, by default all six the
17
+ * runner implements) is `direct`: the runner executes it itself
18
+ * (replay/builtins.ts), or — under `BIR_REPLAY_RECORDED_OUTPUTS=1` — serves
19
+ * its recorded output. The
19
20
  * callers that ask a *different* question, "will a proxy report this call"
20
21
  * (`ReplayController.reachFor`), deliberately pass no allow-list: a built-in
21
22
  * is never proxied, whoever executes it.
@@ -45,7 +45,7 @@ this design has a ladder whose cheapest rung is free — and the more of a stack
45
45
  |---|---|---|
46
46
  | Runs MCP tools itself | No (`replay.md` §12: *"rrepeat never executes tools — not even MCP"*) | **Yes**, on the connection it already owns |
47
47
  | Divergence fallback | A second Agent SDK session — the dominant cost | Direct `tools/call` through the proxies — **$0** |
48
- | Reaches built-ins (`Bash`, `Read`) | Only by steering the live session | Read-only ones (`Read`, `Glob`, `Grep`) run on the runner itself, jailed to the project (`BIR_BUILTIN_TOOLS`); `Edit`/`Write`/`Bash` opt-in, else steered |
48
+ | Reaches built-ins (`Bash`, `Read`) | Only by steering the live session | `Read`, `Glob`, `Grep`, `Edit`, `Write` and `Bash` run on the runner itself, jailed to the project (`BIR_BUILTIN_TOOLS`, `BIR_BASH_ALLOW`); any other built-in is steered |
49
49
  | Needs `@anthropic-ai/*` at runtime | Yes | No (§16.3) |
50
50
 
51
51
  ---
@@ -193,7 +193,7 @@ Every gate that declines logs **one** `replay.decision` line carrying the reason
193
193
  | 1 | **A match arrived** | `getMatch()` resolved `null`, or the budget expired (§10) | (no report — nothing matched) |
194
194
  | 2 | **Replay enabled** | `BIR_REPLAY === "0"`, or, with it unset, the project's stored switch is off (`bir replay off`, or a `bir install` without `--replay`) | `not_steered` |
195
195
  | 3 | **Scenario ready** | `scenario == null`, `state !== "ready"`, or `steps.length === 0` | `not_steered` |
196
- | 4 | **Similarity** | `similarity < BIR_MIN_STEER_SIMILARITY` (default **0.92**) | `not_steered` |
196
+ | 4 | ~~**Similarity**~~ | Retired: a match is the decision to use the scenario (below) | — |
197
197
  | 5 | **Tool coverage** | No step is executable anywhere (§5 → `mode: "none"`) | `not_steered` |
198
198
 
199
199
  Past the table, a plan can still be refused before the model hears of it, each refusal with its own
@@ -205,12 +205,16 @@ after the prompt was answered is retired unannounced (`match_late`, `arm_unannou
205
205
  plan is declined when no running `bir-scenario` has said it serves `run_scenario`
206
206
  (`no_delivery_tool`, §6.3). In plan mode (`permission_mode: "plan"`) nothing is announced at all.
207
207
 
208
- Gate 4 deserves its own note. The server's own detection threshold is 0.9, and it uses it to decide
209
- *"do not record this again"* — a cheap claim: the worst case is a lost recording. Steering makes a
210
- much stronger claim: *"do not think about this again."* A prompt that is 0.90-similar usually covers
211
- only part of the live task, and steering it produces a confident, thin, wrong answer. So the steer
212
- threshold sits **above** the detection threshold by default, and `BIR_MIN_STEER_SIMILARITY` is the
213
- single knob an operator turns after watching `replay.decision` lines for a week.
208
+ Gate 4 is retired. Up to 0.3.6 it declined a match under `BIR_MIN_STEER_SIMILARITY` (0.92),
209
+ above the server's 0.9, on the argument that the server only decided *"do not record this again"*
210
+ while steering claims *"do not think about this again."* The server's match stopped being detection
211
+ alone: it hands a scenario over only when it may run, with a ticket, and it decides the match with
212
+ more than one score — a same-request question under its threshold, and a second pass on the
213
+ prompt's task line. Re-scoring that on this side declined matches the server had made (a checked
214
+ 0.85, a task line at 0.91). **A match is the decision to use the scenario**, so every hand-out
215
+ steers; what a plan cannot do — a parked first step, a tool nothing reaches, a target the prompt
216
+ never named — the gates after it still stop. `BIR_MIN_STEER_SIMILARITY` and
217
+ `BIR_MIN_SEGMENT_STEER_SIMILARITY` are ignored.
214
218
 
215
219
  `not_steered` as the floor is not a failure code. It is the **control group**: the prompt matched,
216
220
  the agent ran the task the ordinary way, and the cost of doing so is a fresh measurement of what this
@@ -248,21 +252,26 @@ export function modeFor(steps: SerializedScenarioStep[], wrapped: ReadonlySet<st
248
252
  | — | `none` | nowhere | declines at gate 5 |
249
253
 
250
254
  `live` covers three genuinely different cases and treats them identically, correctly: a built-in
251
- off the `BIR_BUILTIN_TOOLS` allow-list (by default everything but `Read`, `Glob`, `Grep` — so
252
- `Bash`, `Edit`, `Write`, `Task`, `WebFetch`), an MCP server the user chose not to wrap, and
255
+ off the `BIR_BUILTIN_TOOLS` allow-list (by default everything but the six the runner implements —
256
+ so `Task`, `WebFetch`; up to 0.3.7 also `Bash`, `Edit`, `Write`), an MCP server the user chose not to wrap, and
253
257
  `claude-in-chrome` — which is `scope: "dynamic"`, appears in no config file, and therefore can never
254
258
  be wrapped (v1 design §1). None of them can be reached except through the session.
255
259
 
256
260
  A built-in **on** the allow-list runs on the runner, in the session's own project directory and
257
261
  confined to it — a path that resolves outside the project, symlinks included, is refused and the
258
- step is `not_run`, the clean hand-over. `Bash` needs a second opt-in on top (`BIR_BASH_ALLOW`,
259
- command prefixes), because a runner-side step never reaches the host's permission prompts. The
262
+ step is `not_run`, the clean hand-over. `Bash` has a second switch on top (`BIR_BASH_ALLOW`,
263
+ command prefixes; unset means any command, `none` keeps `Bash` in the session), because a
264
+ runner-side step never reaches the host's permission prompts. The
260
265
  results mimic the hook's view of each tool, which is what the recorder stored for built-in steps
261
266
  and what their `toolOutputLogic` was authored against; they pass through the same
262
267
  `serializeCapped(redact(…))` as every recorded payload. Bash keeps the host's own semantics a
263
268
  recorded chain was written against: each call is a fresh `-l` shell (variables and functions do
264
269
  not persist), while the **working directory carries over** from the plan's previous Bash step —
265
- `cd sub` then `cat file` replays as it ran.
270
+ `cd sub` then `cat file` replays as it ran. Two Bash calls the host runs differently are never
271
+ started on the runner, and are `not_run` instead: one with `run_in_background` (there is no shell
272
+ here to leave it in), and one whose own `timeout` is longer than the step's clock
273
+ (`BIR_STEP_TIMEOUT_MS`, clipped to what the plan has left) — cut off part-way, an `npm i -g` or a
274
+ migration can leave things half done.
266
275
 
267
276
  A **mixed** scenario in `steer` mode still gets direct execution for its wrapped steps once it
268
277
  diverges (§8), so coverage is not all-or-nothing at the divergence boundary — only at the arm-time
@@ -1486,7 +1495,7 @@ plans when it is replayed. The operator's walk-through is
1486
1495
  source run's *recorded* outputs — no real tools, no side effects, one Haiku call. Without `--dry`,
1487
1496
  `bir replay` runs the same plan through the same executor against the live proxies. `bir doctor`
1488
1497
  gains a `replay` block: on/off, the effective `BIR_REPLAY_ALLOW_SERVERS` allowlist,
1489
- `BIR_MIN_STEER_SIMILARITY`, and who derives — this machine, the service, or nobody, which is the
1498
+ and who derives — this machine, the service, or nobody, which is the
1490
1499
  state in which a scenario with a target never runs (§13.2, mitigation 3).
1491
1500
 
1492
1501
  ---
@@ -1627,7 +1636,7 @@ changes the corpus, the threshold and every stored vector. §20 question 1.
1627
1636
  and argument objects the bodies rely on today.
1628
1637
  4. **Should a declined match still arm a *dry* plan?** Deriving parameters and computing the first
1629
1638
  step's input without executing anything would make `bir doctor` able to say *"this scenario would
1630
- have run these three calls"* — excellent for tuning `BIR_MIN_STEER_SIMILARITY`, at the price of a
1639
+ have run these three calls"* — excellent for fixing the plan behind a decline, at the price of a
1631
1640
  Haiku call on turns that replay nothing.
1632
1641
  5. **Should `scenario_steps` carry `recorded_output`?** §8.1 works around its absence with one lazy
1633
1642
  `GET /recordings/runs/:runId`. A column would make it a field read, would let a scenario outlive
@@ -234,7 +234,10 @@ When replay arms:
234
234
  when it plausibly *is* the step — the same command, the same file, an argument in common (design
235
235
  §7) — and never in plan mode.
236
236
  - **Direct steps are not visible to the permission system at all.** They are dispatched straight to a
237
- `bir-proxy`, which calls the upstream on a connection it already holds. Claude Code never sees them.
237
+ `bir-proxy`, which calls the upstream on a connection it already holds — or, for Claude Code's own
238
+ `Read`, `Glob`, `Grep`, `Edit`, `Write` and `Bash`, run by the recorder itself in the project
239
+ folder, any `Bash` command included (`BIR_BUILTIN_TOOLS`, `BIR_BASH_ALLOW`; up to 0.3.7 only the
240
+ first three). Claude Code never sees them.
238
241
  The `run_scenario` call that starts them is answered `allow` too, while the runner has announced a
239
242
  direct plan; your own deny or ask rules for it still apply.
240
243
 
@@ -243,7 +246,9 @@ That is what makes replay fast, and it is the whole risk. Two things follow:
243
246
  1. Only enable it in a project whose scenarios you have dry-replayed and read (§3.1).
244
247
  2. Run `bir replay allow <servers>` (§5.3). An allowlist of the servers you are comfortable having
245
248
  called unattended is cheap insurance, and it is the difference between "replay reads a dashboard"
246
- and "replay can do anything any wrapped server can do".
249
+ and "replay can do anything any wrapped server can do". The same goes for built-ins:
250
+ `BIR_BUILTIN_TOOLS=Read,Glob,Grep` keeps writes and commands with the model, and
251
+ `BIR_BASH_ALLOW=git,npm test` (for example) limits the commands the recorder runs itself.
247
252
 
248
253
  Nothing else is approved on your behalf. Outside a matched scenario's own steps, a wrapped MCP call
249
254
  keeps its permission prompt, as it would without the runner (`bir doctor` says so in a note);
@@ -300,7 +305,7 @@ bir doctor
300
305
  ```
301
306
 
302
307
  ```
303
- Replay : ON servers=chrome-devtools,fleet-api minSimilarity=0.92 source=sidecar
308
+ Replay : ON servers=chrome-devtools,fleet-api source=sidecar
304
309
  derive=the service (no key needed here)
305
310
  Wrapped in config : chrome-devtools, fleet-api
306
311
  Registered proxies: chrome-devtools, fleet-api
@@ -324,7 +329,7 @@ uses, with the account it is signed in as. The two normally agree.
324
329
 
325
330
  ### 5.3 Environment
326
331
 
327
- The first three switches are stored per project, and that is where to set them:
332
+ The first two switches are stored per project, and that is where to set them:
328
333
 
329
334
  ```bash
330
335
  bir replay allow chrome-devtools,fleet-api # the servers bir may call by itself; `allow all` clears it
@@ -344,7 +349,6 @@ so there is exactly one switch and it cannot get out of step with itself.
344
349
  |---|---|---|
345
350
  | `BIR_REPLAY` | *(the stored switch)* | `0` disables replay and any other value turns it on, whatever `bir replay on` or `off` or the install stored. Unset, the stored switch decides: on after `bir setup` or `bir install --replay`, off after a plain `bir install`. Nothing below matters while replay is off |
346
351
  | `BIR_REPLAY_ALLOW_SERVERS` | *(the stored list, else all wrapped)* | Comma-separated server keys bir may call by itself: **direct** execution and divergence recovery. Overrides `bir replay allow` |
347
- | `BIR_MIN_STEER_SIMILARITY` | `0.92` | Below this a match is detected but not replayed (§8) |
348
352
  | `BIR_DERIVE_API_KEY` | *(unset)* | **Not required.** Derivation — reading what this request acts on — is done by the service on its key for a signed-in runner. Set this to keep the reading on this machine instead: the prompt then never leaves it, and it is one round trip faster. Signed out *and* unset, only a scenario with nothing to work out replays. A recorder running in the background (started by the SessionStart hook, `bir up` or `bir setup`) ignores `ANTHROPIC_API_KEY` (usually Claude Code's own, and billed) unless `BIR_USE_ANTHROPIC_API_KEY=1`; a `bir-hooks` you run in a terminal uses it |
349
353
  | `BIR_DERIVE_MODEL` | `claude-haiku-4-5-20251001` | The derivation model, when this machine does the reading |
350
354
  | `BIR_MATCH_BUDGET_MS` | `3000` | How long the prompt hook waits for a match before giving up. After a match slower than `BIR_MATCH_SLOW_MS` (`2000`), prompts stop waiting for one for 5 minutes, doubling up to 30 while the service stays slow |
@@ -355,8 +359,8 @@ so there is exactly one switch and it cannot get out of step with itself.
355
359
  | `BIR_STEP_TIMEOUT_MS` | `60000` | Per-step ceiling for one direct `tools/call`, clipped to what the plan has left |
356
360
  | `BIR_LOGIC_TIMEOUT_MS` | `1000` | How long one scenario logic body may run before it is stopped. At most 10 000 |
357
361
  | `BIR_REPLAY_RECORDED_OUTPUTS` | *(unset)* | `1` lets a step that could not be sent (a built-in off `BIR_BUILTIN_TOOLS`, no proxy, a server off the allowlist) stand on its recorded output instead of stopping the plan. Never for a timeout or an upstream error |
358
- | `BIR_BUILTIN_TOOLS` | `Read,Glob,Grep` | Built-ins the runner executes itself during a replay, in the session's own project directory and confined to it. A chain whose only non-MCP steps are on this list arms `direct`. Add `Edit,Write,Bash` only deliberately — a runner-side step never reaches the host's permission prompts. `none` restores the old behaviour: every built-in hands the turn back |
359
- | `BIR_BASH_ALLOW` | *(unset)* | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset, `Bash` never executes on the runner, even when `BIR_BUILTIN_TOOLS` lists it. A plan's Bash steps share a working directory, exactly as the host's own Bash tool does — variables and functions do not persist, also as the host's |
362
+ | `BIR_BUILTIN_TOOLS` | `Read,Glob,Grep,Edit,Write,Bash` | Built-ins the runner executes itself during a replay, in the session's own project directory and confined to it. A chain whose only non-MCP steps are on this list arms `direct`. A runner-side step never reaches the host's permission prompts, so leave off any tool you want the model to call itself; `Read,Glob,Grep` is the read-only set (the default up to 0.3.7). `none`: every built-in step is steered in the session |
363
+ | `BIR_BASH_ALLOW` | *(unset: any command)* | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset means any command; `none` means no command, and `Bash` steps are steered in the session. A plan's Bash steps share a working directory, exactly as the host's own Bash tool does — variables and functions do not persist, also as the host's. A call with `run_in_background`, or with its own `timeout` longer than the step's clock (`BIR_STEP_TIMEOUT_MS`), is never started on the runner: the plan hands over in front of it |
360
364
  | `BIR_INTENT_ARM` | *(unset)* | `1` lets a match found mid-turn arm a plan in the calling tool's place, which means waiting for it (`BIR_INTENT_ARM_BUDGET_MS`, 500); `0` never waits; unset follows `BIR_SEGMENT_ARM` |
361
365
  | `BIR_VERBOSE` | *(unset)* | `1` adds per-step `replay.*` detail lines |
362
366
 
@@ -515,8 +519,8 @@ a turn cut short says nothing about what the task costs.
515
519
 
516
520
  `not_steered` is not an error, and it is not wasted. A saving is `baseline − actual`, and the baseline
517
521
  is *what this task costs when the agent does it*. Every declined match is a fresh measurement of
518
- exactly that. A deployment that never declines has no baseline to measure against — which is a good
519
- reason to leave `BIR_MIN_STEER_SIMILARITY` where it is rather than driving it to zero.
522
+ exactly that. A deployment that seldom declines measures its baseline mostly from the recording
523
+ itself; `bir replay off` for a few runs of the prompt, then `bir replay on`, gives it more.
520
524
 
521
525
  An **unredeemed** ticket books nothing at all. Silence loses a saving; it never invents one.
522
526
 
@@ -524,25 +528,22 @@ An **unredeemed** ticket books nothing at all. Silence loses a saving; it never
524
528
 
525
529
  ## 8. Tuning
526
530
 
527
- Only one knob matters in practice.
531
+ There is no threshold on this side. **Every match steers**: a match is the service's decision that
532
+ the scenario does what the prompt asks, and bir does not score it again. (Up to 0.3.6 it did:
533
+ `BIR_MIN_STEER_SIMILARITY`, default 0.92, declined matches the service had made. It is ignored now.)
528
534
 
529
- **`BIR_MIN_STEER_SIMILARITY` (default 0.92).** The server detects matches at 0.9 and uses that to
530
- decide *"don't record this again"* — cheap: the worst case is a lost recording. Steering makes a much
531
- stronger claim, *"don't think about this again"*, and a 0.90-similar prompt usually covers only part
532
- of the live task. Steering it produces a confident, thin, wrong answer.
535
+ Whether a prompt matches is decided by the service, in three steps. A prompt at 0.9 or more against
536
+ a recording matches (`SIMILARITY_THRESHOLD`). From 0.7 (`SIMILARITY_CHECK_FLOOR`) up to that, a quick
537
+ model check asks whether the two prompts are the same request, values included, and only a yes
538
+ matches. A prompt that missed both is matched once more on its task line (`PROMPT_INTENT_THRESHOLD`,
539
+ 0.9) — the way the same task with a different value finds its scenario.
533
540
 
534
- Tune it from evidence, not taste. Run for a week and read the declines:
535
-
536
- ```bash
537
- grep 'replay.decision' audit.log | grep 'why="similarity below threshold"'
538
- ```
539
-
540
- - Prompts you *would* have wanted replayed, sitting at 0.90–0.92 → lower it, in steps of 0.01.
541
- - Replays that answered the wrong question → raise it, and re-read the scenario's `intent`.
542
-
543
- If a specific pair of prompts *should* match and does not, the fix is usually upstream of the
544
- threshold: the two prompts genuinely mean different things to the embedding model. Re-record with a
545
- first prompt that states the task plainly.
541
+ - A replay that answered the wrong question → switch that recording off in the console (its
542
+ scenario then never runs), or `bir replay off` in the project, and re-read the scenario's
543
+ `intent`. On a service you run yourself, raise `SIMILARITY_THRESHOLD` or `SIMILARITY_CHECK_FLOOR`.
544
+ - A prompt you *would* have wanted replayed that did not match → the fix is usually upstream of any
545
+ threshold: the two prompts genuinely mean different things to the embedding model. Re-record with
546
+ a first prompt that states the task plainly.
546
547
 
547
548
  ---
548
549
 
@@ -557,7 +558,7 @@ first prompt that states the task plainly.
557
558
  | `replay.vetoed … the model's first call was …, not run_scenario` | A direct plan, and the model reached for another tool first — usually because it has no `run_scenario` | As for `no_delivery_tool`. Nothing ran; the model did the task |
558
559
  | `replay.pin_declined` on most steered turns | The model's calls to the expected tool are not the step — another command, another file — so nothing is pinned; if it then moves on, `replay.pin_abandoned` retires the plan | Harmless for those calls, which ran as the model wrote them. Persistent declines on one step usually mean its arguments cannot be shown in the directive (they depend on an earlier step's output, or the command runs to several lines), and the model writes its own |
559
560
  | `why="no step is executable"` | None of the scenario's tools is a wrapped MCP server and none is a built-in reachable in this session | Wrap the servers the scenario uses (`bir install --server …`) and recalculate |
560
- | `plan.armed mode=steer` where you expected `direct` | At least one step is a built-in off `BIR_BUILTIN_TOOLS` (default `Read,Glob,Grep` — so usually `Edit`, `Write` or `Bash`), an unwrapped MCP server, or `claude-in-chrome` | `bir status` shows what is wrapped; the `plan.armed` line's `coverage=` names the `live` step. Widen `BIR_BUILTIN_TOOLS` (and `BIR_BASH_ALLOW`) only deliberately. `claude-in-chrome` is `scope: "dynamic"` and can never be wrapped |
561
+ | `plan.armed mode=steer` where you expected `direct` | At least one step is a built-in off `BIR_BUILTIN_TOOLS` (by default one the runner does not implement, such as `WebFetch` or `Task`; a `Bash` step too when `BIR_BASH_ALLOW=none`), an unwrapped MCP server, or `claude-in-chrome`. Before 0.3.8 the default list was `Read,Glob,Grep`, so `Edit`, `Write` and `Bash` steered too | `bir status` shows what is wrapped; the `plan.armed` line's `coverage=` names the `live` step. `claude-in-chrome` is `scope: "dynamic"` and can never be wrapped |
561
562
  | Model ignores the directive and diverges every turn | Steering is advisory — the model chooses; `bir` only pins arguments | Expected occasionally. Persistent divergence usually means the scenario's tools do not fit the live task; check `intent` |
562
563
  | A `replay.thread` line with no `emitted=` (under `BIR_VERBOSE=1`) | `toolOutputLogic` derived nothing — usually the output shape changed since recording | Dry-replay (§3.1). If the dry run emits and the live run does not, the tool's output shape has drifted; recalculate |
563
564
  | `replay.thread_fallback` on a wrapped step | The proxy did not report within the grace window | Check that proxy is alive (`bir doctor`). Repeats mean a slow or dying upstream |
@@ -571,8 +571,8 @@ Fleet switches on `bir setup`: `--allow-servers a,b` stores the servers replay
571
571
  may call unattended for this project; `bir replay allow|on|off|status` changes
572
572
  it later. Both live in the project record (§4.3), so a recorder the hook starts
573
573
  — with Claude Code's environment, not the operator's terminal — keeps them;
574
- `BIR_REPLAY`, `BIR_REPLAY_ALLOW_SERVERS` and `BIR_MIN_STEER_SIMILARITY` in the
575
- recorder's environment still override, and `bir doctor` says which source is in
574
+ `BIR_REPLAY` and `BIR_REPLAY_ALLOW_SERVERS` in the recorder's environment still
575
+ override, and `bir doctor` says which source is in
576
576
  effect. `--no-replay` adds no scenario server (one an earlier setup added stays
577
577
  until `bir uninstall --replay`) and stores replay off (`bir replay on` turns it
578
578
  on); `--no-daemon` wires everything and starts
package/docs/t-bench.md CHANGED
@@ -357,9 +357,8 @@ Per arm, per domain: reward (mean), pass^k across trials, `$`/episode and `$`/tu
357
357
  `--output-format json`'s `total_cost_usd`, wall-clock per episode, and turn count.
358
358
 
359
359
  Per replayed turn, from the audit log and the execution ledger: outcome
360
- (`steered_full` / `diverged` / `fell_back` / `not_steered` / `failed`), match similarity against
361
- `BIR_MIN_STEER_SIMILARITY` (default 0.92), direct vs steered mode, `deriveCostUsd` against
362
- `sessionCostUsd`, and the per-step verdicts.
360
+ (`steered_full` / `diverged` / `fell_back` / `not_steered` / `failed`), match similarity, direct
361
+ vs steered mode, `deriveCostUsd` against `sessionCostUsd`, and the per-step verdicts.
363
362
 
364
363
  Two τ-specific splits worth reporting that MCPMark cannot give us:
365
364
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@basein/runner",
3
- "version": "0.3.6",
3
+ "version": "0.3.8",
4
4
  "description": "A recording MCP proxy: sits between any MCP client and its MCP servers, executes each call on the client's behalf, and records the run as a reusable BaseIn scenario.",
5
5
  "type": "module",
6
6
  "license": "MIT",
package/production.md CHANGED
@@ -847,7 +847,7 @@ Truncation is always marked with the original size.
847
847
  | `matchGate` | `{holding, holdMs, until}`: whether prompts have stopped waiting on the match because the service is slow |
848
848
  | `correlation` | `auto`, `allow` or `ask`, or `fingerprint` when correlation is off: who approves wrapped calls |
849
849
  | `deliveryTool`, `scenarioServers[]` | whether a live `bir-scenario` has said hello (`{pid, tools, off, version, ageMs}`). An empty list means no direct plan is announced. |
850
- | `replay` | `{enabled, source, minSimilarity, allowServers, deriveKey, deriveVia, pollingProxies}`. `deriveVia` is `service`, `key` or `samples`; `samples` means no scenario with a target can run. |
850
+ | `replay` | `{enabled, source, allowServers, deriveKey, deriveVia, pollingProxies}`. `deriveVia` is `service`, `key` or `samples`; `samples` means no scenario with a target can run. |
851
851
  | `registeredProxies[]`, `wrappedServers` | proxies that registered (`serverName`, `pid`, `cwd`, `version`) and the wrapped server names |
852
852
  | `hooksVersion`, `pid`, `cwd`, `url` | the recorder's version and identity |
853
853
  | `segmentArm` | whether service-found sub-tasks may run mid-task (default: observe only) |
@@ -981,7 +981,6 @@ Defaults were checked in `src/`.
981
981
  | `BIR_DERIVE_MODEL` | recorder | `claude-haiku-4-5-20251001` | Model for derivation on this machine. |
982
982
  | `BIR_REPLAY` | recorder | on, or the project's stored switch | `0` turns replay off. Any other value forces it on over the stored switch. |
983
983
  | `BIR_REPLAY_ALLOW_SERVERS` | recorder | all wrapped servers | Servers that direct runs and divergence recovery may call unattended (comma-separated). |
984
- | `BIR_MIN_STEER_SIMILARITY` | recorder | 0.92 | Minimum match similarity to replay. |
985
984
  | `BIR_MATCH_BUDGET_MS` | recorder | 3000 | Longest a prompt waits for the match. |
986
985
  | `BIR_MATCH_SLOW_MS` | recorder | 2000 | A match slower than this, or one that timed out, stops prompts waiting for 5 min, doubling to 30 min. |
987
986
  | `BIR_DERIVE_BUDGET_MS` | recorder | 8000 | Derivation wait for `/scenario/run`. Hooks use the smaller of this and `BIR_HOOK_DERIVE_BUDGET_MS`. |
@@ -992,8 +991,8 @@ Defaults were checked in `src/`.
992
991
  | `BIR_STEP_TIMEOUT_MS` | recorder | 60000 | One direct `tools/call`. |
993
992
  | `BIR_LOGIC_TIMEOUT_MS` | recorder | 1000 (capped at 10000) | Synchronous time limit for one scenario logic body. |
994
993
  | `BIR_REPLAY_RECORDED_OUTPUTS` | recorder | unset | `1`: a step that could not be sent may use its recorded output, labelled as such. |
995
- | `BIR_BUILTIN_TOOLS` | recorder | `Read,Glob,Grep` | Built-ins the runner executes itself during a replay, in the session's project directory and confined to it. Add `Edit,Write,Bash` deliberately — runner-side steps bypass the host's permission prompts. `none` turns it off. |
996
- | `BIR_BASH_ALLOW` | recorder | unset | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset, `Bash` never executes on the runner even when listed. A plan's Bash steps share a working directory, as the host's do; variables do not persist, also as the host's. |
994
+ | `BIR_BUILTIN_TOOLS` | recorder | `Read,Glob,Grep,Edit,Write,Bash` | Built-ins the runner executes itself during a replay, in the session's project directory and confined to it (up to 0.3.7 the default was `Read,Glob,Grep`). Runner-side steps bypass the host's permission prompts; leave a tool off to have the model call it. `none` turns it off. |
995
+ | `BIR_BASH_ALLOW` | recorder | unset (any command) | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset means any command; `none` means no command, and `Bash` steps are steered. A plan's Bash steps share a working directory, as the host's do; variables do not persist, also as the host's. |
997
996
  | `BIR_DERIVE_RECENT_RESULTS` | recorder | 5 | How many of this prompt's tool results derivation may read (`0`: none). Never for plans with a command sink. |
998
997
  | `BIR_INTENT_MATCH` | recorder | on | `0` turns mid-turn intent probes off. |
999
998
  | `BIR_INTENT_MATCH_BUDGET_MS` | recorder | 4000 | One probe's own ceiling. It is not a hook wait. |
@@ -1003,7 +1002,6 @@ Defaults were checked in `src/`.
1003
1002
  | `BIR_INTENT_ARM` | recorder | unset (follows `BIR_SEGMENT_ARM`) | `1`: a probe may arm a plan in its own PreToolUse, and that hook waits for it. `0`: never. |
1004
1003
  | `BIR_INTENT_ARM_BUDGET_MS` | recorder | 500 | How long that PreToolUse waits for the probe. |
1005
1004
  | `BIR_SEGMENT_ARM` | recorder | unset (observe only) | `1`: recurring sub-tasks the service found may run mid-task. |
1006
- | `BIR_MIN_SEGMENT_STEER_SIMILARITY` | recorder | 0.95 | Minimum similarity to steer a segment. |
1007
1005
  | `BIR_CONTROL_URL` | proxy, scenario | written into wrapped entries | The project's control server. Together with `BIR_CONTROL_TOKEN` it is the SDK channel and wins over discovery. |
1008
1006
  | `BIR_CONTROL_TOKEN` | proxy, scenario | unset | Bearer token for the SDK channel. |
1009
1007
  | `BIR_PROXY_STAY_UP` | proxy | unset | `1`: keep answering errors after the upstream dies (0.2.12). |