@basein/runner 0.3.7 → 0.3.9

package/docs/calculatedReplay.md

@@ -45,7 +45,7 @@ this design has a ladder whose cheapest rung is free — and the more of a stack
 |---|---|---|
 | Runs MCP tools itself | No (`replay.md` §12: *"rrepeat never executes tools — not even MCP"*) | **Yes**, on the connection it already owns |
 | Divergence fallback | A second Agent SDK session — the dominant cost | Direct `tools/call` through the proxies — **$0** |
-| Reaches built-ins (`Bash`, `Read`) | Only by steering the live session | Read-only ones (`Read`, `Glob`, `Grep`) run on the runner itself, jailed to the project (`BIR_BUILTIN_TOOLS`); `Edit`/`Write`/`Bash` opt-in, else steered |
+| Reaches built-ins (`Bash`, `Read`) | Only by steering the live session | `Read`, `Glob`, `Grep`, `Edit`, `Write` and `Bash` run on the runner itself, jailed to the project (`BIR_BUILTIN_TOOLS`, `BIR_BASH_ALLOW`); any other built-in is steered |
 | Needs `@anthropic-ai/*` at runtime | Yes | No (§16.3) |
 
 ---
@@ -252,21 +252,26 @@ export function modeFor(steps: SerializedScenarioStep[], wrapped: ReadonlySet<st
 | — | `none` | nowhere | declines at gate 5 |
 
 `live` covers three genuinely different cases and treats them identically, correctly: a built-in
-off the `BIR_BUILTIN_TOOLS` allow-list (by default everything but `Read`, `Glob`, `Grep` — so
-`Bash`, `Edit`, `Write`, `Task`, `WebFetch`), an MCP server the user chose not to wrap, and
+off the `BIR_BUILTIN_TOOLS` allow-list (by default everything but the six the runner implements —
+so `Task`, `WebFetch`; up to 0.3.7 also `Bash`, `Edit`, `Write`), an MCP server the user chose not to wrap, and
 `claude-in-chrome` — which is `scope: "dynamic"`, appears in no config file, and therefore can never
 be wrapped (v1 design §1). None of them can be reached except through the session.
 
 A built-in **on** the allow-list runs on the runner, in the session's own project directory and
 confined to it — a path that resolves outside the project, symlinks included, is refused and the
-step is `not_run`, the clean hand-over. `Bash` needs a second opt-in on top (`BIR_BASH_ALLOW`,
-command prefixes), because a runner-side step never reaches the host's permission prompts. The
+step is `not_run`, the clean hand-over. `Bash` has a second switch on top (`BIR_BASH_ALLOW`,
+command prefixes; unset means any command, `none` keeps `Bash` in the session), because a
+runner-side step never reaches the host's permission prompts. The
 results mimic the hook's view of each tool, which is what the recorder stored for built-in steps
 and what their `toolOutputLogic` was authored against; they pass through the same
 `serializeCapped(redact(…))` as every recorded payload. Bash keeps the host's own semantics a
 recorded chain was written against: each call is a fresh `-l` shell (variables and functions do
 not persist), while the **working directory carries over** from the plan's previous Bash step —
-`cd sub` then `cat file` replays as it ran.
+`cd sub` then `cat file` replays as it ran. Two Bash calls the host runs differently are never
+started on the runner, and are `not_run` instead: one with `run_in_background` (there is no shell
+here to leave it in), and one whose own `timeout` is longer than the step's clock
+(`BIR_STEP_TIMEOUT_MS`, clipped to what the plan has left) — cut off part-way, an `npm i -g` or a
+migration can leave things half done.
 
 A **mixed** scenario in `steer` mode still gets direct execution for its wrapped steps once it
 diverges (§8), so coverage is not all-or-nothing at the divergence boundary — only at the arm-time
@@ -898,9 +903,38 @@ steps: Array<{
   stage?: ExecutionStage;     // where in the step, for `failed` / `skipped`
   error?: string;             // message only — this is rendered in a web page
   durationMs?: number;
+  toolInput?: string;         // what the step sent its tool, as JSON text
+  toolOutput?: string;        // what its output logic was given
 }>
 ```
 
+**What the step sent and received.** `toolInput` is the arguments the step's input logic computed
+on this turn, and `toolOutput` what its output logic was given: the tool's answer, or for `recorded`
+the source run's output that stood in. They are this run's, and the service can learn them nowhere
+else — a matched turn is not recorded (§12). Without them the recording page can only put the
+recording's own sample under this run's verdict and time: a transfer of 3500 beside "passed", after
+a replay that sent 3600.
+
+- **Redacted and capped.** Each is serialized and redacted as a recorded step's payload is
+  (`serializeCapped(redact(…))`; an output is redacted again here, as JSON, because the hook's view
+  of a steered built-in is threaded as the host gave it). Then it is cut to 3,800
+  bytes (`REPORT_PAYLOAD_MAX_BYTES`), as text and with a marker that says how much was dropped: these
+  are read on a page, never parsed, and with its marker a cut payload still fits the 4,000
+  characters the service keeps of each — so the page shows this cut and its sizes, not a second one.
+  The recorder redacts once more as it sends.
+- **Inside one small request.** The service reads `POST /scenarios/:id/executions` with its 100 kB
+  JSON parser (only `/recordings` has the 5 MB one) and answers a larger body 413, which would lose
+  the turn's cost with it. So the recorder holds the whole report to 96 KiB (`REPORT_MAX_BYTES`):
+  it writes the costs and the verdicts first, then adds payloads while they fit — the failed steps'
+  first, then every step's input, then the outputs, in the order the steps ran. A payload that does
+  not fit is left out, and its step keeps its verdict.
+- **Absent where there was none.** A step whose input logic threw has no input. A `skipped` step
+  has no output, and neither has a call the host says failed without a result.
+- **Only for the reported scenario's own steps.** A step inside a called segment is reported with
+  its verdict and its `scenarioId`, without payloads: the console still files verdicts by step
+  number alone (the service's plan-services.md, W1.9), and a called segment numbers its steps from
+  zero too.
+
 `ok` and `recorded` are passes (`recorded` with an asterisk: the values came from the source run,
 which happens only with `BIR_REPLAY_RECORDED_OUTPUTS=1` and in `bir replay`). `skipped` produced no
 result here: its call could not be sent (a built-in outside the session, no proxy for its server,
@@ -921,8 +955,10 @@ The verdicts come from `plan.ts`'s `onStep`, which has always known them — wit
 logic failure is announced *before* the throw propagates (`reportingStage`), because after it
 nothing knows which step the stack trace belonged to. In steer mode there is no loop to observe, so
 the controller records the verdict at both hooks: `PreToolUse` for an input-logic throw, `PostToolUse`
-for the pass or the output-logic throw. `ReplayState.stepResults` is keyed by step index so the
-second word on a step replaces the first.
+for the pass or the output-logic throw — and keeps the pinned arguments on the pin, so the verdict
+`PostToolUse` writes can say what the call ran with. `ReplayState.stepResults` is keyed by the
+scenario that owns the step and its index, so the second word on a step replaces the first and a
+called segment's step 0 never replaces the caller's.
 
 One consequence for the rule that a costless decline is not worth sending (`buildReport`, and the
 service's matching `202 { recorded: false }`): it now *is* worth sending when a step broke. No cost
@@ -986,6 +1022,10 @@ trusting it.
 would 404. `run.recording = false` already handles it — `record()` early-returns and `sealRun` skips
 both the final answer and `finishRun`.
 
+What the turn's steps sent and received is not lost with it: the execution report carries them, on
+each step's verdict (§11.2a). That is the only copy the service gets, and it is what the recording
+page shows as the scenario's last run.
+
 Three consequences to keep straight:
 
 1. **Step indices are still allocated.** `ordering` runs regardless, correlations are still minted

package/docs/calculatedReplayGuide.md

@@ -234,7 +234,10 @@ When replay arms:
   when it plausibly *is* the step — the same command, the same file, an argument in common (design
   §7) — and never in plan mode.
 - **Direct steps are not visible to the permission system at all.** They are dispatched straight to a
-  `bir-proxy`, which calls the upstream on a connection it already holds. Claude Code never sees them.
+  `bir-proxy`, which calls the upstream on a connection it already holds — or, for Claude Code's own
+  `Read`, `Glob`, `Grep`, `Edit`, `Write` and `Bash`, run by the recorder itself in the project
+  folder, any `Bash` command included (`BIR_BUILTIN_TOOLS`, `BIR_BASH_ALLOW`; up to 0.3.7 only the
+  first three). Claude Code never sees them.
   The `run_scenario` call that starts them is answered `allow` too, while the runner has announced a
   direct plan; your own deny or ask rules for it still apply.
 
@@ -243,7 +246,9 @@ That is what makes replay fast, and it is the whole risk. Two things follow:
 1. Only enable it in a project whose scenarios you have dry-replayed and read (§3.1).
 2. Run `bir replay allow <servers>` (§5.3). An allowlist of the servers you are comfortable having
    called unattended is cheap insurance, and it is the difference between "replay reads a dashboard"
-   and "replay can do anything any wrapped server can do".
+   and "replay can do anything any wrapped server can do". The same goes for built-ins:
+   `BIR_BUILTIN_TOOLS=Read,Glob,Grep` keeps writes and commands with the model, and
+   `BIR_BASH_ALLOW=git,npm test` (for example) limits the commands the recorder runs itself.
 
 Nothing else is approved on your behalf. Outside a matched scenario's own steps, a wrapped MCP call
 keeps its permission prompt, as it would without the runner (`bir doctor` says so in a note);
@@ -354,8 +359,8 @@ so there is exactly one switch and it cannot get out of step with itself.
 | `BIR_STEP_TIMEOUT_MS` | `60000` | Per-step ceiling for one direct `tools/call`, clipped to what the plan has left |
 | `BIR_LOGIC_TIMEOUT_MS` | `1000` | How long one scenario logic body may run before it is stopped. At most 10 000 |
 | `BIR_REPLAY_RECORDED_OUTPUTS` | *(unset)* | `1` lets a step that could not be sent (a built-in off `BIR_BUILTIN_TOOLS`, no proxy, a server off the allowlist) stand on its recorded output instead of stopping the plan. Never for a timeout or an upstream error |
-| `BIR_BUILTIN_TOOLS` | `Read,Glob,Grep` | Built-ins the runner executes itself during a replay, in the session's own project directory and confined to it. A chain whose only non-MCP steps are on this list arms `direct`. Add `Edit,Write,Bash` only deliberately — a runner-side step never reaches the host's permission prompts. `none` restores the old behaviour: every built-in hands the turn back |
-| `BIR_BASH_ALLOW` | *(unset)* | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset, `Bash` never executes on the runner, even when `BIR_BUILTIN_TOOLS` lists it. A plan's Bash steps share a working directory, exactly as the host's own Bash tool does — variables and functions do not persist, also as the host's |
+| `BIR_BUILTIN_TOOLS` | `Read,Glob,Grep,Edit,Write,Bash` | Built-ins the runner executes itself during a replay, in the session's own project directory and confined to it. A chain whose only non-MCP steps are on this list arms `direct`. A runner-side step never reaches the host's permission prompts, so leave off any tool you want the model to call itself; `Read,Glob,Grep` is the read-only set (the default up to 0.3.7). `none`: every built-in step is steered in the session |
+| `BIR_BASH_ALLOW` | *(unset: any command)* | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset means any command; `none` means no command, and `Bash` steps are steered in the session. A plan's Bash steps share a working directory, exactly as the host's own Bash tool does — variables and functions do not persist, also as the host's. A call with `run_in_background`, or with its own `timeout` longer than the step's clock (`BIR_STEP_TIMEOUT_MS`), is never started on the runner: the plan hands over in front of it |
 | `BIR_INTENT_ARM` | *(unset)* | `1` lets a match found mid-turn arm a plan in the calling tool's place, which means waiting for it (`BIR_INTENT_ARM_BUDGET_MS`, 500); `0` never waits; unset follows `BIR_SEGMENT_ARM` |
 | `BIR_VERBOSE` | *(unset)* | `1` adds per-step `replay.*` detail lines |
 
@@ -553,7 +558,7 @@ matches. A prompt that missed both is matched once more on its task line (`PROMP
 | `replay.vetoed … the model's first call was …, not run_scenario` | A direct plan, and the model reached for another tool first — usually because it has no `run_scenario` | As for `no_delivery_tool`. Nothing ran; the model did the task |
 | `replay.pin_declined` on most steered turns | The model's calls to the expected tool are not the step — another command, another file — so nothing is pinned; if it then moves on, `replay.pin_abandoned` retires the plan | Harmless for those calls, which ran as the model wrote them. Persistent declines on one step usually mean its arguments cannot be shown in the directive (they depend on an earlier step's output, or the command runs to several lines), and the model writes its own |
 | `why="no step is executable"` | None of the scenario's tools is a wrapped MCP server and none is a built-in reachable in this session | Wrap the servers the scenario uses (`bir install --server …`) and recalculate |
-| `plan.armed mode=steer` where you expected `direct` | At least one step is a built-in off `BIR_BUILTIN_TOOLS` (default `Read,Glob,Grep` — so usually `Edit`, `Write` or `Bash`), an unwrapped MCP server, or `claude-in-chrome` | `bir status` shows what is wrapped; the `plan.armed` line's `coverage=` names the `live` step. Widen `BIR_BUILTIN_TOOLS` (and `BIR_BASH_ALLOW`) only deliberately. `claude-in-chrome` is `scope: "dynamic"` and can never be wrapped |
+| `plan.armed mode=steer` where you expected `direct` | At least one step is a built-in off `BIR_BUILTIN_TOOLS` (by default one the runner does not implement, such as `WebFetch` or `Task`; a `Bash` step too when `BIR_BASH_ALLOW=none`), an unwrapped MCP server, or `claude-in-chrome`. Before 0.3.8 the default list was `Read,Glob,Grep`, so `Edit`, `Write` and `Bash` steered too | `bir status` shows what is wrapped; the `plan.armed` line's `coverage=` names the `live` step. `claude-in-chrome` is `scope: "dynamic"` and can never be wrapped |
 | Model ignores the directive and diverges every turn | Steering is advisory — the model chooses; `bir` only pins arguments | Expected occasionally. Persistent divergence usually means the scenario's tools do not fit the live task; check `intent` |
 | A `replay.thread` line with no `emitted=` (under `BIR_VERBOSE=1`) | `toolOutputLogic` derived nothing — usually the output shape changed since recording | Dry-replay (§3.1). If the dry run emits and the live run does not, the tool's output shape has drifted; recalculate |
 | `replay.thread_fallback` on a wrapped step | The proxy did not report within the grace window | Check that proxy is alive (`bir doctor`). Repeats mean a slow or dying upstream |
@@ -567,6 +572,7 @@ matches. A prompt that missed both is matched once more on its task line (`PROMP
 | A parked step ran again after an hour, or a plan was recalculated by itself | The service's grace period (D8): a parked step is served with `stop: null` once an hour for a day and a success clears its count; if it keeps failing, or its tool is gone, the next repeat of the prompt is recorded in full and the plan is calculated again from it | Nothing: that is the repair. `bir investigate` says which it is — a step still being retried, a parked step whose repair is due, or a plan being calculated again. To repair now by hand: `bir scenario calc <runId> --force` for a whole-run plan; from the next version, `bir scenario calc <scnId> --force` works for any plan, a sub-task one included. A plan with steps edited by hand is not repaired by itself: the parked step stays parked, and `bir investigate` says the repair was skipped. Then fix the step (§9.2); or take the edits back with `bir scenario undo`, once per edit, and it is repaired as usual; or recalculate now and drop the edits: `bir scenario calc <scnId> --force --discard-edits` |
 | `sessionCostUsd` grows every prompt in a session | The per-run usage watermark is missing or not taken | Design §11.4. Reports without a mark must carry `measured: false` |
 | Everything works, nothing is saved | A replayed turn is deliberately not recorded (design §12) | Correct. The matched run stays canonical and its `iterations` is bumped |
+| The console shows a replayed step's input and output with the recording's values, beside this run's verdict and time | The run was reported by a runner up to 0.3.8, which sent each step's verdict and duration and nothing of what it sent or received. The page then falls back to the recording's sample (design §11.2a) | Update the runner: a newer one sends each replayed step's input and output with its verdict, redacted and cut to under 4 KB, and the card shows them as *last run*. To see what an older replay ran with: the result of `mcp__bir__run_scenario` in the session's transcript, or `lastParams` in `bir scenario show scn_…` |
 | `bir doctor`: the SessionStart hook points at a Node that is gone | The hook names Node by absolute path. That Node was uninstalled, moved, or was a version manager's per-shell copy; the hook then fails on every session, no recorder is started, and runs record Tier 2 | `bir setup` here again — it rewrites the hook to the Node that runs it |
 | `control.port_busy` in the log; the recorder is not running | Another program holds this project's port, or the system excludes it (on Windows, a range Hyper-V or WSL reserved). A background recorder with a recorded port does not fall back to another, because every hook but SessionStart names this one; only a `bir-hooks` you run by hand in a terminal still does | `bir setup --port <another>` in this directory — the fix the log line itself names. It rewrites the hook URLs and the scenario server's address together and restarts the recorder |
 | The recorder was started by the hook, and `bir doctor` says `servers=(all wrapped)` although you set `BIR_REPLAY_ALLOW_SERVERS` | The hook starts the recorder with Claude Code's environment, not your terminal's; a variable set in one terminal is not seen | `bir replay allow <a,b>` — stored for the project, read by every recorder — then `bir up --restart` |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@basein/runner",
-  "version": "0.3.7",
+  "version": "0.3.9",
   "description": "A recording MCP proxy: sits between any MCP client and its MCP servers, executes each call on the client's behalf, and records the run as a reusable BaseIn scenario.",
   "type": "module",
   "license": "MIT",

package/production.md

@@ -518,6 +518,7 @@ Over 2-minute runs with one turn every 8 s (92 hooks):
 - Non-JSON tool errors: 8 KiB.
 - Report errors: 2 KiB.
 - Tool payloads: 64 KiB.
+- A replayed step's input and output, in the execution report: 3,800 bytes each, redacted like a recorded step's. The whole report stays under 96 KiB, because the service reads that route with its 100 kB parser: a payload that does not fit is left out, and its step keeps its verdict.
 
 Truncation is always marked with the original size.
 
@@ -991,8 +992,8 @@ Defaults were checked in `src/`.
 | `BIR_STEP_TIMEOUT_MS` | recorder | 60000 | One direct `tools/call`. |
 | `BIR_LOGIC_TIMEOUT_MS` | recorder | 1000 (capped at 10000) | Synchronous time limit for one scenario logic body. |
 | `BIR_REPLAY_RECORDED_OUTPUTS` | recorder | unset | `1`: a step that could not be sent may use its recorded output, labelled as such. |
-| `BIR_BUILTIN_TOOLS` | recorder | `Read,Glob,Grep` | Built-ins the runner executes itself during a replay, in the session's project directory and confined to it. Add `Edit,Write,Bash` deliberately — runner-side steps bypass the host's permission prompts. `none` turns it off. |
-| `BIR_BASH_ALLOW` | recorder | unset | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset, `Bash` never executes on the runner even when listed. A plan's Bash steps share a working directory, as the host's do; variables do not persist, also as the host's. |
+| `BIR_BUILTIN_TOOLS` | recorder | `Read,Glob,Grep,Edit,Write,Bash` | Built-ins the runner executes itself during a replay, in the session's project directory and confined to it (up to 0.3.7 the default was `Read,Glob,Grep`). Runner-side steps bypass the host's permission prompts; leave a tool off to have the model call it. `none` turns it off. |
+| `BIR_BASH_ALLOW` | recorder | unset (any command) | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset means any command; `none` means no command, and `Bash` steps are steered. A plan's Bash steps share a working directory, as the host's do; variables do not persist, also as the host's. |
 | `BIR_DERIVE_RECENT_RESULTS` | recorder | 5 | How many of this prompt's tool results derivation may read (`0`: none). Never for plans with a command sink. |
 | `BIR_INTENT_MATCH` | recorder | on | `0` turns mid-turn intent probes off. |
 | `BIR_INTENT_MATCH_BUDGET_MS` | recorder | 4000 | One probe's own ceiling. It is not a hook wait. |