@basein/runner 0.3.9 → 0.3.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/README.md +47 -6
  2. package/dist/auth/client.d.ts +15 -0
  3. package/dist/auth/client.js +34 -12
  4. package/dist/bin/bir-hooks.d.ts +1 -1
  5. package/dist/bin/bir-hooks.js +1 -1
  6. package/dist/bin/bir.d.ts +2 -0
  7. package/dist/bin/bir.js +60 -6
  8. package/dist/bin/investigate.d.ts +5 -0
  9. package/dist/bin/investigate.js +117 -15
  10. package/dist/bin/setup.d.ts +12 -0
  11. package/dist/bin/setup.js +185 -56
  12. package/dist/bin/update.d.ts +81 -0
  13. package/dist/bin/update.js +194 -0
  14. package/dist/config/adapters/claude-code.d.ts +18 -1
  15. package/dist/config/adapters/claude-code.js +30 -3
  16. package/dist/config/generate.d.ts +14 -0
  17. package/dist/control/daemon.d.ts +26 -1
  18. package/dist/control/daemon.js +53 -3
  19. package/dist/control/pin-guard.d.ts +6 -1
  20. package/dist/control/pin-guard.js +15 -1
  21. package/dist/control/recorder-keeper.js +2 -1
  22. package/dist/control/server.d.ts +205 -2
  23. package/dist/control/server.js +766 -56
  24. package/dist/control/transcript.d.ts +4 -0
  25. package/dist/control/transcript.js +4 -0
  26. package/dist/proxy/session.js +1 -0
  27. package/dist/record/housekeeping.d.ts +7 -1
  28. package/dist/record/housekeeping.js +11 -3
  29. package/dist/record/recorder.d.ts +34 -0
  30. package/dist/record/remote-recorder.d.ts +6 -0
  31. package/dist/record/remote-recorder.js +31 -4
  32. package/dist/record/switchable-recorder.js +1 -0
  33. package/dist/replay/controller.d.ts +139 -3
  34. package/dist/replay/controller.js +248 -24
  35. package/dist/replay/derive.js +2 -0
  36. package/dist/replay/handover.d.ts +13 -1
  37. package/dist/replay/handover.js +19 -0
  38. package/dist/replay/safety.d.ts +15 -4
  39. package/dist/replay/safety.js +19 -6
  40. package/dist/replay/source-run.js +2 -1
  41. package/dist/util/service-headers.d.ts +24 -0
  42. package/dist/util/service-headers.js +47 -0
  43. package/docs/BaseInstRunner.md +115 -1
  44. package/docs/calculatedReplay.md +209 -32
  45. package/docs/calculatedReplayGuide.md +133 -12
  46. package/docs/installRun.md +91 -10
  47. package/docs/quickstart.md +64 -9
  48. package/package.json +1 -1
  49. package/production.md +83 -19
  50. package/scripts/install.ps1 +25 -1
@@ -45,7 +45,7 @@ this design has a ladder whose cheapest rung is free — and the more of a stack
45
45
  |---|---|---|
46
46
  | Runs MCP tools itself | No (`replay.md` §12: *"rrepeat never executes tools — not even MCP"*) | **Yes**, on the connection it already owns |
47
47
  | Divergence fallback | A second Agent SDK session — the dominant cost | Direct `tools/call` through the proxies — **$0** |
48
- | Reaches built-ins (`Bash`, `Read`) | Only by steering the live session | `Read`, `Glob`, `Grep`, `Edit`, `Write` and `Bash` run on the runner itself, jailed to the project (`BIR_BUILTIN_TOOLS`, `BIR_BASH_ALLOW`); any other built-in is steered |
48
+ | Reaches built-ins (`Bash`, `Read`) | Only by steering the live session | `Read`, `Glob`, `Grep`, `Edit`, `Write` and `Bash` run on the runner itself, jailed to the project (`BIR_BUILTIN_TOOLS`, `BIR_BASH_ALLOW`); any other built-in is steered. In a host that is not Claude Code a tool with one of these names is the application's own, and the runner runs none of them (§7.4) |
49
49
  | Needs `@anthropic-ai/*` at runtime | Yes | No (§16.3) |
50
50
 
51
51
  ---
@@ -131,6 +131,10 @@ Server-side, and worth stating because the threshold in §4 is a judgement about
131
131
  (`recordings/embeddings.ts`).
132
132
  - Candidates are **the caller's own canonical runs only** (`listUserRunEmbeddings(userId)`);
133
133
  duplicates and sub-threshold runs are excluded from the pool at record time.
134
+ - **Inside one host** — on the service as built, which is not deployed yet. A run created with
135
+ `framework` `langchain` or `langchain:<scope>` (§7.4) is matched only against recordings and
136
+ plans of that same text. Everything else (`baseinstrunner`, `custom`, the built-ins) is Claude
137
+ Code's pool, as before.
134
138
  - The match is the single best cosine ≥ `SIMILARITY_THRESHOLD` (default **0.9**).
135
139
  - A matched run only carries a `scenarioId` when its scenario exists **and** is `state: 'ready'`.
136
140
 
@@ -191,7 +195,7 @@ Every gate that declines logs **one** `replay.decision` line carrying the reason
191
195
  | # | Gate | Declines when | Outcome floor |
192
196
  |---|---|---|---|
193
197
  | 1 | **A match arrived** | `getMatch()` resolved `null`, or the budget expired (§10) | (no report — nothing matched) |
194
- | 2 | **Replay enabled** | `BIR_REPLAY === "0"`, or, with it unset, the project's stored switch is off (`bir replay off`, or a `bir install` without `--replay`) | `not_steered` |
198
+ | 2 | **Replay enabled** | `BIR_REPLAY === "0"`, or, with it unset, the project's stored switch is off (`bir replay off`, or a `bir install` without `--replay`). In a session of another host, also when the run's adapter asked for no plan with its prompt (`host_records_only`, §7.4) | `not_steered` |
195
199
  | 3 | **Scenario ready** | `scenario == null`, `state !== "ready"`, or `steps.length === 0` | `not_steered` |
196
200
  | 4 | ~~**Similarity**~~ | Retired: a match is the decision to use the scenario (below) | — |
197
201
  | 5 | **Tool coverage** | No step is executable anywhere (§5 → `mode: "none"`) | `not_steered` |
@@ -204,6 +208,9 @@ inside the prompt's budget (§10) is dropped (`match_timeout`, `arm_timeout`), a
204
208
  after the prompt was answered is retired unannounced (`match_late`, `arm_unannounced`). A `direct`
205
209
  plan is declined when no running `bir-scenario` has said it serves `run_scenario`
206
210
  (`no_delivery_tool`, §6.3). In plan mode (`permission_mode: "plan"`) nothing is announced at all.
211
+ In a session of another host (§7.4) a plan that armed is taken back, before any step of it runs,
212
+ when it names a tool the agent that asked does not hold (`host_tool_missing`): it is booked like
213
+ any gate's decline, `not_steered`, with no step marked failed.
207
214
 
208
215
  Gate 4 is retired. Up to 0.3.6 it declined a match under `BIR_MIN_STEER_SIMILARITY` (0.92),
209
216
  above the server's 0.9, on the argument that the server only decided *"do not record this again"*
@@ -277,6 +284,12 @@ A **mixed** scenario in `steer` mode still gets direct execution for its wrapped
277
284
  diverges (§8), so coverage is not all-or-nothing at the divergence boundary — only at the arm-time
278
285
  mode choice.
279
286
 
287
+ **None of this applies in a session of another host.** When a session's requests name a host that
288
+ is not Claude Code (§7.4), a tool's name carries no Claude Code meaning: `Bash`, `Read` or an
289
+ `mcp__` prefix name the application's own tools, no proxy of this folder sees their calls, and the
290
+ runner runs none of them, whatever `BIR_BUILTIN_TOOLS` lists. Every step is `live`, so the mode is
291
+ always `steer`.
292
+
280
293
  ---
281
294
 
282
295
  ## 6. Mode A — direct execution
@@ -423,7 +436,9 @@ another way. Divergence (§8) is for steer plans only.
423
436
  The mixed rung. Ported from RRepeat's `replay.md` §3/§12, which is the right design and is already
424
437
  proven: the model chooses *which* tool and *when*; the control server overrides the *arguments* and
425
438
  threads state between steps. The session never leaves the model's hands, so every
426
- `tool_use`/`tool_result` in the transcript is genuine and continuation is automatic.
439
+ `tool_use`/`tool_result` in the transcript is genuine and continuation is automatic. That is Claude
440
+ Code. In a host that is not Claude Code the same mode is stepped through by the host's adapter, in
441
+ the model's place, and a call the model makes itself is never pinned (§7.4).
427
442
 
428
443
  **Arm** at `UserPromptSubmit` — plan on `run.plan`, `run.planToolUse = new Set()`, directive via
429
444
  `additionalContext`, **no `decision`** so the prompt still reaches the model:
@@ -589,15 +604,91 @@ In [`HOOK_ROUTES`](../src/config/adapters/claude-code.ts) neither `PreToolUse` n
589
604
  carries `async: true`, so this already holds — **and must never be relaxed**. Add it to the comment
590
605
  block there, next to the `SessionEnd` note that records the same class of lesson.
591
606
 
607
+ ### 7.4 A host that is not Claude Code
608
+
609
+ Everything above assumes Claude Code: its hooks report the turn, and its model makes the plan's
610
+ calls. The recorder's local protocol also serves a host that has neither. Today that is a
611
+ LangChain application: a Python program whose agent is built with LangChain's `create_agent` and
612
+ the package `langchain-basic2202`, a LangChain middleware that talks to the recorder over that
613
+ protocol. The package is *the adapter* below, and it is not on PyPI yet. Claude Code's hooks are
614
+ unchanged, and every new field is optional. A session whose requests carry
615
+ `host: { app, version?, adapter?, scope? }` with an `app` other than `claude-code` is a session of
616
+ another host ([`control/server.ts`](../src/control/server.ts)). The fields named below are what
617
+ such an adapter adds to the hook routes of the control plane
618
+ ([BaseInstRunner.md](BaseInstRunner.md), Phase 5). `hostFields` in `/health` tells an adapter how
619
+ much of this a recorder understands: with 1 it records, and from 2 on it may also ask for a plan
620
+ and step through it.
621
+
622
+ For such a session:
623
+
624
+ - **The run names its host.** It is created with `framework` = the app, or `app:scope`
625
+ (`langchain`, `langchain:billing`), and `metadata.host`. A Claude Code run is created as before
626
+ (`framework: "baseinstrunner"`).
627
+ - **Only `/session/prompt` opens a run.** Any other hook that finds no open run opens none and
628
+ records nothing: it is answered `{}`, or `{ noPlan: true }` when it asked for a step.
629
+ - **A plan starts only for a prompt that said `replay: true`.** Otherwise the match is declined at
630
+ gate 2 with `host_records_only` (§4). The prompt's answer carries the armed plan as data too
631
+ (`plan: { scenarioId, mode, steps: [{ stepIndex, toolName }], stopAt? }`): the directive beside
632
+ it is a text written for a model.
633
+ - **The mode is always `steer`, and the runner runs no step itself.** The host runs its own tools.
634
+ Tool names carry no Claude Code meaning: `Bash`, `Read`, `TodoWrite`, `ToolSearch`… are the
635
+ application's own tools, and nothing is skipped as housekeeping except the `bir` server's own
636
+ tools. So there is no direct mode there, and no divergence recovery (§8): nothing can be executed
637
+ here in the host's place.
638
+ - **The adapter asks for each step**, with a `/tool/pre` that says `scripted: true`, and the host
639
+ runs its own tool. The answer is the step's arguments (`hookSpecificOutput.updatedInput`), or
640
+ `{ noPlan: true }` when the recorder holds no live plan, or `{ declined: "host_tool_missing" }`
641
+ when the plan names a tool that is not in `tools` (the agent's tools, sent with the first such
642
+ request of a run), or a refusal whose reason is the hand-over note when the plan stopped in that
643
+ very request. The step's output is threaded from the `/tool/post` that follows: every step is
644
+ `live` (§7.2).
645
+ - **The model's own call is never pinned.** A `/tool/pre` without `scripted` is the model's own
646
+ call, and it is never given the plan's arguments. When a plan is live it is handed over (§8): the
647
+ call runs as the model wrote it, and the hand-over note comes with that call's `/tool/post`.
648
+ - **The host may stop a step itself**, and says so with the step's `/tool/post`.
649
+ `host_stop: "rejected"` means a person said no, or answered in the tool's place, or a call limit
650
+ blocked the call; `"edited"` means a person changed the call. Either way the step's verdict is
651
+ `skipped`, not `failed`, nothing is counted against the plan, and the plan hands over (§8). §11.1
652
+ has what the report says.
653
+ - **There is no transcript.** What Claude Code's transcript gives comes with the requests instead:
654
+ with a `/tool/pre`, what the model wrote and reasoned before the call (`context`, `thinking`);
655
+ with `/session/stop`, the answer and the tokens (`final_answer`, `usage`). The cost comes from
656
+ that `usage` (§11.4). The conversation's earlier tool results do not come with the prompt: a new
657
+ prompt's values are read from the prompt alone, here as in Claude Code (§16.3), and a
658
+ `recent_results` an adapter still sends is not read.
659
+ - **A run can end without an ordinary stop.** With `/session/stop`, `failed` closes the run with
660
+ that text as its `error` (cleaned of secrets, cut) and no final answer, and `incomplete` marks
661
+ the finish `incomplete: { lostSteps, lossy }`. A run sealed after two hours of silence is closed
662
+ with `error: "turn ended early: idle"`. A run that is closed while its plan is still live has the
663
+ plan handed over first, so its report says what happened (§11.1). On the service as built (not
664
+ deployed yet), a run with an `error` or with that mark is never picked for automatic calculation
665
+ and no longer catches its prompt, unless its owner preferred it or it already has a plan. Only a
666
+ run of another host sends either today: a Claude Code run that was interrupted or lost steps is
667
+ finished as before.
668
+ - **A value that a plan reads from the prompt is checked against the host's own sinks** before it
669
+ goes into a tool: the `sinks` the host declared with its prompt (§13.2, mitigation 6).
670
+ - **The switch is the same.** `bir setup --host langchain` stores replay on for the folder
671
+ (`--no-replay` stores it off), and `bir replay on` and `bir replay off` work there as anywhere.
672
+ - **The audit lines name the host**: `run.start … host=langchain`,
673
+ `plan.armed … mode=steer host=langchain`, and `run.finish … ended=<reason>` when the run did not
674
+ end with an ordinary stop. `run.incomplete` and `run.host_failed` are what the host said with its
675
+ stop.
676
+
677
+ The how-to for a LangChain developer — setting up the folder, the two lines of Python, what to do
678
+ when something is wrong — is `langChainGuide.md` in the root of the runner's repository (it is not
679
+ in the npm package), and is not repeated here. The design guide with every decision is
680
+ `integratioLangChane.md` in the service's repository.
681
+
592
682
  ---
593
683
 
594
684
  ## 8. Divergence
595
685
 
596
686
  In a steer plan, the model called a tool the plan did not expect, or the call had no `tool_use_id`.
597
- (A direct plan never diverges, §6.4. A `toolInputLogic` that throws is not a divergence either:
598
- re-executing would start at that same step and throw again, so the step is recorded as failed and
599
- the task is handed to the model, with a note in place of the call.) Three tiers, tried in order.
600
- Here is where the §0 asymmetry pays.
687
+ (A direct plan never diverges, §6.4. Nor does a plan in a host that is not Claude Code, §7.4: only
688
+ the host can run its tools, so the plan is handed over and nothing is executed in its place. A
689
+ `toolInputLogic` that throws is not a divergence either: re-executing would start at that same step
690
+ and throw again, so the step is recorded as failed and the task is handed to the model, with a note
691
+ in place of the call.) Three tiers, tried in order. Here is where the §0 asymmetry pays.
601
692
 
602
693
  **Tier 1 — direct-execute the remainder (new; the common case).**
603
694
  Take every not-yet-run step, execute it, and hand the composed result back. This happens inside the
@@ -636,6 +727,12 @@ about it:
636
727
  - beside a pin still in flight: *"… was not run by the scenario: another of its calls was still in
637
728
  flight, and a scenario runs one call at a time."*, and the pin itself *"was started and has not
638
729
  reported back"*.
730
+ - stopped by its host, in a session of another host (§7.4): *"… was not run by the host. The result
731
+ above was written by the host, not by the tool. Do not run the step again unless the user asks."*
732
+ (`host_rejected`), or *"… ran with arguments a person chose, not the scenario's. Its result above
733
+ is real and its side effects are in place. Do not run it again."* (`host_edited`). When no step
734
+ ran before it, the note says *"No step of the scenario ran as it was planned."* instead of *"No
735
+ step … ran"*. The report's `fallbackKind` stays `step_failed`.
639
736
 
640
737
  It ends with the remaining planned steps and the parameters derived for this request, within 4 000
641
738
  characters.
@@ -781,8 +878,8 @@ The hook is on the user's critical path. Every await here is a stall the user wa
781
878
 
782
879
  | Moment | Route | Hook timeout | Answered by | What may be awaited | Budget |
783
880
  |---|---|---|---|---|---|
784
- | `UserPromptSubmit` | `/session/prompt` | 10 s | 7 s | the match; for a scenario with a target, its derivation; a steer plan's argument preview | match `BIR_MATCH_BUDGET_MS` = **3000**; match and arming together ≤ **6 s**; derivation ≤ `BIR_HOOK_DERIVE_BUDGET_MS` = **4000** |
785
- | `PreToolUse` | `/tool/pre` | 15 s | 12 s | `plan.ready()` (derivation); on a divergence, the remaining steps | derivation ≤ min(`BIR_DERIVE_BUDGET_MS`, `BIR_HOOK_DERIVE_BUDGET_MS`) = **4000**; all replay work ≤ `BIR_PRETOOL_REPLAY_BUDGET_MS` = **9000** (held at ≤ 10 000) |
881
+ | `UserPromptSubmit` | `/session/prompt` | 10 s | 8.5 s | the match; for a scenario with a target, its derivation; a steer plan's argument preview | match `BIR_MATCH_BUDGET_MS` = **3000**; match and arming together ≤ **7.5 s**; derivation ≤ `BIR_HOOK_DERIVE_BUDGET_MS` = **8000**, cut short by the 7.5 s |
882
+ | `PreToolUse` | `/tool/pre` | 15 s | 12 s | `plan.ready()` (derivation); on a divergence, the remaining steps | derivation ≤ min(`BIR_DERIVE_BUDGET_MS`, `BIR_HOOK_DERIVE_BUDGET_MS`) = **8000**; all replay work ≤ `BIR_PRETOOL_REPLAY_BUDGET_MS` = **9000** (held at ≤ 10 000) |
786
883
  | `run_scenario` | `/scenario/run` | `bir-scenario`'s own, **290 s** (`BIR_SCENARIO_RUN_TIMEOUT_MS`) | — | `ready()` + N steps | derivation `BIR_DERIVE_BUDGET_MS` = **8000**; plan `BIR_REPLAY_PLAN_BUDGET_MS` = **100000** (`BIR_REPLAY_BUDGET_MS` overrides it where set); per step `BIR_STEP_TIMEOUT_MS` = **60000**, clipped to what the plan has left |
787
884
  | `PostToolUse` | `/tool/post` | 5 s | 2 s | nothing | — |
788
885
  | `Stop` | `/session/stop` | 5 s | 2 s | nothing (report is queued) | — |
@@ -863,6 +960,25 @@ beside a pinned one hands over as `fell_back` or `failed` (§7). A direct run ca
863
960
  (Esc, or `bir-scenario`'s own timeout) books `fell_back` or `failed` with `measured: false`, never
864
961
  `diverged`: that turn's cost says nothing about what the task costs.
865
962
 
963
+ A host that is not Claude Code (§7.4) adds three endings, and none of them is counted against the
964
+ plan. The report says so with two fields: `measured`, and `baselineEligible: false`, which keeps
965
+ the turn's cost out of the baseline samples (absent, the turn is eligible).
966
+
967
+ - **A person's "no".** A step the host did not run (`host_stop: "rejected"`: a person said no, or
968
+ answered in the tool's place, or a call limit blocked the call) is recorded `skipped`, never
969
+ `failed`, and the plan hands over: `fell_back` when a step ran before it, `failed` when none did.
970
+ The report says `measured: false` and `baselineEligible: false`: the task ended early, so the
971
+ turn's cost is no measure of it.
972
+ - **A changed call.** A step that ran with arguments a person chose (`"edited"`) is `skipped` too,
973
+ and the plan hands over the same way. The cost stays measured, with `baselineEligible: false`.
974
+ - **A run closed with its plan live.** The plan is handed over first, so the report says
975
+ `fell_back` (a step ran) or `failed` (none ran), never `not_steered`, with `measured: false` and
976
+ `baselineEligible: false`. A run that ended early for any other reason (it failed, or went
977
+ silent) reports the same two fields.
978
+
979
+ A plan taken back before any step ran (`host_tool_missing`, §4) is a decline like any other:
980
+ `not_steered`.
981
+
866
982
  Direct mode reports `steered_full`. BaseIn's zod enum
867
983
  (mirrored in [`replay/types.ts`](../src/replay/types.ts)) has no `direct` member, and `steered_full`
868
984
  means exactly what happened: every planned step executed under the scenario's control and the
@@ -887,6 +1003,11 @@ POST /scenarios/:scenarioId/executions
887
1003
  An **unredeemed** ticket leaves no execution row at all. That asymmetry is intentional and worth
888
1004
  preserving: silence loses a saving; it never invents one.
889
1005
 
1006
+ **How late a report may arrive.** The deployed service redeems a ticket for one hour after its plan
1007
+ was handed out, and answers a later report `409 invalid_ticket`. The service as built (not deployed
1008
+ yet) redeems it for 24 hours. It also answers a report whose write failed `503 report_not_booked`,
1009
+ and the recorder sends that report once more.
1010
+
890
1011
  ### 11.2a What each step did (the service's errorshandling.md)
891
1012
 
892
1013
  The last five fields are the diagnosis, and none of the money depends on them. The service's
@@ -922,14 +1043,19 @@ a replay that sent 3600.
922
1043
  are read on a page, never parsed, and with its marker a cut payload still fits the 4,000
923
1044
  characters the service keeps of each — so the page shows this cut and its sizes, not a second one.
924
1045
  The recorder redacts once more as it sends.
925
- - **Inside one small request.** The service reads `POST /scenarios/:id/executions` with its 100 kB
926
- JSON parser (only `/recordings` has the 5 MB one) and answers a larger body 413, which would lose
927
- the turn's cost with it. So the recorder holds the whole report to 96 KiB (`REPORT_MAX_BYTES`):
928
- it writes the costs and the verdicts first, then adds payloads while they fit — the failed steps'
929
- first, then every step's input, then the outputs, in the order the steps ran. A payload that does
930
- not fit is left out, and its step keeps its verdict.
1046
+ - **Inside one small request.** The deployed service reads `POST /scenarios/:id/executions` with
1047
+ its 100 kB JSON parser (only `/recordings` has the 5 MB one) and answers a larger body 413, which
1048
+ would lose the turn's cost with it. So the recorder holds the whole report to 96 KiB
1049
+ (`REPORT_MAX_BYTES`): it writes the costs and the verdicts first, then adds payloads while they
1050
+ fit — the failed steps' first, then every step's input, then the outputs, in the order the steps
1051
+ ran. A payload that does not fit is left out, and its step keeps its verdict. The service as
1052
+ built (not deployed yet) accepts 1 MB on this route and on `/scenarios/:id/derive`, and cuts a
1053
+ report's `prompt` at 10,000 characters where the deployed one refuses the whole report with 400.
1054
+ The recorder keeps its report under 100 kB all the same: older services have that limit.
931
1055
  - **Absent where there was none.** A step whose input logic threw has no input. A `skipped` step
932
- has no output, and neither has a call the host says failed without a result.
1056
+ has no output, and neither has a call the host says failed without a result. The exception is a
1057
+ step its host stopped itself (§7.4): its verdict keeps what the host handed back, and after a
1058
+ changed call it has no input, because only the host knows the arguments that call ran with.
933
1059
  - **Only for the reported scenario's own steps.** A step inside a called segment is reported with
934
1060
  its verdict and its `scenarioId`, without payloads: the console still files verdicts by step
935
1061
  number alone (the service's plan-services.md, W1.9), and a called segment numbers its steps from
@@ -938,9 +1064,11 @@ a replay that sent 3600.
938
1064
  `ok` and `recorded` are passes (`recorded` with an asterisk: the values came from the source run,
939
1065
  which happens only with `BIR_REPLAY_RECORDED_OUTPUTS=1` and in `bir replay`). `skipped` produced no
940
1066
  result here: its call could not be sent (a built-in outside the session, no proxy for its server,
941
- only another open session's proxy), or it was sent and timed out or was refused. The plan stops at
942
- a `skipped` step. **`failed` is the only one that means the scenario is broken** rather than
943
- the environment — it is the scenario's own logic throwing, and it is what `Recalculate` fixes.
1067
+ only another open session's proxy), or it was sent and timed out or was refused. A step that its
1068
+ host stopped itself, in a host that is not Claude Code, is `skipped` as well: a person refused it,
1069
+ or changed its call (§7.4). The plan stops at a `skipped` step.
1070
+ **`failed` is the only one that means the scenario is broken** rather than the environment — it is
1071
+ the scenario's own logic throwing, and it is what `Recalculate` fixes.
944
1072
 
945
1073
  Three rules this must keep:
946
1074
 
@@ -969,7 +1097,7 @@ service's matching `202 { recorded: false }`): it now *is* worth sending when a
969
1097
  | Component | Source | Notes |
970
1098
  |---|---|---|
971
1099
  | `deriveCostUsd` | our own Haiku call's `usage`, priced locally | the only tokens replay itself spends |
972
- | `sessionCostUsd` | transcript usage **delta** for this run (§11.4) | the live turn: prompt, tool calls, answer |
1100
+ | `sessionCostUsd` | transcript usage **delta** for this run (§11.4); in a host that is not Claude Code, the `usage` its adapter sends with `/session/stop` | the live turn: prompt, tool calls, answer |
973
1101
  | `fallbackCostUsd` | **0.00** | direct execution spends no tokens (§8) |
974
1102
 
975
1103
  > ### ⚠ Verify the pricing table before copying it
@@ -1010,9 +1138,23 @@ rather than to a negative cost. A run with **no mark** (one opened lazily by a p
1010
1138
  prompt) reports `measured: false` — better to say "unmeasured" than to hand the ledger an inflated
1011
1139
  baseline.
1012
1140
 
1013
- `measured` is `true` only when a mark existed **and** `pricingVersion` matches the server's. The
1014
- server enforces the second half itself and stores a mismatched report as unmeasured rather than
1015
- trusting it.
1141
+ `measured` is `true` only when a mark existed, tokens were found since it, **and** every model of
1142
+ the turn has a price in the runner's table (`priceUsage` and `hasModelPricing`,
1143
+ [`control/server.ts`](../src/control/server.ts)). A model outside the table used to cost $0, and
1144
+ that zero was called measured: a saving nobody made. Now such a turn is not measured, in every
1145
+ host. A `run.unmeasured` line says why, with `why=`, when the run had no mark or when a model has
1146
+ no price (then with `models=`). A turn whose transcript held no tokens since the mark is not
1147
+ measured either, and writes no such line. No price table and no `PRICING_VERSION` changed with this.
1148
+
1149
+ The server judges a report again. It stores one whose `pricingVersion` differs from its own as
1150
+ unmeasured rather than trusting it. As built (not deployed yet) it does the same with a report that
1151
+ says `measured: true` with a session cost of $0, and a finish with a cost of 0 stores no cost.
1152
+
1153
+ **A host that is not Claude Code has no transcript, and no mark** (§7.4). Its cost comes from the
1154
+ `usage` its adapter sends with `/session/stop`, and is measured only when at least one entry has
1155
+ tokens and every model in the list has a price; with no usage at all, the line says
1156
+ `why="the host sent no usage"`. Such a host hands over one number for the whole run, so with more
1157
+ than one plan in one run none of them is measured.
1016
1158
 
1017
1159
  ---
1018
1160
 
@@ -1049,10 +1191,11 @@ bir replay --scenario scn_… --prompt "…" --dry
1049
1191
 
1050
1192
  Runs the same plan through the same executor, against the same live proxies, printing the per-step
1051
1193
  input/output and the response model. It is how you test a scenario without a session, how a Tier 2
1052
- or non-Claude-Code client gets any replay at all, and the first thing to reach for when a steered
1053
- turn behaves oddly. The operator named the scenario, so it keeps its own contract: a step it cannot
1054
- send stands on its recorded output, the allowlist does not gate it, and it is not held to the 270 s
1055
- direct-run ceiling (§10).
1194
+ client — or one that is neither Claude Code nor a host whose adapter steps through plans (§7.4) —
1195
+ gets any replay at all, and the first thing to reach for when a steered turn behaves oddly. The
1196
+ operator named the scenario, so it keeps its own contract: a step it cannot send stands on its
1197
+ recorded output, the allowlist does not gate it, and it is not held to the 270 s direct-run ceiling
1198
+ (§10).
1056
1199
 
1057
1200
  ---
1058
1201
 
@@ -1090,6 +1233,15 @@ What actually constrains it:
1090
1233
  and a `bir install` without it stores replay off in a project that never asked for it. A
1091
1234
  compromised or misconfigured `BIR_AUTH_URL` therefore cannot start executing logic on a machine
1092
1235
  whose operator never asked for replay.
1236
+ - **A small environment in an application's folder.** A recorder started for a folder that is set
1237
+ up for another host and not for Claude Code (`bir setup --host langchain`, `bir up`) gets a small
1238
+ environment, not the whole environment of the window that started it: `PATH`, the home and temp
1239
+ variables, what the system needs to start a process, the proxy and certificate variables, and
1240
+ every `BIR_*` variable (`smallEnv`, [`control/daemon.ts`](../src/control/daemon.ts)). Step code
1241
+ runs inside the recorder with no sandbox, so a model key or a database password of the
1242
+ application must not reach it. A folder that is set up for Claude Code too keeps the whole
1243
+ environment, because Claude Code's own steps may need it. Nothing changes for a Claude Code
1244
+ project.
1093
1245
 
1094
1246
  If that is not enough for a deployment, the mitigation is a real one and is out of scope here:
1095
1247
  evaluate the logic in a separate `node:vm` context with a frozen, minimal global. Filed as §20 question 3.
@@ -1152,6 +1304,9 @@ This is the sharpest edge in the whole design, and it must not be buried.
1152
1304
  answered `allow` while the runner has announced a direct plan, so it brings no surprise prompt
1153
1305
  (and no denial under `claude -p`); the user's own deny or ask rules for it still apply.
1154
1306
  - **Plan mode** (`permission_mode: "plan"`) gets none of it: no directive, no pin, no `allow`.
1307
+ - **A host that is not Claude Code** (§7.4) is neither mode as written here: the runner runs no
1308
+ step. It hands the host's adapter each step's arguments, the host runs its own tool, and the
1309
+ adapter reports a step that a person refused or changed (`host_stop`).
1155
1310
 
1156
1311
  So a scenario is, in effect, a **pre-approved list of tool calls with computed arguments**. That is
1157
1312
  precisely what makes it valuable, and precisely what makes it dangerous. Six mitigations, all
@@ -1159,7 +1314,8 @@ required:
1159
1314
 
1160
1315
  1. `--replay` is a distinct flag: `bir setup` passes it, and a `bir install` without it leaves
1161
1316
  replay off in a project that never asked for it. `bir replay off` or `BIR_REPLAY=0` turns it off
1162
- once installed, and `bir off` takes the whole runner out of the way.
1317
+ once installed — the recorder reads either when it starts, so `bir up --restart` applies it — and
1318
+ `bir off` takes the whole runner out of the way.
1163
1319
  2. `BIR_REPLAY_ALLOW_SERVERS` — a comma-separated allowlist of the server keys bir may call by
1164
1320
  itself: **direct** runs and divergence recovery (§8), though not `bir replay`, which an operator
1165
1321
  runs by hand. Unset means *all wrapped servers*; the guide (§5 of `calculatedReplayGuide.md`)
@@ -1178,7 +1334,11 @@ required:
1178
1334
  have), or the plan hands over; found at arm time, it declines as `missing_target`. A plan whose
1179
1335
  only such steps write files (`Edit`, `MultiEdit`, `Write`, `NotebookEdit`) rejects only line
1180
1336
  breaks and NUL. Plans with a step whose input is run do not send recent tool results to
1181
- derivation, and the derivation prompt is redacted before it leaves the machine.
1337
+ derivation, and the derivation prompt is redacted before it leaves the machine. In a session of
1338
+ another host (§7.4) the tool names are the host's own, and Claude Code's lists of built-ins are
1339
+ not read: a value that a plan reads from the prompt is checked before it goes into a tool the
1340
+ host declared in `sinks` with its prompt (`command`, `write`), or whose plain name has a command
1341
+ word (sql, query, exec, shell, command, script, eval).
1182
1342
 
1183
1343
  ### 13.3 The control plane
1184
1344
 
@@ -1222,7 +1382,13 @@ fail because of BaseInstRunner.**
1222
1382
  | `run_scenario` pending in two sessions and the request names neither | Nothing runs (`replay.run_ambiguous`); each model does its own task |
1223
1383
  | The runner is off (`bir off`, or `BIR_OFF=1` for a session) | Hooks answer `{}` and nothing arms; the `bir` server lists no tools |
1224
1384
  | Plan mode | Nothing is announced, pinned or approved |
1225
- | Ticket redemption fails (offline, 409) | Logged; the turn is unaffected. An unredeemed ticket books nothing |
1385
+ | Another host (§7.4): the run's adapter asked for no plan with its prompt | Declined at gate 2 (`host_records_only`), `not_steered`; the model does the task |
1386
+ | Another host: the plan names a tool the agent that asked does not hold | Taken back before any step runs (`host_tool_missing`), `not_steered`; the model does the task |
1387
+ | Another host: the model makes a call of its own while the plan is live | The plan is handed over. The call runs as the model wrote it, never with the plan's arguments, and the note comes with its `/tool/post` |
1388
+ | Another host: a person refuses a step, or changes its call | The step is `skipped`, never `failed`, and the plan hands over; `baselineEligible: false`, and after a refusal `measured: false` (§11.1) |
1389
+ | Another host: the run fails, goes silent for two hours, or is closed with its plan still live | A live plan is handed over first: `fell_back` or `failed`. The report says `measured: false` and `baselineEligible: false`. A run that ended early is finished with an `error` and no final answer |
1390
+ | The service answers `429`: an account over one of its limits (built on the service, not deployed yet) | A create, a step or a report is sent once more, at most 2 s later, then dropped (`recorder.send_failed` with `HTTP 429`). A dropped step is a hole in its recording, and a run whose create got the 429 runs without its plan. A 429 that the second try did not cure counts as the service being unwell: for 5 to 60 s no prompt of that recorder waits for a match. An intent probe over its own limit is answered `200` with no match, never `429` |
1391
+ | Ticket redemption fails (offline, 409) | Logged; the turn is unaffected. An unredeemed ticket books nothing. A `503 report_not_booked` (the service as built; not deployed yet) is sent once more |
1226
1392
  | Two reports for one ticket (retry, `Stop` racing `SessionEnd`) | Server books once, answers `200 { duplicate: true }` |
1227
1393
  | Replay off (`BIR_REPLAY=0`, `bir replay off`, or never installed with `--replay`) | Exactly v1: stop recording on a match, do nothing else, and no prompt waits for the match |
1228
1394
 
@@ -1291,6 +1457,11 @@ POST /proxy/result { workId: string, result?: unknown, error?: string }
1291
1457
  → 200 { ok: true }
1292
1458
  ```
1293
1459
 
1460
+ The hook routes ([BaseInstRunner.md](BaseInstRunner.md), Phase 5) carry more for a host that is not
1461
+ Claude Code: optional fields that its adapter adds to `/session/prompt`, `/tool/pre`, `/tool/post`
1462
+ and `/session/stop`, and `hostFields` in `/health`. §7.4 names them, and says what they mean for a
1463
+ plan.
1464
+
1294
1465
  ### 16.2 The direct executor
1295
1466
 
1296
1467
  ```ts
@@ -1393,7 +1564,11 @@ same parameters from the same prompt:
1393
1564
  live call, recent results) and a bearer token, `derived` as the service answered. This is the
1394
1565
  ordinary path: nearly every session runs inside Claude Code on a subscription and has no key of
1395
1566
  its own (segmented.md R-PARAM-5). The answer is re-normalised here against this runner's own copy
1396
- of the schema, so the gate reads a `missing` computed from what will actually run.
1567
+ of the schema, so the gate reads a `missing` computed from what will actually run. The service as
1568
+ built (not deployed yet) cuts the prompt it reads new values from at 20,000 characters, and keeps
1569
+ the last five tool results. A plan started at the prompt sends no tool results, in any host: the
1570
+ transcript does not hold the new prompt yet, so its results would be an earlier prompt's, read as
1571
+ this task's (segmented.md R-PARAM-2a). A plan started at a tool call sends this prompt's.
1397
1572
  - **Neither → return every key's recorded `sampleValue`**, `costUsd: 0`, `derived: false`. Settings
1398
1573
  stand on their samples; a scenario with a target declines rather than run on a stale one.
1399
1574
  - With an `apiKey`, `POST https://api.anthropic.com/v1/messages` with the same extraction prompt,
@@ -1523,7 +1698,9 @@ is refused.
1523
1698
  `bir` invocations (and a plain `cd`), are host housekeeping, like `ToolSearch`
1524
1699
  ([housekeeping.ts](../src/record/housekeeping.ts)). They are never recorded as steps, and never
1525
1700
  count as divergence from a plan. A session that fixes a plan must not become a plan that edits
1526
- plans when it is replayed. The operator's walk-through is
1701
+ plans when it is replayed. In a session of another host (§7.4) only the `bir` server's own tools
1702
+ are housekeeping: a tool the application names `Bash`, `ToolSearch` or `TodoWrite` is its own work,
1703
+ and is recorded like any other. The operator's walk-through is
1527
1704
  [calculatedReplayGuide.md](calculatedReplayGuide.md) §9.2.
1528
1705
 
1529
1706
  `--dry` maps to BaseIn's `POST /scenarios/:id/replay`, which evaluates the stored logic against the