@basein/runner 0.2.11 → 0.2.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -148,9 +148,14 @@ Calculation is a background job. Poll `show` until `state` leaves `calculating`:
148
148
  | `failed` | see the `error` field | fix and recalculate with `--force` |
149
149
 
150
150
  `409 scenario_exists` means this run already has one. Re-derive it in place — keeping the id and its
151
- replay history — with `bir scenario calc run_… --force`. To change one step instead, without
152
- recalculating, see §9.2; a plan with steps edited by hand needs `--force --discard-edits` to be
153
- rebuilt.
151
+ replay history — with `bir scenario calc run_… --force`. A `run_` id reaches only the run's
152
+ whole-run scenario. From the next version (not yet released), `bir scenario calc scn_… --force`
153
+ does the same by the scenario's own id, a sub-task scenario included; it is the only way to
154
+ recalculate a sub-task scenario from `bir`.
155
+
156
+ To change one step instead, without recalculating, see §9.2. A plan with steps edited by hand is
157
+ not rebuilt unless you add `--discard-edits`, which replaces those edits. They stay in
158
+ `bir scenario edits`, marked replaced, and can no longer be undone.
154
159
 
155
160
  ### 3.1 Check the scenario before you trust it
156
161
 
@@ -251,8 +256,12 @@ nothing. `bir install --replay` does exactly two things beyond a normal install,
251
256
 
252
257
  - registers a first-party MCP server under the key **`bir`** in the project's `.mcp.json` (never
253
258
  `~/.claude.json`), exposing `mcp__bir__run_scenario` — the channel a fully-wrapped scenario's
254
- results come back through — and three tools that only read a scenario (`scenario_show`,
255
- `scenario_edits`, `investigate`; §9.2, not yet released);
259
+ results come back through. It also offers three tools that only read a scenario
260
+ (`scenario_show`, `scenario_edits`, `investigate`) and, only after `bir scenario editing on`,
261
+ three for changing one (`scenario_check` tries a change, `scenario_edit` saves it,
262
+ `scenario_undo` takes it back). §9.2 lists them with their arguments; the editing it describes
263
+ is built but not yet released. From the next version, a call to any of them except
264
+ `run_scenario` is never recorded;
256
265
  - pre-approves that server in `.claude/settings.local.json` (`enabledMcpjsonServers`), because a
257
266
  "No" to Claude Code's approval dialog leaves a direct replay with nowhere to deliver its results.
258
267
 
@@ -469,8 +478,8 @@ first prompt that states the task plainly.
469
478
  | `run_scenario` returns "no scenario is armed" | The model called it on a turn with no plan | Harmless. It happens when the model remembers the tool from an earlier turn |
470
479
  | `recorder.send_failed … error="HTTP 409 … invalid_ticket"` | The execution report's ticket was already redeemed, or belongs to another scenario | Harmless if the body says `duplicate: true`. Otherwise a rollover raced `SessionEnd`; the server books once |
471
480
  | `savedUsd` looks impossible | Pricing drift between the runner's table and the server's | Compare `PRICING_VERSION` on both sides. Design §11.3 documents a known live drift between two existing copies |
472
- | The hand-over note says a step "did not run: its input needs a judgement" | The service marked the step non-deterministic when it calculated the plan: the generated logic could not compute its input from the prompt or from earlier outputs, so the recorded value was a copy (the service's plan-services design, D8 kind 3 — it lives in the BaseIn repository, not this one). `nondeterministicWhy: by_hand` means someone marked it with `bir scenario edit --freeze` | If the code only has a slip, fix the step yourself: `bir scenario show scn_… --step <n>`, write the input logic to a file, then `bir scenario edit scn_… --step <n> --input-logic <file>` — the service checks it against the recording before it saves it (§9.2). A real judgement — a choice with no fixed rule — cannot be written as code: the agent makes that choice and finishes the task; a model step for it is planned (W3.1). The journal's `plan.armed` line lists the step under `stops=` |
473
- | A parked step ran again after an hour, or a plan was recalculated by itself | The service's grace period (D8): a parked step is served with `stop: null` once an hour for a day and a success clears its count; if it keeps failing, or its tool is gone, the next repeat of the prompt is recorded in full and the plan is calculated again from it | Nothing: that is the repair. `bir investigate` says which it is — a step still being retried, a parked step whose repair is due, or a plan being calculated again. To repair now by hand: `bir scenario calc <runId> --force`. A plan with steps edited by hand is not repaired by itself: the parked step stays parked, and `bir investigate` says the repair was skipped. Fix the step (§9.2), or add `--discard-edits` |
481
+ | The hand-over note says a step "did not run: its input needs a judgement" | The service marked the step non-deterministic when it calculated the plan: the generated logic could not compute its input from the prompt or from earlier outputs, so the recorded value was a copy (the service's plan-services design, D8 kind 3 — it lives in the BaseIn repository, not this one). `nondeterministicWhy: by_hand` means someone marked it with `bir scenario edit --freeze` | If the code only has a slip, fix the step yourself: `bir scenario show scn_… --step <n>`, write the input logic to a file, then `bir scenario edit scn_… --step <n> --input-logic <file>` — the service checks it against the recording before it saves it (§9.2, not yet released). Of the edits, only `--unfreeze` lifts a mark made by hand (`by_hand`); a code edit, even a forced one, leaves it in place. A real judgement — a choice with no fixed rule — cannot be written as code: the agent makes that choice and finishes the task; a model step for it is planned (W3.1). The journal's `plan.armed` line lists the step under `stops=` |
482
+ | A parked step ran again after an hour, or a plan was recalculated by itself | The service's grace period (D8): a parked step is served with `stop: null` once an hour for a day and a success clears its count; if it keeps failing, or its tool is gone, the next repeat of the prompt is recorded in full and the plan is calculated again from it | Nothing: that is the repair. `bir investigate` says which it is — a step still being retried, a parked step whose repair is due, or a plan being calculated again. To repair now by hand: `bir scenario calc <runId> --force` for a whole-run plan; from the next version, `bir scenario calc <scnId> --force` works for any plan, a sub-task one included. A plan with steps edited by hand is not repaired by itself: the parked step stays parked, and `bir investigate` says the repair was skipped. Then fix the step (§9.2); or take the edits back with `bir scenario undo`, once per edit, and it is repaired as usual; or recalculate now and drop the edits: `bir scenario calc <scnId> --force --discard-edits` |
474
483
  | `sessionCostUsd` grows every prompt in a session | The per-run usage watermark is missing or not taken | Design §11.4. Reports without a mark must carry `measured: false` |
475
484
  | Everything works, nothing is saved | A replayed turn is deliberately not recorded (design §12) | Correct. The matched run stays canonical and its `iterations` is bumped |
476
485
  | `bir doctor`: the SessionStart hook points at a Node that is gone | The hook names Node by absolute path. That Node was uninstalled, moved, or was a version manager's per-shell copy; the hook then fails on every session, no recorder is started, and runs record Tier 2 | `bir setup` here again — it rewrites the hook to the Node that runs it |
@@ -512,9 +521,13 @@ You see your own data. An admin of the service sees everyone's and may pass
512
521
 
513
522
  ### 9.2 Fix a step — `bir scenario check`, `edit`, `edits`, `undo`
514
523
 
515
- > Built 2026-09-25, **not yet released**. It needs the next `@basein/runner` version *and* the
516
- > service deploy that adds the `/scenarios/:id/steps/…` routes. The design is `editSteps.md` in the
517
- > BaseIn repository.
524
+ > Built 2026-09-25, **not yet released**. It needs the next `@basein/runner` version *and* a
525
+ > service deploy: the `/scenarios/:id/steps/…` routes and the sandbox the service runs step code
526
+ > in. Version 0.2.11 on npm has an early form of these commands. It is not what this section
527
+ > describes. The design is `editSteps.md` in the BaseIn repository.
528
+ >
529
+ > The hiring-year outputs below were printed by the real `bir`, against the real service, on the
530
+ > real recording. Only the ids are shortened, with `…`.
518
531
 
519
532
  `bir investigate` says *what* is wrong. This section is how you fix it. A calculated step is code:
520
533
  its **input logic** says what to send to the tool, and its **output logic** says what to keep from
@@ -530,17 +543,18 @@ bir scenario check <runId|scnId> --step <n> [--input-logic <file|->] [--output-l
530
543
  bir scenario edit <runId|scnId> --step <n> [--input-logic <file|->] [--output-logic <file|->]
531
544
  [--freeze | --unfreeze] [--note "<why>"] [--force --note "<why>"] [--revision <n>]
532
545
  bir scenario edits <runId|scnId>
533
- bir scenario undo <runId|scnId> [--edit <sedit_id>]
534
- bir scenario calc <runId> [--force [--discard-edits]]
546
+ bir scenario undo <runId|scnId> [--edit <sedit_id> | --step <n>] [--force --note "<why>"]
547
+ bir scenario calc <runId|scnId> [--force [--discard-edits]]
535
548
  bir scenario editing on|off|status
536
549
  ```
537
550
 
538
- A `run_` id finds the run's scenario; a `scn_` id (what `bir investigate` prints) is used as it is.
539
- A sub-task scenario has only a `scn_` id. `--step` is the `stepIndex` that `show` prints; it starts
540
- at 0. `--revision <n>` saves only if the plan is still at revision `n` (the `chainRevision` that
541
- `show` printed). `--json` on `check`, `edit`, `edits` or `undo` prints the service's answer
542
- unchanged. Exit codes: `0`
543
- checked OK, saved or undone · `1` refused, not found, or a service error · `2` a wrong command line.
551
+ A `run_` id finds the run's whole-run scenario. A `scn_` id (what `bir investigate` prints) is used
552
+ as it is. A sub-task scenario has only a `scn_` id. When a run has only sub-task scenarios, `bir`
553
+ lists their `scn_` ids instead of guessing. `--step` is the `stepIndex` that `show` prints; it
554
+ starts at 0. `--revision <n>` saves only if the plan is still at revision `n`: the `chainRevision`
555
+ at the top of `show --step`. `--json` on `check`, `edit`, `edits` or `undo` prints the service's
556
+ answer unchanged. Exit codes: `0` checked OK, saved or undone · `1` refused, not found, or a
557
+ service error · `2` a wrong command line.
544
558
 
545
559
  #### What the code is
546
560
 
@@ -551,17 +565,29 @@ Each logic is the *body* of a JavaScript function. It must end in `return { …
551
565
  | input logic (`toolInputLogic`) | `parameters` — this request's values · `respParams` — what earlier steps kept · `intent` | the tool's input, as an object |
552
566
  | output logic (`toolOutputLogic`) | `toolOutput` — the tool's answer, as text · and the same three | the values to keep for later steps, as an object |
553
567
 
554
- When the service runs it, only JavaScript itself is there: `JSON`, `Math`, `Date`, `RegExp` and the
555
- like. There is no `require`, `process`, `fetch` or timer, and code that uses one fails the check.
568
+ The service runs your code in a sandbox (design §13.1). In the sandbox only JavaScript itself
569
+ exists: `JSON`, `Math`, `Date`, `RegExp` and the like. There is no `require`, `process`, `fetch`,
570
+ `Buffer` or timer. A few built-ins are taken away too: `ArrayBuffer`, the typed arrays such as
571
+ `Uint8Array`, `Atomics`, `WebAssembly`, `FinalizationRegistry` and `Symbol.for`. Code that uses
572
+ one of them with the recording's values throws in the check.
573
+
574
+ **But the check only judges what your code returns for the recording's values.** A `fetch` behind
575
+ a `typeof` guard (`if (typeof fetch === "function") …`), or on a branch the recording's values
576
+ never take, is never reached in the check. A `fetch` inside a `try` fails in the check, but the
577
+ `catch` hides the error. Either way the code passes. On your machine the runner runs the same
578
+ code with `new Function`, where `fetch` and `process` do exist. So read every change before it is
579
+ saved — yours and Claude's.
556
580
 
557
581
  **Only a step's code and its mark can be changed.** Not the parameters, not their sample values, not
558
582
  the logic that builds the final answer. Only tool steps: a sub-task call row belongs to the service.
559
583
 
560
584
  #### The fix, step by step
561
585
 
562
- On 2026-09-25 two SQL steps of the hiring-year scenario were marked *needs a judgement*. Their code
563
- wrote `schema + '.' + summary_table`, but the table name already carried its schema. Step 4 is shown
564
- here; step 3 is the same.
586
+ On 2026-09-25 two SQL steps of the hiring-year scenario, 3 and 4, were marked *needs a judgement*.
587
+ Step 4 is shown here; step 3 was fixed the same way. The calculation's code for step 4 wrote
588
+ `schema + '.' + summary_table`, but the table name already carried its schema, so the code did not
589
+ give the recorded call. The calculation then put a copy of the recorded call in its place, and
590
+ marked the step.
565
591
 
566
592
  **1. Read the step.**
567
593
 
@@ -569,21 +595,35 @@ here; step 3 is the same.
569
595
  bir scenario show scn_4a17a126-… --step 4
570
596
  ```
571
597
 
598
+ You should see this (shortened: each `…` line stands for lines left out):
599
+
572
600
  ```
573
601
  {
602
+ "scenarioId": "scn_4a17a126-…",
603
+ "chainRevision": 1,
574
604
  "stepIndex": 4,
605
+ "kind": "tool",
575
606
  "toolName": "mcp__postgres__execute_sql",
576
607
  "toolInputLogic": "return {\"sql\":\"SELECT * FROM employees.hiring_year_summary ORDER BY hire_year;\"};",
608
+ …
577
609
  "nondeterministic": true,
578
610
  "nondeterministicWhy": "input_mismatch",
611
+ "editedAt": null,
612
+ "editForced": false,
613
+ "stop": {
614
+ "kind": "nondeterministic",
615
+ "why": "input_mismatch"
616
+ },
579
617
  "sampleInput": "{\"sql\":\"SELECT * FROM employees.hiring_year_summary ORDER BY hire_year;\"}",
580
618
  …
581
619
  }
582
620
  ```
583
621
 
584
- The stored input logic is a copy of the recorded call. A copy does the old job, on the old table,
585
- whatever the new request says. That is why the step is marked (`nondeterministic: true`), and why
586
- runs hand over to the agent in front of it.
622
+ `scenarioId` and `chainRevision` come first. They belong to the whole scenario, and
623
+ `chainRevision` is the number `edit --revision` takes. The rest is the step. Its input logic is
624
+ that copy: compare it with `sampleInput`. A copy does the old job, on the old table, whatever the
625
+ new request says. That is why the step is marked (`nondeterministic: true`), and why runs hand over
626
+ to the agent in front of it.
587
627
 
588
628
  **2. Write the new code to a file.**
589
629
 
@@ -607,13 +647,14 @@ bir scenario check scn_4a17a126-… --step 4 --input-logic step4.js
607
647
  ```
608
648
  Step 4 (mcp__postgres__execute_sql) — checked against run_67afb010-…
609
649
  input reproduces the recorded call
610
- later no later steps
650
+ later not affected: only the input logic changed
611
651
  mark needs a judgement (input_mismatch) → runs by itself
612
652
  Nothing saved. To save: bir scenario edit scn_4a17a126-… --step 4 --input-logic step4.js
613
653
  ```
614
654
 
615
655
  - `input` — does the code give the recorded call?
616
- - `later` — do the steps after it still give theirs?
656
+ - `later` — do the steps after it still give theirs? Here only the input logic changed. Nothing a
657
+ later step reads has changed, so no later step was judged.
617
658
  - `mark` — what happens to the *needs a judgement* mark. `→ runs by itself` means the mark goes
618
659
  when you save, and runs stop handing over in front of this step.
619
660
 
@@ -628,9 +669,12 @@ bir scenario edit scn_4a17a126-… --step 4 --input-logic step4.js --note "summa
628
669
  Step 4 (mcp__postgres__execute_sql) — checked against run_67afb010-…
629
670
  input reproduces the recorded call
630
671
  mark needs a judgement (input_mismatch) → runs by itself
631
- Saved as sedit_7c1e… (revision 4). Undo: bir scenario undo scn_4a17a126-… --edit sedit_7c1e…
672
+ Saved as sedit_633dba0d-bbbe-4517-ac29-74cc85a3abda (revision 2). Undo: bir scenario undo scn_4a17a126-… --edit sedit_633dba0d-bbbe-4517-ac29-74cc85a3abda
632
673
  ```
633
674
 
675
+ `edit` prints fewer lines than `check`. It always prints `input`, and the other lines only when
676
+ something moved or broke. Step 3 was then saved the same way, as revision 3.
677
+
634
678
  **5. Try it without real tools.** A dry replay (§3.1) runs the new code over the recorded outputs:
635
679
  `bir scenario replay scn_4a17a126-… --prompt "…" --dry`. The next real prompt uses the new code.
636
680
  There is nothing to restart.
@@ -642,40 +686,96 @@ parameter values, and with what the earlier steps kept from their recorded answe
642
686
 
643
687
  | It asks | It refuses when |
644
688
  |---|---|
645
- | Does the code give the recorded call? Spaces inside strings do not count | it gives something else (`differs`), or it crashes (`throws`) |
646
- | Is it a copy? It changes one parameter, or one earlier output, at a time | the call never moves: the code returns the recorded call whatever the request says |
647
- | Do the later steps still give their calls? | a later step that gave its recorded call before does not any more |
648
- | Does the final answer still get its values? | it did before, and does not after |
689
+ | Does the code give the recorded call? Extra spaces inside strings, and the order of keys, do not count | it gives something else (`differs`), or it crashes (`throws`) |
690
+ | Is it a copy? It changes one parameter, or one earlier output, at a time — only the ones the code reads | the call never moves: the code returns the recorded call whatever the request says |
691
+ | Do the later steps still give their calls? Asked only when the **output** logic changed, and only of later steps that are not marked and can be checked | a later step that gave its recorded call before does not any more |
692
+ | Does the final answer still get its values? Asked only when the **output** logic changed | it did before, and does not after |
693
+ | Can the mark go? | you asked to unmark the step (`--unfreeze`), and its input does not reproduce |
649
694
 
650
695
  A recorded input of `{}` is never a copy. A recorded input that is plain text, not an object, is
651
696
  `not_checked`.
652
697
 
653
- A refused change says where it differs:
698
+ **The check is kept small.** On the service it runs in a worker thread of its own, never in the
699
+ thread that answers requests. By default it may use 5 seconds of sandbox time
700
+ (`EDIT_CHECK_BUDGET_MS`, 5000 ms). Past that it stops, and the change is refused as *too costly to
701
+ check*. The reason `bir` prints is the service's own sentence:
702
+
703
+ ```
704
+ too costly to check: the check used up its 5000 ms of sandbox time (EDIT_CHECK_BUDGET_MS) before it finished.
705
+ ```
706
+
707
+ One run of your code may take 1 second (`LOGIC_TIMEOUT_MS`) and 64 MB of memory
708
+ (`LOGIC_MEMORY_MB`), by default. Code that ran out of time or memory, or stopped the sandbox twice,
709
+ is held back for 10 minutes: every run of that same code fails at once, with the same message.
710
+
711
+ A refused change says where it differs. Here `wrong.js` adds the schema a second time:
654
712
 
655
713
  ```bash
656
- bir scenario edit scn_4a17a126-… --step 4 --input-logic wrong.js
714
+ bir scenario check scn_4a17a126-… --step 4 --input-logic wrong.js
715
+ ```
716
+
717
+ ```
718
+ Step 4 (mcp__postgres__execute_sql) — checked against run_67afb010-…
719
+ input differs from the recorded call at character 32
720
+ computed … FROM employees.employees.hiring_year_summary ORDER BY h…
721
+ recorded … FROM employees.hiring_year_summary ORDER BY hire_year;"…
722
+ later not affected: only the input logic changed
723
+ mark needs a judgement (input_mismatch) → stays marked
724
+ Nothing saved, and `bir scenario edit` would refuse it: the change does not reproduce the recording.
725
+ Fix the code and check again. If the recording itself was wrong, edit can save it on purpose: add --force --note "why".
657
726
  ```
658
727
 
728
+ `bir scenario edit` with the same file refuses it, and saves nothing:
729
+
659
730
  ```
660
731
  Step 4 (mcp__postgres__execute_sql) — checked against run_67afb010-…
661
- input differs from the recorded call at character 14
662
- computed …FROM employees.employees.hiring_year_summary ORDER…
663
- recorded …FROM employees.hiring_year_summary ORDER BY hire_…
732
+ input differs from the recorded call at character 32
733
+ computed … FROM employees.employees.hiring_year_summary ORDER BY h…
734
+ recorded … FROM employees.hiring_year_summary ORDER BY hire_year;"…
664
735
  Not saved: the change does not reproduce the recording.
665
736
  Fix the code and try again, or save it on purpose: add --force --note "why".
666
737
  ```
667
738
 
668
- A pasted value is refused the same way:
669
- `input is a copy: it returns the recorded call whatever the parameters say`.
739
+ A pasted value is refused as a copy. Here `pasted.js` returns the recorded SQL as fixed text:
740
+
741
+ ```bash
742
+ bir scenario check scn_4a17a126-… --step 4 --input-logic pasted.js
743
+ ```
744
+
745
+ ```
746
+ Step 4 (mcp__postgres__execute_sql) — checked against run_67afb010-…
747
+ input is a copy: it returns the recorded call whatever the parameters say
748
+ later not affected: only the input logic changed
749
+ mark needs a judgement (input_mismatch) → stays marked
750
+ Nothing saved, and `bir scenario edit` would refuse it: the input logic is a copy: it returns the recorded call whatever the parameters and earlier outputs say.
751
+ Fix the code and check again. If the recording itself was wrong, edit can save it on purpose: add --force --note "why".
752
+ ```
753
+
754
+ Any other refusal prints the service's own reason, one line for each problem.
755
+
756
+ Code and a mark the step already has change nothing. `check` then says
757
+ `Nothing would change: the step already has this code and mark.` (exit 0), and `edit` refuses:
758
+
759
+ ```bash
760
+ bir scenario edit scn_4a17a126-… --step 4 --input-logic step4.js
761
+ ```
762
+
763
+ ```
764
+ Nothing to change: the step already has this code and this mark.
765
+ ```
670
766
 
671
767
  #### `--force --note`: save it on purpose
672
768
 
673
769
  **Use `--force` only when the recording itself was wrong** — for example, the recorded run sent a
674
770
  call with a mistake you do not want repeated. Then no correct code can reproduce it.
675
771
 
676
- - `--force` needs a `--note` that says why (500 characters at most). Without one: `note_required`.
677
- - A forced save removes the mark, unless you also pass `--freeze`.
678
- - It is saved as *forced*, and `bir investigate` keeps saying so, as a `warn`.
772
+ - `--force` needs a `--note` that says why (500 characters at most). Without one, `bir` stops
773
+ before it sends anything (`note_required`).
774
+ - **When the check found any problem, a forced save is recorded as forced.** `bir` says so under
775
+ the `Saved as …` line and repeats what the check said. `bir investigate` keeps reporting the step
776
+ as a `warn`.
777
+ - A forced save removes a mark **the calculation** set, unless you also pass `--freeze`. A mark
778
+ made **by hand** (`--freeze`) stays. A save lifts it only with `--unfreeze`.
679
779
  - A step after a sub-task call row can only be saved with `--force`: the check cannot rebuild the
680
780
  outputs that step reads.
681
781
 
@@ -683,33 +783,98 @@ call with a mistake you do not want repeated. Then no correct code can reproduce
683
783
 
684
784
  - `--freeze` marks the step *needs a judgement* (`nondeterministicWhy: by_hand`). Runs then hand
685
785
  over to the agent in front of it. No check blocks this: a mark only makes replay more careful.
686
- - `--unfreeze` asks to remove a mark. The check decides: the step's input must reproduce.
786
+ - `--unfreeze` asks to remove a mark. The check decides: the step's input must reproduce. With
787
+ `--force --note "why"` the mark goes anyway, and the edit is recorded as forced.
687
788
  `bir scenario check … --unfreeze` asks the same question without saving.
789
+ - **A mark made by hand is lifted only when someone asks.** A code edit does not lift it, even one
790
+ that reproduces the recording, and neither does a forced save. What lifts it: `--unfreeze`,
791
+ undoing the edit that made it, or `bir scenario calc … --force --discard-edits`.
688
792
 
689
793
  #### The history, and undo
690
794
 
795
+ After both steps were saved:
796
+
691
797
  ```bash
692
798
  bir scenario edits scn_4a17a126-…
693
799
  ```
694
800
 
695
801
  ```
696
- sedit_7c1e… 2026-09-25 09:12:04Z step 4 edit revision 3 → 4 "summary_table already carries its schema"
697
- input logic changed; needs a judgement (input_mismatch) → runs by itself
802
+ sedit_02cb1210-1938-4902-9395-5a82b321276d 2026-09-25 11:10:59Z step 3 edit revision 2 → 3 "use the settings, not the recorded names"
803
+ input logic changed; needs a judgement (input_mismatch) → runs by itself
804
+ sedit_633dba0d-bbbe-4517-ac29-74cc85a3abda 2026-09-25 11:10:59Z step 4 edit revision 1 → 2 "summary_table already carries its schema"
805
+ input logic changed; needs a judgement (input_mismatch) → runs by itself
698
806
  ```
699
807
 
808
+ The newest entry is first. Each id is printed in full, because it is what `undo --edit` takes.
809
+
810
+ Take back the newest edit:
811
+
700
812
  ```bash
701
813
  bir scenario undo scn_4a17a126-…
702
814
  ```
703
815
 
704
816
  ```
705
- Step 4 is back as it was before sedit_7c1e… (revision 5). Redo: bir scenario undo scn_4a17a126-… --edit sedit_9a02…
817
+ Step 3 (mcp__postgres__execute_sql) — checked against run_67afb010-…
818
+ input back as it was before that edit
819
+ mark runs by itself → needs a judgement (input_mismatch)
820
+ Step 3 is back as it was before sedit_02cb1210-1938-4902-9395-5a82b321276d (revision 4). Redo: bir scenario undo scn_4a17a126-… --edit sedit_1bcba735-a526-4a1c-a890-261addf2d82b
821
+ One further back: bir scenario undo scn_4a17a126-…
706
822
  ```
707
823
 
708
- - `undo` puts **one step** back as it was before its newest edit: its code and its mark. Without
709
- `--edit` it undoes the newest edit that can be undone.
710
- - The undo is a new line in the history. Undo the undo to redo.
711
- - Only the newest edit of a step can be undone. For an older one you get `not_latest_edit` and the
712
- id of the newer edit: undo that one first.
824
+ The same command again goes one further back:
825
+
826
+ ```bash
827
+ bir scenario undo scn_4a17a126-…
828
+ ```
829
+
830
+ ```
831
+ Step 4 (mcp__postgres__execute_sql) — checked against run_67afb010-…
832
+ input back as it was before that edit
833
+ mark runs by itself → needs a judgement (input_mismatch)
834
+ Step 4 is back as it was before sedit_633dba0d-bbbe-4517-ac29-74cc85a3abda (revision 5). Redo: bir scenario undo scn_4a17a126-… --edit sedit_8d3da392-1d96-4b2e-8c5d-34f8ab668b53
835
+ One further back: bir scenario undo scn_4a17a126-…
836
+ ```
837
+
838
+ Now the history shows the two undos, and both edits as undone:
839
+
840
+ ```bash
841
+ bir scenario edits scn_4a17a126-…
842
+ ```
843
+
844
+ ```
845
+ sedit_8d3da392-1d96-4b2e-8c5d-34f8ab668b53 2026-09-25 11:11:00Z step 4 revert revision 4 → 5
846
+ undoes sedit_633dba0d-bbbe-4517-ac29-74cc85a3abda; input logic changed; runs by itself → needs a judgement (input_mismatch)
847
+ sedit_1bcba735-a526-4a1c-a890-261addf2d82b 2026-09-25 11:11:00Z step 3 revert revision 3 → 4
848
+ undoes sedit_02cb1210-1938-4902-9395-5a82b321276d; input logic changed; runs by itself → needs a judgement (input_mismatch)
849
+ sedit_02cb1210-1938-4902-9395-5a82b321276d 2026-09-25 11:10:59Z step 3 edit revision 2 → 3 "use the settings, not the recorded names"
850
+ input logic changed; needs a judgement (input_mismatch) → runs by itself; undone 2026-09-25 11:11:00Z
851
+ sedit_633dba0d-bbbe-4517-ac29-74cc85a3abda 2026-09-25 11:10:59Z step 4 edit revision 1 → 2 "summary_table already carries its schema"
852
+ input logic changed; needs a judgement (input_mismatch) → runs by itself; undone 2026-09-25 11:11:00Z
853
+ ```
854
+
855
+ Step 4 is the calculation's code again: `bir scenario show scn_4a17a126-… --step 4` now prints
856
+ `"chainRevision": 5`, `"editedAt": null` and the mark `"nondeterministicWhy": "input_mismatch"`.
857
+
858
+ How undo works:
859
+
860
+ - **Undo walks back.** `bir scenario undo <id>` takes back the newest edit that can be undone. Its
861
+ step goes back as it was before that edit: its code and its mark. Run it again to go one further
862
+ back. Keep going, and each step gets back to the calculation's code, unless an undo is refused
863
+ (below).
864
+ - **`--step <n>` limits it to one step.** `bir scenario undo scn_… --step 4` takes back step 4's
865
+ newest edit, and nothing else.
866
+ - `--edit <sedit_id>` takes back that one entry.
867
+ - An undo is a new line in the history (`revert`). **To redo, undo the undo** by its id. The undo
868
+ prints that command after `Redo:`.
869
+ - **An undo is checked too:** for the later steps and for the final answer, like a save (so only
870
+ when the undo changes the step's output logic). The step itself is not judged: it goes back
871
+ exactly as it was. If the undo would break a later step or the answer, it is refused
872
+ (`undo_refused`). `bir` prints `Not undone:` with the reason, then
873
+ `To undo it anyway, on purpose: add --force --note "why".` An undo forced past a refusal is
874
+ recorded as forced.
875
+ - Once every hand edit of a plan is taken back, the service repairs and rebuilds it as usual again.
876
+ An undo forced past a refusal is the exception: it leaves its step counted as edited and
877
+ forced, so `bir investigate` keeps warning about it.
713
878
 
714
879
  #### What a save changes
715
880
 
@@ -717,8 +882,9 @@ Step 4 is back as it was before sedit_7c1e… (revision 5). Redo: bir scenario u
717
882
  turn that is already running keeps the old plan until it ends.
718
883
  - **The revision goes up by one, and the step's failure count is cleared.** Reports from turns that
719
884
  started before the change do not count against the new code. An undo does the same.
720
- - **A saved step runs unattended, exactly like calculated code** (§5.1). Read your change as
721
- carefully as you read a calculated plan, and dry-replay it (§3.1).
885
+ - **A saved step runs unattended, exactly like calculated code** (§5.1), on this machine, with
886
+ `new Function`. The service's sandbox does not protect this machine. Read your change as carefully
887
+ as you read a calculated plan, and dry-replay it (§3.1).
722
888
  - **Edits are protected.** A recalculation rebuilds every step from the recording, and would throw
723
889
  your fix away. So `calc --force` refuses:
724
890
 
@@ -732,22 +898,31 @@ Not recalculated: steps 3, 4 were edited by hand, and a recalculation rebuilds e
732
898
  (the edits stay in `bir scenario edits`, marked replaced)
733
899
  ```
734
900
 
735
- The console's *Recalculate* is refused too. The service's own rebuilds — the repair of a parked
736
- step (§9) and the adoption of a sub-task — skip a plan with hand edits, and `bir investigate` says
737
- so. After `--discard-edits` the old edits stay readable, but they cannot be undone
738
- (`edit_replaced`).
901
+ A `run_` id reaches only the run's whole-run scenario. `bir scenario calc <scnId> --force`
902
+ recalculates any scenario in place, by its own id, a sub-task scenario included. Add
903
+ `--discard-edits` when it has hand edits. A `scn_` id without `--force` is refused before anything
904
+ is sent: a scenario that exists can only be recalculated.
905
+
906
+ The console's *Recalculate* button (on a sub-task's scenario) is refused too while a plan has hand
907
+ edits; the console does not discard them. The service's own rebuilds — the repair of a parked step
908
+ (§9) and the adoption of a sub-task — skip a plan with hand edits. `bir investigate` says that the
909
+ service does not rebuild the plan on its own, and warns when it skipped a repair
910
+ (`repair_skipped_edits`). After `--discard-edits` the old edits stay readable, but they cannot be
911
+ undone (`edit_replaced`).
739
912
 
740
913
  #### What `bir investigate` says
741
914
 
742
- It names the fix, and it reports edits. Here, after step 4 was saved and before step 3 was:
915
+ It reports every edited step, and names the command that takes it back. After both steps were
916
+ saved, its service findings began like this:
743
917
 
744
918
  ```
745
- 1. [warn] Step 3 (mcp__postgres__execute_sql) needs a judgement, so runs hand over in front of it.
746
- fix: Recalculate once (`bir scenario calc run_… --force`) … Or fix it yourself: read it with
747
- `bir scenario show scn_… --step 3`, write the input logic to a file, and run
748
- `bir scenario edit scn_… --step 3 --input-logic <file>`; the service checks it against the recording.
749
- 2. [info] Step 4 (mcp__postgres__execute_sql) was edited by hand on 2026-09-25 (revision 4).
750
- fix: To see or undo it: `bir scenario edits scn_…`. A recalculation keeps it unless you pass --discard-edits.
919
+ Service findings
920
+ 1. [info] Step 3 (mcp__postgres__execute_sql) was edited by hand on 2026-09-25; the chain is at revision 3.
921
+ why: Its code or its mark was changed by hand rather than by the calculation, and the change was checked against the recording before it was saved. The service does not rebuild a plan with hand edits on its own.
922
+ fix: To see it: `bir scenario edits scn_4a17a126-…`; to take it back: `bir scenario undo scn_4a17a126-… --step 3`. A recalculation will not replace it unless you pass --discard-edits.
923
+ 2. [info] Step 4 (mcp__postgres__execute_sql) was edited by hand on 2026-09-25; the chain is at revision 3.
924
+ why: Its code or its mark was changed by hand rather than by the calculation, and the change was checked against the recording before it was saved. The service does not rebuild a plan with hand edits on its own.
925
+ fix: To see it: `bir scenario edits scn_4a17a126-…`; to take it back: `bir scenario undo scn_4a17a126-… --step 4`. A recalculation will not replace it unless you pass --discard-edits.
751
926
  ```
752
927
 
753
928
  A forced edit is a `warn`, not an `info`. So is a repair the service skipped because of hand edits.
@@ -769,20 +944,43 @@ bir scenario editing off # back to the tools that only read
769
944
  bir scenario editing status # what is stored for this project
770
945
  ```
771
946
 
772
- | Tool (`mcp__bir__…`) | Offered | Runs |
773
- |---|---|---|
774
- | `scenario_show` | always | `bir scenario show` |
775
- | `scenario_edits` | always | `bir scenario edits` |
776
- | `investigate` | always | `bir investigate` |
777
- | `scenario_check` | after `bir scenario editing on` | `bir scenario check` |
778
- | `scenario_edit` | after `bir scenario editing on` | `bir scenario edit` |
779
- | `scenario_undo` | after `bir scenario editing on` | `bir scenario undo` |
780
-
781
- Why it is off by default: the `bir` server runs in every session of every project it is installed
782
- in, a fleet included. A plan must not change because some agent there decided to change it. The
783
- switch is kept in `~/.baseinstrunner/installed.json`, next to the replay switches. It only decides
784
- which tools the `bir` server offers; the `bir scenario` commands work everywhere. **Restart the
785
- Claude Code session after you switch it** — a running session does not see the change.
947
+ The tools, as the `bir` server defines them. **Bold** arguments are required.
948
+
949
+ | Tool (`mcp__bir__…`) | Offered | Arguments | Runs |
950
+ |---|---|---|---|
951
+ | `run_scenario` | always | none | the plan armed for this turn (§6.1) |
952
+ | `scenario_show` | always | **`target`**, `step` | `bir scenario show` |
953
+ | `scenario_edits` | always | **`target`** | `bir scenario edits` |
954
+ | `investigate` | always | `id` — a `run_`, `scn_` or `sexec_` id; without it, the newest turn here | `bir investigate` |
955
+ | `scenario_check` | after `bir scenario editing on` | **`target`**, **`step`**, `inputLogic`, `outputLogic`, `unfreeze` | `bir scenario check` |
956
+ | `scenario_edit` | after `bir scenario editing on` | **`target`**, **`step`**, `inputLogic`, `outputLogic`, `freeze`, `unfreeze`, `force`, `note`, `revision` | `bir scenario edit` |
957
+ | `scenario_undo` | after `bir scenario editing on` | **`target`**, `edit` or `step` (not both), `force`, `note` | `bir scenario undo` |
958
+
959
+ - `target` is a `scn_` id, or the `run_` id of a whole-run scenario. `step` is the `stepIndex`.
960
+ - `inputLogic` and `outputLogic` are the code itself, as text. The server writes them to temporary
961
+ files for `bir`, and removes the files afterwards.
962
+ - `revision` is the `chainRevision` at the top of `scenario_show` with `step`.
963
+ - `force` needs `note`, as on the command line.
964
+ - Each tool runs this package's own `bir` with `--json`. A refusal comes back as an error result
965
+ that carries the check's report, so the model can read why.
966
+ - When the session ends, or you press Esc during a call, that call's `bir` is stopped and its
967
+ temporary files are removed. A request the service had already received may still finish there;
968
+ `bir scenario edits` shows it.
969
+
970
+ Why changing is off by default: the `bir` server runs in every session of every project it is
971
+ installed in, a fleet included. A plan must not change because some agent there decided to change
972
+ it. The switch is kept in `~/.baseinstrunner/installed.json`, next to the replay switches. It only
973
+ decides which tools the `bir` server offers; the `bir scenario` commands work everywhere. The server
974
+ reads the switch when Claude Code lists its tools, and again on every call. **Restart the Claude
975
+ Code session after you switch it** — a running session does not see the change.
976
+
977
+ **`bir`'s own calls are never recorded.** A session that fixes a plan must not become a plan that
978
+ edits plans when it is replayed. So these calls are never recorded as steps, and never count as the
979
+ model leaving a plan: the `bir` server's tools other than `run_scenario`, and a `Bash` or
980
+ `PowerShell` command made only of `bir` commands (a plain `cd` may come with them).
981
+ `npm test && bir investigate` is still recorded: the `npm test` half is real work. So is writing
982
+ the code file, which is an ordinary step. Under `BIR_VERBOSE=1` a skipped call shows as
983
+ `tool.pre.skipped … why="host housekeeping"`.
786
984
 
787
985
  **Do this.** In the Claude Code session, in the project folder, paste:
788
986
 
@@ -794,9 +992,9 @@ words. Do not save anything until I say yes. Then save it with
794
992
  `bir scenario edit` and a --note that says why. Never use --force.
795
993
  ```
796
994
 
797
- For the hiring-year case, Claude's report would be: *step 4 was marked because its code added the
798
- schema twice; the new code builds the query from `summary_table`; the check says it reproduces the
799
- recording, and the step will run by itself.*
995
+ For the hiring-year case, Claude's report would be: *step 4's code is a copy of the recorded query,
996
+ so it was marked; the new code builds the query from `summary_table`; the check says it reproduces
997
+ the recording, and the step will run by itself.*
800
998
 
801
999
  #### When it does not work
802
1000
 
@@ -804,14 +1002,22 @@ recording, and the step will run by itself.*
804
1002
  |---|---|---|
805
1003
  | `input differs from the recorded call at character N` | The code gives a different call from the recorded one | Read the `computed` and `recorded` lines, fix the code, check again |
806
1004
  | `input is a copy` | The code returns the recorded call whatever the request says — usually a pasted value | Build the call from `parameters.…` or `respParams.…` |
807
- | `input throws` | The code crashed; the message is shown | Fix it. `require`, `process`, `fetch` and timers do not exist in the check |
1005
+ | `input throws` | The code crashed; the message is shown | Fix it. `require`, `process`, `fetch`, `Buffer`, timers, `ArrayBuffer`, the typed arrays and `Symbol.for` do not exist in the check |
1006
+ | `… did not finish in N ms`, `… ran out of memory (the limit is N MB)`, or `… stopped the logic sandbox before it finished` | The code went past the sandbox's time or memory limit (`LOGIC_TIMEOUT_MS`, `LOGIC_MEMORY_MB` on the service), or stopped the sandbox twice. That same code is then refused at once for 10 minutes | Fix the loop or the size. Changed code is new code, and is checked again |
808
1007
  | A `later` step no longer reproduces | The change broke a step after it — usually output logic that stopped keeping a value a later step reads | Keep every key the later steps read |
1008
+ | `too costly to check` | The check used up its 5 seconds of sandbox time (`EDIT_CHECK_BUDGET_MS`). Usually code that reads or keeps a great many values, so the check has to run it very many times | Make the code cheaper to run, then check again |
1009
+ | `Nothing to change: the step already has this code and this mark.` | The code and the mark you gave are what the step already has (`nothing_to_change`) | Nothing to do. `check` says `Nothing would change` for the same case |
809
1010
  | `not_a_tool_step` | The index is a sub-task call row, not a tool step | Call rows cannot be edited; the service writes them |
810
1011
  | `note_required` | `--force` without `--note` | Add `--note "why"` |
811
1012
  | `revision_changed` | The plan changed since you read it: another edit, or a recalculation | `show` it again, check again, save again |
812
1013
  | `not_ready` or `scenario_calculating` | The scenario is not `ready`, or a recalculation is running | Wait for it to finish |
813
- | `not_latest_edit` | A newer edit of the same step follows the one you named | Undo the newer one first; the answer names it |
1014
+ | `undo_refused` | The undo would stop a later step from giving its recorded call, or leave the final answer without its values | Fix that later step first. Or undo it on purpose: `--force --note "why"` |
1015
+ | `not_latest_edit` | The step has changed since the edit you named: a newer entry changed it | `bir` prints two commands: undo the newer one first, then yours |
1016
+ | `already_undone` | That edit was undone already | To redo it, undo the undo. `bir` prints the command |
814
1017
  | `edit_replaced` | A recalculation replaced the plan after that edit | Nothing to undo; the edit is history now |
1018
+ | `logic_sandbox_unavailable` | The service could not run step code just now (HTTP 503). Nothing was checked, saved or undone | Try again in a minute |
1019
+ | `invalid_input` | The service found the request invalid; `bir` prints its details. For example, a logic body longer than 20 000 characters, or a note longer than 500 | Shorten it |
1020
+ | `payload_too_large` | The whole request is larger than the service accepts (HTTP 413, over 256 kB) | A logic body may be at most 20 000 characters, and a note 500 |
815
1021
  | `rate_limited` | More checks, edits and undos in a minute than the service allows (`SCENARIO_EDITS_PER_MINUTE`, 30 by default) | Wait a minute |
816
1022
  | `not found` for a scenario you can see | It is not yours. **Only the owner can change a scenario** — an admin gets `not found` too | Ask the owner |
817
1023
  | The `scenario_check`, `scenario_edit` and `scenario_undo` tools are missing | `bir scenario editing on` was not run in this project, or the session started before it | Run it here, then restart the session |
@@ -821,7 +1027,8 @@ recording, and the step will run by itself.*
821
1027
  ## 10. What is built
822
1028
 
823
1029
  All of it. Phases R0–R7 of [calculatedReplay.md](calculatedReplay.md) §17 are implemented and
824
- tested; everything this guide describes is code you can run.
1030
+ tested; everything this guide describes is code you can run. §9.2 is built and tested too, but not
1031
+ yet released.
825
1032
 
826
1033
  | Phase | What landed | Where |
827
1034
  |---|---|---|
@@ -833,9 +1040,10 @@ tested; everything this guide describes is code you can run.
833
1040
  | **R5** | The `bir` MCP server, `/scenario/run`, `--replay` | [bin/bir-scenario.ts](../src/bin/bir-scenario.ts), [bin/bir.ts](../src/bin/bir.ts) |
834
1041
  | **R6** | Divergence, source-run fallback, `bir replay` | `diverge()` in [controller.ts](../src/replay/controller.ts), [source-run.ts](../src/replay/source-run.ts) |
835
1042
  | **R7** | Usage watermark, reporting, ticket, doctor | [control/transcript.ts](../src/control/transcript.ts), [remote-recorder.ts](../src/record/remote-recorder.ts) |
1043
+ | **§9.2** *(not yet released)* | `bir scenario show/check/edit/edits/undo/calc/editing`, the `bir` server's scenario tools, `bir` calls never recorded (`editSteps.md` D13) | [bin/scenario-edit.ts](../src/bin/scenario-edit.ts), [bin/bir-scenario.ts](../src/bin/bir-scenario.ts), [record/housekeeping.ts](../src/record/housekeeping.ts); tests: [scenario-edit.test.ts](../test/scenario-edit.test.ts), [bir-scenario.test.ts](../test/bir-scenario.test.ts), [housekeeping.test.ts](../test/housekeeping.test.ts), [replay-server.test.ts](../test/replay-server.test.ts) |
836
1044
 
837
1045
  ```bash
838
- npm test # ~400 tests, incl. test/replay.test.ts and test/replay-server.test.ts
1046
+ npm test # ~450 tests, incl. test/replay.test.ts and test/replay-server.test.ts
839
1047
  npm run build && npm run pretest && npm run test:smoke # the smoke test needs the test build too
840
1048
  ```
841
1049