@basein/runner 0.3.8 → 0.3.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +46 -5
- package/dist/auth/client.d.ts +15 -0
- package/dist/auth/client.js +34 -12
- package/dist/bin/bir-hooks.d.ts +1 -1
- package/dist/bin/bir-hooks.js +1 -1
- package/dist/bin/bir.d.ts +2 -0
- package/dist/bin/bir.js +60 -6
- package/dist/bin/investigate.d.ts +5 -0
- package/dist/bin/investigate.js +117 -15
- package/dist/bin/setup.d.ts +12 -0
- package/dist/bin/setup.js +185 -56
- package/dist/bin/update.d.ts +81 -0
- package/dist/bin/update.js +194 -0
- package/dist/config/adapters/claude-code.d.ts +17 -0
- package/dist/config/adapters/claude-code.js +28 -1
- package/dist/config/generate.d.ts +14 -0
- package/dist/control/daemon.d.ts +26 -1
- package/dist/control/daemon.js +53 -3
- package/dist/control/pin-guard.d.ts +6 -1
- package/dist/control/pin-guard.js +15 -1
- package/dist/control/recorder-keeper.js +2 -1
- package/dist/control/server.d.ts +200 -2
- package/dist/control/server.js +741 -48
- package/dist/control/transcript.d.ts +4 -0
- package/dist/control/transcript.js +4 -0
- package/dist/proxy/session.js +1 -0
- package/dist/record/housekeeping.d.ts +7 -1
- package/dist/record/housekeeping.js +11 -3
- package/dist/record/recorder.d.ts +34 -0
- package/dist/record/remote-recorder.d.ts +14 -0
- package/dist/record/remote-recorder.js +107 -13
- package/dist/record/switchable-recorder.js +1 -0
- package/dist/record/truncate.d.ts +11 -0
- package/dist/record/truncate.js +11 -0
- package/dist/replay/controller.d.ts +146 -4
- package/dist/replay/controller.js +340 -31
- package/dist/replay/derive.js +2 -0
- package/dist/replay/handover.d.ts +13 -1
- package/dist/replay/handover.js +19 -0
- package/dist/replay/plan.d.ts +6 -0
- package/dist/replay/plan.js +13 -3
- package/dist/replay/safety.d.ts +15 -4
- package/dist/replay/safety.js +19 -6
- package/dist/replay/source-run.js +2 -1
- package/dist/replay/types.d.ts +16 -0
- package/dist/util/service-headers.d.ts +24 -0
- package/dist/util/service-headers.js +47 -0
- package/docs/BaseInstRunner.md +115 -1
- package/docs/calculatedReplay.md +239 -27
- package/docs/calculatedReplayGuide.md +134 -12
- package/docs/installRun.md +91 -10
- package/docs/quickstart.md +64 -9
- package/package.json +1 -1
- package/production.md +81 -16
- package/scripts/install.ps1 +25 -1
|
@@ -5,7 +5,8 @@
|
|
|
5
5
|
>
|
|
6
6
|
> **Read this first.** Replay executes tool calls with arguments a scenario computed, and it does so
|
|
7
7
|
> **without the permission prompts you would normally see** (design §13.2). `bir setup` (§1) turns it
|
|
8
|
-
> on. Read §5 before you leave it on; `bir replay off` turns it off, and
|
|
8
|
+
> on. Read §5 before you leave it on; `bir replay off`, then `bir up --restart`, turns it off, and
|
|
9
|
+
> §2–§4 work either way.
|
|
9
10
|
|
|
10
11
|
---
|
|
11
12
|
|
|
@@ -40,6 +41,11 @@ Tier 1 is not optional for replay. A match is a match on **the prompt**, and a s
|
|
|
40
41
|
sees one (design §1.1). If `bir doctor` says `Recording tier : standalone (Tier 2)`, replay cannot
|
|
41
42
|
arm, and that is the honest ceiling rather than a bug.
|
|
42
43
|
|
|
44
|
+
The one exception is a folder set up for a host that is not Claude Code, today a LangChain
|
|
45
|
+
application (`bir setup --host langchain`). Nothing of Claude Code is wired there: the
|
|
46
|
+
application's adapter reports every call to the recorder, and `bir doctor` says
|
|
47
|
+
`Host : langchain (no Claude Code hooks are needed here)`. §6.5 has the short version.
|
|
48
|
+
|
|
43
49
|
One line, pasted into a terminal opened in the project you start Claude Code in. The console's *Set
|
|
44
50
|
up the runner* page shows it with a one-time setup token filled in. Pasted in the home folder or a
|
|
45
51
|
drive root it stops before downloading anything — `cd` to the project and paste the same line again:
|
|
@@ -255,6 +261,9 @@ keeps its permission prompt, as it would without the runner (`bir doctor` says s
|
|
|
255
261
|
`BIR_CORRELATION_DECISION=allow` where the recorder starts is the explicit opt-in to approve them
|
|
256
262
|
all.
|
|
257
263
|
|
|
264
|
+
All of this is about Claude Code. In a LangChain application the runner runs no step itself,
|
|
265
|
+
whatever a tool is named, and a call the model makes is never given a plan's arguments: §6.5.
|
|
266
|
+
|
|
258
267
|
### 5.2 Enable it
|
|
259
268
|
|
|
260
269
|
`bir setup` already did this: it installed the scenario server in the local scope, which Claude
|
|
@@ -353,13 +362,13 @@ so there is exactly one switch and it cannot get out of step with itself.
|
|
|
353
362
|
| `BIR_DERIVE_MODEL` | `claude-haiku-4-5-20251001` | The derivation model, when this machine does the reading |
|
|
354
363
|
| `BIR_MATCH_BUDGET_MS` | `3000` | How long the prompt hook waits for a match before giving up. After a match slower than `BIR_MATCH_SLOW_MS` (`2000`), prompts stop waiting for one for 5 minutes, doubling up to 30 while the service stays slow |
|
|
355
364
|
| `BIR_DERIVE_BUDGET_MS` | `8000` | How long `run_scenario` waits for parameters |
|
|
356
|
-
| `BIR_HOOK_DERIVE_BUDGET_MS` | `
|
|
365
|
+
| `BIR_HOOK_DERIVE_BUDGET_MS` | `8000` | How long a hook waits for them: the prompt, for a scenario with a target, and the first `PreToolUse` (never longer than `BIR_DERIVE_BUDGET_MS`) |
|
|
357
366
|
| `BIR_PRETOOL_REPLAY_BUDGET_MS` | `9000` | All replay work inside one `PreToolUse`, a divergence's steps included. At most 10 000 |
|
|
358
367
|
| `BIR_REPLAY_PLAN_BUDGET_MS` | `100000` | Whole-plan ceiling for direct execution; `BIR_REPLAY_BUDGET_MS`, where set, overrides it. Past about 110 000 a long run may be moved to the background by Claude Code. Derivation plus plan is held under 270 s |
|
|
359
368
|
| `BIR_STEP_TIMEOUT_MS` | `60000` | Per-step ceiling for one direct `tools/call`, clipped to what the plan has left |
|
|
360
369
|
| `BIR_LOGIC_TIMEOUT_MS` | `1000` | How long one scenario logic body may run before it is stopped. At most 10 000 |
|
|
361
370
|
| `BIR_REPLAY_RECORDED_OUTPUTS` | *(unset)* | `1` lets a step that could not be sent (a built-in off `BIR_BUILTIN_TOOLS`, no proxy, a server off the allowlist) stand on its recorded output instead of stopping the plan. Never for a timeout or an upstream error |
|
|
362
|
-
| `BIR_BUILTIN_TOOLS` | `Read,Glob,Grep,Edit,Write,Bash` | Built-ins the runner executes itself during a replay, in the session's own project directory and confined to it. A chain whose only non-MCP steps are on this list arms `direct`. A runner-side step never reaches the host's permission prompts, so leave off any tool you want the model to call itself; `Read,Glob,Grep` is the read-only set (the default up to 0.3.7). `none`: every built-in step is steered in the session |
|
|
371
|
+
| `BIR_BUILTIN_TOOLS` | `Read,Glob,Grep,Edit,Write,Bash` | Built-ins the runner executes itself during a replay, in the session's own project directory and confined to it. A chain whose only non-MCP steps are on this list arms `direct`. A runner-side step never reaches the host's permission prompts, so leave off any tool you want the model to call itself; `Read,Glob,Grep` is the read-only set (the default up to 0.3.7). `none`: every built-in step is steered in the session. Claude Code sessions only: in a LangChain application a tool with one of these names is the application's own, and the runner runs none (§6.5) |
|
|
363
372
|
| `BIR_BASH_ALLOW` | *(unset: any command)* | Command prefixes a runner-side `Bash` step may run (`*` for all). Unset means any command; `none` means no command, and `Bash` steps are steered in the session. A plan's Bash steps share a working directory, exactly as the host's own Bash tool does — variables and functions do not persist, also as the host's. A call with `run_in_background`, or with its own `timeout` longer than the step's clock (`BIR_STEP_TIMEOUT_MS`), is never started on the runner: the plan hands over in front of it |
|
|
364
373
|
| `BIR_INTENT_ARM` | *(unset)* | `1` lets a match found mid-turn arm a plan in the calling tool's place, which means waiting for it (`BIR_INTENT_ARM_BUDGET_MS`, 500); `0` never waits; unset follows `BIR_SEGMENT_ARM` |
|
|
365
374
|
| `BIR_VERBOSE` | *(unset)* | `1` adds per-step `replay.*` detail lines |
|
|
@@ -442,6 +451,9 @@ way: `replay.pin_abandoned` retires the plan without running anything, and the m
|
|
|
442
451
|
arguments are its own from then on. The plan runs one call at a time: a call made while a pinned
|
|
443
452
|
step is still running hands the rest over (`replay.concurrent_call`).
|
|
444
453
|
|
|
454
|
+
In a LangChain application a plan arms `mode=steer` too, but there the application's adapter makes
|
|
455
|
+
each call in the model's place, and a call the model makes itself is never pinned: §6.5.
|
|
456
|
+
|
|
445
457
|
### 6.3 The model goes off-script
|
|
446
458
|
|
|
447
459
|
```
|
|
@@ -459,7 +471,8 @@ ends the bundle there: `replay.compose` shows `skipped=1 stopped=<n>:<reason>`,
|
|
|
459
471
|
with a note that names the step and says what may be assumed about it, and the model carries on
|
|
460
472
|
from that step. A recorded output stands in for it only with `BIR_REPLAY_RECORDED_OUTPUTS=1`. The
|
|
461
473
|
recovery runs inside the diverging call's `PreToolUse`, so it starts no step after
|
|
462
|
-
`BIR_PRETOOL_REPLAY_BUDGET_MS` (9 s). Direct plans never diverge (§6.1)
|
|
474
|
+
`BIR_PRETOOL_REPLAY_BUDGET_MS` (9 s). Direct plans never diverge (§6.1), and neither do plans in a
|
|
475
|
+
LangChain application: the runner can run none of its tools, so the plan hands over (§6.5).
|
|
463
476
|
|
|
464
477
|
### 6.4 The audit vocabulary
|
|
465
478
|
|
|
@@ -468,7 +481,7 @@ Everything replay decides is one line. `grep` for these:
|
|
|
468
481
|
| Line | Means |
|
|
469
482
|
|---|---|
|
|
470
483
|
| `run.matched` | The server recognised the prompt |
|
|
471
|
-
| `replay.decision … verdict=no-steer` | Matched, and declined. `why=` says which gate, `code=` names it: `match_timeout`, `service_unhealthy`, `arm_timeout`, `arm_unannounced`, `no_delivery_tool`, `missing_target`, … |
|
|
484
|
+
| `replay.decision … verdict=no-steer` | Matched, and declined. `why=` says which gate, `code=` names it: `match_timeout`, `service_unhealthy`, `arm_timeout`, `arm_unannounced`, `no_delivery_tool`, `missing_target`, `host_records_only`, `host_tool_missing`, … |
|
|
472
485
|
| `replay.match_gate` | Prompts stopped waiting on a slow match (`state=holding`), or started again (`state=open`) |
|
|
473
486
|
| `plan.armed` | A plan exists. Carries mode, step count and the ordered tool names — **before** anything runs |
|
|
474
487
|
| `replay.derived` / `replay.derive_failed` / `replay.derive_timeout` | Parameters resolved, fell back to sample values, or ran out of time |
|
|
@@ -489,6 +502,11 @@ Everything replay decides is one line. `grep` for these:
|
|
|
489
502
|
| `control.hook_deadline` | A hook route answered `{}` at its deadline |
|
|
490
503
|
| `replay.done` | Final outcome, steps completed, elapsed |
|
|
491
504
|
| `execution.reported` | The ticket was redeemed and the saving booked |
|
|
505
|
+
| `run.unmeasured` | The turn's cost is not measured (§7). `why=` says what was missing, and `models=` names a model the runner has no price for |
|
|
506
|
+
| `run.start … host=` / `plan.armed … host=` | The turn ran in a host that is not Claude Code, named after `host=` (§6.5). A Claude Code turn has no such field |
|
|
507
|
+
| `replay.own_call` / `replay.step_host_stopped` | Another host only. The model made a call of its own while the plan was live, and the plan handed over / the host stopped a step itself: `stop=rejected` or `stop=edited` |
|
|
508
|
+
| `run.finish … ended=` | Another host only: the run did not end with an ordinary stop, and `ended=` says how (`host_failed`, `idle`, …) |
|
|
509
|
+
| `run.incomplete` / `run.host_failed` | Another host only, said with its stop: the recording is not the whole run / the application's run failed |
|
|
492
510
|
|
|
493
511
|
`scenario.answered` is the `bir` server's own line, on its stderr (Claude Code's log for that MCP
|
|
494
512
|
server): what `run_scenario` answered, and the real reason when the model was told only that the
|
|
@@ -498,6 +516,47 @@ A recorder started in the background — by the SessionStart hook, `bir up` or `
|
|
|
498
516
|
them in `~/.baseinstrunner/logs/<key>.log`; `bir status` names the file. One started in a terminal
|
|
499
517
|
writes them to that terminal; keep them yourself: `bir-hooks 2>&1 | tee -a ~/.baseinstrunner/audit.log`.
|
|
500
518
|
|
|
519
|
+
### 6.5 A host that is not Claude Code
|
|
520
|
+
|
|
521
|
+
A plan can also run in an application that is not Claude Code. Today that is a LangChain
|
|
522
|
+
application: a Python program whose agent is built with LangChain's `create_agent` and the package
|
|
523
|
+
`langchain-basic2202`, in a folder set up with `bir setup --host langchain`. The package tells the
|
|
524
|
+
recorder about the prompt and about every tool call; design §7.4 calls it *the adapter*. The log
|
|
525
|
+
shows such a turn with `host=`: `run.start … host=langchain`, then
|
|
526
|
+
`plan.armed … mode=steer host=langchain`.
|
|
527
|
+
|
|
528
|
+
What is different from §6.1–§6.3:
|
|
529
|
+
|
|
530
|
+
- **The mode is always `steer`**, and there it is the full mode, not a fallback. The runner runs no
|
|
531
|
+
step itself, whatever a tool is named: `Bash`, `Read` or `TodoWrite` are the application's own
|
|
532
|
+
tools there. So `BIR_BUILTIN_TOOLS`, `BIR_BASH_ALLOW` and `bir replay allow` have nothing to act
|
|
533
|
+
on.
|
|
534
|
+
- **The adapter asks the recorder for each step, and the application runs its own tool** with the
|
|
535
|
+
arguments it gets back. The adapter answers in the model's place, so no model call is paid for a
|
|
536
|
+
step.
|
|
537
|
+
- **A call the model makes itself is never pinned.** If it makes one while a plan is live, the plan
|
|
538
|
+
hands over: the call runs as the model wrote it, and the note comes with that call's result.
|
|
539
|
+
- **A plan starts only when the adapter asks for one with the prompt.** Otherwise the match is
|
|
540
|
+
declined (`host_records_only`), and the model does the task. A plan that names a tool the agent
|
|
541
|
+
does not have is declined before any step runs (`host_tool_missing`). §9 has both.
|
|
542
|
+
- **A person can still stop a step.** A step that a person refuses, or changes before it runs, is
|
|
543
|
+
not counted against the plan. §7 has what is booked.
|
|
544
|
+
- **The cost is counted from the tokens the adapter reports**, not from a transcript (§7).
|
|
545
|
+
- **Step code still runs inside the recorder, with no sandbox** (§9.2). So a recorder started for
|
|
546
|
+
a folder that is set up for such a host, and not for Claude Code too, gets a small environment,
|
|
547
|
+
without the application's own keys and passwords (design §13.1).
|
|
548
|
+
- **The switch is the same**: `bir replay off`, then `bir up --restart`, makes the application
|
|
549
|
+
only record.
|
|
550
|
+
|
|
551
|
+
The how-to is `langChainGuide.md`, in the root of the runner's repository (it is not in the npm
|
|
552
|
+
package): setting up the folder, the two lines of Python, the settings, and what to do when
|
|
553
|
+
something is wrong. The rules are in design §7.4.
|
|
554
|
+
|
|
555
|
+
**Where it stands.** The runner's part is in this package. The Python package is not on PyPI yet,
|
|
556
|
+
and the service's part is built and not deployed yet. It was run with a fake model and a stand-in
|
|
557
|
+
for the service (`examples/langchain-proof/` in the repository, not in the npm package); a run
|
|
558
|
+
with a real model and the real service has not been done.
|
|
559
|
+
|
|
501
560
|
---
|
|
502
561
|
|
|
503
562
|
## 7. What each outcome means for your numbers
|
|
@@ -517,10 +576,29 @@ A recorded output that stood in for a step (`BIR_REPLAY_RECORDED_OUTPUTS=1`) is
|
|
|
517
576
|
work done. A direct run you cancel (Esc) is booked `fell_back` or `failed` with `measured: false`:
|
|
518
577
|
a turn cut short says nothing about what the task costs.
|
|
519
578
|
|
|
579
|
+
A turn's cost is **measured** only when its tokens were found and every model of the turn has a
|
|
580
|
+
price in the runner's table. A model with no price used to count as $0, and that zero was called
|
|
581
|
+
measured. Now such a turn is not measured, in Claude Code as in any host, and a `run.unmeasured`
|
|
582
|
+
line says why: `why=`, and `models=` when a model has no price. The service as built (not deployed
|
|
583
|
+
yet) also stores a report that says `measured: true` with a session cost of $0 as not measured.
|
|
584
|
+
|
|
585
|
+
In a host that is not Claude Code (§6.5) the cost comes from the token counts its adapter reports,
|
|
586
|
+
and three more endings have rules of their own. None of them is counted against the plan:
|
|
587
|
+
|
|
588
|
+
| What happened | The step | The report |
|
|
589
|
+
|---|---|---|
|
|
590
|
+
| A person said no to a step, or answered in the tool's place, or a call limit blocked the call | `skipped`, not `failed` | `fell_back` or `failed`, with `measured: false` and `baselineEligible: false` |
|
|
591
|
+
| A person changed the call before it ran | `skipped`, not `failed` | `fell_back` or `failed`; the cost stays measured, with `baselineEligible: false` |
|
|
592
|
+
| The run was closed while its plan was still live | none is marked `failed` | `fell_back` when a step ran, `failed` when none did, with `measured: false` and `baselineEligible: false` |
|
|
593
|
+
|
|
594
|
+
`baselineEligible: false` keeps the turn's cost out of the baseline samples. When more than one
|
|
595
|
+
plan started in one such run, none of them is measured.
|
|
596
|
+
|
|
520
597
|
`not_steered` is not an error, and it is not wasted. A saving is `baseline − actual`, and the baseline
|
|
521
598
|
is *what this task costs when the agent does it*. Every declined match is a fresh measurement of
|
|
522
599
|
exactly that. A deployment that seldom declines measures its baseline mostly from the recording
|
|
523
|
-
itself; `bir replay off` for a few runs of the prompt, then `bir replay on`, gives it more
|
|
600
|
+
itself; `bir replay off` for a few runs of the prompt, then `bir replay on`, gives it more (each
|
|
601
|
+
takes effect at `bir up --restart`).
|
|
524
602
|
|
|
525
603
|
An **unredeemed** ticket books nothing at all. Silence loses a saving; it never invents one.
|
|
526
604
|
|
|
@@ -538,9 +616,15 @@ model check asks whether the two prompts are the same request, values included,
|
|
|
538
616
|
matches. A prompt that missed both is matched once more on its task line (`PROMPT_INTENT_THRESHOLD`,
|
|
539
617
|
0.9) — the way the same task with a different value finds its scenario.
|
|
540
618
|
|
|
619
|
+
On the service as built (not deployed yet), matching also stays inside a host. A run of a
|
|
620
|
+
LangChain application (§6.5) is created with the `framework` `langchain`, or `langchain:<scope>`,
|
|
621
|
+
and is matched only against recordings and plans of that same text. Everything else is Claude
|
|
622
|
+
Code's pool, as before.
|
|
623
|
+
|
|
541
624
|
- A replay that answered the wrong question → switch that recording off in the console (its
|
|
542
|
-
scenario then never runs), or `bir replay off` in the project, and re-read
|
|
543
|
-
`intent`. On a service you run yourself, raise `SIMILARITY_THRESHOLD` or
|
|
625
|
+
scenario then never runs), or `bir replay off` and `bir up --restart` in the project, and re-read
|
|
626
|
+
the scenario's `intent`. On a service you run yourself, raise `SIMILARITY_THRESHOLD` or
|
|
627
|
+
`SIMILARITY_CHECK_FLOOR`.
|
|
544
628
|
- A prompt you *would* have wanted replayed that did not match → the fix is usually upstream of any
|
|
545
629
|
threshold: the two prompts genuinely mean different things to the embedding model. Re-record with
|
|
546
630
|
a first prompt that states the task plainly.
|
|
@@ -554,11 +638,14 @@ matches. A prompt that missed both is matched once more on its task line (`PROMP
|
|
|
554
638
|
| No `run.matched`, ever | Prompt below the server's `SIMILARITY_THRESHOLD` | Rephrase closer, or lower it server-side. Confirm the original run is in the list — a sub-`RECORDING_MIN_ACTIONS` run is never embedded |
|
|
555
639
|
| `run.matched` but no `plan.armed` | A gate declined | Read the `replay.decision` line's `why=`. Ranked by frequency: replay off — `bir replay off`, `BIR_REPLAY=0`, or never switched on here (a `bir install` without `--replay`; `bir replay status` says which) · scenario not `ready` · similarity below threshold · no step is executable |
|
|
556
640
|
| `replay.decision … code=match_timeout` or `code=service_unhealthy` on many prompts | The service answered the match slower than `BIR_MATCH_BUDGET_MS` (3 s), or is failing and in its cool-down | Nothing breaks: those turns run normally, and after a slow match prompts stop waiting on it for a while by themselves (`replay.match_gate`). `bir doctor` shows the service's state. Raise `BIR_MATCH_BUDGET_MS` only if the service is reliably just over it — every prompt pays it |
|
|
641
|
+
| `replay.decision … code=arm_timeout`, `code=missing_target … derivation did not answer within …ms`, or `code=no_derive_key why="service: …"`, on a prompt that names its values | The reading of the prompt's values (the derivation) was too slow for the hook, or failed on the service. A hook waits `BIR_HOOK_DERIVE_BUDGET_MS` (8 s) for it, and the prompt hook never past 6 s from the prompt, match included; the service gives one reading `RUNNER_DERIVE_TIMEOUT_MS` (6 s) and then answers `service: error` or `service: timeout` | Nothing to change in the prompt, and nothing to sign in to: that turn ran normally. The `replay.derived` line that follows says when the reading landed and what it cost. If it is slow every time, the service's reading model is the cause; `BIR_DERIVE_API_KEY` where Claude Code starts moves the reading to this machine |
|
|
557
642
|
| `replay.decision … code=no_delivery_tool` | A direct plan, and no running `bir` server has told the recorder it serves `run_scenario`: the server is missing, disabled or failed in `/mcp`, was not added (a subdirectory or worktree of a repository), or the session was started with `BIR_OFF=1` | `/mcp` shows whether `bir` is connected. `bir install --replay` adds it, in the project's own directory; then start a new session |
|
|
558
643
|
| `replay.vetoed … the model's first call was …, not run_scenario` | A direct plan, and the model reached for another tool first — usually because it has no `run_scenario` | As for `no_delivery_tool`. Nothing ran; the model did the task |
|
|
644
|
+
| `replay.decision … code=host_records_only` | A LangChain application (§6.5) whose adapter asked for no plan with its prompt: it only records. `bir investigate` gives the cause: the runner in that application's folder is older than replay for LangChain needs, so the LangChain package only records | In `bir investigate`'s words: nothing to do while replay for LangChain is not released. After that: update the runner. Until then the model does the task, and the run counts as a sample of the usual cost |
|
|
645
|
+
| `replay.decision … code=host_tool_missing` | A LangChain application: the plan names a tool that the agent which asked does not have. The plan is taken back before any step runs | Give the agent that tool, or delete the recording so the task is recorded again; `bir scenario show <scnId>` lists the plan's tools |
|
|
559
646
|
| `replay.pin_declined` on most steered turns | The model's calls to the expected tool are not the step — another command, another file — so nothing is pinned; if it then moves on, `replay.pin_abandoned` retires the plan | Harmless for those calls, which ran as the model wrote them. Persistent declines on one step usually mean its arguments cannot be shown in the directive (they depend on an earlier step's output, or the command runs to several lines), and the model writes its own |
|
|
560
647
|
| `why="no step is executable"` | None of the scenario's tools is a wrapped MCP server and none is a built-in reachable in this session | Wrap the servers the scenario uses (`bir install --server …`) and recalculate |
|
|
561
|
-
| `plan.armed mode=steer` where you expected `direct` | At least one step is a built-in off `BIR_BUILTIN_TOOLS` (by default one the runner does not implement, such as `WebFetch` or `Task`; a `Bash` step too when `BIR_BASH_ALLOW=none`), an unwrapped MCP server, or `claude-in-chrome`. Before 0.3.8 the default list was `Read,Glob,Grep`, so `Edit`, `Write` and `Bash` steered too | `bir status` shows what is wrapped; the `plan.armed` line's `coverage=` names the `live` step. `claude-in-chrome` is `scope: "dynamic"` and can never be wrapped |
|
|
648
|
+
| `plan.armed mode=steer` where you expected `direct` | At least one step is a built-in off `BIR_BUILTIN_TOOLS` (by default one the runner does not implement, such as `WebFetch` or `Task`; a `Bash` step too when `BIR_BASH_ALLOW=none`), an unwrapped MCP server, or `claude-in-chrome`. Before 0.3.8 the default list was `Read,Glob,Grep`, so `Edit`, `Write` and `Bash` steered too. With `host=langchain` on the line it is not a fallback: in a LangChain application the mode is always `steer` (§6.5) | `bir status` shows what is wrapped; the `plan.armed` line's `coverage=` names the `live` step. `claude-in-chrome` is `scope: "dynamic"` and can never be wrapped |
|
|
562
649
|
| Model ignores the directive and diverges every turn | Steering is advisory — the model chooses; `bir` only pins arguments | Expected occasionally. Persistent divergence usually means the scenario's tools do not fit the live task; check `intent` |
|
|
563
650
|
| A `replay.thread` line with no `emitted=` (under `BIR_VERBOSE=1`) | `toolOutputLogic` derived nothing — usually the output shape changed since recording | Dry-replay (§3.1). If the dry run emits and the live run does not, the tool's output shape has drifted; recalculate |
|
|
564
651
|
| `replay.thread_fallback` on a wrapped step | The proxy did not report within the grace window | Check that proxy is alive (`bir doctor`). Repeats mean a slow or dying upstream |
|
|
@@ -566,12 +653,16 @@ matches. A prompt that missed both is matched once more on its task line (`PROMP
|
|
|
566
653
|
| `run_scenario` answers "The prepared results are not available for this request. Do the task normally with your tools." | The plan could not be delivered — a recorder that does not answer, a timeout, an HTTP error — and the model does the task itself | The `scenario.answered` line on the `bir` server's stderr has the real reason. `BIR_SCENARIO_DEBUG=1` in that server's environment hands the raw reason to the model as an error, for debugging |
|
|
567
654
|
| A hand-over note says a step "was sent but did not answer … may or may not have completed" | The step timed out, or its upstream went away mid-call; nothing after it ran | Check whether its work happened before running it again — the model is told the same. Repeats: look at that upstream, or raise `BIR_STEP_TIMEOUT_MS` |
|
|
568
655
|
| A hand-over note says "a step never runs on another session's upstream" | Two Claude Code sessions are open in the project, and this one has not yet made a call of its own to that server, so bir cannot tell its proxy from the other session's | Expected: the model calls the tool itself. Once the session's own call to the server has gone through, later replays use its proxy |
|
|
569
|
-
|
|
|
656
|
+
| A hand-over note says a step "was not run by the host", or "ran with arguments a person chose, not the scenario's" | A LangChain application (§6.5). A person said no to the step, or answered in the tool's place, or a call limit blocked the call (`host_rejected`); or a person changed the call before it ran (`host_edited`) | Expected, and no fault of the plan: the step is booked `skipped`, the plan stops there and the model takes over. It is told not to run the step again — after a refusal, unless the user asks. §7 has what the report says |
|
|
657
|
+
| `recorder.send_failed … error="HTTP 409 … invalid_ticket"` | The execution report's ticket was already redeemed, or belongs to another scenario. Or the report came more than an hour after its plan was handed out | Harmless if the body says `duplicate: true`. Otherwise a rollover raced `SessionEnd`; the server books once. A report that came too late is not booked; the service as built (not deployed yet) takes a report for 24 hours |
|
|
658
|
+
| `recorder.send_failed … error="HTTP 429 …"` | The account is over one of the service's limits, and the answer is `429` with `Retry-After`. The limits are built on the service and not deployed yet; each is a setting there. By default: 120 runs a minute · 3,000 steps a minute (a tool call is two steps) · 10 title requests a minute · 20 requests in any one hour for Calculate, Prefer and Recalculate together (`bir scenario calc` then prints a line that begins `Could not start calculation (HTTP 429`) | What it costs: a create, a step or a report is sent once more, at most 2 s later, then dropped. A dropped step is a hole in its recording. A run whose create got the 429 runs without its plan. A 429 that the second try did not cure counts as the service being unwell: for 5 to 60 seconds no prompt of that recorder waits for a match. An intent probe over its own limit (1,500 a minute) is answered with no match, never with `429` |
|
|
570
659
|
| `savedUsd` looks impossible | Pricing drift between the runner's table and the server's | Compare `PRICING_VERSION` on both sides. Design §11.3 documents a known live drift between two existing copies |
|
|
571
660
|
| The hand-over note says a step "did not run: its input needs a judgement" | The service marked the step non-deterministic when it calculated the plan: the generated logic could not compute its input from the prompt or from earlier outputs, so the recorded value was a copy (the service's plan-services design, D8 kind 3 — it lives in the BaseIn repository, not this one). `nondeterministicWhy: by_hand` means someone marked it with `bir scenario edit --freeze` | If the code only has a slip, fix the step yourself: `bir scenario show scn_… --step <n>`, write the input logic to a file, then `bir scenario edit scn_… --step <n> --input-logic <file>` — the service checks it against the recording before it saves it (§9.2, not yet released). Of the edits, only `--unfreeze` lifts a mark made by hand (`by_hand`); a code edit, even a forced one, leaves it in place. A real judgement — a choice with no fixed rule — cannot be written as code: the agent makes that choice and finishes the task; a model step for it is planned (W3.1). The journal's `plan.armed` line lists the step under `stops=` |
|
|
572
661
|
| A parked step ran again after an hour, or a plan was recalculated by itself | The service's grace period (D8): a parked step is served with `stop: null` once an hour for a day and a success clears its count; if it keeps failing, or its tool is gone, the next repeat of the prompt is recorded in full and the plan is calculated again from it | Nothing: that is the repair. `bir investigate` says which it is — a step still being retried, a parked step whose repair is due, or a plan being calculated again. To repair now by hand: `bir scenario calc <runId> --force` for a whole-run plan; from the next version, `bir scenario calc <scnId> --force` works for any plan, a sub-task one included. A plan with steps edited by hand is not repaired by itself: the parked step stays parked, and `bir investigate` says the repair was skipped. Then fix the step (§9.2); or take the edits back with `bir scenario undo`, once per edit, and it is repaired as usual; or recalculate now and drop the edits: `bir scenario calc <scnId> --force --discard-edits` |
|
|
573
662
|
| `sessionCostUsd` grows every prompt in a session | The per-run usage watermark is missing or not taken | Design §11.4. Reports without a mark must carry `measured: false` |
|
|
663
|
+
| `run.unmeasured` in the log, and `bir investigate` shows the report as `UNMEASURED` | The turn's cost is not measured. The run opened before any hook carried a transcript path, or the runner has no price for a model of the turn (`models=` names it). In a LangChain application: the adapter sent no usage, the run made no model call, or a model has no price | Update the runner when it has no price for the model. The report is booked as not measured. A model with no price used to count as $0 and measured (§7) |
|
|
574
664
|
| Everything works, nothing is saved | A replayed turn is deliberately not recorded (design §12) | Correct. The matched run stays canonical and its `iterations` is bumped |
|
|
665
|
+
| The console shows a replayed step's input and output with the recording's values, beside this run's verdict and time | The run was reported by a runner up to 0.3.8, which sent each step's verdict and duration and nothing of what it sent or received. The page then falls back to the recording's sample (design §11.2a) | Update the runner: a newer one sends each replayed step's input and output with its verdict, redacted and cut to under 4 KB, and the card shows them as *last run*. To see what an older replay ran with: the result of `mcp__bir__run_scenario` in the session's transcript, or `lastParams` in `bir scenario show scn_…` |
|
|
575
666
|
| `bir doctor`: the SessionStart hook points at a Node that is gone | The hook names Node by absolute path. That Node was uninstalled, moved, or was a version manager's per-shell copy; the hook then fails on every session, no recorder is started, and runs record Tier 2 | `bir setup` here again — it rewrites the hook to the Node that runs it |
|
|
576
667
|
| `control.port_busy` in the log; the recorder is not running | Another program holds this project's port, or the system excludes it (on Windows, a range Hyper-V or WSL reserved). A background recorder with a recorded port does not fall back to another, because every hook but SessionStart names this one; only a `bir-hooks` you run by hand in a terminal still does | `bir setup --port <another>` in this directory — the fix the log line itself names. It rewrites the hook URLs and the scenario server's address together and restarts the recorder |
|
|
577
668
|
| The recorder was started by the hook, and `bir doctor` says `servers=(all wrapped)` although you set `BIR_REPLAY_ALLOW_SERVERS` | The hook starts the recorder with Claude Code's environment, not your terminal's; a variable set in one terminal is not seen | `bir replay allow <a,b>` — stored for the project, read by every recorder — then `bir up --restart` |
|
|
@@ -607,6 +698,33 @@ fix*. The three questions it answers:
|
|
|
607
698
|
| Why did it save so little, or cost more? | the execution's cost breakdown (derive, session, fallback) against the baseline; steer mode with built-in steps is the usual answer, a single-sample baseline the second |
|
|
608
699
|
| What happened, step by step? | the turn's lines in order, the per-step verdicts on the execution, and `replay.*_failed`, `replay.diverge`, `replay.handover` as incidents |
|
|
609
700
|
|
|
701
|
+
**An unmeasured turn** is explained as: *"the turn's cost could not be read
|
|
702
|
+
from the transcript, or the runner has no price for the model: update it."*
|
|
703
|
+
The `run.unmeasured` line names what was missing.
|
|
704
|
+
|
|
705
|
+
**A turn of another host** (a LangChain application, §6.5) is told apart by
|
|
706
|
+
the journal's `run.start` line, which carries `host=<app>`. For such a turn:
|
|
707
|
+
|
|
708
|
+
- there is no `steer_mode` finding: steer is the full mode there;
|
|
709
|
+
- the unmeasured finding has its own words, and names its own causes: too
|
|
710
|
+
old a runner or package, a model with no price, a plan that ended without
|
|
711
|
+
a model call, more than one plan in the run, a run closed early, a
|
|
712
|
+
person's "no";
|
|
713
|
+
- two more lines are incidents: `run.incomplete` (*"the recording is not the
|
|
714
|
+
whole run, so it is never calculated and no longer catches its prompt; run
|
|
715
|
+
the task again to record it whole."*) and `run.host_failed` (*"the
|
|
716
|
+
application's run failed; its recording is closed with that error and
|
|
717
|
+
never becomes a plan."*). The service's side of both — such a run is not
|
|
718
|
+
calculated by itself, and stops catching its prompt — is built and not
|
|
719
|
+
deployed yet;
|
|
720
|
+
- the declines `host_records_only` and `host_tool_missing` come with their
|
|
721
|
+
cause and fix, as in §9.
|
|
722
|
+
|
|
723
|
+
The service's half has its own sentences for those two codes too. For a
|
|
724
|
+
LangChain recording it raises no `steer_only` finding, and it explains an
|
|
725
|
+
unmeasured cost in its own words. That part is built on the service and not
|
|
726
|
+
deployed yet.
|
|
727
|
+
|
|
610
728
|
You see your own data. An admin of the service sees everyone's and may pass
|
|
611
729
|
`--user` to `executions`. `--json` on any form prints the raw merge.
|
|
612
730
|
|
|
@@ -1088,7 +1206,9 @@ model leaving a plan: the `bir` server's tools other than `run_scenario`, and a
|
|
|
1088
1206
|
`PowerShell` command made only of `bir` commands (a plain `cd` may come with them).
|
|
1089
1207
|
`npm test && bir investigate` is still recorded: the `npm test` half is real work. So is writing
|
|
1090
1208
|
the code file, which is an ordinary step. Under `BIR_VERBOSE=1` a skipped call shows as
|
|
1091
|
-
`tool.pre.skipped … why="host housekeeping"`.
|
|
1209
|
+
`tool.pre.skipped … why="host housekeeping"`. In a LangChain application (§6.5) only the `bir`
|
|
1210
|
+
server's own tools are skipped this way: a tool the application names `Bash` or `PowerShell` is its
|
|
1211
|
+
own, and is recorded.
|
|
1092
1212
|
|
|
1093
1213
|
**Do this.** In the Claude Code session, in the project folder, paste:
|
|
1094
1214
|
|
|
@@ -1192,7 +1312,8 @@ Parameter database: target → setting (revision 4). Undo: bir scenario undo scn
|
|
|
1192
1312
|
|
|
1193
1313
|
All of it. Phases R0–R7 of [calculatedReplay.md](calculatedReplay.md) §17 are implemented and
|
|
1194
1314
|
tested; everything this guide describes is code you can run. §9.2 is built and tested too, but not
|
|
1195
|
-
yet released.
|
|
1315
|
+
yet released. §6.5 is built in the runner; its Python package is not on PyPI, and the service's
|
|
1316
|
+
part of it is not deployed yet.
|
|
1196
1317
|
|
|
1197
1318
|
| Phase | What landed | Where |
|
|
1198
1319
|
|---|---|---|
|
|
@@ -1204,6 +1325,7 @@ yet released.
|
|
|
1204
1325
|
| **R5** | The `bir` MCP server, `/scenario/run`, `--replay` | [bin/bir-scenario.ts](../src/bin/bir-scenario.ts), [bin/bir.ts](../src/bin/bir.ts) |
|
|
1205
1326
|
| **R6** | Divergence, source-run fallback, `bir replay` | `diverge()` in [controller.ts](../src/replay/controller.ts), [source-run.ts](../src/replay/source-run.ts) |
|
|
1206
1327
|
| **R7** | Usage watermark, reporting, ticket, doctor | [control/transcript.ts](../src/control/transcript.ts), [remote-recorder.ts](../src/record/remote-recorder.ts) |
|
|
1328
|
+
| **§6.5** *(the package is not on PyPI; the service's part is not deployed)* | A plan stepped through by the adapter of a host that is not Claude Code: the declines `host_records_only` and `host_tool_missing`, the stops `host_rejected` and `host_edited`, the cost from the adapter's `usage` | [control/server.ts](../src/control/server.ts), [controller.ts](../src/replay/controller.ts), [handover.ts](../src/replay/handover.ts), [safety.ts](../src/replay/safety.ts), [bin/investigate.ts](../src/bin/investigate.ts); the proof: `examples/langchain-proof/` in the repository (not in the npm package) |
|
|
1207
1329
|
| **§9.2** *(not yet released)* | `bir scenario show/check/edit/edits/undo/calc/editing`, the `bir` server's scenario tools, `bir` calls never recorded (`editSteps.md` D13) | [bin/scenario-edit.ts](../src/bin/scenario-edit.ts), [bin/bir-scenario.ts](../src/bin/bir-scenario.ts), [record/housekeeping.ts](../src/record/housekeeping.ts); tests: [scenario-edit.test.ts](../test/scenario-edit.test.ts), [bir-scenario.test.ts](../test/bir-scenario.test.ts), [housekeeping.test.ts](../test/housekeeping.test.ts), [replay-server.test.ts](../test/replay-server.test.ts) |
|
|
1208
1330
|
|
|
1209
1331
|
```bash
|
package/docs/installRun.md
CHANGED
|
@@ -42,7 +42,9 @@ distributes. The delivery does not need to be clever.
|
|
|
42
42
|
|
|
43
43
|
The rest of `docs/` **does** ship, and deliberately: the GitHub repository is
|
|
44
44
|
private, so for anyone on a runner machine the tarball is the only place this
|
|
45
|
-
documentation exists.
|
|
45
|
+
documentation exists. Two things about LangChain are outside `files`
|
|
46
|
+
altogether and stay in the repository: `langChainGuide.md`, in its root, and
|
|
47
|
+
`examples/langchain-proof/`.
|
|
46
48
|
|
|
47
49
|
---
|
|
48
50
|
|
|
@@ -305,7 +307,7 @@ under `fnm_multishells` or `.nvm` (§5.1).
|
|
|
305
307
|
`~/.baseinstrunner/installed.json` keeps a record per project directory:
|
|
306
308
|
|
|
307
309
|
```
|
|
308
|
-
projects[<normalised cwd>] = { port, token, replay, invocation, … }
|
|
310
|
+
projects[<normalised cwd>] = { port, token, replay, invocation, hosts, … }
|
|
309
311
|
```
|
|
310
312
|
|
|
311
313
|
The rest is bookkeeping for `bir uninstall`: whether the exclude line was
|
|
@@ -360,6 +362,24 @@ does a `bir install` without `--replay` where no scenario server is installed;
|
|
|
360
362
|
an install with `--replay` turns such an off back on, but never a
|
|
361
363
|
`bir replay off` of your own.
|
|
362
364
|
|
|
365
|
+
**Hosts.** The hosts other than Claude Code the folder was set up for:
|
|
366
|
+
`bir setup --host langchain` (§5.1) writes `["langchain"]`. Such a folder has a
|
|
367
|
+
port, a token and a replay switch like any project, and nothing else — no
|
|
368
|
+
hooks, no wrapped entries, nothing under `.claude/` or in `~/.claude.json`, no
|
|
369
|
+
exclude line. No hook starts its recorder either: `bir setup --host`, `bir up`
|
|
370
|
+
and the application itself do. While the folder is not set up for Claude Code
|
|
371
|
+
as well, that recorder is started with a small environment, because step code
|
|
372
|
+
runs inside it with no sandbox and an application's model key or database
|
|
373
|
+
password must not reach it. It keeps `PATH`, `HOME`, `TMPDIR`, `TEMP`, `TMP`,
|
|
374
|
+
`USERPROFILE`, `HOMEDRIVE`, `HOMEPATH`, `APPDATA`, `LOCALAPPDATA`,
|
|
375
|
+
`SYSTEMROOT`, `SYSTEMDRIVE`, `WINDIR`, `COMSPEC`, `PATHEXT`, `HTTP_PROXY`,
|
|
376
|
+
`HTTPS_PROXY`, `NO_PROXY`, `ALL_PROXY`, `NODE_EXTRA_CA_CERTS`,
|
|
377
|
+
`SSL_CERT_FILE`, `SSL_CERT_DIR` and the `BIR_*` variables (but
|
|
378
|
+
`BIR_SETUP_TOKEN` and `BIR_OFF`, which no recorder is handed), and nothing
|
|
379
|
+
else. A folder set up for both keeps its Claude Code set-up — hooks, wrapped
|
|
380
|
+
servers, port, token, replay switch — as it is, and its recorder keeps the
|
|
381
|
+
whole environment.
|
|
382
|
+
|
|
363
383
|
### 4.4 Wrapped entries go to the local scope
|
|
364
384
|
|
|
365
385
|
`bir install` does not rewrite `.mcp.json` or the user scope. It writes a
|
|
@@ -569,7 +589,8 @@ That is also how a second project on the same machine is added.
|
|
|
569
589
|
|
|
570
590
|
Fleet switches on `bir setup`: `--allow-servers a,b` stores the servers replay
|
|
571
591
|
may call unattended for this project; `bir replay allow|on|off|status` changes
|
|
572
|
-
it later
|
|
592
|
+
it later, from the recorder's next start (`bir up --restart`; the command says
|
|
593
|
+
so). Both live in the project record (§4.3), so a recorder the hook starts
|
|
573
594
|
— with Claude Code's environment, not the operator's terminal — keeps them;
|
|
574
595
|
`BIR_REPLAY` and `BIR_REPLAY_ALLOW_SERVERS` in the recorder's environment still
|
|
575
596
|
override, and `bir doctor` says which source is in
|
|
@@ -578,6 +599,18 @@ until `bir uninstall --replay`) and stores replay off (`bir replay on` turns it
|
|
|
578
599
|
on); `--no-daemon` wires everything and starts
|
|
579
600
|
nothing; `--no-browser` prints the sign-in link without opening one.
|
|
580
601
|
|
|
602
|
+
`--host langchain` is for the folder of a LangChain application, not a Claude
|
|
603
|
+
Code project: `bir setup` signs in, gives the folder its entry (§4.3) and
|
|
604
|
+
starts the recorder, and wires nothing of Claude Code. `--no-replay` stores
|
|
605
|
+
replay off there too, and `--no-daemon` and `--no-browser` work as above. Any
|
|
606
|
+
other name is refused, exit 1 (`--host <x>: not a host this runner knows.
|
|
607
|
+
Known: langchain.`), and a runner older than the switch (0.3.9 or earlier)
|
|
608
|
+
prints its usage and exits 2: neither falls back to a Claude Code set-up. The
|
|
609
|
+
served scripts have no switch for it. It also needs the Python package
|
|
610
|
+
`langchain-basic2202` and a service that has its LangChain part, and neither
|
|
611
|
+
is out yet; the how-to is [../langChainGuide.md](../langChainGuide.md), in the
|
|
612
|
+
repository only (§1).
|
|
613
|
+
|
|
581
614
|
**Permissions on a fleet.** The runner does not approve wrapped MCP tool calls:
|
|
582
615
|
Claude Code asks about them as it would without the runner. (0.2.12 answered
|
|
583
616
|
every one "allow"; §8.) Headless `claude -p` runs must allow those tools
|
|
@@ -701,6 +734,14 @@ MCP calls, and the other projects set up on the machine. `--quick` skips the
|
|
|
701
734
|
network call; `--json` carries `problems`, `warnings`, `notes`, `off`,
|
|
702
735
|
`service`, `otherProjects` and `portProblem`.
|
|
703
736
|
|
|
737
|
+
In a folder set up for another host (§4.3) `bir doctor` prints a `Host` line —
|
|
738
|
+
`langchain (no Claude Code hooks are needed here)`, or `langchain, and Claude
|
|
739
|
+
Code` where the folder has both. Where the folder has no hooks it does not
|
|
740
|
+
report them missing: while no recorder runs, its note reads "the recorder is
|
|
741
|
+
not running; the langchain application starts it when it next runs (`bir up`
|
|
742
|
+
starts it now)". `bir status` prints a `Host` line too and carries `hosts` in
|
|
743
|
+
its `--json`; `bir doctor --json` does not name the host.
|
|
744
|
+
|
|
704
745
|
---
|
|
705
746
|
|
|
706
747
|
## 6. Machines with no registry
|
|
@@ -768,8 +809,11 @@ dropped just the same). The script moves it out of the environment as its first
|
|
|
768
809
|
statement and hands it to `bir setup` through that child's environment only;
|
|
769
810
|
`bir setup` deletes it again the moment it has read it, so the recorder it
|
|
770
811
|
starts never holds it. By hand: `bir setup --token <t>` or
|
|
771
|
-
`bir login --token <t
|
|
772
|
-
|
|
812
|
+
`bir login --token <t>` — on a machine with no stored address yet, with the
|
|
813
|
+
address as well: `--auth-url <api>` for `bir setup`, `BIR_AUTH_URL` for
|
|
814
|
+
either. Without it `bir setup` stops with `No service address. Pass --auth-url
|
|
815
|
+
https://api.your-service, or set BIR_AUTH_URL.` The token signs in one person
|
|
816
|
+
on one machine with no browser step; it is not a fleet credential.
|
|
773
817
|
|
|
774
818
|
What `bir setup` says about a token that is not fresh:
|
|
775
819
|
|
|
@@ -850,6 +894,30 @@ then `bir setup` in each recorded project.
|
|
|
850
894
|
npm i -g @basein/runner@<version> && cd ~/work/repo && bir setup
|
|
851
895
|
```
|
|
852
896
|
|
|
897
|
+
In a folder set up with `--host langchain` (§4.3) the second step is
|
|
898
|
+
`bir setup --host langchain`, or `bir up`: either replaces an older recorder
|
|
899
|
+
that is idle. A plain `bir setup` there would wire the folder for Claude Code
|
|
900
|
+
as well, and its recorder would get the whole environment from then on.
|
|
901
|
+
|
|
902
|
+
`bir update` is the same step as one command, for either kind of install: it
|
|
903
|
+
puts `@basein/runner@latest` where the copy that is running is. A copy the line
|
|
904
|
+
installed (under `~/.baseinstrunner/npm`) is updated by the npm beside the Node
|
|
905
|
+
its launchers name, with `--prefix` that folder and `--use-system-ca` when the
|
|
906
|
+
launchers carry it. A copy `npm install -g` made is updated by
|
|
907
|
+
`npm install -g`, after `npm root -g` has confirmed that npm installs where
|
|
908
|
+
that copy is; when it does not (pnpm, yarn, another Node's prefix), nothing is
|
|
909
|
+
installed, because the result would be a second copy. It exits 1 when npm
|
|
910
|
+
fails, replaces the package only, and names `bir setup` as the next step. It
|
|
911
|
+
always installs `latest`: use the line, or the command above, for a service
|
|
912
|
+
that pins `RUNNER_VERSION`. Typing `npm i -g @basein/runner@latest` over a copy
|
|
913
|
+
the line installed is the mistake it replaces: that adds a second copy in npm's
|
|
914
|
+
own prefix and leaves the launchers' copy as it was.
|
|
915
|
+
|
|
916
|
+
On Windows the line also writes `bir` and `bir-hooks` (sh, no extension) beside
|
|
917
|
+
`bir.cmd` and `bir-hooks.cmd`, because Git Bash, where Claude Code runs its
|
|
918
|
+
commands, never tries a `.cmd` name. A machine set up by an older script has
|
|
919
|
+
only the `.cmd` files: paste the line again.
|
|
920
|
+
|
|
853
921
|
Two rules used to follow, and neither is a step now:
|
|
854
922
|
|
|
855
923
|
- **Restarting the recorder.** A running process is still the old code, and
|
|
@@ -882,8 +950,8 @@ paths:
|
|
|
882
950
|
`claude -p` fleets must allow those tools themselves, or set
|
|
883
951
|
`BIR_CORRELATION_DECISION=allow` (§5.1).
|
|
884
952
|
- In a project with no scenario server (a plain `bir install`), the next
|
|
885
|
-
install without `--replay` switches replay off (§4.3); `bir replay on
|
|
886
|
-
it back on.
|
|
953
|
+
install without `--replay` switches replay off (§4.3); `bir replay on`, then
|
|
954
|
+
`bir up --restart`, turns it back on.
|
|
887
955
|
- Until it is replaced, a 0.2.12 recorder still running keeps approving those
|
|
888
956
|
calls and ignores `bir off`; `bir doctor` warns about a recorder of another
|
|
889
957
|
version, and `bir up --restart` replaces it once its session ends.
|
|
@@ -916,9 +984,13 @@ to report a version is not treated as skewed: silence, not a false alarm.
|
|
|
916
984
|
|
|
917
985
|
### Removing a runner
|
|
918
986
|
|
|
919
|
-
`bir uninstall` covers one project.
|
|
920
|
-
|
|
921
|
-
|
|
987
|
+
`bir uninstall` covers one project. In a folder set up for another host (§4.3)
|
|
988
|
+
that is the folder's entry, which it removes ("← this folder's entry for
|
|
989
|
+
langchain removed (its port, token and replay switch)"), and its recorder,
|
|
990
|
+
which it stops; `--all` covers such folders too. Before
|
|
991
|
+
`npm rm -g @basein/runner` or deleting `~/.baseinstrunner`, run
|
|
992
|
+
`bir uninstall --all` — or paste the install line with `BIR_UNINSTALL=1`, which
|
|
993
|
+
runs just that:
|
|
922
994
|
|
|
923
995
|
```bash
|
|
924
996
|
curl -fsSL https://api.bi2202.com/install.sh | BIR_UNINSTALL=1 sh
|
|
@@ -951,12 +1023,20 @@ npm pack && npm i -g ./basein-runner-*.tgz
|
|
|
951
1023
|
cd /tmp/scratch && bir setup --auth-url <api> --token <t> # or BIR_AUTH_URL=<api> bir login, then bir install --global && bir up
|
|
952
1024
|
bir doctor --json # must exit 0 (--strict: no warnings either)
|
|
953
1025
|
bir uninstall # must end with "Uninstalled." and exit 0
|
|
1026
|
+
cd /tmp/scratch-host && bir setup --host langchain # the other set-up (§4.3): nothing may appear under .claude/
|
|
1027
|
+
bir doctor # must print the Host line and "No problems found."
|
|
1028
|
+
bir uninstall # must print "← this folder's entry for langchain removed (…)", then "Uninstalled."
|
|
954
1029
|
```
|
|
955
1030
|
|
|
956
1031
|
`test:smoke` is the one that matters for packaging: it is the only test that
|
|
957
1032
|
crosses process boundaries, which is exactly where a bad `files` list, a missing
|
|
958
1033
|
shebang, or a broken bin mapping shows up.
|
|
959
1034
|
|
|
1035
|
+
The last three lines are the set-up for another host (§4.3); the sign-in of
|
|
1036
|
+
the lines before them is kept, so they need no token. The same check against a
|
|
1037
|
+
temporary home is `check_setup_host.py` in `examples/langchain-proof/`, which
|
|
1038
|
+
is in the repository and not in the package; its README says how to run it.
|
|
1039
|
+
|
|
960
1040
|
Sandbox a manual run with `BIR_HOME` and `BIR_CLAUDE_JSON` so it cannot touch
|
|
961
1041
|
your real Claude Code configuration. A fresh `BIR_HOME` has no sign-in, which is
|
|
962
1042
|
why the recipe signs in: a bare `bir install --global && bir up` there starts a
|
|
@@ -991,6 +1071,7 @@ runner.
|
|
|
991
1071
|
| 15 | Non-interactive auth grant for fleets | **needs the service** |
|
|
992
1072
|
| 16 | Migrate to trusted publishing before Jan 2027 (§3.3) | scheduled |
|
|
993
1073
|
| 17 | Production readiness: local-scope wrapping (§4.4), no auto-approval of wrapped calls, `bir off` / `bir on`, `bir uninstall --all` and `BIR_UNINSTALL=1` (§8) — contract and measurements in [production.md](../production.md) | done, unreleased |
|
|
1074
|
+
| 18 | `bir setup --host langchain`: the folder of a LangChain application gets an entry and a recorder, and nothing of Claude Code (§4.3, §5.1). It also needs the Python package `langchain-basic2202` and the service's LangChain part, neither out yet | built, unreleased |
|
|
994
1075
|
|
|
995
1076
|
Item 15 is the only one that needs somebody else.
|
|
996
1077
|
|
package/docs/quickstart.md
CHANGED
|
@@ -105,7 +105,8 @@ project's MCP servers now routed through `bir-proxy` under their old names — a
|
|
|
105
105
|
copies in this project's local scope in `~/.claude.json`, so `.mcp.json` itself
|
|
106
106
|
is not changed;
|
|
107
107
|
*replay* says which of them a calculated scenario may call unattended — narrow
|
|
108
|
-
it with `bir replay allow <names>`, turn it off with `bir replay off
|
|
108
|
+
it with `bir replay allow <names>`, turn it off with `bir replay off` (either
|
|
109
|
+
applies once the recorder is restarted: `bir up --restart`), and read
|
|
109
110
|
[calculatedReplayGuide.md §5.1](calculatedReplayGuide.md) before leaving it wide
|
|
110
111
|
anywhere that matters; *tools* is what the `bir` MCP server gives Claude here —
|
|
111
112
|
every tool, and Claude Code runs them without asking, except a change forced
|
|
@@ -184,6 +185,8 @@ came from, and who approves wrapped MCP calls. It sorts the rest into problems
|
|
|
184
185
|
runner is not doing all it should — not recording, say; `--strict` fails on
|
|
185
186
|
these too) and notes. `bir status` prints the service address, the wrapped
|
|
186
187
|
servers, the control server's address and log, and whether the runner is off.
|
|
188
|
+
In a folder set up for LangChain ([below](#a-langchain-application)) both add
|
|
189
|
+
a `Host` line, and `bir doctor` does not report missing hooks there.
|
|
187
190
|
|
|
188
191
|
Then open https://bi2202.com/recordings. The run from Step 2 is there, with
|
|
189
192
|
its prompt, its tool calls and its final answer.
|
|
@@ -196,7 +199,7 @@ its prompt, its tool calls and its final answer.
|
|
|
196
199
|
|---|---|
|
|
197
200
|
| `~/.baseinstrunner/config.json` | the service address; `BIR_AUTH_URL` in the environment overrides it, and `bir setup` warns when the two differ |
|
|
198
201
|
| `~/.baseinstrunner/credentials.json` | the cached session, bound to the service that issued it |
|
|
199
|
-
| `~/.baseinstrunner/installed.json` | what `bir uninstall` needs to undo each change — the original text of the project files it rewrote, with their credential values left out — and each project's port, token and replay switches, whether Claude's `bir` tools may change scenarios there (`bir setup` turns it on, `bir scenario editing off` turns it off),
|
|
202
|
+
| `~/.baseinstrunner/installed.json` | what `bir uninstall` needs to undo each change — the original text of the project files it rewrote, with their credential values left out — and each project's port, token and replay switches, whether Claude's `bir` tools may change scenarios there (`bir setup` turns it on, `bir scenario editing off` turns it off), the permission rule setup added, and the host a folder was set up for when that is not Claude Code (`bir setup --host`) |
|
|
200
203
|
| `~/.baseinstrunner/logs/<key>.log` | the recorder's audit log, one file per project |
|
|
201
204
|
| `~/.baseinstrunner/node/` | a private Node, only if the machine had no usable one; not on PATH |
|
|
202
205
|
| `~/.baseinstrunner/npm/` | the runner package |
|
|
@@ -221,14 +224,61 @@ No token: the sign-in is kept. The project gets its own port, token and log.
|
|
|
221
224
|
|
|
222
225
|
---
|
|
223
226
|
|
|
227
|
+
## A LangChain application
|
|
228
|
+
|
|
229
|
+
A LangChain application — a Python program whose agent is built with
|
|
230
|
+
`create_agent` — is set up in another way. In the folder the application
|
|
231
|
+
starts in:
|
|
232
|
+
|
|
233
|
+
```
|
|
234
|
+
bir setup --host langchain
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
On a machine that never ran `bir setup`, add
|
|
238
|
+
`--auth-url https://api.bi2202.com` the first time. It signs in, gives the
|
|
239
|
+
folder its own port and token, stores replay on (`--no-replay`: off), starts
|
|
240
|
+
the recorder in the background, and prints the two lines of Python to add to
|
|
241
|
+
the agent. Nothing of Claude Code is wired: it writes nothing under `.claude/`
|
|
242
|
+
and nothing into `~/.claude.json`, so of the paths in the table above only
|
|
243
|
+
those under `~/.baseinstrunner` change.
|
|
244
|
+
|
|
245
|
+
There is no install line for it. Pasted in such a folder, the line of Step 1
|
|
246
|
+
sets the folder up for Claude Code; `bir` has to be on the machine already,
|
|
247
|
+
from that line in another project or from `npm i -g @basein/runner`. After
|
|
248
|
+
`bir update`, run the same command there again: a plain `bir setup` would wire
|
|
249
|
+
the folder for Claude Code as well.
|
|
250
|
+
|
|
251
|
+
It also needs the Python package `langchain-basic2202` and a service that has
|
|
252
|
+
its LangChain part, and neither is out yet. The how-to is
|
|
253
|
+
[../langChainGuide.md](../langChainGuide.md), in the repository's root (it is
|
|
254
|
+
not in the npm package).
|
|
255
|
+
|
|
256
|
+
---
|
|
257
|
+
|
|
224
258
|
## Updating
|
|
225
259
|
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
`bir
|
|
260
|
+
```
|
|
261
|
+
bir update
|
|
262
|
+
bir setup # in each project that uses it
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
`bir update` installs the newest version over the copy of bir that is running,
|
|
266
|
+
wherever that copy is: one the console's line installed is updated in
|
|
267
|
+
`~/.baseinstrunner/npm`, by the npm beside its own Node; one you installed with
|
|
268
|
+
`npm i -g @basein/runner` is updated by `npm install -g`. It exits 1 when npm
|
|
269
|
+
fails. `bir setup` then re-wraps any entry whose runner moved.
|
|
270
|
+
|
|
271
|
+
Do not type `npm i -g @basein/runner@latest` over a copy the console's line
|
|
272
|
+
installed: npm puts a second copy in its own folder, and the one your `bir`
|
|
273
|
+
command starts stays as old as it was.
|
|
274
|
+
|
|
275
|
+
A runner older than this command (`bir update` answers with the list of
|
|
276
|
+
commands): paste the line from the console again. The very same line is fine,
|
|
277
|
+
used token and all: `bir setup` says `the setup token was already used; keeping
|
|
278
|
+
the sign-in from before (you@example.com)` and goes on. It installs the newest
|
|
279
|
+
version and runs `bir setup`. If you installed by hand with
|
|
280
|
+
`npm i -g @basein/runner`, upgrade the same way and then run `bir setup` in each
|
|
281
|
+
project.
|
|
232
282
|
|
|
233
283
|
A recorder of an older version is replaced only at the moments that look for
|
|
234
284
|
one — the SessionStart hook, `bir up`, `bir setup` — and only when no session
|
|
@@ -285,7 +335,9 @@ stays up, switched off for this project, and it tells you to restart that
|
|
|
285
335
|
session and run `bir down`. A file the install created — usually
|
|
286
336
|
`.claude/settings.local.json` — stays behind, empty of our entries. If anything
|
|
287
337
|
here still runs through the runner it ends with `Not everything could be
|
|
288
|
-
undone`, names it, and exits 1.
|
|
338
|
+
undone`, names it, and exits 1. In a folder set up with `--host langchain` it
|
|
339
|
+
removes the folder's entry — its port, token and replay switch — and stops its
|
|
340
|
+
recorder.
|
|
289
341
|
|
|
290
342
|
To remove the software, take the runner out of every project first — otherwise
|
|
291
343
|
their wrapped servers and hooks keep naming files you are about to delete:
|
|
@@ -315,8 +367,11 @@ answer. Then:
|
|
|
315
367
|
| `This is your home folder…` from the script | a terminal opened from the Start menu begins in the home folder; the script stopped before downloading anything, so nothing is installed, nobody is signed in and the token is unused | `cd` to the project and paste the same line again |
|
|
316
368
|
| `Signed in as … This is your home folder, so nothing was wired here` from `bir setup` | the same folder, but `bir setup` run by hand with the runner already installed; the sign-in is kept | `cd` to the project and run `bir setup` — no token needed |
|
|
317
369
|
| `… is not a BaseIn service` | the address is the website (docs or app), not the API; or a stale `BIR_AUTH_URL` in a shell profile — `bir setup` warns when the two disagree | `bir setup --auth-url https://api.bi2202.com`; remove the variable from the profile |
|
|
370
|
+
| `No service address. Pass --auth-url https://api.your-service, or set BIR_AUTH_URL.` from `bir setup` | this machine was never told the service's address: `bir setup` was typed by hand before any setup ran here (the console's line gives the address by itself) | `bir setup --auth-url https://api.bi2202.com` — needed once; the address is stored with the sign-in |
|
|
371
|
+
| `bir setup --host langchain` answers with the list of commands | the `bir` that ran is older than `--host` (0.3.9 or earlier): it prints its usage and exits 2. Nothing was set up, and nothing of Claude Code was wired | update the runner ([Updating](#updating); where that means pasting the console's line, paste it in a Claude Code project, not in this folder), then run it again |
|
|
318
372
|
| `control.port_busy` in the log; `bir doctor`: `the recorder could not take port N: …`; the recorder did not start | another program holds this project's port, or the system reserves the range it falls in — on Windows, Hyper-V, WSL and Docker reserve blocks that 53411 sits inside, and they move at boot | `bir setup` — it moves the project to a free port when its own cannot be bound; or `bir setup --port <n>` (below 49152 avoids the reserved ranges; by hand: `bir install --replay --port <n>` — a flag-less install keeps the way the project was installed — then `bir up --restart`) |
|
|
319
373
|
| `recorder.disabled` or `recorder.signed_out` in the log (`not signed in`, `no service address`); `bir doctor` warns the recorder has nowhere to send steps; or `bir setup` ends with ``not signed in — run `bir setup` again (or `bir login`)`` | not signed in, or signed in to another service | `bir setup` — the running recorder starts recording by itself once the sign-in is there, no restart needed |
|
|
374
|
+
| `recorder.send_failed` with `HTTP 429` in the log; or `Could not start calculation (HTTP 429…)` from `bir scenario calc` | the account is over one of the service's limits — by default 120 runs and 3,000 steps a minute (a tool call is two steps), 10 title requests a minute, 20 Calculate, Prefer and Recalculate requests in any one hour; these are built in the service and not deployed yet. The recorder sent it once more, at most 2 s later, then dropped it: a dropped step is a hole in that recording, and a run whose create was refused ran without its plan | nothing to repair on this machine: sessions are not held, and the next sends go out as usual. Each limit is a setting of the service |
|
|
320
375
|
| `- <name> — left unwrapped: Claude Code has not been told to use it yet` from `bir setup` | a `.mcp.json` server Claude Code has not approved (a fresh clone); a copy would start it without asking | start `claude`, approve it, then `bir setup` again; `bir install --server <name>` wraps it anyway |
|
|
321
376
|
| a `claude -p` run is denied a wrapped MCP tool (since upgrading from 0.2.12) | the runner no longer approves wrapped MCP calls; Claude Code's own permissions decide | allow the tools in `permissions.allow` or with `--allowedTools`, or set `BIR_CORRELATION_DECISION=allow` where the recorder starts, then `bir up --restart` from there |
|
|
322
377
|
| a "hook error" on each prompt and tool call | the recorder was stopped (`bir down`) under an open session | `bir up`; `bir off --here` afterwards if you want the runner out of the way; a new session also starts it |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@basein/runner",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.10",
|
|
4
4
|
"description": "A recording MCP proxy: sits between any MCP client and its MCP servers, executes each call on the client's behalf, and records the run as a reusable BaseIn scenario.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|