@basein/runner 0.2.5 → 0.2.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -6
- package/dist/bin/bir-hooks.d.ts +6 -5
- package/dist/bin/bir-hooks.js +7 -6
- package/dist/bin/bir.js +4 -4
- package/dist/replay/controller.d.ts +1 -1
- package/dist/replay/controller.js +1 -1
- package/docs/calculatedReplay.md +3 -3
- package/docs/calculatedReplayGuide.md +5 -5
- package/docs/my-first-sample.md +2 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -110,10 +110,11 @@ between the two is not guaranteed.
|
|
|
110
110
|
is a structural fact, not a limitation of this implementation.
|
|
111
111
|
- **`claude-in-chrome`.** It is `scope: "dynamic"`, present in no config file, so
|
|
112
112
|
there is no entry to rewrite.
|
|
113
|
-
- **Replay, unless you
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
113
|
+
- **Replay, unless you install it.** Recording is the whole of what runs out of
|
|
114
|
+
the box. Calculated replay — running a matched prompt's scenario instead of
|
|
115
|
+
rediscovering it — ships, but needs `bir install --replay`. Once installed it
|
|
116
|
+
is on by default; `BIR_REPLAY=0` turns it off, for the reason in the next
|
|
117
|
+
section.
|
|
117
118
|
- **Change what the model does.** Tool descriptions are never edited. The
|
|
118
119
|
model-visible changes are schema relaxation, below, and — only with
|
|
119
120
|
`--replay` — one first-party tool.
|
|
@@ -134,7 +135,7 @@ record, calculate, check, enable, read the audit lines — is
|
|
|
134
135
|
|
|
135
136
|
```bash
|
|
136
137
|
bir install --replay # adds the `bir` MCP server; raises the prompt-hook timeout
|
|
137
|
-
BIR_REPLAY=
|
|
138
|
+
bir-hooks # replay is on by default; BIR_REPLAY=0 bir-hooks turns it off
|
|
138
139
|
```
|
|
139
140
|
|
|
140
141
|
Two switches, not one, because of what replay is: **a scenario is a pre-approved
|
|
@@ -198,7 +199,7 @@ what to do when they register, so there is one switch and no second copy of it:
|
|
|
198
199
|
|
|
199
200
|
| Variable | Effect |
|
|
200
201
|
|---|---|
|
|
201
|
-
| `BIR_REPLAY=
|
|
202
|
+
| `BIR_REPLAY=0` | Disable replay. It is on by default; nothing below matters while this is `0`. |
|
|
202
203
|
| `BIR_REPLAY_ALLOW_SERVERS` | Server keys eligible for **direct** execution. Unset means every wrapped server; setting it is the recommendation. |
|
|
203
204
|
| `BIR_MIN_STEER_SIMILARITY` | Minimum match similarity to replay (default `0.92`, above the service's own `0.9` detection threshold). |
|
|
204
205
|
| `ANTHROPIC_API_KEY` | **Optional.** Working out what a new request is about — which fleet, which file, which date — is done for you by the service on its own key, as long as you are signed in. Set this only to keep that reading on your machine, on your key. Signed out *and* unset, a scenario whose values change between requests is declined rather than replayed on stale ones. |
|
package/dist/bin/bir-hooks.d.ts
CHANGED
|
@@ -19,11 +19,12 @@
|
|
|
19
19
|
* BIR_NO_CORRELATION=1 never inject a call id; join on fingerprints
|
|
20
20
|
* BIR_VERBOSE=1 per-step detail lines
|
|
21
21
|
*
|
|
22
|
-
* BIR_REPLAY=
|
|
23
|
-
* match (docs/calculatedReplay.md).
|
|
24
|
-
*
|
|
25
|
-
*
|
|
26
|
-
*
|
|
22
|
+
* BIR_REPLAY=0 turn OFF calculated replay — running a scenario on
|
|
23
|
+
* a similar-meaning match (docs/calculatedReplay.md).
|
|
24
|
+
* ON by default; unset or any value other than "0"
|
|
25
|
+
* keeps it on. Read §13.2: a replayed step is
|
|
26
|
+
* auto-approved, and a directly executed one never
|
|
27
|
+
* reaches the permission system at all.
|
|
27
28
|
* BIR_REPLAY_ALLOW_SERVERS comma-separated server keys eligible for *direct*
|
|
28
29
|
* execution. Unset means every wrapped server
|
|
29
30
|
* BIR_MIN_STEER_SIMILARITY minimum match similarity to replay (default 0.92)
|
package/dist/bin/bir-hooks.js
CHANGED
|
@@ -19,11 +19,12 @@
|
|
|
19
19
|
* BIR_NO_CORRELATION=1 never inject a call id; join on fingerprints
|
|
20
20
|
* BIR_VERBOSE=1 per-step detail lines
|
|
21
21
|
*
|
|
22
|
-
* BIR_REPLAY=
|
|
23
|
-
* match (docs/calculatedReplay.md).
|
|
24
|
-
*
|
|
25
|
-
*
|
|
26
|
-
*
|
|
22
|
+
* BIR_REPLAY=0 turn OFF calculated replay — running a scenario on
|
|
23
|
+
* a similar-meaning match (docs/calculatedReplay.md).
|
|
24
|
+
* ON by default; unset or any value other than "0"
|
|
25
|
+
* keeps it on. Read §13.2: a replayed step is
|
|
26
|
+
* auto-approved, and a directly executed one never
|
|
27
|
+
* reaches the permission system at all.
|
|
27
28
|
* BIR_REPLAY_ALLOW_SERVERS comma-separated server keys eligible for *direct*
|
|
28
29
|
* execution. Unset means every wrapped server
|
|
29
30
|
* BIR_MIN_STEER_SIMILARITY minimum match similarity to replay (default 0.92)
|
|
@@ -141,7 +142,7 @@ function parseIntentSources(value) {
|
|
|
141
142
|
* part of the live task.
|
|
142
143
|
*/
|
|
143
144
|
function buildReplayOptions(auth) {
|
|
144
|
-
const enabled = process.env.BIR_REPLAY
|
|
145
|
+
const enabled = process.env.BIR_REPLAY !== "0";
|
|
145
146
|
const allowServers = parseAllowList(process.env.BIR_REPLAY_ALLOW_SERVERS);
|
|
146
147
|
const minSimilarity = Number(process.env.BIR_MIN_STEER_SIMILARITY);
|
|
147
148
|
const opts = {
|
package/dist/bin/bir.js
CHANGED
|
@@ -369,10 +369,10 @@ function install(args) {
|
|
|
369
369
|
}
|
|
370
370
|
if (args.replay) {
|
|
371
371
|
out();
|
|
372
|
-
out("Calculated replay is INSTALLED
|
|
373
|
-
out("`bir-hooks` to
|
|
374
|
-
out("a replayed step is auto-approved,
|
|
375
|
-
out("the permission system at all.");
|
|
372
|
+
out("Calculated replay is INSTALLED and ON by default once `bir-hooks` runs. Set");
|
|
373
|
+
out("BIR_REPLAY=0 where you run `bir-hooks` to turn it off — and read");
|
|
374
|
+
out("docs/calculatedReplayGuide.md §5.1 first: a replayed step is auto-approved,");
|
|
375
|
+
out("and a directly executed one never reaches the permission system at all.");
|
|
376
376
|
}
|
|
377
377
|
return 0;
|
|
378
378
|
}
|
|
@@ -35,7 +35,7 @@ export interface ReplayBudgets {
|
|
|
35
35
|
}
|
|
36
36
|
export declare const DEFAULT_BUDGETS: ReplayBudgets;
|
|
37
37
|
export interface ReplayOptions {
|
|
38
|
-
/** `BIR_REPLAY=
|
|
38
|
+
/** On unless `BIR_REPLAY=0`. Everything here is inert when false. */
|
|
39
39
|
enabled: boolean;
|
|
40
40
|
/** Minimum similarity to steer — deliberately above the service's detection threshold. */
|
|
41
41
|
minSimilarity: number;
|
|
@@ -165,7 +165,7 @@ export class ReplayController {
|
|
|
165
165
|
}
|
|
166
166
|
// Gate 2 — replay enabled.
|
|
167
167
|
if (!this.enabled)
|
|
168
|
-
return decline("BIR_REPLAY
|
|
168
|
+
return decline("BIR_REPLAY=0", "replay_disabled");
|
|
169
169
|
// Gate 3 — a ready scenario with steps.
|
|
170
170
|
const scenario = match.scenario;
|
|
171
171
|
if (!match.scenarioId || !isReadyScenario(scenario)) {
|
package/docs/calculatedReplay.md
CHANGED
|
@@ -59,7 +59,7 @@ Continuing v1's D1–D8.
|
|
|
59
59
|
| **D11** | Execution ownership for MCP steps | **The proxy that owns the upstream** | No second connection, no double-spawned browser, no Agent SDK dependency |
|
|
60
60
|
| **D12** | Control → proxy channel | **Long-poll**, proxy dials out | No new listeners, no new ports, no new tokens (§13.3, §19.2) |
|
|
61
61
|
| **D13** | Result delivery in direct mode | A **first-party MCP server** (`bir`), one tool | The model reads a genuine `tool_result`, never a `deny` reason it distrusts (§19.1) |
|
|
62
|
-
| **D14** | Default state | **
|
|
62
|
+
| **D14** | Default state | **On** once `bir install --replay` has run; `BIR_REPLAY=0` turns it off | Replay bypasses permission prompts (§13.2); the install flag is the opt-in, the env var is the kill switch |
|
|
63
63
|
| **D15** | Parameter derivation | One Haiku call over raw `fetch`; **no API key ⇒ recorded sample values** | Preserves v1's zero-runtime-dependency property, and degrades to a free replay |
|
|
64
64
|
| **D16** | Savings reporting | Every armed match reports to `POST /scenarios/:id/executions`, including declines | A decline is a *baseline sample*, not silence — it is what keeps the ledger honest |
|
|
65
65
|
|
|
@@ -189,7 +189,7 @@ Every gate that declines logs **one** `replay.decision` line carrying the reason
|
|
|
189
189
|
| # | Gate | Declines when | Outcome floor |
|
|
190
190
|
|---|---|---|---|
|
|
191
191
|
| 1 | **A match arrived** | `getMatch()` resolved `null`, or the budget expired (§10) | (no report — nothing matched) |
|
|
192
|
-
| 2 | **Replay enabled** | `BIR_REPLAY
|
|
192
|
+
| 2 | **Replay enabled** | `BIR_REPLAY === "0"` | `not_steered` |
|
|
193
193
|
| 3 | **Scenario ready** | `scenario == null`, `state !== "ready"`, or `steps.length === 0` | `not_steered` |
|
|
194
194
|
| 4 | **Similarity** | `similarity < BIR_MIN_STEER_SIMILARITY` (default **0.92**) | `not_steered` |
|
|
195
195
|
| 5 | **Tool coverage** | No step is executable anywhere (§5 → `mode: "none"`) | `not_steered` |
|
|
@@ -828,7 +828,7 @@ So a scenario is, in effect, a **pre-approved list of tool calls with computed a
|
|
|
828
828
|
precisely what makes it valuable, and precisely what makes it dangerous. Four mitigations, all
|
|
829
829
|
required:
|
|
830
830
|
|
|
831
|
-
1. `
|
|
831
|
+
1. `bir install --replay` a distinct, non-default flag; `BIR_REPLAY=0` a one-line kill switch once installed.
|
|
832
832
|
2. `BIR_REPLAY_ALLOW_SERVERS` — a comma-separated allowlist of server keys eligible for **direct**
|
|
833
833
|
execution. Unset means *all wrapped servers*; the guide (§4 of `calculatedReplayGuide.md`)
|
|
834
834
|
recommends setting it.
|
|
@@ -206,7 +206,7 @@ That is what makes replay fast, and it is the whole risk. Two things follow:
|
|
|
206
206
|
|
|
207
207
|
```bash
|
|
208
208
|
bir install --replay # adds the `bir` MCP server; raises the prompt-hook timeout
|
|
209
|
-
|
|
209
|
+
unset BIR_REPLAY # on by default; only BIR_REPLAY=0 turns it off
|
|
210
210
|
bir-hooks # restart it
|
|
211
211
|
```
|
|
212
212
|
|
|
@@ -243,7 +243,7 @@ there is exactly one switch and it cannot get out of step with itself.
|
|
|
243
243
|
|
|
244
244
|
| Variable | Default | What it does |
|
|
245
245
|
|---|---|---|
|
|
246
|
-
| `BIR_REPLAY` | *(
|
|
246
|
+
| `BIR_REPLAY` | *(on)* | `0` disables replay. Unset or any other value keeps it on; nothing below matters while it is `0` |
|
|
247
247
|
| `BIR_REPLAY_ALLOW_SERVERS` | *(all wrapped)* | Comma-separated server keys eligible for **direct** execution. **Set this** |
|
|
248
248
|
| `BIR_MIN_STEER_SIMILARITY` | `0.92` | Below this a match is detected but not replayed (§8) |
|
|
249
249
|
| `ANTHROPIC_API_KEY` | *(unset)* | **Not required.** Derivation — reading what this request acts on — is done by the service on its key for a signed-in runner. Set this to keep the reading on this machine instead: the prompt then never leaves it, and it is one round trip faster. Signed out *and* unset, only a scenario with nothing to work out replays |
|
|
@@ -381,7 +381,7 @@ first prompt that states the task plainly.
|
|
|
381
381
|
| Symptom | Cause | Fix |
|
|
382
382
|
|---|---|---|
|
|
383
383
|
| No `run.matched`, ever | Prompt below the server's `SIMILARITY_THRESHOLD` | Rephrase closer, or lower it server-side. Confirm the original run is in the list — a sub-`RECORDING_MIN_ACTIONS` run is never embedded |
|
|
384
|
-
| `run.matched` but no `plan.armed` | A gate declined | Read the `replay.decision` line's `why=`. Ranked by frequency: `BIR_REPLAY`
|
|
384
|
+
| `run.matched` but no `plan.armed` | A gate declined | Read the `replay.decision` line's `why=`. Ranked by frequency: `BIR_REPLAY=0` · scenario not `ready` · similarity below threshold · no step is executable |
|
|
385
385
|
| `why="no step is executable"` | None of the scenario's tools is a wrapped MCP server and none is a built-in reachable in this session | Wrap the servers the scenario uses (`bir install --server …`) and recalculate |
|
|
386
386
|
| `plan.armed mode=steer` where you expected `direct` | At least one step is a built-in, an unwrapped MCP server, or `claude-in-chrome` | `bir status` shows what is wrapped. `claude-in-chrome` is `scope: "dynamic"` and can never be wrapped |
|
|
387
387
|
| Model ignores the directive and diverges every turn | Steering is advisory — the model chooses; `bir` only pins arguments | Expected occasionally. Persistent divergence usually means the scenario's tools do not fit the live task; check `intent` |
|
|
@@ -439,10 +439,10 @@ Three things the implementation settled that the design left open:
|
|
|
439
439
|
Three levels, least to most.
|
|
440
440
|
|
|
441
441
|
```bash
|
|
442
|
-
|
|
442
|
+
export BIR_REPLAY=0 # stop arming. Matches still detected; recording still stops on a match
|
|
443
443
|
bir uninstall --replay # remove the `bir` MCP server, restore the prompt-hook timeout
|
|
444
444
|
bir uninstall # remove everything: proxies, hooks, files restored byte-for-byte
|
|
445
445
|
```
|
|
446
446
|
|
|
447
|
-
With `BIR_REPLAY`
|
|
447
|
+
With `BIR_REPLAY=0` the system is exactly v1 again: it recognises a repeated prompt, declines to
|
|
448
448
|
record it a second time, and lets the model do the work.
|
package/docs/my-first-sample.md
CHANGED
|
@@ -357,7 +357,7 @@ Leave this running in its own terminal, **in the run folder**:
|
|
|
357
357
|
|
|
358
358
|
```bash
|
|
359
359
|
export BIR_AUTH_URL=https://api.your-domain.com
|
|
360
|
-
|
|
360
|
+
unset BIR_REPLAY # replay is on by default; BIR_REPLAY=0 turns it off
|
|
361
361
|
export BIR_REPLAY_ALLOW_SERVERS=postgres # servers allowed to run unattended
|
|
362
362
|
bir-hooks 2>&1 | tee -a bir-hooks.log
|
|
363
363
|
```
|
|
@@ -539,7 +539,7 @@ real work for zero tokens, and the saving is real and positive.**
|
|
|
539
539
|
- **[quickstart.md](quickstart.md)** — setting up a fresh machine to record from
|
|
540
540
|
your *own* work, not just this benchmark.
|
|
541
541
|
|
|
542
|
-
> **Turning replay off** is one line: `
|
|
542
|
+
> **Turning replay off** is one line: `export BIR_REPLAY=0`. The system is then
|
|
543
543
|
> exactly a recorder again — it recognises a repeated prompt and declines to
|
|
544
544
|
> record it twice, and the model does the work. Nothing is skipped, nothing is
|
|
545
545
|
> risked.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@basein/runner",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.6",
|
|
4
4
|
"description": "A recording MCP proxy: sits between any MCP client and its MCP servers, executes each call on the client's behalf, and records the run as a reusable BaseIn scenario.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|