@basein/runner 0.1.0

package/docs/quickstart.md

@@ -0,0 +1,201 @@
+# Setting up a runner machine
+
+This guide takes one computer from nothing to recording. It assumes no prior
+knowledge of the project. It should take about ten minutes, most of which is
+waiting for downloads.
+
+**What you are setting up.** BaseInstRunner sits quietly between your AI coding
+assistant and the tools it uses, watches what happens, and saves each session to
+the BaseIn service so it can be reviewed or re-run later. It does not change what
+the assistant does.
+
+---
+
+## Before you start
+
+Three things:
+
+1. **Node 20 or newer.** Check by opening a terminal and typing `node -v`. If you
+   see something like `v20.11.0` or higher, you are fine. If you see an error or
+   a smaller number, install it from [nodejs.org](https://nodejs.org) first.
+2. **The BaseIn service address** — a URL like `https://basein.example.com`.
+   Whoever runs the service gives you this.
+3. **Your BaseIn email and password.** You will type these once.
+
+---
+
+## Step 1 — run one command
+
+Open a terminal **in the folder where you unpacked this project's `scripts`
+folder**, and run the line for your machine. Replace the two placeholder values
+with your real service address and your real project folder.
+
+### Windows
+
+```powershell
+.\scripts\install-runner.ps1 C:\path\to\your\project -AuthUrl https://basein.example.com
+```
+
+### macOS
+
+```bash
+BIR_AUTH_URL=https://basein.example.com ./scripts/install-runner.sh ~/path/to/your/project
+```
+
+> **"Running scripts is disabled on this system"** (Windows only) — Windows blocks
+> unsigned scripts by default. Allow them for your own account with:
+> `Set-ExecutionPolicy -Scope CurrentUser RemoteSigned`, then run the command
+> again.
+
+---
+
+## Step 2 — watch what it does
+
+The script narrates each stage. A healthy run looks like this:
+
+```
+==> Checking Node
+    node v20.11.0
+==> Installing the runner
+    @basein/runner@0.1.0 from the registry
+    bir 0.1.0 -- all four binaries on PATH
+==> Configuring the BaseIn service
+    BIR_AUTH_URL=https://basein.example.com (persisted for this user)
+==> Signing in
+Email: you@example.com
+Password:
+==> Wrapping MCP servers in C:\path\to\your\project
+  + chrome-devtools -> bir-proxy (project scope, upstream: npx)
+  + hooks -> ...\.claude\settings.json
+
+Wrapped 1 server.
+```
+
+The password does not appear as you type it. That is normal.
+
+If anything goes wrong the script stops immediately and says why — it checks
+everything it can *before* changing your machine, so a failed run leaves nothing
+half-installed.
+
+---
+
+## Step 3 — start recording
+
+Two terminals, in the project folder you gave the script.
+
+**First terminal** — leave this one running the whole time:
+
+```
+bir-hooks
+```
+
+**Second terminal** — your normal work:
+
+```
+claude
+```
+
+That is it. Everything you do in the second terminal is now recorded.
+
+When you are finished for the day, press `Ctrl+C` in the first terminal.
+
+---
+
+## Step 4 — check it actually worked
+
+With `bir-hooks` running, in another terminal:
+
+```
+bir doctor
+```
+
+This is the one command worth remembering. It does not read a settings file and
+tell you what *should* happen — it asks the running system what is *actually*
+happening, and it fails loudly when something is wrong.
+
+To see what is set up without needing anything running:
+
+```
+bir status
+```
+
+---
+
+## When something is wrong
+
+### "It says nothing is recorded"
+
+`bir status` will show `BaseIn: (BIR_AUTH_URL not set — nothing is recorded)`.
+The service address did not stick. Close the terminal, open a new one, and check
+`echo $env:BIR_AUTH_URL` (Windows) or `echo $BIR_AUTH_URL` (macOS). If it is
+empty, run the install script again with the `-AuthUrl` / `BIR_AUTH_URL` value.
+
+### "bir is not recognised as a command"
+
+The install worked but your terminal has not noticed yet. **Close the terminal
+and open a new one.** This fixes it almost every time.
+
+### "It records the tools but not what I typed"
+
+`bir-hooks` is not running, or it is running in a different folder. It has to be
+started **in the same folder** as your session — that is how the two find each
+other. Stop it, `cd` to the project folder, and start it again.
+
+### "invalid JSON at ...\.mcp.json"
+
+Something has edited that file into a shape that is no longer valid JSON —
+usually a missing comma or a stray bracket. Open it and check. (A file saved by
+Notepad is fine; that case is handled.)
+
+### Anything else
+
+Run `bir doctor` and keep the output. It says which part of the chain is broken,
+which is most of the way to an answer.
+
+---
+
+## Updating to a newer version
+
+Run the same install script again with the new version number:
+
+```powershell
+.\scripts\install-runner.ps1 C:\path\to\your\project -Version 0.2.0 -AuthUrl https://basein.example.com
+```
+
+```bash
+BIR_VERSION=0.2.0 BIR_AUTH_URL=https://basein.example.com ./scripts/install-runner.sh ~/path/to/your/project
+```
+
+It is safe to re-run: it will not ask you to sign in again, and it will not
+double-wrap anything.
+
+**One thing you must do by hand: stop `bir-hooks` and start it again.** A running
+process keeps using the old version until it is restarted. If you forget, the
+next session logs a `version.skew` warning telling you exactly that.
+
+---
+
+## Undoing it
+
+In the project folder:
+
+```
+bir uninstall
+```
+
+This puts every file it touched back exactly as it found it, byte for byte.
+
+To remove the software entirely as well:
+
+```
+npm rm -g @basein/runner
+```
+
+---
+
+## Where to go next
+
+- [installRun.md](installRun.md) — the fleet version of this: packaging, release
+  process, offline installs, keeping `bir-hooks` alive across reboots.
+- [../README.md](../README.md) — what is recorded, what cannot be, and every
+  configuration switch.

package/docs/t-bench.md

@@ -0,0 +1,394 @@
+# τ²-bench — running it against BaseInstRunner, with Claude Code as the agent
+
+Companion to [benchmark.md](benchmark.md), which recommends MCPMark first and τ²-bench
+second. This is the second half: what τ²-bench actually is, how to read a trajectory,
+and the three structural facts that decide whether the integration measures our product
+or measures nothing at all.
+
+**Read §3 before writing any code.** τ²-bench executes tool calls *inside its own
+orchestrator*. Nothing crosses MCP by default, which means a naive integration produces a
+clean run, a plausible score, and zero recorded steps.
+
+---
+
+## 1. Why this benchmark, in one paragraph
+
+MCPMark proves a replayed scenario survives a **fresh instance of the same task**.
+τ²-bench proves it survives a **different task of the same shape** — "cancel order W1234
+for user A" against "cancel order W5678 for user B". Its domains are built out of
+policy-constrained task families over a shared database, which is exactly the surface that
+per-step parameter derivation exists for. Nothing else public tests the *calculated* half
+of calculated replay this directly. It also carries the name recognition that makes a
+result travel.
+
+Domains: `retail` (115 tasks), `airline` (50), `telecom` (114), plus `mock` and
+`banking_knowledge`. Python ≥3.12 <3.14, `uv sync`, LiteLLM-compatible keys in `.env`.
+
+---
+
+## 2. Reading a trajectory
+
+The link that prompted this doc:
+
+```
+https://taubench.com/trajectory-visualizer?model=qwen3.5-397b-a17b-think_sierra_2026-03-02&domain=retail&task=0
+```
+
+It is a client-side viewer over the simulation JSON that `tau2 run` writes to
+`data/simulations/`. Two modes — **Trajectories** and **Tasks** — and the URL pins one
+`(model, domain, task)` triple. The same data is browsable locally with `tau2 view`, which
+is the version to use while building the adapter, because it reads your own runs.
+
+An episode is a four-party loop, and the visualizer shows it as one interleaved column:
+
+```
+Agent ──message──▶ User ──message──▶ Agent ──tool_call──▶ Environment ──result──▶ Agent
+```
+
+| Element | What it is | What we care about |
+|---|---|---|
+| **UserMessage** | the *simulated* customer, driven by `--user-llm` | turn 1 carries the intent; later turns supply parameters |
+| **AssistantMessage** | the agent's text, and optionally `tool_calls` | the text is what the COMMUNICATE check reads |
+| **ToolMessage** | the environment's result | executed by the orchestrator, not the agent (§3.1) |
+| **evaluation_criteria.actions** | *one* reference solution, replayed on a fresh env to derive the target DB hash | **not** a required path (§3.2) |
+| reward breakdown | DB / COMMUNICATE / ENV_ASSERTION / NL_ASSERTION | the product of whatever `reward_basis` lists |
+
+Episodes end on a stop token, a user-simulator "task complete" transfer, or `max_steps`.
+
+**The one habit worth forming:** when a task scores 0, open the trajectory and check
+*which* component was 0. A DB failure and a COMMUNICATE failure look identical in the
+aggregate and mean opposite things for us — see §3.3.
+
+---
+
+## 3. The three facts that decide the integration
+
+### 3.1 The grade comes from the transcript, not from the world
+
+*Corrected after the first working run. An earlier version of this section said the
+binding constraint was that the orchestrator executes tool calls. That is true — the agent
+returns an `AssistantMessage` containing `tool_calls` and the orchestrator invokes them
+against the domain environment — but it is not the constraint that decides the design.*
+
+The constraint that decides the design is in `evaluator_env.py`: τ² **does not grade the
+environment the episode ran in**. It constructs a *fresh* environment, replays the
+trajectory's `(tool_call, tool_result)` pairs onto it via `set_state`, and hashes that.
+
+> **Consequence:** the reward is earned by what appears in the messages. An agent that
+> executes the whole task perfectly against the live environment and reports it in prose
+> scores **zero**, because the graded environment received nothing.
+
+We measured exactly this. Claude Code solved retail task 0 with a call sequence matching
+the reference trajectory *exactly* — same tools, same item ids, same payment method — and
+scored `DB 0`, `reward 0`. Nothing in the results table said "your harness is broken"; it
+read like a hard task.
+
+τ²-bench also ships no MCP interface: domain tools are Python callables wrapped in `Tool`
+objects and called in-process. So bir sees nothing unless Claude Code makes real MCP calls
+of its own. Those two facts together — the trajectory must carry the calls, *and* Claude
+Code must make them over MCP — are what §4 has to satisfy at once.
+
+### 3.2 Reward is outcome-based, so replay is legitimately scoreable
+
+`evaluation_criteria.actions` is a reference trajectory, replayed on a fresh environment to
+establish a **target database hash**. The DB check compares hashes. Any sequence of tool
+calls reaching an equivalent end state passes. The docs are emphatic: *actions document one
+working solution, not the only acceptable one.*
+
+`ACTION` — exact trajectory match — is a reward component that exists but is **not** in the
+default `reward_basis` for airline, retail or telecom, which is `["DB", "COMMUNICATE"]`.
+
+This is the fact that makes the whole exercise honest. A replayed scenario takes the path
+*a previous run* took, which will not be token-identical to the reference actions. Under an
+action-matching benchmark that would be scored as deviation. Under τ²-bench it is scored as
+what it is: the same outcome, reached differently. We are not exploiting a loophole — we
+are using the benchmark's stated philosophy.
+
+**Check `reward_basis` per task before running.** If a domain or task set includes `ACTION`,
+exclude it and say so, rather than discovering it in the aggregate.
+
+### 3.3 Reward is a *product*, and COMMUNICATE is in the basis
+
+`reward = db_reward × communicate_reward`. `communicate_info` is a list of strings the
+agent must have said to the user, matched as substrings.
+
+Now read that against what direct-mode replay does. From the README: on a scenario whose
+every step is a wrapped MCP tool, the sequence runs through connections the proxies already
+hold — **zero model tokens** — and the model's only job is one tool call that reads the
+results.
+
+> A replay that executes every step perfectly and never *tells the customer the refund
+> amount* scores `1 × 0 = 0`.
+
+The design already accounts for this: `steered_full` means every planned step ran with
+scenario-derived inputs **and the response model was produced**. So the machinery should be
+right. But this is the single highest-value assertion to verify early, because its failure
+signature — every task at exactly 0.0 with a healthy step ledger — is easy to misread as a
+plumbing bug and waste a day on.
+
+τ-bench is, in this specific sense, a *harder* test of replay than MCPMark: MCPMark grades
+the world, τ-bench grades the world **and** what the agent said about it.
+
+---
+
+## 4. Architecture — two environments
+
+Both constraints in §3.1 have to hold at once: Claude Code must make **real MCP calls**
+(or bir records nothing), and the **trajectory must carry those calls** (or the grade is
+zero). One environment cannot satisfy both — if Claude Code executes a call and the
+adapter also hands it to the orchestrator, the orchestrator executes it a second time,
+the exchange fails as already-exchanged, and the trajectory records an error the evaluator
+cannot replay.
+
+Two copies of the domain resolve it. Each executes every call exactly once.
+
+```
+tau2 orchestrator ── graded environment ◀── executes the calls the adapter emits
+  ├── user simulator (--user-llm)              (this is what the trajectory records)
+  └── ClaudeCodeAgent
+        ├── ToolBridge ── private copy of the domain ◀── Claude Code acts here
+        │      (in-process HTTP, ephemeral port, one per episode)
+        └── claude -p / --resume, one call per orchestrator turn
+              └── mcp__tau2__* ─▶ bir-proxy ─▶ node mcp-stdio-bridge.mjs ─▶ ToolBridge
+```
+
+The copies start identical — taken after `_initialize_environment` has applied the task's
+setup — and replay the same calls in the same order, so they stay in step. Claude Code's
+results are real; the orchestrator's execution is what gets graded.
+
+### 4.1 Taking the copy
+
+`build_agent` hands the agent `environment.get_tools()`: live `Tool` objects whose `_func`
+is a bound method of the domain's toolkit, which owns the db. Deep-copying **the toolkit**
+yields an independent domain with its own database:
+
+```python
+toolkit = self.tools[0]._func.__self__      # RetailTools, etc.
+shadow = copy.deepcopy(toolkit)
+shadow_tools = list(shadow.get_tools().values())
+```
+
+Copying the toolkit rather than rebuilding the domain from the registry preserves any
+task-specific initialization that would otherwise have to be reconstructed from the task
+object. Verify independence rather than assuming it: write through the copy and confirm
+`environment.get_db_hash()` does not move. `selftest_bridge.py` is that check.
+
+### 4.2 One Claude Code turn becomes N+1 messages
+
+τ²'s validator (`validate_message_format_default`) rejects a message carrying both text and
+tool calls. So a turn in which Claude Code makes four calls and then replies is *five*
+messages, each separated by the orchestrator executing one call. The adapter queues them
+and hands over one per `generate_next_message`; the tool results the orchestrator returns
+are ignored, because Claude Code already saw the equivalent result from its own copy and
+the run is over by then. Their purpose is the trajectory.
+
+> **Stamp each message when you hand it over, not when you build it.** τ² sorts the final
+> trajectory by timestamp. A queue built in one burst carries timestamps *older* than the
+> tool results answering it, so the sort files every assistant message ahead of every tool
+> message and the evaluator refuses to replay it. It is intermittent — when the clock does
+> not tick between batches the timestamps tie and the stable sort happens to be right — so
+> it presents as roughly one run in three dying with an infrastructure error.
+
+Two smaller things in the same area:
+
+- **Give every assistant message a `cost`, even zero.** `get_cost` discards the whole
+  conversation's cost if any single one is `None`, so the tool-call messages carry `0.0`
+  and the reply carries the run's total.
+- **Translate the token counts.** τ² sums `prompt_tokens` / `completion_tokens`; Claude
+  Code reports `input_tokens`, `output_tokens` and two cache figures. Cached reads are real
+  prompt tokens and are most of the prompt on a resumed session, so they belong in the
+  total or every turn after the first looks nearly free.
+
+### 4.3 Passing the policy
+
+The agent constructor receives `domain_policy` as a string. It goes into the session as
+`--append-system-prompt` (or `--append-system-prompt-file` — retail's `policy.md` is long
+enough that a file is the better shape). Do not paste it into the per-turn prompt: it is
+constant across the episode, and repeating it per turn inflates every cost number we are
+trying to measure.
+
+### 4.4 Per-turn invocation
+
+```bash
+# turn 1
+claude -p "$USER_MESSAGE" \
+  --append-system-prompt-file "$POLICY_MD" \
+  --settings "$BIR_SETTINGS_JSON" \
+  --mcp-config "$EPISODE_MCP_JSON" \
+  --output-format json \
+  --permission-mode dontAsk \
+  --allowedTools "mcp__tau2__*" \
+  --max-turns 30
+
+# turns 2..n
+claude -p "$USER_MESSAGE" --resume "$SESSION_ID" --output-format json ...
+```
+
+`--allowedTools "mcp__tau2__*"` and nothing else. The agent has no business touching the
+filesystem or shell in a customer-service episode, and an allow list that says so turns a
+whole class of confusing failures into a clean permission denial.
+
+Check `system/init`'s `mcp_server_errors` on turn 1 of every episode. A `--mcp-config`
+entry that fails validation is skipped *silently*; the episode then runs with no tools and
+scores 0, which reads exactly like a hard task.
+
+---
+
+## 5. BIR wiring
+
+```bash
+export BIR_AUTH_URL=https://your-basein-service
+export BIR_CONTROL_URL=http://127.0.0.1:53411      # not discovery
+export BIR_CONTROL_TOKEN=...
+export BIR_REPLAY_ALLOW_SERVERS=tau2
+BIR_REPLAY=1 bir-hooks 2>&1 | tee -a results/tau2-audit.log
+```
+
+`BIR_CONTROL_URL` / `BIR_CONTROL_TOKEN` are mandatory, not convenient: the README calls
+them the only reliable channel for an SDK session, and every episode turn is one. Discovery
+across a few hundred harness-spawned processes is how an arm silently degrades to Tier 2.
+
+`BIR_REPLAY_ALLOW_SERVERS=tau2` is the whole allowance. Direct execution dispatches steps on
+connections that are already open and never reaches the permission system, so the list is
+the blast radius.
+
+### 5.1 The per-prompt run boundary — the structural mismatch
+
+This is the one that will shape the results, and it is worth understanding before the first
+run rather than after.
+
+`onPrompt` rolls a **new run on every prompt**. In τ²-bench, every user-simulator turn is a
+prompt. So an episode of six turns is **six runs**, not one, and matching happens per turn
+against a library of per-turn scenarios.
+
+What follows from that:
+
+- **Turn 1 carries the intent** — "I want to cancel my order" — and is where a match should
+  fire. Later turns are mostly parameter supply ("it's W5678") and are short, cheap, and
+  poor matching material.
+- **A scenario is a turn's tool chain, not an episode's.** Cross-episode reuse therefore
+  happens turn-by-turn, and the savings ledger sums over turns.
+- **Per-run usage watermarking matters more here than anywhere.** Six runs share one
+  Claude Code transcript; summing the file gives turn 6 the cost of turns 1–6. The design
+  doc's `markTranscriptUsage` / `usageSince` delta is not an optimisation in this setting,
+  it is the difference between a real number and a fabricated one. Check `measured: true`.
+
+If the results show matching that fires reliably on turn 1 and rarely after, that is not a
+bug — it is the honest shape of the product against a multi-turn benchmark, and it should
+be reported that way.
+
+---
+
+## 6. Running it
+
+```bash
+git clone https://github.com/sierra-research/tau2-bench && cd tau2-bench
+uv sync && cp .env.example .env      # user-simulator key goes here
+
+# smoke: one task, one trial
+tau2 run --domain retail --agent claude_code --user-llm <model> \
+  --num-trials 1 --task-ids 0
+
+tau2 view                            # read the trajectory you just produced
+
+# baseline arm, recording only (BIR_REPLAY unset)
+tau2 run --domain retail --agent claude_code --user-llm <model> \
+  --num-trials 4 --max-concurrency 4
+
+# replay arm, library seeded from the passing runs above
+BIR_REPLAY=1 tau2 run --domain retail --agent claude_code --user-llm <model> \
+  --num-trials 4 --max-concurrency 4
+```
+
+Pin the `--user-llm` model and version across all arms. It is the environment; changing it
+changes the task, and a shifted baseline invalidates every comparison.
+
+---
+
+## 7. Verification gates
+
+Run these on `--task-ids 0`, in order. Each has a distinct failure signature, and they are
+ordered so that the cheapest and most misleading failures are caught first.
+
+| # | Gate | Failure looks like | Means |
+|---|---|---|---|
+| 1 | `mcp_servers` in `system/init` lists `tau2` as connected | score 0, no tool calls | config skipped silently |
+| 2 | `bir doctor` green, `metadata.tier == 1` inside a harness-spawned run | recording exists but thin | Tier 2 — hooks not loading |
+| 3 | The proxy recorded the episode's tool calls | empty step stream on a passing task | traffic bypassed bir |
+| 4 | Reward > 0 on a task the reference solves | *every* task exactly 0.0 | the trajectory carries no tool calls (§3.1) |
+| 5 | No duplicated side effects | reward 0, doubled refund in the env | one environment, executed twice (§4) |
+| 6 | Zero retries across a multi-trial run | ~1 run in 3 dies as "infrastructure error" | the timestamp race (§4.2) |
+| 7 | `Avg Cost/Conversation` is a number, not `n/a` | costs silently absent | an assistant message with `cost=None` |
+| 8 | On the replay arm: DB=1 **and** COMMUNICATE=1 | reward 0, step ledger healthy | **§3.3** — replay produced no response text |
+
+**Every one of these fails quietly.** That is the point of the list. Three of them we hit
+for real, and none announced itself:
+
+- **Gate 4** produced a clean table reading `reward 0.0000, DB match 0/1` on a task the
+  agent had solved *perfectly* — its calls matched the reference exactly. It reads like a
+  hard task, not a broken harness.
+- **Gate 6** was absorbed by τ²'s own retry logic, so the run "succeeded" while costing 3×
+  and mixing infrastructure errors into what would look like task variance across a sweep.
+- A fourth, not a gate but worth knowing: on Windows, npm's `claude.cmd` shim routes
+  arguments through `cmd.exe`, which re-parses them. Any user turn containing `&`, `|`,
+  `%`, `^` or a newline arrives truncated, and the agent answers *"your message came
+  through empty."* Call `claude.exe` directly and refuse to fall back to the shim.
+
+The lesson generalises past this benchmark: a harness bug and a hard task produce the same
+number. Read one transcript per configuration before trusting any aggregate.
+
+---
+
+## 8. What to measure
+
+Per arm, per domain: reward (mean), pass^k across trials, `$`/episode and `$`/turn from
+`--output-format json`'s `total_cost_usd`, wall-clock per episode, and turn count.
+
+Per replayed turn, from the audit log and the execution ledger: outcome
+(`steered_full` / `diverged` / `not_steered` / `failed`), match similarity against
+`BIR_MIN_STEER_SIMILARITY` (default 0.92), direct vs steered mode, `deriveCostUsd` against
+`sessionCostUsd`, and the per-step verdicts.
+
+Two τ-specific splits worth reporting that MCPMark cannot give us:
+
+- **Reward decomposed.** DB and COMMUNICATE separately, always. An arm where DB holds and
+  COMMUNICATE slips is a precise, fixable finding about what replay omits; the product hides
+  it completely.
+- **Match rate by turn index.** Turn 1 versus turns 2..n (§5.1). This is the number that
+  says whether calculated replay generalises across a *task family* or only across
+  restatements of one task, and it is the question τ-bench was chosen to answer.
+
+Then the control from [benchmark.md](benchmark.md) §7 Phase 3, which matters more here than
+anywhere: τ-bench task families differ by *entity*, so a matcher that ignores which order
+id the customer named will replay cheerfully against the wrong order. Build the near-miss
+set by swapping entities between same-shape tasks — same words, different customer — and
+report the **false replay rate**. In a retail domain, that number has an obvious real-world
+reading, which is exactly why it should be published whatever it says.
+
+---
+
+## 9. Known limits
+
+- **The MCP wrapper is ours.** τ²-bench has no MCP interface, so the transport under test is
+  a shim we wrote. A sceptic can question it; the answer is to publish it and to run gate 3
+  and gate 5 in public. MCPMark has no such caveat, which is the other reason it is primary.
+- **The user simulator is an LLM**, so the environment is stochastic across trials. pass^k
+  here mixes agent variance with simulator variance. Pin the simulator model and report it
+  as a limit rather than pretending the variance is all ours.
+- **Six runs per episode** (§5.1) makes every per-run number a per-turn number. Say so in
+  the table headers; a `$`/run figure that a reader takes for `$`/task is off by ~6×.
+- **`banking_knowledge` stays out.** Same reasoning as [benchmark.md](benchmark.md) §5 — it
+  is retrieval over a document set, not a repeatable tool chain, and there is nothing there
+  for a scenario to be.
+- **Voice mode is out of scope.** Full-duplex needs `FullDuplexAgent` and a realtime
+  provider; Claude Code is half-duplex and so is this integration.
+
+---
+
+## Sources
+
+- τ²-bench — [repo](https://github.com/sierra-research/tau2-bench) · [agent developer guide](https://github.com/sierra-research/tau2-bench/blob/main/src/tau2/agent/README.md) · [orchestrator guide](https://github.com/sierra-research/tau2-bench/blob/main/src/tau2/orchestrator/README.md) · [evaluation docs](https://github.com/sierra-research/tau2-bench/blob/main/docs/evaluation.md)
+- Papers — [τ²-bench (arXiv:2506.07982)](https://arxiv.org/pdf/2506.07982) · [τ-bench (arXiv:2406.12045)](https://arxiv.org/pdf/2406.12045)
+- [Trajectory visualizer](https://taubench.com/trajectory-visualizer/?model=qwen3.5-397b-a17b-think_sierra_2026-03-02&domain=retail&task=0) · [leaderboard](https://taubench.com/leaderboard/)
+- [Claude Code headless mode](https://code.claude.com/docs/en/headless)
+- This repo — [benchmark.md](benchmark.md) · [calculatedReplay.md](calculatedReplay.md) · [calculatedReplayGuide.md](calculatedReplayGuide.md)

package/package.json

@@ -0,0 +1,56 @@
+{
+  "name": "@basein/runner",
+  "version": "0.1.0",
+  "description": "A recording MCP proxy: sits between any MCP client and its MCP servers, executes each call on the client's behalf, and records the run as a reusable BaseIn scenario.",
+  "type": "module",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/eran0lavi/BaseInstRunnerMCP.git"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "engines": {
+    "node": ">=20"
+  },
+  "bin": {
+    "bir": "dist/bin/bir.js",
+    "bir-proxy": "dist/bin/bir-proxy.js",
+    "bir-hooks": "dist/bin/bir-hooks.js",
+    "bir-scenario": "dist/bin/bir-scenario.js"
+  },
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "files": [
+    "dist",
+    "!dist/**/*.map",
+    "README.md",
+    "LICENSE",
+    "docs",
+    "!docs/next.md"
+  ],
+  "scripts": {
+    "build": "tsc -p tsconfig.json",
+    "typecheck": "tsc -p tsconfig.test.json --noEmit",
+    "pretest": "tsc -p tsconfig.test.json",
+    "test": "node test/run-tests.mjs",
+    "prepublishOnly": "npm run build",
+    "test:smoke": "node test/smoke-replay.mjs"
+  },
+  "keywords": [
+    "mcp",
+    "proxy",
+    "recorder",
+    "claude-code",
+    "agent",
+    "observability",
+    "replay",
+    "scenario"
+  ],
+  "dependencies": {},
+  "devDependencies": {
+    "@types/node": "^20.14.0",
+    "typescript": "^5.6.0"
+  }
+}