@jr2/orchestrator 0.4.1 → 0.5.1

package/docs/reference/decisions/0010-bdd-acceptance-tests.md

@@ -0,0 +1,160 @@
+# BDD acceptance tests via Cucumber.js (black-box, scenario-isolated)
+
+jr2's per-package tests are `node --test` suites that run in-process and socket-free (e.g. the CLI's `cli.test.ts` calls
+`main(argv, io)` with `fetch` bound to `app.request`). Those are fast and cover unit and integration behavior, but
+nothing exercises the _system_ the way a user does — the real `jr2` binary, a real HTTP socket, exit codes, signals.
+This ADR adds a second, complementary tier: **black-box end-to-end acceptance tests, written in Gherkin and run by
+Cucumber.js**, living in a top-level `./features/` workspace package (`@jr2/e2e`).
+
+## What we decided
+
+- **Two tiers, not one.** In-process `node --test` stays the unit/integration tier (fast, no sockets, per package).
+  Cucumber.js is the **e2e** tier: each step spawns the real `jr2` binary as a child process against a real orchestrator
+  server on an ephemeral port, and asserts only on what a shell sees — captured stdout (the one machine-readable
+  result), stderr (human activity), and the exit code. The e2e tier does not re-test what the in-process tier already
+  covers.
+- **The per-scenario orchestrator is the image entrypoint, not a CLI verb.** There is no host dev mode (ADR-0019), but
+  the Orchestrator's server entrypoint is an ordinary Node process — the test fixture boots exactly that process on the
+  host, with `--url`/token env pointing the spawned `jr2` binary at it. Same real server the cluster runs, no cluster,
+  no user-facing verb. The wire-compatible **stub Harness** (ADR-0011) is a fixture owned by this tier and served the
+  same way.
+- **`Feature` → `Rule` → `Scenario` maps to criteria.** A `Rule` is one acceptance criterion; a `Scenario` is an example
+  of it. This is organizational only — `Rule`s carry no shared state.
+- **`Scenario` is the isolation unit — and the environment is part of it.** Each scenario gets its own scaffolded temp
+  instance (via the real `jr2 init`), its own server process + sqlite store, torn down in an `After` hook. It also gets
+  its own **environment**: the World spawns nothing from the ambient one, because a tier that inherits the developer's
+  shell has made that shell a silent participant in every scenario. Two classes are sealed. Every `JR2_*` variable is
+  scrubbed, since they say where to talk and as whom — an exported `JR2_URL` would point a whole suite at someone's
+  deployed instance, and `JR2_NAMESPACE` reaches the entrypoint's data-plane switch
+  (`composesSandbox && namespace !== undefined`), which would turn `workspace.feature`'s "no cluster, so it faults
+  pointedly" into a real Sandbox provisioned on whatever cluster was configured. Each tier adds back exactly what it
+  means to set, and the variables a scenario's own config reads arrive through the instance's `.env`, a file the step
+  wrote. `KUBECONFIG` is pinned to `/dev/null` wherever there is no cluster in the story, so a run-verb falling through
+  to the kube path finds nothing rather than the last cluster a human used; `@kind` and `@dist` own a namespace on a
+  real cluster and keep the real one. (The leak this closes was not theoretical: `cli-contract.feature`'s exit-code
+  scenario hung for its full 120s timeout against an unreachable home cluster, where it should fail in milliseconds —
+  and it hung only on the machine that had one.) Nothing is shared across scenarios, so the suite is safe under
+  `cucumber-js --parallel`.
+- **An opt-in `@kind` tier for the data plane** (added once the `workspace()` slice landed — ADR-0012). Workspaces are
+  always real Sandboxes, so the only way to test them is against a real cluster: the `@kind` scenarios drive the
+  operator's Sandbox CR, a pod, the node's read-only Repo cache, an in-pod git worktree, and the Harness endpoint —
+  faking only the LLM. (Originally the Sandbox ran a **dev Harness image**, a wire-compatible stub in a container with
+  `git`; [ADR-0038](0038-jr2-up-builds-every-image-it-deploys.md) retired it. The pod runs the STOCK Harness now — real
+  pi, the real Menu through the Custodian, real Working tools — and the substitution moved to the provider: a scripted
+  OpenAI-compatible endpoint the World serves from the host. Which is what makes this tier a second pi canary.) They are
+  tagged `@kind` and **excluded from the default profile**, so the everyday suite needs no docker. Bring-up is the
+  product's own path (ADR-0019): one shared VANILLA kind cluster and nothing else, then **`jr2 up` per scenario into a
+  fresh namespace** — which since ADR-0038 builds and loads every image it deploys, so no image is pre-loaded by hand.
+  Namespace-as-identity makes the scenario the isolation unit here too, so nothing is instance-bound to the cluster.
+  **The host address the pods dial back on is discovered, not assumed.** The scripted provider runs host-side, so the
+  tier needs the one address that routes from a pod back to this process — and it is not the same on every machine. On
+  Linux the daemon IS the host, so the kind bridge's gateway is the answer. Under a VM-backed daemon (colima, Docker
+  Desktop) that gateway is the VM: measured on colima, `172.18.0.1` reaches nothing while `host.docker.internal`,
+  `host.lima.internal` and `192.168.5.2` all answer — and on Linux those names mean nothing. So the candidates are
+  ordered by platform and the winner is VERIFIED, by `curl`ing the provider that is already listening from inside a kind
+  node (a pod's egress routes through it), for the price of one `docker exec` memoized per worker process. Taking the
+  bridge network's first IPAM entry is what broke: docker lists the IPv6 subnet first on some hosts and it carries no
+  gateway at all, so the tier now reads every entry that HAS one, IPv4 first. Failing to find any address is one error
+  naming every candidate tried, not an assertion in each scenario's `Before` hook. Measured: **22/22 on macOS/colima,
+  ~3m20 warm at degree 4** against the ~1m35 Linux figure below — the VM is slower, not different. **The tier runs
+  parallel, and three properties are what make that safe.** Scenario isolation (own namespace, own scripted model port);
+  no shared mutable image name between concurrent converges of one checkout — a Sandbox Image builds straight to its
+  content tag, with no intermediate, because the Harness arrives by volume at pod time
+  ([ADR-0037](0037-an-instance-builds-its-sandbox-images-jr2-injects-the-harness.md)) — while
+  [ADR-0041](0041-a-build-the-host-already-holds-is-not-spent-again.md) makes a warm scenario's converge build nothing
+  (~13s fresh-namespace converge, every build disk-skipped); and connection retries at the two seats that dial a Sandbox
+  before anything else ever has — [ADR-0042](0042-ready-is-not-routable-an-admission-retries-its-connection.md) has the
+  diagnosis (a Sandbox CR at `phase: Ready` is not yet ROUTABLE, so an unretried first dial is a lost turn; the Menu
+  read was a second instance of the same defect). Parallelism does not create those windows, it only samples them more
+  often — which is why any tier flake that vanishes at `--parallel 1` should be read as a routability-class defect
+  first, not as an isolation bug. Measured: **degree 4 → 8 consecutive runs, 88/88 scenarios, 1m30–1m43s each** on a
+  sealed tree (serial: ~5m). **The wired default is now `--parallel 4`** (in the `kind` profile, so a direct
+  `npx cucumber-js --profile kind` gets it too); pass `--parallel 1` to bisect a suspected isolation bug. Three tier
+  changes came out of the hunt and stay. A FAILED `@kind` scenario dumps its evidence (the provider's request count,
+  `jr2 status`, the Harness's `?view=history`, every pod log with timestamps, events, EndpointSlices) to
+  `features/.tmp/kind-failures/` before its namespace is deleted. The tier's own workflow ROUTES `agent.fault` instead
+  of ignoring it — an ignored terminal fault is an invisible one, and a tier that hangs where it could name the reason
+  is a tier that costs a session per defect. And **every** scenario, passing or not, is now scraped for what ADR-0042's
+  retries COST (`jr2.routability seat=… attempts=… ms=… last=…`), against a budget of 30s or 8 attempts judged per
+  worker in `AfterAll` — two meters because a REJECT spends attempts and a dropped SYN spends time, and the tier
+  measured one of each in its first two runs (2 attempts / 10667ms, then 3 attempts / 502ms / `ECONNREFUSED`). That last
+  one is the tier watching the fix rather than merely enjoying it: the retry is absorbed by design, so without it a
+  cluster drifting toward the 90s window would keep passing — slightly slower, silently — until the day it did not.
+- **An opt-in `@console` tier for the Console's UX** (added with
+  [ADR-0032](0032-the-console-unlocks-with-the-instance-token.md)). The Console's risky behavior is interaction —
+  token-mode switching, the frame-triggered gate inbox, selection vs. folding — which no reducer test sees and no CLI
+  step drives. These scenarios hold a **Playwright page inside ordinary Cucumber steps** (Playwright the _library_, not
+  the `@playwright/test` runner): the browser is a second driver of the same black box, reusing the per-scenario
+  orchestrator fixture unchanged, and a scenario may drive both (the CLI starts a run; the browser sees it park). Tagged
+  `@console` and **excluded from the default profile** exactly as `@kind` is — the everyday suite must not require a
+  Chromium install — but they need no docker: a host-booted orchestrator serves the Console fine. Scope discipline: they
+  assert what a user does and sees, never pixels or layout; what the page _believes_ stays in `console-store.test.ts`.
+- **Harness conformance, in `@jr2/harness`, in the default gate**
+  ([ADR-0027](0027-the-harness-is-jr2s-own-server-flue-retires-the-wire-stays.md)). The three tiers above all reach the
+  Harness through the **stub Harness**, which is a hand-written model of the _wire_ — it holds no conversation, so it
+  can report a turn settled `ended` or `aborted` without ever having had a turn to settle. That is the right fixture for
+  testing jr2, and it is blind by construction to what the real runtime keeps, drops, or sends onward. Those claims are
+  covered by the conformance suite in `packages/harness/test/`, driven through the real turn loop — pi at the exact pin,
+  the Menu over a real socket to the Custodian's address, a **scripted provider** — so a turn's shape (where it stalls,
+  what it half-emits) is chosen by the test rather than by a model. The witness is the message array the provider
+  receives on the _next_ turn, which is the only place these claims are visible. Not a separate tier: it runs as part of
+  `pnpm -r test` (no docker, no cluster), and it is the canary for pi bumps — **run it before bumping the pin**. (A
+  predecessor tier that ran the retired foreign harness runtime at its pin dissolved into this suite — ADR-0027.)
+- **The Custodian suite, opt-in, against the real engine**
+  ([ADR-0059](0059-a-harness-holds-stand-ins-and-the-custodian-holds-the-keys.md)). The Custodian's claims — the swap on
+  the wire, the strip, the refusals, the dial guard, TLS verification upstream, SSE timing, abort, and a real SDK
+  through `HTTPS_PROXY` — are Envoy's behaviour under jr2's configuration, so only the pinned Envoy image can prove
+  them. `just custodian-test` runs `node --test` (`packages/cli/test/custodian.docker.ts`) against it under docker, on
+  the host network with free ports, with local HTTPS upstreams under their own CA; about 23 seconds. Opt-in for the
+  `@kind` reason — it needs docker — and NOT part of `pnpm -r test`. What the default gate keeps is the rendering:
+  `custodian.test.ts` pins the bootstrap and script a pod's maker writes, socket-free. The `@kind` tier then proves the
+  Custodian in a real pod, in both placements, against a model provider that refuses any request without the key.
+- **An opt-in `@model` tier, against a real model**
+  ([ADR-0066](0066-a-model-tier-runs-a-real-model-and-a-change-to-what-it-reads-runs-the-tier.md)). Every tier above
+  fakes the model, so none can see a change in what a model DOES with what jr2 shows it — and ADR-0029/0062 measured
+  that such changes are large. This tier is `@kind` with the last fake removed: the same cluster and bring-up, `jr2 up`
+  per scenario into a fresh namespace, and behind the Custodian a real endpoint named by a profile
+  (`features/model-endpoints/<name>.json`, selected by `JR2_MODEL_ENDPOINT`). Its scenarios are CLAIMS — an ADR's
+  measured claim as N trials with a floor, scored into `features/.tmp/model-scorecard.json` against a committed baseline
+  — and THE RUN, one workflow from a failing test to a commit verified in the pod. It runs the endpoint profile's
+  `workers` scenarios at once, and `workers` × the profile's `parallel` runs must stay within what the endpoint serves
+  and what the host holds. Not in CI and not in `pnpm -r test`: `just e2e-model` probes the endpoint and says `skipped:`
+  when nothing answers, and whoever changes what the model reads runs it before calling the change done. A third pi
+  canary.
+- **Shared cluster state is the one thing a scenario cannot own, and the pool is PROCESSES.** `--parallel` runs cucumber
+  workers as separate processes, so a fixture memoized "once per process" runs once per WORKER — four times,
+  concurrently, against one cluster. The tier has exactly one such fixture: the git seed (`steps/seed.ts`) it serves
+  in-cluster, which is cluster-scoped where everything else is namespaced per scenario. `kubectl apply` creates with
+  GET-then-POST, so the workers that lose that race are told `AlreadyExists` — which an `ensure` treats as success,
+  because the object being there is the whole ask. The rollout wait that follows is the real rendezvous, and it is
+  idempotent for every worker. (The same property is why the `@dist` profile is serial: its publish would otherwise
+  stand up one verdaccio per worker on a single port.)
+- **A workspace package, not a bare folder.** `./features/` is `@jr2/e2e` so it owns its own `xstate` (+ cucumber)
+  dependency: a scaffolded instance's `workflows/*.ts` `import "xstate"`, resolved by walking up from the temp dir, and
+  the repo root has no `xstate`. The package's `node_modules` satisfies it. It is top-level (not under `packages/cli/`)
+  because an e2e test exercises most packages at once — the CLI is the common entry point, not the only conceivable one.
+- **Zero-build, like the rest of the repo.** Cucumber.js v13 runs `.ts` ESM step definitions directly under Node 24's
+  native type-stripping — no `ts-node`, no transpile step. (Verified before adopting it.)
+
+## Considered options
+
+- **Express BDD as a convention in `node --test`** (a Given/When/Then helper, no `.feature` files): keeps zero deps, but
+  loses the plain-English acceptance specs that are the point of writing them at the user boundary. Rejected — we want
+  the `.feature` files as readable, reviewable criteria.
+- **In-process e2e** (drive `main(argv, io)` instead of a subprocess): faster, but it is integration, not e2e — it can't
+  catch bin/shebang, exit-code, signal, or attach bugs. That's already the job of the in-process tier; the e2e tier
+  exists precisely to cover the subprocess boundary.
+- **Requiring a cluster for all e2e** (every scenario through `jr2 up`): honest but slow, and it gates the everyday
+  suite on docker. Rejected — the control-plane surface is fully exercisable against the bare server process; only the
+  data plane needs kind, and it has its own tier.
+
+## Consequences
+
+- `@cucumber/cucumber` is the **first test-framework dependency** in a repo that otherwise uses only `node --test` — a
+  deliberate deviation, scoped to `@jr2/e2e` (it does not leak into the kit packages). `playwright` joins it there, same
+  scoping, browser binaries installed only where `@console` runs (CI, or `npx playwright install chromium` locally).
+- The suite is run with `pnpm --filter @jr2/e2e test:e2e`; `@jr2/e2e` exposes **no** `test` script, so `pnpm -r test`
+  stays fast and unit-only.
+- Human-in-the-loop delivery is covered at the gates API (`features/agent-and-gates.feature` — the agent played over
+  `/agents/<iid>/*`, the human over `/runs/:id/gates/:gate/events`); the `jr2 send --gate` CLI verb is covered at the
+  in-process tier. A CLI-driven e2e gate scenario remains a candidate follow-up `Rule`.

package/docs/reference/decisions/0011-workflow-defined-events.md

@@ -0,0 +1,141 @@
+# Control events are workflow-defined and delivered by closure, not routed
+
+Two decisions, out of the jr-parity design exercise: **jr2 ships the event mechanism and zero events**, and **an event
+binds to a Machine state through the actor that exposes it** — delivery is a closure created at invoke time, so no
+routing layer exists anywhere.
+
+## `defineEvent`: mechanism, not policy
+
+`defineEvent({ name, input, audience?, semantics? })` produces a pure-data definition: a name, a zod input schema (flat
+tagged objects, never `oneOf` — the ADR-0006 encoding rule), an optional audience tag (`agent` | `external` | `any`;
+unset, each writing of the event belongs to the party its state, or the nearest state above it, invokes — ADR-0015), and
+an optional semantics tag (`ack`, or the reserved `poll`; a def declares no output — ADR-0006). It is a **pure factory —
+no import-time side effects, no global registry**. Because actor **inputs are serializable** (ADR-0007) they carry event
+_names_, so resolution needs a name→def scope; that scope is **per-Machine**: `jr2Setup` takes the defs as values and
+attaches the vocabulary to the machine (ADR-0015 — discovery reads it there; the module contract is
+`export const machine` alone), and `gate`/`agentRun` resolve names against **the Machine that invoked them** —
+`self._parent.logic`, public xstate API — never a run-wide set. Names are local to their Machine: `coding`'s `approve`
+and `release`'s `approve` may differ, and one run may hold both, because a Machine nested inside another resolves
+against its own defs and no map is ever merged. That is what makes a Machine composable by plain `invoke` (ADR-0049):
+the importing Machine neither re-declares nor sees the nested one's events, and `workspace()`/`pool()` propagate
+nothing. An unlisted name fails at invoke time with an error naming the Machine and its declared set. The attachment is
+keyed on the machine's `config`, which xstate's `.provide()` passes through unchanged, so the host's per-run provide and
+a test's `.provide()` both keep it. We rejected a global name registry (import = registration): it couples every
+workflow in an instance — and any future npm-distributed one — through one namespace, and attribution can't be inferred
+at import time. We also retired the run-scoped set this ADR first specified (`RunBinding.events`, the root's
+vocabulary): it made every nested Machine's events the root's problem to re-declare, and turned a
+same-name-different-payload pair into a collision that the per-Machine defs had already avoided.
+
+`defineEvent` also carries the TypeScript side: `jr2Setup` derives the machine's event union from the defs — one source
+of truth, no schema/type drift, and the machine stays fully typed for xstate tooling.
+
+A def's `input` is what its sender supplies, and nothing the Machine adds. Where a pick starts work,
+`fromPick(def, fill)` carries those params into the invoked Effect's input by name, and `fill` adds the Machine's part;
+the load walk refuses a param nothing reads and a `fill` that overwrites one (ADR-0067, ADR-0015).
+
+There is no jr2-blessed event vocabulary. `request_review`, `review_verdict`, `approve` are the _coding workflow's_
+words; another workflow defines others. jr2 owns only definition, transport, validation, and delivery.
+
+## Agent side: invoke-scoped registration, delivery via `sendBack`
+
+On start, the `agentRun` actor registers, for its instance id, a toolset — the menu derived from its invoking state's
+transitions (ADR-0015) — whose handlers **close over that invocation's `sendBack`**; on stop it deregisters. A tool call
+→ validate against the named schema → `sendBack` a typed event, plus its sender → the event lands on **the state that
+invoked the agent** (bubbling to ancestor states per statechart semantics — a machine can handle a
+`report_blocked`-style event once at its root). The binding _is_ the closure.
+
+- **No routing.** There is no instance→actor table and no workflow-side forwarding boilerplate; delivery works at any
+  nesting depth (workspace wrappers, pool children) with zero authored code.
+- **State-scoped menus for free.** A transition swaps actors, which swaps registrations; the surface an Agent sees is
+  exactly the live registration.
+- **One catch point.** A call for an unregistered iid (settled run, exited state) is rejected at the demux.
+
+**Transport: the Agent's registration is read from the Sandbox** (ADR-0013). The Orchestrator speaks no MCP: it serves
+`GET /agents/:iid/surface` + `POST /agents/:iid/events` under the Sandbox token, and the pod's Harness reads them
+through the Custodian on `localhost`, which holds the token, and presents the surface to its model as the Menu (the
+Harness re-reads per Submission, so the iid in the path is all the addressing needed). Lookup, validation, delivery, and
+lifecycle stay implemented once, in the registration table.
+
+## External side: the same primitive over HTTP (`gate`), and each gate is a resource
+
+`gate` is the symmetric actor for **every non-agent caller** — humans (`jr2 send`, a UI), webhook translators, CI, other
+systems. "Human" is policy, not mechanism, so the actor is not named for one caller. Each invocation is an addressable
+**gate**: input is `{ gate?, meta? }` — `gate` an optional authored id (the id is derived when absent — below), `meta`
+serializable caller/integration context (PR URL, title); its accepted set derives from the gated state's transitions,
+its ancestors' and its plain children's (ADR-0015). Registration is per-gate: `GET /runs/:runId` lists open gates
+(`{ gate, accepts (names + schemas), acceptsNow, meta }` — `acceptsNow` the names the run would take from this Gate now,
+read the way an Agent's Allowed picks are, ADR-0029), `POST /runs/:runId/gates/:gate/events` validates the body against
+the named schema and delivers via `sendBack` into the gated state — or refuses a send the run would not take (ADR-0013)
+— and leaving the state destroys the gate. Gate ids are **run-scoped by mechanism** (the same `system`→run mapping that
+scopes event resolution), so `gate: "F-12"` in two concurrent runs cannot collide.
+
+Gate-as-resource is forced by concurrency: several children park simultaneously (three features in `humanReview` is the
+_normal_ case), all accepting `approve` — a run-level POST is ambiguous. And every real external surface already thinks
+in gates: a CLI listing shows pending decisions and acts on one; a UI inbox renders one card per gate (schemas drive the
+forms, `meta` drives the copy); a forge webhook translator finds its gate by matching `meta` (e.g. `prUrl`), which is
+why `meta` exists. Generalizing beyond humans also closes a hole hard-coded event types left: a push seam like "a
+work-source webhook wakes discovery" is just an idle state holding a gate accepting `work_ready` (the `wake` seam of
+ADR-0017). Two obvious later additions, both deferred: a cross-run inbox (`GET /gates`), and CLI segment-matching over
+derived gate ids (`--gate F-12` resolving an unambiguous segment, git-style like run-id prefixes).
+
+**The gate id is derived; authoring one is the exception.** Absent an authored `gate`, the id is the gate actor's own
+path below the run root, and jr2Setup's menu-derivation walk names an unnamed gate invoke with its state key path so the
+leaf segment is readable — `F-12.body.humanReview`, not xstate's `0.body.humanReview` default. This is unique wherever
+concurrently live siblings have distinct actor ids — the invariant any correct fan-out already maintains, because xstate
+keys children by id (a duplicate silently shadows the children-map entry and breaks `xstate.done.actor.<id>`
+correlation). Authored ids exist for meaningful flat names (jr's feature id) and collision on them is a loud invoke-time
+error. Derivation runs at actor start, not the input mapper — deterministic from structure, so recomputation on every
+(re)start is restore-stable, and one mechanism serves jr2Setup and plain-`setup` machines alike (the walk contributes id
+_quality_ only, never correctness). Rejected: a workflow-computed id as the default (`gate: context.feature.id`) — it
+quietly violated the address doctrine (Consequences below), making the gate id the one caller-facing address a workflow
+computed by hand, and a machine with a static id (`"humanReview"`) was correct standalone but a latent, timing-dependent
+collision once composed under a pool, firing only when two children park concurrently. Also rejected: deriving only in
+the walk (a second dialect and an error case for plain-setup callers), and xstate's default invoke ids as the name
+(invoke-index noise no caller cares about).
+
+The full symmetry: **agents deliver through their per-iid registration (via the Custodian); everything else delivers
+over HTTP (per-gate registration).** Two dialects, one primitive, zero routing. Internally that is literal structure:
+one registration table (`address → { accepted event defs, deliver closure, meta, sender }`) that `agentRun` and `gate`
+register into, with the agent surface and the gates HTTP API as dialect adapters over it — a future caller class (a
+Slack bridge, an email reply) is another adapter, not a third mechanism. The table is implementation structure, not
+vocabulary — workflows speak only `defineEvent` / `agentRun` / `gate`.
+
+**Every delivered event carries its sender.** The table writes `sender: { party: "agent" | "gate", id }` on each event
+it delivers, after the schema parse and from the registration alone: `id` is the Agent's slot, or the Gate's invoke id
+(static in the Machine, the same after a restore — not the caller-facing gate id, which a workflow may compute). It is
+plain data on the event, so it rides the mailbox, a delayed event the run host persists beside the snapshot, and a
+restore; a restored registration writes it again from the persisted input and the invoke id. The Machine's own events
+(`raise`, `sendTo` itself) carry none. `defineEvent` refuses an input that declares `sender`, and both surfaces refuse a
+body that carries it (ADR-0013): no caller and no def can write it. What an author does with it is ADR-0015's.
+
+`CANCEL` stays reserved as the run-level infra interrupt. jr's "exit 3 and let the human drive the bash script" becomes
+"park in a durable state that advertises its human surface."
+
+## Static imports; live objects constructed per-invocation from input
+
+Everything above is reachable by plain `import`. We rejected two alternatives:
+
+- **Injected closures / a `defineWorkflow(kit)` wrapper** — xstate's `provide()` fills only the machine it is called on;
+  it cannot reach actors inside child machines, so host-side injection cannot compose past one level (the finding that
+  retired the templates-with-injected-providers model — see ADR-0015).
+- **Adapter singletons** — a module-global "current adapters" holder cross-contaminates in-process multi-instance use
+  and turns mocking into global state. (The demux registry above is not this: it is rendezvous keyed by globally unique
+  ids, not a swappable adapter.)
+
+The doctrine that makes static import work, sharpening ADR-0007: **actor logic is code; everything live is constructed
+per-invocation from serializable input** — the Harness client from an endpoint, the kube client from ambient config.
+Tests mock by `machine.provide()` at the layer under test.
+
+## Consequences
+
+- Addresses are computed by jr2, never by the workflow — iids always (ADR-0016): fresh per invocation by default (the
+  lossy handoff), or `<run>/<machine actor path>/<agent>` plus its epoch under `continue: true` (ADR-0057); gate ids
+  derived from the gate's actor path by default, authored only to give external callers a meaningful name. Durable
+  handles live in the host ledger (ADR-0016), not in machine contexts.
+- **Test stubbing happens at the wire, not in the actor.** The e2e tier (ADR-0010) hosts a wire-compatible stub Harness
+  beside its orchestrator fixture (admits the agent, holds the stream open, never acts). An endpoint is just a URL, so
+  `agentRun` keeps a single code path and cannot tell it is talking to a fake. Scope: **workspace-less test workflows
+  only**, which pass `endpoint` directly in run input — e2e drives their Machines by playing the Harness's Menu against
+  `/agents/<iid>/*` with no cluster. Workflows that invoke `workspace()` always get real Sandboxes (the data plane is
+  never faked — ADR-0009); their e2e tier requires kind.
+- Unsolicited mid-turn steering remains the residual hard case (ADR-0002/0006).

package/docs/reference/decisions/0012-workspace-wrapper-machine.md

@@ -0,0 +1,103 @@
+# `workspace()` wraps a body Machine and owns only Sandbox lifecycle
+
+jr2 provides Workspaces as a helper so workflows never provision Sandboxes themselves:
+`workspace(body, { input, repos, spec })` is a statically-imported factory (per
+[ADR-0011](0011-workflow-defined-events.md)'s import doctrine) returning a Machine that provisions a Sandbox and its
+worktrees, runs the author's **body** Machine inside it, and tears the Sandbox down when the body finishes. This makes
+GLOSSARY.md's "Workspace = child Machine bound to a unit of work" concrete.
+
+```
+placing:       create Sandbox CR → await the pod scheduled, no deadline (ADR-0064)
+provisioning:  await phase: Ready (watched, ADR-0063) → Harness /attach repos/worktrees (ADR-0004)
+running:       invoke body; input = parent input + { workspace: { repos, branch } }
+teardown:      destroy the CR
+done:          final; workspace output = body output
+```
+
+## Why a wrapper Machine and not actors in the workflow
+
+Teardown forces the shape. In xstate v5 a stop is synchronous — a stopped machine cannot run async work or transition
+through cleanup states — so reliable multi-step teardown must be **internal states the machine reaches by its own
+transitions**. The thing that provisions must therefore also observe the body's completion, which means it wraps the
+body. The persisted `teardown` state is also **durable intent**: a crash mid-teardown restores into it and re-runs the
+idempotent destroy, which host-side lifecycle could only replicate by rebuilding the same ledger the state machine
+already is. When the guarantee still fails (`kill -9`, hard cancel), the operator's heartbeat-lease GC is the backstop
+(ADR-0001) — the reason ADR-0001 made the CR the durable desired state. Author-assembled provisioning/cleanup states
+(the obvious alternative) put the guarantee in every workflow's hands to forget, on every error path.
+
+## Boundary: the spec speaks workspace vocabulary only
+
+The spec — `{ branch, workGroup?, reviewSha? }` — is mapped from the parent's context by the author, but its _shape_ is
+workspace-domain: the branch to cut, and what the attach needs to cut it. The branch itself is the author's: its name
+and its granularity (per feature, per task, from a ticket, from run input) are the Machine's policy, not jr2's; the
+workspace only cuts it. Which Repos to attach is not per-run at all: they are slots on the wrapper's options
+(`repos: { target: open, docs: { url, ref } }`, ADR-0051), static so `jr2 up` can see them, each bound by the Machine,
+by a composer's `customize`, or by a mapper over the door. Workflow configuration (review rounds, budgets, ticket data)
+passes through to the body untouched; the workspace never sees it. In return the body gets what it needs to operate
+inside the Sandbox: `{ repos: Record<slot, path>, branch }`, the handles keyed by the slots the wrapper declared, in
+declaration order. No handle names a primary checkout: which tree an Agent works in is a fact about that Agent's Turn,
+which the body states (ADR-0051). The Harness `endpoint` and the Sandbox name are mechanism-facing, not body-facing —
+`agentRun` resolves them **ambiently** from the enclosing wrapper (a registrar actor co-invoked in `running`, ADR-0016),
+so the body cannot mis-thread them. Keeping this boundary is what lets one helper serve any workflow rather than
+coupling to coding-shaped ones.
+
+The wrapper's `input` beside it is the door the spec maps FROM — the wrapper's own declared run-input schema, which also
+types the mapping ([ADR-0033](0033-a-machine-declares-the-input-that-starts-it.md)). It sits on the wrapper and not on
+the body precisely because of the injection above: the body's input is the door plus `workspace`, which no caller sends.
+That same composition is what the door checks the body against, one way only — the body may not demand more than the
+wrapper hands it; demanding less is safe — so the injection is a compile-time contract, not just a comment. The host's
+own injection into a root machine's input (`instanceId`) counts as handed over for that check, because whether it
+actually reaches this body depends on where the wrapper sits and no type can see that
+([ADR-0033](0033-a-machine-declares-the-input-that-starts-it.md)).
+
+## An Effect runs in its Workspace: the Harness container or the Sandbox Effector
+
+An Effect in the body (ADR-0067) resolves the same ambient handles an Agent does. A `command` that names no secret runs
+in the Harness container of the enclosing Workspace's Sandbox, as the Agent's user. A `push`, an `http` whose body is a
+Worktree file and a `command` that names a secret run in that Workspace's Sandbox Effector, the pod's `effector`
+container ([ADR-0005](0005-sandbox-pod-composition.md),
+[ADR-0068](0068-an-effector-holds-an-effects-secrets-and-signs-its-wire.md)), which the provision adds only when an
+Effect below the `workspace()` is placed there; its image is the Sandbox Image unless the `workspace()` names
+`effector: { image }`. Each one's Worktree follows the Turn's rule (ADR-0057): the one Repo Slot's, or the slot it names
+with `slot`; a Workspace with several and none named refuses it. A push goes to that slot's Binding, and after every
+push whose published commit is not the Agent's own — signed copies, or one squashed commit, signed or not — the
+Workspace's Harness moves the Agent's branch to the published commits (the sync, ADR-0067). An Effect of these kinds
+outside any `workspace()` is refused at invoke, as an error, never as output. A `command` that names a secret runs
+against a pinned commit in a scratch of the Effector's own, and its child is confined with Landlock from the Effector's
+key, the secrets it does not name, the push credentials and the Worktree (ADR-0067, ADR-0068).
+
+## There is no retain policy: parking is retention
+
+Teardown fires **only when the body reaches a final state or fails**. A Failure from the body
+([ADR-0073](0073-a-machine-fails-a-run-with-fail-and-an-uncaught-failure-ends-it-failed.md)) goes to a `failing` state,
+which runs the same `destroy` and then sends the Failure up unchanged; a defect goes up with no `onError`, and its
+Sandbox stays as evidence. A body that needs its Sandbox kept alive — escalation, a human inspecting the worktree, a
+human-review gate — simply parks in a non-final state (typically holding a `gate`, ADR-0011). We considered and rejected
+an `onSettled → retain | destroy` policy knob: parking expresses retention with no new API, and keeps the run visibly
+"waiting" in `jr2 runs`. A parked run's Sandbox stays alive as long as the run does — each live workspace renews its own
+lease, and the operator reaps only Sandboxes whose lease has lapsed (abandonment, not parking — ADR-0001).
+
+## Commits leave via the workflow, never the workspace
+
+Getting commits out of the pod-local clone (push, PR) is **body flow**, decided by the workflow — a coding workflow
+pushes the branch / opens a PR before its human gate, so review is PR-based and the forge owns merging. The workspace
+attaches and destroys; it never touches remotes. A body that finishes without publishing loses its commits by design —
+the same contract as a temp directory.
+
+## Consequences
+
+- ADR-0004's single-active-Agent-per-Workspace invariant becomes structural: one sequential body per workspace, no lock
+  machinery.
+- The workspace labels the CR with its run (`jr2.dev/run`, `jr2.dev/workflow`), which is what `jr2 ls` groups by
+  (ADR-0009).
+- **Lost workspace: the wrapper emits, the body decides.** While the body runs, a lease actor beside it renews the
+  Sandbox's keepalive and follows the Sandbox watch for whether the workspace it attached to is still there (ADR-0021,
+  ADR-0063) — continuously, and on snapshot restore as the first tick of that same loop rather than a separate
+  restore-time path. Intact → carry on (same endpoint — deterministic per Sandbox name; in-flight turns re-attach from
+  the host ledger, ADR-0016). Lost — reaped, deleted, or a replacement pod after an eviction or node loss — → the
+  pod-local clone and any unpushed commits are gone, so silently re-provisioning would resume into an inconsistent world
+  (tickets closed, commits vanished); instead the wrapper delivers a **`workspace.lost`** event into the body — same
+  channel as `agent.fault` — and the body's policy decides (route it to escalation, fail the run, or choose to
+  re-provision and restart). The body must route it: `workspace()` refuses a body with a non-final state that has no
+  route on itself or an ancestor, and an event that no transition takes faults the run
+  ([ADR-0074](0074-an-unrouted-mechanism-event-is-refused-at-load-and-faults-the-run.md)).

package/docs/reference/decisions/0013-the-agent-reaches-its-machine-through-a-container-it-cannot-read.md

@@ -0,0 +1,157 @@
+# The Agent reaches its Machine through a container it cannot read
+
+An earlier cut put the Agent's tool surface on the Orchestrator (`/mcp/:instanceId`), which meant an Agent must dial
+**into** the control plane to drive its Machine. Validating the workspace slice on kind (2026-07-12) showed that leg did
+not exist and could not be safely built as specified: nothing passed a callback URL into a Sandbox, `jr2 dev` bound
+loopback, and — the real problem — an Agent that can reach the delivery API can deliver to **any** registration in its
+run, including its own human-review Gate. The fix is not to hide the API from the Agent. Working tools give the Agent
+code execution in the Harness container, which shares the pod's network namespace, so it can reach any address the pod
+can. The fix is where the credential lives: in a container of the same pod that the Agent executes nothing in — the
+**Custodian** ([ADR-0059](0059-a-harness-holds-stand-ins-and-the-custodian-holds-the-keys.md)). The Orchestrator speaks
+no MCP, and keeps one HTTP surface over the registration table it already has
+([ADR-0011](0011-workflow-defined-events.md)).
+
+## The enabling fact: the Harness reads its Menu per Submission
+
+Each Submission, the Harness reads `GET /agents/<iid>/surface`: the Agent's Menu, which it presents to its model as
+tools named `mcp__jr2__<event>` (shipped instructions and the printer's prefix-stripping depend on that name;
+[ADR-0027](0027-the-harness-is-jr2s-own-server-flue-retires-the-wire-stays.md) states it as explicit jr2 code, never
+written by users). The Harness names the iid itself, so nothing has to _learn_ which turn is live — no push channel, no
+long-poll, no `sandbox → active turn` index, no second inbound port on the pod. The same read carries the Turn's Allowed
+picks, which the Harness states in the Briefing; the Menu itself is fixed for a conversation
+([ADR-0029](0029-a-menu-is-fixed-for-a-conversation-and-a-turn-is-told-its-allowed-picks.md)), so no `list_changed` push
+is needed either (ADR-0006).
+
+## Decision
+
+- **The Custodian carries the Agent's Menu, and holds the credential that makes it work.** It is a jr2-composed
+  container in every Harness pod ([ADR-0005](0005-sandbox-pod-composition.md)). The Sandbox token is mounted into it as
+  a file, and into no other container. The Harness container holds the token's Stand-in,
+  `JR2_SANDBOX_TOKEN=jr2-held-JR2_SANDBOX_TOKEN`, and reaches the Orchestrator only through the Custodian's control
+  listener on `127.0.0.1:8081`. The Custodian forwards four routes, with the token put in place of the Stand-in, and
+  answers every other route itself:
+
+  ```
+  GET  /agents/<iid>/surface          the Menu                            → the Orchestrator
+  POST /agents/<iid>/events           a pick                              → the Orchestrator
+  GET  /agents/<iid>/deliveries/<id>  an outstanding pick's answer        → the Orchestrator
+  POST /fetch                         the ask (ADR-0053), for THIS pod's Sandbox → POST /sandboxes/<name>/fetch
+  anything else                       404
+  ```
+
+  So the Agent's sole control-plane peer is a container it can reach but whose credential it cannot read. That is what
+  makes ADR-0006's "the Agent never steers the workflow" **enforced** rather than advertised.
+
+- **The Harness presents the Menu to its model itself.** There is no MCP server and no MCP client. `menu.ts` turns each
+  accepted event into a Menu item, with the event's JSON Schema as its parameters, and turns a pick into one
+  `POST …/events` — and, for an outstanding pick, the polls of its delivery, inside the same tool call. The engine runs
+  one model answer's calls in the model's order
+  ([ADR-0027](0027-the-harness-is-jr2s-own-server-flue-retires-the-wire-stays.md)), and only the first pick of a model
+  answer runs. It imports nothing from pi: `menu-tools.ts` gives each item to pi as a tool. A harness that only speaks
+  MCP would get a second presenter over the same `menu.ts`, not a second Menu. The receipt comes back as prose
+  ([ADR-0024](0024-an-agents-turn-ends-with-the-state-that-asked-for-it.md)). A 404 on the surface is a Turn that is
+  over, and it is never prompted ([ADR-0026](0026-a-turn-that-is-over-is-never-prompted.md)).
+- **The Orchestrator hosts no MCP, and judges every pick.** The registration table stays the one internal primitive; it
+  keeps two thin HTTP surfaces, neither of them MCP:
+
+  ```
+  # human / webhook / CI — the Gate resource of ADR-0011, authenticated
+  GET  /runs/:id                       → open gates: accepts + schemas + acceptsNow + meta  [Instance token]
+  POST /runs/:id/gates/:gate/events    → validate + deliver, or 409                   [Instance token]
+
+  # the Agent's, through its pod's Custodian
+  GET  /agents/:iid/surface            → Menu + Allowed picks + schemas + semantics [Sandbox token]
+  POST /agents/:iid/events             → validate + deliver → the receipt, or 409  [Sandbox token]
+  GET  /agents/:iid/deliveries/:id     → an outstanding pick's answer, long-polled [Sandbox token]
+  ```
+
+  The receipt is `{ delivered, event, moved, turnComplete, allowed, deliveryId }`. A pick the workflow is still running
+  when the POST answers — it moved the workflow to work inside the Agent's state
+  ([ADR-0024](0024-an-agents-turn-ends-with-the-state-that-asked-for-it.md)) — is `outstanding`, and `deliveryId`
+  addresses its answer: `GET /agents/:iid/deliveries/:id` holds up to about 25 s and answers either `outstanding` (ask
+  again) or the receipt with `answered`, the Allowed picks read after the route, and the answer. A child that replied
+  inside the pick's own send and left the workflow at Rest makes the POST receipt itself the `answered` one. A stopping
+  or parked Orchestrator answers the poll 503, never 404, which says the Turn is over; the delivery is scoped to the
+  Sandbox its registration had when the pick was made, because the answer outlives a registration that a route out of
+  the Agent's state ends. A pick while one is outstanding, or while the Agent's state is busy, is answered
+  `409 { refused: "outstanding", outstanding?, allowed: [] }` and nothing is delivered. Lookup, validation, delivery and
+  lifecycle are implemented once, in the table. The token's scope, the live registration, and the event's name and
+  payload are judged here and nowhere else; the Custodian forwards and decides nothing about a pick. Neither surface
+  takes a body that carries `sender`: the table writes it on every delivered event (ADR-0011), so a client that posts it
+  — the Harness, `jr2 send --input` — is answered 400. The gates surface answers `{ ok: true }` when the run took the
+  send, and only then; a send the run would not take now — its selection holds another party's writing, or no writing
+  would move on this payload (ADR-0015, ADR-0029) — is answered `409 { error, acceptsNow }` and nothing is delivered.
+  `jr2 send --gate` prints that error, which names what the Gate takes now, and exits 1.
+
+- **Bearer tokens, because the boundary is otherwise theater.** The Harness container shares the pod's network
+  namespace, so an Agent can `curl` the Orchestrator directly; a per-pod NetworkPolicy cannot distinguish it from the
+  Custodian. Only authentication closes this.
+  - **Sandbox token** — minted per Sandbox at provision (a signed Sandbox name, so re-provision after a restart yields
+    the same token), delivered as a Secret mounted into the **Custodian container only**. User `harness.env`/`envFrom`
+    land on the Harness container, never the Custodian. Authorizes exactly: deliver to agent registrations **whose
+    Sandbox is this one**, and ask for this Sandbox's fetch. Never a Gate (they are not on that surface at all), never
+    another Sandbox. The Stand-in the Harness holds is refused on every route.
+  - **Instance token** — the human/CLI credential for Gates and run control. Minted into an in-cluster Secret at
+    `jr2 up`; the CLI reads it over the kube API (RBAC is the gate) while port-forwarding the Service (ADR-0019).
+  - **The Instance-token door.** The Instance token may also read any Agent's surface and deliver to it —
+    `GET /agents/:iid/surface`, `POST /agents/:iid/events` — because it is the operator. That is the operator speaking
+    AS the Agent: it reads the Agent's Menu and Allowed picks and its picks are judged as the Agent's — they carry the
+    Agent's sender (ADR-0011) — and no audience filter applies, so it can send a pick no Gate's caller may send
+    ([ADR-0015](0015-authoring-surface-absorbs-the-mechanism.md)). It is deliberate — it is how the e2e tier stands in
+    for an Agent — and it adds no party: neither `jr2 send` nor the Console offers it, and a Sandbox token never opens
+    it for a registration that is not its own.
+  - An agent registration records its Sandbox **ambiently**: `agentRun` resolves the enclosing `workspace()` and records
+    the same deterministic name the token was minted for (ADR-0016), so forgetting to pass it is unrepresentable (the
+    pre-ambient version shipped exactly that footgun: `sandbox` omitted, every tool call 403'd — fail-closed but
+    silent). A run-scoped token instead of a Sandbox-scoped one would be genuinely exploitable: derivable iids plus
+    readable ticket ids would let one feature's coder inject a `review_verdict` into another feature's reviewer.
+- **Reachability, narrowed to one caller.** The Orchestrator is always in-cluster (ADR-0019), so the route home is
+  simply **Service DNS**, which the pod's maker writes into the Custodian's configuration — the Orchestrator at a
+  Sandbox's provision, `jr2 up` for the Instance Harness. The Harness is told the Custodian's loopback address
+  (`JR2_CUSTODIAN_URL`) and nothing else.
+- **A Machine-answered tool result is the held answer, for every pick that runs work.** No def declares it: a pick whose
+  target runs something inside the Agent's state is outstanding until the workflow is back at Rest, and the Harness
+  polls its delivery inside the one tool call (ADR-0002, ADR-0024). The gates surface answers `{ ok: true }` for a send
+  the run took and 409 for one it would not — external callers need no receipt. `deferred` is retired (ADR-0006). `poll`
+  is still in the model and the surface listing still ships each def's `semantics`; registering a `poll` def **fails
+  loudly as unimplemented** — a silent downgrade to `ack` would be a lying tool contract — and the Harness refuses a
+  turn whose surface offers one.
+
+## Considered options
+
+- **Agent talks to the Orchestrator directly** (the original cut, plus a callback URL). Rejected: the credential lands
+  in a container where the Agent has code execution, so the Agent can deliver `approve` to its own human-review Gate —
+  it merges its own PR. The tool menu is advisory once you hold the key to the API.
+- **A jr2-owned Adapter container that serves the Menu over MCP on loopback and holds the token.** It meets the claim,
+  but a whole container, image and package exist to translate two HTTP calls into MCP and back. Model keys need a
+  container the Agent executes nothing in anyway (ADR-0059); one container can hold every credential, and the
+  translation then has no reason to live outside the Harness.
+- **Translation inside the Harness, with the credential there too.** Rejected — same container, same code execution,
+  same credential. The translation is inside the Harness now; the credential is not.
+- **The Orchestrator serves MCP, and the Custodian forwards it.** Rejected: it keeps the MCP transport and its session
+  handling in the control plane and on both ends, to carry two plain HTTP calls.
+- **A uniform `/registrations/:address` API** for every caller. Rejected: it dissolves the run-scoped, discoverable Gate
+  resource ADR-0011 designed on purpose (a webhook translator finds its Gate by `meta.prUrl`; `GET /runs/:id` lists open
+  decisions) in exchange for aesthetic symmetry.
+
+## Consequences
+
+- **The operator needs no change.** ADR-0001 made sidecars generic container fragments the operator schedules without
+  understanding; the Custodian is exactly that. `kubeSandbox` composes it (plus the token Secret, written right after
+  the CR with the CR as its owner from birth, so Kubernetes reaps it with the Sandbox and no path leaks it — ADR-0001).
+- **Nothing in jr2 speaks MCP.** The Orchestrator, the Harness and the CLI carry no MCP dependency.
+- **One container holds the Sandbox token and every model key.** A fault in the Custodian's engine is a fault in the one
+  container that holds both. ADR-0059 records the trade.
+- **The `@kind` e2e tier owns the pod → Orchestrator leg** — the pod's Harness actually calls its tool through the
+  Custodian, so a broken leg is a red test, and a Harness container that posts a pick straight to the Orchestrator with
+  the Stand-in it holds is refused. The mechanics tier plays `/agents/:iid/*` from the host, which is honestly
+  simulating the Harness's Menu, not an Agent.
+- **NetworkPolicy is a second layer, not the control.** It bounds where a pod may talk; only the token bounds what it
+  may _do_. `jr2 up` converges ingress policies that admit only the Orchestrator to a Harness pod
+  ([ADR-0058](0058-a-harness-answers-the-orchestrator-alone.md)); the Orchestrator itself is left reachable, because a
+  Custodian in every Harness pod must reach it and no policy can tell the Custodian's packets from the Agent's.
+- **The Harness wire is authenticated in the other direction.** The Orchestrator bears a token derived for the placement
+  on every Harness call, and the Harness holds only its digest (ADR-0058). Same signed-name idea, reversed caller.
+- **Open: authn for non-kube callers.** The Instance token rides kube RBAC for anyone with cluster creds; what an
+  ingress-exposed jr2 Application uses for human/webhook callers without them (`--url` mode) is out of scope here
+  (ADR-0014 inherits the same question).