@osolmaz/pi-workflows 0.13.4 → 0.15.0

package/docs/WORKFLOW_HOST.md

@@ -0,0 +1,452 @@
+# Workflow host
+
+Status: implemented. [Run workflows outside Pi](2026-08-30-out-of-process-workflow-host-plan.md) records the approved implementation plan.
+
+## Purpose
+
+The workflow host keeps durable workflow state correct when Pi, a workflow, or the host stops unexpectedly. It owns the workflow database and supervises a separate process for each active run. Pi remains the user interface and performs interactive model turns through its documented extension APIs.
+
+The host solves two different failures:
+
+- A busy workflow cannot block the process that renews run claims.
+- A crashed or stale runner cannot leave contradictory state or continue writing after another runner takes over.
+
+## Terms
+
+- **Host:** The single user-level process that owns workflow state, claims, commands, and worker supervision.
+- **Client:** A Pi extension instance or command-line process connected to the host.
+- **Worker:** A child process that loads one workflow and executes one active run generation.
+- **Origin session:** The Pi session that started an interactive run.
+- **Claim:** A time-limited right to change one run.
+- **Generation:** A number increased each time a new owner claims a run. It fences older owners.
+- **Durable boundary:** A committed node or lifecycle transition from which execution can resume.
+- **Interactive request:** A durable agent or assistant-message step that must run in the origin Pi session.
+- **Managed effect:** A side effect reserved and settled through an idempotent durable record.
+
+## Boundaries
+
+The host belongs to the `@osolmaz/pi-workflows` package. It uses the existing SQLite database at `~/.pi/agent/workflows/state.sqlite` and documented Pi extension APIs.
+
+The design does not change Pi source, Pi session files, Pi message schemas, or private Pi APIs. It does not add a remote service or a second database. The package does not install an operating-system service.
+
+SQLite remains local to one machine. The protocol does not provide distributed consensus or network-filesystem safety.
+
+## Process model
+
+One host owns the global workflow database for one user installation.
+
+```text
+Pi extension ─┐
+Pi extension ─┼── local socket ── workflow host ── SQLite
+CLI client ───┘                         │
+                                        ├── run worker A
+                                        ├── run worker B ── headless pi --mode rpc
+                                        ├── controller worker
+                                        └── source resolver
+```
+
+The host may manage runs from more than one project. Each run keeps its canonical project path and source identity.
+
+The host process performs only bounded protocol handling, short SQLite transactions, timers, queue scheduling, and process supervision. It does not import or execute workflow definitions.
+
+A worker loads one workflow source and executes one run generation. It cannot receive a writable `WorkflowRunStore`. It proposes changes to the host over a private child channel. This is an architectural guard against accidental writes. It is not a security sandbox against code running as the same operating-system user.
+
+## Host lifecycle
+
+The package CLI owns host lifecycle commands:
+
+```text
+pi-workflows host start
+pi-workflows host status
+pi-workflows host stop
+pi-workflows host run
+```
+
+`run` stays attached for direct operation and tests. `start` starts the package process on demand and waits for a ready handshake. It does not install systemd, launchd, or another persistent service.
+
+The host uses one global lock and one host epoch. Socket creation and the SQLite host claim must agree before the host accepts commands. A second live host refuses to start. After the old host lease expires, a new host increases the epoch before it handles work. Messages from an older epoch are rejected.
+
+The host stays alive while it has a connected client, an active worker, a scheduled wake, a pending controller, or unsettled work. An idle host may exit after a documented idle period. A later client can start it again.
+
+## Claim rules
+
+A run claim contains:
+
+- owner type and owner ID;
+- token hash;
+- generation;
+- acquisition time;
+- heartbeat time;
+- expiry time.
+
+A protected state write uses one SQLite transaction:
+
+1. Read the expected resource revision and lease.
+2. Compare owner type, owner ID, token hash, and generation.
+3. Require a future expiry time.
+4. Renew heartbeat and expiry for that exact claim.
+5. Apply the domain change.
+6. Add the immutable event and viewer delta.
+7. Increase the resource revision.
+8. Commit.
+
+The transaction fails without changes when any check fails.
+
+An expired claim cannot renew itself, even when the owner ID and token hash still match. Recovery first takes a new claim and increases the generation.
+
+The host also renews active claims from a timer. The timer is a backup for a run with no state writes. Normal write correctness does not depend on the timer.
+
+Claim rejection uses `ClaimLostError` with one internal reason:
+
+- `missingAuthority`
+- `expired`
+- `ownerChanged`
+- `tokenChanged`
+- `generationChanged`
+
+Logs may show the run ID, generation, and reason. They must not show a raw token or token hash.
+
+## Run lifecycle
+
+The run and queue projections follow these states:
+
+| Run state   | Queue state | Claim | Worker   | Meaning                                                 |
+| ----------- | ----------- | ----- | -------- | ------------------------------------------------------- |
+| `queued`    | `queued`    | none  | none     | Ready for host scheduling.                              |
+| `running`   | `starting`  | host  | starting | A worker launch is being recorded.                      |
+| `running`   | `running`   | host  | live     | A worker is executing one node.                         |
+| `running`   | `parked`    | none  | none     | Execution stopped at a durable boundary and can resume. |
+| `waiting`   | `parked`    | none  | none     | A checkpoint or interactive request needs input.        |
+| `completed` | `done`      | none  | none     | The run finished successfully.                          |
+| `failed`    | `failed`    | none  | none     | The run failed with a durable error.                    |
+| `timed_out` | `failed`    | none  | none     | The run exceeded a declared timeout.                    |
+| `cancelled` | `cancelled` | none  | none     | Cancellation completed.                                 |
+
+A lifecycle transaction updates the run, queue, attempt, decision, lease, event, and viewer facts that belong to one transition. The database must not commit a failed event while the run remains running, or a terminal queue row while the run remains nonterminal.
+
+Waiting and paused work does not keep a worker or a live claim. Resume takes a new claim generation and starts a new worker from the last durable boundary.
+
+## Worker lifecycle
+
+A worker launch envelope contains:
+
+```json
+{
+  "schema": "pi-workflows.worker-launch.v1",
+  "runId": "run-id",
+  "generation": 2,
+  "workerEpoch": "opaque-id",
+  "projectPath": "/canonical/project/path",
+  "workflowSource": {
+    "root": {
+      "kind": "file",
+      "path": "/canonical/project/path/.pi/workflows/example.workflow.ts",
+      "hash": "sha256-digest"
+    },
+    "mounted": []
+  },
+  "definitionDigest": "sha256:digest",
+  "inputHash": "sha256:digest",
+  "protocolVersion": 1
+}
+```
+
+Before it loads workflow modules, the worker verifies the root identity and every saved mounted file hash or built-in revision. After loading, it also checks the complete mounted-source map against the saved map. A mismatch parks the run with `workflowSourceChanged`. The normal scheduler does not claim that run again. The operator can restore the recorded source and explicitly resume the run, or cancel it. Changed included code does not execute.
+
+The host records a worker epoch before spawn. The child must return a ready message before the startup deadline. Every later child message includes the run ID, generation, and worker epoch.
+
+The host records one terminal worker outcome:
+
+- `exited`
+- `cancelled`
+- `timedOut`
+- `crashed`
+- `claimLost`
+- `orphaned`
+
+A worker exit is not automatically a run failure. The host decides from the last committed attempt and effect state whether it can resume, must park, or must fail.
+
+## Process supervision
+
+Each worker starts in its own process group. A headless Pi child starts in another process group so normal worker completion can stop all Pi tool descendants without signaling the worker itself. The worker registers that direct child with the host before it sends a prompt and unregisters it only after group shutdown. The host owns the one process registry and reaps a registered child if its worker exits first.
+
+The host enforces:
+
+- a startup handshake deadline;
+- node deadlines already declared by the workflow engine;
+- bounded protocol messages;
+- bounded captured stdout and stderr;
+- cancellation with `SIGTERM` and bounded `SIGKILL` escalation;
+- process-group cleanup;
+- orphan checks after host restart;
+- portable memory or process limits where Node and the operating system support them.
+
+The child protocol must apply backpressure. A child that exceeds message or output limits fails its worker epoch with a clear infrastructure reason. The complete durable workflow result stays in SQLite within the existing value limits.
+
+The process registry includes a process start identity, not only a PID. The host accepts a worker registration only when the PID is a direct child of that active worker. A reused PID cannot let a new host kill an unrelated process.
+
+## Local client protocol
+
+Clients connect through a user-only local socket. Unix socket mode is `0600`. Other platforms use their equivalent local transport and access control.
+
+Messages use newline-delimited canonical JSON. One message is at most 1 MiB, matching the existing durable event limit. The receiver closes only the offending connection when framing or validation fails.
+
+Every request uses this envelope:
+
+```json
+{
+  "schema": "pi-workflows.host-request.v1",
+  "requestId": "opaque-id",
+  "clientId": "opaque-id",
+  "operation": "run.cancel",
+  "runId": "run-id",
+  "expectedRevision": 12,
+  "idempotencyKey": "stable-key",
+  "payload": {}
+}
+```
+
+A response uses:
+
+```json
+{
+  "schema": "pi-workflows.host-response.v1",
+  "requestId": "opaque-id",
+  "outcome": "accepted",
+  "revision": 13,
+  "receipt": {}
+}
+```
+
+Valid outcomes are:
+
+- `accepted`
+- `adopted`
+- `rejected`
+- `conflict`
+- `notFound`
+- `claimLost`
+- `unavailable`
+
+The host commits a command receipt before it acknowledges success. Repeating the same request ID and payload returns the stored receipt. Reusing an ID with another payload returns a conflict.
+
+The first command set is:
+
+- `run.start`
+- `run.pause`
+- `run.resume`
+- `run.cancel`
+- `run.status`
+- `run.list`
+- `checkpoint.answer`
+- `decision.answer`
+- `interaction.submit`
+- `interaction.update`
+- `notification.claim`
+- `notification.deliver`
+- `turn.claim`
+- `turn.resolve`
+- `controller.list`
+- `controller.get`
+- `controller.apply`
+- `controller.reconcile`
+- `controller.delete`
+- `host.status`
+- `host.stop`
+
+Read operations may use the existing read-only store directly in viewers. Mutating Pi and CLI paths use the host.
+
+## Worker protocol
+
+The private worker channel accepts these message kinds:
+
+- `worker.ready`
+- `node.started`
+- `node.update`
+- `node.finished`
+- `node.failed`
+- `run.parked`
+- `run.finished`
+- `interaction.requested`
+- `interaction.accepted`
+- `interaction.rejected`
+- `notification.requested`
+- `presentation.requested`
+- `effect.reserve`
+- `effect.settle`
+- `worker.progress`
+- `worker.exiting`
+
+Every worker message includes the worker launch schema, run ID, generation, worker epoch, attempt ID when applicable, expected revision, and a stable message ID. Headless workers use `process.register` and `process.unregister` operations under `worker.progress` to attach their Pi child group to host supervision. Registration requires the live run claim. Unregistration remains valid after a terminal state releases that claim so cleanup can finish.
+
+The host checks the generation and epoch before it reads the payload. A stale worker gets one claim-loss response and must exit. The host stores receipts for accepted state-changing messages so a retry receives the same answer.
+
+## Durable protocol records
+
+Reuse current rows when they already own a fact:
+
+- `runs`, `run_queue`, `leases`, and `events` own run lifecycle and claims.
+- `node_attempts` owns node execution state and resolved wall-clock deadlines.
+- `human_decisions` and resolution tables own checkpoints.
+- `effects` and `effect_attempts` own side effects and ambiguous outcomes.
+- `notifications` and `turn_intents` own passive and terminal Pi messages.
+- `run_bindings` owns origin session and execution mode.
+
+Add only these records if implementation proves the current rows cannot hold the contract:
+
+### Host commands
+
+`host_commands` stores request ID, client ID, operation, idempotency key, request fingerprint, run ID, accepted revision, outcome, receipt or error hash, and timestamps. The unique request fingerprint prevents one request ID from naming two commands.
+
+### Interactive requests
+
+`interactive_requests` stores request ID, run ID, attempt ID, target session ID, kind, contract hash, pending or settled status, accepted submission ID, and timestamps. One attempt has at most one request. The linked node attempt stores its resolved wall-clock deadline.
+
+`interactive_submissions` stores request ID, submission ID, idempotency key, payload hash, validating, accepted, or rejected outcome, receipt hash, and submission time. Repeated keys return the same receipt.
+
+### Worker epochs
+
+`run_workers` stores run ID, generation, worker epoch, launch envelope hash, process identity, status, start time, ready time, finish time, exit code, signal, and bounded diagnostic hash. One run and generation can have several sequential worker epochs, but only one may be active.
+
+These tables remain part of `pi-workflows-state` schema version 1. The DDL digest changes in place under the alpha policy.
+
+## Interactive Pi execution
+
+Agent and assistant-message steps for an interactive run execute in the origin Pi session.
+
+The worker commits the node's resolved wall-clock deadline before it proposes `interaction.requested`. The host commits the request, changes the node attempt to waiting, parks the queue row, releases the claim, and acknowledges the worker. The worker then exits. The host continues to enforce the durable deadline while no worker exists. If the deadline passes, one control claim atomically closes the stale request and schedules a supervised timeout-resume child. The child preserves the same attempt and deadline, records `timed_out`, and follows any `$result.outcome` edge. A run with no timeout recovery edge becomes terminal and releases its session reservation. Restart recovery starts this timeout path before it schedules other work.
+
+The extension finds pending requests during `session_start`, after `agent_settled`, and once per second while the session is open. It claims one request presentation, sends the step message through documented Pi APIs, and exposes the normal `workflow` tool contract.
+
+A tool update or submission goes to the host. It includes the exact request, node, attempt, expected revision, and tool-call idempotency key. The host first checks this transport contract and records a provisional `validating` submission. It then schedules a supervised workflow child. Only that child loads workflow code and runs the node's `validate` function. The child reports `interaction.accepted` or `interaction.rejected` to the host. The host settles the request only after acceptance. A rejected payload leaves the same request pending and returns the stored actionable error to the model. If the child stops before it reports a result, the host rejects the provisional submission and leaves the request ready for a corrected retry.
+
+An ordinary checkpoint accepts the model-facing `answer` action and starts a continuation run. A protected human decision never accepts that tool action. The extension displays the decision without starting a model turn, and a person answers it with `/workflow answer` through `decision.answer`. When a protected decision reaches its saved `onTimeout` deadline, the host takes a control claim on the waiting parent, atomically records the validated default, closes the pending interaction, releases the parent claim, and reserves the continuation. A human answer cannot win after that deadline.
+
+The session keeps normal Pi entries for prompts, tools, and replies. Pi Workflows stores the public session entry ID used for presentation adoption. It does not edit the Pi session file or schema.
+
+One session presents one workflow interaction at a time. Other requests remain ordered by creation time. A restart or reload can present an unresolved request again, but exact session-entry adoption prevents a second visible message when the first presentation was already recorded.
+
+Notify nodes enqueue passive messages in the existing `notifications` outbox. The extension claims a message through the host, adopts an existing session entry after a crash, and marks delivery through the host. A completed run with a root `presentationPrompt` creates an ineligible `turn_intent` before the terminal commit. The same terminal transaction makes that intent eligible. The extension claims it, starts one normal Pi turn, and records the public session entry ID. No completion turn starts before the completed state is durable.
+
+## Detached execution
+
+A run with headless execution mode uses the existing `pi --mode rpc` integration for agent steps. The Pi child uses a separate process group registered with the host. The worker stops that group during normal completion. Cancellation gives the worker a bounded cleanup interval, and the host reaps the registered group if the worker exits first.
+
+The headless child receives only the workflow step prompt, configured model arguments, and the bridge extension. Its submission uses the same step and attempt contract as origin-session work. A headless run cannot use a visible assistant-message step because it has no origin Pi session.
+
+The run binding records `interactive` or `headless` execution mode. Viewers show that mode without exposing provider credentials.
+
+## Pause and cancellation
+
+Pause atomically commits `paused = 1`, parks the queue, releases the exact claim, and stores the command receipt. The fenced worker process group then stops. An uncommitted pure node can run again after resume. Resume takes a new generation and starts another worker from the last durable boundary.
+
+Cancellation against a live worker atomically commits terminal cancellation, cancels pending attempt and interaction state, settles effect recovery state, releases the exact claim, and stores the command receipt. A pending effect becomes cancelled. An applying effect becomes ambiguous because the host cannot prove its external outcome. The host then stops the fenced worker process group. A host crash after the receipt cannot resume the cancelled run or retry the ambiguous effect. If the child does not stop by the deadline, the host kills its process group.
+
+Cancellation against an expired running row first takes a new control claim. The claim operation must prove that the old lease is absent or expired. The new owner then cancels the active attempt, effect recovery state, and pending interaction or human decision in one lifecycle transaction.
+
+A client cannot force-cancel a live claim through the stale recovery path.
+
+## Recovery
+
+At startup the host:
+
+1. Takes the global host epoch.
+2. Reaps worker records that match an exact stale process identity.
+3. Finds expired running runs and active attempts.
+4. Reads managed effect state before deciding whether work can repeat.
+5. Parks uncertain effects for manual review.
+6. Makes pure and fully settled work claimable.
+7. Restores pending interactive requests and scheduled controller work.
+8. Starts supervised timeout recovery for pending interactive requests whose durable node deadlines expired.
+9. Resumes any remaining provisional `validating` submission in a new supervised child.
+10. Starts no model turn until a matching Pi session connects or headless mode is declared.
+
+Recovery resumes from the last committed boundary. An uncommitted compute node may run again because compute is pure. An action with a stored effect receipt adopts that receipt. An effect in `ambiguous` state requires explicit recovery.
+
+Claim loss is a handoff, not a run failure. The old owner writes no terminal event after claim loss.
+
+## Effects and retry safety
+
+Compute nodes must not perform external side effects. They can repeat after a worker crash.
+
+Side-effecting action and shell behavior must have one of these contracts:
+
+- a managed effect with an external idempotency key;
+- a managed effect with a read-back check that proves whether it applied;
+- an explicit non-resumable result that becomes `ambiguous` after an uncertain crash.
+
+The host reserves an effect before execution. The effect key includes the source resource, effect type, and author-provided idempotency key. The request fingerprint prevents key reuse with another payload.
+
+An applied, rejected, or cancelled effect is terminal. An ambiguous effect is also terminal for automatic retry. An operator may use a separate reviewed recovery action after inspecting the external system.
+
+## Controllers
+
+The global host also reconciles controllers. Controllers keep their existing resource claims, queue, effects, and child workflow request keys.
+
+A controller child run enters the same global run queue and worker process model. It does not need an origin Pi session unless its workflow declares an interactive step. A headless child uses the declared provider path. A child that needs an origin session parks with a clear unsupported-input result unless the controller supplied an approved session binding.
+
+Controller reconcile code runs in a supervised controller worker, not in the host event loop. Controller initialization also runs in a source resolver child. Before `controller.apply` commits, the host checks that the resolved source still matches controller discovery rules and the exact source digest.
+
+## Failure classification
+
+Use separate states and messages for these failures:
+
+- `claimLost`: another generation owns the run, or the claim expired.
+- `workerCrashed`: the child exited without a terminal protocol message.
+- `workerTimedOut`: the child exceeded a declared deadline.
+- `hostUnavailable`: the client cannot reach or start the host.
+- `sourceChanged`: the workflow source does not match the saved identity.
+- `effectAmbiguous`: an external action may have applied without a receipt.
+- `nodeFailed`: workflow code returned a normal failure.
+- `protocolRejected`: a message failed schema, revision, attempt, or idempotency checks.
+
+A failure in one class must not be reported as another. In particular, claim loss does not create a failed run event.
+
+## Status and privacy
+
+`pi-workflows host status` reports:
+
+- host state and epoch;
+- socket availability;
+- active worker count;
+- queued, running, parked, and waiting counts;
+- expired claim count;
+- pending interaction count;
+- ambiguous effect count.
+
+It does not print actor IDs, session IDs, project paths, prompts, outputs, payloads, claim tokens, environment variables, or credentials.
+
+Logs use bounded safe errors. Child stdout and stderr may contain private content and stay in the user-only workflow state directory. Public issue and pull-request text must use generic fixtures and no operator-specific identifiers.
+
+## Alpha state policy
+
+This feature changes the current schema in place while the project is in alpha.
+
+Keep `pi-workflows-state` and schema version 1. Change the DDL digest and current contracts directly. Add no compatibility reader, migration shim, dual read, dual write, alias, feature flag, or parallel state root.
+
+When the installed state has the old digest, fail before mutation with the standard backup and reset instruction. Leave the old database untouched.
+
+## Pi API impact
+
+- **Session state:** Pi appends normal messages and tool results. Pi Workflows does not edit session files.
+- **Other persistent data:** The workflow SQLite shape changes in place and older alpha state requires reset.
+- **Pi internals:** None.
+- **Public API:** The extension uses documented command registration, tool registration, session lifecycle events, message sending, widgets, status, and session IDs.
+
+## Conformance
+
+The implementation conforms when:
+
+- every protected write checks and renews one live claim atomically;
+- an expired or replaced owner cannot write;
+- a blocked worker cannot stop host renewal;
+- Pi can restart while work computes or waits;
+- the host can restart and recover from committed state;
+- run, queue, attempt, decision, lease, event, and viewer projections remain consistent after injected crashes;
+- an expired running row can be resumed or cancelled safely;
+- duplicate commands and submissions return stored receipts;
+- an interactive request appears once in the origin session and survives reload;
+- effects are deduplicated or marked ambiguous;
+- the extension and host run no workflow or controller code in their own event loops;
+- the production package contains no embedded execution fallback;
+- real Pi end-to-end tests, repository checks, reviewer checks, and CI pass.

package/docs/development.md

@@ -3,15 +3,18 @@
 This document covers the standards for working on pi-workflows itself. For
 authoring workflows, see [workflows.md](workflows.md).
 
+The viewer follows the [incremental and virtualized viewer plan](plans/2026-08-28-piw-incremental-viewer-plan.md). Durable writers create revisioned target patches. Rust reads small run-list rows, bounded replay pages, and one shared watched-run projection. Rust and TypeScript share graph fixtures and the same retained scene contract.
+
 ## Layout and boundaries
 
 ```
 src/workflows/   finite graph engine: definitions, execution, SQLite stores, loader
 src/builtins/    default workflows shipped at lowest discovery precedence
 src/controllers/ durable resources, queue, reconciliation, effects, child runs
-src/extension/   pi integration: commands, workflow tool, controller host, widget
+src/host/        global host, local protocols, resolvers, and worker supervision
+src/extension/   thin Pi client: commands, workflow tool, and origin-session bridge
 src/viewer/      standalone read-only views over runs and controller resources
-tui/             Rust piw viewer and live replay server
+tui/             Rust piw viewer, bounded SQLite projection, and replay server
 ```
 
 The dependency direction is enforced by `slophammer.yml`. `src/workflows`
@@ -20,10 +23,12 @@ progress validation, estimation, and text formatting stay in this layer so the
 engine, extension, hosts, and viewers share one contract. `src/builtins` contains
 package-owned definitions and imports only the public workflow engine.
 `src/controllers` may import the public workflow engine for child-run
-scheduling. `src/extension` and `src/host` may also import the built-in catalog.
-The extension and viewer never import each other. The viewer reads
-SQLite runs and opens the controller SQLite database read-only, so it works
-from any process.
+scheduling. `src/host` may import controller stores and workflow stores, but the
+host event loop never loads workflow or controller definitions. Resolver and
+worker child entry points load those definitions. The extension uses static
+built-in metadata and never imports the workflow engine. The extension and
+viewer never import each other. The viewer reads SQLite state in read-only mode,
+so it works from any process.
 
 Within `src/render`, `graph.ts` computes a pure layered layout (ported from
 the acpx replay viewer: labelled switch expansion, DFS back-edge detection,
@@ -31,12 +36,7 @@ longest-path layering, barycenter ordering, virtual pass-through cells for
 long edges), `canvas.ts` is a character grid that merges box-drawing
 characters by connectivity, and `graph-render.ts` turns a SQLite run state plus a
 replay position into the drawn graph in one of two node styles: `box`
-(bordered nodes, used by the viewer and the in-pi widget) or `line`
-(single-line nodes). The widget windows the boxed graph around the active
-node to stay inside pi's 10-line widget cap; `shift+↑`/`shift+↓` shortcuts
-(registered through pi's `registerShortcut`) scroll that window manually, and
-the scroll resets to follow mode when the run records a new step. `render.ts`
-in `src/viewer`
+(bordered nodes) or `line` (single-line nodes). `render.ts` in `src/viewer`
 composes the full detail view (header, graph, step timeline, step inspector)
 and stays pure so tests can assert on rendered lines.
 
@@ -63,18 +63,16 @@ workflow shapes at every replay position; if a rendering change breaks a
 line, misplaces an arrow, or lets a label damage an edge, those tests fail
 with the offending drawing in the assertion message.
 
-Inside the engine, the pi-facing seam is the `AgentStepExecutor` interface.
-The extension implements it on top of the live conversation
-(`src/extension/executor.ts`), and tests implement it with a scripted fake
-(`test/helpers.ts`). Anything that would couple the engine to pi belongs on
-the extension side of that seam.
+Inside the engine, the model-facing seam is the `AgentStepExecutor` interface.
+The production run worker uses an interaction executor for origin-session work
+and `RpcStepExecutor` for headless controller work. The worker receives a
+`HostBackedWorkflowStore`; it cannot open a writable production store.
 
-Temporal session capture follows the same boundary. `src/extension` listens to
-Pi's documented `turn_*`, `message_*`, and `tool_execution_*` hooks and
-normalizes them before passing records to `WorkflowRunStore`. The workflows layer owns persisted shapes and schema validation. It also owns
-ordered append chains but never imports Pi types. High-rate hooks only stamp and enqueue bounded records;
-disk writes run on a separate chain. Capture failures are explicit in
-the `session_segments` capture status and never fail workflow execution.
+The Pi extension uses documented session lifecycle, message, tool, and session
+manager APIs. It writes no Pi session file. It saves host mutations through the
+local protocol and uses the public session branch only to adopt the exact
+visible entry for a durable interaction request. Tests can still use scripted
+executors through the Pi-agnostic engine API.
 
 `src/viewer/session-reducer.ts` and `tui/src/session.rs` implement the same
 sequence-ordered fold. Shared fixtures in `fixtures/session-events/` pin their
@@ -104,6 +102,23 @@ npx slophammer-ts@latest dry .
 npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
 ```
 
+## Viewer performance checks
+
+Create a synthetic growing database and run the release benchmark:
+
+```bash
+mkdir -m 700 /tmp/piw-viewer-benchmark
+npx tsx scripts/generate-viewer-benchmark.ts \
+  /tmp/piw-viewer-benchmark/state.sqlite 44 405 211 1105 1200
+cargo run --release --manifest-path tui/Cargo.toml \
+  --example refresh_benchmark -- \
+  /tmp/piw-viewer-benchmark/state.sqlite 1000
+```
+
+The generator prints only row counts and database size. The benchmark prints run count, query and load counts, payload rows, raw tick times, and peak RSS. It does not print run IDs or payload text.
+
+The deterministic gate is zero payload reads after the first bounded selected window during unchanged idle checks. The measured gates are p99 main-thread checks below 16 ms and peak RSS below 185 MB on a database close to the registered 97 MB fixture. Run the benchmark several times and report the median, range, p99, maximum, and peak memory. Do not select extra complexity from one best run.
+
 ## End-to-end tests
 
 ```bash
@@ -120,10 +135,12 @@ the real pi CLI from `devDependencies` in RPC mode with:
 - `HOME` pointed at a temporary home containing the canonical workflow database,
 - the extension loaded from source with `-e src/extension/index.ts`.
 
-It drives `/workflow` over the RPC protocol and asserts on the resulting SQLite
-rows, including temporal events, final entry linkage, capture integrity, and
-terminal immutability, then renders the finished run through the viewer CLI.
-Nothing outside the temp directories is touched, and no real model is called.
+It drives `/workflow` over the RPC protocol and checks the global host,
+durable interaction request, origin-session presentation, accepted submission,
+worker recovery, final SQLite state, and viewer output. It also restarts the
+real Pi process while one request is pending and proves that the same request is
+not inserted twice. Nothing outside the temp directories is touched, and no
+real model is called.
 
 ## Publishing
 
@@ -155,9 +172,8 @@ before publishing.
 ## Conventions
 
 - Conventional Commits for commit messages and PR titles.
-- Persisted JSON uses camelCase keys and versioned `schema` identifiers; see
-  [SQLITE_STATE.md](SQLITE_STATE.md). Breaking a persisted shape means bumping
-  the schema version string.
+- Persisted JSON uses camelCase keys and current `schema` identifiers; see
+  [SQLITE_STATE.md](SQLITE_STATE.md). During alpha, a breaking shape changes the current contract in place and uses the clear reset failure. Do not add a compatibility path or `v2` only to preserve old alpha state.
 - Every exported API of the engine (`src/workflows/index.ts`) is covered by
   unit tests; new node types or edge semantics need tests in `test/` and a
   section in [workflows.md](workflows.md).