@smthrs/gateway 0.35.0 → 1.0.0-rc.4

package/docs/concepts/projections.md

@@ -0,0 +1,250 @@
+---
+title: "Projections"
+description: "Why the gateway serves folds over control events instead of database rows, what the eight projections answer, and how a node gets its id."
+sidebar:
+  order: 1
+---
+
+A projection is a read model: a row folded from the ordered `ControlEvent`
+deltas [`@smthrs/control`](/api/control) already publishes, plus the run summary
+the control plane already exposes. Subscribing to one never claims a run and
+never writes.
+
+The rule behind every design decision in this package is that the gateway never
+opens the engine database. `Projections` reads through two control-plane
+operations and nothing else: `Control.list` for run and flow listings, and
+`Control.watch` for one run's ordered events. Two consequences follow, and both
+are the point:
+
+- A projection served to a browser through a relay is the same projection a
+  local reader computes. There is no privileged path.
+- A projection cannot drift from the control plane by reading a column the
+  control plane does not expose, because it has no column to read.
+
+That is why a UI depends on this package and on `@smthrs/control`, and never on
+[`@smthrs/engine-store`](/api/engine-store). A projection is the contract; a
+store row is an implementation detail.
+
+## The eight projections
+
+`GatewaySchema.ProjectionName` is the authority for the list, and the set a
+release serves is the set the schema declares.
+
+| Projection       | Selector                | Answers                                                                                         |
+| ---------------- | ----------------------- | ----------------------------------------------------------------------------------------------- |
+| `workspace-runs` | `WorkspaceRunsSelector` | one summary row per run in the workspace                                                        |
+| `run-summary`    | `RunSummarySelector`    | one run's card: status, timing, activity counts, and the diagnosis of what happened to it       |
+| `run-events`     | `RunEventsSelector`     | the run's ordered control events, unfolded                                                      |
+| `transcript`     | `TranscriptSelector`    | one turn-numbered line per reported event                                                       |
+| `run-tree`       | `RunTreeSelector`       | the agent cell calls the run made, keyed `call-1`, `call-2`, and so on                          |
+| `approvals`      | `ApprovalsSelector`     | with a run, that run's gates including decided ones; without one, the workspace's pending gates |
+| `node-output`    | `NodeOutputSelector`    | the value one settled call produced                                                             |
+| `flow-durations` | `FlowDurationsSelector` | how long one flow's nodes take, per action tag, over its newest finished runs                   |
+
+`GatewaySchema.rowSchemaFor` maps a selector to the schema of the rows it
+answers with, so a client decodes a snapshot instead of casting it.
+
+## The rows speak the control plane's vocabulary
+
+Wire names are the flows names. `RunSummaryRow.flowId` carries what an older
+wire split across `workflowKey` and `workflow`; `createdAt` carries what it
+called `createdAtMs`; `ApprovalRow.payload` is the `ApprovalTarget.Node`
+envelope a client submits back unchanged. A client written against the control
+plane reads these rows with no translation table, and no client reconstructs
+authority for itself.
+
+`ControlFacts.fold` is the shared run/approval fold used by both snapshots and
+subscription deltas. Current control lifecycle writers include `factVersion: 1`
+and the complete fenced control snapshot in the event. The first accepted
+snapshot has a `created` baseline. The first upgraded transition of an older run
+has a `legacy` baseline: it establishes state at that sequence and does not
+invent the missing earlier history.
+
+`RunSummaryRow.lifecycleProvenance` explains the boundary. `control: "events"`
+means the latest versioned snapshot agrees with the current coordination row's
+status and update time. `legacy-snapshot` means no versioned baseline is present.
+`unverified-snapshot` means the row is ahead of the covered facts, a lifecycle
+fact is missing, or a producer version is unknown. These last two cases retain
+the observed row; they are never labelled successful replay. A later complete
+snapshot begins a new legacy baseline after a gap.
+
+Engine observations are independent: `execution: "engine-observed"` preserves
+the status and waiting/ancestry fields `ControlLive` read from the executor;
+`engine-missing` records that no execution was visible. Neither is claimed as
+control-journal replay. No projection read writes a migration or acquires a run.
+The field is optional on the wire for older gateway compatibility.
+
+Current approval facts bind the full target, run, request, and digest. Duplicate
+requests cannot reopen a decided gate; exact decisions can precede a request.
+Legacy payloads remain readable, and an unnamed legacy decision can close only
+a legacy request in that run. An unknown current identity never consumes another
+gate. Serving historical approval content grants no authority: the control
+decision endpoint still validates the exact target and principal.
+
+## Committed flow results
+
+`RunSummaryRow.finalOutput` prefers the assistant's final text. Otherwise it
+uses the committed successful result of the `agent/run` execution named by
+`control.engine.bound`. A matching `control.engine.event` must carry the
+completed root's versioned run decision. Child results, unbound events, and
+conflicting evidence do not supply output. Strings are preserved; other JSON
+values are serialized. The binding and result survive in the carried diagnosis
+when older events leave the retained window.
+
+## How a node gets its id
+
+`run-tree` folds agent cell calls, not child runs. A node opens on a call
+invocation and settles on the matching recorded result. The native action owner
+commits `flows.harness.call-fact.v1` inside its attempt transaction, then the
+existing engine bridge copies it into `control.engine.event`. `Diagnosis`'s
+shared `nativeCallEvent` and `uniqueCallEvents` normalize these facts with
+compatible `control.agent.cell-call-started` / `cell-call-settled` telemetry.
+`AgentSession` still emits the latter on its best-effort trace channel.
+The normalized start carries `{callId, flowName, input}` and the settlement
+carries `{callId, flowName, outcome, message, value}`. `callId` is
+`cell-call-v1:` followed by the SHA-256 digest of the canonical JSON dispatch
+identity: session, frame, cell digest, invocation ordinal, declaration digest,
+and ordered active layers. The harness creates that identity before dispatch
+and carries it unchanged into settlement. The same call keeps its identity
+across replay, and two calls of the same flow can settle in either order.
+
+The public node key remains the ordinal of each distinct start: `call-1`,
+`call-2`, and so on. Identified settlements join by `callId`. Repeated starts
+or settlements with the same ID contribute one observation to
+the tree, node output, transcript, and diagnostic counts. A committed native
+fact supersedes identified telemetry while retaining the first observation's
+position and sequence, so existing node references remain stable. Without a
+native fact, the first identified telemetry record remains the observation.
+Transcript rows
+expose the optional `callId`; `run-events` continues to expose every journal
+record, including duplicates, for inspection.
+
+Older records have no `callId` and are read without rewriting history. They
+retain FIFO matching among **unidentified** starts with the same flow name.
+An identified settlement may match an unidentified start when an older parked
+run resumes with the new writer. Neither an unknown ID nor an unidentified
+settlement can consume an identified start. A settlement with no eligible
+start produces no node output. Correlation of overlapping, same-name legacy
+calls cannot be recovered exactly from those old records.
+
+`node-output` keys its rows the same way and advances its ordinal on exactly
+the same distinct starts, so the node id a tree view shows is the node id
+`node-output` answers for. The CLI/MCP node-output reader retains its existing
+`flowName#ordinal` IDs, counting distinct starts within each flow, and uses
+the same identity/legacy matching rule. A fold that skipped a distinct start
+before advancing its ordinal would shift references to later nodes.
+
+A node that never settled stays `running`, which is how a live tree renders
+work in flight.
+
+## A duration is measured, never estimated
+
+`flow-durations` is the one projection that reads across runs. It lists the
+flow's runs through `Control.list`, keeps the newest `Projections.maxDurationRuns`
+that reached a terminal status, and folds each one's node records into samples.
+
+A sample is one node the engine really ran: the distance between the
+`flows.engine.node-scheduled` record that admitted it and the
+`flows.engine.node-settled` record that closed it, both copied into the control
+journal as `control.engine.event` envelopes by the host's engine bridge. Only
+the `built` outcome counts. `clean` was served from records rather than run,
+`failed` measures a collapse, `skipped` was never reached, and `deferred` is
+scheduling debt, so none of them describes how long the work takes.
+
+Samples are grouped by the node's action tag, because the tag is the part of a
+node's key material that survives a re-key: the same step keeps its tag when
+its input changes, and its plan key and dispatch key do not. Both percentiles
+are nearest-rank, so `p50Ms` and `p90Ms` are durations the flow really took.
+`samples` is on the wire beside them, because a percentile over one sample and
+a percentile over twenty are different claims.
+
+One call is one sample. The engine records a `FlowCall` twice: the caller
+admits and settles the node that made the call, and the callee's own execution
+admits and settles its `root` over the same span with the same tag. The fold
+keeps the caller's node and drops the callee's root, which it can tell apart
+because the host's `control.engine.bound` record names the native root the
+control run owns. A run with no such record, such as one no engine bridge
+wrote, keeps both: a fold that cannot name the root cannot name the echo
+either.
+
+A duration is wall time between two engine records, so a node that waits is
+measured through its wait. A flow tag whose flow parks on a `HumanTask`
+measures the park, and so does every flow tag above it. These rows answer how
+long the flow took, not how much compute it spent.
+
+A tag nothing has measured has no row, and a flow nothing has finished answers
+with no rows at all. There is no shape here for an unmeasured prediction.
+
+## What is not in a projection
+
+A host keeps the control plane and the engine in two databases with two
+journals. `Control.watch` reads one run's control partition; the gateway does
+not open the native database. Native call and execution facts become readable
+through the durable engine bridge, which can lag its source. Other engine
+records remain inspection material unless a specific pure fold recognizes
+their versioned producer contract. Model deltas and printed trace text still
+have best-effort delivery; call fact coverage does not make the entire trace
+replayable. Public values are redacted and bounded commitments, not private
+native outcome rows.
+
+## The folds are pure and total
+
+`GatewayProjection.runSummary`, `runTree`, `approvals`, `nodeOutput`, and
+`transcript` are ordinary functions from control facts to rows. They read no
+service, and an event kind outside their vocabulary contributes nothing rather
+than failing the fold. Wire payloads are JSON, so every field read tolerates
+absence: the digest of a malformed journal is a sparse digest, never a throw.
+
+That purity is what makes a delta trustworthy. A subscription recomputes the
+selector's rows from accumulated events rather than patching them, and
+recomputation is only safe because the same events always fold to the same
+rows. `run-events` sends only what one event added. `transcript` normally does
+the same, but a later native fact can correct previously emitted telemetry. It
+then sends the existing `snapshot-start` / `row` / `snapshot-end` reset so the
+subscriber converges to a fresh fold at the same cursor. See
+[Subscriptions and cursors](./subscriptions.md).
+
+## What a projection costs
+
+A projection reads one journal per run, so the read is bounded on purpose:
+
+| Bound                            | Value  | What it caps                                                                |
+| -------------------------------- | ------ | --------------------------------------------------------------------------- |
+| `Projections.maxWorkspaceRuns`   | 500    | runs one workspace projection folds, newest first                           |
+| `Projections.maxEventsPerRun`    | 10,000 | retained events per run                                                     |
+| `Projections.maxEventsPerPage`   | 1,000  | events per `run-events` page                                                |
+| `Projections.maxEventBytes`      | 16 KiB | retained event size before clipping                                         |
+| `Projections.maxProjectionBytes` | 4 MiB  | retained events and digest identity state, event page, or projected row set |
+
+`maxWorkspaceRuns` equals `ControlSchema.maxPageSize`, so the control plane can
+satisfy the whole gateway allowance in one page when it can. A workspace with
+more runs is answered with its newest 500. Older retained events are folded into
+a carried digest. Compact scalar contributions retain call and checkpoint
+identities so replay is counted once and committed native facts can correct
+earlier telemetry across the window boundary. These contributions share the
+retained byte budget; their inputs, outputs, and event bodies are not retained.
+If the exact identity state or a projected row set cannot fit, the read fails
+with `resource_limit`.
+
+`run-events` pages preserve complete `control.engine.event` payloads, including
+native typed results, without the 16 KiB clipping applied to other large events.
+A page stops at its event or byte budget and returns a cursor. One event whose
+encoded size plus array brackets exceeds 4 MiB fails with `resource_limit`.
+
+The approvals inbox is the one workspace projection that filters before it
+counts: it asks the control plane for runs whose status is `waiting-approval`,
+and admits only runs still in that status with at least one pending gate after
+reconciling their journals. Snapshot admission and live refresh use the same
+predicate: cancellation removes a run even when its gate remains pending.
+A run with no pending gate does not consume the source allowance. An inbox
+cannot be exhausted by completed histories.
+
+Native lifecycle uses an independent `ExecutionFact` fold over the host-bound
+engine bridge. Run rows may include `executionProvenance` (`events`,
+`legacy-observation`, or `unverified-observation`) alongside the existing
+`lifecycleProvenance`; engine replay does not become control-stream authority.
+The fold verifies the native root/current-round view against the executor's
+coherent observation. A missing binding, native generation gap, unsupported
+version, or differing observation retains explicit fallback. Snapshot and
+subscription paths share this fold. The protected native wait token remains in
+the execution store; the event carries its SHA-256 digest for comparison.

package/docs/concepts/subscriptions.md

@@ -0,0 +1,204 @@
+---
+title: "Subscriptions and cursors"
+description: "The frame vocabulary a subscription speaks, which deltas replace and which append, what a delta costs, what a cursor is made of, and which subscriptions can resume from one."
+sidebar:
+  order: 2
+---
+
+`Projection.Snapshot` answers the rows a selector currently projects.
+`Projection.Subscribe` answers a stream of tagged frames instead, so a client
+can tell a snapshot from a change without guessing.
+
+## The frames
+
+| Frame                | Means                               |
+| -------------------- | ----------------------------------- |
+| `SnapshotStartFrame` | the snapshot begins, at this cursor |
+| `RowFrame`           | one row of the snapshot             |
+| `SnapshotEndFrame`   | the snapshot is complete            |
+| `DeltaFrame`         | the selector's rows after a change  |
+| `HeartbeatFrame`     | the connection is alive             |
+
+`GatewaySchema.GatewayFrame` is the union of all five. Every frame except the
+heartbeat carries the selector it belongs to and the cursor it was produced at,
+so a client multiplexing several subscriptions on one socket routes on the
+frame rather than on the request id.
+
+## A delta replaces, except where it appends
+
+For `workspace-runs`, `run-summary`, `run-tree`, `approvals`, and
+`node-output`, `DeltaFrame.delta` is the whole row set of the selector,
+recomputed from the events accumulated so far. It is not a patch.
+
+That is a deliberate trade. A projection is a reproducible fold, and
+recomputation is the only delta that cannot disagree with a fresh snapshot. A
+patch would have to encode an inverse for every fold, and any bug in one would
+leave a client's view permanently wrong in a way nothing detects.
+
+Two projections send append deltas: a delta carries only what one event added.
+The client appends it to the rows it already holds.
+
+| Selector     | Delta carries                                     |
+| ------------ | ------------------------------------------------- |
+| `run-events` | the one event that arrived                        |
+| `transcript` | the rows that event contributed, usually one line |
+
+One projection sends no delta at all. `flow-durations` ranks the runs a flow
+has already finished, so no event of a live run changes a row, and the run that
+would change one is outside the window until it settles. Subscribing to it
+answers the snapshot and ends; a client that wants newer rows takes another
+snapshot.
+
+`run-events` rows _are_ the immutable ordered events. `transcript` normally
+adds a row with the same turn counter a full fold would give it. A later native
+call fact can supersede earlier identified telemetry; then the follower sends
+the existing `snapshot-start` / `row` / `snapshot-end` sequence at the new
+cursor to replace those rows. Applying the reset as well as append deltas
+yields exactly the transcript a fresh snapshot would fold. Unidentified legacy
+history is never assigned a guessed call identity to force a match.
+
+Events accumulate in the stream, so recomputing each delta does not re-read
+the journal. Resuming a historical `run-summary` first rebuilds its compacted
+health prefix once through durable follow replay, as described below. During
+ordinary run following, `run-summary`, `run-tree`, and `approvals` re-read the
+run row for current lifecycle state and open human waits.
+
+## What a delta costs
+
+A run subscription folds once per event. A replacing selector sends its whole
+row set on each delta, bounded by `Projections.maxProjectionBytes`; an
+appending selector sends one event's rows, so catching up on E events sends E
+rows in total.
+
+A workspace subscription does not fold per event. The follower gathers the
+events that arrive within a 50 millisecond window, or the first 1024 of them,
+and admits them as one batch. Each run's rows are cached and refolded only
+when that run's journal grew, and a grown run's row is re-read once per batch
+rather than once per event. One batch answers one `DeltaFrame` carrying the
+whole workspace, so a burst across many runs costs one frame plus one run
+lookup per run that changed. The window bounds the frame rate at twenty per
+second no matter how many runs journal at once; a quiet workspace sends no
+delta at all.
+
+## A reconciled snapshot cutoff
+
+The gateway reads each run summary before and after its journal snapshot. If
+the summary changed, it retries with the new summary, up to eight journal
+reads. A row that keeps changing is refused with `run_unavailable`. This keeps
+a terminal event from being acknowledged with the preceding run status.
+
+Snapshot rows and the follow seed share the same final event buffer. Workspace
+snapshots reconcile up to eight runs concurrently. Listing stops at 500 runs,
+an empty page, or a repeated cursor; each continued page adds a run, so at most
+500 pages are read.
+
+An empty event buffer follows from the beginning, including journal sequence
+zero. A buffer that already contains sequence zero suppresses those seeded
+events, including their derived offsets.
+
+## What a cursor is
+
+`GatewaySchema.ProjectionCursor` carries five things:
+
+| Field        | Holds                                                                         |
+| ------------ | ----------------------------------------------------------------------------- |
+| `selector`   | the exact selector that issued it                                             |
+| `projection` | that selector's projection name                                               |
+| `runId`      | the source run, or `null` for a workspace projection                          |
+| `value`      | the journal sequence the read reached                                         |
+| `offset`     | the position within that sequence, for several derived events at one sequence |
+
+The selector is embedded rather than referenced, so two `node-output`
+subscriptions on different nodes of one run cannot share a cursor. The
+`(value, offset)` pair rather than a bare sequence is what lets a client resume
+inside one journal sequence without losing a derived event.
+
+## Resuming
+
+Pass a cursor as `after` and the subscription skips the snapshot, answering the
+deltas after that cursor alone. The read still happens, because a folded
+projection cannot be recomputed from the events after the cursor alone. What
+the client skips is receiving rows it already has.
+
+A resumed run summary rebuilds its health prefix from the durable subscription
+replay before emitting deltas. The latest compacted buffer may have replaced an
+observation that was current at the older cursor; reusing that buffer could let
+a late, weaker reading displace stronger evidence. Replay uses the same bounded
+compaction as a snapshot and emits nothing at or before the issued cursor.
+
+The bounded fold admits only observations for the run's own subject before
+applying evidence precedence or eviction. Foreign-subject observations still
+advance the raw journal cursor. It retains at most 16 health incarnations and
+protects the authoritative run incarnation's strongest evidence from eviction. Late readings
+from former owners cannot clear that reading or let weaker evidence replace it;
+equal evidence advances only with a later durable sequence. Live run summaries
+refresh the run row before compacting each event, and workspace subscriptions
+refresh it before compacting that run's first new event in a batch.
+
+Every frame of one snapshot carries the same cursor, the position the read
+reached, so a cursor from a row proves nothing about which rows arrived. A
+client commits a snapshot and its cursor together, at `snapshot-end`. A
+connection cut before `snapshot-end` leaves no cursor to resume from: the
+client re-subscribes without `after`.
+
+A cursor is refused with `malformed_request` when it:
+
+- names a different projection than the subscription's selector;
+- names a different selector of the same projection;
+- names a different run than the subscription's;
+- is negative, fractional, or not a safe integer in `value` or `offset`;
+- names a position past the run's last position;
+- names a position the run never issued.
+
+## Workspace subscriptions do not resume
+
+Control journal sequences belong to per-run partitions, so there is no
+workspace-wide sequence to advertise. A workspace cursor is therefore always
+`value` 0 with a `null` run, and passing one back is refused: a
+`workspace-runs` or unscoped `approvals` subscription has no resumable cursor
+at all.
+
+It can still follow, and does, by following every partition without a cursor.
+The follower drops replay through each retained run's last sequence and
+duplicate-sequence offset. Each source stores that position and updates it on
+append, so checking replay does not scan its journal.
+
+Sources and cached exclusion verdicts share a `maxWorkspaceRuns` ceiling of
+500 entries. An excluded run retains its journal cutoff and observed position,
+without its history. Replayed events through that cutoff cost no further
+journal reads; later events reconsider the run. Missing run partitions,
+including `plan:` partitions, are skipped after the first lookup failure while
+the verdict is retained. New entries evict the oldest exclusion verdict and
+its cursor state. An evicted run may require another read. At 500 admitted
+sources, a run created after the oldest followed run displaces it, so the
+tail keeps the newest runs the snapshot folded. An older run stays out.
+
+## Heartbeats
+
+An idle subscription emits a `HeartbeatFrame` every
+`Projections.heartbeatIntervalMillis`, which is 30 seconds. A relay cuts an
+idle tunnel at 600 seconds, so a quiet run has to produce a frame well inside
+that window or every follower is disconnected and reconnects. Thirty seconds
+leaves twenty heartbeats of margin. `Projections.layerWith({ heartbeatMillis })`
+shortens it for a relay that cuts sooner. `ServerOptions.heartbeatMillis` at
+the bind re-times both this keepalive and the `Watch` keepalive below, so one
+option governs every followed socket the gateway serves.
+
+The first tick is dropped, so the cadence is what it says it is: an immediate
+keepalive would arrive before the snapshot it is meant to keep alive.
+
+## The other keepalive
+
+`ControlRpcs.Watch` on `/rpc/ws` has the same idle problem and no frame type of
+its own for a keepalive, so the gateway publishes one as a `ControlEvent` whose
+kind no emitter uses: `GatewayServer.watchHeartbeatKind`, the literal
+`control.gateway.heartbeat`. A fold that does not know the kind ignores it,
+which is what every fold in this package already does with an unknown kind.
+
+Two properties make it safe to ignore and safe to read:
+
+- It repeats the sequence and run-scoped cursor of the last event delivered,
+  including any expansion offset. Before delivery it retains the resume cursor.
+- It carries the watched run id, so a client routing by run keeps routing.
+
+A snapshot read, `follow: false`, is left alone. It has to end.