@smthrs/control 0.0.0-stage → 1.0.0-rc.4

package/docs/guides/testing.md

@@ -0,0 +1,139 @@
+---
+title: "Test against the control plane"
+description: "Use the deterministic in-memory stack, swap one collaborator at a time, exercise the pure projections with no stack at all, and hold your own runtime to the contract both shipped ones satisfy."
+sidebar:
+  order: 13
+---
+
+Nothing about the control plane is hard to test, because every nondeterministic
+input is a service: the clock, the runtime, the journal, the notification
+queue, and the executor. A test swaps the service, not the verb.
+
+## Start with the whole stack
+
+`TestControl.layer` bundles the four collaborators `ControlLive` requires,
+already deterministic:
+
+```ts
+import { Control } from "@smthrs/control/Control"
+import type * as ControlRuntime from "@smthrs/control/ControlRuntime"
+import * as TestControl from "@smthrs/control/test/TestControl"
+import * as Effect from "effect/Effect"
+
+const Deploy: ControlRuntime.MemoryFlow = {
+  flowId: "ops/Deploy",
+  description: "Deploys one build",
+  deployClass: true,
+  envelope: { capabilities: [], flows: [], budget: {} }
+}
+
+const stack = TestControl.layer({ flows: [Deploy], now: () => 0 })
+```
+
+It provides `Control` together with every collaborator it built:
+`ControlRuntime`, the in-memory journal bundle, a notification queue over that
+journal, an executor, and an empty registry. Ids are derived from counters, so
+`plan-1`, `run-1`, and `fence-1` are stable across runs.
+
+`MemoryOptions` is the whole configuration surface:
+
+| Option      | Effect                                                                    |
+| ----------- | ------------------------------------------------------------------------- |
+| `flows`     | What the plane may plan. Defaults to the plannable reserved system flows. |
+| `now`       | The clock every timestamp reads. Pass `() => 0` for stable output.        |
+| `principal` | The identity stamped when a caller names none.                            |
+
+## Swap the executor
+
+`TestControl.layer` takes an executor as its second argument, so a test states
+exactly what the engine did:
+
+```ts
+import * as ControlExecutor from "@smthrs/control/ControlExecutor"
+
+const launched: Array<string> = []
+
+const executor = ControlExecutor.makeNoop({
+  launch: ({ run }) =>
+    Effect.sync(() => {
+      launched.push(run.runId)
+      return "accepted" as const
+    })
+})
+
+const stack = TestControl.layer({ flows: [Deploy], now: () => 0 }, executor)
+```
+
+Each answer in the executor's vocabulary is a distinct composition worth a
+test: `pending` releases the row, `accepted` promotes it to `running`, a
+`LaunchFailed` settles it as `failed`, and a `CancelTerminal` reconciles the
+control row onto the engine's status. See
+[Connect an execution engine](./implement-an-executor.md).
+
+The default is `ControlExecutor.makeNoop()`, which answers the honest absence
+for every method, and _no executor at all_ is a different composition again:
+`ControlLive` reads the port optionally, so a plane that starts nothing is a
+supported shape rather than a broken one.
+
+## Test the projections with no stack
+
+`classify`, `remedyFor`, `originOf`, `derive`, `expand`, `attribute`, and
+`steerItem` are pure functions of their arguments. Enumerate them directly:
+
+```ts
+import * as Monitor from "@smthrs/control/Monitor"
+
+expect(Monitor.classify({
+  summary: { runId: "run-1", flowId: "ops/Deploy", status: "running", createdAt: 0, updatedAt: 0 },
+  events: [{ sequence: 1, kind: Monitor.attemptStartedEventType, runId: "run-1", occurredAt: 0, payload: {} }],
+  beatsWithoutProgress: 3,
+  stallBeats: 3
+})).toBe("wedged-node")
+```
+
+Splitting `beatsWithoutProgress` from `stallBeats` is what lets one pure
+function serve a monitor that beats every second and one that beats every hour,
+and it is what lets a test reach a stall without waiting for one.
+
+## Assert on durable evidence
+
+The plane's promises are visible in the journal, so assert there rather than on
+internal state:
+
+```ts
+const kinds = yield * control.watch({ runId, follow: false }).pipe(
+  Stream.map((event) => event.kind),
+  Stream.runCollect
+)
+expect([...kinds]).toEqual(["control.run.accepted", "control.run.pending"])
+```
+
+Use `follow: false`. It ends; the live stream does not.
+
+## Hold your own runtime to the contract
+
+`ControlRuntime.layerMemory` and `SqlControlRuntime.layer` are both held to one
+[shared contract suite](https://github.com/smithersai/smithers/blob/main/packages/smithers/control/test/ControlContract.ts).
+It is not part of the published tarball, so a third implementation copies it
+from the repository and runs it against its own layer. That is what makes
+"behaves like the memory runtime" a checkable claim rather than a hope.
+
+## Assert on the refusals too
+
+Several behaviors are refusals, and they carry the sentence an operator reads:
+
+```ts
+const error = yield * Effect.flip(control.list({ _tag: "runs", limit: 0 }))
+expect(error.issue).toBe("limit: must be an integer between 1 and 500, received 0")
+```
+
+`ControlClient.isControlError` narrows an unknown value to the declared union,
+derived from the same schema the errors are declared in.
+
+## Where to go next
+
+- [Quickstart](../quickstart.md): the smallest complete program on this stack.
+- [Troubleshooting](../troubleshooting.md): the refusals worth a test of their
+  own.
+- [Testing flows on smithers.sh](/docs/guides/testing-flows/): the same habit,
+  one layer down.

package/docs/guides/watch-a-run.md

@@ -0,0 +1,154 @@
+---
+title: "Watch a run's events"
+description: "Read a finite snapshot or follow a live stream of control events, resume at a cursor without seeing an entry twice, and read the lineage and steer-delivery deltas the plane derives."
+sidebar:
+  order: 3
+---
+
+`watch` streams `ControlEvent` values projected from committed journal entries.
+It is a read: nothing you do with the stream changes a run.
+
+## Take a finite snapshot
+
+`follow: false` asks for what is durable when the request is handled. The
+stream ends, which is what makes it assertable in a test or usable in a
+one-shot report:
+
+```ts
+import { Control } from "@smthrs/control/Control"
+import * as Effect from "effect/Effect"
+import * as Stream from "effect/Stream"
+
+const kinds = Effect.gen(function*() {
+  const control = yield* Control
+  return yield* control.watch({ runId: "run-17", follow: false }).pipe(
+    Stream.map((event) => event.kind),
+    Stream.runCollect
+  )
+})
+// [ "control.run.accepted", "control.run.running", "flows.engine.attempt-started", ... ]
+```
+
+## Follow a live stream
+
+Omit `follow` and the stream stays open. This is what a UI subscribes to:
+
+```ts
+const tail = control.watch({ runId: "run-17" }).pipe(
+  Stream.runForEach((event) => Effect.log(`${event.sequence} ${event.kind}`))
+)
+```
+
+A subscriber that arrives late still receives the entries it missed. The
+projection pins a high-water sequence per partition, reads everything at or
+below it from a finite snapshot, and takes everything above it from the
+buffered tail. It is a handoff rather than a deduplicated overlap, so an
+arbitrarily long history cannot make an old entry reappear.
+
+## Resume at a cursor
+
+Store `event.cursor` after processing each event, then pass it back unchanged
+as `afterCursor` with the same `runId`:
+
+```ts
+const resumed = control.watch({ runId: "run-17", afterCursor: lastSeen })
+```
+
+A cursor is `{ sequence: number; offset?: number }`. `sequence` identifies the
+source journal entry. A present `offset` is the zero-based index of the last
+consumed member of that entry's expansion. An absent offset means the entire
+entry was consumed. The last member always carries this completed-entry cursor,
+so the next watch starts after the source row without rereading it.
+
+For a promotion at sequence 12 that delivers two messages, the emitted cursors
+are `{ sequence: 12, offset: 0 }` for the source,
+`{ sequence: 12, offset: 1 }` for the first delivery, and `{ sequence: 12 }`
+for the second. Reconnecting after the source still yields both deliveries.
+This works for finite snapshots and live streams while the source row remains
+in the journal. Commit the checkpoint with your event processing to avoid
+reprocessing an event after a consumer crash.
+
+`afterSequence` remains available to skip a fully processed source entry and
+all its derived events. It cannot checkpoint progress inside an expansion.
+Do not combine it with `afterCursor`.
+
+Both cursor forms require `runId`. Sequences are partition-local:
+
+```text
+InvalidInput: afterCursor: a watch cursor resumes one run, so it requires runId
+```
+
+Raw `Lineage` and `Steering` projections and older providers can omit `cursor`.
+`ControlLive.watch` assigns a cursor to every emitted event.
+
+## Watch everything
+
+Omit `runId` and the stream merges every partition the plane knows: each run,
+and each plan under `plan:<planId>`. Eight partition snapshots are read at a
+time, and the live tail runs beside them so snapshot work never starves it.
+
+The plane lists partitions in insertion order, 100 ids per inventory query,
+and reads the next page only when the snapshot needs it. Each page is one
+indexed seek on the table's row key, so a page costs the same at any table size:
+
+| Resource                 | Bound                                      |
+| ------------------------ | ------------------------------------------ |
+| Rows per inventory query | 101 (one page plus the continuation key)   |
+| Rows scanned per query   | the rows returned; no table scan or sort   |
+| Inventory queries        | `2 + ceil(plans / 100) + ceil(runs / 100)` |
+| Run summaries decoded    | 0                                          |
+| Follow-mode state        | one pinned sequence per partition seen     |
+
+The first page of each inventory pins its newest entry, and the walk stops
+there. A finite watch (`follow: false`) therefore ends even while runs keep
+arriving. A partition that exists when the watch starts is read exactly once;
+one created during the walk is not listed. A followed watch still delivers its
+entries, because the first tail entry for a partition that nothing has pinned
+pins it and reads its history first.
+
+An unscoped watch is the right shape for a dashboard. For a run you can name,
+scope it: the scoped watch is one partition read and it is the only form that
+can resume.
+
+## Read the derived deltas
+
+Two kinds are computed rather than recorded, and arrive beside the entry they
+were derived from:
+
+| Kind                      | Payload                                                     | Means                                        |
+| ------------------------- | ----------------------------------------------------------- | -------------------------------------------- |
+| `control.run.lineage`     | `{ runId, parentRunId, lineageId?, roundOrdinal?, origin }` | A run was spawned, forked, or handed off to. |
+| `control.steer.delivered` | `{ runId, messageId, boundary }`                            | A turn boundary took your steer.             |
+
+Both projections are exported, so a client reading the journal directly reaches
+the same conclusions the server does:
+
+```ts
+import * as Lineage from "@smthrs/control/Lineage"
+import * as Steering from "@smthrs/control/Steering"
+
+const expanded = [...Lineage.expand(event), ...Steering.derive(event)]
+```
+
+`expand` returns the entry plus any delta it discloses; `derive` returns the
+delta alone. See [Journal projections](../concepts/projections.md) for why
+delivery is derived rather than written.
+
+## Failures
+
+Every member of `ControlError` can reach a watch stream. In practice you will
+meet three:
+
+- `InvalidInput` for an unscoped or malformed cursor, or both cursor forms together.
+- `PersistenceError` with operation `watch` when a journal read fails. Its
+  `cause` is the journal's own error.
+- `Unavailable` with feature `watch` when the composition has no open journal.
+
+A watch of a run that does not exist is not an error. The partition is empty.
+
+## Where to go next
+
+- [Journal projections](../concepts/projections.md): partitions, the handoff,
+  and the full list of kinds the plane writes.
+- [Find runs and page through them](./list-runs.md): the point-in-time view.
+- [`smthrs runs logs`](/cli/runs): the operator surface over this verb.

package/docs/installation.md

@@ -0,0 +1,106 @@
+---
+title: "Installation"
+description: "Install @smthrs/control, its runtime requirements, its import forms, and the collaborator packages an in-memory, durable, or remote composition adds."
+sidebar:
+  order: 1
+---
+
+## Install the package
+
+Not on npm yet; see [Installation](https://github.com/smithersai/smithers/blob/main/packages/smithers/flows/flow/docs/installation.md#use-the-libraries).
+
+The package requires Node.js 26.4.0 or later and ships as both ESM and
+CommonJS with TypeScript declarations. Its runtime dependencies install with
+it: [`effect`](https://effect.website) and the `@smthrs/*` packages the plane
+composes.
+
+The package imports no `node:*` module. Identity comes from
+`globalThis.crypto`, encryption comes from Web Crypto, and persistence speaks
+the driver-neutral SQL contract, so the same modules run in Node.js and in a
+browser that supplies a SQL driver. That is a statement about the imports, not
+a tested guarantee: nothing here is exercised in a browser, so verify your own
+bundle before you depend on it.
+
+## Import forms
+
+The root entry point re-exports every module as a namespace:
+
+```ts
+import { Control, ControlLive, Monitor, SqlControlRuntime } from "@smthrs/control"
+```
+
+Each module is also importable from its own subpath, which is the form the
+[API reference](./api.md) uses:
+
+```ts
+import * as ControlExecutor from "@smthrs/control/ControlExecutor"
+import * as ControlSchema from "@smthrs/control/ControlSchema"
+```
+
+The deterministic test stack has its own subpath:
+
+```ts
+import * as TestControl from "@smthrs/control/test/TestControl"
+```
+
+Two subpath families are not public and are blocked in the export map:
+`@smthrs/control/internal/*` and `@smthrs/control/migrations/*`, along with
+every nested `*/index`. `@smthrs/control/package.json` is exported.
+
+## What a composition adds
+
+`ControlLive.layer` requires four collaborators, and a host provides all four:
+
+| Requirement         | Package                                       | What it does here                                                                  |
+| ------------------- | --------------------------------------------- | ---------------------------------------------------------------------------------- |
+| `ControlRuntime`    | this package                                  | Stores plans, tokens, grants, idempotency records, and run rows.                   |
+| `Journal`           | [`@smthrs/journal`](/api/journal)             | Records every decision beside the state change it caused, and backs `watch`.       |
+| `NotificationQueue` | [`@smthrs/notifications`](/api/notifications) | Carries a steer to the turn boundary that delivers it, and counts what is pending. |
+| `Registry`          | [`@smthrs/registry`](/api/registry)           | Answers `list({ _tag: "flows" })` with the flows this host discovered.             |
+
+`ControlExecutor` is optional. A composition that provides none observes and
+records but starts nothing, which is the correct shape for a monitor or a
+read-only dashboard.
+
+### An in-memory composition
+
+`TestControl.layer` bundles all four collaborators with the deterministic
+runtime. Its journal uses a real in-memory SQLite database, so the
+[Quickstart](./quickstart.md) adds the optional Node driver:
+
+```bash
+pnpm add effect@4.0.0-rc.115 @effect/sql-sqlite-node@4.0.0-rc.115
+```
+
+### A durable composition
+
+- [`@smthrs/database`](/api/database) supplies the SQL client and the
+  `DurableWriter` every control write serializes through.
+- [`@smthrs/run-store`](/api/run-store) supplies the fenced run store
+  `SqlControlRuntime` maps the control lifecycle onto.
+
+See [Store control state in a database](./guides/durable-storage.md) for the
+layer stack and the migration order.
+
+### A remote composition
+
+The RPC boundary uses Effect's own HTTP, WebSocket, and RPC modules, which ship
+inside `effect`. A Node host adds the platform bindings and a serialization
+format:
+
+```bash
+pnpm add @effect/platform-node@4.0.0-rc.115 @effect/platform-node-shared@4.0.0-rc.115
+```
+
+See [Serve the control plane over RPC](./guides/serve-over-rpc.md).
+
+### Credential storage
+
+The credential boundary needs a store and a cipher. `CredentialStore.layerMemory`
+and `WebCryptoCipher.layer` need nothing beyond this package;
+`SqlCredentialStore.layer` needs the same database packages a durable
+composition adds. See [Store and resolve a credential](./guides/store-credentials.md).
+
+## Next step
+
+Run one plan through approval and launch in the [Quickstart](./quickstart.md).

package/docs/quickstart.md

@@ -0,0 +1,163 @@
+---
+title: "Quickstart"
+description: "Plan a flow, watch the launch park for approval, approve it, launch it, list the run, and replay the journal, in one in-memory program with no database and no engine."
+sidebar:
+  order: 2
+---
+
+This quickstart drives one plan through the whole gate: a plan card, a refused
+launch, an approval, an accepted launch, a listing, and a replay of the run's
+journal. The `Control` service is the production one. Only its collaborators
+are in memory, so the program is deterministic, needs no database, and starts
+no real work.
+
+By the end you will have seen the two answers that make the control plane
+usable from a script: a `Receipt` that says what happened, and a `ControlEvent`
+stream that says it again from durable evidence.
+
+## Prerequisites
+
+- Node.js 26.4.0 or later.
+- A package with the dependency installed. Not on npm yet; see [Installation](https://github.com/smithersai/smithers/blob/main/packages/smithers/flows/flow/docs/installation.md#use-the-libraries).
+
+## Declare the flow the plane may plan
+
+A control plane plans what its runtime knows about. `MemoryFlow` is that
+entry: an id, a description, whether the flow is deploy class, and the
+capability envelope an approval binds to.
+
+Create `quickstart.ts`:
+
+```ts
+import type * as ControlRuntime from "@smthrs/control/ControlRuntime"
+
+/** One flow this plane may be asked to plan. */
+const Deploy: ControlRuntime.MemoryFlow = {
+  flowId: "quickstart/Deploy",
+  description: "Deploys one build",
+  deployClass: true,
+  envelope: {
+    capabilities: ["process:spawn"],
+    flows: [],
+    budget: { milliseconds: 60_000 }
+  }
+}
+```
+
+The envelope is the authority a reviewer is being asked to grant. It is part of
+the plan digest, so an approval taken on this envelope cannot authorize a
+wider one later.
+
+## Plan, launch, approve, launch again
+
+`plan` returns a `PlanCard`: the flow, a canonical summary of the input, the
+envelope, the keyed node graph, and a digest over all of it. The card starts
+undecided, so the first `run` parks:
+
+```ts
+import { Control } from "@smthrs/control/Control"
+import * as Effect from "effect/Effect"
+import * as Stream from "effect/Stream"
+
+const program = Effect.gen(function*() {
+  const control = yield* Control
+
+  const card = yield* control.plan({
+    flowId: "quickstart/Deploy",
+    input: { build: "v1.4.0" }
+  })
+
+  /** The exact plan, digest, and envelope every launch attempt resubmits. */
+  const launch = {
+    _tag: "Plan" as const,
+    planId: card.planId,
+    digest: card.digest,
+    envelope: card.envelope
+  }
+
+  // Nothing is approved yet, so this starts nothing and says why.
+  const parked = yield* control.run({ ...launch, idempotencyKey: "deploy:v1.4.0" })
+
+  // The card carries the exact payload an approval is taken on, including a
+  // default idempotency key, so a reviewer resubmits it unchanged.
+  yield* control.approve(card.approval)
+
+  // The same call, the same key. Now it launches.
+  const accepted = yield* control.run({ ...launch, idempotencyKey: "deploy:v1.4.0" })
+
+  // And once more, to show what a retry is worth.
+  const replayed = yield* control.run({ ...launch, idempotencyKey: "deploy:v1.4.0" })
+
+  const listed = yield* control.list({ _tag: "runs", filters: {} })
+  const runs = listed._tag === "runs" ? listed.items : []
+
+  const events = yield* control.watch({ runId: runs[0]!.runId, follow: false }).pipe(
+    Stream.map((event) => event.kind),
+    Stream.runCollect
+  )
+
+  return { parked, accepted, replayed, runs, events: [...events] }
+})
+```
+
+## Provide the in-memory stack and run it
+
+`TestControl.layer` bundles the four collaborators `ControlLive` requires: the
+deterministic runtime, an in-memory journal, a notification queue over that
+journal, and an empty registry. Its executor accepts nothing, which is exactly
+the composition a host that only records has.
+
+```ts
+import * as TestControl from "@smthrs/control/test/TestControl"
+
+console.log(
+  await Effect.runPromise(
+    program.pipe(Effect.provide(TestControl.layer({ flows: [Deploy], now: () => 0 })))
+  )
+)
+```
+
+Run the file with your TypeScript runner. The receipts, the listing, and the
+journal replay come back like this:
+
+```text
+{
+  parked: { _tag: 'Parked', receiptId: 'deploy:v1.4.0', planId: 'plan-1', status: 'waiting-approval' },
+  accepted: { _tag: 'Accepted', receiptId: 'deploy:v1.4.0', runId: 'run-1' },
+  replayed: { _tag: 'AlreadyApplied', receiptId: 'deploy:v1.4.0', runId: 'run-1' },
+  runs: [ { runId: 'run-1', flowId: 'quickstart/Deploy', status: 'accepted', ... } ],
+  events: [ 'control.run.accepted', 'control.run.pending' ]
+}
+```
+
+## What just happened
+
+Four things worth naming, because each is a promise the plane keeps everywhere:
+
+- **A launch is not a start.** An undecided plan answers `Parked` with the
+  status it is waiting in. Nothing was created, so nothing has to be cleaned
+  up. See [Gate work behind an approval](./guides/approvals.md).
+- **The same key means the same mutation, once.** The launch and the retry
+  carry one `idempotencyKey`, and the retry answers `AlreadyApplied` with the
+  run the first call created. A parked receipt is deliberately not recorded, so
+  the key was still free when the plan became approvable. See
+  [Receipts and idempotency](./concepts/receipts.md).
+- **An approval is bound to what was reviewed.** `card.approval` carries the
+  target, the digest, the envelope, and a default key. Submit a different
+  digest or a different envelope and the decision is refused rather than
+  re-aimed.
+- **Every decision left evidence.** `watch` replayed the run's journal from
+  durable rows. `control.run.pending` is there because this composition's
+  executor declined the launch, so the plane released the run rather than
+  claiming to drive it. See [Journal projections](./concepts/projections.md).
+
+## Next steps
+
+- [Connect an execution engine](./guides/implement-an-executor.md): make
+  `control.run` start something real.
+- [Store control state in a database](./guides/durable-storage.md): keep the
+  plans, tokens, and runs across a restart.
+- [Serve the control plane over RPC](./guides/serve-over-rpc.md): hand this
+  same program a client instead of a layer.
+- [Authority, not execution](./concepts/authority.md): the model the rest of
+  the package is built on.