okengine 0.18.5 → 0.19.0

package/site/content/docs/elements/signal.mdx

@@ -1,380 +0,0 @@
----
-title: "Signal"
-description: "Data in motion — queues, pub/sub, and streams as one declaration whose delivery physics you choose explicitly."
-icon: "Radio"
-source: "docs/spec/unified-theory.md"
----
-
-Signal is how your app moves data between flows and to clients: background jobs, events, fan-out notifications, live feeds. What other stacks split into a queue library, a pub/sub client, and a streaming platform is **one declaration** here — you pick the delivery physics, and the driver underneath is swappable per environment.
-
-<Callout title="The one rule">
-  `delivery` is **mandatory with no default**. Delivery physics is a semantic decision about your
-  data — guessing it produces silent, expensive bugs, so omitting it is a type error.
-</Callout>
-
-## Quick start
-
-<Steps>
-
-<Step>
-### Declare the signal
-
-A signal is a named, typed channel. The schema describes the payload; `delivery` describes its physics:
-
-```typescript title="src/signals.ts"
-import { signal } from "okengine";
-import { z } from "zod";
-
-export const orderPlaced = signal("order-placed", {
-  schema: z.object({ orderId: z.string(), total: z.number() }),
-  delivery: "once", // queue physics: one consumer, retries, DLQ
-  retries: 3,
-  deadLetter: true,
-});
-```
-
-</Step>
-
-<Step>
-### Emit it from a Flow
-
-Producers never touch a broker client — they call `fx.emit`:
-
-```typescript title="src/flows/orders/place.ts"
-do: async (input, fx) => {
-  await fx.store(db).insert(orders).values({ id: input.id /* … */ });
-  await fx.emit(orderPlaced, { orderId: input.id, total: input.total }, { key: input.id });
-  // optional `key` keeps same-order events serialized under `once`
-};
-```
-
-</Step>
-
-<Step>
-### Consume it with a Flow
-
-A signal is a trigger like any other — the consumer is an ordinary Flow:
-
-```typescript title="src/flows/orders/send-confirmation.ts"
-export const sendConfirmation = on(
-  orderPlaced,
-  flow("orders.sendConfirmation", {
-    in: z.object({ orderId: z.string(), total: z.number() }),
-    do: async (input, fx) => {
-      await fx.send(orderConfirmed, { to: "user-1", data: input });
-    },
-  }),
-);
-```
-
-</Step>
-
-</Steps>
-
-That's the loop for `once` and `broadcast`: declare → `fx.emit` → `on(signal, flow)`.
-`live` keeps the same emit; expose it over HTTP with `.live(signal)` and subscribe from `api.live`.
-
-## The three delivery physics
-
-<SignalDelivery />
-
-The declaration is identical in shape for all three — switching physics later is a one-word change, not a migration to another library.
-
-### Options
-
-| Option        | Type                              | Default          | Meaning                                                                                           |
-| ------------- | --------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------- |
-| `delivery`    | `"once" \| "broadcast" \| "live"` | — (**required**) | Delivery physics                                                                                  |
-| `description` | string                            | —                | Human title in the Console (falls back to the signal name)                                        |
-| `schema`      | zod / Standard Schema             | —                | Payload contract — validated at emit (**OKE1043** on failure); Manifest docs                      |
-| `retries`     | number                            | `3`              | Retry budget for `once` — dead-letter when `attempts > retries` (`retries + 1` total invocations) |
-| `deadLetter`  | boolean                           | `true`           | Preserve exhausted messages in the DLQ (`once`)                                                   |
-| `optional`    | boolean                           | `false`          | Allow emitting while nobody subscribes (skip the orphan check)                                    |
-| `retention`   | `{ maxAge?, maxCount? }`          | unbounded        | Live tape cap (`delivery: "live"` only). Both limits AND-combine when set                         |
-
-## One order, three modes
-
-One placed order can need all three physics at once — pick a signal per job, not one signal stretched across modes:
-
-| Signal          | Mode        | Job                                                         |
-| --------------- | ----------- | ----------------------------------------------------------- |
-| `order-placed`  | `once`      | One job — competing workers, retries, DLQ                   |
-| `order-changed` | `broadcast` | Fan-out — cache invalidation **and** customer notification  |
-| `order-status`  | `live`      | Status feed — retained history a late subscriber can replay |
-
-```typescript title="src/signals.ts"
-import { signal } from "okengine";
-import { z } from "zod";
-
-export const orderPlaced = signal("order-placed", {
-  schema: z.object({ orderId: z.string(), total: z.number() }),
-  delivery: "once",
-  retries: 3,
-  deadLetter: true,
-});
-
-export const orderChanged = signal("order-changed", {
-  schema: z.object({ orderId: z.string(), kind: z.literal("placed") }),
-  delivery: "broadcast",
-});
-
-export const orderStatus = signal("order-status", {
-  schema: z.object({
-    orderId: z.string(),
-    status: z.enum(["placed", "fulfilling", "shipped"]),
-  }),
-  delivery: "live",
-  optional: true, // clients may not be connected yet
-  retention: { maxAge: "24h", maxCount: 500 },
-});
-```
-
-A Flow **writes** all three tapes. `once` / `broadcast` **listen** with `on(signal, flow)`. `live` does not.
-
-```typescript title="src/flows/orders/place.ts"
-export const placeOrder = on(
-  http.post("/orders").public(),
-  flow("orders.place", {
-    in: z.object({ id: z.string(), total: z.number() }),
-    do: async (input, fx) => {
-      await fx.emit(orderPlaced, { orderId: input.id, total: input.total });
-      await fx.emit(orderChanged, { orderId: input.id, kind: "placed" });
-      await fx.emit(orderStatus, { orderId: input.id, status: "placed" });
-      await fx.emit(orderStatus, { orderId: input.id, status: "fulfilling" });
-      await fx.emit(orderStatus, { orderId: input.id, status: "shipped" });
-    },
-  }),
-);
-```
-
-Stacked status emits are so replay is visible in one request. A real app emits as the order moves.
-
-`once` — the listener is `on(orderPlaced, flow)`. Two workers compete; **exactly one** claims each message (at-least-once; make the handler idempotent):
-
-```typescript title="src/flows/orders/send-confirmation.ts"
-export const sendConfirmation = on(
-  orderPlaced,
-  flow("orders.sendConfirmation", {
-    do: async (input, fx) => {
-      await fx.send(orderConfirmed, { to: "user-1", data: input });
-    },
-  }),
-);
-```
-
-`broadcast` — every `on(orderChanged, flow)` gets its own copy. Neither Flow knows the other exists:
-
-```typescript title="src/flows/orders/side-effects.ts"
-export const invalidateCache = on(
-  orderChanged,
-  flow("cache.onOrderChanged", {
-    do: async ({ orderId }, fx) => {
-      /* invalidate order:{orderId} */
-    },
-  }),
-);
-
-export const notifyCustomer = on(
-  orderChanged,
-  flow("notify.onOrderChanged", {
-    do: async ({ orderId }, fx) => {
-      await fx.send(orderConfirmed, { to: "user-1", data: { orderId } });
-    },
-  }),
-);
-```
-
-`live` — retain the tape. A late subscriber replays history (`placed → fulfilling → shipped`), then keeps going. Omit `retention` and that history is **unbounded** — declare `maxAge` / `maxCount` in production.
-
-The listener is **not** `on(orderStatus, flow)` — that trigger does not replay retained payloads. Expose GET SSE with `.live(signal)`, then subscribe from the typed client. `optional: true` is required so emits succeed before anyone is connected.
-
-```typescript
-export const events = on(http.get("/orders/:orderId/events").gate(member).live(orderStatus));
-
-export const adminFeed = on(http.get("/admin/order-status").gate(admin).live(orderStatus));
-
-const stop = api.live(
-  orderStatus,
-  { orderId: "ord_1" },
-  {
-    onEvent: (event) => {
-      /* placed → fulfilling → shipped */
-    },
-  },
-);
-stop();
-```
-
-Same signal, two audiences: member filter on `:orderId`, admin firehose. Duplicate `(signal, gates, match)` is **OKE1013**. Same HTTP path is still **OKE1011**. Firehose default: `on(http.live(orderStatus).gate(member))` → `GET /_oke/live/order-status`.
-
-Server-side `fx.live` **is** the SSE carrier (`JsonStreamResult`). Custom `do` is allowed; authors do not wrap `fx.json.stream`. Reconnects send `Last-Event-ID`; a missing cursor is **OKE1014** / HTTP 410, not a silent full replay. `autoResubscribe: true` backs off 500ms…30s and, after a 410, replays the remaining tape.
-
-**Why separate signals:** delivery is fixed per declaration. Competing work stays on `once`; fan-out stays on `broadcast`; the timeline stays on `live`. Switching a word later is cheap; mixing physics on one name is not.
-
-## When delivery fails
-
-`once` signals retry automatically. Every attempt keeps a **typed failure reason**, and the full attempt history survives into the DLQ — so when a message lands there you see _why_ each attempt failed, not just that it did.
-
-Query that queue from a Flow with `fx.deadLetters(signal)` — same handle as `fx.emit`. The compiler stamps `reads: ["signal:<name>"]`; another signal throws **OKE1001**.
-
-The **Console** (`:6533` → `/overview`) shows emit and consume on the graph, and delivery attempts in Traces — no separate broker UI to run.
-
-```typescript
-on(
-  http.get("/notifications/failed"),
-  flow("notifications.failed", {
-    do: async (input, fx) => fx.json.withQuery(await fx.deadLetters(orderPlaced), input),
-  }),
-);
-```
-
-<Callout title="Orphan emits and schema fail loudly">
-  Zero subscribers throws **OKE1042** unless `optional: true`. A payload that fails the signal's
-  Standard Schema throws **OKE1043** before the message is staged — same `validate()` path as Flow
-  `in`.
-</Callout>
-
-## Per-key ordering (`once`)
-
-`fx.emit(signal, payload, { key })` is optional and additive. When `key` is set, no two messages
-sharing the same `(signal, key)` are claimed concurrently — a per-key serialization lock, not a
-partitioned log.
-
-The lock **is** the in-flight message's visibility lease: while that row is `inflight` with an
-unexpired `leaseExpiresAt`, same-key siblings stay unclaimable. Crash → lease expiry → reclaim
-(same clock as unkeyed `once`). Omit `key` for pure competing-consumer behavior.
-
-```typescript
-await fx.emit(orderPlaced, { orderId, total }, { key: orderId });
-await fx.emit(orderPlaced, { orderId, total: total + fee }, { key: orderId });
-// Same key → never concurrent; complete in emission order across competing workers.
-```
-
-## Durability and leases (`once`)
-
-<SignalOnceLease />
-
-Committed `once` messages survive process death. A claim sets `lockedBy` and a visibility lease (`leaseExpiresAt`, default **30s**). There is no background sweeper: the next consumer reclaim query takes `pending` rows **or** `inflight` rows whose lease has expired. Per-key locks reuse this same lease — no second timeout.
-
-<Callout title="At-least-once, not exactly-once">
-  Delivery is **at-least-once** — finish side effects then die before ack, and the message may run
-  again; make consumers idempotent. Handlers must finish within `leaseMs` (default 30s): a healthy
-  but slower handler can be reclaimed while still running. Lease heartbeat is not offered yet.
-</Callout>
-
-## Live replay
-
-<SignalLiveReplay />
-
-`live` retains delivered messages and replays them to a late subscriber. Omit `retention` for an unbounded tape (a production hazard). `maxAge` (clock duration) and `maxCount` AND-combine when both are set. Drivers prune on write and when `live()` opens — no background sweeper.
-
-Reconnects send SSE `Last-Event-ID`. If that id was pruned or never existed, the server returns **410** `{ error: { code: "LiveResumeGap" } }` (**OKE1014**) before opening the stream. The client clears the cursor; `autoResubscribe` then replays whatever remains.
-
-The Console payload monitor still shows only the newest 50 for display — that cap is not the retained tape.
-
-<Callout title="HTTP live is GET SSE">
-  Expose with `http.get(path).live(signal)` or `http.live(signal)`. Subscribe with `api.live(signal,
-  input?, {onEvent})` — callback + unsubscribe, not `for await`. See
-  [Client](/docs/reference/client#live-signals).
-</Callout>
-
-## Orphaned signal config
-
-When a signal disappears from code, reconciliation marks its config row `orphaned` — it is never deleted. Pending and DLQ messages for that name stay in storage. Drain and replay require an active declaration again: restore the `signal(…)` (and a consumer), then replay from the Console or bus.
-
-## Per-environment drivers
-
-```typescript title="oke.config.ts"
-drivers: {
-  signal: { dev: "redis", test: "memory", prod: "redis" },
-},
-```
-
-| Driver     | Runs as                              | Boot today                                                                  |
-| ---------- | ------------------------------------ | --------------------------------------------------------------------------- |
-| `memory`   | in-process                           | Tests — zero infrastructure                                                 |
-| `redis`    | Redis / Valkey / Dragonfly image pin | Boot-bound — **emit relay** via Streams / pub/sub; consume is process-local |
-| `postgres` | Postgres                             | Driver exists; boot **fails loud** until a LISTEN/NOTIFY SQL client binds   |
-| `nats`     | NATS                                 | Driver exists; boot **fails loud** until a production NATS client binds     |
-
-**Known limit:** `redis` relays emits to Redis, but `once` / `broadcast` / `live` **consume, `live` replay, and `drain` still use a process-local outbox** today. Competing consumers across replicas do not share that outbox until Redis Streams consume ships. Boot prints a one-shot warning when `drivers.signal` is `redis`. Prefer a single consumer instance (or a shared durable outbox path in tests) for multi-process `once` until then.
-
-Pairing an arbitrary Store insert with emit inside **one** shared SQL transaction is not what `fx.emit` does — keep consumers idempotent (at-least-once).
-
-## Troubleshooting
-
-<Accordions>
-<Accordion title="Emit fails with OKE1042 (no subscriber)">
-
-You emitted a signal that nobody currently subscribes to. For `once` and `broadcast`, wire
-`on(signal, flow)` before emit.
-
-`live` listeners are HTTP SSE (`api.live` / `fx.live`), not a Flow — set `optional: true` when they may connect later.
-
-</Accordion>
-<Accordion title="Emit fails with OKE1043 (schema)">
-
-The payload failed the signal's Standard Schema at emit time — nothing was staged. Fix the payload shape, or remove `schema` if the contract was intentional.
-
-</Accordion>
-<Accordion title="A message keeps retrying and then disappears">
-
-When `attempts > retries` the message moves to the DLQ (if `deadLetter: true`) — it is not lost. List it from a Flow with `fx.deadLetters(signal)`, or open Console → Flows, inspect the typed failure, fix the consumer, then replay.
-
-</Accordion>
-<Accordion title="fx.deadLetters throws OKE1001">
-
-The flow read a signal it did not declare. The compiler infers `reads: ["signal:<name>"]` from `fx.deadLetters(signal)` and `fx.live(signal)`. A different handle needs its own read.
-
-</Accordion>
-<Accordion title="Boot fails with OKE1013 (live exposure duplicate)">
-
-Two GET routes expose the same live signal with the same gates and match.
-Change the gate or path-param filter, or drop the extra route (same path is **OKE1011**).
-
-</Accordion>
-<Accordion title="Subscribe gets HTTP 410 LiveResumeGap (OKE1014)">
-
-The `Last-Event-ID` cursor is not on the retained tape (pruned, never existed, or the memory bus restarted). The gap is visible in `onError`. With `autoResubscribe: true` the next request omits the header and replays what remains.
-
-</Accordion>
-<Accordion title="on(liveSignal, flow) never replays history">
-
-`on(signal, flow)` listens for `once` and `broadcast`. `live` replay is HTTP SSE (`api.live` /
-`fx.live`) — a Flow trigger does not replay retained payloads.
-
-</Accordion>
-<Accordion title="once vs broadcast vs live — how do I choose?">
-
-Ask: _how many consumers should process each message?_ One → `once`. All of them → `broadcast`. Clients over time, with full retained replay → `live`. Retries and a DLQ are `once` only.
-
-</Accordion>
-<Accordion title="My emit happened but the row didn't (or vice versa)">
-
-`fx.emit` commits the signal outbox when it resolves; it does not wrap your Store insert. Treat delivery as at-least-once and make the consumer idempotent (or check the row before acting). Remember: `redis` today is an emit relay with a process-local consume outbox — not multi-instance competing consumers yet.
-
-</Accordion>
-<Accordion title="The same once message ran twice">
-
-At-least-once: crash-after-claim reclaims when the lease expires, and a handler slower than `leaseMs` can be reclaimed while still running. Idempotent consumers are required; there is no lease heartbeat yet.
-
-</Accordion>
-</Accordions>
-
-## Learn more
-
-- [Project structure](/docs/get-started/project-structure) — signal files get a name, never a URL; `http.get("/…")` still wins
-- [Flow](/docs/elements/flow) — `on(trigger, flow)` and `fx.emit`
-- [fx](/docs/reference/fx) — `fx.emit` and `fx.deadLetters`
-- [Clock](/docs/elements/clock) — scheduled and delayed work
-
-## Next
-
-<Cards>
-  <Card title="Store" description="Continue to Store." href="/docs/elements/store" />
-  <Card
-    title="Introduction"
-    description="Eight elements overview."
-    href="/docs/get-started/introduction"
-  />
-</Cards>