okengine 0.7.0 → 0.9.0

package/site/content/docs/deployment/kubernetes.mdx

@@ -0,0 +1,176 @@
+---
+title: Kubernetes
+description: Run an OKE app as a plain Deployment — PgDog, readiness vs liveness probes, graceful SIGTERM, shared drivers, and honest multi-instance limits.
+icon: Container
+source: README.md
+---
+
+OKE apps are safe to run as more than one replica when shared drivers back
+Clock, Journal, Gate, and Store. Kubernetes is full orchestration with the
+largest ecosystem — use a **Deployment**, never a StatefulSet.
+
+<Callout title="The one rule">
+  Treat every replica as replaceable. Put durable state in Postgres / Redis / S3, probe `GET
+  /_/ready` for readiness and app `GET /health` for liveness, and set
+  `terminationGracePeriodSeconds` high enough for lease release on SIGTERM.
+</Callout>
+
+## Quick start
+
+<Steps>
+
+<Step>
+### Deploy as a Deployment
+
+Use a plain Kubernetes **Deployment** (or compatible replica set). Do **not**
+use a StatefulSet — there is no per-pod volume identity, ordered startup, or
+sticky network id to preserve. Scale with `replicas: N`.
+
+</Step>
+
+<Step>
+### Share backing services
+
+Pin docker/prod drivers to shared stores:
+
+| Concern                       | Driver                       | Shared backend                                    |
+| ----------------------------- | ---------------------------- | ------------------------------------------------- |
+| SQL / durable journal / clock | `postgres`                   | Postgres (optionally via PgDog)                   |
+| Gate rate counters            | `redis` (`drivers.store.kv`) | Redis / Valkey / Dragonfly (`images["store.kv"]`) |
+| Files                         | `s3`                         | S3-compatible object store                        |
+| Signal                        | see known limits below       | —                                                 |
+
+</Step>
+
+<Step>
+### Probe readiness and liveness
+
+- **Readiness:** `GET /_/ready` — returns `200 { ready: true }` only after boot and the durable orphan scan finish. While booting or scanning orphans it returns `503 { ready: false, reason: "booting" | "orphan_scan" }`.
+- **Liveness:** app-authored `GET /health` (create-oke ships one) — cheap “process is alive,” not orphan-scan aware.
+
+Wire `readinessProbe` to `/_/ready` and `livenessProbe` to `/health`.
+
+</Step>
+
+</Steps>
+
+## PgDog pooling
+
+Bun.SQL defaults to **10** connections per process — fine for one pod.
+Scale the Deployment and `N × pool` can exceed Postgres `max_connections`.
+
+When both `store.sql` and `pgdog` are pinned (docker/prod default in templates),
+`DATABASE_URL` points at PgDog on port `6432` — transaction pooling, wire-protocol
+transparent to Bun.SQL / Drizzle. Point the cluster Service at that pooler.
+
+**Why PgDog.** Transaction pooling fixes the connection math. Naive poolers can
+leak session state (`SET`, RLS vars, `LISTEN`/`NOTIFY`) across clients; PgDog
+re-applies those under transaction mode.
+
+Read-replica routing (`BEGIN READ ONLY` → replica) is documented readiness —
+not configured as a generated Kubernetes manifest this round.
+
+## Redis-protocol images (Redis · Valkey · Dragonfly)
+
+`images["store.kv"]` defaults to Redis (most mature). Pin Valkey or Dragonfly
+the same way — driver id stays `redis`, same `REDIS_URL`. Image table + one-line
+licenses: [Store · KV](/docs/elements/store#kv).
+
+## Readiness vs liveness
+
+| Endpoint       | Role                                                   | Kubernetes                                       |
+| -------------- | ------------------------------------------------------ | ------------------------------------------------ |
+| `GET /_/ready` | Kernel readiness (`booting` → `orphan_scan` → `ready`) | `readinessProbe` — no traffic until Ready        |
+| `GET /health`  | App liveness (create-oke) — cheap “process alive”      | `livenessProbe` — restart if the process is dead |
+
+**Consequence:** orphan scan delays Ready (and traffic) on purpose so a new
+pod is not hit mid-takeover. Do not point liveness at `/_/ready` — a long orphan
+resume would kill the pod instead of waiting.
+
+## Graceful shutdown
+
+On `SIGTERM` / `SIGINT`, `installGracefulShutdown` (wired by `oke dev`’s app
+runner; call it next to `createBunRuntime().serve` in custom entries):
+
+1. Stops accepting new connections
+2. Releases **Clock** cron leases and **Journal** run leases held by this instance
+3. Closes element runtimes and exits
+
+Signal message leases have no proactive release today — survivors reclaim after
+the visibility TTL (default 30s). Set `terminationGracePeriodSeconds` ≥ your
+longest lease TTL (typically ≥ 30s) so release finishes before kubelet SIGKILL.
+
+## Ingress (TLS / routing)
+
+Use cluster-native **Ingress** or Gateway API for HTTPS.
+Compose `images.proxy` recipes (Caddy / Traefik) target plain Docker and Swarm —
+see [Reverse proxy](/docs/deployment/reverse-proxy) when you are not on Kubernetes.
+
+Still set serve `allowedHosts` to the public hostname
+([Security](/docs/reference/security)).
+
+## Known multi-instance limits
+
+Be honest about what is still process-local:
+
+| Surface                                  | Multi-instance today                                                                                       |
+| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
+| Clock (`postgres` / shared `file`)       | Shared leader election                                                                                     |
+| Journal (`postgres` / shared `file`)     | Shared durable runs + orphan resume                                                                        |
+| Gate rates (`drivers.store.kv: redis`)   | Shared counters                                                                                            |
+| Store SQL / KV redis / files `s3`        | Shared                                                                                                     |
+| Store files `fs`                         | **Per-instance filesystem** — boot warns; use `s3` when scaled                                             |
+| Signal `redis`                           | Emit relays to Redis; **consume / `live` / `drain` are process-local outbox** until Streams consume ships  |
+| Channel suppression / consent / receipts | **Default stores are process-local** — opt-out on replica A is invisible to B until a durable driver ships |
+
+## Troubleshooting
+
+<Accordions>
+<Accordion title="Pods pass liveness but get no traffic after deploy">
+
+Check readiness: `/_/ready` stays `503` with `reason: "orphan_scan"` until the
+durable orphan resume finishes. A long resume delays Ready; that is intentional
+so traffic does not hit a pod mid-takeover.
+
+</Accordion>
+<Accordion title="Rate limits look twice as high after scaling to 2">
+
+`drivers.store.kv` must be `redis` with `REDIS_URL` set. A missing URL now
+**fails boot** (no silent memory fallback). Declaring `memory` for local is fine;
+docker/prod should stay on redis.
+
+</Accordion>
+<Accordion title="Files uploaded on pod A are missing on pod B">
+
+`drivers.store.files: "fs"` is single-host. Switch docker/prod to `s3`
+(create-oke templates already do).
+
+</Accordion>
+<Accordion title="Pod killed during orphan scan">
+
+Liveness is pointed at `/_/ready`. Move liveness to `/health`; keep readiness
+on `/_/ready` so the pod stays Alive while Waiting for Ready.
+
+</Accordion>
+</Accordions>
+
+## Learn more
+
+- [Deployment](/docs/deployment) — choose compose, Swarm, or Kubernetes
+- [Docker](/docs/deployment/docker) — plain compose
+- [Docker Swarm](/docs/deployment/docker-swarm) — `docker stack deploy`
+- [Reverse proxy](/docs/deployment/reverse-proxy) — Caddy / Traefik (non-K8s)
+- [Store](/docs/elements/store) — PgDog and facets
+- [Clock](/docs/elements/clock) — shared CronStore / leader election
+- [Flow](/docs/elements/flow) — durable journal
+- [Signal](/docs/elements/signal) — delivery modes and redis honesty
+- [Channel](/docs/elements/channel) — suppression process-local limit
+- [CLI](/docs/reference/cli) — `oke docker --prod`, `oke start`
+
+## Next
+
+<Cards>
+  <Card title="Deployment" href="/docs/deployment" />
+  <Card title="Docker" href="/docs/deployment/docker" />
+  <Card title="Docker Swarm" href="/docs/deployment/docker-swarm" />
+</Cards>

package/site/content/docs/deployment/meta.json

@@ -0,0 +1,5 @@
+{
+  "title": "Deployment",
+  "icon": "Server",
+  "pages": ["index", "docker", "docker-swarm", "kubernetes", "reverse-proxy"]
+}

package/site/content/docs/deployment/reverse-proxy.mdx

@@ -0,0 +1,216 @@
+---
+title: Reverse proxy
+description: Opt-in Caddy or Traefik for TLS termination and routing on plain Docker and Swarm — custom domains, Let's Encrypt, and the socket-proxy security model.
+icon: Globe
+source: README.md
+---
+
+Pin `images.proxy` when you need HTTPS at the edge on plain `docker compose` or
+Docker Swarm. Kubernetes uses cluster Ingress / Gateway API instead — this page
+is for the compose recipes.
+
+<Callout title="The one rule">
+  Leave the no-proxy default until you need TLS or horizontal scale on compose. Then pick **Caddy**
+  (simplest automatic HTTPS) or **Traefik** (label auto-discovery for `--scale app=N`). Never mount
+  the raw Docker socket into Traefik.
+</Callout>
+
+## Quick start
+
+<Steps>
+
+<Step>
+### Opt in (pick one)
+
+```typescript title="oke.config.ts"
+images: {
+  "store.sql": "postgres:18-alpine",
+  // …
+  proxy: "caddy:2-alpine", // or "traefik:v3.3"
+},
+```
+
+Pin **either** Caddy or Traefik — never both. Re-run `oke docker --prod`.
+
+</Step>
+
+<Step>
+### Set the public host and ACME email
+
+```bash title="docker/.env.docker"
+OKE_PROXY_HOST=app.example.com
+OKE_PROXY_ACME_EMAIL=admin@example.com
+```
+
+`.env.docker` is preserved across regeneration. Point DNS `A`/`AAAA` at the host.
+
+</Step>
+
+<Step>
+### Bring the stack up
+
+```bash
+cd docker
+docker compose -f compose.yml -f compose.store.sql.yml \
+  -f compose.proxy.yml -f compose.prod.yml up -d
+```
+
+Pass every `-f` file `oke docker` listed. The edge publishes `80`/`443`; `app`
+stays on the internal `oke` network (no host bind on `6530`).
+
+Add your public hostname to serve `allowedHosts`
+([Security](/docs/reference/security)).
+
+</Step>
+
+</Steps>
+
+## Which proxy?
+
+| Choice                       | Best for                                                   | How routing works                                                  |
+| ---------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------ |
+| **Caddy** (`caddy:2-alpine`) | hello / minimal / standard — single instance, simplest TLS | Generated `Caddyfile` → `reverse_proxy app:6530`; automatic HTTPS  |
+| **Traefik** (`traefik:v3.3`) | Horizontally scaled plain compose                          | Docker labels on `app`; replicas auto-discovered and load-balanced |
+| _(omit `images.proxy`)_      | Local / private networks                                   | App publishes `6530` — **default, unchanged**                      |
+
+**Consequence:** Caddy is the recommended default when you just need a certificate.
+Choose Traefik when `docker compose up --scale app=N` is the deployment shape.
+
+## Caddy
+
+Emits `docker/Caddyfile`:
+
+```text
+{$OKE_PROXY_HOST:localhost} {
+	reverse_proxy app:6530
+}
+```
+
+| Setting                | What it does                                                                                                                                  |
+| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
+| `OKE_PROXY_HOST`       | Site address. `localhost` → local TLS; a public DNS name → Let's Encrypt HTTP-01                                                              |
+| `OKE_PROXY_ACME_EMAIL` | Written into `.env.docker` for the proxy role. Traefik consumes it; for Caddy, add a global `email` block if you want a specific ACME contact |
+
+Custom domain:
+
+1. DNS for `app.example.com` → this host
+2. `OKE_PROXY_HOST=app.example.com` in `docker/.env.docker`
+3. Ports `80`/`443` reachable from the public internet
+4. `allowedHosts` includes `app.example.com`
+
+Override the file via `compose.override.yml` or replace `docker/Caddyfile` —
+`oke docker` regenerates the default on the next run.
+
+## Traefik
+
+Generated `compose.proxy.yml` labels `app`:
+
+| Label                                                     | Meaning                                      |
+| --------------------------------------------------------- | -------------------------------------------- |
+| `traefik.enable=true`                                     | Advertise this service to Traefik            |
+| `traefik.http.routers.app.rule=Host(…)`                   | Match `OKE_PROXY_HOST` (default `localhost`) |
+| `traefik.http.routers.app.entrypoints=websecure`          | HTTPS entrypoint                             |
+| `traefik.http.routers.app.tls.certresolver=letsencrypt`   | ACME via Let's Encrypt                       |
+| `traefik.http.services.app.loadbalancer.server.port=6530` | Upstream port on the compose network         |
+
+HTTP (`:80`) redirects to HTTPS. ACME email comes from
+`OKE_PROXY_ACME_EMAIL` (default `admin@example.com`) on the Traefik command line.
+
+Scaled replicas need no label edits — `docker compose up --scale app=N` registers
+every replica through the Docker provider.
+
+## Why docker-socket-proxy (Traefik)
+
+Traefik's Docker provider watches the Engine API for containers and labels.
+The naive setup mounts `/var/run/docker.sock` into Traefik itself.
+
+That socket is root-equivalent for the host Docker daemon. If the edge process
+is compromised (public `80`/`443`), an attacker with the raw socket can start
+privileged containers, mount the host filesystem, or escape the container.
+
+Traefik [documents this risk](https://doc.traefik.io/traefik/reference/install-configuration/providers/docker/)
+and recommends a filtered proxy. OKE **never** mounts `docker.sock` into Traefik —
+a companion `tecnativa/docker-socket-proxy` mounts it read-only and exposes only:
+
+| Flag         | Why Traefik needs it       |
+| ------------ | -------------------------- |
+| `CONTAINERS` | List / inspect for labels  |
+| `EVENTS`     | Watch start/stop for scale |
+| `PING`       | Health of the Engine API   |
+| `VERSION`    | API negotiation            |
+| `NETWORKS`   | Network membership         |
+
+Destructive Engine calls (`images`, `volumes`, exec, swarm admin, …) stay denied.
+Traefik uses `--providers.docker.endpoint=tcp://socket-proxy:2375` on the
+internal `oke` network only — do **not** publish `2375` on the host.
+
+**Consequence:** a compromised Traefik still talks to a narrow API surface, not
+the full Docker control plane.
+
+## Environment
+
+| Variable               | Default             | Used by                                   |
+| ---------------------- | ------------------- | ----------------------------------------- |
+| `OKE_PROXY_HOST`       | `localhost`         | Caddy site address; Traefik `Host()` rule |
+| `OKE_PROXY_ACME_EMAIL` | `admin@example.com` | Traefik ACME account email                |
+
+## Where this applies
+
+| Surface                                       | Edge / ingress                                           |
+| --------------------------------------------- | -------------------------------------------------------- |
+| [Docker](/docs/deployment/docker)             | Opt-in Caddy or Traefik (`images.proxy`)                 |
+| [Docker Swarm](/docs/deployment/docker-swarm) | Same proxy recipes + Swarm routing mesh                  |
+| [Kubernetes](/docs/deployment/kubernetes)     | Ingress / Gateway API (cluster-native — not this recipe) |
+
+## Troubleshooting
+
+<Accordions>
+<Accordion title="compose up --scale app=N fails with port already allocated">
+
+You still have `6530:6530` on `app`. Confirm `images.proxy` is set and
+`compose.proxy.yml` is in the `-f` list so derivation omitted the host bind.
+
+</Accordion>
+<Accordion title="Let's Encrypt challenge fails">
+
+`OKE_PROXY_HOST` must be a public DNS name pointing at this host; ports `80`/`443`
+reachable. Set `OKE_PROXY_ACME_EMAIL` for Traefik. For localhost-only smoke tests, prefer Caddy.
+
+</Accordion>
+<Accordion title="Traefik starts but never routes to app">
+
+Confirm app labels are present (`traefik.enable=true`) in `compose.proxy.yml`,
+containers share the `oke` network, and `socket-proxy` is healthy. Traefik must
+not mount `docker.sock` itself.
+
+</Accordion>
+<Accordion title="Host header rejected (403)">
+
+Serve validates `Host`. Add your public hostname to `allowedHosts`
+([Security](/docs/reference/security)).
+
+</Accordion>
+<Accordion title="Caddy serves localhost certs on my public host">
+
+`OKE_PROXY_HOST` is still `localhost` (or unset). Set the public DNS name in
+`docker/.env.docker` and recreate the proxy container so it reloads the Caddyfile.
+
+</Accordion>
+</Accordions>
+
+## Learn more
+
+- [Deployment](/docs/deployment) — pick compose, Swarm, or Kubernetes
+- [Docker](/docs/deployment/docker) — plain compose path (includes proxy opt-in)
+- [Docker Swarm](/docs/deployment/docker-swarm) — `docker stack deploy`
+- [CLI](/docs/reference/cli) — `oke docker`, `oke docker --prod`
+- [Configuration](/docs/reference/configuration) — `images` pins
+- [Security](/docs/reference/security) — `allowedHosts` behind a reverse proxy
+
+## Next
+
+<Cards>
+  <Card title="Deployment" href="/docs/deployment" />
+  <Card title="Docker" href="/docs/deployment/docker" />
+  <Card title="Docker Swarm" href="/docs/deployment/docker-swarm" />
+</Cards>

package/site/content/docs/elements/channel.mdx

@@ -85,6 +85,8 @@ Locally the `console` driver captures mail into an inbox instead of sending; in
 
 Opt-out is first-class: a subject who opted out of a medium is **suppressed** — the send resolves without contacting the provider, and the receipt status is `suppressed/opted-out`. You never hand-roll "did they unsubscribe?" checks.
 
+**Known limit:** default suppression, consent, and receipt stores are **process-local memory**. Opt-out / hard-bounce / receipt on replica A is invisible to replica B. Boot prints a one-shot warning when those defaults are used. Inject shared stores for multi-instance, or run a single Channel consumer, until a durable driver ships.
+
 ### Locale resolves, then the catalog
 
 `fx.send` picks a locale: explicit `locale` → `profileLocale` → first
@@ -132,23 +134,34 @@ images: {
 },
 ```
 
-| Driver     | Medium   | Behavior                                                            |
-| ---------- | -------- | ------------------------------------------------------------------- |
-| `console`  | any      | Captures into a readable inbox — local + tests                      |
-| `smtp`     | email    | Real SMTP — Mailpit in docker, your provider in prod                |
-| `resend`   | email    | Resend HTTP API (`RESEND_API_KEY`)                                  |
-| `sndr`     | email    | SNDR HTTP API (`SNDR_API_KEY`)                                      |
-| `taqnyat`  | sms      | Taqnyat SMS (`TAQNYAT_BEARER_TOKEN` + `TAQNYAT_SENDER`)             |
-| `msegat`   | sms      | Msegat SMS (`MSEGAT_USERNAME` + `MSEGAT_API_KEY` + `MSEGAT_SENDER`) |
-| `unifonic` | sms      | Unifonic el.cloud SMS (`UNIFONIC_APPSID`)                           |
-| `wa-cloud` | whatsapp | Meta WhatsApp Cloud API                                             |
-| `webpush`  | push     | Web Push + VAPID (`createPushSender` → `WebPushTransport`)          |
-| `fcm`      | push     | Firebase Cloud Messaging HTTP v1                                    |
+| Driver         | Medium   | Behavior                                                            |
+| -------------- | -------- | ------------------------------------------------------------------- |
+| `console`      | any      | Captures into a readable inbox — local + tests                      |
+| `smtp`         | email    | Real SMTP — Mailpit in docker, your provider in prod                |
+| `resend`       | email    | Resend HTTP API (`RESEND_API_KEY`)                                  |
+| `sndr`         | email    | SNDR HTTP API (`SNDR_API_KEY`)                                      |
+| `taqnyat-mail` | email    | Taqnyat Mail API (`TAQNYAT_MAIL_TOKEN` + `TAQNYAT_CAMPAIGN`)        |
+| `taqnyat`      | sms      | Taqnyat SMS (`TAQNYAT_BEARER_TOKEN` + `TAQNYAT_SENDER`)             |
+| `msegat`       | sms      | Msegat SMS (`MSEGAT_USERNAME` + `MSEGAT_API_KEY` + `MSEGAT_SENDER`) |
+| `unifonic`     | sms      | Unifonic el.cloud SMS (`UNIFONIC_APPSID`)                           |
+| `wa-cloud`     | whatsapp | Meta WhatsApp Cloud API                                             |
+| `webpush`      | push     | Web Push + VAPID (`createPushSender` → `WebPushTransport`)          |
+| `fcm`          | push     | Firebase Cloud Messaging HTTP v1                                    |
 
 Email fallback uses ordered email transports; SMS with multiple SMS drivers uses
 the same failover path. Credentials for boot-opened drivers are listed under
 [Environment Variables](/docs/reference/environment-variables).
 
+### Provider-managed OTP
+
+Flows can also send a provider-generated SMS code with `fx.sendOtp({ to, requestId, lang? })`
+and check it with `fx.verifyOtp({ to, requestId, code })` — Taqnyat's Verify API. Both are
+capability-gated `send` effects and dry-run safe.
+
+They dispatch only when the bound SMS driver supports Verify (`taqnyat`); any other SMS
+driver fails loudly instead of silently falling back to a self-generated code. The
+[OTP](/docs/plugins/otp) Tier 1 uses this path automatically.
+
 `webpush` needs VAPID keys — open it yourself and include it in
 `BootOptions.channel.drivers` (boot does not open push from env):
 

package/site/content/docs/elements/clock.mdx

@@ -73,7 +73,7 @@ export const sendDaily = on(
 ```typescript
 do: async (input, fx) => {
   const now = fx.clock.now(); // epoch-ms, injectable
-  await fx.clock.sleep("wait-for-payment", "7d"); // durable — survives restarts
+  await fx.clock.sleep("wait-for-payment", "7d"); // durable — survives restarts on a shared journal
 };
 ```
 
@@ -103,13 +103,13 @@ difference.
 <ClockSleep />
 
 `fx.clock.sleep(label, duration)` is a **durable** sleep: in a `durable: true` flow the wake time is
-journaled, so the flow resumes after restarts and deploys instead of losing its place. The `label`
-names the step in the journal — it is what the Console shows when you inspect a sleeping run.
+journaled — with a shared journal (`drivers.journal: "postgres"`) the flow resumes after restarts and
+deploys; the `memory` default resumes within the same process only. The `label` names the step — the Console shows it on sleeping runs.
 
 In a non-durable flow the same call resolves immediately, so code reads identically in tests.
 
-Completed journal steps never re-run on resume — a process killed mid-flow continues at the next
-unfinished step.
+Completed journal steps never re-run on resume — with a shared journal, a process killed mid-flow is
+reclaimed after its lease expires and continues at the next unfinished step.
 
 ## What the runtime guarantees
 
@@ -123,8 +123,8 @@ unfinished step.
 | DST detection      | Ambiguous local times get a Console warning — schedules are not rewritten or blocked               |
 
 **Leader election needs a shared store.** The default in-memory CronStore is per process. Multi-replica
-exclusivity only holds when every instance opens the same durable CronStore (file-backed store is
-available for that; a Postgres cron driver is not wired yet).
+exclusivity holds when every instance opens the same durable CronStore — `postgres` (SKIP LOCKED +
+lease reclaim, docker/prod default) or `file` (shared volume, single-host / local).
 
 **Catch-up example:** an hourly clock down for five hours reports `missedRuns: 5` and
 `catchUp: "one"`, then a single tick runs the handler once.
@@ -141,15 +141,16 @@ Detection attaches `dstAmbiguity` on the cron row and the Console shows it. `oke
 
 ```typescript title="oke.config.ts"
 drivers: {
-  clock: { local: "memory", docker: "file", test: "frozen", prod: "file" },
+  clock: { local: "memory", docker: "postgres", test: "frozen", prod: "postgres" },
 },
 ```
 
-| Driver   | Behavior                                                                                 |
-| -------- | ---------------------------------------------------------------------------------------- |
-| `memory` | In-process CronStore + timers — single process; lost on exit                             |
-| `file`   | On-disk CronStore (`.oke/crons.json`) — multi-process leader election on a shared volume |
-| `frozen` | Deterministic test clock — time advances only when the test says so                      |
+| Driver     | Behavior                                                                                       |
+| ---------- | ---------------------------------------------------------------------------------------------- |
+| `memory`   | In-process CronStore + timers — single process; lost on exit                                   |
+| `postgres` | Postgres CronStore (`oke_crons`) — multi-host leader election via SKIP LOCKED + lease reclaim  |
+| `file`     | On-disk CronStore (`.oke/crons.json`) — multi-process on a shared volume (local / single-host) |
+| `frozen`   | Deterministic test clock — time advances only when the test says so                            |
 
 `frozen` is why the no-`Date.now()` rule pays off: tests inject time travel through `fx.clock` and every flow obeys it automatically.
 
@@ -179,12 +180,13 @@ The clock was declared without `overridable: true`. Add it and redeploy — the
 </Accordion>
 <Accordion title="How do I run something once, later — not recurring?">
 
-Emit it from inside a flow with `fx.clock.sleep(label, duration)` before the work, in a `durable: true` flow. The sleep survives restarts, so "remind me in 7 days" is one line, not a cron row.
+Emit it from inside a flow with `fx.clock.sleep(label, duration)` before the work, in a `durable: true` flow. The sleep survives restarts on a shared journal, so "remind me in 7 days" is one line, not a cron row.
 
 </Accordion>
 <Accordion title="Two replicas both ran my cron">
 
-They are not sharing one CronStore. In-memory leases are per process. Point every replica at the same durable cron store, or run a single scheduler instance until the Postgres clock driver ships.
+They are not sharing one CronStore. In-memory leases are per process. Point every replica at the same
+`postgres` CronStore (`DATABASE_URL`), or use `file` on a shared volume for single-host multi-process.
 
 </Accordion>
 </Accordions>

package/site/content/docs/elements/flow.mdx

@@ -219,14 +219,18 @@ export const chargeOrder = flow({
       stripe(fx.vault(stripeKey).reveal()).create(orderId),
     );
 
-    await fx.clock.sleep("verify-window", "2m"); // survives restart and deploy
+    await fx.clock.sleep("verify-window", "2m"); // journals wakeAt in a durable flow
 
     return fx.step("confirm", () => stripe(fx.vault(stripeKey).reveal()).confirm(intent));
   },
 });
 ```
 
-**Consequence:** kill the process between the two steps and the run **resumes at `confirm`** — completed steps replay from the journal, so the card is not charged twice. This is verified by the engine's own test suite: after resume, `create-intent` has run exactly once.
+**Consequence:** completed steps replay from the journal — on resume, `create-intent` does not re-run, so the card is not charged twice.
+
+**Crash recovery needs a shared journal.** Set `drivers.journal: "postgres"` (the docker/prod template default) and every run persists in `oke_journal_runs` under a per-run lease: boot scans for orphaned `running` / due `sleeping` runs and resumes them, and the lease lets exactly one instance execute a run at a time. The default `memory` journal is per process — replay holds for that process only.
+
+Replay is **at-least-once** for a step whose body never persisted (crash mid-step, lease overrun): that step re-runs on reclaim, so keep step bodies short or internally idempotent. A client transport retry starts a _new_ run with a fresh `runId` — request-level idempotency is an application concern, not the journal's.
 
 For flaky sub-steps, wrap the work in `fx.retry` **inside** `fx.step` so a completed charge is never retried on resume. Coarse whole-body retry is also available as `flow({ retry: { retries, delay, backoff, jitter } })` and reuses the same journal session.
 

package/site/content/docs/elements/signal.mdx

@@ -245,14 +245,16 @@ drivers: {
 },
 ```
 
-| Driver     | Runs as        | Boot today                                                                |
-| ---------- | -------------- | ------------------------------------------------------------------------- |
-| `memory`   | in-process     | Local loop + tests — zero infrastructure                                  |
-| `redis`    | Redis / Valkey | Default for docker/prod — Streams + pub/sub via `Bun.redis`               |
-| `postgres` | Postgres       | Driver exists; boot **fails loud** until a LISTEN/NOTIFY SQL client binds |
-| `nats`     | NATS           | Driver exists; boot **fails loud** until a production NATS client binds   |
+| Driver     | Runs as                              | Boot today                                                                  |
+| ---------- | ------------------------------------ | --------------------------------------------------------------------------- |
+| `memory`   | in-process                           | Local loop + tests — zero infrastructure                                    |
+| `redis`    | Redis / Valkey / Dragonfly image pin | Boot-bound — **emit relay** via Streams / pub/sub; consume is process-local |
+| `postgres` | Postgres                             | Driver exists; boot **fails loud** until a LISTEN/NOTIFY SQL client binds   |
+| `nats`     | NATS                                 | Driver exists; boot **fails loud** until a production NATS client binds     |
 
-`redis` is the honestly boot-bound durable path today. Pairing an arbitrary Store insert with emit inside **one** shared SQL transaction is not what `fx.emit` does — keep consumers idempotent (at-least-once).
+**Known limit:** `redis` relays emits to Redis, but `once` / `broadcast` / `live` **consume, `live` replay, and `drain` still use a process-local outbox** today. Competing consumers across replicas do not share that outbox until Redis Streams consume ships. Boot prints a one-shot warning when `drivers.signal` is `redis`. Prefer a single consumer instance (or a shared durable outbox path in tests) for multi-process `once` until then.
+
+Pairing an arbitrary Store insert with emit inside **one** shared SQL transaction is not what `fx.emit` does — keep consumers idempotent (at-least-once).
 
 ## Troubleshooting
 
@@ -279,7 +281,7 @@ Ask: _how many consumers should process each message?_ One → `once`. All of th
 </Accordion>
 <Accordion title="My emit happened but the row didn't (or vice versa)">
 
-`fx.emit` commits the signal outbox when it resolves; it does not wrap your Store insert. Treat delivery as at-least-once and make the consumer idempotent (or check the row before acting). Prefer `redis` (or a future LISTEN-capable postgres bind) when you want durable delivery rather than in-process memory.
+`fx.emit` commits the signal outbox when it resolves; it does not wrap your Store insert. Treat delivery as at-least-once and make the consumer idempotent (or check the row before acting). Remember: `redis` today is an emit relay with a process-local consume outbox — not multi-instance competing consumers yet.
 
 </Accordion>
 <Accordion title="The same once message ran twice">