@fieldnotes/sync-server 0.15.0 → 0.17.0

package/README.md

@@ -1,283 +1,291 @@
-# @fieldnotes/sync-server
-
-Authoritative WebSocket relay server for [`@fieldnotes/sync`](../sync).
-
-The relay holds the canonical per-room canvas state and fans out element
-operations to every other connection in the room. Clients connect, request a
-snapshot to catch up, then stream `upsert` / `remove` / `clear` ops that the hub
-applies and forwards.
-
-## Pieces
-
-- **`SyncHub`** — transport-agnostic relay. Per-room canonical state lives behind
-  an async `HubBackend`; each room processes messages on its own serial queue so
-  concurrent edits to the same room never race, while different rooms run
-  independently.
-- **`MemoryHubBackend`** — in-memory `HubBackend` (the default). Redis-backed and
-  authenticated backends are planned. It reclaims a room's memory on `clear`, but
-  **retains** state for uncleared / abandoned rooms — fine for dev / single-instance
-  use. For a long-lived production process, use a Redis backend with a key TTL, or
-  `clear` rooms you're done with.
-- **`createSyncServer`** — a runnable `ws` reference server. Connect with
-  `?room=<id>` in the query string; a missing room closes the socket with WS code
-  `4400`.
-- **`HubFanout`** — cross-instance live fan-out seam. `SyncHub` publishes each live
-  op to the fanout and forwards ops it receives from other instances to its local
-  connections. The default `InMemoryHubFanout` is in-process (a no-op for a single
-  instance). For multiple relay instances to share live ops, pass a shared fanout
-  via `createSyncServer({ fanout })` (e.g. `RedisHubFanout` from
-  [`@fieldnotes/sync-redis`](../sync-redis)) **together with a shared backend** — a
-  shared fanout alone leaves a new joiner's snapshot stale.
-
-`HubFanout.publish()` may return a promise. `SyncHub` awaits durable mutation publication before
-notifying other local connections, and `handleMessage()` rejects if publication fails after the
-backend commit. Custom server integrations must observe that rejection; `createSyncServer` reports
-it through its relay error path. The room queue recovers so later messages can continue. Presence
-publication remains best-effort because presence is ephemeral.
-
-## Usage
-
-```ts
-import { createSyncServer } from '@fieldnotes/sync-server';
-
-const { close } = createSyncServer({ port: 8080 });
-// ws://localhost:8080?room=my-room
-```
-
-## Server-originated presence
-
-Trusted server integrations can broadcast ephemeral data without accessing the hub's connection
-registries:
-
-```ts
-const { hub } = createSyncServer({ port: 8080, fanout });
-
-const localDeliveries = hub.broadcastPresence('my-room', {
-  kind: 'poke',
-  feature: 'initiative',
-});
-```
-
-Recipients receive the existing presence envelope with the server-owned identity
-`{ from: 'hub', op: { kind: 'presence', data } }`. The event is delivered to every local room
-member and published to the configured `HubFanout`, so other relay instances deliver it to their
-local members too. The return value counts successful sends on the originating hub only; it is not a
-cluster-wide acknowledgement.
-
-Server presence is best-effort, is not persisted, does not enter a room's durable operation queue,
-and is not filtered by `authorize` or `canRead`. Validate and authorize application requests before
-calling this trusted-host API. Data must be JSON-serializable; invalid data throws before delivery.
-The method constructs the presence operation itself and does not accept a raw sync operation or
-caller-controlled sender identity.
-
-## Heartbeat
-
-The server pings every client on an interval and **terminates** any that miss a pong,
-so half-open sockets (a backgrounded iPad, dropped WiFi) are reaped instead of silently
-leaking room membership. Configure via `heartbeatIntervalMs` (default `30000`; `0`
-disables):
-
-```ts
-createSyncServer({ port: 8080, heartbeatIntervalMs: 30000 });
-```
-
-Browsers auto-pong at the protocol level, so **no client change** is needed.
-
-## Graceful shutdown
-
-`close()` stops accepting connections, closes active clients with code `1001`, and waits up to
-`shutdownGraceMs` (default `5000`) before terminating clients that have not completed the close
-handshake. Repeated calls return the same promise, so shutdown hooks can safely converge on it.
-
-```ts
-const server = createSyncServer({ port: 8080, shutdownGraceMs: 5000 });
-await server.close();
-```
-
-## Resource limits
-
-The reference server bounds work and memory per connection by default. Oversized WebSocket messages
-close with code `1009`; message-rate and pending-auth queue violations close with code `4408`.
-Messages with excessive JSON nesting are dropped. Presence sends immediately, then coalesces rapid
-updates so the latest state is forwarded at most once per throttle interval.
-
-```ts
-createSyncServer({
-  port: 8080,
-  maxMessageBytes: 1024 * 1024,
-  maxJsonDepth: 32,
-  maxPendingAuthMessages: 100,
-  maxPendingAuthBytes: 2 * 1024 * 1024,
-  messagesPerSecond: 120,
-  messageBurst: 240,
-  presenceThrottleMs: 50,
-});
-```
-
-These values are the defaults. Tune them to the largest legitimate board operation and expected
-client update rate. `maxMessageBytes` is enforced by the WebSocket parser before a complete message
-is allocated, including fragmented messages. The pending-auth limits bound messages held while an
-asynchronous `authenticate` hook is unresolved. Rate limits are per connection and use a token
-bucket: `messageBurst` is the short spike allowance and `messagesPerSecond` is the refill rate.
-
-## Authentication
-
-Pass an `authenticate` hook to gate connections:
-
-```ts
-import { createSyncServer } from '@fieldnotes/sync-server';
-
-createSyncServer({
-  port: 8080,
-  authenticate: async ({ req, room }) => {
-    const token = new URL(req.url ?? '', 'http://x').searchParams.get('token');
-    const member = await myCampaign.verify(token, room); // your app's check (Redis/DB/JWT)
-    return member ? { userId: member.id, role: member.isDM ? 'dm' : 'player' } : null;
-  },
-});
-```
-
-`authenticate({ req, room }) → { userId, role? } | null` runs on each connection and
-may be sync or async. Returning `null` (or throwing) **rejects** the connection: the
-socket is closed with WS code `4401` and the connection is never admitted to the room —
-no membership, no snapshot. A resolved result admits the connection carrying its
-`userId` and optional `role`.
-
-With **no hook**, rooms stay open: every connection is admitted anonymously with
-`userId = connId`. `role` is captured now and enforced in an upcoming release
-(role-based authorization and per-viewer visibility filtering).
-
-Messages that arrive during an async auth (notably the client's initial
-`request-snapshot`) are queued and replayed once the connection is admitted, so the
-first snapshot is never lost to the auth round-trip.
-
-### Relay identity
-
-The relay assigns every admitted socket an opaque connection identity. That server-owned identity
-is used as `from` on all relayed mutations, presence updates, cross-instance fan-out, and disconnect
-leave events; the client-controlled envelope `from` cannot impersonate another connection. Presence
-identity is intentionally connection-scoped and changes after reconnecting. Application authorization
-continues to use the authenticated `userId` and `role`, not this transport identity.
-
-### Passing a token
-
-A browser `WebSocket` can't set request headers, so pass the token as a URL query
-param (`ws://relay?room=R&token=…`) and read it from `req.url` in `authenticate`. URLs
-land in access/proxy logs, so prefer **short-lived / single-use** tokens. Non-browser
-clients can instead put the token in `req.headers` (e.g. `Authorization`), which
-`authenticate` reads directly.
-
-## Authorization
-
-Pass an `authorize` hook to gate **writes**. The hub calls it before applying each data
-op (`upsert` / `remove` / `clear`) and **drops denied ops** — a denied op is not applied
-to room state and not forwarded to any other connection:
-
-```ts
-authorize(ctx) => boolean | Promise<boolean>
-```
-
-`ctx` is an `AuthorizeContext`:
-
-```ts
-{
-  userId?: string;          // the connection's authenticated user (from authenticate)
-  role?: string;            // the connection's role (from authenticate)
-  room: string;
-  op: SyncOp;               // the incoming upsert / remove / clear
-  currentElement?: OwnedElement; // the STORED element, if this id already exists
-}
-```
-
-`currentElement` is the element currently in room state for an `upsert`/`remove` of an
-**existing** id (typed `OwnedElement = CanvasElement & { ownerId?: string }`), and
-`undefined` for a new/absent id.
-
-**Ownership is server-stamped and un-forgeable.** A new element's `ownerId` is set to the
-authenticated creator; on edit the stored owner is **preserved**; a client-supplied
-`ownerId` is always **discarded**. A policy can therefore trust `currentElement.ownerId`
-to enforce "own elements only".
-
-With **no hook**, rooms are OPEN (allow-all — every op is accepted).
-
-A copy-paste DM / player / display policy:
-
-```ts
-createSyncServer({
-  port: 8080,
-  authenticate: /* … supplies userId + role … */,
-  authorize: ({ role, op, currentElement, userId }) => {
-    if (role === 'dm') return true;
-    if (role === 'display') return false;              // read-only monitor
-    if (role === 'player') {                            // own elements only, never destructive
-      if (op.kind === 'clear') return false;
-      if (op.kind === 'upsert') return !currentElement || currentElement.ownerId === userId;
-      if (op.kind === 'remove') return currentElement?.ownerId === userId;
-      return false;
-    }
-    return false;
-  },
-});
-```
-
-**Important:**
-
-- Ownership-based authz **requires `authenticate` to supply a STABLE `userId`**. The
-  no-hook anonymous default is `userId = connId`, which changes on every reconnect — a
-  user would lose access to their own elements after reconnecting.
-- The authz path adds one `backend.get` (a Redis `HGET`) per data op — negligible for
-  low-write use.
-- Reads / visibility (a player not **receiving** hidden content) are a separate, upcoming
-  concern (D3). `authorize` gates writes only.
-- When `authorize` denies an op, the hub sends the offending client a **corrective op** so its
-  optimistic local edit self-corrects immediately — no client change, no waiting for reconnect:
-  a rejected new element is **removed**, a rejected edit/remove is re-**upserted** from the canonical
-  stored element, and a rejected `clear` gets a fresh **snapshot**. The correction reuses existing op
-  kinds and is sent only to the offending connection. When `canRead` is configured, corrections use
-  the same viewer filter: unreadable canonical upserts become byte-free removes and corrective
-  snapshots omit unreadable elements.
-
-## Read filtering
-
-Pass a `canRead` hook to gate **reads** — what each viewer **receives**. Unlike client-side
-hiding, the hub filters ops before they leave the server, so hidden elements never reach an
-unauthorized client on either path: the live broadcast **and** the snapshot a client gets on join.
-
-```ts
-canRead({ userId, role, room, audience }) => boolean
-```
-
-`audience` is the opaque tag the client stamps on its own outgoing upserts via `@fieldnotes/sync`'s
-`resolveAudience` (e.g. `'dm'` / `'shared'`, derived from the app's layers). With **no hook**,
-everyone sees everything.
-
-A two-tier DM / player policy — the relay's `canRead` paired with the client's `resolveAudience`:
-
-```ts
-// relay
-createSyncServer({
-  port: 8080,
-  authenticate: /* … supplies userId + role … */,
-  canRead: ({ role, audience }) => audience !== 'dm' || role === 'dm',
-});
-
-// client (@fieldnotes/sync)
-new SyncClient({ store, transport, resolveAudience: (el) => layerOf(el) }); // → 'dm' | 'shared'
-```
-
-Moving an element between audiences re-evaluates visibility per viewer: a viewer who loses access
-gets a synthetic **remove**, one who gains it gets an **add**.
-
-**Preconditions:**
-
-- **Stable identity.** Meaningful policy needs a stable `userId`/`role` from `authenticate`; the
-  no-hook anonymous default is per-connection (`userId = connId`) and changes on reconnect.
-- **Same `canRead` on every instance.** Unlike `authorize` (which runs on the write's origin
-  instance only), `canRead` runs on **every** instance — each re-filters fanned-out ops for its own
-  local members — so all relay instances must inject the **same** `canRead`, alongside the existing
-  shared-backend + shared-fanout requirement.
-- **Labeling integrity.** `audience` is client-asserted, so read secrecy is only as trustworthy as
-  the `authorize` (write) policy that stops a player from stamping `dm` on content they shouldn't
-  control.
-
-A Redis `HubBackend` and cross-instance fan-out ship in [`@fieldnotes/sync-redis`](../sync-redis).
+# @fieldnotes/sync-server
+
+Authoritative WebSocket relay server for [`@fieldnotes/sync`](../sync).
+
+The relay holds the canonical per-room canvas state and fans out element
+operations to every other connection in the room. Clients connect, request a
+snapshot to catch up, then stream `upsert` / `remove` / `clear` ops that the hub
+applies and forwards.
+
+## Pieces
+
+- **`SyncHub`** — transport-agnostic relay. Per-room canonical state lives behind
+  an async `HubBackend`; each room processes messages on its own serial queue so
+  concurrent edits to the same room never race, while different rooms run
+  independently.
+- **`MemoryHubBackend`** — in-memory `HubBackend` (the default). Redis-backed and
+  authenticated backends are planned. It reclaims a room's memory on `clear`, but
+  **retains** state for uncleared / abandoned rooms — fine for dev / single-instance
+  use. For a long-lived production process, use a Redis backend with a key TTL, or
+  `clear` rooms you're done with.
+- **`createSyncServer`** — a runnable `ws` reference server. Connect with
+  `?room=<id>` in the query string; a missing room closes the socket with WS code
+  `4400`.
+- **`HubFanout`** — cross-instance live fan-out seam. `SyncHub` publishes each live
+  op to the fanout and forwards ops it receives from other instances to its local
+  connections. The default `InMemoryHubFanout` is in-process (a no-op for a single
+  instance). For multiple relay instances to share live ops, pass a shared fanout
+  via `createSyncServer({ fanout })` (e.g. `RedisHubFanout` from
+  [`@fieldnotes/sync-redis`](../sync-redis)) **together with a shared backend** — a
+  shared fanout alone leaves a new joiner's snapshot stale.
+
+`HubFanout.publish()` may return a promise. `SyncHub` awaits durable mutation publication before
+notifying other local connections, and `handleMessage()` rejects if publication fails after the
+backend commit. Custom server integrations must observe that rejection; `createSyncServer` reports
+it through its relay error path. The room queue recovers so later messages can continue. Presence
+publication remains best-effort because presence is ephemeral.
+
+## Usage
+
+```ts
+import { createSyncServer } from '@fieldnotes/sync-server';
+
+const { close } = createSyncServer({ port: 8080 });
+// ws://localhost:8080?room=my-room
+```
+
+## Server-originated presence
+
+Trusted server integrations can broadcast ephemeral data without accessing the hub's connection
+registries:
+
+```ts
+const { hub } = createSyncServer({ port: 8080, fanout });
+
+const localDeliveries = hub.broadcastPresence('my-room', {
+  kind: 'poke',
+  feature: 'initiative',
+});
+```
+
+Recipients receive the existing presence envelope with the server-owned identity
+`{ from: 'hub', op: { kind: 'presence', data } }`. The event is delivered to every local room
+member and published to the configured `HubFanout`, so other relay instances deliver it to their
+local members too. The return value counts successful sends on the originating hub only; it is not a
+cluster-wide acknowledgement.
+
+Server presence is best-effort, is not persisted, does not enter a room's durable operation queue,
+and is not filtered by `authorize` or `canRead`. Validate and authorize application requests before
+calling this trusted-host API. Data must be JSON-serializable; invalid data throws before delivery.
+The method constructs the presence operation itself and does not accept a raw sync operation or
+caller-controlled sender identity.
+
+## Heartbeat
+
+The server pings every client on an interval and **terminates** any that miss a pong,
+so half-open sockets (a backgrounded iPad, dropped WiFi) are reaped instead of silently
+leaking room membership. Configure via `heartbeatIntervalMs` (default `30000`; `0`
+disables):
+
+```ts
+createSyncServer({ port: 8080, heartbeatIntervalMs: 30000 });
+```
+
+Browsers auto-pong at the protocol level, so **no client change** is needed.
+
+## Graceful shutdown
+
+`close()` stops accepting connections, closes active clients with code `1001`, and waits up to
+`shutdownGraceMs` (default `5000`) before terminating clients that have not completed the close
+handshake. Repeated calls return the same promise, so shutdown hooks can safely converge on it.
+
+```ts
+const server = createSyncServer({ port: 8080, shutdownGraceMs: 5000 });
+await server.close();
+```
+
+## Resource limits
+
+The reference server bounds work and memory per connection by default. Oversized WebSocket messages
+close with code `1009`; message-rate and pending-auth queue violations close with code `4408`.
+Messages with excessive JSON nesting are dropped. Presence sends immediately, then coalesces rapid
+updates so the latest state is forwarded at most once per throttle interval.
+
+```ts
+createSyncServer({
+  port: 8080,
+  maxMessageBytes: 1024 * 1024,
+  maxJsonDepth: 32,
+  maxPendingAuthMessages: 100,
+  maxPendingAuthBytes: 2 * 1024 * 1024,
+  messagesPerSecond: 120,
+  messageBurst: 240,
+  presenceThrottleMs: 50,
+});
+```
+
+These values are the defaults. Tune them to the largest legitimate board operation and expected
+client update rate. `maxMessageBytes` is enforced by the WebSocket parser before a complete message
+is allocated, including fragmented messages. The pending-auth limits bound messages held while an
+asynchronous `authenticate` hook is unresolved. Rate limits are per connection and use a token
+bucket: `messageBurst` is the short spike allowance and `messagesPerSecond` is the refill rate.
+
+## Authentication
+
+Pass an `authenticate` hook to gate connections:
+
+```ts
+import { createSyncServer } from '@fieldnotes/sync-server';
+
+createSyncServer({
+  port: 8080,
+  authenticate: async ({ req, room }) => {
+    const token = new URL(req.url ?? '', 'http://x').searchParams.get('token');
+    const member = await myCampaign.verify(token, room); // your app's check (Redis/DB/JWT)
+    return member ? { userId: member.id, role: member.isDM ? 'dm' : 'player' } : null;
+  },
+});
+```
+
+`authenticate({ req, room }) → { userId, role? } | null` runs on each connection and
+may be sync or async. Returning `null` (or throwing) **rejects** the connection: the
+socket is closed with WS code `4401` and the connection is never admitted to the room —
+no membership, no snapshot. A resolved result admits the connection carrying its
+`userId` and optional `role`.
+
+With **no hook**, rooms stay open: every connection is admitted anonymously with
+`userId = connId`. `role` is captured now and enforced in an upcoming release
+(role-based authorization and per-viewer visibility filtering).
+
+Messages that arrive during an async auth (notably the client's initial
+`request-snapshot`) are queued and replayed once the connection is admitted, so the
+first snapshot is never lost to the auth round-trip.
+
+### Relay identity
+
+The relay assigns every admitted socket an opaque connection identity. That server-owned identity
+is used as `from` on all relayed mutations, presence updates, cross-instance fan-out, and disconnect
+leave events; the client-controlled envelope `from` cannot impersonate another connection. Presence
+identity is intentionally connection-scoped and changes after reconnecting. Application authorization
+continues to use the authenticated `userId` and `role`, not this transport identity.
+
+### Passing a token
+
+A browser `WebSocket` can't set request headers, so pass the token as a URL query
+param (`ws://relay?room=R&token=…`) and read it from `req.url` in `authenticate`. URLs
+land in access/proxy logs, so prefer **short-lived / single-use** tokens. Non-browser
+clients can instead put the token in `req.headers` (e.g. `Authorization`), which
+`authenticate` reads directly.
+
+## Authorization
+
+Pass an `authorize` hook to gate **writes**. The hub calls it before applying each data
+op (`upsert` / `remove` / `clear`) and **drops denied ops** — a denied op is not applied
+to room state and not forwarded to any other connection:
+
+```ts
+authorize(ctx) => boolean | Promise<boolean>
+```
+
+`ctx` is an `AuthorizeContext`:
+
+```ts
+{
+  userId?: string;          // the connection's authenticated user (from authenticate)
+  role?: string;            // the connection's role (from authenticate)
+  room: string;
+  op: SyncOp;               // the incoming upsert / remove / clear
+  currentElement?: OwnedElement; // the STORED element, if this id already exists
+}
+```
+
+`currentElement` is the element currently in room state for an `upsert`/`remove` of an
+**existing** id (typed `OwnedElement = CanvasElement & { ownerId?: string }`), and
+`undefined` for a new/absent id.
+
+**Ownership is server-stamped and un-forgeable.** A new element's `ownerId` is set to the
+authenticated creator; on edit the stored owner is **preserved**; a client-supplied
+`ownerId` is always **discarded**. A policy can therefore trust `currentElement.ownerId`
+to enforce "own elements only".
+
+With **no hook**, rooms are OPEN (allow-all — every op is accepted).
+
+A copy-paste DM / player / display policy:
+
+```ts
+createSyncServer({
+  port: 8080,
+  authenticate: /* … supplies userId + role … */,
+  authorize: ({ role, op, currentElement, userId }) => {
+    if (role === 'dm') return true;
+    if (role === 'display') return false;              // read-only monitor
+    if (role === 'player') {                            // own elements only, never destructive
+      if (op.kind === 'clear') return false;
+      if (op.kind === 'upsert') return !currentElement || currentElement.ownerId === userId;
+      if (op.kind === 'remove') return currentElement?.ownerId === userId;
+      return false;
+    }
+    return false;
+  },
+});
+```
+
+**Important:**
+
+- Ownership-based authz **requires `authenticate` to supply a STABLE `userId`**. The
+  no-hook anonymous default is `userId = connId`, which changes on every reconnect — a
+  user would lose access to their own elements after reconnecting.
+- The authz path adds one `backend.get` (a Redis `HGET`) per data op — negligible for
+  low-write use.
+- Reads / visibility (a player not **receiving** hidden content) are a separate, upcoming
+  concern (D3). `authorize` gates writes only.
+- When `authorize` denies an op, the hub sends the offending client a **corrective op** so its
+  optimistic local edit self-corrects immediately — no client change, no waiting for reconnect:
+  a rejected new element is **removed**, a rejected edit/remove is re-**upserted** from the canonical
+  stored element, and a rejected `clear` gets a fresh **snapshot**. The correction reuses existing op
+  kinds and is sent only to the offending connection. When `canRead` is configured, corrections use
+  the same viewer filter: unreadable canonical upserts become byte-free removes and corrective
+  snapshots omit unreadable elements.
+
+## Read filtering
+
+Pass a `canRead` hook to gate **reads** — what each viewer **receives**. Unlike client-side
+hiding, the hub filters ops before they leave the server, so hidden elements never reach an
+unauthorized client on either path: the live broadcast **and** the snapshot a client gets on join.
+
+```ts
+canRead({ userId, role, room, audience }) => boolean
+```
+
+`audience` is the opaque tag the client stamps on its own outgoing upserts via `@fieldnotes/sync`'s
+`resolveAudience` (e.g. `'dm'` / `'shared'`, derived from the app's layers). With **no hook**,
+everyone sees everything.
+
+A two-tier DM / player policy — the relay's `canRead` paired with the client's `resolveAudience`:
+
+```ts
+// relay
+createSyncServer({
+  port: 8080,
+  authenticate: /* … supplies userId + role … */,
+  canRead: ({ role, audience }) => audience !== 'dm' || role === 'dm',
+});
+
+// client (@fieldnotes/sync)
+new SyncClient({ store, transport, resolveAudience: (el) => layerOf(el) }); // → 'dm' | 'shared'
+```
+
+Moving an element between audiences re-evaluates visibility per viewer: a viewer who loses access
+gets a synthetic **remove**, one who gains it gets an **add**.
+
+**Preconditions:**
+
+- **Stable identity.** Meaningful policy needs a stable `userId`/`role` from `authenticate`; the
+  no-hook anonymous default is per-connection (`userId = connId`) and changes on reconnect.
+- **Same `canRead` on every instance.** Unlike `authorize` (which runs on the write's origin
+  instance only), `canRead` runs on **every** instance — each re-filters fanned-out ops for its own
+  local members — so all relay instances must inject the **same** `canRead`, alongside the existing
+  shared-backend + shared-fanout requirement.
+- **Labeling integrity.** `audience` is client-asserted, so read secrecy is only as trustworthy as
+  the `authorize` (write) policy that stops a player from stamping `dm` on content they shouldn't
+  control.
+
+A Redis `HubBackend` and cross-instance fan-out ship in [`@fieldnotes/sync-redis`](../sync-redis).
+
+## Domain plugins
+
+`SyncHubOptions.plugins` installs ordered `ServerSyncPlugin` middleware. Plugins can own legacy v3
+kinds, register codec-validated extension kinds, provide versioned snapshots, return sender-only
+corrections, and choose local or shared fanout per accepted operation. Fog authorization and state
+application are supplied by `createFogServerPlugin()` from `@fieldnotes/vtt/server`; the generic
+server has no runtime VTT dependency.