@greendrake/rpc-server 0.8.0 → 0.9.0

package/README.md

@@ -1,6 +1,6 @@
 # @greendrake/rpc-server
 
-The server half of the [`@greendrake/rpc`](../rpc) wire format: one dispatch table, an HTTP endpoint and a WebSocket endpoint over it, per-method auth and argument validation, and a connection registry that pushes to clients unasked. **Runs on [bun](https://bun.sh)** — `Bun.serve` owns the sockets. The protocol core (dispatch, auth, nonce, validation) is pure functions over the wire types and touches no runtime API.
+The server half of the [`@greendrake/rpc`](../rpc) wire format: one dispatch table, an HTTP endpoint and a WebSocket endpoint over it, per-method auth and argument validation, and a connection registry that pushes to clients unasked — events, and the state documents every socket keeps. **Runs on [bun](https://bun.sh)** — `Bun.serve` owns the sockets. The protocol core (dispatch, auth, nonce, validation) is pure functions over the wire types and touches no runtime API.
 
 ## Install
 
@@ -101,6 +101,8 @@ A request carrying a `nonce` runs once: `NonceStore.checkAndSet` answers false f
 
 `websocketHandler(dispatcher, registry, { onOpen?, onClose? })` returns `{ upgrade, handlers }`. The handshake credential is the `token` query parameter — the only channel a browser's WebSocket constructor leaves open, having no headers, and one a reconnect carries again for free. Calls dispatch concurrently, correlated by the `id` the client puts on each frame.
 
+Nothing pushed is ever dropped. Bun drops a frame once a socket's send buffer passes its `backpressureLimit`; `handlers` sets `closeOnBackpressureLimit`, so such a socket is closed instead. A client that has stopped reading would otherwise be left silently wrong. Once it reconnects, its re-read (`liveBinding`) or the documents it is greeted with put it right.
+
 The exception is `auth.token` and `auth.logout`, the in-band frames `@greendrake/rpc`'s `WsTransport` sends: they change who every later call on that socket is made by, so each goes on a per-connection gate the calls wait behind. Without it a call sent right after a bind would race it and come back `UNAUTHENTICATED`.
 
 A codec applies to everything on the wire, in both directions, for a service whose clients obscure or encrypt their payloads: `new ConnRegistry(codec)` for the socket, `httpHandler(dispatcher, { codec })` for the endpoint.
@@ -117,6 +119,23 @@ registry.broadcastData('telemetry', reading) // the bare {type, data} form
 
 Push-producing code takes the `EventSink` interface (`emitToUser` + `broadcastEvent`) rather than the class, so it can be driven by anything. On the client, `@greendrake/rpc`'s `liveBinding` subscribes to these, folds each push in or re-reads, and re-reads again after a dropped socket returns.
 
+### Documents
+
+State the whole service shares, kept current on every socket, is a document rather than a stream of events:
+
+```ts
+const cameras = registry.document('cameras', () => fleet.states()) // { gate: {…}, porch: {…} }
+
+fleet.onChange(() => cameras.refresh())
+```
+
+- **Greeting.** `document(type, produce)` produces the state now, and every socket that opens is sent it whole: `{type, data: {doc}}`.
+- **Patches.** Each `refresh()` produces the state again and sends every socket the RFC 7386 merge patch from the last one: `{type, data: {patch}}`. Objects merge, `null` removes a member, and anything else replaces, so collections are objects keyed by id. Nothing is sent when nothing changed, so a producer calls `refresh` on any change it knows of.
+- **Shape.** The state must be a JSON object, or `document` throws. A member produced as `null` is dropped, the same as one left out, since a patch could not tell the two apart.
+- **Ordering.** Sends are synchronous, so no patch can precede or overtake a socket's document.
+
+On the client, `@greendrake/rpc`'s `liveDocument` keeps the document. A reconnect is greeted with it afresh, so no re-read is needed.
+
 ### Topics
 
 For state only the sockets that asked for it want — one record's changes, followed by whoever has it open — a socket joins a topic, and the application publishes to the topic:

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@greendrake/rpc-server",
-    "version": "0.8.0",
+    "version": "0.9.0",
     "description": "RPC server over JSON for bun: method dispatch with auth, nonce and Standard Schema argument validation, HTTP and WebSocket transports, connection registry and server push",
     "license": "MIT",
     "author": "Eugene Greendrake <eugene@greendrake.nz>",
@@ -34,7 +34,7 @@
     },
     "prettier": "@greendrake/dev-config/prettier",
     "dependencies": {
-        "@greendrake/rpc": "^0.8.0",
+        "@greendrake/rpc": "^0.9.0",
         "@standard-schema/spec": "^1.1.0"
     },
     "devDependencies": {

package/src/main.ts

@@ -6,5 +6,5 @@ export { bearerToken, keyAuthResolver } from './auth'
 export { memoryNonceStore } from './nonce'
 export { clientIp, httpHandler, type HttpHandlerOptions } from './http'
 export { websocketHandler, type RpcWebSocket, type WebSocketHandlerOptions } from './ws'
-export { ConnRegistry, type Audience, type Connection, type ConnectionData, type EventSink } from './registry'
+export { ConnRegistry, type Audience, type Connection, type ConnectionData, type Document, type EventSink } from './registry'
 export type { Codec, RpcRequest, RpcResponse } from './types'

package/src/registry.ts

@@ -1,5 +1,5 @@
 import type { ServerWebSocket } from 'bun'
-import { identityCodec } from '@greendrake/rpc'
+import { documentOf, identityCodec, mergePatchBetween, type JsonObject } from '@greendrake/rpc'
 import type { AuthInfo, Codec, EventFrame, ServerFrame } from './types'
 
 // Per-socket state, carried as Bun's websocket `data`.
@@ -26,6 +26,15 @@ export interface EventSink {
     broadcastEvent(event: string, data: unknown): void
 }
 
+// What a producer holds of its document: publishing it is the producer's whole part.
+export interface Document {
+    // Publishes the state as `produce` now has it: what changed since it was last
+    // published goes to every socket as one patch, and nothing goes when nothing
+    // changed. So a producer calls it on any change it knows of; a call too many
+    // costs a diff.
+    refresh(): void
+}
+
 // Whether anybody is connected to hear a broadcast, for a producer whose
 // pushes cost something to produce — a reading taken from somewhere slow —
 // and are worth producing only while somebody is there to be told.
@@ -65,6 +74,13 @@ const eventFrame = (event: string, data: unknown): EventFrame => ({
     data
 })
 
+// A document the registry publishes: what it is called on the wire, and the state as
+// last published, which is what an opening socket is sent.
+interface Published {
+    type: string
+    doc: JsonObject
+}
+
 // Every open socket, and the index from principal to the sockets bound as it —
 // a user with three tabs has three. Holding all of them as well as the index
 // is what lets a broadcast reach anonymous connections too, which a per-user
@@ -73,8 +89,11 @@ const eventFrame = (event: string, data: unknown): EventFrame => ({
 // And the topics sockets have joined: subscriptions a method takes out for its
 // caller (see Topics), published to by the application and dropped with the
 // socket.
+//
+// And the documents every socket keeps (see document()).
 export class ConnRegistry implements EventSink, Audience {
     readonly #connections = new Set<Connection>()
+    readonly #documents: Published[] = []
     readonly #presence = new Set<(present: boolean) => void>()
     readonly #byUser = new Map<unknown, Set<Connection>>()
     // Where each connection is currently filed, so a principal that changes
@@ -100,6 +119,9 @@ export class ConnRegistry implements EventSink, Audience {
     register(conn: Connection): void {
         const arriving = !this.#connections.has(conn)
         this.#connections.add(conn)
+        if (arriving) {
+            this.#documents.forEach(published => this.#greet(conn, published))
+        }
         if (arriving && this.#connections.size === 1) {
             this.#presence.forEach(listener => listener(true))
         }
@@ -197,6 +219,38 @@ export class ConnRegistry implements EventSink, Audience {
         this.#connections.forEach(conn => this.send(conn, { type, data }))
     }
 
+    // A state stream: the whole of some state as one JSON object, kept by every
+    // socket. A socket is sent the document as last published when it opens,
+    // `{type, data: {doc}}`, and each change after that as an RFC 7386 merge
+    // patch over it, `{type, data: {patch}}` — objects merge, null removes a
+    // member, anything else replaces — so what it holds is always the document.
+    // A member produced as null is dropped, the same as one left out, a patch
+    // being unable to tell the two apart (`@greendrake/rpc`'s documentOf, which
+    // the client end shares). `produce` runs now, so every greeting has a
+    // document, and again on each refresh. A state that is no JSON object
+    // throws. Sends are synchronous, so no patch can precede or overtake a
+    // socket's document.
+    document(type: string, produce: () => object): Document {
+        const published: Published = { type, doc: documentOf(type, produce()) }
+        this.#documents.push(published)
+        this.#connections.forEach(conn => this.#greet(conn, published))
+        return {
+            refresh: () => {
+                const next = documentOf(type, produce())
+                const patch = mergePatchBetween(published.doc, next)
+                if (!patch) {
+                    return
+                }
+                published.doc = next
+                this.#connections.forEach(conn => this.send(conn, { type, data: { patch } }))
+            }
+        }
+    }
+
+    #greet(conn: Connection, published: Published): void {
+        this.send(conn, { type: published.type, data: { doc: published.doc } })
+    }
+
     get size(): number {
         return this.#connections.size
     }

package/src/ws.ts

@@ -7,8 +7,8 @@ import { ANONYMOUS } from './types'
 import type { AuthInfo, IncomingRequest, RpcResponse } from './types'
 
 export interface WebSocketHandlerOptions {
-    // Run once the socket can be written to. A service that greets a new
-    // connection with its current state does it here.
+    // Run once the socket can be written to, after the registry has greeted it
+    // with every document.
     onOpen?: (conn: Connection) => void
     // Run when the socket has gone, before it leaves the registry.
     onClose?: (conn: Connection) => void
@@ -89,6 +89,11 @@ export const websocketHandler = (dispatcher: Dispatcher, registry: ConnRegistry,
             return server.upgrade(request, { data })
         },
         handlers: {
+            // A socket whose client has stopped reading, once its send buffer
+            // passes Bun's backpressure limit, is closed rather than having its
+            // frames dropped: a frame lost would leave the client silently wrong,
+            // and a closed socket reconnects to be greeted afresh.
+            closeOnBackpressureLimit: true,
             open(conn) {
                 registry.register(conn)
                 options.onOpen?.(conn)