@ultimat3/realtime 9.0.0 → 11.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +96 -7
- package/README.md +33 -9
- package/package.json +3 -3
- package/src/change-buffer.ts +40 -4
- package/src/client-contract.ts +23 -0
- package/src/client.ts +1 -0
- package/src/cursor.ts +13 -7
- package/src/detach.ts +27 -0
- package/src/drain-evictions.ts +31 -0
- package/src/errors.ts +39 -92
- package/src/hooks.ts +38 -5
- package/src/index.ts +4 -1
- package/src/live-query.ts +11 -2
- package/src/local-store.ts +12 -2
- package/src/nats-lib-client.ts +8 -2
- package/src/nats-transport.ts +3 -1
- package/src/pg-array.ts +141 -0
- package/src/pg-connection.ts +15 -1
- package/src/pg-entity-row.ts +7 -7
- package/src/pg-replication.ts +76 -12
- package/src/pg-values.ts +166 -0
- package/src/pgoutput.ts +15 -72
- package/src/realtime-error.ts +28 -0
- package/src/replication-errors.ts +87 -0
- package/src/server-render-client.ts +96 -0
- package/src/server.ts +3 -0
- package/src/socket.ts +15 -1
- package/src/sync-node.ts +24 -26
- package/src/sync-upgrade.ts +15 -6
- package/src/type-pins.ts +12 -1
package/CLAUDE.md
CHANGED
|
@@ -27,6 +27,14 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
27
27
|
type-only export leaves no runtime entry). `errors.ts` is deliberately whole on `.`: every code
|
|
28
28
|
reaches the wire through `toWireError`, so a client must be able to name any of them, and the
|
|
29
29
|
module is already in the client graph via `sync-protocol`.
|
|
30
|
+
- **`errors.ts` is the code TABLE plus the client-reachable refusals; two neighbours hold the rest,
|
|
31
|
+
and every name is still exported from `./errors`** (2026-08-23, at the 500-line ceiling).
|
|
32
|
+
`realtime-error.ts` holds the base class alone and `replication-errors.ts` the four Postgres ones
|
|
33
|
+
— the only codes no browser can reach. The base needs its own module rather than a re-export:
|
|
34
|
+
`extends` runs at module evaluation and imports hoist above it, so a `replication-errors` that
|
|
35
|
+
imported the base back out of `errors.ts` would read it in its temporal dead zone. Neither
|
|
36
|
+
neighbour runs anything at import, which is what keeps `sideEffects` naming `errors.ts` alone
|
|
37
|
+
true — `registerErrorCodes()` stays there, unconditional, and `bun run side-effects` is the check.
|
|
30
38
|
- **`sideEffects` is the ARRAY `["./src/errors.ts"]`, never `false` and never absent.** Absent was
|
|
31
39
|
what made the failure above unrecoverable — with no field a bundler must assume every module has
|
|
32
40
|
effects, so nothing was tree-shaken and `nats` came along with `useLive`. Measured, not guessed:
|
|
@@ -35,6 +43,15 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
35
43
|
**The array alone would have fixed the build** and is not why the split exists: tree-shaking is
|
|
36
44
|
a bundler's discretion, `export * from` or a namespace import defeats it, and "the client entry
|
|
37
45
|
cannot reach the bus" is a contract rather than an optimisation.
|
|
46
|
+
- **Two entries means a specifier naming a third does not resolve, and a `fix:` is pasted.**
|
|
47
|
+
`local-store.ts`'s `X_NOT_IMPLEMENTED` told the caller to import `createOpfsLocalStore` from
|
|
48
|
+
`@ultimat3/realtime/browser` — a subpath `exports` has never declared — so the one instruction
|
|
49
|
+
the refusal carried ended in a module-resolution failure, in the package whose own rules cite
|
|
50
|
+
axiom 4. Its alternative, `persist: false` on the query, was the same defect twice: `query()`
|
|
51
|
+
does not accept `persist` either. `fix-specifier.test.ts` is the build error — every
|
|
52
|
+
`@ultimat3/realtime/<subpath>` written in shipped source must be a key of `exports`, comments
|
|
53
|
+
included, because a comment naming a subpath that does not exist is the next fix line's source.
|
|
54
|
+
It cannot see WHICH names a fix promises, so the OPFS one is pinned by name beside it.
|
|
38
55
|
- **`@ultimat3/realtime/server` needs its own `paths` entry in `tsconfig.base.json`**, beside
|
|
39
56
|
`@ultimat3/admin/dev`'s. `@ultimat3/*` maps `realtime/server` to `packages/realtime/server/src`,
|
|
40
57
|
which does not exist, and the root program has no `node_modules/@ultimat3` symlink to fall back
|
|
@@ -196,6 +213,17 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
196
213
|
other node renders for a full TTL. During a **rolling restart** that is every room showing each
|
|
197
214
|
user twice for up to 30s, beside the same client's reconnection under a new socket id. One
|
|
198
215
|
`evict(socket, code, reason)`, and every path that ends a socket without a callback takes it.
|
|
216
|
+
- **A `drain()` WAITS for the presence leaves it started; a `close` callback cannot** (2026-08-23).
|
|
217
|
+
`teardown` returns those promises as well as `detach`ing them: Bun's `close` callback is
|
|
218
|
+
synchronous, so there the detach is the whole of it — but a drain has no callback behind it and
|
|
219
|
+
is the one path that can wait. It did not: `release()`, `hub.close()` and the process's exit all
|
|
220
|
+
ran under N·M in-flight KV writes, so every other node rendered every drained member for a full
|
|
221
|
+
TTL — the same rolling-restart double vision `evict` exists to prevent, reached the long way
|
|
222
|
+
round. `evictInChunks` (`drain-evictions.ts`) evicts `DRAIN_EVICT_CHUNK` sockets, waits out what
|
|
223
|
+
they started, then takes the next: one synchronous loop over 50,000 sockets opens a quarter of a
|
|
224
|
+
million writes on one connection at the exact moment the fleet is already restarting.
|
|
225
|
+
`allSettled`, never `all` — a leave that fails is a member left to its TTL, which is what the
|
|
226
|
+
write meant when nobody waited for it at all, and it must not hold up the sockets behind it.
|
|
199
227
|
- **The idle sweep exists, is armed by `start()`, and its budget is an APPLICATION one.**
|
|
200
228
|
`SocketRegistry.sweepIdle` had no caller for as long as it existed, so `touch()`, `idleFor` and
|
|
201
229
|
the 120s default decided nothing and `idleTimeoutMs` was unreachable from `createSyncNode`. The
|
|
@@ -259,6 +287,21 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
259
287
|
whose non-key columns happen to be NULL sends the bytes a FULL one does, so counting absent keys
|
|
260
288
|
would undercount exactly the rows a policy is most likely to misjudge. A hard refusal in the
|
|
261
289
|
`x verify` step is the follow-up and lives in `@ultimat3/cli`.
|
|
290
|
+
- **The replication session pins its own output formats, `As of 2026-08-23`.** Postgres sends every
|
|
291
|
+
WAL value as TEXT and `pg-values.ts` reads a `timestamptz` by matching postgres' ISO spelling,
|
|
292
|
+
keeping the raw text when it does not match — deliberately, because a wrong instant is worse than
|
|
293
|
+
a string. That makes the SERVER's `DateStyle` load-bearing: `SQL`, `German` or `Postgres` sends
|
|
294
|
+
every timestamp down the fallback, the shared window holds `Date`s while the patch holds text,
|
|
295
|
+
`compareValues` falls to string comparison, and one edit to one column jumps its row to the top
|
|
296
|
+
of every `orderBy('createdAt','desc')` feed for every subscriber — the exact defect the decode
|
|
297
|
+
exists to close, re-opened by a GUC. `pg-connection.ts` therefore sends
|
|
298
|
+
`options: '-c datestyle=ISO -c intervalstyle=postgres -c extra_float_digits=3'` in the startup
|
|
299
|
+
packet, byte for byte what postgres' own logical-replication client sends
|
|
300
|
+
(`libpqwalreceiver.c`) — which is why a walsender accepts it. On **every** session this class
|
|
301
|
+
opens, not only the replicating one: one session shape is one thing to reason about, and the
|
|
302
|
+
advisory-lock connection is the same class. A server that refuses one answers `ErrorResponse` at
|
|
303
|
+
startup, so the replicator fails to boot with the server's own words rather than mis-sorting a
|
|
304
|
+
feed behind a warning nobody reads. `pg-connection.test.ts` pins the packet.
|
|
262
305
|
- A change lsn is `<16 hex commit position><8 hex row position in that transaction>`. Never order by
|
|
263
306
|
either half alone: the commit lsn repeats within a transaction, and per-record WAL positions are
|
|
264
307
|
not monotonic across transactions. Never make it depend on wall time, the entity list or a process
|
|
@@ -301,6 +344,38 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
301
344
|
- One registered `LiveClient` per app (`setLiveClient`), and every hook reads it through that seam —
|
|
302
345
|
no hook takes a client argument, and an unregistered one is `X_LIVE_CLIENT_MISSING`, never a
|
|
303
346
|
lazily-constructed default.
|
|
347
|
+
- **A DOM is the whole of the question, and it decides what "no client" MEANS** (2026-08-23, issue
|
|
348
|
+
#271). Deliberately the same rule, the same probe and the same words as `@ultimat3/ui`'s
|
|
349
|
+
`solid()`: with a DOM, a hook that finds no registration is a real bug — the app entry forgot
|
|
350
|
+
`setLiveClient` and every live query on the page is dead — so it stays `X_LIVE_CLIENT_MISSING`.
|
|
351
|
+
Without one there is no socket a client could have been registered *for*; that is a **server
|
|
352
|
+
render**, and it gets `serverRenderLiveClient()`. Before it, a page whose whole body read a live
|
|
353
|
+
query could not server-render at all: `useConnection()` threw and the route answered 500, and the
|
|
354
|
+
existing `hasLiveClient()` guard could not help — it only serves a component that already has a
|
|
355
|
+
static fallback written. `hasLiveClient()` still answers **false** on the server, on purpose,
|
|
356
|
+
because that is exactly what such a component is asking.
|
|
357
|
+
- **The server client serves the first render and opens no socket, so it holds nothing per
|
|
358
|
+
request.** One instance per process, and that is only safe because `useLive` on it registers
|
|
359
|
+
nothing: a client that kept a registration per call would grow by one entry per request forever
|
|
360
|
+
and pin a row window with each. `state()` is **`loading`**, never `offline` and never `live` — the
|
|
361
|
+
rows arrive over a socket this render does not have, so the page's own loading fallback is what
|
|
362
|
+
the document carries. `offline` would be read as a settled answer (`state() !== 'loading'` is the
|
|
363
|
+
gate a page writes), so an empty result set would render "you have no posts" for a feed that has
|
|
364
|
+
some. `connected` is `true` for the mirror-image reason: `useConnection().offline` is a banner
|
|
365
|
+
about this visitor's connectivity, and the request being served is the proof it is up. Everything
|
|
366
|
+
that can only mean "talk to the socket" — `mutate`, `drain` — refuses with
|
|
367
|
+
`X_LIVE_SERVER_RENDER`, because a dropped mutation looks exactly like one that happened.
|
|
368
|
+
- **The hook seam takes `LiveClientLike`, not the `LiveClient` class, and that is a measurement.**
|
|
369
|
+
A value import of the class from `hooks.ts` put the whole connection lifecycle — heartbeat, topic
|
|
370
|
+
book, mutation sender, wire protocol, backoff — into every island that calls `useLive`: a
|
|
371
|
+
`useLive`-only browser chunk went **8,368 B → 26,571 B**. Against the structural shape it is
|
|
372
|
+
9,356 B, and the ~1 kB is the server client and its refusal. `type-pins.ts`
|
|
373
|
+
(`_LiveClientSatisfiesTheHookSeam`) is what keeps the two in step.
|
|
374
|
+
- **A server render that renders is not a live page.** A page component never runs in a browser —
|
|
375
|
+
only an `island()` module does — so `useLive` in a page body server-renders its loading branch and
|
|
376
|
+
nothing replaces it unless that route ships an island that registers a client. The server client
|
|
377
|
+
removes the 500; it does not make a page live, and it must never be described as if it did.
|
|
378
|
+
`examples/dummy`'s `/feed` is exactly that state and its own header says so.
|
|
304
379
|
- Anything a component reads is a **getter or an accessor**, never a value snapshotted at hook time:
|
|
305
380
|
a plain field cannot re-render. `MutatorLike.local` is declared with method syntax so an
|
|
306
381
|
`@ultimat3/action` `Mutator` assigns with no cast — a function-typed property would not.
|
|
@@ -523,12 +598,20 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
523
598
|
what releases the change subscription carrying them. `drain()` + `stop()` are the `close` phase.
|
|
524
599
|
Registered with no phase, both landed in `close` and the node upgraded new websockets until the
|
|
525
600
|
very end. `listenSyncNode` unregisters both on `stop()`.
|
|
526
|
-
- **Readiness
|
|
527
|
-
that passed the
|
|
528
|
-
lands, and the `accept` phase is over by the time it reaches
|
|
529
|
-
a node the load balancer has already stopped routing to, so
|
|
530
|
-
socket count are therefore **functions** on `UpgradeDeps`,
|
|
531
|
-
|
|
601
|
+
- **Readiness AND the connection cap are asked twice, because `authenticate` is app code with an
|
|
602
|
+
await in it.** A request that passed the checks at the top of `handleUpgrade` can be parked in a
|
|
603
|
+
token service when SIGTERM lands, and the `accept` phase is over by the time it reaches
|
|
604
|
+
`server.upgrade` — one more socket on a node the load balancer has already stopped routing to, so
|
|
605
|
+
nothing takes it over. `ready` and the socket count are therefore **functions** on `UpgradeDeps`,
|
|
606
|
+
not values read once.
|
|
607
|
+
**`socketCount()` was the half that was read once and never re-asked** (2026-08-23), which is the
|
|
608
|
+
same staleness with a worse blast radius: a restart storm dials every client of a dead node at
|
|
609
|
+
this one at once and each parks in the token service having passed the cap while the node still
|
|
610
|
+
held nothing, so `maxConnections: 2` with ten parked upgrades took **ten** sockets — reproduced,
|
|
611
|
+
`upgraded 10, shed 0`. Sound because there is no await between the recheck and `server.upgrade`,
|
|
612
|
+
and the count moves INSIDE it: Bun runs `websocket.open` synchronously there, which is where
|
|
613
|
+
`sockets.add` runs. The recheck sheds with the same 503 + `retry-after-ms` and takes no second
|
|
614
|
+
`tryAccept()`: that budget was spent.
|
|
532
615
|
- **A client `send` that returned is not an acknowledgement.** A browser `WebSocket.send` on a
|
|
533
616
|
CLOSING socket discards the frame and returns normally, so a drained mutation is `inflight` until
|
|
534
617
|
the server settles it or `requeueInflight` returns it. Only `pending` is sendable, `drain()` is one
|
|
@@ -690,6 +773,7 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
690
773
|
| `sync-frames.ts` | what a RECEIVED frame does to server state — the node's inbound surface, and the mirror of `client-frames.ts` |
|
|
691
774
|
| `sync-upgrade.ts` | the node's HTTP surface: `/healthz`, `/readyz`, load shedding, and the authenticated upgrade — `WsData` and `UpgradeTarget` are declared with the decision that builds them |
|
|
692
775
|
| `sync-listen.ts` | binding a node to `Bun.serve` and to the shutdown hook — the only `Bun.serve` in the package |
|
|
776
|
+
| `drain-evictions.ts` | evicting every socket a drain holds, in bounded chunks, and waiting out the presence leaves each eviction started |
|
|
693
777
|
| `query-window.ts` | the shared pre-policy window per query id: built once, read once for N subscribers, and replaced when it is known to be wrong |
|
|
694
778
|
| `client-frames.ts` | what a RECEIVED frame does to client state, and `ClientFrameTarget` — the only inbound surface the client exposes. The mirror of `sync-frames.ts` |
|
|
695
779
|
| `client-harness-fixture.ts` | the injected socket + scheduler + harness both client suites drive. Excluded from the tarball |
|
|
@@ -716,10 +800,15 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
716
800
|
## Commands
|
|
717
801
|
|
|
718
802
|
```
|
|
719
|
-
bun test
|
|
803
|
+
bun test packages/realtime/src # from the REPO ROOT, never from packages/realtime
|
|
720
804
|
bun run typecheck
|
|
721
805
|
```
|
|
722
806
|
|
|
807
|
+
**The root is not a preference.** `bunfig.toml`'s preload installs `@ultimat3/testing`'s matchers
|
|
808
|
+
and Bun reads `bunfig.toml` from the cwd, so `bun test` inside this directory loads none and six
|
|
809
|
+
tests fail on a missing matcher — this package's suite reading red for the shell it was run in.
|
|
810
|
+
CI's `package` job spawns `bun test packages/<pkg>` with `cwd` at the root for the same reason.
|
|
811
|
+
|
|
723
812
|
Changing a frame shape means adding a fixture to `sync-protocol.test.ts` — the round-trip test
|
|
724
813
|
fails if a kind has no fixture — and bumping `PROTOCOL_VERSION` **when the change makes an old
|
|
725
814
|
frame unreadable in either direction**. An *additive optional* field (`snapshot.entity`, 2026-08)
|
package/README.md
CHANGED
|
@@ -81,13 +81,23 @@ Client names — `useLive`, `liveHookFor`, `LiveClient`, `OfflineQueue`, `Rebase
|
|
|
81
81
|
| wire | `.` | `PROTOCOL_VERSION`, `encode`, `decode`, `Frame` |
|
|
82
82
|
| halves | both | `LiveClient` on `.`; `createSyncNode` / `listenSyncNode` (`sync` role) on `./server` |
|
|
83
83
|
| a socket's identity | `./server` | `SyncAuthenticator`, `SyncGrant`, `GrantBook`, `sweepGrants`, `DEFAULT_REAUTH_INTERVAL_MS` |
|
|
84
|
-
| hooks | `.` | `setLiveClient`, `useLive`, `useConnection`, `useMutation`, `useMutationQueue` |
|
|
84
|
+
| hooks | `.` | `setLiveClient`, `useLive`, `useConnection`, `useMutation`, `useMutationQueue`, `hasLiveClient` |
|
|
85
|
+
| the server render's client | `.` | `serverRenderLiveClient` — what a hook falls back to with no DOM; `LiveClientLike` is the shape both it and `LiveClient` satisfy |
|
|
85
86
|
| the typed projection | `.` | `liveHookFor` — one query bound to one named hook |
|
|
86
87
|
|
|
87
88
|
## The four hooks
|
|
88
89
|
|
|
89
90
|
Register the client once, in the app entry. Every hook reads it from there — no hook takes a client
|
|
90
|
-
argument, and one that runs before the registration is `X_LIVE_CLIENT_MISSING`,
|
|
91
|
+
argument, and one that runs **in a browser** before the registration is `X_LIVE_CLIENT_MISSING`,
|
|
92
|
+
never a default.
|
|
93
|
+
|
|
94
|
+
**A server render is not a missing registration.** With no DOM there is no socket a client could
|
|
95
|
+
have been registered for, so every hook falls back to `serverRenderLiveClient()`: `useLive` answers
|
|
96
|
+
`state() === 'loading'` with no rows, `useConnection()` reports online, both queue counts are `0`,
|
|
97
|
+
and `mutate` / `drain` refuse with `X_LIVE_SERVER_RENDER`. The page renders its own loading branch
|
|
98
|
+
and a hydrating island takes over. `hasLiveClient()` still answers `false` there, which is what a
|
|
99
|
+
component with a static fallback is asking. `useLive` in a page BODY is not made live by this — a
|
|
100
|
+
page component never runs in a browser; put the live half in an `island()`.
|
|
91
101
|
|
|
92
102
|
```ts
|
|
93
103
|
setLiveClient(new LiveClient({ signal: createSignal, connect, buildId, store, queue }));
|
|
@@ -203,6 +213,9 @@ back in a `finally`, and releasing twice is a no-op.
|
|
|
203
213
|
|
|
204
214
|
The accept budget bounds the accept **rate**; `maxConnections` bounds the **count**, and they are
|
|
205
215
|
two different attacks — 500 accepts/s held open with one keepalive each is 1.8M sockets an hour.
|
|
216
|
+
Both the count and `/readyz` are re-asked **after** `authenticate` resolves and immediately before
|
|
217
|
+
`server.upgrade`: awaiting app code is awaiting a token service, and a restart storm parks every
|
|
218
|
+
client of a dead node in there at once, each having passed a cap the node has since filled.
|
|
206
219
|
The frame budget is per socket and checked at the top of the frame router, before anything a frame
|
|
207
220
|
can reach: a subscribe frame is a database read, a presence write and a fleet-wide publish, and one
|
|
208
221
|
authenticated socket is the cheapest foothold there is.
|
|
@@ -256,7 +269,7 @@ that outlasts the deploy. The design confronts it with exactly two paths and no
|
|
|
256
269
|
| **snapshot** | out of window, past `maxLagMs`, or past `reconnectBudget` | one bounded indexed query |
|
|
257
270
|
|
|
258
271
|
A `LiveCursor` is `lsn` + result-set `digest` + last-seen `ids` + `count`. The digest is
|
|
259
|
-
order-sensitive
|
|
272
|
+
order-sensitive and server-side (see `cursor.ts#digestOf` for why a client cannot reproduce one); the ids let a delta be re-filtered per subscriber, because
|
|
260
273
|
the retained window stores **pre-policy** patches. `resumeFrom()` picks the path,
|
|
261
274
|
`shouldResnapshot()` explains it, and the budget is a cost model in patch-equivalents
|
|
262
275
|
(`snapshotCost: 250` = "replaying 250 patches costs a snapshot") so the expensive path is *chosen*,
|
|
@@ -348,7 +361,9 @@ wire twice by a reconnect that raced an ack.
|
|
|
348
361
|
would put the write the server refused back on the screen. Idempotent for a key the log does not
|
|
349
362
|
hold, because a denial can arrive twice and tier 2 records nothing to undo.
|
|
350
363
|
- **A delta resume leaves the digest unverified** (`DIGEST_UNVERIFIED`). Only a snapshot re-establishes
|
|
351
|
-
it.
|
|
364
|
+
it. The digest is the SERVER's own — nothing on the client reproduces it, and `verifyDigest()`,
|
|
365
|
+
which claimed otherwise and had no caller, is deleted (2026-08-23). What detects drift on the
|
|
366
|
+
client is the server's `desynced` mark and the re-snapshot it triggers.
|
|
352
367
|
- **Backpressure drops patch frames.** That is safe *only* because a re-snapshot is cheap: the drop
|
|
353
368
|
is recorded on the socket (`desynced`) and the next delivery re-snapshots rather than diverging.
|
|
354
369
|
- **A dropped CHANNEL frame is not safe, and is not repaired.** A topic has no cursor, no mark and
|
|
@@ -410,6 +425,11 @@ wire twice by a reconnect that raced an ack.
|
|
|
410
425
|
keeps its patch stream**. The `close` phase is `drain()` then `stop()`. Registered with no phase it
|
|
411
426
|
all landed in `close`, and until that ran the node went on upgrading new websockets onto a process
|
|
412
427
|
that was going away. Both hooks are unregistered by the listener's `stop()`.
|
|
428
|
+
- **`drain()` resolves once the presence leaves have LANDED**, not once they have been started —
|
|
429
|
+
in bounded chunks of sockets, so a node holding tens of thousands does not open a write per topic
|
|
430
|
+
per socket in one go. Started and not waited for, the process could exit with them still on the
|
|
431
|
+
wire, and every other node would render every drained member for a full TTL: the rolling-restart
|
|
432
|
+
double vision the leave exists to prevent.
|
|
413
433
|
- **A full presence frame is capped** at `maxMembers` (256) and carries `total`, so a 5,000-person
|
|
414
434
|
room renders "and 4,744 others" instead of shipping 5,000 members to every joiner. The set itself
|
|
415
435
|
is never capped — the sweep differences it — and one node per topic runs that sweep, elected
|
|
@@ -504,10 +524,13 @@ wire twice by a reconnect that raced an ack.
|
|
|
504
524
|
running half: one per change delivered off a relation that is not FULL, so the decisions it
|
|
505
525
|
actually cost are countable rather than silent. A hard refusal at `x verify` time is the
|
|
506
526
|
follow-up.
|
|
507
|
-
- Tier 3's OPFS SQLite store is browser-only
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
527
|
+
- Tier 3's OPFS SQLite store is browser-only, is **not built**, and throws `X_NOT_IMPLEMENTED` on
|
|
528
|
+
call. `createOpfsLocalStore` is exported from `.` and stays there when it ships — there is no
|
|
529
|
+
third entry to wait for, and the refusal used to name one (`@ultimat3/realtime/browser`, a
|
|
530
|
+
subpath `exports` never declared). `MemoryLocalStore`, beside it on `.`, implements the full
|
|
531
|
+
journal/rollback/replay semantics today and is what the refusal's `fix:` names. It holds
|
|
532
|
+
membership and the journal; the row values are the client's one `IdentityMap`, which is what a
|
|
533
|
+
browser store has to inherit rather than re-implement.
|
|
511
534
|
- The identity map is **per client**, in memory, and it is not a query cache: it answers "what is
|
|
512
535
|
row X now", never "have I run this query before". Nothing evicts by time or size — a row lives
|
|
513
536
|
exactly as long as a window or a table holds it.
|
|
@@ -518,7 +541,8 @@ wire twice by a reconnect that raced an ack.
|
|
|
518
541
|
`X_PROTOCOL_VERSION` · `X_CURSOR_STALE` ·
|
|
519
542
|
`X_REBASE_CONFLICT` · `X_TRANSPORT_UNAVAILABLE` · `X_TRANSPORT_PROTOCOL` ·
|
|
520
543
|
`X_REPLICATION_FAILED` · `X_REPLICATION_PROTOCOL` · `X_REPLICATOR_SLOT_HELD` ·
|
|
521
|
-
`X_LIVE_CLIENT_MISSING` · `
|
|
544
|
+
`X_LIVE_CLIENT_MISSING` · `X_LIVE_SERVER_RENDER` · `X_LIVE_QUERY_UNKNOWN` ·
|
|
545
|
+
`X_LIVE_REPLICA_IDENTITY` ·
|
|
522
546
|
`X_SOCKET_UNAUTHENTICATED` · `X_SOCKET_AUTH_UNAVAILABLE` · `X_NOT_IMPLEMENTED`
|
|
523
547
|
|
|
524
548
|
Topics deny by default: a topic with no matching guard is forbidden. An authz hole is not a config
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/realtime",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "11.0.0",
|
|
4
4
|
"description": "Three-tier realtime: channels, live queries, local-first sync — one protocol, one mutator shape",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -36,8 +36,8 @@
|
|
|
36
36
|
"test": "bun test"
|
|
37
37
|
},
|
|
38
38
|
"dependencies": {
|
|
39
|
-
"@ultimat3/core": "
|
|
40
|
-
"@ultimat3/query": "
|
|
39
|
+
"@ultimat3/core": "11.0.0",
|
|
40
|
+
"@ultimat3/query": "11.0.0",
|
|
41
41
|
"nats": "2.29.3"
|
|
42
42
|
}
|
|
43
43
|
}
|
package/src/change-buffer.ts
CHANGED
|
@@ -51,6 +51,12 @@ function patchBytes(patch: RowPatch): number {
|
|
|
51
51
|
|
|
52
52
|
export class RingChangeBuffer implements ResumeSource {
|
|
53
53
|
readonly #rings = new Map<string, Ring>();
|
|
54
|
+
/**
|
|
55
|
+
* Query hashes whose ring was dropped, so the ring the NEXT change builds knows it does not
|
|
56
|
+
* carry the history before it. A `Set` of ids and not a `Map` of lsns: what the next ring is
|
|
57
|
+
* complete from is its own first patch, which only that patch can name.
|
|
58
|
+
*/
|
|
59
|
+
readonly #forgotten = new Set<string>();
|
|
54
60
|
readonly #capacity: number;
|
|
55
61
|
readonly #maxQueries: number;
|
|
56
62
|
readonly #maxBytesPerQuery: number;
|
|
@@ -71,7 +77,17 @@ export class RingChangeBuffer implements ResumeSource {
|
|
|
71
77
|
|
|
72
78
|
append(qid: string, patch: RowPatch): void {
|
|
73
79
|
const existing = this.#rings.get(qid);
|
|
74
|
-
|
|
80
|
+
// A ring RE-CREATED after a `forget` is complete only from this patch onward: everything before
|
|
81
|
+
// it went with the ring, and — on the `unsubscribe` path — the entry went too, so the changes
|
|
82
|
+
// in between were never appended at all. `evictedThrough` at its own first lsn is what makes
|
|
83
|
+
// `since` refuse a cursor from before that, which it could not do while the field came back
|
|
84
|
+
// `null` and every earlier cursor read as in-window on a ring that had held none of it.
|
|
85
|
+
const reborn = this.#forgotten.delete(qid);
|
|
86
|
+
const ring: Ring = existing ?? {
|
|
87
|
+
patches: [],
|
|
88
|
+
bytes: 0,
|
|
89
|
+
evictedThrough: reborn ? patch.lsn : null,
|
|
90
|
+
};
|
|
75
91
|
ring.patches.push(patch);
|
|
76
92
|
const cost = patchBytes(patch);
|
|
77
93
|
ring.bytes += cost;
|
|
@@ -124,9 +140,29 @@ export class RingChangeBuffer implements ResumeSource {
|
|
|
124
140
|
*/
|
|
125
141
|
forget(qid: string): void {
|
|
126
142
|
const ring = this.#rings.get(qid);
|
|
127
|
-
if (
|
|
128
|
-
|
|
129
|
-
|
|
143
|
+
if (ring !== undefined) {
|
|
144
|
+
this.#bytes -= ring.bytes;
|
|
145
|
+
this.#rings.delete(qid);
|
|
146
|
+
}
|
|
147
|
+
// The qid is remembered, the patches are not — and unconditionally, because the ring being
|
|
148
|
+
// absent is not the history being intact. Both callers lose history here and neither could say
|
|
149
|
+
// so: `LiveQueryRegistry.unsubscribe` drops the ENTRY, so every change until the next
|
|
150
|
+
// subscriber is never appended at all, and the LRU fires on a query that still has LIVE
|
|
151
|
+
// subscribers. Either way the next `append` was building a ring that reported itself complete
|
|
152
|
+
// from the beginning of time, so a client reconnecting inside `maxLagMs` folded a partial patch
|
|
153
|
+
// list onto a stale window with `shouldResnapshot` answering `in-window` and nothing marked
|
|
154
|
+
// desynced — permanently divergent on a healthy socket. What the tombstone costs in exchange is
|
|
155
|
+
// a resume that could have been a delta taking the snapshot path; that is one bounded read, and
|
|
156
|
+
// it is the direction this package errs in everywhere else.
|
|
157
|
+
this.#forgotten.add(qid);
|
|
158
|
+
// Bounded like everything else here: insertion-ordered, so the oldest tombstone goes first.
|
|
159
|
+
// Losing one costs a resume that could have been a delta; keeping them unbounded costs memory
|
|
160
|
+
// a client-chosen input mints at will.
|
|
161
|
+
while (this.#forgotten.size > this.#maxQueries) {
|
|
162
|
+
const oldest = this.#forgotten.values().next();
|
|
163
|
+
if (oldest.done === true) break;
|
|
164
|
+
this.#forgotten.delete(oldest.value);
|
|
165
|
+
}
|
|
130
166
|
}
|
|
131
167
|
|
|
132
168
|
get queryCount(): number {
|
package/src/client-contract.ts
CHANGED
|
@@ -55,6 +55,29 @@ export interface MutatorRef<T extends TableMap = TableMap> {
|
|
|
55
55
|
readonly conflict?: ConflictStrategy;
|
|
56
56
|
}
|
|
57
57
|
|
|
58
|
+
/**
|
|
59
|
+
* What the HOOKS need a client to be — every member `hooks.ts` reads, and not one more.
|
|
60
|
+
*
|
|
61
|
+
* A structural interface rather than the `LiveClient` class, and the reason is measured: a value
|
|
62
|
+
* import of that class from the hook seam put the whole connection lifecycle (heartbeat, topic
|
|
63
|
+
* book, mutation sender, wire protocol, backoff) into every island that calls `useLive`, taking a
|
|
64
|
+
* `useLive`-only browser chunk from 8,368 B to 26,571 B. The server render's client
|
|
65
|
+
* (`server-render-client.ts`) satisfies this and imports no lifecycle at all, so the browser pays
|
|
66
|
+
* nothing for a shape only the server uses. `type-pins.ts` pins that `LiveClient` still satisfies
|
|
67
|
+
* it, so a member added there and not here is a build error rather than a hook that cannot see it.
|
|
68
|
+
*/
|
|
69
|
+
export interface LiveClientLike<T extends TableMap = TableMap> {
|
|
70
|
+
readonly signal: SignalFactory;
|
|
71
|
+
readonly queue: OfflineQueue | undefined;
|
|
72
|
+
readonly connected: boolean;
|
|
73
|
+
readonly reconnectAt: () => number | null;
|
|
74
|
+
readonly appUpdateAvailable: () => string | null;
|
|
75
|
+
useLive<R extends Row>(query: LiveQueryRef, input: JsonValue): LiveHandle<R>;
|
|
76
|
+
mutate(mutator: MutatorRef<T>, input: JsonValue, key?: string): Promise<void>;
|
|
77
|
+
drain(): Promise<void>;
|
|
78
|
+
onQueueChange(listener: () => void): () => void;
|
|
79
|
+
}
|
|
80
|
+
|
|
58
81
|
export interface LiveClientOptions<T extends TableMap = TableMap> {
|
|
59
82
|
readonly signal: SignalFactory;
|
|
60
83
|
/** Called for every connect attempt; returning a fresh socket keeps reconnect logic here. */
|
package/src/client.ts
CHANGED
package/src/cursor.ts
CHANGED
|
@@ -105,17 +105,23 @@ export function makeCursor(
|
|
|
105
105
|
};
|
|
106
106
|
}
|
|
107
107
|
|
|
108
|
-
/**
|
|
108
|
+
/**
|
|
109
|
+
* FNV-1a over `id:row` pairs in result order — order-sensitive, so a re-sort is detected.
|
|
110
|
+
*
|
|
111
|
+
* **Server-side only, and it is not reproducible by a client.** `verifyDigest()` used to sit here
|
|
112
|
+
* and was DELETED (2026-08-23): it was documented as "how a client detects drift", had no caller
|
|
113
|
+
* outside its own test, and could not have had one. Three reasons, any one of them fatal.
|
|
114
|
+
* `canonicalJson` tags a `Date` as `Date(<epoch>)` while the client holds the ISO string
|
|
115
|
+
* `JSON.stringify` sent it. A delta-resumed cursor carries `DIGEST_UNVERIFIED`, so the check
|
|
116
|
+
* answered `false` for every cursor a delta produced — which is the only state drift can be
|
|
117
|
+
* detected in. And `identity-map.ts` MERGES columns across queries on purpose, so a client's row
|
|
118
|
+
* for one id is legitimately a superset of the row any single snapshot sent: an app with two reads
|
|
119
|
+
* over one entity would have reported permanent drift.
|
|
120
|
+
*/
|
|
109
121
|
export function digestOf(rows: readonly Row[]): string {
|
|
110
122
|
return fnv1a(rows.map((row) => `${row.id}:${canonicalJson(row)}`).join(';'));
|
|
111
123
|
}
|
|
112
124
|
|
|
113
|
-
/** Client-side drift check: a mismatch after delta resumes is a request for a fresh snapshot. */
|
|
114
|
-
export function verifyDigest(cursor: LiveCursor, rows: readonly Row[]): boolean {
|
|
115
|
-
if (cursor.digest === DIGEST_UNVERIFIED) return false;
|
|
116
|
-
return cursor.digest === digestOf(rows);
|
|
117
|
-
}
|
|
118
|
-
|
|
119
125
|
export function shouldResnapshot(
|
|
120
126
|
cursor: LiveCursor,
|
|
121
127
|
available: readonly RowPatch[] | null,
|
package/src/detach.ts
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
// Work nobody is waiting on: a presence leave from a synchronous close, a sweep on a timer, a
|
|
2
|
+
// fanout off the change bus. Split out of `sync-node.ts` at the 500-line ceiling, and it is the
|
|
3
|
+
// natural seam — the function closes over nothing the node holds.
|
|
4
|
+
|
|
5
|
+
import { logger, renderThrowable, reportError } from '@ultimat3/core';
|
|
6
|
+
|
|
7
|
+
/**
|
|
8
|
+
* It reaches the bus or a policy, so it can fail; failing must not take a socket or the process
|
|
9
|
+
* with it, and must not be silent either, or "the room still shows someone who left" and "that
|
|
10
|
+
* change reached nobody" have nothing to read. `operation` stays low cardinality so the monitor
|
|
11
|
+
* can group on it; the topic or entity goes in `at`.
|
|
12
|
+
*
|
|
13
|
+
* `renderThrowable` and never `String(error)`: this is the one frame whose whole job is not to
|
|
14
|
+
* throw, and `String()` on a null-prototype throwable raises inside it — the detach's own `catch`,
|
|
15
|
+
* with nothing above it to answer. `channel.ts` already imports it for the same reason.
|
|
16
|
+
*/
|
|
17
|
+
export function detach(work: Promise<unknown>, operation: string, at?: string): void {
|
|
18
|
+
void work.catch((error: unknown) => {
|
|
19
|
+
logger.error(`${operation} failed`, {
|
|
20
|
+
...(at === undefined ? {} : { at }),
|
|
21
|
+
error: renderThrowable(error),
|
|
22
|
+
});
|
|
23
|
+
// Nobody is awaiting this, so the log is the only trace it leaves — and a log is not a signal
|
|
24
|
+
// anyone is paged on. The bus is this node's dependency, never the client's.
|
|
25
|
+
reportError(error, { source: 'realtime', scope: { operation } });
|
|
26
|
+
});
|
|
27
|
+
}
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
// Evicting every socket a drain holds, in bounded chunks, and waiting out what each eviction put
|
|
2
|
+
// on the bus. Split out of `sync-node.ts` at the 500-line ceiling; the loop closes over nothing the
|
|
3
|
+
// node holds, which is the same seam `detach.ts` took.
|
|
4
|
+
|
|
5
|
+
/** Sockets evicted before the leaves they started are awaited. See `evictInChunks`. */
|
|
6
|
+
export const DRAIN_EVICT_CHUNK = 128;
|
|
7
|
+
|
|
8
|
+
/**
|
|
9
|
+
* Every socket released, then the writes those releases started, then the next chunk.
|
|
10
|
+
*
|
|
11
|
+
* Chunked rather than one pass, because an eviction's presence leave is a write per TOPIC per
|
|
12
|
+
* SOCKET: a node holding 50,000 sockets in a handful of rooms each would open a quarter of a
|
|
13
|
+
* million KV writes on one connection in a single synchronous loop, which is a self-inflicted
|
|
14
|
+
* thundering herd on the bus at the exact moment the fleet is already restarting.
|
|
15
|
+
*
|
|
16
|
+
* `allSettled`, never `all`: a leave that fails is a member left to its TTL — the same degradation
|
|
17
|
+
* the write has when nobody waits for it at all — and it must not stop the sockets behind it from
|
|
18
|
+
* being released. The failure itself is already reported, by the `detach` the eviction path
|
|
19
|
+
* attached before handing the promise back here.
|
|
20
|
+
*/
|
|
21
|
+
export async function evictInChunks<Socket>(
|
|
22
|
+
sockets: readonly Socket[],
|
|
23
|
+
evict: (socket: Socket) => readonly Promise<unknown>[],
|
|
24
|
+
chunkSize: number = DRAIN_EVICT_CHUNK,
|
|
25
|
+
): Promise<void> {
|
|
26
|
+
for (let start = 0; start < sockets.length; start += chunkSize) {
|
|
27
|
+
const pending: Promise<unknown>[] = [];
|
|
28
|
+
for (const socket of sockets.slice(start, start + chunkSize)) pending.push(...evict(socket));
|
|
29
|
+
await Promise.allSettled(pending);
|
|
30
|
+
}
|
|
31
|
+
}
|