@ultimat3/realtime 20.2.1 → 22.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +300 -952
- package/README.md +192 -131
- package/package.json +7 -4
- package/src/apply-patches.ts +1 -1
- package/src/boot.ts +72 -0
- package/src/browser-socket.ts +42 -0
- package/src/changefeed.ts +14 -1
- package/src/channel-authz.ts +52 -0
- package/src/channel-bridge.ts +34 -0
- package/src/channel-decl.ts +155 -0
- package/src/channel-describe.ts +35 -0
- package/src/channel-gaps.ts +57 -0
- package/src/channel-logs.ts +134 -0
- package/src/channel-presence.ts +68 -0
- package/src/channel-records.ts +87 -0
- package/src/channel-ref.ts +83 -0
- package/src/channel-registry.ts +35 -0
- package/src/channel-render.ts +37 -0
- package/src/channel-ring.ts +75 -0
- package/src/channel-wire.ts +66 -0
- package/src/channel.ts +147 -157
- package/src/client-channels.ts +359 -0
- package/src/client-contract.ts +35 -65
- package/src/client-frames.ts +42 -110
- package/src/client.ts +150 -195
- package/src/cursor.ts +7 -2
- package/src/errors.ts +55 -101
- package/src/frame-lanes.ts +9 -5
- package/src/idb-fake.ts +133 -0
- package/src/idb-types.ts +48 -0
- package/src/index.ts +80 -75
- package/src/json.ts +5 -0
- package/src/live-contract.ts +5 -0
- package/src/live-definition.ts +15 -4
- package/src/live-fanout.ts +81 -6
- package/src/live-query.ts +11 -0
- package/src/live-record-type.ts +19 -0
- package/src/live-replicator.ts +160 -0
- package/src/live-rows.ts +70 -67
- package/src/local-store-idb.ts +324 -0
- package/src/matcher-bridge.ts +5 -0
- package/src/nats-fake.ts +10 -1
- package/src/nats-jetstream.ts +36 -14
- package/src/nats-transport.ts +2 -2
- package/src/offline-queue.ts +85 -39
- package/src/outbox-slot.ts +31 -0
- package/src/page-errors.ts +124 -0
- package/src/page-outbox.ts +312 -0
- package/src/page-socket.ts +139 -0
- package/src/page-store.ts +138 -0
- package/src/pg-entity-row.ts +37 -184
- package/src/pg-preflight.ts +24 -2
- package/src/pg-replication.ts +28 -8
- package/src/pg-wire.ts +51 -15
- package/src/pgoutput.ts +37 -2
- package/src/policy-fake.ts +14 -0
- package/src/presence.ts +17 -9
- package/src/query-window.ts +38 -21
- package/src/reactivity.ts +70 -0
- package/src/realtime-error.ts +1 -1
- package/src/record-await.ts +102 -0
- package/src/record-key.ts +34 -0
- package/src/record-names.ts +45 -0
- package/src/record-persister.ts +156 -0
- package/src/record-store.ts +364 -0
- package/src/record-synced.ts +100 -0
- package/src/record-tx.ts +145 -0
- package/src/replicator.ts +20 -4
- package/src/server.ts +10 -11
- package/src/socket-drops.ts +30 -0
- package/src/socket-engine.ts +344 -0
- package/src/socket-host.ts +225 -0
- package/src/socket-idle.ts +21 -0
- package/src/socket-port.ts +55 -0
- package/src/socket-routes.ts +170 -0
- package/src/socket.ts +91 -49
- package/src/subscriber-gate.ts +92 -3
- package/src/sync-auth.ts +2 -2
- package/src/sync-frames.ts +41 -114
- package/src/sync-meta.ts +42 -0
- package/src/sync-node-contract.ts +100 -0
- package/src/sync-node.ts +26 -114
- package/src/sync-protocol.ts +63 -212
- package/src/sync-worker.ts +12 -0
- package/src/thundering-herd.ts +31 -12
- package/src/transport-env.ts +55 -14
- package/src/type-pins.ts +30 -61
- package/src/use-channel.ts +88 -0
- package/src/use-connection.ts +59 -0
- package/src/use-mutation.ts +227 -0
- package/src/use-query.ts +260 -0
- package/src/use-record.ts +121 -0
- package/src/wire-channel.ts +116 -0
- package/src/wire-read.ts +86 -0
- package/src/wire-version.ts +44 -0
- package/src/client-mutations.ts +0 -114
- package/src/client-topics.ts +0 -54
- package/src/hooks.ts +0 -277
- package/src/identity-map.ts +0 -141
- package/src/local-store.ts +0 -241
- package/src/query-hook.ts +0 -56
- package/src/rebase.ts +0 -263
- package/src/server-render-client.ts +0 -96
package/CLAUDE.md
CHANGED
|
@@ -6,927 +6,308 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
6
6
|
|
|
7
7
|
| May import | Must not |
|
|
8
8
|
|---|---|
|
|
9
|
-
| `@ultimat3/core`, `@ultimat3/query` | anything tier 4+ (`render`, `pwa`, `mcp`, `ui`, `cli`) |
|
|
9
|
+
| `@ultimat3/core`, `@ultimat3/query`, `@ultimat3/entity` (`/record` only, server-side) | anything tier 4+ (`render`, `pwa`, `mcp`, `ui`, `cli`) |
|
|
10
10
|
| `@ultimat3/policy` **only via** `@ultimat3/query`'s `guard` | a second authz path of any kind |
|
|
11
|
-
| — | `solid-js` (
|
|
12
|
-
| `nats` (the one external dependency, pinned exact) — from `nats-lib-client.ts
|
|
11
|
+
| — | `solid-js` (each island bundle installs its own signal factory: `installRealtime`) |
|
|
12
|
+
| `nats` (the one external dependency, pinned exact) — from `nats-lib-client.ts` and no other file | `nats` from anywhere else. Every other file is written against the port in `nats-client.ts` |
|
|
13
13
|
|
|
14
|
-
##
|
|
14
|
+
## Entries and bundling
|
|
15
15
|
|
|
16
|
-
- **Two entries
|
|
17
|
-
|
|
18
|
-
`
|
|
19
|
-
`thundering-herd
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
`
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
neighbour runs anything at import, which is what keeps `sideEffects` naming `errors.ts` alone
|
|
37
|
-
true — `registerErrorCodes()` stays there, unconditional, and `bun run side-effects` is the check.
|
|
38
|
-
- **`sideEffects` is the ARRAY `["./src/errors.ts"]`, never `false` and never absent.** Absent was
|
|
39
|
-
what made the failure above unrecoverable — with no field a bundler must assume every module has
|
|
40
|
-
effects, so nothing was tree-shaken and `nats` came along with `useLive`. Measured, not guessed:
|
|
41
|
-
`bun run side-effects --explain --json` prints what the tree actually does at import time.
|
|
42
|
-
`false` would drop `registerErrorCodes()` and every `REALTIME_ERROR_TITLES` entry with it.
|
|
43
|
-
**The array alone would have fixed the build** and is not why the split exists: tree-shaking is
|
|
44
|
-
a bundler's discretion, `export * from` or a namespace import defeats it, and "the client entry
|
|
45
|
-
cannot reach the bus" is a contract rather than an optimisation.
|
|
46
|
-
- **Two entries means a specifier naming a third does not resolve, and a `fix:` is pasted.**
|
|
47
|
-
`local-store.ts`'s `X_NOT_IMPLEMENTED` told the caller to import `createOpfsLocalStore` from
|
|
48
|
-
`@ultimat3/realtime/browser` — a subpath `exports` has never declared — so the one instruction
|
|
49
|
-
the refusal carried ended in a module-resolution failure, in the package whose own rules cite
|
|
50
|
-
axiom 4. Its alternative, `persist: false` on the query, was the same defect twice: `query()`
|
|
51
|
-
does not accept `persist` either. `fix-specifier.test.ts` is the build error — every
|
|
52
|
-
`@ultimat3/realtime/<subpath>` written in shipped source must be a key of `exports`, comments
|
|
53
|
-
included, because a comment naming a subpath that does not exist is the next fix line's source.
|
|
54
|
-
It cannot see WHICH names a fix promises, so the OPFS one is pinned by name beside it.
|
|
55
|
-
- **`@ultimat3/realtime/server` needs its own `paths` entry in `tsconfig.base.json`**, beside
|
|
56
|
-
`@ultimat3/admin/dev`'s. `@ultimat3/*` maps `realtime/server` to `packages/realtime/server/src`,
|
|
57
|
-
which does not exist, and the root program has no `node_modules/@ultimat3` symlink to fall back
|
|
58
|
-
through — so `scripts/**` (which the root `tsc -b` compiles) reports `TS2307` without it. The
|
|
59
|
-
workspace packages resolve through their own `node_modules` and never needed it, which is what
|
|
60
|
-
makes the failure look local to `scripts/`.
|
|
16
|
+
- **Two entries; a name lives in exactly ONE.** `.` is the client half (`use-*`, `page-*`,
|
|
17
|
+
`reactivity`, `record-*`, `client*`, `browser-socket`, `live-rows`, `apply-patches`,
|
|
18
|
+
`offline-queue`, `sync-protocol`, `json`, `cursor`, `errors`, the client half of
|
|
19
|
+
`thundering-herd`). `./server` (`server.ts`) is everything touching `nats`, Postgres, the sync node,
|
|
20
|
+
the hub, the registry or the fanout (`nats` requires `stream/web`, which a browser build cannot
|
|
21
|
+
link). `packages/cli/src/realtime-browser-barrel.test.ts` bundles a `useLive`-only entry;
|
|
22
|
+
`barrel-split.test.ts` refuses a name exported from both. `./boot` and `./sync-worker` are the page
|
|
23
|
+
boot and the SharedWorker entry.
|
|
24
|
+
- **`errors.ts` is the code table plus the client-reachable refusals**; `realtime-error.ts` holds
|
|
25
|
+
the base class alone (an `extends` must not read it in a TDZ) and `replication-errors.ts` the
|
|
26
|
+
Postgres ones. Every name is still exported from `./errors`. A browser path loads `page-errors.ts`,
|
|
27
|
+
never the table (`page-errors-bundle.test.ts`).
|
|
28
|
+
- **`sideEffects` is the ARRAY `["./src/errors.ts"]`**, never `false` (drops `registerErrorCodes()`)
|
|
29
|
+
and never absent. `bun run side-effects` is the check.
|
|
30
|
+
- **Every `@ultimat3/realtime/<subpath>` written in shipped source must be a key of `exports`**,
|
|
31
|
+
comments included — `fix-specifier.test.ts`.
|
|
32
|
+
- **`@ultimat3/realtime/server` needs its own `paths` entry in `tsconfig.base.json`** (the wildcard
|
|
33
|
+
maps it to a directory that does not exist; `scripts/**` reports `TS2307` without it).
|
|
34
|
+
- A browser file imports core from `@ultimat3/core/page` (`packages/core/src/page-bundle.test.ts`,
|
|
35
|
+
`packages/query/src/client-bundle.test.ts`).
|
|
61
36
|
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
`
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
- **
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
- **
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
- What `liveQueryDefinition` caches per query id is the compiled source, the shape, the matcher and
|
|
141
|
-
the shared row window. What it must never cache is a decision. It builds that shared half with
|
|
142
|
-
`enforce: false` **on purpose**: a source compiled under the first subscriber's authority and
|
|
143
|
-
then keyed by query id is that subscriber's entitlements becoming everyone's. `authorize` is
|
|
144
|
-
still the subscribe-time decision and still runs per socket.
|
|
145
|
-
- Every policy call in `live-query.ts` takes a `Subscriber`. That is the enforcement: there is no
|
|
146
|
-
path through the gate that reads a query id and no actor.
|
|
147
|
-
- **The row policy always sees the *whole* row from the shared window, never a partial patch — and
|
|
148
|
-
a window that does not hold the row is not a partial one.** An update patch carries the changed
|
|
149
|
-
columns plus the id, so merging it onto nothing and calling that a row hands `visible` a
|
|
150
|
-
`undefined` for every column the change did not touch: fail-closed for `row.ownerId === actor.id`,
|
|
151
|
-
and a leak for every `!row.private`. So a patch whose row the window does not hold is **withheld**
|
|
152
|
-
— dropped, or the one `delete` frame that tells a subscriber holding it that it is gone. It is
|
|
153
|
-
neither `rowsDenied` nor `gateFailures`: nothing decided anything, the window simply *is* the
|
|
154
|
-
result set. The one path that could meet an empty window is a delta resume onto an entry nothing
|
|
155
|
-
has read yet, and `subscribe` fills it first (`entry.lsn === ''`) rather than withholding
|
|
156
|
-
everything — conditional on purpose, because re-reading per resuming subscriber is exactly the
|
|
157
|
-
cost a delta resume exists to skip in a restart storm.
|
|
158
|
-
- **A denial is a decision; everything else is a failure, and the two never share an answer.** A
|
|
159
|
-
bare `catch { return false }` in the row gate read a dead pool as "you may not see this row" —
|
|
160
|
-
the rows left the screen, `live.rows_denied` counted the drop, and the outage shipped as a
|
|
161
|
-
permission change. `visibleWithPolicy` matches `QueryDeniedError` (the only thing `guard` throws
|
|
162
|
-
for a decision) and rethrows the rest; `subscriber-gate.ts` and `reauthorize` ask
|
|
163
|
-
`isPolicyDenial(error)` instead, because `authorize` and `visible` are caller-supplied functions
|
|
164
|
-
and the answer has to come off the error's code. What a failure costs is decided per surface: a
|
|
165
|
-
snapshot **raises** out of `subscribe` (a short result set is indistinguishable from a correct
|
|
166
|
-
one), a delivery desyncs that **one** subscriber and lets the fanout finish, and a `reauthorize`
|
|
167
|
-
keeps the subscription — destroying it would report a timeout as a revoked grant, and a client
|
|
168
|
-
does not resubscribe to a denial. Every failure is counted as `gateFailures` and reported through
|
|
169
|
-
`onGateFailed`, never through `onRowDenied`: an alert fires on one of them.
|
|
170
|
-
- **One serial lane per query id, and it is the only thing that orders a fanout.** Nothing upstream
|
|
171
|
-
does: `sync` fires `void registry.deliver(change)` straight off the bus subscription, so two
|
|
172
|
-
changes arriving back to back both start, both write `entry.rows`/`entry.lsn`, and both await
|
|
173
|
-
their way through a per-subscriber gate in between. Unordered, a subscriber is handed lsn 2 and
|
|
174
|
-
then asked to fold lsn 1 on top of it, its cursor rewound to 1 — a reconnect then replays what it
|
|
175
|
-
already applied, over newer state, and the row stays at the older value. `WindowLock` (`run`)
|
|
176
|
-
gives each entry a FIFO lane. `deliver` *enters* every lane before awaiting any of them, and no
|
|
177
|
-
fanout ever takes a second lane, so holding all of them at once cannot be a cycle — and two
|
|
178
|
-
deliveries queue onto each query id in call order, which is what makes "per query id, not per
|
|
179
|
-
node" true. Awaiting one entry before entering the next was two bugs in one line: one slow policy
|
|
180
|
-
pass set the whole node's pace, and a lane that threw ended the loop, so every entry behind it
|
|
181
|
-
missed the change with **nobody desynced** — the silent divergence `markDesynced` exists to
|
|
182
|
-
prevent. A lane that fails now desyncs its own subscribers and the first failure still reaches
|
|
183
|
-
the caller, but it costs one query id. The lane chains on a settled shadow of each task: one
|
|
184
|
-
fanout that threw must not reject every fanout behind it.
|
|
185
|
-
- **Two reads of one entry are ordered by a READ GENERATION, never by an lsn.** `QueryEntry.lsn`
|
|
186
|
-
is optional — a definition with no lsn provider answers `''` from every snapshot — so the
|
|
187
|
-
never-backwards rule expressed purely in lsn terms read `'' >= ''` as "newer" and let the older
|
|
188
|
-
of two concurrent reads land on top of the newer one's window. The interleaving: a cold
|
|
189
|
-
subscriber issues P1; the change stream skips a sequence and `registry.invalidate()` marks the
|
|
190
|
-
entry; a second cold subscriber forces P2, which **clears `stale` on the way in**; P2 lands with
|
|
191
|
-
the post-gap rows; P1 lands last and overwrites them. `stale` is false, so `fanoutChange`'s
|
|
192
|
-
repair never fires, the next change patches the pre-gap window and re-snapshots every desynced
|
|
193
|
-
subscriber out of it — permanently stale on a healthy socket, which is the exact outcome `stale`
|
|
194
|
-
exists to prevent. `entry.generation` is bumped in `startRead` and `entry.applied` records the
|
|
195
|
-
newest read whose rows are on the window: an *identity* check, the same one `startRead` makes on
|
|
196
|
-
`entry.reading` one function down and `packages/cache/src/single-flight.ts:70` makes for the same
|
|
197
|
-
reason. The lsn guard stays beside it for the other question — a read that resolved behind a
|
|
198
|
-
*change* the fanout already folded — because those are two orderings and neither answers the
|
|
199
|
-
other.
|
|
200
|
-
- **The definition's read is once per entry, not once per subscriber.** A cold subscriber arriving
|
|
201
|
-
while another's read is in flight joins that read — N cold subscribers on one query id being N
|
|
202
|
-
reads is the shared window not existing. It is a share, not a cache: the in-flight promise is
|
|
203
|
-
cleared as it settles, so a later subscriber reads current rows rather than a window that has
|
|
204
|
-
been drifting since boot. The result lands **in the lane and never backwards**: a snapshot that
|
|
205
|
-
resolved after a newer change was already fanned out is discarded and its caller is served from
|
|
206
|
-
the newer window, because rewinding hands that subscriber rows the fanout has moved past and a
|
|
207
|
-
cursor behind the change that would have corrected them.
|
|
208
|
-
- The retained change window stores **pre-policy** patches; resume re-filters them per subscriber.
|
|
209
|
-
- A resume is the one gate pass that runs **outside** the lane, and it reads the live window on
|
|
210
|
-
purpose: the window can only have moved forwards, and a row whose grant was revoked in the
|
|
211
|
-
meantime is one that pass must refuse rather than replay from its state at the cursor's lsn.
|
|
37
|
+
## Numbers and ceilings
|
|
38
|
+
|
|
39
|
+
- **Every numeric option is refused when it is not FINITE** — `@ultimat3/core`'s `finiteOption()`.
|
|
40
|
+
`bun run finite-bounds` reads zero for this package, but it cannot see an option with no `??`
|
|
41
|
+
default or where a screen runs, so audit by reading the option interfaces
|
|
42
|
+
(`SubscriptionBook`'s `maxPerTenant` and `openNatsClient`'s `maxReconnectAttempts` were the two it
|
|
43
|
+
missed). `SyncGrant.expiresAt` is app DATA, not an option, and is deliberately unscreened.
|
|
44
|
+
- **A ceiling is refused where the object is BUILT**, never in a per-connection callback:
|
|
45
|
+
`socketCeilings()` (`sync-node-bounds.ts`) runs once in `createSyncNode`; `SyncSocket` keeps its own
|
|
46
|
+
screen because it is exported. `AcceptBudget`, `SyncSocket` and `SocketRegistry` throw
|
|
47
|
+
`X_INVARIANT` at construction on `NaN`/`±Infinity`.
|
|
48
|
+
- **A ceiling per resource, and the wire's are not options** (table in `README.md`): the accept
|
|
49
|
+
budget bounds the rate, `maxConnections` the count, `socket.frameBudget` an open socket (checked
|
|
50
|
+
at the top of `routeFrame` **before `touch()`**), and `FRAME_LIMITS` in `decode` everything a
|
|
51
|
+
client sizes (narrowable, never widenable). `list()` takes a required `max`. `input` is walked
|
|
52
|
+
ITERATIVELY.
|
|
53
|
+
- **Every `SyncSocket` ceiling is reachable from `createSyncNode`**, forwarded as
|
|
54
|
+
`...(x === undefined ? {} : { x })`.
|
|
55
|
+
- **A `SubscriptionLimitError` names the knob**: every throw site passes `knob` explicitly;
|
|
56
|
+
`channel.test.ts` asserts the hub's two against the option names.
|
|
57
|
+
- **Retained memory is bounded by BYTES**: `RingChangeBuffer` keeps the patch count as a replay
|
|
58
|
+
bound and byte budgets as the memory bound. `forget(qid)` runs when the last subscriber goes and
|
|
59
|
+
from `#dropIfUnheld` after a failed cold subscribe (only if the entry is still in the table, unheld
|
|
60
|
+
and has no read in flight).
|
|
61
|
+
- **Every question a hot path asks is indexed**: `SubscriptionBook` keeps `#bySocket` and a
|
|
62
|
+
per-tenant count; `SocketRegistry` keeps `#byTopic`, changed only by `joinTopic`/`leaveTopic`.
|
|
63
|
+
`reauthorize` calls `book.retenant(socket)`.
|
|
64
|
+
- **A cap is a RESERVATION taken before the first await**: `SubscriptionBook.reserve(socket, sid)`
|
|
65
|
+
decides the sid claim, `maxPerSocket` and `maxPerTenant` in one synchronous step;
|
|
66
|
+
`ChannelHub.subscribe` does the same for its caps. Released in a `finally`, idempotently.
|
|
67
|
+
- **Readiness AND the connection cap are functions on `UpgradeDeps`**, re-asked after
|
|
68
|
+
`authenticate` (app code with an await) and right before `server.upgrade`. The recheck sheds with
|
|
69
|
+
the same 503 + `retry-after-ms` and spends no second `tryAccept()`.
|
|
70
|
+
|
|
71
|
+
## Live queries
|
|
72
|
+
|
|
73
|
+
- Policy is evaluated **once per subscriber**, never once per query (`live-query.test.ts`, and
|
|
74
|
+
`live-definition.test.ts` for a real `query({ live: true })`). Every policy call in `live-query.ts`
|
|
75
|
+
takes a `Subscriber`.
|
|
76
|
+
- **A name nothing registered is `X_LIVE_QUERY_UNKNOWN`**, fix `x queries list --json`. The
|
|
77
|
+
registry is never enumerated over the wire.
|
|
78
|
+
- **One build per `(query, input)`**: `target.live()` returns the descriptor and runs the read
|
|
79
|
+
(`LiveQuery.execute`), never memoised.
|
|
80
|
+
- `liveQueryDefinition` caches per query id the compiled source, shape, matcher and shared window —
|
|
81
|
+
never a decision. The shared half is built with `enforce: false` on purpose; `authorize` runs per socket.
|
|
82
|
+
- **The row policy sees the WHOLE row from the shared window.** A patch whose row the window does not
|
|
83
|
+
hold is WITHHELD (not denied, not a failure). A delta resume onto an unread entry fills it first
|
|
84
|
+
(`entry.lsn === ''`).
|
|
85
|
+
- **A denial is a decision; everything else is a failure.** `visibleWithPolicy` matches
|
|
86
|
+
`QueryDeniedError` and rethrows the rest; `subscriber-gate.ts` and `reauthorize` ask
|
|
87
|
+
`isPolicyDenial(error)`. A failed snapshot raises out of `subscribe`; a failed delivery desyncs that
|
|
88
|
+
one subscriber; a failed `reauthorize` keeps the subscription. Failures count as `gateFailures` via
|
|
89
|
+
`onGateFailed`, never `onRowDenied`.
|
|
90
|
+
- **One serial lane per query id (`WindowLock`) is the only thing that orders a fanout.** `deliver`
|
|
91
|
+
enters every lane before awaiting any; no fanout takes a second lane; a lane that fails desyncs its
|
|
92
|
+
own subscribers; lanes chain on a settled shadow of each task.
|
|
93
|
+
- **Two reads of one entry are ordered by a READ GENERATION**, never by an lsn (`QueryEntry.lsn` may
|
|
94
|
+
be `''`): `entry.generation` / `entry.applied` is an identity check. The lsn guard stays for the
|
|
95
|
+
other question — a read that resolved behind a change the fanout already folded.
|
|
96
|
+
- **The definition's read is once per entry**: a cold subscriber joins the in-flight read (a share,
|
|
97
|
+
cleared on settle), and the result lands in the lane and never backwards.
|
|
98
|
+
- **The shared read carries a deadline and frees the SLOT**: `startRead` races it against
|
|
99
|
+
`DEFAULT_READ_DEADLINE_MS` (30 s; `new LiveQueryRegistry({ readDeadlineMs, schedule })`), rejects
|
|
100
|
+
joined callers with `X_TIMEOUT` (never an empty window), and puts `stale` back. No "off" spelling;
|
|
101
|
+
a non-positive or non-finite value takes the default (`query-window.test.ts`).
|
|
102
|
+
- The retained change window stores **pre-policy** patches; resume re-filters them per subscriber,
|
|
103
|
+
outside the lane, against the live window.
|
|
104
|
+
- **`desynced` is a mark with a reader**: the next delivery serves a fresh snapshot out of the shared
|
|
105
|
+
window and only then clears it. **`result.refill` is checked BEFORE the mark** — a guessed window
|
|
106
|
+
never clears a mark (`live-fanout.test.ts`).
|
|
107
|
+
- **A change the window already holds is refused** (`change.lsn <= entry.lsn`, counted as
|
|
108
|
+
`staleChanges`). **A gap is detected**: the replicator stamps `producer` + `seq`; a skipped
|
|
109
|
+
sequence marks every window stale and every subscriber desynced; `refillWindowInLane` replaces it.
|
|
110
|
+
- **A qid is `@ultimat3/query`'s `queryHash(name, input)`**; this package owns no hash
|
|
111
|
+
(`live-contract.test.ts`). The canonical form is core's and injective (`-0` is wire-reachable).
|
|
112
|
+
- **A `sid` is CLIENT data: a subscription is keyed by `(socket, sid)`**
|
|
113
|
+
(`subscription-book.ts` is the only spelling). Reusing a sid the same socket holds is
|
|
114
|
+
`X_SUBSCRIPTION_ID_TAKEN`.
|
|
212
115
|
- Truth is the server. A client is never the merge authority.
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
socket is the leave — presence has no frame of its own in either direction, because a second way
|
|
217
|
-
to say "I am here" is a client that can be subscribed and invisible at the same time.
|
|
218
|
-
- Expiry is silent by design, so the node sweeps on an interval: without it a member whose node
|
|
219
|
-
died is never announced as gone, and the survivors render a cursor that stopped moving. It has to
|
|
220
|
-
be an interval — a sweep only reports members a previous sweep saw, which is how a member that
|
|
221
|
-
joined on another node becomes leavable here at all.
|
|
222
|
-
- **The wire is the library's, the integration is ours, and `nats-client.ts` is the line between
|
|
223
|
-
them.** `nats` is imported by exactly one file — `nats-lib-client.ts` — and everything else in the
|
|
224
|
-
package, the transport and the JetStream bucket and the KV presence set and every test, is written
|
|
225
|
-
against that port. It is what lets the fake be an in-memory bus with *server* semantics instead of
|
|
226
|
-
431 lines of forged wire bytes, and what made deleting 1,019 LOC of framing, parser, PING/PONG and
|
|
227
|
-
session a swap rather than a rewrite ([`docs/idea/18-build-vs-wrap.md`](../../docs/idea/18-build-vs-wrap.md)).
|
|
228
|
-
The consequence that has to be held: **reconnect and re-subscription are the library's job now.** A
|
|
229
|
-
subscription outlives a dropped connection because the client re-establishes it underneath the
|
|
230
|
-
caller, so `NatsTransport` must never re-grow subscription bookkeeping of its own — a map of wanted
|
|
231
|
-
subjects, a dial promise, a loss counter, a rebind loop. Two things re-subscribing is a doubled
|
|
232
|
-
delivery on every reconnect and a subscription the caller's `unsubscribe` no longer reaches — the
|
|
233
|
-
hand-rolled client's lifecycle bugs were deleted with it rather than fixed for exactly that reason.
|
|
234
|
-
What stays above the port is what the library has no opinion on: our thundering-herd jitter, handed
|
|
235
|
-
over as its `reconnectDelayHandler` because spreading a restart herd is the framework's decision,
|
|
236
|
-
and the KV semantics presence needs (`nats-jetstream.ts`, `nats-kv.ts`) — a per-message TTL and a
|
|
237
|
-
batch direct get, which the library's own KV abstraction cannot express.
|
|
238
|
-
- **Nothing leaves `NatsTransport` uncoded, and `#translating` is where that is enforced — added
|
|
239
|
-
2026-08.** `client.publish` and `client.subscribe` are the port's two SYNCHRONOUS calls, and the
|
|
240
|
-
library refuses locally on both: a bad subject, a payload over the server's `max_payload`, a
|
|
241
|
-
connection torn down between the `#ensure` and the call, a permissions violation on the subject.
|
|
242
|
-
A raw `NatsError` escaped `publish()` into `ChannelHub`'s bridge, `SocketRegistry` and the
|
|
243
|
-
replicator — no code, no `fix:`, nothing an operator can act on — while `InProcessTransport`
|
|
244
|
-
answered `X_TRANSPORT_UNAVAILABLE` for its own one refusal. `transport-parity.test.ts` asserts
|
|
245
|
-
both transports in one test, and a third case proves the wrap still DELIVERS: a translation that
|
|
246
|
-
swallowed a working publish would satisfy the two refusal cases and fan out nothing. An
|
|
247
|
-
`UltimateError` passes through untouched — the port raises its own for a closed client, and
|
|
248
|
-
re-wrapping buries the code a caller branches on. `nats-lib-client.ts`'s header claim that every
|
|
249
|
-
failure leaves *there* as an `UltimateError` is still false for those two calls; the translation
|
|
250
|
-
is deliberately in ONE place, and it is the transport, because `connect` is a public injection
|
|
251
|
-
seam and an app-supplied client throws whatever it likes.
|
|
252
|
-
- **Reusing a client whatever its state is a DECISION, not the other half of that bug.** `#ensure`
|
|
253
|
-
hands back a client that is mid-reconnect on purpose: the library is re-establishing that same
|
|
254
|
-
connection and its subscriptions, and a second dial alongside it doubles every delivery. What a
|
|
255
|
-
caller gets from a client whose reconnect budget is spent is a publish that resolves into
|
|
256
|
-
nothing — reported through `onError` by `#watch`, and visible to `/readyz` through `connected`,
|
|
257
|
-
which is where a dead bus is meant to be caught.
|
|
258
|
-
- One place reads `NATS_URL`, and it is `selectTransport` — a boot that resolved the bus itself
|
|
259
|
-
could reach a different one than the container it is standing in for. The KV bucket and the
|
|
260
|
-
presence TTL come back with the transport for the same reason: they are one decision.
|
|
116
|
+
|
|
117
|
+
## The node, the bus and presence
|
|
118
|
+
|
|
261
119
|
- `sync` is stateless: no sticky sessions, nothing on a socket survives a restart.
|
|
262
|
-
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
`
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
`
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
`
|
|
283
|
-
|
|
284
|
-
- **The idle sweep
|
|
285
|
-
|
|
286
|
-
the
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
`
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
`
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
- **
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
reading their own flag — that flag is false for the whole of a start, so an unguarded teardown
|
|
324
|
-
returns "nothing to do" and leaves behind exactly what it was called to release.
|
|
325
|
-
**A start that FAILS hands the lock back, and `running` is set after the feed is pumping**
|
|
326
|
-
(`As of 2026-09-06`). It was set before `await feed.start()`, so a feed that rejected — a slot
|
|
327
|
-
already `active`, a preflight refusal — left this node holding the advisory lock and claiming to
|
|
328
|
-
run with nothing pumping: the takeover loop's next `start()` was answered `true` by the
|
|
329
|
-
`if (running)` guard without re-entering `begin`, and every standby stayed a standby of a slot
|
|
330
|
-
whose holder was not replicating. The release is best-effort, because the feed's failure is the
|
|
331
|
-
one the operator acts on and a session-scoped lock a dead connection cannot release is released
|
|
332
|
-
by Postgres when that session ends.
|
|
333
|
-
- **The replication pump has one way out, and it closes what it held.** Both exits — a decode error
|
|
334
|
-
and `nextCopyData()` returning `undefined`, which is the walsender ending the copy — run `#die`:
|
|
335
|
-
record `stats().failure`, stop the confirm timer, close the connection and null it. Each one left
|
|
336
|
-
behind is a dead replicator claiming to be a live one. A `null` failure answers `/readyz` ready
|
|
337
|
-
for a loop reading no WAL; a live `#running` makes the next `start()` a silent no-op; a retained
|
|
338
|
-
timer keeps telling the walsender a dead stream is keeping up; a retained `#connection` is a
|
|
339
|
-
socket the next `start()` overwrites rather than closes, holding the slot `active`. A `start()`
|
|
340
|
-
that goes live clears the previous failure, and `stop()` awaits the pump even when the connection
|
|
341
|
-
is already gone — `#die` nulls it *before* closing it, so returning early reports a released slot
|
|
342
|
-
to the supervisor that is about to start the next process.
|
|
343
|
-
- **`#pump` *is* the terminal cleanup, so a restart awaits it before it dials.** `#drain` awaits
|
|
344
|
-
`#die` and `#die` awaits `connection.close()`, but `#die` clears `#running` and nulls
|
|
345
|
-
`#connection` before that close settles: a `start()` checking `#running` alone dialled into a
|
|
346
|
-
slot the dead walsender still owned and replaced `#pump` with its own, so the next `stop()`
|
|
347
|
-
awaited only the new pump. `start()` takes the previous pump and awaits it first; its failure
|
|
348
|
-
path calls `stop().catch(() => undefined)` because the boot diagnosis is the one an operator acts
|
|
349
|
-
on and a teardown that also failed must not replace it.
|
|
350
|
-
- **`stop()` releases everything before it reports anything.** A `#confirm` or an `endCopy` that
|
|
351
|
-
threw skipped the close and the pump await, leaking the socket and telling a supervisor the
|
|
352
|
-
teardown was over before it had begun. Every step runs whatever the step before it did, and the
|
|
353
|
-
first failure is rethrown only once the connection is closed and the pump has ended.
|
|
354
|
-
- **`REPLICA IDENTITY FULL` is asked about at preflight, warned about, and counted — never thrown**
|
|
355
|
-
(added 2026-08-19). `pg-replication.ts`'s `#deliver` hands a `delete` its `message.before`,
|
|
356
|
-
and under any identity but FULL that tuple is the KEY COLUMNS ALONE — which `toRow` accepts,
|
|
357
|
-
because it only requires a text `id`. So `bridgeChange` decided "did this row leave the result
|
|
358
|
-
set" from a one-column row, `visible` read `undefined` for every column the identity did not
|
|
359
|
-
carry, and nothing anywhere recorded that it had happened: no emit, no check, and
|
|
360
|
-
`X_LIVE_REPLICA_IDENTITY` existed in neither the source nor the manifest. The check is a fourth
|
|
361
|
-
`connection.query` in `preflight`, and it **must** stay ahead of
|
|
362
|
-
`pg_create_logical_replication_slot` — changing the identity after a slot exists does not reach
|
|
363
|
-
what that slot decodes. It WARNS (`logger.warn(code, { cause, fix, tables })`, the message being
|
|
364
|
-
the code alone) because a throw would stop every app on the default identity from booting, which
|
|
365
|
-
is a worse outcome than the partial rows. `ReplicationStreamStats.partialBefore` is the runtime
|
|
366
|
-
half, read off `PgRelation.replicaIdentity` and never off the tuple: a DEFAULT-identity table
|
|
367
|
-
whose non-key columns happen to be NULL sends the bytes a FULL one does, so counting absent keys
|
|
368
|
-
would undercount exactly the rows a policy is most likely to misjudge. A hard refusal in the
|
|
369
|
-
`x verify` step is the follow-up and lives in `@ultimat3/cli`.
|
|
370
|
-
- **The replication session pins its own output formats, `As of 2026-08-23`.** Postgres sends every
|
|
371
|
-
WAL value as TEXT and `pg-values.ts` reads a `timestamptz` by matching postgres' ISO spelling,
|
|
372
|
-
keeping the raw text when it does not match — deliberately, because a wrong instant is worse than
|
|
373
|
-
a string. That makes the SERVER's `DateStyle` load-bearing: `SQL`, `German` or `Postgres` sends
|
|
374
|
-
every timestamp down the fallback, the shared window holds `Date`s while the patch holds text,
|
|
375
|
-
`compareValues` falls to string comparison, and one edit to one column jumps its row to the top
|
|
376
|
-
of every `orderBy('createdAt','desc')` feed for every subscriber — the exact defect the decode
|
|
377
|
-
exists to close, re-opened by a GUC. `pg-connection.ts` therefore sends
|
|
378
|
-
`options: '-c datestyle=ISO -c intervalstyle=postgres -c extra_float_digits=3'` in the startup
|
|
379
|
-
packet, byte for byte what postgres' own logical-replication client sends
|
|
380
|
-
(`libpqwalreceiver.c`) — which is why a walsender accepts it. On **every** session this class
|
|
381
|
-
opens, not only the replicating one: one session shape is one thing to reason about, and the
|
|
382
|
-
advisory-lock connection is the same class. A server that refuses one answers `ErrorResponse` at
|
|
383
|
-
startup, so the replicator fails to boot with the server's own words rather than mis-sorting a
|
|
384
|
-
feed behind a warning nobody reads. `pg-connection.test.ts` pins the packet.
|
|
385
|
-
- A change lsn is `<16 hex commit position><8 hex row position in that transaction>`. Never order by
|
|
386
|
-
either half alone: the commit lsn repeats within a transaction, and per-record WAL positions are
|
|
387
|
-
not monotonic across transactions. Never make it depend on wall time, the entity list or a process
|
|
388
|
-
counter — a replay must produce byte-identical lsns or at-least-once turns into duplicate delivery.
|
|
389
|
-
- Slot, publication and entity names are interpolated into a replication command, so they are
|
|
390
|
-
checked against `[a-z_][a-z0-9_]*` first. That regex is a security boundary, not a style rule.
|
|
391
|
-
- Same rule on the bus, for the half that is still ours: a bucket name is interpolated into a
|
|
392
|
-
JetStream stream name and its API subjects, so it is checked first (`assertBucket` in
|
|
393
|
-
`nats-jetstream.ts`, `X_TRANSPORT_PROTOCOL`). Subject validation went with the hand-rolled client —
|
|
394
|
-
the library refuses a malformed subject itself, and a second spelling of that rule here is a second
|
|
395
|
-
place it can drift. A presence key or member id is user data, so it is base64url-encoded
|
|
396
|
-
(`encodeToken`) rather than validated — no name is refused for its spelling.
|
|
397
|
-
- **One row value per `(entity, id)` per client, and `identity-map.ts` is the only place one lives.**
|
|
398
|
-
A live window is an ordered list of ids over that map and a local-store table is membership over
|
|
399
|
-
it — neither holds a row of its own, because two components holding two copies of post #7 is the
|
|
400
|
-
bug the map exists to make unrepresentable. A `LiveClient` takes the map off its store when tier 3
|
|
401
|
-
is configured (`options.store.identity`) and builds one otherwise: a second map here would be that
|
|
402
|
-
same duplication, one level up.
|
|
403
|
-
- **The scope is `(entity, id)`, never `id` alone, and the entity comes from the server.** The
|
|
404
|
-
compiled shape's root entity (`live.shape.entity`) is the one name the live path, `ChangeEvent`,
|
|
405
|
-
a mutator's `tx.<table>` and `rebase`'s `ack.entity` all already agree on; a browser cannot derive
|
|
406
|
-
it, because the shape is compiled out of `sql`. It rides on the `snapshot` frame, and a
|
|
407
|
-
subscription that is told no entity keeps its rows under `?query:<name>` — private, colliding with
|
|
408
|
-
nothing. Wrong sharing merges two entities into one row; no sharing only costs a stale view.
|
|
409
|
-
- **`snapshot.entity` is additive, and that is why `PROTOCOL_VERSION` did NOT move.** The bump rule
|
|
410
|
-
exists for a shape change that makes an old frame unreadable. This one is readable both ways — an
|
|
411
|
-
old node omits the field and the client falls back to the private scope, an old client drops it in
|
|
412
|
-
`decode` — so bumping would refuse every in-flight client during a rolling deploy in exchange for
|
|
413
|
-
nothing. An *incompatible* frame change still bumps, and every kind still needs a fixture.
|
|
414
|
-
- **A value is replaced, never mutated, and a write merges columns rather than replacing the row.**
|
|
415
|
-
A mutated row is a render that never happens — the projections hand rows to a signal, which
|
|
416
|
-
compares by reference. And two queries may project different columns of one row, so a snapshot
|
|
417
|
-
from the narrower one must not blank what the wider one is rendering. Only a `delete` removes.
|
|
418
|
-
- **A row lives exactly as long as something holds it.** Every projection retains its ids and
|
|
419
|
-
releases them when it lets go (`RowWindows` on a re-snapshot, a patch, a close; a table on delete
|
|
420
|
-
and rollback). The last release drops the value — without it an infinite scroll retains every row
|
|
421
|
-
it ever saw. It is what lets a rollback of an optimistic insert leave a row a live window still
|
|
422
|
-
holds: the table's membership goes, the row does not.
|
|
423
|
-
- `local(tx, input)` is pure: no I/O, no `Date.now()`, no `Math.random()`. Rebase replays it.
|
|
424
|
-
- One registered `LiveClient` per app (`setLiveClient`), and every hook reads it through that seam —
|
|
425
|
-
no hook takes a client argument, and an unregistered one is `X_LIVE_CLIENT_MISSING`, never a
|
|
426
|
-
lazily-constructed default.
|
|
427
|
-
- **A DOM is the whole of the question, and it decides what "no client" MEANS** (2026-08-23, issue
|
|
428
|
-
#271). Deliberately the same rule, the same probe and the same words as `@ultimat3/ui`'s
|
|
429
|
-
`solid()`: with a DOM, a hook that finds no registration is a real bug — the app entry forgot
|
|
430
|
-
`setLiveClient` and every live query on the page is dead — so it stays `X_LIVE_CLIENT_MISSING`.
|
|
431
|
-
Without one there is no socket a client could have been registered *for*; that is a **server
|
|
432
|
-
render**, and it gets `serverRenderLiveClient()`. Before it, a page whose whole body read a live
|
|
433
|
-
query could not server-render at all: `useConnection()` threw and the route answered 500, and the
|
|
434
|
-
existing `hasLiveClient()` guard could not help — it only serves a component that already has a
|
|
435
|
-
static fallback written. `hasLiveClient()` still answers **false** on the server, on purpose,
|
|
436
|
-
because that is exactly what such a component is asking.
|
|
437
|
-
- **The server client serves the first render and opens no socket, so it holds nothing per
|
|
438
|
-
request.** One instance per process, and that is only safe because `useLive` on it registers
|
|
439
|
-
nothing: a client that kept a registration per call would grow by one entry per request forever
|
|
440
|
-
and pin a row window with each. `state()` is **`loading`**, never `offline` and never `live` — the
|
|
441
|
-
rows arrive over a socket this render does not have, so the page's own loading fallback is what
|
|
442
|
-
the document carries. `offline` would be read as a settled answer (`state() !== 'loading'` is the
|
|
443
|
-
gate a page writes), so an empty result set would render "you have no posts" for a feed that has
|
|
444
|
-
some. `connected` is `true` for the mirror-image reason: `useConnection().offline` is a banner
|
|
445
|
-
about this visitor's connectivity, and the request being served is the proof it is up. Everything
|
|
446
|
-
that can only mean "talk to the socket" — `mutate`, `drain` — refuses with
|
|
447
|
-
`X_LIVE_SERVER_RENDER`, because a dropped mutation looks exactly like one that happened.
|
|
448
|
-
- **The hook seam takes `LiveClientLike`, not the `LiveClient` class, and that is a measurement.**
|
|
449
|
-
A value import of the class from `hooks.ts` put the whole connection lifecycle — heartbeat, topic
|
|
450
|
-
book, mutation sender, wire protocol, backoff — into every island that calls `useLive`: a
|
|
451
|
-
`useLive`-only browser chunk went **8,368 B → 26,571 B**. Against the structural shape it is
|
|
452
|
-
9,356 B, and the ~1 kB is the server client and its refusal. `type-pins.ts`
|
|
453
|
-
(`_LiveClientSatisfiesTheHookSeam`) is what keeps the two in step.
|
|
454
|
-
- **A server render that renders is not a live page.** A page component never runs in a browser —
|
|
455
|
-
only an `island()` module does — so `useLive` in a page body server-renders its loading branch and
|
|
456
|
-
nothing replaces it unless that route ships an island that registers a client. The server client
|
|
457
|
-
removes the 500; it does not make a page live, and it must never be described as if it did.
|
|
458
|
-
`examples/dummy`'s `/feed` is exactly that state and its own header says so.
|
|
459
|
-
- Anything a component reads is a **getter or an accessor**, never a value snapshotted at hook time:
|
|
460
|
-
a plain field cannot re-render. `MutatorLike.local` is declared with method syntax so an
|
|
461
|
-
`@ultimat3/action` `Mutator` assigns with no cast — a function-typed property would not.
|
|
462
|
-
- `useLive`'s thunk input is read once, at subscribe time. There is no reactive runtime here to
|
|
463
|
-
re-run it, and pretending otherwise would be a silently stale subscription.
|
|
464
|
-
- Every subscription handle client code gets back — `LiveHandle` (`useLive`'s return, and
|
|
465
|
-
`LiveRows` one layer up through the hook), `Unsubscribe` (`client.subscribe(topic, …)`'s return)
|
|
466
|
-
— is `Disposable`. `[Symbol.dispose]` is the exact same function reference as `unsubscribe`,
|
|
467
|
-
never a second implementation that could drift from it, so `using sub = client.useLive(...)` and
|
|
468
|
-
`sub.unsubscribe()` are one teardown path either way. Pinned in `type-pins.ts`
|
|
469
|
-
(`_LiveHandleIsDisposable`, `_LiveRowsIsDisposable`, `_UnsubscribeIsDisposable`) so a refactor
|
|
470
|
-
that drops the member fails the build, not a call site months later.
|
|
471
|
-
- `liveHookFor(query)` is the typed projection the wiki promises as `useLiveFeed({ orgId })`. It
|
|
472
|
-
**binds** `useLive` — it never re-implements a subscribe path, because two of those is two places
|
|
473
|
-
a subscription can be opened wrong. It names `Query`'s shape structurally (`LiveQuerySource`)
|
|
474
|
-
rather than importing `@ultimat3/query` as a value: a hook is browser code, and a value import
|
|
475
|
-
would pull the server's read path into the bundle.
|
|
476
|
-
- The query's name is read **per call**, never captured at bind time. `export const useLiveFeed =
|
|
477
|
-
liveHookFor(liveFeed)` runs at import; `registerQueries()` stamps the name later, at boot.
|
|
478
|
-
- Type claims about the hook go in `type-pins.ts`, never in a `.test.ts` — `tsconfig.json` excludes
|
|
479
|
-
test files, so `tsc -b` never reads one and an assertion written there can never fail.
|
|
480
|
-
- **`backoffDelay` is `@ultimat3/core`'s, and `attempt + 1` is the whole of the seam** (`As of
|
|
481
|
-
2026-08-23`). A client counts its first reconnect as attempt **0** and core counts the first wait
|
|
482
|
-
as attempt **1**, so dropping the shift doubles every reconnect delay in the framework —
|
|
483
|
-
silently, and only under the load this file exists to survive.
|
|
484
|
-
`thundering-herd-core-parity.test.ts` pins it with the numbers (`[500, 1_000, 2_000, 4_000]`) and
|
|
485
|
-
with 17,280 comparisons across every jitter mode, base, cap, factor, attempt and roll.
|
|
486
|
-
`JitterMode` is core's type re-exported and `Rng` is core's `Random` — never a second spelling of
|
|
487
|
-
either. `drainPlan()` and `AcceptBudget` are NOT backoff and keep their own arithmetic: a slot in
|
|
488
|
-
a spread and a token bucket's refusal delay are different questions. `bun run flight-copies` is the
|
|
489
|
-
guard: a second curve-and-jitter function anywhere in `packages/*/src` is a build error, matched
|
|
490
|
-
on the literal shape rather than the name.
|
|
491
|
-
- **The SHARED window read carries a deadline, and it frees the SLOT — it cannot cancel the read**
|
|
492
|
-
(`As of 2026-08-23`). One `definition.snapshot` that never settled pinned `entry.reading` for the
|
|
493
|
-
life of the process, and every later cold subscriber joined a promise nothing would ever resolve:
|
|
494
|
-
one wedged read took every future subscriber of that query id with it. `startRead` now races the
|
|
495
|
-
read against `entry.schedule(…, entry.readDeadlineMs)`, default `DEFAULT_READ_DEADLINE_MS` (30s),
|
|
496
|
-
injectable per entry and through `new LiveQueryRegistry({ readDeadlineMs, schedule })`. Three
|
|
497
|
-
rules, each with its own test in `query-window.test.ts` and each proven by mutation:
|
|
498
|
-
- **A race, not just an eviction.** Freeing the slot alone leaves every caller ALREADY joined
|
|
499
|
-
awaiting a promise nothing settles; they are told instead, with `X_TIMEOUT`.
|
|
500
|
-
- **`X_TIMEOUT`, not a silent empty window.** A superseded read is discarded silently because a
|
|
501
|
-
strictly better window already exists to serve from; a timed-out read has none, and serving
|
|
502
|
-
`rows: []` as the whole result set is the exact fault `live-query.ts`'s `#read` refuses. The
|
|
503
|
-
rejecting-read path (`readSnapshot`'s catch) is the shape this matches, not the superseded one.
|
|
504
|
-
- **`stale` is put back**, for `readSnapshot`'s reason: a read that did not answer must not leave
|
|
505
|
-
the window looking authoritative.
|
|
506
|
-
There is **no "off" spelling** — a shared read with no deadline is the defect itself — and a `0`,
|
|
507
|
-
negative, `NaN` or `Infinity` value falls back to the default rather than to "now".
|
|
508
|
-
- The client owns its own reconnect: a closed socket arms **one** timer through the injected
|
|
509
|
-
`Scheduler`, and that timer calls `connect()`. `reconnectAt` is the render half and never the
|
|
510
|
-
mechanism — publishing it without arming anything is exactly the bug that shipped. Rules that
|
|
511
|
-
hold the arming together: `onClose` nulls `#socket` (a retained dead socket makes `#send` a
|
|
512
|
-
silent no-op), it only schedules when nothing is armed (a `reconnect` frame arms the node's
|
|
513
|
-
spread slot *before* closing, and a local backoff would overwrite it), and `close()` cancels —
|
|
514
|
-
a client whose owner is gone must stop dialling, while `connect()` starts it over. A close speaks
|
|
515
|
-
only for **its own** socket: `onClose` returns before touching any state when `#socket` is no
|
|
516
|
-
longer the socket that closed, because a replaced socket closing late must not mark the live
|
|
517
|
-
connection offline or arm a backoff behind it — and `close()` therefore reports its subscriptions
|
|
518
|
-
offline itself. A dial that throws inside the timer arms the next attempt and is **reported
|
|
519
|
-
through `onError`** (default `console.error`; never `logger`, whose writer is `process.stderr`
|
|
520
|
-
and this is browser code): a socket constructor may refuse, one refusal ending the chain is the
|
|
521
|
-
same outage as never arming, and nothing awaits a timer — a throw out of one is an uncaught
|
|
522
|
-
exception that can kill the process that was going to retry. Only the timer owns the chain and
|
|
523
|
-
only the timer reports — a `connect()` the app called itself throws to the app and arms nothing.
|
|
524
|
-
- **A `sid` is CLIENT data, so a subscription is keyed by `(socket, sid)` — never by `sid` alone.**
|
|
525
|
-
`LiveQueryRegistry.unsubscribe(socketId, sid)` and `.subscription(socketId, sid)` both take the
|
|
526
|
-
owner. Keyed by the sid alone, socket B reusing socket A's sid overwrote A's slot: A's
|
|
527
|
-
subscription stayed in its query entry's `subscribers` map with nothing able to reach it, so
|
|
528
|
-
`unsubscribeSocket(A)` freed nothing, `subscribers.size` never hit zero, and the entry's matcher
|
|
529
|
-
and shared window were pinned for the process's life while every change fanned out to a dead
|
|
530
|
-
socket. A `{op:'drop', sid}` frame from B ended A's stream with no error on either side.
|
|
531
|
-
`sync-node` passes `socket.id` on the drop path for that reason. Reusing a sid the SAME socket
|
|
532
|
-
already holds is `X_SUBSCRIPTION_ID_TAKEN` — refused rather than replaced, because replacing is
|
|
533
|
-
the strand. `subscription-book.ts` owns that identity and is the only place it is spelled: the
|
|
534
|
-
query entry's own `subscribers` map takes the same composite key, so one `unsubscribe` reaches
|
|
535
|
-
both by one identity.
|
|
536
|
-
- **`connect()` closes the socket it is replacing, and a frame speaks only for its own socket.**
|
|
537
|
-
A remount calling `connect()` on a live client left the previous socket open: its `onMessage`
|
|
538
|
-
kept folding patches into the live registrations, and the node held two sockets for one client —
|
|
539
|
-
double presence membership, double fanout — until the tab closed. `#socket` is nulled before the
|
|
540
|
-
close so the corpse's `onClose` takes its early return, and `onMessage` carries the same identity
|
|
541
|
-
guard `onClose` already had.
|
|
542
|
-
- **The grant is recorded BEFORE `server.upgrade`, and released on the path that never opens.**
|
|
543
|
-
Bun runs `websocket.open` SYNCHRONOUSLY inside `server.upgrade` and does not return until it has
|
|
544
|
-
(bun 1.4.0), and `open` is where `sync-node` reads the `GrantBook` for the socket's actor.
|
|
545
|
-
Recorded on the line after the upgrade — which it was — every authenticated socket on a real node
|
|
546
|
-
carried `actor: null`: `ChannelHub.#authorize` denied every topic with `X_TOPIC_FORBIDDEN`,
|
|
547
|
-
`authorize`/`visible` decided about nobody, `maxPerTenant` never applied and `hello.actorId` was
|
|
548
|
-
null. It did not self-repair, because `GrantBook.expired()` skips a grant with no `expiresAt` —
|
|
549
|
-
the shape `authenticate: async () => ({ actor })` produces. `onUngranted` is what makes the
|
|
550
|
-
correct order safe (only a `close` callback deletes a grant, and an upgrade that never took gets
|
|
551
|
-
no callback); it is REQUIRED on `UpgradeDeps`, so a second host of `handleUpgrade` cannot forget
|
|
552
|
-
it. The harness is half the rule: `sync-node-auth.test.ts`'s `upgradeTarget()` returned `true`
|
|
553
|
-
without ever calling `open`, so the bug was invisible to every test here — it now opens the socket
|
|
554
|
-
inside `upgrade()`, the way Bun does.
|
|
555
|
-
- **Every `socket.send` on the node reads its answer, and what a `false` costs is decided per
|
|
556
|
-
frame.** A subscribe reply is REPAIRABLE and was the silent one: `registry.subscribe` has already
|
|
557
|
-
seated the subscription and cleared its desync mark, so a dropped snapshot left the server
|
|
558
|
-
believing a client holding no rows was in sync, and every later change reached it as a patch
|
|
559
|
-
folded onto nothing — on a healthy socket, forever. It is marked desynced, exactly as
|
|
560
|
-
`live-fanout` marks a lost patch. A `rebase`/`ack` has nothing to mark (the node keeps no
|
|
561
|
-
per-mutation state) and a client only returns an `inflight` mutation to its queue when the
|
|
562
|
-
connection dies, so an undeliverable settlement **closes the socket**: the reconnect requeues and
|
|
563
|
-
replays it under the same idempotency key, and acking a rebase that never left is the
|
|
564
|
-
rebase-before-ack order defeated one frame later. A presence roster has no repair at all — the
|
|
565
|
-
membership is already on the shared set and the client's next heartbeat re-rosters — so it is
|
|
566
|
-
logged (`sync.presence_roster_dropped`). The drain's `reconnect` frame is the socket's slot in the
|
|
567
|
-
spread and nothing re-sends it, so `drain()` returns `DrainedSocket[]` with `notified` per socket
|
|
568
|
-
and logs `sync.drain_frames_dropped`.
|
|
569
|
-
- **A socket's actor comes from `createSyncNode({ authenticate })` and from nowhere else.** The node
|
|
570
|
-
imports no authenticator — the app supplies one, exactly as it supplies `onMutate` — and it runs
|
|
571
|
-
on the upgrade *before* `server.upgrade`, so a refused credential never costs a websocket.
|
|
572
|
-
`null` is a **decision** (401, `X_SOCKET_UNAUTHENTICATED`, a client fault that pages nobody); a
|
|
573
|
-
throw is a **failure** (503, `X_SOCKET_AUTH_UNAVAILABLE`, reported) — the same rule the row gate
|
|
574
|
-
follows, one layer out. Absent, every socket is anonymous and `start()` warns: that node is
|
|
575
|
-
single-tenant, and `hub.guard('org.*.feed', ({ actor }) => actor?.orgId === …)` denies everyone.
|
|
576
|
-
The actor is written in exactly one place, the `GrantBook`; `WsData` deliberately carries none,
|
|
577
|
-
because two spellings of one identity disagree the moment a re-auth renews one of them.
|
|
578
|
-
- **A grant expires; a socket does not.** `authenticate` answers a `SyncGrant`, not an `Actor`: a
|
|
579
|
-
15-minute token on a socket that stays up for hours was authorized once and served forever, and
|
|
580
|
-
an active client never idles out either — every inbound frame `touch()`es it. The node re-decides
|
|
581
|
-
an expired grant on an interval and then calls both halves that already existed and had no
|
|
582
|
-
caller: `hub.onActorChange` (topics) and `registry.reauthorize` (subscriptions). `refresh` is the
|
|
583
|
-
app's closure, so the framework retains no credential of its own — re-reading the upgrade
|
|
584
|
-
`Request` would mean holding one per socket. No `refresh` = close with `1008` and let the client
|
|
585
|
-
re-dial. A `refresh` that **raises** keeps the socket and retries: a token service timing out is
|
|
586
|
-
not a revocation.
|
|
587
|
-
- **`desynced` is a mark with a reader.** It is written when a patch is dropped by backpressure,
|
|
588
|
-
when a gate fails, when a window loses its tail and when a re-auth survives; the *next* delivery
|
|
589
|
-
serves that subscriber a fresh snapshot out of the shared window (no DB read) and only then
|
|
590
|
-
clears it. A snapshot the socket refuses leaves the mark, which is the state it is in. Four
|
|
591
|
-
writers and no reader was a subscription that stayed permanently and silently stale on a healthy
|
|
592
|
-
socket, with the server knowing and the client not.
|
|
593
|
-
- **`result.refill` is checked BEFORE the mark, because a repair out of a guessed window clears it.**
|
|
594
|
-
The word "fresh" above is load-bearing: when the matcher lost the window's tail, `entry.rows` is a
|
|
595
|
-
guess, and the same fanout that refuses to send a *patch* derived from it was resnapshotting every
|
|
596
|
-
already-desynced subscriber out of it — and clearing the one mark that would have made the next
|
|
597
|
-
change re-read. That subscriber is then recorded as repaired against rows nothing trusts and gets
|
|
598
|
-
a patch, not the snapshot it is still owed, from the refilled window. A lost tail degrades every
|
|
599
|
-
subscriber the same way, whatever each was holding, and they are all repaired on the next change
|
|
600
|
-
after `refillWindowInLane` has replaced the window. `live-fanout.test.ts` pins both halves.
|
|
601
|
-
- **A change the window already holds is refused on the way in.** The replicator guarded duplicates
|
|
602
|
-
and out-of-order on the *publish* side; `entry.lsn = change.lsn` was unconditional on the
|
|
603
|
-
*consume* side, so a redelivery rewound every subscriber's cursor. `change.lsn <= entry.lsn` is
|
|
604
|
-
dropped and counted as `staleChanges`.
|
|
605
|
-
- **A gap in the change stream is detected, not assumed away.** Fanout is core NATS — at most once
|
|
606
|
-
— and an lsn cannot reveal a gap, because a WAL position is a byte offset and every legitimate
|
|
607
|
-
next change is already an arbitrary jump. The replicator stamps `producer` + `seq`; a skipped
|
|
608
|
-
sequence marks every window `stale` and every subscriber desynced, and the next change to each
|
|
609
|
-
query re-reads. Both fields are optional on the bus: a publisher that does not sequence detects
|
|
610
|
-
nothing rather than crying gap, and a *new* producer restarts the count rather than reading as
|
|
611
|
-
one. A stale window is replaced in the lane (`refillWindowInLane`) — `fillWindow` takes the
|
|
612
|
-
entry's own lane and a lane is not reentrant.
|
|
613
|
-
- **A hub that closed opens nothing, and `#open` is the only thing that can enforce it.** `close()`
|
|
614
|
-
walks `#bridges` and then clears it, which reaches every bridge that is open and none that is
|
|
615
|
-
still opening: a reservation an in-flight `subscribe` has taken is `sub === null`, so
|
|
616
|
-
`unsubscribeWhenOpen` does nothing to it and `clear()` drops the entry. The transport then hands a
|
|
617
|
-
live subscription to a `Bridge` nothing can name — `#release` looks the topic up, misses and
|
|
618
|
-
returns — and its handler keeps calling `deliver` for the life of the process. The same orphan the
|
|
619
|
-
`Bridge` comment describes, one state earlier. So `close()` sets `#closed` **before** the walk and
|
|
620
|
-
`#open` closes its own subscription when it lands after one, dropping the entry with it so a
|
|
621
|
-
second post-close subscribe opens and closes its own rather than double-unsubscribing this handle.
|
|
622
|
-
It then **raises** `X_TRANSPORT_UNAVAILABLE` rather than returning: returning let `subscribe` fall
|
|
623
|
-
through to `joinTopic`, so the socket became a member of a topic nothing on this node is bridged
|
|
624
|
-
to — silent for the life of the connection, no error on either side, and no reason for the client
|
|
625
|
-
to redial. Reachable between `hub.close()` inside `node.drain()` and the last in-flight subscribe.
|
|
626
|
-
`#release` takes the bridge the caller reserved for the same reason: after `close()` cleared the
|
|
627
|
-
table, that topic name may hold a bridge a LATER subscribe opened, and releasing by name alone
|
|
628
|
-
decrements somebody else's refcount.
|
|
629
|
-
- Deny by default on topics. No guard = `X_TOPIC_FORBIDDEN`.
|
|
630
|
-
- **A guard that FAILS is not a guard that denied — the hub's copy of the rule the row gate already
|
|
631
|
-
follows.** On `onActorChange` (the re-auth pass) only a denial unsubscribes; anything else keeps
|
|
632
|
-
the topic, increments `guardFailures` and logs `channel.guard_failed`. A guard is app code and may
|
|
633
|
-
reach a database, so `catch { unsubscribe }` reported a store that timed out as a revoked grant —
|
|
634
|
-
every topic on every re-authenticated socket on the node, silently, with the client never told to
|
|
635
|
-
resubscribe. The initial `subscribe` is deliberately NOT split: there is no subscription to keep,
|
|
636
|
-
so a raising guard refuses that subscribe and the client hears about it.
|
|
637
|
-
- **The `rebase` frame goes out BEFORE its `ack`, and an `ack` refers to what failed.** The ack is
|
|
638
|
-
the receipt and the receipt retires the client's journal row and rebase-log entry, so a rebase
|
|
639
|
-
landing after it has no entry to read `conflict` off — every merge silently becomes `server-wins`
|
|
640
|
-
— and no sequence to decide which later optimistic writes to replay. Two frames on one socket:
|
|
641
|
-
the order is the only coordination there is. `ackRefOf` answers the mutation key for a `mutate`
|
|
642
|
-
and the sid for a `subscribe`; the socket id is only for a frame that could not be decoded, since
|
|
643
|
-
`queue.fail(ref)` looks up by idempotency key and a socket id names a key no queue holds.
|
|
644
|
-
- **Inbound frames run in a lane, and the lane is NEVER the socket.** `sync-node.message` dispatches
|
|
645
|
-
every frame as `void (async () => routeFrame(…))()`, so nothing upstream orders them. A global
|
|
646
|
-
per-socket lane would put every frame behind the slowest one, and the slowest one is a subscribe's
|
|
647
|
-
snapshot read — a DB round trip every reconnecting client pays once per live query, which is the
|
|
648
|
-
restart storm this framework is measured on. `mutate` is one lane per socket, `subscribe` is
|
|
649
|
-
`sub:<sid>` or `topic:<name>`, everything else is unlaned (`frame-lanes.ts`). A lane exists only
|
|
650
|
-
while work is queued on it: keyed by a client-chosen sid, a lane that outlived its work is an
|
|
651
|
-
unbounded map one socket grows at will.
|
|
652
|
-
- **A cap is a RESERVATION taken before the first await, never a check.** A lane makes concurrent
|
|
653
|
-
frames sequential and N sequential subscribes still pass a check-then-act cap N times — and the
|
|
654
|
-
per-tenant cap spans sockets, where no lane can see it at all. `SubscriptionBook.reserve(socket,
|
|
655
|
-
sid)` decides the sid claim, `maxPerSocket` and `maxPerTenant` in one synchronous step;
|
|
656
|
-
`ChannelHub.subscribe` does the same for `maxTopicsPerSocket`, `maxTopicsPerNode` and the node's
|
|
657
|
-
bridge slot, before the guard is awaited. The tenant is captured, not re-derived — a re-auth may
|
|
658
|
-
`retenant` the socket while the read is in flight, and the release has to give the slot back to
|
|
659
|
-
the tenant that took it. Released in a `finally`, and releasing twice is a no-op.
|
|
660
|
-
- **Bun's native pub/sub is deleted, not wired.** Nothing here publishes to a native topic and
|
|
661
|
-
nothing will: a native publish cannot be refused per socket, cannot report the frame it dropped
|
|
662
|
-
and cannot mark a subscriber desynced. `SocketRegistry.deliver` is the one fanout path.
|
|
663
|
-
`WsLike.subscribe`/`unsubscribe` stay declared and unused — a tracked app implements the
|
|
664
|
-
interface structurally, so removing the members is that app's typecheck failure — and the
|
|
665
|
-
declaration says so, because a member that looks live is one someone will call.
|
|
666
|
-
- **A dropped channel frame is counted in three places and repaired in none.** The series
|
|
667
|
-
`channel_frames_dropped_total` (no attributes — a topic is client-chosen, so a per-topic label is
|
|
668
|
-
unbounded series one socket can mint), the log `channel.frames_dropped` with `{ topic, dropped,
|
|
669
|
-
total }`, and `SocketRegistry.droppedChannelFrames` for a test or a bench that cannot scrape.
|
|
670
|
-
Node-wide because a socket past `maxDroppedFrames` is closed and removed — a per-socket count
|
|
671
|
-
leaves exactly when loss is worst — and distinct from `SyncSocket.droppedFrames`, which counts
|
|
672
|
-
every frame kind and dies with its socket. Repair needs a per-topic sequence on the wire: a
|
|
673
|
-
channel's lsn is the publishing hub's own per-node counter, so a client cannot tell a gap from a
|
|
674
|
-
message that came via another node. Declared in `socket.ts`, not core's `runtime-metrics.ts`:
|
|
675
|
-
that file is the series every process emits, this one exists only where channels do.
|
|
676
|
-
- **A qid is `@ultimat3/query`'s `queryHash(name, input)`, and this package derives none of its
|
|
677
|
-
own — `As of 2026-08`.** `qidOf` was the same two lines over a local copy of the canonical form
|
|
678
|
-
(`stableDigest(canonicalJson(input))`), and `canonicalJson`/`stableDigest` were this package's
|
|
679
|
-
third copy of what `@ultimat3/action` and `@ultimat3/query` also each held. They had already
|
|
680
|
-
diverged: `{ a: undefined, b: 1 }` gave `feed:eb8ed3ccb5023093` from `queryHash` and
|
|
681
|
-
`feed:c0bf82ad036cb0a5` from `qidOf`, because query's walk drops an `undefined`-valued key and
|
|
682
|
-
this one rendered `"a":null`. The two are COMPARED in one flow — `@ultimat3/query`'s `planResume`
|
|
683
|
-
decides refetch-vs-resume by comparing a cursor's `queryHash` against the query's, while
|
|
684
|
-
`liveQueryDefinition` keys the shared window by the qid — so keeping both correct was never the
|
|
685
|
-
option; the first time either moved, every resume decision and every window lookup were keyed
|
|
686
|
-
differently. `realtime -> query` is the one declared sideways edge and this package already
|
|
687
|
-
imports it. **`fnv1a` is gone too, `As of 2026-08-24`**: it stayed for one job — the cursor's
|
|
688
|
-
result-set digest — and `LiveCursor.digest` was deleted for having no reader, so this package now
|
|
689
|
-
owns no hash at all. A 32-bit hash nothing calls is one the next caller reaches for as a sharing
|
|
690
|
-
key, which is the single thing `json.test.ts` used to pin it against.
|
|
691
|
-
`live-contract.test.ts` is the pin — it reads `registry.subscriberCount(queryHash(name, input))`
|
|
692
|
-
through a real subscribe, so a local derivation fails it. Cost of the move: none observable on the
|
|
693
|
-
server. Every qid a node computes comes from a DECODED frame, and `JSON.parse` produces no
|
|
694
|
-
`undefined`, no `Date`, no `Map` and no `Set` — the four values the two forms disagree about — so
|
|
695
|
-
no live subscription re-keyed and nothing re-snapshotted.
|
|
696
|
-
- **The canonical form is injective over the values it accepts, and `JSON.stringify` is not** —
|
|
697
|
-
the reason that survives the move, now `@ultimat3/core`'s to enforce. `JSON.stringify` answers
|
|
698
|
-
`"null"` for `NaN` and `±Infinity` and `"0"` for `-0`, so four distinct inputs hashed to one qid
|
|
699
|
-
— and a qid *hit* hands the joiner the first subscriber's compiled source, matcher and seated
|
|
700
|
-
window. Bare `NaN` / `Infinity` / `-Infinity` / `-0` tokens are emitted instead; they are not
|
|
701
|
-
valid JSON, which is correct, because that output is hashed and never parsed. Exposure is
|
|
702
|
-
narrower than it looks and the tests say so rather than overclaiming: `NaN` and `±Infinity` have
|
|
703
|
-
no JSON spelling and so cannot arrive on a `subscribe` frame — they reach the hash only from a
|
|
704
|
-
caller building `input` in JS. **`-0` is wire-reachable**: `JSON.parse('{"a":-0}')` answers `-0`.
|
|
705
|
-
- **Refusing new sockets and draining the ones you have are two shutdown phases.** `stopAccepting()`
|
|
706
|
-
is the `accept` phase: `ready = false`, `/readyz` 503, a late upgrade shed with `retry-after-ms`,
|
|
707
|
-
and every socket untouched — a draining node still owes its clients their patches, and `stop()` is
|
|
708
|
-
what releases the change subscription carrying them. `drain()` + `stop()` are the `close` phase.
|
|
709
|
-
Registered with no phase, both landed in `close` and the node upgraded new websockets until the
|
|
710
|
-
very end. `listenSyncNode` unregisters both on `stop()`.
|
|
711
|
-
- **Readiness AND the connection cap are asked twice, because `authenticate` is app code with an
|
|
712
|
-
await in it.** A request that passed the checks at the top of `handleUpgrade` can be parked in a
|
|
713
|
-
token service when SIGTERM lands, and the `accept` phase is over by the time it reaches
|
|
714
|
-
`server.upgrade` — one more socket on a node the load balancer has already stopped routing to, so
|
|
715
|
-
nothing takes it over. `ready` and the socket count are therefore **functions** on `UpgradeDeps`,
|
|
716
|
-
not values read once.
|
|
717
|
-
**`socketCount()` was the half that was read once and never re-asked** (2026-08-23), which is the
|
|
718
|
-
same staleness with a worse blast radius: a restart storm dials every client of a dead node at
|
|
719
|
-
this one at once and each parks in the token service having passed the cap while the node still
|
|
720
|
-
held nothing, so `maxConnections: 2` with ten parked upgrades took **ten** sockets — reproduced,
|
|
721
|
-
`upgraded 10, shed 0`. Sound because there is no await between the recheck and `server.upgrade`,
|
|
722
|
-
and the count moves INSIDE it: Bun runs `websocket.open` synchronously there, which is where
|
|
723
|
-
`sockets.add` runs. The recheck sheds with the same 503 + `retry-after-ms` and takes no second
|
|
724
|
-
`tryAccept()`: that budget was spent.
|
|
725
|
-
- **A client `send` that returned is not an acknowledgement.** A browser `WebSocket.send` on a
|
|
726
|
-
CLOSING socket discards the frame and returns normally, so a drained mutation is `inflight` until
|
|
727
|
-
the server settles it or `requeueInflight` returns it. Only `pending` is sendable, `drain()` is one
|
|
728
|
-
chained pass at a time (two overlapping passes put one key on the wire twice, and a later pass can
|
|
729
|
-
overtake the one ahead of it), and backpressure over `MAX_BUFFERED_BYTES` declines rather than
|
|
730
|
-
fails — the mutation stays pending and the pass stops instead of reordering the ones behind it.
|
|
731
|
-
- **The lane orders passes; it does not order a socket death, so the queue carries an epoch.**
|
|
732
|
-
`requeueInflight` is not a pass and cannot reach into one parked at `await send(...)`: it hands
|
|
733
|
-
back what was on the dead socket, the parked pass resumes and marks everything *behind* that
|
|
734
|
-
mutation `inflight` for a connection that is gone. `#sendable` excludes `inflight`, so the next
|
|
735
|
-
drain skips them, no ack ever arrives and the writes are lost — invariant 3 inverted. `#epoch` is
|
|
736
|
-
bumped before the requeue scan and read at the top of every `#pass` iteration; a pass whose epoch
|
|
737
|
-
went stale returns and leaves the rest `pending` for the connection that arms the next one.
|
|
738
|
-
- **`#persist` hands the store a SNAPSHOT, never the live entries.** `QueueStore.save` is a durable
|
|
739
|
-
write (OPFS, IndexedDB) and may await before it reads. Given the array itself, a store that
|
|
740
|
-
resolves after the next pass has moved on persists a status that was never true when it was
|
|
741
|
-
called — and `inflight` is the one a reload cannot recover from.
|
|
742
|
-
- **A reconnect replays registrations AND topics, and every socket handler carries the identity
|
|
743
|
-
guard.** A reconnect is one `hello` plus one frame per thing this client holds: a `subscribe` per
|
|
744
|
-
registration, carrying that registration's cursor, and a `subscribe` per topic. Topic membership
|
|
745
|
-
is state on the node's socket, so a channel is silent from the first reconnect while its handler
|
|
746
|
-
is still installed — and its presence membership is swept — unless every one is re-announced.
|
|
747
|
-
`onOpen` needed the `#socket !== socket` guard `onMessage` and `onClose` already had: a replaced
|
|
748
|
-
socket opening late marked the connection up and replayed every subscription onto the current one.
|
|
749
|
-
- **`hello` carries NO cursors, and `HelloFrame.resume` is deleted (2026-08).** It was filled by
|
|
750
|
-
every client on open and read by nobody — the node replied `resume: []` and decided resume per
|
|
751
|
-
subscription from the `subscribe` frame — so every reconnect shipped each cursor twice, up to 512
|
|
752
|
-
ids each, in the restart storm this package is measured on. Wiring it was the wrong half of the
|
|
753
|
-
choice: a cursor's `qid` is `` `${name}:${fingerprint(input)}` ``, so a node reading a resume list
|
|
754
|
-
recovers the query **name** — it is the plaintext prefix — but never the `input`, which is the half
|
|
755
|
-
every decision needs. Without it `definition.authorize({ actor, input })` cannot run and no entry
|
|
756
|
-
can be built; the qid names a window but not a decision, and the retained window holds pre-policy
|
|
757
|
-
patches, so answering from it at `hello` time means answering before the per-subscriber
|
|
758
|
-
authorization pass, for a subscription that does not exist yet. It could only ever restate,
|
|
759
|
-
unauthorized, what `subscribe` decides with the input in hand — and it could not even save the
|
|
760
|
-
bytes, because the cursor still has to ride its `subscribe`. Two places deciding one thing is what axiom 1 refuses. **`PROTOCOL_VERSION` did NOT
|
|
761
|
-
move**, same rule as `snapshot.entity`: `decode` is a whitelist, so a new node drops an old
|
|
762
|
-
client's `resume` and an old node reads a new client's omission as the empty list it always got.
|
|
763
|
-
The one deploy of skew costs nothing in either direction.
|
|
764
|
-
- **The client beats, because only the client can end a half-open socket.** `heartbeatMs` (default
|
|
765
|
-
`DEFAULT_HEARTBEAT_MS`, 15s; `0` disables) sends a `hello` — byte-identical to the opening one,
|
|
766
|
-
since the frame has no resume list to leave out — plus one subscribe frame per topic, which is the
|
|
767
|
-
node's presence heartbeat. It is **not** how a deploy is noticed: `socket.skewed` compares the
|
|
768
|
-
build the client claims (the `hello`'s `buildId`, which `sawHello` records on every one — the
|
|
769
|
-
latest is the record — or `?build=` on the dial) against this node's; a client says the same
|
|
770
|
-
build on every beat and the node's never moves while the socket is open, so every `hello` on one
|
|
771
|
-
socket answers the same forever and `update-available` reaches a client on the
|
|
772
|
-
socket it opens against the *new* node. The hello IS read — until 2026-09-07 only the dial was,
|
|
773
|
-
and a dial without `?build=` was recorded as this node's own id, so a client naming its build only
|
|
774
|
-
in the frame was current forever. Two silent windows and the client closes with `4000` and
|
|
775
|
-
arms the reconnect. It is one
|
|
776
|
-
self-re-arming tick on the injected `Scheduler`, not an interval: a client is either beating on a
|
|
777
|
-
live socket or backing off toward a new one, never both. The 15s is the client's OWN number:
|
|
778
|
-
`realtime.heartbeatMs` was a `RealtimeConfig` key read by nothing and it is **deleted**
|
|
779
|
-
(2026-08-19). The server half of the beat stays derived — `PresenceRegistry.heartbeatMs` is
|
|
780
|
-
`max(1000, floor(ttlMs / 3))`, the same rule `idleSweepPeriodMs` follows, because a second knob
|
|
781
|
-
is a second number that can disagree with the one it is a fraction of.
|
|
782
|
-
- **Every question a hot path asks is indexed, never scanned.** `SubscriptionBook` keeps
|
|
783
|
-
`#bySocket` and a per-tenant count beside `#bySid`, and `SocketRegistry` keeps `#byTopic` beside
|
|
784
|
-
the socket table. Both replaced a walk of the whole node that ran once per socket or once per
|
|
785
|
-
frame: `ofSocket` filtered a copy of every subscription (100,000 entries measured at **17.7s** of
|
|
786
|
-
blocking work per teardown or re-auth sweep — a deploy or a batch of grants expiring together is
|
|
787
|
-
the whole trigger), and the per-tenant cap walked the same map on **every subscribe frame**
|
|
788
|
-
(7.96 ms each at that size). A new index goes where the deaths are seen: topic membership is the
|
|
789
|
-
registry's because `remove` is the one path a close, a drain and the idle sweep all take, and
|
|
790
|
-
`joinTopic`/`leaveTopic` are the only way to change it — two call sites for one membership is how
|
|
791
|
-
an index goes wrong. When an actor changes, `reauthorize` calls `book.retenant(socket)`: an index
|
|
792
|
-
nobody updates is a count that drifts for the rest of the process.
|
|
793
|
-
- **A ceiling per resource, and the wire's are not options.** `README.md` has the table. The rule
|
|
794
|
-
behind it: the accept budget bounds the accept *rate*, so the *count* needs its own
|
|
795
|
-
(`maxConnections`, shed as the same 503 + `retry-after-ms`); a socket that is open needs a frame
|
|
796
|
-
budget (`socket.frameBudget`, checked at the top of `routeFrame` **before `touch()`** — a frame
|
|
797
|
-
this node refuses must not renew the idle window); and anything a client sizes is bounded in
|
|
798
|
-
`decode` by `FRAME_LIMITS`, which a caller may narrow but never widen. `list()` takes a required
|
|
799
|
-
`max` so a new array field on a new frame cannot ship without someone choosing its size.
|
|
800
|
-
`input` is walked ITERATIVELY: the thing being refused is a stack overflow in `canonicalJson`, so
|
|
801
|
-
a recursive check would be the same crash one frame earlier.
|
|
802
|
-
- **Every ceiling on a socket `sync` builds is reachable from `createSyncNode`.** The node
|
|
803
|
-
constructs every `SyncSocket` it holds, so a `SyncSocketOptions` the node does not forward is a
|
|
804
|
-
number an operator can only change by abandoning `createSyncNode` — which is what
|
|
805
|
-
`maxBufferedBytes` and `maxDroppedFrames` were until 2026-08. Forwarded the same way
|
|
806
|
-
`maxFramesPerSecond`/`frameBurst` already are (`...(x === undefined ? {} : { x })`, so an unset
|
|
807
|
-
option keeps `SyncSocket`'s own default rather than overwriting it with `undefined`).
|
|
808
|
-
- **One socket's buffer has one number on the server and a separate one in the browser.**
|
|
809
|
-
`DEFAULT_MAX_BUFFERED_BYTES` (`socket.ts`) is both `SyncSocket`'s send-side ceiling and the
|
|
810
|
-
`backpressureLimit` `sync-node.ts` hands Bun — two spellings of one buffer on one side, and the
|
|
811
|
-
runtime's limit set lower means our check never fires and a frame is dropped with nothing marked
|
|
812
|
-
desynced. `client-mutations.ts`'s `MAX_BUFFERED_BYTES` is deliberately *not* imported from it:
|
|
813
|
-
that is browser code and `socket.ts` is the node's registry, its metrics and its close codes.
|
|
814
|
-
`sync-limits.test.ts` pins the server pair through behaviour, not by comparing two constants that
|
|
815
|
-
are now one declaration — an equality between them is a test that cannot fail.
|
|
816
|
-
- **A `SubscriptionLimitError` names the knob, never the default.** `knob` defaults to
|
|
817
|
-
`maxPerSocket`/`maxPerTenant`, which are `LiveQueryRegistry`'s — so the channel hub's per-socket
|
|
818
|
-
*topic* cap, thrown without one, told an operator to move a number in a different constructor
|
|
819
|
-
that would not have helped. Every throw site passes `knob` explicitly (`maxTopicsPerSocket`,
|
|
820
|
-
`maxTopicsPerNode`, `maxEntries`, `maxPerSocket`, `maxPerTenant`); `channel.test.ts` asserts the
|
|
821
|
-
two hub ones against the option names, because a fix line naming the wrong setting is worse than
|
|
822
|
-
no fix line — it is an instruction that runs and changes nothing.
|
|
823
|
-
- **Retained memory is bounded by BYTES.** `RingChangeBuffer` keeps the patch-count cap as a
|
|
824
|
-
*replay* bound (what a delta resume costs to fold) and adds the byte budgets as the memory one —
|
|
825
|
-
`packages/cache/src/lru.ts:1-2` states why: 4,096 queries x 1,024 patches is 4.19M retained rows
|
|
826
|
-
and no number of bytes at all. `forget(qid)` is called by `LiveQueryRegistry.unsubscribe` when the
|
|
827
|
-
last subscriber of a query id goes; it had no caller, so the ring outlived the entry. It is
|
|
828
|
-
also called by `#dropIfUnheld` on the subscribe path, `As of 2026-09-06`: an entry is created
|
|
829
|
-
BEFORE the snapshot read that fills it, so a cold subscribe the database refused left an entry
|
|
830
|
-
with no subscriber and no removal path — `unsubscribe` can only reach one through a subscription
|
|
831
|
-
that was never attached. `maxEntries` cold failures were therefore enough to answer
|
|
832
|
-
`X_SUBSCRIPTION_LIMIT` to every later subscriber for the life of the process, after the database
|
|
833
|
-
had recovered, because `qid` derives from client-chosen input and distinct inputs mint distinct
|
|
834
|
-
orphans. Dropped only when the entry is still the one in the table, holds no subscriber and has
|
|
835
|
-
no read in flight — a read published on it belongs to a concurrent subscriber that has not
|
|
836
|
-
attached yet, and dropping it there would hand that subscriber a window no change reaches.
|
|
837
|
-
- **A channel patch id carries the NODE that minted it** (`As of 2026-09-06`). `ChannelHub`'s
|
|
838
|
-
`#sequence` counts within one process, and the patch id was that counter alone — so two `sync`
|
|
839
|
-
replicas publishing to one topic minted the same id for the same subscriber, and a channel has
|
|
840
|
-
no cursor and no re-snapshot, so nothing downstream can repair a collision. `nodeId` defaults to
|
|
841
|
-
a per-hub `uuid()` and is declarable (the pod name) when an operator should be able to say which
|
|
842
|
-
node published a frame. The frame's `lsn` is deliberately left as the per-process counter:
|
|
843
|
-
nothing reads a channel frame's lsn as an order across nodes, and `client-frames.ts` advances a
|
|
844
|
-
cursor only for a registered live query, never for a topic.
|
|
845
|
-
- **An error never renders a value that carries a credential.** `parsePgUrl` names `DATABASE_URL`
|
|
846
|
-
rather than echoing the URL it refused — an error reaches a log, `--json`, an agent transcript
|
|
847
|
-
and a ticket, and the password is in the string. Same rule as `packages/mail/src/driver-smtp.ts`.
|
|
848
|
-
- **A full presence frame is capped and says so; the set behind it is never capped.** `roster()` is
|
|
849
|
-
what a frame carries (`maxMembers`, 256, plus `total`); `list()` stays whole because the sweep
|
|
850
|
-
differences it, and a short list would report every member past the cap as having left. `total`
|
|
851
|
-
is set on `sync` only — a `join`/`leave`/`update` frame is a delta, and a count beside one reads
|
|
852
|
-
as truncation.
|
|
853
|
-
- **One node per topic sweeps.** Every node sweeping every room it has seen is one full-set read
|
|
854
|
-
multiplied by the fleet, and the same `leave` frame published N times. The election needs no
|
|
855
|
-
compare-and-set the shared store does not have: the lease key is a *keyed set*, so each node's
|
|
856
|
-
claim is its own member and the leader is the lowest id every claimant can see. Eventually
|
|
857
|
-
consistent on purpose — the worst case is a duplicate `leave` for someone already gone.
|
|
858
|
-
- **Money is THREE physical columns on the wire too, and a live row must equal a repository row.**
|
|
859
|
-
`entityRow` folds `<p>_minor`/`<p>_currency`/`<p>_scale` into one property. It matched two, so a
|
|
860
|
-
scaled amount arrived at every subscriber unscaled *and* carrying a stray physical `priceScale`
|
|
861
|
-
beside `price` — one row, two shapes, no error anywhere. NULL and absent both mean **no `scale`
|
|
862
|
-
key**, never `0` (that is whole units, a 100x reinterpretation of an ordinary price), which is
|
|
863
|
-
exactly what `@ultimat3/entity`'s `moneyOf` does. That equality is the pin:
|
|
864
|
-
`pg-entity-row-parity.test.ts` reads one physical row through both surfaces — this package's fold
|
|
865
|
-
and a real `postgresRepo` — and asserts one object, each side absolutely as well as against the
|
|
866
|
-
other, because equality alone is satisfied by both failing open together. It is the one test here
|
|
867
|
-
that imports `@ultimat3/entity` (tier 2, a legal downward edge, test-only: `*.test.ts` never
|
|
868
|
-
ships), and it has to, or the thing being compared against is a copy of the reader instead of the
|
|
869
|
-
reader. The `0…15` scale bound stays `@ultimat3/schema`'s — enforced by the column CHECK and by
|
|
870
|
-
`parseScale`, never restated here.
|
|
871
|
-
- Never a bare `Error`. Never `any`. Never `Date.now()` — take a `Clock` (`clock.now()` is a `Date`;
|
|
872
|
-
use `monotonic()` for durations).
|
|
873
|
-
- **A test fixture standing in for a FOREIGN error extends `Error` on purpose, and that is not the
|
|
874
|
-
bare-`Error` rule being broken.** `PoolTimeout`, `Denied`, `MutationFailed` and `ThirdPartySdkError`
|
|
875
|
-
(nine sites across `realtime`, `db` and `ai`) simulate a driver, a policy library or an app's
|
|
876
|
-
`onMutate` — values this package did not construct and must handle anyway. `isPolicyDenial`,
|
|
877
|
-
`stringField` and `renderThrowable` all exist *because* such values arrive; rebuilt as
|
|
878
|
-
`UltimateError`s the fixture would prove the framework handles its own errors, which is the
|
|
879
|
-
"equality satisfied by both sides failing open together" failure the row-parity test names. The
|
|
880
|
-
rule governs what this package **throws**, never what a test hands it.
|
|
120
|
+
- **`selectTransport(env, realtime)` is the one place `realtime.transport` + env become a bus**; the
|
|
121
|
+
KV bucket and presence TTL come back with it. The config decides, the env supplies (22.0.0):
|
|
122
|
+
`'nats'` dials the variable `urlEnv` names and refuses (`X_CONFIG_INVALID`) when unset; `'memory'`
|
|
123
|
+
refuses a set `NATS_URL` (or the named variable).
|
|
124
|
+
- **`nats` is imported only by `nats-lib-client.ts`**; reconnect and re-subscription are the
|
|
125
|
+
library's job — `NatsTransport` must never grow subscription bookkeeping. What stays above the
|
|
126
|
+
port: the herd jitter (`reconnectDelayHandler`) and the KV semantics presence needs
|
|
127
|
+
(`nats-jetstream.ts`, `nats-kv.ts`). `nats-fake.ts` is a bus with server semantics.
|
|
128
|
+
- **Nothing leaves `NatsTransport` uncoded** — `#translating` wraps the port's synchronous
|
|
129
|
+
`publish`/`subscribe` refusals as `X_TRANSPORT_UNAVAILABLE`; an `UltimateError` passes through.
|
|
130
|
+
`transport-parity.test.ts` asserts both transports, and that the wrap still delivers. `#ensure`
|
|
131
|
+
reuses a mid-reconnect client on purpose.
|
|
132
|
+
- **A bucket name is checked before interpolation** (`assertBucket`, `X_TRANSPORT_PROTOCOL`); a
|
|
133
|
+
presence key or member id is base64url-encoded (`encodeToken`), never validated.
|
|
134
|
+
- Presence lives in `transport.shared`. The sync node is `PresenceRegistry`'s only caller:
|
|
135
|
+
subscribing to a topic IS joining, repeating the frame is the heartbeat, dropping or closing is the
|
|
136
|
+
leave. Expiry is silent, so the node sweeps on an interval, and **one node per topic sweeps** (lowest
|
|
137
|
+
id in the lease key's keyed set). `roster()` caps a frame at `maxMembers` (256) with `total`;
|
|
138
|
+
`list()` is never capped. `total` is set on `sync` frames only.
|
|
139
|
+
- **A socket the node evicts ITSELF goes through `evict(socket, code, reason)`** (the full
|
|
140
|
+
`teardown`), never `sockets.remove`. **`drain()` waits for the presence leaves it started** —
|
|
141
|
+
`evictInChunks` (`drain-evictions.ts`, `DRAIN_EVICT_CHUNK`, `allSettled`).
|
|
142
|
+
- **The idle sweep is armed by `start()`** (a quarter of `idleTimeoutMs`, floored at 1 s, `.unref()`)
|
|
143
|
+
and measures on `Clock.monotonic()` (`lastSeenMonotonicMs`). `SocketRegistry.idle()` is a query;
|
|
144
|
+
the node evicts.
|
|
145
|
+
- **`drain()` and `stop()` both call one idempotent `release()`**; `drain()` releases after the
|
|
146
|
+
sockets are gone and before `hub.close()`.
|
|
147
|
+
- **Refusing new sockets and draining are two phases**: `stopAccepting()` is `accept`, `drain()` +
|
|
148
|
+
`stop()` are `close`. `listenSyncNode` unregisters both on `stop()`.
|
|
149
|
+
- **A hub that closed opens nothing**: `close()` sets `#closed` before the walk; `#open` closes a
|
|
150
|
+
late subscription and RAISES `X_TRANSPORT_UNAVAILABLE`. `#release` takes the reserved bridge, never
|
|
151
|
+
a name.
|
|
152
|
+
- **A socket's actor comes from `createSyncNode({ authenticate })` only**, run before
|
|
153
|
+
`server.upgrade`. `null` = 401 `X_SOCKET_UNAUTHENTICATED`; a throw = 503
|
|
154
|
+
`X_SOCKET_AUTH_UNAVAILABLE`. Absent = anonymous, and `start()` warns. The actor lives only in the
|
|
155
|
+
`GrantBook`.
|
|
156
|
+
- **The grant is recorded BEFORE `server.upgrade`** (Bun runs `open` synchronously inside it) and
|
|
157
|
+
released on every path that never opens — `onUngranted` is REQUIRED on `UpgradeDeps`, and a
|
|
158
|
+
`server.upgrade` that THROWS releases too. `sync-node-auth.test.ts`'s harness opens inside
|
|
159
|
+
`upgrade()`, as Bun does.
|
|
160
|
+
- **A grant expires; a socket does not**: expired grants are re-decided on an interval through
|
|
161
|
+
`hub.onActorChange` and `registry.reauthorize`. No `refresh` = close with `1008`; a `refresh` that
|
|
162
|
+
raises keeps the socket and retries.
|
|
163
|
+
- **Every `socket.send` reads its answer**: a dropped subscribe reply marks the subscription desynced;
|
|
164
|
+
a dropped presence roster is logged (`sync.presence_roster_dropped`); `drain()` returns
|
|
165
|
+
`DrainedSocket[]` with `notified` and logs `sync.drain_frames_dropped`.
|
|
166
|
+
- **Inbound frames run in a lane, never the socket**: `subscribe` is `sub:<sid>` or `topic:<name>`,
|
|
167
|
+
everything else unlaned (`frame-lanes.ts`); a lane exists only while work is queued.
|
|
168
|
+
- **Bun's native pub/sub is deleted**; `SocketRegistry.deliver` is the one fanout path.
|
|
169
|
+
`WsLike.subscribe`/`unsubscribe` stay declared for structural implementers.
|
|
170
|
+
- **A dropped `records` frame is counted AND repaired**: per-node `seq`/`epoch` per channel; a refused
|
|
171
|
+
frame marks the socket gapped, and the websocket `drain` handler sends `replay-gap`
|
|
172
|
+
(`channel_replay_gaps_total` beside `channel_frames_dropped_total`). The client re-runs the
|
|
173
|
+
channel's `catchUp`, holding frames meanwhile.
|
|
174
|
+
- **A channel is a DECLARATION**: `channel(name, { params, policy, catchUp, records?, events? })`.
|
|
175
|
+
Deny by default, and `policy` is REQUIRED (`X_CHANNEL_DECLARATION_INVALID`; a public channel says
|
|
176
|
+
`policy: allow('public')`). Presence rides `events: true` channels.
|
|
177
|
+
- **A guard that FAILS keeps the topic on re-auth** (`guardFailures`, `channel.guard_failed`); the
|
|
178
|
+
initial subscribe still refuses.
|
|
179
|
+
- **A channel patch id carries the node** (`nodeId`, default a per-hub `uuid()`).
|
|
180
|
+
- **An error never renders a credential**: `parsePgUrl` names `DATABASE_URL`, never the URL.
|
|
881
181
|
|
|
882
|
-
##
|
|
182
|
+
## Replication
|
|
183
|
+
|
|
184
|
+
- Exactly one `replicator` per DB, by a session-level advisory lock. **`start()` and
|
|
185
|
+
`PgAdvisoryLock.tryAcquire` are MEMOISED** (guard and promise published synchronously, cleared on
|
|
186
|
+
settle); `stop()`/`release()` await the in-flight one. A start that FAILS hands the lock back, and
|
|
187
|
+
`running` is set only once the feed pumps.
|
|
188
|
+
- **The pump has one way out, `#die`**, which records the failure, stops the confirm timer, closes and
|
|
189
|
+
nulls the connection. `start()` awaits the previous pump before it dials. **`stop()` releases
|
|
190
|
+
everything before it reports anything.**
|
|
191
|
+
- **`REPLICA IDENTITY FULL` is checked at preflight, warned, and counted — never thrown** — ahead of
|
|
192
|
+
`pg_create_logical_replication_slot`. `ReplicationStreamStats.partialBefore` reads
|
|
193
|
+
`PgRelation.replicaIdentity`, never the tuple.
|
|
194
|
+
- **Every session pins `datestyle=ISO`, `intervalstyle=postgres`, `extra_float_digits=3`** in the
|
|
195
|
+
startup packet (`pg-connection.ts`, pinned by `pg-connection.test.ts`), byte for byte what
|
|
196
|
+
Postgres' own walreceiver sends.
|
|
197
|
+
- A change lsn is `<16 hex commit position><8 hex row position>`; never order by either half alone,
|
|
198
|
+
never depend on wall time or a process counter.
|
|
199
|
+
- Slot, publication and entity names match `[a-z_][a-z0-9_]*` before interpolation — a security
|
|
200
|
+
boundary. A SQLSTATE is data: `pg-wire.ts`'s `FIXES` is read with `Object.hasOwn`.
|
|
201
|
+
- **A live row equals a repository row**: `pg-entity-row.ts` decodes through `@ultimat3/entity`'s
|
|
202
|
+
own `decodeRow` / `entityForTable`, so money (three physical columns), scale and every other kind
|
|
203
|
+
fold exactly as `postgresRepo` folds them.
|
|
204
|
+
- A write's name rides the WAL: `pg-replication.ts` reads it off the transaction's opening
|
|
205
|
+
`pg_logical_emit_message` (prefix `WRITE_ORIGIN_WAL_PREFIX`; `START_REPLICATION` asks
|
|
206
|
+
`messages 'true'`).
|
|
207
|
+
|
|
208
|
+
## The page
|
|
209
|
+
|
|
210
|
+
- **One record store per PAGE (`record-store.ts`)**, keyed `type:key` with the SERVER's key
|
|
211
|
+
(`live-record-type.ts`); the browser never derives one. SYNCED truth plus an OVERLAY of pending
|
|
212
|
+
optimistic writes, REPLAYED over synced truth whenever it moves. Installed as core's `RecordSink` on
|
|
213
|
+
`pageClient().store`.
|
|
214
|
+
- **Page state lives on `globalThis[Symbol.for('ultimate.realtime')]` (`page-store.ts`)**, never module
|
|
215
|
+
scope — every island is its own bundle (`page-client.test.ts` builds the package twice). Never
|
|
216
|
+
`instanceof` across copies.
|
|
217
|
+
- **The signal factory is per BUNDLE**: `installRealtime({ signal, sync })` (`reactivity.ts`). No
|
|
218
|
+
install and a DOM is `X_REALTIME_UNINSTALLED`; no DOM is a server render — hooks answer
|
|
219
|
+
`pending`/online and create no page state. `useRecord` and `useQuery` return `AsyncState` accessors.
|
|
220
|
+
- **One socket per page, built by the first live hook (`page-socket.ts`)**; `browser-socket.ts`
|
|
221
|
+
holds the framework's only `new WebSocket`. `hasPageSocket()` lives in `page-store.ts`. A principal
|
|
222
|
+
change (`rescope`) clears the store and redials.
|
|
223
|
+
- **One socket per ORIGIN and principal, in a SharedWorker**: `socket-engine.ts` holds it and talks
|
|
224
|
+
to tabs over `MessagePort`s (`socket-host.ts`'s virtual socket); wants are ref-counted and frames
|
|
225
|
+
routed; a port silent for `REAP_AFTER_BEATS` is reaped; each tab resubscribes from its OWN cursors.
|
|
226
|
+
No `SharedWorker` → the in-page host runs the same engine over a `MessageChannel`.
|
|
227
|
+
- **The socket is READ-ONLY (protocol 3)**. A write is `useMutation`: the twin into the overlay, then
|
|
228
|
+
`POST actionPath(name)` through core's `clientTransport` with an idempotency key; the answer's
|
|
229
|
+
records are adopted inside the transport call, before `settle`. An `ack` is only a refusal.
|
|
230
|
+
- **`LiveClient` holds no signal** — plain reads plus `onStatus` / `onChange`; hooks wrap them.
|
|
231
|
+
- **An answer that did not carry a touched row keeps that row's overlay** until the server's next row
|
|
232
|
+
for it, bounded by `DEFAULT_AWAIT_SERVER_MS` (10 s).
|
|
233
|
+
- **A `records` frame names the write that produced it** (`ChannelRecordsFrame.write =
|
|
234
|
+
writeDigest(idempotencyKey)`, never the key). `RecordStore.push` digests every overlay key before
|
|
235
|
+
the request leaves (`record-names.ts`); `client-channels.ts` settles inside the frame's own batch.
|
|
236
|
+
A non-digest `write` is a protocol error (`wire-channel.ts`) and dropped on the bus. Rows an answer
|
|
237
|
+
or echo partly confirmed go to `OverlayEntry.confirmed` and stop painting. `write-echo.test.ts`.
|
|
238
|
+
- `local(tx, input)` is pure and CONVERGENT: no I/O, no `Date.now()`, no `Math.random()`.
|
|
239
|
+
- **One conflict vocabulary**: `ConflictPolicy` from `@ultimat3/core`, resolved by core's
|
|
240
|
+
`resolveConflict(overlayRow, serverRow)` in `RecordStore.settle`. A server delete is not a conflict.
|
|
241
|
+
- Anything a component reads is a getter or accessor. `MutatorLike.local` uses method syntax.
|
|
242
|
+
`useQuery`'s input is read once, at call time.
|
|
243
|
+
- Every subscription handle (`LiveHandle`, the hook accessors, `Unsubscribe`) is `Disposable`, and
|
|
244
|
+
`[Symbol.dispose]` is the same function as the release. Type claims go in `type-pins.ts`.
|
|
245
|
+
- **The outbox**: only `pending` is sendable; `drain()` is one chained pass at a time; backpressure
|
|
246
|
+
over `MAX_BUFFERED_BYTES` declines rather than fails. `#epoch` is bumped before the requeue scan and
|
|
247
|
+
read every pass iteration. `#persist` hands the store SNAPSHOTS by KEY (`QueueChange`); one tab
|
|
248
|
+
drains at a time (`navigator.locks`, `page-outbox.ts`'s `exclusive`) and resets a stored `inflight`
|
|
249
|
+
to `pending` under the lock.
|
|
250
|
+
|
|
251
|
+
## The client connection
|
|
252
|
+
|
|
253
|
+
- **The client owns its reconnect**: a closed socket arms ONE timer through the injected `Scheduler`
|
|
254
|
+
that calls `connect()`. `onClose` nulls `#socket`, schedules only when nothing is armed (a
|
|
255
|
+
`reconnect` frame's slot wins), and speaks only for its own socket; `close()` cancels. A dial that
|
|
256
|
+
throws in the timer is reported through `onError` (default `console.error`) and arms the next.
|
|
257
|
+
- **`connect()` closes the socket it replaces**; `onOpen`, `onMessage` and `onClose` all carry the
|
|
258
|
+
identity guard.
|
|
259
|
+
- **A reconnect replays registrations AND topics** — one `hello`, one `subscribe` per registration
|
|
260
|
+
(with its cursor) and per topic.
|
|
261
|
+
- **`hello` carries NO cursors** (`HelloFrame.resume` is deleted): a cursor's qid names a window but
|
|
262
|
+
not the input a decision needs.
|
|
263
|
+
- **The client beats** (`heartbeatMs`, default `DEFAULT_HEARTBEAT_MS` 15 s, `0` disables): a `hello`
|
|
264
|
+
plus one subscribe per topic (the presence heartbeat). Two silent windows close with `4000`. The
|
|
265
|
+
node's `socket.skewed` compares the build the `hello` claims against its own. The server beat is
|
|
266
|
+
derived: `PresenceRegistry.heartbeatMs = max(1000, floor(ttlMs / 3))`.
|
|
267
|
+
- **There is ONE `backoffDelay`, `@ultimat3/core`'s, counted from 1.** `policyDelay(policy, attempt,
|
|
268
|
+
rng)` (internal, not on the barrel) maps a `BackoffPolicy` onto it; a 0-based counter adds 1 at the
|
|
269
|
+
call site (`client.ts`, `socket-engine.ts`, `replicator.ts`'s 0-based `retryDelayMs`).
|
|
270
|
+
`client-reconnect.test.ts`, `socket-engine-reconnect.test.ts`, `nats-transport.test.ts` pin the
|
|
271
|
+
first wait at the base. `drainPlan()` and `AcceptBudget` keep their own arithmetic.
|
|
272
|
+
`bun run flight-copies` refuses a second curve.
|
|
883
273
|
|
|
884
|
-
|
|
274
|
+
## Rules for code here
|
|
275
|
+
|
|
276
|
+
- Never a bare `Error`. Never `any`. Never `Date.now()` — take a `Clock` (`monotonic()` for durations).
|
|
277
|
+
- **A test fixture standing in for a FOREIGN error extends `Error` on purpose** (`PoolTimeout`,
|
|
278
|
+
`Denied`, `ThirdPartySdkError`): it simulates a value this package did not construct. The rule
|
|
279
|
+
governs what this package **throws**, never what a test hands it.
|
|
280
|
+
|
|
281
|
+
## Browser bytes, per hook (`As of 2026-09-22`)
|
|
282
|
+
|
|
283
|
+
`bun build --target=browser --minify`, one entry importing one hook from the barrel. Re-measure
|
|
284
|
+
before quoting.
|
|
285
|
+
|
|
286
|
+
| Hook | Current |
|
|
885
287
|
|---|---|
|
|
886
|
-
| `
|
|
887
|
-
| `
|
|
888
|
-
| `
|
|
889
|
-
| `
|
|
890
|
-
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
|
|
898
|
-
|
|
899
|
-
|
|
900
|
-
|
|
901
|
-
|
|
902
|
-
|
|
903
|
-
|
|
904
|
-
|
|
905
|
-
|
|
906
|
-
|
|
907
|
-
|
|
908
|
-
|
|
909
|
-
| `client-frames.ts` | what a RECEIVED frame does to client state, and `ClientFrameTarget` — the only inbound surface the client exposes. The mirror of `sync-frames.ts` |
|
|
910
|
-
| `client-harness-fixture.ts` | the injected socket + scheduler + harness both client suites drive. Excluded from the tarball |
|
|
911
|
-
| `hooks-fixture.ts` | the same, for the two hook suites (`hooks.test.ts`, `hooks-identity.test.ts`). Excluded from the tarball |
|
|
912
|
-
| `subscription-book.ts` | who holds which subscription, keyed by `(socket, sid)`, and the per-socket/per-tenant caps answered from it |
|
|
913
|
-
| `apply-patches.ts` | folding a patch list onto a row list (`applyPatches`) or onto ids alone (`orderAfterPatches`, what a window uses) — the client's one stateless piece, and one fold, not two |
|
|
914
|
-
| `hooks.ts` | the ambient client seam + the four component hooks — the only file an app imports |
|
|
915
|
-
| `query-hook.ts` | the typed projection: one declared query bound to one named hook |
|
|
916
|
-
| `type-pins.ts` | compile-time assertions `tsc` checks — the hook's input type, its row type, the `Query` seam |
|
|
917
|
-
| `window-lock.ts` | one FIFO lane per query id — the only thing that orders a fanout |
|
|
918
|
-
| `frame-lanes.ts` | the order one socket's INBOUND frames are applied in, and the lane key each kind belongs to. `WindowLock` again, keyed differently — and it bounds no cap |
|
|
919
|
-
| `live-fanout.ts` | what one change does inside one entry's lane: match, fold, one policy pass per subscriber, and the re-snapshot that repairs a desynced one |
|
|
920
|
-
| `client-mutations.ts` | the outbound mutation path — the optimistic twin, the rebase entry, the queue entry, and the sender the drain hands each frame to |
|
|
921
|
-
| `client-heartbeat.ts` | when to beat and when to give up. A policy, which is why it is not in `client.ts`'s connection lifecycle |
|
|
922
|
-
| `client-topics.ts` | the client's channel book, and the one membership frame its two callers (`subscribe`, the reconnect replay) must never spell differently |
|
|
923
|
-
| `client-contract.ts` | the client's injected shapes — `ClientSocket`, `LiveClientOptions`, `LiveHandle` — declared apart from the class that consumes them |
|
|
924
|
-
| `policy-gate.ts` | the only authz seam |
|
|
925
|
-
| `subscriber-gate.ts` | the per-subscriber pass of a definition's row policy, and its two counters — `rowsDenied` and `gateFailures`. Evaluates no policy of its own |
|
|
926
|
-
| `live-contract.ts` | what a live query IS: `LiveQueryDefinition`, `SnapshotResult`, `LiveSubscription`. Four modules need the shape and none of them needs the registry that runs it. The **id** is not here and not anywhere in this package — it is `@ultimat3/query`'s `queryHash` |
|
|
927
|
-
| `json.ts` | the wire's value types. **No hash**: the canonical form and the sharing-key hash are `@ultimat3/core`'s (`canonicalJson`, `fingerprint`), and the 32-bit `fnv1a` that stayed for `LiveCursor.digest` went with it (2026-08-24) |
|
|
928
|
-
| `live-definition.ts` | the only bridge from a declared `query({ live: true })` to a registrable definition — and `policy-gate.ts`'s only caller |
|
|
929
|
-
| `matcher-bridge.ts` | the only `@ultimat3/query` matcher seam — and where a patch row is narrowed to the columns the query returned |
|
|
288
|
+
| `useRecord` | 13,226 |
|
|
289
|
+
| `useMutation` | 21,725 (after the outbox left islands) |
|
|
290
|
+
| `useQuery` | 47,696 |
|
|
291
|
+
| `useChannel` | 45,585 |
|
|
292
|
+
|
|
293
|
+
- **The disk boot is ONE page script (`boot.ts`, `./boot`)**, served at `/_x/page-boot/<hash>.js`
|
|
294
|
+
on a document with the scope tag and a realtime island; it wipes every other principal's stored
|
|
295
|
+
scope, then restores. `booted` reads its promise off `globalThis[Symbol.for('ultimate.page-boot')]`.
|
|
296
|
+
- **An island never carries the outbox**: `boot.ts` builds it; `useMutation` and the page socket read
|
|
297
|
+
it through `outbox-slot.ts` after `page.booted`. No boot ⇒ a refused write is rejected, not queued.
|
|
298
|
+
|
|
299
|
+
## Map
|
|
300
|
+
|
|
301
|
+
`index.ts` / `server.ts` are the two barrels. Server side: `sync-node.ts` (+ `sync-auth`,
|
|
302
|
+
`sync-frames`, `sync-upgrade`, `sync-listen`, `drain-evictions`), `channel.ts`, `presence.ts`,
|
|
303
|
+
`socket.ts`, `live-query.ts` (+ `live-definition`, `query-window`, `live-fanout`, `window-lock`,
|
|
304
|
+
`subscriber-gate`, `policy-gate` — the only authz seam — and `matcher-bridge`), `replicator.ts` /
|
|
305
|
+
`live-replicator.ts`, the Postgres client (`pg-*.ts`, `pgoutput.ts`), the bus (`nats-*.ts`,
|
|
306
|
+
`transport-env.ts`, `fanout.ts`). Client side: `client*.ts`, `socket-engine.ts` / `socket-host.ts` /
|
|
307
|
+
`sync-worker.ts`, `record-*.ts`, `page-*.ts`, `boot.ts`, `offline-queue.ts`, `use-*.ts`,
|
|
308
|
+
`reactivity.ts`. Shared: `sync-protocol.ts` (the wire), `cursor.ts`, `change-buffer.ts`,
|
|
309
|
+
`thundering-herd.ts`, `live-contract.ts`, `json.ts` (no hash), `type-pins.ts`. Each file's header
|
|
310
|
+
states its one job.
|
|
930
311
|
|
|
931
312
|
## Commands
|
|
932
313
|
|
|
@@ -935,45 +316,12 @@ bun test packages/realtime/src # from the REPO ROOT, never from packa
|
|
|
935
316
|
bun run typecheck
|
|
936
317
|
```
|
|
937
318
|
|
|
938
|
-
|
|
939
|
-
and Bun reads `bunfig.toml` from the cwd, so `bun test` inside this directory loads none and six
|
|
940
|
-
tests fail on a missing matcher — this package's suite reading red for the shell it was run in.
|
|
941
|
-
CI's `package` job spawns `bun test packages/<pkg>` with `cwd` at the root for the same reason.
|
|
942
|
-
|
|
943
|
-
Changing a frame shape means adding a fixture to `sync-protocol.test.ts` — the round-trip test
|
|
944
|
-
fails if a kind has no fixture — and bumping `PROTOCOL_VERSION` **when the change makes an old
|
|
945
|
-
frame unreadable in either direction**. An *additive optional* field (`snapshot.entity`, 2026-08)
|
|
946
|
-
is not that: `decode` builds a whitelist, so an old client drops it and a new client reads its
|
|
947
|
-
absence as a defined answer. Neither is *removing a field nothing read* (`hello.resume`, 2026-08):
|
|
948
|
-
the same whitelist drops an old client's copy, and a new client's omission decodes to what the
|
|
949
|
-
field always held. Bumping for either refuses every in-flight client on a rolling deploy and buys
|
|
950
|
-
nothing — the version guards incompatibility, not novelty. Removing a field something *does* read
|
|
951
|
-
is the opposite case and bumps.
|
|
952
|
-
|
|
953
|
-
**"Nothing read it" is decided by the DECODER, not by the callers — and that half is what moved
|
|
954
|
-
`PROTOCOL_VERSION` to 2 (2026-08-24, BREAKING).** `hello.resume` was free because `decode` read it
|
|
955
|
-
through `list()`, which answers `[]` for an absent field; `cursor.digest` and `cursor.count` were
|
|
956
|
-
read through `str()` and `num()`, which **throw**. So deleting two fields no *caller* consumed
|
|
957
|
-
still made the frame unreadable to a peer one deploy behind — in **both** directions, since a
|
|
958
|
-
cursor rides the client's `subscribe` and the node's `snapshot`. Without the bump the skew shows up
|
|
959
|
-
as a per-frame `field "digest" must be a string`, which is the same refusal with none of the
|
|
960
|
-
instruction. Before claiming a removal is free, read the field's line in `decode`: a `list()` is
|
|
961
|
-
free, a `str()`/`num()` is a bump.
|
|
962
|
-
|
|
963
|
-
**A patch carries the result set's columns, never the table's** — `narrowRow` in
|
|
964
|
-
`matcher-bridge.ts`, `As of 2026-08-20`. A `ChangeEvent` carries the whole TABLE row (that is what
|
|
965
|
-
logical replication emits, and what `@ultimat3/entity`'s `setRowObserver` emits), while a live
|
|
966
|
-
query's result set is whatever its `sql` returned. Every patch used to forward the change row
|
|
967
|
-
unnarrowed, so a column a projection exists to withhold went out on the socket the moment it
|
|
968
|
-
CHANGED — `examples/dummy`'s feed projects ten columns and one publish delivered `updatedAt` to
|
|
969
|
-
every subscriber (#230). The per-subscriber gate cannot help: it decides whether a ROW is delivered,
|
|
970
|
-
never which of its columns.
|
|
319
|
+
`bunfig.toml`'s preload installs `@ultimat3/testing`'s matchers and Bun reads it from the cwd.
|
|
971
320
|
|
|
972
|
-
|
|
973
|
-
|
|
974
|
-
|
|
321
|
+
Changing a frame shape means a fixture in `sync-protocol.test.ts` (the round-trip test fails
|
|
322
|
+
without one) and bumping `PROTOCOL_VERSION` **when an old frame becomes unreadable in either
|
|
323
|
+
direction**. `decode` is a whitelist: an additive optional field, or removing a field read through
|
|
324
|
+
`list()`, is free; removing a field read through `str()`/`num()` (which throw) is a bump. Read the
|
|
325
|
+
field's line in `decode` before claiming a removal is free.
|
|
975
326
|
|
|
976
|
-
|
|
977
|
-
lives inside the `sql` provider's closure and there is nothing static to read it from. Learned and
|
|
978
|
-
kept rather than re-derived per fanout, because the case the window's own rows cannot answer is an
|
|
979
|
-
EMPTY window — the first row to arrive would otherwise go out whole.
|
|
327
|
+
Why each rule above is shaped the way it is: [`docs/history/realtime.md`](../../docs/history/realtime.md).
|