@ultimat3/realtime 21.0.0 → 22.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +293 -1009
- package/README.md +78 -12
- package/package.json +4 -4
- package/src/changefeed.ts +7 -1
- package/src/channel-authz.ts +23 -4
- package/src/channel-decl.ts +16 -5
- package/src/channel-describe.ts +7 -5
- package/src/channel-logs.ts +19 -1
- package/src/channel-records.ts +8 -0
- package/src/client-channels.ts +75 -5
- package/src/client.ts +14 -2
- package/src/cursor.ts +5 -0
- package/src/errors.ts +21 -0
- package/src/idb-fake.ts +24 -4
- package/src/idb-types.ts +7 -0
- package/src/index.ts +0 -1
- package/src/live-definition.ts +5 -1
- package/src/live-fanout.ts +51 -2
- package/src/live-query.ts +11 -0
- package/src/live-replicator.ts +160 -0
- package/src/local-store-idb.ts +89 -15
- package/src/matcher-bridge.ts +5 -0
- package/src/nats-fake.ts +10 -1
- package/src/nats-jetstream.ts +36 -14
- package/src/nats-transport.ts +2 -2
- package/src/offline-queue.ts +76 -21
- package/src/page-outbox.ts +80 -10
- package/src/page-socket.ts +39 -8
- package/src/pg-entity-row.ts +37 -184
- package/src/pg-preflight.ts +24 -2
- package/src/pg-replication.ts +19 -6
- package/src/pg-wire.ts +51 -15
- package/src/policy-fake.ts +14 -0
- package/src/query-window.ts +35 -21
- package/src/replicator.ts +13 -3
- package/src/server.ts +8 -3
- package/src/socket-drops.ts +30 -0
- package/src/socket-engine.ts +15 -3
- package/src/socket-host.ts +103 -4
- package/src/socket-idle.ts +21 -0
- package/src/socket.ts +41 -38
- package/src/subscriber-gate.ts +92 -3
- package/src/sync-node.ts +2 -7
- package/src/thundering-herd.ts +12 -11
- package/src/transport-env.ts +55 -14
- package/src/use-mutation.ts +13 -0
- package/src/use-query.ts +10 -5
package/CLAUDE.md
CHANGED
|
@@ -6,991 +6,308 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
6
6
|
|
|
7
7
|
| May import | Must not |
|
|
8
8
|
|---|---|
|
|
9
|
-
| `@ultimat3/core`, `@ultimat3/query`, `@ultimat3/entity` (`/record` only
|
|
9
|
+
| `@ultimat3/core`, `@ultimat3/query`, `@ultimat3/entity` (`/record` only, server-side) | anything tier 4+ (`render`, `pwa`, `mcp`, `ui`, `cli`) |
|
|
10
10
|
| `@ultimat3/policy` **only via** `@ultimat3/query`'s `guard` | a second authz path of any kind |
|
|
11
11
|
| — | `solid-js` (each island bundle installs its own signal factory: `installRealtime`) |
|
|
12
|
-
| `nats` (the one external dependency, pinned exact) — from `nats-lib-client.ts
|
|
12
|
+
| `nats` (the one external dependency, pinned exact) — from `nats-lib-client.ts` and no other file | `nats` from anywhere else. Every other file is written against the port in `nats-client.ts` |
|
|
13
13
|
|
|
14
|
-
##
|
|
14
|
+
## Entries and bundling
|
|
15
15
|
|
|
16
|
-
- **Two entries
|
|
17
|
-
|
|
18
|
-
`
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
`
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
neighbour runs anything at import, which is what keeps `sideEffects` naming `errors.ts` alone
|
|
37
|
-
true — `registerErrorCodes()` stays there, unconditional, and `bun run side-effects` is the check.
|
|
38
|
-
- **`sideEffects` is the ARRAY `["./src/errors.ts"]`, never `false` and never absent.** Absent was
|
|
39
|
-
what made the failure above unrecoverable — with no field a bundler must assume every module has
|
|
40
|
-
effects, so nothing was tree-shaken and `nats` came along with `useLive`. Measured, not guessed:
|
|
41
|
-
`bun run side-effects --explain --json` prints what the tree actually does at import time.
|
|
42
|
-
`false` would drop `registerErrorCodes()` and every `REALTIME_ERROR_TITLES` entry with it.
|
|
43
|
-
**The array alone would have fixed the build** and is not why the split exists: tree-shaking is
|
|
44
|
-
a bundler's discretion, `export * from` or a namespace import defeats it, and "the client entry
|
|
45
|
-
cannot reach the bus" is a contract rather than an optimisation.
|
|
46
|
-
- **Two entries means a specifier naming a third does not resolve, and a `fix:` is pasted.**
|
|
47
|
-
`local-store.ts`'s `X_NOT_IMPLEMENTED` told the caller to import `createOpfsLocalStore` from
|
|
48
|
-
`@ultimat3/realtime/browser` — a subpath `exports` has never declared — so the one instruction
|
|
49
|
-
the refusal carried ended in a module-resolution failure, in the package whose own rules cite
|
|
50
|
-
axiom 4. Its alternative, `persist: false` on the query, was the same defect twice: `query()`
|
|
51
|
-
does not accept `persist` either. `fix-specifier.test.ts` is the build error — every
|
|
52
|
-
`@ultimat3/realtime/<subpath>` written in shipped source must be a key of `exports`, comments
|
|
53
|
-
included, because a comment naming a subpath that does not exist is the next fix line's source.
|
|
54
|
-
The OPFS refusal it was written for is deleted with `local-store.ts` (21.0.0).
|
|
55
|
-
- **`@ultimat3/realtime/server` needs its own `paths` entry in `tsconfig.base.json`**, beside
|
|
56
|
-
`@ultimat3/admin/dev`'s. `@ultimat3/*` maps `realtime/server` to `packages/realtime/server/src`,
|
|
57
|
-
which does not exist, and the root program has no `node_modules/@ultimat3` symlink to fall back
|
|
58
|
-
through — so `scripts/**` (which the root `tsc -b` compiles) reports `TS2307` without it. The
|
|
59
|
-
workspace packages resolve through their own `node_modules` and never needed it, which is what
|
|
60
|
-
makes the failure look local to `scripts/`.
|
|
16
|
+
- **Two entries; a name lives in exactly ONE.** `.` is the client half (`use-*`, `page-*`,
|
|
17
|
+
`reactivity`, `record-*`, `client*`, `browser-socket`, `live-rows`, `apply-patches`,
|
|
18
|
+
`offline-queue`, `sync-protocol`, `json`, `cursor`, `errors`, the client half of
|
|
19
|
+
`thundering-herd`). `./server` (`server.ts`) is everything touching `nats`, Postgres, the sync node,
|
|
20
|
+
the hub, the registry or the fanout (`nats` requires `stream/web`, which a browser build cannot
|
|
21
|
+
link). `packages/cli/src/realtime-browser-barrel.test.ts` bundles a `useLive`-only entry;
|
|
22
|
+
`barrel-split.test.ts` refuses a name exported from both. `./boot` and `./sync-worker` are the page
|
|
23
|
+
boot and the SharedWorker entry.
|
|
24
|
+
- **`errors.ts` is the code table plus the client-reachable refusals**; `realtime-error.ts` holds
|
|
25
|
+
the base class alone (an `extends` must not read it in a TDZ) and `replication-errors.ts` the
|
|
26
|
+
Postgres ones. Every name is still exported from `./errors`. A browser path loads `page-errors.ts`,
|
|
27
|
+
never the table (`page-errors-bundle.test.ts`).
|
|
28
|
+
- **`sideEffects` is the ARRAY `["./src/errors.ts"]`**, never `false` (drops `registerErrorCodes()`)
|
|
29
|
+
and never absent. `bun run side-effects` is the check.
|
|
30
|
+
- **Every `@ultimat3/realtime/<subpath>` written in shipped source must be a key of `exports`**,
|
|
31
|
+
comments included — `fix-specifier.test.ts`.
|
|
32
|
+
- **`@ultimat3/realtime/server` needs its own `paths` entry in `tsconfig.base.json`** (the wildcard
|
|
33
|
+
maps it to a directory that does not exist; `scripts/**` reports `TS2307` without it).
|
|
34
|
+
- A browser file imports core from `@ultimat3/core/page` (`packages/core/src/page-bundle.test.ts`,
|
|
35
|
+
`packages/query/src/client-bundle.test.ts`).
|
|
61
36
|
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
`
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
- **
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
- **
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
- What `liveQueryDefinition` caches per query id is the compiled source, the shape, the matcher and
|
|
141
|
-
the shared row window. What it must never cache is a decision. It builds that shared half with
|
|
142
|
-
`enforce: false` **on purpose**: a source compiled under the first subscriber's authority and
|
|
143
|
-
then keyed by query id is that subscriber's entitlements becoming everyone's. `authorize` is
|
|
144
|
-
still the subscribe-time decision and still runs per socket.
|
|
145
|
-
- Every policy call in `live-query.ts` takes a `Subscriber`. That is the enforcement: there is no
|
|
146
|
-
path through the gate that reads a query id and no actor.
|
|
147
|
-
- **The row policy always sees the *whole* row from the shared window, never a partial patch — and
|
|
148
|
-
a window that does not hold the row is not a partial one.** An update patch carries the changed
|
|
149
|
-
columns plus the id, so merging it onto nothing and calling that a row hands `visible` a
|
|
150
|
-
`undefined` for every column the change did not touch: fail-closed for `row.ownerId === actor.id`,
|
|
151
|
-
and a leak for every `!row.private`. So a patch whose row the window does not hold is **withheld**
|
|
152
|
-
— dropped, or the one `delete` frame that tells a subscriber holding it that it is gone. It is
|
|
153
|
-
neither `rowsDenied` nor `gateFailures`: nothing decided anything, the window simply *is* the
|
|
154
|
-
result set. The one path that could meet an empty window is a delta resume onto an entry nothing
|
|
155
|
-
has read yet, and `subscribe` fills it first (`entry.lsn === ''`) rather than withholding
|
|
156
|
-
everything — conditional on purpose, because re-reading per resuming subscriber is exactly the
|
|
157
|
-
cost a delta resume exists to skip in a restart storm.
|
|
158
|
-
- **A denial is a decision; everything else is a failure, and the two never share an answer.** A
|
|
159
|
-
bare `catch { return false }` in the row gate read a dead pool as "you may not see this row" —
|
|
160
|
-
the rows left the screen, `live.rows_denied` counted the drop, and the outage shipped as a
|
|
161
|
-
permission change. `visibleWithPolicy` matches `QueryDeniedError` (the only thing `guard` throws
|
|
162
|
-
for a decision) and rethrows the rest; `subscriber-gate.ts` and `reauthorize` ask
|
|
163
|
-
`isPolicyDenial(error)` instead, because `authorize` and `visible` are caller-supplied functions
|
|
164
|
-
and the answer has to come off the error's code. What a failure costs is decided per surface: a
|
|
165
|
-
snapshot **raises** out of `subscribe` (a short result set is indistinguishable from a correct
|
|
166
|
-
one), a delivery desyncs that **one** subscriber and lets the fanout finish, and a `reauthorize`
|
|
167
|
-
keeps the subscription — destroying it would report a timeout as a revoked grant, and a client
|
|
168
|
-
does not resubscribe to a denial. Every failure is counted as `gateFailures` and reported through
|
|
169
|
-
`onGateFailed`, never through `onRowDenied`: an alert fires on one of them.
|
|
170
|
-
- **One serial lane per query id, and it is the only thing that orders a fanout.** Nothing upstream
|
|
171
|
-
does: `sync` fires `void registry.deliver(change)` straight off the bus subscription, so two
|
|
172
|
-
changes arriving back to back both start, both write `entry.rows`/`entry.lsn`, and both await
|
|
173
|
-
their way through a per-subscriber gate in between. Unordered, a subscriber is handed lsn 2 and
|
|
174
|
-
then asked to fold lsn 1 on top of it, its cursor rewound to 1 — a reconnect then replays what it
|
|
175
|
-
already applied, over newer state, and the row stays at the older value. `WindowLock` (`run`)
|
|
176
|
-
gives each entry a FIFO lane. `deliver` *enters* every lane before awaiting any of them, and no
|
|
177
|
-
fanout ever takes a second lane, so holding all of them at once cannot be a cycle — and two
|
|
178
|
-
deliveries queue onto each query id in call order, which is what makes "per query id, not per
|
|
179
|
-
node" true. Awaiting one entry before entering the next was two bugs in one line: one slow policy
|
|
180
|
-
pass set the whole node's pace, and a lane that threw ended the loop, so every entry behind it
|
|
181
|
-
missed the change with **nobody desynced** — the silent divergence `markDesynced` exists to
|
|
182
|
-
prevent. A lane that fails now desyncs its own subscribers and the first failure still reaches
|
|
183
|
-
the caller, but it costs one query id. The lane chains on a settled shadow of each task: one
|
|
184
|
-
fanout that threw must not reject every fanout behind it.
|
|
185
|
-
- **Two reads of one entry are ordered by a READ GENERATION, never by an lsn.** `QueryEntry.lsn`
|
|
186
|
-
is optional — a definition with no lsn provider answers `''` from every snapshot — so the
|
|
187
|
-
never-backwards rule expressed purely in lsn terms read `'' >= ''` as "newer" and let the older
|
|
188
|
-
of two concurrent reads land on top of the newer one's window. The interleaving: a cold
|
|
189
|
-
subscriber issues P1; the change stream skips a sequence and `registry.invalidate()` marks the
|
|
190
|
-
entry; a second cold subscriber forces P2, which **clears `stale` on the way in**; P2 lands with
|
|
191
|
-
the post-gap rows; P1 lands last and overwrites them. `stale` is false, so `fanoutChange`'s
|
|
192
|
-
repair never fires, the next change patches the pre-gap window and re-snapshots every desynced
|
|
193
|
-
subscriber out of it — permanently stale on a healthy socket, which is the exact outcome `stale`
|
|
194
|
-
exists to prevent. `entry.generation` is bumped in `startRead` and `entry.applied` records the
|
|
195
|
-
newest read whose rows are on the window: an *identity* check, the same one `startRead` makes on
|
|
196
|
-
`entry.reading` one function down and `packages/cache/src/single-flight.ts:70` makes for the same
|
|
197
|
-
reason. The lsn guard stays beside it for the other question — a read that resolved behind a
|
|
198
|
-
*change* the fanout already folded — because those are two orderings and neither answers the
|
|
199
|
-
other.
|
|
200
|
-
- **The definition's read is once per entry, not once per subscriber.** A cold subscriber arriving
|
|
201
|
-
while another's read is in flight joins that read — N cold subscribers on one query id being N
|
|
202
|
-
reads is the shared window not existing. It is a share, not a cache: the in-flight promise is
|
|
203
|
-
cleared as it settles, so a later subscriber reads current rows rather than a window that has
|
|
204
|
-
been drifting since boot. The result lands **in the lane and never backwards**: a snapshot that
|
|
205
|
-
resolved after a newer change was already fanned out is discarded and its caller is served from
|
|
206
|
-
the newer window, because rewinding hands that subscriber rows the fanout has moved past and a
|
|
207
|
-
cursor behind the change that would have corrected them.
|
|
208
|
-
- The retained change window stores **pre-policy** patches; resume re-filters them per subscriber.
|
|
209
|
-
- A resume is the one gate pass that runs **outside** the lane, and it reads the live window on
|
|
210
|
-
purpose: the window can only have moved forwards, and a row whose grant was revoked in the
|
|
211
|
-
meantime is one that pass must refuse rather than replay from its state at the cursor's lsn.
|
|
37
|
+
## Numbers and ceilings
|
|
38
|
+
|
|
39
|
+
- **Every numeric option is refused when it is not FINITE** — `@ultimat3/core`'s `finiteOption()`.
|
|
40
|
+
`bun run finite-bounds` reads zero for this package, but it cannot see an option with no `??`
|
|
41
|
+
default or where a screen runs, so audit by reading the option interfaces
|
|
42
|
+
(`SubscriptionBook`'s `maxPerTenant` and `openNatsClient`'s `maxReconnectAttempts` were the two it
|
|
43
|
+
missed). `SyncGrant.expiresAt` is app DATA, not an option, and is deliberately unscreened.
|
|
44
|
+
- **A ceiling is refused where the object is BUILT**, never in a per-connection callback:
|
|
45
|
+
`socketCeilings()` (`sync-node-bounds.ts`) runs once in `createSyncNode`; `SyncSocket` keeps its own
|
|
46
|
+
screen because it is exported. `AcceptBudget`, `SyncSocket` and `SocketRegistry` throw
|
|
47
|
+
`X_INVARIANT` at construction on `NaN`/`±Infinity`.
|
|
48
|
+
- **A ceiling per resource, and the wire's are not options** (table in `README.md`): the accept
|
|
49
|
+
budget bounds the rate, `maxConnections` the count, `socket.frameBudget` an open socket (checked
|
|
50
|
+
at the top of `routeFrame` **before `touch()`**), and `FRAME_LIMITS` in `decode` everything a
|
|
51
|
+
client sizes (narrowable, never widenable). `list()` takes a required `max`. `input` is walked
|
|
52
|
+
ITERATIVELY.
|
|
53
|
+
- **Every `SyncSocket` ceiling is reachable from `createSyncNode`**, forwarded as
|
|
54
|
+
`...(x === undefined ? {} : { x })`.
|
|
55
|
+
- **A `SubscriptionLimitError` names the knob**: every throw site passes `knob` explicitly;
|
|
56
|
+
`channel.test.ts` asserts the hub's two against the option names.
|
|
57
|
+
- **Retained memory is bounded by BYTES**: `RingChangeBuffer` keeps the patch count as a replay
|
|
58
|
+
bound and byte budgets as the memory bound. `forget(qid)` runs when the last subscriber goes and
|
|
59
|
+
from `#dropIfUnheld` after a failed cold subscribe (only if the entry is still in the table, unheld
|
|
60
|
+
and has no read in flight).
|
|
61
|
+
- **Every question a hot path asks is indexed**: `SubscriptionBook` keeps `#bySocket` and a
|
|
62
|
+
per-tenant count; `SocketRegistry` keeps `#byTopic`, changed only by `joinTopic`/`leaveTopic`.
|
|
63
|
+
`reauthorize` calls `book.retenant(socket)`.
|
|
64
|
+
- **A cap is a RESERVATION taken before the first await**: `SubscriptionBook.reserve(socket, sid)`
|
|
65
|
+
decides the sid claim, `maxPerSocket` and `maxPerTenant` in one synchronous step;
|
|
66
|
+
`ChannelHub.subscribe` does the same for its caps. Released in a `finally`, idempotently.
|
|
67
|
+
- **Readiness AND the connection cap are functions on `UpgradeDeps`**, re-asked after
|
|
68
|
+
`authenticate` (app code with an await) and right before `server.upgrade`. The recheck sheds with
|
|
69
|
+
the same 503 + `retry-after-ms` and spends no second `tryAccept()`.
|
|
70
|
+
|
|
71
|
+
## Live queries
|
|
72
|
+
|
|
73
|
+
- Policy is evaluated **once per subscriber**, never once per query (`live-query.test.ts`, and
|
|
74
|
+
`live-definition.test.ts` for a real `query({ live: true })`). Every policy call in `live-query.ts`
|
|
75
|
+
takes a `Subscriber`.
|
|
76
|
+
- **A name nothing registered is `X_LIVE_QUERY_UNKNOWN`**, fix `x queries list --json`. The
|
|
77
|
+
registry is never enumerated over the wire.
|
|
78
|
+
- **One build per `(query, input)`**: `target.live()` returns the descriptor and runs the read
|
|
79
|
+
(`LiveQuery.execute`), never memoised.
|
|
80
|
+
- `liveQueryDefinition` caches per query id the compiled source, shape, matcher and shared window —
|
|
81
|
+
never a decision. The shared half is built with `enforce: false` on purpose; `authorize` runs per socket.
|
|
82
|
+
- **The row policy sees the WHOLE row from the shared window.** A patch whose row the window does not
|
|
83
|
+
hold is WITHHELD (not denied, not a failure). A delta resume onto an unread entry fills it first
|
|
84
|
+
(`entry.lsn === ''`).
|
|
85
|
+
- **A denial is a decision; everything else is a failure.** `visibleWithPolicy` matches
|
|
86
|
+
`QueryDeniedError` and rethrows the rest; `subscriber-gate.ts` and `reauthorize` ask
|
|
87
|
+
`isPolicyDenial(error)`. A failed snapshot raises out of `subscribe`; a failed delivery desyncs that
|
|
88
|
+
one subscriber; a failed `reauthorize` keeps the subscription. Failures count as `gateFailures` via
|
|
89
|
+
`onGateFailed`, never `onRowDenied`.
|
|
90
|
+
- **One serial lane per query id (`WindowLock`) is the only thing that orders a fanout.** `deliver`
|
|
91
|
+
enters every lane before awaiting any; no fanout takes a second lane; a lane that fails desyncs its
|
|
92
|
+
own subscribers; lanes chain on a settled shadow of each task.
|
|
93
|
+
- **Two reads of one entry are ordered by a READ GENERATION**, never by an lsn (`QueryEntry.lsn` may
|
|
94
|
+
be `''`): `entry.generation` / `entry.applied` is an identity check. The lsn guard stays for the
|
|
95
|
+
other question — a read that resolved behind a change the fanout already folded.
|
|
96
|
+
- **The definition's read is once per entry**: a cold subscriber joins the in-flight read (a share,
|
|
97
|
+
cleared on settle), and the result lands in the lane and never backwards.
|
|
98
|
+
- **The shared read carries a deadline and frees the SLOT**: `startRead` races it against
|
|
99
|
+
`DEFAULT_READ_DEADLINE_MS` (30 s; `new LiveQueryRegistry({ readDeadlineMs, schedule })`), rejects
|
|
100
|
+
joined callers with `X_TIMEOUT` (never an empty window), and puts `stale` back. No "off" spelling;
|
|
101
|
+
a non-positive or non-finite value takes the default (`query-window.test.ts`).
|
|
102
|
+
- The retained change window stores **pre-policy** patches; resume re-filters them per subscriber,
|
|
103
|
+
outside the lane, against the live window.
|
|
104
|
+
- **`desynced` is a mark with a reader**: the next delivery serves a fresh snapshot out of the shared
|
|
105
|
+
window and only then clears it. **`result.refill` is checked BEFORE the mark** — a guessed window
|
|
106
|
+
never clears a mark (`live-fanout.test.ts`).
|
|
107
|
+
- **A change the window already holds is refused** (`change.lsn <= entry.lsn`, counted as
|
|
108
|
+
`staleChanges`). **A gap is detected**: the replicator stamps `producer` + `seq`; a skipped
|
|
109
|
+
sequence marks every window stale and every subscriber desynced; `refillWindowInLane` replaces it.
|
|
110
|
+
- **A qid is `@ultimat3/query`'s `queryHash(name, input)`**; this package owns no hash
|
|
111
|
+
(`live-contract.test.ts`). The canonical form is core's and injective (`-0` is wire-reachable).
|
|
112
|
+
- **A `sid` is CLIENT data: a subscription is keyed by `(socket, sid)`**
|
|
113
|
+
(`subscription-book.ts` is the only spelling). Reusing a sid the same socket holds is
|
|
114
|
+
`X_SUBSCRIPTION_ID_TAKEN`.
|
|
212
115
|
- Truth is the server. A client is never the merge authority.
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
socket is the leave — presence has no frame of its own in either direction, because a second way
|
|
217
|
-
to say "I am here" is a client that can be subscribed and invisible at the same time.
|
|
218
|
-
- Expiry is silent by design, so the node sweeps on an interval: without it a member whose node
|
|
219
|
-
died is never announced as gone, and the survivors render a cursor that stopped moving. It has to
|
|
220
|
-
be an interval — a sweep only reports members a previous sweep saw, which is how a member that
|
|
221
|
-
joined on another node becomes leavable here at all.
|
|
222
|
-
- **The wire is the library's, the integration is ours, and `nats-client.ts` is the line between
|
|
223
|
-
them.** `nats` is imported by exactly one file — `nats-lib-client.ts` — and everything else in the
|
|
224
|
-
package, the transport and the JetStream bucket and the KV presence set and every test, is written
|
|
225
|
-
against that port. It is what lets the fake be an in-memory bus with *server* semantics instead of
|
|
226
|
-
431 lines of forged wire bytes, and what made deleting 1,019 LOC of framing, parser, PING/PONG and
|
|
227
|
-
session a swap rather than a rewrite ([`docs/idea/18-build-vs-wrap.md`](../../docs/idea/18-build-vs-wrap.md)).
|
|
228
|
-
The consequence that has to be held: **reconnect and re-subscription are the library's job now.** A
|
|
229
|
-
subscription outlives a dropped connection because the client re-establishes it underneath the
|
|
230
|
-
caller, so `NatsTransport` must never re-grow subscription bookkeeping of its own — a map of wanted
|
|
231
|
-
subjects, a dial promise, a loss counter, a rebind loop. Two things re-subscribing is a doubled
|
|
232
|
-
delivery on every reconnect and a subscription the caller's `unsubscribe` no longer reaches — the
|
|
233
|
-
hand-rolled client's lifecycle bugs were deleted with it rather than fixed for exactly that reason.
|
|
234
|
-
What stays above the port is what the library has no opinion on: our thundering-herd jitter, handed
|
|
235
|
-
over as its `reconnectDelayHandler` because spreading a restart herd is the framework's decision,
|
|
236
|
-
and the KV semantics presence needs (`nats-jetstream.ts`, `nats-kv.ts`) — a per-message TTL and a
|
|
237
|
-
batch direct get, which the library's own KV abstraction cannot express.
|
|
238
|
-
- **Nothing leaves `NatsTransport` uncoded, and `#translating` is where that is enforced — added
|
|
239
|
-
2026-08.** `client.publish` and `client.subscribe` are the port's two SYNCHRONOUS calls, and the
|
|
240
|
-
library refuses locally on both: a bad subject, a payload over the server's `max_payload`, a
|
|
241
|
-
connection torn down between the `#ensure` and the call, a permissions violation on the subject.
|
|
242
|
-
A raw `NatsError` escaped `publish()` into `ChannelHub`'s bridge, `SocketRegistry` and the
|
|
243
|
-
replicator — no code, no `fix:`, nothing an operator can act on — while `InProcessTransport`
|
|
244
|
-
answered `X_TRANSPORT_UNAVAILABLE` for its own one refusal. `transport-parity.test.ts` asserts
|
|
245
|
-
both transports in one test, and a third case proves the wrap still DELIVERS: a translation that
|
|
246
|
-
swallowed a working publish would satisfy the two refusal cases and fan out nothing. An
|
|
247
|
-
`UltimateError` passes through untouched — the port raises its own for a closed client, and
|
|
248
|
-
re-wrapping buries the code a caller branches on. `nats-lib-client.ts`'s header claim that every
|
|
249
|
-
failure leaves *there* as an `UltimateError` is still false for those two calls; the translation
|
|
250
|
-
is deliberately in ONE place, and it is the transport, because `connect` is a public injection
|
|
251
|
-
seam and an app-supplied client throws whatever it likes.
|
|
252
|
-
- **Reusing a client whatever its state is a DECISION, not the other half of that bug.** `#ensure`
|
|
253
|
-
hands back a client that is mid-reconnect on purpose: the library is re-establishing that same
|
|
254
|
-
connection and its subscriptions, and a second dial alongside it doubles every delivery. What a
|
|
255
|
-
caller gets from a client whose reconnect budget is spent is a publish that resolves into
|
|
256
|
-
nothing — reported through `onError` by `#watch`, and visible to `/readyz` through `connected`,
|
|
257
|
-
which is where a dead bus is meant to be caught.
|
|
258
|
-
- One place reads `NATS_URL`, and it is `selectTransport` — a boot that resolved the bus itself
|
|
259
|
-
could reach a different one than the container it is standing in for. The KV bucket and the
|
|
260
|
-
presence TTL come back with the transport for the same reason: they are one decision.
|
|
116
|
+
|
|
117
|
+
## The node, the bus and presence
|
|
118
|
+
|
|
261
119
|
- `sync` is stateless: no sticky sessions, nothing on a socket survives a restart.
|
|
262
|
-
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
`
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
`
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
`
|
|
283
|
-
|
|
284
|
-
- **The idle sweep
|
|
285
|
-
|
|
286
|
-
the
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
`
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
`
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
- **
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
reading their own flag — that flag is false for the whole of a start, so an unguarded teardown
|
|
324
|
-
returns "nothing to do" and leaves behind exactly what it was called to release.
|
|
325
|
-
**A start that FAILS hands the lock back, and `running` is set after the feed is pumping**
|
|
326
|
-
(`As of 2026-09-06`). It was set before `await feed.start()`, so a feed that rejected — a slot
|
|
327
|
-
already `active`, a preflight refusal — left this node holding the advisory lock and claiming to
|
|
328
|
-
run with nothing pumping: the takeover loop's next `start()` was answered `true` by the
|
|
329
|
-
`if (running)` guard without re-entering `begin`, and every standby stayed a standby of a slot
|
|
330
|
-
whose holder was not replicating. The release is best-effort, because the feed's failure is the
|
|
331
|
-
one the operator acts on and a session-scoped lock a dead connection cannot release is released
|
|
332
|
-
by Postgres when that session ends.
|
|
333
|
-
- **The replication pump has one way out, and it closes what it held.** Both exits — a decode error
|
|
334
|
-
and `nextCopyData()` returning `undefined`, which is the walsender ending the copy — run `#die`:
|
|
335
|
-
record `stats().failure`, stop the confirm timer, close the connection and null it. Each one left
|
|
336
|
-
behind is a dead replicator claiming to be a live one. A `null` failure answers `/readyz` ready
|
|
337
|
-
for a loop reading no WAL; a live `#running` makes the next `start()` a silent no-op; a retained
|
|
338
|
-
timer keeps telling the walsender a dead stream is keeping up; a retained `#connection` is a
|
|
339
|
-
socket the next `start()` overwrites rather than closes, holding the slot `active`. A `start()`
|
|
340
|
-
that goes live clears the previous failure, and `stop()` awaits the pump even when the connection
|
|
341
|
-
is already gone — `#die` nulls it *before* closing it, so returning early reports a released slot
|
|
342
|
-
to the supervisor that is about to start the next process.
|
|
343
|
-
- **`#pump` *is* the terminal cleanup, so a restart awaits it before it dials.** `#drain` awaits
|
|
344
|
-
`#die` and `#die` awaits `connection.close()`, but `#die` clears `#running` and nulls
|
|
345
|
-
`#connection` before that close settles: a `start()` checking `#running` alone dialled into a
|
|
346
|
-
slot the dead walsender still owned and replaced `#pump` with its own, so the next `stop()`
|
|
347
|
-
awaited only the new pump. `start()` takes the previous pump and awaits it first; its failure
|
|
348
|
-
path calls `stop().catch(() => undefined)` because the boot diagnosis is the one an operator acts
|
|
349
|
-
on and a teardown that also failed must not replace it.
|
|
350
|
-
- **`stop()` releases everything before it reports anything.** A `#confirm` or an `endCopy` that
|
|
351
|
-
threw skipped the close and the pump await, leaking the socket and telling a supervisor the
|
|
352
|
-
teardown was over before it had begun. Every step runs whatever the step before it did, and the
|
|
353
|
-
first failure is rethrown only once the connection is closed and the pump has ended.
|
|
354
|
-
- **`REPLICA IDENTITY FULL` is asked about at preflight, warned about, and counted — never thrown**
|
|
355
|
-
(added 2026-08-19). `pg-replication.ts`'s `#deliver` hands a `delete` its `message.before`,
|
|
356
|
-
and under any identity but FULL that tuple is the KEY COLUMNS ALONE — which `toRow` accepts,
|
|
357
|
-
because it only requires a text `id`. So `bridgeChange` decided "did this row leave the result
|
|
358
|
-
set" from a one-column row, `visible` read `undefined` for every column the identity did not
|
|
359
|
-
carry, and nothing anywhere recorded that it had happened: no emit, no check, and
|
|
360
|
-
`X_LIVE_REPLICA_IDENTITY` existed in neither the source nor the manifest. The check is a fourth
|
|
361
|
-
`connection.query` in `preflight`, and it **must** stay ahead of
|
|
362
|
-
`pg_create_logical_replication_slot` — changing the identity after a slot exists does not reach
|
|
363
|
-
what that slot decodes. It WARNS (`logger.warn(code, { cause, fix, tables })`, the message being
|
|
364
|
-
the code alone) because a throw would stop every app on the default identity from booting, which
|
|
365
|
-
is a worse outcome than the partial rows. `ReplicationStreamStats.partialBefore` is the runtime
|
|
366
|
-
half, read off `PgRelation.replicaIdentity` and never off the tuple: a DEFAULT-identity table
|
|
367
|
-
whose non-key columns happen to be NULL sends the bytes a FULL one does, so counting absent keys
|
|
368
|
-
would undercount exactly the rows a policy is most likely to misjudge. A hard refusal in the
|
|
369
|
-
`x verify` step is the follow-up and lives in `@ultimat3/cli`.
|
|
370
|
-
- **The replication session pins its own output formats, `As of 2026-08-23`.** Postgres sends every
|
|
371
|
-
WAL value as TEXT and `pg-values.ts` reads a `timestamptz` by matching postgres' ISO spelling,
|
|
372
|
-
keeping the raw text when it does not match — deliberately, because a wrong instant is worse than
|
|
373
|
-
a string. That makes the SERVER's `DateStyle` load-bearing: `SQL`, `German` or `Postgres` sends
|
|
374
|
-
every timestamp down the fallback, the shared window holds `Date`s while the patch holds text,
|
|
375
|
-
`compareValues` falls to string comparison, and one edit to one column jumps its row to the top
|
|
376
|
-
of every `orderBy('createdAt','desc')` feed for every subscriber — the exact defect the decode
|
|
377
|
-
exists to close, re-opened by a GUC. `pg-connection.ts` therefore sends
|
|
378
|
-
`options: '-c datestyle=ISO -c intervalstyle=postgres -c extra_float_digits=3'` in the startup
|
|
379
|
-
packet, byte for byte what postgres' own logical-replication client sends
|
|
380
|
-
(`libpqwalreceiver.c`) — which is why a walsender accepts it. On **every** session this class
|
|
381
|
-
opens, not only the replicating one: one session shape is one thing to reason about, and the
|
|
382
|
-
advisory-lock connection is the same class. A server that refuses one answers `ErrorResponse` at
|
|
383
|
-
startup, so the replicator fails to boot with the server's own words rather than mis-sorting a
|
|
384
|
-
feed behind a warning nobody reads. `pg-connection.test.ts` pins the packet.
|
|
385
|
-
- A change lsn is `<16 hex commit position><8 hex row position in that transaction>`. Never order by
|
|
386
|
-
either half alone: the commit lsn repeats within a transaction, and per-record WAL positions are
|
|
387
|
-
not monotonic across transactions. Never make it depend on wall time, the entity list or a process
|
|
388
|
-
counter — a replay must produce byte-identical lsns or at-least-once turns into duplicate delivery.
|
|
389
|
-
- Slot, publication and entity names are interpolated into a replication command, so they are
|
|
390
|
-
checked against `[a-z_][a-z0-9_]*` first. That regex is a security boundary, not a style rule.
|
|
391
|
-
- Same rule on the bus, for the half that is still ours: a bucket name is interpolated into a
|
|
392
|
-
JetStream stream name and its API subjects, so it is checked first (`assertBucket` in
|
|
393
|
-
`nats-jetstream.ts`, `X_TRANSPORT_PROTOCOL`). Subject validation went with the hand-rolled client —
|
|
394
|
-
the library refuses a malformed subject itself, and a second spelling of that rule here is a second
|
|
395
|
-
place it can drift. A presence key or member id is user data, so it is base64url-encoded
|
|
396
|
-
(`encodeToken`) rather than validated — no name is refused for its spelling.
|
|
397
|
-
- **One record store per PAGE, and `record-store.ts` is the only place a record lives (21.0.0).**
|
|
398
|
-
Keyed `type:key` — the entity's NAME and the key the SERVER computed with the entity's own
|
|
399
|
-
projection (`live-record-type.ts`, via `recordProjectionForTable`): an envelope arrives keyed, a
|
|
400
|
-
live snapshot carries `keys` and a patch `key` whenever the record key is not the row's `id`, and a
|
|
401
|
-
non-live `useQuery` window is the key order of the answer's `records[type]`. The browser never
|
|
402
|
-
derives a key; an answer with no envelope holds its own rows, not records. Two layers: SYNCED is server truth, the OVERLAY is every pending optimistic write,
|
|
403
|
-
REPLAYED over synced truth whenever it moves — never a before-image restored over newer truth,
|
|
404
|
-
which is what the old journal did. A live window, a `useQuery` list and a `useRecord` are all
|
|
405
|
-
projections over it; none holds a row of its own. It is core's `RecordSink`, installed on
|
|
406
|
-
`pageClient().store`, so an HTTP answer adopts into it without this package in the call.
|
|
407
|
-
- **Page state lives on `globalThis[Symbol.for('ultimate.realtime')]` (`page-store.ts`), never in
|
|
408
|
-
module scope.** Every island is its own bundle with its own copy of this package, so a module
|
|
409
|
-
singleton is one store and one socket PER ISLAND — the bug the plan-101 major ended.
|
|
410
|
-
`page-client.test.ts` builds this package twice with `Bun.build` and proves one store and one
|
|
411
|
-
`new WebSocket` between the copies. Never `instanceof` across copies: the class is whichever
|
|
412
|
-
bundle's copy made it.
|
|
413
|
-
- **The signal factory is per BUNDLE, the store per PAGE.** Each island carries its own solid-js, so
|
|
414
|
-
a signal must come from the bundle whose effects read it: `installRealtime({ signal, sync })`
|
|
415
|
-
(`reactivity.ts`, module scope on purpose) is what the island bootstrap calls. No install and a
|
|
416
|
-
DOM is `X_REALTIME_UNINSTALLED`; no install and no DOM is a server render — every hook answers
|
|
417
|
-
`pending`/online and creates NO page state (on a server that would be one store for every
|
|
418
|
-
request). `useRecord` and `useQuery` return `AsyncState` accessors the caller releases.
|
|
419
|
-
- **One socket per page, built by the first live hook (`page-socket.ts`), and `browser-socket.ts`
|
|
420
|
-
holds the framework's only `new WebSocket`.** A form-only or `useRecord`-only island never
|
|
421
|
-
imports that module, so it ships none of the lifecycle — measured, a `useRecord`-only chunk
|
|
422
|
-
carries no `WebSocket`. `import()` would buy nothing: islands build with `splitting: false`, and
|
|
423
|
-
Bun inlines a dynamic import there. `hasPageSocket()` lives in `page-store.ts` so asking costs no
|
|
424
|
-
socket bytes either. A principal change (`rescope`) clears the store and redials.
|
|
425
|
-
- **The socket is READ-ONLY (protocol 3, 21.0.0).** `mutate`/`rebase` frames and
|
|
426
|
-
`createSyncNode({ onMutate })` are deleted; an old client's `mutate` decodes to
|
|
427
|
-
`X_PROTOCOL_VERSION`. A write is `useMutation`: the twin into the overlay, then `POST
|
|
428
|
-
actionPath(name)` through core's `clientTransport` with an idempotency key; the answer's records
|
|
429
|
-
are adopted INSIDE the transport call, before `settle` drops the overlay, so a convergent twin
|
|
430
|
-
never flickers. A refusal drops the overlay. An `ack` is now only ever a refusal: `ref` is a
|
|
431
|
-
subscription's sid (that window renders `failed`) or the socket (reported through `onError`).
|
|
432
|
-
- **`LiveClient` holds no signal.** It is shared by every bundle on the page, and a signal belongs to
|
|
433
|
-
one bundle — so status is plain reads plus `onStatus`, and a live window is `LiveHandle` reads
|
|
434
|
-
plus `onChange`. Hooks wrap both in the calling bundle's own signal.
|
|
435
|
-
- **An answer that did not carry a row the write touched does not end its overlay.** `useMutation`
|
|
436
|
-
settles against the answer's own records (`onEnvelope`); a touched row it did not carry (an
|
|
437
|
-
action that returns a view, not the entity) keeps its overlay until the server's next row for it
|
|
438
|
-
— a frame, another answer — bounded by `DEFAULT_AWAIT_SERVER_MS` (10 s), after which the synced
|
|
439
|
-
layer stands. Without it, a like showed `+1`, fell back, then rose again on the frame.
|
|
440
|
-
- **A `records` frame names the write that produced it, and the writing page settles on it
|
|
441
|
-
(21.0.0, additive to protocol 3).** The node fans a commit out before it answers, so the write's
|
|
442
|
-
own frame routinely beats its HTTP answer, and merged under the still-pending twin it painted
|
|
443
|
-
the write twice: measured in `examples/dummy`'s `offline-like` e2e, `2,2 → 3,3 → 2,2` on the
|
|
444
|
-
outbox replay. No client-side rule can tell its own echo from somebody else's change without the
|
|
445
|
-
frame saying which write it is, and deferring frames for touched rows was rejected: with no
|
|
446
|
-
version it reorders an older frame over a newer answer. `ChannelRecordsFrame.write` is
|
|
447
|
-
`writeDigest(idempotencyKey)` (`@ultimat3/core`), never the key, because a frame goes to every
|
|
448
|
-
member. `RecordStore.push` digests every overlay key it takes (`record-names.ts`), before the
|
|
449
|
-
request leaves. `client-channels.ts` calls `settleWrite` inside the frame's own batch, so the
|
|
450
|
-
merge and the settle are one notification. A digest this page does not hold changes nothing.
|
|
451
|
-
Server side, the name comes off `ChangeEvent.write`: `channel-logs.ts` stamps it on the ring
|
|
452
|
-
entry, so a `since` replay names it too. `pg-replication.ts` reads it off the transaction's
|
|
453
|
-
opening `pg_logical_emit_message` (prefix `WRITE_ORIGIN_WAL_PREFIX`, START_REPLICATION asks
|
|
454
|
-
`messages 'true'`), and `@ultimat3/testing`'s in-process replicator reads it off the row
|
|
455
|
-
observer. A frame `write` that is not a digest is a protocol error (`wire-channel.ts`); on the
|
|
456
|
-
bus it is dropped, never trusted (`parseEnvelope`). `write-echo.test.ts` covers the useMutation and
|
|
457
|
-
outbox paths, another writer, an unknown digest, a custom policy, and a partial echo.
|
|
458
|
-
- **An overlay a server answer partly confirmed stops painting on those rows.** `settle` with rows
|
|
459
|
-
missing used to keep the WHOLE twin, so the rows the answer or echo did carry showed the write
|
|
460
|
-
twice until the rest arrived. The carried rows now go into `OverlayEntry.confirmed`, and
|
|
461
|
-
`replayOverlays` leaves them to synced truth. Only the rows still waiting show the twin.
|
|
462
|
-
- `local(tx, input)` is pure and CONVERGENT: no I/O, no `Date.now()`, no `Math.random()`, and
|
|
463
|
-
applying it over its own result changes nothing. The overlay replays it on every server update.
|
|
464
|
-
- Anything a component reads is a **getter or an accessor**, never a value snapshotted at hook time:
|
|
465
|
-
a plain field cannot re-render. `MutatorLike.local` is declared with method syntax so a twin
|
|
466
|
-
typed over its own `tx` assigns with no cast.
|
|
467
|
-
- `useQuery`'s input is read once, at call time. There is no reactive runtime here to re-run it.
|
|
468
|
-
- Every subscription handle client code gets back — `LiveHandle` (`subscribeLive`), the
|
|
469
|
-
`useQuery`/`useRecord` accessors, `Unsubscribe` (`client.subscribe(topic, …)`) — is `Disposable`,
|
|
470
|
-
and `[Symbol.dispose]` is the exact same function reference as the release, never a second
|
|
471
|
-
teardown path. Pinned in `type-pins.ts`.
|
|
472
|
-
- Type claims about the hook go in `type-pins.ts`, never in a `.test.ts` — `tsconfig.json` excludes
|
|
473
|
-
test files, so `tsc -b` never reads one and an assertion written there can never fail.
|
|
474
|
-
- **`backoffDelay` is `@ultimat3/core`'s, and `attempt + 1` is the whole of the seam** (`As of
|
|
475
|
-
2026-08-23`). A client counts its first reconnect as attempt **0** and core counts the first wait
|
|
476
|
-
as attempt **1**, so dropping the shift doubles every reconnect delay in the framework —
|
|
477
|
-
silently, and only under the load this file exists to survive.
|
|
478
|
-
`thundering-herd-core-parity.test.ts` pins it with the numbers (`[500, 1_000, 2_000, 4_000]`) and
|
|
479
|
-
with 17,280 comparisons across every jitter mode, base, cap, factor, attempt and roll.
|
|
480
|
-
`JitterMode` is core's type re-exported and `Rng` is core's `Random` — never a second spelling of
|
|
481
|
-
either. `drainPlan()` and `AcceptBudget` are NOT backoff and keep their own arithmetic: a slot in
|
|
482
|
-
a spread and a token bucket's refusal delay are different questions. `bun run flight-copies` is the
|
|
483
|
-
guard: a second curve-and-jitter function anywhere in `packages/*/src` is a build error, matched
|
|
484
|
-
on the literal shape rather than the name.
|
|
485
|
-
- **The SHARED window read carries a deadline, and it frees the SLOT — it cannot cancel the read**
|
|
486
|
-
(`As of 2026-08-23`). One `definition.snapshot` that never settled pinned `entry.reading` for the
|
|
487
|
-
life of the process, and every later cold subscriber joined a promise nothing would ever resolve:
|
|
488
|
-
one wedged read took every future subscriber of that query id with it. `startRead` now races the
|
|
489
|
-
read against `entry.schedule(…, entry.readDeadlineMs)`, default `DEFAULT_READ_DEADLINE_MS` (30s),
|
|
490
|
-
injectable per entry and through `new LiveQueryRegistry({ readDeadlineMs, schedule })`. Three
|
|
491
|
-
rules, each with its own test in `query-window.test.ts` and each proven by mutation:
|
|
492
|
-
- **A race, not just an eviction.** Freeing the slot alone leaves every caller ALREADY joined
|
|
493
|
-
awaiting a promise nothing settles; they are told instead, with `X_TIMEOUT`.
|
|
494
|
-
- **`X_TIMEOUT`, not a silent empty window.** A superseded read is discarded silently because a
|
|
495
|
-
strictly better window already exists to serve from; a timed-out read has none, and serving
|
|
496
|
-
`rows: []` as the whole result set is the exact fault `live-query.ts`'s `#read` refuses. The
|
|
497
|
-
rejecting-read path (`readSnapshot`'s catch) is the shape this matches, not the superseded one.
|
|
498
|
-
- **`stale` is put back**, for `readSnapshot`'s reason: a read that did not answer must not leave
|
|
499
|
-
the window looking authoritative.
|
|
500
|
-
There is **no "off" spelling** — a shared read with no deadline is the defect itself — and a `0`,
|
|
501
|
-
negative, `NaN` or `Infinity` value falls back to the default rather than to "now".
|
|
502
|
-
- The client owns its own reconnect: a closed socket arms **one** timer through the injected
|
|
503
|
-
`Scheduler`, and that timer calls `connect()`. `reconnectAt` is the render half and never the
|
|
504
|
-
mechanism — publishing it without arming anything is exactly the bug that shipped. Rules that
|
|
505
|
-
hold the arming together: `onClose` nulls `#socket` (a retained dead socket makes `#send` a
|
|
506
|
-
silent no-op), it only schedules when nothing is armed (a `reconnect` frame arms the node's
|
|
507
|
-
spread slot *before* closing, and a local backoff would overwrite it), and `close()` cancels —
|
|
508
|
-
a client whose owner is gone must stop dialling, while `connect()` starts it over. A close speaks
|
|
509
|
-
only for **its own** socket: `onClose` returns before touching any state when `#socket` is no
|
|
510
|
-
longer the socket that closed, because a replaced socket closing late must not mark the live
|
|
511
|
-
connection offline or arm a backoff behind it — and `close()` therefore reports its subscriptions
|
|
512
|
-
offline itself. A dial that throws inside the timer arms the next attempt and is **reported
|
|
513
|
-
through `onError`** (default `console.error`; never `logger`, whose writer is `process.stderr`
|
|
514
|
-
and this is browser code): a socket constructor may refuse, one refusal ending the chain is the
|
|
515
|
-
same outage as never arming, and nothing awaits a timer — a throw out of one is an uncaught
|
|
516
|
-
exception that can kill the process that was going to retry. Only the timer owns the chain and
|
|
517
|
-
only the timer reports — a `connect()` the app called itself throws to the app and arms nothing.
|
|
518
|
-
- **A `sid` is CLIENT data, so a subscription is keyed by `(socket, sid)` — never by `sid` alone.**
|
|
519
|
-
`LiveQueryRegistry.unsubscribe(socketId, sid)` and `.subscription(socketId, sid)` both take the
|
|
520
|
-
owner. Keyed by the sid alone, socket B reusing socket A's sid overwrote A's slot: A's
|
|
521
|
-
subscription stayed in its query entry's `subscribers` map with nothing able to reach it, so
|
|
522
|
-
`unsubscribeSocket(A)` freed nothing, `subscribers.size` never hit zero, and the entry's matcher
|
|
523
|
-
and shared window were pinned for the process's life while every change fanned out to a dead
|
|
524
|
-
socket. A `{op:'drop', sid}` frame from B ended A's stream with no error on either side.
|
|
525
|
-
`sync-node` passes `socket.id` on the drop path for that reason. Reusing a sid the SAME socket
|
|
526
|
-
already holds is `X_SUBSCRIPTION_ID_TAKEN` — refused rather than replaced, because replacing is
|
|
527
|
-
the strand. `subscription-book.ts` owns that identity and is the only place it is spelled: the
|
|
528
|
-
query entry's own `subscribers` map takes the same composite key, so one `unsubscribe` reaches
|
|
529
|
-
both by one identity.
|
|
530
|
-
- **`connect()` closes the socket it is replacing, and a frame speaks only for its own socket.**
|
|
531
|
-
A remount calling `connect()` on a live client left the previous socket open: its `onMessage`
|
|
532
|
-
kept folding patches into the live registrations, and the node held two sockets for one client —
|
|
533
|
-
double presence membership, double fanout — until the tab closed. `#socket` is nulled before the
|
|
534
|
-
close so the corpse's `onClose` takes its early return, and `onMessage` carries the same identity
|
|
535
|
-
guard `onClose` already had.
|
|
536
|
-
- **The grant is recorded BEFORE `server.upgrade`, and released on the path that never opens.**
|
|
537
|
-
Bun runs `websocket.open` SYNCHRONOUSLY inside `server.upgrade` and does not return until it has
|
|
538
|
-
(bun 1.4.0), and `open` is where `sync-node` reads the `GrantBook` for the socket's actor.
|
|
539
|
-
Recorded on the line after the upgrade — which it was — every authenticated socket on a real node
|
|
540
|
-
carried `actor: null`: `ChannelHub.#authorize` denied every topic with `X_TOPIC_FORBIDDEN`,
|
|
541
|
-
`authorize`/`visible` decided about nobody, `maxPerTenant` never applied and `hello.actorId` was
|
|
542
|
-
null. It did not self-repair, because `GrantBook.expired()` skips a grant with no `expiresAt` —
|
|
543
|
-
the shape `authenticate: async () => ({ actor })` produces. `onUngranted` is what makes the
|
|
544
|
-
correct order safe (only a `close` callback deletes a grant, and an upgrade that never took gets
|
|
545
|
-
no callback); it is REQUIRED on `UpgradeDeps`, so a second host of `handleUpgrade` cannot forget
|
|
546
|
-
it. The harness is half the rule: `sync-node-auth.test.ts`'s `upgradeTarget()` returned `true`
|
|
547
|
-
without ever calling `open`, so the bug was invisible to every test here — it now opens the socket
|
|
548
|
-
inside `upgrade()`, the way Bun does.
|
|
549
|
-
- **Every `socket.send` on the node reads its answer, and what a `false` costs is decided per
|
|
550
|
-
frame.** A subscribe reply is REPAIRABLE and was the silent one: `registry.subscribe` has already
|
|
551
|
-
seated the subscription and cleared its desync mark, so a dropped snapshot left the server
|
|
552
|
-
believing a client holding no rows was in sync, and every later change reached it as a patch
|
|
553
|
-
folded onto nothing — on a healthy socket, forever. It is marked desynced, exactly as
|
|
554
|
-
`live-fanout` marks a lost patch. A `rebase`/`ack` has nothing to mark (the node keeps no
|
|
555
|
-
per-mutation state) and a client only returns an `inflight` mutation to its queue when the
|
|
556
|
-
connection dies, so an undeliverable settlement **closes the socket**: the reconnect requeues and
|
|
557
|
-
replays it under the same idempotency key, and acking a rebase that never left is the
|
|
558
|
-
rebase-before-ack order defeated one frame later. A presence roster has no repair at all — the
|
|
559
|
-
membership is already on the shared set and the client's next heartbeat re-rosters — so it is
|
|
560
|
-
logged (`sync.presence_roster_dropped`). The drain's `reconnect` frame is the socket's slot in the
|
|
561
|
-
spread and nothing re-sends it, so `drain()` returns `DrainedSocket[]` with `notified` per socket
|
|
562
|
-
and logs `sync.drain_frames_dropped`.
|
|
563
|
-
- **A socket's actor comes from `createSyncNode({ authenticate })` and from nowhere else.** The node
|
|
564
|
-
imports no authenticator — the app supplies one — and it runs
|
|
565
|
-
on the upgrade *before* `server.upgrade`, so a refused credential never costs a websocket.
|
|
566
|
-
`null` is a **decision** (401, `X_SOCKET_UNAUTHENTICATED`, a client fault that pages nobody); a
|
|
567
|
-
throw is a **failure** (503, `X_SOCKET_AUTH_UNAVAILABLE`, reported) — the same rule the row gate
|
|
568
|
-
follows, one layer out. Absent, every socket is anonymous and `start()` warns: that node is
|
|
569
|
-
single-tenant, and `hub.guard('org.*.feed', ({ actor }) => actor?.orgId === …)` denies everyone.
|
|
570
|
-
The actor is written in exactly one place, the `GrantBook`; `WsData` deliberately carries none,
|
|
571
|
-
because two spellings of one identity disagree the moment a re-auth renews one of them.
|
|
572
|
-
- **A grant expires; a socket does not.** `authenticate` answers a `SyncGrant`, not an `Actor`: a
|
|
573
|
-
15-minute token on a socket that stays up for hours was authorized once and served forever, and
|
|
574
|
-
an active client never idles out either — every inbound frame `touch()`es it. The node re-decides
|
|
575
|
-
an expired grant on an interval and then calls both halves that already existed and had no
|
|
576
|
-
caller: `hub.onActorChange` (topics) and `registry.reauthorize` (subscriptions). `refresh` is the
|
|
577
|
-
app's closure, so the framework retains no credential of its own — re-reading the upgrade
|
|
578
|
-
`Request` would mean holding one per socket. No `refresh` = close with `1008` and let the client
|
|
579
|
-
re-dial. A `refresh` that **raises** keeps the socket and retries: a token service timing out is
|
|
580
|
-
not a revocation.
|
|
581
|
-
- **`desynced` is a mark with a reader.** It is written when a patch is dropped by backpressure,
|
|
582
|
-
when a gate fails, when a window loses its tail and when a re-auth survives; the *next* delivery
|
|
583
|
-
serves that subscriber a fresh snapshot out of the shared window (no DB read) and only then
|
|
584
|
-
clears it. A snapshot the socket refuses leaves the mark, which is the state it is in. Four
|
|
585
|
-
writers and no reader was a subscription that stayed permanently and silently stale on a healthy
|
|
586
|
-
socket, with the server knowing and the client not.
|
|
587
|
-
- **`result.refill` is checked BEFORE the mark, because a repair out of a guessed window clears it.**
|
|
588
|
-
The word "fresh" above is load-bearing: when the matcher lost the window's tail, `entry.rows` is a
|
|
589
|
-
guess, and the same fanout that refuses to send a *patch* derived from it was resnapshotting every
|
|
590
|
-
already-desynced subscriber out of it — and clearing the one mark that would have made the next
|
|
591
|
-
change re-read. That subscriber is then recorded as repaired against rows nothing trusts and gets
|
|
592
|
-
a patch, not the snapshot it is still owed, from the refilled window. A lost tail degrades every
|
|
593
|
-
subscriber the same way, whatever each was holding, and they are all repaired on the next change
|
|
594
|
-
after `refillWindowInLane` has replaced the window. `live-fanout.test.ts` pins both halves.
|
|
595
|
-
- **A change the window already holds is refused on the way in.** The replicator guarded duplicates
|
|
596
|
-
and out-of-order on the *publish* side; `entry.lsn = change.lsn` was unconditional on the
|
|
597
|
-
*consume* side, so a redelivery rewound every subscriber's cursor. `change.lsn <= entry.lsn` is
|
|
598
|
-
dropped and counted as `staleChanges`.
|
|
599
|
-
- **A gap in the change stream is detected, not assumed away.** Fanout is core NATS — at most once
|
|
600
|
-
— and an lsn cannot reveal a gap, because a WAL position is a byte offset and every legitimate
|
|
601
|
-
next change is already an arbitrary jump. The replicator stamps `producer` + `seq`; a skipped
|
|
602
|
-
sequence marks every window `stale` and every subscriber desynced, and the next change to each
|
|
603
|
-
query re-reads. Both fields are optional on the bus: a publisher that does not sequence detects
|
|
604
|
-
nothing rather than crying gap, and a *new* producer restarts the count rather than reading as
|
|
605
|
-
one. A stale window is replaced in the lane (`refillWindowInLane`) — `fillWindow` takes the
|
|
606
|
-
entry's own lane and a lane is not reentrant.
|
|
607
|
-
- **A hub that closed opens nothing, and `#open` is the only thing that can enforce it.** `close()`
|
|
608
|
-
walks `#bridges` and then clears it, which reaches every bridge that is open and none that is
|
|
609
|
-
still opening: a reservation an in-flight `subscribe` has taken is `sub === null`, so
|
|
610
|
-
`unsubscribeWhenOpen` does nothing to it and `clear()` drops the entry. The transport then hands a
|
|
611
|
-
live subscription to a `Bridge` nothing can name — `#release` looks the topic up, misses and
|
|
612
|
-
returns — and its handler keeps calling `deliver` for the life of the process. The same orphan the
|
|
613
|
-
`Bridge` comment describes, one state earlier. So `close()` sets `#closed` **before** the walk and
|
|
614
|
-
`#open` closes its own subscription when it lands after one, dropping the entry with it so a
|
|
615
|
-
second post-close subscribe opens and closes its own rather than double-unsubscribing this handle.
|
|
616
|
-
It then **raises** `X_TRANSPORT_UNAVAILABLE` rather than returning: returning let `subscribe` fall
|
|
617
|
-
through to `joinTopic`, so the socket became a member of a topic nothing on this node is bridged
|
|
618
|
-
to — silent for the life of the connection, no error on either side, and no reason for the client
|
|
619
|
-
to redial. Reachable between `hub.close()` inside `node.drain()` and the last in-flight subscribe.
|
|
620
|
-
`#release` takes the bridge the caller reserved for the same reason: after `close()` cleared the
|
|
621
|
-
table, that topic name may hold a bridge a LATER subscribe opened, and releasing by name alone
|
|
622
|
-
decrements somebody else's refcount.
|
|
623
|
-
- Deny by default on topics. No guard = `X_TOPIC_FORBIDDEN`.
|
|
624
|
-
- **A guard that FAILS is not a guard that denied — the hub's copy of the rule the row gate already
|
|
625
|
-
follows.** On `onActorChange` (the re-auth pass) only a denial unsubscribes; anything else keeps
|
|
626
|
-
the topic, increments `guardFailures` and logs `channel.guard_failed`. A guard is app code and may
|
|
627
|
-
reach a database, so `catch { unsubscribe }` reported a store that timed out as a revoked grant —
|
|
628
|
-
every topic on every re-authenticated socket on the node, silently, with the client never told to
|
|
629
|
-
resubscribe. The initial `subscribe` is deliberately NOT split: there is no subscription to keep,
|
|
630
|
-
so a raising guard refuses that subscribe and the client hears about it.
|
|
631
|
-
- **One conflict vocabulary, and it is not declared here.** A mutator's `conflict` is
|
|
632
|
-
`ConflictPolicy` from `@ultimat3/core`, over ROWS, and `RecordStore.settle` hands it to core's
|
|
633
|
-
`resolveConflict` with the overlay's row and the adopted server row — in that order. Until 21.0.0
|
|
634
|
-
there were two spellings bridged by a filter in `hooks.ts` that dropped one and handed the other
|
|
635
|
-
a `{ local, base, server }` bag, so no merge an app declared ever decided a row. A server delete,
|
|
636
|
-
or a row the client never held, is not a conflict: the server's answer stands, no merge runs.
|
|
637
|
-
- **Inbound frames run in a lane, and the lane is NEVER the socket.** `sync-node.message` dispatches
|
|
638
|
-
every frame as `void (async () => routeFrame(…))()`, so nothing upstream orders them. A global
|
|
639
|
-
per-socket lane would put every frame behind the slowest one, and the slowest one is a subscribe's
|
|
640
|
-
snapshot read — a DB round trip every reconnecting client pays once per live query, which is the
|
|
641
|
-
restart storm this framework is measured on. `mutate` is one lane per socket, `subscribe` is
|
|
642
|
-
`sub:<sid>` or `topic:<name>`, everything else is unlaned (`frame-lanes.ts`). A lane exists only
|
|
643
|
-
while work is queued on it: keyed by a client-chosen sid, a lane that outlived its work is an
|
|
644
|
-
unbounded map one socket grows at will.
|
|
645
|
-
- **A cap is a RESERVATION taken before the first await, never a check.** A lane makes concurrent
|
|
646
|
-
frames sequential and N sequential subscribes still pass a check-then-act cap N times — and the
|
|
647
|
-
per-tenant cap spans sockets, where no lane can see it at all. `SubscriptionBook.reserve(socket,
|
|
648
|
-
sid)` decides the sid claim, `maxPerSocket` and `maxPerTenant` in one synchronous step;
|
|
649
|
-
`ChannelHub.subscribe` does the same for `maxTopicsPerSocket`, `maxTopicsPerNode` and the node's
|
|
650
|
-
bridge slot, before the guard is awaited. The tenant is captured, not re-derived — a re-auth may
|
|
651
|
-
`retenant` the socket while the read is in flight, and the release has to give the slot back to
|
|
652
|
-
the tenant that took it. Released in a `finally`, and releasing twice is a no-op.
|
|
653
|
-
- **Bun's native pub/sub is deleted, not wired.** Nothing here publishes to a native topic and
|
|
654
|
-
nothing will: a native publish cannot be refused per socket, cannot report the frame it dropped
|
|
655
|
-
and cannot mark a subscriber desynced. `SocketRegistry.deliver` is the one fanout path.
|
|
656
|
-
`WsLike.subscribe`/`unsubscribe` stay declared and unused — a tracked app implements the
|
|
657
|
-
interface structurally, so removing the members is that app's typecheck failure — and the
|
|
658
|
-
declaration says so, because a member that looks live is one someone will call.
|
|
659
|
-
- **A dropped `records` frame is counted AND repaired (plan 101, slice 10).** Every declared
|
|
660
|
-
channel carries a per-node `seq` and `epoch`; a frame backpressure refuses marks that socket
|
|
661
|
-
gapped on that channel, and the node sends `replay-gap` the moment the socket drains (the
|
|
662
|
-
websocket `drain` handler → `gapRepairs.repairAll`), counted as `channel_replay_gaps_total` beside
|
|
663
|
-
`channel_frames_dropped_total`. The client re-runs the channel's catch-up query, holds every frame
|
|
664
|
-
that arrives meanwhile, and applies those over the read. The server owns the gap verdict: a
|
|
665
|
-
numeric hole on its own is never one (a row this socket may not see is skipped for it). A resume
|
|
666
|
-
sends `since` = the highest seq with no hole below it; duplicates are dropped, a replay that
|
|
667
|
-
fills a hole is applied (`client-channels.ts`).
|
|
668
|
-
- **A channel is a DECLARATION, never a topic string.** `channel(name, { params, policy, catchUp,
|
|
669
|
-
records?, events? })` spells the topic; a subscribe names the declaration and its params, and a
|
|
670
|
-
name no declaration carries is `X_TOPIC_FORBIDDEN`. Presence rides the channel's `events` frames
|
|
671
|
-
(`readPresence`) — only a channel declared `events: true` has a room. `useChannel` / `usePresence`
|
|
672
|
-
hold one membership per topic per page, however many components ask.
|
|
673
|
-
- **One socket per ORIGIN and principal, in a SharedWorker (plan 101, slice 11).** `socket-engine.ts`
|
|
674
|
-
holds the real socket and talks to every tab only over a `MessagePort`; to each tab's `LiveClient`
|
|
675
|
-
its port IS a socket (`socket-host.ts`'s virtual socket). Channel wants are reference-counted
|
|
676
|
-
across ports and every server frame is ROUTED to the ports that want it; live-query sids carry the
|
|
677
|
-
port. A tab's own beat is the engine's ping: a port silent for `REAP_AFTER_BEATS` is a closed tab
|
|
678
|
-
and is reaped. When the real socket drops, every tab's virtual socket closes and each tab
|
|
679
|
-
resubscribes from its OWN cursors — the engine keeps nothing that could go stale. No
|
|
680
|
-
`SharedWorker`, a constructor that throws, or no built worker: the in-page host runs the same
|
|
681
|
-
engine over a `MessageChannel`, one socket per tab. The worker is named by principal, so two
|
|
682
|
-
principals never share a socket.
|
|
683
|
-
- **A qid is `@ultimat3/query`'s `queryHash(name, input)`, and this package derives none of its
|
|
684
|
-
own — `As of 2026-08`.** `qidOf` was the same two lines over a local copy of the canonical form
|
|
685
|
-
(`stableDigest(canonicalJson(input))`), and `canonicalJson`/`stableDigest` were this package's
|
|
686
|
-
third copy of what `@ultimat3/action` and `@ultimat3/query` also each held. They had already
|
|
687
|
-
diverged: `{ a: undefined, b: 1 }` gave `feed:eb8ed3ccb5023093` from `queryHash` and
|
|
688
|
-
`feed:c0bf82ad036cb0a5` from `qidOf`, because query's walk drops an `undefined`-valued key and
|
|
689
|
-
this one rendered `"a":null`. The two are COMPARED in one flow — `@ultimat3/query`'s `planResume`
|
|
690
|
-
decides refetch-vs-resume by comparing a cursor's `queryHash` against the query's, while
|
|
691
|
-
`liveQueryDefinition` keys the shared window by the qid — so keeping both correct was never the
|
|
692
|
-
option; the first time either moved, every resume decision and every window lookup were keyed
|
|
693
|
-
differently. `realtime -> query` is the one declared sideways edge and this package already
|
|
694
|
-
imports it. **`fnv1a` is gone too, `As of 2026-08-24`**: it stayed for one job — the cursor's
|
|
695
|
-
result-set digest — and `LiveCursor.digest` was deleted for having no reader, so this package now
|
|
696
|
-
owns no hash at all. A 32-bit hash nothing calls is one the next caller reaches for as a sharing
|
|
697
|
-
key, which is the single thing `json.test.ts` used to pin it against.
|
|
698
|
-
`live-contract.test.ts` is the pin — it reads `registry.subscriberCount(queryHash(name, input))`
|
|
699
|
-
through a real subscribe, so a local derivation fails it. Cost of the move: none observable on the
|
|
700
|
-
server. Every qid a node computes comes from a DECODED frame, and `JSON.parse` produces no
|
|
701
|
-
`undefined`, no `Date`, no `Map` and no `Set` — the four values the two forms disagree about — so
|
|
702
|
-
no live subscription re-keyed and nothing re-snapshotted.
|
|
703
|
-
- **The canonical form is injective over the values it accepts, and `JSON.stringify` is not** —
|
|
704
|
-
the reason that survives the move, now `@ultimat3/core`'s to enforce. `JSON.stringify` answers
|
|
705
|
-
`"null"` for `NaN` and `±Infinity` and `"0"` for `-0`, so four distinct inputs hashed to one qid
|
|
706
|
-
— and a qid *hit* hands the joiner the first subscriber's compiled source, matcher and seated
|
|
707
|
-
window. Bare `NaN` / `Infinity` / `-Infinity` / `-0` tokens are emitted instead; they are not
|
|
708
|
-
valid JSON, which is correct, because that output is hashed and never parsed. Exposure is
|
|
709
|
-
narrower than it looks and the tests say so rather than overclaiming: `NaN` and `±Infinity` have
|
|
710
|
-
no JSON spelling and so cannot arrive on a `subscribe` frame — they reach the hash only from a
|
|
711
|
-
caller building `input` in JS. **`-0` is wire-reachable**: `JSON.parse('{"a":-0}')` answers `-0`.
|
|
712
|
-
- **Refusing new sockets and draining the ones you have are two shutdown phases.** `stopAccepting()`
|
|
713
|
-
is the `accept` phase: `ready = false`, `/readyz` 503, a late upgrade shed with `retry-after-ms`,
|
|
714
|
-
and every socket untouched — a draining node still owes its clients their patches, and `stop()` is
|
|
715
|
-
what releases the change subscription carrying them. `drain()` + `stop()` are the `close` phase.
|
|
716
|
-
Registered with no phase, both landed in `close` and the node upgraded new websockets until the
|
|
717
|
-
very end. `listenSyncNode` unregisters both on `stop()`.
|
|
718
|
-
- **Readiness AND the connection cap are asked twice, because `authenticate` is app code with an
|
|
719
|
-
await in it.** A request that passed the checks at the top of `handleUpgrade` can be parked in a
|
|
720
|
-
token service when SIGTERM lands, and the `accept` phase is over by the time it reaches
|
|
721
|
-
`server.upgrade` — one more socket on a node the load balancer has already stopped routing to, so
|
|
722
|
-
nothing takes it over. `ready` and the socket count are therefore **functions** on `UpgradeDeps`,
|
|
723
|
-
not values read once.
|
|
724
|
-
**`socketCount()` was the half that was read once and never re-asked** (2026-08-23), which is the
|
|
725
|
-
same staleness with a worse blast radius: a restart storm dials every client of a dead node at
|
|
726
|
-
this one at once and each parks in the token service having passed the cap while the node still
|
|
727
|
-
held nothing, so `maxConnections: 2` with ten parked upgrades took **ten** sockets — reproduced,
|
|
728
|
-
`upgraded 10, shed 0`. Sound because there is no await between the recheck and `server.upgrade`,
|
|
729
|
-
and the count moves INSIDE it: Bun runs `websocket.open` synchronously there, which is where
|
|
730
|
-
`sockets.add` runs. The recheck sheds with the same 503 + `retry-after-ms` and takes no second
|
|
731
|
-
`tryAccept()`: that budget was spent.
|
|
732
|
-
- **A client `send` that returned is not an acknowledgement.** A browser `WebSocket.send` on a
|
|
733
|
-
CLOSING socket discards the frame and returns normally, so a drained mutation is `inflight` until
|
|
734
|
-
the server settles it or `requeueInflight` returns it. Only `pending` is sendable, `drain()` is one
|
|
735
|
-
chained pass at a time (two overlapping passes put one key on the wire twice, and a later pass can
|
|
736
|
-
overtake the one ahead of it), and backpressure over `MAX_BUFFERED_BYTES` declines rather than
|
|
737
|
-
fails — the mutation stays pending and the pass stops instead of reordering the ones behind it.
|
|
738
|
-
- **The lane orders passes; it does not order a socket death, so the queue carries an epoch.**
|
|
739
|
-
`requeueInflight` is not a pass and cannot reach into one parked at `await send(...)`: it hands
|
|
740
|
-
back what was on the dead socket, the parked pass resumes and marks everything *behind* that
|
|
741
|
-
mutation `inflight` for a connection that is gone. `#sendable` excludes `inflight`, so the next
|
|
742
|
-
drain skips them, no ack ever arrives and the writes are lost — invariant 3 inverted. `#epoch` is
|
|
743
|
-
bumped before the requeue scan and read at the top of every `#pass` iteration; a pass whose epoch
|
|
744
|
-
went stale returns and leaves the rest `pending` for the connection that arms the next one.
|
|
745
|
-
- **`#persist` hands the store a SNAPSHOT, never the live entries.** `QueueStore.save` is a durable
|
|
746
|
-
write (OPFS, IndexedDB) and may await before it reads. Given the array itself, a store that
|
|
747
|
-
resolves after the next pass has moved on persists a status that was never true when it was
|
|
748
|
-
called — and `inflight` is the one a reload cannot recover from.
|
|
749
|
-
- **A reconnect replays registrations AND topics, and every socket handler carries the identity
|
|
750
|
-
guard.** A reconnect is one `hello` plus one frame per thing this client holds: a `subscribe` per
|
|
751
|
-
registration, carrying that registration's cursor, and a `subscribe` per topic. Topic membership
|
|
752
|
-
is state on the node's socket, so a channel is silent from the first reconnect while its handler
|
|
753
|
-
is still installed — and its presence membership is swept — unless every one is re-announced.
|
|
754
|
-
`onOpen` needed the `#socket !== socket` guard `onMessage` and `onClose` already had: a replaced
|
|
755
|
-
socket opening late marked the connection up and replayed every subscription onto the current one.
|
|
756
|
-
- **`hello` carries NO cursors, and `HelloFrame.resume` is deleted (2026-08).** It was filled by
|
|
757
|
-
every client on open and read by nobody — the node replied `resume: []` and decided resume per
|
|
758
|
-
subscription from the `subscribe` frame — so every reconnect shipped each cursor twice, up to 512
|
|
759
|
-
ids each, in the restart storm this package is measured on. Wiring it was the wrong half of the
|
|
760
|
-
choice: a cursor's `qid` is `` `${name}:${fingerprint(input)}` ``, so a node reading a resume list
|
|
761
|
-
recovers the query **name** — it is the plaintext prefix — but never the `input`, which is the half
|
|
762
|
-
every decision needs. Without it `definition.authorize({ actor, input })` cannot run and no entry
|
|
763
|
-
can be built; the qid names a window but not a decision, and the retained window holds pre-policy
|
|
764
|
-
patches, so answering from it at `hello` time means answering before the per-subscriber
|
|
765
|
-
authorization pass, for a subscription that does not exist yet. It could only ever restate,
|
|
766
|
-
unauthorized, what `subscribe` decides with the input in hand — and it could not even save the
|
|
767
|
-
bytes, because the cursor still has to ride its `subscribe`. Two places deciding one thing is what axiom 1 refuses. **`PROTOCOL_VERSION` did NOT
|
|
768
|
-
move**, same rule as `snapshot.entity`: `decode` is a whitelist, so a new node drops an old
|
|
769
|
-
client's `resume` and an old node reads a new client's omission as the empty list it always got.
|
|
770
|
-
The one deploy of skew costs nothing in either direction.
|
|
771
|
-
- **The client beats, because only the client can end a half-open socket.** `heartbeatMs` (default
|
|
772
|
-
`DEFAULT_HEARTBEAT_MS`, 15s; `0` disables) sends a `hello` — byte-identical to the opening one,
|
|
773
|
-
since the frame has no resume list to leave out — plus one subscribe frame per topic, which is the
|
|
774
|
-
node's presence heartbeat. It is **not** how a deploy is noticed: `socket.skewed` compares the
|
|
775
|
-
build the client claims (the `hello`'s `buildId`, which `sawHello` records on every one — the
|
|
776
|
-
latest is the record — or `?build=` on the dial) against this node's; a client says the same
|
|
777
|
-
build on every beat and the node's never moves while the socket is open, so every `hello` on one
|
|
778
|
-
socket answers the same forever and `update-available` reaches a client on the
|
|
779
|
-
socket it opens against the *new* node. The hello IS read — until 2026-09-07 only the dial was,
|
|
780
|
-
and a dial without `?build=` was recorded as this node's own id, so a client naming its build only
|
|
781
|
-
in the frame was current forever. Two silent windows and the client closes with `4000` and
|
|
782
|
-
arms the reconnect. It is one
|
|
783
|
-
self-re-arming tick on the injected `Scheduler`, not an interval: a client is either beating on a
|
|
784
|
-
live socket or backing off toward a new one, never both. The 15s is the client's OWN number:
|
|
785
|
-
`realtime.heartbeatMs` was a `RealtimeConfig` key read by nothing and it is **deleted**
|
|
786
|
-
(2026-08-19). The server half of the beat stays derived — `PresenceRegistry.heartbeatMs` is
|
|
787
|
-
`max(1000, floor(ttlMs / 3))`, the same rule `idleSweepPeriodMs` follows, because a second knob
|
|
788
|
-
is a second number that can disagree with the one it is a fraction of.
|
|
789
|
-
- **Every question a hot path asks is indexed, never scanned.** `SubscriptionBook` keeps
|
|
790
|
-
`#bySocket` and a per-tenant count beside `#bySid`, and `SocketRegistry` keeps `#byTopic` beside
|
|
791
|
-
the socket table. Both replaced a walk of the whole node that ran once per socket or once per
|
|
792
|
-
frame: `ofSocket` filtered a copy of every subscription (100,000 entries measured at **17.7s** of
|
|
793
|
-
blocking work per teardown or re-auth sweep — a deploy or a batch of grants expiring together is
|
|
794
|
-
the whole trigger), and the per-tenant cap walked the same map on **every subscribe frame**
|
|
795
|
-
(7.96 ms each at that size). A new index goes where the deaths are seen: topic membership is the
|
|
796
|
-
registry's because `remove` is the one path a close, a drain and the idle sweep all take, and
|
|
797
|
-
`joinTopic`/`leaveTopic` are the only way to change it — two call sites for one membership is how
|
|
798
|
-
an index goes wrong. When an actor changes, `reauthorize` calls `book.retenant(socket)`: an index
|
|
799
|
-
nobody updates is a count that drifts for the rest of the process.
|
|
800
|
-
- **A ceiling per resource, and the wire's are not options.** `README.md` has the table. The rule
|
|
801
|
-
behind it: the accept budget bounds the accept *rate*, so the *count* needs its own
|
|
802
|
-
(`maxConnections`, shed as the same 503 + `retry-after-ms`); a socket that is open needs a frame
|
|
803
|
-
budget (`socket.frameBudget`, checked at the top of `routeFrame` **before `touch()`** — a frame
|
|
804
|
-
this node refuses must not renew the idle window); and anything a client sizes is bounded in
|
|
805
|
-
`decode` by `FRAME_LIMITS`, which a caller may narrow but never widen. `list()` takes a required
|
|
806
|
-
`max` so a new array field on a new frame cannot ship without someone choosing its size.
|
|
807
|
-
`input` is walked ITERATIVELY: the thing being refused is a stack overflow in `canonicalJson`, so
|
|
808
|
-
a recursive check would be the same crash one frame earlier.
|
|
809
|
-
- **Every ceiling on a socket `sync` builds is reachable from `createSyncNode`.** The node
|
|
810
|
-
constructs every `SyncSocket` it holds, so a `SyncSocketOptions` the node does not forward is a
|
|
811
|
-
number an operator can only change by abandoning `createSyncNode` — which is what
|
|
812
|
-
`maxBufferedBytes` and `maxDroppedFrames` were until 2026-08. Forwarded the same way
|
|
813
|
-
`maxFramesPerSecond`/`frameBurst` already are (`...(x === undefined ? {} : { x })`, so an unset
|
|
814
|
-
option keeps `SyncSocket`'s own default rather than overwriting it with `undefined`).
|
|
815
|
-
- **A `SubscriptionLimitError` names the knob, never the default.** `knob` defaults to
|
|
816
|
-
`maxPerSocket`/`maxPerTenant`, which are `LiveQueryRegistry`'s — so the channel hub's per-socket
|
|
817
|
-
*topic* cap, thrown without one, told an operator to move a number in a different constructor
|
|
818
|
-
that would not have helped. Every throw site passes `knob` explicitly (`maxTopicsPerSocket`,
|
|
819
|
-
`maxTopicsPerNode`, `maxEntries`, `maxPerSocket`, `maxPerTenant`); `channel.test.ts` asserts the
|
|
820
|
-
two hub ones against the option names, because a fix line naming the wrong setting is worse than
|
|
821
|
-
no fix line — it is an instruction that runs and changes nothing.
|
|
822
|
-
- **Retained memory is bounded by BYTES.** `RingChangeBuffer` keeps the patch-count cap as a
|
|
823
|
-
*replay* bound (what a delta resume costs to fold) and adds the byte budgets as the memory one —
|
|
824
|
-
`packages/cache/src/lru.ts:1-2` states why: 4,096 queries x 1,024 patches is 4.19M retained rows
|
|
825
|
-
and no number of bytes at all. `forget(qid)` is called by `LiveQueryRegistry.unsubscribe` when the
|
|
826
|
-
last subscriber of a query id goes; it had no caller, so the ring outlived the entry. It is
|
|
827
|
-
also called by `#dropIfUnheld` on the subscribe path, `As of 2026-09-06`: an entry is created
|
|
828
|
-
BEFORE the snapshot read that fills it, so a cold subscribe the database refused left an entry
|
|
829
|
-
with no subscriber and no removal path — `unsubscribe` can only reach one through a subscription
|
|
830
|
-
that was never attached. `maxEntries` cold failures were therefore enough to answer
|
|
831
|
-
`X_SUBSCRIPTION_LIMIT` to every later subscriber for the life of the process, after the database
|
|
832
|
-
had recovered, because `qid` derives from client-chosen input and distinct inputs mint distinct
|
|
833
|
-
orphans. Dropped only when the entry is still the one in the table, holds no subscriber and has
|
|
834
|
-
no read in flight — a read published on it belongs to a concurrent subscriber that has not
|
|
835
|
-
attached yet, and dropping it there would hand that subscriber a window no change reaches.
|
|
836
|
-
- **A channel patch id carries the NODE that minted it** (`As of 2026-09-06`). `ChannelHub`'s
|
|
837
|
-
`#sequence` counts within one process, and the patch id was that counter alone — so two `sync`
|
|
838
|
-
replicas publishing to one topic minted the same id for the same subscriber, and a channel has
|
|
839
|
-
no cursor and no re-snapshot, so nothing downstream can repair a collision. `nodeId` defaults to
|
|
840
|
-
a per-hub `uuid()` and is declarable (the pod name) when an operator should be able to say which
|
|
841
|
-
node published a frame. The frame's `lsn` is deliberately left as the per-process counter:
|
|
842
|
-
nothing reads a channel frame's lsn as an order across nodes, and `client-frames.ts` advances a
|
|
843
|
-
cursor only for a registered live query, never for a topic.
|
|
844
|
-
- **An error never renders a value that carries a credential.** `parsePgUrl` names `DATABASE_URL`
|
|
845
|
-
rather than echoing the URL it refused — an error reaches a log, `--json`, an agent transcript
|
|
846
|
-
and a ticket, and the password is in the string. Same rule as `packages/mail/src/driver-smtp.ts`.
|
|
847
|
-
- **A full presence frame is capped and says so; the set behind it is never capped.** `roster()` is
|
|
848
|
-
what a frame carries (`maxMembers`, 256, plus `total`); `list()` stays whole because the sweep
|
|
849
|
-
differences it, and a short list would report every member past the cap as having left. `total`
|
|
850
|
-
is set on `sync` only — a `join`/`leave`/`update` frame is a delta, and a count beside one reads
|
|
851
|
-
as truncation.
|
|
852
|
-
- **One node per topic sweeps.** Every node sweeping every room it has seen is one full-set read
|
|
853
|
-
multiplied by the fleet, and the same `leave` frame published N times. The election needs no
|
|
854
|
-
compare-and-set the shared store does not have: the lease key is a *keyed set*, so each node's
|
|
855
|
-
claim is its own member and the leader is the lowest id every claimant can see. Eventually
|
|
856
|
-
consistent on purpose — the worst case is a duplicate `leave` for someone already gone.
|
|
857
|
-
- **Money is THREE physical columns on the wire too, and a live row must equal a repository row.**
|
|
858
|
-
`entityRow` folds `<p>_minor`/`<p>_currency`/`<p>_scale` into one property. It matched two, so a
|
|
859
|
-
scaled amount arrived at every subscriber unscaled *and* carrying a stray physical `priceScale`
|
|
860
|
-
beside `price` — one row, two shapes, no error anywhere. NULL and absent both mean **no `scale`
|
|
861
|
-
key**, never `0` (that is whole units, a 100x reinterpretation of an ordinary price), which is
|
|
862
|
-
exactly what `@ultimat3/entity`'s `moneyOf` does. That equality is the pin:
|
|
863
|
-
`pg-entity-row-parity.test.ts` reads one physical row through both surfaces — this package's fold
|
|
864
|
-
and a real `postgresRepo` — and asserts one object, each side absolutely as well as against the
|
|
865
|
-
other, because equality alone is satisfied by both failing open together. It is the one test here
|
|
866
|
-
that imports `@ultimat3/entity` (tier 2, a legal downward edge, test-only: `*.test.ts` never
|
|
867
|
-
ships), and it has to, or the thing being compared against is a copy of the reader instead of the
|
|
868
|
-
reader. The `0…15` scale bound stays `@ultimat3/schema`'s — enforced by the column CHECK and by
|
|
869
|
-
`parseScale`, never restated here.
|
|
870
|
-
- Never a bare `Error`. Never `any`. Never `Date.now()` — take a `Clock` (`clock.now()` is a `Date`;
|
|
871
|
-
use `monotonic()` for durations).
|
|
872
|
-
- **A test fixture standing in for a FOREIGN error extends `Error` on purpose, and that is not the
|
|
873
|
-
bare-`Error` rule being broken.** `PoolTimeout`, `Denied`, `MutationFailed` and `ThirdPartySdkError`
|
|
874
|
-
(nine sites across `realtime`, `db` and `ai`) simulate a driver, a policy library or an app's
|
|
875
|
-
`onMutate` — values this package did not construct and must handle anyway. `isPolicyDenial`,
|
|
876
|
-
`stringField` and `renderThrowable` all exist *because* such values arrive; rebuilt as
|
|
877
|
-
`UltimateError`s the fixture would prove the framework handles its own errors, which is the
|
|
878
|
-
"equality satisfied by both sides failing open together" failure the row-parity test names. The
|
|
879
|
-
rule governs what this package **throws**, never what a test hands it.
|
|
120
|
+
- **`selectTransport(env, realtime)` is the one place `realtime.transport` + env become a bus**; the
|
|
121
|
+
KV bucket and presence TTL come back with it. The config decides, the env supplies (22.0.0):
|
|
122
|
+
`'nats'` dials the variable `urlEnv` names and refuses (`X_CONFIG_INVALID`) when unset; `'memory'`
|
|
123
|
+
refuses a set `NATS_URL` (or the named variable).
|
|
124
|
+
- **`nats` is imported only by `nats-lib-client.ts`**; reconnect and re-subscription are the
|
|
125
|
+
library's job — `NatsTransport` must never grow subscription bookkeeping. What stays above the
|
|
126
|
+
port: the herd jitter (`reconnectDelayHandler`) and the KV semantics presence needs
|
|
127
|
+
(`nats-jetstream.ts`, `nats-kv.ts`). `nats-fake.ts` is a bus with server semantics.
|
|
128
|
+
- **Nothing leaves `NatsTransport` uncoded** — `#translating` wraps the port's synchronous
|
|
129
|
+
`publish`/`subscribe` refusals as `X_TRANSPORT_UNAVAILABLE`; an `UltimateError` passes through.
|
|
130
|
+
`transport-parity.test.ts` asserts both transports, and that the wrap still delivers. `#ensure`
|
|
131
|
+
reuses a mid-reconnect client on purpose.
|
|
132
|
+
- **A bucket name is checked before interpolation** (`assertBucket`, `X_TRANSPORT_PROTOCOL`); a
|
|
133
|
+
presence key or member id is base64url-encoded (`encodeToken`), never validated.
|
|
134
|
+
- Presence lives in `transport.shared`. The sync node is `PresenceRegistry`'s only caller:
|
|
135
|
+
subscribing to a topic IS joining, repeating the frame is the heartbeat, dropping or closing is the
|
|
136
|
+
leave. Expiry is silent, so the node sweeps on an interval, and **one node per topic sweeps** (lowest
|
|
137
|
+
id in the lease key's keyed set). `roster()` caps a frame at `maxMembers` (256) with `total`;
|
|
138
|
+
`list()` is never capped. `total` is set on `sync` frames only.
|
|
139
|
+
- **A socket the node evicts ITSELF goes through `evict(socket, code, reason)`** (the full
|
|
140
|
+
`teardown`), never `sockets.remove`. **`drain()` waits for the presence leaves it started** —
|
|
141
|
+
`evictInChunks` (`drain-evictions.ts`, `DRAIN_EVICT_CHUNK`, `allSettled`).
|
|
142
|
+
- **The idle sweep is armed by `start()`** (a quarter of `idleTimeoutMs`, floored at 1 s, `.unref()`)
|
|
143
|
+
and measures on `Clock.monotonic()` (`lastSeenMonotonicMs`). `SocketRegistry.idle()` is a query;
|
|
144
|
+
the node evicts.
|
|
145
|
+
- **`drain()` and `stop()` both call one idempotent `release()`**; `drain()` releases after the
|
|
146
|
+
sockets are gone and before `hub.close()`.
|
|
147
|
+
- **Refusing new sockets and draining are two phases**: `stopAccepting()` is `accept`, `drain()` +
|
|
148
|
+
`stop()` are `close`. `listenSyncNode` unregisters both on `stop()`.
|
|
149
|
+
- **A hub that closed opens nothing**: `close()` sets `#closed` before the walk; `#open` closes a
|
|
150
|
+
late subscription and RAISES `X_TRANSPORT_UNAVAILABLE`. `#release` takes the reserved bridge, never
|
|
151
|
+
a name.
|
|
152
|
+
- **A socket's actor comes from `createSyncNode({ authenticate })` only**, run before
|
|
153
|
+
`server.upgrade`. `null` = 401 `X_SOCKET_UNAUTHENTICATED`; a throw = 503
|
|
154
|
+
`X_SOCKET_AUTH_UNAVAILABLE`. Absent = anonymous, and `start()` warns. The actor lives only in the
|
|
155
|
+
`GrantBook`.
|
|
156
|
+
- **The grant is recorded BEFORE `server.upgrade`** (Bun runs `open` synchronously inside it) and
|
|
157
|
+
released on every path that never opens — `onUngranted` is REQUIRED on `UpgradeDeps`, and a
|
|
158
|
+
`server.upgrade` that THROWS releases too. `sync-node-auth.test.ts`'s harness opens inside
|
|
159
|
+
`upgrade()`, as Bun does.
|
|
160
|
+
- **A grant expires; a socket does not**: expired grants are re-decided on an interval through
|
|
161
|
+
`hub.onActorChange` and `registry.reauthorize`. No `refresh` = close with `1008`; a `refresh` that
|
|
162
|
+
raises keeps the socket and retries.
|
|
163
|
+
- **Every `socket.send` reads its answer**: a dropped subscribe reply marks the subscription desynced;
|
|
164
|
+
a dropped presence roster is logged (`sync.presence_roster_dropped`); `drain()` returns
|
|
165
|
+
`DrainedSocket[]` with `notified` and logs `sync.drain_frames_dropped`.
|
|
166
|
+
- **Inbound frames run in a lane, never the socket**: `subscribe` is `sub:<sid>` or `topic:<name>`,
|
|
167
|
+
everything else unlaned (`frame-lanes.ts`); a lane exists only while work is queued.
|
|
168
|
+
- **Bun's native pub/sub is deleted**; `SocketRegistry.deliver` is the one fanout path.
|
|
169
|
+
`WsLike.subscribe`/`unsubscribe` stay declared for structural implementers.
|
|
170
|
+
- **A dropped `records` frame is counted AND repaired**: per-node `seq`/`epoch` per channel; a refused
|
|
171
|
+
frame marks the socket gapped, and the websocket `drain` handler sends `replay-gap`
|
|
172
|
+
(`channel_replay_gaps_total` beside `channel_frames_dropped_total`). The client re-runs the
|
|
173
|
+
channel's `catchUp`, holding frames meanwhile.
|
|
174
|
+
- **A channel is a DECLARATION**: `channel(name, { params, policy, catchUp, records?, events? })`.
|
|
175
|
+
Deny by default, and `policy` is REQUIRED (`X_CHANNEL_DECLARATION_INVALID`; a public channel says
|
|
176
|
+
`policy: allow('public')`). Presence rides `events: true` channels.
|
|
177
|
+
- **A guard that FAILS keeps the topic on re-auth** (`guardFailures`, `channel.guard_failed`); the
|
|
178
|
+
initial subscribe still refuses.
|
|
179
|
+
- **A channel patch id carries the node** (`nodeId`, default a per-hub `uuid()`).
|
|
180
|
+
- **An error never renders a credential**: `parsePgUrl` names `DATABASE_URL`, never the URL.
|
|
880
181
|
|
|
881
|
-
##
|
|
182
|
+
## Replication
|
|
882
183
|
|
|
883
|
-
|
|
884
|
-
|
|
184
|
+
- Exactly one `replicator` per DB, by a session-level advisory lock. **`start()` and
|
|
185
|
+
`PgAdvisoryLock.tryAcquire` are MEMOISED** (guard and promise published synchronously, cleared on
|
|
186
|
+
settle); `stop()`/`release()` await the in-flight one. A start that FAILS hands the lock back, and
|
|
187
|
+
`running` is set only once the feed pumps.
|
|
188
|
+
- **The pump has one way out, `#die`**, which records the failure, stops the confirm timer, closes and
|
|
189
|
+
nulls the connection. `start()` awaits the previous pump before it dials. **`stop()` releases
|
|
190
|
+
everything before it reports anything.**
|
|
191
|
+
- **`REPLICA IDENTITY FULL` is checked at preflight, warned, and counted — never thrown** — ahead of
|
|
192
|
+
`pg_create_logical_replication_slot`. `ReplicationStreamStats.partialBefore` reads
|
|
193
|
+
`PgRelation.replicaIdentity`, never the tuple.
|
|
194
|
+
- **Every session pins `datestyle=ISO`, `intervalstyle=postgres`, `extra_float_digits=3`** in the
|
|
195
|
+
startup packet (`pg-connection.ts`, pinned by `pg-connection.test.ts`), byte for byte what
|
|
196
|
+
Postgres' own walreceiver sends.
|
|
197
|
+
- A change lsn is `<16 hex commit position><8 hex row position>`; never order by either half alone,
|
|
198
|
+
never depend on wall time or a process counter.
|
|
199
|
+
- Slot, publication and entity names match `[a-z_][a-z0-9_]*` before interpolation — a security
|
|
200
|
+
boundary. A SQLSTATE is data: `pg-wire.ts`'s `FIXES` is read with `Object.hasOwn`.
|
|
201
|
+
- **A live row equals a repository row**: `pg-entity-row.ts` decodes through `@ultimat3/entity`'s
|
|
202
|
+
own `decodeRow` / `entityForTable`, so money (three physical columns), scale and every other kind
|
|
203
|
+
fold exactly as `postgresRepo` folds them.
|
|
204
|
+
- A write's name rides the WAL: `pg-replication.ts` reads it off the transaction's opening
|
|
205
|
+
`pg_logical_emit_message` (prefix `WRITE_ORIGIN_WAL_PREFIX`; `START_REPLICATION` asks
|
|
206
|
+
`messages 'true'`).
|
|
885
207
|
|
|
886
|
-
|
|
887
|
-
|---|---|---|---|
|
|
888
|
-
| `useRecord` | 34,838 | 19,404 | **13,226** |
|
|
889
|
-
| `useMutation` | 37,073 | 35,480 | **29,907** |
|
|
890
|
-
| `useQuery` | 64,183 | 62,596 | **56,647** |
|
|
891
|
-
| `useChannel` | 62,155 | 60,566 | **54,621** |
|
|
892
|
-
| `@ultimat3/realtime/boot` (once per page, cached immutable) | — | 34,884 | not re-measured |
|
|
208
|
+
## The page
|
|
893
209
|
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
|
|
898
|
-
|
|
899
|
-
`
|
|
210
|
+
- **One record store per PAGE (`record-store.ts`)**, keyed `type:key` with the SERVER's key
|
|
211
|
+
(`live-record-type.ts`); the browser never derives one. SYNCED truth plus an OVERLAY of pending
|
|
212
|
+
optimistic writes, REPLAYED over synced truth whenever it moves. Installed as core's `RecordSink` on
|
|
213
|
+
`pageClient().store`.
|
|
214
|
+
- **Page state lives on `globalThis[Symbol.for('ultimate.realtime')]` (`page-store.ts`)**, never module
|
|
215
|
+
scope — every island is its own bundle (`page-client.test.ts` builds the package twice). Never
|
|
216
|
+
`instanceof` across copies.
|
|
217
|
+
- **The signal factory is per BUNDLE**: `installRealtime({ signal, sync })` (`reactivity.ts`). No
|
|
218
|
+
install and a DOM is `X_REALTIME_UNINSTALLED`; no DOM is a server render — hooks answer
|
|
219
|
+
`pending`/online and create no page state. `useRecord` and `useQuery` return `AsyncState` accessors.
|
|
220
|
+
- **One socket per page, built by the first live hook (`page-socket.ts`)**; `browser-socket.ts`
|
|
221
|
+
holds the framework's only `new WebSocket`. `hasPageSocket()` lives in `page-store.ts`. A principal
|
|
222
|
+
change (`rescope`) clears the store and redials.
|
|
223
|
+
- **One socket per ORIGIN and principal, in a SharedWorker**: `socket-engine.ts` holds it and talks
|
|
224
|
+
to tabs over `MessagePort`s (`socket-host.ts`'s virtual socket); wants are ref-counted and frames
|
|
225
|
+
routed; a port silent for `REAP_AFTER_BEATS` is reaped; each tab resubscribes from its OWN cursors.
|
|
226
|
+
No `SharedWorker` → the in-page host runs the same engine over a `MessageChannel`.
|
|
227
|
+
- **The socket is READ-ONLY (protocol 3)**. A write is `useMutation`: the twin into the overlay, then
|
|
228
|
+
`POST actionPath(name)` through core's `clientTransport` with an idempotency key; the answer's
|
|
229
|
+
records are adopted inside the transport call, before `settle`. An `ack` is only a refusal.
|
|
230
|
+
- **`LiveClient` holds no signal** — plain reads plus `onStatus` / `onChange`; hooks wrap them.
|
|
231
|
+
- **An answer that did not carry a touched row keeps that row's overlay** until the server's next row
|
|
232
|
+
for it, bounded by `DEFAULT_AWAIT_SERVER_MS` (10 s).
|
|
233
|
+
- **A `records` frame names the write that produced it** (`ChannelRecordsFrame.write =
|
|
234
|
+
writeDigest(idempotencyKey)`, never the key). `RecordStore.push` digests every overlay key before
|
|
235
|
+
the request leaves (`record-names.ts`); `client-channels.ts` settles inside the frame's own batch.
|
|
236
|
+
A non-digest `write` is a protocol error (`wire-channel.ts`) and dropped on the bus. Rows an answer
|
|
237
|
+
or echo partly confirmed go to `OverlayEntry.confirmed` and stop painting. `write-echo.test.ts`.
|
|
238
|
+
- `local(tx, input)` is pure and CONVERGENT: no I/O, no `Date.now()`, no `Math.random()`.
|
|
239
|
+
- **One conflict vocabulary**: `ConflictPolicy` from `@ultimat3/core`, resolved by core's
|
|
240
|
+
`resolveConflict(overlayRow, serverRow)` in `RecordStore.settle`. A server delete is not a conflict.
|
|
241
|
+
- Anything a component reads is a getter or accessor. `MutatorLike.local` uses method syntax.
|
|
242
|
+
`useQuery`'s input is read once, at call time.
|
|
243
|
+
- Every subscription handle (`LiveHandle`, the hook accessors, `Unsubscribe`) is `Disposable`, and
|
|
244
|
+
`[Symbol.dispose]` is the same function as the release. Type claims go in `type-pins.ts`.
|
|
245
|
+
- **The outbox**: only `pending` is sendable; `drain()` is one chained pass at a time; backpressure
|
|
246
|
+
over `MAX_BUFFERED_BYTES` declines rather than fails. `#epoch` is bumped before the requeue scan and
|
|
247
|
+
read every pass iteration. `#persist` hands the store SNAPSHOTS by KEY (`QueueChange`); one tab
|
|
248
|
+
drains at a time (`navigator.locks`, `page-outbox.ts`'s `exclusive`) and resets a stored `inflight`
|
|
249
|
+
to `pending` under the lock.
|
|
900
250
|
|
|
901
|
-
|
|
902
|
-
`feed` 95,983 → 94,174, `like` 84,351 → 82,503; `settings` and `contact-sales` reach no realtime
|
|
903
|
-
and are unchanged.
|
|
251
|
+
## The client connection
|
|
904
252
|
|
|
905
|
-
- **The
|
|
906
|
-
|
|
907
|
-
|
|
908
|
-
the
|
|
909
|
-
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
913
|
-
|
|
914
|
-
|
|
915
|
-
|
|
916
|
-
|
|
917
|
-
|
|
918
|
-
|
|
919
|
-
|
|
920
|
-
|
|
921
|
-
|
|
922
|
-
`
|
|
923
|
-
|
|
924
|
-
`
|
|
925
|
-
`examples/dummy` via `buildIslands`: `feed` 101,261 → 92,973, `like` 86,325 → 78,050.
|
|
926
|
-
- **A browser path never loads realtime's code table.** The refusals a browser can reach live in
|
|
927
|
-
`page-errors.ts`; `errors.ts` re-exports them and keeps the table and its one
|
|
928
|
-
`registerErrorCodes()`. Measured the same day, same method: `useRecord` 21,629 → 18,600,
|
|
929
|
-
`useMutation` 38,225 → 35,194, `useQuery` 65,964 → 62,935 (`useChannel` 60,903 after).
|
|
930
|
-
`page-errors-bundle.test.ts` fails if `errors.ts` comes back into `useRecord` or `useMutation`.
|
|
931
|
-
In a browser that loaded no table a code titles itself from its name; `code`, `cause` and `fix`
|
|
932
|
-
are unchanged. The ~7.6 kB of core's `UltimateError` chain that was left under `useRecord` is now
|
|
933
|
-
the class and the lookup alone (~2.4 kB): core's titles table is an anchor only its barrel loads.
|
|
253
|
+
- **The client owns its reconnect**: a closed socket arms ONE timer through the injected `Scheduler`
|
|
254
|
+
that calls `connect()`. `onClose` nulls `#socket`, schedules only when nothing is armed (a
|
|
255
|
+
`reconnect` frame's slot wins), and speaks only for its own socket; `close()` cancels. A dial that
|
|
256
|
+
throws in the timer is reported through `onError` (default `console.error`) and arms the next.
|
|
257
|
+
- **`connect()` closes the socket it replaces**; `onOpen`, `onMessage` and `onClose` all carry the
|
|
258
|
+
identity guard.
|
|
259
|
+
- **A reconnect replays registrations AND topics** — one `hello`, one `subscribe` per registration
|
|
260
|
+
(with its cursor) and per topic.
|
|
261
|
+
- **`hello` carries NO cursors** (`HelloFrame.resume` is deleted): a cursor's qid names a window but
|
|
262
|
+
not the input a decision needs.
|
|
263
|
+
- **The client beats** (`heartbeatMs`, default `DEFAULT_HEARTBEAT_MS` 15 s, `0` disables): a `hello`
|
|
264
|
+
plus one subscribe per topic (the presence heartbeat). Two silent windows close with `4000`. The
|
|
265
|
+
node's `socket.skewed` compares the build the `hello` claims against its own. The server beat is
|
|
266
|
+
derived: `PresenceRegistry.heartbeatMs = max(1000, floor(ttlMs / 3))`.
|
|
267
|
+
- **There is ONE `backoffDelay`, `@ultimat3/core`'s, counted from 1.** `policyDelay(policy, attempt,
|
|
268
|
+
rng)` (internal, not on the barrel) maps a `BackoffPolicy` onto it; a 0-based counter adds 1 at the
|
|
269
|
+
call site (`client.ts`, `socket-engine.ts`, `replicator.ts`'s 0-based `retryDelayMs`).
|
|
270
|
+
`client-reconnect.test.ts`, `socket-engine-reconnect.test.ts`, `nats-transport.test.ts` pin the
|
|
271
|
+
first wait at the base. `drainPlan()` and `AcceptBudget` keep their own arithmetic.
|
|
272
|
+
`bun run flight-copies` refuses a second curve.
|
|
934
273
|
|
|
935
|
-
##
|
|
274
|
+
## Rules for code here
|
|
275
|
+
|
|
276
|
+
- Never a bare `Error`. Never `any`. Never `Date.now()` — take a `Clock` (`monotonic()` for durations).
|
|
277
|
+
- **A test fixture standing in for a FOREIGN error extends `Error` on purpose** (`PoolTimeout`,
|
|
278
|
+
`Denied`, `ThirdPartySdkError`): it simulates a value this package did not construct. The rule
|
|
279
|
+
governs what this package **throws**, never what a test hands it.
|
|
280
|
+
|
|
281
|
+
## Browser bytes, per hook (`As of 2026-09-22`)
|
|
282
|
+
|
|
283
|
+
`bun build --target=browser --minify`, one entry importing one hook from the barrel. Re-measure
|
|
284
|
+
before quoting.
|
|
936
285
|
|
|
937
|
-
|
|
|
286
|
+
| Hook | Current |
|
|
938
287
|
|---|---|
|
|
939
|
-
| `
|
|
940
|
-
| `
|
|
941
|
-
| `
|
|
942
|
-
| `
|
|
943
|
-
|
|
944
|
-
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
948
|
-
|
|
949
|
-
|
|
950
|
-
|
|
951
|
-
|
|
952
|
-
|
|
953
|
-
|
|
954
|
-
|
|
955
|
-
|
|
956
|
-
|
|
957
|
-
|
|
958
|
-
|
|
959
|
-
|
|
960
|
-
|
|
961
|
-
|
|
962
|
-
| `use-record.ts` / `use-query.ts` / `use-mutation.ts` / `use-connection.ts` / `use-channel.ts` | the hooks — the only surface an island calls |
|
|
963
|
-
| `client-channels.ts` | the client's declared channels: one membership per topic, the seq/epoch cursor, the catch-up read and the frames held during it |
|
|
964
|
-
| `socket-engine.ts` / `socket-host.ts` / `sync-worker.ts` | the one socket per origin: the port-driven engine (dial, beat, redial, reap), the tab's host choice and virtual socket, the SharedWorker entry (`./sync-worker`) |
|
|
965
|
-
| `socket-port.ts` / `socket-routes.ts` | the engine ⇄ tab port messages; the multiplexing — one membership per topic, every frame routed only to the ports that want it |
|
|
966
|
-
| `sync-meta.ts` | the sync target and worker URL read off the document's `<meta>` tags (core's names) |
|
|
967
|
-
| `live-record-type.ts` | the live path's record type and record key per table, from the entity's own projection |
|
|
968
|
-
| `live-rows.ts` | one live subscription's window over the store — its record type, its order, its retain/release, and `Registration` itself |
|
|
969
|
-
| `offline-queue.ts` | the durable outbox queue; `page-outbox.ts` opens one per principal and replays it in order over HTTP |
|
|
970
|
-
| `client.ts` / `sync-node.ts` | the two halves — connection lifecycle and subscriptions; the socket carries no writes |
|
|
971
|
-
| `sync-auth.ts` | what a socket's identity IS (`SyncGrant`), the book that holds one per socket, and the pass that re-decides an expired one |
|
|
972
|
-
| `sync-frames.ts` | what a RECEIVED frame does to server state — the node's inbound surface, and the mirror of `client-frames.ts` |
|
|
973
|
-
| `sync-upgrade.ts` | the node's HTTP surface: `/healthz`, `/readyz`, load shedding, and the authenticated upgrade — `WsData` and `UpgradeTarget` are declared with the decision that builds them |
|
|
974
|
-
| `sync-listen.ts` | binding a node to `Bun.serve` and to the shutdown hook — the only `Bun.serve` in the package |
|
|
975
|
-
| `drain-evictions.ts` | evicting every socket a drain holds, in bounded chunks, and waiting out the presence leaves each eviction started |
|
|
976
|
-
| `query-window.ts` | the shared pre-policy window per query id: built once, read once for N subscribers, and replaced when it is known to be wrong |
|
|
977
|
-
| `client-frames.ts` | what a RECEIVED frame does to client state, and `ClientFrameTarget` — the only inbound surface the client exposes. The mirror of `sync-frames.ts` |
|
|
978
|
-
| `client-harness-fixture.ts` | the injected socket + scheduler + harness both client suites drive. Excluded from the tarball |
|
|
979
|
-
| `hooks-fixture.ts` | the page harness the hook and frame suites drive: a fake socket, a page reset, a fetch double. Excluded from the tarball |
|
|
980
|
-
| `subscription-book.ts` | who holds which subscription, keyed by `(socket, sid)`, and the per-socket/per-tenant caps answered from it |
|
|
981
|
-
| `apply-patches.ts` | folding a patch list onto a row list (`applyPatches`) or onto ids alone (`orderAfterPatches`, what a window uses) — the client's one stateless piece, and one fold, not two |
|
|
982
|
-
| `type-pins.ts` | compile-time assertions `tsc` checks — the hooks answer `AsyncState`, a query ref has no server field, every handle is `Disposable` |
|
|
983
|
-
| `window-lock.ts` | one FIFO lane per query id — the only thing that orders a fanout |
|
|
984
|
-
| `frame-lanes.ts` | the order one socket's INBOUND frames are applied in, and the lane key each kind belongs to. `WindowLock` again, keyed differently — and it bounds no cap |
|
|
985
|
-
| `live-fanout.ts` | what one change does inside one entry's lane: match, fold, one policy pass per subscriber, and the re-snapshot that repairs a desynced one |
|
|
986
|
-
| `client-heartbeat.ts` | when to beat and when to give up. A policy, which is why it is not in `client.ts`'s connection lifecycle |
|
|
987
|
-
| `client-contract.ts` | the client's injected shapes — `ClientSocket`, `LiveClientOptions`, `LiveHandle` — declared apart from the class that consumes them |
|
|
988
|
-
| `policy-gate.ts` | the only authz seam |
|
|
989
|
-
| `subscriber-gate.ts` | the per-subscriber pass of a definition's row policy, and its two counters — `rowsDenied` and `gateFailures`. Evaluates no policy of its own |
|
|
990
|
-
| `live-contract.ts` | what a live query IS: `LiveQueryDefinition`, `SnapshotResult`, `LiveSubscription`. Four modules need the shape and none of them needs the registry that runs it. The **id** is not here and not anywhere in this package — it is `@ultimat3/query`'s `queryHash` |
|
|
991
|
-
| `json.ts` | the wire's value types. **No hash**: the canonical form and the sharing-key hash are `@ultimat3/core`'s (`canonicalJson`, `fingerprint`), and the 32-bit `fnv1a` that stayed for `LiveCursor.digest` went with it (2026-08-24) |
|
|
992
|
-
| `live-definition.ts` | the only bridge from a declared `query({ live: true })` to a registrable definition — and `policy-gate.ts`'s only caller |
|
|
993
|
-
| `matcher-bridge.ts` | the only `@ultimat3/query` matcher seam — and where a patch row is narrowed to the columns the query returned |
|
|
288
|
+
| `useRecord` | 13,226 |
|
|
289
|
+
| `useMutation` | 21,725 (after the outbox left islands) |
|
|
290
|
+
| `useQuery` | 47,696 |
|
|
291
|
+
| `useChannel` | 45,585 |
|
|
292
|
+
|
|
293
|
+
- **The disk boot is ONE page script (`boot.ts`, `./boot`)**, served at `/_x/page-boot/<hash>.js`
|
|
294
|
+
on a document with the scope tag and a realtime island; it wipes every other principal's stored
|
|
295
|
+
scope, then restores. `booted` reads its promise off `globalThis[Symbol.for('ultimate.page-boot')]`.
|
|
296
|
+
- **An island never carries the outbox**: `boot.ts` builds it; `useMutation` and the page socket read
|
|
297
|
+
it through `outbox-slot.ts` after `page.booted`. No boot ⇒ a refused write is rejected, not queued.
|
|
298
|
+
|
|
299
|
+
## Map
|
|
300
|
+
|
|
301
|
+
`index.ts` / `server.ts` are the two barrels. Server side: `sync-node.ts` (+ `sync-auth`,
|
|
302
|
+
`sync-frames`, `sync-upgrade`, `sync-listen`, `drain-evictions`), `channel.ts`, `presence.ts`,
|
|
303
|
+
`socket.ts`, `live-query.ts` (+ `live-definition`, `query-window`, `live-fanout`, `window-lock`,
|
|
304
|
+
`subscriber-gate`, `policy-gate` — the only authz seam — and `matcher-bridge`), `replicator.ts` /
|
|
305
|
+
`live-replicator.ts`, the Postgres client (`pg-*.ts`, `pgoutput.ts`), the bus (`nats-*.ts`,
|
|
306
|
+
`transport-env.ts`, `fanout.ts`). Client side: `client*.ts`, `socket-engine.ts` / `socket-host.ts` /
|
|
307
|
+
`sync-worker.ts`, `record-*.ts`, `page-*.ts`, `boot.ts`, `offline-queue.ts`, `use-*.ts`,
|
|
308
|
+
`reactivity.ts`. Shared: `sync-protocol.ts` (the wire), `cursor.ts`, `change-buffer.ts`,
|
|
309
|
+
`thundering-herd.ts`, `live-contract.ts`, `json.ts` (no hash), `type-pins.ts`. Each file's header
|
|
310
|
+
states its one job.
|
|
994
311
|
|
|
995
312
|
## Commands
|
|
996
313
|
|
|
@@ -999,45 +316,12 @@ bun test packages/realtime/src # from the REPO ROOT, never from packa
|
|
|
999
316
|
bun run typecheck
|
|
1000
317
|
```
|
|
1001
318
|
|
|
1002
|
-
|
|
1003
|
-
and Bun reads `bunfig.toml` from the cwd, so `bun test` inside this directory loads none and six
|
|
1004
|
-
tests fail on a missing matcher — this package's suite reading red for the shell it was run in.
|
|
1005
|
-
CI's `package` job spawns `bun test packages/<pkg>` with `cwd` at the root for the same reason.
|
|
1006
|
-
|
|
1007
|
-
Changing a frame shape means adding a fixture to `sync-protocol.test.ts` — the round-trip test
|
|
1008
|
-
fails if a kind has no fixture — and bumping `PROTOCOL_VERSION` **when the change makes an old
|
|
1009
|
-
frame unreadable in either direction**. An *additive optional* field (`snapshot.entity`, 2026-08)
|
|
1010
|
-
is not that: `decode` builds a whitelist, so an old client drops it and a new client reads its
|
|
1011
|
-
absence as a defined answer. Neither is *removing a field nothing read* (`hello.resume`, 2026-08):
|
|
1012
|
-
the same whitelist drops an old client's copy, and a new client's omission decodes to what the
|
|
1013
|
-
field always held. Bumping for either refuses every in-flight client on a rolling deploy and buys
|
|
1014
|
-
nothing — the version guards incompatibility, not novelty. Removing a field something *does* read
|
|
1015
|
-
is the opposite case and bumps.
|
|
1016
|
-
|
|
1017
|
-
**"Nothing read it" is decided by the DECODER, not by the callers — and that half is what moved
|
|
1018
|
-
`PROTOCOL_VERSION` to 2 (2026-08-24, BREAKING).** `hello.resume` was free because `decode` read it
|
|
1019
|
-
through `list()`, which answers `[]` for an absent field; `cursor.digest` and `cursor.count` were
|
|
1020
|
-
read through `str()` and `num()`, which **throw**. So deleting two fields no *caller* consumed
|
|
1021
|
-
still made the frame unreadable to a peer one deploy behind — in **both** directions, since a
|
|
1022
|
-
cursor rides the client's `subscribe` and the node's `snapshot`. Without the bump the skew shows up
|
|
1023
|
-
as a per-frame `field "digest" must be a string`, which is the same refusal with none of the
|
|
1024
|
-
instruction. Before claiming a removal is free, read the field's line in `decode`: a `list()` is
|
|
1025
|
-
free, a `str()`/`num()` is a bump.
|
|
1026
|
-
|
|
1027
|
-
**A patch carries the result set's columns, never the table's** — `narrowRow` in
|
|
1028
|
-
`matcher-bridge.ts`, `As of 2026-08-20`. A `ChangeEvent` carries the whole TABLE row (that is what
|
|
1029
|
-
logical replication emits, and what `@ultimat3/entity`'s `setRowObserver` emits), while a live
|
|
1030
|
-
query's result set is whatever its `sql` returned. Every patch used to forward the change row
|
|
1031
|
-
unnarrowed, so a column a projection exists to withhold went out on the socket the moment it
|
|
1032
|
-
CHANGED — `examples/dummy`'s feed projects ten columns and one publish delivered `updatedAt` to
|
|
1033
|
-
every subscriber (#230). The per-subscriber gate cannot help: it decides whether a ROW is delivered,
|
|
1034
|
-
never which of its columns.
|
|
319
|
+
`bunfig.toml`'s preload installs `@ultimat3/testing`'s matchers and Bun reads it from the cwd.
|
|
1035
320
|
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
321
|
+
Changing a frame shape means a fixture in `sync-protocol.test.ts` (the round-trip test fails
|
|
322
|
+
without one) and bumping `PROTOCOL_VERSION` **when an old frame becomes unreadable in either
|
|
323
|
+
direction**. `decode` is a whitelist: an additive optional field, or removing a field read through
|
|
324
|
+
`list()`, is free; removing a field read through `str()`/`num()` (which throw) is a bump. Read the
|
|
325
|
+
field's line in `decode` before claiming a removal is free.
|
|
1039
326
|
|
|
1040
|
-
|
|
1041
|
-
lives inside the `sql` provider's closure and there is nothing static to read it from. Learned and
|
|
1042
|
-
kept rather than re-derived per fanout, because the case the window's own rows cannot answer is an
|
|
1043
|
-
EMPTY window — the first row to arrive would otherwise go out whole.
|
|
327
|
+
Why each rule above is shaped the way it is: [`docs/history/realtime.md`](../../docs/history/realtime.md).
|