@ultimat3/realtime 23.0.0 → 25.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +118 -126
- package/README.md +229 -54
- package/package.json +9 -6
- package/src/advisory-lock.ts +62 -0
- package/src/boot.ts +25 -15
- package/src/change-buffer.ts +96 -41
- package/src/changefeed-env.ts +9 -9
- package/src/changefeed.ts +81 -19
- package/src/channel-authz.ts +48 -17
- package/src/channel-describe.ts +1 -1
- package/src/channel-latch.ts +52 -0
- package/src/channel-logs.ts +36 -15
- package/src/channel-presence.ts +3 -3
- package/src/channel-records.ts +35 -4
- package/src/channel-ref.ts +2 -2
- package/src/channel-registry.ts +2 -2
- package/src/channel-sids.ts +37 -0
- package/src/channel.ts +148 -48
- package/src/client-channels.ts +12 -1
- package/src/client-contract.ts +4 -1
- package/src/client-frames.ts +16 -3
- package/src/client-heartbeat.ts +39 -9
- package/src/client.ts +30 -2
- package/src/cursor.ts +11 -0
- package/src/error-titles.ts +91 -0
- package/src/errors.ts +113 -105
- package/src/fanout.ts +13 -0
- package/src/first-paint-hold.ts +49 -0
- package/src/idb-types.ts +2 -2
- package/src/index.ts +25 -7
- package/src/json.ts +16 -3
- package/src/live-contract.ts +14 -0
- package/src/live-definition.ts +34 -9
- package/src/live-deliver.ts +55 -0
- package/src/live-fanout.ts +122 -29
- package/src/live-floor.ts +35 -0
- package/src/live-query-options.ts +44 -0
- package/src/live-query.ts +96 -175
- package/src/live-reauth.ts +127 -0
- package/src/live-replicator.ts +28 -8
- package/src/live-resume.ts +44 -8
- package/src/live-rows.ts +32 -3
- package/src/live-spend.ts +50 -0
- package/src/local-store-idb.ts +5 -0
- package/src/matcher-bridge.ts +11 -5
- package/src/nats-client.ts +37 -6
- package/src/nats-lib-client.ts +36 -7
- package/src/nats-subscriptions.ts +63 -0
- package/src/nats-transport.ts +149 -15
- package/src/offline-queue.ts +53 -5
- package/src/page-errors.ts +25 -9
- package/src/page-outbox.ts +125 -54
- package/src/page-runtime-wait.ts +72 -0
- package/src/page-runtime.ts +32 -0
- package/src/page-socket.ts +7 -16
- package/src/page-store.ts +83 -16
- package/src/pg-advisory-lock.ts +54 -5
- package/src/pg-auth.ts +9 -4
- package/src/pg-bytes.ts +4 -1
- package/src/pg-connection.ts +126 -22
- package/src/pg-entity-row.ts +16 -0
- package/src/pg-identity-tables.ts +25 -0
- package/src/pg-preflight.ts +50 -4
- package/src/pg-replication-row.ts +82 -0
- package/src/pg-replication.ts +50 -51
- package/src/pg-socket.ts +26 -7
- package/src/pg-tls.ts +18 -0
- package/src/pg-values.ts +29 -14
- package/src/pg-wire.ts +5 -3
- package/src/pgoutput.ts +19 -9
- package/src/policy-gate.ts +7 -7
- package/src/presence.ts +87 -10
- package/src/principal-sockets.ts +75 -0
- package/src/query-window.ts +17 -5
- package/src/record-persister.ts +7 -2
- package/src/record-publisher.ts +178 -0
- package/src/record-store.ts +25 -4
- package/src/record-synced.ts +43 -1
- package/src/replication-errors.ts +11 -7
- package/src/replicator-envelope.ts +160 -0
- package/src/replicator-row.ts +75 -0
- package/src/replicator.ts +313 -191
- package/src/server.ts +45 -23
- package/src/socket-defaults.ts +20 -0
- package/src/socket-engine.ts +14 -2
- package/src/socket-idle.ts +3 -1
- package/src/socket-routes.ts +8 -3
- package/src/socket.ts +14 -19
- package/src/subscriber-gate.ts +11 -1
- package/src/subscription-book.ts +129 -71
- package/src/subscription-count.ts +151 -0
- package/src/sync-actor-change.ts +56 -0
- package/src/sync-auth.ts +47 -2
- package/src/sync-bus-handlers.ts +131 -0
- package/src/sync-frames.ts +74 -21
- package/src/sync-node-bounds.ts +16 -1
- package/src/sync-node-contract.ts +36 -4
- package/src/sync-node.ts +163 -82
- package/src/sync-origin.ts +27 -12
- package/src/sync-protocol.ts +86 -16
- package/src/sync-upgrade.ts +115 -8
- package/src/type-pins.ts +9 -1
- package/src/use-channel.ts +19 -14
- package/src/use-connection.ts +3 -3
- package/src/use-mutation.ts +41 -20
- package/src/use-query.ts +11 -11
- package/src/use-record.ts +89 -10
- package/src/wire-channel.ts +18 -9
- package/src/wire-read.ts +2 -2
- package/src/idb-fake.ts +0 -133
- package/src/policy-fake.ts +0 -14
package/CLAUDE.md
CHANGED
|
@@ -7,7 +7,7 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
7
7
|
| May import | Must not |
|
|
8
8
|
|---|---|
|
|
9
9
|
| `@ultimat3/core`, `@ultimat3/query`, `@ultimat3/entity` (`/record` only, server-side) | anything tier 4+ (`render`, `pwa`, `mcp`, `ui`, `cli`) |
|
|
10
|
-
| `@ultimat3/policy` **only via** `@ultimat3/query`'s `
|
|
10
|
+
| `@ultimat3/policy` **only via** `@ultimat3/query`'s `guardQuery` for a decision (`policyCapability` / `policyPermissions` are display reads) | a second authz path of any kind |
|
|
11
11
|
| — | `solid-js` (each island bundle installs its own signal factory: `installRealtime`) |
|
|
12
12
|
| `nats` (the one external dependency, pinned exact) — from `nats-lib-client.ts` and no other file | `nats` from anywhere else. Every other file is written against the port in `nats-client.ts` |
|
|
13
13
|
|
|
@@ -21,28 +21,22 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
21
21
|
link). `packages/cli/src/realtime-browser-barrel.test.ts` bundles a `useLive`-only entry;
|
|
22
22
|
`barrel-split.test.ts` refuses a name exported from both. `./boot` and `./sync-worker` are the page
|
|
23
23
|
boot and the SharedWorker entry.
|
|
24
|
-
- **`
|
|
25
|
-
|
|
26
|
-
Postgres
|
|
27
|
-
|
|
28
|
-
- **`sideEffects` is
|
|
29
|
-
|
|
24
|
+
- **`error-titles.ts` holds the codes, titles and `registerErrorCodes()`**; `errors.ts` the server
|
|
25
|
+
refusals, `page-errors.ts` the browser ones, `realtime-error.ts` the base class (TDZ),
|
|
26
|
+
`replication-errors.ts` Postgres. All re-exported from `./errors`. A hook's chunk never loads
|
|
27
|
+
`errors.ts` (`page-errors-bundle.test.ts`).
|
|
28
|
+
- **`sideEffects` is `["./src/error-titles.ts"]`**, imported bare by the barrel (`refusalError` and
|
|
29
|
+
`page-errors.ts` title by code); never `false` or absent.
|
|
30
30
|
- **Every `@ultimat3/realtime/<subpath>` written in shipped source must be a key of `exports`**,
|
|
31
31
|
comments included — `fix-specifier.test.ts`.
|
|
32
|
-
-
|
|
33
|
-
maps it to a directory that does not exist; `scripts/**` reports `TS2307` without it).
|
|
34
|
-
- A browser file imports core from `@ultimat3/core/page` (`packages/core/src/page-bundle.test.ts`,
|
|
35
|
-
`packages/query/src/client-bundle.test.ts`).
|
|
32
|
+
- A browser file imports core from `@ultimat3/core/page` (`packages/core/src/page-bundle.test.ts`).
|
|
36
33
|
|
|
37
34
|
## Numbers and ceilings
|
|
38
35
|
|
|
39
36
|
- **Every numeric option is refused when it is not FINITE** — `@ultimat3/core`'s `finiteOption()`.
|
|
40
|
-
`bun run finite-bounds`
|
|
41
|
-
default or where a screen runs, so audit by reading the option interfaces
|
|
42
|
-
(`SubscriptionBook`'s `maxPerTenant` and `openNatsClient`'s `maxReconnectAttempts` were the two it
|
|
43
|
-
missed). `SyncGrant.expiresAt` is app DATA, not an option, and is deliberately unscreened.
|
|
37
|
+
`bun run finite-bounds` cannot see an option with no `??` default: read the option interfaces. `SyncGrant.expiresAt` is app DATA, deliberately unscreened.
|
|
44
38
|
- **A ceiling is refused where the object is BUILT**, never in a per-connection callback:
|
|
45
|
-
`socketCeilings()` (`sync-node-bounds.ts`) runs once in `
|
|
39
|
+
`socketCeilings()` (`sync-node-bounds.ts`) runs once in `syncNode`; `SyncSocket` keeps its own
|
|
46
40
|
screen because it is exported. `AcceptBudget`, `SyncSocket` and `SocketRegistry` throw
|
|
47
41
|
`X_INVARIANT` at construction on `NaN`/`±Infinity`.
|
|
48
42
|
- **A ceiling per resource, and the wire's are not options** (table in `README.md`): the accept
|
|
@@ -50,19 +44,19 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
50
44
|
at the top of `routeFrame` **before `touch()`**), and `FRAME_LIMITS` in `decode` everything a
|
|
51
45
|
client sizes (narrowable, never widenable). `list()` takes a required `max`. `input` is walked
|
|
52
46
|
ITERATIVELY.
|
|
53
|
-
- **Every `SyncSocket` ceiling is reachable from `createSyncNode`**, forwarded as
|
|
54
|
-
`...(x === undefined ? {} : { x })`.
|
|
55
47
|
- **A `SubscriptionLimitError` names the knob**: every throw site passes `knob` explicitly;
|
|
56
48
|
`channel.test.ts` asserts the hub's two against the option names.
|
|
57
|
-
- **Retained memory is bounded by BYTES
|
|
58
|
-
bound
|
|
59
|
-
|
|
60
|
-
|
|
49
|
+
- **Retained memory is bounded by BYTES** (`RingChangeBuffer`: patch count bounds replay, bytes
|
|
50
|
+
bound memory). `forget(qid)` runs when the last subscriber goes and from `#dropIfUnheld`.
|
|
51
|
+
- **The ring floor is one rule (`change-buffer.ts`, `Ring`)**: only a read that LANDED floors it
|
|
52
|
+
(`fillWindow().landed`), a forced one exclusively; a position-less read is marked with the
|
|
53
|
+
node's origin and answers that cursor alone below its first patch; a stale entry serves cold
|
|
54
|
+
starts only.
|
|
61
55
|
- **Every question a hot path asks is indexed**: `SubscriptionBook` keeps `#bySocket` and a
|
|
62
56
|
per-tenant count; `SocketRegistry` keeps `#byTopic`, changed only by `joinTopic`/`leaveTopic`.
|
|
63
57
|
`reauthorize` calls `book.retenant(socket)`.
|
|
64
58
|
- **A cap is a RESERVATION taken before the first await**: `SubscriptionBook.reserve(socket, sid)`
|
|
65
|
-
decides the sid claim, `maxPerSocket` and `
|
|
59
|
+
decides the sid claim, `maxPerSocket`, `maxPerTenant` and `maxPerActor` in one synchronous step;
|
|
66
60
|
`ChannelHub.subscribe` does the same for its caps. Released in a `finally`, idempotently.
|
|
67
61
|
- **Readiness AND the connection cap are functions on `UpgradeDeps`**, re-asked after
|
|
68
62
|
`authenticate` (app code with an await) and right before `server.upgrade`. The recheck sheds with
|
|
@@ -85,8 +79,8 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
85
79
|
- **A denial is a decision; everything else is a failure.** `visibleWithPolicy` matches
|
|
86
80
|
`QueryDeniedError` and rethrows the rest; `subscriber-gate.ts` and `reauthorize` ask
|
|
87
81
|
`isPolicyDenial(error)`. A failed snapshot raises out of `subscribe`; a failed delivery desyncs that
|
|
88
|
-
one subscriber; a failed `reauthorize` keeps the subscription,
|
|
89
|
-
|
|
82
|
+
one subscriber; a failed `reauthorize` keeps the subscription, a DENIED one is refused under its sid
|
|
83
|
+
(`refuseSubscription`) — as is one whose org moved, and a failed re-seat. Failures count as
|
|
90
84
|
`gateFailures` via `onGateFailed`, never `onRowDenied`.
|
|
91
85
|
- **One serial lane per query id (`WindowLock`) is the only thing that orders a fanout.** `deliver`
|
|
92
86
|
enters every lane before awaiting any; no fanout takes a second lane; a lane that fails desyncs its
|
|
@@ -107,10 +101,10 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
107
101
|
never clears a mark (`live-fanout.test.ts`).
|
|
108
102
|
- **A change the window already holds is refused** (`change.lsn <= entry.lsn`, counted as
|
|
109
103
|
`staleChanges`). **A gap is detected**: the replicator stamps `producer` + `seq`; a skipped
|
|
110
|
-
sequence
|
|
111
|
-
- **A bulk write stales only
|
|
112
|
-
`onBulk
|
|
113
|
-
on a non-
|
|
104
|
+
sequence — or a NEW producer after a known one — stales every window; `refillWindowInLane` replaces it.
|
|
105
|
+
- **A bulk write stales only its entity's readers**: windows + channel topics, `invalidate(entity)`
|
|
106
|
+
on both from `onBulk`. **A stale window's re-read re-snapshots EVERY subscriber**, before the lsn
|
|
107
|
+
guard, even on a non-match (`live-replicator-bulk`, `live-fanout` tests).
|
|
114
108
|
- **A WINDOW is per TENANT**: its id is `windowId(queryHash(name, input), tenant)` (`live-tenant.ts`;
|
|
115
109
|
tenant = the subscriber's `actor.orgId`). This package owns no hash. The read runs `readFor`
|
|
116
110
|
(`live-definition.ts`): the node's ctx, a service actor carrying ONLY that org — never the
|
|
@@ -119,47 +113,47 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
119
113
|
- **A `sid` is CLIENT data: a subscription is keyed by `(socket, sid)`**
|
|
120
114
|
(`subscription-book.ts` is the only spelling). Reusing a sid the same socket holds is
|
|
121
115
|
`X_SUBSCRIPTION_ID_TAKEN`.
|
|
116
|
+
- **`rateLimit:` is spent once per NEW subscription** (`live-spend.ts`).
|
|
122
117
|
- Truth is the server. A client is never the merge authority.
|
|
123
118
|
|
|
124
119
|
## The node, the bus and presence
|
|
125
120
|
|
|
126
121
|
- `sync` is stateless: no sticky sessions, nothing on a socket survives a restart.
|
|
127
|
-
- **`selectTransport(env, realtime)` is the one place `realtime.transport` + env become a bus
|
|
128
|
-
|
|
129
|
-
`
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
`publish`/`subscribe` refusals as `X_TRANSPORT_UNAVAILABLE`; an `UltimateError` passes through.
|
|
137
|
-
`transport-parity.test.ts` asserts both transports, and that the wrap still delivers. `#ensure`
|
|
138
|
-
reuses a mid-reconnect client on purpose.
|
|
122
|
+
- **`selectTransport(env, realtime)` is the one place `realtime.transport` + env become a bus** (KV
|
|
123
|
+
bucket and presence TTL come back with it). The config decides, the env supplies: `'nats'` refuses
|
|
124
|
+
(`X_CONFIG_INVALID`) when the `urlEnv` variable is unset; `'memory'` refuses a set one.
|
|
125
|
+
- **`nats` is imported only by `nats-lib-client.ts`**; its reconnect re-subscribes. Only when it
|
|
126
|
+
GIVES UP (`onClosed`) does `NatsTransport` re-dial on backoff and re-bind kept subscriptions
|
|
127
|
+
(`nats-subscriptions.ts`); callers meanwhile are refused. KV semantics: `nats-jetstream`/`nats-kv`.
|
|
128
|
+
- **Nothing leaves `NatsTransport` uncoded** — `#translating` wraps `publish`/`subscribe` refusals
|
|
129
|
+
as `X_TRANSPORT_UNAVAILABLE`. `#ensure` reuses a mid-reconnect client on purpose;
|
|
130
|
+
`Transport.onReconnect(listener)` (required) announces each recovery.
|
|
139
131
|
- **A bucket name is checked before interpolation** (`assertBucket`, `X_TRANSPORT_PROTOCOL`); a
|
|
140
132
|
presence key or member id is base64url-encoded (`encodeToken`), never validated.
|
|
141
133
|
- Presence lives in `transport.shared`. The sync node is `PresenceRegistry`'s only caller:
|
|
142
134
|
subscribing to a topic IS joining, repeating the frame is the heartbeat, dropping or closing is the
|
|
143
135
|
leave. Expiry is silent, so the node sweeps on an interval, and **one node per topic sweeps** (lowest
|
|
144
|
-
id in the lease
|
|
136
|
+
id in the lease's keyed set; the lease carries the roster followers record). `roster()` caps a frame at `maxMembers` (256) with `total`;
|
|
145
137
|
`list()` is never capped. `total` is set on `sync` frames only.
|
|
146
138
|
- **A socket the node evicts ITSELF goes through `evict(socket, code, reason)`** (the full
|
|
147
139
|
`teardown`), never `sockets.remove`. **`drain()` waits for the presence leaves it started** —
|
|
148
140
|
`evictInChunks` (`drain-evictions.ts`, `DRAIN_EVICT_CHUNK`, `allSettled`).
|
|
149
|
-
- **The idle sweep is armed by `start()`** (
|
|
150
|
-
|
|
151
|
-
the node evicts.
|
|
141
|
+
- **The idle sweep is armed by `start()`** (`idleTimeoutMs / 4`, floored at 1 s, `.unref()`), on
|
|
142
|
+
`Clock.monotonic()`. `SocketRegistry.idle()` is a query; the node evicts.
|
|
152
143
|
- **`drain()` and `stop()` both call one idempotent `release()`**; `drain()` releases after the
|
|
153
144
|
sockets are gone and before `hub.close()`.
|
|
154
|
-
- **Refusing
|
|
155
|
-
`
|
|
145
|
+
- **Refusing and draining are two phases**: `stopAccepting()` is `accept`, `drain()` + `stop()`
|
|
146
|
+
`close`; `listenSyncNode` unregisters both on `stop()`.
|
|
156
147
|
- **A hub that closed opens nothing**: `close()` sets `#closed` before the walk; `#open` closes a
|
|
157
148
|
late subscription and RAISES `X_TRANSPORT_UNAVAILABLE`. `#release` takes the reserved bridge, never
|
|
158
149
|
a name.
|
|
159
|
-
- **An upgrade from
|
|
160
|
-
(`sync-origin.ts
|
|
161
|
-
`
|
|
162
|
-
|
|
150
|
+
- **An upgrade from another origin is `403 X_SOCKET_ORIGIN_REFUSED`** before `authenticate`
|
|
151
|
+
(`sync-origin.ts`): `allowedOrigins`, when declared, is the WHOLE list; undeclared, the
|
|
152
|
+
`Host`-derived origin (and its `https` spelling when reached over plain http). The refusal names
|
|
153
|
+
the asker and the list. `x dev` keeps the reached-on origin (`admitReachedOrigin`) and adds
|
|
154
|
+
its web role's beside a declared list; a container adds nothing. `AcceptBudget` is reserved before
|
|
155
|
+
`authenticate`, `refund()`ed on every exit that takes no socket.
|
|
156
|
+
- **A socket's actor comes from `syncNode({ authenticate })` only**, run before
|
|
163
157
|
`server.upgrade`. `null` = 401 `X_SOCKET_UNAUTHENTICATED`; a throw = 503
|
|
164
158
|
`X_SOCKET_AUTH_UNAVAILABLE`. Absent = anonymous, and `start()` warns. The actor lives only in the
|
|
165
159
|
`GrantBook`.
|
|
@@ -167,57 +161,57 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
167
161
|
released on every path that never opens — `onUngranted` is REQUIRED on `UpgradeDeps`, and a
|
|
168
162
|
`server.upgrade` that THROWS releases too. `sync-node-auth.test.ts`'s harness opens inside
|
|
169
163
|
`upgrade()`, as Bun does.
|
|
170
|
-
- **A grant expires; a socket does not**: expired grants are re-decided on an interval
|
|
171
|
-
`hub.onActorChange
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
a dropped presence roster is logged (`sync.presence_roster_dropped`); `drain()` returns
|
|
175
|
-
`DrainedSocket[]` with `notified` and logs `sync.drain_frames_dropped`.
|
|
164
|
+
- **A grant expires; a socket does not**: expired grants are re-decided on an interval
|
|
165
|
+
(`hub.onActorChange`, `registry.reauthorize`). No `refresh` = close `1008`; one that raises retries.
|
|
166
|
+
- **Every `socket.send` reads its answer**: a dropped subscribe reply marks it desynced; a dropped
|
|
167
|
+
roster logs `sync.presence_roster_dropped`; `drain()` returns `DrainedSocket[]` (`notified`).
|
|
176
168
|
- **Inbound frames run in a lane, never the socket**: `subscribe` is `sub:<sid>` or `topic:<name>`,
|
|
177
169
|
everything else unlaned (`frame-lanes.ts`); a lane exists only while work is queued.
|
|
178
170
|
- **A dropped `records` frame is counted AND repaired**: per-node `seq`/`epoch` per channel; a refused
|
|
179
|
-
frame marks the socket gapped, and the
|
|
180
|
-
(`channel_replay_gaps_total`
|
|
181
|
-
channel's `catchUp`, holding frames meanwhile.
|
|
171
|
+
frame marks the socket gapped, and the `drain` handler sends `replay-gap`
|
|
172
|
+
(`channel_replay_gaps_total`). The client re-runs `catchUp`, holding frames meanwhile.
|
|
182
173
|
- **A channel is a DECLARATION**: `channel(name, { params, policy, catchUp, records?, events? })`.
|
|
183
174
|
Deny by default, and `policy` is REQUIRED (`X_CHANNEL_DECLARATION_INVALID`; a public channel says
|
|
184
175
|
`policy: allow('public')`). Presence rides `events: true` channels.
|
|
185
|
-
- **A guard that FAILS
|
|
186
|
-
|
|
187
|
-
- **A
|
|
188
|
-
-
|
|
176
|
+
- **A guard that FAILS on re-auth SUSPENDS the seat** (`guardFailures`): kept, silent until a pass.
|
|
177
|
+
A `null` actor on a `row` channel and a tenancy refusal are DENIALS.
|
|
178
|
+
- **A seat is decided for the actor ON the socket when the guard resolves** (`#settle`); a denied
|
|
179
|
+
re-ask on a suspended seat is refused and latched. The latch is capped at `maxTopicsPerSocket`.
|
|
180
|
+
- **A missed change reopens every records topic** (`ChannelHub.invalidate()`, beside both
|
|
181
|
+
`registry.invalidate()` sites; `sync-bus-handlers.test.ts`).
|
|
182
|
+
- **A `recordPublisher` change (`source: 'publisher'`) feeds channels ONLY** — no commit position:
|
|
183
|
+
never `registry.deliver`, its gap stales no window; a new publisher is no gap. One producer KIND
|
|
184
|
+
per table per node (`ProducerKinds`, `X_REALTIME_PRODUCER_CONFLICT`); the x dev bridge skips a
|
|
185
|
+
claimed table (`isPublishedTable`). `ChangeEvent.table` is the relation on every producer.
|
|
186
|
+
- **An error never renders a credential**: `parsePgUrl`/`parseNatsUrl` name the variable.
|
|
189
187
|
|
|
190
188
|
## Replication
|
|
191
189
|
|
|
192
190
|
- Exactly one `replicator` per DB, by a session-level advisory lock. **`start()` and
|
|
193
|
-
`
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
- **
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
- **
|
|
207
|
-
|
|
208
|
-
- **
|
|
209
|
-
|
|
191
|
+
`PostgresAdvisoryLock.tryAcquire` are MEMOISED**; `stop()`/`release()` await the in-flight one. A start
|
|
192
|
+
that FAILS hands the lock back; `running` is set once the feed pumps.
|
|
193
|
+
- **The pump has one way out, `#die`**: failure recorded, timer stopped, connection closed, then
|
|
194
|
+
`onEnd(reason)`. `start()` awaits the previous pump. **`stop()` releases everything first.**
|
|
195
|
+
- **A stream that ENDS is restarted**: feed then lock ABANDONED (synchronous, never asked);
|
|
196
|
+
`stop()` bounds both goodbyes by `STOP_DEADLINE_MS`; redial on `retryDelayMs` (injected
|
|
197
|
+
`schedule`) from `lastLsn` until `stop()`. The delay resets only when a change is PUBLISHED. A
|
|
198
|
+
rejected `publish` is one such end, and so is a LOST lock (`AdvisoryLock.onLost`), mid-dial
|
|
199
|
+
included. `running` = pumping AND no refused change owed; `onChange` is fenced per run and
|
|
200
|
+
REFUSES, never drops.
|
|
201
|
+
- **A row crosses the bus through its entity's columns** (`replicator-row.ts`): `parseEnvelope`
|
|
202
|
+
revives by `entityForTable` + `$parse`, bytes as base64 — by declared kind, never by shape.
|
|
203
|
+
- **Cleartext/md5 auth is answered only under `sslmode` `require`, `verify-*` or `disable`**.
|
|
204
|
+
- **The replicator ensures its publication** (`pg-publication.ts`): `CREATE … FOR TABLE` every
|
|
205
|
+
entity table, or `ALTER … ADD TABLE` the missing; never a DROP, never `FOR ALL TABLES`.
|
|
206
|
+
- **TLS is libpq's `sslmode`** (`pg-tls.ts`): only `verify-*` verify; the runtime never rejects,
|
|
207
|
+
`judgeHandshake` decides; the raw socket is DEAF after `upgradeTLS`.
|
|
208
|
+
- **Every session pins `datestyle=ISO`, `intervalstyle=postgres`, `extra_float_digits=3`, `TimeZone=UTC`**.
|
|
210
209
|
- A change lsn is `<16 hex commit position><8 hex row position>`; never order by one half, a wall
|
|
211
210
|
time or a counter.
|
|
212
211
|
- Slot, publication and entity names match `[a-z_][a-z0-9_]*` before interpolation — a security
|
|
213
|
-
boundary (`pg-identifier.ts
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
own `decodeRow` / `entityForTable`, so money (three physical columns), scale and every other kind
|
|
217
|
-
fold exactly as `postgresRepo` folds them.
|
|
218
|
-
- A write's name rides the WAL: `pg-replication.ts` reads it off the transaction's opening
|
|
219
|
-
`pg_logical_emit_message` (prefix `WRITE_ORIGIN_WAL_PREFIX`; `START_REPLICATION` asks
|
|
220
|
-
`messages 'true'`).
|
|
212
|
+
boundary (`pg-identifier.ts`). A SQLSTATE is data: `FIXES` is read with `Object.hasOwn`.
|
|
213
|
+
- **A live row equals a repository row**: `pg-entity-row.ts` decodes by `@ultimat3/entity`'s
|
|
214
|
+
`decodeRow` / `entityForTable`.
|
|
221
215
|
|
|
222
216
|
## The page
|
|
223
217
|
|
|
@@ -226,14 +220,14 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
226
220
|
optimistic writes, REPLAYED over synced truth whenever it moves. Installed as core's `RecordSink` on
|
|
227
221
|
`pageClient().store`.
|
|
228
222
|
- **Page state lives on `globalThis[Symbol.for('ultimate.realtime')]` (`page-store.ts`)**, never module
|
|
229
|
-
scope — every island is its own bundle (`page-client.test.ts` builds
|
|
223
|
+
scope — every island is its own bundle (`page-client.test.ts` builds two islands). Never
|
|
230
224
|
`instanceof` across copies.
|
|
231
225
|
- **The signal factory is per BUNDLE**: `installRealtime({ signal, sync })` (`reactivity.ts`). No
|
|
232
226
|
install and a DOM is `X_REALTIME_UNINSTALLED`; no DOM is a server render — hooks answer
|
|
233
227
|
`pending`/online and create no page state. `useRecord` and `useQuery` return `AsyncState` accessors.
|
|
234
|
-
- **
|
|
235
|
-
|
|
236
|
-
|
|
228
|
+
- **The page runtime (`page-runtime.ts`) is installed once per page, never in an island**;
|
|
229
|
+
hooks call `installedPage(hook).services`. One socket per page (`page-socket.ts`);
|
|
230
|
+
`browser-socket.ts` holds the only `new WebSocket`. `rescope` clears the store, redials.
|
|
237
231
|
- **One socket per ORIGIN and principal, in a SharedWorker**: `socket-engine.ts` holds it and talks
|
|
238
232
|
to tabs over `MessagePort`s (`socket-host.ts`'s virtual socket); wants are ref-counted and frames
|
|
239
233
|
routed; a port silent for `REAP_AFTER_BEATS` is reaped; each tab resubscribes from its OWN cursors.
|
|
@@ -244,23 +238,27 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
244
238
|
- **`LiveClient` holds no signal** — plain reads plus `onStatus` / `onChange`; hooks wrap them.
|
|
245
239
|
- **An answer that did not carry a touched row keeps that row's overlay** until the server's next row
|
|
246
240
|
for it, bounded by `DEFAULT_AWAIT_SERVER_MS` (10 s).
|
|
247
|
-
- **A
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
241
|
+
- **A frame names the write that produced it**: `records.write`, `patch.writes` (a live patch
|
|
242
|
+
one, a resume each it replays, off `RowPatch.write` in the ring) — `writeDigest(key)`, from a
|
|
243
|
+
REQUIRED `ChangeEvent.write: string | null`. `RecordStore.push` digests each overlay key first
|
|
244
|
+
(`record-names.ts`); `client-channels.ts` / `live-rows.ts` settle in the frame's own batch. A
|
|
245
|
+
non-digest is a protocol error, dropped on the bus. Partly confirmed rows go to
|
|
246
|
+
`OverlayEntry.confirmed`. `write-echo.test.ts`, `live-write-echo.test.ts`.
|
|
252
247
|
- `local(tx, input)` is pure and CONVERGENT: no I/O, no `Date.now()`, no `Math.random()`.
|
|
253
248
|
- **One conflict vocabulary**: `ConflictPolicy` from `@ultimat3/core`, resolved by core's
|
|
254
249
|
`resolveConflict(overlayRow, serverRow)` in `RecordStore.settle`. A server delete is not a conflict.
|
|
255
250
|
- Anything a component reads is a getter or accessor. `MutatorLike.local` uses method syntax.
|
|
256
251
|
`useQuery`'s input is read once, at call time.
|
|
257
|
-
-
|
|
258
|
-
|
|
259
|
-
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
252
|
+
- **`useRecords(type, selection)` holds the TYPE** (`retainType`): `ready` at once, recomputed per
|
|
253
|
+
change to that type, no row of it evicted while held; the last type hold evicts what it spared.
|
|
254
|
+
- Every subscription handle is `Disposable`; `[Symbol.dispose]` IS the release. Type claims go in
|
|
255
|
+
`type-pins.ts`.
|
|
256
|
+
- **The outbox**: only `pending` is sendable; `drain()` is one chained pass at a time; `#epoch` is
|
|
257
|
+
bumped before the requeue scan, read every pass. `#persist` hands SNAPSHOTS by KEY; one tab drains
|
|
258
|
+
(`navigator.locks`), resetting a stored `inflight`. A `rescope` abandons the queue and bumps the
|
|
259
|
+
epoch synchronously; `enqueue` decides WHOSE queue before its first await
|
|
260
|
+
(`X_OFFLINE_QUEUE_ABANDONED`); `useMutation` fences its POST on its scope epoch. A refused disk is
|
|
261
|
+
warned `X_LOCAL_STORE_UNAVAILABLE`, never rejecting `ready`. An entry leaves disk at its ack.
|
|
264
262
|
|
|
265
263
|
## The client connection
|
|
266
264
|
|
|
@@ -273,28 +271,23 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
|
|
|
273
271
|
- **A refusal `ack` is revived as a branded `UltimateError`** (`refusalError`, `client-frames.ts`).
|
|
274
272
|
- **A reconnect replays registrations AND topics** — one `hello`, one `subscribe` per registration
|
|
275
273
|
(with its cursor) and per topic.
|
|
276
|
-
- **The client beats** (`
|
|
277
|
-
plus one subscribe per topic
|
|
278
|
-
|
|
279
|
-
|
|
274
|
+
- **The client beats** (`DEFAULT_HEARTBEAT_MS` 10 s = default ttl / 3, pinned by
|
|
275
|
+
`client-heartbeat.test.ts`; `0` disables): a `hello` plus one subscribe per topic. Two silent
|
|
276
|
+
windows close with `4000`. **The node names the beat: `hello.heartbeatMs`** — `Heartbeat.follow`
|
|
277
|
+
per socket; the engine reaps ports on the longer of its own and the node's.
|
|
280
278
|
- **There is ONE `backoffDelay`, `@ultimat3/core`'s, counted from 1** — `policyDelay()` maps a
|
|
281
|
-
`BackoffPolicy` onto it; `bun run flight-copies` refuses a second curve.
|
|
282
|
-
`docs/history/realtime.md`, "Moved 2026-10-01".
|
|
279
|
+
`BackoffPolicy` onto it; `bun run flight-copies` refuses a second curve.
|
|
283
280
|
|
|
284
281
|
## Rules for code here
|
|
285
282
|
|
|
286
283
|
- Never a bare `Error`. Never `any`. Never `Date.now()` — take a `Clock` (`monotonic()` for durations).
|
|
287
|
-
- **A test fixture standing in for a FOREIGN error extends `Error` on purpose** (`PoolTimeout`,
|
|
288
|
-
`Denied`, `ThirdPartySdkError`): it simulates a value this package did not construct. The rule
|
|
289
|
-
governs what this package **throws**, never what a test hands it.
|
|
290
284
|
|
|
291
285
|
## The page boot
|
|
292
286
|
|
|
293
|
-
Browser bytes per hook, as last measured: `docs/history/realtime.md`. Re-measure before quoting.
|
|
294
|
-
|
|
295
287
|
- **The disk boot is ONE page script (`boot.ts`, `./boot`)**, served at `/_x/page-boot/<hash>.js`
|
|
296
|
-
on a document with the scope tag and a realtime island; it wipes every other
|
|
297
|
-
scope, then restores. `booted` reads its promise off
|
|
288
|
+
on a document with the scope tag and a realtime island; it installs the runtime, wipes every other
|
|
289
|
+
principal's stored scope (read at the wipe), then restores. `booted` reads its promise off
|
|
290
|
+
`globalThis[Symbol.for('ultimate.page-boot')]`. No boot: islands load its chunk.
|
|
298
291
|
- **An island never carries the outbox**: `boot.ts` builds it; `useMutation` and the page socket read
|
|
299
292
|
it through `outbox-slot.ts` after `page.booted`. No boot ⇒ a refused write is rejected, not queued.
|
|
300
293
|
|
|
@@ -303,27 +296,26 @@ Browser bytes per hook, as last measured: `docs/history/realtime.md`. Re-measure
|
|
|
303
296
|
`index.ts` / `server.ts` are the two barrels. Server side: `sync-node.ts` (+ `sync-auth`,
|
|
304
297
|
`sync-frames`, `sync-upgrade`, `sync-listen`, `drain-evictions`), `channel.ts`, `presence.ts`,
|
|
305
298
|
`socket.ts`, `live-query.ts` (+ `live-definition`, `live-refusal`, `live-tenant`, `live-resume`, `query-window`, `live-fanout`, `window-lock`,
|
|
306
|
-
`subscriber-gate`, `policy-gate` — the only authz seam — and `matcher-bridge`), `replicator.ts` /
|
|
299
|
+
`subscriber-gate`, `policy-gate` — the only authz seam — and `matcher-bridge`), `replicator.ts` (+ `replicator-envelope`) /
|
|
307
300
|
`live-replicator.ts`, the Postgres client (`pg-*.ts`, `pgoutput.ts`), the bus (`nats-*.ts`,
|
|
308
301
|
`transport-env.ts`, `fanout.ts`). Client side: `client*.ts`, `socket-engine.ts` / `socket-host.ts` /
|
|
309
302
|
`sync-worker.ts`, `record-*.ts`, `page-*.ts`, `boot.ts`, `offline-queue.ts`, `use-*.ts`,
|
|
310
303
|
`reactivity.ts`. Shared: `sync-protocol.ts` (the wire), `cursor.ts`, `change-buffer.ts`,
|
|
311
|
-
`thundering-herd.ts`, `live-contract.ts`, `json.ts` (no hash), `type-pins.ts`.
|
|
312
|
-
states its one job.
|
|
304
|
+
`thundering-herd.ts`, `live-contract.ts`, `json.ts` (no hash), `type-pins.ts`.
|
|
313
305
|
|
|
314
306
|
## Commands
|
|
315
307
|
|
|
316
308
|
```
|
|
317
|
-
bun test packages/realtime/src
|
|
309
|
+
bun test packages/realtime/src # from the REPO ROOT
|
|
318
310
|
bun run typecheck
|
|
319
311
|
```
|
|
320
312
|
|
|
321
|
-
`bunfig.toml`'s preload installs `@ultimat3/testing`'s matchers
|
|
313
|
+
`bunfig.toml`'s preload installs `@ultimat3/testing`'s matchers; Bun reads it from the cwd.
|
|
322
314
|
|
|
323
315
|
Changing a frame shape means a fixture in `sync-protocol.test.ts` (the round-trip test fails
|
|
324
316
|
without one) and bumping `PROTOCOL_VERSION` **when an old frame becomes unreadable in either
|
|
325
317
|
direction**. `decode` is a whitelist: an additive optional field, or removing a field read through
|
|
326
|
-
`list()`, is free; removing a field read through `str()`/`num()` (which throw) is a bump.
|
|
327
|
-
field's line in `decode` before claiming a removal is free.
|
|
318
|
+
`list()`, is free; removing a field read through `str()`/`num()` (which throw) is a bump.
|
|
328
319
|
|
|
329
|
-
Why each rule
|
|
320
|
+
Why each rule is shaped so, and browser bytes per hook (re-measure before quoting):
|
|
321
|
+
[`docs/history/realtime.md`](../../docs/history/realtime.md).
|