@ultimat3/realtime 27.14.0 → 27.15.0

package/CLAUDE.md

@@ -103,10 +103,10 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
   `staleChanges`). **A gap is detected**: the replicator stamps `producer` + `seq`; a skipped
   sequence — or a NEW producer after a known one — stales every window; `refillWindowInLane` replaces it.
 - **A bulk write stales only its entity's readers**: windows + channel topics, `invalidate(entity)`
-  on both from `onBulk`. **A stale window's re-read re-snapshots EVERY subscriber**, before the lsn
+  on both from `onBulk` (channels: not a `recordPublisher` table). **A stale window's re-read re-snapshots EVERY subscriber**, before the lsn
   guard, even on a non-match (`live-replicator-bulk`, `live-fanout` tests).
 - **A WINDOW is per TENANT**: its id is `windowId(queryHash(name, input), tenant)` (`live-tenant.ts`;
-  tenant = the subscriber's `actor.orgId`). This package owns no hash. The read runs `readFor`
+  tenant = the subscriber's `actor.orgId`). The read runs `readFor`
   (`live-definition.ts`): the node's ctx, a service actor carrying ONLY that org — never the
   subscriber. A cursor naming another window is a cold start; `reauthorize` re-seats a
   subscription whose actor changed org (`live-tenancy.test.ts`; `docs/history/realtime.md`).
@@ -278,7 +278,7 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
   (with its cursor) and per topic.
 - **The client beats** (`DEFAULT_HEARTBEAT_MS` 10 s = default ttl / 3, pinned by
   `client-heartbeat.test.ts`; `0` disables): a `hello` plus one subscribe per topic. Two silent
-  windows close with `4000`. **The node names the beat: `hello.heartbeatMs`** — `Heartbeat.follow`
+  windows after an UNANSWERED beat close with `4000`. **The node names the beat: `hello.heartbeatMs`** — `Heartbeat.follow`
   per socket; the engine reaps ports on the longer of its own and the node's.
 - **There is ONE `backoffDelay`, `@ultimat3/core`'s, counted from 1** — `policyDelay()` maps a
   `BackoffPolicy` onto it; `bun run flight-copies` refuses a second curve.
@@ -312,7 +312,7 @@ bun test packages/realtime/src   # from the REPO ROOT
 bun run typecheck
 ```
 
-`bunfig.toml`'s preload installs `@ultimat3/testing`'s matchers; Bun reads it from the cwd.
+`bunfig.toml`'s preload (matchers) is read from the cwd.
 
 Changing a frame shape means a fixture in `sync-protocol.test.ts` (the round-trip test fails
 without one) and bumping `PROTOCOL_VERSION` **when an old frame becomes unreadable in either

package/README.md

@@ -84,7 +84,7 @@ class — stay on `.`.
 | wire | `.` | `PROTOCOL_VERSION` (3), `encode`, `decode`, `Frame` |
 | the node | `./server` | `syncNode` / `listenSyncNode` (`sync` role) |
 | a socket's identity | `./server` | `SyncAuthenticator`, `SyncGrant`, `GrantBook`, `sweepGrants`, `DEFAULT_REAUTH_INTERVAL_MS` |
-| hooks | `.` | `useQuery`, `useRecord`, `useRecords` (by key, or a whole type: `RecordSelection`), `useMutation`, `useMutationQueue`, `useOutbox`, `useConnection`, `useChannel`, `usePresence`, `hasPageSocket`, `installRealtime` |
+| hooks | `.` | `useQuery`, `useRecord`, `useRecords` (by key, or a whole type: `RecordSelection`), `useRecordChanges`, `useMutation`, `useMutationQueue`, `useOutbox`, `useConnection`, `useChannel`, `usePresence`, `hasPageSocket`, `installRealtime` |
 | channel records, no replicator | `./server` | `recordPublisher` — an app's committed rows as channel `records`, sequenced per producer ([below](#channel-records-with-no-replicator)) |
 | channels | `.` | `channel`, `channelRef`, `ChannelHandle`, `topic`, `readPresence`, the channel frame types |
 | first paint | `.` | `holdFirstPaint`, `FIRST_PAINT_HOLD_MS` (1 s) — what the island bootstrap awaits before `mount`, for an island that reads the store or the outbox: the boot's restore and the open outbox, capped (#506). An island taking only `useChannel` / `usePresence` / `useConnection` never awaits it and is never hidden |
@@ -279,6 +279,7 @@ same type `@ultimat3/ui` renders.
 | A query ref is `{ name, live?, entity? }`, never the query VALUE | importing a `query()` drags its read path into the island (698,801 B measured) |
 | Lists hold keys; rows are the store's | a record updated by any answer or frame re-renders every list showing it, with no refetch |
 | A list that must GROW with the store is `useRecords(type, { where?, order? })` | a `useQuery` list holds its own answer's keys and `useRecords(type, keys)` the keys it was given, so a record a channel's `records` frame adopts reaches neither. The whole-type form is the channel-records counterpart of a live window: `ready` from its first read, every record of the type the store holds (optimistic ones included), default order by key, and none evicted while it is held — a row it alone kept goes when the last reader of the type lets go (#680) |
+| Knowing THAT records moved is `useRecordChanges(types, listener)` | one call per store update, the moved keys per type, and no hold on any row or type, so a page that re-renders on a channel's `records` pins nothing. `useChannel(...).error()` is the node's refusal of a `failed` membership |
 | Every write is HTTP through core's `clientTransport` | one write path: the action's authz, idempotency and contract; the socket carries none |
 | The answer's records are adopted before the overlay goes | a convergent twin never flickers back to the pre-write value |
 | **No `solid-js` import.** Each bundle installs its own `SignalFactory` | every island carries its own solid-js; a signal from another bundle is invisible to its effects |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ultimat3/realtime",
-  "version": "27.14.0",
+  "version": "27.15.0",
   "description": "Three-tier realtime: channels, live queries, local-first sync — one protocol, one mutator shape",
   "license": "MIT",
   "type": "module",
@@ -40,10 +40,10 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@ultimat3/core": "27.14.0",
-    "@ultimat3/entity": "27.14.0",
-    "@ultimat3/policy": "27.14.0",
-    "@ultimat3/query": "27.14.0",
+    "@ultimat3/core": "27.15.0",
+    "@ultimat3/entity": "27.15.0",
+    "@ultimat3/policy": "27.15.0",
+    "@ultimat3/query": "27.15.0",
     "nats": "2.29.3"
   }
 }

package/src/client-heartbeat.ts

@@ -39,6 +39,8 @@ export class Heartbeat {
   readonly #options: HeartbeatOptions;
   #cancel: (() => void) | null = null;
   #lastSeen = 0;
+  /** Whether a frame arrived since this socket last asked (the opening `hello`, then each beat). */
+  #answered = false;
   /** The beat in force on THIS socket: the option's, until the node's `hello` names another. */
   #intervalMs: number;
 
@@ -58,6 +60,7 @@ export class Heartbeat {
     this.#intervalMs = this.#options.intervalMs;
     if (this.#intervalMs <= 0) return;
     this.#lastSeen = now;
+    this.#answered = false;
     this.#arm();
   }
 
@@ -78,6 +81,7 @@ export class Heartbeat {
   /** A frame arrived. Anything counts: the point is that bytes still cross in this direction. */
   saw(now: number): void {
     this.#lastSeen = now;
+    this.#answered = true;
   }
 
   stop(): void {
@@ -98,10 +102,14 @@ export class Heartbeat {
     // Two windows, not one: a beat and the answer to it share the window they were sent in, so a
     // single quiet interval is a slow round trip and not a dead socket. Nothing is re-armed after
     // a silence — `onSilence` drops the socket, and the next `start()` is the next connection's.
-    if (now - this.#lastSeen > this.#intervalMs * 2) {
+    // And only when the last beat went UNANSWERED: a hidden tab's timers are throttled (Chrome runs
+    // a chained timer about once a minute), so a late tick on a socket that answered every beat it
+    // was sent is the page's clock, not the socket — dropping it churned every background tab.
+    if (!this.#answered && now - this.#lastSeen > this.#intervalMs * 2) {
       this.#options.onSilence();
       return;
     }
+    this.#answered = false;
     this.#options.beat();
     this.#arm();
   }

package/src/client.ts

@@ -62,6 +62,8 @@ export class LiveClient {
   #attempt = 0;
   /** The armed reconnect's canceller, and the flag for "one timer in flight, the first wins". */
   #reconnectTimer: (() => void) | null = null;
+  /** The armed wait is a node's drain slot: `wake()` leaves it alone, or the herd re-clusters. */
+  #armedByNode = false;
   /** Set by `close()`: an explicit teardown must not be undone by the close it just triggered. */
   #closed = false;
   #connected = false;
@@ -157,7 +159,6 @@ export class LiveClient {
       // and the one handler that had none. A replaced socket opening late would otherwise mark the
       // live connection up and replay every subscription onto whatever socket is current.
       if (this.#socket !== socket) return;
-      this.#attempt = 0;
       this.#setStatus({ connected: true, reconnectAt: null });
       // `hello` announces the connection and nothing else. Each cursor rides its own `subscribe`
       // frame below, which is the only place resume is decided — sending it here too shipped every
@@ -174,6 +175,10 @@ export class LiveClient {
       // that is still draining bytes would otherwise fold its patches into the live registrations
       // a second time, over newer state.
       if (this.#socket !== socket) return;
+      // The curve resets on the node's first word, not on the open: a proxy or an overloaded node
+      // that accepts the upgrade and drops it at once would otherwise reset it every time, and the
+      // tab redialled every half second for as long as that lasted.
+      this.#attempt = 0;
       this.#heartbeat.saw(this.#clock.now().getTime());
       applyFrame(decode(data), this.#frameTarget);
     });
@@ -227,6 +232,24 @@ export class LiveClient {
     this.#offline();
   }
 
+  /**
+   * The network or the tab is back (`online`, a tab made visible): dial now instead of sitting out
+   * the armed backoff. Only a local backoff is cut short — a node's drain slot is how a deploy's
+   * herd is spread, so it is kept — and only while a reconnect is actually armed.
+   */
+  wake(): void {
+    if (this.#closed || this.#connected || this.#reconnectTimer === null || this.#armedByNode)
+      return;
+    this.#cancelReconnect();
+    this.#attempt = 0;
+    try {
+      this.connect();
+    } catch (error) {
+      this.#scheduleReconnect(null);
+      this.#onError(error);
+    }
+  }
+
   /**
    * Subscribe to a live query. The window holds ids; the rows are the page store's, so a record
    * updated by an HTTP response or another window re-renders here too, with no second copy.
@@ -411,6 +434,7 @@ export class LiveClient {
       // `#attempt` counts reconnects already scheduled, from 0; the wait being armed is the next one.
       serverDelayMs ?? policyDelay(this.#options.backoff ?? browserBackoff, this.#attempt + 1, rng);
     this.#attempt += 1;
+    this.#armedByNode = serverDelayMs !== null;
     this.#setStatus({ reconnectAt: this.#clock.now().getTime() + delay });
     const schedule = this.#options.scheduler ?? timeoutScheduler;
     this.#reconnectTimer = schedule(() => {

package/src/index.ts

@@ -202,3 +202,8 @@ export {
   useRecord,
   useRecords,
 } from './use-record';
+export {
+  type RecordChanges,
+  type RecordChangesHandle,
+  useRecordChanges,
+} from './use-record-changes';

package/src/json.ts

@@ -74,9 +74,24 @@ export function changedColumns(before: JsonObject | null, after: JsonObject): Js
   return out;
 }
 
-function sameJson(a: JsonValue | undefined, b: JsonValue | undefined): boolean {
-  if (a === b) return true;
-  if (a === undefined || b === undefined || a === null || b === null) return false;
-  if (typeof a !== 'object' || typeof b !== 'object') return false;
-  return JSON.stringify(a) === JSON.stringify(b);
+/**
+ * Structural equality over the value domain: arrays by position, plain objects by key whatever
+ * their order (Postgres `jsonb` reorders keys), a `Date` by its time. A JSON column is a fresh
+ * object on every read, so identity said "changed" for every re-read of an unchanged row.
+ */
+export function sameJson(a: unknown, b: unknown): boolean {
+  if (Object.is(a, b)) return true;
+  if (typeof a !== 'object' || typeof b !== 'object' || a === null || b === null) return false;
+  if (a instanceof Date || b instanceof Date) {
+    return a instanceof Date && b instanceof Date && a.getTime() === b.getTime();
+  }
+  if (Array.isArray(a) || Array.isArray(b)) {
+    if (!Array.isArray(a) || !Array.isArray(b) || a.length !== b.length) return false;
+    return a.every((item, index) => sameJson(item, b[index]));
+  }
+  const left = a as Record<string, unknown>;
+  const right = b as Record<string, unknown>;
+  const keys = Object.keys(left);
+  if (keys.length !== Object.keys(right).length) return false;
+  return keys.every((key) => Object.hasOwn(right, key) && sameJson(left[key], right[key]));
 }

package/src/live-replicator.ts

@@ -151,8 +151,11 @@ export async function startLiveReplicator(options: LiveReplicatorOptions): Promi
       if (stopped) return;
       const table = tableOf(change.entity);
       registry.invalidate(table);
-      // Channels too, or a tab seated on one held the pre-update rows with nothing to re-read.
-      options.channels?.invalidate(table);
+      // Channels too, or a tab seated on one held the pre-update rows with nothing to re-read —
+      // unless a `recordPublisher` claims the table, as `onChange` asks: the app's own publish is
+      // its channel delivery, bulk writes included, and a replay-gap on every `updateWhere` sent
+      // every seated tab back to a full catch-up read for rows it was already being sent.
+      if (!isPublishedTable(table)) options.channels?.invalidate(table);
     },
   };
 

package/src/page-socket.ts

@@ -66,16 +66,24 @@ export function pageSocket(hook: string): LiveClient {
     host.rehost();
     client.connect();
   };
+  // The network or the tab came back: dial now, not at the end of a backoff armed while it was gone.
+  const wake = (): void => {
+    if (typeof document === 'undefined' || document.visibilityState !== 'hidden') client.wake();
+  };
   const listens = typeof addEventListener === 'function';
   if (listens) {
     addEventListener('pagehide', bye);
     addEventListener('pageshow', restored);
+    addEventListener('online', wake);
+    if (typeof document !== 'undefined') document.addEventListener('visibilitychange', wake);
   }
   teardowns.add(() => {
     offRescope();
     if (listens) {
       removeEventListener('pagehide', bye);
       removeEventListener('pageshow', restored);
+      removeEventListener('online', wake);
+      if (typeof document !== 'undefined') document.removeEventListener('visibilitychange', wake);
     }
     client.close();
     host.bye();

package/src/record-synced.ts

@@ -3,6 +3,7 @@
 // state — it answers what moved, and the store decides who hears about it.
 
 import type { Row } from '@ultimat3/core/page';
+import { sameJson } from './json';
 import type { RecordKey } from './record-key';
 
 export class SyncedLayer {
@@ -44,7 +45,9 @@ export class SyncedLayer {
   /**
    * One server write. Merged, never replaced: a projection that selected fewer columns must not
    * blank the columns another is rendering, so an omitted (or `undefined`) field is left alone.
-   * Answers the row as it now stands, and whether anything in it changed.
+   * Answers the row as it now stands, and whether anything in it changed. Compared STRUCTURALLY:
+   * a JSON column is a fresh array or object on every read, and by identity every catch-up, poll
+   * and reconnect moved every row. An unmoved column keeps its object, so its readers stay put.
    */
   merge(rk: RecordKey, columns: Row): { readonly row: Row; readonly changed: boolean } {
     // A restored row is a guess from disk: the server's first answer is the whole truth.
@@ -53,7 +56,8 @@ export class SyncedLayer {
     let changed = current === undefined;
     for (const [column, value] of Object.entries(columns)) {
       if (value === undefined) continue;
-      if (current === undefined || current[column] !== value) changed = true;
+      if (current !== undefined && sameJson(current[column], value)) continue;
+      changed = true;
       next[column] = value;
     }
     if (!changed && current !== undefined) return { row: current, changed: false };

package/src/record-tx.ts

@@ -3,6 +3,7 @@
 // over the OLD row must become `+ 1` over the row the server just sent, never the stale sum.
 
 import type { ConflictPolicy, Row } from '@ultimat3/core/page';
+import { sameJson } from './json';
 
 /** One table as a mutator's `local` half sees it, keyed by record type (the entity's name). */
 export interface LocalTable<R extends Row = Row> {
@@ -45,13 +46,16 @@ export interface ReplayResult {
   readonly failed: readonly string[];
 }
 
-/** Shallow content equality — a replay producing the same columns must not re-render anything. */
+/**
+ * Content equality, column by column and STRUCTURAL within one (`sameJson`) — a replay producing
+ * the same columns must not re-render anything, and a twin's array is a fresh one every replay.
+ */
 export function sameRow(a: Row | null, b: Row | null): boolean {
   if (a === b) return true;
   if (a === null || b === null) return false;
   const keys = Object.keys(a);
   if (keys.length !== Object.keys(b).length) return false;
-  return keys.every((key) => Object.hasOwn(b, key) && a[key] === b[key]);
+  return keys.every((key) => Object.hasOwn(b, key) && sameJson(a[key], b[key]));
 }
 
 export function replayOverlays(

package/src/use-channel.ts

@@ -8,8 +8,10 @@ import { installedPage } from './page-store';
 import { isServerRender, signalFor } from './reactivity';
 import type { PresenceMember } from './sync-protocol';
 
-/** The membership's state, callable, plus its release (Solid: `onCleanup`). */
+/** The membership's state, callable, plus its refusal and its release (Solid: `onCleanup`). */
 export type ChannelAccessor = (() => ChannelState) & {
+  /** What the node answered when it refused the membership (`failed`); `undefined` otherwise. */
+  error(): unknown;
   release(): void;
   [Symbol.dispose](): void;
 };
@@ -29,6 +31,7 @@ export function useChannel<K extends string>(
   const signal = signalFor('useChannel');
   if (isServerRender()) {
     return Object.assign((): ChannelState => 'joining', {
+      error: (): unknown => undefined,
       release: nothing,
       [Symbol.dispose]: nothing,
     });
@@ -42,11 +45,16 @@ export function useChannel<K extends string>(
     version();
     return membership.state();
   };
+  // Read through the same version, so a component rendering the refusal re-renders with it.
+  const error = (): unknown => {
+    version();
+    return membership.error();
+  };
   const release = (): void => {
     off();
     membership.release();
   };
-  return Object.assign(read, { release, [Symbol.dispose]: release });
+  return Object.assign(read, { error, release, [Symbol.dispose]: release });
 }
 
 /** Who is in a channel's room, as the node's presence set says — one entry per member id. */

package/src/use-record-changes.ts

@@ -0,0 +1,47 @@
+// Which records of some types moved, without holding any of them. A whole-type `useRecords` is the
+// way to SHOW every record of a type, and holding the type is its price: none of its rows is
+// evicted while the reader lives. A reader that only wants to know THAT something moved — re-render
+// a server page, flash a counter — pays nothing here: it holds no row and no type.
+
+import { pageStore } from './page-store';
+import { isServerRender } from './reactivity';
+
+/** Per type, the keys of its records that one store update moved (written, removed or evicted). */
+export type RecordChanges = ReadonlyMap<string, readonly string[]>;
+
+/** The listener's undo — call it, `release()` it or `using` it (Solid: `onCleanup`). */
+export type RecordChangesHandle = (() => void) & { release(): void; [Symbol.dispose](): void };
+
+const nothing = (): void => undefined;
+
+/**
+ * Calls `listener` once per store update that moved at least one record of `types`, with what
+ * moved, grouped by type. Holds nothing: a record no other reader holds is evicted as before.
+ */
+export function useRecordChanges(
+  types: readonly string[],
+  listener: (changed: RecordChanges) => void,
+): RecordChangesHandle {
+  if (isServerRender())
+    return Object.assign(nothing, { release: nothing, [Symbol.dispose]: nothing });
+  const watched = new Set(types);
+  const unsubscribe = pageStore('useRecordChanges').subscribe((changed) => {
+    const moved = new Map<string, string[]>();
+    for (const rk of changed) {
+      const at = rk.indexOf(':');
+      const type = rk.slice(0, at);
+      if (!watched.has(type)) continue;
+      const keys = moved.get(type) ?? [];
+      keys.push(rk.slice(at + 1));
+      moved.set(type, keys);
+    }
+    if (moved.size > 0) listener(moved);
+  });
+  let held = true;
+  const release = (): void => {
+    if (!held) return;
+    held = false;
+    unsubscribe();
+  };
+  return Object.assign(release, { release, [Symbol.dispose]: release });
+}