@ultimat3/realtime 2.0.0 → 3.0.0

package/CLAUDE.md

@@ -79,6 +79,21 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
   prevent. A lane that fails now desyncs its own subscribers and the first failure still reaches
   the caller, but it costs one query id. The lane chains on a settled shadow of each task: one
   fanout that threw must not reject every fanout behind it.
+- **Two reads of one entry are ordered by a READ GENERATION, never by an lsn.** `QueryEntry.lsn`
+  is optional — a definition with no lsn provider answers `''` from every snapshot — so the
+  never-backwards rule expressed purely in lsn terms read `'' >= ''` as "newer" and let the older
+  of two concurrent reads land on top of the newer one's window. The interleaving: a cold
+  subscriber issues P1; the change stream skips a sequence and `registry.invalidate()` marks the
+  entry; a second cold subscriber forces P2, which **clears `stale` on the way in**; P2 lands with
+  the post-gap rows; P1 lands last and overwrites them. `stale` is false, so `fanoutChange`'s
+  repair never fires, the next change patches the pre-gap window and re-snapshots every desynced
+  subscriber out of it — permanently stale on a healthy socket, which is the exact outcome `stale`
+  exists to prevent. `entry.generation` is bumped in `startRead` and `entry.applied` records the
+  newest read whose rows are on the window: an *identity* check, the same one `startRead` makes on
+  `entry.reading` one function down and `packages/cache/src/single-flight.ts:70` makes for the same
+  reason. The lsn guard stays beside it for the other question — a read that resolved behind a
+  *change* the fanout already folded — because those are two orderings and neither answers the
+  other.
 - **The definition's read is once per entry, not once per subscriber.** A cold subscriber arriving
   while another's read is in flight joins that read — N cold subscribers on one query id being N
   reads is the shared window not existing. It is a share, not a cache: the in-flight promise is
@@ -141,6 +156,34 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
   could reach a different one than the container it is standing in for. The KV bucket and the
   presence TTL come back with the transport for the same reason: they are one decision.
 - `sync` is stateless: no sticky sessions, nothing on a socket survives a restart.
+- **A socket the node evicts ITSELF is released through `teardown`, never through
+  `sockets.remove`.** Bun's `close` callback runs `teardown`; a drain and the idle sweep have no
+  callback behind them — Bun's fires a tick later and `sockets.get` misses by then — so whatever
+  they do instead *is* the whole release. `drain()` inlined three of `teardown`'s five steps
+  (`close`, `sockets.remove`, `grants.delete`) and skipped the two the rest of the fleet can see:
+  `registry.unsubscribeSocket` and `presence.leave` per topic. What that left is a `QueryEntry`
+  whose `subscribers` map never empties — matcher, shared window and `WindowLock` pinned, and
+  `source.forget(qid)` never called — and, worse because it is cross-node, a presence member every
+  other node renders for a full TTL. During a **rolling restart** that is every room showing each
+  user twice for up to 30s, beside the same client's reconnection under a new socket id. One
+  `evict(socket, code, reason)`, and every path that ends a socket without a callback takes it.
+- **The idle sweep exists, is armed by `start()`, and its budget is an APPLICATION one.**
+  `SocketRegistry.sweepIdle` had no caller for as long as it existed, so `touch()`, `idleFor` and
+  the 120s default decided nothing and `idleTimeoutMs` was unreachable from `createSyncNode`. The
+  only live guard was `websocket.idleTimeout: 120` handed to Bun — which Bun's own ping/pong
+  renews, so a client whose frame loop is wedged answers pings and keeps its `GrantBook` entry,
+  its `SubscriptionBook` entries and its `#byTopic` membership forever. It is now
+  `SocketRegistry.idle()`, a **query**: this table is three of the five things a socket holds, so
+  the object that can evict one is the node and not the registry. `start()` arms one `.unref()`ed
+  pass every `idleSweepPeriodMs(idleTimeoutMs)` — a quarter of the budget, floored at a second,
+  derived rather than configured because a second knob is a second number that can disagree with
+  the one it is a fraction of — and `release()` clears it beside the presence sweep. **It measures
+  on `Clock.monotonic()`**, the clock `AcceptBudget` already uses: the sweep compares a DURATION,
+  and a duration read off `now().getTime()` is decided by whatever NTP last wrote — a step forward
+  evicts every socket that is talking, a step backward makes `idleFor` negative and spares every
+  socket that is dead, and the sweep had only just gained its first caller when both became
+  reachable. The field is named `lastSeenMonotonicMs` so nobody hands it to `new Date()`;
+  `openedAt` is the wall-clock one and stays that way, because a human reads it.
 - **`drain()` and `stop()` both release what `start()` acquired, and releasing twice is a no-op.**
   A `drain()` is terminal on its own — it closes the hub and evicts every socket — and nothing
   obliges a `stop()` to follow it, so leaving the change subscription and the presence sweep to
@@ -321,6 +364,13 @@ Tier 3 package. Channels, live queries, local-first sync. One protocol for all t
   `Bridge` comment describes, one state earlier. So `close()` sets `#closed` **before** the walk and
   `#open` closes its own subscription when it lands after one, dropping the entry with it so a
   second post-close subscribe opens and closes its own rather than double-unsubscribing this handle.
+  It then **raises** `X_TRANSPORT_UNAVAILABLE` rather than returning: returning let `subscribe` fall
+  through to `joinTopic`, so the socket became a member of a topic nothing on this node is bridged
+  to — silent for the life of the connection, no error on either side, and no reason for the client
+  to redial. Reachable between `hub.close()` inside `node.drain()` and the last in-flight subscribe.
+  `#release` takes the bridge the caller reserved for the same reason: after `close()` cleared the
+  table, that topic name may hold a bridge a LATER subscribe opened, and releasing by name alone
+  decrements somebody else's refcount.
 - Deny by default on topics. No guard = `X_TOPIC_FORBIDDEN`.
 - **A guard that FAILS is not a guard that denied — the hub's copy of the rule the row gate already
   follows.** On `onActorChange` (the re-auth pass) only a denial unsubscribes; anything else keeps

package/README.md

@@ -168,6 +168,7 @@ wire.
 | distinct channel topics per node | 10,000 | `new ChannelHub({ maxTopicsPerNode })` | `X_SUBSCRIPTION_LIMIT` |
 | outbound bytes buffered on one socket | 1 MiB | `createSyncNode({ maxBufferedBytes })` | the frame is dropped and `send` answers `false` |
 | dropped frames before that socket is closed | 32 | `createSyncNode({ maxDroppedFrames })` | close `1013` (`overloaded`), reason `backpressure` |
+| time one socket may route no frame | 120s | `createSyncNode({ idleTimeoutMs })` | close `4001` (`idle`), reason `idle timeout` |
 | retained patch bytes per node | 64 MiB | `new RingChangeBuffer({ maxBytes, maxBytesPerQuery })` | eviction, then a re-snapshot on resume |
 | array lengths and `input` nesting in a frame | `FRAME_LIMITS` | none — a hard ceiling | `X_PROTOCOL_VERSION` |
 
@@ -362,6 +363,18 @@ wire twice by a reconnect that raced an ack.
   revoked grant — every topic on every re-authenticated socket, silently, with the client never told
   to resubscribe. The initial `subscribe` is deliberately not split that way: there is no
   subscription to keep, so a guard that raises refuses that subscribe and the client hears about it.
+- **An idle socket is swept, and the sweep is an APPLICATION budget, not Bun's.** Bun's own
+  `idleTimeout` is renewed by its ping/pong, so a client whose frame loop is wedged answers pings
+  and keeps its grant, its live subscriptions and its topic membership indefinitely. `start()`
+  arms one `.unref()`ed pass every `idleTimeoutMs / 4` (floored at a second, derived rather than
+  configured) and evicts anything past the budget the same way a close does — through the node's
+  `teardown`, never `SocketRegistry.remove`. `SocketRegistry.idle()` is a *query* for that reason:
+  the socket table is three of the five things a socket holds, and the other two are its live
+  subscriptions and its presence membership on the shared set. `sweepIdle` — which closed and
+  removed here, and had no caller at all — is gone. The budget is measured on `Clock.monotonic()`,
+  so `SyncSocket.lastSeenMonotonicMs` is a duration's start and not an instant: an NTP step forward
+  would otherwise evict every socket that is talking, and a step backward would spare every socket
+  that is dead. `openedAt` stays on the wall clock — it is a value a human reads.
 - **A `sync` node shuts down in two phases.** The `accept` phase calls `stopAccepting()`: `/readyz`
   answers 503 and a late upgrade is shed with `retry-after-ms`, while **every socket the node holds
   keeps its patch stream**. The `close` phase is `drain()` then `stop()`. Registered with no phase it
@@ -381,7 +394,10 @@ wire twice by a reconnect that raced an ack.
   caller.
 - **A cold subscribe reads once per query id.** Subscribers arriving during a read join it and each
   runs its own policy pass over the result. A read that resolves behind a change already fanned out
-  is discarded rather than written back: the window only ever moves forwards.
+  is discarded rather than written back: the window only ever moves forwards. Two reads are ordered
+  by a monotonic **read generation** and never by lsn — a definition with no lsn provider answers
+  `''` for every read, and `'' >= ''` let the older of two concurrent reads land on top of the
+  newer one's gap repair, with `stale` already cleared and therefore nothing left to re-read.
 - **A denial drops a row; a gate that could not decide does not.** A policy answer (`X_FORBIDDEN`,
   `X_UNAUTHENTICATED`) is a decision and costs the row, counted as `rowsDenied`. Anything else a
   gate throws — a rule whose lookup timed out, a predicate with a typo — is counted as

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ultimat3/realtime",
-  "version": "2.0.0",
+  "version": "3.0.0",
   "description": "Three-tier realtime: channels, live queries, local-first sync — one protocol, one mutator shape",
   "license": "MIT",
   "type": "module",
@@ -32,8 +32,8 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@ultimat3/core": "2.0.0",
-    "@ultimat3/query": "2.0.0",
+    "@ultimat3/core": "3.0.0",
+    "@ultimat3/query": "3.0.0",
     "nats": "2.29.3"
   }
 }

package/src/channel.ts

@@ -6,7 +6,12 @@
 
 import { type Actor, logger, renderThrowable } from '@ultimat3/core';
 import { formatLsn } from './changefeed';
-import { isPolicyDenial, SubscriptionLimitError, TopicForbiddenError } from './errors';
+import {
+  isPolicyDenial,
+  SubscriptionLimitError,
+  TopicForbiddenError,
+  TransportUnavailableError,
+} from './errors';
 import { subjectMatches, type Transport, type TransportSubscription } from './fanout';
 import type { JsonObject } from './json';
 import type { SocketRegistry, SyncSocket } from './socket';
@@ -152,8 +157,11 @@ export class ChannelHub {
       await this.#authorize(socket.actor, name);
       await this.#open(name, bridge);
     } catch (error) {
-      // The slot this subscribe took, given back on the one path that will never fill it.
-      this.#release(name);
+      // The slot this subscribe took, given back on the one path that will never fill it — and
+      // given back to the bridge this subscribe actually reserved. `close()` clears the table, so
+      // a later subscribe may have put a DIFFERENT bridge under this name in the meantime, and
+      // decrementing that one's refs releases a topic somebody else is holding.
+      this.#release(name, bridge);
       throw error;
     } finally {
       const held = this.#claimed.get(socket) ?? 1;
@@ -164,7 +172,7 @@ export class ChannelHub {
     // membership this socket's close will give back, so the reference taken above has to go now or
     // it is a bridge nothing will ever release.
     if (socket.topics.has(name)) {
-      this.#release(name);
+      this.#release(name, bridge);
       return;
     }
     // Through the registry, never `socket.subscribeTopic` directly: membership and the index the
@@ -297,12 +305,31 @@ export class ChannelHub {
     if (this.#closed) {
       unsubscribeWhenOpen(bridge);
       if (this.#bridges.get(name) === bridge) this.#bridges.delete(name);
+      // RAISED, not returned. Returning let `subscribe` fall through to `joinTopic`, so the socket
+      // became a member of a topic nothing on this node is bridged to: silent for the life of the
+      // connection, with no error on either side and nothing telling the client to redial. The
+      // same refusal the transport itself answers when it is gone, because from the client's side
+      // that is what happened — this node's bus for that topic is closed.
+      throw new TransportUnavailableError({
+        transport: 'channel',
+        reason: `the hub closed while "${name}" was opening`,
+        // The reader is a browser websocket client, which cannot run a CLI — so pure command
+        // advice would be worse than prose here. The shape that satisfies axiom 4 anyway is
+        // `http/src/error-map.ts`'s: a command that SHIPS (`x errors explain`, unlike the planned
+        // `x logs tail`) for whoever is holding a terminal, then the instruction as a comment.
+        fix: 'x errors explain X_TRANSPORT_UNAVAILABLE --json   # then reconnect and resubscribe: this node is draining',
+      });
     }
   }
 
-  #release(name: Topic): void {
+  /**
+   * `expected` is the bridge the caller reserved. Without it a release looks the topic up by name,
+   * and after a `close()` cleared the table that name may hold a bridge a LATER subscribe opened.
+   */
+  #release(name: Topic, expected?: Bridge): void {
     const bridge = this.#bridges.get(name);
     if (!bridge) return;
+    if (expected !== undefined && bridge !== expected) return;
     bridge.refs -= 1;
     if (bridge.refs > 0) return;
     unsubscribeWhenOpen(bridge);

package/src/index.ts

@@ -263,6 +263,7 @@ export {
   createEntry,
   fillWindow,
   orgIdOf,
+  type PendingRead,
   type QueryEntry,
   refillWindowInLane,
 } from './query-window';
@@ -299,8 +300,10 @@ export {
   actorIdOf,
   CLOSE,
   DEFAULT_FRAME_BURST,
+  DEFAULT_IDLE_TIMEOUT_MS,
   DEFAULT_MAX_BUFFERED_BYTES,
   DEFAULT_MAX_FRAMES_PER_SECOND,
+  idleSweepPeriodMs,
   SocketRegistry,
   type SocketRegistryOptions,
   SyncSocket,

package/src/query-window.ts

@@ -32,7 +32,24 @@ export interface QueryEntry {
   /** Serial lane over `rows`/`lsn`. Every fanout and every window assignment takes its turn here. */
   readonly lock: WindowLock;
   /** The read in flight, shared by every subscriber that arrives during it. `null` between reads. */
-  reading: Promise<SnapshotResult> | null;
+  reading: PendingRead | null;
+  /**
+   * Reads issued against this entry, ever. It is the ORDER of two reads, which nothing else here
+   * can answer: a definition with no lsn provider returns `''` from every snapshot.
+   */
+  generation: number;
+  /** The generation of the newest read whose rows are in `rows`. `0` before the first one lands. */
+  applied: number;
+}
+
+/**
+ * One read and which read it is. They are one fact — a joiner needs the promise AND the generation
+ * it will have to compare against when it lands — and two fields on the entry is two writes a later
+ * edit can separate.
+ */
+export interface PendingRead {
+  readonly generation: number;
+  readonly result: Promise<SnapshotResult>;
 }
 
 export function createEntry(
@@ -64,6 +81,8 @@ export function createEntry(
     stale: false,
     lock: new WindowLock(),
     reading: null,
+    generation: 0,
+    applied: 0,
   };
 }
 
@@ -82,18 +101,21 @@ export async function fillWindow(
   // Read before `startRead` clears it: a second caller arriving during the read joins it and is
   // not the one that forced it, which is what keeps one forced read from becoming N.
   const forced = entry.stale;
-  const result = await (forced || entry.reading === null ? startRead(entry) : entry.reading);
+  const pending = forced || entry.reading === null ? startRead(entry) : entry.reading;
+  const result = await pending.result;
   return await entry.lock.run(async () => {
-    if (forced) {
-      // A forced read replaces the window whatever its lsn says: it was issued *because* what is
-      // under it is wrong, and a definition with no lsn provider answers `''` — which the
-      // never-backwards rule below would read as older than what we hold and discard, leaving
-      // every subscriber served from the window the gap already invalidated.
-      entry.rows = result.rows;
-      if (result.lsn > entry.lsn) entry.lsn = result.lsn;
-    } else if (result.lsn >= entry.lsn) {
-      entry.rows = result.rows;
-      entry.lsn = result.lsn;
+    // Two rules, and neither can stand in for the other. Against another READ it is identity —
+    // the same check `startRead` makes on `entry.reading` one function down, and the one
+    // `packages/cache/src/single-flight.ts` makes for the same reason — because an lsn cannot
+    // order two reads at all: a definition with no lsn provider answers `''` for both, and
+    // `'' >= ''` let the older one overwrite the gap repair the newer one had just landed, with
+    // `stale` already cleared by its issue and therefore nothing left to re-read. Against a
+    // CHANGE it is still the lsn, because a fanout moved `entry.lsn` forwards while this read was
+    // in flight and rewinding to what the read saw hands that subscriber rows the fanout has
+    // moved past — except for a forced read, which was issued *because* what is under it is
+    // wrong.
+    if (isNewestRead(entry, pending) && (forced || result.lsn >= entry.lsn)) {
+      applyRead(entry, pending, result);
     }
     return { rows: entry.rows, lsn: entry.lsn };
   });
@@ -105,22 +127,36 @@ export async function fillWindow(
  * that repairs a stale window mid-fanout is spelled here rather than deadlocking on the other.
  */
 export async function refillWindowInLane(entry: QueryEntry): Promise<void> {
-  const result = await startRead(entry);
+  const pending = startRead(entry);
+  const result = await pending.result;
+  // Same identity rule as `fillWindow`: a read issued before this one may still be in flight, and
+  // whichever was issued LAST is the one the window keeps.
+  if (isNewestRead(entry, pending)) applyRead(entry, pending, result);
+}
+
+/** Is this the newest read to have landed? An older one's rows are behind the window, not on it. */
+function isNewestRead(entry: QueryEntry, pending: PendingRead): boolean {
+  return pending.generation > entry.applied;
+}
+
+function applyRead(entry: QueryEntry, pending: PendingRead, result: SnapshotResult): void {
+  entry.applied = pending.generation;
   entry.rows = result.rows;
   if (result.lsn > entry.lsn) entry.lsn = result.lsn;
 }
 
 /** Publishes the in-flight read, and clears it as it settles — the share is per read, not a cache. */
-function startRead(entry: QueryEntry): Promise<SnapshotResult> {
+function startRead(entry: QueryEntry): PendingRead {
   // Cleared here rather than when the read lands: the read about to be issued is the one that
   // answers the staleness, so a second caller must join it instead of forcing another.
   entry.stale = false;
-  const reading = readSnapshot(entry);
+  entry.generation += 1;
+  const reading: PendingRead = { generation: entry.generation, result: readSnapshot(entry) };
   entry.reading = reading;
   const done = (): void => {
     if (entry.reading === reading) entry.reading = null;
   };
-  void reading.then(done, done);
+  void reading.result.then(done, done);
   return reading;
 }
 

package/src/socket.ts

@@ -126,7 +126,13 @@ export class SyncSocket {
   readonly frameBudget: AcceptBudget;
 
   actor: Actor | null;
-  lastSeenAt: number;
+  /**
+   * MONOTONIC milliseconds, not an instant — the idle sweep measures a duration, and a duration
+   * read off the wall clock is decided by whatever NTP last wrote. A step forward evicts sockets
+   * that are talking; a step backward makes `idleFor` negative and spares sockets that are dead.
+   * Named for the units so nobody hands it to `new Date()`; `openedAt` is the wall-clock one.
+   */
+  lastSeenMonotonicMs: number;
   droppedFrames = 0;
   sentFrames = 0;
 
@@ -150,8 +156,10 @@ export class SyncSocket {
       burst: options.frameBurst ?? DEFAULT_FRAME_BURST,
       clock: this.#clock,
     });
+    // Two clocks on purpose: `openedAt` is an instant a human reads, `lastSeenMonotonicMs` is the
+    // start of a duration only this process compares.
     this.openedAt = this.#clock.now().getTime();
-    this.lastSeenAt = this.openedAt;
+    this.lastSeenMonotonicMs = this.#clock.monotonic();
   }
 
   get actorId(): string | null {
@@ -206,11 +214,12 @@ export class SyncSocket {
   }
 
   touch(): void {
-    this.lastSeenAt = this.#clock.now().getTime();
+    this.lastSeenMonotonicMs = this.#clock.monotonic();
   }
 
-  idleFor(now: number): number {
-    return now - this.lastSeenAt;
+  /** `nowMonotonicMs` comes from the SAME clock — `Clock.monotonic()`, never `now().getTime()`. */
+  idleFor(nowMonotonicMs: number): number {
+    return nowMonotonicMs - this.lastSeenMonotonicMs;
   }
 
   close(code: number = CLOSE.normal, reason = ''): void {
@@ -220,9 +229,28 @@ export class SyncSocket {
   }
 }
 
+/**
+ * How long a socket may route no frame before `sync-node` evicts it. It is an APPLICATION
+ * inactivity budget and not Bun's transport one: Bun's `idleTimeout` is renewed by its own
+ * ping/pong, so a client whose TCP stack still answers pings while its frame loop is wedged holds
+ * its grant, its subscriptions and its topic membership forever. A beating client sends a `hello`
+ * every `DEFAULT_HEARTBEAT_MS` (15s), so this is eight missed beats.
+ */
+export const DEFAULT_IDLE_TIMEOUT_MS = 120_000;
+
+/**
+ * How often to ask. A quarter of the budget, floored at a second: a socket is evicted within 25%
+ * of its window of going quiet, and a node holding 50,000 of them pays one pass over the table
+ * four times per window rather than once a second. Derived rather than configured — a second knob
+ * is a second number that can disagree with the one it is a fraction of.
+ */
+export function idleSweepPeriodMs(idleTimeoutMs: number): number {
+  return Math.max(1_000, Math.floor(idleTimeoutMs / 4));
+}
+
 export interface SocketRegistryOptions {
   readonly clock?: Clock;
-  /** Bun also enforces its own `idleTimeout`; this sweep catches half-open connections. */
+  /** Bun's own `idleTimeout` is renewed by its ping/pong; this budget counts routed FRAMES. */
   readonly idleTimeoutMs?: number;
 }
 
@@ -244,7 +272,7 @@ export class SocketRegistry {
 
   constructor(options: SocketRegistryOptions = {}) {
     this.#clock = options.clock ?? systemClock;
-    this.#idleTimeoutMs = options.idleTimeoutMs ?? 120_000;
+    this.#idleTimeoutMs = options.idleTimeoutMs ?? DEFAULT_IDLE_TIMEOUT_MS;
   }
 
   /**
@@ -310,19 +338,26 @@ export class SocketRegistry {
     return this.#sockets.size;
   }
 
-  /** Closes and returns everything past the idle budget. Called on an interval by `sync-node`. */
-  sweepIdle(): SyncSocket[] {
-    const now = this.#clock.now().getTime();
-    const closed: SyncSocket[] = [];
-    for (const socket of this.#sockets.values()) {
-      if (socket.idleFor(now) > this.#idleTimeoutMs) {
-        socket.close(CLOSE.idle, 'idle timeout');
-        // Through `remove`, not the map: the sweep is exactly the abnormal close a gauge leaks on.
-        this.remove(socket.id);
-        closed.push(socket);
-      }
-    }
-    return closed;
+  /** The budget `idle()` answers against, so a caller can size its own sweep from one number. */
+  get idleTimeoutMs(): number {
+    return this.#idleTimeoutMs;
+  }
+
+  /**
+   * Everything past the idle budget. A QUERY, and deliberately not an eviction: this table is three
+   * of the five things a socket holds, and the other two — its live subscriptions and its presence
+   * membership on the SHARED set — are only reachable from `sync-node`'s `teardown`. A sweep that
+   * closed and `remove`d here left a member every other node renders until its TTL and a
+   * `QueryEntry` whose `subscribers` map never empties. `sync-node` is the one caller and it
+   * releases each one the way the close callback does.
+   */
+  idle(): SyncSocket[] {
+    // Monotonic, because this is the one comparison an operator's clock could otherwise decide:
+    // `sync-node` hands this registry and every socket it builds the same `Clock`.
+    const now = this.#clock.monotonic();
+    return [...this.#sockets.values()].filter(
+      (socket) => socket.idleFor(now) > this.#idleTimeoutMs,
+    );
   }
 
   /**

package/src/sync-node.ts

@@ -14,6 +14,7 @@ import { CHANGE_SUBJECT_PREFIX, parseEnvelope, SeqGapDetector } from './replicat
 import {
   CLOSE,
   DEFAULT_MAX_BUFFERED_BYTES,
+  idleSweepPeriodMs,
   SocketRegistry,
   SyncSocket,
   type WsLike,
@@ -76,6 +77,12 @@ export interface SyncNodeOptions {
    */
   readonly maxBufferedBytes?: number;
   readonly maxDroppedFrames?: number;
+  /**
+   * How long a socket may route no frame before this node evicts it. Every ceiling on a socket
+   * `sync` builds has to be reachable from here, and this one was not: `SocketRegistry`'s default
+   * was only settable by constructing the registry yourself, and nothing swept it either way.
+   */
+  readonly idleTimeoutMs?: number;
   readonly onMutate?: MutationHandler;
   /**
    * Who is dialling. Injected for the same reason `onMutate` is: `sync` owns no business logic and
@@ -131,7 +138,11 @@ export interface SyncNode {
 
 export function createSyncNode(options: SyncNodeOptions): SyncNode {
   const sockets =
-    options.sockets ?? new SocketRegistry({ ...(options.clock ? { clock: options.clock } : {}) });
+    options.sockets ??
+    new SocketRegistry({
+      ...(options.clock ? { clock: options.clock } : {}),
+      ...(options.idleTimeoutMs === undefined ? {} : { idleTimeoutMs: options.idleTimeoutMs }),
+    });
   const clock = options.clock ?? systemClock;
   const accept = options.accept ?? new AcceptBudget({ perSecond: 500, burst: 2000, clock });
   const maxConnections = options.maxConnections ?? DEFAULT_MAX_CONNECTIONS;
@@ -143,6 +154,7 @@ export function createSyncNode(options: SyncNodeOptions): SyncNode {
   let changes: TransportSubscription | null = null;
   let sweeping: ReturnType<typeof setInterval> | null = null;
   let reauthing: ReturnType<typeof setInterval> | null = null;
+  let idling: ReturnType<typeof setInterval> | null = null;
 
   /**
    * Work nobody is waiting on — a presence leave from a synchronous close, a sweep on a timer, a
@@ -178,6 +190,8 @@ export function createSyncNode(options: SyncNodeOptions): SyncNode {
     sweeping = null;
     if (reauthing !== null) clearInterval(reauthing);
     reauthing = null;
+    if (idling !== null) clearInterval(idling);
+    idling = null;
     gaps.forget();
   };
 
@@ -200,6 +214,17 @@ export function createSyncNode(options: SyncNodeOptions): SyncNode {
     }
   };
 
+  /**
+   * The node's one eviction: close, then release everything the socket held. Every path that ends
+   * a socket without a `close` callback behind it — the drain, the idle sweep — goes through it,
+   * because dropping the socket from the table is three of `teardown`'s five steps and the two it
+   * misses are the ones another node can see.
+   */
+  const evict = (socket: SyncSocket, code: number, reason: string): void => {
+    socket.close(code, reason);
+    teardown(socket);
+  };
+
   /**
    * One pass over the grants whose window has closed. This is the half R2 was missing: `reauthorize`
    * and `onActorChange` were both written and neither had a caller, so a socket that was accepted
@@ -277,6 +302,14 @@ export function createSyncNode(options: SyncNodeOptions): SyncNode {
         );
         sweeping.unref();
       }
+      // The half-open connection Bun's own `idleTimeout` renews through its ping/pong: a client
+      // whose frame loop is wedged answers pings and keeps its grant, its subscriptions and its
+      // topic membership. `sweepIdle` was written for this and never called, so `touch()` and the
+      // 120s budget under it decided nothing.
+      idling = setInterval(() => {
+        for (const socket of sockets.idle()) evict(socket, CLOSE.idle, 'idle timeout');
+      }, idleSweepPeriodMs(sockets.idleTimeoutMs));
+      idling.unref();
       if (options.authenticate) {
         reauthing = setInterval(
           () => detach(reauthenticate(), 'sync.reauthenticate'),
@@ -419,11 +452,13 @@ export function createSyncNode(options: SyncNodeOptions): SyncNode {
       }
       const graceMs = drainOptions.graceMs ?? 5_000;
       if (graceMs > 0) await new Promise((resolve) => setTimeout(resolve, graceMs));
-      for (const socket of [...sockets.all()]) {
-        socket.close(CLOSE.goingAway, 'drain');
-        sockets.remove(socket.id);
-        grants.delete(socket.id);
-      }
+      // Through `evict`, never `sockets.remove` + `grants.delete`: those are three of `teardown`'s
+      // five steps, and the two they skip are the ones the rest of the fleet can see. A drained
+      // socket that never left its presence set is a member every other node renders for a full
+      // TTL — during a rolling restart, beside the same client's reconnection under a new id —
+      // and its live subscriptions stay in the registry, so `entry.subscribers` never empties and
+      // the matcher, the shared window and the retained ring are pinned for the process's life.
+      for (const socket of [...sockets.all()]) evict(socket, CLOSE.goingAway, 'drain');
       // Released once the sockets are gone rather than at the top: a client is entitled to its
       // patches for the whole grace window, and it is entitled to them *before* the hub the
       // fanout writes through is closed.