@dokebi/websocket-hub 0.4.1 → 0.4.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AI.md CHANGED
@@ -1,4 +1,4 @@
1
- # AI guide — @dokebi/websocket-hub 0.4.1
1
+ # AI guide — @dokebi/websocket-hub
2
2
 
3
3
  The hub that manages WebSocket connection lists, rooms, broadcasts and kicks (it replaces the old `dokebi-sync`). A broadcast is **at most once**, with no receipt ACK or resend ([D-0057](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0057-websocket-broadcast-without-ack.md)). Usually **@dokebi/websocket uses it alone**, and app code uses websocket's `defineSocket`. Direct use is mostly operational status (`status()`, `dokebi-hub-status`) and replay defense for sealed frames (`nonces()`). Node >=24, core ^0.4.0, peers: @dokebi/redis, @dokebi/nats (cluster mode). Details are in the [README](README.md).
4
4
 
@@ -10,6 +10,7 @@ The hub that manages WebSocket connection lists, rooms, broadcasts and kicks (it
10
10
  | Needs | nothing | Redis (required), NATS (optional: instant leader change notices; otherwise Redis polling) |
11
11
  | Failure | state disappears with the process, no handover | every worker is a hub candidate, the leader is elected by a Redis lease, and when the leader dies real sockets stay and reconnect to the new hub |
12
12
  | cluster workers | refused with `LOCAL_IN_CLUSTER` (allowed only with `allowInCluster: true`) | normal |
13
+ | production (`NODE_ENV=production` in a deployment; not an app the CLI runs here, `DOKEBI_LOCAL_RUN=1`) | refused with `LOCAL_IN_PRODUCTION` (a restart loses the presence and the replay nonces; D-0161) | normal |
13
14
 
14
15
  The API and broadcast semantics (at most once) are the same in both modes. `mode` keeps its name in 0.4.0: it names the deployment style, not a driver ([D-0071](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0071-option-vocabulary.md)).
15
16
 
@@ -41,6 +42,9 @@ dokebi-websocket-hub:
41
42
  | `maxFrameBytes` / `maxPayloadBytes` | integer | 1048576 / 65536 | Hub TCP frame / message |
42
43
  | `maxConnections` / `maxOwners` / `maxTotalConnections` | integer | 128 / 64 / 8192 | Per server / number of servers / all connections |
43
44
  | `maxPending` / `maxPendingBytes` | integer | 128 / 4194304 | Pending limits (more broadcasts in progress than this give `BUSY`) |
45
+ | `broadcastShare` | number 0.05–1 | 0.5 | local mode: the share of the event queue (`maxPending` events, `maxPendingBytes`) broadcasts may fill before they are refused with `BUSY`; the rest stays for presence and kick events, which are never refused (an overflow of those closes the hub). Raise it when broadcasts dominate and presence is rare |
46
+ | `maxNonces` | integer 1–10000000 | 100000 | local mode: live seal replay nonces the in-process store keeps; more refuses new sealed frames (`NONCE_STORE_FULL`, fail closed). Size it above the sealed frames of 2 × the seal drift window (about 60 bytes each); cluster mode keeps them in Redis instead |
47
+ | `logRepeatWindowMs` | integer 1000–3600000 | 10000 | window of every repeated hub log (refusals, peer frame and send failures, `NONCE_STORE_FULL`): the first is written, the rest of the window are counted (`repeated`) |
44
48
 
45
49
  Keys renamed in 0.4.0 (D-0071). The old name is accepted with the warning `OPTION_DEPRECATED`; old and new together fail with `INVALID_OPTIONS`; 0.5.0 rejects it.
46
50
 
@@ -54,7 +58,7 @@ Keys renamed in 0.4.0 (D-0071). The old name is accepted with the warning `OPTIO
54
58
 
55
59
  **Checks (0.4.0, [D-0063](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0063-core-primitives.md))**: the options are one `defineOptions` declaration in `src/options.ts` that is the parser, and `options.schema.json` is generated from it at build time. Rules that only the hand parser used to keep are in the schema now: connection names (`namespace`, `serverId`, `redisConnection`, `natsConnection`) `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`, no spaces or control characters in `host` and `advertiseHost`, and the defaults the parser filled in (`namespace` and `serverId` = `local`, `host` and `advertiseHost` = `127.0.0.1`). Cross-field rules are unchanged: `pollIntervalMs × 3 ≤ leaseMs`, `ownerTtlMs ≥ leaseMs × 3`, base64(`maxPayloadBytes`) + 8192 ≤ `maxFrameBytes`, `maxPendingBytes ≥ maxFrameBytes`. `secret` must be 32–256 **characters** and 32–256 **bytes** (the old schema counted characters, the parser only bytes). On failure the core `INVALID_OPTIONS` is the cause of `INVALID_PLUGIN_OPTIONS`, with paths and issue codes in `details.issues` (no values): `REDIS_REQUIRED` in cluster mode without a Redis connection, `SECRET_REQUIRED` when the secret is missing or has the wrong byte count, otherwise `required`, `pattern`, `check` and so on.
56
60
 
57
- While a server with different settings (secret, limits) is alive in the same namespace, a new one is refused at once with `CONFIG_MISMATCH`; once every server with the old settings is down, the new settings take over.
61
+ While a server with **another secret** is alive in the same namespace, a new one is refused at once with `CONFIG_MISMATCH`; once every server with the old secret is down, the new secret takes over. **Limits may differ between servers** (0.4.2, D-0136), so a rolling deploy can change `leaseMs`, `ownerTtlMs`, `maxConnections`, `maxOwners`, `maxTotalConnections`, `maxFrameBytes` or `maxPayloadBytes`: a server reads what other hub servers send against the largest values any server may set (4 MiB frames, 256 KiB payloads, 4096 connections per server, 1024 servers, 100000 connections), never against its own options. Each server applies its own `maxConnections` and `maxPayloadBytes` to its own clients; the leader's `maxOwners`, `maxTotalConnections` and `maxFrameBytes` (the presence snapshot) and its `maxPayloadBytes` (a larger broadcast is refused for that call with `INVALID_ARGUMENT`) are the shared ones, so during an overlap the stricter of the producer's and the leader's limits applies. Keep every server's `pollIntervalMs` below the leader's `ownerTtlMs` (the per-server checks ensure it when those two are not changed together). Upgrade servers of 0.4.1 or earlier to this version first with unchanged limits, then change limits: older servers still compare every limit.
58
62
 
59
63
  ## 3. API (`import { dependency } from '@dokebi/websocket-hub/dependency'` → `getDependency`)
60
64
 
@@ -62,9 +66,10 @@ The provider descriptor lives in the subpath `@dokebi/websocket-hub/dependency`
62
66
 
63
67
  - `state` (`starting`, `ready`, `recovering`, `closing`, `closed`), `role` (`active`, `standby`), `ownerEpoch`, `leader`, `whenReady(timeoutMs)` → boolean.
64
68
  - `register(...)` → `HubConnection` (`identity`, `join`, `leave`, `updateMetadata`, `close`), `list(query)` → `HubSnapshot`, `broadcast(query, bytes)`, `kick(query)`, `events()`: the low-level API websocket uses. A query (`HubQuery`) is a required `tenant` plus optional `room`, `member`, `connection` (AND). **`events()` has a single consumer**; when websocket uses it, never read it separately. The `message` event (`{ id, connections, data }`) is the whole delivery: there is nothing to confirm (`acknowledge` was removed in 0.3.0).
65
- - `broadcast` → receipt `HubReceipt` `{ id, recipients }`. Failures: `RECOVERING` (not connected to the leader, or the leader is changing; not sent), `OUTCOME_UNKNOWN` (the connection dropped after sending, before the answer; it may have been sent), `BUSY` (broadcasts in progress reached `maxPending`; in local mode, half of the event queue is still unread). The hub never resends. `kick` → `HubKickReceipt` `{ authorityEpoch, id }` (default code 4403).
69
+ - **The newest registration of a `(tenant, member, clientId)` wins** (D-0025 as changed by the 2026-10-26 audit): `register` of a clientId that another connection holds, on this server or any other, is accepted, and the older connection is closed with **4409 `Replaced`** (a `kick` event; websocket closes the socket with that code and reason). Its record leaves the roster first, then the new one joins: presence shows one leave and one join. Two registrations at once: the one the leader (local: this process) handles last wins, and the other is replaced. A server whose link to the leader dropped (a network blip, a failover) reconnects with its whole list and is never refused: records of it that newer registrations took meanwhile (a tab that reconnected elsewhere while its socket here was half open) are left out and the leader tells it to close them (4409 `Replaced`). After a failover a tab of a dead server reconnects elsewhere at once: its record on the dead server is replaced (that server's expiry is kept). The code is the one clients never reconnect on: a replaced tab that reconnected with the same clientId would replace the newer one in turn. A tab that must stay live (a duplicated tab copies sessionStorage) takes a new clientId first. Mixed versions during a rolling deploy: a server before this rule (no `newestWins` in its hello) keeps the old refusal (`DUPLICATE_CLIENT`) for its own registrations, and its records can still be replaced by newer servers' registrations (it gets a kick 4409 `Replaced` for them; until those sockets have closed its lists still carry them, and the leader leaves them out instead of refusing the list, so its other tabs keep registering). Registrations batched into one roster update are decided one by one: a limit fails only its own registration, never the other tabs of the batch.
70
+ - `broadcast` → receipt `HubReceipt` `{ id, recipients }`. Failures: `RECOVERING` (not connected to the leader, or the leader is changing; not sent), `OUTCOME_UNKNOWN` (the connection dropped after sending, before the answer; it may have been sent), `BUSY` (broadcasts in progress reached `maxPending`; in local mode, `broadcastShare` (half) of the event queue is still unread). The hub never resends. `kick` → `HubKickReceipt` `{ authorityEpoch, id }` (default code 4403). Every server connected to the leader that holds matching connections gets the kick; when one holder is not connected (after a failover, a server that has not reconnected yet or died with the old leader, until its record expires) or its frame failed, the kick fails with `OUTCOME_UNKNOWN` after the others were sent.
66
71
  - `status()` (D-0038) → `HubStatus` `{ mode, state, role, leaseState?, leader?, leaseRemainingMs?, servers?, connections?, serverId? }`. Unknown values are **left out**, not filled with 0 (`leaseState: 'unreachable'` when Redis does not answer).
67
- - `nonces()` (D-0038) → `HubNonceStore` `{ shared, setIfNotExists(key, ttlSeconds) }`. cluster shares it between servers through Redis `SET NX PX` (`shared: true`); local keeps it in the process (`shared: false`; used in a cluster worker it warns `HUB_NONCES_NOT_SHARED` once). Pass it to the replay defense of sealed frames (@dokebi/seal `SealTerminator`). Only the shape matches; seal is never imported. The local store holds at most 100000 live nonces and **fails closed** ([D-0075](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0075-replay-capacity.md)): it never forgets a live nonce; expired ones make room first (every expired entry, earliest expiry first, through an expiry heap), otherwise `setIfNotExists` rejects with `HubError` `NONCE_STORE_FULL` (seal answers it as a retryable `SEAL_REPLAY_CAPACITY`: web 503, websocket 1013) and warns `NONCE_STORE_FULL` once per 10 seconds with `max`, `live` and `repeated`. Nonces live 2 × the seal drift (120 s by default), so one local process takes about 830 sealed frames or bodies per second at most; above that use `mode: cluster` (Redis `SET NX PX`, no such limit).
72
+ - `nonces()` (D-0038) → `HubNonceStore` `{ shared, setIfNotExists(key, ttlSeconds), lostAt() }`. `lostAt()` (seal `NonceStore.lostAt`) says since when the store may have forgotten nonces: the local store its creation (a new process knows nothing of the previous one's), the cluster store the epoch of its Redis nonce keys — a key without expiry, checked in the same Lua step as each `SET NX PX` and written again with the current Redis `TIME` when it is missing (a restart without persistence, a FLUSHALL, an eviction; info `HUB_NONCES_EPOCH`). It is primed when the node starts (a failure is error `HUB_NONCES_EPOCH_FAILED`; the first claim writes it then). @dokebi/seal refuses a sealed frame stamped before it for 2 × drift, so a frame captured before the loss is not accepted once more on any server. cluster shares it between servers through Redis `SET NX PX` (`shared: true`); local keeps it in the process (`shared: false`; used in a cluster worker it warns `HUB_NONCES_NOT_SHARED` once). Pass it to the replay defense of sealed frames (@dokebi/seal `SealTerminator`). Only the shape matches; seal is never imported. The local store holds at most 100000 live nonces and **fails closed** ([D-0075](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0075-replay-capacity.md)): it never forgets a live nonce; expired ones make room first (every expired entry, earliest expiry first, through an expiry heap), otherwise `setIfNotExists` rejects with `HubError` `NONCE_STORE_FULL` (seal answers it as a retryable `SEAL_REPLAY_CAPACITY`: web 503, websocket 1013) and warns `NONCE_STORE_FULL` once per 10 seconds with `max`, `live` and `repeated`. Nonces live 2 × the seal drift (120 s by default), so one local process takes about 830 sealed frames or bodies per second at most; above that use `mode: cluster` (Redis `SET NX PX`, no such limit).
68
73
  - Exports: `createPlugin`, `dependency`, `HubError`, types `HubApi`, `HubConnection`, `HubRegistration`, `HubRecord`, `HubQuery`, `HubLeader`, `HubSnapshot`, `HubReceipt`, `HubKickReceipt`, `HubEvent`, `HubLease`, `HubStatus`, `HubNonceStore`.
69
74
 
70
75
  ## 4. Operations command `dokebi-hub-status`
@@ -74,7 +79,7 @@ npx dokebi-hub-status --config dokebi.yaml # human-readable
74
79
  npx dokebi-hub-status --config dokebi.yaml --json # for machines
75
80
  ```
76
81
 
77
- From outside the app process it reads only Redis and shows the leader, the remaining lease time, the number of live servers and the connection count (it never starts a hub or joins the election). When Redis does not answer it exits with code 1. In local mode it says the state lives only inside the process.
82
+ Without `--config` it reads the file `DOKEBI_CONFIG` names (the one the app reads; `src/main.ts` and the CLI set it), else `./dokebi.yaml`. From outside the app process it reads only Redis and shows the leader, the remaining lease time, the number of live servers and the connection count (it never starts a hub or joins the election). When Redis does not answer it exits with code 1. In local mode it says the state lives only inside the process.
78
83
 
79
84
  ## 5. Delivery semantics (D-0057)
80
85
 
@@ -89,7 +94,7 @@ From outside the app process it reads only Redis and shows the leader, the remai
89
94
  ## 6. Index and performance ([D-0068](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0068-performance-stage4.md) §3)
90
95
 
91
96
  - `HubIndex` (`src/hub-index.ts`) keeps tenant → member → connections, tenant → room → connections (insertion-ordered sets), owner → connection → record and recipientKey → record. local updates it on every register, change and close; the leader when the server list changes (`setOwner`/`removeOwner`, rebuilt from the snapshot at start).
92
- - Selection walks only the smallest matching set (a named connection → one; member and room → the smaller; member; room; the whole tenant) and checks each with `matches`, so the answer equals a full scan (receipt semantics, presence and roster unchanged). The `DUPLICATE_CLIENT` check is O(1); `list` and `kick` use the same index.
97
+ - Selection walks only the smallest matching set (a named connection → one; member and room → the smaller; member; room; the whole tenant) and checks each with `matches`, so the answer equals a full scan (receipt semantics, presence and roster unchanged). The clientId lookup of the newest-wins rule is O(1) per record; `list` and `kick` use the same index.
93
98
  - Evidence: a randomized 2000-step test (add, room change, remove, owner swap) compares every filter with the full scan; `benchmarks/scenarios/micro/hub-select.mjs` measures `select`; `websocket-load.bench.mjs 1024 300` passed with no missed, duplicated or misdirected messages and baseline receive p50/p95/p99 6.8/10.8/17.0 ms.
94
99
 
95
100
  ## 7. Errors
@@ -101,13 +106,14 @@ From outside the app process it reads only Redis and shows the leader, the remai
101
106
  | Code | When |
102
107
  |---|---|
103
108
  | `LOCAL_IN_CLUSTER` | local mode started in a node:cluster worker (the role comes from core `context.runtime.role`). `fix`: `mode: cluster` or `allowInCluster: true`. Before 0.4.0 this error was HUB_LOCAL_IN_CLUSTER; the error log keeps that code |
109
+ | `LOCAL_IN_PRODUCTION` | local mode started in production (`NODE_ENV=production`, D-0161). `fix`: `mode: cluster` (Redis) |
104
110
  | `LISTEN_FAILED` · `STARTUP_TIMEOUT` | Could not open the hub TCP · could not join the leader within `startupTimeoutMs` (with `fix`) |
105
111
  | `CONFIG_MISMATCH` · `PROTOCOL_MISMATCH` · `UNAUTHORIZED` | Different settings, version or secret in the same namespace (with `fix`) |
106
112
  | `RECOVERING` · `OUTCOME_UNKNOWN` · `BUSY` | Broadcast failures (§5) |
107
113
  | `STORE_UNAVAILABLE` | Redis failure (the cause is in `cause`) |
108
114
  | `INVALID_ARGUMENT` · `INVALID_OPTIONS` | Wrong API arguments or hub frame values (frame value codes are compared between servers, so they stay as in 0.3) |
109
- | `NONCE_STORE_FULL` | The local nonce store holds 100000 live nonces: the new nonce is refused (fail closed, D-0075). Retryable; `fix`: `mode: cluster` or a smaller seal drift |
110
- | `CONNECTION_LIMIT` · `MESSAGE_TOO_LARGE` · `DUPLICATE_CLIENT` · `DUPLICATE_CONNECTION` · `CLOSED` · `EVENT_OVERFLOW` | Limits, duplicates, shutdown |
115
+ | `NONCE_STORE_FULL` | The local nonce store holds `maxNonces` (100000) live nonces: the new nonce is refused (fail closed, D-0075). Retryable; `fix`: `mode: cluster` or a smaller seal drift |
116
+ | `CONNECTION_LIMIT` · `MESSAGE_TOO_LARGE` · `DUPLICATE_CLIENT` · `DUPLICATE_CONNECTION` · `CLOSED` · `EVENT_OVERFLOW` | Limits, duplicates, shutdown (`DUPLICATE_CLIENT` only from a leader before the newest-wins rule, §3) |
111
117
 
112
118
  ## 8. Cautions
113
119
 
@@ -116,7 +122,8 @@ From outside the app process it reads only Redis and shows the leader, the remai
116
122
  - Each server needs a different `serverId` and an `advertiseHost` and TCP port reachable from other servers (firewalls). The hub TCP has no built-in TLS; use a trusted private network or an encrypted tunnel.
117
123
  - Never close the hub, Redis or NATS before the whole drain (core ordering ensures it).
118
124
  - **Operations ([D-0049](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0049-redis-rejections-and-eviction.md))**: Redis for cluster mode uses `maxmemory-policy noeviction`. With an evicting policy the leader key gets evicted and the hub goes through `STALE_LEADER` and `RECOVERING` (OPS-002 G). Put caches on a separate Redis (`allkeys-lru`). At node start it reads `CONFIG GET maxmemory*` once; with `maxmemory > 0` and a policy other than `noeviction` it logs **error `REDIS_EVICTION_UNSAFE`** (`policy`, `maxmemory`, `connection`) and **starts anyway** (an operator may accept the risk). When safe, debug `REDIS_EVICTION_CHECKED`; a managed Redis or ACL that blocks CONFIG gives debug `REDIS_EVICTION_CHECK_SKIPPED` (`reason: rejected` + the full `error`); an older @dokebi/redis the same code (`reason: unsupported`); any other failure error `REDIS_EVICTION_CHECK_FAILED`. Redis error replies (OOM and so on) now stay as `COMMAND_REJECTED` with the server reply in the cause of `STORE_UNAVAILABLE`.
119
- - Logs ([D-0047](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0047-logging-and-no-silent-errors.md)): exceptions are never dropped silently. Failures that cannot be turned into a defined error or response code are logged at `error`: a fixed English message + `code` + `error` (the full original error: name, whole message, stack and `Caused by: …` for each cause, address, port and errno text included; only URL credentials masked as `//***@`; cut only beyond 16 KB). Examples: lost leader connection `Hub connection dropped` (debug when the node itself closed it with `RECOVERING` or `CLOSED`, during `bye`, or after the leader announced `leaving` — D-0056), recovery start `Hub transport or store unavailable; recovering`, peer send failure `PEER_SEND_FAILED` (backpressure included), peer socket and frame `PEER_SOCKET_ERROR`, `PEER_FRAME_REJECTED` (a socket error before hello is debug `Hub peer connection failed before it was established`, and the startup retry or recovery log reports it with the cause), when the leader vanished first and `bye` could not be sent but Redis `leave` worked, debug `Hub goodbye not delivered; left through Redis` (a successful stop), a leader of another version `PROTOCOL_MISMATCH` (error; the refusing leader logs warn), removed options `OPTION_REMOVED` (warn, once per key at start), NATS `NATS_PUBLISH_FAILED`, `NATS_SUBSCRIPTION_FAILED`, local event queue overflow `EVENT_OVERFLOW`, a Redis failure of `status()` `STATUS_UNREACHABLE`. Redis failures stay as the `cause` of `STORE_UNAVAILABLE`. Places that run per frame or message write the same message and code once per interval (10 seconds) and group the rest as a `repeated` count (the remaining count is written at shutdown too). Bodies, secrets and tokens are never logged.
125
+ - Logs ([D-0047](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0047-logging-and-no-silent-errors.md)): exceptions are never dropped silently. Failures that cannot be turned into a defined error or response code are logged at `error`: a fixed English message + `code` + `error` (the full original error: name, whole message, stack and `Caused by: …` for each cause, address, port and errno text included; only URL credentials masked as `//***@`; cut only beyond 16 KB). Examples: a lost leader connection other than a closed socket `Hub connection dropped` (debug when the node itself closed it with `RECOVERING` or `CLOSED`, during `bye`, or after the leader announced `leaving` — D-0056), recovery start after a Redis or protocol failure `Hub transport or store unavailable; recovering`, a failover that takes longer than its bound (below), peer send failure `PEER_SEND_FAILED` (backpressure included), peer socket and frame `PEER_SOCKET_ERROR` (on the leader side; on the leader connection of a peer it becomes the cause of the failover's `DISCONNECTED`), `PEER_FRAME_REJECTED` (a socket error before hello is debug `Hub peer connection failed before it was established`, and the startup retry or recovery log reports it with the cause), when the leader vanished first and `bye` could not be sent but Redis `leave` worked, debug `Hub goodbye not delivered; left through Redis` (a successful stop), a leader of another version `PROTOCOL_MISMATCH` (error; the refusing leader logs warn), removed options `OPTION_REMOVED` (warn, once per key at start), NATS `NATS_PUBLISH_FAILED`, `NATS_SUBSCRIPTION_FAILED` (the control subscription that a full connection buffer or an oversized message ended is opened again at once, followed by a poll, and logged once per `logRepeatWindowMs` with `reason` and `repeated`; any other end leaves Redis polling alone), local event queue overflow `EVENT_OVERFLOW`, a roster change the leader answered that this server could not take (its `Replaced` kick did not fit the event queue) `ROSTER_COMMIT_FAILED` (the changes of that batch are refused, never left waiting, and the server reconnects unless it is closing), a Redis failure of `status()` `STATUS_UNREACHABLE`. Redis failures stay as the `cause` of `STORE_UNAVAILABLE`. Places that run per frame or message write the same message and code once per interval (10 seconds) and group the rest as a `repeated` count (the remaining count is written at shutdown too). Bodies, secrets and tokens are never logged.
126
+ - **Failover is a warn ([D-0097](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0097-public-plugin-routes-failover-logs-plugin-messages.md) §2)**: when the leader dies (a crash, `kill -9` of its worker, a lost host) the other servers lose their connection to it and reconnect until its lease runs out and a new leader is elected. That is expected and loses nothing, so a started server logs warn `Hub lost the leader connection; failing over` (`code` such as `DISCONNECTED`, `leader`, `boundMs`, the full `error`), each failed reconnect at debug `Hub failover attempt failed; retrying` (`attempts`, `ms`), and the end at warn `Hub failed over to a new leader` (`attempts`, `ms`, `leader`, `previous`, `firstError`, `lastError`, both full with their stacks). The bound is `2 × leaseMs + pollIntervalMs` (default 6.5 s: the old lease runs out, then electing and reaching the next leader takes at most one connect timeout and a poll). A failover still failing past it is error `Hub failover exceeded its bound; still recovering`, once (`code`, `attempts`, `ms`, `boundMs`, `firstError` and the full last `error` with its stack); its end is info `Hub recovered` (`ms`, `attempts`). A leader that announced `leaving` (a planned stop) moves the servers at debug (`Hub moved to the next leader`), with the same bound. Reconnect attempts count only failures of the leader link (`DISCONNECTED`, `RECOVERING`, `OUTCOME_UNKNOWN`, `STALE_LEADER`, `STALE_PEER`, `CONNECT_TIMEOUT`, `ECONNREFUSED`, `ECONNRESET`, `ECONNABORTED`, `EPIPE`, `ETIMEDOUT`, `EHOSTUNREACH`, `ENETUNREACH`); a Redis failure is an error at once.
120
127
  - debug (`DOKEBI_LOG_LEVEL=debug`): election, leader start and stop, lease loss, leader connection and reconnect (`Hub connected to leader`, ms taken, resync count), server registration, removal and expiry, connection list changes, broadcasts `Hub broadcast sent` (leader and local: `id`, `recipients`, `owners`, `ms`; no body), broadcasts of unknown outcome `OUTCOME_UNKNOWN` (not resent), refusals (`BUSY`, `STALE_*`), failures while still recovering (`Hub still recovering`), and the recovery time at info (`Hub recovered`).
121
128
  - `dokebi-hub-status` writes the cause of a Redis failure to stderr (the stdout JSON stays as it is).
122
129
  - Extension hooks for the hub are listed as candidates in [D-0064](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/decisions/D-0064-extension-contract.md) §9; there is no extension class yet.
package/README.md CHANGED
@@ -14,7 +14,7 @@ There are two modes ([D-0029](https://git.nyx-zone.com/dokebi/framework/-/blob/m
14
14
  - NATS (`natsConnection`) is optional. With it, leader changes are announced at once; without it other servers notice through Redis polling (`pollIntervalMs`).
15
15
  - When the hub dies, the real WebSockets of healthy servers stay and reconnect to the new hub.
16
16
  - **Install check:** the install module's `check` looks at the mode and reports the required plugins: Redis in cluster mode, and NATS too when a NATS connection is set.
17
- - **`whenReady(timeoutMs)` (D-0034):** true once ready, false when the time passes or it closes. websocket uses it to wait out short reconnects when accepting upgrades. It logs why the leader connection dropped (`Hub connection dropped`), failures while recovery continues (`Hub still recovering`) and the recovery time (`Hub recovered`).
17
+ - **`whenReady(timeoutMs)` (D-0034):** true once ready, false when the time passes or it closes. websocket uses it to wait out short reconnects when accepting upgrades. It logs why the leader connection dropped (`Hub connection dropped`), failures while recovery continues (`Hub still recovering`) and the recovery time (`Hub recovered`). A leader failover (the leader died) is a warn, `Hub lost the leader connection; failing over` and `Hub failed over to a new leader` with the attempt count and the first and last error; it becomes an error only past `2 × leaseMs + pollIntervalMs` (D-0097 §2).
18
18
  - **presence events:** they come as the full list per owner (server) when connections come and go. When a peer reconnects or the leader changes, owners that differ from the snapshot and owners that disappeared are announced again as presence events, so consumers never miss connections that left meanwhile.
19
19
 
20
20
  ```yaml
@@ -39,7 +39,7 @@ options:
39
39
  secret: replace-with-your-random-secret-at-least-32-bytes
40
40
  ```
41
41
 
42
- `serverId`/`advertiseHost` are per server; `namespace`, `secret`, shared stores and distributed limits must match across the area. Each worker opens its own TCP port (`port` default 0), which must be reachable from other servers. The TCP has no built-in TLS, so use a trusted private network or an encrypted tunnel.
42
+ `serverId`/`advertiseHost` are per server; `namespace`, `secret` and shared stores must match across the area; limits may differ between servers (a rolling deploy that changes them, D-0136). Each worker opens its own TCP port (`port` default 0), which must be reachable from other servers. The TCP has no built-in TLS, so use a trusted private network or an encrypted tunnel.
43
43
 
44
44
  ```ts snippet
45
45
  import { Application, loadConfig } from '@dokebi/core';
@@ -92,7 +92,7 @@ A full snapshot also stays within the frame limit. Check the [option schema](opt
92
92
  - **App guideline:** never leave work that must not be missed (order state, payment results) to a broadcast. Store it in the DB and let the page read the latest state from the server when it reconnects. Use broadcasts as "it changed, read again" notices. Work that must happen goes to @dokebi/job.
93
93
  - **Removed options:** `deliveryTimeoutMs`, `retryMs`, `maxDeliveries`, `deliveryBytes`. Start still works when they remain; each key logs warn `OPTION_REMOVED` once and is ignored. The schema keeps them as deprecated (removed in the next version).
94
94
  - **Version between hub servers:** hello carries `protocol: 2`. The leader refuses servers of another version with `PROTOCOL_MISMATCH` (warn), and the refused server logs an error with both versions and retries. A 0.3 server connected to a 0.2 leader is cut too. When versions mix during a rolling restart, old servers cannot join the new leader, so upgrade every server of the namespace to the same version. The configuration fingerprint stays the same as 0.2 so the version check tells them apart (against a 0.1.x Redis outbox it is still `CONFIG_MISMATCH`).
95
- - `clientId` is an identifier per authenticated device or tab. A concurrent registration of the same `(tenant, member, clientId)` is refused with `DUPLICATE_CLIENT`.
95
+ - `clientId` is an identifier per authenticated device or tab. The newest registration of a `(tenant, member, clientId)` wins: the older connection, on any server, is closed with 4409 `Replaced` (a tab that reconnected elsewhere while its old socket was half open keeps working; a replaced tab must not reconnect with the same clientId). A server that reconnects to the leader is never refused for a clientId another server took meanwhile: those connections of it are closed instead.
96
96
 
97
97
  ## Operational status and replay defense (D-0038)
98
98
 
@@ -115,7 +115,7 @@ Example 18 in the repository covers the public API and a local TCP baseline; exa
115
115
 
116
116
  ## Configuration fingerprint (2026-10-14)
117
117
 
118
- Servers of one namespace need the same secret, lease and limit settings (the configuration fingerprint in Redis). While a server with different settings is alive, a new server fails to start at once with `CONFIG_MISMATCH` and logs an error (`code: CONFIG_MISMATCH`). Once every leader and owner with the old settings is gone (a full restart), the new settings replace the fingerprint. So rotating the secret is a full stop and start, without deleting Redis keys ([ISSUE-0005](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/issues/ISSUE-0005-playground-findings.md) PG-16).
118
+ Servers of one namespace need the same secret (the shared configuration fingerprint in Redis). While a server with another secret is alive, a new server fails to start at once with `CONFIG_MISMATCH` and logs an error (`code: CONFIG_MISMATCH`). Once every leader and owner with the old secret is gone (a full restart), the new one replaces the fingerprint. Since 0.4.2 (D-0136) lease, TTL and size limits may differ between servers: each reads the others' frames against the protocol maxima and applies its own limits to its own clients, the leader's to the shared presence. The full fingerprint of 0.4.1 is still written once (never overwritten) so that running older servers keep matching during the upgrade; upgrade first, then change limits. So rotating the secret is a full stop and start, without deleting Redis keys ([ISSUE-0005](https://git.nyx-zone.com/dokebi/framework/-/blob/main/docs/issues/ISSUE-0005-playground-findings.md) PG-16).
119
119
 
120
120
  ## Shutdown cleanup and startup logs (2026-10-14)
121
121
 
@@ -1,4 +1,14 @@
1
1
  import type { HubQuery, HubRecord } from './types.js';
2
+ /**
3
+ * The close of a connection whose (tenant, member, clientId) registered again, on any server:
4
+ * the newest registration wins (D-0025 as changed in the 2026-10-26 audit). The code is the one
5
+ * clients already never reconnect on: a replaced tab that reconnected with the same clientId would
6
+ * replace the newer one in turn.
7
+ */
8
+ export declare const REPLACED: Readonly<{
9
+ code: 4409;
10
+ reason: "Replaced";
11
+ }>;
2
12
  /**
3
13
  * The identity of a connection in a broadcast receipt: SHA-256 (base64url) of tenant, member and
4
14
  * clientId, or of the owner and connection when the client has no clientId.
package/dist/delivery.js CHANGED
@@ -1,5 +1,12 @@
1
1
  import { createHash } from 'node:crypto';
2
2
  import { matches } from './protocol.js';
3
+ /**
4
+ * The close of a connection whose (tenant, member, clientId) registered again, on any server:
5
+ * the newest registration wins (D-0025 as changed in the 2026-10-26 audit). The code is the one
6
+ * clients already never reconnect on: a replaced tab that reconnected with the same clientId would
7
+ * replace the newer one in turn.
8
+ */
9
+ export const REPLACED = Object.freeze({ code: 4409, reason: 'Replaced' });
3
10
  const recipientKeys = new WeakMap();
4
11
  /**
5
12
  * The identity of a connection in a broadcast receipt: SHA-256 (base64url) of tenant, member and
package/dist/index.js CHANGED
@@ -1,6 +1,5 @@
1
1
  import { definePlugin, providerDependency } from '@dokebi/core';
2
2
  import { LocalHub } from './local.js';
3
- import { HubNode } from './node.js';
4
3
  import { options as declaration } from './options.js';
5
4
  import { HubError } from './types.js';
6
5
  export * from './types.js';
@@ -38,6 +37,16 @@ class HubRuntime {
38
37
  });
39
38
  throw new HubError('LOCAL_IN_CLUSTER');
40
39
  }
40
+ // D-0161: production keeps shared state in Redis too (a restart loses the local hub's).
41
+ if (!context.runtime.describing &&
42
+ !worker &&
43
+ process.env.NODE_ENV === 'production' &&
44
+ process.env.DOKEBI_LOCAL_RUN !== '1') {
45
+ context.logger.log('error', 'Local WebSocket hub in production', {
46
+ code: 'HUB_LOCAL_IN_PRODUCTION',
47
+ });
48
+ throw new HubError('LOCAL_IN_PRODUCTION');
49
+ }
41
50
  const hub = new LocalHub(options, context.logger, worker);
42
51
  context.defer(() => hub.close());
43
52
  context.logger.log('debug', 'Local WebSocket hub started', {
@@ -46,10 +55,19 @@ class HubRuntime {
46
55
  });
47
56
  return hub.api;
48
57
  }
58
+ // Cluster mode only: the node, its leader and peer client load here, not with the local hub.
59
+ const { HubNode } = await import('./node.js');
49
60
  const node = new HubNode(context, options, context.getDependency(redis).connection(options.redisConnection), options.natsConnection === undefined
50
61
  ? undefined
51
62
  : context.getDependency(nats).connection(options.natsConnection));
52
63
  context.defer(() => node.close());
64
+ // D-0077 B: `dokebi describe` sets up without connecting: the cluster node is not started.
65
+ if (context.runtime.describing) {
66
+ context.logger.log('debug', 'WebSocket hub node not started in describe mode', {
67
+ mode: options.mode,
68
+ });
69
+ return node.api;
70
+ }
53
71
  await node.start();
54
72
  return node.api;
55
73
  }
@@ -57,8 +75,8 @@ class HubRuntime {
57
75
  /** Factory input: `{ app, options? }`. Options default to `app.config['dokebi-websocket-hub']`. */
58
76
  export const createPlugin = definePlugin({
59
77
  name: 'dokebi-websocket-hub',
60
- version: '0.4.1',
61
- core: '^0.4.0',
78
+ version: '0.4.2',
79
+ core: '^0.4.2',
62
80
  options: declaration,
63
81
  // local: nothing. cluster: Redis, and NATS only when natsConnection is set (D-0029).
64
82
  dependencies: ({ options }) => options.mode === 'local'
package/dist/leader.d.ts CHANGED
@@ -1,8 +1,15 @@
1
1
  import type { Socket } from 'node:net';
2
2
  import type { Logger } from '@dokebi/core';
3
- import type { HubOptions } from './options.js';
3
+ import { type HubOptions } from './options.js';
4
4
  import type { HubStore } from './store.js';
5
5
  import { type HubLeader } from './types.js';
6
+ /**
7
+ * UTF-8 bytes of `JSON.stringify({ owners: [[owner, list], …] })` (what a reconnecting peer's hello
8
+ * answer carries) from each owner's id and the bytes of its list's JSON: `{"owners":[` and `]}`,
9
+ * then `["<owner>",<list>]` per owner, separated by commas. Owner ids are epochs (UUIDs): no
10
+ * escaping.
11
+ */
12
+ export declare function snapshotBytes(lists: Iterable<readonly [owner: string, listBytes: number]>): number;
6
13
  export declare class Leader {
7
14
  #private;
8
15
  readonly identity: HubLeader;