@luckystack/sync 0.10.2 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +9 -0
- package/CHANGELOG.md +7 -0
- package/README.md +2 -150
- package/dist/_shared/clientFanout.d.ts +39 -0
- package/dist/_shared/errorBuilders.d.ts +35 -0
- package/dist/_shared/errorCodes.d.ts +3 -0
- package/dist/_shared/logFlags.d.ts +2 -0
- package/dist/_shared/receiverAuth.d.ts +34 -0
- package/dist/_shared/redactToken.d.ts +10 -0
- package/dist/_shared/streamEmitters.d.ts +51 -0
- package/dist/_shared/syncTypes.d.ts +74 -0
- package/dist/_shared/validationMode.d.ts +2 -0
- package/dist/client.d.ts +4 -129
- package/dist/handleHttpSyncRequest.d.ts +33 -0
- package/dist/handleSyncRequest.d.ts +7 -0
- package/dist/index.d.ts +7 -91
- package/dist/index.js.map +1 -1
- package/dist/streamThrottle.d.ts +45 -0
- package/dist/syncRequest.d.ts +126 -0
- package/docs/callback-registration.md +3 -255
- package/docs/error-states.md +11 -251
- package/docs/ignore-self.md +3 -160
- package/docs/room-fanout.md +11 -247
- package/docs/server-vs-client-handlers.md +5 -321
- package/docs/streaming.md +3 -346
- package/docs/sync-request.md +3 -282
- package/docs/version-policy.md +3 -360
- package/package.json +4 -4
- package/CLAUDE.md +0 -128
- package/dist/errorCodes-B8GoTenr.d.ts +0 -5
package/docs/room-fanout.md
CHANGED
|
@@ -1,257 +1,21 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Room fanout
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A receiver supplied by a client must be authorized before delivery. Server-side policy limits global broadcast and room targeting; membership for socket requests comes from the connection, while HTTP/SSE uses persisted session room membership because it has no originator socket. Do not use recipient-specific handlers as a substitute for this sender gate. See `../src/_shared/receiverAuth.ts`, `../src/handleSyncRequest.ts`, `../src/handleHttpSyncRequest.ts`.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
The two membership sources are deliberate (ADR 0059). The socket check tests the room the fanout will actually deliver to. HTTP has no originator socket, and resolving the caller's sockets across instances before authorization would add a Redis round-trip to every HTTP sync. Consequence: on routed HTTP, a successful sync does not prove that the caller is in the room or will receive the fanout. A session can still list a room that its socket has not re-joined since a reconnect. Re-joining after a reconnect is the consumer's job.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Each authenticated socket's private token room is for session-targeted messages; never expose token room names to unrelated callers. Tenant room formatting must be consistent when joining and fanning out (`../../core/src/roomNameFormatterRegistry.ts`). The `userId` a formatter receives is context only. It is the joiner on join and the sender on fanout. A formatter that builds it into a content room's physical name therefore splits join and send into different rooms, and every broadcast reaches nobody (ADR 0060). Put per-user isolation in the room code itself.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
## Recipients are resolved across instances, before the handler commits
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
Both transports enumerate the room across every instance that shares the adapter, after authorization and before the `_server` handler (ADR 0063). An empty list or a failed lookup is therefore reported before anything is persisted (see `./error-states.md`). The accepted cost: the recipient snapshot is taken one handler execution earlier, so a socket that joins while the handler runs misses that one fanout.
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
A recipient on another instance is a full recipient. Its per-recipient handler runs on the instance handling the request, and the result is delivered through the remote socket handle. Correctness does not need sticky sessions.
|
|
14
14
|
|
|
15
|
-
|
|
16
|
-
2. **Enumerate across instances — before `_server` runs.** The framework asks Socket.io for the members of that physical room on **every** instance sharing the Redis adapter (`io.in(room).fetchSockets()`), or for every socket everywhere when the receiver is `'all'`. The result is a list of `RemoteSocket`s — never a lookup in this process's local room map. This happens after auth, rate-limit and the authorize hooks but **before** input validation and the `_server` handler, so nothing has been persisted yet when it fails.
|
|
17
|
-
3. **Empty → `sync.noReceiversFound`.** An empty list ends the request with that error on **both** transports, before `_server` runs, before `preSyncFanout` fires and before any recipient is touched. See §8.
|
|
18
|
-
4. `_server` (validate input, execute, persist), then `preSyncFanout`, then the per-recipient loop (§3).
|
|
15
|
+
Never build a fanout target, presence list or room snapshot from the per-instance socket maps (`io.sockets.adapter.rooms`, `adapter.sids` or enumerating `io.sockets.sockets`). Under the Redis adapter they return a partial view that looks complete, with no error, and silently drop the rest of the cluster (ADR 0061). Use `getRoomSockets` (`../../core/src/roomSockets.ts`) or `io.in(room).fetchSockets()`. A lint rule, `no-local-socket-enumeration`, and a guard on `getIoInstance()` that throws outside production both catch this mistake. Deliberate per-instance work opts out explicitly with a stated reason. See `../../core/src/localSocketEnumerationGuard.ts`, `../../core/docs/redis-adapter.md`.
|
|
19
16
|
|
|
20
|
-
|
|
17
|
+
## Per-recipient hook and counts
|
|
21
18
|
|
|
22
|
-
|
|
19
|
+
`preSyncRecipient` acts on one recipient, on both transports and for remote recipients too. A stop skips only that recipient. A stop with an override output delivers a redacted payload instead. Its `recipientUserId` is always null, because resolving a user per socket would cost one session read per recipient on the hot path. A handler that needs the user reads the session by `recipientToken`. Payload shape: `HookPayloads` in `@luckystack/core`.
|
|
23
20
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
---
|
|
27
|
-
|
|
28
|
-
## 2. Room conventions
|
|
29
|
-
|
|
30
|
-
Rooms are managed by `@luckystack/core` (client) via `joinRoom(code)` / `leaveRoom(code)`. Conventions in production code:
|
|
31
|
-
|
|
32
|
-
| Room name | Membership | Used for |
|
|
33
|
-
|---|---|---|
|
|
34
|
-
| `<sessionToken>` (auto-joined) | Every socket of one user | `streamTo(token, payload)` reaches all that user's devices. |
|
|
35
|
-
| `<sharedCode>` (manual join) | Multiple users by app logic | Collab editors, multiplayer rooms, chat. |
|
|
36
|
-
| `'all'` (sentinel, not a real room) | Every connected socket | Global broadcasts. Avoid in production. |
|
|
37
|
-
|
|
38
|
-
### Why every socket auto-joins a room named after its session token
|
|
39
|
-
|
|
40
|
-
`@luckystack/server` joins each socket to a room with that socket's session token at connect time. This is what makes `streamTo(['userToken1', 'userToken2'], payload)` work — without it the framework would have to maintain a separate `tokenToSockets` map.
|
|
41
|
-
|
|
42
|
-
Side effect: if a user is connected from three devices (three sockets, three different `socket.id`s, one session token), `streamTo` reaches all three because they all live in the same per-token room.
|
|
43
|
-
|
|
44
|
-
---
|
|
45
|
-
|
|
46
|
-
## 3. The fanout loop
|
|
47
|
-
|
|
48
|
-
After auth, rate-limit, validation, and `_server` execution succeed, `handleSyncRequest` enters the per-recipient loop:
|
|
49
|
-
|
|
50
|
-
```
|
|
51
|
-
preSyncFanout (stop signal aborts before any recipient is touched)
|
|
52
|
-
|
|
|
53
|
-
v
|
|
54
|
-
for each socket in <resolved sockets>:
|
|
55
|
-
yield every N (configurable)
|
|
56
|
-
skip if ignoreSelf && token === recipientToken
|
|
57
|
-
preSyncRecipient (stop without overrideOutput skips this recipient, uncounted;
|
|
58
|
-
stop WITH overrideOutput replaces serverOutput for this recipient)
|
|
59
|
-
recipientCount++
|
|
60
|
-
if _client exists:
|
|
61
|
-
run clientHandler
|
|
62
|
-
emit per-recipient result (success or normalized error)
|
|
63
|
-
else:
|
|
64
|
-
emit { serverOutput, clientOutput: {}, status: 'success', cb, fullName }
|
|
65
|
-
|
|
|
66
|
-
v
|
|
67
|
-
postSyncFanout({ recipientCount, ...payload })
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
Per-recipient errors do **not** abort the loop. A single recipient failing `_client` execution receives a `sync.clientExecutionFailed` frame; everyone else still receives their merged payload.
|
|
71
|
-
|
|
72
|
-
---
|
|
73
|
-
|
|
74
|
-
## 4. Event-loop yielding (`sync.fanoutYieldEvery`, `sync.fanoutYieldMs`)
|
|
75
|
-
|
|
76
|
-
```ts
|
|
77
|
-
const { fanoutYieldEvery, fanoutYieldMs } = getProjectConfig().sync;
|
|
78
|
-
let tempCount = 1;
|
|
79
|
-
for (const socketEntry of sockets) {
|
|
80
|
-
tempCount++;
|
|
81
|
-
if (tempCount % fanoutYieldEvery === 0) {
|
|
82
|
-
await new Promise(resolve => setTimeout(resolve, fanoutYieldMs));
|
|
83
|
-
}
|
|
84
|
-
// ... fanout to this recipient ...
|
|
85
|
-
}
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
Why: a `receiver: 'all'` fanout against thousands of sockets would otherwise block the event loop for the whole iteration. Yielding every `fanoutYieldEvery` recipients (default: see `projectConfig.sync.fanoutYieldEvery`) for `fanoutYieldMs` (default: a few ms) lets other socket events, API requests, and the heartbeat handlers run.
|
|
89
|
-
|
|
90
|
-
Tuning:
|
|
91
|
-
|
|
92
|
-
- **Higher `fanoutYieldEvery` + lower `fanoutYieldMs`** = faster fanout, less responsiveness for other requests.
|
|
93
|
-
- **Lower `fanoutYieldEvery` + higher `fanoutYieldMs`** = smoother for concurrent traffic but slower fanout.
|
|
94
|
-
- For typical room sizes (<100 recipients), the yield never triggers — defaults are tuned for `receiver: 'all'` worst cases.
|
|
95
|
-
|
|
96
|
-
Both transports yield on the same schedule — the HTTP handler's loop reads the same two config keys. An HTTP-invoked fanout still runs on the event loop of the instance that received the request, so a giant `'all'` fanout would starve it just the same.
|
|
97
|
-
|
|
98
|
-
---
|
|
99
|
-
|
|
100
|
-
## 5. Hooks dispatched during fanout
|
|
101
|
-
|
|
102
|
-
### `preSyncFanout`
|
|
103
|
-
|
|
104
|
-
```ts
|
|
105
|
-
{
|
|
106
|
-
routeName: string, // e.g. 'board/moveCard/v1'
|
|
107
|
-
data: Record<string, unknown>, // clientInput
|
|
108
|
-
user: SessionLayout | null, // sender's session
|
|
109
|
-
receiver: string, // resolved roomCode or 'all'
|
|
110
|
-
serverOutput: unknown, // what _server returned (minus status)
|
|
111
|
-
}
|
|
112
|
-
```
|
|
113
|
-
|
|
114
|
-
Fires **after** `_server` runs successfully and the recipient set is resolved, **before** any recipient receives the payload. Stop signal converts to an originator-side error envelope with the hook's `errorCode` / `httpStatus`. Use for:
|
|
115
|
-
|
|
116
|
-
- "Don't fanout this mutation to a degraded region while we drain traffic."
|
|
117
|
-
- "Throttle fanout-heavy routes during peak load."
|
|
118
|
-
- "Inject a cross-room replication hop before the room receives the payload."
|
|
119
|
-
|
|
120
|
-
### `preSyncRecipient`
|
|
121
|
-
|
|
122
|
-
Fires once per resolved recipient, after the `ignoreSelf` skip and before that one socket receives anything. It carries the route name, the receiver, the recipient's socket id, the recipient's session token (`recipientToken`, `null` for an anonymous socket) and the `serverOutput` about to be sent. Exact shape: `HookPayloads` in `@luckystack/core`.
|
|
123
|
-
|
|
124
|
-
Semantics that differ from the other fanout hooks:
|
|
125
|
-
|
|
126
|
-
- A stop signal **without** `overrideOutput` skips just this recipient — the loop continues, and the skipped socket is not counted in `recipientCount`.
|
|
127
|
-
- A stop signal **with** `overrideOutput` still delivers to this recipient, but with the override in place of `serverOutput` — per-recipient redaction without a `_client` file.
|
|
128
|
-
- `recipientUserId` is **always `null`**. There is no resolver option: resolving a user per recipient would cost a session read per socket on the hot path. A handler that needs the user reads the session itself by `recipientToken` (the token is already known in the loop, so it costs nothing to expose).
|
|
129
|
-
|
|
130
|
-
Fires on both transports and for remote recipients too (§7).
|
|
131
|
-
|
|
132
|
-
### `postSyncFanout`
|
|
133
|
-
|
|
134
|
-
```ts
|
|
135
|
-
{
|
|
136
|
-
routeName: string,
|
|
137
|
-
data: Record<string, unknown>,
|
|
138
|
-
user: SessionLayout | null,
|
|
139
|
-
receiver: string,
|
|
140
|
-
serverOutput: unknown,
|
|
141
|
-
recipientCount: number, // actual number of sockets emitted to (NOT room size)
|
|
142
|
-
}
|
|
143
|
-
```
|
|
144
|
-
|
|
145
|
-
Fires after the last recipient's emit. Observation-only — there is no stop signal because the fanout has already happened. Use for:
|
|
146
|
-
|
|
147
|
-
- Audit logs ("this mutation reached N viewers").
|
|
148
|
-
- Metrics (`fanout_size_histogram.observe(recipientCount)`).
|
|
149
|
-
- Cross-region eventual-consistency markers.
|
|
150
|
-
|
|
151
|
-
### `rateLimitExceeded`
|
|
152
|
-
|
|
153
|
-
```ts
|
|
154
|
-
// Scope 'user' or 'route' (per-token / per-IP per-route bucket):
|
|
155
|
-
{
|
|
156
|
-
scope: 'user' | 'route',
|
|
157
|
-
key: string, // 'token:<token>:sync:<route>' or 'ip:<ip>:sync:<route>'
|
|
158
|
-
limit: number,
|
|
159
|
-
windowMs: number,
|
|
160
|
-
count: number,
|
|
161
|
-
route: string,
|
|
162
|
-
userId: string | undefined,
|
|
163
|
-
}
|
|
164
|
-
|
|
165
|
-
// Scope 'ip' (global per-IP cross-route bucket):
|
|
166
|
-
{
|
|
167
|
-
scope: 'ip',
|
|
168
|
-
key: string, // 'ip:<ip>:sync:all'
|
|
169
|
-
limit: number,
|
|
170
|
-
windowMs: number,
|
|
171
|
-
count: number,
|
|
172
|
-
ip: string,
|
|
173
|
-
}
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
Fires before fanout begins, when either bucket rejects. Used to surface abusive senders and feed automated mitigation.
|
|
177
|
-
|
|
178
|
-
---
|
|
179
|
-
|
|
180
|
-
## 6. `recipientCount` vs raw room size
|
|
181
|
-
|
|
182
|
-
`recipientCount` differs from the size of the resolved recipient list in two cases, identically on both transports:
|
|
183
|
-
|
|
184
|
-
1. **`ignoreSelf: true`** — every socket whose extracted token matches the sender's is skipped. If a user has 3 sockets in the room and triggered the sync themselves, `recipientCount` is `size - 3`.
|
|
185
|
-
2. **`preSyncRecipient` stop without `overrideOutput`** — the recipient is skipped and not counted (§5).
|
|
186
|
-
|
|
187
|
-
What does **not** reduce it:
|
|
188
|
-
|
|
189
|
-
- **Sockets disappearing mid-fanout.** The recipient list is a snapshot of `RemoteSocket`s taken once, before the loop. A socket that disconnects between the snapshot and its turn is still iterated: the loop emits to the remote handle, the adapter drops the emit because the target is gone, and the recipient is still counted. There is no per-iteration "does this socket still exist" re-check against a local map — that check would only be answerable for local sockets anyway.
|
|
190
|
-
- **A failing `_client`.** The recipient is counted before `_client` runs, so a recipient that received an error frame counts the same as one that received a success frame.
|
|
191
|
-
|
|
192
|
-
So `recipientCount` is "how many recipients the loop emitted to", not "how many received the payload" — the second number does not exist on the server side of an adapter. Observers wanting delivery confirmation need a client-side ack of their own.
|
|
193
|
-
|
|
194
|
-
---
|
|
195
|
-
|
|
196
|
-
## 7. Cross-instance fanout via Redis adapter
|
|
197
|
-
|
|
198
|
-
Single-instance fanout is built into Socket.io. **Cross-instance fanout requires the Redis adapter**, which `@luckystack/server` attaches on every backend (see [`/docs/ARCHITECTURE_SOCKET.md`](../../../docs/ARCHITECTURE_SOCKET.md)). Two mechanisms ride on it:
|
|
199
|
-
|
|
200
|
-
- `io.to(room).emit(...)` — used by the streaming emitters — publishes to Redis; every instance delivers to its own members of that room.
|
|
201
|
-
- `io.in(room).fetchSockets()` + `RemoteSocket.emit()` — used by the regular sync fanout — enumerates the room across all instances and delivers to each member individually.
|
|
202
|
-
|
|
203
|
-
**The per-recipient `_client` handler runs for remote recipients too.** The fanout loop iterates the cross-instance list, runs `_client` on the instance that is handling the request (it has the route code, the `serverOutput`, and the recipient's handshake headers and token), and delivers the result through the recipient's `RemoteSocket`. No sticky sessions are needed for correctness; spreading one room's members across instances is fine. The cost model (one Redis round-trip per fanout plus one emit per remote recipient) is in [`/docs/ARCHITECTURE_MULTI_INSTANCE.md`](../../../docs/ARCHITECTURE_MULTI_INSTANCE.md).
|
|
204
|
-
|
|
205
|
-
> **Warning — `io.sockets.adapter.rooms`, `io.sockets.adapter.sids` and enumerating `io.sockets.sockets` are per-instance maps.** They answer "who is connected to *this process*", which under the Redis adapter is a partial view that looks complete: no error, just missing members. A helper that builds a room snapshot, a presence list or a broadcast target from them works on one instance and silently drops the rest of the cluster on two. Use instead:
|
|
206
|
-
>
|
|
207
|
-
> - `getRoomSockets(room, { userId? })` from `@luckystack/core` — routes the room through the formatter under `'broadcast'` and returns the cross-instance `RemoteSocket[]` (`'all'` = every socket everywhere). It throws when no Socket.io server is registered, because a silent empty list is exactly the failure this exists to prevent.
|
|
208
|
-
> - `io.in(room).fetchSockets()` / `io.to(room).emit()` directly when you already hold the physical room name.
|
|
209
|
-
>
|
|
210
|
-
> Three guards catch the mistake before production does: the ESLint rule `luckystack/no-local-socket-enumeration` in `@luckystack/core/eslint` flags the patterns statically; outside production `getIoInstance()` returns a guarded view that **throws** on those accessors (`sockets.sockets.get(id)` stays allowed); and `getIoInstance({ raw: true })` is the explicit opt-in for deliberate per-instance work — a sweep cleaning up its own connections, or sampling this process's backpressure. In production the raw instance is always returned, so the guard costs nothing on the hot path.
|
|
211
|
-
|
|
212
|
-
`recipientCount` in `postSyncFanout` counts recipients on every instance, not just local ones.
|
|
213
|
-
|
|
214
|
-
---
|
|
215
|
-
|
|
216
|
-
## 8. `sync.noReceiversFound`
|
|
217
|
-
|
|
218
|
-
Triggered on **both transports** when the cross-instance recipient list (§1) comes back empty. Before 0.10.0 only the socket transport rejected this; the HTTP/SSE path returned `success` with zero recipients, so an HTTP caller could not tell "delivered" from "nobody there". Typical causes:
|
|
219
|
-
|
|
220
|
-
- The room name was misspelled.
|
|
221
|
-
- Every member already disconnected before the request reached the fanout step.
|
|
222
|
-
- A bug had the client `joinRoom`-ing under a different name than the sender's `receiver` argument.
|
|
223
|
-
- A room-name formatter that folds the caller's `userId` into the physical name — join and fanout then land in different physical rooms (see the formatter contract in `packages/core/docs/socket-bootstrap.md`).
|
|
224
|
-
- On `routed-http`: the caller's session still lists the room in `roomCodes` (so the membership check passed) but no socket has re-joined it — see the membership section in [`/docs/ARCHITECTURE_SYNC.md`](../../../docs/ARCHITECTURE_SYNC.md).
|
|
225
|
-
- `receiver: 'all'` while no sockets are connected at all (development edge case).
|
|
226
|
-
|
|
227
|
-
Surfaced to the originator as:
|
|
228
|
-
|
|
229
|
-
```ts
|
|
230
|
-
{ status: 'error', errorCode: 'sync.noReceiversFound', message: '<localized>', httpStatus: 404 }
|
|
231
|
-
```
|
|
232
|
-
|
|
233
|
-
Default `httpStatus` mapping for this code comes from `defaultHttpStatusForResponse` in `@luckystack/core`.
|
|
234
|
-
|
|
235
|
-
This is not necessarily a bug — sending to an empty room is legal if the sender doesn't yet know the room is empty. Recipients are resolved **before** `_server` runs (§1), so on this error nothing has been persisted: the mutation did NOT happen, and a retry once someone is listening is safe. (Earlier versions ran the check after `_server`, so the UI had to read this error as "saved, but nobody was listening"; that reading is now wrong.)
|
|
236
|
-
|
|
237
|
-
---
|
|
238
|
-
|
|
239
|
-
## 9. Sanity-check checklist
|
|
240
|
-
|
|
241
|
-
- Are sockets joining the right room? `socket.rooms` on the recipient side lists every room they're in (including the auto-joined `<sessionToken>` room and `<socket.id>` self-room).
|
|
242
|
-
- Is the Redis adapter wired in production? `getIoInstance().of('/').adapter` should be `RedisAdapter`, not `Adapter`.
|
|
243
|
-
- Are giant fanouts hot in the profiler? Bump `fanoutYieldEvery` higher OR move to per-token rooms instead of `receiver: 'all'`.
|
|
244
|
-
- Is `recipientCount` consistently below room size? Probably `ignoreSelf: true` + multi-tab usage. Expected; not a bug.
|
|
245
|
-
|
|
246
|
-
---
|
|
247
|
-
|
|
248
|
-
## 10. Related
|
|
249
|
-
|
|
250
|
-
- Originator API: [`./sync-request.md`](./sync-request.md)
|
|
251
|
-
- Handler authoring: [`./server-vs-client-handlers.md`](./server-vs-client-handlers.md)
|
|
252
|
-
- Skip-self semantics: [`./ignore-self.md`](./ignore-self.md)
|
|
253
|
-
- Streaming fanout: [`./streaming.md`](./streaming.md)
|
|
254
|
-
- Error catalog (including `sync.noReceiversFound`): [`./error-states.md`](./error-states.md)
|
|
255
|
-
- Socket.io + Redis adapter: [`/docs/ARCHITECTURE_SOCKET.md`](../../../docs/ARCHITECTURE_SOCKET.md)
|
|
256
|
-
- Hook payload shapes: `@luckystack/core` `HookPayloads`
|
|
257
|
-
- Config: `projectConfig.sync.fanoutYieldEvery`, `projectConfig.sync.fanoutYieldMs`
|
|
21
|
+
`recipientCount` counts the recipients the loop emitted to, not the ones that received the payload. A socket that disconnected after the snapshot and a recipient whose handler failed are still counted. Only `ignoreSelf` and a skipping `preSyncRecipient` stop lower the count. Across an adapter, the server has no delivery confirmation, so an observer that needs one must add a client-side acknowledgement. Fanout yields to the event loop on both transports, because a fanout started over HTTP still runs on the receiving instance's event loop.
|
|
@@ -1,325 +1,9 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Server versus recipient sync handlers
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
The server handler owns the request's shared validation, mutation and originator result. An optional per-recipient handler exists only when a viewer needs different output, such as filtering private fields or adapting presentation. Do not add an empty recipient handler: it adds work without changing the audience. Source: `../src/handleSyncRequest.ts`, `../src/handleHttpSyncRequest.ts`, `../src/_shared/clientFanout.ts`.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
**Security invariant:** the sender's authority to target a room must be checked before the shared mutation or fanout (`../src/_shared/receiverAuth.ts`). A recipient handler is not an originator auth gate, and one recipient's failure does not roll back the mutation or stop other deliveries. Assume shared server output can reach every permitted recipient; keep secrets out of it unless the server-side audience is constrained. Resolve recipient-specific identity from that recipient's session, not the sender's. The originator acknowledgement reports the server result independently of recipient delivery.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
**Ordering invariant:** recipients are resolved, across instances, before the server handler runs, on both transports (ADR 0063). The framework cannot undo a handler's side effects, so any failure it can find before the mutation must be reported before the mutation. An empty room or a failed recipient lookup therefore never reaches the server handler. Consequence for handler authors: the audience is fixed before your handler runs, and a socket that joins the room during your handler misses that fanout.
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
Given the file-based router (see `/docs/ARCHITECTURE_ROUTING.md`):
|
|
12
|
-
|
|
13
|
-
```
|
|
14
|
-
src/board/_sync/moveCard_server_v1.ts <- REQUIRED: validates + produces serverOutput (runs once)
|
|
15
|
-
src/board/_sync/moveCard_client_v1.ts <- OPTIONAL: per-recipient logic (runs once per socket in the room)
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
Route name: `board/moveCard` at version `v1`. The framework loads both files via `getRuntimeSyncMaps()` and keys them as `board/moveCard/v1_server` and `board/moveCard/v1_client`.
|
|
19
|
-
|
|
20
|
-
A route is valid if **at least one** of the two files exists. If neither exists, `handleSyncRequest` rejects with `sync.notFound`.
|
|
21
|
-
|
|
22
|
-
---
|
|
23
|
-
|
|
24
|
-
## 2. `_server_v{N}.ts` contract — runs ONCE per request
|
|
25
|
-
|
|
26
|
-
```ts
|
|
27
|
-
import type { SessionLayout, Functions } from '@luckystack/core';
|
|
28
|
-
import type { AuthProps } from '@luckystack/login';
|
|
29
|
-
|
|
30
|
-
export const auth: AuthProps = { login: true };
|
|
31
|
-
|
|
32
|
-
export interface SyncParams {
|
|
33
|
-
clientInput: { /* validated against generated Zod schema */ };
|
|
34
|
-
user: SessionLayout | null; // null only when auth.login === false
|
|
35
|
-
functions: Functions; // tryCatch, db, redis, notify, ...
|
|
36
|
-
roomCode: string; // the `receiver` from syncRequest
|
|
37
|
-
stream: (payload?: Record<string, unknown>) => void; // originator only
|
|
38
|
-
broadcastStream: (payload?: Record<string, unknown>) => void; // entire room
|
|
39
|
-
streamTo: (tokens: string | string[], payload?: Record<string, unknown>) => void;
|
|
40
|
-
}
|
|
41
|
-
|
|
42
|
-
export const main = async ({ clientInput, user, functions, roomCode, broadcastStream }: SyncParams) => {
|
|
43
|
-
// 1. Mutate persistent state.
|
|
44
|
-
await functions.db.card.update({ where: { id: clientInput.cardId }, data: { laneId: clientInput.toLane } });
|
|
45
|
-
|
|
46
|
-
// 2. Optionally stream progress / tokens / diffs (broadcastStream goes to everyone in roomCode).
|
|
47
|
-
// 3. Return the success envelope. Everything except `status` becomes `serverOutput` for the recipients.
|
|
48
|
-
return { status: 'success', cardId: clientInput.cardId, movedBy: user!.id };
|
|
49
|
-
};
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
Key rules:
|
|
53
|
-
|
|
54
|
-
- **`auth: AuthProps` is mandatory.** Drives the framework's auth gate (`auth.login` -> require session, `auth.additional` -> run `validateRequest` predicates). See `@luckystack/login` `AuthProps`.
|
|
55
|
-
- **Return must include `status: 'success' | 'error'`.** Anything else collapses to `sync.invalidServerResponse`. The rest of the object is forwarded as `serverOutput` (after stripping `status`).
|
|
56
|
-
- **Streaming primitives are received as params.** See [`./streaming.md`](./streaming.md) for the audience matrix.
|
|
57
|
-
- **Optional `export const validation`** toggles runtime input validation, exactly like the API handler. `'strict'` (default, or omit the export) validates `clientInput` against the generated Zod schema; `'relaxed'` or `{ input: 'skip' }` skips validation entirely — use it for routes whose payload shape can't be modelled in TypeScript (third-party webhook fan-in, dynamic blobs). Honored identically by both the Socket.io and HTTP/SSE transports.
|
|
58
|
-
- **Optional `export const errorFormatter`** overrides the error envelope for this route only (falls back to the global `registerErrorFormatter`, then the framework default). Both transports honor it.
|
|
59
|
-
- **Never `try/catch` manually.** Wrap async ops in `functions.tryCatch` (or the destructured `tryCatch` from `functions`) — the framework already wraps the whole `main()` in its own `tryCatch` for registered error-tracker capture, but per-operation `tryCatch` lets you surface targeted error codes.
|
|
60
|
-
|
|
61
|
-
---
|
|
62
|
-
|
|
63
|
-
## 3. `_client_v{N}.ts` contract — runs ONCE per recipient socket
|
|
64
|
-
|
|
65
|
-
```ts
|
|
66
|
-
import type { Functions } from '@luckystack/core';
|
|
67
|
-
|
|
68
|
-
export interface SyncClientParams {
|
|
69
|
-
clientInput: { /* same shape as _server */ };
|
|
70
|
-
token: string | null; // recipient's session token (NOT the sender's)
|
|
71
|
-
functions: Functions;
|
|
72
|
-
serverOutput: unknown; // exactly what _server returned (minus `status`)
|
|
73
|
-
roomCode: string;
|
|
74
|
-
stream: (payload?: Record<string, unknown>) => void; // per-recipient stream
|
|
75
|
-
}
|
|
76
|
-
|
|
77
|
-
export const main = async ({ clientInput, token, functions, serverOutput, roomCode }: SyncClientParams) => {
|
|
78
|
-
// 1. Optionally fetch this recipient's session — DO NOT receive `user`, only `token`.
|
|
79
|
-
// The framework deliberately does not pre-resolve sessions per recipient because
|
|
80
|
-
// most clients do not need them. Resolving N sessions for a 50-user room is wasteful.
|
|
81
|
-
const recipient = await functions.session.getSession(token);
|
|
82
|
-
if (!recipient) return { status: 'success' }; // anonymous viewer — leave as-is
|
|
83
|
-
|
|
84
|
-
// 2. Filter / brand / translate based on the recipient.
|
|
85
|
-
if (recipient.role === 'guest' && serverOutput.privateNotes) {
|
|
86
|
-
return { status: 'success', visibleNotes: null }; // strip the private field
|
|
87
|
-
}
|
|
88
|
-
|
|
89
|
-
// 3. Return a `clientOutput` envelope. Same status rules as _server.
|
|
90
|
-
return { status: 'success', visibleNotes: serverOutput.privateNotes };
|
|
91
|
-
};
|
|
92
|
-
```
|
|
93
|
-
|
|
94
|
-
Key rules:
|
|
95
|
-
|
|
96
|
-
- **No `user` param.** Receives `token` instead. Call `functions.session.getSession(token)` only when you actually need the session — most `_client` files don't.
|
|
97
|
-
- **Cannot validate or reject the request.** Validation already happened in `_server`. `_client` only customizes per-recipient output. Its error path emits `sync.clientExecutionFailed` / `sync.invalidClientResponse` / `sync.clientRejected` to that specific recipient, but does NOT abort the fanout to the others.
|
|
98
|
-
- **No `broadcastStream` / `streamTo`.** A per-recipient handler cannot fanout to other recipients — that contract is server-scope only.
|
|
99
|
-
|
|
100
|
-
---
|
|
101
|
-
|
|
102
|
-
## 4. When to create `_client_v{N}.ts`
|
|
103
|
-
|
|
104
|
-
| Goal | Need `_client`? | Why |
|
|
105
|
-
|---|---|---|
|
|
106
|
-
| Mutate state, tell the room what changed | No | `serverOutput` reaches every recipient unchanged. |
|
|
107
|
-
| Hide a field from non-owner viewers | Yes | Per-recipient filter. |
|
|
108
|
-
| Brand the payload (translate strings, swap CDN host, inject feature flags) | Yes | Per-recipient customization. |
|
|
109
|
-
| Per-recipient auth (e.g. "guests get a redacted version") | Yes | Use `token` -> `getSession` -> branch. |
|
|
110
|
-
| Per-recipient stream (different chunks per viewer) | Yes | Only `_client` has the per-target `stream(...)` primitive. |
|
|
111
|
-
| Just return `{ status: 'success' }` | **No — delete the file** | Pure overhead: framework already emits success with `clientOutput: {}` when `_client` is absent. |
|
|
112
|
-
|
|
113
|
-
### The "empty client" anti-pattern
|
|
114
|
-
|
|
115
|
-
```ts
|
|
116
|
-
// BAD — adds avoidable per-recipient await + tryCatch + emit overhead for every socket in the room.
|
|
117
|
-
export const main = async () => ({ status: 'success' });
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
If the file would only return `{ status: 'success' }`, leave it out. The framework's no-client branch emits:
|
|
121
|
-
|
|
122
|
-
```ts
|
|
123
|
-
{
|
|
124
|
-
cb,
|
|
125
|
-
fullName: resolvedName,
|
|
126
|
-
serverOutput,
|
|
127
|
-
clientOutput: {},
|
|
128
|
-
message: `${resolvedName} sync success`,
|
|
129
|
-
status: 'success',
|
|
130
|
-
}
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
…directly to each recipient with zero per-recipient handler invocation. That path is materially cheaper, especially when `receiver: 'all'`.
|
|
134
|
-
|
|
135
|
-
Rule of thumb: a `_client` file exists **only** when it makes a decision the server cannot make once.
|
|
136
|
-
|
|
137
|
-
---
|
|
138
|
-
|
|
139
|
-
## 5. Full lifecycle of `handleSyncRequest`
|
|
140
|
-
|
|
141
|
-
```
|
|
142
|
-
incoming msg (Socket.io 'sync' event)
|
|
143
|
-
|
|
|
144
|
-
v
|
|
145
|
-
1. validate msg shape -> sync.invalidRequest
|
|
146
|
-
2. parseTransportRouteName(name) -> routing.invalidServiceRouteName
|
|
147
|
-
3. validate cb -> sync.invalidCallback
|
|
148
|
-
4. validate receiver -> sync.missingReceiver
|
|
149
|
-
5. getSession(token) -> propagate identity to registered error trackers
|
|
150
|
-
6. getRuntimeSyncMaps()
|
|
151
|
-
no _server AND no _client -> sync.notFound
|
|
152
|
-
|
|
|
153
|
-
v
|
|
154
|
-
7. If _server exists:
|
|
155
|
-
AuthProps gate (auth.login) -> auth.required
|
|
156
|
-
validateRequest(auth.additional) -> auth.forbidden
|
|
157
|
-
8. dispatchHook('preSyncAuthorize') -> stop signal becomes error envelope
|
|
158
|
-
9. applySyncRateLimits() -> sync.rateLimitExceeded
|
|
159
|
-
10. Resolve recipients (cross-instance, both transports, BEFORE anything is persisted — see ./room-fanout.md §1):
|
|
160
|
-
receiver === 'all' -> every socket on every instance (fetchSockets())
|
|
161
|
-
otherwise -> formatRoomName(receiver, 'broadcast') -> io.in(room).fetchSockets()
|
|
162
|
-
fetchSockets throws -> sync.serverExecutionFailed (adapter request timeout; nothing persisted)
|
|
163
|
-
empty list -> sync.noReceiversFound (nothing persisted)
|
|
164
|
-
11. If _server exists:
|
|
165
|
-
validateInputByType(clientInput) -> sync.invalidInputType
|
|
166
|
-
tryCatch(serverMain(...)) -> sync.serverExecutionFailed
|
|
167
|
-
status !== 'success' | 'error' -> sync.invalidServerResponse
|
|
168
|
-
status === 'error' -> server's errorCode (normalized)
|
|
169
|
-
status === 'success' -> serverOutput = result (minus status)
|
|
170
|
-
12. dispatchHook('preSyncFanout') -> stop signal becomes error envelope
|
|
171
|
-
13. Per-recipient loop (with periodic event-loop yield):
|
|
172
|
-
if (ignoreSelf && token === recipientToken) continue
|
|
173
|
-
recipientCount++
|
|
174
|
-
if (_client exists):
|
|
175
|
-
tryCatch(clientHandler(...)) -> sync.clientExecutionFailed to that recipient
|
|
176
|
-
status='error' -> normalized error to that recipient
|
|
177
|
-
status='success' -> emit { serverOutput, clientOutput, ... }
|
|
178
|
-
else:
|
|
179
|
-
emit { serverOutput, clientOutput: {}, ... }
|
|
180
|
-
14. dispatchHook('postSyncFanout', { recipientCount })
|
|
181
|
-
15. ack originator: emit(buildSyncResponseEventName(responseIndex), { status, result: serverOutput })
|
|
182
|
-
```
|
|
183
|
-
|
|
184
|
-
Per-recipient failures inside step 13 do **not** abort the fanout — every other recipient still receives the merged payload. The originator's ack always reflects the `_server` outcome, never the per-recipient outcomes.
|
|
185
|
-
|
|
186
|
-
---
|
|
187
|
-
|
|
188
|
-
## 6. Failure-mode matrix
|
|
189
|
-
|
|
190
|
-
| Stage | Trigger | Error code | Visibility |
|
|
191
|
-
|---|---|---|---|
|
|
192
|
-
| Msg shape | Non-object `msg` | `sync.invalidRequest` | originator ack |
|
|
193
|
-
| Routing | Bad `name` shape | `routing.invalidServiceRouteName` | originator ack |
|
|
194
|
-
| Routing | No `cb` | `sync.invalidCallback` | originator ack |
|
|
195
|
-
| Routing | No receiver | `sync.missingReceiver` | originator ack |
|
|
196
|
-
| Route lookup | No `_server` AND no `_client` | `sync.notFound` | originator ack |
|
|
197
|
-
| Auth | `auth.login` + no session | `auth.required` | originator ack |
|
|
198
|
-
| Auth | `validateRequest` reject | `auth.forbidden` (or specific predicate code) | originator ack |
|
|
199
|
-
| Auth | `preSyncAuthorize` stop | hook's `errorCode` | originator ack |
|
|
200
|
-
| Rate limit | Per-route or per-IP bucket | `sync.rateLimitExceeded` | originator ack |
|
|
201
|
-
| Validation | Zod fail | `sync.invalidInputType` | originator ack |
|
|
202
|
-
| Server execution | Thrown | `sync.serverExecutionFailed` | originator ack |
|
|
203
|
-
| Server return | `status: 'error'` from `_server` | route-supplied `errorCode` | originator ack |
|
|
204
|
-
| Server return | Anything other than `success`/`error` | `sync.invalidServerResponse` | originator ack |
|
|
205
|
-
| Recipients (before `_server`) | No sockets in room | `sync.noReceiversFound` | originator ack |
|
|
206
|
-
| Recipients (before `_server`) | `fetchSockets()` threw (adapter request timeout) | `sync.serverExecutionFailed` | originator ack |
|
|
207
|
-
| Fanout | `preSyncFanout` stop | hook's `errorCode` | originator ack |
|
|
208
|
-
| Per-recipient | `_client` thrown | `sync.clientExecutionFailed` | that recipient only |
|
|
209
|
-
| Per-recipient | `_client` returned `status: 'error'` | route code or `sync.clientRejected` | that recipient only |
|
|
210
|
-
| Per-recipient | `_client` returned non-success | `sync.invalidClientResponse` | that recipient only |
|
|
211
|
-
|
|
212
|
-
Full catalog including HTTP-status mapping in [`./error-states.md`](./error-states.md).
|
|
213
|
-
|
|
214
|
-
---
|
|
215
|
-
|
|
216
|
-
## 7. Example A — server-only sync (no `_client`)
|
|
217
|
-
|
|
218
|
-
```ts
|
|
219
|
-
// src/notifications/_sync/markRead_server_v1.ts
|
|
220
|
-
export const auth = { login: true };
|
|
221
|
-
|
|
222
|
-
export interface SyncParams {
|
|
223
|
-
clientInput: { notificationId: string };
|
|
224
|
-
user: SessionLayout;
|
|
225
|
-
functions: Functions;
|
|
226
|
-
roomCode: string;
|
|
227
|
-
}
|
|
228
|
-
|
|
229
|
-
export const main = async ({ clientInput, user, functions }: SyncParams) => {
|
|
230
|
-
await functions.db.notification.update({
|
|
231
|
-
where: { id: clientInput.notificationId, userId: user.id },
|
|
232
|
-
data: { readAt: new Date() },
|
|
233
|
-
});
|
|
234
|
-
return { status: 'success', notificationId: clientInput.notificationId };
|
|
235
|
-
};
|
|
236
|
-
```
|
|
237
|
-
|
|
238
|
-
No `_client` file. Every recipient in `roomCode` receives `{ serverOutput: { notificationId }, clientOutput: {} }`.
|
|
239
|
-
|
|
240
|
-
---
|
|
241
|
-
|
|
242
|
-
## 8. Example B — server + client per-recipient filter
|
|
243
|
-
|
|
244
|
-
```ts
|
|
245
|
-
// src/board/_sync/showCard_server_v1.ts
|
|
246
|
-
export const auth = { login: true };
|
|
247
|
-
|
|
248
|
-
export const main = async ({ clientInput, functions }: SyncParams) => {
|
|
249
|
-
const card = await functions.db.card.findUnique({ where: { id: clientInput.cardId } });
|
|
250
|
-
if (!card) return { status: 'error', errorCode: 'board.cardNotFound' };
|
|
251
|
-
return { status: 'success', card };
|
|
252
|
-
};
|
|
253
|
-
```
|
|
254
|
-
|
|
255
|
-
```ts
|
|
256
|
-
// src/board/_sync/showCard_client_v1.ts — strip privateNotes from non-owners
|
|
257
|
-
export const main = async ({ token, serverOutput, functions }: SyncClientParams) => {
|
|
258
|
-
const recipient = await functions.session.getSession(token);
|
|
259
|
-
if (!recipient) return { status: 'success', card: { ...serverOutput.card, privateNotes: null } };
|
|
260
|
-
|
|
261
|
-
if (recipient.id !== serverOutput.card.ownerId) {
|
|
262
|
-
return { status: 'success', card: { ...serverOutput.card, privateNotes: null } };
|
|
263
|
-
}
|
|
264
|
-
return { status: 'success', card: serverOutput.card };
|
|
265
|
-
};
|
|
266
|
-
```
|
|
267
|
-
|
|
268
|
-
Recipients see:
|
|
269
|
-
|
|
270
|
-
- Owner: `{ serverOutput, clientOutput: { card: { privateNotes: '...' } } }`
|
|
271
|
-
- Anyone else: `{ serverOutput, clientOutput: { card: { privateNotes: null } } }`
|
|
272
|
-
|
|
273
|
-
Note: `serverOutput` is the same payload for every recipient — only `clientOutput` differs. The client-side handler chooses which to render.
|
|
274
|
-
|
|
275
|
-
---
|
|
276
|
-
|
|
277
|
-
## 9. Example C — server + client custom `clientOutput` (translation)
|
|
278
|
-
|
|
279
|
-
```ts
|
|
280
|
-
// src/chat/_sync/announce_server_v1.ts
|
|
281
|
-
export const main = async ({ clientInput }: SyncParams) => ({
|
|
282
|
-
status: 'success',
|
|
283
|
-
i18nKey: clientInput.i18nKey,
|
|
284
|
-
params: clientInput.params,
|
|
285
|
-
});
|
|
286
|
-
```
|
|
287
|
-
|
|
288
|
-
```ts
|
|
289
|
-
// src/chat/_sync/announce_client_v1.ts — pre-translate per recipient locale
|
|
290
|
-
export const main = async ({ token, serverOutput, functions }: SyncClientParams) => {
|
|
291
|
-
const recipient = await functions.session.getSession(token);
|
|
292
|
-
const locale = recipient?.language ?? 'en';
|
|
293
|
-
const text = functions.translator.translate({ key: serverOutput.i18nKey, params: serverOutput.params, locale });
|
|
294
|
-
return { status: 'success', text };
|
|
295
|
-
};
|
|
296
|
-
```
|
|
297
|
-
|
|
298
|
-
This is the canonical reason to add `_client`: the translation cost is per-locale, the source key is shared, and centralizing it in `_client` keeps the room mutation single-source.
|
|
299
|
-
|
|
300
|
-
---
|
|
301
|
-
|
|
302
|
-
## 10. Related hooks
|
|
303
|
-
|
|
304
|
-
| Hook | Stage | Stop semantics |
|
|
305
|
-
|---|---|---|
|
|
306
|
-
| `preSyncAuthorize` | After `AuthProps` gate, before rate-limit | Stop -> originator error envelope with hook's `errorCode` |
|
|
307
|
-
| `preSyncFanout` | After `_server` runs, before any recipient receives | Stop -> originator error envelope; nobody gets the payload |
|
|
308
|
-
| `postSyncFanout` | After all recipients emitted | Observation-only; receives `recipientCount` |
|
|
309
|
-
| `rateLimitExceeded` | When per-route or per-IP bucket rejects | Observation-only |
|
|
310
|
-
| `preSyncStream` / `postSyncStream` | Per stream chunk emit | Observation-only |
|
|
311
|
-
|
|
312
|
-
Hook payload shapes live in `@luckystack/core` (`HookPayloads`).
|
|
313
|
-
|
|
314
|
-
---
|
|
315
|
-
|
|
316
|
-
## 11. Related
|
|
317
|
-
|
|
318
|
-
- Originator API: [`./sync-request.md`](./sync-request.md)
|
|
319
|
-
- Streaming primitives: [`./streaming.md`](./streaming.md)
|
|
320
|
-
- Room mechanics + fanout hooks: [`./room-fanout.md`](./room-fanout.md)
|
|
321
|
-
- Skip-self semantics: [`./ignore-self.md`](./ignore-self.md)
|
|
322
|
-
- Recipient subscription: [`./callback-registration.md`](./callback-registration.md)
|
|
323
|
-
- Version coexistence: [`./version-policy.md`](./version-policy.md)
|
|
324
|
-
- Full architecture: [`/docs/ARCHITECTURE_SYNC.md`](../../../docs/ARCHITECTURE_SYNC.md)
|
|
325
|
-
- Routing conventions: [`/docs/ARCHITECTURE_ROUTING.md`](../../../docs/ARCHITECTURE_ROUTING.md)
|
|
9
|
+
See `./room-fanout.md` for room trust and `./streaming.md` for stream audiences. Route declarations and generated types own signatures.
|