@kumiai/hub-server 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +84 -0
- package/lib/handlers.d.ts +55 -2
- package/lib/handlers.js +171 -44
- package/lib/hub.d.ts +10 -3
- package/lib/hub.js +1 -1
- package/lib/index.d.ts +1 -1
- package/lib/memoryStore.d.ts +26 -4
- package/lib/memoryStore.js +259 -105
- package/lib/registry.d.ts +31 -2
- package/lib/registry.js +41 -10
- package/package.json +14 -15
package/README.md
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# @kumiai/hub-server
|
|
2
|
+
|
|
3
|
+
The hub server: `@kumiai/hub-protocol`'s procedure handlers wired to an `@enkaku` server, a live
|
|
4
|
+
client registry for push fan-out, and an in-memory `HubStore` for development and tests.
|
|
5
|
+
|
|
6
|
+
## Exports
|
|
7
|
+
|
|
8
|
+
- `createHub({ transport, store, identity, ... })` — the whole server: handlers, registry, access
|
|
9
|
+
rules, rate limits, and an optional scheduled purge. Returns `{ registry, server }`.
|
|
10
|
+
- `createHandlers({ registry, store, ... })` — the handlers alone, for a host assembling its own
|
|
11
|
+
server.
|
|
12
|
+
- `createMemoryStore({ maxDepth?, retention? })` — an in-memory `HubStore`. For testing and
|
|
13
|
+
development; it is also what runs the `@kumiai/hub-conformance` suites in this repo.
|
|
14
|
+
- `HubClientRegistry` — the live-connection table.
|
|
15
|
+
- `createRateLimiter`, `DEFAULT_RATE_LIMITS`, `DEFAULT_KEYPACKAGE_FETCH_LIMITS`,
|
|
16
|
+
`DEFAULT_HUB_ACCESS_RULES`.
|
|
17
|
+
|
|
18
|
+
```ts
|
|
19
|
+
import { createHub, createMemoryStore } from '@kumiai/hub-server'
|
|
20
|
+
|
|
21
|
+
const { server } = createHub({ transport, identity, store: createMemoryStore() })
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
## The hub is blind, and identity comes only from the signature
|
|
25
|
+
|
|
26
|
+
Topic IDs are opaque and payloads are ciphertext the hub never opens. Every handler takes the caller
|
|
27
|
+
DID from the **verified issuer** of the signed message, never from a wire field — so
|
|
28
|
+
`hub/v1/topic/fetch` cannot be pointed at someone else's subscription, and a publish cannot claim
|
|
29
|
+
another sender. The `identity` param is required for that reason.
|
|
30
|
+
|
|
31
|
+
Authorization is two layers, and only the second one knows about topics: `accessRules` gate the
|
|
32
|
+
procedures (the default lets any authenticated DID call them), and the optional
|
|
33
|
+
`authorize(request)` hook decides publish and subscribe per topic. It takes one discriminated
|
|
34
|
+
`AuthorizeRequest` — narrow on `request.action` — and returns `AuthorizeDecision`, either a boolean
|
|
35
|
+
or `{ allow, reason?, code?, retryAfterMs? }`. `AuthorizeRequest` names six actions, but only
|
|
36
|
+
`publish` and `subscribe` are dispatched today. The default allows any authenticated DID.
|
|
37
|
+
|
|
38
|
+
## Durable subscription, ephemeral connection
|
|
39
|
+
|
|
40
|
+
Subscription state lives in the store. `HubClientRegistry` holds only currently-connected clients and
|
|
41
|
+
their live `hub/v1/receive` writers, so it routes push fan-out and nothing else — a restart loses no
|
|
42
|
+
subscription.
|
|
43
|
+
|
|
44
|
+
Binding a receive channel **evicts** whatever held the lane for that DID. A reconnect happens because
|
|
45
|
+
the old connection broke and the server learns that last, so the stale writer must give way to the
|
|
46
|
+
live one, not the other way round. The evicted channel is resolved rather than thrown: being replaced
|
|
47
|
+
is not its error, and the client that replaced it is the same client.
|
|
48
|
+
|
|
49
|
+
`hub/v1/receive` is always added to the server's `longLivedProcedures`, so open mailbox channels are
|
|
50
|
+
exempt from `controllerTimeoutMs` and from the `maxConcurrentHandlers` cap. A host passing its own
|
|
51
|
+
`limits` does not need to remember this.
|
|
52
|
+
|
|
53
|
+
## What live fan-out does and does not push
|
|
54
|
+
|
|
55
|
+
A **deduped** publish fans out to nobody. It appended nothing — the frame was already accepted and
|
|
56
|
+
already delivered to whoever was subscribed then, and its sequenceID may since have been acked and
|
|
57
|
+
its delivery row removed — so re-running the loop would push a frame every current subscriber has
|
|
58
|
+
already applied, named by a dead sequenceID. Fan-out is for a genuine append only.
|
|
59
|
+
|
|
60
|
+
The sender is excluded from its own fan-out. `hub/v1/topic/fetch` makes no such exclusion — a topic's
|
|
61
|
+
log holds every log-class frame including the caller's own — so a reader that must not see its own
|
|
62
|
+
messages twice filters them itself, as `@kumiai/rpc`'s drain does.
|
|
63
|
+
|
|
64
|
+
A pushed frame carries `logPosition` **only** when it is log-class, and the key is spread in rather
|
|
65
|
+
than assigned, because `logPosition: undefined` becomes a present key with a falsy value once it is
|
|
66
|
+
off the wire — and the entire point of the field is that a reader can tell "no place in any log" from
|
|
67
|
+
a place.
|
|
68
|
+
|
|
69
|
+
## Retention: the store refuses, the hub schedules
|
|
70
|
+
|
|
71
|
+
Two different knobs, easily confused. `createHub`'s `purge.olderThan` (default 7 days) is the **age
|
|
72
|
+
bound** the scheduled sweep applies to a topic no subscriber asked to keep longer; the sweep runs on
|
|
73
|
+
`purge.interval` (default 1 hour) and stops with the server. The **ceiling** on what a subscriber may
|
|
74
|
+
request belongs to the store, because the store is what refuses the subscribe —
|
|
75
|
+
`createMemoryStore`'s is 30 days, finite by design, since an unbounded ceiling lets any subscriber
|
|
76
|
+
pin a topic's frames forever with `subscribe({ retention: 2 ** 31 })`. A subscribe above the ceiling
|
|
77
|
+
is refused with `RetentionExceededError`, never clamped.
|
|
78
|
+
|
|
79
|
+
`createMemoryStore`'s `maxDepth` (default 1000) evicts the oldest **log** frames beyond that count.
|
|
80
|
+
Mailbox frames on the same topic are not counted, so a member cannot evict the log with a mailbox
|
|
81
|
+
flood.
|
|
82
|
+
|
|
83
|
+
Publish rate limits are per DID (20/s, burst 50) and per topic (100/s, burst 200), merged over
|
|
84
|
+
`DEFAULT_RATE_LIMITS`; `hub/v1/keypackage/fetch` has its own request quota per requester DID.
|
package/lib/handlers.d.ts
CHANGED
|
@@ -2,8 +2,61 @@ import { type ProcedureHandlers } from '@enkaku/server';
|
|
|
2
2
|
import type { HubProtocol, HubStore } from '@kumiai/hub-protocol';
|
|
3
3
|
import { type RateLimitConfig } from './rateLimit.js';
|
|
4
4
|
import type { HubClientRegistry } from './registry.js';
|
|
5
|
-
|
|
6
|
-
|
|
5
|
+
/**
|
|
6
|
+
* A single request to authorize. All six variants ship even though only `publish` and
|
|
7
|
+
* `subscribe` are currently enforced: the union is the exhaustive-switch surface a host's own
|
|
8
|
+
* `switch (req.action)` closes over, so adding a seventh variant later is the compatibility break
|
|
9
|
+
* this type exists to avoid.
|
|
10
|
+
*
|
|
11
|
+
* A host's `switch` need not handle every action today — an unrecognized action should default to
|
|
12
|
+
* allow, so a hook written before a new variant shipped doesn't silently start refusing a
|
|
13
|
+
* procedure that was previously ungated.
|
|
14
|
+
*/
|
|
15
|
+
export type AuthorizeRequest = {
|
|
16
|
+
action: 'publish';
|
|
17
|
+
did: string;
|
|
18
|
+
topicID: string;
|
|
19
|
+
retain: 'log' | 'mailbox';
|
|
20
|
+
payloadSize: number;
|
|
21
|
+
} | {
|
|
22
|
+
action: 'subscribe';
|
|
23
|
+
did: string;
|
|
24
|
+
topicID: string;
|
|
25
|
+
retention?: number;
|
|
26
|
+
} | {
|
|
27
|
+
action: 'unsubscribe';
|
|
28
|
+
did: string;
|
|
29
|
+
topicID: string;
|
|
30
|
+
} | {
|
|
31
|
+
action: 'topic/fetch';
|
|
32
|
+
did: string;
|
|
33
|
+
topicID: string;
|
|
34
|
+
} | {
|
|
35
|
+
action: 'keypackage/upload';
|
|
36
|
+
did: string;
|
|
37
|
+
count: number;
|
|
38
|
+
} | {
|
|
39
|
+
action: 'keypackage/fetch';
|
|
40
|
+
did: string;
|
|
41
|
+
targetDID: string;
|
|
42
|
+
count: number;
|
|
43
|
+
};
|
|
44
|
+
/**
|
|
45
|
+
* A plain `boolean` is shorthand for `{ allow: boolean }`, with no reason, code, or retry hint.
|
|
46
|
+
*
|
|
47
|
+
* **`reason` is the only field a caller sees today.** `code` and `retryAfterMs` are reserved for
|
|
48
|
+
* enforcement that hasn't been built yet — a hook may return them, but both are silently dropped
|
|
49
|
+
* before the refusal reaches the client. They ship now because widening this union later is the
|
|
50
|
+
* break the surface exists to avoid; until enforcement lands, don't depend on either reaching a
|
|
51
|
+
* caller.
|
|
52
|
+
*/
|
|
53
|
+
export type AuthorizeDecision = boolean | {
|
|
54
|
+
allow: boolean;
|
|
55
|
+
reason?: string;
|
|
56
|
+
code?: string;
|
|
57
|
+
retryAfterMs?: number;
|
|
58
|
+
};
|
|
59
|
+
export type AuthorizeHook = (req: AuthorizeRequest) => AuthorizeDecision | Promise<AuthorizeDecision>;
|
|
7
60
|
export type HubRateLimits = {
|
|
8
61
|
perDID: RateLimitConfig;
|
|
9
62
|
perTopic: RateLimitConfig;
|
package/lib/handlers.js
CHANGED
|
@@ -1,6 +1,12 @@
|
|
|
1
1
|
import { HandlerError } from '@enkaku/server';
|
|
2
|
+
import { hubErrorCodeOf } from '@kumiai/hub-protocol';
|
|
2
3
|
import { fromB64, toB64 } from '@sozai/codec';
|
|
3
4
|
import { createRateLimiter } from './rateLimit.js';
|
|
5
|
+
function normalizeAuthorizeDecision(decision) {
|
|
6
|
+
return typeof decision === 'boolean' ? {
|
|
7
|
+
allow: decision
|
|
8
|
+
} : decision;
|
|
9
|
+
}
|
|
4
10
|
export const DEFAULT_RATE_LIMITS = {
|
|
5
11
|
perDID: {
|
|
6
12
|
rate: 20,
|
|
@@ -23,6 +29,20 @@ function getClientDID(ctx) {
|
|
|
23
29
|
}
|
|
24
30
|
return iss;
|
|
25
31
|
}
|
|
32
|
+
/**
|
|
33
|
+
* Re-raise a named store error with its wire code, so the caller can tell a lost compare-and-set
|
|
34
|
+
* from an unreachable hub. Anything else is not ours and passes through untouched.
|
|
35
|
+
*/ function rethrowAsHandlerError(error) {
|
|
36
|
+
const code = hubErrorCodeOf(error);
|
|
37
|
+
if (code != null) {
|
|
38
|
+
throw new HandlerError({
|
|
39
|
+
code,
|
|
40
|
+
message: error instanceof Error ? error.message : code,
|
|
41
|
+
cause: error
|
|
42
|
+
});
|
|
43
|
+
}
|
|
44
|
+
throw error;
|
|
45
|
+
}
|
|
26
46
|
export function createHandlers(params) {
|
|
27
47
|
const { store, registry } = params;
|
|
28
48
|
const authorize = params.authorize ?? (()=>true);
|
|
@@ -66,13 +86,32 @@ export function createHandlers(params) {
|
|
|
66
86
|
window.count++;
|
|
67
87
|
}
|
|
68
88
|
return {
|
|
69
|
-
'hub/publish': async (ctx)=>{
|
|
89
|
+
'hub/v1/publish': async (ctx)=>{
|
|
70
90
|
const { topicID, payload } = ctx.param;
|
|
71
91
|
const senderDID = getClientDID(ctx);
|
|
72
|
-
|
|
92
|
+
// Decoded before authorize (not after) so the hook can see the real size. Two consequences
|
|
93
|
+
// are deliberate: an unauthorized caller sending malformed base64 gets a decode error instead
|
|
94
|
+
// of EK02, and decoding runs before both rate limiters, so oversized payloads are decoded
|
|
95
|
+
// before being throttled (bounded by the transport's own message limit). Deriving the size
|
|
96
|
+
// arithmetically from the base64 length instead was rejected — it drifts from
|
|
97
|
+
// `@sozai/codec`'s actual decoding and can hand the hook a wrong number.
|
|
98
|
+
//
|
|
99
|
+
// `retain` defaults to 'mailbox' here, matching the store's own default (`memoryStore.ts`) —
|
|
100
|
+
// the hook must never see an absent retain, since `AuthorizeRequest`'s `publish` variant
|
|
101
|
+
// declares it required.
|
|
102
|
+
const payloadBytes = fromB64(payload);
|
|
103
|
+
const retain = ctx.param.retain ?? 'mailbox';
|
|
104
|
+
const decision = normalizeAuthorizeDecision(await authorize({
|
|
105
|
+
action: 'publish',
|
|
106
|
+
did: senderDID,
|
|
107
|
+
topicID,
|
|
108
|
+
retain,
|
|
109
|
+
payloadSize: payloadBytes.length
|
|
110
|
+
}));
|
|
111
|
+
if (!decision.allow) {
|
|
73
112
|
throw new HandlerError({
|
|
74
113
|
code: 'EK02',
|
|
75
|
-
message: 'Not authorized to publish to topic'
|
|
114
|
+
message: decision.reason ?? 'Not authorized to publish to topic'
|
|
76
115
|
});
|
|
77
116
|
}
|
|
78
117
|
if (!didLimiter.tryConsume(senderDID)) {
|
|
@@ -87,45 +126,111 @@ export function createHandlers(params) {
|
|
|
87
126
|
message: 'Publish rate limit exceeded for topic'
|
|
88
127
|
});
|
|
89
128
|
}
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
129
|
+
let sequenceID;
|
|
130
|
+
let deduped;
|
|
131
|
+
try {
|
|
132
|
+
const result = await store.publish({
|
|
133
|
+
senderDID,
|
|
134
|
+
topicID,
|
|
135
|
+
payload: payloadBytes,
|
|
136
|
+
retain: ctx.param.retain,
|
|
137
|
+
// Absent and null are different requests: null is the empty-topic sentinel, absent is
|
|
138
|
+
// an unconditional publish. Spreading only when present keeps them apart.
|
|
139
|
+
...'expectedHead' in ctx.param ? {
|
|
140
|
+
expectedHead: ctx.param.expectedHead
|
|
141
|
+
} : {},
|
|
142
|
+
publishID: ctx.param.publishID
|
|
143
|
+
});
|
|
144
|
+
sequenceID = result.sequenceID;
|
|
145
|
+
deduped = result.deduped;
|
|
146
|
+
} catch (error) {
|
|
147
|
+
rethrowAsHandlerError(error);
|
|
148
|
+
}
|
|
149
|
+
// A deduped publish appended nothing: already fanned out to whoever was subscribed then, and
|
|
150
|
+
// its sequenceID may since be acked and its delivery row gone. Re-running would push a frame
|
|
151
|
+
// every current subscriber already has, named by a dead sequenceID.
|
|
152
|
+
if (!deduped) {
|
|
153
|
+
// Only a log-class publish carries a logPosition. This sequenceID IS the log position (the
|
|
154
|
+
// store just minted it for this accepted append); a mailbox publish has none, since its
|
|
155
|
+
// recipient's own delivery-queue position isn't usable as a durable log cursor.
|
|
156
|
+
const logPosition = ctx.param.retain === 'log' ? {
|
|
157
|
+
logPosition: sequenceID
|
|
158
|
+
} : {};
|
|
159
|
+
// Live-deliver to currently-connected subscribers (minus the sender).
|
|
160
|
+
const subscribers = await store.getSubscribers(topicID);
|
|
161
|
+
for (const recipientDID of subscribers){
|
|
162
|
+
if (recipientDID === senderDID) continue;
|
|
163
|
+
const client = registry.getClient(recipientDID);
|
|
164
|
+
if (client?.sendMessage != null) {
|
|
165
|
+
client.sendMessage({
|
|
166
|
+
sequenceID,
|
|
167
|
+
senderDID,
|
|
168
|
+
topicID,
|
|
169
|
+
payload: payloadBytes,
|
|
170
|
+
...logPosition
|
|
171
|
+
});
|
|
172
|
+
}
|
|
108
173
|
}
|
|
109
174
|
}
|
|
110
175
|
return {
|
|
111
176
|
sequenceID
|
|
112
177
|
};
|
|
113
178
|
},
|
|
114
|
-
'hub/subscribe': async (ctx)=>{
|
|
115
|
-
const { topicID } = ctx.param;
|
|
179
|
+
'hub/v1/subscribe': async (ctx)=>{
|
|
180
|
+
const { topicID, retention } = ctx.param;
|
|
116
181
|
const clientDID = getClientDID(ctx);
|
|
117
|
-
|
|
182
|
+
const decision = normalizeAuthorizeDecision(await authorize({
|
|
183
|
+
action: 'subscribe',
|
|
184
|
+
did: clientDID,
|
|
185
|
+
topicID,
|
|
186
|
+
retention
|
|
187
|
+
}));
|
|
188
|
+
if (!decision.allow) {
|
|
118
189
|
throw new HandlerError({
|
|
119
190
|
code: 'EK02',
|
|
120
|
-
message: 'Not authorized to subscribe to topic'
|
|
191
|
+
message: decision.reason ?? 'Not authorized to subscribe to topic'
|
|
192
|
+
});
|
|
193
|
+
}
|
|
194
|
+
try {
|
|
195
|
+
await store.subscribe({
|
|
196
|
+
subscriberDID: clientDID,
|
|
197
|
+
topicID,
|
|
198
|
+
retention
|
|
121
199
|
});
|
|
200
|
+
} catch (error) {
|
|
201
|
+
rethrowAsHandlerError(error);
|
|
122
202
|
}
|
|
123
|
-
await store.subscribe(clientDID, topicID);
|
|
124
203
|
return {
|
|
125
204
|
subscribed: true
|
|
126
205
|
};
|
|
127
206
|
},
|
|
128
|
-
'hub/
|
|
207
|
+
'hub/v1/topic/fetch': async (ctx)=>{
|
|
208
|
+
const { topicID, after, limit } = ctx.param;
|
|
209
|
+
// subscriberDID is the authenticated caller (verified issuer), never a wire field — or any
|
|
210
|
+
// member could read another's topic log by naming them.
|
|
211
|
+
const subscriberDID = getClientDID(ctx);
|
|
212
|
+
try {
|
|
213
|
+
const result = await store.fetchTopic({
|
|
214
|
+
subscriberDID,
|
|
215
|
+
topicID,
|
|
216
|
+
after,
|
|
217
|
+
limit
|
|
218
|
+
});
|
|
219
|
+
return {
|
|
220
|
+
messages: result.messages.map((message)=>({
|
|
221
|
+
sequenceID: message.sequenceID,
|
|
222
|
+
senderDID: message.senderDID,
|
|
223
|
+
topicID: message.topicID,
|
|
224
|
+
payload: toB64(message.payload)
|
|
225
|
+
})),
|
|
226
|
+
head: result.head,
|
|
227
|
+
oldest: result.oldest
|
|
228
|
+
};
|
|
229
|
+
} catch (error) {
|
|
230
|
+
rethrowAsHandlerError(error);
|
|
231
|
+
}
|
|
232
|
+
},
|
|
233
|
+
'hub/v1/unsubscribe': async (ctx)=>{
|
|
129
234
|
const { topicID } = ctx.param;
|
|
130
235
|
const clientDID = getClientDID(ctx);
|
|
131
236
|
await store.unsubscribe(clientDID, topicID);
|
|
@@ -133,27 +238,40 @@ export function createHandlers(params) {
|
|
|
133
238
|
unsubscribed: true
|
|
134
239
|
};
|
|
135
240
|
},
|
|
136
|
-
'hub/receive': async (ctx)=>{
|
|
241
|
+
'hub/v1/receive': async (ctx)=>{
|
|
137
242
|
const clientDID = getClientDID(ctx);
|
|
138
243
|
const { after } = ctx.param ?? {};
|
|
139
244
|
registry.register(clientDID);
|
|
140
|
-
if (registry.isWriterBound(clientDID)) {
|
|
141
|
-
throw new HandlerError({
|
|
142
|
-
code: 'EK01',
|
|
143
|
-
message: `receive writer already bound for DID ${clientDID}`
|
|
144
|
-
});
|
|
145
|
-
}
|
|
146
245
|
const writer = ctx.writable.getWriter();
|
|
147
246
|
const reader = ctx.readable.getReader();
|
|
247
|
+
// Ends this channel when a newer one for the same DID takes the lane. Resolved, not thrown —
|
|
248
|
+
// replacement isn't this channel's error, and it's the same client that replaced it.
|
|
249
|
+
let endEvicted;
|
|
250
|
+
const evictedHere = new Promise((resolve)=>{
|
|
251
|
+
endEvicted = resolve;
|
|
252
|
+
});
|
|
253
|
+
let receiveToken;
|
|
148
254
|
try {
|
|
149
|
-
|
|
255
|
+
// Binding evicts whatever held the lane: a reconnect means the old connection broke and the
|
|
256
|
+
// server learns last, so the stale writer must give way to the live one. See
|
|
257
|
+
// `HubClientRegistry.bindReceiveWriter`.
|
|
258
|
+
const { token, evicted } = registry.bindReceiveWriter(clientDID, (message)=>{
|
|
150
259
|
writer.write({
|
|
151
260
|
sequenceID: message.sequenceID,
|
|
152
261
|
senderDID: message.senderDID,
|
|
153
262
|
topicID: message.topicID,
|
|
154
|
-
payload: toB64(message.payload)
|
|
263
|
+
payload: toB64(message.payload),
|
|
264
|
+
// Spread, not plain assignment — `logPosition: undefined` becomes a present key
|
|
265
|
+
// once serialized, defeating the field's whole point (telling absent from present).
|
|
266
|
+
...message.logPosition != null ? {
|
|
267
|
+
logPosition: message.logPosition
|
|
268
|
+
} : {}
|
|
155
269
|
}).catch(()=>{});
|
|
156
|
-
});
|
|
270
|
+
}, ()=>endEvicted?.());
|
|
271
|
+
receiveToken = token;
|
|
272
|
+
// After the bind, so the lane is never unheld between the two — a frame arriving in that
|
|
273
|
+
// gap would have nowhere to go, and mailbox frames drop when nobody's there.
|
|
274
|
+
evicted?.();
|
|
157
275
|
let cursor = after;
|
|
158
276
|
while(true){
|
|
159
277
|
const result = await store.fetch({
|
|
@@ -166,19 +284,23 @@ export function createHandlers(params) {
|
|
|
166
284
|
sequenceID: msg.sequenceID,
|
|
167
285
|
senderDID: msg.senderDID,
|
|
168
286
|
topicID: msg.topicID,
|
|
169
|
-
payload: toB64(msg.payload)
|
|
287
|
+
payload: toB64(msg.payload),
|
|
288
|
+
...msg.logPosition != null ? {
|
|
289
|
+
logPosition: msg.logPosition
|
|
290
|
+
} : {}
|
|
170
291
|
});
|
|
171
292
|
}
|
|
172
293
|
cursor = result.cursor;
|
|
173
294
|
if (!result.hasMore) break;
|
|
174
295
|
}
|
|
175
296
|
} catch (error) {
|
|
176
|
-
registry.
|
|
297
|
+
if (receiveToken != null) registry.releaseReceiveWriter(clientDID, receiveToken);
|
|
177
298
|
registry.unregisterIfIdle(clientDID);
|
|
178
299
|
reader.cancel().catch(()=>{});
|
|
179
300
|
writer.abort(error).catch(()=>{});
|
|
180
301
|
throw error;
|
|
181
302
|
}
|
|
303
|
+
const token = receiveToken;
|
|
182
304
|
void (async ()=>{
|
|
183
305
|
try {
|
|
184
306
|
while(true){
|
|
@@ -196,18 +318,23 @@ export function createHandlers(params) {
|
|
|
196
318
|
}
|
|
197
319
|
})();
|
|
198
320
|
return new Promise((resolve)=>{
|
|
199
|
-
|
|
200
|
-
|
|
321
|
+
const finish = ()=>{
|
|
322
|
+
// Token-scoped, so an already-evicted channel can't unbind the one that replaced it.
|
|
323
|
+
if (token != null) registry.releaseReceiveWriter(clientDID, token);
|
|
201
324
|
registry.unregisterIfIdle(clientDID);
|
|
202
325
|
reader.cancel().catch(()=>{});
|
|
203
326
|
writer.close().catch(()=>{});
|
|
204
327
|
resolve(undefined);
|
|
205
|
-
}
|
|
328
|
+
};
|
|
329
|
+
// Evicted by a newer channel: end this one; `finish` releases only its own token, which is
|
|
330
|
+
// no longer current.
|
|
331
|
+
void evictedHere.then(finish);
|
|
332
|
+
ctx.signal.addEventListener('abort', finish, {
|
|
206
333
|
once: true
|
|
207
334
|
});
|
|
208
335
|
});
|
|
209
336
|
},
|
|
210
|
-
'hub/keypackage/upload': async (ctx)=>{
|
|
337
|
+
'hub/v1/keypackage/upload': async (ctx)=>{
|
|
211
338
|
const { keyPackages } = ctx.param;
|
|
212
339
|
const clientDID = getClientDID(ctx);
|
|
213
340
|
await Promise.all(keyPackages.map((kp)=>store.storeKeyPackage(clientDID, kp)));
|
|
@@ -215,7 +342,7 @@ export function createHandlers(params) {
|
|
|
215
342
|
stored: keyPackages.length
|
|
216
343
|
};
|
|
217
344
|
},
|
|
218
|
-
'hub/keypackage/fetch': async (ctx)=>{
|
|
345
|
+
'hub/v1/keypackage/fetch': async (ctx)=>{
|
|
219
346
|
const requesterDID = getClientDID(ctx);
|
|
220
347
|
assertKeyPackageFetchAllowed(requesterDID);
|
|
221
348
|
const { did, count } = ctx.param;
|
package/lib/hub.d.ts
CHANGED
|
@@ -13,7 +13,11 @@ export declare const DEFAULT_HUB_ACCESS_RULES: AccessRules;
|
|
|
13
13
|
export type HubPurgeOptions = {
|
|
14
14
|
/** Interval between purge runs in milliseconds. Default: 3600000 (1 hour) */
|
|
15
15
|
interval?: number;
|
|
16
|
-
/**
|
|
16
|
+
/**
|
|
17
|
+
* The hub's default retention in seconds: the age bound applied to a topic no subscriber asked
|
|
18
|
+
* to keep for longer. Default: 604800 (7 days). The maximum a subscriber may ask for is the
|
|
19
|
+
* store's, since the store is the thing that refuses the subscribe.
|
|
20
|
+
*/
|
|
17
21
|
olderThan?: number;
|
|
18
22
|
};
|
|
19
23
|
export type CreateHubParams = {
|
|
@@ -30,12 +34,15 @@ export type CreateHubParams = {
|
|
|
30
34
|
authorize?: AuthorizeHook;
|
|
31
35
|
/** Publish rate limits. Merged over {@link DEFAULT_RATE_LIMITS}. */
|
|
32
36
|
rateLimits?: Partial<HubRateLimits>;
|
|
33
|
-
/**
|
|
37
|
+
/**
|
|
38
|
+
* Quotas applied to hub/v1/keypackage/fetch. Merged over
|
|
39
|
+
* {@link DEFAULT_KEYPACKAGE_FETCH_LIMITS}.
|
|
40
|
+
*/
|
|
34
41
|
keyPackageFetchLimits?: Partial<KeyPackageFetchLimits>;
|
|
35
42
|
/** Scheduled purge of expired stored messages. Set to `false` to disable. */
|
|
36
43
|
purge?: HubPurgeOptions | false;
|
|
37
44
|
/**
|
|
38
|
-
* Server resource limits. `hub/receive` is always added to
|
|
45
|
+
* Server resource limits. `hub/v1/receive` is always added to
|
|
39
46
|
* `longLivedProcedures` so open mailbox channels are exempt from
|
|
40
47
|
* `controllerTimeoutMs` and from the `maxConcurrentHandlers` cap.
|
|
41
48
|
*/
|
package/lib/hub.js
CHANGED
package/lib/index.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
export type {
|
|
1
|
+
export type { AuthorizeDecision, AuthorizeHook, AuthorizeRequest, CreateHandlersParams, HubRateLimits, KeyPackageFetchLimits, } from './handlers.js';
|
|
2
2
|
export { createHandlers, DEFAULT_KEYPACKAGE_FETCH_LIMITS, DEFAULT_RATE_LIMITS } from './handlers.js';
|
|
3
3
|
export type { CreateHubParams, HubInstance, HubPurgeOptions } from './hub.js';
|
|
4
4
|
export { createHub, DEFAULT_HUB_ACCESS_RULES } from './hub.js';
|
package/lib/memoryStore.d.ts
CHANGED
|
@@ -1,12 +1,34 @@
|
|
|
1
1
|
import type { HubStore } from '@kumiai/hub-protocol';
|
|
2
|
+
export type MemoryStoreRetention = {
|
|
3
|
+
/** Floor, in seconds, on how long a topic's frames are kept. Default 0. */
|
|
4
|
+
default?: number;
|
|
5
|
+
/**
|
|
6
|
+
* Ceiling, in seconds, on what a subscriber may request. Above it: RetentionExceededError.
|
|
7
|
+
* Default {@link DEFAULT_MAX_RETENTION} (30 days), finite by design — an unbounded ceiling lets
|
|
8
|
+
* any subscriber pin a topic's frames forever with `subscribe({ retention: 2**31 })`. A host
|
|
9
|
+
* wanting no ceiling sets `Number.POSITIVE_INFINITY` explicitly.
|
|
10
|
+
*/
|
|
11
|
+
max?: number;
|
|
12
|
+
};
|
|
2
13
|
export type MemoryStoreOptions = {
|
|
3
|
-
/** Per-topic max retained
|
|
14
|
+
/** Per-topic max retained entries; oldest are trimmed beyond this. Default 1000. */
|
|
4
15
|
maxDepth?: number;
|
|
16
|
+
retention?: MemoryStoreRetention;
|
|
5
17
|
};
|
|
6
18
|
/**
|
|
7
|
-
* In-memory
|
|
19
|
+
* In-memory HubStore for testing and development.
|
|
20
|
+
*
|
|
21
|
+
* Retention is a class (per publish) and a duration (per subscribe):
|
|
22
|
+
* - `'mailbox'` (default) is delivery-derived: readers are known at publish time, so the last ack
|
|
23
|
+
* frees the frame, and a publish nobody is subscribed to is dropped outright.
|
|
24
|
+
* - `'log'` is not: a subscriber that must read a frame may not exist when it is published, so no
|
|
25
|
+
* refcount over current subscribers can free it. It is appended whether or not anyone is
|
|
26
|
+
* subscribed, and only `trim` or the age bound removes it.
|
|
27
|
+
*
|
|
28
|
+
* The store is told the class; it never infers it, and never reads a payload.
|
|
8
29
|
*
|
|
9
|
-
*
|
|
10
|
-
*
|
|
30
|
+
* Both classes are age-bounded: frames live for the longest retention any subscriber asked for,
|
|
31
|
+
* floored at the hub's default. For mailbox that bound sits alongside the ack GC, ack usually
|
|
32
|
+
* winning.
|
|
11
33
|
*/
|
|
12
34
|
export declare function createMemoryStore(options?: MemoryStoreOptions): HubStore;
|
package/lib/memoryStore.js
CHANGED
|
@@ -1,138 +1,251 @@
|
|
|
1
|
+
import { HeadMismatchError, NotSubscribedError, RetentionExceededError } from '@kumiai/hub-protocol';
|
|
1
2
|
import { EventEmitter } from '@sozai/event';
|
|
2
3
|
const DEFAULT_MAX_DEPTH = 1000;
|
|
4
|
+
/** 30 days, in seconds. A finite default ceiling on requested retention. */ const DEFAULT_MAX_RETENTION = 2_592_000;
|
|
3
5
|
function formatSequenceID(counter) {
|
|
4
6
|
return String(counter).padStart(12, '0');
|
|
5
7
|
}
|
|
6
8
|
/**
|
|
7
|
-
*
|
|
9
|
+
* The entry as a reader sees it, with the ONE piece of routing metadata a reader cannot recover for
|
|
10
|
+
* itself: where the frame sits in its topic's log.
|
|
8
11
|
*
|
|
9
|
-
*
|
|
10
|
-
*
|
|
12
|
+
* Present iff the entry is log-class, and the key is left off entirely otherwise — a mailbox frame
|
|
13
|
+
* has no place in a log, and a falsy placeholder is a position a reader's cursor would move to,
|
|
14
|
+
* skipping every log frame below it. This store mints one sequence for both, so the log position of
|
|
15
|
+
* a log-class frame IS its sequenceID here; a store that mints a separate per-recipient delivery
|
|
16
|
+
* sequence reports its log's own value, and the conformance suite pins the equality that matters —
|
|
17
|
+
* a pushed log frame's `logPosition` is the position `fetchTopic` serves that same frame at.
|
|
18
|
+
*/ function asStoredMessage(entry) {
|
|
19
|
+
return {
|
|
20
|
+
sequenceID: entry.sequenceID,
|
|
21
|
+
senderDID: entry.senderDID,
|
|
22
|
+
topicID: entry.topicID,
|
|
23
|
+
payload: entry.payload,
|
|
24
|
+
...entry.retain === 'log' ? {
|
|
25
|
+
logPosition: entry.sequenceID
|
|
26
|
+
} : {}
|
|
27
|
+
};
|
|
28
|
+
}
|
|
29
|
+
/**
|
|
30
|
+
* In-memory HubStore for testing and development.
|
|
31
|
+
*
|
|
32
|
+
* Retention is a class (per publish) and a duration (per subscribe):
|
|
33
|
+
* - `'mailbox'` (default) is delivery-derived: readers are known at publish time, so the last ack
|
|
34
|
+
* frees the frame, and a publish nobody is subscribed to is dropped outright.
|
|
35
|
+
* - `'log'` is not: a subscriber that must read a frame may not exist when it is published, so no
|
|
36
|
+
* refcount over current subscribers can free it. It is appended whether or not anyone is
|
|
37
|
+
* subscribed, and only `trim` or the age bound removes it.
|
|
38
|
+
*
|
|
39
|
+
* The store is told the class; it never infers it, and never reads a payload.
|
|
40
|
+
*
|
|
41
|
+
* Both classes are age-bounded: frames live for the longest retention any subscriber asked for,
|
|
42
|
+
* floored at the hub's default. For mailbox that bound sits alongside the ack GC, ack usually
|
|
43
|
+
* winning.
|
|
11
44
|
*/ export function createMemoryStore(options = {}) {
|
|
12
45
|
const maxDepth = options.maxDepth ?? DEFAULT_MAX_DEPTH;
|
|
46
|
+
const defaultRetention = options.retention?.default ?? 0;
|
|
47
|
+
const maxRetention = options.retention?.max ?? DEFAULT_MAX_RETENTION;
|
|
13
48
|
let counter = 0;
|
|
14
|
-
const
|
|
49
|
+
const entries = new Map();
|
|
50
|
+
const topicLogs = new Map();
|
|
51
|
+
const heads = new Map();
|
|
15
52
|
const deliveries = new Map();
|
|
16
|
-
|
|
17
|
-
|
|
53
|
+
/**
|
|
54
|
+
* publishID -> the sequenceID it was accepted as. Not a log entry and not reachable from one: no
|
|
55
|
+
* deleter takes a publishID, so `removeEntry`, `trim` and `purge` cannot touch this map even by
|
|
56
|
+
* mistake. Retained indefinitely — it is the only thing that lets a peer replaying its journal
|
|
57
|
+
* learn that a commit it never saw acknowledged had in fact landed.
|
|
58
|
+
*/ const publishRecords = new Map();
|
|
59
|
+
/** Per topic: the subscribers, each with the retention it asked for. */ const subscriptions = new Map();
|
|
18
60
|
const keyPackages = new Map();
|
|
19
61
|
const events = new EventEmitter();
|
|
20
|
-
//
|
|
21
|
-
// and the
|
|
22
|
-
function
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
const
|
|
62
|
+
// The age bound for a topic: the longest retention any of its subscribers asked for, floored
|
|
63
|
+
// by the hub's default and by the caller's own bound.
|
|
64
|
+
function retentionOf(topicID, floor) {
|
|
65
|
+
let retention = Math.max(floor, defaultRetention);
|
|
66
|
+
const subscribers = subscriptions.get(topicID);
|
|
67
|
+
if (subscribers != null) {
|
|
68
|
+
for (const requested of subscribers.values()){
|
|
69
|
+
if (requested > retention) retention = requested;
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
return retention;
|
|
73
|
+
}
|
|
74
|
+
// The only removal path for an entry. Pending deliveries of it go with it: they reference an
|
|
75
|
+
// entry that no longer exists, so they can no longer be pushed. `heads` is never touched — an
|
|
76
|
+
// empty log still has a head.
|
|
77
|
+
function removeEntry(sequenceID) {
|
|
78
|
+
const entry = entries.get(sequenceID);
|
|
79
|
+
if (entry == null) return;
|
|
80
|
+
for (const recipientDID of entry.pendingFor){
|
|
81
|
+
const list = deliveries.get(recipientDID);
|
|
27
82
|
if (list != null) {
|
|
28
83
|
const index = list.indexOf(sequenceID);
|
|
29
84
|
if (index !== -1) list.splice(index, 1);
|
|
30
85
|
}
|
|
31
86
|
}
|
|
32
|
-
const
|
|
33
|
-
if (
|
|
34
|
-
const index =
|
|
35
|
-
if (index !== -1)
|
|
36
|
-
if (
|
|
87
|
+
const log = topicLogs.get(entry.topicID);
|
|
88
|
+
if (log != null) {
|
|
89
|
+
const index = log.indexOf(sequenceID);
|
|
90
|
+
if (index !== -1) log.splice(index, 1);
|
|
91
|
+
if (log.length === 0) topicLogs.delete(entry.topicID);
|
|
37
92
|
}
|
|
38
|
-
|
|
93
|
+
entries.delete(sequenceID);
|
|
39
94
|
}
|
|
40
|
-
// Drop one
|
|
41
|
-
//
|
|
42
|
-
|
|
95
|
+
// Drop one recipient's pending delivery. A mailbox frame whose last delivery is gone has been
|
|
96
|
+
// read by everyone who was ever going to read it, so it goes with it. A log frame does not: the
|
|
97
|
+
// subscriber that needs it may not exist yet.
|
|
98
|
+
function dropDelivery(recipientDID, sequenceID) {
|
|
43
99
|
const list = deliveries.get(recipientDID);
|
|
44
100
|
if (list != null) {
|
|
45
101
|
const index = list.indexOf(sequenceID);
|
|
46
102
|
if (index !== -1) list.splice(index, 1);
|
|
47
103
|
}
|
|
48
|
-
const
|
|
49
|
-
if (
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
}
|
|
104
|
+
const entry = entries.get(sequenceID);
|
|
105
|
+
if (entry == null) return;
|
|
106
|
+
entry.pendingFor.delete(recipientDID);
|
|
107
|
+
if (entry.retain === 'mailbox' && entry.pendingFor.size === 0) {
|
|
108
|
+
removeEntry(sequenceID);
|
|
54
109
|
}
|
|
55
110
|
}
|
|
56
111
|
return {
|
|
57
112
|
events,
|
|
58
113
|
async publish (params) {
|
|
114
|
+
const retain = params.retain ?? 'mailbox';
|
|
115
|
+
// The dedup check comes BEFORE the CAS; the order is load-bearing. A replay carries the
|
|
116
|
+
// already-accepted publishID and the journalled expectedHead, which the accepted publish
|
|
117
|
+
// itself made stale — comparing first would raise HeadMismatchError and tell the caller its
|
|
118
|
+
// commit was lost when it landed. The returned sequenceID may name an already-trimmed frame,
|
|
119
|
+
// which is correct: a replay asks "did my publish land?", not "give me my frame".
|
|
120
|
+
if (params.publishID != null) {
|
|
121
|
+
const accepted = publishRecords.get(params.publishID);
|
|
122
|
+
if (accepted !== undefined) {
|
|
123
|
+
return {
|
|
124
|
+
sequenceID: accepted,
|
|
125
|
+
deduped: true
|
|
126
|
+
};
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
// The CAS, before anything is minted or written. Head comparison, sequence mint, append and
|
|
130
|
+
// head advance are one indivisible step; a loser leaves log, head and sequence untouched (no
|
|
131
|
+
// entry, no delivery, no sequenceID consumed). Absent expectedHead this is skipped — the
|
|
132
|
+
// unconditional fast path every mailbox frame takes.
|
|
133
|
+
if (params.expectedHead !== undefined) {
|
|
134
|
+
const head = heads.get(params.topicID) ?? null;
|
|
135
|
+
if (head !== params.expectedHead) {
|
|
136
|
+
throw new HeadMismatchError(`Publish to ${params.topicID} expected head ${params.expectedHead ?? 'null'}, but the head is ${head ?? 'null'}`);
|
|
137
|
+
}
|
|
138
|
+
}
|
|
59
139
|
counter++;
|
|
60
140
|
const sequenceID = formatSequenceID(counter);
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
141
|
+
if (params.publishID != null) {
|
|
142
|
+
publishRecords.set(params.publishID, sequenceID);
|
|
143
|
+
}
|
|
144
|
+
// Only a log publish moves the head. A head naming a mailbox frame is a head that the
|
|
145
|
+
// frame's own last ack deletes, leaving readers of the log a head they can never reach.
|
|
146
|
+
if (retain === 'log') {
|
|
147
|
+
heads.set(params.topicID, sequenceID);
|
|
148
|
+
}
|
|
149
|
+
// Recipients = current subscribers minus the sender.
|
|
64
150
|
const recipients = new Set();
|
|
151
|
+
const subscribers = subscriptions.get(params.topicID);
|
|
65
152
|
if (subscribers != null) {
|
|
66
|
-
for (const
|
|
67
|
-
if (
|
|
153
|
+
for (const subscriberDID of subscribers.keys()){
|
|
154
|
+
if (subscriberDID !== params.senderDID) recipients.add(subscriberDID);
|
|
68
155
|
}
|
|
69
156
|
}
|
|
70
|
-
|
|
71
|
-
|
|
157
|
+
// A mailbox frame with no recipients has already been read by everyone who was going to
|
|
158
|
+
// read it. A log frame is kept regardless: its reader may not exist yet.
|
|
159
|
+
if (retain === 'mailbox' && recipients.size === 0) {
|
|
160
|
+
return {
|
|
161
|
+
sequenceID,
|
|
162
|
+
deduped: false
|
|
163
|
+
};
|
|
72
164
|
}
|
|
73
|
-
const
|
|
165
|
+
const entry = {
|
|
74
166
|
sequenceID,
|
|
75
167
|
senderDID: params.senderDID,
|
|
76
168
|
topicID: params.topicID,
|
|
77
169
|
payload: params.payload,
|
|
78
|
-
|
|
79
|
-
|
|
170
|
+
storedAt: Date.now(),
|
|
171
|
+
retain,
|
|
172
|
+
pendingFor: recipients
|
|
80
173
|
};
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
174
|
+
entries.set(sequenceID, entry);
|
|
175
|
+
let log = topicLogs.get(params.topicID);
|
|
176
|
+
if (log == null) {
|
|
177
|
+
log = [];
|
|
178
|
+
topicLogs.set(params.topicID, log);
|
|
179
|
+
}
|
|
180
|
+
log.push(sequenceID);
|
|
181
|
+
for (const recipientDID of recipients){
|
|
182
|
+
let list = deliveries.get(recipientDID);
|
|
84
183
|
if (list == null) {
|
|
85
184
|
list = [];
|
|
86
|
-
deliveries.set(
|
|
185
|
+
deliveries.set(recipientDID, list);
|
|
87
186
|
}
|
|
88
187
|
list.push(sequenceID);
|
|
89
188
|
}
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
189
|
+
// Depth bound: a trim like any other (moves oldest, leaves head alone), counting LOG frames
|
|
190
|
+
// only. A mailbox frame is bounded by ack and age, not depth; counting it here would let any
|
|
191
|
+
// member evict the commit log with a mailbox flood. Only a log publish can push the log-class
|
|
192
|
+
// count over the bound, so this is skipped for the mailbox fast path.
|
|
193
|
+
if (retain === 'log') {
|
|
194
|
+
let logDepth = 0;
|
|
195
|
+
for (const id of log){
|
|
196
|
+
if (entries.get(id)?.retain === 'log') logDepth++;
|
|
197
|
+
}
|
|
198
|
+
while(logDepth > maxDepth){
|
|
199
|
+
const oldestLog = log.find((id)=>entries.get(id)?.retain === 'log');
|
|
200
|
+
if (oldestLog == null) break;
|
|
201
|
+
removeEntry(oldestLog);
|
|
202
|
+
logDepth--;
|
|
203
|
+
}
|
|
99
204
|
}
|
|
100
|
-
return
|
|
205
|
+
return {
|
|
206
|
+
sequenceID,
|
|
207
|
+
deduped: false
|
|
208
|
+
};
|
|
101
209
|
},
|
|
102
210
|
async fetch (params) {
|
|
103
211
|
if (params.ack != null && params.ack.length > 0) {
|
|
104
212
|
for (const sequenceID of params.ack){
|
|
105
|
-
|
|
213
|
+
dropDelivery(params.recipientDID, sequenceID);
|
|
106
214
|
}
|
|
107
215
|
}
|
|
108
|
-
const
|
|
109
|
-
if (
|
|
216
|
+
const pending = deliveries.get(params.recipientDID);
|
|
217
|
+
if (pending == null || pending.length === 0) {
|
|
110
218
|
return {
|
|
111
219
|
messages: [],
|
|
112
220
|
cursor: null
|
|
113
221
|
};
|
|
114
222
|
}
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
223
|
+
// `after` is an exclusive cursor: the first pending delivery with sequenceID strictly greater
|
|
224
|
+
// than it. Pending is in append order (strictly increasing), so this is the position just
|
|
225
|
+
// past `after` whether or not `after` is still pending. Matching on the value (indexOf) would
|
|
226
|
+
// fall back to index 0 once the cursored delivery is acked or aged out between pages of a
|
|
227
|
+
// drain, silently restarting from the top and re-serving frames already handed out. Scanning
|
|
228
|
+
// for `> after` resumes where the cursor pointed even after its own entry is gone.
|
|
229
|
+
let startIndex = pending.length;
|
|
230
|
+
if (params.after == null) {
|
|
231
|
+
startIndex = 0;
|
|
232
|
+
} else {
|
|
233
|
+
for(let index = 0; index < pending.length; index++){
|
|
234
|
+
if (pending[index] > params.after) {
|
|
235
|
+
startIndex = index;
|
|
236
|
+
break;
|
|
237
|
+
}
|
|
120
238
|
}
|
|
121
239
|
}
|
|
122
|
-
const available =
|
|
240
|
+
const available = pending.slice(startIndex);
|
|
123
241
|
const limit = params.limit ?? available.length;
|
|
124
242
|
const selected = available.slice(0, limit);
|
|
125
243
|
const hasMore = available.length > limit;
|
|
126
244
|
const resultMessages = [];
|
|
127
245
|
for (const sequenceID of selected){
|
|
128
|
-
const
|
|
129
|
-
if (
|
|
130
|
-
resultMessages.push(
|
|
131
|
-
sequenceID: record.sequenceID,
|
|
132
|
-
senderDID: record.senderDID,
|
|
133
|
-
topicID: record.topicID,
|
|
134
|
-
payload: record.payload
|
|
135
|
-
});
|
|
246
|
+
const entry = entries.get(sequenceID);
|
|
247
|
+
if (entry != null) {
|
|
248
|
+
resultMessages.push(asStoredMessage(entry));
|
|
136
249
|
}
|
|
137
250
|
}
|
|
138
251
|
const cursor = resultMessages.length > 0 ? resultMessages[resultMessages.length - 1].sequenceID : null;
|
|
@@ -145,20 +258,68 @@ function formatSequenceID(counter) {
|
|
|
145
258
|
}
|
|
146
259
|
return result;
|
|
147
260
|
},
|
|
261
|
+
async fetchTopic (params) {
|
|
262
|
+
if (!subscriptions.get(params.topicID)?.has(params.subscriberDID)) {
|
|
263
|
+
throw new NotSubscribedError(`${params.subscriberDID} is not a subscriber of ${params.topicID}`);
|
|
264
|
+
}
|
|
265
|
+
// A topic's LOG is its log-class frames and nothing else. A mailbox frame published to a log
|
|
266
|
+
// topic is still delivered (push is untouched) but never enters the log: it does not move the
|
|
267
|
+
// head, so a reader that saw one would hold a cursor naming a position the head can never
|
|
268
|
+
// reach, and every CAS anchored there would lose forever. Only the publisher's own `retain`
|
|
269
|
+
// sets the class.
|
|
270
|
+
const log = (topicLogs.get(params.topicID) ?? []).filter((sequenceID)=>entries.get(sequenceID)?.retain === 'log');
|
|
271
|
+
const after = params.after;
|
|
272
|
+
// Filter to the class BEFORE the limit: a page of mailbox frames must not eat the limit and
|
|
273
|
+
// hand back an empty page while log frames wait — a caller draining until a short page would
|
|
274
|
+
// stop early and strand itself.
|
|
275
|
+
const selected = after == null ? log : log.filter((sequenceID)=>sequenceID > after);
|
|
276
|
+
const limited = params.limit == null ? selected : selected.slice(0, params.limit);
|
|
277
|
+
const messages = [];
|
|
278
|
+
for (const sequenceID of limited){
|
|
279
|
+
const entry = entries.get(sequenceID);
|
|
280
|
+
if (entry != null) {
|
|
281
|
+
messages.push(asStoredMessage(entry));
|
|
282
|
+
}
|
|
283
|
+
}
|
|
284
|
+
return {
|
|
285
|
+
messages,
|
|
286
|
+
head: heads.get(params.topicID) ?? null,
|
|
287
|
+
oldest: log.length > 0 ? log[0] : null
|
|
288
|
+
};
|
|
289
|
+
},
|
|
290
|
+
async trim (params) {
|
|
291
|
+
const log = topicLogs.get(params.topicID);
|
|
292
|
+
if (log == null) return;
|
|
293
|
+
for (const sequenceID of [
|
|
294
|
+
...log
|
|
295
|
+
]){
|
|
296
|
+
// Log-class frames only. A mailbox frame shares the topic's array but not its log: it is
|
|
297
|
+
// delivery-derived, freed by its last ack or by age, and trim never removes it — trimming
|
|
298
|
+
// the mixed array silently drops pending mail below the bound.
|
|
299
|
+
if (sequenceID < params.before && entries.get(sequenceID)?.retain === 'log') {
|
|
300
|
+
removeEntry(sequenceID);
|
|
301
|
+
}
|
|
302
|
+
}
|
|
303
|
+
},
|
|
148
304
|
async ack (params) {
|
|
149
305
|
for (const sequenceID of params.sequenceIDs){
|
|
150
|
-
|
|
306
|
+
dropDelivery(params.recipientDID, sequenceID);
|
|
151
307
|
}
|
|
152
308
|
},
|
|
153
309
|
async purge (params) {
|
|
154
|
-
|
|
310
|
+
// The age bound, for both classes: the same removal path, the same invariants — head is
|
|
311
|
+
// untouched.
|
|
312
|
+
const now = Date.now();
|
|
155
313
|
const purgedIDs = [];
|
|
156
|
-
for (const [sequenceID,
|
|
157
|
-
|
|
314
|
+
for (const [sequenceID, entry] of entries){
|
|
315
|
+
const retention = retentionOf(entry.topicID, params.olderThan);
|
|
316
|
+
if (entry.storedAt <= now - retention * 1000) {
|
|
158
317
|
purgedIDs.push(sequenceID);
|
|
159
|
-
deleteMessage(sequenceID);
|
|
160
318
|
}
|
|
161
319
|
}
|
|
320
|
+
for (const sequenceID of purgedIDs){
|
|
321
|
+
removeEntry(sequenceID);
|
|
322
|
+
}
|
|
162
323
|
if (purgedIDs.length > 0) {
|
|
163
324
|
await events.emit('purge', {
|
|
164
325
|
sequenceIDs: purgedIDs
|
|
@@ -166,50 +327,43 @@ function formatSequenceID(counter) {
|
|
|
166
327
|
}
|
|
167
328
|
return purgedIDs;
|
|
168
329
|
},
|
|
169
|
-
async subscribe (
|
|
170
|
-
|
|
171
|
-
if (
|
|
172
|
-
|
|
173
|
-
|
|
330
|
+
async subscribe (params) {
|
|
331
|
+
const requested = params.retention ?? defaultRetention;
|
|
332
|
+
if (requested > maxRetention) {
|
|
333
|
+
throw new RetentionExceededError(`Requested retention of ${requested}s exceeds the maximum of ${maxRetention}s`);
|
|
334
|
+
}
|
|
335
|
+
let subscribers = subscriptions.get(params.topicID);
|
|
336
|
+
if (subscribers == null) {
|
|
337
|
+
subscribers = new Map();
|
|
338
|
+
subscriptions.set(params.topicID, subscribers);
|
|
174
339
|
}
|
|
175
|
-
|
|
340
|
+
subscribers.set(params.subscriberDID, requested);
|
|
176
341
|
},
|
|
177
342
|
async unsubscribe (subscriberDID, topicID) {
|
|
178
|
-
const
|
|
179
|
-
if (
|
|
180
|
-
|
|
181
|
-
if (
|
|
343
|
+
const subscribers = subscriptions.get(topicID);
|
|
344
|
+
if (subscribers != null) {
|
|
345
|
+
subscribers.delete(subscriberDID);
|
|
346
|
+
if (subscribers.size === 0) {
|
|
182
347
|
subscriptions.delete(topicID);
|
|
183
348
|
}
|
|
184
349
|
}
|
|
185
|
-
//
|
|
186
|
-
|
|
187
|
-
|
|
350
|
+
// Drops this subscriber's pending deliveries for the topic — freeing a mailbox frame whose
|
|
351
|
+
// last delivery this was, and leaving a log frame standing.
|
|
352
|
+
const pending = deliveries.get(subscriberDID);
|
|
353
|
+
if (pending != null) {
|
|
188
354
|
for (const sequenceID of [
|
|
189
|
-
...
|
|
355
|
+
...pending
|
|
190
356
|
]){
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
removeDelivery(subscriberDID, sequenceID);
|
|
194
|
-
}
|
|
195
|
-
}
|
|
196
|
-
}
|
|
197
|
-
// Last subscriber gone → drop the whole topic log immediately.
|
|
198
|
-
if (!subscriptions.has(topicID)) {
|
|
199
|
-
const topicLog = topicMessages.get(topicID);
|
|
200
|
-
if (topicLog != null) {
|
|
201
|
-
for (const sequenceID of [
|
|
202
|
-
...topicLog
|
|
203
|
-
]){
|
|
204
|
-
deleteMessage(sequenceID);
|
|
357
|
+
if (entries.get(sequenceID)?.topicID === topicID) {
|
|
358
|
+
dropDelivery(subscriberDID, sequenceID);
|
|
205
359
|
}
|
|
206
360
|
}
|
|
207
361
|
}
|
|
208
362
|
},
|
|
209
363
|
async getSubscribers (topicID) {
|
|
210
|
-
const
|
|
211
|
-
return
|
|
212
|
-
...
|
|
364
|
+
const subscribers = subscriptions.get(topicID);
|
|
365
|
+
return subscribers == null ? [] : [
|
|
366
|
+
...subscribers.keys()
|
|
213
367
|
];
|
|
214
368
|
},
|
|
215
369
|
async storeKeyPackage (ownerDID, keyPackage) {
|
package/lib/registry.d.ts
CHANGED
|
@@ -2,6 +2,10 @@ import type { StoredMessage } from '@kumiai/hub-protocol';
|
|
|
2
2
|
export type ClientEntry = {
|
|
3
3
|
did: string;
|
|
4
4
|
sendMessage: ((message: StoredMessage) => void) | null;
|
|
5
|
+
/** Ends the currently bound receive channel. Set with the writer and cleared with it. */
|
|
6
|
+
endReceive: (() => void) | null;
|
|
7
|
+
/** Identifies the binding, so a channel can only ever release its OWN. */
|
|
8
|
+
token: symbol | null;
|
|
5
9
|
};
|
|
6
10
|
/**
|
|
7
11
|
* Tracks online clients and their live receive-channel writers. Subscription
|
|
@@ -14,8 +18,33 @@ export declare class HubClientRegistry {
|
|
|
14
18
|
unregister(did: string): void;
|
|
15
19
|
/** Removes the entry only when no receive writer is bound. */
|
|
16
20
|
unregisterIfIdle(did: string): void;
|
|
17
|
-
|
|
18
|
-
|
|
21
|
+
/**
|
|
22
|
+
* Bind this DID's receive writer, EVICTING whatever was bound before it.
|
|
23
|
+
*
|
|
24
|
+
* One writer per DID, and the newest one wins. The alternative — refuse the second — reads as
|
|
25
|
+
* the safer rule and is not: a client whose socket died has a writer the server still believes
|
|
26
|
+
* in, because nothing has told it otherwise yet, and that belief outlives the socket by however
|
|
27
|
+
* long the transport takes to notice. Refusing on it means the reconnect that the client makes
|
|
28
|
+
* precisely BECAUSE its connection dropped is the one that gets turned away, and the member has
|
|
29
|
+
* no push lane until a timeout it cannot see or influence expires. Last-writer-wins costs a
|
|
30
|
+
* live connection only when the same authenticated DID opens a second channel, which is the
|
|
31
|
+
* same client and its own business.
|
|
32
|
+
*
|
|
33
|
+
* The eviction is handed back rather than run here: ending a channel is the handler's to do,
|
|
34
|
+
* and a registry that closed streams would be doing two jobs.
|
|
35
|
+
*
|
|
36
|
+
* The TOKEN is what keeps the eviction from unbinding its successor. The evicted handler runs
|
|
37
|
+
* its own cleanup afterwards — it does not know why it ended — and a `clear(did)` there would
|
|
38
|
+
* null the writer the new channel has just bound, leaving the DID online with nothing behind
|
|
39
|
+
* it. So a release names the binding it is releasing, and one that is no longer current is a
|
|
40
|
+
* no-op.
|
|
41
|
+
*/
|
|
42
|
+
bindReceiveWriter(did: string, writer: (message: StoredMessage) => void, endReceive: () => void): {
|
|
43
|
+
token: symbol;
|
|
44
|
+
evicted: (() => void) | null;
|
|
45
|
+
};
|
|
46
|
+
/** Clear the binding this token names. A token that is no longer current is a no-op. */
|
|
47
|
+
releaseReceiveWriter(did: string, token: symbol): void;
|
|
19
48
|
getClient(did: string): ClientEntry | undefined;
|
|
20
49
|
isOnline(did: string): boolean;
|
|
21
50
|
isWriterBound(did: string): boolean;
|
package/lib/registry.js
CHANGED
|
@@ -11,7 +11,9 @@
|
|
|
11
11
|
}
|
|
12
12
|
const entry = {
|
|
13
13
|
did,
|
|
14
|
-
sendMessage: null
|
|
14
|
+
sendMessage: null,
|
|
15
|
+
endReceive: null,
|
|
16
|
+
token: null
|
|
15
17
|
};
|
|
16
18
|
this.#clients.set(did, entry);
|
|
17
19
|
return entry;
|
|
@@ -25,19 +27,48 @@
|
|
|
25
27
|
this.#clients.delete(did);
|
|
26
28
|
}
|
|
27
29
|
}
|
|
28
|
-
|
|
30
|
+
/**
|
|
31
|
+
* Bind this DID's receive writer, EVICTING whatever was bound before it.
|
|
32
|
+
*
|
|
33
|
+
* One writer per DID, and the newest one wins. The alternative — refuse the second — reads as
|
|
34
|
+
* the safer rule and is not: a client whose socket died has a writer the server still believes
|
|
35
|
+
* in, because nothing has told it otherwise yet, and that belief outlives the socket by however
|
|
36
|
+
* long the transport takes to notice. Refusing on it means the reconnect that the client makes
|
|
37
|
+
* precisely BECAUSE its connection dropped is the one that gets turned away, and the member has
|
|
38
|
+
* no push lane until a timeout it cannot see or influence expires. Last-writer-wins costs a
|
|
39
|
+
* live connection only when the same authenticated DID opens a second channel, which is the
|
|
40
|
+
* same client and its own business.
|
|
41
|
+
*
|
|
42
|
+
* The eviction is handed back rather than run here: ending a channel is the handler's to do,
|
|
43
|
+
* and a registry that closed streams would be doing two jobs.
|
|
44
|
+
*
|
|
45
|
+
* The TOKEN is what keeps the eviction from unbinding its successor. The evicted handler runs
|
|
46
|
+
* its own cleanup afterwards — it does not know why it ended — and a `clear(did)` there would
|
|
47
|
+
* null the writer the new channel has just bound, leaving the DID online with nothing behind
|
|
48
|
+
* it. So a release names the binding it is releasing, and one that is no longer current is a
|
|
49
|
+
* no-op.
|
|
50
|
+
*/ bindReceiveWriter(did, writer, endReceive) {
|
|
29
51
|
const entry = this.#clients.get(did);
|
|
30
|
-
if (entry == null) return
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
}
|
|
52
|
+
if (entry == null) return {
|
|
53
|
+
token: Symbol('unbound'),
|
|
54
|
+
evicted: null
|
|
55
|
+
};
|
|
56
|
+
const evicted = entry.endReceive;
|
|
57
|
+
const token = Symbol('receive');
|
|
34
58
|
entry.sendMessage = writer;
|
|
59
|
+
entry.endReceive = endReceive;
|
|
60
|
+
entry.token = token;
|
|
61
|
+
return {
|
|
62
|
+
token,
|
|
63
|
+
evicted
|
|
64
|
+
};
|
|
35
65
|
}
|
|
36
|
-
|
|
66
|
+
/** Clear the binding this token names. A token that is no longer current is a no-op. */ releaseReceiveWriter(did, token) {
|
|
37
67
|
const entry = this.#clients.get(did);
|
|
38
|
-
if (entry
|
|
39
|
-
|
|
40
|
-
|
|
68
|
+
if (entry == null || entry.token !== token) return;
|
|
69
|
+
entry.sendMessage = null;
|
|
70
|
+
entry.endReceive = null;
|
|
71
|
+
entry.token = null;
|
|
41
72
|
}
|
|
42
73
|
getClient(did) {
|
|
43
74
|
return this.#clients.get(did);
|
package/package.json
CHANGED
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@kumiai/hub-server",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"license": "MIT",
|
|
3
|
+
"version": "0.4.0",
|
|
5
4
|
"description": "Hub server for Enkaku E2EE group message routing",
|
|
6
5
|
"keywords": [
|
|
7
6
|
"hub",
|
|
@@ -14,37 +13,37 @@
|
|
|
14
13
|
"url": "https://github.com/TairuFramework/kumiai",
|
|
15
14
|
"directory": "packages/hub-server"
|
|
16
15
|
},
|
|
16
|
+
"license": "MIT",
|
|
17
|
+
"sideEffects": false,
|
|
17
18
|
"type": "module",
|
|
18
|
-
"main": "lib/index.js",
|
|
19
|
-
"types": "lib/index.d.ts",
|
|
20
19
|
"exports": {
|
|
21
20
|
".": "./lib/index.js"
|
|
22
21
|
},
|
|
22
|
+
"main": "lib/index.js",
|
|
23
|
+
"types": "lib/index.d.ts",
|
|
23
24
|
"files": [
|
|
24
25
|
"lib/*"
|
|
25
26
|
],
|
|
26
|
-
"sideEffects": false,
|
|
27
27
|
"dependencies": {
|
|
28
|
-
"@sozai/codec": "^0.2.0",
|
|
29
|
-
"@sozai/event": "^0.1.1",
|
|
30
28
|
"@enkaku/protocol": "^0.19.0",
|
|
31
29
|
"@enkaku/server": "^0.19.0",
|
|
32
|
-
"@
|
|
33
|
-
"@
|
|
34
|
-
"@
|
|
35
|
-
"@kumiai/hub-protocol": "^0.
|
|
30
|
+
"@kokuin/token": "^0.3.0",
|
|
31
|
+
"@sozai/codec": "^0.2.0",
|
|
32
|
+
"@sozai/event": "^0.1.1",
|
|
33
|
+
"@kumiai/hub-protocol": "^0.4.0"
|
|
36
34
|
},
|
|
37
35
|
"devDependencies": {
|
|
38
36
|
"@enkaku/client": "^0.19.0",
|
|
39
|
-
"@enkaku/transport": "^0.19.1"
|
|
37
|
+
"@enkaku/transport": "^0.19.1",
|
|
38
|
+
"@kumiai/hub-conformance": "^0.4.0"
|
|
40
39
|
},
|
|
41
40
|
"scripts": {
|
|
41
|
+
"build": "pnpm run build:clean && pnpm run build:js && pnpm run build:types",
|
|
42
42
|
"build:clean": "del lib",
|
|
43
43
|
"build:js": "swc src -d ./lib --config-file ../../node_modules/@kigu/dev/swc.json --strip-leading-paths",
|
|
44
44
|
"build:types": "tsc --emitDeclarationOnly --skipLibCheck",
|
|
45
|
-
"
|
|
45
|
+
"test": "pnpm run test:types && pnpm run test:unit",
|
|
46
46
|
"test:types": "tsc --noEmit --skipLibCheck -p tsconfig.test.json",
|
|
47
|
-
"test:unit": "vitest run"
|
|
48
|
-
"test": "pnpm run test:types && pnpm run test:unit"
|
|
47
|
+
"test:unit": "vitest run"
|
|
49
48
|
}
|
|
50
49
|
}
|