clanka 0.2.39 → 0.2.41

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,1032 @@
1
+ const spec = String.raw`# EventLogServerUnencrypted
2
+
3
+ ## Overview
4
+
5
+ Update \
6
+ t the unencrypted event-log server specification so the server can append its own
7
+ client-visible events, while still supporting client-originated writes over the
8
+ existing ` + "`EventLogRemote`" + String.raw` websocket protocol.
9
+
10
+ The unencrypted server must no longer treat transport history as purely
11
+ per-public-key. Instead it must introduce a persisted **store id** concept:
12
+
13
+ - every client ` + "`publicKey`" + String.raw` resolves to a ` + "`StoreId`" + String.raw`
14
+ - transport history is stored and sequenced per ` + "`StoreId`" + String.raw`, not per public key
15
+ - multiple public keys may resolve to the same store id and therefore observe
16
+ the same outbound history and sequence numbers
17
+ - client writes are appended to the resolved store feed
18
+ - server-authored writes target a store id directly and are broadcast to all
19
+ clients mapped to that store
20
+
21
+ The module remains intentionally **read-time compaction, not ingest-time
22
+ compaction**:
23
+
24
+ - all accepted client writes are stored and processed immediately
25
+ - server-authored writes are stored immediately and optionally processed through
26
+ handlers / reactivity depending on configuration
27
+ - compaction only affects what the server sends back from
28
+ ` + "`RequestChanges(startSequence)`" + String.raw`, and only for entries older than a
29
+ configurable fixed duration
30
+
31
+ The design should still reuse existing EventLog patterns as much as possible:
32
+
33
+ - ` + "`EventLog.group(...)`" + String.raw` remains the handler authoring API
34
+ - grouping / payload decoding behavior from ` + "`EventLog.groupCompaction(...)`" + String.raw`
35
+ should be reused or factored into shared internal helpers
36
+ - Reactivity invalidation semantics should match ` + "`EventLog`" + String.raw` whenever handler
37
+ replay is enabled
38
+ - ` + "`SqlEventLogJournal`" + String.raw` remains the preferred journal implementation for
39
+ the server-side processing journal when persistence is needed
40
+ - transport history and publicKey-to-store mapping should stay behind narrow
41
+ service contracts so a future SQL-backed implementation can be added without
42
+ reshaping the runtime
43
+
44
+ ## User Decisions Captured
45
+
46
+ The specification reflects the following clarified requirements:
47
+
48
+ 1. authorization is handled by a new required ` + "`EventLogServerAuth`" + String.raw` service
49
+ for **client-originated** reads and writes
50
+ 2. backend handlers should reuse ` + "`EventLog.group(...)`" + String.raw`
51
+ 3. compaction applies only to events older than a configurable fixed duration
52
+ 4. unauthorized client writes reject the whole batch
53
+ 5. ` + "`Ack.sequenceNumbers`" + String.raw` must still use one sequence per originally
54
+ submitted event slot
55
+ 6. server reactivity invalidation must run for all accepted incoming client
56
+ events and for server-authored writes only when handler replay is enabled
57
+ 7. unencrypted protocol responses need an explicit error message type
58
+ 8. outbound compaction applies to the feed returned from
59
+ ` + "`RequestChanges(startSequence)`" + String.raw`
60
+ 9. server-authored events must be able to target a ` + "`StoreId`" + String.raw` directly
61
+ 10. public keys must map to store ids through a persistable mapping abstraction
62
+ 11. server-authored writes bypass ` + "`EventLogServerAuth`" + String.raw`
63
+ 12. whether server-authored writes run handlers / reactivity must be
64
+ configurable per write
65
+
66
+ ## Goals
67
+
68
+ - Make ` + "`EventLogServerUnencrypted`" + String.raw` feature-complete enough to back a real
69
+ backend event-log endpoint.
70
+ - Allow multiple client identities to share a single store feed through a
71
+ persisted publicKey-to-store mapping.
72
+ - Allow the backend to append its own events to a store feed and have those
73
+ events broadcast to all mapped clients.
74
+ - Allow the backend to execute the same event handlers authored with
75
+ ` + "`EventLog.group(...)`" + String.raw` that are already used by clients.
76
+ - Ensure accepted client writes trigger the same Reactivity invalidation pattern
77
+ used by ` + "`EventLog`" + String.raw`, and ensure server-authored writes can opt into that
78
+ same processing path.
79
+ - Finish the already-started unencrypted protocol support in ` + "`EventLogRemote`" + String.raw`.
80
+
81
+ ## Non-Goals
82
+
83
+ - Adding encrypted-protocol error frames in the same change.
84
+ - Mutating or deleting stored raw events as part of compaction. Compaction is a
85
+ read-time projection.
86
+ - Replacing ` + "`EventLog.group(...)`" + String.raw` with a new handler DSL.
87
+ - Requiring a dedicated SQL transport-storage adapter in the same PR.
88
+ - Requiring a production-ready SQL implementation of the publicKey-to-store
89
+ registry in the same PR, as long as the service contract is explicitly
90
+ durable-friendly and the shipped runtime does not hard-code ephemeral
91
+ derivation.
92
+ - Adding cross-store fan-out APIs in the initial change. Targeting one store per
93
+ write is sufficient.
94
+
95
+ ## Module Surface
96
+
97
+ ### Primary module
98
+
99
+ - ` + "`packages/effect/src/unstable/eventlog/EventLogServerUnencrypted.ts`" + String.raw`
100
+
101
+ ### Existing modules that must change
102
+
103
+ - ` + "`packages/effect/src/unstable/eventlog/EventLogRemote.ts`" + String.raw`
104
+ - ` + "`packages/effect/src/unstable/eventlog/EventLog.ts`" + String.raw` (internal refactors /
105
+ shared helpers only, unless a small public adjustment is the cleanest option)
106
+ - optionally ` + "`packages/effect/src/unstable/eventlog/EventJournal.ts`" + String.raw` only if a
107
+ small helper type is needed; avoid unnecessary public API churn
108
+
109
+ ### Tests
110
+
111
+ - ` + "`packages/effect/test/unstable/eventlog/EventLogServerUnencrypted.test.ts`" + String.raw`
112
+ - update/add targeted tests around ` + "`EventLogRemote.ts`" + String.raw` if needed
113
+
114
+ ## Public API
115
+
116
+ ### 1. ` + "`StoreId`" + String.raw`
117
+
118
+ Add a stable server-visible store identifier.
119
+
120
+ Suggested shape:
121
+
122
+ ```ts
123
+ export type StoreIdTypeId = "effect/eventlog/EventLogServerUnencrypted/StoreId"
124
+
125
+ export type StoreId = string & Brand<StoreIdTypeId>
126
+
127
+ export const StoreId = Schema.String.pipe(Schema.brand(StoreIdTypeId))
128
+ ```
129
+
130
+ The exact encoding can differ, but it must be:
131
+
132
+ - stable across process restarts
133
+ - serializable / persistable by user-provided infrastructure
134
+ - suitable as the key for transport history and store-mapping persistence
135
+
136
+ ### 2. ` + "`EventLogServerStoreRegistry`" + String.raw`
137
+
138
+ Add a required service responsible for resolving public keys to store ids.
139
+
140
+ ```ts
141
+ export class EventLogServerStoreRegistry extends ServiceMap.Service<EventLogServerStoreRegistry, {
142
+ readonly resolve: (publicKey: string) => Effect.Effect<StoreId, EventLogServerStoreRegistryError>
143
+ readonly assign: (options: {
144
+ readonly publicKey: string
145
+ readonly storeId: StoreId
146
+ }) => Effect.Effect<void, EventLogServerStoreRegistryError>
147
+ }>()("effect/eventlog/EventLogServerUnencrypted/EventLogServerStoreRegistry") {}
148
+ ```
149
+
150
+ #### Required semantics
151
+
152
+ - ` + "`resolve(publicKey)`" + String.raw` is used by all client-originated paths:
153
+ - ` + "`WriteEntriesUnencrypted`" + String.raw`
154
+ - ` + "`RequestChanges`" + String.raw`
155
+ - two public keys assigned the same store id must observe the same outbound
156
+ history and sequence numbers
157
+ - the mapping is domain state, not an ephemeral hash or in-memory-only default
158
+ - the main server runtime must require this service explicitly
159
+ - a helper such as ` + "`layerStoreRegistryMemory`" + String.raw` is recommended for tests and
160
+ local development
161
+ - the service contract must support durable implementations, even if only a
162
+ memory helper ships in the first PR
163
+
164
+ #### Notes
165
+
166
+ - ` + "`assign(...)`" + String.raw` is included so tests and local setups can configure mappings
167
+ without reaching into implementation details
168
+ - additional helpers such as ` + "`assignMany`" + String.raw`, ` + "`remove`" + String.raw`, or read-only admin APIs
169
+ are acceptable but not required
170
+
171
+ ### 3. ` + "`EventLogServerStoreRegistryError`" + String.raw`
172
+
173
+ Add a structured error for store-resolution failures.
174
+
175
+ ```ts
176
+ export class EventLogServerStoreRegistryError extends Data.TaggedError("EventLogServerStoreRegistryError")<{
177
+ readonly reason: "NotMapped" | "AlreadyAssigned" | "Unavailable"
178
+ readonly publicKey?: string | undefined
179
+ readonly storeId?: StoreId | undefined
180
+ readonly message?: string | undefined
181
+ }> {}
182
+ ```
183
+
184
+ Required behavior:
185
+
186
+ - an unmapped client public key must not silently derive a store id
187
+ - write / read requests for an unmapped public key must fail before any
188
+ transport persistence or subscription creation
189
+ - protocol mapping for ` + "`NotMapped`" + String.raw` should use ` + "`ErrorUnencrypted`" + String.raw` with code
190
+ ` + '"InvalidRequest"' + String.raw` unless the implementation intentionally wants to conceal
191
+ store existence behind an auth failure; whichever mapping is chosen must be
192
+ consistent across read and write paths
193
+
194
+ ### 4. ` + "`EventLogServerAuth`" + String.raw`
195
+
196
+ Keep the required auth service from the original spec for client-originated
197
+ traffic.
198
+
199
+ ```ts
200
+ export class EventLogServerAuth extends ServiceMap.Service<EventLogServerAuth, {
201
+ readonly authorizeWrite: (options: {
202
+ readonly publicKey: string
203
+ readonly entries: ReadonlyArray<Entry>
204
+ }) => Effect.Effect<void, EventLogServerAuthError>
205
+ readonly authorizeRead: (publicKey: string) => Effect.Effect<void, EventLogServerAuthError>
206
+ }>()("effect/eventlog/EventLogServerUnencrypted/EventLogServerAuth") {}
207
+ ```
208
+
209
+ #### Semantics
210
+
211
+ - ` + "`authorizeWrite`" + String.raw` is called once per inbound client write request before any
212
+ persistence, handler execution, reactivity invalidation, or store resolution
213
+ - authorization is batch-wide: if it fails, the entire write request is
214
+ rejected
215
+ - ` + "`authorizeRead`" + String.raw` is called before creating a ` + "`RequestChanges`" + String.raw` subscription
216
+ - server-authored writes targeting a store id bypass this service entirely
217
+ - a helper layer such as ` + "`layerAuthAllowAll`" + String.raw` is recommended for tests and local
218
+ development, but the service itself must remain required by the main server
219
+ constructor APIs
220
+
221
+ ### 5. ` + "`EventLogServerAuthError`" + String.raw`
222
+
223
+ Add a tagged error type for auth rejections.
224
+
225
+ ```ts
226
+ export class EventLogServerAuthError extends Data.TaggedError("EventLogServerAuthError")<{
227
+ readonly reason: "Unauthorized" | "Forbidden"
228
+ readonly publicKey: string
229
+ readonly message?: string | undefined
230
+ }> {}
231
+ ```
232
+
233
+ Exact fields may vary, but the type must be structured enough to map to an
234
+ unencrypted protocol error frame.
235
+
236
+ ### 6. ` + "`Storage`" + String.raw`
237
+
238
+ Update the transport-history storage abstraction so it is keyed by ` + "`StoreId`" + String.raw`, not
239
+ public key.
240
+
241
+ ```ts
242
+ export class Storage extends ServiceMap.Service<Storage, {
243
+ readonly getId: Effect.Effect<RemoteId>
244
+ readonly write: (options: {
245
+ readonly storeId: StoreId
246
+ readonly entries: ReadonlyArray<Entry>
247
+ }) => Effect.Effect<{
248
+ readonly sequenceNumbers: ReadonlyArray<number>
249
+ readonly committed: ReadonlyArray<RemoteEntry>
250
+ }>
251
+ readonly entries: (
252
+ storeId: StoreId,
253
+ startSequence: number
254
+ ) => Effect.Effect<ReadonlyArray<RemoteEntry>>
255
+ readonly changes: (
256
+ storeId: StoreId,
257
+ startSequence: number
258
+ ) => Effect.Effect<Queue.Dequeue<RemoteEntry, Cause.Done>, never, Scope.Scope>
259
+ }>()("effect/eventlog/EventLogServerUnencrypted/Storage") {}
260
+ ```
261
+
262
+ #### Required semantics
263
+
264
+ - transport history is sequenced independently per store id
265
+ - duplicate detection is per store id and entry id
266
+ - ` + "`sequenceNumbers`" + String.raw` must contain one output sequence per input slot, even when
267
+ an input is a duplicate
268
+ - ` + "`committed`" + String.raw` contains only newly committed entries that still need replay into
269
+ the processing journal / handlers
270
+ - ` + "`entries`" + String.raw` and ` + "`changes`" + String.raw` expose the shared store feed used by all mapped
271
+ public keys
272
+
273
+ A helper layer such as ` + "`layerStorageMemory`" + String.raw` is recommended for tests and local
274
+ workflows.
275
+
276
+ ### 7. ` + "`EventLogServerUnencrypted`" + String.raw` runtime service
277
+
278
+ Add a server runtime service, parallel to the client-side ` + "`EventLog`" + String.raw` runtime,
279
+ responsible for:
280
+
281
+ - registering outbound compactors
282
+ - registering reactivity mappings
283
+ - processing accepted client entries through the server journal + handlers
284
+ - serving store-backed outbound change feeds to client public keys
285
+ - exposing a server-authored write path that targets store ids directly
286
+
287
+ Suggested shape:
288
+
289
+ ```ts
290
+ export class EventLogServerUnencrypted extends ServiceMap.Service<EventLogServerUnencrypted, {
291
+ readonly ingestClient: (options: {
292
+ readonly publicKey: string
293
+ readonly entries: ReadonlyArray<Entry>
294
+ }) => Effect.Effect<{
295
+ readonly storeId: StoreId
296
+ readonly sequenceNumbers: ReadonlyArray<number>
297
+ readonly committed: ReadonlyArray<RemoteEntry>
298
+ }, EventLogServerAuthError | EventLogServerStoreRegistryError | EventJournal.EventJournalError>
299
+ readonly requestChanges: (
300
+ publicKey: string,
301
+ startSequence: number
302
+ ) => Effect.Effect<Queue.Dequeue<RemoteEntry, EventLogRemote.EventLogRemoteError>, never, Scope.Scope>
303
+ readonly writeToStore: (options: {
304
+ readonly storeId: StoreId
305
+ readonly entries: ReadonlyArray<Entry>
306
+ readonly runHandlers?: boolean | undefined
307
+ }) => Effect.Effect<{
308
+ readonly sequenceNumbers: ReadonlyArray<number>
309
+ readonly committed: ReadonlyArray<RemoteEntry>
310
+ }, EventJournal.EventJournalError>
311
+ readonly registerCompaction: (options: {
312
+ readonly events: ReadonlyArray<string>
313
+ readonly olderThan: Duration.DurationInput
314
+ readonly effect: (options: {
315
+ readonly entries: ReadonlyArray<Entry>
316
+ readonly write: (entry: Entry) => Effect.Effect<void>
317
+ }) => Effect.Effect<void>
318
+ }) => Effect.Effect<void, never, Scope.Scope>
319
+ readonly registerReactivity: (keys: Record<string, ReadonlyArray<string>>) => Effect.Effect<void, never, Scope.Scope>
320
+ }>()("effect/eventlog/EventLogServerUnencrypted") {}
321
+ ```
322
+
323
+ #### Required behavior
324
+
325
+ - ` + "`ingestClient(...)`" + String.raw` is the client-originated write path used by the websocket
326
+ handler
327
+ - ` + "`writeToStore(...)`" + String.raw` is the server-authored write path
328
+ - ` + "`writeToStore(...)`" + String.raw` bypasses auth and store resolution because the caller already
329
+ supplies a target store id
330
+ - when ` + "`runHandlers`" + String.raw` is omitted, it should default to ` + "`true`" + String.raw`
331
+ - when ` + "`runHandlers: true`" + String.raw`, committed entries must be replayed into the processing
332
+ journal and must trigger the same handler / reactivity behavior as client
333
+ writes
334
+ - when ` + "`runHandlers: false`" + String.raw`, committed entries are transport-only:
335
+ - they are persisted to storage
336
+ - they are visible to ` + "`RequestChanges`" + String.raw` readers for that store
337
+ - they are **not** replayed into the processing journal
338
+ - they do **not** execute handlers
339
+ - they do **not** trigger reactivity invalidation
340
+ - they do **not** participate in conflict detection until a future feature
341
+ explicitly introduces a second processing mode
342
+
343
+ ### 8. ` + "`layer`" + String.raw`
344
+
345
+ Add a layer constructor, parallel to ` + "`EventLog.layer(...)`" + String.raw`, for building the
346
+ server runtime from handler layers and a processing journal.
347
+
348
+ ```ts
349
+ export const layer: <Groups extends EventGroup.Any>(
350
+ schema: EventLog.EventLogSchema<Groups>
351
+ ) => Layer.Layer<
352
+ EventLogServerUnencrypted,
353
+ never,
354
+ EventGroup.ToService<Groups>
355
+ | EventJournal.EventJournal
356
+ | EventLogServerStoreRegistry
357
+ | EventLogServerAuth
358
+ | Storage
359
+ >
360
+ ```
361
+
362
+ #### Notes
363
+
364
+ - handlers are still authored with ` + "`EventLog.group(...)`" + String.raw`
365
+ - the server runtime should read those handler registrations from the produced
366
+ service map exactly like ` + "`EventLog`" + String.raw` does
367
+ - this keeps domain handler code portable between client and server runtimes
368
+
369
+ ### 9. ` + "`groupCompaction`" + String.raw`
370
+
371
+ Keep the existing compaction helper from the original spec, but clarify that it
372
+ now operates on per-store transport history served through client public keys.
373
+
374
+ ```ts
375
+ export const groupCompaction: <Events extends Event.Any, R>(
376
+ group: EventGroup.EventGroup<Events>,
377
+ options: {
378
+ readonly olderThan: Duration.DurationInput
379
+ },
380
+ effect: (options: {
381
+ readonly primaryKey: string
382
+ readonly entries: ReadonlyArray<Entry>
383
+ readonly events: ReadonlyArray<Event.TaggedPayload<Events>>
384
+ readonly write: <Tag extends Event.Tag<Events>>(
385
+ tag: Tag,
386
+ payload: Event.PayloadWithTag<Events, Tag>
387
+ ) => Effect.Effect<void, never, Event.PayloadSchemaWithTag<Events, Tag>["EncodingServices"]>
388
+ }) => Effect.Effect<void, never, R>
389
+ ) => Layer.Layer<never, never, EventLogServerUnencrypted | R | Event.PayloadSchema<Events>["DecodingServices"]>
390
+ ```
391
+
392
+ Behavior remains the same as the prior spec except that:
393
+
394
+ - eligible entries come from the resolved store feed
395
+ - a compacted response is shared by all public keys reading that store from the
396
+ same cursor
397
+
398
+ ### 10. ` + "`groupReactivity`" + String.raw`
399
+
400
+ Add a server-local helper mirroring ` + "`EventLog.groupReactivity(...)`" + String.raw` so a server
401
+ runtime can register per-event invalidation keys without depending on the
402
+ client-side ` + "`EventLog`" + String.raw` service.
403
+
404
+ ### 11. ` + "`makeHandler`" + String.raw` and ` + "`makeHandlerHttp`" + String.raw`
405
+
406
+ Add websocket / HTTP upgrade handlers parallel to ` + "`EventLogServer.makeHandler`" + String.raw`
407
+ and ` + "`makeHandlerHttp`" + String.raw`.
408
+
409
+ ```ts
410
+ export const makeHandler: Effect.Effect<
411
+ (socket: Socket.Socket) => Effect.Effect<void, Socket.SocketError>,
412
+ never,
413
+ EventLogServerUnencrypted | Storage | EventLogServerAuth | EventLogServerStoreRegistry
414
+ >
415
+
416
+ export const makeHandlerHttp: Effect.Effect<
417
+ Effect.Effect<
418
+ HttpServerResponse.HttpServerResponse,
419
+ HttpServerError.HttpServerError | Socket.SocketError,
420
+ HttpServerRequest.HttpServerRequest | Scope.Scope
421
+ >,
422
+ never,
423
+ EventLogServerUnencrypted | Storage | EventLogServerAuth | EventLogServerStoreRegistry
424
+ >
425
+ ```
426
+
427
+ ## EventLogRemote Changes
428
+
429
+ The existing unencrypted support in ` + "`EventLogRemote.ts`" + String.raw` is incomplete and must
430
+ be finished as part of this work.
431
+
432
+ ### Protocol additions
433
+
434
+ Add an explicit unencrypted error response:
435
+
436
+ ```ts
437
+ export class ErrorUnencrypted extends Schema.Class<ErrorUnencrypted>(
438
+ "effect/eventlog/EventLogRemote/ErrorUnencrypted"
439
+ )({
440
+ _tag: Schema.tag("Error"),
441
+ requestTag: Schema.String,
442
+ id: Schema.optional(Schema.Number),
443
+ publicKey: Schema.optional(Schema.String),
444
+ code: Schema.Literal("Unauthorized", "InvalidRequest", "InternalServerError"),
445
+ message: Schema.String
446
+ }) {}
447
+ ```
448
+
449
+ The exact schema can differ, but it must support:
450
+
451
+ - correlating write failures to a pending ` + "`WriteEntries`" + String.raw` request id
452
+ - indicating read-subscription failures for ` + "`RequestChanges`" + String.raw`
453
+ - communicating auth failures in a machine-readable form
454
+ - communicating store-resolution failures for unmapped public keys
455
+
456
+ Update:
457
+
458
+ - ` + "`ProtocolResponseUnencrypted`" + String.raw`
459
+ - ` + "`ProtocolResponseUnencryptedMsgpack`" + String.raw`
460
+ - ` + "`decodeResponseUnencrypted`" + String.raw`
461
+ - ` + "`encodeResponseUnencrypted`" + String.raw`
462
+
463
+ ### Codec correctness
464
+
465
+ Complete the unfinished unencrypted protocol path in ` + "`EventLogRemote.ts`" + String.raw`:
466
+
467
+ - ` + "`fromSocketUnencrypted`" + String.raw` must use the unencrypted request encoder and response
468
+ decoder consistently
469
+ - the misspelled ` + "`encodeRequestUnsencrypted`" + String.raw` helper should be corrected to
470
+ ` + "`encodeRequestUnencrypted`" + String.raw`
471
+ - if the typo is already exported, keep a temporary backwards-compatible alias
472
+ or deprecate it in-place rather than silently breaking consumers
473
+
474
+ ### Error surfacing
475
+
476
+ Because the server now sends explicit unencrypted protocol errors, the remote
477
+ client API must surface them.
478
+
479
+ #### Required behavior
480
+
481
+ - a write rejection for a pending request must fail the corresponding
482
+ ` + "`remote.write(...)`" + String.raw` effect with ` + "`EventLogRemoteError`" + String.raw`
483
+ - a rejected ` + "`RequestChanges`" + String.raw` subscription must fail the returned dequeue, or an
484
+ equivalent fallible subscription mechanism, with ` + "`EventLogRemoteError`" + String.raw`
485
+ - downstream ` + "`EventLog.registerRemote(...)`" + String.raw` consumption must handle those
486
+ failures by logging and continuing its normal reconnection / retry behavior
487
+
488
+ Since these APIs are unstable, it is acceptable to widen the public error types
489
+ if needed.
490
+
491
+ ## Runtime Architecture
492
+
493
+ ### 1. Separate transport history, store mapping, and processing journal
494
+
495
+ The server has three distinct responsibilities:
496
+
497
+ 1. maintain a per-store transport feed used by ` + "`RequestChanges`" + String.raw`
498
+ 2. resolve client public keys to store ids through a durable-friendly registry
499
+ 3. run handlers / conflicts / reactivity against a backend processing journal
500
+
501
+ Use:
502
+
503
+ - the new unencrypted ` + "`Storage`" + String.raw` service for (1)
504
+ - the new ` + "`EventLogServerStoreRegistry`" + String.raw` service for (2)
505
+ - the existing ` + "`EventJournal`" + String.raw` service for (3)
506
+
507
+ This keeps the design aligned with existing abstractions while allowing store
508
+ broadcast semantics.
509
+
510
+ ### 2. Stable journal origin per store id
511
+
512
+ Accepted client writes and server-authored writes that replay handlers must be
513
+ replayed into the processing journal via ` + "`journal.writeFromRemote(...)`" + String.raw`. That
514
+ requires a stable ` + "`RemoteId`" + String.raw`.
515
+
516
+ The runtime should derive a deterministic ` + "`RemoteId`" + String.raw` from ` + "`StoreId`" + String.raw`, not from
517
+ public key.
518
+
519
+ Recommended approach:
520
+
521
+ - compute a stable 16-byte digest from the UTF-8 encoded store id
522
+ - brand the first 16 bytes as ` + "`RemoteId`" + String.raw`
523
+ - keep this derivation internal to the server runtime
524
+
525
+ This ensures:
526
+
527
+ - all public keys mapped to the same store share the same processing-journal
528
+ remote origin
529
+ - server-authored writes targeting that store participate in the same remote
530
+ sequence space whenever handler replay is enabled
531
+
532
+ ### 3. Client ingest pipeline
533
+
534
+ For each ` + "`WriteEntriesUnencrypted`" + String.raw` request:
535
+
536
+ 1. decode request
537
+ 2. authorize the entire batch via ` + "`EventLogServerAuth.authorizeWrite`" + String.raw`
538
+ 3. resolve ` + "`publicKey -> storeId`" + String.raw` via the store registry
539
+ 4. if auth or store resolution fails, send ` + "`ErrorUnencrypted`" + String.raw` and do not persist
540
+ anything
541
+ 5. persist raw entries into transport ` + "`Storage.write({ storeId, entries })`" + String.raw`
542
+ 6. send ` + "`Ack`" + String.raw` with ` + "`sequenceNumbers.length === request.entries.length`" + String.raw`
543
+ 7. replay only ` + "`committed`" + String.raw` entries into ` + "`EventJournal.writeFromRemote(...)`" + String.raw`
544
+ 8. during replay, run backend handlers and reactivity invalidation exactly once
545
+ per committed entry
546
+ 9. do not compact during ingest
547
+
548
+ ### 4. Server-authored write pipeline
549
+
550
+ For each ` + "`writeToStore({ storeId, entries, runHandlers })`" + String.raw` call:
551
+
552
+ 1. validate the supplied entries and target store id
553
+ 2. persist raw entries into transport ` + "`Storage.write({ storeId, entries })`" + String.raw`
554
+ 3. broadcast those committed entries to all current and future readers of that
555
+ store via the normal ` + "`RequestChanges`" + String.raw` path
556
+ 4. if ` + "`runHandlers`" + String.raw` is ` + "`true`" + String.raw`, replay committed entries into the
557
+ processing journal using the deterministic store remote id
558
+ 5. if ` + "`runHandlers`" + String.raw` is ` + "`false`" + String.raw`, stop after transport persistence and
559
+ do not touch the processing journal
560
+
561
+ This path bypasses ` + "`EventLogServerAuth`" + String.raw` entirely.
562
+
563
+ ### 5. Shared-store visibility semantics
564
+
565
+ Because the feed is keyed by store id:
566
+
567
+ - two public keys assigned to the same store must receive the same event stream
568
+ when reading from the same ` + "`startSequence`" + String.raw`
569
+ - a client write from public key A must be visible to public key B if both map to
570
+ the same store
571
+ - a server-authored write to store S must be visible to all clients mapped to S
572
+ - a newly mapped public key requesting sequence ` + "`0`" + String.raw` for an existing store should
573
+ observe the full store backlog
574
+
575
+ ### 6. Duplicate handling
576
+
577
+ If the same entry id is written again for the same store:
578
+
579
+ - ` + "`Ack.sequenceNumbers`" + String.raw` or equivalent server-write metadata still includes a
580
+ sequence for that input slot
581
+ - the duplicate entry is not re-committed to storage
582
+ - the duplicate entry is not re-run through handlers
583
+ - the duplicate entry does not re-trigger reactivity invalidation
584
+
585
+ Duplicate identity is store-local. The same entry id written to two different
586
+ stores may be treated as distinct transport entries.
587
+
588
+ ### 7. Handler execution semantics
589
+
590
+ Server-side handler execution should match the remote-consumption path already
591
+ implemented inside ` + "`EventLog`" + String.raw`.
592
+
593
+ For each committed entry whose replay path is enabled:
594
+
595
+ - locate the handler via the service key compiled from ` + "`EventLog.group(...)`" + String.raw`
596
+ - decode the payload with the event schema
597
+ - decode conflict payloads with the same schema
598
+ - merge handler-required services into the effect environment
599
+ - execute the handler
600
+ - log handler failures the same way ` + "`EventLog`" + String.raw` does, rather than failing the
601
+ websocket protocol loop
602
+
603
+ Implementation should extract or share the common decode / execute / invalidate
604
+ logic from ` + "`EventLog.ts`" + String.raw` rather than copying it into two places.
605
+
606
+ ### 8. Reactivity semantics
607
+
608
+ Reactivity invalidation must behave the same as accepted remote writes in the
609
+ client-side ` + "`EventLog`" + String.raw` runtime whenever replay is enabled.
610
+
611
+ For each committed entry whose replay path is enabled:
612
+
613
+ - look up registered reactivity keys for ` + "`entry.event`" + String.raw`
614
+ - invalidate ` + "`{ [key]: [entry.primaryKey] }`" + String.raw` for each configured key
615
+ - perform invalidation after successful handler execution, matching current
616
+ ` + "`EventLog`" + String.raw` semantics
617
+
618
+ When ` + "`runHandlers: false`" + String.raw` is used for a server-authored write, reactivity does not
619
+ run.
620
+
621
+ ## Outbound Compaction
622
+
623
+ ### 1. When compaction runs
624
+
625
+ Compaction runs only while serving ` + "`RequestChanges(publicKey, startSequence)`" + String.raw`.
626
+
627
+ - the server first resolves ` + "`publicKey -> storeId`" + String.raw`
628
+ - compaction operates on that store's transport history
629
+ - recent entries remain unmodified in the outbound stream
630
+ - only entries older than the configured threshold are eligible
631
+ - authorization happens before the subscription is created
632
+
633
+ ### 2. Eligibility rule
634
+
635
+ For a compactor registered with ` + "`olderThan`" + String.raw`, an entry is eligible when:
636
+
637
+ ```ts
638
+ entry.createdAtMillis <= now - olderThan
639
+ ```
640
+
641
+ If an entry is newer than the cutoff, it must be sent through unchanged even if
642
+ its event tag belongs to a compaction group.
643
+
644
+ ### 3. Compaction grouping model
645
+
646
+ Use the same high-level grouping semantics as ` + "`EventLog.groupCompaction(...)`" + String.raw`:
647
+
648
+ - only event tags registered by the compactor participate
649
+ - decode payloads with the event schemas from the ` + "`EventGroup`" + String.raw`
650
+ - group the candidate entries by ` + "`primaryKey`" + String.raw`
651
+ - pass ` + "`{ primaryKey, entries, events, write(...) }`" + String.raw` to the compaction effect
652
+ - ` + "`write(...)`" + String.raw` encodes replacement entries back into ` + "`Entry`" + String.raw`
653
+
654
+ ### 4. Sequence assignment for compacted outputs
655
+
656
+ Compaction happens on raw transport history that already has ` + "`remoteSequence`" + String.raw`
657
+ values. Replacement entries sent to clients must preserve cursor monotonicity.
658
+
659
+ Rule:
660
+
661
+ - each compacted output entry must be emitted as a ` + "`RemoteEntry`" + String.raw`
662
+ - its ` + "`remoteSequence`" + String.raw` must be the **maximum** raw sequence consumed for that
663
+ output entry's source bucket
664
+ - when a single primary-key bucket produces multiple replacement entries, each
665
+ emitted replacement entry may share that maximum sequence
666
+
667
+ ### 5. Filtering relative to ` + "`startSequence`" + String.raw`
668
+
669
+ The server must compact first, then apply the outbound sequence filter using the
670
+ representative ` + "`remoteSequence`" + String.raw` of the emitted entries.
671
+
672
+ That means old raw entries below ` + "`startSequence`" + String.raw` may still influence a compacted
673
+ replacement entry whose representative sequence is at or above the cursor.
674
+
675
+ ### 6. Streaming behavior
676
+
677
+ ` + "`requestChanges(publicKey, startSequence)`" + String.raw` should:
678
+
679
+ 1. authorize the read request
680
+ 2. resolve the public key to a store id
681
+ 3. subscribe to raw storage changes for that store
682
+ 4. read the initial backlog plus live updates
683
+ 5. compact the eligible historical portion on each emission batch
684
+ 6. emit recent raw entries unchanged
685
+ 7. preserve monotonic ordering by ` + "`remoteSequence`" + String.raw`
686
+
687
+ A simple and acceptable first implementation may compact the initial backlog and
688
+ then pass through live updates unchanged until they age past the threshold on a
689
+ future subscription.
690
+
691
+ ## Authorization, Store Resolution, and Protocol Errors
692
+
693
+ ### Client write rejection
694
+
695
+ If ` + "`authorizeWrite`" + String.raw` fails:
696
+
697
+ - send ` + "`ErrorUnencrypted`" + String.raw` with a write-correlated request id
698
+ - do not send ` + "`Ack`" + String.raw`
699
+ - do not resolve the store id
700
+ - do not persist transport entries
701
+ - do not replay into the processing journal
702
+ - do not run handlers
703
+ - do not invalidate reactivity
704
+
705
+ ### Client read rejection
706
+
707
+ If ` + "`authorizeRead`" + String.raw` fails:
708
+
709
+ - send ` + "`ErrorUnencrypted`" + String.raw` describing the rejected ` + "`RequestChanges`" + String.raw`
710
+ - do not resolve the store id
711
+ - do not create a subscription
712
+ - do not leak queue resources or a live FiberMap registration
713
+
714
+ ### Store resolution failure
715
+
716
+ If the public key cannot be resolved to a store id:
717
+
718
+ - send ` + "`ErrorUnencrypted`" + String.raw` for the relevant request
719
+ - do not persist transport entries
720
+ - do not create a subscription
721
+ - do not leak resources
722
+ - do not fall back to hashing the public key into an implicit store id
723
+
724
+ ### Internal failures
725
+
726
+ Internal operational failures should be mapped as follows:
727
+
728
+ - expected auth failures -> ` + "`ErrorUnencrypted`" + String.raw`
729
+ - expected store-resolution failures -> ` + "`ErrorUnencrypted`" + String.raw`
730
+ - expected request validation failures in the unencrypted path ->
731
+ ` + "`ErrorUnencrypted`" + String.raw`
732
+ - unexpected defects / decode corruption / chunk reassembly corruption -> log and
733
+ use the existing connection failure behavior unless an explicit protocol error
734
+ can be sent safely
735
+
736
+ ## Persistence Requirements
737
+
738
+ ### 1. Store mapping persistence
739
+
740
+ The publicKey-to-store mapping is durable domain state.
741
+
742
+ Therefore:
743
+
744
+ - the runtime must not derive store ids from public keys on the fly as its only
745
+ mapping strategy
746
+ - the mapping must live behind the required
747
+ ` + "`EventLogServerStoreRegistry`" + String.raw` service
748
+ - the service contract must support implementations backed by durable storage
749
+ - a memory helper is acceptable for tests and local development only
750
+
751
+ ### 2. Transport history persistence
752
+
753
+ This specification still does **not** require ` + "`SqlEventLogJournal`" + String.raw` to also become the
754
+ transport history store used by outbound ` + "`RequestChanges`" + String.raw`.
755
+
756
+ Reason:
757
+
758
+ - transport history is keyed by ` + "`StoreId`" + String.raw`
759
+ - read-time compaction needs direct access to each store's outbound stream
760
+ - keeping the transport feed as a dedicated server storage abstraction keeps the
761
+ protocol-serving path simpler and mirrors the existing encrypted server module
762
+
763
+ ### 3. Relationship to ` + "`SqlEventLogJournal`" + String.raw`
764
+
765
+ ` + "`SqlEventLogJournal`" + String.raw` remains a good fit for the **processing journal** side of
766
+ this feature because it already supports:
767
+
768
+ - ` + "`writeFromRemote(...)`" + String.raw`
769
+ - remote sequence tracking
770
+ - conflict-aware effect execution
771
+ - local journal persistence
772
+
773
+ Implementation should keep the transport-store and store-registry contracts
774
+ narrow enough that future SQL-backed adapters can be added without refactoring
775
+ the runtime.
776
+
777
+ ## Testing Requirements
778
+
779
+ Add focused tests that follow existing repository patterns.
780
+
781
+ ### Core server tests
782
+
783
+ Create ` + "`packages/effect/test/unstable/eventlog/EventLogServerUnencrypted.test.ts`" + String.raw`
784
+ covering at minimum:
785
+
786
+ 1. **client writes run handlers for the resolved store**
787
+ - compose a runtime from ` + "`EventLog.group(...)`" + String.raw`
788
+ - map a public key to a store id
789
+ - write an unencrypted batch through the server handler or runtime ingest path
790
+ - assert handler side effects ran
791
+
792
+ 2. **two public keys mapped to the same store share transport history**
793
+ - assign public keys A and B to the same store
794
+ - write from A
795
+ - request changes from B
796
+ - assert B receives the same entries and sequence numbers
797
+
798
+ 3. **server-authored writes broadcast to mapped clients**
799
+ - assign two public keys to the same store
800
+ - call ` + "`writeToStore({ storeId, ... })`" + String.raw`
801
+ - assert both readers observe the appended entries
802
+
803
+ 4. **server-authored writes bypass auth**
804
+ - provide an auth service that rejects all client traffic
805
+ - call ` + "`writeToStore`" + String.raw`
806
+ - assert the write still succeeds and is observable by mapped clients
807
+
808
+ 5. **server-authored writes with ` + "`runHandlers: true`" + String.raw` run handlers and reactivity**
809
+ - register a handler and Reactivity observer
810
+ - write to the store with replay enabled
811
+ - assert handler side effects and invalidation occur exactly once per
812
+ committed entry
813
+
814
+ 6. **server-authored writes with ` + "`runHandlers: false`" + String.raw` are transport-only**
815
+ - write to the store with replay disabled
816
+ - assert clients receive the events
817
+ - assert handlers do not run
818
+ - assert reactivity does not invalidate
819
+
820
+ 7. **unauthorized client write rejects the full batch**
821
+ - ` + "`authorizeWrite`" + String.raw` fails
822
+ - assert ` + "`ErrorUnencrypted`" + String.raw` is returned
823
+ - assert no ` + "`Ack`" + String.raw`
824
+ - assert storage and handler state remain unchanged
825
+
826
+ 8. **unmapped public key rejects read and write requests**
827
+ - do not assign a store id
828
+ - assert both client write and ` + "`RequestChanges`" + String.raw` fail with the expected
829
+ protocol error
830
+
831
+ 9. **duplicate writes are idempotent but still ack every input slot**
832
+ - write the same entries twice into the same store
833
+ - assert the second ack has the same number of sequence numbers as inputs
834
+ - assert handlers/reactivity only run once per unique entry id
835
+
836
+ 10. **read-time compaction only affects old entries in a shared store feed**
837
+ - store an old history batch and a recent batch
838
+ - assert only the old portion is compacted
839
+ - assert recent entries pass through raw
840
+
841
+ 11. **compacted outputs preserve cursor progression across public keys mapped to the same store**
842
+ - request from sequence ` + "`0`" + String.raw` and from a later cursor through different keys
843
+ - assert representative output sequences are monotonic and compatible with
844
+ follow-up requests
845
+
846
+ ### Remote client tests
847
+
848
+ Add or update tests around ` + "`EventLogRemote`" + String.raw` to verify:
849
+
850
+ 1. ` + "`fromSocketUnencrypted`" + String.raw` uses the unencrypted codec path
851
+ 2. ` + "`ErrorUnencrypted`" + String.raw` fails pending writes with ` + "`EventLogRemoteError`" + String.raw`
852
+ 3. rejected read subscriptions surface an ` + "`EventLogRemoteError`" + String.raw`
853
+ 4. store-resolution failures surface as the expected unencrypted protocol error
854
+ 5. chunked unencrypted messages round-trip correctly
855
+
856
+ ## Validation Expectations
857
+
858
+ Any implementation produced from this spec must run:
859
+
860
+ - ` + "`pnpm lint-fix`" + String.raw`
861
+ - targeted tests, at minimum:
862
+ - ` + "`pnpm test packages/effect/test/unstable/eventlog/EventLogServerUnencrypted.test.ts`" + String.raw`
863
+ - any updated ` + "`EventLog`" + String.raw` / ` + "`EventLogRemote`" + String.raw` tests
864
+ - ` + "`pnpm check:tsgo`" + String.raw`
865
+ - ` + "`pnpm docgen`" + String.raw`
866
+ - ` + "`pnpm codegen`" + String.raw` if new module exports or barrel changes are introduced
867
+ - add a changeset for the unstable eventlog package changes
868
+
869
+ ## Implementation Plan
870
+
871
+ The work should be broken into the following atomic, validation-safe tasks.
872
+ Tasks are intentionally grouped so each step can pass linting, tests, and type
873
+ checking on its own.
874
+
875
+ ### Task 1: Finish unencrypted protocol plumbing in ` + "`EventLogRemote`" + String.raw`
876
+
877
+ Scope:
878
+
879
+ - add ` + "`ErrorUnencrypted`" + String.raw` to the protocol model
880
+ - fix unencrypted encode/decode usage in ` + "`fromSocketUnencrypted`" + String.raw`
881
+ - correct the misspelled unencrypted request encoder export
882
+ - plumb protocol errors into pending write handling and read-subscription
883
+ handling
884
+ - update/add focused remote protocol tests, including store-resolution failures
885
+
886
+ Why this task is grouped:
887
+
888
+ - protocol message additions and client-side handling must land together or the
889
+ code will not typecheck and tests will fail
890
+ - this task is independently shippable because it only completes the unfinished
891
+ unencrypted client transport behavior
892
+
893
+ Validation for this task:
894
+
895
+ - targeted ` + "`EventLogRemote`" + String.raw` tests
896
+ - any affected ` + "`EventLog`" + String.raw` tests if public remote signatures widen
897
+
898
+ ### Task 2: Add store-id, registry, auth, and transport-storage foundations
899
+
900
+ Scope:
901
+
902
+ - define ` + "`StoreId`" + String.raw`, ` + "`EventLogServerStoreRegistry`" + String.raw`, and
903
+ ` + "`EventLogServerStoreRegistryError`" + String.raw`
904
+ - define ` + "`EventLogServerAuth`" + String.raw` and ` + "`EventLogServerAuthError`" + String.raw`
905
+ - define store-keyed ` + "`Storage`" + String.raw`, ` + "`layerStorageMemory`" + String.raw`, and a memory
906
+ store-registry helper
907
+ - implement duplicate-aware write results with one ack sequence per input entry
908
+ - add focused registry / storage / auth tests
909
+
910
+ Why this task is grouped:
911
+
912
+ - store resolution, auth, and per-store transport sequencing are inseparable
913
+ foundations for every later runtime behavior
914
+ - splitting duplicate-aware sequencing away from storage would create failing
915
+ intermediate states
916
+
917
+ Validation for this task:
918
+
919
+ - new registry / storage / auth tests
920
+ - compile checks for the new public APIs
921
+
922
+ ### Task 3: Build the store-aware client ingest runtime and handler/reactivity integration
923
+
924
+ Scope:
925
+
926
+ - add ` + "`EventLogServerUnencrypted`" + String.raw` runtime service and ` + "`layer(...)`" + String.raw`
927
+ - wire accepted client writes through the store registry, transport storage, and
928
+ processing journal using deterministic store-derived remote ids
929
+ - reuse ` + "`EventLog.group(...)`" + String.raw`-compiled handlers
930
+ - extract or share common handler execution / reactivity helpers from
931
+ ` + "`EventLog.ts`" + String.raw`
932
+ - add ` + "`groupReactivity`" + String.raw`
933
+ - add tests proving handlers and reactivity fire exactly once per committed
934
+ client entry
935
+ - add tests proving two public keys mapped to the same store share history
936
+
937
+ Why this task is grouped:
938
+
939
+ - handler execution, deterministic store identity, and shared-feed semantics all
940
+ depend on the same ingest path
941
+ - the resulting runtime is independently shippable even before server-authored
942
+ writes are exposed
943
+
944
+ Validation for this task:
945
+
946
+ - new server runtime tests
947
+ - existing ` + "`EventLog`" + String.raw` tests to ensure shared refactors do not regress client
948
+ behavior
949
+
950
+ ### Task 4: Add server-authored store writes with configurable replay
951
+
952
+ Scope:
953
+
954
+ - add ` + "`writeToStore(...)`" + String.raw` or the equivalent public server-authored write API
955
+ - bypass auth for that path
956
+ - support ` + "`runHandlers: true | false`" + String.raw` semantics
957
+ - ensure transport-only writes are still broadcast to clients
958
+ - add tests for broadcast visibility, auth bypass, replay-enabled behavior, and
959
+ replay-disabled behavior
960
+
961
+ Why this task is grouped:
962
+
963
+ - the write API, replay switch, and broadcast tests must land together to avoid
964
+ ambiguous intermediate behavior
965
+ - this task is independently shippable once the client ingest runtime exists
966
+
967
+ Validation for this task:
968
+
969
+ - focused ` + "`EventLogServerUnencrypted`" + String.raw` tests for server-authored writes
970
+ - any affected ` + "`EventLog`" + String.raw` tests if shared helpers change again
971
+
972
+ ### Task 5: Add read-time compaction over store feeds
973
+
974
+ Scope:
975
+
976
+ - add server-side ` + "`groupCompaction(...)`" + String.raw` with ` + "`olderThan`" + String.raw`
977
+ - implement compaction of outbound initial backlog / eligible batches for a
978
+ resolved store feed
979
+ - preserve representative remote sequences for cursor compatibility across all
980
+ public keys mapped to the same store
981
+ - add tests for old-vs-recent eligibility and cursor progression
982
+
983
+ Why this task is grouped:
984
+
985
+ - helper registration, runtime projection logic, and cursor-preservation tests
986
+ must land together to avoid broken feed semantics
987
+ - it is independently shippable once store-aware ingest and server writes exist
988
+
989
+ Validation for this task:
990
+
991
+ - server compaction tests
992
+ - existing event log compaction tests, if any shared helper extraction affects
993
+ client behavior
994
+
995
+ ### Task 6: Expose websocket / HTTP handlers and complete integration coverage
996
+
997
+ Scope:
998
+
999
+ - implement ` + "`makeHandler`" + String.raw` and ` + "`makeHandlerHttp`" + String.raw`
1000
+ - connect auth, store resolution, storage, runtime, chunk handling, ack/error
1001
+ responses, and change subscriptions
1002
+ - add end-to-end websocket-level tests for authorized writes, unmapped-key
1003
+ errors, server-authored broadcast behavior, and change subscriptions
1004
+ - regenerate exports if necessary and add the required changeset
1005
+
1006
+ Why this task is grouped:
1007
+
1008
+ - the transport handlers are the first point where all preceding contracts meet;
1009
+ partial integration would leave validation broken
1010
+ - this task yields the complete user-facing module
1011
+
1012
+ Validation for this task:
1013
+
1014
+ - end-to-end ` + "`EventLogServerUnencrypted`" + String.raw` tests
1015
+ - ` + "`pnpm lint-fix`" + String.raw`
1016
+ - ` + "`pnpm check:tsgo`" + String.raw`
1017
+ - ` + "`pnpm docgen`" + String.raw`
1018
+ - ` + "`pnpm codegen`" + String.raw` if needed
1019
+
1020
+ ## Open Follow-Ups
1021
+
1022
+ These are explicitly out of scope for the initial implementation but should be
1023
+ kept in mind while designing the API:
1024
+
1025
+ - a SQL-backed transport storage implementation for unencrypted server history
1026
+ - a SQL-backed ` + "`EventLogServerStoreRegistry`" + String.raw` helper or companion module
1027
+ - optional success acknowledgements for ` + "`RequestChanges`" + String.raw` subscriptions
1028
+ - extending explicit protocol error frames to the encrypted server path
1029
+ - richer typed convenience APIs for server-authored writes so callers do not
1030
+ need to manually construct ` + "`Entry`" + String.raw` values when that ergonomics becomes
1031
+ important
1032
+ `