@kairos-es/read 0.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (82) hide show
  1. package/LICENSE +28 -0
  2. package/README.md +538 -0
  3. package/dist/cjs/EventLogDurability.js +184 -0
  4. package/dist/cjs/EventLogDurability.js.map +1 -0
  5. package/dist/cjs/ProjectionRunner.js +478 -0
  6. package/dist/cjs/ProjectionRunner.js.map +1 -0
  7. package/dist/cjs/ProjectionStore.js +233 -0
  8. package/dist/cjs/ProjectionStore.js.map +1 -0
  9. package/dist/cjs/foldIntoRef.js +36 -0
  10. package/dist/cjs/foldIntoRef.js.map +1 -0
  11. package/dist/cjs/inMemoryProjectionStore.js +138 -0
  12. package/dist/cjs/inMemoryProjectionStore.js.map +1 -0
  13. package/dist/cjs/index.js +128 -0
  14. package/dist/cjs/index.js.map +1 -0
  15. package/dist/cjs/projectionWiringFault.js +532 -0
  16. package/dist/cjs/projectionWiringFault.js.map +1 -0
  17. package/dist/cjs/runProjection.js +117 -0
  18. package/dist/cjs/runProjection.js.map +1 -0
  19. package/dist/cjs/runProjections.js +144 -0
  20. package/dist/cjs/runProjections.js.map +1 -0
  21. package/dist/cjs/superviseOnProgress.js +580 -0
  22. package/dist/cjs/superviseOnProgress.js.map +1 -0
  23. package/dist/cjs/testing.js +143 -0
  24. package/dist/cjs/testing.js.map +1 -0
  25. package/dist/dts/EventLogDurability.d.ts +182 -0
  26. package/dist/dts/EventLogDurability.d.ts.map +1 -0
  27. package/dist/dts/ProjectionRunner.d.ts +557 -0
  28. package/dist/dts/ProjectionRunner.d.ts.map +1 -0
  29. package/dist/dts/ProjectionStore.d.ts +475 -0
  30. package/dist/dts/ProjectionStore.d.ts.map +1 -0
  31. package/dist/dts/foldIntoRef.d.ts +39 -0
  32. package/dist/dts/foldIntoRef.d.ts.map +1 -0
  33. package/dist/dts/inMemoryProjectionStore.d.ts +11 -0
  34. package/dist/dts/inMemoryProjectionStore.d.ts.map +1 -0
  35. package/dist/dts/index.d.ts +185 -0
  36. package/dist/dts/index.d.ts.map +1 -0
  37. package/dist/dts/projectionWiringFault.d.ts +260 -0
  38. package/dist/dts/projectionWiringFault.d.ts.map +1 -0
  39. package/dist/dts/runProjection.d.ts +185 -0
  40. package/dist/dts/runProjection.d.ts.map +1 -0
  41. package/dist/dts/runProjections.d.ts +480 -0
  42. package/dist/dts/runProjections.d.ts.map +1 -0
  43. package/dist/dts/superviseOnProgress.d.ts +587 -0
  44. package/dist/dts/superviseOnProgress.d.ts.map +1 -0
  45. package/dist/dts/testing.d.ts +207 -0
  46. package/dist/dts/testing.d.ts.map +1 -0
  47. package/dist/esm/EventLogDurability.js +175 -0
  48. package/dist/esm/EventLogDurability.js.map +1 -0
  49. package/dist/esm/ProjectionRunner.js +468 -0
  50. package/dist/esm/ProjectionRunner.js.map +1 -0
  51. package/dist/esm/ProjectionStore.js +223 -0
  52. package/dist/esm/ProjectionStore.js.map +1 -0
  53. package/dist/esm/foldIntoRef.js +29 -0
  54. package/dist/esm/foldIntoRef.js.map +1 -0
  55. package/dist/esm/inMemoryProjectionStore.js +131 -0
  56. package/dist/esm/inMemoryProjectionStore.js.map +1 -0
  57. package/dist/esm/index.js +185 -0
  58. package/dist/esm/index.js.map +1 -0
  59. package/dist/esm/package.json +4 -0
  60. package/dist/esm/projectionWiringFault.js +524 -0
  61. package/dist/esm/projectionWiringFault.js.map +1 -0
  62. package/dist/esm/runProjection.js +109 -0
  63. package/dist/esm/runProjection.js.map +1 -0
  64. package/dist/esm/runProjections.js +137 -0
  65. package/dist/esm/runProjections.js.map +1 -0
  66. package/dist/esm/superviseOnProgress.js +571 -0
  67. package/dist/esm/superviseOnProgress.js.map +1 -0
  68. package/dist/esm/testing.js +133 -0
  69. package/dist/esm/testing.js.map +1 -0
  70. package/package.json +41 -0
  71. package/src/EventLogDurability.ts +201 -0
  72. package/src/ProjectionRunner.ts +923 -0
  73. package/src/ProjectionStore.ts +528 -0
  74. package/src/foldIntoRef.ts +63 -0
  75. package/src/inMemoryProjectionStore.ts +163 -0
  76. package/src/index.ts +218 -0
  77. package/src/projectionWiringFault.ts +694 -0
  78. package/src/runProjection.ts +270 -0
  79. package/src/runProjections.ts +623 -0
  80. package/src/superviseOnProgress.ts +897 -0
  81. package/src/testing.ts +290 -0
  82. package/testing/package.json +6 -0
package/LICENSE ADDED
@@ -0,0 +1,28 @@
1
+ BSD 3-Clause License
2
+
3
+ Copyright (c) 2026, Neverbland
4
+
5
+ Redistribution and use in source and binary forms, with or without
6
+ modification, are permitted provided that the following conditions are met:
7
+
8
+ 1. Redistributions of source code must retain the above copyright notice, this
9
+ list of conditions and the following disclaimer.
10
+
11
+ 2. Redistributions in binary form must reproduce the above copyright notice,
12
+ this list of conditions and the following disclaimer in the documentation
13
+ and/or other materials provided with the distribution.
14
+
15
+ 3. Neither the name of the copyright holder nor the names of its
16
+ contributors may be used to endorse or promote products derived from
17
+ this software without specific prior written permission.
18
+
19
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
package/README.md ADDED
@@ -0,0 +1,538 @@
1
+ # @kairos-es/read
2
+
3
+ The kairos-es read side: a query-driven projection runner and the
4
+ `ProjectionStore` checkpoint port with its in-memory implementation (ADR-0007).
5
+
6
+ Platform-neutral — `effect`, `@kairos-es/core` and `@kairos-es/codec` are peer
7
+ dependencies and nothing else, so the all-in-memory permutation — a property of one
8
+ materialisation's (log, view) pairing — needs no database. The durable
9
+ `ProjectionStore` lives in `@kairos-es/read-postgres`.
10
+
11
+ Everything here is the read side and only the read side. `retryOnConflict`, the
12
+ decider-level retry for `AppendConditionFailed` (ADR-0008), is a **write**-side
13
+ combinator and lives in [`@kairos-es/core`](../core) beside the error it matches,
14
+ so retrying your own appends never means depending on this package.
15
+
16
+ ## Wiring a read model
17
+
18
+ A read model is a view store bound to the `CheckpointKey` it advances
19
+ (`forKey(store, key)`), the codec slices whose union query drives its
20
+ subscription, and a per-batch `apply`. `runProjection` forks the daemon into the
21
+ enclosing `Scope`; `projectionLayer` is the same thing as a `Layer` whose scope
22
+ owns it. Both also require an `EventLogDurability` declaration, provided **once**
23
+ beside the `DcbEventStore` layer.
24
+
25
+ ```ts
26
+ import { event, projection, SerializerDefault } from '@kairos-es/codec'
27
+ import {
28
+ composeProjections,
29
+ DcbEventStoreInMemory,
30
+ tagIdentity,
31
+ } from '@kairos-es/core'
32
+ import {
33
+ checkpointKey,
34
+ EphemeralEventLog,
35
+ foldIntoRef,
36
+ forKey,
37
+ makeInMemoryProjectionStore,
38
+ ProjectionId,
39
+ projectionLayer,
40
+ runProjection,
41
+ } from '@kairos-es/read'
42
+ import { Effect, Layer, Ref, Schema } from 'effect'
43
+
44
+ const CourseId = tagIdentity('course')
45
+
46
+ const SeatTaken = event('SeatTaken', { courseId: Schema.String }, (payload) => [
47
+ CourseId.make(payload.courseId),
48
+ ])
49
+
50
+ // Tag-narrowed, never type-only: every read-model slice carries at least one tag,
51
+ // so the composed subscription's union query stays servable.
52
+ const slices = {
53
+ seats: projection<number>(0)
54
+ .on(SeatTaken, (count) => count + 1)
55
+ .build({ tags: [CourseId.make('c1')] }),
56
+ }
57
+
58
+ // The read model, plus the view it materialises into. The view is returned
59
+ // alongside it because a read model is queried through its VIEW, never through
60
+ // the runner — `projectionLayer` deliberately provides nothing.
61
+ const makeCourseSeats = Effect.gen(function* () {
62
+ const store = yield* makeInMemoryProjectionStore
63
+ const view = yield* Ref.make(composeProjections(slices).initialState)
64
+ return {
65
+ view,
66
+ readModel: {
67
+ // `ProjectionStore` is multi-key, because one store instance serves many
68
+ // read models. `forKey` binds it to this one's checkpoint, so `commit` has
69
+ // no key argument left to get wrong — and the read model carries the pair
70
+ // rather than two fields that could disagree.
71
+ store: forKey(store, checkpointKey(ProjectionId.make('course-seats'))),
72
+ slices,
73
+ // The folded-state shape: fold the batch purely, then ONE write. This is
74
+ // what a non-transactional view store's contract requires.
75
+ apply: foldIntoRef(slices, view),
76
+ },
77
+ }
78
+ })
79
+
80
+ // `EphemeralEventLog` sits BESIDE the store layer, because what it declares is a
81
+ // fact about the LOG, not about any one read model: `DcbEventStore` exposes no
82
+ // durability, so the application that chose the layer is the only honest source,
83
+ // and one declaration here satisfies every runner in the graph. Swap it for
84
+ // `DurableEventLog` when the log is.
85
+ const inMemory = Layer.mergeAll(
86
+ DcbEventStoreInMemory,
87
+ EphemeralEventLog,
88
+ SerializerDefault,
89
+ )
90
+
91
+ // Either fork the daemon into a scope you own — catch-up starts at once, and
92
+ // closing the scope interrupts it and waits for it...
93
+ const program = Effect.scoped(
94
+ Effect.gen(function* () {
95
+ const { readModel, view } = yield* makeCourseSeats
96
+ yield* runProjection(readModel)
97
+ // ...append through `DcbEventStore` and let the runner catch up, then read
98
+ // the materialised view.
99
+ return yield* Ref.get(view)
100
+ }),
101
+ ).pipe(Effect.provide(inMemory))
102
+
103
+ // ...or let a Layer own it, so the daemon starts when the application's layer
104
+ // graph is built and is interrupted when the graph is released.
105
+ //
106
+ // `onStalled` is wired here rather than left out, because this shape DISCARDS the
107
+ // runner: `projectionLayer` provides nothing, so there is no fibre to await and a
108
+ // `ProjectionStalled` would otherwise reach you only as a log line. The restart is
109
+ // the metric; the stall is the page.
110
+
111
+ // Read by this process's readiness probe.
112
+ const stalledProjections = new Set<string>()
113
+
114
+ const courseSeats = Layer.unwrapEffect(
115
+ Effect.map(makeCourseSeats, ({ readModel }) =>
116
+ projectionLayer(readModel, {
117
+ // Whatever "a human must look at this" means in your deployment: fail a
118
+ // health check, exit non-zero, fire a pager. Infallible by signature, so it
119
+ // can neither fail the daemon nor widen its requirements.
120
+ onStalled: ({ key }) =>
121
+ Effect.sync(() => stalledProjections.add(key.projection)),
122
+ }),
123
+ ),
124
+ )
125
+
126
+ const main = Layer.launch(Layer.provide(courseSeats, inMemory))
127
+ ```
128
+
129
+ ## Testing a read model: `@kairos-es/read/testing`
130
+
131
+ A projection is a **daemon**, so `runProjection` returns as soon as the fibre is
132
+ forked — which is the only honest thing it can do, but it means every test of a
133
+ read model needs one more step: *run this until it has consumed the log, then
134
+ assert the view*. A published subpath ships that step, so nobody has to invent it
135
+ (and nobody has to guess a `sleep`, which is wrong when it is short and expensive
136
+ when it is long).
137
+
138
+ ```ts
139
+ import { runProjectionUntil, storedCheckpoint } from '@kairos-es/read/testing'
140
+
141
+ const { readModel, view } = yield* makeCourseSeats
142
+ const positions = yield* appendSomeEvents
143
+
144
+ // Fork the daemon, then wait until the stored checkpoint reaches that position.
145
+ // Returns the `ProjectionRunner`, so `Fiber.poll`/`Fiber.interrupt` stay available.
146
+ yield* runProjectionUntil(readModel, lastPosition(positions), {
147
+ // The runner's own tuning and the wait's, in ONE flat object.
148
+ batchSize: BatchSize.make(2),
149
+ batchWindow: '5 millis',
150
+ timeout: '20 seconds',
151
+ })
152
+
153
+ expect(yield* Ref.get(view)).toEqual(/* … */)
154
+ expect(yield* storedCheckpoint(readModel.store)).toBe(lastPosition(positions))
155
+ ```
156
+
157
+ `awaitCheckpoint(readModel.store, target, options?)` is the wait on its own, for a
158
+ projection started some other way (`projectionLayer`, a scope you own). It POLLS
159
+ the stored checkpoint rather than sleeping, so a slow machine takes longer instead
160
+ of failing, and a genuine hang becomes a failed assertion naming the position that
161
+ was never reached. `storedCheckpoint` reads it once, and `lastPosition` turns what
162
+ `append` reported into a target, loudly, rather than yielding `undefined` from an
163
+ empty fixture.
164
+
165
+ `awaitCheckpoint` and `storedCheckpoint` take the **keyed** store a read model
166
+ already carries (`readModel.store`) and `runProjectionUntil` takes the read model
167
+ itself, so there is no key to pass and none to transpose; `lastPosition` is a pure
168
+ function over what `append` returned, and takes no store at all. It adds no peer
169
+ dependency — no test framework, no `expect`, just `Effect`s (and that one pure
170
+ function) you run under whatever runner you already have — which is why it is a
171
+ subpath of this package rather than a package of its own. It is deliberately a
172
+ **testing** surface: there is no lag reporting and no production "wait until my
173
+ write is projected" hook, because a read model is eventually consistent by
174
+ construction.
175
+
176
+ ## One store, many read models — one read model, one key per store
177
+
178
+ `ProjectionStore` is **multi-key**: `readCheckpoint(key)`, `resetCheckpoint(key)`
179
+ and `commit({ key, … })`, because one store instance (one `SqlClient`, one pool)
180
+ serves however many read models an application runs, and keeping their keys
181
+ independent is contract behaviour the shared suite proves against every backend.
182
+
183
+ A read model has exactly **one** key, so it is given `forKey(store, key)`: the
184
+ same three operations with the key already supplied, so there is none to
185
+ transpose, omit, or read from the wrong variable. `ReadModel` carries that bound
186
+ store **instead of** a `key` field, which is what turns "one materialisation, one
187
+ key, one runner" from a convention every wiring happened to follow into something a
188
+ wiring cannot express otherwise. The key is still readable as `store.key` — the
189
+ runner's log annotations and its `ProjectionStalled` payload name it.
190
+
191
+ A key is unique **within one store**, which is a consequence of R1 rather than a
192
+ policy: the checkpoint is co-located with its view so the pair commits atomically,
193
+ so a key is only ever resolved against the store holding it and there is no global
194
+ namespace to collide in. That is the direction this heading did not use to name.
195
+ ONE read model materialised in several stores carries ONE `ProjectionId` across
196
+ them — that shared id is what its single identity means, and it is what the
197
+ graduation case asserts as its acceptance criterion. Distinctness is forced only
198
+ where two materialisations of one read model cohabit a store, which is what
199
+ `PartitionId` is for: two of them in one store under one key would trade cursors,
200
+ each advancing the other's, and the wiring gate refuses that pairing at
201
+ construction. That rung is the one only `runProjections` can reach, a single
202
+ materialisation having no sibling to collide with.
203
+
204
+ `forKey` is a **free function over the port**, not a method on it: one
205
+ implementation every backend gets for nothing, a port that stays three methods,
206
+ and — the sharp reason — a store **decorator** (`{ ...inner, commit: … }`) cannot
207
+ copy a `forKey` that closed over the undecorated store and silently route past
208
+ its own decoration.
209
+
210
+ Every read-model slice must carry at least one tag (a `system:…` tag for a
211
+ genuinely global slice), so a composed subscription's union query stays servable.
212
+ The wiring gate **checks this at construction**, for either entry point, and the
213
+ wiring is refused as a defect naming the convention — including for a lone tagless
214
+ slice, which the store's own servable grammar would
215
+ accept (a single type-only item is the indexed fast path) but which breaks the day
216
+ a second, tagged slice is composed with it.
217
+
218
+ The same wiring over a real domain, with the events coming from the write side's
219
+ own deciders rather than from hand-assembled fixtures, is
220
+ [`examples/course-subscriptions/src/courseRoster.ts`](../../examples/course-subscriptions/src/courseRoster.ts) —
221
+ a course's capacity, roster and seats remaining, run and resumed from its
222
+ checkpoint in
223
+ [`test/courseRoster.spec.ts`](../../examples/course-subscriptions/test/courseRoster.spec.ts).
224
+
225
+ ## Graduating a slice from memory to Postgres
226
+
227
+ Because the view store is a **value** selected per MATERIALISATION, a slice that
228
+ started in an in-memory view graduates to a durable one with its **slices, its
229
+ composed query, its `CheckpointKey` and its runner untouched**. Two things change
230
+ and only two: the `store` it is handed, and the `apply` that writes through it —
231
+ `foldIntoRef` becomes the view's own statements, which also widens the read
232
+ model's `E` from `never` to whatever those writes can fail with. The runner is
233
+ generic in that `E` and needs no retuning; a fault from `apply` travels through
234
+ `commit` to the supervisor, which is what makes "retry with backoff" the response
235
+ to a flaky view store.
236
+
237
+ That is demonstrated rather than asserted, and it is the same roster both ways.
238
+ [`examples/course-subscriptions/src/courseRosterSql.ts`](../../examples/course-subscriptions/src/courseRosterSql.ts)
239
+ re-points that example's roster at Postgres — importing its slices, its key
240
+ derivation and its pure `courseRosterOf` verbatim — and is the repo's worked SQL
241
+ `apply`, hybrid on purpose: the capacity row's whole batch coalesces into one
242
+ upsert, while each seat is its own row and its own insert.
243
+ [`read-postgres/test/courseRosterGraduation.test.ts`](../read-postgres/test/courseRosterGraduation.test.ts)
244
+ then runs that one slice record into a `Ref` and into Postgres **side by side over
245
+ one durable log** — legal permutations 2 and 3 at once — and asserts the two
246
+ composed queries are equal, the two checkpoint keys are equal, and the two rosters
247
+ agree field for field.
248
+
249
+ So that pair demonstrates two things rather than one. Graduation is the sequential
250
+ reading — memory *then* Postgres, with the `store` and the `apply` the only things
251
+ that change. Running both **at once** is the other, and it is the section below. The
252
+ two wirings live in the example, which is where a consumer reads worked code and
253
+ where `courseRoster.ts` still needs no database at all; what lives in
254
+ `@kairos-es/read-postgres`'s test directory is the COMPARISON, because running that
255
+ needs a container and a live Postgres log.
256
+
257
+ ## Several materialisations of one read model
258
+
259
+ Per-read-model view-store selection is a **floor rather than a ceiling**. One slice
260
+ record can feed a durable Postgres materialisation for the big queryable read model
261
+ and an ephemeral in-memory one for hot in-process reads, at the same time, over one
262
+ log.
263
+
264
+ R1 and R2 fix the shape: one atomic `commit` cannot span two stores, and one
265
+ checkpoint cannot carry two durability classes, so this is N checkpoints, N runners
266
+ and N subscriptions. They do not contend — each owns its checkpoint in its own
267
+ store, and the guarded compare-and-set arbitrates without any lease. Nor are they
268
+ shards: no stream is divided and no work is shared, so the sharding of one read
269
+ model that ADR-0007 refuses is not what this is.
270
+
271
+ `runProjections({ slices, projection, materialisations })` wires it, holding one
272
+ `slices` value and one `ProjectionId` for the whole call so the subscription query
273
+ and the decode table cannot diverge across materialisations. Each entry — a
274
+ `Materialisation` — keeps its own **unbound** `store`, its own `apply` and its own
275
+ tuning and supervision hooks, and the call applies `forKey` per entry itself. The
276
+ guarantee is exactly **one query, one decode, N applies**. What it does not hold is
277
+ the folds: a SQL `apply` re-implements the fold rather than reusing `foldIntoRef`,
278
+ so agreement between materialisations is something a test demonstrates rather than
279
+ something the type system gives you.
280
+
281
+ One consequence surprises, so it is worth stating plainly: keys are store-local, so
282
+ under one `ProjectionId` and the default partition the N materialisations'
283
+ `CheckpointKey`s are **identical**. That is the intended shape — a key names a
284
+ cursor *within* a store — and the discriminator is the store. Each runner carries
285
+ its own, so `runners[i].store` is entry *i*'s cursor and
286
+ `awaitCheckpoint(runners[i].store, target)` observes how far that one materialisation
287
+ has got, with no `forKey` for the caller to re-derive. What the store cannot reach is
288
+ a **hook**: `onRestart`/`onStalled` are handed a payload rather than a handle, and
289
+ `info.key` does not tell the entries apart, so which entry's hook fired is the
290
+ discriminator there — which is why the tuning and the hooks are per entry rather than
291
+ per call.
292
+
293
+ Two materialisations that would share one store **under the same resolved
294
+ `CheckpointKey`** are a **construction-time fault** — both conditions, since
295
+ sharing a store is perfectly legal under distinct partitions and equal keys across
296
+ DIFFERENT stores are the intended shape of a read model materialised twice. The fix
297
+ is usually that they were never two materialisations: several views in one store
298
+ want ONE MATERIALISATION whose single `apply` writes them all inside the one
299
+ `commit`, under one checkpoint. Use a distinct partition only where two
300
+ materialisations genuinely must cohabit a store.
301
+
302
+ What the gate cannot see is the other side of the same mistake: two entries whose
303
+ `apply`s write **the same view**. Different stores satisfy every rule it has, and
304
+ the view is opaque inside `apply`, so nothing refuses it — and both runners then
305
+ fold the whole stream into that one target under independent checkpoints, applying
306
+ every event twice. **One materialisation owns one view target**, and that one is on
307
+ you rather than on the gate. Every other rule the read side enforces runs in the
308
+ **same pass** — two of them per entry, two once over the shared query and slice
309
+ record — so a call refused for any reason at all has forked nothing, not even the
310
+ entries that were sound.
311
+
312
+ And if a read model is small enough to fold per query, consider not maintaining it
313
+ at all. `modelAtHead` from `@kairos-es/codec` reads the log at query time and is
314
+ consistent as of the `head` its read returned, which makes it MORE current than any
315
+ maintained view of the same slices — a maintained view is consistent as of a
316
+ checkpoint that trails that head. No table, no checkpoint, no runner, and no rung
317
+ of the construction gate applies to it.
318
+
319
+ All three over one course's roster:
320
+ [`examples/course-subscriptions/src/courseRosterMaterialisations.ts`](../../examples/course-subscriptions/src/courseRosterMaterialisations.ts),
321
+ whose
322
+ [`test/courseRosterMaterialisations.spec.ts`](../../examples/course-subscriptions/test/courseRosterMaterialisations.spec.ts)
323
+ is what establishes that the three folds agree — empirically, by comparing the
324
+ materialised figures, because nothing structural gives it.
325
+
326
+ ## Supervision: retry while the checkpoint moves
327
+
328
+ The runner's supervisor is a module of its own — `superviseOnProgress`, which is
329
+ **package-internal and deliberately not exported**: it is this runner's supervisor,
330
+ not a general combinator on offer, and its signature says so (a `CheckpointKey`, a
331
+ `Position`, a `ProjectionStoreError`). What you tune is exported
332
+ (`ProjectionSupervisionOptions`, `MaxNoProgressRestarts`) and what you catch is
333
+ exported (`ProjectionStalled`); the mechanism between them is ours to change.
334
+
335
+ What it does is **retry an effect while an externally observed progress signal keeps
336
+ moving, and give up with `ProjectionStalled` when it stops.** For a projection the
337
+ signal is the stored checkpoint, re-read through `readCheckpoint` after every
338
+ failure. It is separate from the runner because supervision is the hardest thing
339
+ here to test through the whole pipeline — it knows nothing of `subscribe`, decoding,
340
+ batching or committing, so every subtle part of it is pinned against a stub progress
341
+ signal instead of an event store, a rigged view store and a hand-driven clock loop.
342
+
343
+ It splits two decisions that are usually conflated. The **schedule** decides how
344
+ fast to restart — a capped, jittered exponential that resets after
345
+ `restartResetAfter`, so a sustained outage sawtooths back to `restartMinDelay`
346
+ rather than pinning at the ceiling. The **circuit-breaker** decides whether to
347
+ keep restarting, by re-reading `progress` after every failure and counting only
348
+ the *consecutive* restarts that moved it not at all.
349
+
350
+ Keying on the signal rather than on a failure count is what makes a competing
351
+ writer behave: two processes accidentally maintaining one read model both lose
352
+ their guard on every batch, but the winner keeps advancing the shared checkpoint,
353
+ so every one of the loser's restarts scores as progress, its counter resets, and
354
+ it resynchronises at the schedule's bounded rate. Only a signal that is genuinely
355
+ stuck — a poison event, a persistently broken view store — trips the breaker. The
356
+ budget for that is sized in **wall clock**, roughly five minutes on the defaults,
357
+ so a view store that is merely *unavailable* is back inside it.
358
+
359
+ What that paragraph describes is one store under one key, and it is a **fault**: two
360
+ *deliberate* materialisations of one read model never reach it, their checkpoints
361
+ living in different stores with no shared cursor to contend for. That is why
362
+ `runProjections` refuses the one-store pairing at construction rather than leaving
363
+ this mechanism to absorb it — it would, indefinitely and silently, each
364
+ materialisation holding an arbitrary subset of the events.
365
+
366
+ Tuning is two documented option sets in **one flat object**.
367
+ `ProjectionPipelineOptions` (`batchSize`, `batchWindow`) sizes a transaction and a
368
+ latency ceiling; `ProjectionSupervisionOptions` (`restartMinDelay`,
369
+ `restartMaxDelay`, `restartResetAfter`, `maxNoProgressRestarts`, `onRestart`,
370
+ `onStalled`) sizes a recovery policy. `ProjectionRunnerOptions` is their union —
371
+ flat, because the ubiquitous `{ ...OPTIONS, oneField }` override would silently
372
+ drop its siblings under a nested shape. Every figure is bounded at construction:
373
+ the two counts are branded (`BatchSize.make(…)`, `MaxNoProgressRestarts.make(…)`),
374
+ and the four `Duration`s are rejected as defects before anything is forked —
375
+ including under `runProjections`, where every entry's figures are judged before the
376
+ first daemon exists. Retune `maxNoProgressRestarts` by the wall clock it buys, never
377
+ by the count.
378
+
379
+ ## The restart is the metric, the stall is the page
380
+
381
+ The supervisor has two outcomes and one hook each, and they are not equally
382
+ important. `onRestart` fires on **every restart**, before the backoff sleep — the
383
+ outcome that resolves itself, so it is a rate to graph and to alert on only if it
384
+ climbs. `onStalled` fires **once**, at the give-up, immediately after the error is
385
+ logged and immediately before `ProjectionStalled` is raised — a poison event or a
386
+ view store broken rather than merely unavailable, both of which need a human.
387
+
388
+ **Wire `onStalled` in production.** The recommended shape is `projectionLayer`,
389
+ which provides nothing and therefore discards the daemon fibre — deliberately,
390
+ because a projection is a background process and a read model is queried through
391
+ its view — so there is nothing to `Fiber.await`, and a stall's only other route out
392
+ of the process is a single `Effect.logError`. The hook is what turns that into a
393
+ failed health check, a non-zero exit, or a page.
394
+
395
+ Its payload is the `ProjectionStalled` about to be raised, field for field and
396
+ value for value: `key`, `noProgressRestarts`, the stuck `position` (so the
397
+ offending event is one query away) and the raw `reason` rather than the string the
398
+ log line carries. Both hooks are `Effect<void>`, so neither can fail the daemon nor
399
+ widen its requirements — and neither traps a **defect**, which takes the fibre down
400
+ as any other bug would. The ordering is what makes that safe: the log line has
401
+ already landed by the time your hook is called, so a broken pager cannot buy
402
+ silence.
403
+
404
+ **The hooks are the primary channel; the log lines are a thin lossy default.**
405
+ Every annotation on the two supervision lines is a string or a number, on purpose
406
+ — JSON-safe rather than uniformly stringy. Be precise about which `bigint` is the
407
+ hazard, because the obvious reading is wrong: `effect@3.22`'s `Logger.json`
408
+ special-cases a **top-level** annotation value, so a bare `bigint` renders fine. A
409
+ `bigint` **nested inside** an annotation's object is the one that survives into the
410
+ single `JSON.stringify` of the whole record, with nothing catching the throw, and
411
+ raises `TypeError: Do not know how to serialize a BigInt` — losing the entire line
412
+ and taking the daemon with it as a defect. A raw fault is exactly such an object:
413
+ `CheckpointSuperseded` carries `expected: Position`. So the log line gets
414
+ `String(fault)` and the hooks get the fault untouched: anything that needs to match
415
+ on a tag, read a payload, or count by fault type takes `onRestart`/`onStalled`; the
416
+ log line is for everyone else.
417
+
418
+ **The label is the fault's, not the supervisor's.** `String(fault)` is the whole
419
+ reduction — there is no table of tag-to-label anywhere, and nothing to hand the
420
+ supervisor. `Error.prototype.toString` is `name: message`, and `Data.TaggedError`
421
+ sets `name` to the tag, so a fault that fills `message` arrives as
422
+ `ProjectionStoreError: connection reset` and one that does not arrives as its bare
423
+ tag. The library's own faults all fill it from the field that says why —
424
+ `ProjectionStoreError` and `CodecError` from `reason`, `PipelineDied` from its
425
+ `defect`, `CheckpointSuperseded` from `expected`, and `ProjectionStalled` from the
426
+ budget it spent and the position it is stuck at. A `bigint` field is no obstacle:
427
+ `message` is typed `string`, so interpolating one yields digits, and the hazard
428
+ above — which is about the annotation values a logger is handed — never arises
429
+ through it. **Your read model's `E` gets the same treatment**, so put a `message` on
430
+ any error you want to recognise in a log: it costs one getter over the field you
431
+ already have, and it pays in `Cause.pretty` and every other reader of that error,
432
+ not only here.
433
+
434
+ ## R2: one log, one declaration
435
+
436
+ The three legal (event log, view store) pairings are all-in-memory, durable log
437
+ with an in-memory view, and durable log with a durable view. A **durable view
438
+ over an in-memory log is rejected at construction** (R2): its checkpoint would
439
+ outlive the log, leaving the view frozen and permanently stale without ever
440
+ erroring.
441
+
442
+ Each pairing is a property of ONE materialisation, not of a read model: a read
443
+ model materialised twice can have two of them in play at once over one log, which
444
+ is a deployment property rather than a fourth permutation.
445
+
446
+ The comparison's two inputs are declared in the two places that know them. The
447
+ view store carries its own class on the `ProjectionStore` it implements. The log's
448
+ is the **`EventLogDurability` service** — `DurableEventLog` or
449
+ `EphemeralEventLog`, provided beside the `DcbEventStore` layer — because
450
+ `DcbEventStore` deliberately exposes no durability and the application that chose
451
+ the layer is the only honest source.
452
+
453
+ A **context tag rather than a field on `ReadModel`**, because one log is *one*
454
+ fact. Authored per read model, it would be stated N times at N sites, N copies
455
+ could disagree over the identical store, and nothing anywhere could notice:
456
+ `runProjection` only ever sees the pair it was handed. Since horizontal scale here
457
+ is a runner per read model — and one read model materialised N ways is N runners
458
+ too — N > 1 is the expected case.
459
+
460
+ The cost is that `EventLogDurability` is a **required** input of `runProjection`,
461
+ `projectionLayer` and `runProjections` alike, with no default — and it is costliest
462
+ to forget at `runProjections`, where one missing layer refuses N materialisations at
463
+ once. That is deliberate in both directions:
464
+ defaulting to `'ephemeral'` would make the production wiring the one you have to
465
+ remember, and defaulting to `'durable'` would turn an omission into the frozen
466
+ view R2 exists to prevent. The store layers do not supply it either — the tag is a
467
+ read-side concept and `@kairos-es/core` knows nothing of the read side, a backend
468
+ declaring it would move the fact away from the only party that knows it, and two
469
+ providers of one tag resolve by merge order rather than loudly.
470
+
471
+ ## More apply throughput in one runner: coalesce the batch per row
472
+
473
+ ADR-0007's escape hatch for when apply **throughput** rather than isolation is the
474
+ constraint (horizontal scale of one read model — sharding, competing consumers —
475
+ is deliberately not offered). It is a technique, not a library symbol, because all
476
+ of it is a **fold your read model performs over its own batch**: collapse each
477
+ touched row's N events into the one write that row needs, then issue those writes
478
+ inside the one `commit`. A batch of N events touching R rows costs R statements
479
+ rather than N. There is still one writer, one transaction and one checkpoint
480
+ advance.
481
+
482
+ Why per **row** and not per event: one DCB event fans out to several rows, so no
483
+ event-derived key can own a row — which is precisely what sank sharding, and the
484
+ argument lives in ADR-0007 beside the alternative it rejects.
485
+
486
+ Coalescing need not be total, and the **hybrid** is the shape worth seeing: one
487
+ row's whole batch becomes an upsert while a row written per event stays N inserts.
488
+ A plain sequential list is all the ordering that needs, since it preserves exactly
489
+ the order the fold emitted — which is ascending position order, because the store
490
+ delivers strictly ascending and `groupedWithin` keeps that.
491
+
492
+ ```ts
493
+ const apply = (batch: Arr.NonEmptyReadonlyArray<DecodedEvent>) =>
494
+ // `Effect.suspend` so the fold runs when this effect RUNS — inside `commit`'s
495
+ // bracket, where a transaction can roll it back — and not when the runner calls
496
+ // `apply` to build the effect it hands to `commit`.
497
+ Effect.suspend(() =>
498
+ Effect.all(
499
+ // One write per row it can collapse, plus whatever cannot fold into that.
500
+ Arr.flatMap([...coalesceByRow(batch)], ([row, delta]) => [
501
+ upsertTotals(row, delta.applied),
502
+ appendAudit(row, delta.positions),
503
+ ]),
504
+ { discard: true },
505
+ ),
506
+ )
507
+ ```
508
+
509
+ **On a transactional SQL view store these writes cannot run in parallel, and do
510
+ not need to.** `sql.withTransaction` acquires one connection and pins it into the
511
+ fibre context, and `pg` dispatches one statement at a time per connection, so the
512
+ statements queue in the driver whatever a caller does. What coalescing buys there
513
+ is far **fewer round trips** on that pinned connection, never parallel ones.
514
+
515
+ On a **non-transactional** view store the port already requires a single atomic
516
+ per-batch write, and `foldIntoRef` is coalescing taken to its limit: fold the whole
517
+ batch, then write exactly once.
518
+
519
+ ## Where the checkpoint cannot travel with the view
520
+
521
+ R1 says a read model's checkpoint lives with its view, committed atomically with
522
+ it — which is what `ProjectionStore.commit` is. A view store offering **no
523
+ transaction at all** (a search index, an object store, an HTTP API) cannot
524
+ satisfy that: there is nothing to bundle the checkpoint into, so the mode is
525
+ at-least-once and the projection's own writes must be **idempotent**.
526
+
527
+ The recommended shape is a **position-keyed write that no-ops on replay** — in
528
+ SQL terms an `INSERT … ON CONFLICT DO NOTHING` keyed by the event's position, and
529
+ in a search index or an object store the equivalent: derive the document or
530
+ object key from the position so re-applying the same event overwrites itself
531
+ instead of accumulating. The runner's guarantee then degrades from exactly-once
532
+ to "every event applied at least once, and applying it twice is
533
+ indistinguishable from applying it once".
534
+
535
+ This is documented rather than enforced because the library cannot force a read
536
+ model's schema to be position-keyed: the key is a property of the reader's own
537
+ rows, which the port has no knowledge of. No helper is shipped for it, and no
538
+ such view store is implemented here.