@zio.dev/zio-blocks 0.0.51 → 0.0.56

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (166) hide show
  1. package/adr/2026-07-18-data-migration.md +123 -0
  2. package/guides/async-getting-started.md +687 -0
  3. package/guides/compile-time-resource-safety-with-scope.md +6 -0
  4. package/guides/getting-started-with-mux.md +0 -112
  5. package/guides/query-dsl-extending.md +1 -1
  6. package/guides/query-dsl-fluent-builder.md +1 -1
  7. package/guides/query-dsl-reified-optics.md +1 -1
  8. package/guides/query-dsl-sql.md +395 -1
  9. package/guides/sql-checked-interpolation.md +173 -0
  10. package/guides/sql-transactions.md +286 -0
  11. package/guides/telemetry-guide.md +131 -70
  12. package/guides/zio-schema-migration.md +6 -6
  13. package/index.md +200 -559
  14. package/package.json +1 -1
  15. package/reference/async.md +1379 -531
  16. package/reference/chunk.md +3 -3
  17. package/reference/codegen/index.md +1 -1
  18. package/reference/combinators.md +4 -4
  19. package/reference/config/config-decoder.md +460 -0
  20. package/reference/config/config-source.md +489 -0
  21. package/reference/config/errors.md +278 -0
  22. package/reference/config/flags.md +369 -0
  23. package/reference/config/formats.md +314 -0
  24. package/reference/config/index.md +304 -0
  25. package/reference/config/rollout.md +336 -0
  26. package/reference/context.md +6 -49
  27. package/reference/data-migration.md +269 -0
  28. package/reference/datastar/attributes.md +302 -0
  29. package/reference/datastar/events.md +234 -0
  30. package/reference/datastar/index.md +256 -0
  31. package/reference/datastar/signals.md +230 -0
  32. package/reference/datastar/sse.md +295 -0
  33. package/reference/datastar.md +2 -2
  34. package/reference/docs.md +2 -2
  35. package/reference/endpoint/bulk-creation.md +96 -0
  36. package/reference/endpoint/endpoint.md +1 -0
  37. package/reference/endpoint/index.md +9 -89
  38. package/reference/endpoint/path-codec.md +12 -24
  39. package/reference/endpoint/route-pattern.md +4 -6
  40. package/reference/endpoint/segment-codec.md +19 -32
  41. package/reference/html.md +313 -9
  42. package/reference/htmx/index.md +4 -52
  43. package/reference/htmx/response-headers.md +240 -0
  44. package/reference/http-model/headers.md +735 -0
  45. package/reference/http-model/index.md +3 -1
  46. package/reference/http-model/model.md +107 -71
  47. package/reference/http-model/schema-codecs.md +522 -0
  48. package/reference/http-model/schema.md +6 -3
  49. package/reference/http-model/server-sent-event.md +341 -0
  50. package/reference/jwt.md +195 -0
  51. package/reference/maybe.md +128 -11
  52. package/reference/media-type.md +2 -2
  53. package/reference/mux.mdx +7 -2
  54. package/reference/openapi.md +3 -3
  55. package/reference/projection.md +654 -0
  56. package/reference/resource-management/index.md +1 -1
  57. package/reference/resource-management/resource.md +2 -98
  58. package/reference/resource-management/scope.md +1 -209
  59. package/reference/resource-management/wire.md +4 -50
  60. package/reference/ringbuffer/advanced.mdx +1 -1
  61. package/reference/ringbuffer/index.mdx +3 -3
  62. package/reference/ringbuffer/mpmc.mdx +38 -4
  63. package/reference/ringbuffer/mpsc.mdx +36 -4
  64. package/reference/ringbuffer/spmc.mdx +1 -1
  65. package/reference/ringbuffer/spsc.mdx +87 -15
  66. package/reference/schema/allows.md +0 -96
  67. package/reference/schema/binding.md +2 -2
  68. package/reference/schema/built-in-codecs/avro.md +2 -2
  69. package/reference/schema/built-in-codecs/bson.md +50 -20
  70. package/reference/schema/built-in-codecs/csv.md +2 -2
  71. package/reference/schema/built-in-codecs/index.md +3 -3
  72. package/reference/schema/built-in-codecs/json/index.md +2 -2
  73. package/reference/schema/built-in-codecs/json/json.md +1 -0
  74. package/reference/schema/built-in-codecs/messagepack.md +3 -3
  75. package/reference/schema/built-in-codecs/thrift.md +2 -2
  76. package/reference/schema/built-in-codecs/toon.md +3 -3
  77. package/reference/schema/built-in-codecs/yaml.md +2 -2
  78. package/reference/schema/codec.md +11 -11
  79. package/reference/schema/dynamic-optic.md +48 -3
  80. package/reference/schema/dynamic-schema.md +3 -3
  81. package/reference/schema/index.md +2 -0
  82. package/reference/schema/path-interpolator.md +2 -0
  83. package/reference/schema/reflect-transformer.md +140 -0
  84. package/reference/schema/schema-evolution/as.md +4 -4
  85. package/reference/schema/schema-evolution/into.md +2 -2
  86. package/reference/schema/schema-expr.md +2 -2
  87. package/reference/schema/schema-search.md +263 -0
  88. package/reference/schema/schema.md +10 -2
  89. package/reference/schema/type-class-derivation.md +1 -1
  90. package/reference/smithy.md +502 -3
  91. package/reference/sql/db-codec-deriver.md +3 -3
  92. package/reference/sql/db-codec.md +22 -22
  93. package/reference/sql/db-con.md +4 -4
  94. package/reference/sql/db-connection.md +1 -1
  95. package/reference/sql/db-param.md +1 -1
  96. package/reference/sql/db-result-reader.md +4 -2
  97. package/reference/sql/db-tx.md +46 -14
  98. package/reference/sql/ddl.md +1 -1
  99. package/reference/sql/frag.md +44 -10
  100. package/reference/sql/index.md +7 -7
  101. package/reference/sql/repo.md +15 -15
  102. package/reference/sql/sql-dialect.md +1 -1
  103. package/reference/sql/sql-logger.md +1 -1
  104. package/reference/sql/sql-name-mapper.md +3 -3
  105. package/reference/sql/table-metadata.md +3 -3
  106. package/reference/sql/table.md +10 -10
  107. package/reference/sql/transactor-zio.md +1 -1
  108. package/reference/sql/transactor.md +21 -11
  109. package/reference/sql-zio.md +2 -2
  110. package/reference/streams/core/index.md +32 -0
  111. package/reference/streams/{pipeline.md → core/pipeline.md} +210 -74
  112. package/reference/streams/{sink.md → core/sink.md} +331 -353
  113. package/reference/streams/{stream.md → core/stream.md} +919 -209
  114. package/reference/streams/execution-and-compatibility/async-execution.md +822 -0
  115. package/reference/streams/execution-and-compatibility/index.md +35 -0
  116. package/reference/streams/execution-and-compatibility/platform-differences.md +297 -0
  117. package/reference/streams/execution-and-compatibility/scala-2-compatibility.md +88 -0
  118. package/reference/streams/execution-and-compatibility/zero-boxing.md +393 -0
  119. package/reference/streams/index.md +140 -67
  120. package/reference/streams/primitives/index.md +30 -0
  121. package/reference/streams/primitives/reader.md +1992 -0
  122. package/reference/streams/{writer.md → primitives/writer.md} +254 -98
  123. package/reference/telemetry/common/any-value.md +90 -0
  124. package/reference/telemetry/common/attribute-key.md +87 -0
  125. package/reference/telemetry/common/attributes.md +118 -0
  126. package/reference/telemetry/common/index.md +39 -0
  127. package/reference/telemetry/common/instrumentation-scope.md +24 -0
  128. package/reference/telemetry/common/resource.md +34 -0
  129. package/reference/telemetry/index.md +311 -0
  130. package/reference/telemetry/logging/index.md +197 -0
  131. package/reference/telemetry/logging/log-enrichment.md +72 -0
  132. package/reference/telemetry/logging/log-formatter.md +100 -0
  133. package/reference/telemetry/logging/log-record-processor.md +56 -0
  134. package/reference/telemetry/logging/log-record.md +44 -0
  135. package/reference/telemetry/logging/log-writer.md +64 -0
  136. package/reference/telemetry/logging/logger-provider.md +142 -0
  137. package/reference/telemetry/logging/logger.md +83 -0
  138. package/reference/telemetry/logging/severity.md +62 -0
  139. package/reference/telemetry/metrics/index.md +150 -0
  140. package/reference/telemetry/metrics/instruments.md +183 -0
  141. package/reference/telemetry/metrics/labeled-instruments.md +74 -0
  142. package/reference/telemetry/metrics/meter-provider.md +76 -0
  143. package/reference/telemetry/metrics/meter.md +98 -0
  144. package/reference/telemetry/metrics/metric-data.md +57 -0
  145. package/reference/telemetry/otel/custom-exporter.md +216 -0
  146. package/reference/telemetry/otel/index.md +212 -0
  147. package/reference/telemetry/tracing/index.md +155 -0
  148. package/reference/telemetry/tracing/sampler.md +89 -0
  149. package/reference/telemetry/tracing/span-builder.md +57 -0
  150. package/reference/telemetry/tracing/span-context.md +39 -0
  151. package/reference/telemetry/tracing/span-data.md +32 -0
  152. package/reference/telemetry/tracing/span-kind.md +55 -0
  153. package/reference/telemetry/tracing/span-processor.md +53 -0
  154. package/reference/telemetry/tracing/span-status.md +47 -0
  155. package/reference/telemetry/tracing/span.md +117 -0
  156. package/reference/telemetry/tracing/tracer-provider.md +91 -0
  157. package/reference/telemetry/tracing/tracer.md +52 -0
  158. package/reference/typeid.md +0 -64
  159. package/sidebars.js +365 -185
  160. package/undocumented-report.md +528 -270
  161. package/reference/config.md +0 -158
  162. package/reference/streams/concurrent-operators.md +0 -106
  163. package/reference/streams/reader.md +0 -1284
  164. package/reference/streams/scala-2-compatibility.md +0 -55
  165. package/reference/streams/zero-boxing.md +0 -275
  166. package/reference/telemetry.md +0 -693
@@ -0,0 +1,654 @@
1
+ ---
2
+ id: projection
3
+ title: "Projection"
4
+ ---
5
+
6
+ `zio-blocks-projection` provides event-sourced projections backed by per-entity
7
+ SQLite files. Each projection type gets its own isolated `.db` file, giving you
8
+ WAL-mode writes, offline queryability, and zero contention between unrelated
9
+ entities.
10
+
11
+ ## Installation
12
+
13
+ ```scala
14
+ libraryDependencies += "dev.zio" %% "zio-blocks-projection" % "0.0.56"
15
+
16
+ // JVM only — SQLite-backed stores
17
+ libraryDependencies += "dev.zio" %% "zio-blocks-projection" % "0.0.56"
18
+
19
+ // ZIO integration (transactor cache, engine runtime)
20
+ libraryDependencies += "dev.zio" %% "zio-blocks-sql-zio" % "0.0.56"
21
+
22
+ // for SQLite persistence add: libraryDependencies += "org.xerial" % "sqlite-jdbc" % "3.53.4.0" // or InMemory fallback if not present
23
+ libraryDependencies += "org.xerial" % "sqlite-jdbc" % "3.53.4.0"
24
+ ```
25
+
26
+ `sqlite-jdbc` is `% Test` by default in `zio-blocks-projection`; consumers that
27
+ want SQLite persistence at runtime must add the dependency above explicitly.
28
+ If it is not present the engine falls back to `InMemoryProjectionStore`.
29
+
30
+ ## Overview
31
+
32
+ Projections turn an event stream into queryable read models. Instead of
33
+ querying the event store directly, you define how events map to entities, and
34
+ the engine materializes those entities into SQLite files you can read
35
+ anytime.
36
+
37
+ **Why per-entity SQLite?**
38
+
39
+ A single shared database means every projection contends on the same locks,
40
+ the same WAL, and the same file. With per-entity files, each projection type
41
+ gets isolated storage:
42
+
43
+ | Approach | Contention | WAL contention | Offline access | Backup granularity |
44
+ |----------|-----------|---------------|----------------|-------------------|
45
+ | Single DB | High | High | Yes | Full DB |
46
+ | Per-entity SQLite | Zero | Zero | Yes | Per projection |
47
+
48
+ Each projection store lives at `<basePath>/<specName>.db`. Global
49
+ aggregates go to `global/<specName>.db`. The `TransactorCache` reuses
50
+ connections via an LRU cache so you don't open hundreds of file handles.
51
+
52
+ **Three projection scopes:**
53
+
54
+ - **PerEntity** (default): one row per entity ID. Events routed by
55
+ `ctx.entityId`. Good for user profiles, order state, anything keyed by
56
+ a single ID.
57
+ - **CrossEntity**: events routed by an arbitrary key extracted from the
58
+ event. Multiple event types converge on the same projection. Good for
59
+ "all repos owned by user X" views.
60
+ - **Global**: a single shared row (or a few rows keyed by a grouping
61
+ string). Events from all sources funnel into atomic counter updates.
62
+ Good for dashboards, daily aggregates, live counters.
63
+
64
+ ## Quick Start
65
+
66
+ Define your events, your projection entity, wire up a spec, and start the
67
+ engine:
68
+
69
+ ```scala
70
+ import zio.*
71
+ import zio.blocks.projection.*
72
+ import zio.blocks.projection.testing.TestEngine
73
+ import zio.blocks.schema.{Modifier, Schema}
74
+
75
+ // 1. Define events
76
+ case class UserCreated(name: String, email: String)
77
+ object UserCreated {
78
+ implicit val schema: Schema[UserCreated] = Schema.derived[UserCreated]
79
+ }
80
+
81
+ // 2. Define the projection entity
82
+ case class UserProfile(@Modifier.id id: String, name: String, email: String)
83
+ object UserProfile {
84
+ implicit val schema: Schema[UserProfile] = Schema.derived[UserProfile]
85
+ implicit val entityPath: EntityPath[UserProfile] = EntityPath.derived[UserProfile]
86
+ }
87
+
88
+ // 3. Define the projection
89
+ val projection = Projection[UserProfile]("userProfiles")
90
+ .from("users")
91
+ .routeToSelf
92
+ .on[UserCreated]
93
+ .insert((e, ctx) => UserProfile(ctx.entityId, e.name, e.email))
94
+
95
+ // 4. Create a test engine (auto-creates stores and hubs)
96
+ val program: ZIO[Scope, Throwable, Unit] = for {
97
+ engine <- TestEngine.make(projection)
98
+ // Append events
99
+ _ <- engine.append("user-1", UserCreated("Alice", "alice@example.com"))
100
+ _ <- engine.append("user-2", UserCreated("Bob", "bob@example.com"))
101
+ // Query
102
+ u1 <- engine.query(projection, "user-1")
103
+ u2 <- engine.query(projection, "user-2")
104
+ } yield ()
105
+ ```
106
+
107
+ The engine starts a catch-up fiber that reads all historical events, then
108
+ switches to live mode via the Hub subscription. Query results appear once
109
+ the engine processes the relevant events.
110
+
111
+ ## Entity Path Conventions
112
+
113
+ `EntityPath[A]` tells the engine two things: which folder stores the SQLite
114
+ file, and which field holds the entity ID. The ID field is identified via
115
+ `@Modifier.id` (from `zio.blocks.schema.Modifier`) or, for ergonomics, a
116
+ field named `id`.
117
+
118
+ ```scala
119
+ import zio.blocks.projection.*
120
+ import zio.blocks.schema.{Modifier, Schema}
121
+
122
+ case class Order(@Modifier.id id: Long, total: BigDecimal)
123
+ object Order {
124
+ implicit val schema: Schema[Order] = Schema.derived[Order]
125
+ }
126
+
127
+ val orderId = "42" // String id
128
+ val raw = orderId // String = "42"
129
+ ```
130
+
131
+ ```scala
132
+ import zio.blocks.projection.*
133
+ import zio.blocks.schema.{Modifier, Schema}
134
+
135
+ case class UserProfile(@Modifier.id userId: String, name: String, email: String)
136
+ object UserProfile {
137
+ implicit val schema: Schema[UserProfile] = Schema.derived[UserProfile]
138
+ implicit val entityPath: EntityPath[UserProfile] = EntityPath.derived[UserProfile]
139
+ }
140
+ // basePath = "users", entityIdField = "userId"
141
+ ```
142
+
143
+ **Derivation rules** (`EntityPath.derived[A]`):
144
+
145
+ 1. Look for a field annotated with `@Modifier.id`. If found, use it as
146
+ the entity ID field.
147
+ 2. Otherwise, find the field named `id`.
148
+ 3. If neither is found, fail with `Entity must have @Modifier.id field`.
149
+ 4. Derive the folder name from the ID field name:
150
+ - Strip trailing `Id` suffix: `userId` → `user`
151
+ - Convert to snake_case: `userId` → `user_id`
152
+ - Pluralize: `user` → `users`
153
+ - Result: `basePath = "users"`, `entityIdField = "userId"`
154
+
155
+ If the field is exactly `id`, `basePath` is `ids` — prefer `userId` or use `@path` to override.
156
+
157
+ **Override with `@path`:**
158
+
159
+ ```scala
160
+ import zio.blocks.projection.*
161
+ import zio.blocks.schema.{Modifier, Schema}
162
+
163
+ @path("custom_users")
164
+ case class UserProfile(@Modifier.id id: String, name: String)
165
+ object UserProfile {
166
+ implicit val schema: Schema[UserProfile] = Schema.derived[UserProfile]
167
+ implicit val entityPath: EntityPath[UserProfile] = EntityPath.derived[UserProfile]
168
+ }
169
+ // basePath = "custom_users", entityIdField = "id"
170
+ ```
171
+
172
+ **Manual construction:**
173
+
174
+ ```scala
175
+ import zio.blocks.projection.*
176
+ import zio.blocks.schema.{Modifier, Schema}
177
+
178
+ case class MyEntity(@Modifier.id id: String, value: Int)
179
+ object MyEntity {
180
+ implicit val schema: Schema[MyEntity] = Schema.derived[MyEntity]
181
+ implicit val entityPath: EntityPath[MyEntity] = EntityPath[MyEntity]("my_entities", "id")
182
+ }
183
+ ```
184
+
185
+ ## Multi-Source Projections
186
+
187
+ Cross-entity projections receive events from multiple sources and route them
188
+ by a key extracted from the event:
189
+
190
+ ```scala
191
+ import zio.blocks.projection.*
192
+ import zio.blocks.schema.{Modifier, Schema}
193
+
194
+ case class RepoCreated(ownerId: String, repoName: String)
195
+ object RepoCreated {
196
+ implicit val schema: Schema[RepoCreated] = Schema.derived[RepoCreated]
197
+ }
198
+
199
+ case class RepoListEntry(@Modifier.id id: String, ownerId: String, repoName: String)
200
+ object RepoListEntry {
201
+ implicit val schema: Schema[RepoListEntry] = Schema.derived[RepoListEntry]
202
+ implicit val entityPath: EntityPath[RepoListEntry] = EntityPath.derived[RepoListEntry]
203
+ }
204
+
205
+ val spec = Projection[RepoListEntry]("repoListEntries")
206
+ .from("repos")
207
+ .routedBy[RepoCreated](_.ownerId)
208
+ .on[RepoCreated]
209
+ .custom((e, _) => ProjectionAction.Upsert(RepoListEntry(e.ownerId, e.ownerId, e.repoName)))
210
+
211
+ // spec.scope == ProjectionScope.CrossEntity(extractor)
212
+ ```
213
+
214
+ The `routedBy` call tells the engine to extract a routing key from each
215
+ event. Events with the same key go to the same shard store, so querying
216
+ `engine.query(spec, "alice")` searches Alice's shard first.
217
+
218
+ **Routing modes:**
219
+
220
+ | Mode | Behavior | Scope derived |
221
+ |------|----------|---------------|
222
+ | `.routeToSelf` | Uses `ctx.entityId` as key | `PerEntity` |
223
+ | `.routedBy[E](_.field)` | Extracts key from event | `CrossEntity` |
224
+ | `.routeToAll` | All events go to same store | `Global` (if `isGlobal`) |
225
+
226
+ ## Aggregate Projections
227
+
228
+ Global projections aggregate events from multiple sources into a single
229
+ row (or a few rows keyed by a grouping string). The engine applies atomic
230
+ counter updates:
231
+
232
+ ```scala
233
+ import zio.blocks.projection.*
234
+ import zio.blocks.schema.{Modifier, Schema}
235
+
236
+ case class UserCreated(name: String, email: String)
237
+ object UserCreated {
238
+ implicit val schema: Schema[UserCreated] = Schema.derived[UserCreated]
239
+ }
240
+ case class RepoCreated(ownerId: String, repoName: String)
241
+ object RepoCreated {
242
+ implicit val schema: Schema[RepoCreated] = Schema.derived[RepoCreated]
243
+ }
244
+
245
+ case class DailyStats(@Modifier.id date: String, userCount: Int, repoCount: Int)
246
+ object DailyStats {
247
+ implicit val schema: Schema[DailyStats] = Schema.derived[DailyStats]
248
+ implicit val entityPath: EntityPath[DailyStats] = EntityPath.derived[DailyStats]
249
+ }
250
+
251
+ val spec = Projection
252
+ .global[DailyStats]("dailyStats")
253
+ .from("users")
254
+ .routeToAll
255
+ .on[UserCreated]
256
+ .aggregate(FieldUpdate.Increment("user_count", 1L))
257
+ .from("repos")
258
+ .routeToAll
259
+ .on[RepoCreated]
260
+ .aggregate(FieldUpdate.Increment("repo_count", 1L))
261
+ ```
262
+
263
+ **FieldUpdate operations:**
264
+
265
+ | Operation | SQL translation | Description |
266
+ |-----------|----------------|-------------|
267
+ | `Set(field, value)` | `SET col = ?` | Replace field value |
268
+ | `Increment(field, by)` | `SET col = COALESCE(col,0) + ?` | Atomic increment |
269
+ | `Decrement(field, by)` | `SET col = COALESCE(col,0) - ?` | Atomic decrement |
270
+ | `Max(field, value)` | `SET col = MAX(COALESCE(col, ?), ?)` | Keep highest value |
271
+ | `Min(field, value)` | `SET col = MIN(COALESCE(col, ?), ?)` | Keep lowest value |
272
+
273
+ The `COALESCE` handles missing rows. The engine runs `INSERT OR IGNORE`
274
+ before any `UPDATE`, so incrementing a counter that doesn't exist yet
275
+ creates the row with value `1` (not `0 + 1`).
276
+
277
+ **Concurrent safety:** Counter operations are atomic at the SQL level. Ten
278
+ fibers each incrementing the same counter produce the correct total without
279
+ lost updates.
280
+
281
+ ## Schema Evolution
282
+
283
+ When you change a projection entity's schema (add a field, rename a column),
284
+ the engine detects the mismatch and rebuilds the projection from events.
285
+
286
+ ### How it works
287
+
288
+ 1. On startup, `ProjectionEngine` computes `SchemaHash.compute[A]` for
289
+ each spec's entity type. This is a SHA-256 hash of the schema structure
290
+ (field names, types, order).
291
+ 2. The hash is stored in `_projection_meta.schema_hash`.
292
+ 3. If the stored hash doesn't match the current hash, the engine rebuilds:
293
+ - Truncate the projection store
294
+ - Replay all events from the EventStore through the spec's handlers
295
+ - Store the new hash
296
+
297
+ ```scala
298
+ import zio.blocks.projection.*
299
+ import zio.blocks.schema.{Modifier, Schema}
300
+
301
+ case class UserProfileV1(@Modifier.id id: String, name: String)
302
+ object UserProfileV1 {
303
+ implicit val schema: Schema[UserProfileV1] = Schema.derived[UserProfileV1]
304
+ }
305
+
306
+ case class UserProfileV2(@Modifier.id id: String, name: String, email: String)
307
+ object UserProfileV2 {
308
+ implicit val schema: Schema[UserProfileV2] = Schema.derived[UserProfileV2]
309
+ }
310
+
311
+ val hashV1 = SchemaHash.compute[UserProfileV1]
312
+ val hashV2 = SchemaHash.compute[UserProfileV2]
313
+ // hashV1 != hashV2 because V2 has an extra "email" field
314
+ ```
315
+
316
+ ### Lazy rebuild
317
+
318
+ Set `lazyRebuild = true` in `ProjectionEngineConfig` to defer rebuilds.
319
+ The engine marks specs needing rebuild but doesn't block startup. The first
320
+ query to a spec triggers its rebuild:
321
+
322
+ ```scala
323
+ import zio.blocks.projection.ProjectionEngineConfig
324
+
325
+ val config = ProjectionEngineConfig(
326
+ batchSize = 100,
327
+ batchTimeout = zio.Duration.fromMillis(50),
328
+ ringCapacity = 4096,
329
+ rebuildParallelism = 4,
330
+ lazyRebuild = true
331
+ )
332
+ ```
333
+
334
+ ### Migration shortcut
335
+
336
+ For simple `AddField` migrations (adding columns with defaults), the engine
337
+ uses `ALTER TABLE ADD COLUMN` instead of a full rebuild. This is much faster
338
+ for large projections:
339
+
340
+ ```scala
341
+ import zio.blocks.schema.migration.Migration
342
+
343
+ // Assumes UserProfileV1 and UserProfileV2 are defined as above
344
+ val migration = Migration
345
+ .newBuilder[UserProfileV1, UserProfileV2]
346
+ .addField(_.email, "")
347
+ .build
348
+
349
+ // Engine detects this is a simple AddField migration
350
+ // and uses ALTER TABLE instead of full rebuild
351
+ ```
352
+
353
+ ## Event Tagging and Migration
354
+
355
+ Event tags identify variant cases for serialization. By default, the tag is
356
+ the case class name (e.g., `"UserCreated"`). You can override with numeric
357
+ tags or rename cases across versions.
358
+
359
+ ### String tags (default)
360
+
361
+ For a sealed trait `UserEvent` with cases `UserCreated` and `UserDeleted`,
362
+ the tags are `"UserCreated"` and `"UserDeleted"`.
363
+
364
+ ### Numeric tags
365
+
366
+ Use the `@eventTag` annotation for stable numeric identifiers:
367
+
368
+ ```scala
369
+ import zio.blocks.projection.eventTag
370
+ import zio.blocks.schema.{Modifier, Schema}
371
+
372
+ sealed trait UserEvent
373
+ object UserEvent {
374
+ @eventTag(1)
375
+ case class Created(name: String) extends UserEvent
376
+ object Created {
377
+ implicit val schema: Schema[Created] = Schema.derived[Created]
378
+ }
379
+
380
+ @eventTag(2)
381
+ case class Deleted(userId: String) extends UserEvent
382
+ object Deleted {
383
+ implicit val schema: Schema[Deleted] = Schema.derived[Deleted]
384
+ }
385
+
386
+ implicit val schema: Schema[UserEvent] = Schema.derived[UserEvent]
387
+ }
388
+ ```
389
+
390
+ ### Tag aliases via Migration
391
+
392
+ When you rename a case, old events in the database still have the old tag.
393
+ Use `Migration.renameCase` to create an alias:
394
+
395
+ ```scala
396
+ import zio.blocks.schema.migration.Migration
397
+ import zio.blocks.projection.TagResolver
398
+ import zio.blocks.schema.{Modifier, Schema}
399
+
400
+ sealed trait LoginEvent
401
+ object LoginEvent {
402
+ case class UserLoggedIn(userId: String) extends LoginEvent
403
+ object UserLoggedIn {
404
+ implicit val loggedInSchema: Schema[UserLoggedIn] = Schema.derived
405
+ }
406
+ case class UserAuthenticated(userId: String) extends LoginEvent
407
+ object UserAuthenticated {
408
+ implicit val authSchema: Schema[UserAuthenticated] = Schema.derived
409
+ }
410
+ implicit val loginEventSchema: Schema[LoginEvent] = Schema.derived
411
+ }
412
+
413
+ val migration = Migration
414
+ .newBuilder[LoginEvent, LoginEvent]
415
+ .renameCase("UserLoggedIn", "UserAuthenticated")
416
+ .build
417
+
418
+ val tagInfo = TagResolver.resolve[LoginEvent](migration)
419
+ // tagInfo.aliases == Map("UserLoggedIn" -> "UserAuthenticated")
420
+ // tagInfo.allTags == Set("UserLoggedIn", "UserAuthenticated")
421
+ ```
422
+
423
+ The `TagInfo` object provides:
424
+
425
+ - `expandRequested(tags)` — returns the union of requested tags plus all
426
+ their old aliases, so selective reads fetch both old and new events
427
+ - `isOldTag(tag)` — true if the tag was renamed
428
+ - `currentTagFor(oldTag)` — returns the current tag for an old tag
429
+ - `normalize(tag)` — maps old tags to current tags
430
+
431
+ ### Transitive chains
432
+
433
+ If you rename `A → B` then `B → C`, the resolver collapses this to `A → C`.
434
+ All three tags are queryable.
435
+
436
+ ### Startup warning
437
+
438
+ On startup, the `SQLiteEventStore` checks all distinct tags in the database
439
+ against the known tag set. Unknown tags produce a warning log, not an error.
440
+ This catches typos or events from a different schema version without crashing
441
+ the application.
442
+
443
+ ## Testing
444
+
445
+ The projection module ships with in-memory implementations for unit tests.
446
+ No SQLite required.
447
+
448
+ ### InMemoryProjectionStore
449
+
450
+ A `ProjectionStore[A]` backed by `Ref[Map[String, A]]`. Supports all
451
+ operations: insert, upsert, updateFields, delete, truncate, schema hash
452
+ tracking.
453
+
454
+ ```scala
455
+ import zio.*
456
+ import zio.blocks.projection.*
457
+ import zio.blocks.projection.testing.InMemoryProjectionStore
458
+ import zio.blocks.schema.{Modifier, Schema}
459
+
460
+ case class UserProfile(@Modifier.id id: String, name: String, email: String)
461
+ object UserProfile {
462
+ implicit val schema: Schema[UserProfile] = Schema.derived[UserProfile]
463
+ implicit val entityPath: EntityPath[UserProfile] = EntityPath.derived[UserProfile]
464
+ }
465
+
466
+ val test: ZIO[Any, Throwable, Unit] = for {
467
+ store <- InMemoryProjectionStore.make[UserProfile]
468
+ _ <- store.insert(UserProfile("u1", "Alice", "alice@example.com"))
469
+ alice <- store.findById("u1")
470
+ // alice == Some(UserProfile("u1", "Alice", "alice@example.com"))
471
+ } yield ()
472
+ ```
473
+
474
+ ### TestEngine
475
+
476
+ A simple test helper that auto-creates `InMemoryProjectionStore` per
477
+ projection and `Hub` per source. No manual `makeWithStores` wiring:
478
+
479
+ ```scala
480
+ import zio.*
481
+ import zio.blocks.projection.*
482
+ import zio.blocks.projection.testing.TestEngine
483
+ import zio.blocks.schema.{Modifier, Schema}
484
+
485
+ case class UserProfile(@Modifier.id id: String, name: String, email: String)
486
+ object UserProfile {
487
+ implicit val schema: Schema[UserProfile] = Schema.derived[UserProfile]
488
+ implicit val entityPath: EntityPath[UserProfile] = EntityPath.derived[UserProfile]
489
+ }
490
+ case class UserCreated(name: String, email: String)
491
+ object UserCreated {
492
+ implicit val schema: Schema[UserCreated] = Schema.derived[UserCreated]
493
+ }
494
+
495
+ val projection = Projection[UserProfile]("userProfiles")
496
+ .from("users").routeToSelf
497
+ .on[UserCreated]
498
+ .insert((e, ctx) => UserProfile(ctx.entityId, e.name, e.email))
499
+
500
+ val test: ZIO[Scope, Throwable, Unit] = for {
501
+ engine <- TestEngine.make(projection)
502
+ _ <- engine.append("user-1", UserCreated("Alice", "alice@example.com"))
503
+ result <- engine.query(projection, "user-1")
504
+ // result == Some(UserProfile("user-1", "Alice", "alice@example.com"))
505
+ } yield ()
506
+ ```
507
+
508
+ ### TestContext
509
+
510
+ Factory for `ProjectionContext` values in tests:
511
+
512
+ ```scala
513
+ import zio.blocks.projection.testing.TestContext
514
+ import java.time.Instant
515
+
516
+ val ctx1 = TestContext.make(entityId = "user-1")
517
+ // ProjectionContext("user-1", now, 0, None)
518
+
519
+ val ctx2 = TestContext.makeWithSource(entityId = "user-1", sourceEntityId = "src-1")
520
+ // ProjectionContext("user-1", now, 0, Some("src-1"))
521
+
522
+ val ctx3 = TestContext.withSeq(entityId = "user-1", seq = 42L)
523
+ // ProjectionContext("user-1", now, 42, None)
524
+ ```
525
+
526
+ ## Configuration
527
+
528
+ ### ProjectionEngineConfig
529
+
530
+ Controls batching, ring buffer size, rebuild behavior, and schema
531
+ evolution:
532
+
533
+ ```scala
534
+ import zio.*
535
+ import zio.blocks.projection.ProjectionEngineConfig
536
+
537
+ val config = ProjectionEngineConfig(
538
+ batchSize = 100, // events per batch in catch-up
539
+ batchTimeout = 50.millis, // max wait before flushing partial batch
540
+ ringCapacity = 4096, // Hub capacity for live events
541
+ rebuildParallelism = 4, // concurrent rebuild fibers
542
+ lazyRebuild = false // true = defer rebuilds to first query
543
+ )
544
+ ```
545
+
546
+ | Parameter | Default | Description |
547
+ |-----------|---------|-------------|
548
+ | `batchSize` | 100 | Number of events processed per batch during catch-up |
549
+ | `batchTimeout` | 50ms | Maximum wait before flushing a partial batch |
550
+ | `ringCapacity` | 4096 | Capacity of the Hub ring buffer for live events |
551
+ | `rebuildParallelism` | 4 | Number of concurrent rebuild fibers |
552
+ | `lazyRebuild` | false | Defer schema rebuilds to first query |
553
+ | `evolution` | default | `SchemaEvolutionConfig` sub-config |
554
+
555
+ ### TransactorCacheConfig
556
+
557
+ Controls the SQLite transactor cache:
558
+
559
+ ```scala
560
+ import zio.blocks.projection.TransactorCacheConfig
561
+
562
+ val cacheConfig = TransactorCacheConfig(maxSize = 256)
563
+ ```
564
+
565
+ | Parameter | Default | Description |
566
+ |-----------|---------|-------------|
567
+ | `maxSize` | 256 | Maximum number of cached SQLite connections |
568
+
569
+ The cache uses LRU eviction. Each cached transactor sets `PRAGMA journal_mode=WAL`
570
+ and `PRAGMA synchronous=NORMAL` on first open. The cache is `Scope`-managed:
571
+ all transactors close when the scope exits.
572
+
573
+ ### Scope lifecycle
574
+
575
+ The `ProjectionEngine.make` and `TransactorCache.make` methods require
576
+ `ZIO[Scope, ...]`. This ensures resources are cleaned up when your
577
+ application shuts down:
578
+
579
+ ```scala
580
+ import zio.*
581
+ import zio.blocks.projection.*
582
+
583
+ val app: ZIO[Scope, Throwable, Unit] = for {
584
+ cache <- TransactorCache.make()
585
+ engine <- ProjectionEngine.makeWithConfig(
586
+ ProjectionEngineConfig.default,
587
+ /* specs... */
588
+ )
589
+ _ <- engine.start
590
+ } yield ()
591
+ // cache and engine cleaned up when scope closes
592
+ ```
593
+
594
+ ## API Reference
595
+
596
+ | Type | Package | Description |
597
+ |------|---------|-------------|
598
+ | `@Modifier.id` | `zio.blocks.schema.Modifier` | Annotation marking the entity ID field |
599
+ | `EntityPath[A]` | `zio.blocks.projection` | Folder name + ID field for a projection entity |
600
+ | `@path(name)` | `zio.blocks.projection` | Annotation to override the derived folder name |
601
+ | `@eventTag(n)` | `zio.blocks.projection` | Annotation for numeric event tags |
602
+ | `Projection[A]` | `zio.blocks.projection` | Defines a projection: name, schema, handlers, routing |
603
+ | `ProjectionAction[+A]` | `zio.blocks.projection` | Enum: Insert, Upsert, Update, Delete, Truncate, Noop |
604
+ | `FieldUpdate` | `zio.blocks.projection` | Enum: Set, Increment, Decrement, Max, Min |
605
+ | `ProjectionContext` | `zio.blocks.projection` | Event metadata: entityId, timestamp, seq, sourceEntityId |
606
+ | `EventEnvelope[+E]` | `zio.blocks.projection` | Event wrapper: seq, tag, event, timestamp, entityId |
607
+ | `EventStore[E]` | `zio.blocks.projection` | Trait: append, readFrom, readAll, subscribe |
608
+ | `SQLiteEventStore[E]` | `zio.blocks.projection` | SQLite-backed EventStore with tag migration |
609
+ | `ProjectionStore[A]` | `zio.blocks.projection` | Trait: insert, upsert, updateFields, delete, truncate, findById |
610
+ | `InMemoryProjectionStore[A]` | `zio.blocks.projection.testing` | In-memory ProjectionStore for tests |
611
+ | `TransactorCache` | `zio.blocks.projection` | LRU cache of SQLite Transactors |
612
+ | `TransactorCacheConfig` | `zio.blocks.projection` | Configuration for TransactorCache |
613
+ | `ProjectionEngine` | `zio.blocks.projection` | Orchestrates catch-up + live processing |
614
+ | `ProjectionEngineConfig` | `zio.blocks.projection` | Engine configuration (batching, rebuild, etc.) |
615
+ | `SchemaHash` | `zio.blocks.projection` | SHA-256 hash of Schema structure for evolution detection |
616
+ | `SchemaEvolution` | `zio.blocks.projection` | Hash check, rebuild, migration shortcut |
617
+ | `SchemaEvolutionConfig` | `zio.blocks.projection` | Evolution config (parallelism, lazy, migration shortcut) |
618
+ | `TagResolver` | `zio.blocks.projection` | Builds TagInfo from Schema + optional Migration |
619
+ | `TagInfo` | `zio.blocks.projection` | Alias map, all tags, old tag detection, value migration |
620
+ | `AggregateProjection` | `zio.blocks.projection` | Helpers for global aggregate specs with counters |
621
+ | `TestEngine` | `zio.blocks.projection.testing` | Simple test engine with auto-created stores and hubs |
622
+ | `TestProjectionEngine` | `zio.blocks.projection.testing` | Synchronous test engine (deprecated, use TestEngine) |
623
+ | `TestContext` | `zio.blocks.projection.testing` | Factory for ProjectionContext in tests |
624
+
625
+ ### Projection methods
626
+
627
+ | Method | Description |
628
+ |--------|-------------|
629
+ | `Projection[A](name)` | Create a per-entity spec (requires `Schema[A]` + `EntityPath[A]`) |
630
+ | `Projection.global[A](name)` | Create a global spec (requires `Schema[A]`) |
631
+ | `.from(sourceName)` | Bind to a named event source |
632
+ | `.on[E]` | Register a handler for event type `E` |
633
+ | `.insert((E, ProjectionContext) => A)` | Handler: insert a new entity |
634
+ | `.update((E, ProjectionContext) => A)` | Handler: replace the entity |
635
+ | `.updateWithField(fieldName, (E, ProjectionContext) => Any)` | Handler: update a single field |
636
+ | `.delete` | Handler: delete the entity |
637
+ | `.custom((E, ProjectionContext) => ProjectionAction[A])` | Handler: arbitrary action |
638
+ | `.aggregate(FieldUpdate)` | Handler: atomic counter update |
639
+ | `.routedBy[E](E => String)` | Route events by extracted key |
640
+ | `.routeToSelf` | Route events by `ctx.entityId` |
641
+ | `.routeToAll` | Send all events to same store |
642
+ | `spec.scope` | Derived scope: `PerEntity`, `CrossEntity`, or `Global` |
643
+
644
+ ### ProjectionEngine methods
645
+
646
+ | Method | Description |
647
+ |--------|-------------|
648
+ | `ProjectionEngine.make(specs*)` | Create engine with default config (requires `Scope`) |
649
+ | `ProjectionEngine.makeWithConfig(config, specs*)` | Create engine with custom config |
650
+ | `engine.start` | Start catch-up + live processing (requires `Scope`) |
651
+ | `engine.query(spec, entityId)` | Query an entity by ID |
652
+ | `engine.queryByName(specName, entityId)` | Query by spec name string |
653
+ | `engine.registerMigration(spec, migration)` | Register a migration for schema evolution |
654
+ | `engine.transactorCache` | Access the underlying TransactorCache |
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  id: index
3
- title: "Resource Management & Dependency Injection"
3
+ title: "Resource Management"
4
4
  ---
5
5
 
6
6
  ## Introduction