@zio.dev/zio-blocks 0.0.51 → 0.0.56
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/adr/2026-07-18-data-migration.md +123 -0
- package/guides/async-getting-started.md +687 -0
- package/guides/compile-time-resource-safety-with-scope.md +6 -0
- package/guides/getting-started-with-mux.md +0 -112
- package/guides/query-dsl-extending.md +1 -1
- package/guides/query-dsl-fluent-builder.md +1 -1
- package/guides/query-dsl-reified-optics.md +1 -1
- package/guides/query-dsl-sql.md +395 -1
- package/guides/sql-checked-interpolation.md +173 -0
- package/guides/sql-transactions.md +286 -0
- package/guides/telemetry-guide.md +131 -70
- package/guides/zio-schema-migration.md +6 -6
- package/index.md +200 -559
- package/package.json +1 -1
- package/reference/async.md +1379 -531
- package/reference/chunk.md +3 -3
- package/reference/codegen/index.md +1 -1
- package/reference/combinators.md +4 -4
- package/reference/config/config-decoder.md +460 -0
- package/reference/config/config-source.md +489 -0
- package/reference/config/errors.md +278 -0
- package/reference/config/flags.md +369 -0
- package/reference/config/formats.md +314 -0
- package/reference/config/index.md +304 -0
- package/reference/config/rollout.md +336 -0
- package/reference/context.md +6 -49
- package/reference/data-migration.md +269 -0
- package/reference/datastar/attributes.md +302 -0
- package/reference/datastar/events.md +234 -0
- package/reference/datastar/index.md +256 -0
- package/reference/datastar/signals.md +230 -0
- package/reference/datastar/sse.md +295 -0
- package/reference/datastar.md +2 -2
- package/reference/docs.md +2 -2
- package/reference/endpoint/bulk-creation.md +96 -0
- package/reference/endpoint/endpoint.md +1 -0
- package/reference/endpoint/index.md +9 -89
- package/reference/endpoint/path-codec.md +12 -24
- package/reference/endpoint/route-pattern.md +4 -6
- package/reference/endpoint/segment-codec.md +19 -32
- package/reference/html.md +313 -9
- package/reference/htmx/index.md +4 -52
- package/reference/htmx/response-headers.md +240 -0
- package/reference/http-model/headers.md +735 -0
- package/reference/http-model/index.md +3 -1
- package/reference/http-model/model.md +107 -71
- package/reference/http-model/schema-codecs.md +522 -0
- package/reference/http-model/schema.md +6 -3
- package/reference/http-model/server-sent-event.md +341 -0
- package/reference/jwt.md +195 -0
- package/reference/maybe.md +128 -11
- package/reference/media-type.md +2 -2
- package/reference/mux.mdx +7 -2
- package/reference/openapi.md +3 -3
- package/reference/projection.md +654 -0
- package/reference/resource-management/index.md +1 -1
- package/reference/resource-management/resource.md +2 -98
- package/reference/resource-management/scope.md +1 -209
- package/reference/resource-management/wire.md +4 -50
- package/reference/ringbuffer/advanced.mdx +1 -1
- package/reference/ringbuffer/index.mdx +3 -3
- package/reference/ringbuffer/mpmc.mdx +38 -4
- package/reference/ringbuffer/mpsc.mdx +36 -4
- package/reference/ringbuffer/spmc.mdx +1 -1
- package/reference/ringbuffer/spsc.mdx +87 -15
- package/reference/schema/allows.md +0 -96
- package/reference/schema/binding.md +2 -2
- package/reference/schema/built-in-codecs/avro.md +2 -2
- package/reference/schema/built-in-codecs/bson.md +50 -20
- package/reference/schema/built-in-codecs/csv.md +2 -2
- package/reference/schema/built-in-codecs/index.md +3 -3
- package/reference/schema/built-in-codecs/json/index.md +2 -2
- package/reference/schema/built-in-codecs/json/json.md +1 -0
- package/reference/schema/built-in-codecs/messagepack.md +3 -3
- package/reference/schema/built-in-codecs/thrift.md +2 -2
- package/reference/schema/built-in-codecs/toon.md +3 -3
- package/reference/schema/built-in-codecs/yaml.md +2 -2
- package/reference/schema/codec.md +11 -11
- package/reference/schema/dynamic-optic.md +48 -3
- package/reference/schema/dynamic-schema.md +3 -3
- package/reference/schema/index.md +2 -0
- package/reference/schema/path-interpolator.md +2 -0
- package/reference/schema/reflect-transformer.md +140 -0
- package/reference/schema/schema-evolution/as.md +4 -4
- package/reference/schema/schema-evolution/into.md +2 -2
- package/reference/schema/schema-expr.md +2 -2
- package/reference/schema/schema-search.md +263 -0
- package/reference/schema/schema.md +10 -2
- package/reference/schema/type-class-derivation.md +1 -1
- package/reference/smithy.md +502 -3
- package/reference/sql/db-codec-deriver.md +3 -3
- package/reference/sql/db-codec.md +22 -22
- package/reference/sql/db-con.md +4 -4
- package/reference/sql/db-connection.md +1 -1
- package/reference/sql/db-param.md +1 -1
- package/reference/sql/db-result-reader.md +4 -2
- package/reference/sql/db-tx.md +46 -14
- package/reference/sql/ddl.md +1 -1
- package/reference/sql/frag.md +44 -10
- package/reference/sql/index.md +7 -7
- package/reference/sql/repo.md +15 -15
- package/reference/sql/sql-dialect.md +1 -1
- package/reference/sql/sql-logger.md +1 -1
- package/reference/sql/sql-name-mapper.md +3 -3
- package/reference/sql/table-metadata.md +3 -3
- package/reference/sql/table.md +10 -10
- package/reference/sql/transactor-zio.md +1 -1
- package/reference/sql/transactor.md +21 -11
- package/reference/sql-zio.md +2 -2
- package/reference/streams/core/index.md +32 -0
- package/reference/streams/{pipeline.md → core/pipeline.md} +210 -74
- package/reference/streams/{sink.md → core/sink.md} +331 -353
- package/reference/streams/{stream.md → core/stream.md} +919 -209
- package/reference/streams/execution-and-compatibility/async-execution.md +822 -0
- package/reference/streams/execution-and-compatibility/index.md +35 -0
- package/reference/streams/execution-and-compatibility/platform-differences.md +297 -0
- package/reference/streams/execution-and-compatibility/scala-2-compatibility.md +88 -0
- package/reference/streams/execution-and-compatibility/zero-boxing.md +393 -0
- package/reference/streams/index.md +140 -67
- package/reference/streams/primitives/index.md +30 -0
- package/reference/streams/primitives/reader.md +1992 -0
- package/reference/streams/{writer.md → primitives/writer.md} +254 -98
- package/reference/telemetry/common/any-value.md +90 -0
- package/reference/telemetry/common/attribute-key.md +87 -0
- package/reference/telemetry/common/attributes.md +118 -0
- package/reference/telemetry/common/index.md +39 -0
- package/reference/telemetry/common/instrumentation-scope.md +24 -0
- package/reference/telemetry/common/resource.md +34 -0
- package/reference/telemetry/index.md +311 -0
- package/reference/telemetry/logging/index.md +197 -0
- package/reference/telemetry/logging/log-enrichment.md +72 -0
- package/reference/telemetry/logging/log-formatter.md +100 -0
- package/reference/telemetry/logging/log-record-processor.md +56 -0
- package/reference/telemetry/logging/log-record.md +44 -0
- package/reference/telemetry/logging/log-writer.md +64 -0
- package/reference/telemetry/logging/logger-provider.md +142 -0
- package/reference/telemetry/logging/logger.md +83 -0
- package/reference/telemetry/logging/severity.md +62 -0
- package/reference/telemetry/metrics/index.md +150 -0
- package/reference/telemetry/metrics/instruments.md +183 -0
- package/reference/telemetry/metrics/labeled-instruments.md +74 -0
- package/reference/telemetry/metrics/meter-provider.md +76 -0
- package/reference/telemetry/metrics/meter.md +98 -0
- package/reference/telemetry/metrics/metric-data.md +57 -0
- package/reference/telemetry/otel/custom-exporter.md +216 -0
- package/reference/telemetry/otel/index.md +212 -0
- package/reference/telemetry/tracing/index.md +155 -0
- package/reference/telemetry/tracing/sampler.md +89 -0
- package/reference/telemetry/tracing/span-builder.md +57 -0
- package/reference/telemetry/tracing/span-context.md +39 -0
- package/reference/telemetry/tracing/span-data.md +32 -0
- package/reference/telemetry/tracing/span-kind.md +55 -0
- package/reference/telemetry/tracing/span-processor.md +53 -0
- package/reference/telemetry/tracing/span-status.md +47 -0
- package/reference/telemetry/tracing/span.md +117 -0
- package/reference/telemetry/tracing/tracer-provider.md +91 -0
- package/reference/telemetry/tracing/tracer.md +52 -0
- package/reference/typeid.md +0 -64
- package/sidebars.js +365 -185
- package/undocumented-report.md +528 -270
- package/reference/config.md +0 -158
- package/reference/streams/concurrent-operators.md +0 -106
- package/reference/streams/reader.md +0 -1284
- package/reference/streams/scala-2-compatibility.md +0 -55
- package/reference/streams/zero-boxing.md +0 -275
- package/reference/telemetry.md +0 -693
|
@@ -0,0 +1,314 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: formats
|
|
3
|
+
title: "File Formats"
|
|
4
|
+
sidebar_label: "File Formats"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Three optional modules add file-format support: `config-yaml`, `config-json`, and `config-hocon`. Each contributes one constructor to the `ConfigSource` companion, parses its format into an in-memory tree, and flattens that tree into the dot-separated namespace every `ConfigSource` exposes. Supporting types: `YamlConfigSource`, `HoconValue`, `HoconError`. One constructor per format:
|
|
8
|
+
|
|
9
|
+
```scala
|
|
10
|
+
// with `import zio.blocks.config.yaml._`
|
|
11
|
+
ConfigSource.fromYaml(yaml: String, sourceId: String = "yaml:string"): Either[ConfigError, ConfigSource]
|
|
12
|
+
|
|
13
|
+
// with `import zio.blocks.config.json._`
|
|
14
|
+
ConfigSource.fromJson(json: String, sourceId: String = "json:string"): Either[ConfigError, ConfigSource]
|
|
15
|
+
|
|
16
|
+
// with `import zio.blocks.config.hocon._`
|
|
17
|
+
ConfigSource.fromHocon(hocon: String, sourceId: String = "hocon:string"): Either[ConfigError, ConfigSource]
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Motivation
|
|
21
|
+
|
|
22
|
+
A decoder that understood YAML, JSON, and HOCON natively would be three decoders with three sets of bugs. Flattening avoids that: each format is parsed by code that knows only that format, and the result is the same flat map of dotted keys the core module already handles.
|
|
23
|
+
|
|
24
|
+
The consequence is that everything above the source layer is format-blind. `ConfigSource#orElse` layers a YAML file under environment variables. `ConfigDecoder` reads a JSON document with the same rules it uses for a `Map`. Provenance reports `yaml:string` where it would otherwise report `env`. Nothing in the decoding path branches on format.
|
|
25
|
+
|
|
26
|
+
Keeping the adapters in separate artifacts means a service that only reads environment variables does not pull a YAML parser, and adding HOCON support is a build change rather than a code change.
|
|
27
|
+
|
|
28
|
+
## Common Flattening Rules
|
|
29
|
+
|
|
30
|
+
All three adapters agree on how a tree becomes flat keys. Learning the rules once tells you what key any document produces:
|
|
31
|
+
|
|
32
|
+
| Tree shape | Flattened form | Example |
|
|
33
|
+
| ------------------------- | ----------------------------------------- | ------------------------------------------- |
|
|
34
|
+
| Nested object or mapping | Keys joined with `.` | `db: { host: x }` → `db.host` = `x` |
|
|
35
|
+
| Array or sequence | Zero-based numeric segment | `hosts: [a, b]` → `hosts.0` = `a`, `hosts.1` = `b` |
|
|
36
|
+
| Scalar | The value as a string | `port: 8080` → `port` = `"8080"` |
|
|
37
|
+
| Null | **Omitted entirely** | `tag: null` → no `tag` key |
|
|
38
|
+
|
|
39
|
+
Two of these have consequences worth stating plainly.
|
|
40
|
+
|
|
41
|
+
Indexed sequences are exactly the shape `ConfigDecoder` probes for, so a YAML list decodes into a `List` without any extra configuration. See [Config Decoder](./config-decoder.md).
|
|
42
|
+
|
|
43
|
+
Null omission means an explicit null is indistinguishable from an absent key. A field of type `Option[A]` becomes `None`, a field with a default takes its default, and a required field reports `ConfigError.MissingKey`. There is no way to express "present but null" through a flattened source.
|
|
44
|
+
|
|
45
|
+
:::warning[Nulls disappear]
|
|
46
|
+
Writing `tag: null` to unset a value from a lower-priority layer does not work. `ConfigSource#orElse` sees no key at all, so the fallback still wins. Override with a real value instead.
|
|
47
|
+
:::
|
|
48
|
+
|
|
49
|
+
## YAML
|
|
50
|
+
|
|
51
|
+
`config-yaml` reads YAML through the `zio-blocks-schema-yaml` reader:
|
|
52
|
+
|
|
53
|
+
```scala
|
|
54
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-config-yaml" % "0.0.56"
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Importing the adapter package makes the constructor available on the `ConfigSource` companion:
|
|
58
|
+
|
|
59
|
+
```scala
|
|
60
|
+
import zio.blocks.config._
|
|
61
|
+
import zio.blocks.config.yaml._
|
|
62
|
+
|
|
63
|
+
val yamlText =
|
|
64
|
+
"""|db:
|
|
65
|
+
| host: localhost
|
|
66
|
+
| port: 5432
|
|
67
|
+
|hosts:
|
|
68
|
+
| - alpha
|
|
69
|
+
| - beta
|
|
70
|
+
|""".stripMargin
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Nested mappings become dotted keys and the sequence becomes indexed keys, all as strings:
|
|
74
|
+
|
|
75
|
+
```scala
|
|
76
|
+
ConfigSource.fromYaml(yamlText, "app.yaml")
|
|
77
|
+
// res0: Either[ConfigError, ConfigSource] = Right(
|
|
78
|
+
// MapSource(
|
|
79
|
+
// map = Map(
|
|
80
|
+
// "hosts.0" -> "alpha",
|
|
81
|
+
// "hosts.1" -> "beta",
|
|
82
|
+
// "db.port" -> "5432",
|
|
83
|
+
// "db.host" -> "localhost"
|
|
84
|
+
// ),
|
|
85
|
+
// sourceId = "app.yaml"
|
|
86
|
+
// )
|
|
87
|
+
// )
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
The resulting source behaves like any other, so decoding a case class needs nothing format-specific:
|
|
91
|
+
|
|
92
|
+
```scala
|
|
93
|
+
import zio.blocks.schema.Schema
|
|
94
|
+
|
|
95
|
+
case class Db(host: String, port: Int)
|
|
96
|
+
|
|
97
|
+
object Db {
|
|
98
|
+
implicit val schema: Schema[Db] = Schema.derived[Db]
|
|
99
|
+
}
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Re-rooting with `ConfigSource#prefix` reads the `db` section:
|
|
103
|
+
|
|
104
|
+
```scala
|
|
105
|
+
ConfigSource.fromYaml(yamlText, "app.yaml").map(src => Config.load[Db](src.prefix("db")))
|
|
106
|
+
// res1: Either[ConfigError, Either[::[ConfigError], Db]] = Right(
|
|
107
|
+
// Right(Db(host = "localhost", port = 5432))
|
|
108
|
+
// )
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
A document that will not parse yields `ConfigError.InvalidValue` whose `value` is the first 100 characters of the input, truncated with an ellipsis, and whose `cause` is the underlying reader exception. `YamlConfigSource.fromString` is the same operation without the syntax import, for code that prefers an explicit call.
|
|
112
|
+
|
|
113
|
+
Non-scalar mapping keys are skipped rather than rejected, since a dotted key namespace cannot represent them.
|
|
114
|
+
|
|
115
|
+
## JSON
|
|
116
|
+
|
|
117
|
+
`config-json` reads JSON through the `zio-blocks-schema-json` parser:
|
|
118
|
+
|
|
119
|
+
```scala
|
|
120
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-config-json" % "0.0.56"
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
The import and the constructor mirror the YAML adapter:
|
|
124
|
+
|
|
125
|
+
```scala
|
|
126
|
+
import zio.blocks.config._
|
|
127
|
+
import zio.blocks.config.json._
|
|
128
|
+
|
|
129
|
+
val jsonText =
|
|
130
|
+
"""|{
|
|
131
|
+
| "db": { "host": "localhost", "port": 5432 },
|
|
132
|
+
| "hosts": ["alpha", "beta"],
|
|
133
|
+
| "debug": true
|
|
134
|
+
|}
|
|
135
|
+
|""".stripMargin
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Numbers and booleans are rendered with their `toString`, so every value arrives as a string for the decoder to parse:
|
|
139
|
+
|
|
140
|
+
```scala
|
|
141
|
+
ConfigSource.fromJson(jsonText, "app.json")
|
|
142
|
+
// res3: Either[ConfigError, ConfigSource] = Right(
|
|
143
|
+
// MapSource(
|
|
144
|
+
// map = Map(
|
|
145
|
+
// "db.port" -> "5432",
|
|
146
|
+
// "hosts.0" -> "alpha",
|
|
147
|
+
// "hosts.1" -> "beta",
|
|
148
|
+
// "debug" -> "true",
|
|
149
|
+
// "db.host" -> "localhost"
|
|
150
|
+
// ),
|
|
151
|
+
// sourceId = "app.json"
|
|
152
|
+
// )
|
|
153
|
+
// )
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
A malformed document yields `ConfigError.InvalidValue` with a 100-character excerpt of the input and the parser's `SchemaError` as the cause.
|
|
157
|
+
|
|
158
|
+
## HOCON
|
|
159
|
+
|
|
160
|
+
`config-hocon` has its own parser with no external dependency, and it is the only adapter that supports substitutions and includes:
|
|
161
|
+
|
|
162
|
+
```scala
|
|
163
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-config-hocon" % "0.0.56"
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
### Parsing a String
|
|
167
|
+
|
|
168
|
+
The package object mixes in both the shared and platform-specific syntax, so one import brings in every constructor available on the current platform:
|
|
169
|
+
|
|
170
|
+
```scala
|
|
171
|
+
import zio.blocks.config._
|
|
172
|
+
import zio.blocks.config.hocon._
|
|
173
|
+
|
|
174
|
+
val hoconText =
|
|
175
|
+
"""|db {
|
|
176
|
+
| host = "localhost"
|
|
177
|
+
| port = 5432
|
|
178
|
+
|}
|
|
179
|
+
|""".stripMargin
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
Braces nest into dotted keys, and an integral number flattens without a decimal point — `5432`, not `5432.0`:
|
|
183
|
+
|
|
184
|
+
```scala
|
|
185
|
+
ConfigSource.fromHocon(hoconText, "app.conf")
|
|
186
|
+
// res5: Either[ConfigError, ConfigSource] = Right(
|
|
187
|
+
// MapSource(
|
|
188
|
+
// map = Map("db.host" -> "localhost", "db.port" -> "5432"),
|
|
189
|
+
// sourceId = "app.conf"
|
|
190
|
+
// )
|
|
191
|
+
// )
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
That integral-number handling matters because a value rendered as `5432.0` would not parse as an `Int`. Non-integral and infinite values keep their `Double` rendering.
|
|
195
|
+
|
|
196
|
+
### Substitutions
|
|
197
|
+
|
|
198
|
+
Parsing runs in two passes: the text is read into a tree that may contain unresolved `${...}` references, then those references are resolved. Substitution therefore happens **before** flattening, so a substituted value reaches the source layer already expanded:
|
|
199
|
+
|
|
200
|
+
```scala
|
|
201
|
+
import zio.blocks.config._
|
|
202
|
+
import zio.blocks.config.hocon._
|
|
203
|
+
|
|
204
|
+
val withSubstitution =
|
|
205
|
+
"""|base = "/srv/app"
|
|
206
|
+
|paths {
|
|
207
|
+
| data = ${base}/data
|
|
208
|
+
| logs = ${base}/logs
|
|
209
|
+
|}
|
|
210
|
+
|""".stripMargin
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
The flattened keys hold expanded strings, and nothing downstream needs to know a substitution was involved:
|
|
214
|
+
|
|
215
|
+
```scala
|
|
216
|
+
ConfigSource.fromHocon(withSubstitution, "app.conf")
|
|
217
|
+
// res7: Either[ConfigError, ConfigSource] = Right(
|
|
218
|
+
// MapSource(
|
|
219
|
+
// map = Map(
|
|
220
|
+
// "base" -> "/srv/app",
|
|
221
|
+
// "paths.data" -> "/srv/app/data",
|
|
222
|
+
// "paths.logs" -> "/srv/app/logs"
|
|
223
|
+
// ),
|
|
224
|
+
// sourceId = "app.conf"
|
|
225
|
+
// )
|
|
226
|
+
// )
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
The resolution pass detects circular references and reports them as a parse failure rather than looping.
|
|
230
|
+
|
|
231
|
+
### Includes
|
|
232
|
+
|
|
233
|
+
`include "file"` directives are resolved through a callback, because the core parser does not assume a filesystem. `HoconParser.parse` accepts a `String => Option[String]` that returns the contents of a named resource, which lets includes come from a classpath, an archive, or a test fixture.
|
|
234
|
+
|
|
235
|
+
On the JVM, `ConfigSource.fromFile` wires that callback to the filesystem for you.
|
|
236
|
+
|
|
237
|
+
### Loading from a File
|
|
238
|
+
|
|
239
|
+
`ConfigSource.fromFile` is JVM-only and resolves includes relative to the including file's directory:
|
|
240
|
+
|
|
241
|
+
```scala
|
|
242
|
+
import zio.blocks.config._
|
|
243
|
+
import zio.blocks.config.hocon._
|
|
244
|
+
|
|
245
|
+
ConfigSource.fromFile("conf/application.conf")
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
Two parameters guard against a hostile or accidentally-recursive configuration tree:
|
|
249
|
+
|
|
250
|
+
| Parameter | Default | Effect |
|
|
251
|
+
| ------------------ | ------- | -------------------------------------------------------------------------------- |
|
|
252
|
+
| `allowedBase` | `None` | When set, both the file and every include must resolve inside this directory. |
|
|
253
|
+
| `maxIncludeDepth` | `10` | Maximum nesting depth for includes; exceeding it is a parse failure. |
|
|
254
|
+
|
|
255
|
+
Pass `allowedBase` whenever the path comes from outside the program. Paths are canonicalized before the check, so `../` sequences cannot escape:
|
|
256
|
+
|
|
257
|
+
```scala
|
|
258
|
+
import java.io.File
|
|
259
|
+
import zio.blocks.config._
|
|
260
|
+
import zio.blocks.config.hocon._
|
|
261
|
+
|
|
262
|
+
ConfigSource.fromFile(
|
|
263
|
+
path = "conf/application.conf",
|
|
264
|
+
allowedBase = Some(new File("conf"))
|
|
265
|
+
)
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
Failures are reported as `ConfigError.ParseError`: a missing file expects `"existing file"`, a path outside `allowedBase` expects `"path inside <base>"`, and a malformed document carries the `HoconError` as its cause. The `sourceId` of a file-loaded source is `hocon:` followed by the file's name.
|
|
269
|
+
|
|
270
|
+
### HoconValue
|
|
271
|
+
|
|
272
|
+
`HoconValue` is the parsed tree, exposed for code that needs the structure rather than a flat source:
|
|
273
|
+
|
|
274
|
+
```scala
|
|
275
|
+
sealed trait HoconValue
|
|
276
|
+
|
|
277
|
+
object HoconValue {
|
|
278
|
+
final case class Obj(fields: Map[String, HoconValue]) extends HoconValue
|
|
279
|
+
final case class Arr(elements: Seq[HoconValue]) extends HoconValue
|
|
280
|
+
final case class Str(value: String) extends HoconValue
|
|
281
|
+
final case class Num(value: Double) extends HoconValue
|
|
282
|
+
final case class Bool(value: Boolean) extends HoconValue
|
|
283
|
+
case object Null extends HoconValue
|
|
284
|
+
}
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
`HoconValue.flatten` performs the flattening the adapter uses, and `HoconValue.deepMerge` merges two trees recursively with the right side winning on conflict — the operation a layered HOCON setup needs before flattening, as opposed to `ConfigSource#orElse`, which layers after.
|
|
288
|
+
|
|
289
|
+
`HoconError` carries a message with the line and column where parsing failed, and extends `Exception` so it can be thrown or attached as a cause:
|
|
290
|
+
|
|
291
|
+
```scala
|
|
292
|
+
final case class HoconError(message: String, line: Int, column: Int)
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
## Choosing a Format
|
|
296
|
+
|
|
297
|
+
The adapters differ in capability, not in how their output is consumed:
|
|
298
|
+
|
|
299
|
+
| Capability | YAML | JSON | HOCON |
|
|
300
|
+
| ----------------------------- | ---- | ---- | ----- |
|
|
301
|
+
| Nested objects and arrays | Yes | Yes | Yes |
|
|
302
|
+
| Comments | Yes | No | Yes |
|
|
303
|
+
| Substitutions (`${...}`) | No | No | Yes |
|
|
304
|
+
| Includes | No | No | Yes |
|
|
305
|
+
| Load from a file | No | No | Yes (JVM) |
|
|
306
|
+
| Scala.js | Yes | Yes | Yes (string only) |
|
|
307
|
+
|
|
308
|
+
For everything else, prefer the format your deployment already uses. Because all three flatten identically, switching later changes one constructor call.
|
|
309
|
+
|
|
310
|
+
## Integration Points
|
|
311
|
+
|
|
312
|
+
Each adapter depends on the core `config` module for `ConfigSource` and `ConfigError`. YAML and JSON additionally depend on `zio-blocks-schema-yaml` and `zio-blocks-schema-json` for parsing; HOCON has no dependency beyond the core module.
|
|
313
|
+
|
|
314
|
+
Adapters produce nothing but a `ConfigSource`, so see [ConfigSource](./config-source.md) for composition and provenance, and [Config Decoder](./config-decoder.md) for how the flattened keys become typed values.
|
|
@@ -0,0 +1,304 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: index
|
|
3
|
+
title: "Configuration & Feature Flags"
|
|
4
|
+
sidebar_label: "Configuration & Feature Flags"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
`zio.blocks.config` loads typed configuration from string-keyed sources, tracks where every resolved value came from, and evaluates feature flags with percentage-based rollouts. The module is synchronous and zero-dependency. Core types: `ConfigSource`, `ConfigDecoder`, `ConfigError`, `Provenance`, `FlagSource`, `StaticFlag`, `DynamicFlag`, `Rollout`. The shapes at the center of the module:
|
|
8
|
+
|
|
9
|
+
```scala
|
|
10
|
+
trait ConfigSource extends FlagSource {
|
|
11
|
+
def sourceId: String
|
|
12
|
+
def get(key: String): Maybe[SourceValue[String]]
|
|
13
|
+
def all(prefix: String): Map[String, SourceValue[String]]
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
trait ConfigDecoder[A] {
|
|
17
|
+
def decode(source: ConfigSource, prefix: String): Either[::[ConfigError], A]
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
final case class SourceValue[A](value: A, provenance: Provenance)
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Introduction
|
|
24
|
+
|
|
25
|
+
Configuration in this module is a two-layer story. The bottom layer is a flat, string-to-string namespace: a `ConfigSource` answers `get("db.host")` with a raw string plus a record of where that string came from. The top layer turns that namespace into typed values: a `ConfigDecoder[A]`, derived from `Schema[A]`, walks your case class and pulls one key per field.
|
|
26
|
+
|
|
27
|
+
Everything in between — composing sources, renaming keys to match an environment's conventions, accumulating errors instead of failing on the first one, redacting secrets when you print what was loaded — happens without an effect type. There is no `IO` wrapper: loading configuration is a synchronous function call that returns `Either`.
|
|
28
|
+
|
|
29
|
+
Feature flags reuse the same source abstraction. `FlagSource` is the scalar half of `ConfigSource`, so one source object can answer both typed config lookups and flag lookups.
|
|
30
|
+
|
|
31
|
+
## Motivation
|
|
32
|
+
|
|
33
|
+
Most configuration libraries make you choose between two unpleasant options: a stringly-typed map that compiles but fails at runtime, or a typed API bolted to a specific effect system and a specific file format. This module avoids both.
|
|
34
|
+
|
|
35
|
+
- **Typed without ceremony.** A `Schema[A]` you already have — or that `Schema.derived` writes for you — is all a decoder needs. There is no separate config-description DSL to learn and keep in sync with the case class.
|
|
36
|
+
- **Errors accumulate.** A five-field config with three bad fields reports three errors, not the first one. `ConfigError.Composite` carries them all.
|
|
37
|
+
- **Provenance is not an afterthought.** Every value arrives paired with the source that produced it, so "why is this port 8080?" has an answer you can print.
|
|
38
|
+
- **Format-agnostic.** YAML, JSON, and HOCON adapters all flatten to the same dot-separated namespace, so decoding, composition, and provenance work identically regardless of where the bytes came from.
|
|
39
|
+
- **No dependencies, both platforms.** The core module runs on the JVM and Scala.js. Only file-based HOCON loading is JVM-only.
|
|
40
|
+
|
|
41
|
+
## Installation
|
|
42
|
+
|
|
43
|
+
The core module provides sources, decoding, and flags:
|
|
44
|
+
|
|
45
|
+
```scala
|
|
46
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-config" % "0.0.56"
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Each file format is a separate artifact, so you depend only on the ones you use:
|
|
50
|
+
|
|
51
|
+
```scala
|
|
52
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-config-yaml" % "0.0.56"
|
|
53
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-config-json" % "0.0.56"
|
|
54
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-config-hocon" % "0.0.56"
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
For Scala.js, use `%%%` instead of `%%`:
|
|
58
|
+
|
|
59
|
+
```scala
|
|
60
|
+
libraryDependencies += "dev.zio" %%% "zio-blocks-config" % "0.0.56"
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Supported Scala versions: 2.13.x and 3.x.
|
|
64
|
+
|
|
65
|
+
## Overview
|
|
66
|
+
|
|
67
|
+
The module divides into three groups of types: the source layer that produces raw strings, the decoding layer that produces typed values, and the flag layer that resolves runtime switches.
|
|
68
|
+
|
|
69
|
+
### The Source Layer
|
|
70
|
+
|
|
71
|
+
`ConfigSource` is a flat key-value namespace with dot-separated paths. `ConfigSource.fromMap` builds one from a `Map`, `EnvSource` reads environment variables, and `SysPropSource` reads system properties. Sources compose with `ConfigSource#orElse` and can be re-rooted with `ConfigSource#prefix`. See [Config Source](./config-source.md).
|
|
72
|
+
|
|
73
|
+
`SourceValue` pairs a raw value with a `Provenance` describing its origin, and `ProvenanceMap` lets you query and print those origins after a successful load. Also on that page: `KeyMapper` and `KeyFormat`, which translate between your canonical camelCase field names and whatever casing the environment actually uses.
|
|
74
|
+
|
|
75
|
+
### The Decoding Layer
|
|
76
|
+
|
|
77
|
+
`ConfigDecoder[A]` turns a source into an `A`, accumulating every failure. `ConfigDecoderDeriver` is the `Deriver` that builds those decoders from `Schema[A]`, and it defines the mapping rules: how nested case classes become dotted paths, how sealed traits pick a case from a discriminator key, how sequences and maps are keyed. See [Config Decoder](./config-decoder.md).
|
|
78
|
+
|
|
79
|
+
`ConfigError` is the failure type, split into four category traits so you can match on the kind of problem rather than the specific constructor. See [Errors](./errors.md).
|
|
80
|
+
|
|
81
|
+
### The Flag Layer
|
|
82
|
+
|
|
83
|
+
`StaticFlag[A]` resolves once at class-load time and never changes. `DynamicFlag[A]` holds a rollout expression that can be updated or reloaded while the process runs. Both derive their names from the enclosing Scala object and register themselves globally. See [Flags](./flags.md).
|
|
84
|
+
|
|
85
|
+
`Rollout` is the expression language behind dynamic flags: a semicolon-separated list of choices, each optionally targeted at a path pattern and a percentage bucket. See [Rollout](./rollout.md).
|
|
86
|
+
|
|
87
|
+
### The Format Adapters
|
|
88
|
+
|
|
89
|
+
`config-yaml`, `config-json`, and `config-hocon` each add one `ConfigSource` constructor. They differ in what they support — only HOCON has substitutions and includes — but they agree on the flattening rules. See [File Formats](./formats.md).
|
|
90
|
+
|
|
91
|
+
## How They Work Together
|
|
92
|
+
|
|
93
|
+
A typed load moves through four stages. The source produces raw strings, the decoder walks the schema requesting one key per field, errors accumulate rather than short-circuit, and the result is either a fully-built value or every problem found along the way:
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
1. Build a source ConfigSource.fromMap / fromYaml / EnvSource
|
|
97
|
+
2. Compose and re-root source.orElse(fallback).prefix("app")
|
|
98
|
+
3. Derive a decoder ConfigDecoder.derive[A] (from Schema[A])
|
|
99
|
+
4. Decode decoder.decode(source, "")
|
|
100
|
+
│
|
|
101
|
+
├─ Right(a) all fields resolved
|
|
102
|
+
└─ Left(errors) every failure, accumulated
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
The type relationships behind those stages:
|
|
106
|
+
|
|
107
|
+
```
|
|
108
|
+
FlagSource ─────────────────┐ get(name): Maybe[SourceValue[String]]
|
|
109
|
+
▲ │
|
|
110
|
+
│ extends │
|
|
111
|
+
ConfigSource ───────────────┤ + all(prefix): Map[String, SourceValue[String]]
|
|
112
|
+
│ │
|
|
113
|
+
├─ MapSource │
|
|
114
|
+
├─ EnvSource │ DB_HOST ← db.host
|
|
115
|
+
├─ SysPropSource │
|
|
116
|
+
└─ (yaml/json/hocon) │ flattened to dotted keys
|
|
117
|
+
│
|
|
118
|
+
└─> SourceValue ──> Provenance
|
|
119
|
+
├─ Resolved(sourceId, key, rawValue)
|
|
120
|
+
└─ Default
|
|
121
|
+
|
|
122
|
+
Schema[A] ──deriving──> ConfigDecoderDeriver ──> ConfigDecoder[A]
|
|
123
|
+
│
|
|
124
|
+
decode(source, prefix)
|
|
125
|
+
│
|
|
126
|
+
┌─────────────────┴──────────────────┐
|
|
127
|
+
▼ ▼
|
|
128
|
+
Right(A) Left(::[ConfigError])
|
|
129
|
+
├─ MissingKey
|
|
130
|
+
├─ InvalidValue
|
|
131
|
+
├─ ParseError
|
|
132
|
+
└─ Composite(all)
|
|
133
|
+
|
|
134
|
+
StaticFlag[A] ──resolves once──> Registry → sysprop → env → default
|
|
135
|
+
DynamicFlag[A] ──evaluates──> Rollout.Choices x bucket → value
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
## Common Patterns
|
|
139
|
+
|
|
140
|
+
Four shapes cover most usage: a straight typed load, a layered load where environment variables win over file defaults, wiring config into a dependency graph, and loading with provenance so you can explain what happened.
|
|
141
|
+
|
|
142
|
+
### Loading a Typed Value
|
|
143
|
+
|
|
144
|
+
`Config.load` derives a decoder and runs it, returning every error it found:
|
|
145
|
+
|
|
146
|
+
```scala
|
|
147
|
+
import zio.blocks.config._
|
|
148
|
+
import zio.blocks.schema.Schema
|
|
149
|
+
|
|
150
|
+
case class Db(host: String, port: Int)
|
|
151
|
+
|
|
152
|
+
object Db {
|
|
153
|
+
implicit val schema: Schema[Db] = Schema.derived[Db]
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
val source = ConfigSource.fromMap(Map("host" -> "localhost", "port" -> "5432"), "example")
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
Decoding returns `Either[::[ConfigError], Db]`, where `::` is Scala's non-empty list, so a `Left` always carries at least one error:
|
|
160
|
+
|
|
161
|
+
```scala
|
|
162
|
+
Config.load[Db](source)
|
|
163
|
+
// res0: Either[::[ConfigError], Db] = Right(
|
|
164
|
+
// Db(host = "localhost", port = 5432)
|
|
165
|
+
// )
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
When a key is missing, the error names both the path and the source that was searched:
|
|
169
|
+
|
|
170
|
+
```scala
|
|
171
|
+
Config.load[Db](ConfigSource.fromMap(Map("host" -> "localhost"), "example"))
|
|
172
|
+
// res1: Either[::[ConfigError], Db] = Left(
|
|
173
|
+
// List(MissingKey(path = "port", source = "example"))
|
|
174
|
+
// )
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
### Layering Sources
|
|
178
|
+
|
|
179
|
+
Production configuration usually comes from more than one place: a file provides defaults and the environment overrides them. `ConfigSource#orElse` consults the receiver first and falls back to the argument, and provenance still records which one actually answered:
|
|
180
|
+
|
|
181
|
+
```scala
|
|
182
|
+
import zio.blocks.config._
|
|
183
|
+
import zio.blocks.schema.Schema
|
|
184
|
+
|
|
185
|
+
case class Db(host: String, port: Int)
|
|
186
|
+
|
|
187
|
+
object Db {
|
|
188
|
+
implicit val schema: Schema[Db] = Schema.derived[Db]
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
val defaults = ConfigSource.fromMap(Map("host" -> "localhost", "port" -> "5432"), "defaults")
|
|
192
|
+
val envOverrides = ConfigSource.fromMap(Map("host" -> "db.prod.internal"), "env")
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
The override wins for `host`, and `port` falls through to the defaults:
|
|
196
|
+
|
|
197
|
+
```scala
|
|
198
|
+
Config.load[Db](envOverrides.orElse(defaults))
|
|
199
|
+
// res3: Either[::[ConfigError], Db] = Right(
|
|
200
|
+
// Db(host = "db.prod.internal", port = 5432)
|
|
201
|
+
// )
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
### Wiring Config into a Dependency Graph
|
|
205
|
+
|
|
206
|
+
Rather than decoding at startup and threading the result through constructors, `Config.wire` produces a `Wire.Shared[ConfigSource, A]` so decoding becomes a node in the graph. The `ConfigSource` is the injected input; the typed config is the derived output:
|
|
207
|
+
|
|
208
|
+
```scala
|
|
209
|
+
import zio.blocks.config._
|
|
210
|
+
import zio.blocks.context.Context
|
|
211
|
+
import zio.blocks.schema.Schema
|
|
212
|
+
import zio.blocks.scope.{Resource, Scope, Unscoped, Wire}
|
|
213
|
+
|
|
214
|
+
case class Db(host: String, port: Int)
|
|
215
|
+
|
|
216
|
+
object Db {
|
|
217
|
+
implicit val schema: Schema[Db] = Schema.derived[Db]
|
|
218
|
+
implicit val unscoped: Unscoped[Db] = Unscoped.derived[Db]
|
|
219
|
+
}
|
|
220
|
+
|
|
221
|
+
final class DbService(val config: Db)
|
|
222
|
+
|
|
223
|
+
val source = ConfigSource.fromMap(Map("host" -> "localhost", "port" -> "5432"), "graph")
|
|
224
|
+
val resource = Resource.from[DbService](Wire(source), Config.wire[Db])
|
|
225
|
+
|
|
226
|
+
Scope.global.scoped { scope =>
|
|
227
|
+
import scope._
|
|
228
|
+
val service = allocate(resource)
|
|
229
|
+
$(service)(_.config)
|
|
230
|
+
}
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
`Config.wire[A](prefix)` re-roots the injected source before decoding, which is how you keep several config sections in one graph without giving each its own source.
|
|
234
|
+
|
|
235
|
+
### Loading with Provenance
|
|
236
|
+
|
|
237
|
+
`Config.loadWithProvenance` returns a `ProvenanceMap[A]` alongside the value, which answers per-key origin questions and renders a table for startup logs:
|
|
238
|
+
|
|
239
|
+
```scala
|
|
240
|
+
import zio.blocks.config._
|
|
241
|
+
import zio.blocks.schema.Schema
|
|
242
|
+
|
|
243
|
+
case class Db(host: String, port: Int)
|
|
244
|
+
|
|
245
|
+
object Db {
|
|
246
|
+
implicit val schema: Schema[Db] = Schema.derived[Db]
|
|
247
|
+
}
|
|
248
|
+
|
|
249
|
+
val source = ConfigSource.fromMap(Map("host" -> "localhost", "port" -> "5432"), "startup")
|
|
250
|
+
val loaded = Config.loadWithProvenance[Db](source)
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
Querying a single key reports the source id, the source-facing key, and the raw string:
|
|
254
|
+
|
|
255
|
+
```scala
|
|
256
|
+
loaded.map(_.provenanceOf("host"))
|
|
257
|
+
// res6: Either[::[ConfigError], Maybe[Provenance]] = Right(
|
|
258
|
+
// Resolved(sourceId = "startup", key = "host", rawValue = "localhost")
|
|
259
|
+
// )
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
`ProvenanceMap#dump` renders every visible key as a table, redacting values whose key names look sensitive:
|
|
263
|
+
|
|
264
|
+
```scala
|
|
265
|
+
loaded.map(_.dump()).foreach(println)
|
|
266
|
+
// ┌──────┬───────────┬─────────┐
|
|
267
|
+
// │ Key │ Value │ Source │
|
|
268
|
+
// ├──────┼───────────┼─────────┤
|
|
269
|
+
// │ host │ localhost │ startup │
|
|
270
|
+
// │ port │ 5432 │ startup │
|
|
271
|
+
// └──────┴───────────┴─────────┘
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
## Entry Points
|
|
275
|
+
|
|
276
|
+
`Config` is the front door for typed loading. Which method you want depends on how you handle failure and whether you need provenance:
|
|
277
|
+
|
|
278
|
+
| Method | Returns | Use when |
|
|
279
|
+
| ----------------------------- | ------------------------------------ | --------------------------------------------------------------- |
|
|
280
|
+
| `Config.load[A]` | `Either[::[ConfigError], A]` | You want to inspect or report errors yourself. |
|
|
281
|
+
| `Config.loadOrThrow[A]` | `A` | Failure should abort startup; throws `ConfigLoadException`. |
|
|
282
|
+
| `Config.loadWithProvenance[A]`| `Either[::[ConfigError], ProvenanceMap[A]]` | You need to explain where values came from. |
|
|
283
|
+
| `Config.wire[A]` | `Wire.Shared[ConfigSource, A]` | Decoding belongs inside a dependency graph. |
|
|
284
|
+
|
|
285
|
+
`Config.load` derives a fresh decoder on every call. When you load the same type repeatedly — during reloads, or per request — derive once with `ConfigDecoder.derive[A]` and reuse the result.
|
|
286
|
+
|
|
287
|
+
:::warning[Derivation is not free]
|
|
288
|
+
`Schema.derived` runs at compile time, but turning a `Schema[A]` into a `ConfigDecoder[A]` happens at runtime and walks the whole schema. Calling `Config.load[A]` in a hot path re-does that walk every time.
|
|
289
|
+
:::
|
|
290
|
+
|
|
291
|
+
## Integration Points
|
|
292
|
+
|
|
293
|
+
The module sits on top of three other blocks and hands its output to a fourth:
|
|
294
|
+
|
|
295
|
+
- **`zio-blocks-schema`** supplies `Schema[A]`, and `ConfigDecoderDeriver` is an ordinary `Deriver[ConfigDecoder]`. Anything the schema layer can describe — records, variants, sequences, maps, wrappers, dynamic values — has a decoding rule.
|
|
296
|
+
- **`zio-blocks-maybe`** supplies `Maybe`, the allocation-free option type returned by every source lookup.
|
|
297
|
+
- **`zio-blocks-scope`** supplies `Wire` and `Resource`, which is what `Config.wire` produces so config can participate in dependency graphs.
|
|
298
|
+
- **`zio-blocks-schema-yaml`** and **`zio-blocks-schema-json`** back the YAML and JSON adapters; the HOCON adapter has its own parser and no external dependency.
|
|
299
|
+
|
|
300
|
+
Within the module, the dependency direction is one-way: sources know nothing about decoders, decoders know nothing about flags, and flags reuse `FlagSource` without reaching into `ConfigDecoder`.
|
|
301
|
+
|
|
302
|
+
## Next Steps
|
|
303
|
+
|
|
304
|
+
Start with [Config Source](./config-source.md) to build and compose sources, then [Config Decoder](./config-decoder.md) for the mapping rules from case classes to keys. [Errors](./errors.md) covers the failure model, [Flags](./flags.md) and [Rollout](./rollout.md) cover runtime switches, and [File Formats](./formats.md) covers YAML, JSON, and HOCON.
|