@zio.dev/zio-blocks 0.0.51 → 0.0.56
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/adr/2026-07-18-data-migration.md +123 -0
- package/guides/async-getting-started.md +687 -0
- package/guides/compile-time-resource-safety-with-scope.md +6 -0
- package/guides/getting-started-with-mux.md +0 -112
- package/guides/query-dsl-extending.md +1 -1
- package/guides/query-dsl-fluent-builder.md +1 -1
- package/guides/query-dsl-reified-optics.md +1 -1
- package/guides/query-dsl-sql.md +395 -1
- package/guides/sql-checked-interpolation.md +173 -0
- package/guides/sql-transactions.md +286 -0
- package/guides/telemetry-guide.md +131 -70
- package/guides/zio-schema-migration.md +6 -6
- package/index.md +200 -559
- package/package.json +1 -1
- package/reference/async.md +1379 -531
- package/reference/chunk.md +3 -3
- package/reference/codegen/index.md +1 -1
- package/reference/combinators.md +4 -4
- package/reference/config/config-decoder.md +460 -0
- package/reference/config/config-source.md +489 -0
- package/reference/config/errors.md +278 -0
- package/reference/config/flags.md +369 -0
- package/reference/config/formats.md +314 -0
- package/reference/config/index.md +304 -0
- package/reference/config/rollout.md +336 -0
- package/reference/context.md +6 -49
- package/reference/data-migration.md +269 -0
- package/reference/datastar/attributes.md +302 -0
- package/reference/datastar/events.md +234 -0
- package/reference/datastar/index.md +256 -0
- package/reference/datastar/signals.md +230 -0
- package/reference/datastar/sse.md +295 -0
- package/reference/datastar.md +2 -2
- package/reference/docs.md +2 -2
- package/reference/endpoint/bulk-creation.md +96 -0
- package/reference/endpoint/endpoint.md +1 -0
- package/reference/endpoint/index.md +9 -89
- package/reference/endpoint/path-codec.md +12 -24
- package/reference/endpoint/route-pattern.md +4 -6
- package/reference/endpoint/segment-codec.md +19 -32
- package/reference/html.md +313 -9
- package/reference/htmx/index.md +4 -52
- package/reference/htmx/response-headers.md +240 -0
- package/reference/http-model/headers.md +735 -0
- package/reference/http-model/index.md +3 -1
- package/reference/http-model/model.md +107 -71
- package/reference/http-model/schema-codecs.md +522 -0
- package/reference/http-model/schema.md +6 -3
- package/reference/http-model/server-sent-event.md +341 -0
- package/reference/jwt.md +195 -0
- package/reference/maybe.md +128 -11
- package/reference/media-type.md +2 -2
- package/reference/mux.mdx +7 -2
- package/reference/openapi.md +3 -3
- package/reference/projection.md +654 -0
- package/reference/resource-management/index.md +1 -1
- package/reference/resource-management/resource.md +2 -98
- package/reference/resource-management/scope.md +1 -209
- package/reference/resource-management/wire.md +4 -50
- package/reference/ringbuffer/advanced.mdx +1 -1
- package/reference/ringbuffer/index.mdx +3 -3
- package/reference/ringbuffer/mpmc.mdx +38 -4
- package/reference/ringbuffer/mpsc.mdx +36 -4
- package/reference/ringbuffer/spmc.mdx +1 -1
- package/reference/ringbuffer/spsc.mdx +87 -15
- package/reference/schema/allows.md +0 -96
- package/reference/schema/binding.md +2 -2
- package/reference/schema/built-in-codecs/avro.md +2 -2
- package/reference/schema/built-in-codecs/bson.md +50 -20
- package/reference/schema/built-in-codecs/csv.md +2 -2
- package/reference/schema/built-in-codecs/index.md +3 -3
- package/reference/schema/built-in-codecs/json/index.md +2 -2
- package/reference/schema/built-in-codecs/json/json.md +1 -0
- package/reference/schema/built-in-codecs/messagepack.md +3 -3
- package/reference/schema/built-in-codecs/thrift.md +2 -2
- package/reference/schema/built-in-codecs/toon.md +3 -3
- package/reference/schema/built-in-codecs/yaml.md +2 -2
- package/reference/schema/codec.md +11 -11
- package/reference/schema/dynamic-optic.md +48 -3
- package/reference/schema/dynamic-schema.md +3 -3
- package/reference/schema/index.md +2 -0
- package/reference/schema/path-interpolator.md +2 -0
- package/reference/schema/reflect-transformer.md +140 -0
- package/reference/schema/schema-evolution/as.md +4 -4
- package/reference/schema/schema-evolution/into.md +2 -2
- package/reference/schema/schema-expr.md +2 -2
- package/reference/schema/schema-search.md +263 -0
- package/reference/schema/schema.md +10 -2
- package/reference/schema/type-class-derivation.md +1 -1
- package/reference/smithy.md +502 -3
- package/reference/sql/db-codec-deriver.md +3 -3
- package/reference/sql/db-codec.md +22 -22
- package/reference/sql/db-con.md +4 -4
- package/reference/sql/db-connection.md +1 -1
- package/reference/sql/db-param.md +1 -1
- package/reference/sql/db-result-reader.md +4 -2
- package/reference/sql/db-tx.md +46 -14
- package/reference/sql/ddl.md +1 -1
- package/reference/sql/frag.md +44 -10
- package/reference/sql/index.md +7 -7
- package/reference/sql/repo.md +15 -15
- package/reference/sql/sql-dialect.md +1 -1
- package/reference/sql/sql-logger.md +1 -1
- package/reference/sql/sql-name-mapper.md +3 -3
- package/reference/sql/table-metadata.md +3 -3
- package/reference/sql/table.md +10 -10
- package/reference/sql/transactor-zio.md +1 -1
- package/reference/sql/transactor.md +21 -11
- package/reference/sql-zio.md +2 -2
- package/reference/streams/core/index.md +32 -0
- package/reference/streams/{pipeline.md → core/pipeline.md} +210 -74
- package/reference/streams/{sink.md → core/sink.md} +331 -353
- package/reference/streams/{stream.md → core/stream.md} +919 -209
- package/reference/streams/execution-and-compatibility/async-execution.md +822 -0
- package/reference/streams/execution-and-compatibility/index.md +35 -0
- package/reference/streams/execution-and-compatibility/platform-differences.md +297 -0
- package/reference/streams/execution-and-compatibility/scala-2-compatibility.md +88 -0
- package/reference/streams/execution-and-compatibility/zero-boxing.md +393 -0
- package/reference/streams/index.md +140 -67
- package/reference/streams/primitives/index.md +30 -0
- package/reference/streams/primitives/reader.md +1992 -0
- package/reference/streams/{writer.md → primitives/writer.md} +254 -98
- package/reference/telemetry/common/any-value.md +90 -0
- package/reference/telemetry/common/attribute-key.md +87 -0
- package/reference/telemetry/common/attributes.md +118 -0
- package/reference/telemetry/common/index.md +39 -0
- package/reference/telemetry/common/instrumentation-scope.md +24 -0
- package/reference/telemetry/common/resource.md +34 -0
- package/reference/telemetry/index.md +311 -0
- package/reference/telemetry/logging/index.md +197 -0
- package/reference/telemetry/logging/log-enrichment.md +72 -0
- package/reference/telemetry/logging/log-formatter.md +100 -0
- package/reference/telemetry/logging/log-record-processor.md +56 -0
- package/reference/telemetry/logging/log-record.md +44 -0
- package/reference/telemetry/logging/log-writer.md +64 -0
- package/reference/telemetry/logging/logger-provider.md +142 -0
- package/reference/telemetry/logging/logger.md +83 -0
- package/reference/telemetry/logging/severity.md +62 -0
- package/reference/telemetry/metrics/index.md +150 -0
- package/reference/telemetry/metrics/instruments.md +183 -0
- package/reference/telemetry/metrics/labeled-instruments.md +74 -0
- package/reference/telemetry/metrics/meter-provider.md +76 -0
- package/reference/telemetry/metrics/meter.md +98 -0
- package/reference/telemetry/metrics/metric-data.md +57 -0
- package/reference/telemetry/otel/custom-exporter.md +216 -0
- package/reference/telemetry/otel/index.md +212 -0
- package/reference/telemetry/tracing/index.md +155 -0
- package/reference/telemetry/tracing/sampler.md +89 -0
- package/reference/telemetry/tracing/span-builder.md +57 -0
- package/reference/telemetry/tracing/span-context.md +39 -0
- package/reference/telemetry/tracing/span-data.md +32 -0
- package/reference/telemetry/tracing/span-kind.md +55 -0
- package/reference/telemetry/tracing/span-processor.md +53 -0
- package/reference/telemetry/tracing/span-status.md +47 -0
- package/reference/telemetry/tracing/span.md +117 -0
- package/reference/telemetry/tracing/tracer-provider.md +91 -0
- package/reference/telemetry/tracing/tracer.md +52 -0
- package/reference/typeid.md +0 -64
- package/sidebars.js +365 -185
- package/undocumented-report.md +528 -270
- package/reference/config.md +0 -158
- package/reference/streams/concurrent-operators.md +0 -106
- package/reference/streams/reader.md +0 -1284
- package/reference/streams/scala-2-compatibility.md +0 -55
- package/reference/streams/zero-boxing.md +0 -275
- package/reference/telemetry.md +0 -693
|
@@ -3,21 +3,21 @@ id: index
|
|
|
3
3
|
title: "Streams"
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
`zio.blocks.streams` is a **
|
|
6
|
+
`zio.blocks.streams` is a **pull-based** streaming library for **Scala 3** (and Scala 2.13) with synchronous and asynchronous readers, typed errors, resource safety, and primitive specialization. Streams are lazy descriptions -- nothing executes until a terminal operation is driven. Cross-platform terminals ending in `Async` return `Async[Either[E, Z]]`; the JVM also provides blocking terminals returning `Either[E, Z]`. The library has zero runtime dependencies beyond ZIO Blocks modules, and avoids boxing on primitive element types through JVM-type-specialized internal readers.
|
|
7
7
|
|
|
8
8
|
ZIO Blocks Streams is built on three composable primitives:
|
|
9
9
|
|
|
10
10
|
| Type | Description | Key operation |
|
|
11
11
|
|----------------------------------------|----------------------------------------------------------------------|-----------------------|
|
|
12
|
-
| [`Stream[+E, +A]`](./stream.md) | A lazy, pull-based sequence of elements that may fail with error `E` | `stream.via(pipe)` |
|
|
13
|
-
| [`Pipeline[-In, +Out]`](./pipeline.md) | A reusable, composable stream-to-stream transformation | `pipe.andThen(other)` |
|
|
14
|
-
| [`Sink[+E, -A, +Z]`](./sink.md) | A stream consumer that produces a typed result `Z` | `stream.run(sink)` |
|
|
12
|
+
| [`Stream[+E, +A]`](./core/stream.md) | A lazy, pull-based sequence of elements that may fail with error `E` | `stream.via(pipe)` |
|
|
13
|
+
| [`Pipeline[-In, +Out]`](./core/pipeline.md) | A reusable, composable stream-to-stream transformation | `pipe.andThen(other)` |
|
|
14
|
+
| [`Sink[+E, -A, +Z]`](./core/sink.md) | A stream consumer that produces a typed result `Z` | `stream.run(sink)` |
|
|
15
15
|
|
|
16
16
|
## Overview
|
|
17
17
|
|
|
18
18
|
ZIO Blocks Streams is designed around three core principles:
|
|
19
19
|
|
|
20
|
-
**
|
|
20
|
+
**Dual execution.** Cross-platform `*Async` terminals drive either synchronous or asynchronous sources without blocking and require no ZIO runtime. On the JVM, plain terminals such as `run`, `runCollect`, and `head` are blocking compatibility twins.
|
|
21
21
|
|
|
22
22
|
**Pull-based evaluation.** Execution is driven from the consumer (Sink) backward through the pipeline to the source (Stream). This enables natural short-circuiting: if a sink only needs the first three elements, the stream stops producing after three elements — no work is wasted.
|
|
23
23
|
|
|
@@ -25,7 +25,9 @@ ZIO Blocks Streams is designed around three core principles:
|
|
|
25
25
|
|
|
26
26
|
## Quick Start
|
|
27
27
|
|
|
28
|
-
Here's a minimal example. Streams are lazy descriptions — nothing executes until you call a terminal operation like `runCollect`.
|
|
28
|
+
Here's a minimal JVM example. Streams are lazy descriptions — nothing executes until you call a terminal operation like `runCollect`. Use `runCollectAsync` for the cross-platform form.
|
|
29
|
+
|
|
30
|
+
Unless a section says otherwise, examples using plain terminals (`run`, `runCollect`, `head`, and their peers) are JVM-only shorthand. Replace them with the matching `*Async` terminal in shared JVM/Scala.js code.
|
|
29
31
|
|
|
30
32
|
```scala
|
|
31
33
|
import zio.blocks.streams.*
|
|
@@ -46,58 +48,46 @@ val result = stream.take(5).runCollect
|
|
|
46
48
|
Add the Streams module to your SBT build:
|
|
47
49
|
|
|
48
50
|
```scala
|
|
49
|
-
libraryDependencies += "dev.zio" %% "zio-blocks-streams" % "0.0.
|
|
51
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-streams" % "0.0.56"
|
|
50
52
|
```
|
|
51
53
|
|
|
52
54
|
For Scala.js (JavaScript/Node.js):
|
|
53
55
|
|
|
54
56
|
```scala
|
|
55
|
-
libraryDependencies += "dev.zio" %%% "zio-blocks-streams" % "0.0.
|
|
57
|
+
libraryDependencies += "dev.zio" %%% "zio-blocks-streams" % "0.0.56"
|
|
56
58
|
```
|
|
57
59
|
|
|
58
60
|
Supported Scala versions: 2.13.x and 3.x.
|
|
59
61
|
|
|
60
62
|
## Why Streams?
|
|
61
63
|
|
|
62
|
-
Streaming libraries in the Scala ecosystem typically require an effect system. fs2
|
|
64
|
+
Streaming libraries in the Scala ecosystem typically require an effect system. fs2 runs in a `cats.effect`-compatible `F[_]`, Kyo Streams needs the Kyo runtime, and Pekko (formerly Akka) Streams needs the actor runtime. When your code is synchronous and you want streaming without pulling in an effect monad, the options narrow considerably.
|
|
63
65
|
|
|
64
66
|
`zio.blocks.streams` fills that gap:
|
|
65
67
|
|
|
66
68
|
| Feature | ZB Streams | fs2 | Kyo | Ox | Pekko |
|
|
67
69
|
|---------------------------|-------------------------|----------------------|---------------|-------------------------|-----------------|
|
|
68
|
-
| Effect system required | No | Yes (cats-effect) | Yes (Kyo) | No (virtual threads) | Yes (
|
|
69
|
-
| Execution model |
|
|
70
|
-
| Typed errors | `Either[E, Z]` |
|
|
71
|
-
| Primitive specialization | Yes (zero boxing) |
|
|
72
|
-
| Stack-safe deep pipelines | Yes (trampolined) |
|
|
70
|
+
| Effect system required | No | Yes (cats-effect) | Yes (Kyo) | No (virtual threads) | Yes (Pekko) |
|
|
71
|
+
| Execution model | Sync/async, pull-based | Async, pull-based | Async, chunk | Synchronous, pull-based | Async, push |
|
|
72
|
+
| Typed errors | `Either[E, Z]` | Not verified here | Not verified here | Not verified here | Not verified here |
|
|
73
|
+
| Primitive specialization | Yes (zero boxing) | Not verified here | Not verified here | Not verified here | Not verified here |
|
|
74
|
+
| Stack-safe deep pipelines | Yes (trampolined) | Not verified here | Not verified here | Not verified here | Not verified here |
|
|
73
75
|
| Resource safety | Scope integration | Resource/bracket | Kyo resources | try/finally | Graph lifecycle |
|
|
74
|
-
| Dependencies | chunk
|
|
75
|
-
|
|
76
|
-
## Benchmarks
|
|
76
|
+
| Dependencies | scope, chunk, combinators, ringbuffer, async | fs2-core | kyo-prelude, kyo-core | ox core | pekko-stream |
|
|
77
77
|
|
|
78
|
-
|
|
78
|
+
The provider columns name the artifacts and versions this repository pins for benchmarking — fs2 3.14.0, Pekko 1.7.0, Kyo 1.0.0-RC6, Ox 1.0.6 (`build.sbt`, `streams-benchmark/benchmark-manifest.json`). Nothing outside the ZB Streams column is measured or verified in this repository.
|
|
79
79
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
| Benchmark | ZB Streams | Ox | Kyo | fs2 | Pekko |
|
|
83
|
-
|----------------------|------------|--------|--------|--------|-------|
|
|
84
|
-
| drain | 179,872 | 54,512 | 31,777 | 20,795 | 4,381 |
|
|
85
|
-
| map | 161,920 | 42,007 | 12,012 | 13,295 | 2,259 |
|
|
86
|
-
| filter | 168,541 | 47,933 | 19,962 | 14,977 | 2,901 |
|
|
87
|
-
| flatMap | 49,165 | 30,506 | 28,303 | 748 | 742 |
|
|
88
|
-
| take/drop | 322,470 | 28,708 | 64,640 | 28,836 | 2,379 |
|
|
89
|
-
| map+filter+flatMap | 980 | 508 | 602 | 19 | 16 |
|
|
90
|
-
| mixed depth 1 | 47,459 | 19,449 | 13,427 | 257 | 639 |
|
|
91
|
-
| mixed depth 2 | 33,859 | 15,336 | 7,328 | 208 | 459 |
|
|
92
|
-
| mixed depth 3 | 23,610 | 11,878 | 3,174 | 139 | 256 |
|
|
93
|
-
| nested flatMap (10K) | 8,161 | -- | -- | 937 | -- |
|
|
94
|
-
| nested concat (10K) | 6,140 | -- | 3 | 1,065 | 1 |
|
|
80
|
+
## Benchmarks
|
|
95
81
|
|
|
96
|
-
|
|
82
|
+
The repository carries a JMH benchmark suite under `streams-benchmark`. Provider comparisons are
|
|
83
|
+
governed by the allowlist and provider versions recorded in
|
|
84
|
+
`streams-benchmark/benchmark-manifest.json`; results are only comparable within a single benchmark
|
|
85
|
+
class and contract, and are not aggregated into a ranking here. Re-run the benchmarks on your target
|
|
86
|
+
environment before drawing any performance conclusion.
|
|
97
87
|
|
|
98
|
-
|
|
88
|
+
If you are evaluating Scala 2 compatibility work, read the [Scala 2 compatibility design note](./execution-and-compatibility/scala-2-compatibility.md) before moving any `Stream` or `Sink` hot-path combinators behind version-specific seams.
|
|
99
89
|
|
|
100
|
-
## Core
|
|
90
|
+
## Core Mental Model
|
|
101
91
|
|
|
102
92
|
To understand ZIO Blocks Streams fully, it's helpful to see how the three primitives fit together and how data flows through a pipeline from source to sink. This section walks through the architecture and explains each component in depth.
|
|
103
93
|
|
|
@@ -128,17 +118,24 @@ Operations on streams transform the pipeline and ultimately run it against a sin
|
|
|
128
118
|
.run(sink)
|
|
129
119
|
│
|
|
130
120
|
┌──────────────────▼───────────────┐
|
|
131
|
-
│ Either[E, Z]
|
|
132
|
-
│ (
|
|
121
|
+
│ Async[Either[E, Z]] │
|
|
122
|
+
│ (or blocking Either on JVM) │
|
|
133
123
|
└──────────────────────────────────┘
|
|
134
124
|
```
|
|
135
125
|
|
|
126
|
+
The last box is where the flow forks. The same `Stream` description is materialized either as a
|
|
127
|
+
synchronous reader, drained on the calling thread by a blocking terminal such as `run` or
|
|
128
|
+
`runCollect`, or as an asynchronous reader, driven without blocking by the matching `*Async`
|
|
129
|
+
terminal. Which engine runs is decided by the source and operators the pipeline is built from, not
|
|
130
|
+
by the terminal you call. See [Async Execution](./execution-and-compatibility/async-execution.md) for how that classification
|
|
131
|
+
works.
|
|
136
132
|
|
|
137
|
-
|
|
133
|
+
|
|
134
|
+
### 1) `Stream[E, A]` -- a Lazy Sequence
|
|
138
135
|
|
|
139
136
|
A `Stream[+E, +A]` is a **description** of a potentially infinite sequence of elements of type `A` that may fail with an error of type `E`. It is covariant in both type parameters.
|
|
140
137
|
|
|
141
|
-
Nothing happens when you construct a stream or chain transformations. Execution only begins when you
|
|
138
|
+
Nothing happens when you construct a stream or chain transformations. Execution only begins when you drive a terminal operation. Cross-platform terminals (`runAsync`, `runCollectAsync`, `headAsync`, `countAsync`, etc.) return `Async[Either[E, Z]]`; plain blocking terminals return `Either[E, Z]` on the JVM:
|
|
142
139
|
|
|
143
140
|
- `Left(e)` -- a typed stream error
|
|
144
141
|
- `Right(z)` -- the successful result
|
|
@@ -170,7 +167,7 @@ This makes debugging and logging straightforward -- you can see exactly what tra
|
|
|
170
167
|
|
|
171
168
|
---
|
|
172
169
|
|
|
173
|
-
### 2) `Sink[E, A, Z]` -- a
|
|
170
|
+
### 2) `Sink[E, A, Z]` -- a Consumer
|
|
174
171
|
|
|
175
172
|
A `Sink[+E, -A, +Z]` consumes elements of type `A` from a stream and produces a final result of type `Z`. Sinks are passed to `Stream.run`:
|
|
176
173
|
|
|
@@ -199,7 +196,7 @@ Sinks compose with `contramap` (pre-process input) and `map` (post-process resul
|
|
|
199
196
|
|
|
200
197
|
```scala
|
|
201
198
|
val lengthSink: Sink[Nothing, String, Long] =
|
|
202
|
-
Sink.sumInt.contramap[String](_.length)
|
|
199
|
+
Sink.sumInt.contramap[Int, String](_.length)
|
|
203
200
|
|
|
204
201
|
val doubled: Sink[Nothing, Int, Long] =
|
|
205
202
|
Sink.sumInt.map(_ * 2)
|
|
@@ -207,7 +204,7 @@ val doubled: Sink[Nothing, Int, Long] =
|
|
|
207
204
|
|
|
208
205
|
---
|
|
209
206
|
|
|
210
|
-
### 3) `Pipeline[In, Out]` --
|
|
207
|
+
### 3) `Pipeline[In, Out]` -- Reusable Transformation
|
|
211
208
|
|
|
212
209
|
A `Pipeline[-In, +Out]` is a reusable stream transformation. It decouples the transformation logic from any specific stream, so you can define it once and apply it many times.
|
|
213
210
|
|
|
@@ -243,6 +240,31 @@ val countLong: Sink[Nothing, String, Long] =
|
|
|
243
240
|
.andThenSink(Sink.sumInt)
|
|
244
241
|
```
|
|
245
242
|
|
|
243
|
+
## Synchronous and Asynchronous Execution
|
|
244
|
+
|
|
245
|
+
There is one `Stream` type. It serves both execution modes, and there is no mode type parameter, no
|
|
246
|
+
`AsyncStream`, and no annotation to write.
|
|
247
|
+
|
|
248
|
+
The type that decides is [`Reader`](./primitives/reader.md), not `Stream`. Materializing a stream yields either
|
|
249
|
+
a `Reader.SyncReader[A]`, whose pulls return values directly, or a `Reader.AsyncReader[A]`, whose
|
|
250
|
+
pulls return `Async` values. A pipeline that is synchronous end to end materializes as the former; a
|
|
251
|
+
single asynchronous source or operator anywhere in it makes the whole pipeline asynchronous.
|
|
252
|
+
|
|
253
|
+
The `*Async` terminals -- `runAsync`, `runCollectAsync`, `runDrainAsync`, `runFoldAsync`,
|
|
254
|
+
`countAsync`, `headAsync`, and their peers -- are the cross-platform API. They all return
|
|
255
|
+
`Async[Either[E, Z]]`: the typed error `E` stays inside the `Either`, while the outer `Async` fails
|
|
256
|
+
only on a defect. They drive a synchronous pipeline just as correctly as an asynchronous one, so
|
|
257
|
+
shared JVM/Scala.js code can use them unconditionally.
|
|
258
|
+
|
|
259
|
+
The blocking terminals -- `run`, `runCollect`, `runDrain`, `runFold`, `count`, `head`, and their
|
|
260
|
+
peers -- are **JVM-only** compatibility twins returning a bare `Either[E, Z]`. They do not exist on
|
|
261
|
+
Scala.js, and cross-compiled sources cannot call them.
|
|
262
|
+
|
|
263
|
+
- [Async Execution](./execution-and-compatibility/async-execution.md) -- the full execution model: classification, the `Reader`
|
|
264
|
+
union, the async source constructors, operators, and terminals.
|
|
265
|
+
- [Platform Differences](./execution-and-compatibility/platform-differences.md) -- the availability matrix of every member that
|
|
266
|
+
differs between the JVM and Scala.js.
|
|
267
|
+
|
|
246
268
|
## Error Handling
|
|
247
269
|
|
|
248
270
|
Streams distinguish between two kinds of failures:
|
|
@@ -260,7 +282,7 @@ recovered.runCollect // Right(Chunk(1, 2, 3, 99))
|
|
|
260
282
|
|
|
261
283
|
## Resource Management
|
|
262
284
|
|
|
263
|
-
Streams integrate with `zio.blocks.scope.Scope` for deterministic resource cleanup. The `fromAcquireRelease` constructor guarantees that a resource is acquired lazily (when the stream runs), used to produce elements, and then released — even if the stream is short-circuited early via `take()`, fails with an error, or succeeds normally. The release function is wired into a finally block, ensuring cleanup always happens.
|
|
285
|
+
Streams integrate with [`zio.blocks.scope.Scope`](../resource-management/scope.md) for deterministic resource cleanup. The `fromAcquireRelease` constructor guarantees that a resource is acquired lazily (when the stream runs), used to produce elements, and then released — even if the stream is short-circuited early via `take()`, fails with an error, or succeeds normally. The release function is wired into a finally block, ensuring cleanup always happens.
|
|
264
286
|
|
|
265
287
|
```scala
|
|
266
288
|
import zio.blocks.streams.*
|
|
@@ -278,7 +300,9 @@ This eliminates the need for manual try/finally when working with resources —
|
|
|
278
300
|
|
|
279
301
|
## Primitive Specialization
|
|
280
302
|
|
|
281
|
-
ZB Streams
|
|
303
|
+
ZB Streams carries the JVM representation of the element type through the whole pipeline, so a stream of primitives is not boxed at each stage boundary. Specialization is not limited to `Int`, `Long`, `Float`, and `Double`: there are **nine logical lanes** -- the eight primitive pull identities `Boolean`, `Byte`, `Short`, `Char`, `Int`, `Long`, `Float`, and `Double`, plus the reference fallback -- and the synchronous interpreter compacts them into **five storage lanes**: int-like (`Boolean`/`Byte`/`Short`/`Char`/`Int`), `Long`, `Float`, `Double`, and reference. Nine logical lanes therefore does not mean nine interpreter arrays; the operator tag selects the identity-specific reads over the shared storage.
|
|
304
|
+
|
|
305
|
+
[Zero-Boxing Streams](./execution-and-compatibility/zero-boxing.md) explains how a lane is chosen and what the specialization evidence is for.
|
|
282
306
|
|
|
283
307
|
```scala
|
|
284
308
|
import zio.blocks.streams.*
|
|
@@ -302,18 +326,18 @@ This matters most for numeric workloads — data processing, statistics, encodin
|
|
|
302
326
|
- **Use the auto-closing I/O constructors** (`fromInputStream`, `fromJavaReader`, `NioStreams.fromChannel`) by default. Only use the `Unmanaged` variants when you need to borrow a resource whose lifetime is managed elsewhere.
|
|
303
327
|
- **Use `Pipeline`** when you have a transformation you want to reuse across multiple streams or apply to sinks.
|
|
304
328
|
- **Use `&&` for zipping** instead of manual zip calls. Tuples flatten automatically: `a && b && c` produces `(A, B, C)` not `((A, B), C)`.
|
|
305
|
-
- **Leverage primitive specialization** for numeric workloads. Streams of
|
|
329
|
+
- **Leverage primitive specialization** for numeric workloads. Streams of any primitive element type avoid boxing automatically on the synchronous path; use `Sink.sumInt`, `runFold(0)(_ + _)`, etc. for allocation-free folds.
|
|
306
330
|
- **Use `scan` for running accumulators**, `grouped` for batching, and `sliding` for windowed computations.
|
|
307
331
|
- **Use `render`/`toString`** to inspect pipeline structure during debugging — it shows each transformation stage without executing the stream.
|
|
308
|
-
- **Use `Sink.create`** as an escape hatch when none of the built-in sinks fit.
|
|
332
|
+
- **Use `Sink.create`** (JVM only) as an escape hatch when none of the built-in sinks fit; on Scala.js and in cross-compiled code use `Sink.createAsync` or `Sink.createBoth`.
|
|
309
333
|
- **`suspend`** is your friend for recursive or self-referential stream definitions, preventing stack overflow during construction.
|
|
310
334
|
- **Typed errors vs. defects**: use `Stream.fail` for expected domain errors and `Stream.die` for programmer errors. Use `catchAll` for the former, `catchDefect` for the latter.
|
|
311
335
|
|
|
312
|
-
## Usage
|
|
336
|
+
## Usage Examples
|
|
313
337
|
|
|
314
338
|
This section shows practical examples of using streams in real-world scenarios. Each subsection demonstrates a different aspect of the API with runnable code examples.
|
|
315
339
|
|
|
316
|
-
### Creating
|
|
340
|
+
### Creating Streams
|
|
317
341
|
|
|
318
342
|
Here are the most common ways to construct a stream. Choose the constructor that best fits your data source:
|
|
319
343
|
|
|
@@ -347,8 +371,6 @@ Stream.fail("error") // Stream[String, Nothing]
|
|
|
347
371
|
|
|
348
372
|
// Generators
|
|
349
373
|
Stream.repeat(1) // infinite stream of 1s
|
|
350
|
-
// Stream.iterate(1)(_ * 2) // 1, 2, 4, 8, 16, ...
|
|
351
|
-
// Stream.repeatThunk(scala.util.Random.nextInt(100)) // infinite random ints
|
|
352
374
|
Stream.unfold(0)(n => // 0, 1, 2, ..., 9
|
|
353
375
|
if n < 10 then Some((n, n + 1)) else None
|
|
354
376
|
)
|
|
@@ -362,23 +384,23 @@ Stream.attemptEval(riskyEffect()) // same, for Unit-returning effe
|
|
|
362
384
|
Stream.suspend(expensiveStreamBuilder())
|
|
363
385
|
|
|
364
386
|
// I/O sources (auto-closing) - JVM only
|
|
365
|
-
Stream.fromInputStream(inputStream) // Stream[IOException,
|
|
387
|
+
Stream.fromInputStream(inputStream) // Stream[IOException, Byte] (auto-closes)
|
|
366
388
|
Stream.fromJavaReader(javaReader) // Stream[IOException, Char] (auto-closes)
|
|
367
389
|
|
|
368
390
|
// I/O sources (borrowing -- caller manages lifetime) - JVM only
|
|
369
|
-
Stream.fromInputStreamUnmanaged(inputStream) // Stream[IOException,
|
|
391
|
+
Stream.fromInputStreamUnmanaged(inputStream) // Stream[IOException, Byte] (does NOT close)
|
|
370
392
|
Stream.fromJavaReaderUnmanaged(javaReader) // Stream[IOException, Char] (does NOT close)
|
|
371
393
|
```
|
|
372
394
|
|
|
373
395
|
---
|
|
374
396
|
|
|
375
|
-
### Transforming
|
|
397
|
+
### Transforming Streams
|
|
376
398
|
|
|
377
|
-
Streams support many transformation operations. Use `map` for element-wise changes, `filter` for selection, and `flatMap` for expanding elements into sub-streams. See the [Stream reference](./stream.md) page for comprehensive examples of all transformation methods including `map`, `filter`, `flatMap`, `collect`, `scan`, `mapAccum`, `distinct`, `intersperse`, and more.
|
|
399
|
+
Streams support many transformation operations. Use `map` for element-wise changes, `filter` for selection, and `flatMap` for expanding elements into sub-streams. See the [Stream reference](./core/stream.md) page for comprehensive examples of all transformation methods including `map`, `filter`, `flatMap`, `collect`, `scan`, `mapAccum`, `distinct`, `intersperse`, and more.
|
|
378
400
|
|
|
379
401
|
---
|
|
380
402
|
|
|
381
|
-
### Zipping
|
|
403
|
+
### Zipping Streams with `&&`
|
|
382
404
|
|
|
383
405
|
The `&&` operator zips two streams element-by-element into tuples. The resulting stream ends when either input is exhausted.
|
|
384
406
|
|
|
@@ -398,7 +420,7 @@ val triples = names && ages && ids
|
|
|
398
420
|
triples.runCollect // Right(Chunk(("Alice", 30, 1L), ("Bob", 25, 2L), ("Charlie", 35, 3L)))
|
|
399
421
|
```
|
|
400
422
|
|
|
401
|
-
When the error types differ, they widen
|
|
423
|
+
When the error types differ, they widen through `Concat` — to a union `E1 | E2` on Scala 3, and to a meaningful least upper bound (or `Either[E1, E2]` for disjoint types) on Scala 2.13:
|
|
402
424
|
|
|
403
425
|
```scala
|
|
404
426
|
import zio.blocks.streams.*
|
|
@@ -412,9 +434,9 @@ val s2: Stream[OtherError, Int] = Stream.fromIterable(List(4, 5, 6))
|
|
|
412
434
|
|
|
413
435
|
---
|
|
414
436
|
|
|
415
|
-
### Primitive
|
|
437
|
+
### Primitive Specialization
|
|
416
438
|
|
|
417
|
-
|
|
439
|
+
All nine logical lanes are specialized, not only `Int`, `Long`, `Float`, and `Double`. Every intermediate step uses the identity-specific read (`readInt`, `readByte`, `readChar`, and so on), so no `java.lang.Integer` wrappers are allocated between stages.
|
|
418
440
|
|
|
419
441
|
```scala
|
|
420
442
|
// This entire pipeline runs with ZERO boxing of the Int elements.
|
|
@@ -426,11 +448,11 @@ val sum: Either[Nothing, Long] =
|
|
|
426
448
|
.runFold(0L)(_ + _) // Long-specialized accumulator
|
|
427
449
|
```
|
|
428
450
|
|
|
429
|
-
This matters most for numeric workloads -- data processing, statistics, encoding/decoding -- where millions of elements flow through multi-stage pipelines.
|
|
451
|
+
This matters most for numeric workloads -- data processing, statistics, encoding/decoding -- where millions of elements flow through multi-stage pipelines.
|
|
430
452
|
|
|
431
453
|
---
|
|
432
454
|
|
|
433
|
-
### Consuming
|
|
455
|
+
### Consuming Streams
|
|
434
456
|
|
|
435
457
|
Terminal operations run the stream and produce a final result. Use `runCollect` to gather all elements, `runDrain` to discard them, or specialized operations like `head`, `count`, and `foldLeft`:
|
|
436
458
|
|
|
@@ -467,7 +489,50 @@ s.run(Sink.take(3)) // Right(Chunk(1, 2, 3))
|
|
|
467
489
|
|
|
468
490
|
---
|
|
469
491
|
|
|
470
|
-
###
|
|
492
|
+
### Async Execution
|
|
493
|
+
|
|
494
|
+
`runCollectAsync` is the cross-platform twin of `runCollect`. It returns `Async[Either[E, Chunk[A]]]`
|
|
495
|
+
rather than `Either[E, Chunk[A]]`, so it never blocks and compiles on both the JVM and Scala.js:
|
|
496
|
+
|
|
497
|
+
```scala
|
|
498
|
+
import zio.blocks.streams._
|
|
499
|
+
import zio.blocks.chunk.Chunk
|
|
500
|
+
import zio.blocks.async._
|
|
501
|
+
|
|
502
|
+
val evens: Stream[Nothing, Int] =
|
|
503
|
+
Stream.range(1, 100).filter(_ % 2 == 0).map(_ * 3)
|
|
504
|
+
|
|
505
|
+
// Still a description -- nothing has run.
|
|
506
|
+
val pending: Async[Either[Nothing, Chunk[Int]]] = evens.take(5).runCollectAsync
|
|
507
|
+
|
|
508
|
+
// Stay in Async: map the result rather than extracting it.
|
|
509
|
+
val described: Async[String] = pending.map {
|
|
510
|
+
case Right(values) => s"collected ${values.length} elements"
|
|
511
|
+
case Left(error) => s"failed: $error"
|
|
512
|
+
}
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
Something has to drive the `Async` at the edge of the world. On the JVM that is `.block`, which
|
|
516
|
+
belongs in `main` or a test and nowhere else:
|
|
517
|
+
|
|
518
|
+
```scala
|
|
519
|
+
import zio.blocks.streams._
|
|
520
|
+
import zio.blocks.chunk.Chunk
|
|
521
|
+
import zio.blocks.async._
|
|
522
|
+
|
|
523
|
+
// JVM only: Async#block throws on Scala.js.
|
|
524
|
+
val result: Either[Nothing, Chunk[Int]] =
|
|
525
|
+
Stream.range(1, 100).filter(_ % 2 == 0).take(5).runCollectAsync.block
|
|
526
|
+
// Right(Chunk(2, 4, 6, 8, 10))
|
|
527
|
+
```
|
|
528
|
+
|
|
529
|
+
Scala.js code keeps the `Async` and hands it to the host runtime instead. See
|
|
530
|
+
[Async Execution](./execution-and-compatibility/async-execution.md) for the full terminal family and
|
|
531
|
+
[Platform Differences](./execution-and-compatibility/platform-differences.md) for what is available where.
|
|
532
|
+
|
|
533
|
+
---
|
|
534
|
+
|
|
535
|
+
### Error Handling Patterns
|
|
471
536
|
|
|
472
537
|
Streams support two types of failures: typed errors that you can handle explicitly, and defects (exceptions) that propagate. Here are common patterns for dealing with both:
|
|
473
538
|
|
|
@@ -507,7 +572,7 @@ val handled = risky.catchDefect {
|
|
|
507
572
|
|
|
508
573
|
---
|
|
509
574
|
|
|
510
|
-
### Resource
|
|
575
|
+
### Resource Safety Patterns
|
|
511
576
|
|
|
512
577
|
When working with files, network connections, or other resources, use the resource-safe constructors to guarantee cleanup. Here are the most common patterns:
|
|
513
578
|
|
|
@@ -547,11 +612,11 @@ val withDefer =
|
|
|
547
612
|
|
|
548
613
|
---
|
|
549
614
|
|
|
550
|
-
### NIO
|
|
615
|
+
### NIO Integration (JVM Only)
|
|
551
616
|
|
|
552
617
|
On the JVM, `NioStreams` and `NioSinks` provide zero-copy integration with `java.nio` buffers and channels.
|
|
553
618
|
|
|
554
|
-
#### `NioStreams` --
|
|
619
|
+
#### `NioStreams` -- Creating Streams From NIO Sources
|
|
555
620
|
|
|
556
621
|
```scala
|
|
557
622
|
import zio.blocks.streams.*
|
|
@@ -582,7 +647,7 @@ val bytes2 = NioStreams.fromChannelUnmanaged(ch2, bufSize = 4096).runCollect
|
|
|
582
647
|
ch2.close() // caller is responsible for closing
|
|
583
648
|
```
|
|
584
649
|
|
|
585
|
-
#### `NioSinks` --
|
|
650
|
+
#### `NioSinks` -- Writing to NIO Targets
|
|
586
651
|
|
|
587
652
|
```scala
|
|
588
653
|
import zio.blocks.streams.*
|
|
@@ -612,7 +677,7 @@ finally {
|
|
|
612
677
|
|
|
613
678
|
---
|
|
614
679
|
|
|
615
|
-
### Pipeline
|
|
680
|
+
### Pipeline Composition
|
|
616
681
|
|
|
617
682
|
Pipelines are composable transformations that can be reused across different streams. Build complex transformations by chaining pipelines together with `andThen`:
|
|
618
683
|
|
|
@@ -651,3 +716,11 @@ Stream.fromIterable(List("10", "abc", "-3", "7", "0", "25"))
|
|
|
651
716
|
.run(sumPositiveDoubled)
|
|
652
717
|
// Right(84L)
|
|
653
718
|
```
|
|
719
|
+
|
|
720
|
+
## See Also
|
|
721
|
+
|
|
722
|
+
- [Async Execution](./execution-and-compatibility/async-execution.md) -- the synchronous/asynchronous execution model, the `Reader` union, and the `*Async` terminal family
|
|
723
|
+
- [Platform Differences](./execution-and-compatibility/platform-differences.md) -- which members exist on the JVM, on Scala.js, and on both
|
|
724
|
+
- [Zero-Boxing Streams](./execution-and-compatibility/zero-boxing.md) -- the primitive lanes and how one is chosen
|
|
725
|
+
- [Async](../async.md) -- the `Async` effect type the cross-platform terminals return
|
|
726
|
+
- [Mux](../mux.mdx) -- coordinating many keyed streams over one shared transport
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: index
|
|
3
|
+
title: "Low-Level Primitives"
|
|
4
|
+
description: "Low-Level Primitives index: Reader and Writer, the live cursors a Stream and Sink compile into at execution time."
|
|
5
|
+
keywords:
|
|
6
|
+
- "Pull-Based Streaming"
|
|
7
|
+
- "Push-Based Writing"
|
|
8
|
+
- "Primitives Overview"
|
|
9
|
+
- "Reader"
|
|
10
|
+
- "Writer"
|
|
11
|
+
sidebar_label: "Low-Level Primitives"
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
`Reader` and `Writer` are the live, stateful objects a pipeline compiles into and runs on. Where the [Core Types](../core/index.md) are inert descriptions, these hold real resources — an open file, a socket, a position in a buffer — until closed.
|
|
15
|
+
|
|
16
|
+
A terminal such as `stream.runAsync(sink)` compiles to and drains a `Reader` for you. Work with these types directly only when writing a custom source or sink, wrapping a native I/O API, or driving a compiled stream by hand.
|
|
17
|
+
|
|
18
|
+
## Reader
|
|
19
|
+
|
|
20
|
+
[`Reader[+Elem]`](./reader.md) is the pull side: the cursor a `Stream` compiles into. It has two kinds — `Reader.SyncReader[Elem]`, whose pulls return directly, and `Reader.AsyncReader[Elem]`, whose pulls return `Async`. An asynchronous reader is single-consumer, with exactly one owner responsible for awaiting its `close()`.
|
|
21
|
+
|
|
22
|
+
## Writer
|
|
23
|
+
|
|
24
|
+
[`Writer[-Elem]`](./writer.md) is the push side: a destination you feed elements into one at a time until it closes or fills. Reach for it when adapting an `OutputStream`-shaped API, or implementing a sink that a producer outside the pipeline writes into.
|
|
25
|
+
|
|
26
|
+
## See Also
|
|
27
|
+
|
|
28
|
+
- [Core Types](../core/index.md) — the descriptions that compile into these primitives
|
|
29
|
+
- [Streams Reference](../index.md) — module overview
|
|
30
|
+
- [Asynchronous Stream Execution](../execution-and-compatibility/async-execution.md) — how a graph decides between a `SyncReader` and an `AsyncReader`
|