@zio.dev/zio-blocks 0.0.33 → 0.0.51
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/guides/compile-time-resource-safety-with-scope.md +16 -17
- package/guides/getting-started-with-mux.md +1507 -0
- package/guides/query-dsl-extending.md +161 -102
- package/guides/query-dsl-fluent-builder.md +217 -157
- package/guides/query-dsl-reified-optics.md +12 -10
- package/guides/query-dsl-sql.md +246 -165
- package/guides/telemetry-guide.md +1069 -0
- package/guides/zio-schema-migration.md +29 -22
- package/index.md +292 -50
- package/package.json +1 -1
- package/plans/config-follow-up-prs.md +188 -0
- package/plans/config-pr-assessment-roadmap.md +310 -0
- package/reference/MuxDataFlow.jsx +250 -0
- package/reference/async.md +651 -0
- package/reference/chunk.md +3533 -308
- package/reference/codegen/case-class.md +436 -0
- package/reference/codegen/emitter-config.md +383 -0
- package/reference/codegen/examples.md +664 -0
- package/reference/codegen/field.md +316 -0
- package/reference/codegen/index.md +317 -0
- package/reference/codegen/scala-emitter.md +392 -0
- package/reference/codegen/scala-file.md +276 -0
- package/reference/codegen/sealed-trait.md +408 -0
- package/reference/codegen/type-definition.md +340 -0
- package/reference/codegen/type-ref.md +201 -0
- package/reference/combinators.md +347 -117
- package/reference/config.md +158 -0
- package/reference/context.md +4 -4
- package/reference/datastar.md +346 -0
- package/reference/docs.md +1461 -345
- package/reference/endpoint/auth-type.md +146 -0
- package/reference/endpoint/endpoint.md +297 -0
- package/reference/endpoint/http-codec.md +249 -0
- package/reference/endpoint/index.md +825 -0
- package/reference/endpoint/path-codec.md +237 -0
- package/reference/endpoint/route-pattern.md +196 -0
- package/reference/endpoint/route-tree.md +111 -0
- package/reference/endpoint/segment-codec.md +212 -0
- package/reference/html.md +1120 -0
- package/reference/htmx/attribute-values.md +359 -0
- package/reference/htmx/hx-encoding.md +111 -0
- package/reference/htmx/hx-params.md +204 -0
- package/reference/htmx/hx-swap.md +276 -0
- package/reference/htmx/hx-sync.md +251 -0
- package/reference/htmx/hx-target.md +314 -0
- package/reference/htmx/hx-trigger.md +457 -0
- package/reference/htmx/hx-url-update.md +239 -0
- package/reference/htmx/index.md +855 -0
- package/reference/http-model/index.md +47 -0
- package/reference/http-model/model.md +1481 -0
- package/reference/http-model/schema.md +747 -0
- package/reference/maybe.md +826 -0
- package/reference/media-type.md +2 -2
- package/reference/mux.mdx +823 -0
- package/reference/openapi.md +1351 -0
- package/reference/resource-management/defer-handle.md +1 -1
- package/reference/resource-management/resource.md +31 -2
- package/reference/resource-management/scope.md +28 -12
- package/reference/resource-management/wire.md +3 -7
- package/reference/ringbuffer/MpmcDiagram.jsx +717 -0
- package/reference/ringbuffer/MpscDiagram.jsx +618 -0
- package/reference/ringbuffer/SpmcDiagram.jsx +680 -0
- package/reference/ringbuffer/SpscDiagram.jsx +677 -0
- package/reference/ringbuffer/advanced.mdx +109 -0
- package/reference/ringbuffer/index.mdx +145 -0
- package/reference/ringbuffer/mpmc.mdx +151 -0
- package/reference/ringbuffer/mpsc.mdx +132 -0
- package/reference/ringbuffer/spmc.mdx +108 -0
- package/reference/ringbuffer/spsc.mdx +344 -0
- package/reference/{allows.md → schema/allows.md} +4 -4
- package/reference/{binding-resolver.md → schema/binding-resolver.md} +1 -1
- package/reference/{binding.md → schema/binding.md} +2 -3
- package/reference/schema/built-in-codecs/avro.md +451 -0
- package/reference/schema/built-in-codecs/bson.md +480 -0
- package/reference/schema/built-in-codecs/csv.md +564 -0
- package/reference/schema/built-in-codecs/index.md +77 -0
- package/reference/schema/built-in-codecs/json/index.md +295 -0
- package/reference/schema/built-in-codecs/json/json-config.md +217 -0
- package/reference/{json-patch.md → schema/built-in-codecs/json/json-patch.md} +5 -5
- package/reference/{json-schema.md → schema/built-in-codecs/json/json-schema.md} +14 -47
- package/reference/schema/built-in-codecs/json/json-selection.md +322 -0
- package/reference/{json.md → schema/built-in-codecs/json/json.md} +32 -64
- package/reference/schema/built-in-codecs/messagepack.md +508 -0
- package/reference/schema/built-in-codecs/thrift.md +433 -0
- package/reference/schema/built-in-codecs/toon.md +1078 -0
- package/reference/{xml.md → schema/built-in-codecs/xml.md} +13 -9
- package/reference/schema/built-in-codecs/yaml.md +552 -0
- package/reference/{codec.md → schema/codec.md} +10 -10
- package/reference/{dynamic-optic.md → schema/dynamic-optic.md} +151 -5
- package/reference/{dynamic-schema.md → schema/dynamic-schema.md} +8 -8
- package/reference/schema/format.md +92 -0
- package/reference/schema/index.md +50 -0
- package/reference/schema/migration.md +297 -0
- package/reference/{modifier.md → schema/modifier.md} +58 -7
- package/reference/{optics.md → schema/optics.md} +2 -2
- package/reference/{patch.md → schema/patch.md} +1 -1
- package/{path-interpolator.md → reference/schema/path-interpolator.md} +165 -72
- package/reference/{schema-evolution → schema/schema-evolution}/as.md +8 -8
- package/reference/{schema-evolution → schema/schema-evolution}/index.md +2 -2
- package/reference/{schema-evolution → schema/schema-evolution}/into.md +8 -8
- package/reference/{schema-expr.md → schema/schema-expr.md} +110 -175
- package/reference/{schema.md → schema/schema.md} +12 -0
- package/reference/{structural-types.md → schema/structural-types.md} +1 -1
- package/reference/{type-class-derivation.md → schema/type-class-derivation.md} +63 -1
- package/reference/smithy.md +533 -0
- package/reference/sql/db-codec-deriver.md +71 -0
- package/reference/sql/db-codec.md +687 -0
- package/reference/sql/db-con.md +271 -0
- package/reference/sql/db-connection.md +153 -0
- package/reference/sql/db-param-writer.md +77 -0
- package/reference/sql/db-param.md +66 -0
- package/reference/sql/db-result-reader.md +146 -0
- package/reference/sql/db-tx.md +82 -0
- package/reference/sql/db-value.md +41 -0
- package/reference/sql/ddl.md +85 -0
- package/reference/sql/frag.md +254 -0
- package/reference/sql/index.md +341 -0
- package/reference/sql/repo.md +600 -0
- package/reference/sql/sql-dialect.md +73 -0
- package/reference/sql/sql-logger.md +62 -0
- package/reference/sql/sql-name-mapper.md +70 -0
- package/reference/sql/table-metadata.md +134 -0
- package/reference/sql/table.md +448 -0
- package/reference/sql/transactor-zio.md +399 -0
- package/reference/sql/transactor.md +353 -0
- package/reference/sql-zio.md +112 -0
- package/reference/streams/concurrent-operators.md +106 -0
- package/reference/streams/index.md +653 -0
- package/reference/streams/pipeline.md +718 -0
- package/reference/streams/reader.md +1284 -0
- package/reference/streams/scala-2-compatibility.md +55 -0
- package/reference/streams/sink.md +1426 -0
- package/reference/streams/stream.md +2526 -0
- package/reference/streams/writer.md +1045 -0
- package/reference/streams/zero-boxing.md +275 -0
- package/reference/telemetry.md +693 -0
- package/reference/typeid.md +5 -19
- package/sidebars.js +238 -43
- package/reference/formats.md +0 -694
- package/reference/http-model.md +0 -1716
- package/reference/streams.md +0 -989
- package/ringbuffer.md +0 -249
- /package/reference/{json-differ.md → schema/built-in-codecs/json/json-differ.md} +0 -0
- /package/reference/{dynamic-value.md → schema/dynamic-value.md} +0 -0
- /package/reference/{lazy.md → schema/lazy.md} +0 -0
- /package/reference/{reflect.md → schema/reflect.md} +0 -0
- /package/reference/{registers.md → schema/registers.md} +0 -0
- /package/reference/{schema-error.md → schema/schema-error.md} +0 -0
- /package/reference/{syntax.md → schema/syntax.md} +0 -0
- /package/reference/{validation.md → schema/validation.md} +0 -0
package/reference/streams.md
DELETED
|
@@ -1,989 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
id: streams
|
|
3
|
-
title: "Streams"
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
`zio.blocks.streams` is a **synchronous, pull-based** streaming library for **Scala 3** (and Scala 2.13) with typed errors, resource safety, and primitive specialization. Streams are lazy descriptions -- nothing executes until a terminal operation is called. All results are returned as `Either[E, Z]`, keeping error handling explicit and typed. The library has zero runtime dependencies beyond `zio.blocks.chunk` and `zio.blocks.scope`, and achieves zero-boxing on primitive element types (`Int`, `Long`, `Float`, `Double`) through JVM-type-specialized internal readers.
|
|
7
|
-
|
|
8
|
-
## Why Streams?
|
|
9
|
-
|
|
10
|
-
Streaming libraries in the Scala ecosystem typically require an effect system. fs2 needs `cats.effect.IO`, Kyo Streams needs the Kyo runtime, and Pekko (formerly Akka) Streams needs the actor runtime. When your code is synchronous and you want streaming without pulling in an effect monad, the options narrow considerably.
|
|
11
|
-
|
|
12
|
-
`zio.blocks.streams` fills that gap:
|
|
13
|
-
|
|
14
|
-
| Feature | ZB Streams | fs2 | Kyo | Ox | Pekko |
|
|
15
|
-
|---|---|---|---|---|---|
|
|
16
|
-
| Effect system required | No | Yes (cats-effect) | Yes (Kyo) | No (virtual threads) | Yes (Akka) |
|
|
17
|
-
| Execution model | Synchronous, pull-based | Async, pull-based | Async, chunk-based | Synchronous, pull-based | Async, push-based |
|
|
18
|
-
| Typed errors | `Either[E, Z]` | ApplicativeError | Kyo effects | Exceptions | No |
|
|
19
|
-
| Primitive specialization | Yes (zero boxing) | No | No | No | No |
|
|
20
|
-
| Internal chunking | No (element-at-a-time) | Yes (Chunk) | Yes (Chunk) | No | No |
|
|
21
|
-
| Stack-safe deep pipelines | Yes (trampolined) | Yes (Pull) | Yes | No (SO on deep flatMap) | N/A |
|
|
22
|
-
| Resource safety | Scope integration | Resource/bracket | Kyo resources | try/finally | Graph lifecycle |
|
|
23
|
-
| Dependencies | chunk + scope | cats-effect + scodec | Kyo core | Ox core | Akka actor |
|
|
24
|
-
|
|
25
|
-
Key properties:
|
|
26
|
-
|
|
27
|
-
- **No effect system** -- streams run on the calling thread; `run` returns `Either[E, Z]` directly
|
|
28
|
-
- **Pull-based** -- the consumer drives evaluation; elements are produced on demand. Contrast with push-based systems (like Pekko) where the producer drives and the consumer must keep up.
|
|
29
|
-
- **Primitive specialization** -- `Int`, `Long`, `Float`, and `Double` streams avoid boxing through specialized `readInt`, `readLong`, `readFloat`, `readDouble` methods on `Reader`
|
|
30
|
-
- **Resource safe** -- integrates with `zio.blocks.scope.Scope` for deterministic finalization; also provides `fromAcquireRelease`, `ensuring`, and `defer` for standalone resource management
|
|
31
|
-
- **Lazy** -- a `Stream[E, A]` is a description; construction is free and nothing executes until a terminal operation (`run`, `runCollect`, `head`, etc.)
|
|
32
|
-
- **Debuggable** -- streams render their pipeline structure via `toString`/`render`, so you can inspect what a stream does without running it
|
|
33
|
-
|
|
34
|
-
---
|
|
35
|
-
|
|
36
|
-
## Quick start
|
|
37
|
-
|
|
38
|
-
```scala
|
|
39
|
-
import zio.blocks.streams.*
|
|
40
|
-
import zio.blocks.chunk.Chunk
|
|
41
|
-
|
|
42
|
-
// Create a stream, transform it, consume it
|
|
43
|
-
val result: Either[Nothing, Chunk[Int]] =
|
|
44
|
-
Stream.range(1, 11) // 1 to 10
|
|
45
|
-
.filter(_ % 2 == 0) // keep evens
|
|
46
|
-
.map(_ * 10) // multiply by 10
|
|
47
|
-
.runCollect // collect into a Chunk
|
|
48
|
-
|
|
49
|
-
// result: Right(Chunk(20, 40, 60, 80, 100))
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
Key points:
|
|
53
|
-
|
|
54
|
-
- `Stream.range(1, 11)` creates a lazy stream of integers 1 through 10 (exclusive upper bound)
|
|
55
|
-
- `.filter` and `.map` add transformations without executing anything
|
|
56
|
-
- `.runCollect` is the terminal operation -- it drives evaluation and returns `Either[E, Chunk[A]]`
|
|
57
|
-
- Since `Stream.range` cannot fail, the error type is `Nothing` and the result is always `Right`
|
|
58
|
-
|
|
59
|
-
A stream that can fail:
|
|
60
|
-
|
|
61
|
-
```scala
|
|
62
|
-
val fallible: Either[String, Chunk[Int]] =
|
|
63
|
-
Stream.range(1, 6)
|
|
64
|
-
.flatMap { n =>
|
|
65
|
-
if n == 3 then Stream.fail("boom at 3")
|
|
66
|
-
else Stream.succeed(n)
|
|
67
|
-
}
|
|
68
|
-
.runCollect
|
|
69
|
-
|
|
70
|
-
// fallible: Left("boom at 3")
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
---
|
|
74
|
-
|
|
75
|
-
## Benchmarks
|
|
76
|
-
|
|
77
|
-
All benchmarks use 10,000 elements, measured in operations per second (higher is better). Run on Apple M-series, JDK 25, Scala 3.7.4.
|
|
78
|
-
|
|
79
|
-
| Benchmark | ZB Streams | Ox | Kyo | fs2 | Pekko |
|
|
80
|
-
|---|---|---|---|---|---|
|
|
81
|
-
| drain | 179,872 | 54,512 | 31,777 | 20,795 | 4,381 |
|
|
82
|
-
| map | 161,920 | 42,007 | 12,012 | 13,295 | 2,259 |
|
|
83
|
-
| filter | 168,541 | 47,933 | 19,962 | 14,977 | 2,901 |
|
|
84
|
-
| flatMap | 49,165 | 30,506 | 28,303 | 748 | 742 |
|
|
85
|
-
| take/drop | 322,470 | 28,708 | 64,640 | 28,836 | 2,379 |
|
|
86
|
-
| map+filter+flatMap | 980 | 508 | 602 | 19 | 16 |
|
|
87
|
-
| mixed depth 1 | 47,459 | 19,449 | 13,427 | 257 | 639 |
|
|
88
|
-
| mixed depth 2 | 33,859 | 15,336 | 7,328 | 208 | 459 |
|
|
89
|
-
| mixed depth 3 | 23,610 | 11,878 | 3,174 | 139 | 256 |
|
|
90
|
-
| nested flatMap (10K) | 8,161 | -- | -- | 937 | -- |
|
|
91
|
-
| nested concat (10K) | 6,140 | -- | 3 | 1,065 | 1 |
|
|
92
|
-
|
|
93
|
-
"--" indicates the benchmark was not run or the library crashed.
|
|
94
|
-
|
|
95
|
-
ZB Streams leads in every single-operator benchmark and maintains its advantage as pipeline depth increases. The "mixed depth" rows show cascading `map`/`filter`/`flatMap` stages -- ZB Streams degrades gracefully thanks to its trampolined execution model, while libraries without stack-safety (Ox) or with high per-element overhead (fs2, Pekko) fall off sharply.
|
|
96
|
-
|
|
97
|
-
---
|
|
98
|
-
|
|
99
|
-
## Core mental model
|
|
100
|
-
|
|
101
|
-
### 1) `Stream[E, A]` -- a lazy sequence
|
|
102
|
-
|
|
103
|
-
A `Stream[+E, +A]` is a **description** of a potentially infinite sequence of elements of type `A` that may fail with an error of type `E`. It is covariant in both type parameters.
|
|
104
|
-
|
|
105
|
-
Nothing happens when you construct a stream or chain transformations. Execution only begins when you call a terminal operation (`run`, `runCollect`, `runDrain`, `head`, `count`, etc.). Terminal operations return `Either[E, Z]`:
|
|
106
|
-
|
|
107
|
-
- `Left(e)` -- a typed stream error
|
|
108
|
-
- `Right(z)` -- the successful result
|
|
109
|
-
|
|
110
|
-
Untyped defects (unexpected exceptions) propagate as thrown exceptions, not as `Left` values.
|
|
111
|
-
|
|
112
|
-
```scala
|
|
113
|
-
// This does nothing -- it's just a description
|
|
114
|
-
val description: Stream[Nothing, Int] =
|
|
115
|
-
Stream.range(0, 1_000_000)
|
|
116
|
-
.filter(_ % 7 == 0)
|
|
117
|
-
.map(_ * 2)
|
|
118
|
-
.take(100)
|
|
119
|
-
|
|
120
|
-
// Only this line executes the pipeline
|
|
121
|
-
val result: Either[Nothing, Chunk[Int]] = description.runCollect
|
|
122
|
-
```
|
|
123
|
-
|
|
124
|
-
Streams render their pipeline structure as a human-readable string:
|
|
125
|
-
|
|
126
|
-
```scala
|
|
127
|
-
val s = Stream.range(0, 100).map(_ + 1).filter(_ > 50).take(10)
|
|
128
|
-
println(s) // Stream.range(0, 100).map(...).filter(...).take(10)
|
|
129
|
-
```
|
|
130
|
-
|
|
131
|
-
This makes debugging and logging straightforward -- you can see exactly what transformations a stream applies without running it.
|
|
132
|
-
|
|
133
|
-
---
|
|
134
|
-
|
|
135
|
-
### 2) `Sink[E, A, Z]` -- a consumer
|
|
136
|
-
|
|
137
|
-
A `Sink[+E, -A, +Z]` consumes elements of type `A` from a stream and produces a final result of type `Z`. Sinks are passed to `Stream.run`:
|
|
138
|
-
|
|
139
|
-
```scala
|
|
140
|
-
val stream = Stream.range(1, 101)
|
|
141
|
-
|
|
142
|
-
// Built-in sinks
|
|
143
|
-
val total: Either[Nothing, Long] = stream.run(Sink.count)
|
|
144
|
-
val items: Either[Nothing, Chunk[Int]] = stream.run(Sink.collectAll)
|
|
145
|
-
val sum: Either[Nothing, Long] = stream.run(Sink.sumInt)
|
|
146
|
-
val first: Either[Nothing, Option[Int]] = stream.run(Sink.head)
|
|
147
|
-
```
|
|
148
|
-
|
|
149
|
-
Most sinks also have convenience methods directly on `Stream`:
|
|
150
|
-
|
|
151
|
-
```scala
|
|
152
|
-
stream.count // Either[Nothing, Long]
|
|
153
|
-
stream.runCollect // Either[Nothing, Chunk[Int]]
|
|
154
|
-
stream.head // Either[Nothing, Option[Int]]
|
|
155
|
-
stream.last // Either[Nothing, Option[Int]]
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
Sinks compose with `contramap` (pre-process input) and `map` (post-process result):
|
|
159
|
-
|
|
160
|
-
```scala
|
|
161
|
-
val lengthSink: Sink[Nothing, String, Long] =
|
|
162
|
-
Sink.sumInt.contramap[String](_.length)
|
|
163
|
-
|
|
164
|
-
val doubled: Sink[Nothing, Int, Long] =
|
|
165
|
-
Sink.sumInt.map(_ * 2)
|
|
166
|
-
```
|
|
167
|
-
|
|
168
|
-
Built-in sinks:
|
|
169
|
-
|
|
170
|
-
| Sink | Result type | Description |
|
|
171
|
-
|---|---|---|
|
|
172
|
-
| `Sink.collectAll` | `Chunk[A]` | Collects all elements |
|
|
173
|
-
| `Sink.drain` | `Unit` | Consumes and discards all elements |
|
|
174
|
-
| `Sink.count` | `Long` | Counts elements |
|
|
175
|
-
| `Sink.foldLeft(z)(f)` | `Z` | Left fold with initial value |
|
|
176
|
-
| `Sink.foreach(f)` | `Unit` | Side-effect per element |
|
|
177
|
-
| `Sink.head` | `Option[A]` | First element |
|
|
178
|
-
| `Sink.last` | `Option[A]` | Last element |
|
|
179
|
-
| `Sink.take(n)` | `Chunk[A]` | First n elements |
|
|
180
|
-
| `Sink.exists(p)` | `Boolean` | Short-circuiting existential |
|
|
181
|
-
| `Sink.forall(p)` | `Boolean` | Short-circuiting universal |
|
|
182
|
-
| `Sink.find(p)` | `Option[A]` | First matching element |
|
|
183
|
-
| `Sink.sumInt` | `Long` | Sum of Ints (zero-boxing) |
|
|
184
|
-
| `Sink.sumLong` | `Long` | Sum of Longs (zero-boxing) |
|
|
185
|
-
| `Sink.sumFloat` | `Double` | Sum of Floats (zero-boxing) |
|
|
186
|
-
| `Sink.sumDouble` | `Double` | Sum of Doubles (zero-boxing) |
|
|
187
|
-
| `Sink.fromOutputStream(os)` | `Unit` | Writes bytes to an `OutputStream` |
|
|
188
|
-
| `Sink.fromJavaWriter(w)` | `Unit` | Writes chars to a `java.io.Writer` |
|
|
189
|
-
| `Sink.fail(e)` | `Nothing` | Fails immediately |
|
|
190
|
-
| `Sink.create(f)` | `Z` | Custom sink from `Reader[A] => Z` |
|
|
191
|
-
|
|
192
|
-
---
|
|
193
|
-
|
|
194
|
-
### 3) `Pipeline[In, Out]` -- reusable transformation
|
|
195
|
-
|
|
196
|
-
A `Pipeline[-In, +Out]` is a reusable stream transformation. It decouples the transformation logic from any specific stream, so you can define it once and apply it many times.
|
|
197
|
-
|
|
198
|
-
```scala
|
|
199
|
-
// Define a reusable pipeline
|
|
200
|
-
val normalize: Pipeline[Int, Double] =
|
|
201
|
-
Pipeline.filter[Int](_ > 0)
|
|
202
|
-
.andThen(Pipeline.map[Int, Double](_.toDouble / 100.0))
|
|
203
|
-
|
|
204
|
-
// Apply to different streams
|
|
205
|
-
val result1 = Stream.range(-10, 10).via(normalize).runCollect
|
|
206
|
-
val result2 = Stream(42, -5, 100, 0).via(normalize).runCollect
|
|
207
|
-
```
|
|
208
|
-
|
|
209
|
-
Pipelines compose with `andThen`:
|
|
210
|
-
|
|
211
|
-
```scala
|
|
212
|
-
val step1: Pipeline[String, Int] =
|
|
213
|
-
Pipeline.map[String, Int](_.length)
|
|
214
|
-
|
|
215
|
-
val step2: Pipeline[Int, Int] =
|
|
216
|
-
Pipeline.filter[Int](_ > 3)
|
|
217
|
-
|
|
218
|
-
val combined: Pipeline[String, Int] =
|
|
219
|
-
step1.andThen(step2)
|
|
220
|
-
```
|
|
221
|
-
|
|
222
|
-
You can also apply a pipeline to a sink with `andThenSink` / `applyToSink`, which pre-processes the sink's input:
|
|
223
|
-
|
|
224
|
-
```scala
|
|
225
|
-
val countLong: Sink[Nothing, String, Long] =
|
|
226
|
-
Pipeline.map[String, Int](_.length)
|
|
227
|
-
.andThenSink(Sink.sumInt)
|
|
228
|
-
```
|
|
229
|
-
|
|
230
|
-
Built-in pipeline factories:
|
|
231
|
-
|
|
232
|
-
| Factory | Description |
|
|
233
|
-
|---|---|
|
|
234
|
-
| `Pipeline.map(f)` | Transform each element |
|
|
235
|
-
| `Pipeline.filter(p)` | Keep elements matching predicate |
|
|
236
|
-
| `Pipeline.collect(pf)` | Partial function -- filter + map |
|
|
237
|
-
| `Pipeline.take(n)` | Keep first n elements |
|
|
238
|
-
| `Pipeline.drop(n)` | Skip first n elements |
|
|
239
|
-
| `Pipeline.identity` | Pass-through (useful as a base for composition) |
|
|
240
|
-
|
|
241
|
-
---
|
|
242
|
-
|
|
243
|
-
### 4) `Reader[A]` -- low-level pull source
|
|
244
|
-
|
|
245
|
-
`Reader[+Elem]` is the low-level, pull-based source that backs every stream. Most users will never interact with `Reader` directly; it is the compilation target when a stream runs.
|
|
246
|
-
|
|
247
|
-
The protocol is simple:
|
|
248
|
-
|
|
249
|
-
- `read(sentinel)` -- returns the next element, or `sentinel` when exhausted
|
|
250
|
-
- `close()` -- signal the consumer is done
|
|
251
|
-
- `isClosed` -- check whether the reader has been closed
|
|
252
|
-
|
|
253
|
-
For primitive types, specialized methods avoid boxing:
|
|
254
|
-
|
|
255
|
-
- `readInt(sentinel: Long): Long`
|
|
256
|
-
- `readLong(sentinel: Long): Long`
|
|
257
|
-
- `readFloat(sentinel: Double): Double`
|
|
258
|
-
- `readDouble(sentinel: Double): Double`
|
|
259
|
-
|
|
260
|
-
You interact with `Reader` in two situations:
|
|
261
|
-
|
|
262
|
-
1. **Custom sources** -- create a stream from a `Reader` via `Stream.fromReader`
|
|
263
|
-
2. **Manual pull** -- open a stream for element-by-element control via `stream.start`
|
|
264
|
-
|
|
265
|
-
```scala
|
|
266
|
-
import zio.blocks.scope.*
|
|
267
|
-
|
|
268
|
-
Scope.global.scoped { scope =>
|
|
269
|
-
import scope.*
|
|
270
|
-
|
|
271
|
-
// Open a stream for manual pulling
|
|
272
|
-
val reader: $[Reader[Int]] = Stream.range(1, 6).start(using scope)
|
|
273
|
-
|
|
274
|
-
$(reader) { r =>
|
|
275
|
-
var v = r.read(-1)
|
|
276
|
-
while v != -1 do
|
|
277
|
-
println(v) // prints 1, 2, 3, 4, 5
|
|
278
|
-
v = r.read(-1)
|
|
279
|
-
}
|
|
280
|
-
// reader is closed automatically when scope exits
|
|
281
|
-
}
|
|
282
|
-
```
|
|
283
|
-
|
|
284
|
-
---
|
|
285
|
-
|
|
286
|
-
### 5) Error handling
|
|
287
|
-
|
|
288
|
-
Streams distinguish between two kinds of failures:
|
|
289
|
-
|
|
290
|
-
- **Typed errors** (`E`) -- domain errors you expect and handle. These appear as `Left` in the `Either` result.
|
|
291
|
-
- **Defects** (`Throwable`) -- unexpected exceptions. These propagate as thrown exceptions, bypassing the `Either` channel.
|
|
292
|
-
|
|
293
|
-
```scala
|
|
294
|
-
// Create a failing stream
|
|
295
|
-
val failing: Stream[String, Int] =
|
|
296
|
-
Stream(1, 2, 3) ++ Stream.fail("oops") ++ Stream(4, 5)
|
|
297
|
-
|
|
298
|
-
// catchAll: recover from typed errors
|
|
299
|
-
val recovered: Stream[Nothing, Int] =
|
|
300
|
-
failing.catchAll(_ => Stream(99))
|
|
301
|
-
|
|
302
|
-
recovered.runCollect // Right(Chunk(1, 2, 3, 99))
|
|
303
|
-
```
|
|
304
|
-
|
|
305
|
-
Error handling operators:
|
|
306
|
-
|
|
307
|
-
| Operator | Description |
|
|
308
|
-
|---|---|
|
|
309
|
-
| `catchAll(f: E => Stream[E2, A])` | Recover from all typed errors |
|
|
310
|
-
| `catchDefect(pf: PartialFunction[Throwable, Stream[E1, A]])` | Recover from matching defects |
|
|
311
|
-
| `mapError(f: E => E2)` | Transform the error type |
|
|
312
|
-
| `orElse(that)` / `\|\|(that)` | Fall back to another stream on error |
|
|
313
|
-
|
|
314
|
-
---
|
|
315
|
-
|
|
316
|
-
### 6) Resource safety
|
|
317
|
-
|
|
318
|
-
Streams integrate with `zio.blocks.scope.Scope` for deterministic finalization. Several constructors guarantee that acquired resources are released when the stream closes, whether it completes normally, short-circuits, or fails.
|
|
319
|
-
|
|
320
|
-
#### `fromAcquireRelease`
|
|
321
|
-
|
|
322
|
-
The primary resource-safe constructor. Acquires a resource, uses it to produce a stream, and guarantees the release function runs on close:
|
|
323
|
-
|
|
324
|
-
```scala
|
|
325
|
-
import java.io.BufferedReader
|
|
326
|
-
import java.io.FileReader
|
|
327
|
-
|
|
328
|
-
val lines: Stream[Nothing, String] =
|
|
329
|
-
Stream.fromAcquireRelease(
|
|
330
|
-
acquire = new BufferedReader(new FileReader("data.txt")),
|
|
331
|
-
release = _.close()
|
|
332
|
-
) { reader =>
|
|
333
|
-
Stream.unfold(()) { _ =>
|
|
334
|
-
Option(reader.readLine()).map(line => (line, ()))
|
|
335
|
-
}
|
|
336
|
-
}
|
|
337
|
-
|
|
338
|
-
// The BufferedReader is closed when the stream finishes,
|
|
339
|
-
// even if the consumer takes only a few lines
|
|
340
|
-
lines.take(5).runCollect
|
|
341
|
-
```
|
|
342
|
-
|
|
343
|
-
If the resource is `AutoCloseable`, the release function defaults to calling `close()`:
|
|
344
|
-
|
|
345
|
-
```scala
|
|
346
|
-
val lines: Stream[Nothing, String] =
|
|
347
|
-
Stream.fromAcquireRelease(
|
|
348
|
-
acquire = new BufferedReader(new FileReader("data.txt"))
|
|
349
|
-
) { reader =>
|
|
350
|
-
Stream.unfold(()) { _ =>
|
|
351
|
-
Option(reader.readLine()).map(line => (line, ()))
|
|
352
|
-
}
|
|
353
|
-
}
|
|
354
|
-
```
|
|
355
|
-
|
|
356
|
-
#### `fromResource`
|
|
357
|
-
|
|
358
|
-
Integrates with `zio.blocks.scope.Resource` directly:
|
|
359
|
-
|
|
360
|
-
```scala
|
|
361
|
-
import zio.blocks.scope.Resource
|
|
362
|
-
|
|
363
|
-
val resource: Resource[BufferedReader] =
|
|
364
|
-
Resource.fromAutoCloseable(new BufferedReader(new FileReader("data.txt")))
|
|
365
|
-
|
|
366
|
-
val lines: Stream[Nothing, String] =
|
|
367
|
-
Stream.fromResource(resource) { reader =>
|
|
368
|
-
Stream.unfold(()) { _ =>
|
|
369
|
-
Option(reader.readLine()).map(line => (line, ()))
|
|
370
|
-
}
|
|
371
|
-
}
|
|
372
|
-
```
|
|
373
|
-
|
|
374
|
-
#### `ensuring` and `defer`
|
|
375
|
-
|
|
376
|
-
For attaching finalizers to existing streams:
|
|
377
|
-
|
|
378
|
-
```scala
|
|
379
|
-
// ensuring: run a finalizer when the stream closes
|
|
380
|
-
val withCleanup: Stream[Nothing, Int] =
|
|
381
|
-
Stream.range(1, 11).ensuring(println("stream closed"))
|
|
382
|
-
|
|
383
|
-
// defer: register a release action (runs on close, not on construction)
|
|
384
|
-
val withDefer: Stream[Nothing, Int] =
|
|
385
|
-
Stream.defer(println("cleanup")) ++ Stream.range(1, 6)
|
|
386
|
-
```
|
|
387
|
-
|
|
388
|
-
#### `start` with `Scope`
|
|
389
|
-
|
|
390
|
-
For manual pull-based consumption with scope-managed lifetime:
|
|
391
|
-
|
|
392
|
-
```scala
|
|
393
|
-
import zio.blocks.scope.*
|
|
394
|
-
|
|
395
|
-
Scope.global.scoped { scope =>
|
|
396
|
-
import scope.*
|
|
397
|
-
val reader = Stream.range(1, 100).start(using scope)
|
|
398
|
-
// reader is automatically closed when scope exits
|
|
399
|
-
}
|
|
400
|
-
```
|
|
401
|
-
|
|
402
|
-
---
|
|
403
|
-
|
|
404
|
-
## Usage examples
|
|
405
|
-
|
|
406
|
-
### Creating streams
|
|
407
|
-
|
|
408
|
-
```scala
|
|
409
|
-
import zio.blocks.streams.*
|
|
410
|
-
import zio.blocks.chunk.Chunk
|
|
411
|
-
|
|
412
|
-
// From explicit elements
|
|
413
|
-
Stream(1, 2, 3) // Stream[Nothing, Int]
|
|
414
|
-
Stream("a", "b", "c") // Stream[Nothing, String]
|
|
415
|
-
|
|
416
|
-
// From collections
|
|
417
|
-
Stream.fromChunk(Chunk(1, 2, 3)) // Stream[Nothing, Int]
|
|
418
|
-
Stream.fromIterable(List("x", "y", "z")) // Stream[Nothing, String]
|
|
419
|
-
Stream.fromIterator(Iterator.from(1)) // Stream[Nothing, Int] (lazy)
|
|
420
|
-
|
|
421
|
-
// Ranges
|
|
422
|
-
Stream.range(0, 100) // 0 to 99
|
|
423
|
-
Stream.fromRange(1 to 50) // 1 to 50
|
|
424
|
-
|
|
425
|
-
// Single values (primitive-specialized)
|
|
426
|
-
Stream.succeed(42) // Stream[Nothing, Int]
|
|
427
|
-
Stream.succeed(3.14) // Stream[Nothing, Double]
|
|
428
|
-
Stream.succeed("hello") // Stream[Nothing, String]
|
|
429
|
-
|
|
430
|
-
// Special streams
|
|
431
|
-
Stream.empty // Stream[Nothing, Nothing]
|
|
432
|
-
Stream.fail("error") // Stream[String, Nothing]
|
|
433
|
-
Stream.die(new Exception("defect")) // throws on evaluation
|
|
434
|
-
|
|
435
|
-
// Generators
|
|
436
|
-
Stream.repeat(1) // infinite stream of 1s
|
|
437
|
-
Stream.iterate(1)(_ * 2) // 1, 2, 4, 8, 16, ...
|
|
438
|
-
Stream.repeatThunk(scala.util.Random.nextInt(100)) // infinite random ints
|
|
439
|
-
Stream.unfold(0)(n => // 0, 1, 2, ..., 9
|
|
440
|
-
if n < 10 then Some((n, n + 1)) else None
|
|
441
|
-
)
|
|
442
|
-
|
|
443
|
-
// Side-effects
|
|
444
|
-
Stream.eval(println("hello")) // prints, emits nothing
|
|
445
|
-
Stream.attempt(someFallibleCall()) // captures exceptions as typed errors
|
|
446
|
-
Stream.attemptEval(riskyEffect()) // same, for Unit-returning effects
|
|
447
|
-
|
|
448
|
-
// Deferred construction (useful for recursion)
|
|
449
|
-
Stream.suspend(expensiveStreamBuilder())
|
|
450
|
-
|
|
451
|
-
// I/O sources (auto-closing)
|
|
452
|
-
Stream.fromInputStream(inputStream) // Stream[IOException, Int] (bytes as 0-255, auto-closes)
|
|
453
|
-
Stream.fromJavaReader(javaReader) // Stream[IOException, Char] (auto-closes)
|
|
454
|
-
|
|
455
|
-
// I/O sources (borrowing -- caller manages lifetime)
|
|
456
|
-
Stream.fromInputStreamUnmanaged(inputStream) // Stream[IOException, Int] (does NOT close)
|
|
457
|
-
Stream.fromJavaReaderUnmanaged(javaReader) // Stream[IOException, Char] (does NOT close)
|
|
458
|
-
```
|
|
459
|
-
|
|
460
|
-
---
|
|
461
|
-
|
|
462
|
-
### Transforming streams
|
|
463
|
-
|
|
464
|
-
```scala
|
|
465
|
-
val s = Stream.range(1, 21) // 1 to 20
|
|
466
|
-
|
|
467
|
-
// map: transform each element
|
|
468
|
-
s.map(_ * 2) // 2, 4, 6, ..., 40
|
|
469
|
-
|
|
470
|
-
// filter: keep matching elements
|
|
471
|
-
s.filter(_ % 3 == 0) // 3, 6, 9, 12, 15, 18
|
|
472
|
-
|
|
473
|
-
// flatMap: expand each element into a sub-stream
|
|
474
|
-
s.flatMap(n => Stream(n, n * 10)) // 1, 10, 2, 20, 3, 30, ...
|
|
475
|
-
|
|
476
|
-
// collect: partial function (filter + map)
|
|
477
|
-
s.collect { case n if n % 2 == 0 => n / 2 } // 1, 2, 3, ..., 10
|
|
478
|
-
|
|
479
|
-
// take / drop / takeWhile
|
|
480
|
-
s.take(5) // 1, 2, 3, 4, 5
|
|
481
|
-
s.drop(15) // 16, 17, 18, 19, 20
|
|
482
|
-
s.takeWhile(_ < 8) // 1, 2, 3, 4, 5, 6, 7
|
|
483
|
-
|
|
484
|
-
// scan: running accumulator (emits initial value + one value per element)
|
|
485
|
-
Stream(1, 2, 3, 4).scan(0)(_ + _) // 0, 1, 3, 6, 10
|
|
486
|
-
|
|
487
|
-
// mapAccum: stateful transformation
|
|
488
|
-
s.mapAccum(0) { (acc, n) =>
|
|
489
|
-
val newAcc = acc + n
|
|
490
|
-
(newAcc, newAcc)
|
|
491
|
-
}
|
|
492
|
-
// running sum: 1, 3, 6, 10, 15, ...
|
|
493
|
-
|
|
494
|
-
// grouped: collect into fixed-size chunks
|
|
495
|
-
Stream.range(1, 11).grouped(3)
|
|
496
|
-
// Chunk(1,2,3), Chunk(4,5,6), Chunk(7,8,9), Chunk(10)
|
|
497
|
-
|
|
498
|
-
// sliding: overlapping windows
|
|
499
|
-
Stream.range(1, 7).sliding(3, 1)
|
|
500
|
-
// Chunk(1,2,3), Chunk(2,3,4), Chunk(3,4,5), Chunk(4,5,6)
|
|
501
|
-
|
|
502
|
-
// intersperse: insert separator between elements
|
|
503
|
-
Stream("a", "b", "c").intersperse(",") // "a", ",", "b", ",", "c"
|
|
504
|
-
|
|
505
|
-
// distinct / distinctBy: deduplication
|
|
506
|
-
Stream(1, 2, 2, 3, 1, 3).distinct // 1, 2, 3
|
|
507
|
-
Stream("ab", "cd", "ae").distinctBy(_.head) // "ab", "cd"
|
|
508
|
-
|
|
509
|
-
// zipWithIndex: pair elements with their 0-based index
|
|
510
|
-
Stream("a", "b", "c").zipWithIndex
|
|
511
|
-
// ("a", 0L), ("b", 1L), ("c", 2L)
|
|
512
|
-
|
|
513
|
-
// tapEach: side-effect without changing elements
|
|
514
|
-
s.tapEach(n => println(s"processing $n"))
|
|
515
|
-
|
|
516
|
-
// concat: sequence two streams
|
|
517
|
-
Stream(1, 2) ++ Stream(3, 4) // 1, 2, 3, 4
|
|
518
|
-
|
|
519
|
-
// repeated: restart on completion
|
|
520
|
-
Stream(1, 2, 3).repeated.take(8) // 1, 2, 3, 1, 2, 3, 1, 2
|
|
521
|
-
|
|
522
|
-
// via: apply a Pipeline
|
|
523
|
-
s.via(Pipeline.filter[Int](_ > 10)) // 11, 12, ..., 20
|
|
524
|
-
```
|
|
525
|
-
|
|
526
|
-
---
|
|
527
|
-
|
|
528
|
-
### Zipping streams with `&&`
|
|
529
|
-
|
|
530
|
-
The `&&` operator zips two streams element-by-element into tuples. The resulting stream ends when either input is exhausted.
|
|
531
|
-
|
|
532
|
-
```scala
|
|
533
|
-
val names: Stream[Nothing, String] = Stream("Alice", "Bob", "Charlie")
|
|
534
|
-
val ages: Stream[Nothing, Int] = Stream(30, 25, 35)
|
|
535
|
-
val ids: Stream[Nothing, Long] = Stream(1L, 2L, 3L)
|
|
536
|
-
|
|
537
|
-
// Two-way zip
|
|
538
|
-
val pairs: Stream[Nothing, (String, Int)] = names && ages
|
|
539
|
-
pairs.runCollect // Right(Chunk(("Alice", 30), ("Bob", 25), ("Charlie", 35)))
|
|
540
|
-
|
|
541
|
-
// Three-way zip -- tuples flatten automatically
|
|
542
|
-
val triples: Stream[Nothing, (String, Int, Long)] = names && ages && ids
|
|
543
|
-
triples.runCollect // Right(Chunk(("Alice", 30, 1L), ("Bob", 25, 2L), ("Charlie", 35, 3L)))
|
|
544
|
-
```
|
|
545
|
-
|
|
546
|
-
When the error types differ, they widen via union:
|
|
547
|
-
|
|
548
|
-
```scala
|
|
549
|
-
val s1: Stream[String, Int] = Stream(1, 2, 3)
|
|
550
|
-
val s2: Stream[IOException, Int] = Stream(4, 5, 6)
|
|
551
|
-
val zipped: Stream[String | IOException, (Int, Int)] = s1 && s2
|
|
552
|
-
```
|
|
553
|
-
|
|
554
|
-
---
|
|
555
|
-
|
|
556
|
-
### Primitive specialization
|
|
557
|
-
|
|
558
|
-
ZB Streams eliminates boxing for `Int`, `Long`, `Float`, and `Double` elements throughout the entire pipeline. Every intermediate step uses specialized `readInt`/`writeInt` (or the corresponding type) methods, so no `java.lang.Integer` wrappers are allocated.
|
|
559
|
-
|
|
560
|
-
```scala
|
|
561
|
-
// This entire pipeline runs with ZERO boxing of the Int elements.
|
|
562
|
-
// Every step uses specialized readInt/writeInt internally.
|
|
563
|
-
val sum: Either[Nothing, Long] =
|
|
564
|
-
Stream.range(0, 1_000_000) // Int-specialized source
|
|
565
|
-
.filter(_ % 2 == 0) // Int-specialized filter
|
|
566
|
-
.map(_ * 3) // Int->Int specialized map
|
|
567
|
-
.runFold(0L)(_ + _) // Long-specialized accumulator
|
|
568
|
-
|
|
569
|
-
// Compare: in fs2 or ZIO Streams, every Int would be boxed to java.lang.Integer
|
|
570
|
-
// at each pipeline stage boundary.
|
|
571
|
-
```
|
|
572
|
-
|
|
573
|
-
This matters most for numeric workloads -- data processing, statistics, encoding/decoding -- where millions of elements flow through multi-stage pipelines. The benchmark results above reflect this advantage directly.
|
|
574
|
-
|
|
575
|
-
---
|
|
576
|
-
|
|
577
|
-
### Consuming streams
|
|
578
|
-
|
|
579
|
-
```scala
|
|
580
|
-
val s = Stream.range(1, 11) // 1 to 10
|
|
581
|
-
|
|
582
|
-
// Collect all elements
|
|
583
|
-
s.runCollect // Right(Chunk(1, 2, 3, ..., 10))
|
|
584
|
-
|
|
585
|
-
// Discard all elements (run for side-effects only)
|
|
586
|
-
s.tapEach(println).runDrain
|
|
587
|
-
|
|
588
|
-
// Fold
|
|
589
|
-
s.runFold(0)(_ + _) // Right(55) (Int accumulator)
|
|
590
|
-
s.runFold(0L)(_ + _) // Right(55L) (Long accumulator)
|
|
591
|
-
s.runFold(0.0)(_ + _) // Right(55.0) (Double accumulator)
|
|
592
|
-
|
|
593
|
-
// Foreach
|
|
594
|
-
s.runForeach(n => println(n))
|
|
595
|
-
s.foreach(n => println(n)) // alias
|
|
596
|
-
|
|
597
|
-
// Aggregates
|
|
598
|
-
s.count // Right(10L)
|
|
599
|
-
s.head // Right(Some(1))
|
|
600
|
-
s.last // Right(Some(10))
|
|
601
|
-
s.exists(_ > 5) // Right(true)
|
|
602
|
-
s.forall(_ > 0) // Right(true)
|
|
603
|
-
s.find(_ > 7) // Right(Some(8))
|
|
604
|
-
|
|
605
|
-
// Run with an explicit Sink
|
|
606
|
-
s.run(Sink.sumInt) // Right(55L)
|
|
607
|
-
s.run(Sink.take(3)) // Right(Chunk(1, 2, 3))
|
|
608
|
-
```
|
|
609
|
-
|
|
610
|
-
---
|
|
611
|
-
|
|
612
|
-
### Error handling patterns
|
|
613
|
-
|
|
614
|
-
```scala
|
|
615
|
-
// Typed error: appears in Either
|
|
616
|
-
val result = Stream.fail("not found").runCollect
|
|
617
|
-
// result: Left("not found")
|
|
618
|
-
|
|
619
|
-
// Recover and continue
|
|
620
|
-
val safe =
|
|
621
|
-
Stream(1, 2) ++ Stream.fail("oops") ++ Stream(3)
|
|
622
|
-
val recovered = safe.catchAll(_ => Stream(99)).runCollect
|
|
623
|
-
// Right(Chunk(1, 2, 99))
|
|
624
|
-
|
|
625
|
-
// Transform error type
|
|
626
|
-
val mapped =
|
|
627
|
-
Stream.fail("bad input")
|
|
628
|
-
.mapError(msg => new IllegalArgumentException(msg))
|
|
629
|
-
// Stream[IllegalArgumentException, Nothing]
|
|
630
|
-
|
|
631
|
-
// Fallback stream
|
|
632
|
-
val primary: Stream[String, Int] = Stream.fail("down")
|
|
633
|
-
val backup: Stream[String, Int] = Stream(1, 2, 3)
|
|
634
|
-
val result2 = (primary || backup).runCollect
|
|
635
|
-
// Right(Chunk(1, 2, 3))
|
|
636
|
-
|
|
637
|
-
// Catch defects (unexpected exceptions)
|
|
638
|
-
val risky: Stream[Nothing, Int] =
|
|
639
|
-
Stream(1, 2, 3).map { n =>
|
|
640
|
-
if n == 2 then throw new ArithmeticException("boom")
|
|
641
|
-
else n
|
|
642
|
-
}
|
|
643
|
-
|
|
644
|
-
val handled = risky.catchDefect {
|
|
645
|
-
case _: ArithmeticException => Stream(0)
|
|
646
|
-
}.runCollect
|
|
647
|
-
// Right(Chunk(1, 0))
|
|
648
|
-
```
|
|
649
|
-
|
|
650
|
-
---
|
|
651
|
-
|
|
652
|
-
### Resource safety patterns
|
|
653
|
-
|
|
654
|
-
```scala
|
|
655
|
-
import zio.blocks.streams.*
|
|
656
|
-
import zio.blocks.scope.*
|
|
657
|
-
|
|
658
|
-
// Bracket pattern: acquire/use/release
|
|
659
|
-
def fileLines(path: String): Stream[Nothing, String] =
|
|
660
|
-
Stream.fromAcquireRelease(
|
|
661
|
-
acquire = scala.io.Source.fromFile(path),
|
|
662
|
-
release = _.close()
|
|
663
|
-
) { source =>
|
|
664
|
-
Stream.fromIterable(source.getLines().toList)
|
|
665
|
-
}
|
|
666
|
-
|
|
667
|
-
// Compose resource-safe streams -- both resources are released
|
|
668
|
-
val merged =
|
|
669
|
-
fileLines("input1.txt") ++ fileLines("input2.txt")
|
|
670
|
-
|
|
671
|
-
// Only reads 10 lines; both files are still closed properly
|
|
672
|
-
merged.take(10).runCollect
|
|
673
|
-
|
|
674
|
-
// ensuring: attach a finalizer
|
|
675
|
-
var cleaned = false
|
|
676
|
-
Stream.range(1, 6)
|
|
677
|
-
.ensuring { cleaned = true }
|
|
678
|
-
.take(2)
|
|
679
|
-
.runDrain
|
|
680
|
-
// cleaned == true, even though only 2 of 5 elements were consumed
|
|
681
|
-
|
|
682
|
-
// defer: register cleanup that runs on stream close
|
|
683
|
-
val withDefer =
|
|
684
|
-
Stream.defer(println("releasing lock")) ++
|
|
685
|
-
Stream.range(1, 100)
|
|
686
|
-
```
|
|
687
|
-
|
|
688
|
-
---
|
|
689
|
-
|
|
690
|
-
### NIO integration (JVM only)
|
|
691
|
-
|
|
692
|
-
On the JVM, `NioStreams` and `NioSinks` provide zero-copy integration with `java.nio` buffers and channels.
|
|
693
|
-
|
|
694
|
-
#### `NioStreams` -- creating streams from NIO sources
|
|
695
|
-
|
|
696
|
-
```scala
|
|
697
|
-
import zio.blocks.streams.*
|
|
698
|
-
import java.nio.ByteBuffer
|
|
699
|
-
import java.nio.channels.FileChannel
|
|
700
|
-
import java.nio.file.{Paths, StandardOpenOption}
|
|
701
|
-
|
|
702
|
-
// From a ByteBuffer
|
|
703
|
-
val buf = ByteBuffer.wrap(Array[Byte](1, 2, 3, 4, 5))
|
|
704
|
-
NioStreams.fromByteBuffer(buf).runCollect
|
|
705
|
-
// Right(Chunk(1, 2, 3, 4, 5))
|
|
706
|
-
|
|
707
|
-
// Typed buffer views (zero-boxing)
|
|
708
|
-
val intBuf = ByteBuffer.allocate(16).putInt(1).putInt(2).putInt(3).putInt(4).flip()
|
|
709
|
-
NioStreams.fromByteBufferInt(intBuf).runCollect
|
|
710
|
-
// Right(Chunk(1, 2, 3, 4))
|
|
711
|
-
|
|
712
|
-
// Similarly: fromByteBufferLong, fromByteBufferFloat, fromByteBufferDouble
|
|
713
|
-
|
|
714
|
-
// From a ReadableByteChannel (auto-closing)
|
|
715
|
-
val ch = FileChannel.open(Paths.get("data.bin"), StandardOpenOption.READ)
|
|
716
|
-
val bytes = NioStreams.fromChannel(ch, bufSize = 4096).runCollect
|
|
717
|
-
// ch is closed automatically when the stream completes
|
|
718
|
-
|
|
719
|
-
// From a ReadableByteChannel (borrowing -- caller manages lifetime)
|
|
720
|
-
val ch2 = FileChannel.open(Paths.get("data.bin"), StandardOpenOption.READ)
|
|
721
|
-
val bytes2 = NioStreams.fromChannelUnmanaged(ch2, bufSize = 4096).runCollect
|
|
722
|
-
ch2.close() // caller is responsible for closing
|
|
723
|
-
```
|
|
724
|
-
|
|
725
|
-
#### `NioSinks` -- writing to NIO targets
|
|
726
|
-
|
|
727
|
-
```scala
|
|
728
|
-
import java.nio.ByteBuffer
|
|
729
|
-
import java.nio.channels.FileChannel
|
|
730
|
-
import java.nio.file.{Paths, StandardOpenOption}
|
|
731
|
-
|
|
732
|
-
// Write to a ByteBuffer
|
|
733
|
-
val outBuf = ByteBuffer.allocate(1024)
|
|
734
|
-
Stream.fromInputStream(inputStream).run(NioSinks.fromByteBuffer(outBuf))
|
|
735
|
-
|
|
736
|
-
// Typed buffer sinks (zero-boxing)
|
|
737
|
-
Stream.range(1, 5).run(NioSinks.fromByteBufferInt(outBuf))
|
|
738
|
-
// Also: fromByteBufferLong, fromByteBufferFloat, fromByteBufferDouble
|
|
739
|
-
|
|
740
|
-
// Write to a WritableByteChannel (buffered)
|
|
741
|
-
val outCh = FileChannel.open(
|
|
742
|
-
Paths.get("output.bin"),
|
|
743
|
-
StandardOpenOption.WRITE, StandardOpenOption.CREATE
|
|
744
|
-
)
|
|
745
|
-
Stream.fromInputStream(inputStream).run(NioSinks.fromChannel(outCh))
|
|
746
|
-
outCh.close()
|
|
747
|
-
```
|
|
748
|
-
|
|
749
|
-
---
|
|
750
|
-
|
|
751
|
-
### Pipeline composition
|
|
752
|
-
|
|
753
|
-
```scala
|
|
754
|
-
import zio.blocks.streams.*
|
|
755
|
-
|
|
756
|
-
// Build reusable transformation steps
|
|
757
|
-
val parseInts: Pipeline[String, Int] =
|
|
758
|
-
Pipeline.collect[String, Int] {
|
|
759
|
-
case s if s.matches("-?\\d+") => s.toInt
|
|
760
|
-
}
|
|
761
|
-
|
|
762
|
-
val positiveOnly: Pipeline[Int, Int] =
|
|
763
|
-
Pipeline.filter[Int](_ > 0)
|
|
764
|
-
|
|
765
|
-
val doubled: Pipeline[Int, Int] =
|
|
766
|
-
Pipeline.map[Int, Int](_ * 2)
|
|
767
|
-
|
|
768
|
-
// Compose into a single pipeline
|
|
769
|
-
val fullPipeline: Pipeline[String, Int] =
|
|
770
|
-
parseInts
|
|
771
|
-
.andThen(positiveOnly)
|
|
772
|
-
.andThen(doubled)
|
|
773
|
-
|
|
774
|
-
// Apply to any stream of strings
|
|
775
|
-
Stream("10", "abc", "-3", "7", "0", "25")
|
|
776
|
-
.via(fullPipeline)
|
|
777
|
-
.runCollect
|
|
778
|
-
// Right(Chunk(20, 14, 50))
|
|
779
|
-
|
|
780
|
-
// Apply to a sink (pre-process the sink's input)
|
|
781
|
-
val sumPositiveDoubled: Sink[Nothing, String, Long] =
|
|
782
|
-
fullPipeline.andThenSink(Sink.sumInt)
|
|
783
|
-
|
|
784
|
-
Stream("10", "abc", "-3", "7", "0", "25")
|
|
785
|
-
.run(sumPositiveDoubled)
|
|
786
|
-
// Right(84L)
|
|
787
|
-
```
|
|
788
|
-
|
|
789
|
-
---
|
|
790
|
-
|
|
791
|
-
## API reference
|
|
792
|
-
|
|
793
|
-
### `Stream[+E, +A]`
|
|
794
|
-
|
|
795
|
-
#### Constructors
|
|
796
|
-
|
|
797
|
-
```scala
|
|
798
|
-
object Stream:
|
|
799
|
-
def apply[A](as: A*): Stream[Nothing, A]
|
|
800
|
-
val empty: Stream[Nothing, Nothing]
|
|
801
|
-
def succeed[A](a: A): Stream[Nothing, A] // also specialized for primitives
|
|
802
|
-
def fail[E](error: E): Stream[E, Nothing]
|
|
803
|
-
def die(t: Throwable): Stream[Nothing, Nothing]
|
|
804
|
-
|
|
805
|
-
def fromChunk[A](chunk: Chunk[A]): Stream[Nothing, A]
|
|
806
|
-
def fromIterable[A](it: Iterable[A]): Stream[Nothing, A]
|
|
807
|
-
def fromIterator[A](it: => Iterator[A]): Stream[Nothing, A]
|
|
808
|
-
def range(from: Int, until: Int): Stream[Nothing, Int]
|
|
809
|
-
def fromRange(range: Range): Stream[Nothing, Int]
|
|
810
|
-
|
|
811
|
-
def repeat[A](a: A): Stream[Nothing, A] // infinite
|
|
812
|
-
def iterate[A](init: A)(f: A => A): Stream[Nothing, A] // infinite: init, f(init), f(f(init)), ...
|
|
813
|
-
def repeatThunk[A](thunk: => A): Stream[Nothing, A] // infinite: thunk() per element
|
|
814
|
-
def unfold[S, A](s: S)(f: S => Option[(A, S)]): Stream[Nothing, A]
|
|
815
|
-
|
|
816
|
-
def eval(f: => Any): Stream[Nothing, Nothing] // side-effect, no output
|
|
817
|
-
def attempt[A](f: => A): Stream[Throwable, A] // captures exceptions
|
|
818
|
-
def attemptEval(f: => Any): Stream[Throwable, Nothing]
|
|
819
|
-
def suspend[E, A](stream: => Stream[E, A]): Stream[E, A]
|
|
820
|
-
def defer(f: => Unit): Stream[Nothing, Nothing] // register finalizer
|
|
821
|
-
|
|
822
|
-
def fromInputStream(is: => InputStream): Stream[IOException, Int] // auto-closes
|
|
823
|
-
def fromInputStreamUnmanaged(is: InputStream): Stream[IOException, Int] // borrowing
|
|
824
|
-
def fromJavaReader(r: => java.io.Reader): Stream[IOException, Char] // auto-closes
|
|
825
|
-
def fromJavaReaderUnmanaged(r: java.io.Reader): Stream[IOException, Char] // borrowing
|
|
826
|
-
def fromReader[E, A](mkReader: => Reader[A]): Stream[E, A]
|
|
827
|
-
|
|
828
|
-
def fromAcquireRelease[R, E, A](acquire: => R, release: R => Unit)(use: R => Stream[E, A]): Stream[E, A]
|
|
829
|
-
def fromResource[R, E, A](resource: Resource[R])(use: R => Stream[E, A]): Stream[E, A]
|
|
830
|
-
def flattenAll[E, A](streams: Stream[E, Stream[E, A]]): Stream[E, A]
|
|
831
|
-
```
|
|
832
|
-
|
|
833
|
-
#### Transformations (return `Stream`)
|
|
834
|
-
|
|
835
|
-
```scala
|
|
836
|
-
abstract class Stream[+E, +A]:
|
|
837
|
-
def map[B](f: A => B): Stream[E, B]
|
|
838
|
-
def flatMap[E2, B](f: A => Stream[E2, B]): Stream[E | E2, B]
|
|
839
|
-
def filter(pred: A => Boolean): Stream[E, A]
|
|
840
|
-
def collect[B](pf: PartialFunction[A, B]): Stream[E, B]
|
|
841
|
-
def scan[S](init: S)(f: (S, A) => S): Stream[E, S]
|
|
842
|
-
def mapAccum[S, B](init: S)(f: (S, A) => (S, B)): Stream[E, B]
|
|
843
|
-
def tapEach(f: A => Unit): Stream[E, A]
|
|
844
|
-
|
|
845
|
-
def take(n: Long): Stream[E, A]
|
|
846
|
-
def drop(n: Long): Stream[E, A]
|
|
847
|
-
def takeWhile(pred: A => Boolean): Stream[E, A]
|
|
848
|
-
|
|
849
|
-
def grouped(n: Int): Stream[E, Chunk[A]]
|
|
850
|
-
def sliding(size: Int, step: Int): Stream[E, Chunk[A]]
|
|
851
|
-
def intersperse[A1 >: A](sep: A1): Stream[E, A1]
|
|
852
|
-
def distinct: Stream[E, A]
|
|
853
|
-
def distinctBy[B](f: A => B): Stream[E, A]
|
|
854
|
-
def zipWithIndex: Stream[E, (A, Long)]
|
|
855
|
-
|
|
856
|
-
def concat[E2, A2](that: Stream[E2, A2]): Stream[E | E2, A | A2] // alias: ++
|
|
857
|
-
def &&[E2, B](that: Stream[E2, B]): Stream[E | E2, (A, B)] // zip with tuple flattening
|
|
858
|
-
def repeated: Stream[E, A]
|
|
859
|
-
|
|
860
|
-
def catchAll[E2, A1](f: E => Stream[E2, A1]): Stream[E2, A | A1]
|
|
861
|
-
def catchDefect[E1, A1](f: PartialFunction[Throwable, Stream[E1, A1]]): Stream[E | E1, A | A1]
|
|
862
|
-
def mapError[E2](f: E => E2): Stream[E2, A]
|
|
863
|
-
def orElse[E2, A1](that: => Stream[E2, A1]): Stream[E2, A | A1] // alias: ||
|
|
864
|
-
|
|
865
|
-
def ensuring(finalizer: => Unit): Stream[E, A]
|
|
866
|
-
def via[B](pipe: Pipeline[A, B]): Stream[E, B]
|
|
867
|
-
|
|
868
|
-
def render: String // human-readable pipeline description
|
|
869
|
-
override def toString: String // alias for render
|
|
870
|
-
```
|
|
871
|
-
|
|
872
|
-
#### Terminal operations (return `Either[E, Z]`)
|
|
873
|
-
|
|
874
|
-
```scala
|
|
875
|
-
abstract class Stream[+E, +A]:
|
|
876
|
-
def run[E2 >: E, Z](sink: Sink[E2, A, Z]): Either[E2, Z]
|
|
877
|
-
def runCollect: Either[E, Chunk[A]]
|
|
878
|
-
def runDrain: Either[E, Unit]
|
|
879
|
-
def runFold[Z](z: Z)(f: (Z, A) => Z): Either[E, Z] // also specialized for Int, Long, Double
|
|
880
|
-
def runForeach(f: A => Unit): Either[E, Unit]
|
|
881
|
-
def foreach(f: A => Unit): Either[E, Unit] // alias
|
|
882
|
-
|
|
883
|
-
def head: Either[E, Option[A]]
|
|
884
|
-
def last: Either[E, Option[A]]
|
|
885
|
-
def count: Either[E, Long]
|
|
886
|
-
def exists(pred: A => Boolean): Either[E, Boolean]
|
|
887
|
-
def forall(pred: A => Boolean): Either[E, Boolean]
|
|
888
|
-
def find(pred: A => Boolean): Either[E, Option[A]]
|
|
889
|
-
|
|
890
|
-
def start(using scope: Scope): scope.$[Reader[A]] // manual pull
|
|
891
|
-
```
|
|
892
|
-
|
|
893
|
-
---
|
|
894
|
-
|
|
895
|
-
### `Sink[+E, -A, +Z]`
|
|
896
|
-
|
|
897
|
-
```scala
|
|
898
|
-
abstract class Sink[+E, -A, +Z]:
|
|
899
|
-
def contramap[A2](g: A2 => A): Sink[E, A2, Z]
|
|
900
|
-
def map[Z2](f: Z => Z2): Sink[E, A, Z2]
|
|
901
|
-
def mapError[E2](f: E => E2): Sink[E2, A, Z]
|
|
902
|
-
|
|
903
|
-
object Sink:
|
|
904
|
-
def collectAll[A]: Sink[Nothing, A, Chunk[A]]
|
|
905
|
-
val drain: Sink[Nothing, Any, Unit]
|
|
906
|
-
val count: Sink[Nothing, Any, Long]
|
|
907
|
-
def foldLeft[A, Z](z: Z)(f: (Z, A) => Z): Sink[Nothing, A, Z]
|
|
908
|
-
def foreach[A](f: A => Unit): Sink[Nothing, A, Unit]
|
|
909
|
-
def head[A]: Sink[Nothing, A, Option[A]]
|
|
910
|
-
def last[A]: Sink[Nothing, A, Option[A]]
|
|
911
|
-
def take[A](n: Int): Sink[Nothing, A, Chunk[A]]
|
|
912
|
-
def exists[A](pred: A => Boolean): Sink[Nothing, A, Boolean]
|
|
913
|
-
def forall[A](pred: A => Boolean): Sink[Nothing, A, Boolean]
|
|
914
|
-
def find[A](pred: A => Boolean): Sink[Nothing, A, Option[A]]
|
|
915
|
-
val sumInt: Sink[Nothing, Int, Long]
|
|
916
|
-
val sumLong: Sink[Nothing, Long, Long]
|
|
917
|
-
val sumFloat: Sink[Nothing, Float, Double]
|
|
918
|
-
val sumDouble: Sink[Nothing, Double, Double]
|
|
919
|
-
def fromOutputStream(os: OutputStream): Sink[Nothing, Byte, Unit]
|
|
920
|
-
def fromJavaWriter(w: java.io.Writer): Sink[Nothing, Char, Unit]
|
|
921
|
-
def fail[E](e: E): Sink[E, Any, Nothing]
|
|
922
|
-
def create[E, A, Z](f: Reader[A] => Z): Sink[E, A, Z]
|
|
923
|
-
```
|
|
924
|
-
|
|
925
|
-
---
|
|
926
|
-
|
|
927
|
-
### `Pipeline[-In, +Out]`
|
|
928
|
-
|
|
929
|
-
```scala
|
|
930
|
-
abstract class Pipeline[-In, +Out]:
|
|
931
|
-
def andThen[C](that: Pipeline[Out, C]): Pipeline[In, C]
|
|
932
|
-
def andThenSink[E, Z](sink: Sink[E, Out, Z]): Sink[E, In, Z]
|
|
933
|
-
def applyToStream[E](stream: Stream[E, In]): Stream[E, Out]
|
|
934
|
-
def applyToSink[E, Z](sink: Sink[E, Out, Z]): Sink[E, In, Z]
|
|
935
|
-
|
|
936
|
-
object Pipeline:
|
|
937
|
-
def map[A, B](f: A => B): Pipeline[A, B]
|
|
938
|
-
def filter[A](pred: A => Boolean): Pipeline[A, A]
|
|
939
|
-
def collect[A, B](pf: PartialFunction[A, B]): Pipeline[A, B]
|
|
940
|
-
def take[A](n: Long): Pipeline[A, A]
|
|
941
|
-
def drop[A](n: Long): Pipeline[A, A]
|
|
942
|
-
def identity[A]: Pipeline[A, A]
|
|
943
|
-
```
|
|
944
|
-
|
|
945
|
-
---
|
|
946
|
-
|
|
947
|
-
### `NioStreams` (JVM only)
|
|
948
|
-
|
|
949
|
-
```scala
|
|
950
|
-
object NioStreams:
|
|
951
|
-
def fromByteBuffer(buf: ByteBuffer): Stream[Nothing, Byte]
|
|
952
|
-
def fromByteBufferInt(buf: ByteBuffer): Stream[Nothing, Int]
|
|
953
|
-
def fromByteBufferLong(buf: ByteBuffer): Stream[Nothing, Long]
|
|
954
|
-
def fromByteBufferFloat(buf: ByteBuffer): Stream[Nothing, Float]
|
|
955
|
-
def fromByteBufferDouble(buf: ByteBuffer): Stream[Nothing, Double]
|
|
956
|
-
def fromChannel(ch: => ReadableByteChannel, bufSize: Int = 8192): Stream[IOException, Byte] // auto-closes
|
|
957
|
-
def fromChannelUnmanaged(ch: ReadableByteChannel, bufSize: Int = 8192): Stream[IOException, Byte] // borrowing
|
|
958
|
-
```
|
|
959
|
-
|
|
960
|
-
### `NioSinks` (JVM only)
|
|
961
|
-
|
|
962
|
-
```scala
|
|
963
|
-
object NioSinks:
|
|
964
|
-
def fromByteBuffer(buf: ByteBuffer): Sink[Nothing, Byte, Unit]
|
|
965
|
-
def fromByteBufferInt(buf: ByteBuffer): Sink[Nothing, Int, Unit]
|
|
966
|
-
def fromByteBufferLong(buf: ByteBuffer): Sink[Nothing, Long, Unit]
|
|
967
|
-
def fromByteBufferFloat(buf: ByteBuffer): Sink[Nothing, Float, Unit]
|
|
968
|
-
def fromByteBufferDouble(buf: ByteBuffer): Sink[Nothing, Double, Unit]
|
|
969
|
-
def fromChannel(ch: WritableByteChannel, bufSize: Int = 8192): Sink[Nothing, Byte, Unit]
|
|
970
|
-
```
|
|
971
|
-
|
|
972
|
-
---
|
|
973
|
-
|
|
974
|
-
## Practical guidance
|
|
975
|
-
|
|
976
|
-
- **Start with `Stream` constructors and terminal operations.** You can get very far with `Stream.range`, `Stream.fromIterable`, `.map`, `.filter`, and `.runCollect`.
|
|
977
|
-
- **Use `Either` pattern matching** to handle the result: `Right(value)` for success, `Left(error)` for typed failures.
|
|
978
|
-
- **Prefer `Stream.fromAcquireRelease`** when wrapping resources (files, connections, etc.) over manual try/finally. It guarantees cleanup even on early termination via `take`, `head`, or error.
|
|
979
|
-
- **Use the auto-closing I/O constructors** (`fromInputStream`, `fromJavaReader`, `NioStreams.fromChannel`) by default. Only use the `Unmanaged` variants when you need to borrow a resource whose lifetime is managed elsewhere.
|
|
980
|
-
- **Use `Pipeline`** when you have a transformation you want to reuse across multiple streams or apply to sinks. Pipelines have two type parameters: `Pipeline[In, Out]`.
|
|
981
|
-
- **Use `&&` for zipping** instead of manual `zip` calls. Tuples flatten automatically for three or more streams: `a && b && c` produces `(A, B, C)` not `((A, B), C)`.
|
|
982
|
-
- **Leverage primitive specialization** for numeric workloads. Streams of `Int`, `Long`, `Float`, and `Double` avoid boxing automatically; use `Sink.sumInt`, `Sink.sumLong`, `runFold(0)(_ + _)`, etc. for zero-allocation folds.
|
|
983
|
-
- **Use `scan` for running accumulators**, `grouped` for batching, and `sliding` for windowed computations.
|
|
984
|
-
- **Use `render`/`toString`** to inspect pipeline structure during debugging -- it shows each transformation stage without executing the stream.
|
|
985
|
-
- **Use `Sink.create`** as an escape hatch when none of the built-in sinks fit. It gives you direct access to the `Reader` for custom consumption logic.
|
|
986
|
-
- **Use `NioStreams` / `NioSinks`** on the JVM for efficient NIO buffer and channel integration.
|
|
987
|
-
- **Avoid holding references** to a `Reader` obtained via `start` outside its `Scope`. The scope guarantees cleanup; escaping the reader defeats that guarantee.
|
|
988
|
-
- **`suspend`** is your friend for recursive or self-referential stream definitions, preventing stack overflow during construction.
|
|
989
|
-
- **Typed errors vs. defects**: use `Stream.fail` for expected domain errors and `Stream.die` for programmer errors. Use `catchAll` for the former, `catchDefect` for the latter.
|