@zio.dev/zio-blocks 0.0.33 → 0.0.55

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (215) hide show
  1. package/adr/2026-07-18-data-migration.md +123 -0
  2. package/guides/async-getting-started.md +687 -0
  3. package/guides/compile-time-resource-safety-with-scope.md +21 -16
  4. package/guides/getting-started-with-mux.md +1395 -0
  5. package/guides/query-dsl-extending.md +161 -102
  6. package/guides/query-dsl-fluent-builder.md +217 -157
  7. package/guides/query-dsl-reified-optics.md +12 -10
  8. package/guides/query-dsl-sql.md +640 -165
  9. package/guides/sql-checked-interpolation.md +173 -0
  10. package/guides/sql-transactions.md +286 -0
  11. package/guides/telemetry-guide.md +1130 -0
  12. package/guides/zio-schema-migration.md +29 -22
  13. package/index.md +248 -389
  14. package/package.json +1 -1
  15. package/plans/config-follow-up-prs.md +188 -0
  16. package/plans/config-pr-assessment-roadmap.md +310 -0
  17. package/reference/MuxDataFlow.jsx +250 -0
  18. package/reference/async.md +1499 -0
  19. package/reference/chunk.md +3533 -308
  20. package/reference/codegen/case-class.md +436 -0
  21. package/reference/codegen/emitter-config.md +383 -0
  22. package/reference/codegen/examples.md +664 -0
  23. package/reference/codegen/field.md +316 -0
  24. package/reference/codegen/index.md +317 -0
  25. package/reference/codegen/scala-emitter.md +392 -0
  26. package/reference/codegen/scala-file.md +276 -0
  27. package/reference/codegen/sealed-trait.md +408 -0
  28. package/reference/codegen/type-definition.md +340 -0
  29. package/reference/codegen/type-ref.md +201 -0
  30. package/reference/combinators.md +347 -117
  31. package/reference/config/config-decoder.md +460 -0
  32. package/reference/config/config-source.md +489 -0
  33. package/reference/config/errors.md +278 -0
  34. package/reference/config/flags.md +369 -0
  35. package/reference/config/formats.md +314 -0
  36. package/reference/config/index.md +304 -0
  37. package/reference/config/rollout.md +336 -0
  38. package/reference/context.md +9 -52
  39. package/reference/data-migration.md +269 -0
  40. package/reference/datastar/attributes.md +302 -0
  41. package/reference/datastar/events.md +234 -0
  42. package/reference/datastar/index.md +256 -0
  43. package/reference/datastar/signals.md +230 -0
  44. package/reference/datastar/sse.md +295 -0
  45. package/reference/datastar.md +346 -0
  46. package/reference/docs.md +1461 -345
  47. package/reference/endpoint/auth-type.md +146 -0
  48. package/reference/endpoint/bulk-creation.md +96 -0
  49. package/reference/endpoint/endpoint.md +297 -0
  50. package/reference/endpoint/http-codec.md +249 -0
  51. package/reference/endpoint/index.md +745 -0
  52. package/reference/endpoint/path-codec.md +225 -0
  53. package/reference/endpoint/route-pattern.md +194 -0
  54. package/reference/endpoint/route-tree.md +111 -0
  55. package/reference/endpoint/segment-codec.md +199 -0
  56. package/reference/html.md +1424 -0
  57. package/reference/htmx/attribute-values.md +359 -0
  58. package/reference/htmx/hx-encoding.md +111 -0
  59. package/reference/htmx/hx-params.md +204 -0
  60. package/reference/htmx/hx-swap.md +276 -0
  61. package/reference/htmx/hx-sync.md +251 -0
  62. package/reference/htmx/hx-target.md +314 -0
  63. package/reference/htmx/hx-trigger.md +457 -0
  64. package/reference/htmx/hx-url-update.md +239 -0
  65. package/reference/htmx/index.md +807 -0
  66. package/reference/htmx/response-headers.md +240 -0
  67. package/reference/http-model/headers.md +735 -0
  68. package/reference/http-model/index.md +49 -0
  69. package/reference/http-model/model.md +1517 -0
  70. package/reference/http-model/schema-codecs.md +522 -0
  71. package/reference/http-model/schema.md +750 -0
  72. package/reference/http-model/server-sent-event.md +341 -0
  73. package/reference/jwt.md +195 -0
  74. package/reference/maybe.md +943 -0
  75. package/reference/media-type.md +2 -2
  76. package/reference/mux.md +254 -0
  77. package/reference/mux.mdx +828 -0
  78. package/reference/openapi.md +1351 -0
  79. package/reference/projection.md +654 -0
  80. package/reference/resource-management/defer-handle.md +1 -1
  81. package/reference/resource-management/resource.md +31 -98
  82. package/reference/resource-management/scope.md +28 -220
  83. package/reference/resource-management/wire.md +5 -55
  84. package/reference/ringbuffer/MpmcDiagram.jsx +717 -0
  85. package/reference/ringbuffer/MpscDiagram.jsx +618 -0
  86. package/reference/ringbuffer/SpmcDiagram.jsx +680 -0
  87. package/reference/ringbuffer/SpscDiagram.jsx +677 -0
  88. package/reference/ringbuffer/advanced.mdx +109 -0
  89. package/reference/ringbuffer/index.mdx +145 -0
  90. package/reference/ringbuffer/mpmc.mdx +185 -0
  91. package/reference/ringbuffer/mpsc.mdx +164 -0
  92. package/reference/ringbuffer/spmc.mdx +108 -0
  93. package/reference/ringbuffer/spsc.mdx +416 -0
  94. package/reference/{allows.md → schema/allows.md} +4 -100
  95. package/reference/{binding-resolver.md → schema/binding-resolver.md} +1 -1
  96. package/reference/{binding.md → schema/binding.md} +3 -4
  97. package/reference/schema/built-in-codecs/avro.md +451 -0
  98. package/reference/schema/built-in-codecs/bson.md +510 -0
  99. package/reference/schema/built-in-codecs/csv.md +564 -0
  100. package/reference/schema/built-in-codecs/index.md +77 -0
  101. package/reference/schema/built-in-codecs/json/index.md +295 -0
  102. package/reference/schema/built-in-codecs/json/json-config.md +217 -0
  103. package/reference/{json-patch.md → schema/built-in-codecs/json/json-patch.md} +5 -5
  104. package/reference/{json-schema.md → schema/built-in-codecs/json/json-schema.md} +14 -47
  105. package/reference/schema/built-in-codecs/json/json-selection.md +322 -0
  106. package/reference/{json.md → schema/built-in-codecs/json/json.md} +32 -64
  107. package/reference/schema/built-in-codecs/messagepack.md +508 -0
  108. package/reference/schema/built-in-codecs/thrift.md +433 -0
  109. package/reference/schema/built-in-codecs/toon.md +1078 -0
  110. package/reference/{xml.md → schema/built-in-codecs/xml.md} +13 -9
  111. package/reference/schema/built-in-codecs/yaml.md +552 -0
  112. package/reference/{codec.md → schema/codec.md} +11 -11
  113. package/reference/{dynamic-optic.md → schema/dynamic-optic.md} +196 -5
  114. package/reference/{dynamic-schema.md → schema/dynamic-schema.md} +8 -8
  115. package/reference/schema/format.md +92 -0
  116. package/reference/schema/index.md +52 -0
  117. package/reference/schema/migration.md +297 -0
  118. package/reference/{modifier.md → schema/modifier.md} +58 -7
  119. package/reference/{optics.md → schema/optics.md} +2 -2
  120. package/reference/{patch.md → schema/patch.md} +1 -1
  121. package/{path-interpolator.md → reference/schema/path-interpolator.md} +167 -72
  122. package/reference/schema/reflect-transformer.md +140 -0
  123. package/reference/{schema-evolution → schema/schema-evolution}/as.md +8 -8
  124. package/reference/{schema-evolution → schema/schema-evolution}/index.md +2 -2
  125. package/reference/{schema-evolution → schema/schema-evolution}/into.md +8 -8
  126. package/reference/{schema-expr.md → schema/schema-expr.md} +110 -175
  127. package/reference/schema/schema-search.md +263 -0
  128. package/reference/{schema.md → schema/schema.md} +22 -2
  129. package/reference/{structural-types.md → schema/structural-types.md} +1 -1
  130. package/reference/{type-class-derivation.md → schema/type-class-derivation.md} +63 -1
  131. package/reference/smithy.md +1032 -0
  132. package/reference/sql/db-codec-deriver.md +71 -0
  133. package/reference/sql/db-codec.md +687 -0
  134. package/reference/sql/db-con.md +271 -0
  135. package/reference/sql/db-connection.md +153 -0
  136. package/reference/sql/db-param-writer.md +77 -0
  137. package/reference/sql/db-param.md +66 -0
  138. package/reference/sql/db-result-reader.md +148 -0
  139. package/reference/sql/db-tx.md +114 -0
  140. package/reference/sql/db-value.md +41 -0
  141. package/reference/sql/ddl.md +85 -0
  142. package/reference/sql/frag.md +288 -0
  143. package/reference/sql/index.md +341 -0
  144. package/reference/sql/repo.md +600 -0
  145. package/reference/sql/sql-dialect.md +73 -0
  146. package/reference/sql/sql-logger.md +62 -0
  147. package/reference/sql/sql-name-mapper.md +70 -0
  148. package/reference/sql/table-metadata.md +134 -0
  149. package/reference/sql/table.md +448 -0
  150. package/reference/sql/transactor-zio.md +399 -0
  151. package/reference/sql/transactor.md +363 -0
  152. package/reference/sql-zio.md +112 -0
  153. package/reference/streams/core/index.md +32 -0
  154. package/reference/streams/core/pipeline.md +854 -0
  155. package/reference/streams/core/sink.md +1404 -0
  156. package/reference/streams/core/stream.md +3236 -0
  157. package/reference/streams/execution-and-compatibility/async-execution.md +822 -0
  158. package/reference/streams/execution-and-compatibility/index.md +35 -0
  159. package/reference/streams/execution-and-compatibility/platform-differences.md +297 -0
  160. package/reference/streams/execution-and-compatibility/scala-2-compatibility.md +88 -0
  161. package/reference/streams/execution-and-compatibility/zero-boxing.md +393 -0
  162. package/reference/streams/index.md +726 -0
  163. package/reference/streams/primitives/index.md +30 -0
  164. package/reference/streams/primitives/reader.md +1992 -0
  165. package/reference/streams/primitives/writer.md +1201 -0
  166. package/reference/telemetry/common/any-value.md +90 -0
  167. package/reference/telemetry/common/attribute-key.md +87 -0
  168. package/reference/telemetry/common/attributes.md +118 -0
  169. package/reference/telemetry/common/index.md +39 -0
  170. package/reference/telemetry/common/instrumentation-scope.md +24 -0
  171. package/reference/telemetry/common/resource.md +34 -0
  172. package/reference/telemetry/index.md +311 -0
  173. package/reference/telemetry/logging/index.md +197 -0
  174. package/reference/telemetry/logging/log-enrichment.md +72 -0
  175. package/reference/telemetry/logging/log-formatter.md +100 -0
  176. package/reference/telemetry/logging/log-record-processor.md +56 -0
  177. package/reference/telemetry/logging/log-record.md +44 -0
  178. package/reference/telemetry/logging/log-writer.md +64 -0
  179. package/reference/telemetry/logging/logger-provider.md +142 -0
  180. package/reference/telemetry/logging/logger.md +83 -0
  181. package/reference/telemetry/logging/severity.md +62 -0
  182. package/reference/telemetry/metrics/index.md +150 -0
  183. package/reference/telemetry/metrics/instruments.md +183 -0
  184. package/reference/telemetry/metrics/labeled-instruments.md +74 -0
  185. package/reference/telemetry/metrics/meter-provider.md +76 -0
  186. package/reference/telemetry/metrics/meter.md +98 -0
  187. package/reference/telemetry/metrics/metric-data.md +57 -0
  188. package/reference/telemetry/otel/custom-exporter.md +216 -0
  189. package/reference/telemetry/otel/index.md +212 -0
  190. package/reference/telemetry/tracing/index.md +155 -0
  191. package/reference/telemetry/tracing/sampler.md +89 -0
  192. package/reference/telemetry/tracing/span-builder.md +57 -0
  193. package/reference/telemetry/tracing/span-context.md +39 -0
  194. package/reference/telemetry/tracing/span-data.md +32 -0
  195. package/reference/telemetry/tracing/span-kind.md +55 -0
  196. package/reference/telemetry/tracing/span-processor.md +53 -0
  197. package/reference/telemetry/tracing/span-status.md +47 -0
  198. package/reference/telemetry/tracing/span.md +117 -0
  199. package/reference/telemetry/tracing/tracer-provider.md +91 -0
  200. package/reference/telemetry/tracing/tracer.md +52 -0
  201. package/reference/typeid.md +5 -83
  202. package/sidebars.js +376 -43
  203. package/undocumented-report.md +528 -270
  204. package/reference/formats.md +0 -694
  205. package/reference/http-model.md +0 -1716
  206. package/reference/streams.md +0 -989
  207. package/ringbuffer.md +0 -249
  208. /package/reference/{json-differ.md → schema/built-in-codecs/json/json-differ.md} +0 -0
  209. /package/reference/{dynamic-value.md → schema/dynamic-value.md} +0 -0
  210. /package/reference/{lazy.md → schema/lazy.md} +0 -0
  211. /package/reference/{reflect.md → schema/reflect.md} +0 -0
  212. /package/reference/{registers.md → schema/registers.md} +0 -0
  213. /package/reference/{schema-error.md → schema/schema-error.md} +0 -0
  214. /package/reference/{syntax.md → schema/syntax.md} +0 -0
  215. /package/reference/{validation.md → schema/validation.md} +0 -0
@@ -0,0 +1,3236 @@
1
+ ---
2
+ id: stream
3
+ title: "Stream"
4
+ sidebar_label: "Stream"
5
+ description: "The Stream data type: construction, transformation, resource safety, and the cross-platform async and JVM-only blocking terminal families."
6
+ keywords:
7
+ - "Pull-Based Streams"
8
+ - "Stateful Transformations"
9
+ - "Async Operators"
10
+ - "Blocking Terminals"
11
+ - "Stream"
12
+ ---
13
+
14
+ import Tabs from '@theme/Tabs';
15
+ import TabItem from '@theme/TabItem';
16
+
17
+ `Stream[+E, +A]` is a **lazy, pull-based, typed-error stream** of elements that may fail with an error of type `E`. Nothing executes until a terminal operation is driven. Cross-platform asynchronous terminals return `Async[Either[E, Z]]`; the JVM also provides the existing blocking terminal family returning `Either[E, Z]`. Typed errors surface as `Left(e)`, while defects and cleanup failures fail the outer `Async` or propagate from a JVM blocking terminal:
18
+
19
+ ```scala
20
+ abstract class Stream[+E, +A] {
21
+ def runAsync[ES, E3, Z](sink: Sink[ES, A, Z])(implicit
22
+ errorConcat: Concat.WithOut[E, ES, E3]
23
+ ): Async[Either[E3, Z]]
24
+ def runCollectAsync: Async[Either[E, Chunk[A]]]
25
+
26
+ // JVM only
27
+ def run[ES, E3, Z](sink: Sink[ES, A, Z])(implicit
28
+ errorConcat: Concat.WithOut[E, ES, E3]
29
+ ): Either[E3, Z]
30
+ }
31
+ ```
32
+
33
+ `Stream` is purely functional, referentially transparent, and resource-safe:
34
+ - **Lazy**: descriptions of pipelines, not eager computations
35
+ - **Nonblocking across platforms**: `*Async` terminals drive synchronous or asynchronous readers without blocking JavaScript
36
+ - **JVM-compatible**: plain blocking terminals remain available on the JVM
37
+ - **Pull-based**: execution is driven from the sink backward through the pipeline
38
+ - **Typed errors**: distinguish recoverable errors (`E`) from untyped defects (`Throwable`)
39
+ - **Resource-safe**: RAII semantics ensure resources are released in all cases
40
+
41
+ ### Asynchronous Source Constructors
42
+
43
+ The companion constructors whose names end in `Async` — `attemptAsync`, `attemptEvalAsync`, `deferAsync`, `evalAsync`, `fromAcquireReleaseAsync`, `fromIteratorAsync`, `fromReaderAsync`, and `unfoldAsync` — defer their `Async` thunk until the first reader operation is driven, and `Stream.unwrap` flattens an `Async[Stream[E, A]]` so that ordinary operators such as `flatMap`, `catchAll`, and `flatMapPar` compose with asynchronously produced streams. [Async Source Constructors](../execution-and-compatibility/async-execution.md#async-source-constructors) documents each of them, along with the laziness and error conventions they share; this page does not repeat them.
44
+
45
+ ### Source Compatibility
46
+
47
+ `Reader` is an ordinary `abstract class`, not a sealed one, but every reader the library hands you is a `Reader.SyncReader[A]` or a `Reader.AsyncReader[A]`, so code that implements or accepts a reader must choose one kind or match both with a fallback case. Custom sinks cannot be written by subclassing `Sink`, whose two abstract drains are `private[streams]`; use `Sink.createAsync`, `Sink.createBoth`, or the JVM-only `Sink.create`. Plain terminals, `start`, `AsyncReader#toSync`, and `Sink.create` are JVM-only, so shared sources should migrate to `run*Async`, `startAsync`/`useReaderAsync`, and `createAsync`.
48
+
49
+ Async constructor callbacks remain lazy until the first drive, and managed/unmanaged names encode ownership. Do not compensate by eagerly opening a resource before constructing the stream. Cancellation closes an acquired reader and awaits its finalizer; `startAsync` is the exception because it explicitly transfers that responsibility to its caller.
50
+
51
+ ## Motivation
52
+
53
+ Traditional eager sequences (like Scala `List`) fall short in **three critical dimensions**. Here's what `Stream[E, A]` solves for each:
54
+
55
+ **1. Efficiency — Wasteful Computation**
56
+
57
+ With eager evaluation, the entire dataset is processed upfront, regardless of how many elements you actually need. This example shows how much work is wasted:
58
+
59
+ ```scala
60
+ // With Scala List (eager evaluation)
61
+ val data = (1 to 1_000_000).toList
62
+ val result = data
63
+ .map(_ * 2) // eagerly: 1M multiplications
64
+ .filter(_ > 10) // eagerly: 1M comparisons
65
+ .take(10) // finally: keep only 10
66
+ // ❌ Wasted work: computed and discarded 999,990 elements!
67
+ ```
68
+
69
+ The problem: `List` eagerly applies `.map` and `.filter` to all 1 million elements, even though only the first 10 passing elements matter. In data processing pipelines (parsing CSV files, filtering logs, transforming sensor streams), this is enormously wasteful.
70
+
71
+ With `Stream[E, A]`, the architecture is **inverted**: the **sink (consumer) pulls** from the stream. If the sink asks for only 10 elements, only ~20 calculations occur (enough to find 10 valid results after filtering):
72
+
73
+ ```scala
74
+ import zio.blocks.streams.*
75
+
76
+ // With Stream (lazy, pull-based evaluation)
77
+ val result: Either[Nothing, zio.blocks.chunk.Chunk[Int]] =
78
+ Stream.fromRange((1 to 100))
79
+ .map(_ * 2)
80
+ .filter(_ > 10)
81
+ .run(Sink.take(10))
82
+ // ✓ Computation stops after 10 valid elements are produced
83
+ // Only necessary work: ~20 multiplications, ~20 comparisons
84
+ ```
85
+
86
+ This **short-circuiting** behavior is automatic and requires no special syntax.
87
+
88
+ **2. Resource Management — Error-Prone Cleanup**
89
+
90
+ When you open resources (file handles, network connections, database cursors), you must release them in **all** code paths—success, error, and even mid-stream cancellation. With eager sequences, this burden falls on the caller:
91
+
92
+ ```
93
+ // ❌ With traditional Scala (manual resource management, uses var for mutable state)
94
+ import java.io.*
95
+
96
+ var file: BufferedReader = null
97
+ try {
98
+ file = new BufferedReader(new FileReader("build.sbt"))
99
+ var count = 0L
100
+ var char = file.read()
101
+ while (char != -1) {
102
+ if (!Character.isWhitespace(char)) {
103
+ count += 1
104
+ }
105
+ char = file.read()
106
+ }
107
+ count
108
+ } catch {
109
+ case e: IOException =>
110
+ throw e
111
+ } finally {
112
+ if (file != null) file.close() // ✓ Manual cleanup in finally
113
+ }
114
+ // ❌ Problem: You must remember the finally block
115
+ // ❌ Problem: If an exception occurs in the loop, cleanup must still run (easy to forget!)
116
+ // ❌ Problem: Scale to 10 resources? 50 resources? Manually nesting becomes error-prone
117
+ ```
118
+
119
+ With `Stream[E, A]`, resource cleanup is **automatic, composable, and guaranteed**—even on error or if the sink cancels early:
120
+
121
+ ```scala
122
+ import zio.blocks.streams.*
123
+ import java.io.*
124
+
125
+ // With Stream (resource-safe RAII)
126
+ // Open a file and count non-whitespace characters
127
+ val charCount: Either[IOException, Long] =
128
+ Stream
129
+ .fromJavaReader(new FileReader("build.sbt")) // lazily acquires file handle
130
+ .filter(!_.isWhitespace) // process only non-whitespace
131
+ .count // count all matching characters
132
+ // ✓ File automatically closes in finally block (success or error)
133
+ // ✓ If FileReader throws, or filter throws, or count throws—cleanup still runs
134
+ // ✓ No manual try/finally needed; no resource leak risk
135
+ // ✓ Multiple resources (files, connections, etc.) compose naturally
136
+ ```
137
+
138
+ The key difference: `Stream` releases resources via **RAII** (Resource Acquisition Is Initialization) — the resource's lifetime is bound to the compiled stream's `close()` method, which the terminal operation (`run`) always calls in a `finally` block.
139
+
140
+ **3. Error Handling — Untyped Errors**
141
+
142
+ Traditional error handling conflates two categories: recoverable **domain errors** (e.g., parsing failed, validation failed) and fatal **defects** (e.g., `OutOfMemoryError`, `NullPointerException`). This makes it hard to write correct error recovery code:
143
+
144
+ ```scala
145
+ import scala.util.Try
146
+
147
+ // With Try/catch (untyped errors)
148
+ case class ParseError(msg: String)
149
+
150
+ def parseLines(lines: List[String]): Try[List[Int]] = Try {
151
+ lines.map { line =>
152
+ line.toInt // throws NumberFormatException (defect, not domain error!)
153
+ }
154
+ }
155
+
156
+ val result = parseLines(List("1", "abc", "3"))
157
+ result match {
158
+ case util.Success(nums) => println(s"Parsed: $nums")
159
+ case util.Failure(e) =>
160
+ // ❌ Can't tell if 'e' is a parse error or a JVM defect
161
+ // ❌ Must handle *all* exceptions the same way
162
+ // ❌ Domain logic mixed with system-level exception handling
163
+ println(s"Error: $e")
164
+ }
165
+ ```
166
+
167
+ With `Stream[E, A]`, typed errors (`E`) are distinct from untyped defects (`Throwable`), enabling proper error recovery:
168
+
169
+ ```scala
170
+ import zio.blocks.streams.*
171
+
172
+ case class ParseError(msg: String)
173
+
174
+ // With Stream (typed errors)
175
+ val result: Either[ParseError, zio.blocks.chunk.Chunk[Int]] =
176
+ Stream
177
+ .fromIterable(List("1", "abc", "3"))
178
+ .flatMap { line =>
179
+ try {
180
+ Stream.succeed(line.toInt) // success path
181
+ } catch {
182
+ case _: NumberFormatException =>
183
+ Stream.fail(ParseError(s"Not a number: $line")) // typed error
184
+ }
185
+ }
186
+ .runCollect
187
+
188
+ result match {
189
+ case Left(parseError) =>
190
+ // ✓ This branch is *only* for domain errors we chose to surface
191
+ println(s"Parse error: ${parseError.msg}")
192
+ case Right(nums) =>
193
+ // ✓ Untyped defects (OutOfMemoryError, etc.) propagate as exceptions
194
+ // ✓ Clear separation: Either[E, Z] is for recovery, uncaught exceptions are fatal
195
+ println(s"Parsed: ${nums}")
196
+ }
197
+ ```
198
+
199
+ The key distinction: `Either[ParseError, Z]` means domain errors are *recoverable* via `Left`; any uncaught `Throwable` defect propagates as an exception, which is correct—you cannot recover from running out of memory, only from bad input.
200
+
201
+ ## Construction
202
+
203
+ Streams can be created from constants, collections, resources, and pull-based sources:
204
+
205
+ ### Constant Streams
206
+
207
+ The simplest streams are single-element or empty streams.
208
+
209
+ #### `Stream.empty`
210
+
211
+ An empty stream that emits no elements and succeeds immediately:
212
+
213
+ ```scala
214
+ object Stream {
215
+ val empty: Stream[Nothing, Nothing]
216
+ }
217
+ ```
218
+
219
+ The empty stream is useful as a base case in recursive stream builders or as a neutral element when concatenating:
220
+
221
+ ```scala
222
+ import zio.blocks.streams.*
223
+
224
+ val emptyStream = Stream.empty
225
+ // emptyStream: Stream[Nothing, Nothing] = Stream.empty
226
+ val result = emptyStream.runCollect
227
+ // result: Either[Nothing, Chunk[Nothing]] = Right(IndexedSeq())
228
+ // emptyStream contains no elements
229
+ ```
230
+
231
+ #### `Stream.succeed[A]`
232
+
233
+ Wraps a single value of any type. Specialized overloads avoid boxing for primitives:
234
+
235
+ ```scala
236
+ object Stream {
237
+ def succeed[A](a: A): Stream[Nothing, A]
238
+ def succeed(a: Int): Stream[Nothing, Int]
239
+ def succeed(a: Long): Stream[Nothing, Long]
240
+ def succeed(a: Double): Stream[Nothing, Double]
241
+ // ... and Byte, Short, Char, Float, Boolean variants
242
+ }
243
+ ```
244
+
245
+ When you call `Stream.succeed(value)`, the stream emits exactly one element and completes successfully. This is useful for wrapping a computed value into the stream abstraction:
246
+
247
+ ```scala
248
+ import zio.blocks.streams.*
249
+
250
+ val singleElement = Stream.succeed(42)
251
+ // singleElement: Stream[Nothing, Int] = Stream.succeed(...)
252
+ val result = singleElement.runCollect
253
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(42))
254
+ ```
255
+
256
+ The `Byte` overload remains a byte stream: `Stream.succeed(1.toByte)` has type `Stream[Nothing, Byte]`, uses the `Byte` representation lane, and compiles through `Reader.singleByte` rather than widening the element type to `Int`.
257
+
258
+ #### `Stream.fail[E]`
259
+
260
+ Creates a stream that fails immediately with a typed error:
261
+
262
+ ```scala
263
+ object Stream {
264
+ def fail[E](error: E): Stream[E, Nothing]
265
+ }
266
+ ```
267
+
268
+ Use `fail` when you need to short-circuit a stream with a known error:
269
+
270
+ ```scala
271
+ import zio.blocks.streams.*
272
+
273
+ sealed trait ApiError
274
+ case class NotFound(id: String) extends ApiError
275
+
276
+ val failedStream = Stream.fail(NotFound("user-123"))
277
+ // failedStream: Stream[NotFound, Nothing] = Stream.fail(...)
278
+ val result = failedStream.runDrain
279
+ // result: Either[NotFound, Unit] = Left(NotFound("user-123"))
280
+ // result is Left(NotFound("user-123"))
281
+ ```
282
+
283
+ #### `Stream.die`
284
+
285
+ Throws an untyped defect (exception) immediately:
286
+
287
+ ```scala
288
+ object Stream {
289
+ def die(t: Throwable): Stream[Nothing, Nothing]
290
+ }
291
+ ```
292
+
293
+ Use `die` for truly exceptional, unrecoverable conditions that should not be caught as typed errors:
294
+
295
+ ```scala
296
+ import zio.blocks.streams.*
297
+
298
+ val dieStream = Stream.die(new Exception("System failure"))
299
+ // dieStream: Stream[Nothing, Nothing] = Stream.die(...)
300
+ ```
301
+
302
+ ### From Collections
303
+
304
+ Streams can be created from existing collections and iterables, making it easy to convert `List`, `Array`, [`Chunk`](../../chunk.md), or custom iterables into lazy streams:
305
+
306
+ #### `Stream.apply[A]`
307
+
308
+ Wraps a variable number of arguments into a stream:
309
+
310
+ ```scala
311
+ object Stream {
312
+ def apply[A](as: A*)(implicit jt: JvmType.Infer[A]): Stream[Nothing, A]
313
+ }
314
+ ```
315
+
316
+ This is the most natural way to lift a list of values:
317
+
318
+ ```scala
319
+ import zio.blocks.streams.*
320
+
321
+ val numbers = Stream(1, 2, 3, 4, 5)
322
+ // numbers: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5)
323
+ val result = numbers.runCollect
324
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 3, 4, 5))
325
+ ```
326
+
327
+ #### `Stream.fromChunk[A]`
328
+
329
+ Converts a `Chunk` into a stream. Chunks are immutable, indexed sequences optimized for high-performance operations:
330
+
331
+ ```scala
332
+ object Stream {
333
+ def fromChunk[A](chunk: Chunk[A])(implicit jt: JvmType.Infer[A]): Stream[Nothing, A]
334
+ }
335
+ ```
336
+
337
+ Use this when you already have a `Chunk`:
338
+
339
+ ```scala
340
+ import zio.blocks.streams.*
341
+ import zio.blocks.chunk.Chunk
342
+
343
+ val chunk = Chunk(10, 20, 30)
344
+ // chunk: Chunk[Int] = IndexedSeq(10, 20, 30)
345
+ val stream = Stream.fromChunk(chunk)
346
+ // stream: Stream[Nothing, Int] = Stream.fromChunk(...)
347
+ val result = stream.runCollect
348
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(10, 20, 30))
349
+ ```
350
+
351
+ #### `Stream.fromIterable[A]`
352
+
353
+ Converts any `Iterable[A]` (List, Set, Vector, etc.) into a stream:
354
+
355
+ ```scala
356
+ object Stream {
357
+ def fromIterable[A](it: Iterable[A])(implicit jtA: JvmType.Infer[A]): Stream[Nothing, A]
358
+ }
359
+ ```
360
+
361
+ This is useful when integrating with legacy Scala collections:
362
+
363
+ ```scala
364
+ import zio.blocks.streams.*
365
+
366
+ val list = List("a", "b", "c")
367
+ // list: List[String] = List("a", "b", "c")
368
+ val stream = Stream.fromIterable(list)
369
+ // stream: Stream[Nothing, String] = Stream.fromIterable(...)
370
+ val result = stream.runCollect
371
+ // result: Either[Nothing, Chunk[String]] = Right(IndexedSeq("a", "b", "c"))
372
+ ```
373
+
374
+ #### `Stream.fromIterator[A]`
375
+
376
+ Converts an `Iterator[A]` into a stream. The iterator is consumed lazily:
377
+
378
+ ```scala
379
+ object Stream {
380
+ def fromIterator[A](it: => Iterator[A])(implicit jtA: JvmType.Infer[A]): Stream[Nothing, A]
381
+ }
382
+ ```
383
+
384
+ Create a stream from an iterator and collect all elements:
385
+
386
+ ```scala
387
+ import zio.blocks.streams.*
388
+
389
+ val iter = Iterator(10, 20, 30, 40)
390
+ // iter: Iterator[Int] = empty iterator
391
+ val stream = Stream.fromIterator(iter)
392
+ // stream: Stream[Nothing, Int] = Stream.fromIterator(...)
393
+ val result = stream.runCollect
394
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(10, 20, 30, 40))
395
+ ```
396
+
397
+ ### From Ranges
398
+
399
+ Streams can be created from numeric ranges, providing an efficient way to generate sequences of integers without allocating memory upfront:
400
+
401
+ #### `Stream.range`
402
+
403
+ Emits integers from `from` (inclusive) to `until` (exclusive):
404
+
405
+ ```scala
406
+ object Stream {
407
+ def range(from: Int, until: Int): Stream[Nothing, Int]
408
+ }
409
+ ```
410
+
411
+ This is memory-efficient (does not allocate intermediate collections):
412
+
413
+ ```scala
414
+ import zio.blocks.streams.*
415
+
416
+ val nums = Stream.range(0, 5)
417
+ // nums: Stream[Nothing, Int] = Stream.range(0, 5)
418
+ val result = nums.runCollect
419
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(0, 1, 2, 3, 4))
420
+ ```
421
+
422
+ #### `Stream.fromRange`
423
+
424
+ Converts a Scala `Range` object:
425
+
426
+ ```scala
427
+ object Stream {
428
+ def fromRange(range: Range): Stream[Nothing, Int]
429
+ }
430
+ ```
431
+
432
+ Create a stream from a `Range` and collect elements:
433
+
434
+ ```scala
435
+ import zio.blocks.streams.*
436
+
437
+ val range = 1 to 10 by 2
438
+ // range: Range = Range(1, 3, 5, 7, 9)
439
+ val stream = Stream.fromRange(range)
440
+ // stream: Stream[Nothing, Int] = Stream.fromRange(...)
441
+ val result = stream.runCollect
442
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 3, 5, 7, 9))
443
+ ```
444
+
445
+ ### Generators
446
+
447
+ These constructors create streams from functions and logic, useful for synthesizing infinite or computed sequences:
448
+
449
+ #### `Stream.repeat[A]`
450
+
451
+ Emits the same value infinitely:
452
+
453
+ ```scala
454
+ object Stream {
455
+ def repeat[A](a: A)(implicit jt: JvmType.Infer[A]): Stream[Nothing, A]
456
+ }
457
+ ```
458
+
459
+ Infinite streams are safe because streams are lazy; nothing runs until you call a terminal operation with a stopping condition (like `take`):
460
+
461
+ ```scala
462
+ import zio.blocks.streams.*
463
+
464
+ val infinite = Stream.repeat(42)
465
+ // infinite: Stream[Nothing, Int] = Stream.repeat(...)
466
+ val first5 = infinite.take(5)
467
+ // first5: Stream[Nothing, Int] = Stream.repeat(...).take(5)
468
+ val result = first5.runCollect
469
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(42, 42, 42, 42, 42))
470
+ ```
471
+
472
+ #### `Stream.unfold[S, A]`
473
+
474
+ A stateful generator that emits elements based on a fold-like transition function:
475
+
476
+ ```scala
477
+ object Stream {
478
+ def unfold[S, A](s: S)(f: S => Option[(A, S)])(implicit jtA: JvmType.Infer[A]): Stream[Nothing, A]
479
+ }
480
+ ```
481
+
482
+ Each iteration, `f` receives the current state and returns either `None` (stop) or `Some((element, nextState))`. This is useful for generating Fibonacci numbers or other sequences defined by a recurrence relation:
483
+
484
+ ```scala
485
+ import zio.blocks.streams.*
486
+
487
+ val fibonacci = Stream.unfold((0, 1)) {
488
+ case (a, b) => Some((a, (b, a + b)))
489
+ }
490
+ // fibonacci: Stream[Nothing, Int] = Stream.unfold(...)
491
+ val first10 = fibonacci.take(10)
492
+ // first10: Stream[Nothing, Int] = Stream.unfold(...).take(10)
493
+ val result = first10.runCollect
494
+ // result: Either[Nothing, Chunk[Int]] = Right(
495
+ // IndexedSeq(0, 1, 1, 2, 3, 5, 8, 13, 21, 34)
496
+ // )
497
+ ```
498
+
499
+ ### Side Effects
500
+
501
+ These constructors embed effects and deferred computation into streams, running actions at stream execution time:
502
+
503
+ #### `Stream.eval[A]`
504
+
505
+ Runs an arbitrary side effect and emits nothing:
506
+
507
+ ```scala
508
+ object Stream {
509
+ def eval(f: => Any): Stream[Nothing, Nothing]
510
+ }
511
+ ```
512
+
513
+ Use `eval` when you want a side effect in a stream (e.g., logging, metrics) but no element:
514
+
515
+ ```scala
516
+ import zio.blocks.streams.*
517
+
518
+ val sideEffect = Stream.eval(println("Executing side effect"))
519
+ // sideEffect: Stream[Nothing, Nothing] = Stream.suspend(...)
520
+ val result = sideEffect.runDrain
521
+ // Executing side effect
522
+ // result: Either[Nothing, Unit] = Right(())
523
+ ```
524
+
525
+ #### `Stream.attempt[A]`
526
+
527
+ Wraps a potentially throwing computation, converting non-fatal `Throwable`s into a typed error. Fatal errors (like `OutOfMemoryError`) are not caught and propagate as exceptions:
528
+
529
+ ```scala
530
+ object Stream {
531
+ def attempt[A](f: => A)(implicit jtA: JvmType.Infer[A]): Stream[Throwable, A]
532
+ }
533
+ ```
534
+
535
+ Use `attempt` when you have legacy code that throws exceptions:
536
+
537
+ ```scala
538
+ import zio.blocks.streams.*
539
+
540
+ def unsafeJsonParse(s: String): Int = s.toInt
541
+
542
+ val parsed = Stream.attempt(unsafeJsonParse("42"))
543
+ // parsed: Stream[Throwable, Int] = Stream.attempt(...)
544
+ val result = parsed.runCollect
545
+ // result: Either[Throwable, Chunk[Int]] = Right(IndexedSeq(42))
546
+ ```
547
+
548
+ #### `Stream.attemptEval`
549
+
550
+ Evaluates a side effect and converts any thrown exception into a typed `Throwable` error. Unlike `eval`, this captures exceptions and emits nothing:
551
+
552
+ ```scala
553
+ object Stream {
554
+ def attemptEval(f: => Any): Stream[Throwable, Nothing]
555
+ }
556
+ ```
557
+
558
+ Use `attemptEval` when you need to safely execute an effect that might throw, but you don't need to emit any elements:
559
+
560
+ ```scala
561
+ import zio.blocks.streams.*
562
+
563
+ val effect = Stream.attemptEval {
564
+ val file = new java.io.File("nonexistent.txt")
565
+ if (!file.exists()) throw new java.io.FileNotFoundException("File not found")
566
+ }
567
+ val result = effect.runDrain
568
+ ```
569
+
570
+ #### `Stream.defer[A]`
571
+
572
+ Creates an empty stream that lazily registers `f` as a release action. `f` runs exactly once when each materialization closes, including after failure, early termination, or cancellation; exceptions are defects:
573
+
574
+ ```scala
575
+ object Stream {
576
+ def defer(f: => Unit): Stream[Nothing, Nothing]
577
+ }
578
+ ```
579
+
580
+ Register a release action that runs when the stream closes:
581
+
582
+ ```scala
583
+ import zio.blocks.streams.*
584
+
585
+ val deferred = Stream.defer(println("Release action runs when the stream closes"))
586
+ // deferred: Stream[Nothing, Nothing] = Stream.defer(...)
587
+ val result = deferred.runDrain
588
+ // Release action runs when the stream closes
589
+ // result: Either[Nothing, Unit] = Right(())
590
+ ```
591
+
592
+ #### `Stream.suspend[E, A]`
593
+
594
+ Defers the creation of a stream until run time, useful for recursive stream definitions:
595
+
596
+ ```scala
597
+ object Stream {
598
+ def suspend[E, A](stream: => Stream[E, A]): Stream[E, A]
599
+ }
600
+ ```
601
+
602
+ Define a recursive stream safely:
603
+
604
+ ```scala
605
+ import zio.blocks.streams.*
606
+
607
+ def countDown(n: Int): Stream[Nothing, Int] =
608
+ if (n <= 0) Stream.empty
609
+ else Stream.suspend(Stream.succeed(n) ++ countDown(n - 1))
610
+
611
+ val result = countDown(5).runCollect
612
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(5, 4, 3, 2, 1))
613
+ ```
614
+
615
+ ### I/O
616
+
617
+ Streams can read from external I/O sources like files and readers, automatically managing resource cleanup:
618
+
619
+ #### `Stream.fromInputStream`
620
+
621
+ Reads bytes from a Java `InputStream`, managing the resource:
622
+
623
+ ```scala
624
+ object Stream {
625
+ def fromInputStream(is: java.io.InputStream): Stream[java.io.IOException, Byte]
626
+ }
627
+ ```
628
+
629
+ The stream automatically closes the input stream when done:
630
+
631
+ ```scala
632
+ import zio.blocks.streams.*
633
+ import java.io.ByteArrayInputStream
634
+
635
+ val data = new ByteArrayInputStream("Hello".getBytes)
636
+ // data: ByteArrayInputStream = java.io.ByteArrayInputStream@2c6b522e
637
+ val bytes = Stream.fromInputStream(data)
638
+ // bytes: Stream[IOException, Byte] = Stream.fromAcquireRelease(...)
639
+ val result = bytes.runCollect
640
+ // result: Either[IOException, Chunk[Byte]] = Right(
641
+ // IndexedSeq(72, 101, 108, 108, 111)
642
+ // )
643
+ ```
644
+
645
+ #### `Stream.fromJavaReader`
646
+
647
+ Reads characters from a Java `Reader`:
648
+
649
+ ```scala
650
+ object Stream {
651
+ def fromJavaReader(r: java.io.Reader): Stream[java.io.IOException, Char]
652
+ }
653
+ ```
654
+
655
+ Read characters from a string reader:
656
+
657
+ ```scala
658
+ import zio.blocks.streams.*
659
+ import java.io.StringReader
660
+
661
+ val reader = new StringReader("hello world")
662
+ // reader: StringReader = java.io.StringReader@e4d1116
663
+ val stream = Stream.fromJavaReader(reader)
664
+ // stream: Stream[IOException, Char] = Stream.fromAcquireRelease(...)
665
+ val result = stream.runCollect
666
+ // result: Either[IOException, Chunk[Char]] = Right(
667
+ // IndexedSeq('h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd')
668
+ // )
669
+ ```
670
+
671
+ #### `Stream.fromInputStreamUnmanaged`
672
+
673
+ Reads bytes from a Java `InputStream` without automatic resource management. The caller is responsible for closing the stream:
674
+
675
+ ```scala
676
+ object Stream {
677
+ def fromInputStreamUnmanaged(is: java.io.InputStream): Stream[java.io.IOException, Byte]
678
+ }
679
+ ```
680
+
681
+ Use this when you need to manage the stream's lifecycle yourself, for example when the stream is created from a long-lived resource:
682
+
683
+ ```scala
684
+ import zio.blocks.streams.*
685
+ import java.io.ByteArrayInputStream
686
+
687
+ val data = new ByteArrayInputStream("Data".getBytes)
688
+ val bytes = Stream.fromInputStreamUnmanaged(data)
689
+ val result = bytes.runCollect
690
+ // Caller must close data when done
691
+ ```
692
+
693
+ #### `Stream.fromJavaReaderUnmanaged`
694
+
695
+ Reads characters from a Java `Reader` without automatic resource management. The caller is responsible for closing the reader:
696
+
697
+ ```scala
698
+ object Stream {
699
+ def fromJavaReaderUnmanaged(r: java.io.Reader): Stream[java.io.IOException, Char]
700
+ }
701
+ ```
702
+
703
+ Use this when you need to manage the reader's lifecycle yourself:
704
+
705
+ ```scala
706
+ import zio.blocks.streams.*
707
+ import java.io.StringReader
708
+
709
+ val reader = new StringReader("managed externally")
710
+ val stream = Stream.fromJavaReaderUnmanaged(reader)
711
+ val result = stream.runCollect
712
+ // Caller must close reader when done
713
+ ```
714
+
715
+ ## Transformations
716
+
717
+ Streams provide powerful operations for transforming elements, flattening nested structures, filtering, and managing state:
718
+
719
+ ### Element-wise Transformations
720
+
721
+ These operations apply functions to stream elements one-by-one, applying the transformation lazily as elements are pulled:
722
+
723
+ #### `Stream#map[B]`
724
+
725
+ Applies a function to each element:
726
+
727
+ ```scala
728
+ abstract class Stream[+E, +A] {
729
+ def map[B](f: A => B)(implicit jtB: JvmType.Infer[B]): Stream[E, B]
730
+ }
731
+ ```
732
+
733
+ `map` does not run immediately; it builds up a description of the transformation. Only when you call a terminal operation does the mapping happen:
734
+
735
+ ```scala
736
+ import zio.blocks.streams.*
737
+
738
+ val nums = Stream(1, 2, 3)
739
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3)
740
+ val doubled = nums.map(_ * 2)
741
+ // doubled: Stream[Nothing, Int] = Stream(1, 2, 3).map(...)
742
+ val result = doubled.runCollect
743
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(2, 4, 6))
744
+ ```
745
+
746
+ For bounded concurrency, [`Stream#mapPar`](#streammappar) applies a synchronous function with up to `n` applications active and [`Stream#mapParAsync`](#streammapparasync) keeps up to `n` `Async` callbacks in flight; both are unordered. See [Bounded Concurrency](#bounded-concurrency).
747
+
748
+ **Key point:** `Stream#map` is covariant in the output type because it preserves the error type and only transforms elements. Output-changing operations such as `map`, `collect`, `flatMap`, `mapAccum`, `scan`, and `zipWith` take `JvmType.Infer` evidence for their result type; that result evidence selects the physical output lane. Type-preserving operations retain the source's known lane, including when the static element type is widened.
749
+
750
+ #### `Stream#mapError[E2]`
751
+
752
+ Transforms typed errors without affecting elements:
753
+
754
+ ```scala
755
+ abstract class Stream[+E, +A] {
756
+ def mapError[E2](f: E => E2): Stream[E2, A]
757
+ }
758
+ ```
759
+
760
+ Use `mapError` to convert one error type to another:
761
+
762
+ ```scala
763
+ import zio.blocks.streams.*
764
+
765
+ sealed trait ApiError
766
+ case class ServerError(msg: String) extends ApiError
767
+ case class NetworkError() extends ApiError
768
+
769
+ val mayFail: Stream[NetworkError, String] = Stream.fail(NetworkError())
770
+ val mapped = mayFail.mapError(e => ServerError("Connection failed"))
771
+ ```
772
+
773
+ #### `Stream#filter`
774
+
775
+ Emits only elements that satisfy a predicate:
776
+
777
+ ```scala
778
+ abstract class Stream[+E, +A] {
779
+ def filter(pred: A => Boolean): Stream[E, A]
780
+ }
781
+ ```
782
+
783
+ Short-circuits: as soon as the sink says "stop," filtering stops:
784
+
785
+ ```scala
786
+ import zio.blocks.streams.Stream
787
+
788
+ val nums = Stream(1, 2, 3, 4, 5)
789
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5)
790
+ val evens = nums.filter(_ % 2 == 0)
791
+ // evens: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5).filter(...)
792
+ val result = evens.runCollect
793
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(2, 4))
794
+ ```
795
+
796
+ #### `Stream#collect[B]`
797
+
798
+ Applies a partial function, emitting only defined results:
799
+
800
+ ```scala
801
+ abstract class Stream[+E, +A] {
802
+ def collect[B](pf: PartialFunction[A, B])(implicit jtB: JvmType.Infer[B]): Stream[E, B]
803
+ }
804
+ ```
805
+
806
+ This combines filtering and mapping in one step:
807
+
808
+ ```scala
809
+ import zio.blocks.streams.*
810
+
811
+ val mixed = Stream(1, "a", 2, "b", 3)
812
+ // mixed: Stream[Nothing, Int | String] = Stream(1, a, 2, b, 3)
813
+ val numbers = mixed.collect { case n: Int => n }
814
+ // numbers: Stream[Nothing, Int] = Stream(1, a, 2, b, 3).collect(...)
815
+ val result = numbers.runCollect
816
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 3))
817
+ ```
818
+
819
+ ### Stateful Transformations
820
+
821
+ These operations maintain internal state while processing elements, allowing you to fold computations into the transformation:
822
+
823
+ #### `Stream#mapAccum[S, B]`
824
+
825
+ Maintains state while transforming each element:
826
+
827
+ ```scala
828
+ abstract class Stream[+E, +A] {
829
+ def mapAccum[S, B](init: S)(f: (S, A) => (S, B))(implicit jtB: JvmType.Infer[B]): Stream[E, B]
830
+ }
831
+ ```
832
+
833
+ `mapAccum` threads a state value through the transformation. At each step, you receive the current state and the element, return a new state and output element:
834
+
835
+ ```scala
836
+ import zio.blocks.streams.*
837
+
838
+ val nums = Stream(1, 2, 3)
839
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3)
840
+ val indexed = nums.mapAccum(0)((idx, x) => (idx + 1, (idx, x)))
841
+ // indexed: Stream[Nothing, Tuple2[Int, Int]] = Stream.suspend(...)
842
+ val result = indexed.runCollect
843
+ // result: Either[Nothing, Chunk[Tuple2[Int, Int]]] = Right(
844
+ // IndexedSeq((0, 1), (1, 2), (2, 3))
845
+ // )
846
+ ```
847
+
848
+ #### `Stream#mapAccumAsync[S, B]`
849
+
850
+ The asynchronous twin of `mapAccum`: the step returns an `Async`, and the state is still threaded strictly in order.
851
+
852
+ ```scala
853
+ abstract class Stream[+E, +A] {
854
+ def mapAccumAsync[S, B](init: S)(f: (S, A) => Async[(S, B)])(implicit
855
+ jtB: JvmType.Infer[B]
856
+ ): Stream[E, B]
857
+ }
858
+ ```
859
+
860
+ At most one invocation of `f` is active at a time, which is what keeps the accumulator meaningful — there is no concurrency here to reorder the steps or to hand two invocations the same state. A failure inside `f` is a defect, not a typed error. The implicit `JvmType.Infer[B]` records the physical lane of the new output type and is supplied by the compiler.
861
+
862
+ ```scala
863
+ import zio.blocks.async.*
864
+ import zio.blocks.streams.*
865
+
866
+ val events = Stream("open", "write", "close")
867
+ val numbered = events.mapAccumAsync(0L)((seq, event) => Async.succeed((seq + 1, s"$seq:$event")))
868
+ ```
869
+
870
+ This operator is also catalogued with the rest of the sequential asynchronous family in [Async Operators](../execution-and-compatibility/async-execution.md#streammapaccumasync), and [Stateful Asynchronous Operators](#stateful-asynchronous-operators) runs it end to end alongside `scanAsync`, `takeWhileAsync`, and `ensuringAsync`.
871
+
872
+ #### `Stream#scan[S]`
873
+
874
+ Like `mapAccum`, but emits the accumulator rather than a mapped value, starting with `init` — so the output stream has one more element than the input:
875
+
876
+ ```scala
877
+ abstract class Stream[+E, +A] {
878
+ def scan[S](init: S)(f: (S, A) => S)(implicit jtS: JvmType.Infer[S]): Stream[E, S]
879
+ }
880
+ ```
881
+
882
+ This is useful for computing running totals, moving averages, or other cumulative statistics:
883
+
884
+ ```scala
885
+ import zio.blocks.streams.*
886
+
887
+ val nums = Stream(1, 2, 3, 4)
888
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4)
889
+ val cumsum = nums.scan(0)(_ + _)
890
+ // cumsum: Stream[Nothing, Int] = Stream(1, 2, 3, 4).scan(...)
891
+ val result = cumsum.runCollect
892
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(0, 1, 3, 6, 10))
893
+ ```
894
+
895
+ #### `Stream#scanAsync[S]`
896
+
897
+ The asynchronous twin of `scan`: the fold step returns an `Async`, and the accumulator is still emitted at each step.
898
+
899
+ ```scala
900
+ abstract class Stream[+E, +A] {
901
+ def scanAsync[S](init: S)(f: (S, A) => Async[S])(implicit jtS: JvmType.Infer[S]): Stream[E, S]
902
+ }
903
+ ```
904
+
905
+ The output stream carries one more element than the input, because `init` is emitted before the first step runs. As with `mapAccumAsync`, the steps are sequential and a failure inside `f` is a defect.
906
+
907
+ ```scala
908
+ import zio.blocks.async.*
909
+ import zio.blocks.streams.*
910
+
911
+ val amounts = Stream(120, -40, 75)
912
+ val balances = amounts.scanAsync(0L)((balance, amount) => Async.succeed(balance + amount))
913
+ ```
914
+
915
+ `balances` emits `0`, `120`, `80`, `155`. See [Async Operators](../execution-and-compatibility/async-execution.md#streamscanasync) for the same entry alongside the rest of the asynchronous family.
916
+
917
+ ### Flat-Mapping (Nested Streams)
918
+
919
+ `flatMap[E2, E3, B]` — Maps each element to a stream and flattens the results.:
920
+
921
+ ```scala
922
+ abstract class Stream[+E, +A] {
923
+ def flatMap[E2, E3, B](f: A => Stream[E2, B])(implicit
924
+ errorConcat: Concat.WithOut[E, E2, E3],
925
+ jtB: JvmType.Infer[B]
926
+ ): Stream[E3, B]
927
+ }
928
+ ```
929
+
930
+ `Stream#flatMap` is sequential: streams are processed one at a time, in order. This is essential for resource safety: if each inner stream acquires a resource, `Stream#flatMap` ensures they are released in proper FIFO order:
931
+
932
+ ```scala
933
+ import zio.blocks.streams.*
934
+
935
+ val ids = Stream(1, 2, 3)
936
+ // ids: Stream[Nothing, Int] = Stream(1, 2, 3)
937
+ val expanded = ids.flatMap(id => Stream(s"${id}-a", s"${id}-b"))
938
+ // expanded: Stream[Nothing, String] = Stream(1, 2, 3).flatMap(...)
939
+ val result = expanded.runCollect
940
+ // result: Either[Nothing, Chunk[String]] = Right(
941
+ // IndexedSeq("1-a", "1-b", "2-a", "2-b", "3-a", "3-b")
942
+ // )
943
+ ```
944
+
945
+ For the concurrent counterpart, which merges up to `n` inner streams at once in arrival order, see [`Stream#flatMapPar`](#streamflatmappar) in [Bounded Concurrency](#bounded-concurrency).
946
+
947
+ #### `Stream.flattenAll[E, A]`
948
+
949
+ Flattens a stream of streams into a single stream, processing them sequentially:
950
+
951
+ ```scala
952
+ object Stream {
953
+ def flattenAll[E, A](streams: Stream[E, Stream[E, A]])(implicit jtA: JvmType.Infer[A]): Stream[E, A]
954
+ }
955
+ ```
956
+
957
+ This is equivalent to `flatMap(identity)`. Use `flattenAll` when you already have a stream of streams and want to flatten it without applying a transformation:
958
+
959
+ ```scala
960
+ import zio.blocks.streams.*
961
+
962
+ val nested = Stream.fromIterable(List(
963
+ Stream(1, 2),
964
+ Stream(3, 4)
965
+ ))
966
+ // nested: Stream[Nothing, Stream[Nothing, Int]] = Stream.fromIterable(...)
967
+ val flat = Stream.flattenAll(nested)
968
+ // flat: Stream[Nothing, Int] = Stream.fromIterable(...).flatMap(...)
969
+ val result = flat.runCollect
970
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 3, 4))
971
+ ```
972
+
973
+ To flatten a stream of streams concurrently instead of sequentially, the companion also offers [`Stream.mergeAll`](#streammergeall), which drains up to `maxOpen` inner streams at a time. See [Bounded Concurrency](#bounded-concurrency).
974
+
975
+ ## Windowing
976
+
977
+ Streams can be grouped, sliced, and scanned to process data in temporal windows. These operations group elements into chunks and slide windows over the stream for batch processing:
978
+
979
+ ### `Stream#grouped[A]`
980
+
981
+ Collects elements into fixed-size chunks:
982
+
983
+ ```scala
984
+ abstract class Stream[+E, +A] {
985
+ def grouped(n: Int): Stream[E, Chunk[A]]
986
+ }
987
+ ```
988
+
989
+ The last chunk may contain fewer than `n` elements:
990
+
991
+ ```scala
992
+ import zio.blocks.streams.*
993
+
994
+ val nums = Stream(1, 2, 3, 4, 5)
995
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5)
996
+ val groups = nums.grouped(2)
997
+ // groups: Stream[Nothing, Chunk[Int]] = Stream(1, 2, 3, 4, 5).chunked(2)
998
+ val result = groups.runCollect
999
+ // result: Either[Nothing, Chunk[Chunk[Int]]] = Right(
1000
+ // IndexedSeq(IndexedSeq(1, 2), IndexedSeq(3, 4), IndexedSeq(5))
1001
+ // )
1002
+ ```
1003
+
1004
+ ### `Stream#sliding[A]`
1005
+
1006
+ Creates a sliding window of size `n`, optionally stepping by `step` elements:
1007
+
1008
+ ```scala
1009
+ abstract class Stream[+E, +A] {
1010
+ def sliding(n: Int, step: Int = 1): Stream[E, Chunk[A]]
1011
+ }
1012
+ ```
1013
+
1014
+ This is useful for computing local statistics or detecting patterns in sequences:
1015
+
1016
+ ```scala
1017
+ import zio.blocks.streams.*
1018
+
1019
+ val nums = Stream(1, 2, 3, 4, 5)
1020
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5)
1021
+ val windows = nums.sliding(3, step = 1)
1022
+ // windows: Stream[Nothing, Chunk[Int]] = Stream(1, 2, 3, 4, 5).sliding(3, 1)
1023
+ val result = windows.runCollect
1024
+ // result: Either[Nothing, Chunk[Chunk[Int]]] = Right(
1025
+ // IndexedSeq(IndexedSeq(1, 2, 3), IndexedSeq(2, 3, 4), IndexedSeq(3, 4, 5))
1026
+ // )
1027
+ ```
1028
+
1029
+ ## Combining Streams
1030
+
1031
+ Streams can be sequentially concatenated, zipped together, or merged:
1032
+
1033
+ ### Sequential Concatenation
1034
+
1035
+ `++[E2, E3, A2, A3]` or `concat[E2, E3, A2, A3]` — Emits all elements of the first stream, then all elements of the second stream:
1036
+
1037
+ ```scala
1038
+ abstract class Stream[+E, +A] {
1039
+ final def ++[E2, E3, A2, A3](that: Stream[E2, A2])(implicit
1040
+ errorConcat: Concat.WithOut[E, E2, E3],
1041
+ valueConcat: Concat.WithOut[A, A2, A3],
1042
+ jtA3: JvmType.Infer[A3]
1043
+ ): Stream[E3, A3] = concat(that)
1044
+ }
1045
+ ```
1046
+
1047
+ The result type follows the same widening rules as Scala 3 unions:
1048
+
1049
+ - identical types stay unchanged (`A ++ A => A`)
1050
+ - subtypes widen to the supertype (`Dog ++ Animal => Animal`)
1051
+ - siblings with a common meaningful supertype widen to that supertype (`Dog ++ Cat => Animal`, when both extend a sealed `Animal`)
1052
+ - otherwise the result is a disjoint union (`String ++ Int => String | Int`)
1053
+
1054
+ On Scala 3, disjoint concat results are native unions. On Scala 2, the same/subtype and sibling cases collapse to the wider existing type (zero-cost, values are reused as-is); only types without a shared meaningful supertype fall back to `Either[L, R]`.
1055
+
1056
+ Evaluation is sequential: the second stream only starts when the first completes:
1057
+
1058
+ ```scala
1059
+ import zio.blocks.streams.*
1060
+
1061
+ val first = Stream(1, 2)
1062
+ // first: Stream[Nothing, Int] = Stream(1, 2)
1063
+ val second = Stream(3, 4)
1064
+ // second: Stream[Nothing, Int] = Stream(3, 4)
1065
+ val combined = first ++ second
1066
+ // combined: Stream[Nothing, Int] = Stream(1, 2) ++ Stream(3, 4)
1067
+ val result = combined.runCollect
1068
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 3, 4))
1069
+ ```
1070
+
1071
+ For unrelated element types, Scala 3 produces a direct union while Scala 2 produces `Either`:
1072
+
1073
+ <Tabs groupId="scala-version" defaultValue="scala2">
1074
+ <TabItem value="scala2" label="Scala 2.13">
1075
+
1076
+ ```scala
1077
+ val combined: Stream[Nothing, Either[String, Int]] =
1078
+ Stream.succeed("left") ++ Stream.succeed(1)
1079
+ ```
1080
+
1081
+ </TabItem>
1082
+ <TabItem value="scala3" label="Scala 3.x">
1083
+
1084
+ ```scala
1085
+ val combined: Stream[Nothing, String | Int] =
1086
+ Stream.succeed("left") ++ Stream.succeed(1)
1087
+ ```
1088
+
1089
+ </TabItem>
1090
+ </Tabs>
1091
+
1092
+ ```scala
1093
+ import zio.blocks.streams.*
1094
+ import zio.blocks.chunk.Chunk
1095
+
1096
+ val concatResult = (Stream.succeed("left") ++ Stream.succeed(1)).runCollect
1097
+ // concatResult: Either[Nothing, Chunk[String | Int]] = Right(
1098
+ // IndexedSeq("left", 1)
1099
+ // )
1100
+
1101
+ assert(concatResult == Right(Chunk[String | Int]("left", 1)))
1102
+ ```
1103
+
1104
+ The error channel follows the same rules. Same/subtype errors collapse; unrelated errors remain disjoint:
1105
+
1106
+ ```scala
1107
+ import zio.blocks.streams.*
1108
+
1109
+ sealed trait LeftError
1110
+ case class Boom(msg: String) extends LeftError
1111
+ case class Missing(code: Int)
1112
+
1113
+ val left: Stream[LeftError, String] = Stream.fail(Boom("boom"))
1114
+ // left: Stream[LeftError, String] = Stream.fail(...)
1115
+ val right = Stream.succeed(true)
1116
+ // right: Stream[Nothing, Boolean] = Stream.succeed(...)
1117
+
1118
+ left.runCollect
1119
+ // res39: Either[LeftError, Chunk[String]] = Left(Boom("boom"))
1120
+
1121
+ val failed = left ++ (Stream.fail(Missing(404)): Stream[Missing, Boolean])
1122
+ // failed: Stream[LeftError | Missing, String | Boolean] = Stream.fail(...) ++ Stream.fail(...)
1123
+ failed.runCollect
1124
+ // res40: Either[LeftError | Missing, Chunk[String | Boolean]] = Left(
1125
+ // Boom("boom")
1126
+ // )
1127
+ ```
1128
+
1129
+ There is no separate `choice` operator anymore. Use `++` / `concat` for all sequential combination; the result type already reflects the Scala 3-style union semantics.
1130
+
1131
+ ### Zipping
1132
+
1133
+ Zips two streams together as tuples:
1134
+
1135
+ ```scala
1136
+ abstract class Stream[+E, +A] {
1137
+ def &&[E2, E3, B, C](that: Stream[E2, B])(implicit
1138
+ errorConcat: Concat.WithOut[E, E2, E3],
1139
+ zip: Stream.Zip[A, B, C],
1140
+ jtC: JvmType.Infer[C]
1141
+ ): Stream[E3, C]
1142
+ }
1143
+ ```
1144
+
1145
+ The error type `E3` is the `Concat` of the two error types, and the element type `C` is chosen by the `Stream.Zip` evidence, which flattens nested pairs so that `a && b && c` produces a `Stream` of `(A, B, C)`.
1146
+
1147
+ The result streams have the same length as the shorter input:
1148
+
1149
+ ```scala
1150
+ import zio.blocks.streams.*
1151
+
1152
+ val nums = Stream(1, 2, 3)
1153
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3)
1154
+ val chars = Stream('a', 'b')
1155
+ // chars: Stream[Nothing, Char] = Stream(a, b)
1156
+ val zipped = nums && chars
1157
+ // zipped: Stream[Nothing, Tuple2[Int, Char]] = Stream(1, 2, 3) && Stream(a, b)
1158
+ val result = zipped.runCollect
1159
+ // result: Either[Nothing, Chunk[Tuple2[Int, Char]]] = Right(
1160
+ // IndexedSeq((1, 'a'), (2, 'b'))
1161
+ // )
1162
+ ```
1163
+
1164
+ ## Bounded Concurrency
1165
+
1166
+ Four operators bound the concurrency of a stream. `Stream#mapPar`, `Stream#flatMapPar`, and `Stream.mergeAll` take synchronous callbacks; `Stream#mapParAsync` takes a callback that returns an `Async`. All four run on either of two execution paths — a synchronous one backed by worker threads, or an asynchronous one — and all four carry an `n = 1` degradation guarantee.
1167
+
1168
+ The two execution paths are different engines with different bounds, and which one a program gets is decided by the kind of reader its pipeline compiles to — not by which operator it called.
1169
+
1170
+ ### The Operators
1171
+
1172
+ | Operator | Element callback | What `n` bounds |
1173
+ |-------------------------------------|------------------------------------------------|--------------------------------|
1174
+ | `Stream#mapPar(n)(f)` | `A => B` | concurrent applications of `f` |
1175
+ | `Stream#mapParAsync(n)(f)` | `A => Async[B]` | callbacks in flight |
1176
+ | `Stream#flatMapPar(n)(f)` | `A => Stream[E1, B]` | open inner streams |
1177
+ | `Stream.mergeAll(maxOpen)(streams)` | none; `streams` is a `Stream[E, Stream[E, A]]` | open inner streams |
1178
+
1179
+ Each of the four begins with `require` on its parallelism argument, so passing zero or a negative number raises `IllegalArgumentException` at description time rather than producing an empty or sequential stream.
1180
+
1181
+ #### `Stream#mapPar`
1182
+
1183
+ ```scala
1184
+ def mapPar[B](n: Int)(f: A => B)(implicit jtB: JvmType.Infer[B]): Stream[E, B]
1185
+ ```
1186
+
1187
+ Applies `f` to each element with up to `n` applications active. Output is unordered: elements leave in the order their applications finish, not the order they entered. `f` is synchronous, so this is the operator for CPU-bound or blocking work on the JVM — and, as [Threading and Platform Behaviour](#threading-and-platform-behaviour) explains, the operator that overlaps nothing at all once the pipeline is on the asynchronous lane.
1188
+
1189
+ #### `Stream#mapParAsync`
1190
+
1191
+ ```scala
1192
+ def mapParAsync[B](n: Int)(f: A => Async[B])(implicit jtB: JvmType.Infer[B]): Stream[E, B]
1193
+ ```
1194
+
1195
+ Keeps at most `n` `Async` callbacks in flight and emits each result in completion order. It is the only member of the family whose callback can suspend: `f` returns a description, so the engine holds `n` unfinished effects rather than `n` busy threads. A failure inside the callback is a defect, not a typed error, and it fails the outer terminal effect.
1196
+
1197
+ Unlike `mapPar`, this operator has no synchronous materialization at all. It compiles to the shared asynchronous concurrent reader on both platforms, which is why its `n` counts suspended callbacks rather than workers.
1198
+
1199
+ #### `Stream#flatMapPar`
1200
+
1201
+ ```scala
1202
+ def flatMapPar[E1 >: E, B](n: Int)(f: A => Stream[E1, B])(implicit jtB: JvmType.Infer[B]): Stream[E1, B]
1203
+ ```
1204
+
1205
+ Applies `f` to each element to produce an inner stream, then merges up to `n` inner streams concurrently. It has no engine of its own. Past the `n = 1` branch, its body is a single delegation:
1206
+
1207
+ ```scala
1208
+ Stream.mergeAll[E1, B](n)(
1209
+ this.asInstanceOf[Stream[E1, A]].map(f)(JvmType.Infer.boxed[Stream[E1, B]])
1210
+ )(jtB)
1211
+ ```
1212
+
1213
+ One fan-in engine, not two. Everything below about slots, admission, ordering, and shutdown is stated for `mergeAll`, and `flatMapPar` inherits all of it unchanged.
1214
+
1215
+ #### `Stream.mergeAll`
1216
+
1217
+ ```scala
1218
+ def mergeAll[E, A](maxOpen: Int)(streams: Stream[E, Stream[E, A]])(implicit jtA: JvmType.Infer[A]): Stream[E, A]
1219
+ ```
1220
+
1221
+ Merges up to `maxOpen` inner streams concurrently into one output stream, with elements arriving in completion order. Note the argument: `streams` is a *stream of streams*, not a varargs list, so a fixed collection of sources is fed in through a constructor such as `Stream.fromIterable`.
1222
+
1223
+ ```scala
1224
+ import zio.blocks.streams._
1225
+
1226
+ val sources: Stream[Nothing, Stream[Nothing, Int]] =
1227
+ Stream.fromIterable((0 until 10).map(i => Stream.range(i * 100, (i + 1) * 100)))
1228
+
1229
+ val merged: Stream[Nothing, Int] = Stream.mergeAll(4)(sources)
1230
+ ```
1231
+
1232
+ ### Semantics
1233
+
1234
+ #### What `n` Means
1235
+
1236
+ `n` is a count of selector slots, not of threads. The concurrent reader allocates one `AsyncSelector` with `n + 1` entries: `n` entries for inner work, and one final entry reserved for the outer source.
1237
+
1238
+ ```
1239
+ ┌────────────────────────────────────────────────────────────────────┐
1240
+ │ Async.selectorWithCapacity(n + 1) one selector, n + 1 entries │
1241
+ ├────────────────────────────────────────────────────────────────────┤
1242
+ │ entry 0 inner work: callback in flight, or an open inner │
1243
+ │ entry 1 inner work: callback in flight, or an open inner │
1244
+ │ ... up to n of these; `active` counts the occupied ones │
1245
+ │ entry n-1 inner work: callback in flight, or an open inner │
1246
+ ├────────────────────────────────────────────────────────────────────┤
1247
+ │ entry n the outer source re-armed only while active < n │
1248
+ └────────────────────────────────────────────────────────────────────┘
1249
+ ```
1250
+
1251
+ The extra entry is what keeps the operator from pulling ahead. The source entry is re-armed only while `active < n`, so the reader stops asking upstream for elements the moment every inner slot is taken, and resumes the instant one frees. Nothing queues behind a full set of slots.
1252
+
1253
+ Whether a slot corresponds to a thread depends entirely on which engine materialized. On the JVM's synchronous lane each slot does have a worker thread behind it; on the asynchronous lane a slot is one pending `Async` and there are no threads involved.
1254
+
1255
+ #### Ordering
1256
+
1257
+ All four operators are unordered with respect to input position. An element leaves when its work finishes, so output is in arrival order. The property tests compare results with `.toSet` for exactly this reason: there is no input-order assertion available to make.
1258
+
1259
+ The single exception is `n = 1`, which is exactly sequential — and the `n = 1` tests do assert exact [`Chunk`](../../chunk.md) equality, because at that value the operator is not the concurrent engine at all. See [The `n = 1` Guarantee](#the-n-1-guarantee).
1260
+
1261
+ :::warning[Unordered means unordered on Scala.js too]
1262
+ Single-threaded execution does not restore input order. The concurrent engine runs on Scala.js with immediately-ready effects, and its arrival-order semantics are retained there. Code that depends on Scala.js emitting source order is a bug that will not reproduce on the JVM.
1263
+ :::
1264
+
1265
+ If input order is what you need, use sequential `map`, `mapAsync`, or `flatMap`. Sorting afterwards — `.runCollectAsync.map(_.map(_.sorted))` — recovers *a* total order, but not the input one unless the elements happen to sort that way.
1266
+
1267
+ #### Boundedness
1268
+
1269
+ Two separate things are bounded, and conflating them leads to the wrong buffer size.
1270
+
1271
+ The first is the number of open inner streams. Admission is guarded by an active count: `occupied` is a `Boolean` array of length `n` and `active` is the number of `true` entries, and a new element is accepted only into a free index. A full set of slots stops admission at the source rather than accumulating work anywhere.
1272
+
1273
+ The second is the number of buffered elements, and it exists only on the JVM's synchronous lane. `ConcurrentMapParReader` allocates `n` input and `n` output `SpscRingBuffer`s, each of `bufferSize` capacity, so a `mapPar(8)` with the default buffer holds at most 1024 elements in transit. The concurrent merge readers allocate one output ring per slot on the same basis.
1274
+
1275
+ The asynchronous engine has no rings of its own. `AsyncConcurrentReaders.mapPar` does not even take a buffer size: each slot holds exactly one unfinished effect, so in-flight work is bounded at `n` and nothing more. `AsyncConcurrentReaders.merge` does take one, but spends it on compiling each inner stream (`stream.compile(0, bufferSize)`), where it sizes whatever buffered stages that inner stream contains.
1276
+
1277
+ #### Admission and Replenishment
1278
+
1279
+ A slot is occupied before the work in it begins, and — for merge — freed only after that work has fully closed.
1280
+
1281
+ ```
1282
+ free
1283
+ │ outer element arrives: occupied(i) = true, active += 1
1284
+ ▼
1285
+ constructing the Async that builds the child runs HERE, in the slot
1286
+ │ child installed
1287
+ ▼
1288
+ draining elements reach the consumer in arrival order
1289
+ │ MapWorkerEnd: slot entry replaced by the child's close effect
1290
+ ▼
1291
+ closing slot still held; no successor may be admitted yet
1292
+ │ MapWorkerClosed: releaseSlot(i), then armSource()
1293
+ ▼
1294
+ free
1295
+ ```
1296
+
1297
+ That two-phase ending is the part worth remembering. When an inner stream reaches its end the engine does not free the slot; it replaces the slot's selector entry with the inner reader's own close effect, which resolves to a second signal. Only then does `releaseSlot` clear `occupied(i)`, decrement `active`, and re-arm the source. A slot is therefore never handed to a successor before its predecessor's finalizers have run to completion.
1298
+
1299
+ `mapPar` and `mapParAsync` have the shorter version of this, because a callback has no reader to close: the slot is released in the same step that hands the value to the consumer, and the source is re-armed there.
1300
+
1301
+ The "constructing" phase is where the slot accounting surprises people. With `flatMapPar(n)(a => Stream.unwrap(f(a)))`, the effect `f(a)` that *produces* the child runs inside the slot the child will later occupy — they share one of the `n`, they do not get one each. [Async children and slot accounting](#async-children-and-slot-accounting) below demonstrates this with a running program.
1302
+
1303
+ #### The `n = 1` Guarantee {#the-n-1-guarantee}
1304
+
1305
+ At `n = 1` each operator degrades to its sequential twin. This is a guarantee about the code path, not an optimization note: the branch sits in the public operator body, above every allocation.
1306
+
1307
+ ```scala
1308
+ def mapPar[B](n: Int)(f: A => B)(implicit jtB: JvmType.Infer[B]): Stream[E, B] = {
1309
+ require(n >= 1, s"mapPar requires n >= 1, got $n")
1310
+ if (n == 1) map(f)
1311
+ else {
1312
+ jtB.jvmType
1313
+ new Stream.MapPar[E, A, B](this, n, f, elementRepresentation, jtB.jvmType)
1314
+ }
1315
+ }
1316
+ ```
1317
+
1318
+ The other three are shaped identically: `mapParAsync(1)` returns `mapAsync(f)`, `flatMapPar(1)` returns `flatMap(f)`, and `mergeAll(1)` returns `streams.flatMap(identity)` — or, when the outer stream is already a `Mapped` node, the fused `mapped.self.flatMap(mapped.f)` that skips the intermediate stream entirely. No reader, no selector, no thread, and no ring is allocated in any of those cases, because the decision is made before the concurrent node is ever constructed.
1319
+
1320
+ The practical consequence is that `n` can be a configuration value that is allowed to be `1`. A deployment that dials concurrency down to one gets the sequential operator, with its exact input ordering, rather than a concurrent engine running at width one.
1321
+
1322
+ ### Buffer Sizing
1323
+
1324
+ Concurrent readers on the synchronous lane use ring-buffer queues sized by the enclosing buffer-size region. The default is 64 (the library-internal `Stream.DefaultBufferSize`, which is not part of the public API).
1325
+
1326
+ ```scala
1327
+ import zio.blocks.streams._
1328
+
1329
+ def heavyComputation(n: Int): Int = n * n
1330
+
1331
+ val sized: Stream[Nothing, Int] =
1332
+ Stream.bufferSize(256) {
1333
+ Stream.range(0, 1000000).mapPar(8)(heavyComputation)
1334
+ }
1335
+ ```
1336
+
1337
+ `Stream.bufferSize(n)` requires a positive power of two and rejects anything else with `IllegalArgumentException`:
1338
+
1339
+ ```scala
1340
+ require(n >= 1 && (n & (n - 1)) == 0, s"bufferSize must be a positive power of 2, got $n")
1341
+ ```
1342
+
1343
+ Nested regions use the innermost size. Larger buffers absorb bursty producers; smaller ones cut memory when many slots are open at once. The default suits most workloads. On the asynchronous lane its reach is narrower: `mapPar` and `mapParAsync` ignore it entirely, because each slot holds exactly one unfinished effect and the bound is the slot count, while `mergeAll` and `flatMapPar` still pass it down, where it sizes the buffered stages inside each compiled inner stream.
1344
+
1345
+ `Pipeline.buffer(n)` is a different tool for a different job: it inserts a bounded buffer between two stages rather than resizing the queues inside one concurrent reader, and it participates in the asynchronous reader graph on both platforms.
1346
+
1347
+ ### Error Behaviour
1348
+
1349
+ First failure wins. The concurrent reader commits a terminal state exactly once, and the first typed source or inner-stream error to reach that commit becomes the result; the fold short-circuits and the operation surfaces as `Left(e)`.
1350
+
1351
+ ```scala
1352
+ import zio.blocks.streams._
1353
+
1354
+ // A typed error anywhere upstream terminates every worker.
1355
+ val fromUpstream: Stream[String, Int] =
1356
+ Stream
1357
+ .range(0, 1000)
1358
+ .flatMap(n => if (n == 500) Stream.fail("bad element") else Stream.succeed(n))
1359
+ .mapPar(4)(identity)
1360
+
1361
+ // A typed error in one inner stream terminates the merge.
1362
+ val fromInner: Stream[String, Int] =
1363
+ Stream.mergeAll(4)(
1364
+ Stream.fromIterable(
1365
+ List(Stream.range(0, 100), Stream.fail("inner error"), Stream.range(200, 300))
1366
+ )
1367
+ )
1368
+ ```
1369
+
1370
+ Both of those collect to a `Left`. Elements that were already emitted stay emitted — ordering is arrival-based, so a consumer may well have seen output from other slots before the failing one reached the commit point.
1371
+
1372
+ Once a failure is committed, cleanup runs and every sibling is torn down. A failure *during* that cleanup is attached to the primary failure rather than substituted for it: the engine calls `StreamError.attachCleanupReplay(primary, secondary)`, so the error a caller sees is still the one that caused the termination, with the cleanup problem carried alongside it. The same rule holds on the successful path, where a cleanup failure with no primary becomes the failure via `StreamError.attachCleanup(null, cleanupFailure)`.
1373
+
1374
+ :::note[What the tests actually prove]
1375
+ `MergeInnerErrorSpec` asserts `result.isLeft` across the generic, `Int`, `Long`, `Float`, and `Double` lanes under a ten-second timeout, and again with four coordinated simultaneous failures per lane under a thirty-second one. That establishes the general shape — a failing inner terminates the merge promptly and does not hang — but it does not pin down *which* failure wins when several race at `n > 1`. Do not write code that depends on a particular one of several concurrent errors being the one reported.
1376
+ :::
1377
+
1378
+ A defect is different from a typed error. The `mapParAsync` callback failing, or an `ensuring` finalizer throwing, is a defect and fails the outer terminal effect rather than appearing in the `E` channel.
1379
+
1380
+ ### Cancellation and Shutdown
1381
+
1382
+ Shutdown is cooperative and ordered, and it is driven from the consumer end. Closing, failing, or cancelling the reader runs the same owned-cleanup path.
1383
+
1384
+ For `mergeAll` and `flatMapPar` that path is three steps, in order: shut the selector down, close every installed inner reader, then close the outer source. The steps are joined rather than sequenced-and-abandoned, so a failure in one does not skip the others — each later close still runs, and its failure is attached to the first. For `mapPar` and `mapParAsync` there are no inner readers, so it is the selector shutdown followed by the upstream close.
1385
+
1386
+ Cancellation goes through the `Async` cancellation protocol rather than thread interruption: the in-flight child run is cancelled with cleanup, and the reader settles its completion from the cancellation's outcome. A cancellation that arrives after the reader is already closed is recognised as stale and does not turn a clean close into a failure.
1387
+
1388
+ Closing is idempotent and it waits. The reader will not report closed until the cleanup it owns has finished, which is the property the slot lifecycle above depends on — a slot's successor cannot start while the predecessor's finalizers are still running.
1389
+
1390
+ [Cancellation](../execution-and-compatibility/async-execution.md#cancellation) covers the protocol these readers participate in, and [Async.Running#cancel](../../async.md#runningcancel) documents the primitive underneath it.
1391
+
1392
+ ### Threading and Platform Behaviour
1393
+
1394
+ Which engine a concurrent operator materializes depends on the kind of reader its upstream compiled to, and the two engines differ far more than the two platforms do.
1395
+
1396
+ | Materialization | JVM reader | Scala.js reader | What actually overlaps |
1397
+ |---------------------------------|-----------------------------------------|---------------------------------|--------------------------------------------|
1398
+ | `mapPar`, synchronous upstream | `ConcurrentMapParReader` family | `Reader.MappedInt` and siblings | one worker thread per slot |
1399
+ | `mapPar`, asynchronous upstream | `AsyncConcurrentReaders.mapPar` | `AsyncConcurrentReaders.mapPar` | nothing; `f` is wrapped in `Async.succeed` |
1400
+ | `mapParAsync`, always | `AsyncConcurrentReaders.mapPar` | `AsyncConcurrentReaders.mapPar` | whatever the `Async` callbacks suspend on |
1401
+ | `mergeAll` from a `Reader` | `AsyncConcurrentReaders.merge` | `AsyncConcurrentReaders.merge` | whatever the inner streams suspend on |
1402
+ | `mergeAll` via the interpreter | `IntConcurrentMergeReader` and siblings | `Reader.FlatMappedRef` | one drainer thread per slot on the JVM |
1403
+
1404
+ Read the second row before choosing an operator. Once the upstream compiles to an `AsyncReader`, `Platform.createMapParReaderFromReader` routes `mapPar` to the shared asynchronous engine with the mapping function wrapped as `Async.succeed(f(a))` — an already-complete effect. The slots and the selector are all still there, but there is nothing for them to overlap, on either platform. `mapParAsync` exists precisely because a callback that returns a real `Async` is the only way to get concurrency out of that lane.
1405
+
1406
+ #### The JVM
1407
+
1408
+ Workers on the synchronous lane run on virtual threads where the runtime provides them. `Platform.startVirtualThread` obtains `Thread.ofVirtual()` reflectively, so the module builds and runs on any supported JDK and uses virtual threads on JDK 21 and later; if the reflective lookup fails for any reason it starts a named daemon platform thread instead.
1409
+
1410
+ The threads are named, which makes them identifiable in a thread dump. `mapPar` names its workers `zio-blocks-mappar-worker-<n>-<index>` and its dispatcher `zio-blocks-mappar-coordinator-<n>`, where `<n>` counts reader instances and `<index>` identifies the worker within one reader; merge uses `zio-blocks-merge-drainer-<n>-<index>` and `zio-blocks-merge-coordinator-<n>` on the same scheme. Those strings are prefixes rather than final names: the virtual-thread builder is created with `Thread.ofVirtual().name(prefix, 0L)`, whose two-argument form appends a counter, so a virtual worker appears in a dump as `zio-blocks-mappar-worker-<n>-<index>0`. Only the platform-thread fallback, which calls `setName` directly, uses the name verbatim.
1411
+
1412
+ #### Scala.js
1413
+
1414
+ There is no parallelism on Scala.js either way — `Platform.supportsConcurrency` is `false` and `Platform.startVirtualThread` throws `UnsupportedOperationException`. But "no parallelism" resolves into two different mechanisms, and only one of them is sequential in the sense the scaladoc suggests.
1415
+
1416
+ When the pipeline compiles end to end to a `SyncReader`, the operator really is sequential: `Platform.createMapParReaderFromReader` builds an ordinary mapped reader (`Reader.MappedIntInt`, `Reader.MappedInt`, `Reader.MappedLong`, and the rest), and the synchronous merge path builds a `Reader.FlatMappedRef` — one that throws `UnsupportedOperationException` if an inner stream turns out to be asynchronous.
1417
+
1418
+ When the pipeline is on the asynchronous lane, the concurrent engine runs, with immediately-ready effects standing in for suspension. The effect is still sequential, but the *semantics* are the concurrent engine's: **unordered arrival is retained**.
1419
+
1420
+ :::warning[The scaladoc is a throughput claim, not an ordering claim]
1421
+ The scaladoc on `Stream#mapPar`, `Stream#flatMapPar`, and `Stream.mergeAll` says that on Scala.js each "degrades to sequential `map`" or "sequential `flatMap`". That describes the synchronous-lane case as though it were the whole story. Never read it as a promise about element order.
1422
+ :::
1423
+
1424
+ [Platform Differences](../execution-and-compatibility/platform-differences.md#concurrency-and-threading) states the same split from the platform side, including the full capability surface of `Platform`.
1425
+
1426
+ ### Laws
1427
+
1428
+ `ConcurrentLawsSpec` asserts three equalities, all of them as set equality (`.toSet == .toSet`), since neither side of any of them is ordered:
1429
+
1430
+ - `Stream.mergeAll(1)(streams)` equals `streams.flatMap(identity)`
1431
+ - `stream.mapPar(1)(f)` equals `stream.map(f)`
1432
+ - `stream.flatMapPar(n)(f)` equals `Stream.mergeAll(n)(stream.map(f))`
1433
+
1434
+ The first two additionally hold as exact equality, and not because the engine happens to preserve order — at `n = 1` the operator body *returns the sequential operator itself*, so the two sides are the same description. The third holds only as a set equality: both sides run the concurrent engine at width `n`, so both are in arrival order and neither has an input order to compare against.
1435
+
1436
+ The third law is also a statement about the implementation rather than a coincidence, since `flatMapPar` is defined as that right-hand side. Reading it as "there is one fan-in engine" is more useful than reading it as a property that had to be checked.
1437
+
1438
+ ### Bounded Concurrency Examples
1439
+
1440
+ #### `mapPar`, `mergeAll`, and `flatMapPar` on the JVM
1441
+
1442
+ The three snippets below use blocking terminals, which exist only on the JVM. In cross-platform code, swap the terminal for its `*Async` twin — `runCollectAsync`, `runFoldAsync` — and the operator itself is unchanged.
1443
+
1444
+ Expensive per-element work across eight slots:
1445
+
1446
+ ```scala
1447
+ import zio.blocks.streams._
1448
+
1449
+ val doubled = Stream
1450
+ .range(0, 1000)
1451
+ .mapPar(8) { n =>
1452
+ Thread.sleep(1)
1453
+ n * 2
1454
+ }
1455
+ .runCollect
1456
+ ```
1457
+
1458
+ `doubled` is a `Right` holding all thousand elements, in arrival order rather than `0, 2, 4, …`.
1459
+
1460
+ Ten sources drained four at a time:
1461
+
1462
+ ```scala
1463
+ import zio.blocks.streams._
1464
+
1465
+ val sources = Stream.fromIterable((0 until 10).map(i => Stream.range(i * 100, (i + 1) * 100)))
1466
+ val summed = Stream.mergeAll(4)(sources).runFold(0L)(_ + _)
1467
+ ```
1468
+
1469
+ A sum is order-insensitive, which is what makes it a safe thing to compute over an unordered merge: `summed` is `Right(499500)` on every run.
1470
+
1471
+ One sub-stream per element, eight drained at a time:
1472
+
1473
+ ```scala
1474
+ import zio.blocks.streams._
1475
+
1476
+ val flattened = Stream
1477
+ .range(0, 50)
1478
+ .flatMapPar(8)(i => Stream.range(i * 20, (i + 1) * 20))
1479
+ .runFold(0L)(_ + _)
1480
+ ```
1481
+
1482
+ Again `Right(499500)`: the same 1000 integers, reached through 50 inner streams instead of 10.
1483
+
1484
+ #### A Worked `mapParAsync`
1485
+
1486
+ The two runnable files below live in the `streams-examples` module. This one makes arrival order visible rather than asserting it: every callback hands back an unresolved `Completer`, and a driver thread then settles the four of them in reverse. The collected chunk comes back reversed with respect to the input.
1487
+
1488
+ ```scala title="streams-examples/src/main/scala/stream/MapParAsyncExample.scala"
1489
+ /*
1490
+ * Copyright 2024-2026 John A. De Goes and the ZIO Contributors
1491
+ *
1492
+ * Licensed under the Apache License, Version 2.0 (the "License");
1493
+ * you may not use this file except in compliance with the License.
1494
+ */
1495
+ package stream
1496
+
1497
+ import java.util.concurrent.{ConcurrentHashMap, CountDownLatch}
1498
+
1499
+ import zio.blocks.async._
1500
+ import zio.blocks.chunk.Chunk
1501
+ import zio.blocks.streams.Stream
1502
+
1503
+ /**
1504
+ * `mapParAsync` keeps at most `n` `Async` callbacks in flight and emits each
1505
+ * result the moment it completes. Output is therefore in arrival order, not
1506
+ * input order.
1507
+ *
1508
+ * This example makes that visible rather than asserting it. Each callback hands
1509
+ * back a `Completer` instead of a value, so nothing completes on its own; a
1510
+ * driver thread then settles the four callbacks in reverse order. The collected
1511
+ * chunk comes back reversed with respect to the input.
1512
+ */
1513
+ object MapParAsyncExample {
1514
+ def main(args: Array[String]): Unit = {
1515
+ val inputs = Chunk(1, 2, 3, 4)
1516
+
1517
+ // Every callback registers its completer and reports for duty. Nothing
1518
+ // resolves until the driver thread below decides that it should.
1519
+ val pending = new ConcurrentHashMap[Int, Completer[Int]]
1520
+ val allInFlight = new CountDownLatch(inputs.length)
1521
+
1522
+ val arrivals: Async[Either[Nothing, Chunk[Int]]] =
1523
+ Stream
1524
+ .fromChunk(inputs)
1525
+ .mapParAsync(4) { value =>
1526
+ val completer = new Completer[Int]
1527
+ pending.put(value, completer)
1528
+ allInFlight.countDown()
1529
+ completer
1530
+ }
1531
+ .runCollectAsync
1532
+
1533
+ // n = 4 and there are four elements, so all four callbacks are admitted
1534
+ // before any of them completes. The driver settles 4 first and 1 last.
1535
+ val driver = new Thread(() => {
1536
+ allInFlight.await()
1537
+ List(4, 3, 2, 1).foreach { value =>
1538
+ pending.get(value).succeed(value * 10)
1539
+ Thread.sleep(100L)
1540
+ }
1541
+ })
1542
+ driver.setDaemon(true)
1543
+ driver.start()
1544
+
1545
+ // `.block` is JVM-only; on Scala.js drive the Async with map/flatMap.
1546
+ val collected = arrivals.block
1547
+
1548
+ println(s"input order: ${inputs.map(_ * 10)}")
1549
+ println(s"arrival order: $collected")
1550
+
1551
+ // The elements are all there...
1552
+ require(
1553
+ collected.map(_.toSet) == Right(Set(10, 20, 30, 40)),
1554
+ s"unexpected elements: $collected"
1555
+ )
1556
+ // ...but not in the order they went in.
1557
+ require(
1558
+ collected != Right(inputs.map(_ * 10)),
1559
+ "mapParAsync emitted input order; arrival order was expected"
1560
+ )
1561
+ }
1562
+ }
1563
+ ```
1564
+
1565
+ Run it with:
1566
+
1567
+ ```bash
1568
+ sbt "streams-examples/runMain stream.MapParAsyncExample"
1569
+ ```
1570
+
1571
+ It prints the two orders side by side:
1572
+
1573
+ ```
1574
+ input order: Chunk(10,20,30,40)
1575
+ arrival order: Right(Chunk(40,30,20,10))
1576
+ ```
1577
+
1578
+ #### Async Children and Slot Accounting
1579
+
1580
+ `flatMapPar(n)(a => Stream.unwrap(f(a)))` is the idiom for asynchronously produced children, and this example pins down what a slot covers. A gauge is incremented when child construction starts and decremented by the child's finalizer, so it counts exactly the elements occupying a slot; a `CyclicBarrier(2)` forces each construction to wait for a partner, so the program can only finish if two really are in flight at once.
1581
+
1582
+ ```scala title="streams-examples/src/main/scala/stream/FlatMapParAsyncChildrenExample.scala"
1583
+ /*
1584
+ * Copyright 2024-2026 John A. De Goes and the ZIO Contributors
1585
+ *
1586
+ * Licensed under the Apache License, Version 2.0 (the "License");
1587
+ * you may not use this file except in compliance with the License.
1588
+ */
1589
+ package stream
1590
+
1591
+ import java.util.concurrent.CyclicBarrier
1592
+ import java.util.concurrent.atomic.AtomicInteger
1593
+
1594
+ import zio.blocks.async._
1595
+ import zio.blocks.streams.Stream
1596
+
1597
+ /**
1598
+ * `flatMapPar(n)(a => Stream.unwrap(f(a)))` is the idiom for asynchronously
1599
+ * produced children. The point of this example is the slot accounting: the
1600
+ * effect that *builds* the child and the child that is then *drained* share one
1601
+ * of the `n` slots, so a slot is occupied from the moment construction starts
1602
+ * until the child has finished closing.
1603
+ *
1604
+ * `inFlight` is incremented when construction starts and decremented by the
1605
+ * child's finalizer, so it counts exactly the elements occupying a slot. With
1606
+ * `n = 2` its peak is 2 and never 4, even though four outer elements are
1607
+ * available immediately: the engine re-arms the outer source only while
1608
+ * `active < n`, so it will not pull ahead to start a third construction.
1609
+ *
1610
+ * The `CyclicBarrier(2)` proves the lower bound from the other side. Each
1611
+ * construction waits for a partner, so the run can only finish if two
1612
+ * constructions really are in flight at once — and it pairs off exactly twice.
1613
+ */
1614
+ object FlatMapParAsyncChildrenExample {
1615
+ private val inFlight = new AtomicInteger
1616
+ private val peak = new AtomicInteger
1617
+ private val pairUp = new CyclicBarrier(2)
1618
+
1619
+ /**
1620
+ * Produces a value on a thread of its own. Blocking inside an `Async` would
1621
+ * stall the driver that is running the stream, so the barrier wait has to
1622
+ * happen somewhere else.
1623
+ */
1624
+ private def deferred[A](value: => A): Async[A] = {
1625
+ val completer = new Completer[A]
1626
+ val thread = new Thread(() => completer.succeed(value))
1627
+ thread.setDaemon(true)
1628
+ thread.start()
1629
+ completer
1630
+ }
1631
+
1632
+ /** The asynchronously produced child for one outer element. */
1633
+ private def child(value: Int): Async[Stream[Nothing, Int]] =
1634
+ deferred {
1635
+ val current = inFlight.incrementAndGet()
1636
+ peak.getAndUpdate(seen => if (current > seen) current else seen)
1637
+ pairUp.await()
1638
+ Stream(value * 10, value * 10 + 1).ensuring {
1639
+ inFlight.decrementAndGet()
1640
+ ()
1641
+ }
1642
+ }
1643
+
1644
+ def main(args: Array[String]): Unit = {
1645
+ // `.block` is JVM-only; on Scala.js drive the Async with map/flatMap.
1646
+ val collected = Stream(1, 2, 3, 4)
1647
+ .flatMapPar(2)(value => Stream.unwrap(child(value)))
1648
+ .runCollectAsync
1649
+ .block
1650
+
1651
+ println(s"elements: ${collected.map(_.toSet.toList.sorted)}")
1652
+ println(s"peak slot occupancy: ${peak.get()} of 2")
1653
+
1654
+ require(
1655
+ collected.map(_.toSet) == Right(Set(10, 11, 20, 21, 30, 31, 40, 41)),
1656
+ s"unexpected elements: $collected"
1657
+ )
1658
+ // Two slots, so at most two elements are ever being constructed or drained.
1659
+ require(peak.get() == 2, s"peak occupancy was ${peak.get()}, expected 2")
1660
+ // Every slot was released, which means every child was closed.
1661
+ require(inFlight.get() == 0, s"${inFlight.get()} children were left open")
1662
+ }
1663
+ }
1664
+ ```
1665
+
1666
+ Run it with:
1667
+
1668
+ ```bash
1669
+ sbt "streams-examples/runMain stream.FlatMapParAsyncChildrenExample"
1670
+ ```
1671
+
1672
+ ```
1673
+ elements: Right(List(10, 11, 20, 21, 30, 31, 40, 41))
1674
+ peak slot occupancy: 2 of 2
1675
+ ```
1676
+
1677
+ Peak occupancy is 2 and not 4, even though all four outer elements are available immediately, because the construction effect holds the slot its child will use. For sequential asynchronous children, `flatMap(a => Stream.unwrap(f(a)))` is the operator you want instead.
1678
+
1679
+ ### Comparison with Other Libraries
1680
+
1681
+ The concurrent surface is small, and what distinguishes it is less the operator names than what a caller has to bring along to use them.
1682
+
1683
+ | Feature | ZIO Blocks Streams | fs2 | Kyo | Ox | Pekko |
1684
+ |----------------------|------------------------------------|--------------------|-------------------|-----------------------|----------------------------|
1685
+ | Concurrent operators | `mapPar`, `mergeAll`, `flatMapPar` | `parEvalMap` | `mapParUnordered` | `mapPar` | `mapAsync`, `flatMapMerge` |
1686
+ | Effect system | none required | cats-effect | Kyo | none; virtual threads | Akka |
1687
+ | Typed errors | `Either[E, Z]` | `ApplicativeError` | Kyo effects | exceptions | none |
1688
+
1689
+ The table compares contracts, not speed. Cross-provider throughput rankings require the specific benchmark classes that were built to compare like with like, and none of the numbers from those runs belong in a row next to a feature name.
1690
+
1691
+ ## Other Operations
1692
+
1693
+ Common utilities for deduplication, draining, and error recovery:
1694
+
1695
+ ### Filtering Duplicates
1696
+
1697
+ These operations remove duplicate elements, useful for deduplicating streams before processing:
1698
+
1699
+ #### `Stream#distinct[A]`
1700
+
1701
+ Emits only unique elements (using a mutable `HashSet` internally):
1702
+
1703
+ ```scala
1704
+ abstract class Stream[+E, +A] {
1705
+ def distinct: Stream[E, A]
1706
+ }
1707
+ ```
1708
+
1709
+ This consumes memory proportional to the number of unique elements:
1710
+
1711
+ ```scala
1712
+ import zio.blocks.streams.*
1713
+
1714
+ val nums = Stream(1, 2, 2, 3, 3, 3)
1715
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 2, 3, 3, ...)
1716
+ val unique = nums.distinct
1717
+ // unique: Stream[Nothing, Int] = Stream.suspend(...)
1718
+ val result = unique.runCollect
1719
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 3))
1720
+ ```
1721
+
1722
+ #### `Stream#distinctBy[K]`
1723
+
1724
+ Emits only elements whose key (computed by `f`) has not been seen before:
1725
+
1726
+ ```scala
1727
+ abstract class Stream[+E, +A] {
1728
+ def distinctBy[K](f: A => K): Stream[E, A]
1729
+ }
1730
+ ```
1731
+
1732
+ This deduplicates elements by a computed key, keeping only the first occurrence of each key:
1733
+
1734
+ ```scala
1735
+ import zio.blocks.streams.*
1736
+
1737
+ case class Person(id: Int, name: String)
1738
+
1739
+ val people = Stream(
1740
+ Person(1, "Alice"),
1741
+ Person(2, "Bob"),
1742
+ Person(1, "Alice2"), // same id as first, dropped
1743
+ Person(3, "Charlie")
1744
+ )
1745
+
1746
+ val unique = people.distinctBy(_.id)
1747
+ val result = unique.runCollect
1748
+ ```
1749
+
1750
+ ### Skipping and Taking
1751
+
1752
+ These operations skip or limit elements, allowing you to keep or drop unwanted portions of the stream:
1753
+
1754
+ #### `Stream#drop`
1755
+
1756
+ Skips the first `n` elements:
1757
+
1758
+ ```scala
1759
+ abstract class Stream[+E, +A] {
1760
+ def drop(n: Long): Stream[E, A]
1761
+ }
1762
+ ```
1763
+
1764
+ Dropping the first 3 elements and collecting the remainder:
1765
+
1766
+ ```scala
1767
+ import zio.blocks.streams.*
1768
+
1769
+ val nums = Stream(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
1770
+ val remaining = nums.drop(3)
1771
+ val result = remaining.runCollect
1772
+ ```
1773
+
1774
+ #### `Stream#take`
1775
+
1776
+ Emits at most the first `n` elements, then stops:
1777
+
1778
+ ```scala
1779
+ abstract class Stream[+E, +A] {
1780
+ def take(n: Long): Stream[E, A]
1781
+ }
1782
+ ```
1783
+
1784
+ This naturally short-circuits: the stream stops pulling from upstream:
1785
+
1786
+ ```scala
1787
+ import zio.blocks.streams.*
1788
+
1789
+ val nums = Stream.range(0, 1000)
1790
+ // nums: Stream[Nothing, Int] = Stream.range(0, 1000)
1791
+ val first10 = nums.take(10)
1792
+ // first10: Stream[Nothing, Int] = Stream.range(0, 1000).take(10)
1793
+ val result = first10.runCollect
1794
+ // result: Either[Nothing, Chunk[Int]] = Right(
1795
+ // IndexedSeq(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
1796
+ // )
1797
+ ```
1798
+
1799
+ #### `Stream#takeWhile`
1800
+
1801
+ Emits elements while a predicate is true, then stops:
1802
+
1803
+ ```scala
1804
+ abstract class Stream[+E, +A] {
1805
+ def takeWhile(pred: A => Boolean): Stream[E, A]
1806
+ }
1807
+ ```
1808
+
1809
+ Taking elements while they are less than 6 stops early without processing the rest:
1810
+
1811
+ ```scala
1812
+ import zio.blocks.streams.*
1813
+
1814
+ val nums = Stream(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
1815
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5, ...)
1816
+ val firstFive = nums.takeWhile(_ < 6)
1817
+ // firstFive: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5, ...).takeWhile(...)
1818
+ val result = firstFive.runCollect
1819
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 3, 4, 5))
1820
+ ```
1821
+
1822
+ #### `Stream#takeWhileAsync`
1823
+
1824
+ The asynchronous twin of `takeWhile`, for a predicate that has to await something before it can answer:
1825
+
1826
+ ```scala
1827
+ abstract class Stream[+E, +A] {
1828
+ def takeWhileAsync(pred: A => Async[Boolean]): Stream[E, A]
1829
+ }
1830
+ ```
1831
+
1832
+ Elements are tested sequentially, and the first `false` closes upstream — so the element that failed the test is not emitted, and nothing beyond it is pulled. Predicate failure is a defect. No `JvmType.Infer` evidence appears here because the element type does not change.
1833
+
1834
+ ```scala
1835
+ import zio.blocks.async.*
1836
+ import zio.blocks.streams.*
1837
+
1838
+ val feed = Stream(120, -40, 75, 0, 999)
1839
+ val untilZero = feed.takeWhileAsync(amount => Async.succeed(amount != 0))
1840
+ ```
1841
+
1842
+ The same entry appears in [Async Operators](../execution-and-compatibility/async-execution.md#streamtakewhileasync).
1843
+
1844
+ ### Interspersing
1845
+
1846
+ `intersperse[A2, A3]` — Inserts a separator value between every two elements.:
1847
+
1848
+ ```scala
1849
+ abstract class Stream[+E, +A] {
1850
+ def intersperse[A2, A3](sep: A2)(implicit
1851
+ valueConcat: Concat.WithOut[A, A2, A3],
1852
+ jtA3: JvmType.Infer[A3]
1853
+ ): Stream[E, A3]
1854
+ }
1855
+ ```
1856
+
1857
+ This is useful for rendering comma-separated lists or row delimiters:
1858
+
1859
+ ```scala
1860
+ import zio.blocks.streams.*
1861
+
1862
+ val items = Stream("a", "b", "c")
1863
+ // items: Stream[Nothing, String] = Stream(a, b, c)
1864
+ val separated = items.intersperse(", ")
1865
+ // separated: Stream[Nothing, String] = Stream(a, b, c).intersperse(...)
1866
+ val result = separated.runCollect
1867
+ // result: Either[Nothing, Chunk[String]] = Right(
1868
+ // IndexedSeq("a", ", ", "b", ", ", "c")
1869
+ // )
1870
+ ```
1871
+
1872
+ ### Repeating
1873
+
1874
+ `repeated` — Rematerializes the stream after each clean completion, emitting the whole sequence again indefinitely. A typed error or a defect terminates the repetition.:
1875
+
1876
+ ```scala
1877
+ abstract class Stream[+E, +A] {
1878
+ def repeated: Stream[E, A]
1879
+ }
1880
+ ```
1881
+
1882
+ This creates an infinite repetition of the stream:
1883
+
1884
+ ```scala
1885
+ import zio.blocks.streams.*
1886
+
1887
+ val original = Stream(1, 2)
1888
+ // original: Stream[Nothing, Int] = Stream(1, 2)
1889
+ val repeated = original.repeated.take(6)
1890
+ // repeated: Stream[Nothing, Int] = Stream(1, 2).repeated.take(6)
1891
+ val result = repeated.runCollect
1892
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 1, 2, 1, 2))
1893
+ ```
1894
+
1895
+ ### Side Effects
1896
+
1897
+ `tapEach` — Applies a function to each element for side effects, passing the element through unchanged.:
1898
+
1899
+ ```scala
1900
+ abstract class Stream[+E, +A] {
1901
+ def tapEach(f: A => Unit): Stream[E, A]
1902
+ }
1903
+ ```
1904
+
1905
+ Use `tapEach` for logging or metrics:
1906
+
1907
+ ```scala
1908
+ import zio.blocks.streams.*
1909
+
1910
+ val nums = Stream(1, 2, 3)
1911
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3)
1912
+ val logged = nums.tapEach(x => println(s"Element: $x"))
1913
+ // logged: Stream[Nothing, Int] = Stream(1, 2, 3).filter(...)
1914
+ val result = logged.runCollect
1915
+ // Element: 1
1916
+ // Element: 2
1917
+ // Element: 3
1918
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 3))
1919
+ ```
1920
+
1921
+ ## Error Handling
1922
+
1923
+ Streams distinguish between recoverable business errors and unexpected exceptions, providing separate recovery mechanisms for each:
1924
+
1925
+ ### Typed Error Vs Untyped Defect
1926
+
1927
+ ZIO Blocks distinguishes two error channels:
1928
+
1929
+ - **Typed errors (`E`)**: Recoverable business logic errors. Returned as `Left(e)` from terminal operations.
1930
+ - **Untyped defects (`Throwable`)**: Unexpected exceptions (bugs, system failures). Propagate as thrown exceptions.
1931
+
1932
+ Internally, typed errors are wrapped in `StreamError` (a non-fatal exception) and caught by the terminal operation to surface as `Left(e)`. Untyped `Throwable`s are not caught and propagate upward.
1933
+
1934
+ This separation allows you to:
1935
+ - Use `catchAll` and `orElse` for business logic errors
1936
+ - Use `catchDefect` or try-catch for unexpected exceptions
1937
+ - Avoid accidentally silencing real bugs by catching all errors
1938
+
1939
+ Streams distinguish between recoverable domain errors and fatal defects, with flexible recovery patterns:
1940
+
1941
+ ### Recovering From Typed Errors
1942
+
1943
+ These operations handle typed errors gracefully by recovering with alternative streams:
1944
+
1945
+ #### `Stream#catchAll[E2, A2, A3]`
1946
+
1947
+ Recovers from any typed error by switching to a recovery stream:
1948
+
1949
+ ```scala
1950
+ abstract class Stream[+E, +A] {
1951
+ def catchAll[E2, A2, A3](f: E => Stream[E2, A2])(implicit
1952
+ valueConcat: Concat.WithOut[A, A2, A3],
1953
+ jtA3: JvmType.Infer[A3]
1954
+ ): Stream[E2, A3]
1955
+ }
1956
+ ```
1957
+
1958
+ The recovery function receives the error and can return a new stream:
1959
+
1960
+ ```scala
1961
+ import zio.blocks.streams.*
1962
+
1963
+ sealed trait Error
1964
+ case object NotFound extends Error
1965
+
1966
+ val mayFail: Stream[Error, String] = Stream.fail(NotFound)
1967
+ // mayFail: Stream[Error, String] = Stream.fail(...)
1968
+ val recovered = mayFail.catchAll(_ => Stream.succeed("default"))
1969
+ // recovered: Stream[Nothing, String] = Stream.fail(...).catchAll(...)
1970
+ val result = recovered.runCollect
1971
+ // result: Either[Nothing, Chunk[String]] = Right(IndexedSeq("default"))
1972
+ ```
1973
+
1974
+ #### `Stream#orElse[E2, A2, A3]`
1975
+
1976
+ If this stream fails, tries the fallback stream. The fallback is evaluated lazily, only on error:
1977
+
1978
+ ```scala
1979
+ abstract class Stream[+E, +A] {
1980
+ def orElse[E2, A2, A3](that: => Stream[E2, A2])(implicit
1981
+ valueConcat: Concat.WithOut[A, A2, A3],
1982
+ jtA3: JvmType.Infer[A3]
1983
+ ): Stream[E2, A3]
1984
+ }
1985
+ ```
1986
+
1987
+ `||` is an alias for `orElse`:
1988
+
1989
+ ```scala
1990
+ import zio.blocks.streams._
1991
+
1992
+ val primary = Stream.fail("error")
1993
+ // primary: Stream[String, Nothing] = Stream.fail(...)
1994
+ val fallback = Stream.succeed(42)
1995
+ // fallback: Stream[Nothing, Int] = Stream.succeed(...)
1996
+ val result = (primary || fallback).runCollect
1997
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(42))
1998
+ ```
1999
+
2000
+ ### Recovering From Defects
2001
+
2002
+ `catchDefect[E2, E3, A2, A3]` — Catches untyped defects (exceptions not wrapped as typed errors) using a partial function.:
2003
+
2004
+ ```scala
2005
+ abstract class Stream[+E, +A] {
2006
+ def catchDefect[E2, E3, A2, A3](
2007
+ f: PartialFunction[Throwable, Stream[E2, A2]]
2008
+ )(implicit
2009
+ errorConcat: Concat.WithOut[E, E2, E3],
2010
+ valueConcat: Concat.WithOut[A, A2, A3],
2011
+ jtA3: JvmType.Infer[A3]
2012
+ ): Stream[E3, A3]
2013
+ }
2014
+ ```
2015
+
2016
+ Use `catchDefect` when you need to handle unexpected exceptions that were not wrapped by `attempt`:
2017
+
2018
+ ```scala
2019
+ import zio.blocks.streams.*
2020
+
2021
+ val risky = Stream.die(new IllegalArgumentException("Not allowed"))
2022
+ val safe = risky.catchDefect {
2023
+ case e: IllegalArgumentException => Stream.succeed(-1)
2024
+ }
2025
+ val result = safe.runCollect
2026
+ ```
2027
+
2028
+ ## Resource Management
2029
+
2030
+ **The Problem:** Resources like files, database connections, and network sockets must be explicitly closed after use. If you just process them in a stream and forget to close, you leak resources. If an error occurs during processing, manual cleanup code might be skipped.
2031
+
2032
+ **The Solution:** ZIO Blocks streams provide three patterns for safe, automatic resource cleanup:
2033
+
2034
+ ### `Stream.fromAcquireRelease[R, E, A]`
2035
+
2036
+ Acquires a resource, uses it in a stream, and **guarantees cleanup regardless of success or failure**:
2037
+
2038
+ ```scala
2039
+ object Stream {
2040
+ def fromAcquireRelease[R, E, A](
2041
+ acquire: => R, // How to open the resource
2042
+ release: R => Unit = (r: R) => // How to close it (defaults to .close())
2043
+ r match {
2044
+ case ac: AutoCloseable => ac.close()
2045
+ case _ => ()
2046
+ }
2047
+ )(use: R => Stream[E, A]): Stream[E, A]
2048
+ }
2049
+ ```
2050
+
2051
+ This is the fundamental pattern for safe resource handling:
2052
+ 1. **Acquire** — opens the resource (runs once, before streaming)
2053
+ 2. **Use** — streams elements from the resource
2054
+ 3. **Release** — closes the resource in a `finally` block (always runs, even on error)
2055
+
2056
+ Here's an example with automatic cleanup:
2057
+
2058
+ ```scala
2059
+ import zio.blocks.streams.*
2060
+
2061
+ case class DatabaseConnection(id: String) {
2062
+ def close(): Unit = println(s"Closing connection $id")
2063
+ def query(q: String): List[String] = List("result1", "result2")
2064
+ }
2065
+
2066
+ val managed = Stream.fromAcquireRelease(
2067
+ acquire = {
2068
+ println("Opening database connection")
2069
+ DatabaseConnection("db-1")
2070
+ },
2071
+ release = _.close() // Guaranteed to run even if streaming fails
2072
+ )(conn => Stream.fromIterable(conn.query("SELECT *")))
2073
+
2074
+ val result = managed.runCollect
2075
+ // Output:
2076
+ // Opening database connection
2077
+ // Closing database connection <-- always happens
2078
+ ```
2079
+
2080
+ Even if the stream fails, cleanup runs:
2081
+
2082
+ ```scala
2083
+ import zio.blocks.streams.*
2084
+
2085
+ val managed = Stream.fromAcquireRelease(
2086
+ acquire = { println("Opening"); "resource" },
2087
+ release = { r => println(s"Closing $r") }
2088
+ )(_ => Stream.fail("error occurred"))
2089
+
2090
+ val result = managed.runCollect
2091
+ // Output:
2092
+ // Opening
2093
+ // Closing resource <-- cleanup still runs even with error
2094
+ // result: Either[String, Chunk[Nothing]] = Left("error occurred")
2095
+ ```
2096
+
2097
+ ### `Stream.fromResource[R, E, A]`
2098
+
2099
+ Uses a ZIO Blocks `Resource[R]` (more abstract, composable resource type) within a stream:
2100
+
2101
+ ```scala
2102
+ object Stream {
2103
+ def fromResource[R, E, A](resource: Resource[R])(use: R => Stream[E, A]): Stream[E, A]
2104
+ }
2105
+ ```
2106
+
2107
+ Use `fromResource` when you already have a `Resource` value, or when you need resource composition. The resource is acquired at stream start and released when the stream terminates:
2108
+
2109
+ ```scala
2110
+ import zio.blocks.streams.*
2111
+ import zio.blocks.scope.Resource
2112
+
2113
+ val resource = Resource.acquireRelease(acquire = {
2114
+ println("Acquiring resource")
2115
+ 42
2116
+ })(release = { value =>
2117
+ println(s"Releasing resource with value: $value")
2118
+ })
2119
+
2120
+ val stream = Stream.fromResource(resource) { value =>
2121
+ Stream(value, value * 2, value * 3)
2122
+ }
2123
+
2124
+ val result = stream.runCollect
2125
+ ```
2126
+
2127
+ ### `Stream#ensuring`
2128
+
2129
+ Adds a **cleanup action to any stream**, regardless of how it is created. The finalizer runs in a `finally` block:
2130
+
2131
+ ```scala
2132
+ abstract class Stream[+E, +A] {
2133
+ def ensuring(finalizer: => Unit): Stream[E, A]
2134
+ }
2135
+ ```
2136
+
2137
+ Use `ensuring` for simple cleanup tasks that don't fit the acquire-release pattern:
2138
+
2139
+ ```scala
2140
+ import zio.blocks.streams.*
2141
+
2142
+ val stream = Stream(1, 2, 3)
2143
+ .ensuring {
2144
+ println("Stream finished (success or error)")
2145
+ }
2146
+
2147
+ val result = stream.runCollect
2148
+ ```
2149
+
2150
+ The finalizer always runs, in a `finally` block:
2151
+
2152
+ ```scala
2153
+ import zio.blocks.streams.*
2154
+
2155
+ val managed = Stream(1, 2, 3)
2156
+ .ensuring { println("Cleaned up") }
2157
+
2158
+ val result = managed.runCollect
2159
+ ```
2160
+
2161
+ ### `Stream#ensuringAsync`
2162
+
2163
+ The asynchronous twin of `ensuring`, for cleanup that is itself an `Async` — closing a socket, flushing a remote session, releasing a lease:
2164
+
2165
+ ```scala
2166
+ abstract class Stream[+E, +A] {
2167
+ def ensuringAsync(finalizer: => Async[Unit]): Stream[E, A]
2168
+ }
2169
+ ```
2170
+
2171
+ The finalizer is registered lazily and awaited exactly once when the materialized stream closes: on normal completion, on failure, on early termination such as `take` or `takeWhileAsync` cutting the stream short, and on cancellation. Awaited, not merely started — the close does not complete until the finalizer does. A failure inside the finalizer is a defect, and when the stream had already failed, that defect is attached to the primary failure rather than replacing it.
2172
+
2173
+ ```scala
2174
+ import zio.blocks.async.*
2175
+ import zio.blocks.streams.*
2176
+
2177
+ val session = Stream(1, 2, 3)
2178
+ .ensuringAsync(Async.succeed(println("session closed")))
2179
+
2180
+ val closed = session.runCollectAsync
2181
+ ```
2182
+
2183
+ What drives that close is the terminal. Under `runCollectAsync` and every other terminal the library closes the reader for you, and so does `useReaderAsync`; under `startAsync` you own the reader, and the finalizer has not run until you await `close()`. [Resource Management](../execution-and-compatibility/async-execution.md#resource-management) works through the three cases.
2184
+
2185
+ ## Running Streams
2186
+
2187
+ There are two terminal families, and which one you reach for is a platform decision rather than a stylistic one.
2188
+
2189
+ The **cross-platform** family is the one whose names end in `Async`: `runAsync`, `runCollectAsync`, `runDrainAsync`, `runFoldAsync`, `runForeachAsync`/`foreachAsync`, `countAsync`, `existsAsync`, `findAsync`, `forallAsync`, `headAsync`, and `lastAsync`. Each returns `Async[Either[E, Z]]`, stays lazy until driven, and awaits reader cleanup on success, failure, or cancellation. It drives a synchronous and an asynchronous pipeline alike, and it is the only family that compiles for both targets. [Async Terminals](../execution-and-compatibility/async-execution.md#async-terminals) documents the family and the `Async[Either[E, Z]]` convention it follows.
2190
+
2191
+ The **JVM-only** family is everything documented in the rest of this section — `run`, `runCollect`, `runDrain`, `runFold`, `runForeach`/`foreach`, `count`, `exists`, `find`, `forall`, `head`, and `last` — together with `start`, covered under [Manual Pull via `start`](#manual-pull-via-start). Those sixteen members, counting `runFold`'s four overloads, are the entire JVM-only surface of `Stream`: the terminals among them park a thread and return a plain `Either[E, Z]`, and `start` hands back a blocking `Reader.SyncReader[A]`. They live in `StreamPlatformSpecific`, whose Scala.js copy has an empty body, so calling one from shared code fails to compile for the JavaScript target rather than failing at runtime. The [availability matrix](../execution-and-compatibility/platform-differences.md#availability-matrix) lists them member by member.
2192
+
2193
+ Each heading below therefore carries a one-line note naming its cross-platform form.
2194
+
2195
+ ### Collecting Results
2196
+
2197
+ These operations accumulate or examine stream results, running the entire stream to completion:
2198
+
2199
+ #### `Stream#runCollect`
2200
+
2201
+ JVM only. The cross-platform form is `runCollectAsync`.
2202
+
2203
+ Collects all elements into a `Chunk[A]`:
2204
+
2205
+ ```scala
2206
+ abstract class Stream[+E, +A] {
2207
+ def runCollect: Either[E, Chunk[A]]
2208
+ }
2209
+ ```
2210
+
2211
+ This is the most common terminal operation for extracting results:
2212
+
2213
+ ```scala
2214
+ import zio.blocks.streams.*
2215
+
2216
+ val nums = Stream(1, 2, 3, 4, 5)
2217
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5)
2218
+ val result = nums.runCollect
2219
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2, 3, 4, 5))
2220
+ // result is Right(Chunk(1, 2, 3, 4, 5))
2221
+ ```
2222
+
2223
+ #### `Stream#run[E2, Z]`
2224
+
2225
+ JVM only. The cross-platform form is `runAsync`.
2226
+
2227
+ Runs the stream with a custom sink, producing result `Z`:
2228
+
2229
+ ```scala
2230
+ abstract class Stream[+E, +A] {
2231
+ def run[ES, E3, Z](sink: Sink[ES, A, Z])(implicit
2232
+ errorConcat: Concat.WithOut[E, ES, E3]
2233
+ ): Either[E3, Z]
2234
+ }
2235
+ ```
2236
+
2237
+ Use `run` when you need a specialized sink operation:
2238
+
2239
+ ```scala
2240
+ import zio.blocks.streams.*
2241
+
2242
+ val nums = Stream(1, 2, 3, 4, 5)
2243
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5)
2244
+ val sum = nums.run(Sink.foldLeft(0)((acc, x) => acc + x))
2245
+ // sum: Either[Nothing, Int] = Right(15)
2246
+ // sum is Right(15)
2247
+ ```
2248
+
2249
+ ### Discarding Results
2250
+
2251
+ These operations consume streams without collecting their elements, useful when you only care about side effects:
2252
+
2253
+ #### `Stream#runDrain`
2254
+
2255
+ JVM only. The cross-platform form is `runDrainAsync`.
2256
+
2257
+ Consumes all elements and discards them, returning `Unit`:
2258
+
2259
+ ```scala
2260
+ abstract class Stream[+E, +A] {
2261
+ def runDrain: Either[E, Unit]
2262
+ }
2263
+ ```
2264
+
2265
+ Use `runDrain` when you only care about side effects:
2266
+
2267
+ ```scala
2268
+ import zio.blocks.streams.*
2269
+
2270
+ val nums = Stream(1, 2, 3)
2271
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3)
2272
+ val sideEffect = nums.tapEach(x => println(s"Processing $x"))
2273
+ // sideEffect: Stream[Nothing, Int] = Stream(1, 2, 3).filter(...)
2274
+ val result = sideEffect.runDrain
2275
+ // Processing 1
2276
+ // Processing 2
2277
+ // Processing 3
2278
+ // result: Either[Nothing, Unit] = Right(())
2279
+ ```
2280
+
2281
+ #### `Stream#runForeach`
2282
+
2283
+ JVM only. The cross-platform forms are `runForeachAsync` and its alias `foreachAsync`.
2284
+
2285
+ Applies a function to each element for side effects:
2286
+
2287
+ ```scala
2288
+ abstract class Stream[+E, +A] {
2289
+ def runForeach(f: A => Unit): Either[E, Unit]
2290
+ }
2291
+ ```
2292
+
2293
+ Alias `foreach` also exists:
2294
+
2295
+ ```scala
2296
+ import zio.blocks.streams.*
2297
+
2298
+ val nums = Stream(1, 2, 3)
2299
+ val result = nums.foreach(x => println(s"Got: $x"))
2300
+ ```
2301
+
2302
+ ### Aggregations
2303
+
2304
+ These operations reduce streams to single values, aggregating elements into results:
2305
+
2306
+ #### `Stream#runFold[Z]`
2307
+
2308
+ JVM only. The cross-platform form is `runFoldAsync`.
2309
+
2310
+ Folds all elements using an accumulator, returning the final result:
2311
+
2312
+ ```scala
2313
+ abstract class Stream[+E, +A] {
2314
+ def runFold[Z](z: Z)(f: (Z, A) => Z)(implicit jtZ: JvmType.Infer[Z]): Either[E, Z]
2315
+ }
2316
+ ```
2317
+
2318
+ This is the most general aggregation, equivalent to `reduce` or `fold` on eager sequences:
2319
+
2320
+ ```scala
2321
+ import zio.blocks.streams.*
2322
+
2323
+ val nums = Stream(1, 2, 3, 4)
2324
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4)
2325
+ val sum = nums.runFold(0)(_ + _)
2326
+ // sum: Either[Nothing, Int] = Right(10)
2327
+ ```
2328
+
2329
+ Specialized overloads for primitives avoid boxing:
2330
+
2331
+ ```scala
2332
+ def runFold(z: Int)(f: (Int, A) => Int): Either[E, Int]
2333
+ def runFold(z: Long)(f: (Long, A) => Long): Either[E, Long]
2334
+ def runFold(z: Double)(f: (Double, A) => Double): Either[E, Double]
2335
+ ```
2336
+
2337
+ #### `Stream#count`
2338
+
2339
+ JVM only. The cross-platform form is `countAsync`.
2340
+
2341
+ Returns the number of elements:
2342
+
2343
+ ```scala
2344
+ abstract class Stream[+E, +A] {
2345
+ def count: Either[E, Long]
2346
+ }
2347
+ ```
2348
+
2349
+ Counting elements in a stream:
2350
+
2351
+ ```scala
2352
+ import zio.blocks.streams.*
2353
+
2354
+ val nums = Stream(10, 20, 30, 40, 50)
2355
+ // nums: Stream[Nothing, Int] = Stream(10, 20, 30, 40, 50)
2356
+ val total = nums.count
2357
+ // total: Either[Nothing, Long] = Right(5L)
2358
+ ```
2359
+
2360
+ #### `Stream#head`
2361
+
2362
+ JVM only. The cross-platform form is `headAsync`.
2363
+
2364
+ Returns the first element (or `None` if empty):
2365
+
2366
+ ```scala
2367
+ abstract class Stream[+E, +A] {
2368
+ def head: Either[E, Option[A]]
2369
+ }
2370
+ ```
2371
+
2372
+ Getting the first element without collecting the entire stream:
2373
+
2374
+ ```scala
2375
+ import zio.blocks.streams.*
2376
+
2377
+ val nums = Stream(10, 20, 30, 40, 50)
2378
+ // nums: Stream[Nothing, Int] = Stream(10, 20, 30, 40, 50)
2379
+ val first = nums.head
2380
+ // first: Either[Nothing, Option[Int]] = Right(Some(10))
2381
+ ```
2382
+
2383
+ #### `Stream#last`
2384
+
2385
+ JVM only. The cross-platform form is `lastAsync`.
2386
+
2387
+ Returns the last element (or `None` if empty):
2388
+
2389
+ ```scala
2390
+ abstract class Stream[+E, +A] {
2391
+ def last: Either[E, Option[A]]
2392
+ }
2393
+ ```
2394
+
2395
+ Getting the final element of the stream:
2396
+
2397
+ ```scala
2398
+ import zio.blocks.streams.*
2399
+
2400
+ val nums = Stream(10, 20, 30, 40, 50)
2401
+ // nums: Stream[Nothing, Int] = Stream(10, 20, 30, 40, 50)
2402
+ val last = nums.last
2403
+ // last: Either[Nothing, Option[Int]] = Right(Some(50))
2404
+ ```
2405
+
2406
+ #### `Stream#find[A]`
2407
+
2408
+ JVM only. The cross-platform form is `findAsync`.
2409
+
2410
+ Returns the first element satisfying a predicate:
2411
+
2412
+ ```scala
2413
+ abstract class Stream[+E, +A] {
2414
+ def find(pred: A => Boolean): Either[E, Option[A]]
2415
+ }
2416
+ ```
2417
+
2418
+ Finding the first element matching a condition:
2419
+
2420
+ ```scala
2421
+ import zio.blocks.streams.*
2422
+
2423
+ val nums = Stream(10, 20, 30, 40, 50)
2424
+ // nums: Stream[Nothing, Int] = Stream(10, 20, 30, 40, 50)
2425
+ val firstEven = nums.find(_ % 2 == 0)
2426
+ // firstEven: Either[Nothing, Option[Int]] = Right(Some(10))
2427
+ ```
2428
+
2429
+ #### `Stream#exists[A]`
2430
+
2431
+ JVM only. The cross-platform form is `existsAsync`.
2432
+
2433
+ Returns `true` if any element satisfies a predicate, short-circuiting:
2434
+
2435
+ ```scala
2436
+ abstract class Stream[+E, +A] {
2437
+ def exists(pred: A => Boolean): Either[E, Boolean]
2438
+ }
2439
+ ```
2440
+
2441
+ Checking if any element is greater than 35:
2442
+
2443
+ ```scala
2444
+ import zio.blocks.streams.*
2445
+
2446
+ val nums = Stream(10, 20, 30, 40, 50)
2447
+ // nums: Stream[Nothing, Int] = Stream(10, 20, 30, 40, 50)
2448
+ val hasLargeValue = nums.exists(_ > 35)
2449
+ // hasLargeValue: Either[Nothing, Boolean] = Right(true)
2450
+ ```
2451
+
2452
+ #### `Stream#forall[A]`
2453
+
2454
+ JVM only. The cross-platform form is `forallAsync`.
2455
+
2456
+ Returns `true` if all elements satisfy a predicate, short-circuiting:
2457
+
2458
+ ```scala
2459
+ abstract class Stream[+E, +A] {
2460
+ def forall(pred: A => Boolean): Either[E, Boolean]
2461
+ }
2462
+ ```
2463
+
2464
+ Checking if all elements are positive:
2465
+
2466
+ ```scala
2467
+ import zio.blocks.streams.*
2468
+
2469
+ val nums = Stream(10, 20, 30, 40, 50)
2470
+ // nums: Stream[Nothing, Int] = Stream(10, 20, 30, 40, 50)
2471
+ val allPositive = nums.forall(_ > 0)
2472
+ // allPositive: Either[Nothing, Boolean] = Right(true)
2473
+ ```
2474
+
2475
+ ## Integration with Pipeline and Sink
2476
+
2477
+ Streams compose with pipelines and sinks to form complete data processing flows:
2478
+
2479
+ ### Using Pipelines
2480
+
2481
+ `via[B]` — Applies a `Pipeline[A, B]` transformation to the stream.:
2482
+
2483
+ ```scala
2484
+ abstract class Stream[+E, +A] {
2485
+ final def via[B](pipe: Pipeline[A, B]): Stream[E, B]
2486
+ }
2487
+ ```
2488
+
2489
+ Pipelines are composable transformations that can be reused across streams and sinks. Common pipelines include `Pipeline.map`, `Pipeline.filter`, `Pipeline.take`, and `Pipeline.drop`:
2490
+
2491
+ ```scala
2492
+ import zio.blocks.streams.*
2493
+
2494
+ val nums = Stream(1, 2, 3, 4, 5)
2495
+ // nums: Stream[Nothing, Int] = Stream(1, 2, 3, 4, 5)
2496
+ val pipe = Pipeline.filter((x: Int) => x > 2).andThen(Pipeline.map((x: Int) => x * 10))
2497
+ // pipe: Pipeline[Int, Int] = zio.blocks.streams.Pipeline$Composed@34830fae
2498
+ val result = nums.via(pipe).runCollect
2499
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(30, 40, 50))
2500
+ ```
2501
+
2502
+ Pipelines are useful when you want to build reusable transformation logic:
2503
+
2504
+ ```scala
2505
+ import zio.blocks.streams.*
2506
+
2507
+ def positiveIntsPipe: Pipeline[Int, Int] =
2508
+ Pipeline.filter((x: Int) => x > 0)
2509
+
2510
+ val mixed = Stream(-2, -1, 0, 1, 2)
2511
+ // mixed: Stream[Nothing, Int] = Stream(-2, -1, 0, 1, 2)
2512
+ val positives = mixed.via(positiveIntsPipe)
2513
+ // positives: Stream[Nothing, Int] = Stream(-2, -1, 0, 1, 2).filter(...)
2514
+ val result = positives.runCollect
2515
+ // result: Either[Nothing, Chunk[Int]] = Right(IndexedSeq(1, 2))
2516
+ ```
2517
+
2518
+ ### Understanding Sinks
2519
+
2520
+ A `Sink[+E, -A, +Z]` is a consumer of elements of type `A` that produces a result `Z` or fails with `E`. Sinks are contravariant in `A` (they can accept a supertype of what they expect). Common sinks include:
2521
+
2522
+ - `Sink.collectAll: Sink[Nothing, A, Chunk[A]]` — collects all elements
2523
+ - `Sink.drain: Sink[Nothing, Any, Unit]` — discards all elements
2524
+ - `Sink.count: Sink[Nothing, Any, Long]` — counts elements
2525
+ - `Sink.foldLeft: Sink[Nothing, A, Z]` — folds elements with an accumulator
2526
+ - `Sink.head: Sink[Nothing, A, Option[A]]` — takes the first element
2527
+ - `Sink.foreach: Sink[Nothing, A, Unit]` — applies a function to each element
2528
+
2529
+ When you call `stream.run(sink)`, the stream is compiled to a `Reader` and the sink drains it, consuming all elements and producing the result.
2530
+
2531
+ ## Low-Level Pull with Reader
2532
+
2533
+ `Reader[+Elem]` is the low-level, pull-based source that backs every stream at execution time. Use cross-platform `startAsync` for a caller-owned `Reader.AsyncReader`, or `useReaderAsync` for bracketed access that awaits close on every outcome. The JVM additionally provides blocking `start` with a [`Scope`](../../resource-management/scope.md).
2534
+
2535
+ ### Manual Pull via `start`
2536
+
2537
+ `startAsync` transfers ownership to its caller, which must await `close()`. Prefer `useReaderAsync` when ownership need not escape. On the JVM, `start` opens a blocking reader within a `Scope`, which closes it when the scope exits:
2538
+
2539
+ ```scala
2540
+ abstract class Stream[+E, +A] {
2541
+ def startAsync: Async[Reader.AsyncReader[A]]
2542
+ def useReaderAsync[Z](f: Reader.AsyncReader[A] => Async[Z]): Async[Z]
2543
+ // JVM only
2544
+ def start(implicit scope: Scope): scope.$[Reader.SyncReader[A]]
2545
+ }
2546
+ ```
2547
+
2548
+ `start` is JVM only, and its result type is `Reader.SyncReader[A]` rather than an undifferentiated `Reader[A]`: a pipeline with asynchronous stages still works under it, because the JVM runtime bridges those boundaries by blocking. Shared code has no such member and must use `startAsync` or `useReaderAsync` instead, both of which hand back a `Reader.AsyncReader[A]`. [Manual Pull Across Platforms](../execution-and-compatibility/platform-differences.md#manual-pull-across-platforms) compares the three, and [`Stream#startAsync`](../execution-and-compatibility/async-execution.md#streamstartasync) covers the ownership rules that come with the asynchronous form.
2549
+
2550
+ Use `start` to manually pull elements within a resource scope:
2551
+
2552
+ ```scala title="streams-examples/src/main/scala/stream/ManualPullUsingStart.scala"
2553
+ package stream
2554
+
2555
+ import zio.blocks.streams.*
2556
+ import zio.blocks.streams.io.Reader
2557
+ import zio.blocks.scope.*
2558
+
2559
+ object ManualPullUsingStart extends App {
2560
+ Scope.global.scoped { scope =>
2561
+ import scope.*
2562
+
2563
+ // Open a stream for manual pulling
2564
+ val reader: $[Reader.SyncReader[Int]] = Stream.range(1, 6).start(using scope)
2565
+
2566
+ $(reader) { r =>
2567
+ // Iterate through reader values using the protocol directly
2568
+ // (cannot use scoped value in nested function, so use it directly)
2569
+ var current = r.read(-1)
2570
+ while (current != -1) {
2571
+ println(current) // prints 1, 2, 3, 4, 5
2572
+ current = r.read(-1)
2573
+ }
2574
+ }
2575
+ // reader is closed automatically when scope exits
2576
+ }
2577
+ }
2578
+ ```
2579
+
2580
+ Use these methods when you need element-by-element control rather than running through a Sink. Do not overlap asynchronous pulls: an async reader allows one active pull, and its lifecycle operations must be awaited.
2581
+
2582
+ ### The Reader Protocol
2583
+
2584
+ The pull protocol uses a **sentinel value** to signal end-of-stream:
2585
+
2586
+ - `read(sentinel)` — returns the next element, or `sentinel` when exhausted
2587
+ - `close()` — signals the consumer is done
2588
+ - `isClosed` — checks whether the reader is closed
2589
+
2590
+ For primitive types, specialized methods avoid boxing:
2591
+
2592
+ - `readBoolean(sentinel: Int): Int`
2593
+ - `readByte(): Int`
2594
+ - `readChar(sentinel: Int): Int`
2595
+ - `readShort(sentinel: Int): Int`
2596
+ - `readInt(sentinel: Long): Long`
2597
+ - `readLong(sentinel: Long): Long`
2598
+ - `readFloat(sentinel: Double): Double`
2599
+ - `readDouble(sentinel: Double): Double`
2600
+
2601
+ These are the eight exact physical methods selected by `Reader.jvmType`. The tag describes the reader's actual representation and method contract, so a known primitive lane survives type-preserving wrappers and static widening. Operations that produce a different element type select their output lane from result `JvmType.Infer` evidence.
2602
+
2603
+ :::note
2604
+ Avoid holding references to a `Reader` obtained via `start` outside its `Scope`. The scope guarantees cleanup; escaping the reader defeats that guarantee.
2605
+ :::
2606
+
2607
+ ## Implementation Notes
2608
+
2609
+ ZIO Blocks Streams achieves zero-boxing via compile-time type detection and dual compilation strategies:
2610
+
2611
+ ### JVM Primitive Specialization
2612
+
2613
+ By default, Scala's type system boxes primitive values into objects, which wastes memory and is slower. ZIO Blocks specializes all eight JVM primitive representations: `Boolean`, `Byte`, `Char`, `Short`, `Int`, `Long`, `Float`, and `Double`. `JvmType.Infer[A]` records a result type's physical lane at construction and at output-changing operations; type-preserving operations carry an already-known lane forward.
2614
+
2615
+ For example, an `Int` pipeline uses `readInt(Long.MinValue)` instead of boxing. A `Long` or `Double` internal pull cannot use a collision-free scalar sentinel, so it calls `readLongs` or `readDoubles` with a length-one array and interprets the returned count as EOF/data status. This preserves every `Long` and `Double` value:
2616
+
2617
+ ```scala
2618
+ if (jvmType eq JvmType.Int) {
2619
+ val i = source.readInt(Long.MinValue)
2620
+ // ... unboxed, fast path
2621
+ } else {
2622
+ val o = reader.read(EndOfStream) // generic boxed path
2623
+ // ...
2624
+ }
2625
+ ```
2626
+
2627
+ This optimization is transparent: you write normal, high-level code, and the compiler and runtime automatically use the fast path for primitives.
2628
+
2629
+ ### Dual Compilation: Recursive Vs Interpreter
2630
+
2631
+ Each stream node compiles in two ways:
2632
+
2633
+ 1. **Recursive (`compile`)**: Builds a tree of `Reader` objects, where each operation wraps the previous one. This is fast for shallow pipelines (< 100 operations).
2634
+
2635
+ 2. **Flat-Array Interpreter (`compileInterpreter`)**: For deep pipelines (> 100 operations), the recursive approach hits Scala's default stack-depth limit (~100) and risks `StackOverflowError`. Instead, the interpreter compiles the entire pipeline into a flat array of operations, executed iteratively.
2636
+
2637
+ The switch happens at `DepthCutoff = 100`. You should never see this in normal use, but it ensures that pipelines of any depth are safe.
2638
+
2639
+ A second split sits alongside this one: a graph made only of synchronous nodes compiles to the synchronous engine, and a graph containing any asynchronous node compiles to a separate, heap-allocated asynchronous engine. That choice is made once per materialization, never per element, and never from an annotation you write. [Why two engines](../execution-and-compatibility/async-execution.md#why-two-engines) explains what it buys, and why a purely synchronous stream pays nothing for it.
2640
+
2641
+ ## Running the Examples
2642
+
2643
+ All code from this guide is available as runnable examples in the `streams-examples` module.
2644
+
2645
+ Clone the repository and navigate to the project:
2646
+
2647
+ ```bash
2648
+ git clone https://github.com/zio/zio-blocks.git
2649
+ cd zio-blocks
2650
+ ```
2651
+
2652
+ **2. Run individual examples with sbt.** Here are the available examples:
2653
+
2654
+ ---
2655
+
2656
+ ### Basic Usage
2657
+
2658
+ This example demonstrates constructing streams from collections, transforming elements with `Stream#map` and `Stream#filter`, and collecting results:
2659
+
2660
+ ```scala title="streams-examples/src/main/scala/stream/StreamBasicUsageExample.scala"
2661
+ package stream
2662
+
2663
+ import zio.blocks.streams.Stream
2664
+ import zio.sbt.ExprEval.show
2665
+
2666
+ object StreamBasicUsageExample extends App {
2667
+ println("=== Stream Basic Usage ===\n")
2668
+
2669
+ // Construction from values
2670
+ println("1. Creating a stream from values:")
2671
+ val nums = Stream(1, 2, 3, 4, 5)
2672
+ show(nums.runCollect)
2673
+
2674
+ // Map transformation
2675
+ println("\n2. Transforming with map:")
2676
+ val doubled = Stream(1, 2, 3).map(_ * 2)
2677
+ show(doubled.runCollect)
2678
+
2679
+ // Filter operation
2680
+ println("\n3. Filtering elements:")
2681
+ val evens = Stream(1, 2, 3, 4, 5, 6).filter(_ % 2 == 0)
2682
+ show(evens.runCollect)
2683
+
2684
+ // Chaining operations
2685
+ println("\n4. Chaining multiple operations:")
2686
+ val result = Stream(1, 2, 3, 4, 5)
2687
+ .map(_ * 2)
2688
+ .filter(_ > 4)
2689
+ .runCollect
2690
+ show(result)
2691
+
2692
+ // Count operation
2693
+ println("\n5. Counting elements:")
2694
+ val count = Stream(1, 2, 3, 4, 5).count
2695
+ show(count)
2696
+
2697
+ // Take operation (short-circuiting)
2698
+ println("\n6. Taking first n elements (short-circuits):")
2699
+ val first3 = Stream.range(0, 1000).take(3).runCollect
2700
+ show(first3)
2701
+
2702
+ // Drop operation
2703
+ println("\n7. Dropping first n elements:")
2704
+ val afterDrop = Stream(1, 2, 3, 4, 5).drop(2).runCollect
2705
+ show(afterDrop)
2706
+
2707
+ // Empty stream
2708
+ println("\n8. Working with empty streams:")
2709
+ val empty = Stream.empty.runCollect
2710
+ show(empty)
2711
+
2712
+ // Concatenation
2713
+ println("\n9. Concatenating streams:")
2714
+ val combined = (Stream(1, 2) ++ Stream(3, 4)).runCollect
2715
+ show(combined)
2716
+ }
2717
+ ```
2718
+
2719
+ To run this example:
2720
+
2721
+ ```bash
2722
+ sbt "streams-examples/runMain stream.StreamBasicUsageExample"
2723
+ ```
2724
+
2725
+ ### Flat-Mapping Nested Streams
2726
+
2727
+ This example shows how `Stream#flatMap` sequences multiple streams and flattens the results:
2728
+
2729
+ ```scala title="streams-examples/src/main/scala/stream/StreamFlatMapExample.scala"
2730
+ package stream
2731
+
2732
+ import zio.blocks.streams.Stream
2733
+ import zio.sbt.ExprEval.show
2734
+
2735
+ object StreamFlatMapExample extends App {
2736
+ println("=== Stream FlatMap and Nested Streams ===\n")
2737
+
2738
+ // Basic flatMap
2739
+ println("1. Basic flatMap - expand each element into a stream:")
2740
+ val expanded = Stream(1, 2, 3).flatMap(x => Stream(x, x * 10))
2741
+ show(expanded.runCollect)
2742
+
2743
+ // FlatMap with different stream sizes
2744
+ println("\n2. FlatMap with varying sizes:")
2745
+ val varySizes = Stream(1, 2, 3).flatMap(x => Stream.range(0, x))
2746
+ show(varySizes.runCollect)
2747
+
2748
+ // FlatMap with string expansion
2749
+ println("\n3. Expanding into string streams:")
2750
+ val ids = Stream("a", "b")
2751
+ val expanded_ids = ids.flatMap(id => Stream(s"${id}_1", s"${id}_2", s"${id}_3"))
2752
+ show(expanded_ids.runCollect)
2753
+
2754
+ // FlattenAll for deeply nested streams
2755
+ println("\n4. Flattening nested streams with flattenAll:")
2756
+ val nested = Stream(
2757
+ Stream(1, 2),
2758
+ Stream(3, 4),
2759
+ Stream(5, 6)
2760
+ )
2761
+ val flat = Stream.flattenAll(nested)
2762
+ show(flat.runCollect)
2763
+
2764
+ // Sequential processing guarantees
2765
+ println("\n5. Sequential processing (important for side effects and resources):")
2766
+ var order = scala.collection.mutable.Buffer[String]()
2767
+ val tracked = Stream(1, 2, 3).flatMap { x =>
2768
+ order += s"expand($x)"
2769
+ Stream(x, x + 100).tapEach(y => order += s"emit($y)")
2770
+ }
2771
+ val _ = tracked.runCollect
2772
+ show(order.toList)
2773
+
2774
+ // FlatMap with error recovery
2775
+ println("\n6. FlatMap can propagate errors:")
2776
+ sealed trait Error
2777
+ case object InvalidId extends Error
2778
+
2779
+ val mayFail = Stream(1, 2, -1, 3).flatMap { x =>
2780
+ if (x < 0) Stream.fail(InvalidId)
2781
+ else Stream(x, x * 2)
2782
+ }
2783
+ show(mayFail.runCollect)
2784
+ }
2785
+ ```
2786
+
2787
+ Run this example:
2788
+
2789
+ ```bash
2790
+ sbt "streams-examples/runMain stream.StreamFlatMapExample"
2791
+ ```
2792
+
2793
+ ### Error Handling
2794
+
2795
+ This example demonstrates typed error recovery with `fail`, `catchAll`, and `orElse`:
2796
+
2797
+ ```scala title="streams-examples/src/main/scala/stream/StreamErrorHandlingExample.scala"
2798
+ package stream
2799
+
2800
+ import zio.blocks.streams.Stream
2801
+ import zio.sbt.ExprEval.show
2802
+
2803
+ object StreamErrorHandlingExample extends App {
2804
+ println("=== Stream Error Handling ===\n")
2805
+
2806
+ sealed trait ApiError
2807
+ case object NotFound extends ApiError
2808
+ case class ValidationError(msg: String) extends ApiError
2809
+ case class ServerError(code: Int) extends ApiError
2810
+
2811
+ // Basic fail
2812
+ println("1. Creating a failing stream:")
2813
+ val failed: Stream[ApiError, String] = Stream.fail(NotFound)
2814
+ show(failed.runCollect)
2815
+
2816
+ // catchAll for recovery
2817
+ println("\n2. Recovering from errors with catchAll:")
2818
+ val recovered = Stream.fail(NotFound).catchAll(_ => Stream.succeed("default-value"))
2819
+ show(recovered.runCollect)
2820
+
2821
+ // orElse for recovery
2822
+ println("\n3. Using orElse (lazy fallback evaluation):")
2823
+ val fallback = Stream.fail(NotFound) || Stream(1, 2, 3)
2824
+ show(fallback.runCollect)
2825
+
2826
+ // Error transformation with error-producing flatMap
2827
+ println("\n4. Producing typed errors in flatMap:")
2828
+ val errorExample = Stream(1, 2, 3, 4).flatMap { x =>
2829
+ if (x == 3) Stream.fail[ApiError](ValidationError("cannot process"))
2830
+ else Stream(x)
2831
+ }
2832
+ show(errorExample.runCollect)
2833
+
2834
+ // Handling errors in flatMap chains
2835
+ println("\n5. Error handling in flatMap chains:")
2836
+ val chain = Stream(1, 2, 3, 4).flatMap { x =>
2837
+ if (x == 3) Stream.fail(ValidationError(s"Cannot process $x"))
2838
+ else Stream(x * 10)
2839
+ }
2840
+ show(chain.runCollect)
2841
+
2842
+ // Recovering from errors in flatMap
2843
+ println("\n6. Recovering from errors with catchAll in chains:")
2844
+ val recovered_chain = Stream(1, 2, 3, 4).flatMap { x =>
2845
+ if (x == 3) Stream.fail(ValidationError(s"Cannot process $x"))
2846
+ else Stream(x * 10)
2847
+ }
2848
+ .catchAll(_ => Stream.succeed(-1))
2849
+
2850
+ show(recovered_chain.runCollect)
2851
+
2852
+ // Handling typed errors from attempt
2853
+ println("\n7. Recovering typed errors from Stream.attempt with catchAll:")
2854
+ val risky = Stream.attempt("not-a-number".toInt)
2855
+ val safe = risky.catchAll { case _: NumberFormatException =>
2856
+ Stream.succeed(-1)
2857
+ }
2858
+ show(safe.runCollect)
2859
+
2860
+ // Multiple error branches
2861
+ println("\n8. Distinguishing error types in recovery:")
2862
+ val multi_errors = Stream(1, 2, 3, 4).flatMap { x =>
2863
+ x match {
2864
+ case 2 => Stream.fail(NotFound)
2865
+ case 3 => Stream.fail(ValidationError("Invalid data"))
2866
+ case _ => Stream(x * 10)
2867
+ }
2868
+ }.catchAll {
2869
+ case NotFound => Stream("missing")
2870
+ case ValidationError(msg) => Stream(s"invalid: $msg")
2871
+ case _ => Stream("unknown error")
2872
+ }
2873
+
2874
+ show(multi_errors.runCollect)
2875
+ }
2876
+ ```
2877
+
2878
+ Run this example:
2879
+
2880
+ ```bash
2881
+ sbt "streams-examples/runMain stream.StreamErrorHandlingExample"
2882
+ ```
2883
+
2884
+ ### Resource Management
2885
+
2886
+ This example shows how `fromAcquireRelease` and `ensuring` manage resources safely:
2887
+
2888
+ ```scala title="streams-examples/src/main/scala/stream/StreamResourceExample.scala"
2889
+ package stream
2890
+
2891
+ import zio.blocks.streams.Stream
2892
+ import zio.sbt.ExprEval.show
2893
+ import scala.collection.mutable.Buffer
2894
+
2895
+ object StreamResourceExample extends App {
2896
+ println("=== Stream Resource Management ===\n")
2897
+
2898
+ // Simulated resource type
2899
+ case class Database(name: String) {
2900
+ private var closed = false
2901
+
2902
+ def query(q: String): List[String] = {
2903
+ if (closed) throw new Exception("Database is closed")
2904
+ q match {
2905
+ case "users" => List("Alice", "Bob", "Charlie")
2906
+ case "ids" => List("1", "2", "3")
2907
+ case _ => List()
2908
+ }
2909
+ }
2910
+
2911
+ def close(): Unit = {
2912
+ println(s" → Closing database: $name")
2913
+ closed = true
2914
+ }
2915
+
2916
+ def isClosed: Boolean = closed
2917
+ }
2918
+
2919
+ // Basic resource management
2920
+ println("1. Basic fromAcquireRelease - automatic cleanup:")
2921
+ val log = Buffer[String]()
2922
+
2923
+ val managed = Stream.fromAcquireRelease(
2924
+ acquire = {
2925
+ log += "opened"
2926
+ Database("main")
2927
+ },
2928
+ release = { db =>
2929
+ db.close()
2930
+ log += "closed"
2931
+ }
2932
+ )(db => Stream.fromIterable(db.query("users")))
2933
+
2934
+ val result1 = managed.runCollect
2935
+ show(result1)
2936
+ show(log.toList)
2937
+
2938
+ // Ensuring cleanup
2939
+ println("\n2. Using ensuring for guaranteed cleanup:")
2940
+ log.clear()
2941
+ var finalizing = false
2942
+
2943
+ val withEnsure = Stream(1, 2, 3)
2944
+ .tapEach(x => log += s"processing $x")
2945
+ .ensuring {
2946
+ finalizing = true
2947
+ log += "finalizing"
2948
+ }
2949
+
2950
+ val result2 = withEnsure.runCollect
2951
+ show(result2)
2952
+ show(log.toList)
2953
+ show(finalizing)
2954
+
2955
+ // Error safety
2956
+ println("\n3. Cleanup happens even on error:")
2957
+ log.clear()
2958
+
2959
+ sealed trait Error
2960
+ case object ProcessingFailed extends Error
2961
+
2962
+ val errorStream = Stream.fromAcquireRelease(
2963
+ acquire = {
2964
+ log += "opened"
2965
+ Database("error-test")
2966
+ },
2967
+ release = { db =>
2968
+ db.close()
2969
+ log += "closed"
2970
+ }
2971
+ )(db =>
2972
+ Stream(1, 2, 3).flatMap { x =>
2973
+ if (x == 2) Stream.fail(ProcessingFailed)
2974
+ else Stream(x)
2975
+ }
2976
+ )
2977
+
2978
+ val result3 = errorStream.runCollect
2979
+ show(result3)
2980
+ show(log.toList)
2981
+
2982
+ // Multiple nested resources
2983
+ println("\n4. Multiple nested resources with proper cleanup order:")
2984
+ log.clear()
2985
+
2986
+ val nested = Stream.fromAcquireRelease(
2987
+ acquire = {
2988
+ log += "open db1"
2989
+ Database("db1")
2990
+ },
2991
+ release = db => {
2992
+ db.close()
2993
+ log += "close db1"
2994
+ }
2995
+ )(db1 =>
2996
+ Stream.fromAcquireRelease(
2997
+ acquire = {
2998
+ log += "open db2"
2999
+ Database("db2")
3000
+ },
3001
+ release = db2 => {
3002
+ db2.close()
3003
+ log += "close db2"
3004
+ }
3005
+ ) { db2 =>
3006
+ val data = db1.query("users") ++ db2.query("ids")
3007
+ Stream.fromIterable(data).tapEach(x => log += s"emit $x")
3008
+ }
3009
+ )
3010
+
3011
+ val result4 = nested.runCollect
3012
+ show(result4)
3013
+ show(log.toList)
3014
+
3015
+ // AutoCloseable integration
3016
+ println("\n5. Using AutoCloseable for simpler cleanup:")
3017
+ log.clear()
3018
+
3019
+ class AutoCloseableDb extends AutoCloseable {
3020
+ def close(): Unit =
3021
+ log += "auto-closed"
3022
+ }
3023
+
3024
+ val autoCloseable = Stream.fromAcquireRelease(
3025
+ acquire = {
3026
+ log += "acquired"
3027
+ new AutoCloseableDb
3028
+ }
3029
+ // release defaults to calling .close() on AutoCloseable
3030
+ )(db => Stream.succeed(42))
3031
+
3032
+ val result5 = autoCloseable.runCollect
3033
+ show(result5)
3034
+ show(log.toList)
3035
+ }
3036
+
3037
+ object X extends App {
3038
+ import zio.blocks.streams.*
3039
+ import java.io.*
3040
+
3041
+ val charCount: Either[IOException, Long] =
3042
+ Stream
3043
+ .fromJavaReader(new StringReader("Hello\nWorld")) // lazily acquires reader
3044
+ .filter(!_.isWhitespace) // process only non-whitespace
3045
+ .count // count all matching characters
3046
+
3047
+ println(charCount) // prints Right(10) — count of non-whitespace characters
3048
+ }
3049
+ ```
3050
+
3051
+ Run this example:
3052
+
3053
+ ```bash
3054
+ sbt "streams-examples/runMain stream.StreamResourceExample"
3055
+ ```
3056
+
3057
+ ### Windowing and Scanning
3058
+
3059
+ This example demonstrates `grouped`, `sliding`, and `scan` for windowing and stateful transformations:
3060
+
3061
+ ```scala title="streams-examples/src/main/scala/stream/StreamWindowingExample.scala"
3062
+ package stream
3063
+
3064
+ import zio.blocks.streams.Stream
3065
+ import zio.sbt.ExprEval.show
3066
+
3067
+ object StreamWindowingExample extends App {
3068
+ println("=== Stream Windowing and Stateful Transformations ===\n")
3069
+
3070
+ // Grouped - fixed-size windows
3071
+ println("1. Grouping into fixed-size chunks:")
3072
+ val nums = Stream(1, 2, 3, 4, 5, 6, 7)
3073
+ val grouped = nums.grouped(3)
3074
+ show(grouped.runCollect)
3075
+
3076
+ // Grouped with incomplete last chunk
3077
+ println("\n2. Last chunk may be smaller:")
3078
+ val ungrouped = Stream(1, 2, 3, 4, 5).grouped(2)
3079
+ show(ungrouped.runCollect)
3080
+
3081
+ // Sliding window
3082
+ println("\n3. Sliding window (default step = 1):")
3083
+ val sliding1 = Stream(1, 2, 3, 4, 5).sliding(3)
3084
+ show(sliding1.runCollect)
3085
+
3086
+ // Sliding with custom step
3087
+ println("\n4. Sliding window with step > 1:")
3088
+ val sliding2 = Stream(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, step = 2)
3089
+ show(sliding2.runCollect)
3090
+
3091
+ // Scan - running aggregate
3092
+ println("\n5. Scan for running sum (accumulator pattern):")
3093
+ val cumsum = Stream(1, 2, 3, 4, 5).scan(0)(_ + _)
3094
+ show(cumsum.runCollect)
3095
+
3096
+ // Scan for running product
3097
+ println("\n6. Scan for running product:")
3098
+ val cumprod = Stream(1, 2, 3, 4).scan(1)(_ * _)
3099
+ show(cumprod.runCollect)
3100
+
3101
+ // MapAccum - accumulator + transform
3102
+ println("\n7. MapAccum for indexed transformation:")
3103
+ val indexed = Stream("a", "b", "c").mapAccum(0)((idx, x) => (idx + 1, (idx, x)))
3104
+ show(indexed.runCollect)
3105
+
3106
+ // MapAccum with state structure
3107
+ println("\n8. MapAccum with complex state:")
3108
+ case class Stats(count: Int, sum: Int, max: Int)
3109
+
3110
+ val stats = Stream(5, 3, 8, 2, 9).mapAccum(Stats(0, 0, Int.MinValue)) { case (s, x) =>
3111
+ (
3112
+ Stats(s.count + 1, s.sum + x, math.max(s.max, x)),
3113
+ (x, Stats(s.count + 1, s.sum + x, math.max(s.max, x)))
3114
+ )
3115
+ }
3116
+ show(stats.runCollect)
3117
+
3118
+ // Combining windowing with filtering
3119
+ println("\n9. Windowing + filtering (only windows with sum > 5):")
3120
+ val filtered = Stream(1, 2, 3, 4, 5, 6)
3121
+ .sliding(3, step = 1)
3122
+ .filter(chunk => chunk.foldLeft(0)(_ + _) > 5)
3123
+ show(filtered.runCollect)
3124
+
3125
+ // Chaining scan with other operations
3126
+ println("\n10. Scan + filter for conditional processing:")
3127
+ val conditional = Stream(1, 1, 2, 1, 1, 3)
3128
+ .scan(0)(_ + _)
3129
+ .filter(_ >= 3) // emit when cumsum >= 3
3130
+ show(conditional.runCollect)
3131
+
3132
+ // Intersperse - useful with grouping
3133
+ println("\n11. Intersperse (insert separator between elements):")
3134
+ val separated = Stream(1, 2, 3).intersperse(0)
3135
+ show(separated.runCollect)
3136
+
3137
+ // Grouped + intersperse for row formatting
3138
+ println("\n12. Grouped + intersperse for CSV-like output:")
3139
+ val rows = Stream(1, 2, 3, 4, 5, 6)
3140
+ .grouped(2)
3141
+ .map(chunk => chunk.toList.mkString(","))
3142
+ .intersperse("\n")
3143
+ show(rows.runCollect)
3144
+ }
3145
+ ```
3146
+
3147
+ Run this example:
3148
+
3149
+ ```bash
3150
+ sbt "streams-examples/runMain stream.StreamWindowingExample"
3151
+ ```
3152
+
3153
+ ### Stateful Asynchronous Operators
3154
+
3155
+ This example runs `takeWhileAsync`, `mapAccumAsync`, `scanAsync`, and `ensuringAsync` in a single pipeline, with every callback completing on another thread. It is JVM-only, because it ends in `.block`:
3156
+
3157
+ ```scala title="streams-examples/src/main/scala/stream/StreamAsyncStatefulExample.scala"
3158
+ package stream
3159
+
3160
+ import java.util.concurrent.atomic.AtomicInteger
3161
+
3162
+ import zio.blocks.async._
3163
+ import zio.blocks.chunk.Chunk
3164
+ import zio.blocks.streams.Stream
3165
+
3166
+ /**
3167
+ * The stateful asynchronous operators in one pipeline: `takeWhileAsync`,
3168
+ * `mapAccumAsync`, `scanAsync`, and `ensuringAsync`.
3169
+ *
3170
+ * A day's ledger feed is cut at its end-of-day marker, numbered, folded into
3171
+ * running balances, and closed by an asynchronous finalizer. Every step
3172
+ * suspends on another thread, and every step is still sequential: at most one
3173
+ * invocation of each callback is active at a time, so the accumulator is never
3174
+ * shared across concurrent work.
3175
+ *
3176
+ * JVM only: the closing `.block` parks the calling thread until the `Async`
3177
+ * completes. On Scala.js, keep the `Async` and let the host drive it.
3178
+ */
3179
+ object StreamAsyncStatefulExample {
3180
+ final case class Posting(seq: Long, amount: Int)
3181
+
3182
+ /** Completes on another thread, so each callback below really suspends. */
3183
+ def deferred[A](value: => A): Async[A] = {
3184
+ val completer = new Completer[A]
3185
+ val thread = new Thread(() => completer.succeed(value))
3186
+ thread.start()
3187
+ completer
3188
+ }
3189
+
3190
+ def main(args: Array[String]): Unit = {
3191
+ val sessionsClosed = new AtomicInteger
3192
+
3193
+ // `0` is the end-of-day marker, not an amount. Everything after it belongs
3194
+ // to the next day and must not be posted.
3195
+ val feed: Stream[Nothing, Int] = Stream(120, -40, 75, 0, 999)
3196
+
3197
+ val balances: Stream[Nothing, Long] = feed
3198
+ // Stops at the marker and closes upstream, so `999` is never pulled.
3199
+ .takeWhileAsync(amount => deferred(amount != 0))
3200
+ // Threads a sequence number through while numbering each entry.
3201
+ .mapAccumAsync(0L)((seq, amount) => deferred((seq + 1, Posting(seq + 1, amount))))
3202
+ // Emits the accumulator at each step, starting with `0L`, so the output
3203
+ // carries one more element than the input.
3204
+ .scanAsync(0L)((balance, posting) => deferred(balance + posting.amount))
3205
+ // Awaited exactly once when the materialized stream closes.
3206
+ .ensuringAsync(deferred { sessionsClosed.incrementAndGet(); () })
3207
+
3208
+ val result: Either[Nothing, Chunk[Long]] = balances.runCollectAsync.block
3209
+
3210
+ require(result == Right(Chunk(0L, 120L, 80L, 155L)), s"unexpected balances: $result")
3211
+ require(sessionsClosed.get() == 1, s"finalizer ran ${sessionsClosed.get()} times")
3212
+
3213
+ println(s"running balances -> $result")
3214
+ println(s"sessions closed -> ${sessionsClosed.get()}")
3215
+ }
3216
+ }
3217
+ ```
3218
+
3219
+ Run this example:
3220
+
3221
+ ```bash
3222
+ sbt "streams-examples/runMain stream.StreamAsyncStatefulExample"
3223
+ ```
3224
+
3225
+ ## Native Asynchronous Byte Readers
3226
+
3227
+ Each platform ships adapters that turn a native byte source into a `Reader.AsyncReader[Byte]`, which `Stream.fromReader` then lifts into a stream: `AsyncNioReaders.fromChannel` and `fromSocket` on the JVM, `ReadableStreamReaders.fromReadableStream` on Scala.js, each with an `Unmanaged` variant that leaves the native source caller-owned. [Reader](../primitives/reader.md#from-native-asynchronous-sources) documents them, together with their chunking and EOF rules and the way source failures reach the typed error channel.
3228
+
3229
+ ## See Also
3230
+
3231
+ - [Asynchronous Stream Execution](../execution-and-compatibility/async-execution.md) — the full asynchronous constructor, operator, and terminal API, and how one `Stream` type describes both execution modes
3232
+ - [Platform Differences](../execution-and-compatibility/platform-differences.md) — which members exist on the JVM, which exist on Scala.js, and why the blocking family is JVM-only
3233
+ - [Reader](../primitives/reader.md) — the `SyncReader` / `AsyncReader` union that decides which engine materializes behind the bounded-concurrency operators, and the adapters behind the native asynchronous byte readers
3234
+ - [Async Reference](../../async.md) — `Async.promise` and `Completer` bridge callback-based APIs into async values that can feed stream sources; `Async.Running` carries a synchronous cancellation handle that complements stream resource management
3235
+ - [Async](../../async.md#asyncselector) — `AsyncSelector`, the primitive the bounded-concurrency slot machinery is built from
3236
+ - [Scope Reference](../../resource-management/scope.md) — compile-time resource safety for stream acquisition and release; `fromAcquireRelease` follows the same ownership rules as Scope-managed resources