@zio.dev/zio-blocks 0.0.33 → 0.0.51
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/guides/compile-time-resource-safety-with-scope.md +16 -17
- package/guides/getting-started-with-mux.md +1507 -0
- package/guides/query-dsl-extending.md +161 -102
- package/guides/query-dsl-fluent-builder.md +217 -157
- package/guides/query-dsl-reified-optics.md +12 -10
- package/guides/query-dsl-sql.md +246 -165
- package/guides/telemetry-guide.md +1069 -0
- package/guides/zio-schema-migration.md +29 -22
- package/index.md +292 -50
- package/package.json +1 -1
- package/plans/config-follow-up-prs.md +188 -0
- package/plans/config-pr-assessment-roadmap.md +310 -0
- package/reference/MuxDataFlow.jsx +250 -0
- package/reference/async.md +651 -0
- package/reference/chunk.md +3533 -308
- package/reference/codegen/case-class.md +436 -0
- package/reference/codegen/emitter-config.md +383 -0
- package/reference/codegen/examples.md +664 -0
- package/reference/codegen/field.md +316 -0
- package/reference/codegen/index.md +317 -0
- package/reference/codegen/scala-emitter.md +392 -0
- package/reference/codegen/scala-file.md +276 -0
- package/reference/codegen/sealed-trait.md +408 -0
- package/reference/codegen/type-definition.md +340 -0
- package/reference/codegen/type-ref.md +201 -0
- package/reference/combinators.md +347 -117
- package/reference/config.md +158 -0
- package/reference/context.md +4 -4
- package/reference/datastar.md +346 -0
- package/reference/docs.md +1461 -345
- package/reference/endpoint/auth-type.md +146 -0
- package/reference/endpoint/endpoint.md +297 -0
- package/reference/endpoint/http-codec.md +249 -0
- package/reference/endpoint/index.md +825 -0
- package/reference/endpoint/path-codec.md +237 -0
- package/reference/endpoint/route-pattern.md +196 -0
- package/reference/endpoint/route-tree.md +111 -0
- package/reference/endpoint/segment-codec.md +212 -0
- package/reference/html.md +1120 -0
- package/reference/htmx/attribute-values.md +359 -0
- package/reference/htmx/hx-encoding.md +111 -0
- package/reference/htmx/hx-params.md +204 -0
- package/reference/htmx/hx-swap.md +276 -0
- package/reference/htmx/hx-sync.md +251 -0
- package/reference/htmx/hx-target.md +314 -0
- package/reference/htmx/hx-trigger.md +457 -0
- package/reference/htmx/hx-url-update.md +239 -0
- package/reference/htmx/index.md +855 -0
- package/reference/http-model/index.md +47 -0
- package/reference/http-model/model.md +1481 -0
- package/reference/http-model/schema.md +747 -0
- package/reference/maybe.md +826 -0
- package/reference/media-type.md +2 -2
- package/reference/mux.mdx +823 -0
- package/reference/openapi.md +1351 -0
- package/reference/resource-management/defer-handle.md +1 -1
- package/reference/resource-management/resource.md +31 -2
- package/reference/resource-management/scope.md +28 -12
- package/reference/resource-management/wire.md +3 -7
- package/reference/ringbuffer/MpmcDiagram.jsx +717 -0
- package/reference/ringbuffer/MpscDiagram.jsx +618 -0
- package/reference/ringbuffer/SpmcDiagram.jsx +680 -0
- package/reference/ringbuffer/SpscDiagram.jsx +677 -0
- package/reference/ringbuffer/advanced.mdx +109 -0
- package/reference/ringbuffer/index.mdx +145 -0
- package/reference/ringbuffer/mpmc.mdx +151 -0
- package/reference/ringbuffer/mpsc.mdx +132 -0
- package/reference/ringbuffer/spmc.mdx +108 -0
- package/reference/ringbuffer/spsc.mdx +344 -0
- package/reference/{allows.md → schema/allows.md} +4 -4
- package/reference/{binding-resolver.md → schema/binding-resolver.md} +1 -1
- package/reference/{binding.md → schema/binding.md} +2 -3
- package/reference/schema/built-in-codecs/avro.md +451 -0
- package/reference/schema/built-in-codecs/bson.md +480 -0
- package/reference/schema/built-in-codecs/csv.md +564 -0
- package/reference/schema/built-in-codecs/index.md +77 -0
- package/reference/schema/built-in-codecs/json/index.md +295 -0
- package/reference/schema/built-in-codecs/json/json-config.md +217 -0
- package/reference/{json-patch.md → schema/built-in-codecs/json/json-patch.md} +5 -5
- package/reference/{json-schema.md → schema/built-in-codecs/json/json-schema.md} +14 -47
- package/reference/schema/built-in-codecs/json/json-selection.md +322 -0
- package/reference/{json.md → schema/built-in-codecs/json/json.md} +32 -64
- package/reference/schema/built-in-codecs/messagepack.md +508 -0
- package/reference/schema/built-in-codecs/thrift.md +433 -0
- package/reference/schema/built-in-codecs/toon.md +1078 -0
- package/reference/{xml.md → schema/built-in-codecs/xml.md} +13 -9
- package/reference/schema/built-in-codecs/yaml.md +552 -0
- package/reference/{codec.md → schema/codec.md} +10 -10
- package/reference/{dynamic-optic.md → schema/dynamic-optic.md} +151 -5
- package/reference/{dynamic-schema.md → schema/dynamic-schema.md} +8 -8
- package/reference/schema/format.md +92 -0
- package/reference/schema/index.md +50 -0
- package/reference/schema/migration.md +297 -0
- package/reference/{modifier.md → schema/modifier.md} +58 -7
- package/reference/{optics.md → schema/optics.md} +2 -2
- package/reference/{patch.md → schema/patch.md} +1 -1
- package/{path-interpolator.md → reference/schema/path-interpolator.md} +165 -72
- package/reference/{schema-evolution → schema/schema-evolution}/as.md +8 -8
- package/reference/{schema-evolution → schema/schema-evolution}/index.md +2 -2
- package/reference/{schema-evolution → schema/schema-evolution}/into.md +8 -8
- package/reference/{schema-expr.md → schema/schema-expr.md} +110 -175
- package/reference/{schema.md → schema/schema.md} +12 -0
- package/reference/{structural-types.md → schema/structural-types.md} +1 -1
- package/reference/{type-class-derivation.md → schema/type-class-derivation.md} +63 -1
- package/reference/smithy.md +533 -0
- package/reference/sql/db-codec-deriver.md +71 -0
- package/reference/sql/db-codec.md +687 -0
- package/reference/sql/db-con.md +271 -0
- package/reference/sql/db-connection.md +153 -0
- package/reference/sql/db-param-writer.md +77 -0
- package/reference/sql/db-param.md +66 -0
- package/reference/sql/db-result-reader.md +146 -0
- package/reference/sql/db-tx.md +82 -0
- package/reference/sql/db-value.md +41 -0
- package/reference/sql/ddl.md +85 -0
- package/reference/sql/frag.md +254 -0
- package/reference/sql/index.md +341 -0
- package/reference/sql/repo.md +600 -0
- package/reference/sql/sql-dialect.md +73 -0
- package/reference/sql/sql-logger.md +62 -0
- package/reference/sql/sql-name-mapper.md +70 -0
- package/reference/sql/table-metadata.md +134 -0
- package/reference/sql/table.md +448 -0
- package/reference/sql/transactor-zio.md +399 -0
- package/reference/sql/transactor.md +353 -0
- package/reference/sql-zio.md +112 -0
- package/reference/streams/concurrent-operators.md +106 -0
- package/reference/streams/index.md +653 -0
- package/reference/streams/pipeline.md +718 -0
- package/reference/streams/reader.md +1284 -0
- package/reference/streams/scala-2-compatibility.md +55 -0
- package/reference/streams/sink.md +1426 -0
- package/reference/streams/stream.md +2526 -0
- package/reference/streams/writer.md +1045 -0
- package/reference/streams/zero-boxing.md +275 -0
- package/reference/telemetry.md +693 -0
- package/reference/typeid.md +5 -19
- package/sidebars.js +238 -43
- package/reference/formats.md +0 -694
- package/reference/http-model.md +0 -1716
- package/reference/streams.md +0 -989
- package/ringbuffer.md +0 -249
- /package/reference/{json-differ.md → schema/built-in-codecs/json/json-differ.md} +0 -0
- /package/reference/{dynamic-value.md → schema/dynamic-value.md} +0 -0
- /package/reference/{lazy.md → schema/lazy.md} +0 -0
- /package/reference/{reflect.md → schema/reflect.md} +0 -0
- /package/reference/{registers.md → schema/registers.md} +0 -0
- /package/reference/{schema-error.md → schema/schema-error.md} +0 -0
- /package/reference/{syntax.md → schema/syntax.md} +0 -0
- /package/reference/{validation.md → schema/validation.md} +0 -0
package/ringbuffer.md
DELETED
|
@@ -1,249 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
id: ringbuffer
|
|
3
|
-
title: "Ring Buffer"
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# ZIO Blocks — Ring Buffer (`zio.blocks.ringbuffer`)
|
|
7
|
-
|
|
8
|
-
`zio.blocks.ringbuffer` is a family of **high-performance, bounded ring buffers** for the JVM (and Scala.js). Each variant is optimized for a specific producer/consumer threading pattern—pick the one that matches your use case and get the fastest possible inter-thread communication with zero dependencies.
|
|
9
|
-
|
|
10
|
-
All variants expose `offer` (returns `false` if full) and `take` (returns `null` if empty)—both non-blocking. Capacity must be a **power of two** (enables bitwise masking instead of modulo). Elements must be **non-null reference types** (`A <: AnyRef`).
|
|
11
|
-
|
|
12
|
-
## Ring buffer variants
|
|
13
|
-
|
|
14
|
-
| Type | Producers | Consumers | Algorithm |
|
|
15
|
-
|------|-----------|-----------|-----------|
|
|
16
|
-
| `SpscRingBuffer` | 1 | 1 | FastFlow (null/non-null signaling) |
|
|
17
|
-
| `SpmcRingBuffer` | 1 | Many | Index-based capacity + CAS consumers |
|
|
18
|
-
| `MpscRingBuffer` | Many | 1 | CAS producers + relaxed poll |
|
|
19
|
-
| `MpmcRingBuffer` | Many | Many | Vyukov/Dmitry sequence buffer |
|
|
20
|
-
|
|
21
|
-
**Naming convention:** `S` = single, `M` = multi, `p` = producer, `c` = consumer.
|
|
22
|
-
|
|
23
|
-
---
|
|
24
|
-
|
|
25
|
-
## Installation
|
|
26
|
-
|
|
27
|
-
```scala
|
|
28
|
-
libraryDependencies += "dev.zio" %% "zio-blocks-ringbuffer" % "0.0.33"
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
---
|
|
32
|
-
|
|
33
|
-
## Quick start
|
|
34
|
-
|
|
35
|
-
```scala
|
|
36
|
-
import zio.blocks.ringbuffer.SpscRingBuffer
|
|
37
|
-
|
|
38
|
-
val buf = SpscRingBuffer[String](1024) // capacity must be a power of 2
|
|
39
|
-
|
|
40
|
-
// Producer thread
|
|
41
|
-
buf.offer("hello") // true if inserted, false if full
|
|
42
|
-
|
|
43
|
-
// Consumer thread
|
|
44
|
-
val msg: String = buf.take() // element or null if empty
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
---
|
|
48
|
-
|
|
49
|
-
## API
|
|
50
|
-
|
|
51
|
-
Every ring buffer provides:
|
|
52
|
-
|
|
53
|
-
```scala
|
|
54
|
-
def offer(a: A): Boolean // insert; returns false if full
|
|
55
|
-
def take(): A // remove; returns null if empty
|
|
56
|
-
def size: Int // approximate element count
|
|
57
|
-
def isEmpty: Boolean // approximate emptiness check
|
|
58
|
-
def isFull: Boolean // approximate fullness check
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
`SpscRingBuffer` additionally provides batch operations:
|
|
62
|
-
|
|
63
|
-
```scala
|
|
64
|
-
def drain(consumer: A => Unit, limit: Int): Int // drain up to limit elements
|
|
65
|
-
def fill(supplier: () => A, limit: Int): Int // fill up to limit slots
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
All `size`/`isEmpty`/`isFull` values are **approximate** under concurrency—they are snapshots that may be stale by the time the caller acts on them.
|
|
69
|
-
|
|
70
|
-
---
|
|
71
|
-
|
|
72
|
-
## Choosing a variant
|
|
73
|
-
|
|
74
|
-
Use the most constrained variant that fits your threading model:
|
|
75
|
-
|
|
76
|
-
| Scenario | Recommended |
|
|
77
|
-
|----------|-------------|
|
|
78
|
-
| Dedicated pipeline: one writer thread, one reader thread | `SpscRingBuffer` |
|
|
79
|
-
| Fan-in: many writers, one reader (e.g., logging, event aggregation) | `MpscRingBuffer` |
|
|
80
|
-
| Fan-out: one writer, many readers (e.g., work distribution) | `SpmcRingBuffer` |
|
|
81
|
-
| General purpose: any number of writers and readers | `MpmcRingBuffer` |
|
|
82
|
-
|
|
83
|
-
---
|
|
84
|
-
|
|
85
|
-
## Usage examples
|
|
86
|
-
|
|
87
|
-
### SPSC with drain/fill
|
|
88
|
-
|
|
89
|
-
```scala
|
|
90
|
-
import zio.blocks.ringbuffer.SpscRingBuffer
|
|
91
|
-
|
|
92
|
-
val buf = SpscRingBuffer[java.lang.Integer](64)
|
|
93
|
-
|
|
94
|
-
// Producer: batch-fill from a data source
|
|
95
|
-
var seq = 0
|
|
96
|
-
val filled = buf.fill(() => { seq += 1; Integer.valueOf(seq) }, 32)
|
|
97
|
-
println(s"Filled $filled elements")
|
|
98
|
-
|
|
99
|
-
// Consumer: batch-drain into a processor
|
|
100
|
-
val drained = buf.drain(e => println(s"Processing $e"), 32)
|
|
101
|
-
println(s"Drained $drained elements")
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
### MPSC fan-in (multiple producers, single consumer)
|
|
105
|
-
|
|
106
|
-
```scala
|
|
107
|
-
import zio.blocks.ringbuffer.MpscRingBuffer
|
|
108
|
-
|
|
109
|
-
val buf = MpscRingBuffer[String](256)
|
|
110
|
-
|
|
111
|
-
// Multiple producer threads
|
|
112
|
-
for (i <- 0 until 4) {
|
|
113
|
-
new Thread(() => {
|
|
114
|
-
for (j <- 0 until 100)
|
|
115
|
-
buf.offer(s"producer-$i: message-$j")
|
|
116
|
-
}).start()
|
|
117
|
-
}
|
|
118
|
-
|
|
119
|
-
// Single consumer thread
|
|
120
|
-
new Thread(() => {
|
|
121
|
-
var msg = buf.take()
|
|
122
|
-
while (msg != null) {
|
|
123
|
-
println(msg)
|
|
124
|
-
msg = buf.take()
|
|
125
|
-
}
|
|
126
|
-
}).start()
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
### SPMC fan-out (single producer, multiple consumers)
|
|
130
|
-
|
|
131
|
-
```scala
|
|
132
|
-
import zio.blocks.ringbuffer.SpmcRingBuffer
|
|
133
|
-
|
|
134
|
-
val buf = SpmcRingBuffer[String](256)
|
|
135
|
-
|
|
136
|
-
// Single producer thread
|
|
137
|
-
new Thread(() => {
|
|
138
|
-
for (i <- 0 until 400)
|
|
139
|
-
while (!buf.offer(s"task-$i")) {} // retry if full
|
|
140
|
-
}).start()
|
|
141
|
-
|
|
142
|
-
// Multiple consumer (worker) threads
|
|
143
|
-
for (w <- 0 until 4) {
|
|
144
|
-
new Thread(() => {
|
|
145
|
-
var msg = buf.take()
|
|
146
|
-
while (msg != null) {
|
|
147
|
-
println(s"worker-$w: $msg")
|
|
148
|
-
msg = buf.take()
|
|
149
|
-
}
|
|
150
|
-
}).start()
|
|
151
|
-
}
|
|
152
|
-
```
|
|
153
|
-
|
|
154
|
-
### Non-blocking try-once with fallback
|
|
155
|
-
|
|
156
|
-
```scala
|
|
157
|
-
import zio.blocks.ringbuffer.MpmcRingBuffer
|
|
158
|
-
|
|
159
|
-
val buf = MpmcRingBuffer[String](64)
|
|
160
|
-
|
|
161
|
-
// Try to offer without blocking; handle backpressure yourself
|
|
162
|
-
if (!buf.offer("data")) {
|
|
163
|
-
// buffer is full — drop, log, or retry later
|
|
164
|
-
println("Buffer full, applying backpressure")
|
|
165
|
-
}
|
|
166
|
-
|
|
167
|
-
// Try to take without blocking
|
|
168
|
-
val result = buf.take()
|
|
169
|
-
if (result != null) {
|
|
170
|
-
println(s"Got: $result")
|
|
171
|
-
} else {
|
|
172
|
-
// buffer is empty — do other work
|
|
173
|
-
}
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
---
|
|
177
|
-
|
|
178
|
-
## Design notes
|
|
179
|
-
|
|
180
|
-
### FastFlow pattern (SPSC)
|
|
181
|
-
|
|
182
|
-
`SpscRingBuffer` uses the FastFlow algorithm: the **null/non-null state of an array slot** is the synchronization signal. The producer never reads `consumerIndex`; the consumer never reads `producerIndex`. This minimizes cross-core cache traffic to a single cache line per operation.
|
|
183
|
-
|
|
184
|
-
A **look-ahead step** (`capacity/4`, capped at 4096) lets the producer batch-check multiple future slots at once, further reducing the frequency of slow-path reads.
|
|
185
|
-
|
|
186
|
-
### Vyukov/Dmitry sequence buffer (MPMC)
|
|
187
|
-
|
|
188
|
-
`MpmcRingBuffer` uses a parallel `Long` sequence buffer alongside the data array. Each slot carries a stamp indicating whether it is available for writing or reading. Both `producerIndex` and `consumerIndex` are advanced via CAS, allowing any number of threads on both sides. The minimum capacity is 2 (the algorithm requires at least 2 slots to distinguish written from consumed).
|
|
189
|
-
|
|
190
|
-
### CAS-based producers (MPSC)
|
|
191
|
-
|
|
192
|
-
`MpscRingBuffer` follows the JCTools `MpscArrayQueue` design: producers claim a slot via CAS on `producerIndex`, then write the element with release semantics. A cached `producerLimit` avoids reading `consumerIndex` on every offer, reducing cross-core traffic. The consumer side uses relaxed poll semantics—a `null` slot means either empty or a producer mid-write.
|
|
193
|
-
|
|
194
|
-
### Index-based SPMC
|
|
195
|
-
|
|
196
|
-
`SpmcRingBuffer` uses index-based capacity checking on the producer side (no CAS needed for a single producer). Consumers use a CAS loop on `consumerIndex`. Consumers read the element *before* the CAS to avoid a race with the producer overwriting the slot. Consumers do not null array slots after reading—the producer overwrites them on the next lap.
|
|
197
|
-
|
|
198
|
-
### Cache-line padding
|
|
199
|
-
|
|
200
|
-
All ring buffers use **128-byte padding regions** (16 `Long` fields) between producer and consumer fields. This prevents false sharing on all architectures, including Apple Silicon which uses 128-byte cache lines (most x86 CPUs use 64-byte lines).
|
|
201
|
-
|
|
202
|
-
The padding is implemented via a class hierarchy:
|
|
203
|
-
|
|
204
|
-
```
|
|
205
|
-
Pad0 → ProducerFields → Pad1 → ConsumerFields → Pad2
|
|
206
|
-
```
|
|
207
|
-
|
|
208
|
-
### VarHandle for memory ordering
|
|
209
|
-
|
|
210
|
-
All JVM implementations use `java.lang.invoke.VarHandle` (Java 9+) for acquire/release semantics instead of `sun.misc.Unsafe`. This is the recommended modern approach for lock-free data structures on the JVM.
|
|
211
|
-
|
|
212
|
-
### Power-of-two masking
|
|
213
|
-
|
|
214
|
-
Capacity must be a power of two. This allows `index & (capacity - 1)` instead of `index % capacity`, which is significantly faster because bitwise AND compiles to a single CPU instruction.
|
|
215
|
-
|
|
216
|
-
---
|
|
217
|
-
|
|
218
|
-
## Thread-safety contract
|
|
219
|
-
|
|
220
|
-
Violating the threading contract (e.g., calling `take` from multiple threads on an `SpscRingBuffer`) results in **undefined behavior**. No runtime check is performed—this is enforced by contract for maximum performance.
|
|
221
|
-
|
|
222
|
-
| Type | `offer` | `take` |
|
|
223
|
-
|------|---------|--------|
|
|
224
|
-
| `SpscRingBuffer` | Single producer thread only | Single consumer thread only |
|
|
225
|
-
| `SpmcRingBuffer` | Single producer thread only | Any number of consumer threads |
|
|
226
|
-
| `MpscRingBuffer` | Any number of producer threads | Single consumer thread only |
|
|
227
|
-
| `MpmcRingBuffer` | Any number of producer threads | Any number of consumer threads |
|
|
228
|
-
|
|
229
|
-
---
|
|
230
|
-
|
|
231
|
-
## Performance characteristics
|
|
232
|
-
|
|
233
|
-
| Operation | Complexity |
|
|
234
|
-
|-----------|-----------|
|
|
235
|
-
| `offer` | Lock-free (SPSC/SPMC: wait-free) |
|
|
236
|
-
| `take` | Lock-free (SPSC/MPSC: wait-free) |
|
|
237
|
-
|
|
238
|
-
**SPSC** is the fastest: no CAS, no locks, minimal cache-line traffic. Use it whenever your threading model allows a dedicated producer-consumer pair.
|
|
239
|
-
|
|
240
|
-
---
|
|
241
|
-
|
|
242
|
-
## Cross-platform support
|
|
243
|
-
|
|
244
|
-
On **Scala.js**, all ring buffer types compile and provide the same API surface. Since Scala.js is single-threaded, the JS implementations use plain reads and writes with no memory ordering primitives.
|
|
245
|
-
|
|
246
|
-
| Platform | Support |
|
|
247
|
-
|----------|---------|
|
|
248
|
-
| JVM | Full concurrency support |
|
|
249
|
-
| Scala.js | Sequential (same API) |
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|