@zio.dev/zio-blocks 0.0.32 → 0.0.51
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/guides/compile-time-resource-safety-with-scope.md +16 -17
- package/guides/getting-started-with-mux.md +1507 -0
- package/guides/query-dsl-extending.md +161 -102
- package/guides/query-dsl-fluent-builder.md +217 -157
- package/guides/query-dsl-reified-optics.md +12 -10
- package/guides/query-dsl-sql.md +246 -165
- package/guides/telemetry-guide.md +1069 -0
- package/guides/zio-schema-migration.md +29 -22
- package/index.md +293 -51
- package/package.json +1 -1
- package/plans/config-follow-up-prs.md +188 -0
- package/plans/config-pr-assessment-roadmap.md +310 -0
- package/reference/MuxDataFlow.jsx +250 -0
- package/reference/async.md +651 -0
- package/reference/chunk.md +3533 -308
- package/reference/codegen/case-class.md +436 -0
- package/reference/codegen/emitter-config.md +383 -0
- package/reference/codegen/examples.md +664 -0
- package/reference/codegen/field.md +316 -0
- package/reference/codegen/index.md +317 -0
- package/reference/codegen/scala-emitter.md +392 -0
- package/reference/codegen/scala-file.md +276 -0
- package/reference/codegen/sealed-trait.md +408 -0
- package/reference/codegen/type-definition.md +340 -0
- package/reference/codegen/type-ref.md +201 -0
- package/reference/combinators.md +347 -117
- package/reference/config.md +158 -0
- package/reference/context.md +4 -4
- package/reference/datastar.md +346 -0
- package/reference/docs.md +1461 -345
- package/reference/endpoint/auth-type.md +146 -0
- package/reference/endpoint/endpoint.md +297 -0
- package/reference/endpoint/http-codec.md +249 -0
- package/reference/endpoint/index.md +825 -0
- package/reference/endpoint/path-codec.md +237 -0
- package/reference/endpoint/route-pattern.md +196 -0
- package/reference/endpoint/route-tree.md +111 -0
- package/reference/endpoint/segment-codec.md +212 -0
- package/reference/html.md +1120 -0
- package/reference/htmx/attribute-values.md +359 -0
- package/reference/htmx/hx-encoding.md +111 -0
- package/reference/htmx/hx-params.md +204 -0
- package/reference/htmx/hx-swap.md +276 -0
- package/reference/htmx/hx-sync.md +251 -0
- package/reference/htmx/hx-target.md +314 -0
- package/reference/htmx/hx-trigger.md +457 -0
- package/reference/htmx/hx-url-update.md +239 -0
- package/reference/htmx/index.md +855 -0
- package/reference/http-model/index.md +47 -0
- package/reference/http-model/model.md +1481 -0
- package/reference/http-model/schema.md +747 -0
- package/reference/maybe.md +826 -0
- package/reference/media-type.md +2 -2
- package/reference/mux.mdx +823 -0
- package/reference/openapi.md +1351 -0
- package/reference/resource-management/defer-handle.md +1 -1
- package/reference/resource-management/resource.md +31 -2
- package/reference/resource-management/scope.md +28 -12
- package/reference/resource-management/wire.md +3 -7
- package/reference/ringbuffer/MpmcDiagram.jsx +717 -0
- package/reference/ringbuffer/MpscDiagram.jsx +618 -0
- package/reference/ringbuffer/SpmcDiagram.jsx +680 -0
- package/reference/ringbuffer/SpscDiagram.jsx +677 -0
- package/reference/ringbuffer/advanced.mdx +109 -0
- package/reference/ringbuffer/index.mdx +145 -0
- package/reference/ringbuffer/mpmc.mdx +151 -0
- package/reference/ringbuffer/mpsc.mdx +132 -0
- package/reference/ringbuffer/spmc.mdx +108 -0
- package/reference/ringbuffer/spsc.mdx +344 -0
- package/reference/{allows.md → schema/allows.md} +4 -4
- package/reference/{binding-resolver.md → schema/binding-resolver.md} +1 -1
- package/reference/{binding.md → schema/binding.md} +2 -3
- package/reference/schema/built-in-codecs/avro.md +451 -0
- package/reference/schema/built-in-codecs/bson.md +480 -0
- package/reference/schema/built-in-codecs/csv.md +564 -0
- package/reference/schema/built-in-codecs/index.md +77 -0
- package/reference/schema/built-in-codecs/json/index.md +295 -0
- package/reference/schema/built-in-codecs/json/json-config.md +217 -0
- package/reference/{json-patch.md → schema/built-in-codecs/json/json-patch.md} +3 -3
- package/reference/{json-schema.md → schema/built-in-codecs/json/json-schema.md} +14 -47
- package/reference/schema/built-in-codecs/json/json-selection.md +322 -0
- package/reference/{json.md → schema/built-in-codecs/json/json.md} +32 -64
- package/reference/schema/built-in-codecs/messagepack.md +508 -0
- package/reference/schema/built-in-codecs/thrift.md +433 -0
- package/reference/schema/built-in-codecs/toon.md +1078 -0
- package/reference/{xml.md → schema/built-in-codecs/xml.md} +34 -192
- package/reference/schema/built-in-codecs/yaml.md +552 -0
- package/reference/{codec.md → schema/codec.md} +10 -10
- package/reference/{dynamic-optic.md → schema/dynamic-optic.md} +151 -5
- package/reference/{dynamic-schema.md → schema/dynamic-schema.md} +8 -8
- package/reference/schema/format.md +92 -0
- package/reference/schema/index.md +50 -0
- package/reference/schema/migration.md +297 -0
- package/reference/{modifier.md → schema/modifier.md} +58 -7
- package/reference/{optics.md → schema/optics.md} +2 -2
- package/reference/{patch.md → schema/patch.md} +1 -1
- package/{path-interpolator.md → reference/schema/path-interpolator.md} +165 -72
- package/reference/{schema-evolution → schema/schema-evolution}/as.md +5 -5
- package/reference/{schema-evolution → schema/schema-evolution}/index.md +2 -2
- package/reference/{schema-evolution → schema/schema-evolution}/into.md +3 -3
- package/reference/{schema-expr.md → schema/schema-expr.md} +110 -175
- package/reference/{schema.md → schema/schema.md} +13 -1
- package/reference/{structural-types.md → schema/structural-types.md} +1 -1
- package/reference/{type-class-derivation.md → schema/type-class-derivation.md} +63 -1
- package/reference/smithy.md +533 -0
- package/reference/sql/db-codec-deriver.md +71 -0
- package/reference/sql/db-codec.md +687 -0
- package/reference/sql/db-con.md +271 -0
- package/reference/sql/db-connection.md +153 -0
- package/reference/sql/db-param-writer.md +77 -0
- package/reference/sql/db-param.md +66 -0
- package/reference/sql/db-result-reader.md +146 -0
- package/reference/sql/db-tx.md +82 -0
- package/reference/sql/db-value.md +41 -0
- package/reference/sql/ddl.md +85 -0
- package/reference/sql/frag.md +254 -0
- package/reference/sql/index.md +341 -0
- package/reference/sql/repo.md +600 -0
- package/reference/sql/sql-dialect.md +73 -0
- package/reference/sql/sql-logger.md +62 -0
- package/reference/sql/sql-name-mapper.md +70 -0
- package/reference/sql/table-metadata.md +134 -0
- package/reference/sql/table.md +448 -0
- package/reference/sql/transactor-zio.md +399 -0
- package/reference/sql/transactor.md +353 -0
- package/reference/sql-zio.md +112 -0
- package/reference/streams/concurrent-operators.md +106 -0
- package/reference/streams/index.md +653 -0
- package/reference/streams/pipeline.md +718 -0
- package/reference/streams/reader.md +1284 -0
- package/reference/streams/scala-2-compatibility.md +55 -0
- package/reference/streams/sink.md +1426 -0
- package/reference/streams/stream.md +2526 -0
- package/reference/streams/writer.md +1045 -0
- package/reference/streams/zero-boxing.md +275 -0
- package/reference/telemetry.md +693 -0
- package/reference/typeid.md +2922 -583
- package/sidebars.js +238 -43
- package/superpowers/plans/2026-03-19-docs-critique-subagent.md +407 -0
- package/superpowers/specs/2026-03-19-docs-critique-subagent-design.md +222 -0
- package/reference/formats.md +0 -694
- package/reference/http-model.md +0 -1716
- package/reference/streams.md +0 -989
- package/ringbuffer.md +0 -249
- /package/reference/{json-differ.md → schema/built-in-codecs/json/json-differ.md} +0 -0
- /package/reference/{dynamic-value.md → schema/dynamic-value.md} +0 -0
- /package/reference/{lazy.md → schema/lazy.md} +0 -0
- /package/reference/{reflect.md → schema/reflect.md} +0 -0
- /package/reference/{registers.md → schema/registers.md} +0 -0
- /package/reference/{schema-error.md → schema/schema-error.md} +0 -0
- /package/reference/{syntax.md → schema/syntax.md} +0 -0
- /package/reference/{validation.md → schema/validation.md} +0 -0
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: advanced
|
|
3
|
+
title: "Advanced Topics"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
## Thread Safety and Correctness
|
|
7
|
+
|
|
8
|
+
Ring buffers are **lock-free** but must be used correctly:
|
|
9
|
+
|
|
10
|
+
- **Wrong thread access causes undefined behavior**: Using `SpscRingBuffer` from multiple producer threads produces data races, silent data loss, or crashes. Always use the implementation matching your thread pattern.
|
|
11
|
+
- **`SpscRingBuffer#offer` and `SpscRingBuffer#take` thread contract**: The producer thread must be the sole caller of `SpscRingBuffer#offer`; the consumer thread must be the sole caller of `SpscRingBuffer#take`. They may be the same physical thread (as in single-threaded environments like Scala.js or unit tests) or different threads.
|
|
12
|
+
- **State queries are approximate**: Under concurrency, `SpscRingBuffer#size`, `SpscRingBuffer#isEmpty`, and `SpscRingBuffer#isFull` may stay stale by the time they return. Do not rely on them for exact synchronization — use `SpscRingBuffer#offer`'s return value for backpressure instead.
|
|
13
|
+
- **Null elements are forbidden**: All implementations reject `null` with `NullPointerException`. If you need to store nullable values, wrap them in `Option` or another container.
|
|
14
|
+
|
|
15
|
+
:::warning
|
|
16
|
+
**Critical:** Ring buffers do not enforce thread-safety at runtime. Using the wrong implementation for your thread pattern or calling methods from the wrong thread **does not throw an exception** — it silently corrupts data. Test thoroughly and document your threading contract.
|
|
17
|
+
:::
|
|
18
|
+
|
|
19
|
+
## Advanced Usage: Cache-Line Padding
|
|
20
|
+
|
|
21
|
+
Ring buffers use **cache-line padding** to prevent false sharing between producer and consumer indices on modern CPUs. The padding is transparent to users but enables dramatically lower latency on multi-core systems.
|
|
22
|
+
|
|
23
|
+
Each implementation pads its internal index fields (producer index, consumer index) to occupy a full cache line (128 bytes on Apple Silicon, 64 bytes on most other architectures). This ensures that when one thread reads its index, it does not invalidate the cache line holding the other thread's index, eliminating costly cache-coherency traffic.
|
|
24
|
+
|
|
25
|
+
This optimization is automatic and requires no configuration. Ring buffers are inherently more efficient than comparable Scala and Java implementations because of this padding.
|
|
26
|
+
|
|
27
|
+
## Designing With Ring Buffers
|
|
28
|
+
|
|
29
|
+
Common patterns for using ring buffers include:
|
|
30
|
+
|
|
31
|
+
### Pattern: Producer-Consumer Pipeline
|
|
32
|
+
|
|
33
|
+
Ring buffers form the backbone of producer-consumer pipelines where one or more producers generate work and one or more consumers process it:
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
┌──────────┐ ┌──────────────┐ ┌──────────┐
|
|
37
|
+
│Producer 1├──────>│ RingBuffer │<──────┤Consumer 1│
|
|
38
|
+
│Producer 2├──────>│ (MPMC, cap=N)│<──────┤Consumer 2│
|
|
39
|
+
└──────────┘ └──────────────┘ └──────────┘
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
In this pattern:
|
|
43
|
+
- Producers call `MpmcRingBuffer#offer` and handle backpressure if `false` is returned (e.g., retry, queue internally, apply rate limiting).
|
|
44
|
+
- Consumers call `MpmcRingBuffer#take` in a tight loop, checking for `null` to detect empty buffers.
|
|
45
|
+
- Ring buffer capacity bounds memory and provides natural backpressure.
|
|
46
|
+
|
|
47
|
+
### Pattern: Batch Processing
|
|
48
|
+
|
|
49
|
+
For workloads where producers batch elements together, use `SpscRingBuffer#fill` (SPSC) or call `SpscRingBuffer#offer` in a loop:
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
Producer fills batch of N items
|
|
53
|
+
↓
|
|
54
|
+
Ring Buffer (growing)
|
|
55
|
+
↓
|
|
56
|
+
Consumer drain()s batch of M items
|
|
57
|
+
↓
|
|
58
|
+
Process batch
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Batching reduces per-element synchronization costs.
|
|
62
|
+
|
|
63
|
+
### Pattern: Work Stealing with Multiple Consumers (MPMC)
|
|
64
|
+
|
|
65
|
+
When multiple workers consume from the same queue, use `MpmcRingBuffer`. Each worker calls `MpmcRingBuffer#take` to grab the next item atomically:
|
|
66
|
+
|
|
67
|
+
```scala mdoc:compile-only
|
|
68
|
+
import zio.blocks.ringbuffer.MpmcRingBuffer
|
|
69
|
+
|
|
70
|
+
case class Task(id: Int, work: String)
|
|
71
|
+
|
|
72
|
+
val queue = MpmcRingBuffer[Task](256)
|
|
73
|
+
|
|
74
|
+
def worker(): Unit = {
|
|
75
|
+
while (true) {
|
|
76
|
+
val task = queue.take()
|
|
77
|
+
if (task ne null) {
|
|
78
|
+
println(s"Processing: ${task.work}")
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
The CAS loop in `MpmcRingBuffer#take` ensures no two workers grab the same task.
|
|
85
|
+
|
|
86
|
+
## Performance Characteristics
|
|
87
|
+
|
|
88
|
+
All ring buffer implementations provide O(1) time complexity for `offer`, `take`, `size`, `isEmpty`, and `isFull` operations.
|
|
89
|
+
|
|
90
|
+
- **SPSC** (FastFlow) — Fastest: avoids volatile reads on the fast path, minimal cache traffic
|
|
91
|
+
- **SPMC** — Fast: producer uses index-based checking; consumers CAS on a shared index
|
|
92
|
+
- **MPSC** — Fast: producers CAS on a shared index with a cached limit; consumer uses FastFlow relaxed-poll
|
|
93
|
+
- **MPMC** — Slightly slower: uses sequence buffer stamps for coordination; all indices use CAS
|
|
94
|
+
|
|
95
|
+
Actual performance depends on:
|
|
96
|
+
- **CPU cache architecture** — 64-byte vs 128-byte cache lines affect padding efficiency
|
|
97
|
+
- **Contention level** — high contention increases CAS failure rates and retries
|
|
98
|
+
- **Element size** — larger elements may affect cache locality
|
|
99
|
+
- **Platform** — JVM JIT warmup, Scala.js compiled code, GraalVM-generated native image
|
|
100
|
+
|
|
101
|
+
Micro-benchmark your specific workload if latency is critical.
|
|
102
|
+
|
|
103
|
+
## Integration with Other ZIO Blocks Types
|
|
104
|
+
|
|
105
|
+
Ring buffers are standalone data structures and do not depend on other ZIO Blocks types. However, they integrate well with:
|
|
106
|
+
|
|
107
|
+
- **Threading models**: Ring buffers work on raw JVM threads, virtual threads (Loom), or platform-specific threads. Pair with `ZIO.fork` or `Thread` as needed.
|
|
108
|
+
- **Reactive streams**: Ring buffers can back reactive sources, where producers feed a `Source` and consumers pull from it. The ring buffer provides natural backpressure via `offer`'s return value.
|
|
109
|
+
- **Event loops**: In game engines or event-driven systems, ring buffers connect event producers (input, network) to event dispatchers (main loop) with predictable latency.
|
|
@@ -0,0 +1,145 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: index
|
|
3
|
+
title: "RingBuffer"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Ring buffers are **fixed-size, lock-free queues** for efficiently exchanging elements between producer and consumer threads with minimal contention and cache-line effects. They use a circular array to recycle memory, eliminating garbage collection pressure from transient allocations. The `zio.blocks.ringbuffer` module provides four specialized implementations tuned for different producer/consumer thread patterns:
|
|
7
|
+
|
|
8
|
+
```scala
|
|
9
|
+
final class SpscRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
10
|
+
def offer(a: A): Boolean
|
|
11
|
+
def take(): A
|
|
12
|
+
def size: Int
|
|
13
|
+
def isEmpty: Boolean
|
|
14
|
+
def isFull: Boolean
|
|
15
|
+
def drain(consumer: A => Unit, limit: Int): Int
|
|
16
|
+
def fill(supplier: () => A, limit: Int): Int
|
|
17
|
+
}
|
|
18
|
+
|
|
19
|
+
final class SpmcRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
20
|
+
def offer(a: A): Boolean
|
|
21
|
+
def take(): A
|
|
22
|
+
def size: Int
|
|
23
|
+
def isEmpty: Boolean
|
|
24
|
+
def isFull: Boolean
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
final class MpscRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
28
|
+
def offer(a: A): Boolean
|
|
29
|
+
def take(): A
|
|
30
|
+
def size: Int
|
|
31
|
+
def isEmpty: Boolean
|
|
32
|
+
def isFull: Boolean
|
|
33
|
+
def drain(consumer: A => Unit, limit: Int): Int
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
final class MpmcRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
37
|
+
def offer(a: A): Boolean
|
|
38
|
+
def take(): A
|
|
39
|
+
def size: Int
|
|
40
|
+
def isEmpty: Boolean
|
|
41
|
+
def isFull: Boolean
|
|
42
|
+
}
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Motivation
|
|
46
|
+
|
|
47
|
+
Building low-latency systems — trading platforms, game engines, real-time event processors — requires careful control over memory allocation and CPU cache behavior. Standard JVM collections like `LinkedList` or `ArrayDeque` are convenient but have a cost: every enqueue/dequeue pair allocates a node, triggering garbage collection pauses that can destroy millisecond-scale latencies.
|
|
48
|
+
|
|
49
|
+
A naive approach is to pre-allocate a large `Array[A]` and manually manage head/tail indices. This code shows a simplified single-threaded version:
|
|
50
|
+
|
|
51
|
+
```scala mdoc:compile-only
|
|
52
|
+
class SimpleRingBuffer {
|
|
53
|
+
// WARNING: Mutable state (vars) demonstrated here to show the problem;
|
|
54
|
+
// this is NOT how ring buffers actually solve concurrency.
|
|
55
|
+
private var head = 0
|
|
56
|
+
private var tail = 0
|
|
57
|
+
private val array = new Array[String](1024)
|
|
58
|
+
|
|
59
|
+
def offer(x: String): Boolean = {
|
|
60
|
+
if (tail - head >= array.length) false // full
|
|
61
|
+
else {
|
|
62
|
+
array(tail % array.length) = x
|
|
63
|
+
tail += 1
|
|
64
|
+
true
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
def take(): String = {
|
|
69
|
+
if (head == tail) null // empty
|
|
70
|
+
else {
|
|
71
|
+
val x = array(head % array.length)
|
|
72
|
+
head += 1
|
|
73
|
+
x
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
This works for single-threaded code, but introduces a critical problem under concurrency: both threads read and write `head` and `tail` without synchronization, leading to lost updates, stale reads, and silent data corruption. Adding `synchronized` blocks solves the data race but reintroduces contention and latency pauses.
|
|
80
|
+
|
|
81
|
+
Ring buffers solve both problems with **lock-free algorithms** and **cache-line padding**. A ring buffer provides:
|
|
82
|
+
- **No garbage collection** — reuses the same array forever
|
|
83
|
+
- **Lock-free access** — uses atomic compare-and-swap (CAS) for coordination, avoiding mutex contention
|
|
84
|
+
- **Predictable latency** — no surprise GC pauses or lock waits
|
|
85
|
+
- **Thread-specialized variants** — choose SPSC, MPSC, SPMC, or MPMC based on your thread pattern
|
|
86
|
+
|
|
87
|
+
This module provides four implementations tuned for maximum throughput and minimal latency across all producer/consumer combinations.
|
|
88
|
+
|
|
89
|
+
## Overview
|
|
90
|
+
|
|
91
|
+
Ring buffers are high-performance data structures for:
|
|
92
|
+
|
|
93
|
+
- **Event queues** in IO, networking, and game engines where throughput and latency matter
|
|
94
|
+
- **Thread-safe work queues** with bounded capacity to backpressure senders
|
|
95
|
+
- **Inter-thread communication** between producers and consumers with predictable latency
|
|
96
|
+
- **Concurrent batch processing** where producers fill elements and consumers drain them
|
|
97
|
+
|
|
98
|
+
## Why Ring Buffers
|
|
99
|
+
|
|
100
|
+
Ring buffers excel when you need:
|
|
101
|
+
|
|
102
|
+
- **Ultra-low latency** — lock-free algorithms and cache-line padding eliminate pauses from contention
|
|
103
|
+
- **Predictable throughput** — no GC overhead since the same memory array is reused forever
|
|
104
|
+
- **Bounded resources** — fixed capacity prevents runaway memory growth and enables backpressure
|
|
105
|
+
- **High concurrency** — multiple implementations optimized for different thread patterns (SPSC, MPMC, etc.)
|
|
106
|
+
|
|
107
|
+
How do ring buffers compare to other queue and collection types?
|
|
108
|
+
|
|
109
|
+
### Comparison with Java and Scala Alternatives
|
|
110
|
+
|
|
111
|
+
| Property | RingBuffer (ZIO Blocks) | `java.util.Queue` (ConcurrentLinkedQueue) | `scala.collection.concurrent.TrieMap` | Array + manual index | Disruptor |
|
|
112
|
+
|-------------------------|----------------------------------------|-------------------------------------------|---------------------------------------|--------------------------|--------------------------|
|
|
113
|
+
| **Allocation** | Single upfront | Per-element nodes | Per-node allocations | Single upfront | Single upfront |
|
|
114
|
+
| **Lock-free** | Yes (CAS-based) | Yes | Yes (CASes on trie nodes) | Yes (if single-threaded) | Yes (CAS) |
|
|
115
|
+
| **GC pressure** | Minimal (reuses slots) | High (node garbage) | High (trie node garbage) | Minimal | Minimal |
|
|
116
|
+
| **Bounded** | Fixed capacity | Unbounded | Unbounded | Fixed | Fixed (bounded capacity) |
|
|
117
|
+
| **Thread patterns** | Four variants (SPSC, MPSC, SPMC, MPMC) | Multi-producer/multi-consumer | Multi-reader/multi-writer | Limited (see impl) | Multi-producer/consumer |
|
|
118
|
+
| **Predictable latency** | High (no GC) | Medium (GC pauses) | Medium (GC pauses) | High (if no contention) | High |
|
|
119
|
+
|
|
120
|
+
RingBuffer is ideal when you control both producer and consumer thread counts and want maximum performance. `java.util.Queue` is better if you need unbounded capacity; Disruptor is a comparable JVM alternative with similar guarantees.
|
|
121
|
+
|
|
122
|
+
## Installation
|
|
123
|
+
|
|
124
|
+
Add the ZIO Blocks Ring Buffer module to your `build.sbt`:
|
|
125
|
+
|
|
126
|
+
```scala
|
|
127
|
+
libraryDependencies += "dev.zio" %% "zio-blocks-ringbuffer" % "@VERSION@"
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
For Scala.js cross-platform support:
|
|
131
|
+
|
|
132
|
+
```scala
|
|
133
|
+
libraryDependencies += "dev.zio" %%% "zio-blocks-ringbuffer" % "@VERSION@"
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
## Implementations
|
|
137
|
+
|
|
138
|
+
ZIO Blocks provides four optimized ring buffer implementations. Choose the one matching your producer/consumer thread counts:
|
|
139
|
+
|
|
140
|
+
- **SPSC** — Single Producer, Single Consumer (⚡⚡⚡ **Fastest**)
|
|
141
|
+
- **SPMC** — Single Producer, Multiple Consumers
|
|
142
|
+
- **MPSC** — Multiple Producers, Single Consumer
|
|
143
|
+
- **MPMC** — Multiple Producers, Multiple Consumers
|
|
144
|
+
|
|
145
|
+
Use the sidebar for the per-implementation guides and the advanced topics page covering thread safety, cache-line padding, design patterns, and performance characteristics.
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: mpmc
|
|
3
|
+
title: "MPMC RingBuffer"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
import MpmcDiagram from './MpmcDiagram.jsx';
|
|
7
|
+
|
|
8
|
+
`MpmcRingBuffer[A]` is the fully general-purpose implementation for systems with multiple producer and consumer threads. It uses the **Vyukov/Dmitry sequence-buffer algorithm**, a sophisticated lock-free design that coordinates all access through monotonically increasing indices and per-slot sequence stamps—making every slot's state self-describing at any moment.
|
|
9
|
+
|
|
10
|
+
## Algorithm
|
|
11
|
+
|
|
12
|
+
`MpmcRingBuffer` handles the hardest case: **many producers and many consumers** all accessing the same buffer at once. It uses the **Vyukov/Dmitry algorithm**, which is a lock-free MPMC queue design.
|
|
13
|
+
|
|
14
|
+
The challenge: when multiple producers might try to write to the same slot, and multiple consumers might try to read the same slot, we need a fair way to say "this slot is mine" without using locks.
|
|
15
|
+
|
|
16
|
+
**The trick: Give each slot a "ticket number" that changes in a predictable cycle.**
|
|
17
|
+
|
|
18
|
+
The algorithm runs two parallel arrays of the same length:
|
|
19
|
+
|
|
20
|
+
- `buffer[i]` — holds the actual element at position `i`
|
|
21
|
+
- `seqBuf[i]` — holds a *sequence stamp* at position `i`
|
|
22
|
+
|
|
23
|
+
The sequence stamps are the heart of the algorithm. Each stamp encodes the *ownership state* of its slot: who is allowed to act on it and what they are allowed to do.
|
|
24
|
+
|
|
25
|
+
On initialization, `seqBuf[i]` is set to `i`. The producer index `pIdx` and consumer index `cIdx` both start at zero. From here, every operation follows the same pattern: read the stamp, compute a single difference (`diff`), and branch on whether `diff` is zero, negative, or positive.
|
|
26
|
+
|
|
27
|
+
At any moment, slot `i` (where `i = index & mask`) is in exactly one of three states:
|
|
28
|
+
|
|
29
|
+
| stamp value | meaning |
|
|
30
|
+
|-----------------------------|--------------------------------------------------|
|
|
31
|
+
| if `seq == pIdx` | Slot is free — the producer at `pIdx` may write |
|
|
32
|
+
| if `seq == cIdx + 1` | Data written — the consumer at `cIdx` may read |
|
|
33
|
+
| if `seq == cIdx + capacity` | Slot consumed — free for the producer's next lap |
|
|
34
|
+
|
|
35
|
+
The producer looks for `diff = seq - pIdx == 0`. The consumer looks for `diff = seq - (cIdx + 1) == 0`. In both cases, a negative diff means the other side has fallen behind (buffer full or empty), and a positive diff means another thread already claimed this slot and you should retry.
|
|
36
|
+
|
|
37
|
+
### Diagram
|
|
38
|
+
|
|
39
|
+
To see the sequence buffer in action, use this interactive stepper. The component below implements the algorithm faithfully in React. Type any label, click **Offer** to enqueue or **Take** to dequeue, and watch the trace panel show every intermediate variable — `pIdx`, `slot`, `seq`, `diff` — and the exact decision the algorithm makes from them.
|
|
40
|
+
|
|
41
|
+
<MpmcDiagram />
|
|
42
|
+
|
|
43
|
+
Click "Step Producer" and "Step Consumer" to see how the algorithm coordinates access handoff without locks.
|
|
44
|
+
|
|
45
|
+
Here is a complete walkthrough of every variable in the trace, in the order the algorithm computes them.
|
|
46
|
+
|
|
47
|
+
### `pIdx` / `cIdx` — the monotonic counters
|
|
48
|
+
|
|
49
|
+
These two numbers are the heartbeat of the entire algorithm. `pIdx` is the producer's counter and `cIdx` is the consumer's counter. They start at zero and *only ever increase* — they never wrap, never reset, never go backwards. After a million operations `pIdx` might be 1,000,000 and `cIdx` might be 999,996. The raw slot position is derived from them rather than stored directly, which is what makes the algorithm safe for multiple concurrent threads.
|
|
50
|
+
|
|
51
|
+
### `slot = idx & mask` — the circular array index
|
|
52
|
+
|
|
53
|
+
Because the buffer has a power-of-two capacity (4 in the demo), `mask = capacity - 1 = 3`, which in binary is `0011`. The bitwise AND strips everything above the lowest two bits, giving a number in the range `[0, 3]`. This is mathematically identical to `idx % capacity` but costs a single CPU instruction instead of a division. So `pIdx = 7` maps to slot `7 & 3 = 3`, and `pIdx = 8` maps back to slot `8 & 3 = 0` — that is the wrap-around.
|
|
54
|
+
|
|
55
|
+
### `seq = seqBuf[slot]` — the sequence stamp
|
|
56
|
+
|
|
57
|
+
Every slot carries its own stamp, completely independent of the other slots. The stamp is not a lock and not a boolean "occupied/free" flag — it is a number that encodes the *exact generation* of the slot. On construction `seqBuf[i] = i`, so slot 0 starts at 0, slot 1 at 1, and so on. After each write the stamp advances by 1. After each consume it advances by `capacity`. Because of this, slot 0's stamp trail across three laps looks like `0 → 1 → 4 → 5 → 8 → 9 → 12 → 13 …` — it grows forever and never repeats, which is what prevents the ABA problem entirely.
|
|
58
|
+
|
|
59
|
+
### `expected` (take only) `= cIdx + 1`
|
|
60
|
+
|
|
61
|
+
The producer, after winning its CAS and writing data, stamps the slot with `pIdx + 1`. So if a producer claimed slot 0 when `pIdx` was 4, it leaves `seqBuf[0] = 5`. The consumer that arrives with `cIdx = 4` therefore looks for `seqBuf[0] == cIdx + 1 == 5`. The `+ 1` is the handshake signal: *"a producer has finished writing here, and you are the right consumer to read it."* The offer trace does not need an `expected` row because the producer compares `seq` directly against `pIdx` (not `pIdx + 1`) — the slot is free when the stamp equals the producer index exactly.
|
|
62
|
+
|
|
63
|
+
### `diff` — the three-way decision
|
|
64
|
+
|
|
65
|
+
This is the key insight of the Vyukov algorithm. A single subtraction replaces what would otherwise be a tangle of conditional checks.
|
|
66
|
+
|
|
67
|
+
For **offer**: `diff = seq − pIdx`
|
|
68
|
+
- `diff == 0` — the stamp exactly matches the producer index, meaning no one has touched this slot since it was last released. The slot is yours. The thread does a CAS on `pIdx`, writes the element, then stamps `seqBuf[slot] = pIdx + 1`.
|
|
69
|
+
- `diff < 0` — the stamp is *behind* the producer index. This means the slot is still occupied by data from the current lap that has not been consumed yet. The buffer is full. Return `false`.
|
|
70
|
+
- `diff > 0` — the stamp is *ahead* of the producer index. Another producer already claimed this slot and advanced past it. Retry the loop with a fresh read of `pIdx`.
|
|
71
|
+
|
|
72
|
+
For **take**: `diff = seq − expected` where `expected = cIdx + 1`
|
|
73
|
+
- `diff == 0` — the stamp matches exactly what the producer left. Data is ready. CAS on `cIdx`, read the element, null out the slot for GC, stamp `seqBuf[slot] = cIdx + capacity` to release the slot for a future producer on the next lap.
|
|
74
|
+
- `diff < 0` — the stamp is behind what the consumer expects, meaning the producer has not finished writing yet (or has not written at all). The buffer appears empty from this consumer's perspective. Return `null`.
|
|
75
|
+
- `diff > 0` — another consumer already read this slot and advanced past it. Retry.
|
|
76
|
+
|
|
77
|
+
### Why all three decisions are safe without any lock
|
|
78
|
+
|
|
79
|
+
The diff check and the subsequent CAS form an atomic claim. Two producers might both read the same `pIdx` and both see `diff == 0`, but only one will win the CAS that advances `pIdx`. The loser sees the CAS fail, loops back, reads the new `pIdx`, and naturally ends up looking at the next slot. No explicit coordination between threads is ever needed — the sequence stamps and the monotonically increasing indices together make every slot's state self-describing at any point in time.
|
|
80
|
+
|
|
81
|
+
## Creating Instances
|
|
82
|
+
|
|
83
|
+
`MpmcRingBuffer[A]` is the fully general-purpose implementation supporting any number of producers and consumers. It uses the Vyukov/Dmitry algorithm with a parallel sequence buffer to coordinate access safely.
|
|
84
|
+
|
|
85
|
+
```scala
|
|
86
|
+
object MpmcRingBuffer {
|
|
87
|
+
def apply[A <: AnyRef](capacity: Int): MpmcRingBuffer[A]
|
|
88
|
+
}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Creates a new MPMC ring buffer with the given capacity. The capacity must be a power of two >= 2 (the sequence buffer algorithm requires at least 2 slots). A capacity of 1 is valid for `SpscRingBuffer`, `SpmcRingBuffer`, and `MpscRingBuffer`, but `MpmcRingBuffer` needs at least 2 because its sequence stamp mechanism requires a second slot to distinguish the written-but-not-consumed state from the empty state.
|
|
92
|
+
|
|
93
|
+
To create an MPMC buffer:
|
|
94
|
+
|
|
95
|
+
```scala mdoc:compile-only
|
|
96
|
+
import zio.blocks.ringbuffer.MpmcRingBuffer
|
|
97
|
+
|
|
98
|
+
val rb = MpmcRingBuffer[String](128)
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## Operations
|
|
102
|
+
|
|
103
|
+
`MpmcRingBuffer` provides operations for multiple producers and consumers to exchange elements:
|
|
104
|
+
|
|
105
|
+
### Inserting Elements — `MpmcRingBuffer#offer`
|
|
106
|
+
|
|
107
|
+
Use `MpmcRingBuffer#offer` to insert an element with this signature:
|
|
108
|
+
|
|
109
|
+
```scala
|
|
110
|
+
final class MpmcRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
111
|
+
def offer(a: A): Boolean
|
|
112
|
+
}
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Inserts the element without blocking. Returns `true` on successful insertion, `false` when the buffer is full. Raises `NullPointerException` if the element is `null`. Thread-safe; multiple producer threads may call this concurrently.
|
|
116
|
+
|
|
117
|
+
### Removing Elements — `MpmcRingBuffer#take`
|
|
118
|
+
|
|
119
|
+
Use `MpmcRingBuffer#take` to remove an element with this signature:
|
|
120
|
+
|
|
121
|
+
```scala
|
|
122
|
+
final class MpmcRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
123
|
+
def take(): A
|
|
124
|
+
}
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
Retrieves and removes an element from the front of the buffer. Returns immediately without blocking, providing the element or `null` if the buffer is empty. Thread-safe; multiple consumer threads may call this concurrently.
|
|
128
|
+
|
|
129
|
+
### Checking State — `size`, `isEmpty`, `isFull`
|
|
130
|
+
|
|
131
|
+
Ring buffers provide three query methods to check their state. Note that under concurrent access, these results are **approximate** — by the time the method returns, other threads may modify the buffer.
|
|
132
|
+
|
|
133
|
+
All four implementations provide the same method signatures: `size`, `isEmpty`, and `isFull`. See the [`SpscRingBuffer` Operations](./spsc.mdx#operations) section for detailed descriptions and examples of these methods.
|
|
134
|
+
|
|
135
|
+
## Examples
|
|
136
|
+
|
|
137
|
+
### MPMC: General-Purpose Queue
|
|
138
|
+
|
|
139
|
+
For workloads with multiple producers and consumers, use `MpmcRingBuffer`. This example demonstrates how multiple workers coordinate to process tasks from a shared queue.
|
|
140
|
+
|
|
141
|
+
```scala mdoc:passthrough
|
|
142
|
+
import docs.SourceFile
|
|
143
|
+
|
|
144
|
+
SourceFile.print("zio-blocks-examples/src/main/scala/ringbuffer/MpmcExample.scala")
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
([source](https://github.com/zio/zio-blocks/blob/main/zio-blocks-examples/src/main/scala/ringbuffer/MpmcExample.scala))
|
|
148
|
+
|
|
149
|
+
```bash
|
|
150
|
+
sbt "zio-blocks-examples/runMain ringbuffer.MpmcExample"
|
|
151
|
+
```
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: mpsc
|
|
3
|
+
title: "MPSC RingBuffer"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
import MpscDiagram from './MpscDiagram.jsx';
|
|
7
|
+
|
|
8
|
+
`MpscRingBuffer[A]` handles the inverse case: multiple producer threads safely offering elements to a single consumer thread. It uses a **hybrid design** combining producer-side CAS coordination with a FastFlow-style relaxed-poll consumer, balancing producer contention while keeping the consumer blazingly fast.
|
|
9
|
+
|
|
10
|
+
## Algorithm
|
|
11
|
+
|
|
12
|
+
`MpscRingBuffer` handles multiple producers with a single consumer, based on the **JCTools `MpscArrayQueue`** design. It's a hybrid: producers use CAS among themselves, while the consumer remains FastFlow-style.
|
|
13
|
+
|
|
14
|
+
**How it works:**
|
|
15
|
+
|
|
16
|
+
- The **producers** (multiple) coordinate via CAS on the shared `producerIndex`. Each producer claims a slot by atomically incrementing `producerIndex`, then writes its element with release semantics. A cached `producerLimit` (initialized to capacity) reduces volatile reads of `consumerIndex`.
|
|
17
|
+
- The **consumer** (single) uses **FastFlow relaxed-poll semantics**: it reads array slots directly. A `null` slot indicates either the buffer is empty or a producer has claimed the slot via CAS but has not yet written the element (mid-write). In both cases, `take` returns `null` rather than spinning. The consumer clears the slot after reading.
|
|
18
|
+
- The `producerLimit` is updated opportunistically (racy updates are benign) to reflect approximate available capacity.
|
|
19
|
+
|
|
20
|
+
**Why hybrid?** Multiple producers need CAS to coordinate their offers, but the single consumer can achieve maximum speed using pure FastFlow (no CAS, no reading producer state).
|
|
21
|
+
|
|
22
|
+
### Diagram
|
|
23
|
+
|
|
24
|
+
To see the hybrid algorithm in action, use this interactive stepper. Type any label, click **Offer** to enqueue or **Take** to dequeue, and watch the trace panel show every intermediate variable — `pIdx`, `cIdx`, `size`, `slot`, `value` — and the exact decision the algorithm makes.
|
|
25
|
+
|
|
26
|
+
<MpscDiagram />
|
|
27
|
+
|
|
28
|
+
Here is a complete walkthrough of every variable in the trace, in the order the algorithm computes them.
|
|
29
|
+
|
|
30
|
+
### `pIdx` / `cIdx` — the monotonic counters
|
|
31
|
+
|
|
32
|
+
Like the MPMC algorithm, both counters start at zero and only ever increase — they never wrap, never reset, never go backwards. `pIdx` is shared by all producer threads: each producer atomically claims the next slot by performing a compare-and-swap (CAS) from `pIdx` to `pIdx + 1`. Whoever wins the CAS owns that slot. `cIdx` is read with a plain load because only one thread ever consumes, so no CAS is needed.
|
|
33
|
+
|
|
34
|
+
### `size = pIdx − cIdx` — the occupancy check
|
|
35
|
+
|
|
36
|
+
Before claiming a slot, the producer checks whether the buffer is full: if `pIdx − cIdx == capacity`, every slot is occupied and `offer` returns `false` immediately. The difference `pIdx − cIdx` is the number of elements currently in the buffer. Under real concurrent access, the JVM implementation maintains a cached `producerLimit` to avoid reading the volatile `consumerIndex` on every `offer` call — but the fundamental check is the same occupancy test shown here.
|
|
37
|
+
|
|
38
|
+
### `slot = pIdx & mask` — the circular array index
|
|
39
|
+
|
|
40
|
+
Because capacity is a power of two, `mask = capacity - 1`, and the bitwise AND strips high bits to give a slot number in `[0, capacity)`. This is mathematically equivalent to `pIdx % capacity` but costs a single CPU instruction. So `pIdx = 7` maps to slot `7 & 3 = 3`, and `pIdx = 8` wraps back to slot `0`.
|
|
41
|
+
|
|
42
|
+
### `value = buf[slot]` — the FastFlow slot check
|
|
43
|
+
|
|
44
|
+
The consumer uses the **FastFlow relaxed-poll pattern**: read the array slot directly with acquire semantics. If the slot is `null`, either the buffer is empty, or a producer has won the CAS but has not yet finished writing the element (mid-write). In both cases, `take` returns `null` rather than blocking or spinning — this is the *relaxed poll* semantic. This is why the consumer never reads `pIdx`; it derives all its information from the slot value itself.
|
|
45
|
+
|
|
46
|
+
Once the producer wins its CAS, it writes the element with release semantics. The acquire/release pair guarantees the consumer will see the fully written element as soon as the slot is non-null.
|
|
47
|
+
|
|
48
|
+
## Creating Instances
|
|
49
|
+
|
|
50
|
+
`MpscRingBuffer[A]` allows multiple producer threads to offer elements concurrently via compare-and-swap on the producer index, while a single consumer thread takes elements efficiently.
|
|
51
|
+
|
|
52
|
+
```scala
|
|
53
|
+
object MpscRingBuffer {
|
|
54
|
+
def apply[A <: AnyRef](capacity: Int): MpscRingBuffer[A]
|
|
55
|
+
}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Creates a new MPSC ring buffer with the given capacity. The capacity must be a positive power of two.
|
|
59
|
+
|
|
60
|
+
To create an MPSC buffer:
|
|
61
|
+
|
|
62
|
+
```scala mdoc:compile-only
|
|
63
|
+
import zio.blocks.ringbuffer.MpscRingBuffer
|
|
64
|
+
|
|
65
|
+
val rb = MpscRingBuffer[java.lang.Long](32)
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Operations
|
|
69
|
+
|
|
70
|
+
`MpscRingBuffer` supports inserting elements from multiple producers, removing from a single consumer, and batch operations:
|
|
71
|
+
|
|
72
|
+
### Inserting Elements — `MpscRingBuffer#offer`
|
|
73
|
+
|
|
74
|
+
Use `MpscRingBuffer#offer` to insert an element with this signature:
|
|
75
|
+
|
|
76
|
+
```scala
|
|
77
|
+
final class MpscRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
78
|
+
def offer(a: A): Boolean
|
|
79
|
+
}
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Inserts the element without blocking. Returns `true` on successful insertion, `false` when the buffer is full. Raises `NullPointerException` if the element is `null`. Thread-safe; multiple producer threads may call this concurrently.
|
|
83
|
+
|
|
84
|
+
### Removing Elements — `MpscRingBuffer#take`
|
|
85
|
+
|
|
86
|
+
Use `MpscRingBuffer#take` to remove an element with this signature:
|
|
87
|
+
|
|
88
|
+
```scala
|
|
89
|
+
final class MpscRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
90
|
+
def take(): A
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Retrieves and removes an element from the front of the buffer. Returns immediately without blocking, providing the element or `null` if the buffer is empty. Call only from the single consumer thread.
|
|
95
|
+
|
|
96
|
+
### Checking State — `size`, `isEmpty`, `isFull`
|
|
97
|
+
|
|
98
|
+
Ring buffers provide three query methods to check their state. Note that under concurrent access, these results are **approximate** — by the time the method returns, other threads may modify the buffer.
|
|
99
|
+
|
|
100
|
+
All four implementations provide the same method signatures: `size`, `isEmpty`, and `isFull`. See the [`SpscRingBuffer` Operations](./spsc.mdx#operations) section for detailed descriptions and examples of these methods.
|
|
101
|
+
|
|
102
|
+
### Batch Operations — `MpscRingBuffer#drain`
|
|
103
|
+
|
|
104
|
+
Use `MpscRingBuffer#drain` to consume up to N elements with this signature:
|
|
105
|
+
|
|
106
|
+
```scala
|
|
107
|
+
final class MpscRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
108
|
+
def drain(consumer: A => Unit, limit: Int): Int
|
|
109
|
+
}
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Removes up to `limit` elements from the buffer, passing each to the `consumer` callback. Returns the number of elements actually drained. Raises `IllegalArgumentException` if `limit` is negative. Call only from the consumer thread. O(n) where n is the number of elements drained.
|
|
113
|
+
|
|
114
|
+
**Note**: Uses relaxed poll semantics and stops at the first `null` slot, which may indicate either an empty buffer or a producer that has claimed a slot but has not yet written its element (mid-write). In the mid-write case, fewer than `limit` elements are returned even though more elements will become available shortly. If the callback throws, all elements passed to it up to that point remain consumed and the buffer is in a consistent state.
|
|
115
|
+
|
|
116
|
+
## Examples
|
|
117
|
+
|
|
118
|
+
### MPSC: Multiple Producers, Single Aggregator
|
|
119
|
+
|
|
120
|
+
When multiple threads produce work for a single processor, use `MpscRingBuffer`. This example shows how three producer threads safely offer items to a single consumer.
|
|
121
|
+
|
|
122
|
+
```scala mdoc:passthrough
|
|
123
|
+
import docs.SourceFile
|
|
124
|
+
|
|
125
|
+
SourceFile.print("zio-blocks-examples/src/main/scala/ringbuffer/MpscExample.scala")
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
([source](https://github.com/zio/zio-blocks/blob/main/zio-blocks-examples/src/main/scala/ringbuffer/MpscExample.scala))
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
sbt "zio-blocks-examples/runMain ringbuffer.MpscExample"
|
|
132
|
+
```
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: spmc
|
|
3
|
+
title: "SPMC RingBuffer"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
import SpmcDiagram from './SpmcDiagram.jsx';
|
|
7
|
+
|
|
8
|
+
`SpmcRingBuffer[A]` allows a single producer thread to efficiently feed multiple consumer threads. It uses an **index-based algorithm** where slot validity is determined by comparing producer and consumer indices, allowing multiple consumers to coordinate safely via CAS operations on a shared consumer index.
|
|
9
|
+
|
|
10
|
+
## Algorithm
|
|
11
|
+
|
|
12
|
+
`SpmcRingBuffer` allows a single producer to feed multiple consumers. The algorithm is **index-based**: slot validity is determined by comparing `producerIndex` and `consumerIndex`, not by null-checking slots.
|
|
13
|
+
|
|
14
|
+
**How it works:**
|
|
15
|
+
|
|
16
|
+
- The **producer** is single-threaded and never uses CAS. It maintains a `producerLimit` derived from reading the volatile `consumerIndex`. On the fast path, it checks its local limit; when exhausted, it refreshes by reading `consumerIndex`. This design avoids the producer ever reading consumer state on the fast path.
|
|
17
|
+
- The **consumers** (any number) coordinate via a CAS loop on `consumerIndex`. Each consumer reads the element at its claimed index, then attempts to CAS `consumerIndex` forward. If CAS fails, it retries with a refreshed index. This ensures each element is claimed by exactly one consumer.
|
|
18
|
+
- **No slot clearing by consumers:** Consumers do not write `null` after reading. Instead, the producer safely overwrites slots once its `producerLimit` check (based on `consumerIndex`) confirms all consumers have advanced past them. This eliminates the race that would occur if a consumer claimed a slot but hadn't yet cleared it.
|
|
19
|
+
|
|
20
|
+
**Trade-off:** Consumer CAS introduces overhead under contention, but the producer remains extremely fast (no synchronization).
|
|
21
|
+
|
|
22
|
+
### Diagram
|
|
23
|
+
|
|
24
|
+
To see the single-producer multi-consumer algorithm in action, use this interactive stepper. Type any label, click **Offer** to enqueue (single producer) or **Take** to dequeue (any consumer), and watch the trace panel show every intermediate variable — `pIdx`, `cIdx`, `size`, `slot`, `element` — and the exact decision each side makes.
|
|
25
|
+
|
|
26
|
+
<SpmcDiagram />
|
|
27
|
+
|
|
28
|
+
Here is a complete walkthrough of every variable in the trace, in the order the algorithm computes them.
|
|
29
|
+
|
|
30
|
+
### `pIdx` — the monotonic producer counter
|
|
31
|
+
|
|
32
|
+
The producer has its own index that starts at zero and only ever increases. Because only one thread produces, `pIdx` is read and written with plain loads and stores — no CAS is needed. The producer's job is simple: check capacity (`pIdx - cIdx < capacity`), write to the slot, then advance `pIdx`.
|
|
33
|
+
|
|
34
|
+
### `cIdx` — the monotonic consumer counter (multi-threaded)
|
|
35
|
+
|
|
36
|
+
All consumers share a single `consumerIndex` counter. Because multiple consumers may be operating concurrently, each consumer reads `cIdx` with volatile semantics to see the most recent updates, then attempts to atomically advance it via CAS. If the CAS fails, another consumer claimed the slot first and the reader discards their read and retries.
|
|
37
|
+
|
|
38
|
+
### `size = pIdx − cIdx` — the occupancy check
|
|
39
|
+
|
|
40
|
+
Both sides use the same occupancy formula: `pIdx − cIdx` is the number of elements currently in the buffer. The producer checks if `size == capacity` to detect a full buffer. The consumer checks if `size == 0` to detect an empty buffer.
|
|
41
|
+
|
|
42
|
+
### `slot = pIdx & mask` or `slot = cIdx & mask` — the circular array index
|
|
43
|
+
|
|
44
|
+
Identical to all other ring buffer variants: because capacity is a power of two, the bitwise AND `& mask` replaces modulo division, giving a slot number in `[0, capacity)`. So `pIdx = 7` maps to slot `7 & 3 = 3`, and `pIdx = 8` wraps back to slot `0`.
|
|
45
|
+
|
|
46
|
+
### `element = buf[slot]` — read-before-CAS
|
|
47
|
+
|
|
48
|
+
This is the critical SPMC distinction. The consumer reads the element *before* attempting the CAS on `cIdx`. This ordering is essential: once the CAS succeeds and `cIdx` advances, the producer is permitted to overwrite that slot on its next lap. By reading the element first, the consumer guarantees it captures the value while the slot is still logically owned.
|
|
49
|
+
|
|
50
|
+
In the trace, when a consumer takes an element, you will see the `element` row highlighted — this captures the read that must happen before the CAS.
|
|
51
|
+
|
|
52
|
+
### Why SPMC doesn't clear slots
|
|
53
|
+
|
|
54
|
+
Unlike MPSC (which uses FastFlow's null-check pattern), SPMC determines slot validity purely by index comparison: if `cIdx ≤ slot < pIdx`, the slot is live. After a consumer advances `cIdx`, the slot becomes "stale" — no longer live, but still holding the old value until the producer wraps around and overwrites it. This eliminates the race that would occur if a consumer had to null out the slot while holding the CAS; instead, the producer simply overwrites when it knows the consumer has advanced.
|
|
55
|
+
|
|
56
|
+
The diagram marks stale slots with a dashed border and a "stale" label in grey.
|
|
57
|
+
|
|
58
|
+
## Creating Instances
|
|
59
|
+
|
|
60
|
+
`SpmcRingBuffer[A]` allows a single producer thread to offer elements while multiple consumer threads concurrently take elements via compare-and-swap on the consumer index. Use the companion object to instantiate a buffer:
|
|
61
|
+
|
|
62
|
+
```scala
|
|
63
|
+
object SpmcRingBuffer {
|
|
64
|
+
def apply[A <: AnyRef](capacity: Int): SpmcRingBuffer[A]
|
|
65
|
+
}
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
The capacity must be a positive power of two. To create an SPMC buffer:
|
|
69
|
+
|
|
70
|
+
```scala mdoc:compile-only
|
|
71
|
+
import zio.blocks.ringbuffer.SpmcRingBuffer
|
|
72
|
+
|
|
73
|
+
val rb = SpmcRingBuffer[java.lang.Integer](64)
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
## Operations
|
|
77
|
+
|
|
78
|
+
`SpmcRingBuffer` provides three core operations for sending and receiving elements:
|
|
79
|
+
|
|
80
|
+
### Inserting Elements — `SpmcRingBuffer#offer`
|
|
81
|
+
|
|
82
|
+
Use `SpmcRingBuffer#offer` to insert an element with this signature:
|
|
83
|
+
|
|
84
|
+
```scala
|
|
85
|
+
final class SpmcRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
86
|
+
def offer(a: A): Boolean
|
|
87
|
+
}
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Inserts the element without blocking. Returns `true` on successful insertion, `false` when the buffer is full. Raises `NullPointerException` if the element is `null`. Call only from the single producer thread; concurrent calls from multiple threads cause undefined behavior.
|
|
91
|
+
|
|
92
|
+
### Removing Elements — `SpmcRingBuffer#take`
|
|
93
|
+
|
|
94
|
+
Use `SpmcRingBuffer#take` to remove an element with this signature:
|
|
95
|
+
|
|
96
|
+
```scala
|
|
97
|
+
final class SpmcRingBuffer[A <: AnyRef](val capacity: Int) {
|
|
98
|
+
def take(): A
|
|
99
|
+
}
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Retrieves and removes an element from the front of the buffer. Returns immediately without blocking. Returns the element, or `null` if the buffer is empty. Thread-safe; multiple consumer threads may call this concurrently.
|
|
103
|
+
|
|
104
|
+
### Checking State — `size`, `isEmpty`, `isFull`
|
|
105
|
+
|
|
106
|
+
Ring buffers provide three query methods to check their state. Note that under concurrent access, these results are **approximate** — by the time the method returns, other threads may already modify the buffer.
|
|
107
|
+
|
|
108
|
+
All four implementations provide the same method signatures: `size`, `isEmpty`, and `isFull`. See the [`SpscRingBuffer` Operations](./spsc.mdx#operations) section for detailed descriptions and examples of these methods.
|