cdc-concurrent 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cbc0a0ebe15a9254e91b40caa48db7f7f26e4b01933be88e5fd148ea8c3371d0
-  data.tar.gz: 38888cc8ade72e4611e352fa534a25876cbc71d6ef74c2f725b5bb3ef1c4c069
+  metadata.gz: e7567d8160a1a8b4f930c718574ed7a6e7c181266d62572bbad2362c5b6a480f
+  data.tar.gz: b0867ca4e10b7e26e4c355420183517bc34e13bf4a72b49f91cb6f63227670fe
 SHA512:
-  metadata.gz: c3315531e5f0cbe43f86461acabf835ec5f187d65dace206d1640a213b42702a0ee8e1088421f1e2f05e076ce9dfcdd7625f309f837b613c5f13dfc459b07152
-  data.tar.gz: 351566c71b92c033c2c35b2639653eabf6e75d3bacee667f643dde9af47bffaf7dbeee4f802eb8e237bd5e9a5bc7957d43269b323fc9c17b89bedda1ecc7dd8b
+  metadata.gz: b145904c468fdc9ad5bcf492635da16d75371952f04ab247e220d7f3d6fa0a269a32e4255dc810b6e3661fa8bb1c11f1d5ca9306f72c4ce9475a8ef60d7b0dd8
+  data.tar.gz: ce8a9edee9f28b4c372b187fa1d8a871f31209321cf4b76b536d44e88b5e0e55006529c0af4fd0c35f60219822da93498e160a51b397c14b015c2fec8915b3a4

data/CHANGELOG.md

@@ -1,3 +1,24 @@
+## [0.1.1] - 2026-06-12
+
+  Tightened the type surface and switched the signatures over to the published
+  `cdc-core` 0.1.3 RBS files.
+
+  ### Added
+
+  - Published `cdc-core` RBS files are now loaded directly by Steep.
+  - `CDC::Concurrent` signatures are tighter around runtime, router, and pool
+    boundaries.
+
+  ### Changed
+
+  - Removed local RBS shims for `cdc-core` and `async`.
+  - Updated the gem dependency floor to `cdc-core >= 0.1.3`.
+
+  ### Fixed
+
+  - `TransactionPool` now handles failure results with a missing error value
+    defensively.
+
 ## [0.1.0] - 2026-06-04
 
   Initial release of `cdc-concurrent`.

data/README.md

@@ -1,5 +1,10 @@
 # cdc-concurrent
 
+[![Gem Version](https://badge.fury.io/rb/cdc-conxurrent.svg)](https://badge.fury.io/rb/cdc-conxurrent)
+[![CI](https://github.com/kanutocd/cdc-conxurrent/workflows/CI/badge.svg)](https://github.com/kanutocd/cdc-concurrent/actions)
+[![Ruby Version](https://img.shields.io/badge/ruby-%3E%3D%203.4-ruby.svg)](https://www.ruby-lang.org/en/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
 Optional I/O-concurrent runtime adapter for `cdc-core`.
 
 `cdc-concurrent` executes `CDC::Core::Processor` objects with Fiber-scheduler-based I/O concurrency using `async`. It is the I/O-bound twin of `cdc-parallel`.
@@ -73,6 +78,10 @@ results = runtime.process_many(events)
 
 Results preserve input order by default. Set `preserve_order: false` when completion order is acceptable.
 
+For I/O-bound throughput, `process_many` is the primary path. Repeated
+single-event `process` calls still pay one Async dispatch per event, while
+`process_many` lets the runtime overlap waits across the submitted batch.
+
 ## Transaction Processing
 
 ```ruby
@@ -99,10 +108,17 @@ CDC::Concurrent::UnsafeProcessorError
 
 A concurrent-safe processor should avoid unsafe shared mutable instance state. This runtime runs tasks concurrently in one Ruby process; it does not isolate mutable objects like Ractors do.
 
+`concurrent_safe!` is a declaration of processor intent, not a proof. The
+runtime cannot verify that a processor avoids unsafe shared mutable state or
+uses scheduler-compatible I/O.
+
 ## Important Limits
 
 `cdc-concurrent` improves throughput only for I/O that cooperates with Ruby's Fiber scheduler. Blocking libraries that do not yield to the scheduler will still block the process.
 
+Timeouts are applied per event task. They are not whole-batch or
+whole-transaction deadlines.
+
 For CPU-bound processing, use `cdc-parallel`.
 
 ## Roadmap
@@ -114,6 +130,112 @@ For CPU-bound processing, use `cdc-parallel`.
 - Sink abstractions
 - Async Redis/OpenSearch integrations
 
+## Test Organization
+
+The test suite is grouped by intent so the same structure can be reused across CDC ecosystem gems.
+
+```text
+test/unit/          focused class and branch coverage
+test/integration/   component interaction and runtime integration
+test/behavior/      ecosystem contracts and guardrails
+test/performance/   opt-in smoke benchmarks
+```
+
+Run the default quality suite:
+
+```bash
+bundle exec rake test
+```
+
+Run a specific group:
+
+```bash
+bundle exec rake test:unit
+bundle exec rake test:integration
+bundle exec rake test:behavior
+bundle exec rake test:performance
+```
+
+The default `test` task runs unit, integration, and behavior tests. Performance tests are intentionally separate because they are environment-sensitive.
+
+## Benchmarking
+
+`cdc-concurrent` includes reproducible benchmarks that compare serial processor execution against the Async-backed processor pool.
+
+The benchmark focuses on three workload categories:
+
+| Workload | Purpose                                      |
+| -------- | -------------------------------------------- |
+| tiny     | Measure dispatch overhead                    |
+| io       | Measure scheduler-friendly I/O concurrency   |
+| batch    | Measure batched CDC event I/O fanout         |
+
+See [benchmark/README.md](benchmark/README.md) for the full benchmark methodology,
+configuration reference, report schema, and interpretation guidance.
+
+`cdc-parallel` and `cdc-concurrent` benchmark different bottlenecks.
+`cdc-parallel` measures CPU parallelism; `cdc-concurrent` measures I/O wait
+overlap. Their speedup ratios are not directly comparable.
+
+### Quick Start
+
+Default I/O workload:
+
+```bash
+bundle exec rake benchmark:processor_pool
+```
+
+Tiny overhead workload:
+
+```bash
+BENCHMARK_WORKLOAD=tiny \
+bundle exec rake benchmark:processor_pool
+```
+
+Batch workload:
+
+```bash
+BENCHMARK_WORKLOAD=batch \
+BENCHMARK_BATCH_SIZE=1000 \
+bundle exec rake benchmark:processor_pool
+```
+
+Concurrency sweep:
+
+```bash
+BENCHMARK_WORKLOAD=io \
+BENCHMARK_CONCURRENCY_COUNTS=1,10,50,100 \
+bundle exec rake benchmark:processor_pool
+```
+
+Credibility controls:
+
+```bash
+BENCHMARK_TRIALS=7 \
+BENCHMARK_MIN_DURATION=0.25 \
+BENCHMARK_ITERATIONS=1000 \
+bundle exec rake benchmark:processor_pool
+```
+
+### Benchmark Docker Image
+
+Build and run the reusable Docker image:
+
+```bash
+bundle exec rake benchmark:docker_build
+bundle exec rake benchmark:docker_run
+```
+
+Or run the image directly after it is published to GitHub Container Registry:
+
+```bash
+docker run --rm ghcr.io/kanutocd/cdc-concurrent-benchmark:main
+```
+
+The benchmark image is intended to follow the shared performance validation
+pattern across CDC Ecosystem gems, enabling reproducible benchmark execution
+locally, in CI, and across different development environments.
+
 ## License
 
-MIT.
+[MIT](LICENSE.txt)

data/benchmark/README.md

@@ -0,0 +1,165 @@
+# cdc-concurrent Benchmarking
+
+This directory contains the reproducible benchmark harness for `cdc-concurrent`.
+
+The benchmark compares direct serial processor execution against the
+Async-backed `CDC::Concurrent::ProcessorPool`.
+
+## Goals
+
+The benchmark is designed to answer practical runtime questions:
+
+- What is the dispatch overhead for tiny work?
+- When does Async-backed I/O concurrency amortize its overhead?
+- How much does batched `process_many` improve throughput?
+- How does throughput change as concurrency changes?
+- Are results stable across multiple trials?
+
+## Workloads
+
+| Workload | Purpose                                      | Default options          |
+| -------- | -------------------------------------------- | ------------------------ |
+| tiny     | Measures processor-pool dispatch cost        | none                     |
+| io       | Measures scheduler-friendly I/O wait overlap | `io_sleep: 0.001`        |
+| batch    | Measures CDC-style batch I/O fanout          | `batch_size: 100`, `io_sleep: 0.001` |
+
+Tiny workloads intentionally do almost no work. They are useful for measuring
+runtime overhead, but they are not expected to make concurrent execution look
+faster than direct method calls.
+
+I/O and batch workloads are better indicators of useful concurrent throughput.
+
+## Execution Modes
+
+The benchmark compares three execution modes.
+
+| Mode             | Meaning                                      |
+| ---------------- | -------------------------------------------- |
+| serial           | Direct processor execution                   |
+| repeated_process | Repeated `ProcessorPool#process` calls       |
+| process_many     | Batched `ProcessorPool#process_many` calls   |
+
+`serial` is measured once per benchmark run. `repeated_process` and
+`process_many` are measured once for each configured concurrency count.
+
+For `cdc-concurrent`, `process_many` is the primary throughput path. Repeated
+single-event `process` calls are included to expose per-dispatch overhead, but
+they do not represent the best way to overlap I/O waits.
+
+## Configuration
+
+| Environment variable             | Default | Meaning                                      |
+| -------------------------------- | ------- | -------------------------------------------- |
+| `BENCHMARK_WORKLOAD`             | `io`    | `tiny`, `io`, or `batch`                     |
+| `BENCHMARK_ITERATIONS`           | `1000`  | Work items submitted per pass                |
+| `BENCHMARK_WARMUP`               | `100`   | Warmup work items before measurement         |
+| `BENCHMARK_TRIALS`               | `5`     | Number of measured trials                    |
+| `BENCHMARK_MIN_DURATION`         | `0.1`   | Minimum seconds per trial                    |
+| `BENCHMARK_CONCURRENCY`          | `100`   | Single concurrency count when no sweep is given |
+| `BENCHMARK_CONCURRENCY_COUNTS`   | unset   | Comma-separated concurrency sweep, e.g. `10,50,100` |
+| `BENCHMARK_IO_SLEEP`             | `0.001` | Seconds slept by I/O-like workloads          |
+| `BENCHMARK_BATCH_SIZE`           | `100`   | Events inside each batch workload item       |
+
+`BENCHMARK_CONCURRENCY_COUNTS` takes precedence over `BENCHMARK_CONCURRENCY`.
+
+## Examples
+
+Run the default I/O workload:
+
+```bash
+bundle exec rake benchmark:processor_pool
+```
+
+Run the tiny overhead workload:
+
+```bash
+BENCHMARK_WORKLOAD=tiny \
+bundle exec rake benchmark:processor_pool
+```
+
+Run the I/O workload with a custom concurrency sweep:
+
+```bash
+BENCHMARK_WORKLOAD=io \
+BENCHMARK_CONCURRENCY_COUNTS=10,50,100 \
+bundle exec rake benchmark:processor_pool
+```
+
+Run a longer batch benchmark:
+
+```bash
+BENCHMARK_WORKLOAD=batch \
+BENCHMARK_TRIALS=9 \
+BENCHMARK_MIN_DURATION=0.5 \
+BENCHMARK_CONCURRENCY_COUNTS=10,50,100 \
+bundle exec rake benchmark:processor_pool
+```
+
+Build and run the reusable Docker benchmark image:
+
+```bash
+bundle exec rake benchmark:docker_build
+bundle exec rake benchmark:docker_run
+```
+
+## Report Shape
+
+The benchmark prints JSON.
+
+Top-level fields:
+
+| Field               | Meaning                                      |
+| ------------------- | -------------------------------------------- |
+| `benchmark`         | Benchmark name                               |
+| `gem`               | Gem name                                     |
+| `timestamp`         | UTC timestamp                                |
+| `environment`       | Ruby, platform, host, CPU, and uname metadata |
+| `config`            | Benchmark configuration                      |
+| `workload_options`  | Workload-specific options                    |
+| `serial`            | Serial execution distribution                |
+| `concurrency_sweep` | Concurrent mode distributions by concurrency |
+| `interpretation`    | Ratio interpretation guide                   |
+
+Each distribution includes:
+
+| Field    | Meaning                          |
+| -------- | -------------------------------- |
+| `min`    | Fastest observed value           |
+| `median` | Median observed value            |
+| `max`    | Slowest observed value           |
+| `p95`    | 95th percentile observed value   |
+
+Each mode also includes `raw_trials` so results can be inspected or reprocessed.
+
+## Interpretation
+
+`ratio_to_serial_median_events_per_second` compares a concurrent mode's median
+throughput against serial median throughput.
+
+```text
+ratio_to_serial_median_events_per_second > 1.0  => concurrent mode faster
+ratio_to_serial_median_events_per_second = 1.0  => equivalent
+ratio_to_serial_median_events_per_second < 1.0  => serial faster
+```
+
+Tiny workloads primarily measure dispatch overhead, so serial execution may be
+faster. I/O-bound and batched workloads are better indicators of useful
+concurrent throughput.
+
+`cdc-parallel` and `cdc-concurrent` benchmark different bottlenecks.
+`cdc-parallel` measures CPU parallelism; `cdc-concurrent` measures I/O wait
+overlap. Their speedup ratios are not directly comparable.
+
+## Reproducibility
+
+Benchmark results vary depending on:
+
+- CPU model
+- operating system
+- Ruby version
+- background system activity
+- scheduler-compatible I/O behavior
+- thermal and power-management state
+
+Use multiple trials, a minimum measurement duration, and concurrency sweeps when
+comparing results across machines or releases.