cdc-parallel 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7b66764665f18a0f84cc6b45c5138d932fb000d01be2bef02a26504e7013c5cb
-  data.tar.gz: 12daffb757cae8c4e9c5330cce505cc964834ac205fc89d3febc29944772f815
+  metadata.gz: 6e16d6352e78132e2e0f488542b17117bda8733ca15b3117f8151b6acbfc3567
+  data.tar.gz: 3e28c6c37d5078696ab334f5c8b15409172bebb8102747c2e7558f0089d4ad64
 SHA512:
-  metadata.gz: e7e3bd0dcd37187e7f31a9930fdea00680f898ce52c31cf33b232004bd9da4ec07cb132fc1bd71a337c7a8db99b7888618d96fc435132e16eb3038e1ddb8588f
-  data.tar.gz: f9af8ecfb077ef1898d312f45169b6b0b81d3ea641af1e9f00374999cd06542bcda7561fc8878b147f61ebcd4c0f5503f88d3f332f1657be19bc45504d77de49
+  metadata.gz: 4e71ef2eeda63a9d6f6c59d49ba140b73a62d0e37894e168060a4c796f28a7d6790b37b891c7188fff78de1e843386c0f892f0720ba3de16ea9bfe198e719382
+  data.tar.gz: de5eb4cb7861e263402305e361d4ac49d335d9b657562cca93544c64f34f76646289212647506c87f4d6ef7eb5ec586cbb58596ad9427f8becb99ca1af82127c

data/CHANGELOG.md

@@ -4,11 +4,35 @@ All notable changes to this project will be documented in this file.
 
 The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
 
-## [Unreleased]
+## [0.2.0] - 2026-06-03
 
 ### Added
 
-- Placeholder for future development.
+- Pre-warmed persistent Ractor worker pool implementation.
+- `ProcessorPool#process_many` for batched dispatch.
+- Tiny workload benchmark for dispatch overhead analysis.
+- CPU-bound workload benchmark for throughput analysis.
+- Batch workload benchmark for CDC-style event processing.
+- Performance test suite guarded by `CDC_PARALLEL_PERFORMANCE_TESTS=1`.
+- Reusable benchmark Docker image.
+- `benchmark:processor_pool` Rake task.
+- `benchmark:docker_build` Rake task.
+- `benchmark:docker_run` Rake task.
+- Benchmark documentation and reproducibility guidance.
+
+### Changed
+
+- Processor workers are now initialized once and reused for the lifetime of the pool.
+- Benchmark methodology updated to measure pre-warmed worker execution.
+- README updated with benchmark execution instructions and example results.
+
+### Performance
+
+Local benchmark results on Ruby 4.0.5 (4 workers) demonstrated measurable throughput improvements for CPU-bound workloads using pre-warmed worker pools compared to serial execution.
+
+Benchmark results vary by hardware, operating system, Ruby version, and workload characteristics. Users are encouraged to reproduce results on their own systems using the included benchmark suite.
+
+
 
 ## [0.1.0] - 2026-05-31
 
@@ -28,3 +52,10 @@ The format is based on Keep a Changelog, and this project adheres to Semantic Ve
 - Added Minitest suite.
 - Added README and example.
 - Added CI and release workflows.
+
+## [0.1.1] - 2026-06-03
+
+No code changes.
+
+Improves RubyGems metadata and documentation wording to
+explicitly identify CDC as Change Data Capture.

data/README.md

@@ -3,7 +3,7 @@
 [![Gem Version](https://badge.fury.io/rb/cdc-parallel.svg)](https://badge.fury.io/rb/cdc-parallel)
 [![CI](https://github.com/kanutocd/cdc-parallel/workflows/CI/badge.svg)](https://github.com/kanutocd/cdc-parallel/actions)
 [![Coverage Status](https://codecov.io/gh/kanutocd/cdc-parallel/branch/main/graph/badge.svg)](https://codecov.io/gh/kanutocd/cdc-parallel)
-[![Ruby Version](https://img.shields.io/badge/ruby-%3E%3D%203.4-ruby.svg)](https://www.ruby-lang.org/en/)
+[![Ruby Version](https://img.shields.io/badge/ruby-%3E%3D%204.0-ruby.svg)](https://www.ruby-lang.org/en/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 Optional high-throughput Ractor runtime for `cdc-core`.
@@ -161,3 +161,106 @@ The default `test` task runs unit, integration, and behavior tests. Performance
 ## License
 
 MIT.
+
+
+## Benchmarking
+
+`cdc-parallel` includes reproducible benchmarks that compare serial processor execution against the pre-warmed Ractor worker pool.
+
+The benchmark focuses on three workload categories:
+
+| Workload | Purpose                                         |
+| -------- | ----------------------------------------------- |
+| tiny     | Measure dispatch overhead                       |
+| cpu      | Measure CPU-bound processing throughput         |
+| batch    | Measure batched CDC event processing throughput |
+
+### Running Benchmarks
+
+Tiny workload:
+
+```bash
+BENCHMARK_WORKLOAD=tiny \
+bundle exec rake benchmark:processor_pool
+```
+
+CPU-bound workload:
+
+```bash
+BENCHMARK_WORKLOAD=cpu \
+BENCHMARK_CPU_ROUNDS=5000 \
+bundle exec rake benchmark:processor_pool
+```
+
+Batch workload:
+
+```bash
+BENCHMARK_WORKLOAD=batch \
+BENCHMARK_BATCH_SIZE=10000 \
+bundle exec rake benchmark:processor_pool
+```
+
+### Benchmark Docker Image
+
+Build and run the reusable Docker image:
+
+```bash
+bundle exec rake benchmark:docker_build
+bundle exec rake benchmark:docker_run
+```
+
+Or run the image directly after it is published to GitHub Container Registry:
+
+```bash
+docker run --rm ghcr.io/kanutocd/cdc-parallel-benchmark:main
+```
+
+The benchmark image is intended to become the shared performance validation
+pattern across CDC Ecosystem gems, enabling reproducible benchmark execution
+locally, in CI, and across different development environments.
+
+### Example Result
+
+Environment:
+
+* Ruby 4.0.5
+* x86_64 Linux
+* 4 workers
+
+CPU workload (`BENCHMARK_CPU_ROUNDS=5000`):
+
+```json
+{
+  "serial": {
+    "events_per_second": 120.26
+  },
+  "parallel": {
+    "events_per_second": 250.15
+  },
+  "ratio": {
+    "parallel_to_serial": 2.08
+  }
+}
+```
+
+### Interpretation
+
+A ratio greater than `1.0` indicates that the pre-warmed Ractor worker pool outperformed serial execution.
+
+```text
+ratio > 1.0  => parallel faster
+ratio = 1.0  => equivalent
+ratio < 1.0  => serial faster
+```
+
+### Reproducibility
+
+Benchmark results vary depending on:
+
+* CPU model
+* Core count
+* Operating system
+* Ruby version
+* Background system activity
+
+The benchmark suite is provided so that users can reproduce and validate results on their own hardware.

data/lib/cdc/parallel/processor_pool.rb

@@ -2,12 +2,12 @@
 
 module CDC
   module Parallel
-    # Executes one Ractor-safe processor in isolated Ractor workers.
+    # Executes one Ractor-safe processor in pre-warmed persistent Ractor workers.
     #
-    # This v0.1 implementation intentionally uses one-shot worker Ractors for
-    # deterministic synchronous semantics while preserving the public pool API.
-    # The parallel-pool dependency is kept as the runtime foundation for later
-    # async/throughput-focused versions.
+    # Workers are created during initialization and reused for every dispatch.
+    # This pays Ractor startup cost once, keeps workers alive after processor
+    # failures, and provides both synchronous single-item processing and batched
+    # dispatch for throughput-oriented benchmarks and runtimes.
     class ProcessorPool
       # @param processor [CDC::Core::Processor]
       # @param size [Integer]
@@ -18,47 +18,107 @@ module CDC
 
         @processor = ::Ractor.make_shareable(processor)
         @configuration = Configuration.new(size:, timeout:)
+        @workers = Array.new(@configuration.size) do
+          build_worker(@processor)
+        end.freeze
+
+        @next_worker = 0
         @shutdown = false
       end
 
-      # Process one ChangeEvent.
+      # Process one work item synchronously.
       #
-      # @param event [CDC::Core::ChangeEvent]
+      # @param item [Object]
       # @return [CDC::Core::ProcessorResult]
-      def process(event)
+      def process(item)
+        process_many([item]).fetch(0)
+      end
+
+      # Process many work items using the pre-warmed worker pool.
+      #
+      # Results are returned in the same order as the supplied work items.
+      #
+      # @param items [Array<Object>]
+      # @return [Array<CDC::Core::ProcessorResult>]
+      def process_many(items)
         raise ShutdownError, "processor pool has been shut down" if @shutdown
 
-        work = ::Ractor.make_shareable(event)
-        worker = ::Ractor.new(@processor, work) do |processor, item|
-          CDC::Parallel::ResultCollector.normalize(processor.process(item))
-        rescue StandardError => e
-          CDC::Parallel::ResultCollector.worker_failure(e)
+        work_items = items.map { |item| ::Ractor.make_shareable(item) }
+        reply_port = ::Ractor::Port.new
+
+        work_items.each_with_index do |item, index|
+          next_worker.send([index, item, reply_port])
         end
 
-        ResultCollector.normalize(take(worker))
+        collect_results(reply_port, work_items.length)
+      ensure
+        reply_port&.close
       end
 
       # Shut down the pool.
       #
       # @return [void]
       def shutdown
+        return if @shutdown
+
         @shutdown = true
+
+        @workers.each do |worker|
+          worker.send(nil)
+        rescue Ractor::ClosedError
+          # Already stopped.
+        end
       end
 
       private
 
       def validate_processor!(processor)
-        return if processor.class.respond_to?(:ractor_safe?) && processor.class.ractor_safe?
+        return if processor.class.respond_to?(:ractor_safe?) &&
+                  processor.class.ractor_safe?
 
-        raise UnsafeProcessorError, "#{processor.class} must declare ractor_safe!"
+        raise UnsafeProcessorError,
+              "#{processor.class} must declare ractor_safe!"
       end
 
-      def take(worker)
-        if worker.respond_to?(:value)
-          worker.value
-        else
-          worker.take
+      def build_worker(processor)
+        ::Ractor.new(processor) do |safe_processor|
+          loop do
+            message = ::Ractor.receive
+            break if message.nil?
+
+            index, item, reply_port = message
+
+            response = begin
+              CDC::Parallel::ResultCollector.worker_success(
+                safe_processor.process(item)
+              )
+            rescue StandardError => e
+              CDC::Parallel::ResultCollector.worker_failure(e)
+            end
+
+            reply_port << [index, response]
+          end
+        end
+      end
+
+      def next_worker
+        worker = @workers[@next_worker]
+
+        @next_worker += 1
+        @next_worker = 0 if @next_worker >= @workers.length
+
+        worker
+      end
+
+      def collect_results(reply_port, count)
+        results = Array.new(count)
+
+        count.times do
+          index, response = reply_port.receive
+          results[index] = ResultCollector.normalize(response)
         end
+
+        results.freeze
       end
     end
   end

data/lib/cdc/parallel/result_collector.rb

@@ -6,6 +6,14 @@ module CDC
     class ResultCollector
       FAILURE_MARKER = :__cdc_parallel_failure__
 
+      # Build a shareable success payload that can safely cross a Ractor boundary.
+      #
+      # @param value [Object]
+      # @return [Object]
+      def self.worker_success(value)
+        ::Ractor.make_shareable(value)
+      end
+
       # Build a shareable failure payload that can safely cross a Ractor boundary.
       #
       # @param error [Exception]

data/lib/cdc/parallel/version.rb

@@ -3,6 +3,6 @@
 module CDC
   module Parallel
     # Current cdc-parallel version.
-    VERSION = "0.1.0"
+    VERSION = "0.2.0"
   end
 end

data/lib/cdc/parallel.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 require "etc"
-require "ractor-pool"
 
 require_relative "parallel/version"
 require_relative "parallel/errors"
@@ -13,7 +12,7 @@ require_relative "parallel/router"
 require_relative "parallel/runtime"
 
 module CDC
-  # Optional high-throughput Ractor runtime for cdc-core processors.
+  # Optional parallel Change Data Capture runtime for cdc-core processors.
   module Parallel
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: cdc-parallel
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Ken C. Demanawa
@@ -39,8 +39,8 @@ dependencies:
         version: 0.4.0
 description: |
   cdc-parallel provides optional Ractor-backed parallel execution for
-  cdc-core. It accelerates Change Data Capture (CDC) pipelines while
-  preserving the simplicity and composability of the CDC Ecosystem.
+  cdc-core. It accelerates PostgreSQL Change Data Capture (CDC) event
+  processing while preserving the cdc-core programming model.
 email:
 - kenneth.c.demanawa@gmail.com
 executables: []
@@ -98,5 +98,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 4.0.10
 specification_version: 4
-summary: Optional parallel runtime for the CDC Ecosystem.
+summary: Optional parallel Change Data Capture (CDC) runtime for cdc-core.
 test_files: []