sidekiq-ratomic-pool 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 73e5869196bd322f91cea20c1332e39b2c2e3c9d81e99717ac74210fb766be36
+  data.tar.gz: 4b830bdc58727ba25cc653011bf52d1394d2344ad5de996a7f20965b9b404e57
+SHA512:
+  metadata.gz: 6272ccc17b365724a851f3d6b39cb82d4ce109068cc854a640ab0b649f2c062bf4ee6afdeccf0c6e442789dfe03be411eb3bf8abd8dab2da36d8f9b5f8d09e00
+  data.tar.gz: 71f001b3aca75e43792bf760795d037a6f7a220eb15de33da7b1802eab89e976186ebce147f6eeb21f95e88c702ffd05c8309b06a410831609b58ff613133417

data/CHANGELOG.md

@@ -0,0 +1,13 @@
+## [Unreleased]
+
+- Added Sidekiq middleware with Ratomic `LocalPool` resource injection,
+  health validation, exponential retries, and circuit-breaker fast failure.
+- Added thread-boundary pool behavior tests and synchronized RBS signatures for the
+  Ractor-local implementation.
+- Added the `quality` Rake task covering tests, RuboCop, Steep, and YARD validation.
+- Enforced at least 99% line and branch coverage through SimpleCov.
+- Used Ratomic's native `Counter` primitive.
+- Added YARD validation and documented the public pool API.
+- Added argument validation and current-Ractor pool shutdown.
+- Added configurable retryable I/O errors and real exponential backoff delays.
+- Prevented non-retryable worker exceptions from affecting the circuit breaker.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Kenneth C. Demanawa
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,82 @@
+# sidekiq-ratomic-pool
+
+[![Gem Version](https://badge.fury.io/rb/sidekiq-ratomic-pool.svg)](https://badge.fury.io/rb/sidekiq-ratomic-pool)
+[![CI](https://github.com/kanutocd/sidekiq-ratomic-pool/workflows/CI/badge.svg)](https://github.com/kanutocd/sidekiq-ratomic-pool/actions)
+[![Ruby Version](https://img.shields.io/badge/ruby-%3E%3D%204.0-ruby.svg)](https://www.ruby-lang.org/en/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+
+Sidekiq server middleware leveraging `Ratomic::LocalPool` for Ractor-local resource ownership. It includes automated connection health validation, exponential backoff retries, and an integrated **Circuit Breaker** to help prevent cascading resource failures.
+
+## Installation
+
+Add to your `Gemfile`:
+
+```ruby
+gem "sidekiq-ratomic-pool"
+```
+
+## Sidekiq dependency
+
+Despite its name, this gem does not declare `sidekiq` as a runtime dependency.
+It provides a `Sidekiq::Ratomic::Pool` middleware implementation using the
+standard `call(job, payload, queue) { ... }` middleware contract. Sidekiq is the
+primary supported integration and the reason for the gem name, but other
+Sidekiq-compatible job frameworks can use it if they support the same contract
+and worker pool accessor pattern.
+
+## Features
+
+- **Ractor-Local Isolation**: Each Ractor lazily owns its resources through `Ratomic::LocalPool`; threads within the same Ractor share that Ractor-local pool.
+- **Circuit Breaker Pattern**: Trips open after a configurable threshold of checkout, health-check, or configured retryable I/O failures.
+- **Exponential Backoff**: Applies increasing retry delays to transient checkout and retryable resource-operation failures.
+- **Automated Health Probes**: Validates resources with `ping`, `active?`, or a caller-supplied validator before use.
+- **Configurable Failure Policy**: Non-retryable worker exceptions propagate without changing circuit state, avoiding accidental duplicate work.
+
+## Usage
+
+Factories must be Ractor-shareable because `Ratomic::LocalPool` creates resources lazily
+inside each Ractor. A small frozen factory object is suitable for production use:
+
+```ruby
+RedisFactory = Data.define(:url) do
+  def call
+    RedisClient.new(url:)
+  end
+end
+
+Sidekiq.configure_server do |config|
+  config.server_middleware do |chain|
+    chain.add Sidekiq::Ratomic::Pool,
+      pool_name: :redis_pool,
+      size: 10,
+      max_retries: 3,
+      retry_delay: 0.2,
+      cb_threshold: 5,
+      cb_timeout: 30,
+      factory: RedisFactory.new(ENV.fetch('REDIS_URL')).freeze
+  end
+end
+```
+
+Workers use the injected pool with `with`:
+
+```ruby
+redis_pool.with { |redis| redis.call('PING') }
+```
+
+Resource checkout/health failures and configured retryable I/O errors use exponential backoff.
+Other exceptions raised by the worker block propagate without being retried, preventing
+accidental duplication of non-idempotent work.
+
+## Smoke test
+
+The [`smoke_test/`](smoke_test/) harness runs Redis in Docker, starts a standalone
+Sidekiq server, enqueues jobs from a separate client process, and verifies the
+results through real Redis connections. It also prints `ps -L` snapshots showing
+Sidekiq worker threads and the CPU core (`PSR`) on which they were recently scheduled:
+
+```bash
+cd smoke_test
+./run.sh
+```

data/lib/sidekiq/ratomic/pool/errors.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+# Sidekiq integration namespace.
+module Sidekiq
+  # Ratomic-backed Sidekiq middleware namespace.
+  module Ratomic
+    class Pool
+      # Base error for pool failures.
+      class Error < StandardError; end
+
+      # Raised when a checked-out resource fails validation.
+      class CheckoutError < Error; end
+
+      # Raised when the circuit breaker is open.
+      class CircuitOpenError < Error; end
+    end
+  end
+end

data/lib/sidekiq/ratomic/pool/version.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+# Sidekiq integration namespace.
+module Sidekiq
+  # Ratomic-backed Sidekiq middleware namespace.
+  module Ratomic
+    class Pool
+      # Current gem version.
+      VERSION = '0.1.0'
+    end
+  end
+end

data/lib/sidekiq/ratomic/pool.rb

@@ -0,0 +1,169 @@
+# frozen_string_literal: true
+
+require 'ratomic'
+require 'timeout'
+require_relative 'pool/errors'
+require_relative 'pool/version'
+
+# Sidekiq integration namespace.
+module Sidekiq
+  # Ratomic-backed Sidekiq middleware namespace.
+  module Ratomic
+    # Sidekiq server middleware that exposes a Ractor-local resource pool.
+    #
+    # Resources are validated before checkout and transient failures are retried
+    # with exponential backoff. Persistent failures open the circuit breaker.
+    class Pool
+      attr_reader :pool_name, :size, :max_retries, :retry_delay, :validator, :cb_threshold, :cb_timeout,
+                  :retryable_errors
+
+      def initialize(pool_name:, size: 10, max_retries: 3, retry_delay: 0.2,
+                     cb_threshold: 5, cb_timeout: 30, validator: nil,
+                     retryable_errors: [IOError, SystemCallError, Timeout::Error], factory: nil, &block)
+        factory ||= block
+        raise ArgumentError, 'A resource factory must be provided' unless factory
+
+        validate_options!(size, max_retries, retry_delay, cb_threshold, cb_timeout)
+
+        @pool_name = pool_name.to_sym
+        @size = size
+        @max_retries = max_retries
+        @retry_delay = retry_delay
+        @cb_threshold = cb_threshold
+        @cb_timeout = cb_timeout
+        @validator = validator || method(:default_validator)
+        @retryable_errors = retryable_errors.freeze
+        @state_mutex = Mutex.new
+        @failure_count = ::Ratomic::Counter.new
+        @state = :closed
+        @last_state_change = monotonic_time
+
+        shareable_factory = Ractor.make_shareable(factory)
+        @local_pool = ::Ratomic::LocalPool.new(size: @size, factory: shareable_factory)
+      end
+
+      # Inject this pool into a worker's configured pool accessor.
+      def call(job_instance, _job_payload, _queue)
+        setter = "#{@pool_name}="
+        job_instance.public_send(setter, self) if job_instance.respond_to?(setter)
+        yield
+      end
+
+      # Close resources owned by the current Ractor.
+      def close
+        @local_pool.close
+      end
+
+      alias shutdown close
+
+      # Check out a healthy resource and yield it to the caller.
+      def with
+        check_circuit_state!
+        attempts = 0
+        work_failed = false
+
+        begin
+          attempts += 1
+          @local_pool.with do |resource|
+            raise Pool::CheckoutError, 'Resource connection health check failed' unless verify_health(resource)
+
+            begin
+              result = yield resource
+            rescue StandardError
+              work_failed = true
+              raise
+            end
+            record_success
+            result
+          end
+        rescue StandardError => e
+          raise unless !work_failed || retryable_error?(e)
+
+          record_failure
+          if attempts <= @max_retries && state != :open
+            delay = @retry_delay * (2**(attempts - 1))
+            sleep(delay) if delay.positive?
+            retry
+          end
+
+          raise
+        end
+      end
+
+      # Return the current circuit-breaker state.
+      def state
+        @state_mutex.synchronize do
+          if @state == :open && monotonic_time - @last_state_change > @cb_timeout
+            @state = :half_open
+            @last_state_change = monotonic_time
+          end
+          @state
+        end
+      end
+
+      private
+
+      def check_circuit_state!
+        return unless state == :open
+
+        raise Pool::CircuitOpenError, 'Circuit breaker is open'
+      end
+
+      def verify_health(resource)
+        @validator.call(resource)
+      rescue StandardError
+        false
+      end
+
+      def retryable_error?(error)
+        !work_error?(error) || @retryable_errors.any? { |error_class| error.is_a?(error_class) }
+      end
+
+      def work_error?(error)
+        error.is_a?(StandardError) && !error.is_a?(Pool::CheckoutError)
+      end
+
+      def validate_options!(size, max_retries, retry_delay, cb_threshold, cb_timeout)
+        validations = [
+          [size.is_a?(Integer) && size.positive?, 'size must be a positive Integer'],
+          [max_retries.is_a?(Integer) && max_retries >= 0, 'max_retries must be a non-negative Integer'],
+          [retry_delay.is_a?(Numeric) && retry_delay >= 0, 'retry_delay must be non-negative'],
+          [cb_threshold.is_a?(Integer) && cb_threshold.positive?, 'cb_threshold must be a positive Integer'],
+          [cb_timeout.is_a?(Numeric) && cb_timeout >= 0, 'cb_timeout must be non-negative']
+        ]
+        validations.each do |valid, message|
+          raise ArgumentError, message unless valid
+        end
+      end
+
+      def default_validator(resource)
+        return resource.ping if resource.respond_to?(:ping)
+        return resource.active? if resource.respond_to?(:active?)
+
+        true
+      end
+
+      def record_success
+        @state_mutex.synchronize do
+          failure_count = @failure_count.value
+          @failure_count.decrement(failure_count) unless failure_count.zero?
+          @state = :closed if @state == :half_open
+        end
+      end
+
+      def record_failure
+        @state_mutex.synchronize do
+          @failure_count.increment(1)
+          if @failure_count.value >= @cb_threshold || @state == :half_open
+            @state = :open
+            @last_state_change = monotonic_time
+          end
+        end
+      end
+
+      def monotonic_time
+        Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      end
+    end
+  end
+end

data/lib/sidekiq_ratomic_pool.rb

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+
+require_relative 'sidekiq/ratomic/pool'

data/sig/sidekiq/ratomic/pool.rbs

@@ -0,0 +1,62 @@
+module Sidekiq
+  module Ratomic
+    class Pool
+      VERSION: String
+
+      class Error < StandardError
+      end
+
+      class CheckoutError < Error
+      end
+
+      class CircuitOpenError < Error
+      end
+
+      attr_reader pool_name: Symbol
+      attr_reader size: Integer
+      attr_reader max_retries: Integer
+      attr_reader retry_delay: untyped
+      attr_reader validator: ^(untyped) -> bool
+      attr_reader cb_threshold: Integer
+      attr_reader cb_timeout: Numeric
+      attr_reader retryable_errors: Array[Class]
+
+      def initialize: (
+        pool_name: Symbol | String,
+        ?size: Integer,
+        ?max_retries: Integer,
+        ?retry_delay: Numeric,
+        ?cb_threshold: Integer,
+        ?cb_timeout: Numeric,
+        ?validator: ^(untyped) -> bool,
+        ?retryable_errors: Array[Class],
+        ?factory: _SidekiqRatomicFactory
+      ) { () -> untyped } -> void
+
+      def call: (untyped, untyped, untyped) { () -> untyped } -> untyped
+      def close: () -> nil
+      def shutdown: () -> nil
+      def with: () { (untyped) -> untyped } -> untyped
+      def state: () -> Symbol
+      private
+      def validate_options!: (untyped, untyped, untyped, untyped, untyped) -> void
+      def check_circuit_state!: () -> void
+      def verify_health: (untyped) -> bool
+      def retryable_error?: (StandardError) -> bool
+      def work_error?: (StandardError) -> bool
+      def default_validator: (untyped) -> bool
+      def record_success: () -> void
+      def record_failure: () -> void
+      def monotonic_time: () -> Float
+    end
+  end
+end
+
+interface _SidekiqRatomicFactory
+  def call: () -> untyped
+end
+
+module Timeout
+  class Error < StandardError
+  end
+end

data/smoke_test/client.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+require 'sidekiq'
+require 'redis-client'
+require_relative 'worker'
+
+redis_url = ENV.fetch('REDIS_URL', 'redis://127.0.0.1:6379/0').freeze
+job_count = Integer(ENV.fetch('SMOKE_JOB_COUNT', '20'))
+timeout = Integer(ENV.fetch('SMOKE_TIMEOUT', '30'))
+run_id = ENV.fetch('SMOKE_RUN_ID', Process.clock_gettime(Process::CLOCK_REALTIME, :nanosecond).to_s)
+redis = RedisClient.config(url: redis_url).new_client
+Sidekiq.configure_client { |config| config.redis = { url: redis_url } }
+
+redis.call('DEL', SmokeTest::COUNT_KEY, SmokeTest::RESULTS_KEY)
+job_ids = job_count.times.map do |index|
+  job_id = "#{run_id}-#{index}"
+  Sidekiq::Client.push('class' => SmokeTest::RedisSmokeWorker, 'args' => [job_id], 'queue' => 'smoke')
+  job_id
+end
+
+deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + timeout
+loop do
+  processed = Integer(redis.call('GET', SmokeTest::COUNT_KEY) || 0)
+  results = Integer(redis.call('HLEN', SmokeTest::RESULTS_KEY) || 0)
+  break if processed >= job_count && results >= job_count
+
+  if Process.clock_gettime(Process::CLOCK_MONOTONIC) >= deadline
+    abort "Timed out waiting for #{job_count} jobs (processed=#{processed}, results=#{results})"
+  end
+
+  sleep 0.1
+end
+
+missing = job_ids.reject { |job_id| redis.call('HGET', SmokeTest::RESULTS_KEY, job_id) == 'processed' }
+abort "Missing processed jobs: #{missing.join(', ')}" unless missing.empty?
+
+puts "Smoke test passed: #{job_count} jobs processed by standalone Sidekiq through real Redis connections"

data/smoke_test/server.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+require 'sidekiq'
+require 'redis-client'
+require 'sidekiq_ratomic_pool'
+require_relative 'worker'
+
+RedisFactory = Data.define(:url) do
+  def call
+    RedisClient.config(url:).new_client
+  end
+end
+
+redis_port = ENV.fetch('REDIS_PORT', '6379')
+redis_url = ENV.fetch('REDIS_URL', "redis://127.0.0.1:#{redis_port}/0").freeze
+
+Sidekiq.configure_server do |config|
+  config.redis = { url: redis_url }
+  config.server_middleware do |chain|
+    chain.add(
+      Sidekiq::Ratomic::Pool,
+      pool_name: :redis_pool,
+      size: Integer(ENV.fetch('RATOMIC_POOL_SIZE', '4')),
+      max_retries: 3,
+      retry_delay: 0.05,
+      cb_threshold: 5,
+      cb_timeout: 5,
+      validator: ->(redis) { redis.call('PING') == 'PONG' },
+      factory: RedisFactory.new(redis_url).freeze
+    )
+  end
+end

data/smoke_test/worker.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module SmokeTest
+  COUNT_KEY = 'sidekiq-ratomic-pool:smoke:processed'
+  RESULTS_KEY = 'sidekiq-ratomic-pool:smoke:results'
+
+  # Sidekiq job that performs real Redis work through the injected pool.
+  class RedisSmokeWorker
+    include Sidekiq::Job
+
+    sidekiq_options queue: 'smoke', retry: 2
+
+    attr_accessor :redis_pool
+
+    def perform(job_id)
+      redis_pool.with do |redis|
+        redis.call('INCR', SmokeTest::COUNT_KEY)
+        redis.call('HSET', SmokeTest::RESULTS_KEY, job_id, 'processed')
+        work_seconds = Float(ENV.fetch('SMOKE_WORK_SECONDS', '0.05'))
+        sleep work_seconds if work_seconds.positive?
+      end
+    end
+  end
+end

metadata

@@ -0,0 +1,72 @@
+--- !ruby/object:Gem::Specification
+name: sidekiq-ratomic-pool
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Ken C. Demanawa
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: ratomic
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.4.3
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.4.3
+description: |
+  Thread and Ractor safe connection pooling utilizing Ratomic's LocalPool for Sidekiq middleware
+  with health validation, exponential retries, and circuit breakers.
+email:
+- kenneth.c.demanawa@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- lib/sidekiq/ratomic/pool.rb
+- lib/sidekiq/ratomic/pool/errors.rb
+- lib/sidekiq/ratomic/pool/version.rb
+- lib/sidekiq_ratomic_pool.rb
+- sig/sidekiq/ratomic/pool.rbs
+- smoke_test/client.rb
+- smoke_test/server.rb
+- smoke_test/worker.rb
+homepage: https://kanutocd.github.io/sidekiq-ratomic-pool
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://kanutocd.github.io/sidekiq-ratomic-pool
+  source_code_uri: https://github.com/kanutocd/sidekiq-ratomic-pool
+  changelog_uri: https://github.com/kanutocd/sidekiq-ratomic-pool/blob/main/CHANGELOG.md
+  documentation_uri: https://kanutocd.github.io/sidekiq-ratomic-pool
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '4.0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.18
+specification_version: 4
+summary: Ractor-safe connection pooling for Sidekiq utilizing Ratomic's LocalPool.
+test_files: []