ratomic 0.3.5 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6cce49de24aecb066a0fcab6361ab4637b138ea1162888678701ba9030bfe2d1
-  data.tar.gz: 93ff58b2084bb8fd10edccf7b059e5e2b6503f0814256ecabba9065c9a5e4139
+  metadata.gz: a08c265d5b7a1c6f780a36dc2448b7cb49421c53f15957a242dd1457503b33c1
+  data.tar.gz: 5dcbb8b1fd92a0e162dcc4daa9ca6cec31117c918bb984ddf03a5ccb68df8779
 SHA512:
-  metadata.gz: 49cb72854fd73aa9e0cc5c3ee9614ba01433748101bd6f7037d8b498d3b9853761293efe1a680b2c3e9251c5b85274c250e068674696959e84513080682b8cdf
-  data.tar.gz: 63f5f4e7584a46d5f3fd34f611fce08e0d6d451d72b91907a947018ce5b2a14ffca87e551189ee92dec64be3989ed56f062cf8f96a62a4c5a1df42890b456093
+  metadata.gz: 0dccc6d4fcceed16ac0437d2550c25deb6ad83c4de57a5b0465d0de604f4376baa776312607d9bd361073c53eeef14e88c651db5cf3efb09bedcd3a38e9ee93d
+  data.tar.gz: b6e9154857321453b8e0b2919ee35551beab7763b80c02358172537e35375628420f10c434d3e97432f87f99e567794de6343855a791d74aa0540d7d11e5ea7c

data/CHANGELOG.md

@@ -1,5 +1,35 @@
 ## [Unreleased]
 
+## [0.4.0] - 2026-06-10
+
+### Added
+
+* Introduced `Ratomic::LocalPool`, a new pooling primitive for live resources that should remain local to the Ractor that created them.
+* Added Redis-based smoke tests demonstrating safe operation across both Threads and Ractors.
+* Added queue producer/consumer Redis examples.
+* Added RBS definitions for `LocalPool`.
+* Expanded API documentation and usage examples.
+
+### Design
+
+`Ratomic::Pool` and `Ratomic::LocalPool` serve different ownership models:
+
+* `Pool` — ownership transfer
+* `LocalPool` — ownership preservation
+
+`LocalPool` is intended for resources such as:
+
+* Redis clients
+* Database connections
+* HTTP clients
+* Kafka producers
+* Other stateful network resources
+
+### Notes
+
+`LocalPool` is implemented in pure Ruby and is not backed by Ratomic's Rust extension.
+
+
 ## [0.3.5] - 2026-06-06
 
 - Fix the native loader so development loads the compiled extension from the

data/Cargo.toml

@@ -2,7 +2,14 @@
 members = ["ext/ratomic"]
 resolver = "2"
 
+[profile.dev]
+debug = true
+
 [profile.release]
 panic = "abort"
 lto = true
 codegen-units = 1
+# By default, debug symbols are stripped from the final binary which makes it
+# harder to debug if something goes wrong. It's recommended to keep debug
+# symbols in the release build so that you can debug the final binary if needed.
+debug = false

data/README.md

@@ -5,7 +5,7 @@
 [![Ruby Version](https://img.shields.io/badge/ruby-%3E%3D%204.0-ruby.svg)](https://www.ruby-lang.org/en/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-Ratomic provides mutable data structures for Ruby Ractors. Its primitives are backed by native Rust concurrency libraries so Ruby code can share useful state across Ractors without falling back to one global lock. `Pool` uses Ruby Ractor ownership-transfer primitives instead of the native Rust path.
+Ratomic provides mutable data structures for Ruby Ractors. Its core shared primitives are backed by native Rust concurrency libraries so Ruby code can share useful state across Ractors without falling back to one global lock. `Pool` and `LocalPool` are pure Ruby primitives that use Ruby Ractor ownership and locality semantics instead of the native Rust path.
 
 ## Project Direction
 
@@ -43,18 +43,27 @@ RBS signatures are included under `sig/` for downstream type checking.
 ## Examples And Benchmarks
 
 - [`redis_poc`](./redis_poc) contains local Redis scripts that exercise
-  `Ratomic::Map`, `Ratomic::Counter`, and `Ratomic::Pool` under Thread and
+  `Ratomic::Map`, `Ratomic::Counter`, and `Ratomic::LocalPool` under Thread and
   Ractor workloads.
+- [`pgoutput-parser`](https://github.com/kanutocd/pgoutput-parser#relation-metadata-tracking)
+  uses `Ratomic::Map` for relation metadata tracking in a real CDC pipeline
+  POC, with a matching benchmark and deeper implementation notes in
+  [docs/relation_tracker.md](https://github.com/kanutocd/pgoutput-parser/blob/main/docs/relation_tracker.md).
+- [`sidekiq-tenant-policy-cache`](https://github.com/kanutocd/sidekiq-tenant-policy-cache)
+  shows `Ratomic::Map` and `Ratomic::Counter` in Sidekiq middleware for tenant
+  policy caching and cache-hit / cache-miss tracking, with a benchmarked
+  cache-vs-policy-every-job comparison.
 - The [`cdc-parallel` Ratomic benchmark][cdc-parallel-ratomic] demonstrates
   Ractor workers updating shared CDC processing metrics through `Ratomic::Map`
   and `Ratomic::Counter`.
 
 ## Usage
 
-Ratomic provides two safety models:
+Ratomic provides three safety models:
 
 - `Counter`, `Map`, and `Queue` are shared concurrent structures.
-- `Pool` transfers ownership of mutable objects between Ractors.
+- `Pool` transfers ownership of plain mutable objects between Ractors.
+- `LocalPool` keeps live resources local to the Ractor that created them.
 
 That distinction matters. A mutable pooled object is not shared by multiple Ractors
 at the same time. It is moved to the caller on checkout and moved back to the pool
@@ -119,6 +128,11 @@ groups.append("jobs", "import") # => ["import"]
 groups.add_to_set("workers", "alpha") # => #<Set: {"alpha"}>
 ```
 
+Some `Map` methods hold an internal guard while a block runs or while a
+reference is live. Avoid re-entering the same map from inside those blocks or
+mutating the same key while holding a reference from `get` or `[]`. The API
+docs cover the exact locking caveats.
+
 ### `Ratomic::Queue`
 
 `Ratomic::Queue` is a Ractor-shareable multi-producer, multi-consumer queue.
@@ -219,6 +233,142 @@ The lower-level `Ratomic::FixedSizeObjectPool` native class may still exist, but
 `Ratomic::Pool` does not inherit from it. The public `Pool` API is implemented
 in Ruby so it can use Ruby's Ractor ownership primitives directly.
 
+
+### `Ratomic::LocalPool`
+
+`Ratomic::LocalPool` is the safe pool shape for live resources that should stay
+local to the Ractor that created them.
+
+Use it for resources such as:
+
+- Redis clients
+- database connections
+- HTTP clients
+- Kafka producers
+- OpenSearch clients
+- per-worker caches, buffers, encoders, or aggregators
+
+Unlike `Ratomic::Pool`, `LocalPool` does **not** move pooled objects between
+Ractors. The `LocalPool` instance is a shareable facade. Each Ractor lazily
+creates and owns its own private, thread-safe resource pool behind that facade.
+Threads inside the same Ractor share that local pool, but different Ractors
+never share the live resources.
+
+```ruby
+require "ratomic"
+require "redis-client"
+
+RedisFactory = Data.define(:host) do
+  def call
+    RedisClient.new(host: host)
+  end
+end
+
+REDIS = Ratomic::LocalPool.new(
+  size: 10,
+  timeout: 1,
+  factory: RedisFactory.new("127.0.0.1".freeze)
+)
+
+REDIS.with do |client|
+  client.call("ping")
+end
+```
+
+Use `Pool` for plain mutable values where ownership transfer is the intended
+safety model. Use `LocalPool` for live resources that should be created, used,
+and reused inside the same Ractor.
+
+The intended topology is:
+
+```text
+shareable LocalPool facade
+        ↓
+one local resource pool per Ractor
+        ↓
+threads inside that Ractor share local resources
+```
+
+The mental model is intentionally close to a Ruby local variable: the resource is
+local to the execution scope that owns it. For `LocalPool`, that scope is the
+current Ractor.
+
+#### Pure Ruby implementation
+
+`LocalPool` is implemented in pure Ruby.
+
+Unlike `Counter`, `Map`, and `Queue`, it is not backed by the Rust native
+extension. Its safety comes from Ruby Ractor ownership boundaries and locality,
+not from Rust synchronization primitives.
+
+#### Why not use `Pool` for Redis clients?
+
+`Pool` moves checked-out objects between Ractors. That is correct for plain
+mutable Ruby values such as arrays or buffers, but it is a poor fit for live I/O
+resources. Redis clients, database connections, sockets, and similar resources
+carry internal connection state. Moving those objects across Ractor boundaries can
+leave nested internal state unusable, producing errors such as
+`Ractor::MovedError`.
+
+`LocalPool` avoids that class of bug by not moving live resources at all. Work
+moves between Ractors. Live resources stay local.
+
+#### Redis smoke-test snapshot
+
+The Redis POC includes two scripts under `redis_poc/`.
+
+`basic_redis.rb` exercises repeated Redis operations from both Threads and
+Ractors:
+
+```text
+Thread
+[{"one" => 31501}, {"two" => 31379}, {"three" => 31320}, {"four" => 31410}, {"five" => 31454}]
+
+Ractor
+[{"one" => 42419}, {"two" => 42186}, {"three" => 42400}, {"four" => 42206}, {"five" => 42568}]
+```
+
+`queue_redis.rb` exercises a producer/consumer Redis queue workload:
+
+```text
+[:start, Ractor, 2026-06-10 01:17:57.584727984 +0800]
+[[:producer_done, 0], [:producer_done, 1]]
+[[:consumer_done, 0, 7998], [:consumer_done, 1, 7987], [:consumer_done, 2, 7984], [:consumer_done, 3, 8035], [:consumer_done, 4, 7996]]
+[{"one" => 0}, {"two" => 0}, {"three" => 0}, {"four" => 0}, {"five" => 0}]
+[:end, 2026-06-10 01:18:02.226310862 +0800]
+```
+
+These numbers are a smoke-test snapshot, not a formal benchmark claim. The
+important interpretation is:
+
+- no `Ractor::MovedError`
+- no `Ractor::IsolationError`
+- no process crash
+- all produced queue items were consumed
+- Redis queues drained to zero
+- live Redis clients remained owned by the Ractor that created them
+
+#### Inception pool
+
+Internally, `LocalPool` follows the "inception pool" shape discovered while
+experimenting with Redis clients under Ruby Ractors:
+
+```text
+pool facade
+  ↓
+local pool
+  ↓
+resource
+```
+
+That same ownership pattern appears in hybrid execution runtimes: put parallel
+workers on the outside, keep I/O concurrency and live resources inside the worker
+that owns them. Ratomic keeps the primitive general-purpose and independent of
+any specific runtime, scheduler, database, or message system.
+
+`LocalPool#close` closes only the current Ractor's local pool. Other Ractors own
+their own pools and must close them independently when needed.
+
 ## Contributing
 
 Please read the [Code of Conduct](./CODE_OF_CONDUCT.md) before contributing.

data/lib/ratomic/local_pool.rb

@@ -0,0 +1,246 @@
+# frozen_string_literal: true
+
+require "timeout"
+
+module Ratomic
+  # Design Note
+  #
+  # LocalPool originated while investigating Redis clients under Ruby
+  # Ractors. The original goal was to reuse Pool, but ownership-transfer
+  # semantics proved incompatible with live resources containing internal
+  # state.
+  #
+  # The resulting architecture became known as the "Inception Pool"
+  # design:
+  #
+  #   LocalPool facade
+  #          ↓
+  #   Ractor-local pool
+  #          ↓
+  #   Live resources
+  #
+  # or informally:
+  #
+  #   Pool
+  #     ↓
+  #   Pool
+  #     ↓
+  #   Resource
+  #
+  # The public API intentionally uses the more descriptive name `LocalPool`.
+  #
+  # A shareable facade over resources that stay local to each Ractor.
+  #
+  # LocalPool is intended for live resources such as Redis clients,
+  # database connections, sockets, and other objects which must not be moved
+  # between Ractors. The facade itself is shareable, but each Ractor lazily
+  # creates and owns an independent thread-safe local pool. Threads inside the
+  # same Ractor share that local pool; different Ractors never share the live
+  # resources.
+  #
+  # This is the correct shape for resources with process, socket, connection,
+  # or native state. Move work across Ractor boundaries, not live clients.
+  #
+  # The factory must be Ractor-shareable because the facade stores it and each
+  # Ractor calls it when its local resource pool needs to create a resource. Prefer a
+  # small immutable callable object instead of a block when the pool will be
+  # used from multiple Ractors.
+  #
+  # @example Redis clients owned by each Ractor
+  #   RedisFactory = Data.define(:host) do
+  #     def call
+  #       RedisClient.new(host: host)
+  #     end
+  #   end
+  #
+  #   REDIS = Ratomic::LocalPool.new(
+  #     size: 5,
+  #     timeout: 1,
+  #     factory: RedisFactory.new("127.0.0.1".freeze)
+  #   )
+  #
+  #   REDIS.with { |client| client.call("ping") }
+  #
+  # @note LocalPool is implemented in pure Ruby. It is not backed by the Rust
+  #   native extension used by Counter, Map, and Queue. Its safety comes from
+  #   Ruby Ractor locality: live resources are created and reused inside the
+  #   Ractor that owns them.
+  #
+  # @see Pool Use Pool for plain mutable Ruby values where ownership transfer
+  #   is the desired safety model.
+  class LocalPool
+    # Create a per-Ractor local pool facade.
+    #
+    # @param size [Integer] maximum number of resources in each Ractor-local pool
+    # @param timeout [Numeric, nil] checkout timeout in seconds, or nil to wait indefinitely
+    # @param factory [#call, nil] shareable object factory
+    # @yieldreturn [Object] resource created inside the current Ractor
+    # @raise [ArgumentError] if size, timeout, or factory is invalid
+    # @raise [LocalJumpError] if no factory or block is given
+    def initialize(size: 5, timeout: 1.0, factory: nil, &block)
+      raise ArgumentError, "pool size must be positive" unless size.is_a?(Integer) && size.positive?
+      raise ArgumentError, "pool timeout must be numeric or nil" unless timeout.nil? || timeout.is_a?(Numeric)
+      raise ArgumentError, "pool timeout must be non-negative" if timeout && timeout.negative?
+      raise ArgumentError, "use either factory: or block, not both" if factory && block
+
+      factory ||= block
+      raise LocalJumpError, "no factory given" unless factory
+      raise ArgumentError, "factory must respond to #call" unless factory.respond_to?(:call)
+      raise ArgumentError, "factory must be Ractor-shareable" unless Ractor.shareable?(factory)
+
+      @size = size
+      @timeout = timeout&.to_f
+      @factory = factory
+      @storage_key = :"ratomic_local_pool_#{object_id}"
+
+      freeze
+      Ractor.make_shareable(self)
+    end
+
+    # Checkout a current-Ractor-owned resource, yield it, then return it to the
+    # same Ractor-local pool.
+    #
+    # No resource is moved between Ractors. The yielded object belongs to the
+    # Ractor which called this method.
+    #
+    # @yieldparam object [Object] current-Ractor-owned resource
+    # @raise [Ratomic::Error] if checkout times out
+    # @return [Object] the block return value
+    def with
+      local_pool.with { |object| yield object }
+    rescue Timeout::Error
+      raise Ratomic::Error, "pool checkout timeout"
+    end
+
+    # Close the current Ractor's local pool, if it has been initialized.
+    #
+    # Other Ractors own independent local pools and are not affected. Available
+    # resources are closed if they respond to #close. Resources currently
+    # checked out by threads in this Ractor are closed when returned.
+    #
+    # @return [nil]
+    def close
+      pool = Ractor.current[@storage_key]
+      return nil unless pool
+
+      Ractor.current[@storage_key] = nil
+      pool.close
+      nil
+    end
+
+    # Minimal thread-safe resource pool used inside exactly one Ractor.
+    class ResourcePool
+      def initialize(size:, timeout:, factory:)
+        @size = size
+        @timeout = timeout
+        @factory = factory
+        @available = []
+        @created = 0
+        @closed = false
+        @mutex = Mutex.new
+        @condition = ConditionVariable.new
+      end
+
+      def with
+        object = checkout
+        yield object
+      ensure
+        checkin(object) if object
+      end
+
+      def close
+        objects = nil
+
+        @mutex.synchronize do
+          @closed = true
+          objects = @available.dup
+          @available.clear
+          @condition.broadcast
+        end
+
+        objects.each { |object| close_object(object) }
+        nil
+      end
+
+      private
+
+      def checkout
+        deadline = monotonic_deadline
+
+        should_create = @mutex.synchronize do
+          raise IOError, "pool is closed" if @closed
+
+          loop do
+            object = @available.pop
+            return object if object
+
+            if @created < @size
+              @created += 1
+              break true
+            end
+
+            wait_for_available(deadline)
+            raise IOError, "pool is closed" if @closed
+          end
+        end
+
+        create_object if should_create
+      end
+
+      def checkin(object)
+        close_now = false
+
+        @mutex.synchronize do
+          if @closed
+            close_now = true
+          else
+            @available << object
+            @condition.signal
+          end
+        end
+
+        close_object(object) if close_now
+        nil
+      end
+
+      def create_object
+        @factory.call
+      rescue Exception # rubocop:disable Lint/RescueException
+        @mutex.synchronize do
+          @created -= 1
+          @condition.signal
+        end
+        raise
+      end
+
+      def close_object(object)
+        object.close if object.respond_to?(:close)
+      end
+
+      def monotonic_deadline
+        return nil unless @timeout
+
+        Process.clock_gettime(Process::CLOCK_MONOTONIC) + @timeout
+      end
+
+      def wait_for_available(deadline)
+        if deadline
+          remaining = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
+          raise Timeout::Error, "pool checkout timeout" if remaining <= 0
+
+          @condition.wait(@mutex, remaining)
+        else
+          @condition.wait(@mutex)
+        end
+      end
+    end
+
+    private
+
+    def local_pool
+      Ractor.current[@storage_key] ||= ResourcePool.new(size: @size, timeout: @timeout, factory: @factory)
+    end
+
+    private_constant :ResourcePool
+  end
+end

data/lib/ratomic/map.rb

@@ -11,6 +11,11 @@ module Ratomic
   # This is not a full Hash replacement. Iteration and arbitrary mutable object
   # borrowing are intentionally absent.
   #
+  # Some methods on this type hold an internal entry guard while the block runs
+  # or while a reference is live. Do not call back into the same map from inside
+  # those blocks, and do not hold a reference from #get or #[] while mutating
+  # the same key. That can deadlock the underlying DashMap bucket.
+  #
   # @example Store pipeline offsets
   #   OFFSETS = Ratomic::Map.new
   #   OFFSETS[:source_a] = 42
@@ -22,6 +27,10 @@ module Ratomic
   #   Missing keys return nil, so use #key? or #fetch when stored nil values
   #   need to be distinguished from missing entries.
   #
+  #   If you keep the returned reference alive and then mutate the same key, you
+  #   can deadlock the underlying DashMap bucket. Copy out what you need and let
+  #   the reference go out of scope before mutating.
+  #
   #   @param key [Object] lookup key
   #   @return [Object, nil] the stored value, or nil when the key is missing
   #
@@ -102,6 +111,10 @@ module Ratomic
   #   into the same map from inside the block. Prefer native update helpers such
   #   as #increment when they fit the workflow.
   #
+  #   Never call this method from inside another live reference to the same
+  #   entry or from a block that already holds the same map's guard. That can
+  #   deadlock the bucket.
+  #
   #   If the block raises, the previous value is preserved. If the key was
   #   missing, no entry is inserted.
   #
@@ -141,6 +154,10 @@ module Ratomic
   #   into the same map from inside the block. Prefer native update helpers such
   #   as #increment when they fit the workflow.
   #
+  #   Never call this method from inside another live reference to the same
+  #   entry or from a block that already holds the same map's guard. That can
+  #   deadlock the bucket.
+  #
   #   If the block raises, the previous value is preserved.
   #
   #   @param key [Object] key to update

data/lib/ratomic/version.rb

@@ -2,5 +2,5 @@
 
 module Ratomic
   # Current gem version string.
-  VERSION = "0.3.5"
+  VERSION = "0.4.0"
 end

data/lib/ratomic.rb

@@ -4,10 +4,11 @@ require "rbconfig"
 
 # Ratomic provides mutable data structures for Ruby Ractors. Its primitives
 # are backed by native Rust concurrency libraries so Ruby code can share useful
-# state across Ractors without falling back to one global lock. Pool uses Ruby
-# Ractor ownership-transfer primitives instead of the native Rust path.
+# state across Ractors without falling back to one global lock. `Pool` and
+# `LocalPool` are pure Ruby primitives which use Ruby Ractor ownership and
+# locality semantics instead of the native Rust path.
 #
-# The public API currently includes {Counter}, {Map}, {Queue}, and {Pool}.
+# The public API currently includes {Counter}, {Map}, {Queue}, {Pool}, and {LocalPool}.
 module Ratomic
   # Base error class for Ratomic-specific failures.
   class Error < StandardError; end
@@ -53,3 +54,5 @@ require "ratomic/counter"
 require "ratomic/map"
 require "ratomic/queue"
 require "ratomic/pool"
+
+require "ratomic/local_pool"

data/sig/ratomic.rbs

@@ -62,6 +62,14 @@ module Ratomic
     def length: () -> Integer
   end
 
+  class LocalPool
+    def self.new: (?size: Integer, ?timeout: (Numeric | nil), factory: untyped) -> LocalPool
+                | (?size: Integer, ?timeout: (Numeric | nil)) { () -> untyped } -> LocalPool
+
+    def with: () { (untyped object) -> untyped } -> untyped
+    def close: () -> nil
+  end
+
   class Pool
     def self.new: (?Integer size, ?(Numeric | nil) timeout) { () -> untyped } -> Pool
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ratomic
 version: !ruby/object:Gem::Version
-  version: 0.3.5
+  version: 0.4.0
 platform: ruby
 authors:
 - Mike Perham
@@ -50,6 +50,7 @@ files:
 - ext/ratomic/src/sem.rs
 - lib/ratomic.rb
 - lib/ratomic/counter.rb
+- lib/ratomic/local_pool.rb
 - lib/ratomic/map.rb
 - lib/ratomic/pool.rb
 - lib/ratomic/queue.rb