connection_pool 1.0.0 → 2.5.5

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 24c74a1caa5e04827c47155b75e19805bcda4c329e28793309dfa95b5881c4bd
+  data.tar.gz: 23aadf2da494be8c3314039700606cd4e01b58d6cae1f45e3b7cdef57a98e7bd
+SHA512:
+  metadata.gz: 0f2385ddf4619ebc1bf9e2f202024b330c170ae2e4e7e63480be0f556242140af75eb3d272f6d262b2fc859625f0861c560af6a58c20bea3d69b005e2d248472
+  data.tar.gz: 0ac5103c69aa2e8226d2fa872714f9bd611b24ea171e91ce3d7a3c9be62b9887e24916cf4b6b1e5bc8c407349f87c3ea855c4659dbf5f3f137cd45738301d1a6

data/Changes.md

@@ -1,3 +1,153 @@
+# connection_pool Changelog
+
+2.5.5
+------
+
+- Support `ConnectionPool::TimedStack#pop(exception: false)` [#207]
+  to avoid using exceptions as control flow.
+
+2.5.4
+------
+
+- Add ability to remove a broken connection from the pool [#204, womblep]
+
+2.5.3
+------
+
+- Fix TruffleRuby/JRuby crash [#201]
+
+2.5.2
+------
+
+- Rollback inadvertant change to `auto_reload_after_fork` default. [#200]
+
+2.5.1
+------
+
+- Pass options to TimedStack in `checkout` [#195]
+- Optimize connection lookup [#196]
+- Fixes for use with Ractors
+
+2.5.0
+------
+
+- Reap idle connections [#187]
+```ruby
+idle_timeout = 60
+pool = ConnectionPool.new ...
+pool.reap(idle_timeout, &:close)
+```
+- `ConnectionPool#idle` returns the count of connections not in use [#187]
+
+2.4.1
+------
+
+- New `auto_reload_after_fork` config option to disable auto-drop [#177, shayonj]
+
+2.4.0
+------
+
+- Automatically drop all connections after fork [#166]
+
+2.3.0
+------
+
+- Minimum Ruby version is now 2.5.0
+- Add pool size to TimeoutError message
+
+2.2.5
+------
+
+- Fix argument forwarding on Ruby 2.7 [#149]
+
+2.2.4
+------
+
+- Add `reload` to close all connections, recreating them afterwards [Andrew Marshall, #140]
+- Add `then` as a way to use a pool or a bare connection with the same code path [#138]
+
+2.2.3
+------
+
+- Pool now throws `ConnectionPool::TimeoutError` on timeout. [#130]
+- Use monotonic clock present in all modern Rubies [Tero Tasanen, #109]
+- Remove code hacks necessary for JRuby 1.7
+- Expose wrapped pool from ConnectionPool::Wrapper [Thomas Lecavelier, #113]
+
+2.2.2
+------
+
+- Add pool `size` and `available` accessors for metrics and monitoring
+  purposes [#97, robholland]
+
+2.2.1
+------
+
+- Allow CP::Wrapper to use an existing pool [#87, etiennebarrie]
+- Use monotonic time for more accurate timeouts [#84, jdantonio]
+
+2.2.0
+------
+
+- Rollback `Timeout` handling introduced in 2.1.1 and 2.1.2.  It seems
+  impossible to safely work around the issue. Please never, ever use
+  `Timeout.timeout` in your code or you will see rare but mysterious bugs. [#75]
+
+2.1.3
+------
+
+- Don't increment created count until connection is successfully
+  created. [mylesmegyesi, #73]
+
+2.1.2
+------
+
+- The connection\_pool will now close any connections which respond to
+  `close` (Dalli) or `disconnect!` (Redis).  This ensures discarded connections
+  from the fix in 2.1.1 are torn down ASAP and don't linger open.
+
+
+2.1.1
+------
+
+- Work around a subtle race condition with code which uses `Timeout.timeout` and
+  checks out a connection within the timeout block.  This might cause
+  connections to get into a bad state and raise very odd errors. [tamird, #67]
+
+
+2.1.0
+------
+
+- Refactoring to better support connection pool subclasses [drbrain,
+  #55]
+- `with` should return value of the last expression [#59]
+
+
+2.0.0
+-----
+
+- The connection pool is now lazy.  Connections are created as needed
+  and retained until the pool is shut down. [drbrain, #52]
+
+1.2.0
+-----
+
+- Add `with(options)` and `checkout(options)`. [mattcamuto]
+  Allows the caller to override the pool timeout.
+```ruby
+@pool.with(:timeout => 2) do |conn|
+end
+```
+
+1.1.0
+-----
+
+- New `#shutdown` method (simao)
+
+    This method accepts a block and calls the block for each
+    connection in the pool. After calling this method, trying to get a
+    connection from the pool raises `PoolShuttingDownError`.
+
 1.0.0
 -----
 

data/README.md

@@ -1,60 +1,69 @@
-connection_pool
-======================
+connection\_pool
+=================
+[![Build Status](https://github.com/mperham/connection_pool/actions/workflows/ci.yml/badge.svg)](https://github.com/mperham/connection_pool/actions/workflows/ci.yml)
 
 Generic connection pooling for Ruby.
 
-MongoDB has its own connection pool.  ActiveRecord has its own connection pool.  This is a generic connection pool that can be used with anything, e.g. Redis, Dalli and other Ruby network clients.
-
-
-Install
-------------
-
-    gem install connection_pool
-
-
-Notes
-------------
-
-- Connections are eager created when the pool is created.
-- There is no provision for repairing or checking the health of a
-  connection; connections should be self-repairing.  This is
-true of the dalli and redis clients.
-
+MongoDB has its own connection pool.
+ActiveRecord has its own connection pool.
+This is a generic connection pool that can be used with anything, e.g. Redis, Dalli and other Ruby network clients.
 
 Usage
-------------
+-----
 
 Create a pool of objects to share amongst the fibers or threads in your Ruby application:
 
 ``` ruby
-@memcached = ConnectionPool.new(:size => 5, :timeout => 5) { Dalli::Client.new }
+$memcached = ConnectionPool.new(size: 5, timeout: 5) { Dalli::Client.new }
 ```
 
 Then use the pool in your application:
 
 ``` ruby
-@memcached.with do |dalli|
-  dalli.get('some-count')
+$memcached.with do |conn|
+  conn.get('some-count')
 end
 ```
 
 If all the objects in the connection pool are in use, `with` will block
-until one becomes available.  If no object is available within `:timeout` seconds,
-`with` will raise a `Timeout::Error`.
+until one becomes available.
+If no object is available within `:timeout` seconds,
+`with` will raise a `ConnectionPool::TimeoutError` (a subclass of `Timeout::Error`).
 
-You can use `ConnectionPool::Wrapper` to wrap a single global connection, making
-it easier to port your connection code over time:
+You can also use `ConnectionPool#then` to support _both_ a
+connection pool and a raw client.
+
+```ruby
+# Compatible with a raw Redis::Client, and ConnectionPool Redis
+$redis.then { |r| r.set 'foo' 'bar' }
+```
+
+Optionally, you can specify a timeout override using the with-block semantics:
 
 ``` ruby
-$redis = ConnectionPool::Wrapper.new(:size => 5, :timeout => 3) { Redis.connect }
+$memcached.with(timeout: 2.0) do |conn|
+  conn.get('some-count')
+end
+```
+
+This will only modify the resource-get timeout for this particular
+invocation.
+This is useful if you want to fail-fast on certain non-critical
+sections when a resource is not available, or conversely if you are comfortable blocking longer on a particular resource.
+This is not implemented in the `ConnectionPool::Wrapper` class.
+
+## Migrating to a Connection Pool
+
+You can use `ConnectionPool::Wrapper` to wrap a single global connection, making it easier to migrate existing connection code over time:
+
+``` ruby
+$redis = ConnectionPool::Wrapper.new(size: 5, timeout: 3) { Redis.new }
 $redis.sadd('foo', 1)
 $redis.smembers('foo')
 ```
 
-The Wrapper uses `method_missing` to checkout a connection, run the
-requested method and then immediately check the connection back into the
-pool.  It's **not** high-performance so you'll want to port your
-performance sensitive code to use `with` as soon as possible.
+The wrapper uses `method_missing` to checkout a connection, run the requested method and then immediately check the connection back into the pool.
+It's **not** high-performance so you'll want to port your performance sensitive code to use `with` as soon as possible.
 
 ``` ruby
 $redis.with do |conn|
@@ -63,10 +72,117 @@ $redis.with do |conn|
 end
 ```
 
-Once you've ported your entire system to use `with`, you can simply
-remove ::Wrapper and use a simple, fast ConnectionPool.
+Once you've ported your entire system to use `with`, you can simply remove `Wrapper` and use the simpler and faster `ConnectionPool`.
+
+
+## Shutdown
+
+You can shut down a ConnectionPool instance once it should no longer be used.
+Further checkout attempts will immediately raise an error but existing checkouts will work.
+
+```ruby
+cp = ConnectionPool.new { Redis.new }
+cp.shutdown { |c| c.close }
+```
+
+Shutting down a connection pool will block until all connections are checked in and closed.
+**Note that shutting down is completely optional**; Ruby's garbage collector will reclaim unreferenced pools under normal circumstances.
+
+## Reload
+
+You can reload a ConnectionPool instance in the case it is desired to close all connections to the pool and, unlike `shutdown`, afterwards recreate connections so the pool may continue to be used.
+Reloading may be useful after forking the process.
+
+```ruby
+cp = ConnectionPool.new { Redis.new }
+cp.reload { |conn| conn.quit }
+cp.with { |conn| conn.get('some-count') }
+```
+
+Like `shutdown`, this will block until all connections are checked in and closed.
+
+## Reap
+
+You can reap idle connections in the ConnectionPool instance to close connections that were created but have not been used for a certain amount of time. This can be useful to run periodically in a separate thread especially if keeping the connection open is resource intensive.
+
+You can specify how many seconds the connections have to be idle for them to be reaped.
+Defaults to 60 seconds.
+
+```ruby
+cp = ConnectionPool.new { Redis.new }
+cp.reap(300) { |conn| conn.close } # Reaps connections that have been idle for 300 seconds (5 minutes).
+```
+
+### Reaper Thread
+
+You can start your own reaper thread to reap idle connections in the ConnectionPool instance on a regular interval.
+
+```ruby
+cp = ConnectionPool.new { Redis.new }
+
+# Start a reaper thread to reap connections that have been idle for 300 seconds (5 minutes).
+Thread.new do
+  loop do
+    cp.reap(300) { |conn| conn.close }
+    sleep 300
+  end
+end
+```
+
+## Discarding Connections
+
+You can discard connections in the ConnectionPool instance to remove connections that are broken and can't be restarted. 
+
+NOTE: the connection is not closed. It will just be removed from the pool so it won't be selected again.
+
+It can only be done inside the block passed to `with` or `with_timeout`.
+
+Takes an optional block that will be executed with the connection.
+
+```ruby
+   pool.with do |conn|
+     begin
+       conn.execute("SELECT 1")
+     rescue SomeConnectionError
+       pool.discard_current_connection  # remove the connection from the pool
+       raise
+     end
+   end
+```
+
+## Current State
+
+There are several methods that return information about a pool.
+
+```ruby
+cp = ConnectionPool.new(size: 10) { Redis.new }
+cp.size # => 10
+cp.available # => 10
+cp.idle # => 0
+
+cp.with do |conn|
+  cp.size # => 10
+  cp.available # => 9
+  cp.idle # => 0
+end
+
+cp.idle # => 1
+```
+
+Notes
+-----
+
+- Connections are lazily created as needed.
+- There is no provision for repairing or checking the health of a connection;
+  connections should be self-repairing. This is true of the Dalli and Redis
+  clients.
+- **WARNING**: Don't ever use `Timeout.timeout` in your Ruby code or you will see
+  occasional silent corruption and mysterious errors. The Timeout API is unsafe
+  and cannot be used correctly, ever. Use proper socket timeout options as
+  exposed by Net::HTTP, Redis, Dalli, etc.
+
 
 Author
---------------
+------
 
-Mike Perham, [@mperham](https://twitter.com/mperham), <http://mikeperham.com>
+Mike Perham, [@getajobmike](https://twitter.com/getajobmike), <https://www.mikeperham.com>

data/connection_pool.gemspec

@@ -1,17 +1,24 @@
-# -*- encoding: utf-8 -*-
 require "./lib/connection_pool/version"
 
 Gem::Specification.new do |s|
-  s.name        = "connection_pool"
-  s.version     = ConnectionPool::VERSION
-  s.platform    = Gem::Platform::RUBY
-  s.authors     = ["Mike Perham"]
-  s.email       = ["mperham@gmail.com"]
-  s.homepage    = "https://github.com/mperham/connection_pool"
-  s.description = s.summary = %q{Generic connection pool for Ruby}
+  s.name = "connection_pool"
+  s.version = ConnectionPool::VERSION
+  s.platform = Gem::Platform::RUBY
+  s.authors = ["Mike Perham", "Damian Janowski"]
+  s.email = ["mperham@gmail.com", "damian@educabilia.com"]
+  s.homepage = "https://github.com/mperham/connection_pool"
+  s.description = s.summary = "Generic connection pool for Ruby"
 
-  s.files         = `git ls-files`.split("\n")
-  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
-  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
+  s.files = ["Changes.md", "LICENSE", "README.md", "connection_pool.gemspec",
+    "lib/connection_pool.rb", "lib/connection_pool/timed_stack.rb",
+    "lib/connection_pool/version.rb", "lib/connection_pool/wrapper.rb"]
+  s.executables = []
   s.require_paths = ["lib"]
+  s.license = "MIT"
+  s.add_development_dependency "bundler"
+  s.add_development_dependency "minitest", ">= 5.0.0"
+  s.add_development_dependency "rake"
+  s.required_ruby_version = ">= 2.5.0"
+
+  s.metadata = {"changelog_uri" => "https://github.com/mperham/connection_pool/blob/main/Changes.md", "rubygems_mfa_required" => "true"}
 end

data/lib/connection_pool/timed_stack.rb

@@ -1,42 +1,237 @@
-require 'thread'
-require 'timeout'
-
+##
+# The TimedStack manages a pool of homogeneous connections (or any resource
+# you wish to manage). Connections are created lazily up to a given maximum
+# number.
+#
+# Examples:
+#
+#    ts = TimedStack.new(1) { MyConnection.new }
+#
+#    # fetch a connection
+#    conn = ts.pop
+#
+#    # return a connection
+#    ts.push conn
+#
+#    conn = ts.pop
+#    ts.pop timeout: 5
+#    #=> raises ConnectionPool::TimeoutError after 5 seconds
 class ConnectionPool::TimedStack
-  def initialize(size = 0)
-    @que = Array.new(size) { yield }
-    @mutex = Mutex.new
-    @resource = ConditionVariable.new
+  attr_reader :max
+
+  ##
+  # Creates a new pool with +size+ connections that are created from the given
+  # +block+.
+  def initialize(size = 0, &block)
+    @create_block = block
+    @created = 0
+    @que = []
+    @max = size
+    @mutex = Thread::Mutex.new
+    @resource = Thread::ConditionVariable.new
+    @shutdown_block = nil
   end
 
-  def push(obj)
+  ##
+  # Returns +obj+ to the stack. +options+ is ignored in TimedStack but may be
+  # used by subclasses that extend TimedStack.
+  def push(obj, options = {})
     @mutex.synchronize do
-      @que.push obj
+      if @shutdown_block
+        @created -= 1 unless @created == 0
+        @shutdown_block.call(obj)
+      else
+        store_connection obj, options
+      end
+
       @resource.broadcast
     end
   end
   alias_method :<<, :push
 
-  def pop(timeout=0.5)
-    deadline = Time.now + timeout
+  ##
+  # Retrieves a connection from the stack. If a connection is available it is
+  # immediately returned. If no connection is available within the given
+  # timeout a ConnectionPool::TimeoutError is raised.
+  #
+  # @option options [Float] :timeout (0.5) Wait this many seconds for an available entry
+  # @option options [Class] :exception (ConnectionPool::TimeoutError) Exception class to raise
+  #   if an entry was not available within the timeout period. Use `exception: false` to return nil.
+  #
+  # The +timeout+ argument will be removed in 3.0.
+  # Other options may be used by subclasses that extend TimedStack.
+  def pop(timeout = 0.5, options = {})
+    options, timeout = timeout, 0.5 if Hash === timeout
+    timeout = options.fetch :timeout, timeout
+
+    deadline = current_time + timeout
     @mutex.synchronize do
       loop do
-        return @que.pop unless @que.empty?
-        to_wait = deadline - Time.now
-        raise Timeout::Error, "Waited #{timeout} sec" if to_wait <= 0
+        raise ConnectionPool::PoolShuttingDownError if @shutdown_block
+        if (conn = try_fetch_connection(options))
+          return conn
+        end
+
+        connection = try_create(options)
+        return connection if connection
+
+        to_wait = deadline - current_time
+        if to_wait <= 0
+          exc = options.fetch(:exception, ConnectionPool::TimeoutError)
+          if exc
+            raise ConnectionPool::TimeoutError, "Waited #{timeout} sec, #{length}/#{@max} available"
+          else
+            return nil
+          end
+        end
         @resource.wait(@mutex, to_wait)
       end
     end
   end
 
-  def empty?
-    @que.empty?
+  ##
+  # Shuts down the TimedStack by passing each connection to +block+ and then
+  # removing it from the pool. Attempting to checkout a connection after
+  # shutdown will raise +ConnectionPool::PoolShuttingDownError+ unless
+  # +:reload+ is +true+.
+  def shutdown(reload: false, &block)
+    raise ArgumentError, "shutdown must receive a block" unless block
+
+    @mutex.synchronize do
+      @shutdown_block = block
+      @resource.broadcast
+
+      shutdown_connections
+      @shutdown_block = nil if reload
+    end
+  end
+
+  ##
+  # Reaps connections that were checked in more than +idle_seconds+ ago.
+  def reap(idle_seconds, &block)
+    raise ArgumentError, "reap must receive a block" unless block
+    raise ArgumentError, "idle_seconds must be a number" unless idle_seconds.is_a?(Numeric)
+    raise ConnectionPool::PoolShuttingDownError if @shutdown_block
+
+    idle.times do
+      conn =
+        @mutex.synchronize do
+          raise ConnectionPool::PoolShuttingDownError if @shutdown_block
+
+          reserve_idle_connection(idle_seconds)
+        end
+      break unless conn
+
+      block.call(conn)
+    end
   end
 
-  def clear
-    @que.clear
+  ##
+  # Returns +true+ if there are no available connections.
+  def empty?
+    (@created - @que.length) >= @max
   end
 
+  ##
+  # The number of connections available on the stack.
   def length
+    @max - @created + @que.length
+  end
+
+  ##
+  # The number of connections created and available on the stack.
+  def idle
     @que.length
   end
+
+  ##
+  # Reduce the created count
+  def decrement_created
+    @created -= 1 unless @created == 0
+  end
+
+  private
+
+  def current_time
+    Process.clock_gettime(Process::CLOCK_MONOTONIC)
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must returns a connection from the stack if one exists. Allows
+  # subclasses with expensive match/search algorithms to avoid double-handling
+  # their stack.
+  def try_fetch_connection(options = nil)
+    connection_stored?(options) && fetch_connection(options)
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must returns true if a connection is available on the stack.
+  def connection_stored?(options = nil)
+    !@que.empty?
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must return a connection from the stack.
+  def fetch_connection(options = nil)
+    @que.pop&.first
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must shut down all connections on the stack.
+  def shutdown_connections(options = nil)
+    while (conn = try_fetch_connection(options))
+      @created -= 1 unless @created == 0
+      @shutdown_block.call(conn)
+    end
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method returns the oldest idle connection if it has been idle for more than idle_seconds.
+  # This requires that the stack is kept in order of checked in time (oldest first).
+  def reserve_idle_connection(idle_seconds)
+    return unless idle_connections?(idle_seconds)
+
+    @created -= 1 unless @created == 0
+
+    @que.shift.first
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # Returns true if the first connection in the stack has been idle for more than idle_seconds
+  def idle_connections?(idle_seconds)
+    connection_stored? && (current_time - @que.first.last > idle_seconds)
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must return +obj+ to the stack.
+  def store_connection(obj, options = nil)
+    @que.push [obj, current_time]
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must create a connection if and only if the total number of
+  # connections allowed has not been met.
+  def try_create(options = nil)
+    unless @created == @max
+      object = @create_block.call
+      @created += 1
+      object
+    end
+  end
 end

data/lib/connection_pool/version.rb

@@ -1,3 +1,3 @@
 class ConnectionPool
-  VERSION = "1.0.0"
+  VERSION = "2.5.5"
 end

data/lib/connection_pool/wrapper.rb

@@ -0,0 +1,56 @@
+class ConnectionPool
+  class Wrapper < ::BasicObject
+    METHODS = [:with, :pool_shutdown, :wrapped_pool]
+
+    def initialize(options = {}, &block)
+      @pool = options.fetch(:pool) { ::ConnectionPool.new(options, &block) }
+    end
+
+    def wrapped_pool
+      @pool
+    end
+
+    def with(&block)
+      @pool.with(&block)
+    end
+
+    def pool_shutdown(&block)
+      @pool.shutdown(&block)
+    end
+
+    def pool_size
+      @pool.size
+    end
+
+    def pool_available
+      @pool.available
+    end
+
+    def respond_to?(id, *args)
+      METHODS.include?(id) || with { |c| c.respond_to?(id, *args) }
+    end
+
+    # rubocop:disable Style/MissingRespondToMissing
+    if ::RUBY_VERSION >= "3.0.0"
+      def method_missing(name, *args, **kwargs, &block)
+        with do |connection|
+          connection.send(name, *args, **kwargs, &block)
+        end
+      end
+    elsif ::RUBY_VERSION >= "2.7.0"
+      ruby2_keywords def method_missing(name, *args, &block)
+        with do |connection|
+          connection.send(name, *args, &block)
+        end
+      end
+    else
+      def method_missing(name, *args, &block)
+        with do |connection|
+          connection.send(name, *args, &block)
+        end
+      end
+    end
+    # rubocop:enable Style/MethodMissingSuper
+    # rubocop:enable Style/MissingRespondToMissing
+  end
+end

data/lib/connection_pool.rb

@@ -1,16 +1,30 @@
-require_relative 'connection_pool/version'
+require "timeout"
+require_relative "connection_pool/version"
 
-# Generic connection pool class for e.g. sharing a limited number of network connections
-# among many threads.  Note: Connections are eager created.
+class ConnectionPool
+  class Error < ::RuntimeError; end
+
+  class PoolShuttingDownError < ::ConnectionPool::Error; end
+
+  class TimeoutError < ::Timeout::Error; end
+end
+
+# Generic connection pool class for sharing a limited number of objects or network connections
+# among many threads.  Note: pool elements are lazily created.
 #
 # Example usage with block (faster):
 #
 #    @pool = ConnectionPool.new { Redis.new }
-#
 #    @pool.with do |redis|
 #      redis.lpop('my-list') if redis.llen('my-list') > 0
 #    end
 #
+# Using optional timeout override (for that single invocation)
+#
+#    @pool.with(timeout: 2.0) do |redis|
+#      redis.lpop('my-list') if redis.llen('my-list') > 0
+#    end
+#
 # Example usage replacing an existing connection (slower):
 #
 #    $redis = ConnectionPool.wrap { Redis.new }
@@ -22,80 +36,198 @@ require_relative 'connection_pool/version'
 # Accepts the following options:
 # - :size - number of connections to pool, defaults to 5
 # - :timeout - amount of time to wait for a connection if none currently available, defaults to 5 seconds
+# - :auto_reload_after_fork - automatically drop all connections after fork, defaults to true
 #
 class ConnectionPool
-  DEFAULTS = {size: 5, timeout: 5}
+  DEFAULTS = {size: 5, timeout: 5, auto_reload_after_fork: true}.freeze
 
   def self.wrap(options, &block)
     Wrapper.new(options, &block)
   end
 
+  if Process.respond_to?(:fork)
+    INSTANCES = ObjectSpace::WeakMap.new
+    private_constant :INSTANCES
+
+    def self.after_fork
+      INSTANCES.values.each do |pool|
+        next unless pool.auto_reload_after_fork
+
+        # We're on after fork, so we know all other threads are dead.
+        # All we need to do is to ensure the main thread doesn't have a
+        # checked out connection
+        pool.checkin(force: true)
+        pool.reload do |connection|
+          # Unfortunately we don't know what method to call to close the connection,
+          # so we try the most common one.
+          connection.close if connection.respond_to?(:close)
+        end
+      end
+      nil
+    end
+
+    if ::Process.respond_to?(:_fork) # MRI 3.1+
+      module ForkTracker
+        def _fork
+          pid = super
+          if pid == 0
+            ConnectionPool.after_fork
+          end
+          pid
+        end
+      end
+      Process.singleton_class.prepend(ForkTracker)
+    end
+  else
+    INSTANCES = nil
+    private_constant :INSTANCES
+
+    def self.after_fork
+      # noop
+    end
+  end
+
   def initialize(options = {}, &block)
-    raise ArgumentError, 'Connection pool requires a block' unless block
+    raise ArgumentError, "Connection pool requires a block" unless block
 
     options = DEFAULTS.merge(options)
 
-    @size = options.fetch(:size)
+    @size = Integer(options.fetch(:size))
     @timeout = options.fetch(:timeout)
+    @auto_reload_after_fork = options.fetch(:auto_reload_after_fork)
 
     @available = TimedStack.new(@size, &block)
-    @key = :"current-#{@available.object_id}"
+    @key = :"pool-#{@available.object_id}"
+    @key_count = :"pool-#{@available.object_id}-count"
+    @discard_key = :"pool-#{@available.object_id}-discard"
+    INSTANCES[self] = self if @auto_reload_after_fork && INSTANCES
   end
 
-  def with
-    conn = checkout
-    begin
-      yield conn
-    ensure
-      checkin
+  def with(options = {})
+    # We need to manage exception handling manually here in order
+    # to work correctly with `Timeout.timeout` and `Thread#raise`.
+    # Otherwise an interrupted Thread can leak connections.
+    Thread.handle_interrupt(Exception => :never) do
+      conn = checkout(options)
+      begin
+        Thread.handle_interrupt(Exception => :immediate) do
+          yield conn
+        end
+      ensure
+        checkin
+      end
     end
   end
+  alias_method :then, :with
+
+  ##
+  # Marks the current thread's checked-out connection for discard.
+  #
+  # When a connection is marked for discard, it will not be returned to the pool
+  # when checked in. Instead, the connection will be discarded.
+  # This is useful when a connection has become invalid or corrupted
+  # and should not be reused.
+  #
+  # Takes an optional block that will be called with the connection to be discarded.
+  # The block should perform any necessary clean-up on the connection.
+  #
+  # @yield [conn]
+  # @yieldparam conn [Object] The connection to be discarded.
+  # @yieldreturn [void]
+  #
+  #
+  # Note: This only affects the connection currently checked out by the calling thread.
+  # The connection will be discarded when +checkin+ is called.
+  #
+  # @return [void]
+  #
+  # @example
+  #   pool.with do |conn|
+  #     begin
+  #       conn.execute("SELECT 1")
+  #     rescue SomeConnectionError
+  #       pool.discard_current_connection  # Mark connection as bad
+  #       raise
+  #     end
+  #   end
+  def discard_current_connection(&block)
+    ::Thread.current[@discard_key] = block || proc { |conn| conn }
+  end
 
-  def checkout
-    stack = ::Thread.current[@key] ||= []
-
-    if stack.empty?
-      conn = @available.pop(@timeout)
+  def checkout(options = {})
+    if ::Thread.current[@key]
+      ::Thread.current[@key_count] += 1
+      ::Thread.current[@key]
     else
-      conn = stack.last
+      ::Thread.current[@key_count] = 1
+      ::Thread.current[@key] = @available.pop(options[:timeout] || @timeout, options)
     end
-
-    stack.push conn
-    conn
   end
 
-  def checkin
-    stack = ::Thread.current[@key]
-    conn = stack.pop
-    if stack.empty?
-      @available << conn
+  def checkin(force: false)
+    if ::Thread.current[@key]
+      if ::Thread.current[@key_count] == 1 || force
+        if ::Thread.current[@discard_key]
+          begin
+            @available.decrement_created
+            ::Thread.current[@discard_key].call(::Thread.current[@key])
+          rescue
+            nil
+          ensure
+            ::Thread.current[@discard_key] = nil
+          end
+        else
+          @available.push(::Thread.current[@key])
+        end
+        ::Thread.current[@key] = nil
+        ::Thread.current[@key_count] = nil
+      else
+        ::Thread.current[@key_count] -= 1
+      end
+    elsif !force
+      raise ConnectionPool::Error, "no connections are checked out"
     end
+
     nil
   end
 
-  class Wrapper < ::BasicObject
-    METHODS = [:with]
+  ##
+  # Shuts down the ConnectionPool by passing each connection to +block+ and
+  # then removing it from the pool. Attempting to checkout a connection after
+  # shutdown will raise +ConnectionPool::PoolShuttingDownError+.
+  def shutdown(&block)
+    @available.shutdown(&block)
+  end
 
-    def initialize(options = {}, &block)
-      @pool = ::ConnectionPool.new(options, &block)
-    end
+  ##
+  # Reloads the ConnectionPool by passing each connection to +block+ and then
+  # removing it the pool. Subsequent checkouts will create new connections as
+  # needed.
+  def reload(&block)
+    @available.shutdown(reload: true, &block)
+  end
 
-    def with
-      yield @pool.checkout
-    ensure
-      @pool.checkin
-    end
+  ## Reaps idle connections that have been idle for over +idle_seconds+.
+  # +idle_seconds+ defaults to 60.
+  def reap(idle_seconds = 60, &block)
+    @available.reap(idle_seconds, &block)
+  end
 
-    def respond_to?(id, *args)
-      METHODS.include?(id) || @pool.with { |c| c.respond_to?(id, *args) }
-    end
+  # Size of this connection pool
+  attr_reader :size
+  # Automatically drop all connections after fork
+  attr_reader :auto_reload_after_fork
 
-    def method_missing(name, *args, &block)
-      @pool.with do |connection|
-        connection.send(name, *args, &block)
-      end
-    end
+  # Number of pool entries available for checkout at this instant.
+  def available
+    @available.length
+  end
+
+  # Number of pool entries created and idle in the pool.
+  def idle
+    @available.idle
   end
 end
 
-require_relative 'connection_pool/timed_stack'
+require_relative "connection_pool/timed_stack"
+require_relative "connection_pool/wrapper"

metadata

@@ -1,59 +1,94 @@
 --- !ruby/object:Gem::Specification
 name: connection_pool
 version: !ruby/object:Gem::Version
-  version: 1.0.0
-  prerelease: 
+  version: 2.5.5
 platform: ruby
 authors:
 - Mike Perham
-autorequire: 
+- Damian Janowski
 bindir: bin
 cert_chain: []
-date: 2012-12-19 00:00:00.000000000 Z
-dependencies: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 5.0.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 5.0.0
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Generic connection pool for Ruby
 email:
 - mperham@gmail.com
+- damian@educabilia.com
 executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- .gitignore
 - Changes.md
-- Gemfile
 - LICENSE
 - README.md
-- Rakefile
 - connection_pool.gemspec
 - lib/connection_pool.rb
 - lib/connection_pool/timed_stack.rb
 - lib/connection_pool/version.rb
-- test/helper.rb
-- test/test_connection_pool.rb
+- lib/connection_pool/wrapper.rb
 homepage: https://github.com/mperham/connection_pool
-licenses: []
-post_install_message: 
+licenses:
+- MIT
+metadata:
+  changelog_uri: https://github.com/mperham/connection_pool/blob/main/Changes.md
+  rubygems_mfa_required: 'true'
 rdoc_options: []
 require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
-  none: false
   requirements:
-  - - ! '>='
+  - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 2.5.0
 required_rubygems_version: !ruby/object:Gem::Requirement
-  none: false
   requirements:
-  - - ! '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 1.8.24
-signing_key: 
-specification_version: 3
+rubygems_version: 3.6.9
+specification_version: 4
 summary: Generic connection pool for Ruby
-test_files:
-- test/helper.rb
-- test/test_connection_pool.rb
+test_files: []

data/.gitignore

@@ -1,4 +0,0 @@
-*.gem
-.bundle
-Gemfile.lock
-pkg/*

data/Gemfile

@@ -1,7 +0,0 @@
-source "http://rubygems.org"
-
-# Specify your gem's dependencies in connection_pool.gemspec
-gemspec
-
-gem 'rake'
-gem 'minitest'

data/Rakefile

@@ -1,14 +0,0 @@
-begin
-  require 'bundler'
-  Bundler::GemHelper.install_tasks
-rescue LoadError
-end
-
-require 'rake/testtask'
-Rake::TestTask.new(:test) do |test|
-  test.libs << 'test'
-  test.warning = true
-  test.pattern = 'test/**/test_*.rb'
-end
-
-task :default => :test

data/test/helper.rb

@@ -1,19 +0,0 @@
-require 'rubygems'
-require 'minitest/pride'
-require 'minitest/autorun'
-
-puts RUBY_DESCRIPTION
-
-class MiniTest::Unit::TestCase
-
-  def async_test(time=0.5)
-    q = TimedQueue.new
-    yield Proc.new { q << nil }
-    q.timed_pop(time)
-  end
-
-end
-
-$VERBOSE = 1
-
-require_relative '../lib/connection_pool'

data/test/test_connection_pool.rb

@@ -1,150 +0,0 @@
-Thread.abort_on_exception = true
-require 'helper'
-
-class TestConnectionPool < MiniTest::Unit::TestCase
-
-  class NetworkConnection
-    def initialize
-      @x = 0
-    end
-
-    def do_something
-      @x += 1
-      sleep 0.05
-      @x
-    end
-
-    def fast
-      @x += 1
-    end
-
-    def do_something_with_block
-      @x += yield
-      sleep 0.05
-      @x
-    end
-
-    def respond_to?(method_id, *args)
-      method_id == :do_magic || super(method_id, *args)
-    end
-  end
-
-  def test_basic_multithreaded_usage
-    pool = ConnectionPool.new(:size => 5) { NetworkConnection.new }
-    threads = []
-    15.times do
-      threads << Thread.new do
-        pool.with do |net|
-          net.do_something
-        end
-      end
-    end
-
-    a = Time.now
-    result = threads.map(&:value)
-    b = Time.now
-    assert_operator((b - a), :>, 0.125)
-    assert_equal([1,2,3].cycle(5).sort, result.sort)
-  end
-
-  def test_timeout
-    pool = ConnectionPool.new(:timeout => 0.05, :size => 1) { NetworkConnection.new }
-    Thread.new do
-      pool.with do |net|
-        net.do_something
-        sleep 0.1
-      end
-    end
-    sleep 0.05
-    assert_raises Timeout::Error do
-      pool.with { |net| net.do_something }
-    end
-
-    sleep 0.05
-    pool.with do |conn|
-      refute_nil conn
-    end
-  end
-
-  def test_passthru
-    pool = ConnectionPool.wrap(:timeout => 0.1, :size => 1) { NetworkConnection.new }
-    assert_equal 1, pool.do_something
-    assert_equal 2, pool.do_something
-    assert_equal 5, pool.do_something_with_block { 3 }
-    assert_equal 6, pool.with { |net| net.fast }
-  end
-
-  def test_passthru_respond_to
-    pool = ConnectionPool.wrap(:timeout => 0.1, :size => 1) { NetworkConnection.new }
-    assert pool.respond_to?(:with)
-    assert pool.respond_to?(:do_something)
-    assert pool.respond_to?(:do_magic)
-    refute pool.respond_to?(:do_lots_of_magic)
-  end
-
-  def test_return_value
-    pool = ConnectionPool.new(:timeout => 0.1, :size => 1) { NetworkConnection.new }
-    result = pool.with do |net|
-      net.fast
-    end
-    assert_equal 1, result
-  end
-
-  def test_heavy_threading
-    pool = ConnectionPool.new(:timeout => 0.5, :size => 3) { NetworkConnection.new }
-    20.times do
-      Thread.new do
-        pool.with do |net|
-          sleep 0.05
-        end
-      end
-    end
-    sleep 0.5
-  end
-
-  def test_reuses_objects_when_pool_not_saturated
-    pool = ConnectionPool.new(:size => 5) { NetworkConnection.new }
-
-    ids = 10.times.map do
-      pool.with { |c| c.object_id }
-    end
-
-    assert_equal 1, ids.uniq.size
-  end
-
-  class Recorder
-    def initialize
-      @calls = []
-    end
-
-    attr_reader :calls
-
-    def do_work(label)
-      @calls << label
-    end
-  end
-
-  def test_nested_checkout
-    recorder = Recorder.new
-    pool = ConnectionPool.new(:size => 1) { recorder }
-    pool.with do |r_outer|
-      @other = Thread.new do |t|
-        pool.with do |r_other|
-          r_other.do_work('other')
-        end
-      end
-
-      pool.with do |r_inner|
-        r_inner.do_work('inner')
-      end
-
-      sleep 0.1
-
-      r_outer.do_work('outer')
-    end
-
-    @other.join
-
-    assert_equal ['inner', 'outer', 'other'], recorder.calls
-  end
-end