healthy_pools 2.2.3

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 7920f22d2bf5c1bb58d00ac099f15d2c074587e8
+  data.tar.gz: b090b648aa7186b23387ac584d4851bf7ee3d947
+SHA512:
+  metadata.gz: 27e4679398276cc5f2fe3086a61cdaf487f7c90b3d1a3b1432ae5c3f88587f8eaa5164613c5880ea6a390d5c55fe11917d6024b2f13cd4a83e522cfe8f0b4819
+  data.tar.gz: df8a0a87bed874e383c1516218e0e1678610e2d87e805952bb7f4a1222a498cd9d75e9b113fc2d1286de22debf5732573902fa5e48b9df78d0cc91f23776c34c

data/.gitignore

@@ -0,0 +1,4 @@
+*.gem
+.bundle
+Gemfile.lock
+pkg/*

data/.travis.yml

@@ -0,0 +1,14 @@
+---
+sudo: false
+cache: bundler
+language: ruby
+rvm:
+  - 2.0.0
+  - 2.1.6
+  - 2.2.2
+  - 2.3.1
+  - 2.4.1
+  - jruby
+notifications:
+  email:
+    - marcojoemontagna@gmail.com

data/Changes.md

@@ -0,0 +1,127 @@
+healthy\_pools changelog
+---------------------------
+
+HEAD
+------
+
+2.2.3
+------
+- Fork connection_pool to new name healthy_pools.
+- Add support for checking connection health.
+- Add pool `size` and `available` accessors for metrics and monitoring
+  purposes [#97, robholland]
+
+2.2.1
+------
+
+- Allow CP::Wrapper to use an existing pool [#87, etiennebarrie]
+- Use monotonic time for more accurate timeouts [#84, jdantonio]
+
+2.2.0
+------
+
+- Rollback `Timeout` handling introduced in 2.1.1 and 2.1.2.  It seems
+  impossible to safely work around the issue. Please never, ever use
+  `Timeout.timeout` in your code or you will see rare but mysterious bugs. [#75]
+
+2.1.3
+------
+
+- Don't increment created count until connection is successfully
+  created. [mylesmegyesi, #73]
+
+2.1.2
+------
+
+- The connection\_pool will now close any connections which respond to
+  `close` (Dalli) or `disconnect!` (Redis).  This ensures discarded connections
+  from the fix in 2.1.1 are torn down ASAP and don't linger open.
+
+
+2.1.1
+------
+
+- Work around a subtle race condition with code which uses `Timeout.timeout` and
+  checks out a connection within the timeout block.  This might cause
+  connections to get into a bad state and raise very odd errors. [tamird, #67]
+
+
+2.1.0
+------
+
+- Refactoring to better support connection pool subclasses [drbrain,
+  #55]
+- `with` should return value of the last expression [#59]
+
+
+2.0.0
+-----
+
+- The connection pool is now lazy.  Connections are created as needed
+  and retained until the pool is shut down. [drbrain, #52]
+
+1.2.0
+-----
+
+- Add `with(options)` and `checkout(options)`. [mattcamuto]
+  Allows the caller to override the pool timeout.
+```ruby
+@pool.with(:timeout => 2) do |conn|
+end
+```
+
+1.1.0
+-----
+
+- New `#shutdown` method (simao)
+
+    This method accepts a block and calls the block for each
+    connection in the pool. After calling this method, trying to get a
+    connection from the pool raises `PoolShuttingDownError`.
+
+1.0.0
+-----
+
+- `#with_connection` is now gone in favor of `#with`.
+
+- We no longer pollute the top level namespace with our internal
+`TimedStack` class.
+
+0.9.3
+--------
+
+- `#with_connection` is now deprecated in favor of `#with`.
+
+    A warning will be issued in the 0.9 series and the method will be
+    removed in 1.0.
+
+- We now reuse objects when possible.
+
+    This means that under no contention, the same object will be checked
+    out from the pool after subsequent calls to `ConnectionPool#with`.
+
+    This change should have no impact on end user performance. If
+    anything, it should be an improvement, depending on what objects you
+    are pooling.
+
+0.9.2
+--------
+
+- Fix reentrant checkout leading to early checkin.
+
+0.9.1
+--------
+
+- Fix invalid superclass in version.rb
+
+0.9.0
+--------
+
+- Move method\_missing magic into ConnectionPool::Wrapper (djanowski)
+- Remove BasicObject superclass (djanowski)
+
+0.1.0
+--------
+
+- More precise timeouts and better error message
+- ConnectionPool now subclasses BasicObject so `method_missing` is more effective.

data/Gemfile

@@ -0,0 +1,3 @@
+source "https://rubygems.org"
+
+gemspec(development_group: :runtime)

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2011 Mike Perham
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,114 @@
+connection\_pool
+=================
+[![Build Status](https://travis-ci.org/marcomontagna/healthy_pools.svg)](https://travis-ci.org/marcomontagna/healthy_pools)
+
+Generic connection pooling for Ruby.
+
+MongoDB has its own connection pool.  ActiveRecord has its own connection pool.
+This is a generic connection pool that can be used with anything, e.g. Redis,
+Dalli and other Ruby network clients.
+
+This is a fork of [connection_pool](https://github.com/mperham/connection_pool) with 
+support for checking connection health.
+
+
+Usage
+-----
+
+Create a pool of objects to share amongst the fibers or threads in your Ruby
+application:
+
+``` ruby
+$memcached = ConnectionPool.new(size: 5, timeout: 5) { Dalli::Client.new }
+```
+
+Then use the pool in your application:
+
+``` ruby
+$memcached.with do |conn|
+  conn.get('some-count')
+end
+```
+
+If all the objects in the connection pool are in use, `with` will block
+until one becomes available.  If no object is available within `:timeout` seconds,
+`with` will raise a `Timeout::Error`.
+
+Optionally, you can specify a timeout override using the with-block semantics:
+
+``` ruby
+$memcached.with(timeout: 2.0) do |conn|
+  conn.get('some-count')
+end
+```
+
+This will only modify the resource-get timeout for this particular
+invocation. This is useful if you want to fail-fast on certain non critical
+sections when a resource is not available, or conversely if you are comfortable
+blocking longer on a particular resource. This is not implemented in the below
+`ConnectionPool::Wrapper` class.
+
+## Migrating to a Connection Pool
+
+You can use `ConnectionPool::Wrapper` to wrap a single global connection,
+making it easier to migrate existing connection code over time:
+
+``` ruby
+$redis = ConnectionPool::Wrapper.new(size: 5, timeout: 3) { Redis.connect }
+$redis.sadd('foo', 1)
+$redis.smembers('foo')
+```
+
+The wrapper uses `method_missing` to checkout a connection, run the requested
+method and then immediately check the connection back into the pool.  It's
+**not** high-performance so you'll want to port your performance sensitive code
+to use `with` as soon as possible.
+
+``` ruby
+$redis.with do |conn|
+  conn.sadd('foo', 1)
+  conn.smembers('foo')
+end
+```
+
+Once you've ported your entire system to use `with`, you can simply remove
+`Wrapper` and use the simpler and faster `ConnectionPool`.
+
+
+## Shutdown
+
+You can shut down a ConnectionPool instance once it should no longer be used.
+Further checkout attempts will immediately raise an error but existing checkouts
+will work.
+
+```ruby
+cp = ConnectionPool.new { Redis.new }
+cp.shutdown { |conn| conn.quit }
+```
+
+Shutting down a connection pool will block until all connections are checked in and closed.
+**Note that shutting down is completely optional**; Ruby's garbage collector will reclaim
+unreferenced pools under normal circumstances.
+
+## Managing Health of Connections
+
+You can pass a health check lambda or Proc to the connection pool which is called 
+before a connection is returned to clients to verify the connection is still good.
+If the health check returns false or throws an exception the connection is removed
+from the pool and a new one is created.
+
+
+```ruby
+cp = ConnectionPool.new(health_check: lambda {|conn| conn.exec('select 1')}) { PG.connect }
+```
+
+
+Notes
+-----
+
+- Connections are lazily created as needed.
+- **WARNING**: Don't ever use `Timeout.timeout` in your Ruby code or you will see
+  occasional silent corruption and mysterious errors.  The Timeout API is unsafe
+  and cannot be used correctly, ever.  Use proper socket timeout options as
+  exposed by Net::HTTP, Redis, Dalli, etc.
+

data/Rakefile

@@ -0,0 +1,6 @@
+require 'bundler/gem_tasks'
+
+require 'rake/testtask'
+Rake::TestTask.new
+
+task :default => :test

data/healthy_pools.gemspec

@@ -0,0 +1,21 @@
+# -*- encoding: utf-8 -*-
+require "./lib/connection_pool/version"
+
+Gem::Specification.new do |s|
+  s.name        = "healthy_pools"
+  s.version     = ConnectionPool::VERSION
+  s.platform    = Gem::Platform::RUBY
+  s.authors     = ["Marco Montagna"]
+  s.email       = ["marcojoemontagna@gmail.com"]
+  s.homepage    = "https://github.com/mmontagna/healthy_pools"
+  s.description = s.summary = %q{Generic connection pool for Ruby}
+
+  s.files         = `git ls-files`.split("\n")
+  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
+  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
+  s.require_paths = ["lib"]
+  s.license = "MIT"
+  s.add_development_dependency 'bundler'
+  s.add_development_dependency 'minitest', '>= 5.0.0'
+  s.add_development_dependency 'rake'
+end

data/lib/connection_pool/monotonic_time.rb

@@ -0,0 +1,66 @@
+# Global monotonic clock from Concurrent Ruby 1.0.
+# Copyright (c) Jerry D'Antonio -- released under the MIT license.
+# Slightly modified; used with permission.
+# https://github.com/ruby-concurrency/concurrent-ruby
+
+require 'thread'
+
+class ConnectionPool
+
+  class_definition = Class.new do
+
+    if defined?(Process::CLOCK_MONOTONIC)
+
+      # @!visibility private
+      def get_time
+        Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      end
+
+    elsif defined?(RUBY_ENGINE) && RUBY_ENGINE == 'jruby'
+
+      # @!visibility private
+      def get_time
+        java.lang.System.nanoTime() / 1_000_000_000.0
+      end
+
+    else
+
+      # @!visibility private
+      def initialize
+        @mutex = Mutex.new
+        @last_time = Time.now.to_f
+      end
+
+      # @!visibility private
+      def get_time
+        @mutex.synchronize do
+          now = Time.now.to_f
+          if @last_time < now
+            @last_time = now
+          else # clock has moved back in time
+            @last_time += 0.000_001
+          end
+        end
+      end
+    end
+  end
+
+  ##
+  # Clock that cannot be set and represents monotonic time since
+  # some unspecified starting point.
+  #
+  # @!visibility private
+  GLOBAL_MONOTONIC_CLOCK = class_definition.new
+  private_constant :GLOBAL_MONOTONIC_CLOCK
+
+  class << self
+    ##
+    # Returns the current time a tracked by the application monotonic clock.
+    #
+    # @return [Float] The current monotonic time when `since` not given else
+    #   the elapsed monotonic time between `since` and the current time
+    def monotonic_time
+      GLOBAL_MONOTONIC_CLOCK.get_time
+    end
+  end
+end

data/lib/connection_pool/timed_stack.rb

@@ -0,0 +1,201 @@
+require 'thread'
+require 'timeout'
+require_relative 'monotonic_time'
+
+##
+# Raised when you attempt to retrieve a connection from a pool that has been
+# shut down.
+
+class ConnectionPool::PoolShuttingDownError < RuntimeError; end
+
+##
+# The TimedStack manages a pool of homogeneous connections (or any resource
+# you wish to manage).  Connections are created lazily up to a given maximum
+# number.
+
+# Examples:
+#
+#    ts = TimedStack.new(1) { MyConnection.new }
+#
+#    # fetch a connection
+#    conn = ts.pop
+#
+#    # return a connection
+#    ts.push conn
+#
+#    conn = ts.pop
+#    ts.pop timeout: 5
+#    #=> raises Timeout::Error after 5 seconds
+
+class ConnectionPool::TimedStack
+  attr_reader :max
+
+  ##
+  # Creates a new pool with +size+ connections that are created from the given
+  # +block+.
+
+  def initialize(size = 0, health_check: nil, &block)
+    @create_block = block
+    @health_check = health_check
+    @created = 0
+    @que = []
+    @max = size
+    @mutex = Mutex.new
+    @resource = ConditionVariable.new
+    @shutdown_block = nil
+  end
+
+  ##
+  # Removes +conn+ from the stack. This is useful if connections
+  # die or otherwise become stale.
+
+  def delete(conn)
+    if @mutex.owned?
+      @created -= 1
+    else
+      @mutex.synchronize do
+        @created -= 1
+      end
+    end
+  end
+
+  ##
+  # Returns +obj+ to the stack.  +options+ is ignored in TimedStack but may be
+  # used by subclasses that extend TimedStack.
+
+  def push(obj, options = {})
+    @mutex.synchronize do
+      if @shutdown_block
+        @shutdown_block.call(obj)
+      else
+        store_connection obj, options
+      end
+
+      @resource.broadcast
+    end
+  end
+  alias_method :<<, :push
+
+  ##
+  # Retrieves a connection from the stack.  If a connection is available it is
+  # immediately returned.  If no connection is available within the given
+  # timeout a Timeout::Error is raised.
+  #
+  # +:timeout+ is the only checked entry in +options+ and is preferred over
+  # the +timeout+ argument (which will be removed in a future release).  Other
+  # options may be used by subclasses that extend TimedStack.
+
+  def pop(timeout = 0.5, options = {})
+    options, timeout = timeout, 0.5 if Hash === timeout
+    timeout = options.fetch :timeout, timeout
+
+    deadline = ConnectionPool.monotonic_time + timeout
+    @mutex.synchronize do
+      loop do
+        raise ConnectionPool::PoolShuttingDownError if @shutdown_block
+        while connection_stored?(options)
+          conn = fetch_connection(options)
+          begin
+            if @health_check.nil? || @health_check.call(conn)
+              return conn
+            end
+          rescue
+          end
+          # Health check has failed, delete conn and retry.
+          delete(conn)
+        end
+
+        connection = try_create(options)
+        return connection if connection
+
+        to_wait = deadline - ConnectionPool.monotonic_time
+        raise Timeout::Error, "Waited #{timeout} sec" if to_wait <= 0
+        @resource.wait(@mutex, to_wait)
+      end
+    end
+  end
+
+  ##
+  # Shuts down the TimedStack which prevents connections from being checked
+  # out.  The +block+ is called once for each connection on the stack.
+
+  def shutdown(&block)
+    raise ArgumentError, "shutdown must receive a block" unless block_given?
+
+    @mutex.synchronize do
+      @shutdown_block = block
+      @resource.broadcast
+
+      shutdown_connections
+    end
+  end
+
+  ##
+  # Returns +true+ if there are no available connections.
+
+  def empty?
+    (@created - @que.length) >= @max
+  end
+
+  ##
+  # The number of connections available on the stack.
+
+  def length
+    @max - @created + @que.length
+  end
+
+  private
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must returns true if a connection is available on the stack.
+
+  def connection_stored?(options = nil)
+    !@que.empty?
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must return a connection from the stack.
+
+  def fetch_connection(options = nil)
+    @que.pop
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must shut down all connections on the stack.
+
+  def shutdown_connections(options = nil)
+    while connection_stored?(options)
+      conn = fetch_connection(options)
+      @shutdown_block.call(conn)
+    end
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must return +obj+ to the stack.
+
+  def store_connection(obj, options = nil)
+    @que.push obj
+  end
+
+  ##
+  # This is an extension point for TimedStack and is called with a mutex.
+  #
+  # This method must create a connection if and only if the total number of
+  # connections allowed has not been met.
+
+  def try_create(options = nil)
+    unless @created == @max
+      object = @create_block.call
+      @created += 1
+      object
+    end
+  end
+end

data/lib/connection_pool/version.rb

@@ -0,0 +1,3 @@
+class ConnectionPool
+  VERSION = "2.2.3"
+end