pond 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
---- 
-SHA1: 
-  metadata.gz: 60082a3f35a388cf665c609388ee4025e58006d1
-  data.tar.gz: 1e0ed085f1a3f3943c85ce1f00976f2d892c1e33
-SHA512: 
-  metadata.gz: 7e1b8250177dbd7b8ee56d1750debade92eabb7ad22b9b778d72f8465083d3c0071f1674713a0d44a06aa3d6df18ac549e7bfade50c3a24293a68168dbf0dde7
-  data.tar.gz: 24b0bd2e914f6932bdb4ea2e4c532543a3a7928e8c01d11326ed44b5254c261b1ee3032408c0f42092a5a8a6bd48e6d8406b6f1ddaee3dce5ddf4fd5202b2529
+---
+SHA1:
+  metadata.gz: c9254999a6eb11a5873947aa1fee8bc5ad326be3
+  data.tar.gz: fc080fdb56ada8713ffff8bbe2b82a535a4125c3
+SHA512:
+  metadata.gz: aecc96803e996c38b4168fa287cd3f4f39eb0d8b2d448b9a6c36d9fa63fcb2cba14e2fc06f4377238d7bf6632410a1ebff38e460d8200c975249b77b1b9f7eae
+  data.tar.gz: ded459d09b79d4017db0e878a0cbccc44c99c47998a6224ebdb34ec8bd91dddf15763cda57d435a9f79f299fb0f316c1b2ed66de5b7223c25fb2bae1d91638b5

data/CHANGELOG.md

@@ -1,3 +1,8 @@
+### 0.2.0 (2016-02-05)
+
+*   Add an option for a detach_if callable, which can contain logic to
+    determine whether to remove an object from the pool.
+
 ### 0.1.0 (2014-02-14)
 
 *   Initial release.

data/README.md

@@ -1,6 +1,11 @@
 # Pond
 
-Pond is a gem that offers thread-safe object pooling. It can wrap anything that is costly to instantiate, but is usually used for connections. It is intentionally very similar to the `connection_pool` gem, but is intended to be more efficient and flexible. It instantiates objects lazily by default, which is important for things with high overhead like Postgres connections. It can also be dynamically resized.
+Pond is a gem that offers thread-safe object pooling. It can wrap anything
+that is costly to instantiate, but is usually used for connections. It is
+intentionally very similar to the `connection_pool` gem, but is intended to be
+more efficient and flexible. It instantiates objects lazily by default, which
+is important for things with high overhead like Postgres connections. It can
+also be dynamically resized, and does not block on object instantiation.
 
 Also, it was pretty fun to write.
 
@@ -20,36 +25,78 @@ Or install it yourself as:
 
 ## Usage
 
-    require 'pond'
-    require 'redis'
+```ruby
+require 'pond'
+require 'redis'
 
-    $redis_pond = Pond.new(:maximum_size => 5, :timeout => 0.5) { Redis.new }
+$redis_pond = Pond.new(:maximum_size => 5, :timeout => 0.5) { Redis.new }
 
-    # No connections are established until we need one:
-    $redis_pond.checkout do |redis|
-      redis.incr 'my_counter'
-      redis.lpush 'my_list', 'item'
-    end
+# No connections are established until we need one:
+$redis_pond.checkout do |redis|
+  redis.incr 'my_counter'
+  redis.lpush 'my_list', 'item'
+end
 
-    # Alternatively, wrap it:
-    $redis = Pond.wrap(:maximum_size => 5, :timeout => 0.5) { Redis.new }
+# Alternatively, wrap it:
+$redis = Pond.wrap(:maximum_size => 5, :timeout => 0.5) { Redis.new }
 
-    # You can now use $redis as you normally would.
-    $redis.incr 'my_counter'
-    $redis.lpush 'my_list', 'item'
+# You can now use $redis as you normally would.
+$redis.incr 'my_counter'
+$redis.lpush 'my_list', 'item'
 
-    $redis.pipelined do
-      # All these commands go to the same Redis connection, and so are pipelined correctly.
-      $redis.incr 'my_counter'
-      $redis.lpush 'my_list', 'item'
-    end
+$redis.pipelined do
+  # All these commands go to the same Redis connection, and so are pipelined correctly.
+  $redis.incr 'my_counter'
+  $redis.lpush 'my_list', 'item'
+end
+```
 
 Options:
-* :maximum_size - The maximum number of objects you want the pool to contain. The default is 10.
-* :timeout - When attempting to check out an object but none are available, how many seconds to wait before raising a `Pond::Timeout` error. The default is 1.
-* :collection - How to manage the objects in the pool. The default is :queue, meaning that pond.checkout will yield the object that hasn't been used in the longest period of time. This is to prevent connections from becoming 'stale'. The alternative is :stack, so checkout will yield the object that has most recently been returned to the pool. This would be preferable if you're using connections that have their own logic for becoming idle in periods of low activity.
-* :eager - Set this to true to fill the pool with instantiated objects when it is created, similar to how `connection_pool` works.
+
+* :maximum_size - The maximum number of objects you want the pool to contain.
+  The default is 10.
+* :timeout - When attempting to check out an object but none are available,
+  how many seconds to wait before raising a `Pond::Timeout` error. The
+  default is 1. Integers or floats are both accepted.
+* :collection - How to manage the objects in the pool. The default is :queue,
+  meaning that pond.checkout will yield the object that hasn't been used in
+  the longest period of time. This is to prevent connections from becoming
+  'stale'. The alternative is :stack, so checkout will yield the object that
+  has most recently been returned to the pool. This would be preferable if
+  you're using connections that have their own logic for becoming idle in
+  periods of low activity.
+* :eager - Set this to true to fill the pool with instantiated objects (up to
+  the maximum size) when it is created, similar to how the `connection_pool`
+  gem works.
+* :detach_if - Set this to a callable object that can determine whether
+  objects should be returned to the pool or not. See the following example for
+  more information.
+
+### Detaching Objects
+
+Sometimes objects in the pool outlive their usefulness (connections may fail)
+and it becomes necessary to remove them. Pond's detach_if option is useful for
+this - you can pass it any callable object, and Pond will pass it objects from
+the pool that have been checked out before they are checked back in. For
+example, when using Pond with PostgreSQL connections:
+
+```ruby
+require 'pond'
+require 'pg'
+
+$pg_pond = Pond.new(:detach_if => lambda {|c| c.finished?}) do
+  PG.connect(:dbname => "pond_test")
+end
+```
+
+Now, after a PostgreSQL connection has been used, but before it is returned to
+the pool, it will be passed to that lambda to see if it should be detached or
+not. If the lambda returns truthy, the connection will be detached (and made
+available for garbage collection), and a new one will be instantiated to
+replace it as necessary (until the pool returns to its maximum size).
 
 ## Contributing
 
-I don't plan on adding too many more features to Pond, since I want to keep its design simple. If there's something you'd like to see it do, open an issue so we can discuss it before going to the trouble of creating a pull request.
+I don't plan on adding too many more features to Pond, since I want to keep
+its design simple. If there's something you'd like to see it do, open an issue
+so we can discuss it before going to the trouble of creating a pull request.

data/Rakefile

@@ -5,3 +5,5 @@ require 'rspec/core/rake_task'
 RSpec::Core::RakeTask.new :default do |spec|
   spec.pattern = './spec/**/*_spec.rb'
 end
+
+Dir[File.dirname(__FILE__) + '/tasks/**/*.rake'].sort.each &method(:load)

data/lib/pond.rb

@@ -5,7 +5,7 @@ require 'pond/version'
 class Pond
   class Timeout < StandardError; end
 
-  attr_reader :allocated, :available, :timeout, :collection, :maximum_size
+  attr_reader :allocated, :available, :timeout, :collection, :maximum_size, :detach_if
 
   def initialize(options = {}, &block)
     @block   = block
@@ -19,6 +19,7 @@ class Pond
 
     self.timeout      = options.fetch :timeout, 1
     self.collection   = options.fetch :collection, :queue
+    self.detach_if    = options.fetch :detach_if, lambda { |_| false }
     self.maximum_size = maximum_size
   end
 
@@ -52,6 +53,11 @@ class Pond
     end
   end
 
+  def detach_if=(callable)
+    raise "Object given for Pond detach_if must respond to #call" unless callable.respond_to?(:call)
+    sync { @detach_if = callable }
+  end
+
   private
 
   def checkout_object
@@ -83,11 +89,23 @@ class Pond
   end
 
   def unlock_object
+    object               = nil
+    detach_if            = nil
+    should_return_object = nil
+
     sync do
-      object = @allocated.delete(Thread.current)
+      object               = current_object
+      detach_if            = self.detach_if
+      should_return_object = object && object != @block && size <= maximum_size
+    end
 
-      if object && object != @block && size < maximum_size
-        @available << object
+    begin
+      should_return_object = !detach_if.call(object) if should_return_object
+      detach_check_finished = true
+    ensure
+      sync do
+        @available << object if detach_check_finished && should_return_object
+        @allocated.delete(Thread.current)
         @cv.signal
       end
     end

data/lib/pond/version.rb

@@ -1,3 +1,3 @@
 class Pond
-  VERSION = '0.1.0'
+  VERSION = '0.2.0'
 end

data/spec/checkout_spec.rb

@@ -233,7 +233,7 @@ describe Pond, "#checkout" do
 
   it "should not block other threads if the object instantiation takes a long time" do
     t = nil
-    q1, q2, q3 = Queue.new, Queue.new, Queue.new
+    q1, q2 = Queue.new, Queue.new
     pond = Pond.new do
       q1.push nil
       q2.pop
@@ -289,5 +289,109 @@ describe Pond, "#checkout" do
     pond.size.should == 1
     pond.allocated.should == {}
     pond.available.should == [1]
+
+    error = false
+
+    pond.checkout do |i|
+      i.should == 1
+
+      t = Thread.new do
+        pond.checkout { |j| j.should == 2 }
+      end
+
+      t.join
+    end
+
+    pond.size.should == 2
+    pond.allocated.should == {}
+    pond.available.should == [2, 1]
+  end
+
+  it "removes the object from the pool if the detach_if block returns true" do
+    int = 0
+    pond = Pond.new(detach_if: lambda { |obj| obj < 2 }) { int += 1 }
+    pond.available.should == []
+
+    # allocate 1, should not check back in
+    pond.checkout {|i| i.should == 1}
+    pond.available.should == []
+
+    # allocate 2, should be nothing else in the pond
+    pond.checkout do |i|
+      i.should == 2
+      pond.available.should == []
+    end
+
+    # 2 should still be in the pond
+    pond.available.should == [2]
+  end
+
+  it "should not block other threads if the detach_if block takes a long time" do
+    i = 0
+    q1, q2 = Queue.new, Queue.new
+
+    detach_if = proc do
+      q1.push nil
+      q2.pop
+    end
+
+    pond = Pond.new(:detach_if => detach_if) do
+      i += 1
+    end
+
+    t = Thread.new do
+      pond.checkout do |i|
+        i.should == 1
+      end
+    end
+
+    q1.pop
+    pond.available.should == []
+    pond.allocated.should == {t => 1}
+
+    # t is in the middle of invoking detach_if, we should still be able to
+    # instantiate new objects and check them out.
+    pond.checkout do |i|
+      i.should == 2
+      q2.push nil
+      q2.push nil
+    end
+
+    t.join
+  end
+
+  it "should not leave the Pond in a bad state if the detach_if block fails" do
+    i = 0
+    error = false
+    detach_if = proc do |obj|
+      raise "Detach Error!" if error
+      false
+    end
+
+    pond = Pond.new(:eager => true, :maximum_size => 5, :detach_if => detach_if) do
+      i += 1
+    end
+
+    pond.size.should == 5
+    pond.allocated.should == {}
+    pond.available.should == [1, 2, 3, 4, 5]
+
+    pond.checkout {}
+
+    pond.available.should == [2, 3, 4, 5, 1]
+
+    error = true
+
+    checked_out = nil
+    proc {
+      pond.checkout do |i|
+        checked_out = i
+      end
+    }.should raise_error RuntimeError, "Detach Error!"
+
+    checked_out.should == 2
+
+    pond.available.should == [3, 4, 5, 1]
+    pond.allocated.should == {}
   end
 end

data/spec/spec_helper.rb

@@ -1 +1,5 @@
 require 'pond'
+
+RSpec.configure do |config|
+  config.expect_with(:rspec) { |c| c.syntax = [:expect, :should] }
+end

data/spec/wrapper_spec.rb

@@ -2,6 +2,11 @@ require 'spec_helper'
 
 describe Pond::Wrapper do
   class Wrapped
+    # JRuby implements BasicObject#object_id, so we need a minor workaround.
+    def id
+      object_id
+    end
+
     def pipelined(&block)
       yield
     end
@@ -17,34 +22,34 @@ describe Pond::Wrapper do
 
     @wrapper.class.should == Wrapped
     @wrapper.respond_to?(:pipelined).should == true
-    object_id = @wrapper.object_id
+    id = @wrapper.id
 
     @pond.size.should == 1
     @pond.allocated.should == {}
-    @pond.available.map(&:object_id).should == [object_id]
+    @pond.available.map(&:id).should == [id]
   end
 
   it "should return the same object within a block passed to one of its methods" do
     q1, q2 = Queue.new, Queue.new
-    oid1, oid2 = nil, nil
+    id1, id2 = nil, nil
 
     @wrapper.pipelined do
-      oid1 = @wrapper.object_id
+      id1 = @wrapper.id
 
       t = Thread.new do
         @wrapper.pipelined do
           q1.push nil
           q2.pop
 
-          oid2 = @wrapper.object_id
-          oid2.should == @wrapper.object_id
+          id2 = @wrapper.id
+          id2.should == @wrapper.id
           @wrapper
         end
       end
 
       q1.pop
 
-      @wrapper.object_id.should == oid1
+      @wrapper.id.should == id1
 
       @pond.allocated.keys.should == [Thread.current, t]
       @pond.available.should == []
@@ -52,11 +57,11 @@ describe Pond::Wrapper do
       q2.push nil
       t.join
 
-      @wrapper.object_id.should == oid1
-      @wrapper.object_id.should == oid1
+      @wrapper.id.should == id1
+      @wrapper.id.should == id1
     end
   
     @pond.allocated.should == {}
-    @pond.available.map(&:object_id).should == [oid2, oid1]
+    @pond.available.map(&:id).should == [id2, id1]
   end
 end

data/tasks/stress.rake

@@ -0,0 +1,34 @@
+require 'pond'
+
+desc "Stress test the Pond gem to check for concurrency issues."
+task :stress do
+  detach_if = proc do |obj|
+    raise "Bad Detach!" if rand < 0.05
+    obj != "Good!"
+  end
+
+  pond = Pond.new(detach_if: detach_if) do
+    raise "Bad Instantiation!" if rand < 0.05
+    "Good!"
+  end
+
+  threads =
+    20.times.map do
+      Thread.new do
+        10_000.times do
+          begin
+            pond.checkout do |o|
+              raise "Uh-oh!" unless o == "Good!"
+              o.replace "Bad!" if rand < 0.05
+            end
+          rescue => e
+            raise e unless ["Bad Detach!", "Bad Instantiation!"].include?(e.message)
+          end
+        end
+      end
+    end
+
+  threads.each(&:join)
+
+  puts "Stress test succeeded!"
+end

metadata

@@ -1,59 +1,66 @@
---- !ruby/object:Gem::Specification 
+--- !ruby/object:Gem::Specification
 name: pond
-version: !ruby/object:Gem::Version 
-  version: 0.1.0
+version: !ruby/object:Gem::Version
+  version: 0.2.0
 platform: ruby
-authors: 
+authors:
 - Chris Hanks
 autorequire: 
 bindir: bin
 cert_chain: []
-
-date: 2014-02-15 00:00:00 Z
-dependencies: 
-- !ruby/object:Gem::Dependency 
+date: 2016-02-05 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
   name: bundler
-  prerelease: false
-  requirement: &id001 !ruby/object:Gem::Requirement 
-    requirements: 
-    - - ~>
-      - !ruby/object:Gem::Version 
-        version: "1.3"
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
   type: :development
-  version_requirements: *id001
-- !ruby/object:Gem::Dependency 
-  name: rspec
   prerelease: false
-  requirement: &id002 !ruby/object:Gem::Requirement 
-    requirements: 
-    - - ~>
-      - !ruby/object:Gem::Version 
-        version: "2.14"
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.14'
   type: :development
-  version_requirements: *id002
-- !ruby/object:Gem::Dependency 
-  name: rake
   prerelease: false
-  requirement: &id003 !ruby/object:Gem::Requirement 
-    requirements: 
-    - &id004 
-      - ">="
-      - !ruby/object:Gem::Version 
-        version: "0"
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.14'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
   type: :development
-  version_requirements: *id003
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: A simple, generic, thread-safe pool for connections or whatever else
-email: 
+email:
 - christopher.m.hanks@gmail.com
 executables: []
-
 extensions: []
-
 extra_rdoc_files: []
-
-files: 
-- .gitignore
-- .rspec
+files:
+- ".gitignore"
+- ".rspec"
 - CHANGELOG.md
 - Gemfile
 - LICENSE.txt
@@ -66,30 +73,32 @@ files:
 - spec/config_spec.rb
 - spec/spec_helper.rb
 - spec/wrapper_spec.rb
+- tasks/stress.rake
 homepage: https://github.com/chanks/pond
-licenses: 
+licenses:
 - MIT
 metadata: {}
-
 post_install_message: 
 rdoc_options: []
-
-require_paths: 
+require_paths:
 - lib
-required_ruby_version: !ruby/object:Gem::Requirement 
-  requirements: 
-  - *id004
-required_rubygems_version: !ruby/object:Gem::Requirement 
-  requirements: 
-  - *id004
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
 requirements: []
-
 rubyforge_project: 
-rubygems_version: 2.2.2
+rubygems_version: 2.5.1
 signing_key: 
 specification_version: 4
 summary: A simple, generic, thread-safe pool
-test_files: 
+test_files:
 - spec/checkout_spec.rb
 - spec/config_spec.rb
 - spec/spec_helper.rb