fiber_connection_pool 0.1.2 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 654c0a80fecfc77daa838a3a0611ad534e4413fc
-  data.tar.gz: cbb3bd2ccb7f50fc534d98f976cfe2ec42c87302
+  metadata.gz: 3805c4817fd7ddab4eaafcdf30bacdd99dd79646
+  data.tar.gz: 2af6d4c85fd8d672b7d8c0b9eaea6a02a76e3658
 SHA512:
-  metadata.gz: 4f01cfc453ecfc286069f18ac34545c079a64214b2e99dee2a2ef3ef6c1414e72c2da586d653d23e61a35d363c62f28bcd45f160bab9d5d93323cf92e2d9765a
-  data.tar.gz: c0995f388b1552a246cc9f68ab1ebcc929cac66a088b0aa1a4d0743bae84b5f34f853e34b0164d1c323e458e81be4a249c0d0d52fe7fc9347b8e610ac76af2a2
+  metadata.gz: 65a29c9021d57a6f7de8390aa28cf56a5d330ed7b1259dd760b6b4dd5e5be67d8a7e5b1d0c127fb62d7b7c0d252746866269e4fd2a134cf2ab92e8dcba5a9b8c
+  data.tar.gz: ae6b1518828313527b91be4005e6cdc510f364158ed7626df6dff083feee43166006267baba0aae70f519df54bc20ee93bdf0b94e0fd97072bacf3fd6b4e823b

data/.travis.yml

@@ -4,9 +4,3 @@ rvm:
   - 1.9.3
   - 2.0.0
   - rbx-19mode
-
-matrix:
-  allow_failures:
-    - branches:
-      except:
-        - master

data/README.md

@@ -47,25 +47,31 @@ It just keeps an array (the internal pool) holding the result of running
 the given block _size_ times. Inside the reactor loop (either EventMachine's or Celluloid's),
 each request is wrapped on a Fiber, and then `pool` plays its magic.
 
-When a method `query_me` is called on `pool` and it's not one of its own methods,
-then it:
+``` ruby
+results = pool.query_me(sql)
+```
 
-1. reserves one connection from the internal pool and associates it __with the current Fiber__
-2. if no connection is available, then that Fiber stays on a _pending_ queue, and __is yielded__
-3. when a connection is available, then the pool calls `query_me` on that `MyFancyConnection` instance
-4. when `query_me` returns, the reserved instance is released again,
-and the next Fiber on the _pending_ queue __is resumed__
-5. the return value is sent back to the caller
+When a method `query_me` is called on `pool` it:
+
+1. Reserves one connection from the internal pool and associates it __with the current fiber__.
+2. If no connection is available, then that fiber stays on a _pending_ queue,
+and __is yielded__ until another connection is released.
+3. When a connection is available, then the pool calls `query_me` on that `MyFancyConnection` instance.
+4. When `query_me` returns, the reserved instance is released again,
+and the next fiber on the _pending_ queue __is resumed__.
+5. The return value is sent back to the caller.
 
 Methods from `MyFancyConnection` instance should yield the fiber before
 perform any blocking IO. That returns control to te underlying reactor,
 that spawns another fiber to process the next request, while the previous
 one is still waiting for the IO response. That new fiber will get its own
 connection from the pool, or else it will yield until there
-is one available.
+is one available. That behaviour is implemented on `Mysql2::EM::Client`
+from [em-synchrony](https://github.com/igrigorik/em-synchrony),
+and on a patched version of [ruby-mysql](https://github.com/rubencaro/ruby-mysql), for example.
 
-The whole process looks synchronous from the Fiber perspective, _because it is_.
-The Fiber will really block ( _yield_ ) until it gets the result.
+The whole process looks synchronous from the fiber perspective, _because it is_ indeed.
+The fiber will really block ( or _yield_ ) until it gets the result.
 
 ``` ruby
 results = pool.query_me(sql)
@@ -77,21 +83,21 @@ The magic resides on the fact that other fibers are being processed while this o
 Not thread-safe
 ------------------
 
-`FiberConnectionPool` is not thread-safe right now. You will not be able to use it
+`FiberConnectionPool` is not thread-safe. You will not be able to use it
 from different threads, as eventually it will try to resume a Fiber that resides
 on a different Thread. That will raise a FiberError( _"calling a fiber across threads"_ ).
-Maybe one day we add that feature too.
+Maybe one day we add that feature too. Or maybe it's not worth the added code complexity.
 
-We have tested it on Goliath servers having one pool on each server instance, and on Reel servers
-having one pool on each Actor thread. Take a look at the `examples` folder for details.
+We use it with no need to be thread-safe on Goliath servers having one pool on each server instance,
+and on Reel servers having one pool on each Actor thread. Take a look at the `examples` folder for details.
 
-MySQL specific
+Generic
 ------------------
 
-By now we have only thought and tested it to be used with MySQL connections.
-For EventMachine by using `Mysql2::EM::Client` from [em-synchrony](https://github.com/igrigorik/em-synchrony).
+We use it extensively with MySQL connections with Goliath servers by using `Mysql2::EM::Client`
+from [em-synchrony](https://github.com/igrigorik/em-synchrony).
 And for Celluloid by using a patched version of [ruby-mysql](https://github.com/rubencaro/ruby-mysql).
-We plan on removing any MySQL specific code, so it becomes completely generic. Does not seem so hard to achieve.
+By >=0.2 there is no MySQL-specific code, so it can be used with any kind of connection that can be fibered.
 
 Reacting to connection failure
 ------------------
@@ -102,8 +108,81 @@ react as you would do normally.
 
 You have to be aware that the connection instance will remain in the pool, and other fibers
 will surely use it. If the Exception you rescued indicates that the connection should be
-recreated, you can call `recreate_connection` passing it a new instance. The instance that
-just failed will be replaced inside the pool by the brand new connection.
+recreated or treated somehow, there's a way to access that particular connection:
+
+``` ruby
+begin
+
+  pool.bad_query('will make me worse')
+
+rescue BadQueryMadeMeWorse
+
+  pool.with_failed_connection do |connection|
+    puts "Replacing #{connection.inspect} with a new one!"
+    MyFancyConnection.new
+  end
+
+end
+```
+
+The pool saves the connection when it raises an exception on a fiber, and with `with_failed_connection` lets
+you execute a block of code over it. It must return a connection instance, and it will be put inside the pool
+in place of the failed one. It can be the same instance after being fixed, or maybe a new one.
+The call to `with_failed_connection` must be made from the very same
+fiber that raised the exception.
+
+Also the reference to the failed connection will be lost after any method execution from that
+fiber. So you must call `with_failed_connection` before any other method that may acquire a new instance from the pool.
+
+Any reference to a failed connection is released when the fiber is dead, but as you must access it from the fiber itself, worry should not.
+
+Save data
+-------------------
+
+Sometimes we need to get something more than de return value from the `query_me` call, but that _something_ is related to _that_ call on _that_ connection.
+For example, maybe you need to call `affected_rows` right after the query was made on that particular connection.
+If you make that extra calls on the `pool` object, it will acquire a new connection from the pool an run on it. So it's useless.
+There is a way to gather all that data from the connection so we can work on it, but also release the connection for other fiber to use it.
+
+``` ruby
+# define the pool
+pool = FiberConnectionPool.new(:size => 5){ MyFancyConnection.new }
+
+# add a request to save data for each successful call on a connection
+# will save the return value inside a hash on the key ':affected_rows'
+# and make it available for the fiber that made the call
+pool.save_data(:affected_rows) do |connection|
+  connection.affected_rows
+end
+```
+
+Then from our fiber:
+
+``` ruby
+pool.query_me('affecting 5 rows right now')
+
+# recover gathered data for this fiber
+puts pool.gathered_data
+  => { :affected_rows => 5 }
+```
+
+You must access the gathered data from the same fiber that triggered its gathering.
+Also any new call to `query_me` or any other method from the connection would execute the block again,
+overwriting that position on the hash (unless you code to prevent it, of course). Usually you would use the gathered data
+right after you made the query that generated it. But you could:
+
+``` ruby
+# save only the first run
+pool.save_data(:affected_rows) do |connection|
+  pool.gathered_data[:affected_rows] || connection.affected_rows
+end
+```
+
+You can define as much `save_data` blocks as you want, and run any wonder ruby lets you. But great power comes with great responsability.
+You must consider that any requests for saving data are executed for _every call_ on the pool from that fiber.
+So keep it stupid simple, and blindly fast. At least as much as you can. That would affect performance otherwise.
+
+Any gathered_data is released when the fiber is dead, but as you must access it from the fiber itself, worry should not.
 
 Supported Platforms
 -------------------
@@ -111,11 +190,6 @@ Supported Platforms
 Used in production environments on Ruby 1.9.3 and 2.0.0.
 Tested against Ruby 1.9.3, 2.0.0, and rbx-19mode ([See details..](http://travis-ci.org/rubencaro/fiber_connection_pool)).
 
-TODOS
+More to come !
 -------------------
-
-* no MySQL-specific code
-* better testing
-* improve reaction to failure
-* better in-code docs
-* make thread-safe
+See [issues](https://github.com/rubencaro/fiber_connection_pool/issues?direction=desc&sort=updated&state=open)

data/fiber_connection_pool.gemspec

@@ -6,7 +6,7 @@ Gem::Specification.new do |s|
   s.version     = FiberConnectionPool::VERSION
   s.platform    = Gem::Platform::RUBY
   s.authors     = ["Ruben Caro", "Oriol Francès"]
-  s.email       = ["ruben@lanuez.org"]
+  s.email       = ["ruben.caro@lanuez.org"]
   s.homepage    = "https://github.com/rubencaro/fiber_connection_pool"
   s.summary = "Fiber-based generic connection pool for Ruby"
   s.description = "Fiber-based generic connection pool for Ruby, allowing
@@ -14,8 +14,7 @@ Gem::Specification.new do |s|
                   as provided by EventMachine or Celluloid."
 
   s.files         = `git ls-files`.split("\n")
-  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
-  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
+  s.test_files    = `git ls-files -- test/*`.split("\n")
   s.require_paths = ["lib"]
   s.license = "GPLv3"
 

data/lib/fiber_connection_pool/exceptions.rb

@@ -0,0 +1,6 @@
+
+class NoBackupConnection < Exception
+  def initialize
+    super "No backup connection for this fiber!"
+  end
+end

data/lib/fiber_connection_pool.rb

@@ -1,9 +1,13 @@
 require 'fiber'
+require_relative 'fiber_connection_pool/exceptions'
 
 class FiberConnectionPool
-  VERSION = '0.1.2'
+  VERSION = '0.2.0'
 
-  attr_accessor :saved_data
+  RESERVED_BACKUP_TTL_SECS = 30 # reserved backup cleanup trigger
+  SAVED_DATA_TTL_SECS = 30 # saved_data cleanup trigger
+
+  attr_accessor :saved_data, :reserved_backup
 
   # Initializes the pool with 'size' instances
   # running the given block to get each one. Ex:
@@ -16,33 +20,113 @@ class FiberConnectionPool
     @saved_data = {} # placeholder for requested save data
     @reserved  = {}   # map of in-progress connections
     @reserved_backup = {}   # backup map of in-progress connections, to catch failures
+    @last_backup_cleanup = Time.now # reserved backup cleanup trigger
     @available = []   # pool of free connections
     @pending   = []   # pending reservations (FIFO)
+    @save_data_requests = {} # blocks to be yielded to save data
+    @last_data_cleanup = Time.now # saved_data cleanup trigger
 
     @available = Array.new(opts[:size].to_i) { yield }
   end
 
+  # DEPRECATED: use save_data
   def save_data_for_fiber
-    @saved_data[Fiber.current.object_id] ||= {}
+    nil
   end
 
+  # DEPRECATED: use release_data
   def stop_saving_data_for_fiber
-    @saved_data.delete Fiber.current.object_id
+    @saved_data.delete Fiber.current
   end
 
-  ##
-  # avoid method_missing for most common methods
+  # Add a save_data request to the pool.
+  # The given block will be executed after each successful
+  # call to -any- method on the connection.
+  # The connection and the method name are passed to the block.
+  #
+  # The returned value will be saved in pool.saved_data[Fiber.current][key],
+  # and will be kept as long as the fiber stays alive.
+  #
+  # Ex:
+  #
+  #   # (...right after pool's creation...)
+  #   pool.save_data(:hey_or_hoo) do |conn, method|
+  #     return 'hey' if method == 'query'
+  #     'hoo'
+  #   end
+  #
+  #   # (...from a reactor fiber...)
+  #   myfiber = Fiber.current
+  #   pool.query('select anything from anywhere')
+  #   puts pool.saved_data[myfiber][:hey_or_hoo]
+  #     => 'hey'
+  #
+  #   # (...eventually fiber dies...)
+  #   puts pool.saved_data[myfiber].inspect
+  #     => nil
+  #
+  def save_data(key, &block)
+    @save_data_requests[key] = block
+  end
+
+  # Return the gathered data for this fiber
+  #
+  def gathered_data
+    @saved_data[Fiber.current]
+  end
+
+  # Clear any save_data requests in the pool.
+  # No data will be saved after this, unless new requests are added with #save_data.
+  #
+  def clear_save_data_requests
+    @save_data_requests = {}
+  end
+
+  # Delete any saved_data for given fiber
+  #
+  def release_data(fiber)
+    @saved_data.delete(fiber)
+  end
+
+  # Delete any saved_data held for dead fibers
+  #
+  def save_data_cleanup
+    @saved_data.dup.each do |k,v|
+      @saved_data.delete(k) if not k.alive?
+    end
+    @last_data_cleanup = Time.now
+  end
+
+  # Avoid method_missing stack for 'query'
   #
   def query(sql)
-    execute(false,'query') do |conn|
+    execute('query') do |conn|
       conn.query sql
     end
   end
 
+  # True if the given connection is anywhere inside the pool
+  #
+  def has_connection?(conn)
+    (@available + @reserved.values).include?(conn)
+  end
 
+  # DEPRECATED: use with_failed_connection
   def recreate_connection(new_conn)
-    bad_conn = @reserved_backup[Fiber.current.object_id]
-    release_backup Fiber.current
+    with_failed_connection { new_conn }
+  end
+
+  # Identify the connection that just failed for current fiber.
+  # Pass it to the given block, which must return a valid instance of connection.
+  # After that, put the new connection into the pool in failed connection's place.
+  # Raises NoBackupConnection if cannot find the failed connection instance.
+  #
+  def with_failed_connection
+    f = Fiber.current
+    bad_conn = @reserved_backup[f]
+    raise NoBackupConnection.new if bad_conn.nil?
+    new_conn = yield bad_conn
+    release_backup f
     @available.reject!{ |v| v == bad_conn }
     @reserved.reject!{ |k,v| v == bad_conn }
     @available.push new_conn
@@ -53,35 +137,62 @@ class FiberConnectionPool
     end
   end
 
+  # Delete any backups held for dead fibers
+  #
+  def backup_cleanup
+    @reserved_backup.dup.each do |k,v|
+      @reserved_backup.delete(k) if not k.alive?
+    end
+    @last_backup_cleanup = Time.now
+  end
+
   private
 
   # Choose first available connection and pass it to the supplied
-  # block. This will block indefinitely until there is an available
+  # block. This will block (yield) indefinitely until there is an available
   # connection to service the request.
-  def execute(async,method)
+  #
+  # After running the block, save requested data and release the connection.
+  #
+  def execute(method)
     f = Fiber.current
-
     begin
+      # get a connection and use it
       conn = acquire(f)
       retval = yield conn
-      if !@saved_data[Fiber.current.object_id].nil?
-        @saved_data[Fiber.current.object_id]['affected_rows'] = conn.affected_rows
-      end
-      release_backup(f) if !async and method == 'query'
+
+      # save anything requested
+      process_save_data(f, conn, method)
+
+      # successful run, release_backup
+      release_backup(f)
+
       retval
     ensure
-      release(f) if not async
+      release(f)
     end
   end
 
-  # Acquire a lock on a connection and assign it to executing fiber
-  # - if connection is available, pass it back to the calling block
-  # - if pool is full, yield the current fiber until connection is available
-  def acquire(fiber)
+  # Run each save_data_block over the given connection
+  # and save the data for the given fiber.
+  # Also perform cleanup if TTL is past
+  #
+  def process_save_data(fiber, conn, method)
+    @save_data_requests.each do |key,block|
+      @saved_data[fiber] ||= {}
+      @saved_data[fiber][key] = block.call(conn, method)
+    end
+    # try cleanup
+    save_data_cleanup if (Time.now - @last_data_cleanup) >= SAVED_DATA_TTL_SECS
+  end
 
+  # Acquire a lock on a connection and assign it to given fiber
+  # If no connection is available, yield the given fiber on the pending array
+  #
+  def acquire(fiber)
     if conn = @available.pop
       @reserved[fiber.object_id] = conn
-      @reserved_backup[fiber.object_id] = conn
+      @reserved_backup[fiber] = conn
       conn
     else
       Fiber.yield @pending.push fiber
@@ -90,13 +201,18 @@ class FiberConnectionPool
   end
 
   # Release connection from the backup hash
+  # Also perform cleanup if TTL is past
+  #
   def release_backup(fiber)
-    @reserved_backup.delete(fiber.object_id)
+    @reserved_backup.delete(fiber)
+    # try cleanup
+    backup_cleanup if (Time.now - @last_backup_cleanup) >= RESERVED_BACKUP_TTL_SECS
   end
 
   # Release connection assigned to the supplied fiber and
   # resume any other pending connections (which will
   # immediately try to run acquire on the pool)
+  #
   def release(fiber)
     @available.push(@reserved.delete(fiber.object_id)).compact!
 
@@ -107,29 +223,13 @@ class FiberConnectionPool
 
   # Allow the pool to behave as the underlying connection
   #
-  # If the requesting method begins with "a" prefix, then
-  # hijack the callbacks and errbacks to fire a connection
-  # pool release whenever the request is complete. Otherwise
-  # yield the connection within execute method and release
-  # once it is complete (assumption: fiber will yield until
-  # data is available, or request is complete)
+  # Yield the connection within execute method and release
+  # once it is complete (assumption: fiber will yield while
+  # waiting for IO, allowing the reactor run other fibers)
   #
   def method_missing(method, *args, &blk)
-    async = (method[0,1] == "a")
-
-    execute(async,method) do |conn|
-      df = conn.send(method, *args, &blk)
-
-      if async
-        fiber = Fiber.current
-        df.callback do
-          release(fiber)
-          release_backup(fiber)
-        end
-        df.errback { release(fiber) }
-      end
-
-      df
+    execute(method) do |conn|
+      conn.send(method, *args, &blk)
     end
   end
 end

data/test/fiber_connection_pool_test.rb

@@ -1,4 +1,3 @@
-Thread.abort_on_exception = true
 require 'helper'
 
 class TestFiberConnectionPool < Minitest::Test
@@ -6,11 +5,12 @@ class TestFiberConnectionPool < Minitest::Test
   def test_blocking_behaviour
     # get pool and fibers
     pool = FiberConnectionPool.new(:size => 5) { ::BlockingConnection.new(:delay => 0.05) }
+    info = { :threads => [], :fibers => [], :instances => []}
 
-    fibers = Array.new(15){ Fiber.new { pool.do_something } }
+    fibers = Array.new(15){ Fiber.new { pool.do_something(info) } }
 
     a = Time.now
-    result = fibers.map(&:resume)
+    fibers.each{ |f| f.resume }
     b = Time.now
 
     # 15 fibers on a size 5 pool, but -blocking- connections
@@ -20,36 +20,28 @@ class TestFiberConnectionPool < Minitest::Test
     # Also we only use the first connection from the pool,
     # because as we are -blocking- it's always available
     # again for the next request
-    assert_equal 1, result.uniq.count
+    # we should have visited 1 thread, 15 fibers and 1 instances
+    info.dup.each{ |k,v| info[k] = v.uniq }
+    assert_equal 1, info[:threads].count
+    assert_equal 15, info[:fibers].count
+    assert_equal 1, info[:instances].count
   end
 
   def test_em_synchrony_behaviour
-    require 'em-synchrony'
-
-    a = b = nil
     info = { :threads => [], :fibers => [], :instances => []}
 
-    EM.synchrony do
-      # get pool and fibers
-      pool = FiberConnectionPool.new(:size => 5) { ::EMSynchronyConnection.new(:delay => 0.05) }
+    # get pool and fibers
+    pool = FiberConnectionPool.new(:size => 5) { ::EMSynchronyConnection.new(:delay => 0.05) }
 
-      fibers = Array.new(15){ Fiber.new { pool.do_something(info) } }
+    fibers = Array.new(15){ Fiber.new { pool.do_something(info) } }
 
-      a = Time.now
-      fibers.each{ |f| f.resume }
-      # wait all fibers to end
-      while fibers.any?{ |f| f.alive? } do
-        EM::Synchrony.sleep 0.01
-      end
-      b = Time.now
-      EM.stop
-    end
+    lapse = run_em_reactor fibers
 
     # 15 fibers on a size 5 pool, and -non-blocking- connections
     # with a 0.05 delay we expect to spend at least: 0.05*15/5 = 0.15
     # plus some breeze lost on precision on the wait loop
     # then we should be under 0.20 for sure
-    assert_operator((b - a), :<, 0.20)
+    assert_operator(lapse, :<, 0.20)
 
     # we should have visited 1 thread, 15 fibers and 5 instances
     info.dup.each{ |k,v| info[k] = v.uniq }
@@ -58,6 +50,11 @@ class TestFiberConnectionPool < Minitest::Test
     assert_equal 5, info[:instances].count
   end
 
+  def test_celluloid_behaviour
+    skip 'Could not test celluloid 0.15.0pre, as it would not start reactor on test environment.
+          See the examples folder for a working celluloid (reel) server.'
+  end
+
   def test_size_is_mandatory
     assert_raises ArgumentError do
       FiberConnectionPool.new { ::BlockingConnection.new }
@@ -70,5 +67,170 @@ class TestFiberConnectionPool < Minitest::Test
     end
   end
 
+  def test_failure_reaction
+    info = { :instances => [] }
+
+    # get pool and fibers
+    pool = FiberConnectionPool.new(:size => 5) { ::EMSynchronyConnection.new(:delay => 0.05) }
+
+    fibers = Array.new(14){ Fiber.new { pool.do_something(info) } }
+
+    failing_fiber = Fiber.new do
+      begin
+        pool.fail(info)
+      rescue
+        pool.with_failed_connection do |connection|
+          info[:repaired_connection] = connection
+          # replace it in the pool
+          ::EMSynchronyConnection.new(:delay => 0.05)
+        end
+      end
+    end
+    # put it among others, not the first or the last
+    # so we see it does not mistake the failing connection
+    fibers.insert 7,failing_fiber
+
+    run_em_reactor fibers
+
+    # we should have visited 1 thread, 15 fibers and 6 instances (including failed)
+    info.dup.each{ |k,v| info[k] = v.uniq if v.is_a?(Array) }
+    assert_equal 6, info[:instances].count
+
+    # assert we do not lose track of failing connection
+    assert_equal info[:repaired_connection], info[:failing_connection]
+
+    # assert we replaced it
+    refute pool.has_connection?(info[:failing_connection])
+
+    # nothing left
+    assert_equal(0, pool.reserved_backup.count)
+
+    # if dealing with failed connection where you shouldn't...
+    assert_raises NoBackupConnection do
+      pool.with_failed_connection{ |c| 'boo' }
+    end
+  end
+
+  def test_reserved_backups
+    # create pool, run fibers and gather info
+    pool, info = run_reserved_backups
+
+    # one left
+    assert_equal(1, pool.reserved_backup.count)
+
+    # fire cleanup
+    pool.backup_cleanup
+
+    # nothing left
+    assert_equal(0, pool.reserved_backup.count)
+
+    # assert we did not replace it
+    assert pool.has_connection?(info[:failing_connection])
+  end
+
+  def test_auto_cleanup_reserved_backups
+    # lower ttl to force auto cleanup
+    prev_ttl = force_constant FiberConnectionPool, :RESERVED_BACKUP_TTL_SECS, 0
+
+    # create pool, run fibers and gather info
+    pool, info = run_reserved_backups
+
+    # nothing left, because failing fiber was not the last to run
+    # the following fiber made the cleanup
+    assert_equal(0, pool.reserved_backup.count)
+
+    # assert we did not replace it
+    assert pool.has_connection?(info[:failing_connection])
+  ensure
+    # restore
+    force_constant FiberConnectionPool, :RESERVED_BACKUP_TTL_SECS, prev_ttl
+  end
+
+  def test_save_data
+    # create pool, run fibers and gather info
+    pool, fibers, info = run_saved_data
+
+    # gathered data for all 4 fibers
+    assert fibers.all?{ |f| not pool.saved_data[f].nil? },
+        "fibers: #{fibers}, saved_data: #{pool.saved_data}"
+
+    # gathered 2 times each connection
+    connection_ids = pool.saved_data.values.map{ |v| v[:connection_id] }
+    assert info[:instances].all?{ |i| connection_ids.count(i) == 2 },
+        "info: #{info}, saved_data: #{pool.saved_data}"
+
+    # fire cleanup
+    pool.save_data_cleanup
+
+    # nothing left
+    assert_equal(0, pool.saved_data.count)
+  end
+
+  def test_auto_cleanup_saved_data
+    # lower ttl to force auto cleanup
+    prev_ttl = force_constant FiberConnectionPool, :SAVED_DATA_TTL_SECS, 0
+
+    # create pool, run fibers and gather info
+    pool, _, _ = run_saved_data
+
+    # only the last run left
+    # that fiber was the one making the cleanup, so it was still alive
+    assert_equal(1, pool.saved_data.count)
+  ensure
+    # restore
+    force_constant FiberConnectionPool, :SAVED_DATA_TTL_SECS, prev_ttl
+  end
+
+  private
+
+  def run_reserved_backups
+    info = { :instances => [] }
+
+    # get pool and fibers
+    pool = FiberConnectionPool.new(:size => 2) { ::EMSynchronyConnection.new(:delay => 0.05) }
+
+    fibers = Array.new(4){ Fiber.new { pool.do_something(info) } }
+
+    # we do not repair it, backup associated with this Fiber stays in the pool
+    failing_fiber = Fiber.new { pool.fail(info) rescue nil }
+
+    # put it among others, not the first or the last
+    # so we see it does not mistake the failing connection
+    fibers.insert 2,failing_fiber
+
+    run_em_reactor fibers
+
+    # we should have visited only 2 instances (no instance added by repairing broken one)
+    info.dup.each{ |k,v| info[k] = v.uniq if v.is_a?(Array) }
+    assert_equal 2, info[:instances].count
+
+    [ pool, info ]
+  end
+
+  def run_saved_data
+    info = { :instances => [] }
+
+    # get pool and fibers
+    pool = FiberConnectionPool.new(:size => 2) { ::EMSynchronyConnection.new(:delay => 0.05) }
+
+    # ask to save some data
+    pool.save_data(:connection_id) { |conn| conn.object_id }
+    pool.save_data(:fiber_id) { |conn| Fiber.current.object_id }
+
+    fibers = Array.new(4) do
+      Fiber.new do
+        pool.do_something(info)
+        assert_equal Fiber.current.object_id, pool.gathered_data[:fiber_id]
+      end
+    end
+
+    run_em_reactor fibers
+
+    # we should have visited 2 instances
+    info.dup.each{ |k,v| info[k] = v.uniq if v.is_a?(Array) }
+    assert_equal 2, info[:instances].count
+
+    [ pool, fibers, info ]
+  end
 
 end

data/test/helper.rb

@@ -1,5 +1,6 @@
 require 'minitest/pride'
 require 'minitest/autorun'
+require 'em-synchrony'
 
 require_relative '../lib/fiber_connection_pool'
 
@@ -8,17 +9,50 @@ class BlockingConnection
     @delay = opts[:delay] || 0.05
   end
 
-  def do_something
+  def do_something(info = {})
+    fill_info info
     sleep @delay
-    self.object_id
+  end
+
+  def fill_info(info = {})
+    info[:threads] << Thread.current.object_id if info[:threads]
+    info[:fibers] << Fiber.current.object_id if info[:fibers]
+    info[:instances] << self.object_id if info[:instances]
   end
 end
 
 class EMSynchronyConnection < BlockingConnection
-  def do_something(info)
-    info[:threads] << Thread.current.object_id
-    info[:fibers] << Fiber.current.object_id
-    info[:instances] << self.object_id
+  def do_something(info = {})
+    fill_info info
     EM::Synchrony.sleep @delay
   end
+
+  def fail(info)
+    fill_info info
+    info[:failing_connection] = self
+    raise "Sadly failing here..."
+  end
+end
+
+# start an EM reactor and run given fibers
+# return time spent
+def run_em_reactor(fibers)
+  a = b = nil
+  EM.synchrony do
+    a = Time.now
+    fibers.each{ |f| f.resume }
+    # wait all fibers to end
+    while fibers.any?{ |f| f.alive? } do
+      EM::Synchrony.sleep 0.01
+    end
+    b = Time.now
+    EM.stop
+  end
+  b-a
+end
+
+def force_constant(klass, name, value)
+  previous_value = klass.send(:remove_const, name)
+  klass.const_set name.to_s, value
+  previous_value
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fiber_connection_pool
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.2.0
 platform: ruby
 authors:
 - Ruben Caro
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-08-16 00:00:00.000000000 Z
+date: 2013-08-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -44,7 +44,7 @@ description: |-
                     non-blocking IO behaviour on the same thread
                     as provided by EventMachine or Celluloid.
 email:
-- ruben@lanuez.org
+- ruben.caro@lanuez.org
 executables: []
 extensions: []
 extra_rdoc_files: []
@@ -63,6 +63,7 @@ files:
 - examples/reel_server/main.rb
 - fiber_connection_pool.gemspec
 - lib/fiber_connection_pool.rb
+- lib/fiber_connection_pool/exceptions.rb
 - test/fiber_connection_pool_test.rb
 - test/helper.rb
 homepage: https://github.com/rubencaro/fiber_connection_pool
@@ -85,9 +86,11 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.0.3
+rubygems_version: 2.0.6
 signing_key: 
 specification_version: 4
 summary: Fiber-based generic connection pool for Ruby
-test_files: []
+test_files:
+- test/fiber_connection_pool_test.rb
+- test/helper.rb
 has_rdoc: