fiber_connection_pool 0.1.1 → 0.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 6d5107300e5f00c78557fabe1ef21a964bc0d929
-  data.tar.gz: 06f8d340506aba8db7035d0c5f17fb6ddc4e563b
+  metadata.gz: 654c0a80fecfc77daa838a3a0611ad534e4413fc
+  data.tar.gz: cbb3bd2ccb7f50fc534d98f976cfe2ec42c87302
 SHA512:
-  metadata.gz: d258dabb23ce59218aa85cb5271a6efebc91817bd74748f2ad9933001a5b160f8373c38c2768f26694f3ff5bde7a1331455f6f515fd7af7ebfe8c24af81bc245
-  data.tar.gz: 03f4d89e6235a16501a1998752e6693d7b8e8cd824a5d7c49d956482c667549fd73d59ea0d3b7f9028d0023c67cd5bbf1c78c7359732c36a9874693ddbe285eb
+  metadata.gz: 4f01cfc453ecfc286069f18ac34545c079a64214b2e99dee2a2ef3ef6c1414e72c2da586d653d23e61a35d363c62f28bcd45f160bab9d5d93323cf92e2d9765a
+  data.tar.gz: c0995f388b1552a246cc9f68ab1ebcc929cac66a088b0aa1a4d0743bae84b5f34f853e34b0164d1c323e458e81be4a249c0d0d52fe7fc9347b8e610ac76af2a2

data/README.md

@@ -6,17 +6,20 @@ fiber_connection_pool
 
 Fiber-based generic connection pool
 
+A connection pool meant to be used inside a Fiber-based _reactor_,
+such as any [EventMachine](https://github.com/eventmachine/eventmachine)
+or [Celluloid](http://celluloid.io/) server.
+
 Widely based on `ConnectionPool`
 from [em-synchrony](https://github.com/igrigorik/em-synchrony) gem, and
 some things borrowed also from
 threaded [connection_pool](https://github.com/mperham/connection_pool) gem.
-
 Used in production environments
 with [Goliath](https://github.com/postrank-labs/goliath)
-([EventMachine](https://github.com/eventmachine/eventmachine) based) servers,
+(EventMachine based) servers,
 and in promising experiments with
 [Reel](https://github.com/celluloid/reel)
-([Celluloid](http://celluloid.io/) based) servers.
+(Celluloid based) servers.
 
 Install
 ----------------
@@ -33,10 +36,86 @@ Inside of your Ruby program, require FiberConnectionPool with:
 
     require 'fiber_connection_pool'
 
+How It Works
+-------------------
+
+```  ruby
+pool = FiberConnectionPool.new(:size => 5){ MyFancyConnection.new }
+```
+
+It just keeps an array (the internal pool) holding the result of running
+the given block _size_ times. Inside the reactor loop (either EventMachine's or Celluloid's),
+each request is wrapped on a Fiber, and then `pool` plays its magic.
+
+When a method `query_me` is called on `pool` and it's not one of its own methods,
+then it:
+
+1. reserves one connection from the internal pool and associates it __with the current Fiber__
+2. if no connection is available, then that Fiber stays on a _pending_ queue, and __is yielded__
+3. when a connection is available, then the pool calls `query_me` on that `MyFancyConnection` instance
+4. when `query_me` returns, the reserved instance is released again,
+and the next Fiber on the _pending_ queue __is resumed__
+5. the return value is sent back to the caller
+
+Methods from `MyFancyConnection` instance should yield the fiber before
+perform any blocking IO. That returns control to te underlying reactor,
+that spawns another fiber to process the next request, while the previous
+one is still waiting for the IO response. That new fiber will get its own
+connection from the pool, or else it will yield until there
+is one available.
+
+The whole process looks synchronous from the Fiber perspective, _because it is_.
+The Fiber will really block ( _yield_ ) until it gets the result.
+
+``` ruby
+results = pool.query_me(sql)
+puts "I waited for this: #{results}"
+```
+
+The magic resides on the fact that other fibers are being processed while this one is waiting.
+
+Not thread-safe
+------------------
+
+`FiberConnectionPool` is not thread-safe right now. You will not be able to use it
+from different threads, as eventually it will try to resume a Fiber that resides
+on a different Thread. That will raise a FiberError( _"calling a fiber across threads"_ ).
+Maybe one day we add that feature too.
+
+We have tested it on Goliath servers having one pool on each server instance, and on Reel servers
+having one pool on each Actor thread. Take a look at the `examples` folder for details.
+
+MySQL specific
+------------------
+
+By now we have only thought and tested it to be used with MySQL connections.
+For EventMachine by using `Mysql2::EM::Client` from [em-synchrony](https://github.com/igrigorik/em-synchrony).
+And for Celluloid by using a patched version of [ruby-mysql](https://github.com/rubencaro/ruby-mysql).
+We plan on removing any MySQL specific code, so it becomes completely generic. Does not seem so hard to achieve.
+
+Reacting to connection failure
+------------------
+
+When the call to a method raises an Exception it will raise as if there was no pool between
+your code and the connetion itself. You can rescue the Exception as usual and
+react as you would do normally.
+
+You have to be aware that the connection instance will remain in the pool, and other fibers
+will surely use it. If the Exception you rescued indicates that the connection should be
+recreated, you can call `recreate_connection` passing it a new instance. The instance that
+just failed will be replaced inside the pool by the brand new connection.
+
 Supported Platforms
 -------------------
 
 Used in production environments on Ruby 1.9.3 and 2.0.0.
 Tested against Ruby 1.9.3, 2.0.0, and rbx-19mode ([See details..](http://travis-ci.org/rubencaro/fiber_connection_pool)).
 
-TODO: sparkling docs
+TODOS
+-------------------
+
+* no MySQL-specific code
+* better testing
+* improve reaction to failure
+* better in-code docs
+* make thread-safe

data/examples/goliath_server/Gemfile

@@ -0,0 +1,5 @@
+source 'https://rubygems.org'
+gem 'goliath'
+gem 'em-synchrony'
+gem 'mysql2'
+gem 'fiber_connection_pool'

data/examples/goliath_server/config/main.rb

@@ -0,0 +1,13 @@
+require 'fiber_connection_pool'
+require 'mysql2'
+require 'em-synchrony'
+require 'em-synchrony/mysql2'
+
+config['db'] = FiberConnectionPool.new(:size => 5) do
+                 Mysql2::EM::Client.new({
+                    host: 'localhost',
+                    username: 'user',
+                    password: 'pass',
+                    database: 'bogusdb'
+                  })
+               end

data/examples/goliath_server/main.rb

@@ -0,0 +1,13 @@
+require 'goliath'
+require 'fiber'
+
+class Main < Goliath::API
+
+  def response(env)
+    print '.'
+    db.query 'select sleep(2);'
+    puts "Done #{Thread.current.to_s}, #{Fiber.current.to_s}"
+    [200,{"Content-Type" => "text/html"},"hello, world! #{Time.now.strftime('%T')}"]
+  end
+
+end

data/lib/fiber_connection_pool.rb

@@ -1,7 +1,7 @@
 require 'fiber'
 
 class FiberConnectionPool
-  VERSION = '0.1.1'
+  VERSION = '0.1.2'
 
   attr_accessor :saved_data
 
@@ -30,22 +30,6 @@ class FiberConnectionPool
     @saved_data.delete Fiber.current.object_id
   end
 
-  # Choose first available connection and pass it to the supplied
-  # block. This will block indefinitely until there is an available
-  # connection to service the request.
-  def execute(async,method)
-    f = Fiber.current
-
-    begin
-      conn = acquire(f)
-      retval = yield conn
-      release_backup(f) if !async and method == 'query'
-      retval
-    ensure
-      release(f) if not async
-    end
-  end
-
   ##
   # avoid method_missing for most common methods
   #
@@ -71,62 +55,81 @@ class FiberConnectionPool
 
   private
 
-    # Acquire a lock on a connection and assign it to executing fiber
-    # - if connection is available, pass it back to the calling block
-    # - if pool is full, yield the current fiber until connection is available
-    def acquire(fiber)
-
-      if conn = @available.pop
-        @reserved[fiber.object_id] = conn
-        @reserved_backup[fiber.object_id] = conn
-        conn
-      else
-        Fiber.yield @pending.push fiber
-        acquire(fiber)
+  # Choose first available connection and pass it to the supplied
+  # block. This will block indefinitely until there is an available
+  # connection to service the request.
+  def execute(async,method)
+    f = Fiber.current
+
+    begin
+      conn = acquire(f)
+      retval = yield conn
+      if !@saved_data[Fiber.current.object_id].nil?
+        @saved_data[Fiber.current.object_id]['affected_rows'] = conn.affected_rows
       end
+      release_backup(f) if !async and method == 'query'
+      retval
+    ensure
+      release(f) if not async
     end
+  end
 
-    # Release connection from the backup hash
-    def release_backup(fiber)
-      @reserved_backup.delete(fiber.object_id)
+  # Acquire a lock on a connection and assign it to executing fiber
+  # - if connection is available, pass it back to the calling block
+  # - if pool is full, yield the current fiber until connection is available
+  def acquire(fiber)
+
+    if conn = @available.pop
+      @reserved[fiber.object_id] = conn
+      @reserved_backup[fiber.object_id] = conn
+      conn
+    else
+      Fiber.yield @pending.push fiber
+      acquire(fiber)
     end
+  end
 
-    # Release connection assigned to the supplied fiber and
-    # resume any other pending connections (which will
-    # immediately try to run acquire on the pool)
-    def release(fiber)
-      @available.push(@reserved.delete(fiber.object_id)).compact!
+  # Release connection from the backup hash
+  def release_backup(fiber)
+    @reserved_backup.delete(fiber.object_id)
+  end
 
-      if pending = @pending.shift
-        pending.resume
-      end
+  # Release connection assigned to the supplied fiber and
+  # resume any other pending connections (which will
+  # immediately try to run acquire on the pool)
+  def release(fiber)
+    @available.push(@reserved.delete(fiber.object_id)).compact!
+
+    if pending = @pending.shift
+      pending.resume
     end
+  end
 
-    # Allow the pool to behave as the underlying connection
-    #
-    # If the requesting method begins with "a" prefix, then
-    # hijack the callbacks and errbacks to fire a connection
-    # pool release whenever the request is complete. Otherwise
-    # yield the connection within execute method and release
-    # once it is complete (assumption: fiber will yield until
-    # data is available, or request is complete)
-    #
-    def method_missing(method, *args, &blk)
-      async = (method[0,1] == "a")
-
-      execute(async,method) do |conn|
-        df = conn.send(method, *args, &blk)
-
-        if async
-          fiber = Fiber.current
-          df.callback do
-            release(fiber)
-            release_backup(fiber)
-          end
-          df.errback { release(fiber) }
-        end
+  # Allow the pool to behave as the underlying connection
+  #
+  # If the requesting method begins with "a" prefix, then
+  # hijack the callbacks and errbacks to fire a connection
+  # pool release whenever the request is complete. Otherwise
+  # yield the connection within execute method and release
+  # once it is complete (assumption: fiber will yield until
+  # data is available, or request is complete)
+  #
+  def method_missing(method, *args, &blk)
+    async = (method[0,1] == "a")
+
+    execute(async,method) do |conn|
+      df = conn.send(method, *args, &blk)
 
-        df
+      if async
+        fiber = Fiber.current
+        df.callback do
+          release(fiber)
+          release_backup(fiber)
+        end
+        df.errback { release(fiber) }
       end
+
+      df
     end
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fiber_connection_pool
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.1.2
 platform: ruby
 authors:
 - Ruben Caro
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-08-15 00:00:00.000000000 Z
+date: 2013-08-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -56,6 +56,9 @@ files:
 - README.md
 - Rakefile
 - examples/.placeholder
+- examples/goliath_server/Gemfile
+- examples/goliath_server/config/main.rb
+- examples/goliath_server/main.rb
 - examples/reel_server/Gemfile
 - examples/reel_server/main.rb
 - fiber_connection_pool.gemspec