fiber_connection_pool 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 6d5107300e5f00c78557fabe1ef21a964bc0d929
4
- data.tar.gz: 06f8d340506aba8db7035d0c5f17fb6ddc4e563b
3
+ metadata.gz: 654c0a80fecfc77daa838a3a0611ad534e4413fc
4
+ data.tar.gz: cbb3bd2ccb7f50fc534d98f976cfe2ec42c87302
5
5
  SHA512:
6
- metadata.gz: d258dabb23ce59218aa85cb5271a6efebc91817bd74748f2ad9933001a5b160f8373c38c2768f26694f3ff5bde7a1331455f6f515fd7af7ebfe8c24af81bc245
7
- data.tar.gz: 03f4d89e6235a16501a1998752e6693d7b8e8cd824a5d7c49d956482c667549fd73d59ea0d3b7f9028d0023c67cd5bbf1c78c7359732c36a9874693ddbe285eb
6
+ metadata.gz: 4f01cfc453ecfc286069f18ac34545c079a64214b2e99dee2a2ef3ef6c1414e72c2da586d653d23e61a35d363c62f28bcd45f160bab9d5d93323cf92e2d9765a
7
+ data.tar.gz: c0995f388b1552a246cc9f68ab1ebcc929cac66a088b0aa1a4d0743bae84b5f34f853e34b0164d1c323e458e81be4a249c0d0d52fe7fc9347b8e610ac76af2a2
data/README.md CHANGED
@@ -6,17 +6,20 @@ fiber_connection_pool
6
6
 
7
7
  Fiber-based generic connection pool
8
8
 
9
+ A connection pool meant to be used inside a Fiber-based _reactor_,
10
+ such as any [EventMachine](https://github.com/eventmachine/eventmachine)
11
+ or [Celluloid](http://celluloid.io/) server.
12
+
9
13
  Widely based on `ConnectionPool`
10
14
  from [em-synchrony](https://github.com/igrigorik/em-synchrony) gem, and
11
15
  some things borrowed also from
12
16
  threaded [connection_pool](https://github.com/mperham/connection_pool) gem.
13
-
14
17
  Used in production environments
15
18
  with [Goliath](https://github.com/postrank-labs/goliath)
16
- ([EventMachine](https://github.com/eventmachine/eventmachine) based) servers,
19
+ (EventMachine based) servers,
17
20
  and in promising experiments with
18
21
  [Reel](https://github.com/celluloid/reel)
19
- ([Celluloid](http://celluloid.io/) based) servers.
22
+ (Celluloid based) servers.
20
23
 
21
24
  Install
22
25
  ----------------
@@ -33,10 +36,86 @@ Inside of your Ruby program, require FiberConnectionPool with:
33
36
 
34
37
  require 'fiber_connection_pool'
35
38
 
39
+ How It Works
40
+ -------------------
41
+
42
+ ``` ruby
43
+ pool = FiberConnectionPool.new(:size => 5){ MyFancyConnection.new }
44
+ ```
45
+
46
+ It just keeps an array (the internal pool) holding the result of running
47
+ the given block _size_ times. Inside the reactor loop (either EventMachine's or Celluloid's),
48
+ each request is wrapped on a Fiber, and then `pool` plays its magic.
49
+
50
+ When a method `query_me` is called on `pool` and it's not one of its own methods,
51
+ then it:
52
+
53
+ 1. reserves one connection from the internal pool and associates it __with the current Fiber__
54
+ 2. if no connection is available, then that Fiber stays on a _pending_ queue, and __is yielded__
55
+ 3. when a connection is available, then the pool calls `query_me` on that `MyFancyConnection` instance
56
+ 4. when `query_me` returns, the reserved instance is released again,
57
+ and the next Fiber on the _pending_ queue __is resumed__
58
+ 5. the return value is sent back to the caller
59
+
60
+ Methods from `MyFancyConnection` instance should yield the fiber before
61
+ perform any blocking IO. That returns control to te underlying reactor,
62
+ that spawns another fiber to process the next request, while the previous
63
+ one is still waiting for the IO response. That new fiber will get its own
64
+ connection from the pool, or else it will yield until there
65
+ is one available.
66
+
67
+ The whole process looks synchronous from the Fiber perspective, _because it is_.
68
+ The Fiber will really block ( _yield_ ) until it gets the result.
69
+
70
+ ``` ruby
71
+ results = pool.query_me(sql)
72
+ puts "I waited for this: #{results}"
73
+ ```
74
+
75
+ The magic resides on the fact that other fibers are being processed while this one is waiting.
76
+
77
+ Not thread-safe
78
+ ------------------
79
+
80
+ `FiberConnectionPool` is not thread-safe right now. You will not be able to use it
81
+ from different threads, as eventually it will try to resume a Fiber that resides
82
+ on a different Thread. That will raise a FiberError( _"calling a fiber across threads"_ ).
83
+ Maybe one day we add that feature too.
84
+
85
+ We have tested it on Goliath servers having one pool on each server instance, and on Reel servers
86
+ having one pool on each Actor thread. Take a look at the `examples` folder for details.
87
+
88
+ MySQL specific
89
+ ------------------
90
+
91
+ By now we have only thought and tested it to be used with MySQL connections.
92
+ For EventMachine by using `Mysql2::EM::Client` from [em-synchrony](https://github.com/igrigorik/em-synchrony).
93
+ And for Celluloid by using a patched version of [ruby-mysql](https://github.com/rubencaro/ruby-mysql).
94
+ We plan on removing any MySQL specific code, so it becomes completely generic. Does not seem so hard to achieve.
95
+
96
+ Reacting to connection failure
97
+ ------------------
98
+
99
+ When the call to a method raises an Exception it will raise as if there was no pool between
100
+ your code and the connetion itself. You can rescue the Exception as usual and
101
+ react as you would do normally.
102
+
103
+ You have to be aware that the connection instance will remain in the pool, and other fibers
104
+ will surely use it. If the Exception you rescued indicates that the connection should be
105
+ recreated, you can call `recreate_connection` passing it a new instance. The instance that
106
+ just failed will be replaced inside the pool by the brand new connection.
107
+
36
108
  Supported Platforms
37
109
  -------------------
38
110
 
39
111
  Used in production environments on Ruby 1.9.3 and 2.0.0.
40
112
  Tested against Ruby 1.9.3, 2.0.0, and rbx-19mode ([See details..](http://travis-ci.org/rubencaro/fiber_connection_pool)).
41
113
 
42
- TODO: sparkling docs
114
+ TODOS
115
+ -------------------
116
+
117
+ * no MySQL-specific code
118
+ * better testing
119
+ * improve reaction to failure
120
+ * better in-code docs
121
+ * make thread-safe
@@ -0,0 +1,5 @@
1
+ source 'https://rubygems.org'
2
+ gem 'goliath'
3
+ gem 'em-synchrony'
4
+ gem 'mysql2'
5
+ gem 'fiber_connection_pool'
@@ -0,0 +1,13 @@
1
+ require 'fiber_connection_pool'
2
+ require 'mysql2'
3
+ require 'em-synchrony'
4
+ require 'em-synchrony/mysql2'
5
+
6
+ config['db'] = FiberConnectionPool.new(:size => 5) do
7
+ Mysql2::EM::Client.new({
8
+ host: 'localhost',
9
+ username: 'user',
10
+ password: 'pass',
11
+ database: 'bogusdb'
12
+ })
13
+ end
@@ -0,0 +1,13 @@
1
+ require 'goliath'
2
+ require 'fiber'
3
+
4
+ class Main < Goliath::API
5
+
6
+ def response(env)
7
+ print '.'
8
+ db.query 'select sleep(2);'
9
+ puts "Done #{Thread.current.to_s}, #{Fiber.current.to_s}"
10
+ [200,{"Content-Type" => "text/html"},"hello, world! #{Time.now.strftime('%T')}"]
11
+ end
12
+
13
+ end
@@ -1,7 +1,7 @@
1
1
  require 'fiber'
2
2
 
3
3
  class FiberConnectionPool
4
- VERSION = '0.1.1'
4
+ VERSION = '0.1.2'
5
5
 
6
6
  attr_accessor :saved_data
7
7
 
@@ -30,22 +30,6 @@ class FiberConnectionPool
30
30
  @saved_data.delete Fiber.current.object_id
31
31
  end
32
32
 
33
- # Choose first available connection and pass it to the supplied
34
- # block. This will block indefinitely until there is an available
35
- # connection to service the request.
36
- def execute(async,method)
37
- f = Fiber.current
38
-
39
- begin
40
- conn = acquire(f)
41
- retval = yield conn
42
- release_backup(f) if !async and method == 'query'
43
- retval
44
- ensure
45
- release(f) if not async
46
- end
47
- end
48
-
49
33
  ##
50
34
  # avoid method_missing for most common methods
51
35
  #
@@ -71,62 +55,81 @@ class FiberConnectionPool
71
55
 
72
56
  private
73
57
 
74
- # Acquire a lock on a connection and assign it to executing fiber
75
- # - if connection is available, pass it back to the calling block
76
- # - if pool is full, yield the current fiber until connection is available
77
- def acquire(fiber)
78
-
79
- if conn = @available.pop
80
- @reserved[fiber.object_id] = conn
81
- @reserved_backup[fiber.object_id] = conn
82
- conn
83
- else
84
- Fiber.yield @pending.push fiber
85
- acquire(fiber)
58
+ # Choose first available connection and pass it to the supplied
59
+ # block. This will block indefinitely until there is an available
60
+ # connection to service the request.
61
+ def execute(async,method)
62
+ f = Fiber.current
63
+
64
+ begin
65
+ conn = acquire(f)
66
+ retval = yield conn
67
+ if !@saved_data[Fiber.current.object_id].nil?
68
+ @saved_data[Fiber.current.object_id]['affected_rows'] = conn.affected_rows
86
69
  end
70
+ release_backup(f) if !async and method == 'query'
71
+ retval
72
+ ensure
73
+ release(f) if not async
87
74
  end
75
+ end
88
76
 
89
- # Release connection from the backup hash
90
- def release_backup(fiber)
91
- @reserved_backup.delete(fiber.object_id)
77
+ # Acquire a lock on a connection and assign it to executing fiber
78
+ # - if connection is available, pass it back to the calling block
79
+ # - if pool is full, yield the current fiber until connection is available
80
+ def acquire(fiber)
81
+
82
+ if conn = @available.pop
83
+ @reserved[fiber.object_id] = conn
84
+ @reserved_backup[fiber.object_id] = conn
85
+ conn
86
+ else
87
+ Fiber.yield @pending.push fiber
88
+ acquire(fiber)
92
89
  end
90
+ end
93
91
 
94
- # Release connection assigned to the supplied fiber and
95
- # resume any other pending connections (which will
96
- # immediately try to run acquire on the pool)
97
- def release(fiber)
98
- @available.push(@reserved.delete(fiber.object_id)).compact!
92
+ # Release connection from the backup hash
93
+ def release_backup(fiber)
94
+ @reserved_backup.delete(fiber.object_id)
95
+ end
99
96
 
100
- if pending = @pending.shift
101
- pending.resume
102
- end
97
+ # Release connection assigned to the supplied fiber and
98
+ # resume any other pending connections (which will
99
+ # immediately try to run acquire on the pool)
100
+ def release(fiber)
101
+ @available.push(@reserved.delete(fiber.object_id)).compact!
102
+
103
+ if pending = @pending.shift
104
+ pending.resume
103
105
  end
106
+ end
104
107
 
105
- # Allow the pool to behave as the underlying connection
106
- #
107
- # If the requesting method begins with "a" prefix, then
108
- # hijack the callbacks and errbacks to fire a connection
109
- # pool release whenever the request is complete. Otherwise
110
- # yield the connection within execute method and release
111
- # once it is complete (assumption: fiber will yield until
112
- # data is available, or request is complete)
113
- #
114
- def method_missing(method, *args, &blk)
115
- async = (method[0,1] == "a")
116
-
117
- execute(async,method) do |conn|
118
- df = conn.send(method, *args, &blk)
119
-
120
- if async
121
- fiber = Fiber.current
122
- df.callback do
123
- release(fiber)
124
- release_backup(fiber)
125
- end
126
- df.errback { release(fiber) }
127
- end
108
+ # Allow the pool to behave as the underlying connection
109
+ #
110
+ # If the requesting method begins with "a" prefix, then
111
+ # hijack the callbacks and errbacks to fire a connection
112
+ # pool release whenever the request is complete. Otherwise
113
+ # yield the connection within execute method and release
114
+ # once it is complete (assumption: fiber will yield until
115
+ # data is available, or request is complete)
116
+ #
117
+ def method_missing(method, *args, &blk)
118
+ async = (method[0,1] == "a")
119
+
120
+ execute(async,method) do |conn|
121
+ df = conn.send(method, *args, &blk)
128
122
 
129
- df
123
+ if async
124
+ fiber = Fiber.current
125
+ df.callback do
126
+ release(fiber)
127
+ release_backup(fiber)
128
+ end
129
+ df.errback { release(fiber) }
130
130
  end
131
+
132
+ df
131
133
  end
134
+ end
132
135
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fiber_connection_pool
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ruben Caro
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2013-08-15 00:00:00.000000000 Z
12
+ date: 2013-08-16 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: minitest
@@ -56,6 +56,9 @@ files:
56
56
  - README.md
57
57
  - Rakefile
58
58
  - examples/.placeholder
59
+ - examples/goliath_server/Gemfile
60
+ - examples/goliath_server/config/main.rb
61
+ - examples/goliath_server/main.rb
59
62
  - examples/reel_server/Gemfile
60
63
  - examples/reel_server/main.rb
61
64
  - fiber_connection_pool.gemspec