connection_pool 2.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: cf8177bf07f6c8e68b13f28416f6ccc0c64803af
4
+ data.tar.gz: ff9c836ce5576d4d0cb1edf8d858a761f982e437
5
+ SHA512:
6
+ metadata.gz: 73bda4d8d0b3cc9daef5c93bb98b7649d811bd5c8f467e25c3aaf7d3353541225e7b30e1efd481a14e45f5c56b5fc846d49dabdd9a167702a71c4a9631a9b6a2
7
+ data.tar.gz: a331275f67b3635e09792e646113e32bdc74254a2cdff032e972c6f2ebedd6391a1e508c36be7c4e33b40e2a07efe029a2ad0267c3ba4dc20b22813df86427bb
@@ -0,0 +1,4 @@
1
+ *.gem
2
+ .bundle
3
+ Gemfile.lock
4
+ pkg/*
@@ -0,0 +1,10 @@
1
+ ---
2
+ sudo: false
3
+ cache: bundler
4
+ language: ruby
5
+ rvm:
6
+ - 2.2.9
7
+ - 2.3.6
8
+ - 2.4.3
9
+ - 2.5.0
10
+ - jruby
@@ -0,0 +1,123 @@
1
+ connection\_pool changelog
2
+ ---------------------------
3
+
4
+ 2.2.2
5
+ ------
6
+
7
+ - Add pool `size` and `available` accessors for metrics and monitoring
8
+ purposes [#97, robholland]
9
+
10
+ 2.2.1
11
+ ------
12
+
13
+ - Allow CP::Wrapper to use an existing pool [#87, etiennebarrie]
14
+ - Use monotonic time for more accurate timeouts [#84, jdantonio]
15
+
16
+ 2.2.0
17
+ ------
18
+
19
+ - Rollback `Timeout` handling introduced in 2.1.1 and 2.1.2. It seems
20
+ impossible to safely work around the issue. Please never, ever use
21
+ `Timeout.timeout` in your code or you will see rare but mysterious bugs. [#75]
22
+
23
+ 2.1.3
24
+ ------
25
+
26
+ - Don't increment created count until connection is successfully
27
+ created. [mylesmegyesi, #73]
28
+
29
+ 2.1.2
30
+ ------
31
+
32
+ - The connection\_pool will now close any connections which respond to
33
+ `close` (Dalli) or `disconnect!` (Redis). This ensures discarded connections
34
+ from the fix in 2.1.1 are torn down ASAP and don't linger open.
35
+
36
+
37
+ 2.1.1
38
+ ------
39
+
40
+ - Work around a subtle race condition with code which uses `Timeout.timeout` and
41
+ checks out a connection within the timeout block. This might cause
42
+ connections to get into a bad state and raise very odd errors. [tamird, #67]
43
+
44
+
45
+ 2.1.0
46
+ ------
47
+
48
+ - Refactoring to better support connection pool subclasses [drbrain,
49
+ #55]
50
+ - `with` should return value of the last expression [#59]
51
+
52
+
53
+ 2.0.0
54
+ -----
55
+
56
+ - The connection pool is now lazy. Connections are created as needed
57
+ and retained until the pool is shut down. [drbrain, #52]
58
+
59
+ 1.2.0
60
+ -----
61
+
62
+ - Add `with(options)` and `checkout(options)`. [mattcamuto]
63
+ Allows the caller to override the pool timeout.
64
+ ```ruby
65
+ @pool.with(:timeout => 2) do |conn|
66
+ end
67
+ ```
68
+
69
+ 1.1.0
70
+ -----
71
+
72
+ - New `#shutdown` method (simao)
73
+
74
+ This method accepts a block and calls the block for each
75
+ connection in the pool. After calling this method, trying to get a
76
+ connection from the pool raises `PoolShuttingDownError`.
77
+
78
+ 1.0.0
79
+ -----
80
+
81
+ - `#with_connection` is now gone in favor of `#with`.
82
+
83
+ - We no longer pollute the top level namespace with our internal
84
+ `TimedStack` class.
85
+
86
+ 0.9.3
87
+ --------
88
+
89
+ - `#with_connection` is now deprecated in favor of `#with`.
90
+
91
+ A warning will be issued in the 0.9 series and the method will be
92
+ removed in 1.0.
93
+
94
+ - We now reuse objects when possible.
95
+
96
+ This means that under no contention, the same object will be checked
97
+ out from the pool after subsequent calls to `ConnectionPool#with`.
98
+
99
+ This change should have no impact on end user performance. If
100
+ anything, it should be an improvement, depending on what objects you
101
+ are pooling.
102
+
103
+ 0.9.2
104
+ --------
105
+
106
+ - Fix reentrant checkout leading to early checkin.
107
+
108
+ 0.9.1
109
+ --------
110
+
111
+ - Fix invalid superclass in version.rb
112
+
113
+ 0.9.0
114
+ --------
115
+
116
+ - Move method\_missing magic into ConnectionPool::Wrapper (djanowski)
117
+ - Remove BasicObject superclass (djanowski)
118
+
119
+ 0.1.0
120
+ --------
121
+
122
+ - More precise timeouts and better error message
123
+ - ConnectionPool now subclasses BasicObject so `method_missing` is more effective.
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "https://rubygems.org"
2
+
3
+ gemspec(development_group: :runtime)
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2011 Mike Perham
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,107 @@
1
+ connection\_pool
2
+ =================
3
+ [![Build Status](https://travis-ci.org/mperham/connection_pool.svg)](https://travis-ci.org/mperham/connection_pool)
4
+
5
+ Generic connection pooling for Ruby.
6
+
7
+ MongoDB has its own connection pool. ActiveRecord has its own connection pool.
8
+ This is a generic connection pool that can be used with anything, e.g. Redis,
9
+ Dalli and other Ruby network clients.
10
+
11
+
12
+ Usage
13
+ -----
14
+
15
+ Create a pool of objects to share amongst the fibers or threads in your Ruby
16
+ application:
17
+
18
+ ``` ruby
19
+ $memcached = ConnectionPool.new(size: 5, timeout: 5) { Dalli::Client.new }
20
+ ```
21
+
22
+ Then use the pool in your application:
23
+
24
+ ``` ruby
25
+ $memcached.with do |conn|
26
+ conn.get('some-count')
27
+ end
28
+ ```
29
+
30
+ If all the objects in the connection pool are in use, `with` will block
31
+ until one becomes available. If no object is available within `:timeout` seconds,
32
+ `with` will raise a `Timeout::Error`.
33
+
34
+ Optionally, you can specify a timeout override using the with-block semantics:
35
+
36
+ ``` ruby
37
+ $memcached.with(timeout: 2.0) do |conn|
38
+ conn.get('some-count')
39
+ end
40
+ ```
41
+
42
+ This will only modify the resource-get timeout for this particular
43
+ invocation. This is useful if you want to fail-fast on certain non critical
44
+ sections when a resource is not available, or conversely if you are comfortable
45
+ blocking longer on a particular resource. This is not implemented in the below
46
+ `ConnectionPool::Wrapper` class.
47
+
48
+ ## Migrating to a Connection Pool
49
+
50
+ You can use `ConnectionPool::Wrapper` to wrap a single global connection,
51
+ making it easier to migrate existing connection code over time:
52
+
53
+ ``` ruby
54
+ $redis = ConnectionPool::Wrapper.new(size: 5, timeout: 3) { Redis.connect }
55
+ $redis.sadd('foo', 1)
56
+ $redis.smembers('foo')
57
+ ```
58
+
59
+ The wrapper uses `method_missing` to checkout a connection, run the requested
60
+ method and then immediately check the connection back into the pool. It's
61
+ **not** high-performance so you'll want to port your performance sensitive code
62
+ to use `with` as soon as possible.
63
+
64
+ ``` ruby
65
+ $redis.with do |conn|
66
+ conn.sadd('foo', 1)
67
+ conn.smembers('foo')
68
+ end
69
+ ```
70
+
71
+ Once you've ported your entire system to use `with`, you can simply remove
72
+ `Wrapper` and use the simpler and faster `ConnectionPool`.
73
+
74
+
75
+ ## Shutdown
76
+
77
+ You can shut down a ConnectionPool instance once it should no longer be used.
78
+ Further checkout attempts will immediately raise an error but existing checkouts
79
+ will work.
80
+
81
+ ```ruby
82
+ cp = ConnectionPool.new { Redis.new }
83
+ cp.shutdown { |conn| conn.quit }
84
+ ```
85
+
86
+ Shutting down a connection pool will block until all connections are checked in and closed.
87
+ **Note that shutting down is completely optional**; Ruby's garbage collector will reclaim
88
+ unreferenced pools under normal circumstances.
89
+
90
+
91
+ Notes
92
+ -----
93
+
94
+ - Connections are lazily created as needed.
95
+ - There is no provision for repairing or checking the health of a connection;
96
+ connections should be self-repairing. This is true of the Dalli and Redis
97
+ clients.
98
+ - **WARNING**: Don't ever use `Timeout.timeout` in your Ruby code or you will see
99
+ occasional silent corruption and mysterious errors. The Timeout API is unsafe
100
+ and cannot be used correctly, ever. Use proper socket timeout options as
101
+ exposed by Net::HTTP, Redis, Dalli, etc.
102
+
103
+
104
+ Author
105
+ ------
106
+
107
+ Mike Perham, [@mperham](https://twitter.com/mperham), <http://mikeperham.com>
@@ -0,0 +1,6 @@
1
+ require 'bundler/gem_tasks'
2
+
3
+ require 'rake/testtask'
4
+ Rake::TestTask.new
5
+
6
+ task :default => :test
@@ -0,0 +1,21 @@
1
+ # -*- encoding: utf-8 -*-
2
+ require "./lib/connection_pool/version"
3
+
4
+ Gem::Specification.new do |s|
5
+ s.name = "connection_pool"
6
+ s.version = ConnectionPool::VERSION
7
+ s.platform = Gem::Platform::RUBY
8
+ s.authors = ["Mike Perham", "Damian Janowski"]
9
+ s.email = ["mperham@gmail.com", "damian@educabilia.com"]
10
+ s.homepage = "https://github.com/mperham/connection_pool"
11
+ s.description = s.summary = %q{Generic connection pool for Ruby}
12
+
13
+ s.files = `git ls-files`.split("\n")
14
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
15
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
16
+ s.require_paths = ["lib"]
17
+ s.license = "MIT"
18
+ s.add_development_dependency 'bundler'
19
+ s.add_development_dependency 'minitest', '>= 5.0.0'
20
+ s.add_development_dependency 'rake'
21
+ end
@@ -0,0 +1,161 @@
1
+ require_relative 'connection_pool/version'
2
+ require_relative 'connection_pool/timed_stack'
3
+
4
+
5
+ # Generic connection pool class for e.g. sharing a limited number of network connections
6
+ # among many threads. Note: Connections are lazily created.
7
+ #
8
+ # Example usage with block (faster):
9
+ #
10
+ # @pool = ConnectionPool.new { Redis.new }
11
+ #
12
+ # @pool.with do |redis|
13
+ # redis.lpop('my-list') if redis.llen('my-list') > 0
14
+ # end
15
+ #
16
+ # Using optional timeout override (for that single invocation)
17
+ #
18
+ # @pool.with(timeout: 2.0) do |redis|
19
+ # redis.lpop('my-list') if redis.llen('my-list') > 0
20
+ # end
21
+ #
22
+ # Example usage replacing an existing connection (slower):
23
+ #
24
+ # $redis = ConnectionPool.wrap { Redis.new }
25
+ #
26
+ # def do_work
27
+ # $redis.lpop('my-list') if $redis.llen('my-list') > 0
28
+ # end
29
+ #
30
+ # Accepts the following options:
31
+ # - :size - number of connections to pool, defaults to 5
32
+ # - :timeout - amount of time to wait for a connection if none currently available, defaults to 5 seconds
33
+ #
34
+ class ConnectionPool
35
+ DEFAULTS = {size: 5, timeout: 5}
36
+
37
+ class Error < RuntimeError
38
+ end
39
+
40
+ def self.wrap(options, &block)
41
+ Wrapper.new(options, &block)
42
+ end
43
+
44
+ def initialize(options = {}, &block)
45
+ raise ArgumentError, 'Connection pool requires a block' unless block
46
+
47
+ options = DEFAULTS.merge(options)
48
+
49
+ @size = options.fetch(:size)
50
+ @timeout = options.fetch(:timeout)
51
+
52
+ @available = TimedStack.new(@size, &block)
53
+ @key = :"current-#{@available.object_id}"
54
+ @key_count = :"current-#{@available.object_id}-count"
55
+ end
56
+
57
+ if Thread.respond_to?(:handle_interrupt)
58
+
59
+ # MRI
60
+ def with(options = {})
61
+ Thread.handle_interrupt(Exception => :never) do
62
+ conn = checkout(options)
63
+ begin
64
+ Thread.handle_interrupt(Exception => :immediate) do
65
+ yield conn
66
+ end
67
+ ensure
68
+ checkin
69
+ end
70
+ end
71
+ end
72
+
73
+ else
74
+
75
+ # jruby 1.7.x
76
+ def with(options = {})
77
+ conn = checkout(options)
78
+ begin
79
+ yield conn
80
+ ensure
81
+ checkin
82
+ end
83
+ end
84
+
85
+ end
86
+
87
+ def checkout(options = {})
88
+ if ::Thread.current[@key]
89
+ ::Thread.current[@key_count]+= 1
90
+ ::Thread.current[@key]
91
+ else
92
+ ::Thread.current[@key_count]= 1
93
+ ::Thread.current[@key]= @available.pop(options[:timeout] || @timeout)
94
+ end
95
+ end
96
+
97
+ def checkin
98
+ if ::Thread.current[@key]
99
+ if ::Thread.current[@key_count] == 1
100
+ @available.push(::Thread.current[@key])
101
+ ::Thread.current[@key]= nil
102
+ else
103
+ ::Thread.current[@key_count]-= 1
104
+ end
105
+ else
106
+ raise ConnectionPool::Error, 'no connections are checked out'
107
+ end
108
+
109
+ nil
110
+ end
111
+
112
+ def shutdown(&block)
113
+ @available.shutdown(&block)
114
+ end
115
+
116
+ # Size of this connection pool
117
+ def size
118
+ @size
119
+ end
120
+
121
+ # Number of pool entries available for checkout at this instant.
122
+ def available
123
+ @available.length
124
+ end
125
+
126
+ private
127
+
128
+ class Wrapper < ::BasicObject
129
+ METHODS = [:with, :pool_shutdown]
130
+
131
+ def initialize(options = {}, &block)
132
+ @pool = options.fetch(:pool) { ::ConnectionPool.new(options, &block) }
133
+ end
134
+
135
+ def with(&block)
136
+ @pool.with(&block)
137
+ end
138
+
139
+ def pool_shutdown(&block)
140
+ @pool.shutdown(&block)
141
+ end
142
+
143
+ def pool_size
144
+ @pool.size
145
+ end
146
+
147
+ def pool_available
148
+ @pool.available
149
+ end
150
+
151
+ def respond_to?(id, *args)
152
+ METHODS.include?(id) || with { |c| c.respond_to?(id, *args) }
153
+ end
154
+
155
+ def method_missing(name, *args, &block)
156
+ with do |connection|
157
+ connection.send(name, *args, &block)
158
+ end
159
+ end
160
+ end
161
+ end