connection_pool 2.5.0 → 3.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Changes.md +52 -0
- data/README.md +49 -34
- data/connection_pool.gemspec +16 -5
- data/lib/connection_pool/fork.rb +40 -0
- data/lib/connection_pool/timed_stack.rb +63 -51
- data/lib/connection_pool/version.rb +1 -1
- data/lib/connection_pool/wrapper.rb +15 -28
- data/lib/connection_pool.rb +72 -73
- metadata +12 -10
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a064d41333b8b92fcb23617701011f1b34e6348b324048ca16e5cb758d31f05f
|
|
4
|
+
data.tar.gz: 7ca1cc56ff7d020f2b7f2cee01b2008502faadf563dad010c2b8eee4a0b83dcd
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 1c7554d540f6aefcd356154246a1cdcab5e4ddea745e9a580261fba0635e6b711a9e0781691503f20aca7faaf982e2ba8f5f48cc71ff2a4f67053039e947ba3d
|
|
7
|
+
data.tar.gz: 431dcf74c39f6a9db1503290a227f5780c4b03e392ef25709245e63d0ca46a2f0f6cc0792a5241e615eafd92663aa38c1165aae99f6dffb0783f5b442846582f
|
data/Changes.md
CHANGED
|
@@ -1,5 +1,57 @@
|
|
|
1
1
|
# connection_pool Changelog
|
|
2
2
|
|
|
3
|
+
3.0.2
|
|
4
|
+
------
|
|
5
|
+
|
|
6
|
+
- Support :name keyword for backwards compatibility [#210]
|
|
7
|
+
|
|
8
|
+
3.0.1
|
|
9
|
+
------
|
|
10
|
+
|
|
11
|
+
- Add missing `fork.rb` to gemspec.
|
|
12
|
+
|
|
13
|
+
3.0.0
|
|
14
|
+
------
|
|
15
|
+
|
|
16
|
+
- **BREAKING CHANGES** `ConnectionPool` and `ConnectionPool::TimedStack` now
|
|
17
|
+
use keyword arguments rather than positional arguments everywhere. Expected impact is minimal as most people use the `with` API, which is unchanged.
|
|
18
|
+
```ruby
|
|
19
|
+
pool = ConnectionPool.new(size: 5, timeout: 5)
|
|
20
|
+
pool.checkout(1) # 2.x
|
|
21
|
+
pool.reap(30) # 2.x
|
|
22
|
+
pool.checkout(timeout: 1) # 3.x
|
|
23
|
+
pool.reap(idle_seconds: 30) # 3.x
|
|
24
|
+
```
|
|
25
|
+
- Dropped support for Ruby <3.2.0
|
|
26
|
+
|
|
27
|
+
2.5.5
|
|
28
|
+
------
|
|
29
|
+
|
|
30
|
+
- Support `ConnectionPool::TimedStack#pop(exception: false)` [#207]
|
|
31
|
+
to avoid using exceptions as control flow.
|
|
32
|
+
|
|
33
|
+
2.5.4
|
|
34
|
+
------
|
|
35
|
+
|
|
36
|
+
- Add ability to remove a broken connection from the pool [#204, womblep]
|
|
37
|
+
|
|
38
|
+
2.5.3
|
|
39
|
+
------
|
|
40
|
+
|
|
41
|
+
- Fix TruffleRuby/JRuby crash [#201]
|
|
42
|
+
|
|
43
|
+
2.5.2
|
|
44
|
+
------
|
|
45
|
+
|
|
46
|
+
- Rollback inadvertant change to `auto_reload_after_fork` default. [#200]
|
|
47
|
+
|
|
48
|
+
2.5.1
|
|
49
|
+
------
|
|
50
|
+
|
|
51
|
+
- Pass options to TimedStack in `checkout` [#195]
|
|
52
|
+
- Optimize connection lookup [#196]
|
|
53
|
+
- Fixes for use with Ractors
|
|
54
|
+
|
|
3
55
|
2.5.0
|
|
4
56
|
------
|
|
5
57
|
|
data/README.md
CHANGED
|
@@ -38,7 +38,7 @@ connection pool and a raw client.
|
|
|
38
38
|
$redis.then { |r| r.set 'foo' 'bar' }
|
|
39
39
|
```
|
|
40
40
|
|
|
41
|
-
Optionally, you can specify a timeout override
|
|
41
|
+
Optionally, you can specify a timeout override:
|
|
42
42
|
|
|
43
43
|
``` ruby
|
|
44
44
|
$memcached.with(timeout: 2.0) do |conn|
|
|
@@ -46,11 +46,9 @@ $memcached.with(timeout: 2.0) do |conn|
|
|
|
46
46
|
end
|
|
47
47
|
```
|
|
48
48
|
|
|
49
|
-
This will only modify the
|
|
50
|
-
invocation.
|
|
49
|
+
This will only modify the timeout for this particular invocation.
|
|
51
50
|
This is useful if you want to fail-fast on certain non-critical
|
|
52
51
|
sections when a resource is not available, or conversely if you are comfortable blocking longer on a particular resource.
|
|
53
|
-
This is not implemented in the `ConnectionPool::Wrapper` class.
|
|
54
52
|
|
|
55
53
|
## Migrating to a Connection Pool
|
|
56
54
|
|
|
@@ -72,7 +70,7 @@ $redis.with do |conn|
|
|
|
72
70
|
end
|
|
73
71
|
```
|
|
74
72
|
|
|
75
|
-
Once you've ported your entire system to use `with`, you can
|
|
73
|
+
Once you've ported your entire system to use `with`, you can remove `::Wrapper` and use `ConnectionPool` directly.
|
|
76
74
|
|
|
77
75
|
|
|
78
76
|
## Shutdown
|
|
@@ -90,41 +88,50 @@ Shutting down a connection pool will block until all connections are checked in
|
|
|
90
88
|
|
|
91
89
|
## Reload
|
|
92
90
|
|
|
93
|
-
You can reload a ConnectionPool instance
|
|
94
|
-
|
|
91
|
+
You can reload a ConnectionPool instance if it is necessary to close all existing connections and continue to use the pool.
|
|
92
|
+
ConnectionPool will automatically reload if the process is forked.
|
|
93
|
+
Use `auto_reload_after_fork: false` if you don't want this behavior.
|
|
95
94
|
|
|
96
95
|
```ruby
|
|
97
|
-
cp = ConnectionPool.new { Redis.new }
|
|
98
|
-
cp.reload { |conn| conn.quit }
|
|
96
|
+
cp = ConnectionPool.new(auto_reload_after_fork: false) { Redis.new }
|
|
97
|
+
cp.reload { |conn| conn.quit } # reload manually
|
|
99
98
|
cp.with { |conn| conn.get('some-count') }
|
|
100
99
|
```
|
|
101
100
|
|
|
102
|
-
Like `shutdown`,
|
|
101
|
+
Like `shutdown`, `reload` will block until all connections are checked in and closed.
|
|
103
102
|
|
|
104
103
|
## Reap
|
|
105
104
|
|
|
106
|
-
You can reap
|
|
105
|
+
You can call `reap` periodically on the ConnectionPool instance to close connections that were created but have not been used for a certain amount of time. This can be useful in environments where connections are expensive.
|
|
107
106
|
|
|
108
|
-
You can specify how many seconds the connections have to be idle for them to be reaped.
|
|
109
|
-
Defaults to 60 seconds.
|
|
107
|
+
You can specify how many seconds the connections have to be idle for them to be reaped, defaulting to 60 seconds.
|
|
110
108
|
|
|
111
109
|
```ruby
|
|
112
110
|
cp = ConnectionPool.new { Redis.new }
|
|
113
|
-
|
|
111
|
+
|
|
112
|
+
# Start a reaper thread to reap connections that have been
|
|
113
|
+
# idle more than 300 seconds (5 minutes)
|
|
114
|
+
Thread.new do
|
|
115
|
+
loop do
|
|
116
|
+
cp.reap(idle_seconds: 300, &:close)
|
|
117
|
+
sleep 30
|
|
118
|
+
end
|
|
119
|
+
end
|
|
114
120
|
```
|
|
115
121
|
|
|
116
|
-
|
|
122
|
+
## Discarding Connections
|
|
117
123
|
|
|
118
|
-
You can
|
|
124
|
+
You can discard connections in the ConnectionPool instance to remove connections that are broken and can't be repaired.
|
|
125
|
+
It can only be done inside the block passed to `with`.
|
|
126
|
+
Takes an optional block that will be executed with the connection.
|
|
119
127
|
|
|
120
128
|
```ruby
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
sleep 300
|
|
129
|
+
pool.with do |conn|
|
|
130
|
+
begin
|
|
131
|
+
conn.execute("SELECT 1")
|
|
132
|
+
rescue SomeConnectionError
|
|
133
|
+
pool.discard_current_connection(&:close) # remove the connection from the pool
|
|
134
|
+
raise
|
|
128
135
|
end
|
|
129
136
|
end
|
|
130
137
|
```
|
|
@@ -148,20 +155,28 @@ end
|
|
|
148
155
|
cp.idle # => 1
|
|
149
156
|
```
|
|
150
157
|
|
|
151
|
-
|
|
152
|
-
|
|
158
|
+
## Upgrading from ConnectionPool 2
|
|
159
|
+
|
|
160
|
+
* Support for Ruby <3.2 has been removed.
|
|
161
|
+
* ConnectionPool's APIs now consistently use keyword arguments everywhere.
|
|
162
|
+
Positional arguments must be converted to keywords:
|
|
163
|
+
```ruby
|
|
164
|
+
pool = ConnectionPool.new(size: 5, timeout: 5)
|
|
165
|
+
pool.checkout(1) # 2.x
|
|
166
|
+
pool.reap(30) # 2.x
|
|
167
|
+
pool.checkout(timeout: 1) # 3.x
|
|
168
|
+
pool.reap(idle_seconds: 30) # 3.x
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
## Notes
|
|
153
172
|
|
|
154
173
|
- Connections are lazily created as needed.
|
|
155
|
-
-
|
|
156
|
-
connections should be self-repairing. This is true of the Dalli and Redis
|
|
157
|
-
clients.
|
|
158
|
-
- **WARNING**: Don't ever use `Timeout.timeout` in your Ruby code or you will see
|
|
174
|
+
- **WARNING**: Avoid `Timeout.timeout` in your Ruby code or you can see
|
|
159
175
|
occasional silent corruption and mysterious errors. The Timeout API is unsafe
|
|
160
|
-
and
|
|
161
|
-
|
|
176
|
+
and dangerous to use. Use proper socket timeout options as exposed by
|
|
177
|
+
Net::HTTP, Redis, Dalli, etc.
|
|
162
178
|
|
|
163
179
|
|
|
164
|
-
Author
|
|
165
|
-
------
|
|
180
|
+
## Author
|
|
166
181
|
|
|
167
|
-
Mike Perham, [@getajobmike](https://
|
|
182
|
+
Mike Perham, [@getajobmike](https://ruby.social/@getajobmike), <https://www.mikeperham.com>
|
data/connection_pool.gemspec
CHANGED
|
@@ -10,15 +10,26 @@ Gem::Specification.new do |s|
|
|
|
10
10
|
s.description = s.summary = "Generic connection pool for Ruby"
|
|
11
11
|
|
|
12
12
|
s.files = ["Changes.md", "LICENSE", "README.md", "connection_pool.gemspec",
|
|
13
|
-
"lib/connection_pool.rb",
|
|
14
|
-
"lib/connection_pool/
|
|
13
|
+
"lib/connection_pool.rb",
|
|
14
|
+
"lib/connection_pool/timed_stack.rb",
|
|
15
|
+
"lib/connection_pool/version.rb",
|
|
16
|
+
"lib/connection_pool/fork.rb",
|
|
17
|
+
"lib/connection_pool/wrapper.rb"]
|
|
15
18
|
s.executables = []
|
|
16
19
|
s.require_paths = ["lib"]
|
|
17
20
|
s.license = "MIT"
|
|
21
|
+
|
|
22
|
+
s.required_ruby_version = ">= 3.2.0"
|
|
18
23
|
s.add_development_dependency "bundler"
|
|
19
|
-
s.add_development_dependency "
|
|
24
|
+
s.add_development_dependency "maxitest"
|
|
20
25
|
s.add_development_dependency "rake"
|
|
21
|
-
s.required_ruby_version = ">= 2.5.0"
|
|
22
26
|
|
|
23
|
-
s.metadata = {
|
|
27
|
+
s.metadata = {
|
|
28
|
+
"bug_tracker_uri" => "https://github.com/mperham/connection_pool/issues",
|
|
29
|
+
"documentation_uri" => "https://github.com/mperham/connection_pool/wiki",
|
|
30
|
+
"changelog_uri" => "https://github.com/mperham/connection_pool/blob/main/Changes.md",
|
|
31
|
+
"source_code_uri" => "https://github.com/mperham/connection_pool",
|
|
32
|
+
"homepage_uri" => "https://github.com/mperham/connection_pool",
|
|
33
|
+
"rubygems_mfa_required" => "true"
|
|
34
|
+
}
|
|
24
35
|
end
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
class ConnectionPool
|
|
2
|
+
if Process.respond_to?(:fork)
|
|
3
|
+
INSTANCES = ObjectSpace::WeakMap.new
|
|
4
|
+
private_constant :INSTANCES
|
|
5
|
+
|
|
6
|
+
def self.after_fork
|
|
7
|
+
INSTANCES.each_value do |pool|
|
|
8
|
+
# We're in after_fork, so we know all other threads are dead.
|
|
9
|
+
# All we need to do is ensure the main thread doesn't have a
|
|
10
|
+
# checked out connection
|
|
11
|
+
pool.checkin(force: true)
|
|
12
|
+
pool.reload do |connection|
|
|
13
|
+
# Unfortunately we don't know what method to call to close the connection,
|
|
14
|
+
# so we try the most common one.
|
|
15
|
+
connection.close if connection.respond_to?(:close)
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
nil
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
module ForkTracker
|
|
22
|
+
def _fork
|
|
23
|
+
pid = super
|
|
24
|
+
if pid == 0
|
|
25
|
+
ConnectionPool.after_fork
|
|
26
|
+
end
|
|
27
|
+
pid
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
Process.singleton_class.prepend(ForkTracker)
|
|
31
|
+
else
|
|
32
|
+
# JRuby, et al
|
|
33
|
+
INSTANCES = nil
|
|
34
|
+
private_constant :INSTANCES
|
|
35
|
+
|
|
36
|
+
def self.after_fork
|
|
37
|
+
# noop
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
end
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
##
|
|
2
2
|
# The TimedStack manages a pool of homogeneous connections (or any resource
|
|
3
|
-
# you wish to manage).
|
|
3
|
+
# you wish to manage). Connections are created lazily up to a given maximum
|
|
4
4
|
# number.
|
|
5
|
-
|
|
5
|
+
#
|
|
6
6
|
# Examples:
|
|
7
7
|
#
|
|
8
|
-
# ts = TimedStack.new(1) { MyConnection.new }
|
|
8
|
+
# ts = TimedStack.new(size: 1) { MyConnection.new }
|
|
9
9
|
#
|
|
10
10
|
# # fetch a connection
|
|
11
11
|
# conn = ts.pop
|
|
@@ -16,15 +16,13 @@
|
|
|
16
16
|
# conn = ts.pop
|
|
17
17
|
# ts.pop timeout: 5
|
|
18
18
|
# #=> raises ConnectionPool::TimeoutError after 5 seconds
|
|
19
|
-
|
|
20
19
|
class ConnectionPool::TimedStack
|
|
21
20
|
attr_reader :max
|
|
22
21
|
|
|
23
22
|
##
|
|
24
23
|
# Creates a new pool with +size+ connections that are created from the given
|
|
25
24
|
# +block+.
|
|
26
|
-
|
|
27
|
-
def initialize(size = 0, &block)
|
|
25
|
+
def initialize(size: 0, &block)
|
|
28
26
|
@create_block = block
|
|
29
27
|
@created = 0
|
|
30
28
|
@que = []
|
|
@@ -35,16 +33,15 @@ class ConnectionPool::TimedStack
|
|
|
35
33
|
end
|
|
36
34
|
|
|
37
35
|
##
|
|
38
|
-
# Returns +obj+ to the stack.
|
|
36
|
+
# Returns +obj+ to the stack. Additional kwargs are ignored in TimedStack but may be
|
|
39
37
|
# used by subclasses that extend TimedStack.
|
|
40
|
-
|
|
41
|
-
def push(obj, options = {})
|
|
38
|
+
def push(obj, **)
|
|
42
39
|
@mutex.synchronize do
|
|
43
40
|
if @shutdown_block
|
|
44
41
|
@created -= 1 unless @created == 0
|
|
45
42
|
@shutdown_block.call(obj)
|
|
46
43
|
else
|
|
47
|
-
store_connection obj,
|
|
44
|
+
store_connection obj, **
|
|
48
45
|
end
|
|
49
46
|
|
|
50
47
|
@resource.broadcast
|
|
@@ -53,29 +50,35 @@ class ConnectionPool::TimedStack
|
|
|
53
50
|
alias_method :<<, :push
|
|
54
51
|
|
|
55
52
|
##
|
|
56
|
-
# Retrieves a connection from the stack.
|
|
57
|
-
# immediately returned.
|
|
53
|
+
# Retrieves a connection from the stack. If a connection is available it is
|
|
54
|
+
# immediately returned. If no connection is available within the given
|
|
58
55
|
# timeout a ConnectionPool::TimeoutError is raised.
|
|
59
56
|
#
|
|
60
|
-
#
|
|
61
|
-
#
|
|
62
|
-
#
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
timeout = options.fetch :timeout, timeout
|
|
67
|
-
|
|
57
|
+
# @option options [Float] :timeout (0.5) Wait this many seconds for an available entry
|
|
58
|
+
# @option options [Class] :exception (ConnectionPool::TimeoutError) Exception class to raise
|
|
59
|
+
# if an entry was not available within the timeout period. Use `exception: false` to return nil.
|
|
60
|
+
#
|
|
61
|
+
# Other options may be used by subclasses that extend TimedStack.
|
|
62
|
+
def pop(timeout: 0.5, exception: ConnectionPool::TimeoutError, **)
|
|
68
63
|
deadline = current_time + timeout
|
|
69
64
|
@mutex.synchronize do
|
|
70
65
|
loop do
|
|
71
66
|
raise ConnectionPool::PoolShuttingDownError if @shutdown_block
|
|
72
|
-
|
|
67
|
+
if (conn = try_fetch_connection(**))
|
|
68
|
+
return conn
|
|
69
|
+
end
|
|
73
70
|
|
|
74
|
-
connection = try_create(
|
|
71
|
+
connection = try_create(**)
|
|
75
72
|
return connection if connection
|
|
76
73
|
|
|
77
74
|
to_wait = deadline - current_time
|
|
78
|
-
|
|
75
|
+
if to_wait <= 0
|
|
76
|
+
if exception
|
|
77
|
+
raise exception, "Waited #{timeout} sec, #{length}/#{@max} available"
|
|
78
|
+
else
|
|
79
|
+
return nil
|
|
80
|
+
end
|
|
81
|
+
end
|
|
79
82
|
@resource.wait(@mutex, to_wait)
|
|
80
83
|
end
|
|
81
84
|
end
|
|
@@ -86,7 +89,6 @@ class ConnectionPool::TimedStack
|
|
|
86
89
|
# removing it from the pool. Attempting to checkout a connection after
|
|
87
90
|
# shutdown will raise +ConnectionPool::PoolShuttingDownError+ unless
|
|
88
91
|
# +:reload+ is +true+.
|
|
89
|
-
|
|
90
92
|
def shutdown(reload: false, &block)
|
|
91
93
|
raise ArgumentError, "shutdown must receive a block" unless block
|
|
92
94
|
|
|
@@ -101,34 +103,31 @@ class ConnectionPool::TimedStack
|
|
|
101
103
|
|
|
102
104
|
##
|
|
103
105
|
# Reaps connections that were checked in more than +idle_seconds+ ago.
|
|
104
|
-
def reap(idle_seconds
|
|
105
|
-
raise ArgumentError, "reap must receive a block" unless
|
|
106
|
+
def reap(idle_seconds:)
|
|
107
|
+
raise ArgumentError, "reap must receive a block" unless block_given?
|
|
106
108
|
raise ArgumentError, "idle_seconds must be a number" unless idle_seconds.is_a?(Numeric)
|
|
107
109
|
raise ConnectionPool::PoolShuttingDownError if @shutdown_block
|
|
108
110
|
|
|
109
|
-
idle
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
end
|
|
111
|
+
count = idle
|
|
112
|
+
count.times do
|
|
113
|
+
conn = @mutex.synchronize do
|
|
114
|
+
raise ConnectionPool::PoolShuttingDownError if @shutdown_block
|
|
115
|
+
reserve_idle_connection(idle_seconds)
|
|
116
|
+
end
|
|
116
117
|
break unless conn
|
|
117
118
|
|
|
118
|
-
|
|
119
|
+
yield conn
|
|
119
120
|
end
|
|
120
121
|
end
|
|
121
122
|
|
|
122
123
|
##
|
|
123
124
|
# Returns +true+ if there are no available connections.
|
|
124
|
-
|
|
125
125
|
def empty?
|
|
126
126
|
(@created - @que.length) >= @max
|
|
127
127
|
end
|
|
128
128
|
|
|
129
129
|
##
|
|
130
130
|
# The number of connections available on the stack.
|
|
131
|
-
|
|
132
131
|
def length
|
|
133
132
|
@max - @created + @que.length
|
|
134
133
|
end
|
|
@@ -139,6 +138,12 @@ class ConnectionPool::TimedStack
|
|
|
139
138
|
@que.length
|
|
140
139
|
end
|
|
141
140
|
|
|
141
|
+
##
|
|
142
|
+
# Reduce the created count
|
|
143
|
+
def decrement_created
|
|
144
|
+
@created -= 1 unless @created == 0
|
|
145
|
+
end
|
|
146
|
+
|
|
142
147
|
private
|
|
143
148
|
|
|
144
149
|
def current_time
|
|
@@ -148,9 +153,18 @@ class ConnectionPool::TimedStack
|
|
|
148
153
|
##
|
|
149
154
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
150
155
|
#
|
|
151
|
-
# This method must returns
|
|
156
|
+
# This method must returns a connection from the stack if one exists. Allows
|
|
157
|
+
# subclasses with expensive match/search algorithms to avoid double-handling
|
|
158
|
+
# their stack.
|
|
159
|
+
def try_fetch_connection(**)
|
|
160
|
+
connection_stored?(**) && fetch_connection(**)
|
|
161
|
+
end
|
|
152
162
|
|
|
153
|
-
|
|
163
|
+
##
|
|
164
|
+
# This is an extension point for TimedStack and is called with a mutex.
|
|
165
|
+
#
|
|
166
|
+
# This method must returns true if a connection is available on the stack.
|
|
167
|
+
def connection_stored?(**)
|
|
154
168
|
!@que.empty?
|
|
155
169
|
end
|
|
156
170
|
|
|
@@ -158,8 +172,7 @@ class ConnectionPool::TimedStack
|
|
|
158
172
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
159
173
|
#
|
|
160
174
|
# This method must return a connection from the stack.
|
|
161
|
-
|
|
162
|
-
def fetch_connection(options = nil)
|
|
175
|
+
def fetch_connection(**)
|
|
163
176
|
@que.pop&.first
|
|
164
177
|
end
|
|
165
178
|
|
|
@@ -167,10 +180,8 @@ class ConnectionPool::TimedStack
|
|
|
167
180
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
168
181
|
#
|
|
169
182
|
# This method must shut down all connections on the stack.
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
while connection_stored?(options)
|
|
173
|
-
conn = fetch_connection(options)
|
|
183
|
+
def shutdown_connections(**)
|
|
184
|
+
while (conn = try_fetch_connection(**))
|
|
174
185
|
@created -= 1 unless @created == 0
|
|
175
186
|
@shutdown_block.call(conn)
|
|
176
187
|
end
|
|
@@ -181,12 +192,13 @@ class ConnectionPool::TimedStack
|
|
|
181
192
|
#
|
|
182
193
|
# This method returns the oldest idle connection if it has been idle for more than idle_seconds.
|
|
183
194
|
# This requires that the stack is kept in order of checked in time (oldest first).
|
|
184
|
-
|
|
185
195
|
def reserve_idle_connection(idle_seconds)
|
|
186
196
|
return unless idle_connections?(idle_seconds)
|
|
187
197
|
|
|
188
198
|
@created -= 1 unless @created == 0
|
|
189
199
|
|
|
200
|
+
# Most active elements are at the tail of the array.
|
|
201
|
+
# Most idle will be at the head so `shift` rather than `pop`.
|
|
190
202
|
@que.shift.first
|
|
191
203
|
end
|
|
192
204
|
|
|
@@ -194,17 +206,18 @@ class ConnectionPool::TimedStack
|
|
|
194
206
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
195
207
|
#
|
|
196
208
|
# Returns true if the first connection in the stack has been idle for more than idle_seconds
|
|
197
|
-
|
|
198
209
|
def idle_connections?(idle_seconds)
|
|
199
|
-
connection_stored?
|
|
210
|
+
return unless connection_stored?
|
|
211
|
+
# Most idle will be at the head so `first`
|
|
212
|
+
age = (current_time - @que.first.last)
|
|
213
|
+
age > idle_seconds
|
|
200
214
|
end
|
|
201
215
|
|
|
202
216
|
##
|
|
203
217
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
204
218
|
#
|
|
205
219
|
# This method must return +obj+ to the stack.
|
|
206
|
-
|
|
207
|
-
def store_connection(obj, options = nil)
|
|
220
|
+
def store_connection(obj, **)
|
|
208
221
|
@que.push [obj, current_time]
|
|
209
222
|
end
|
|
210
223
|
|
|
@@ -213,8 +226,7 @@ class ConnectionPool::TimedStack
|
|
|
213
226
|
#
|
|
214
227
|
# This method must create a connection if and only if the total number of
|
|
215
228
|
# connections allowed has not been met.
|
|
216
|
-
|
|
217
|
-
def try_create(options = nil)
|
|
229
|
+
def try_create(**)
|
|
218
230
|
unless @created == @max
|
|
219
231
|
object = @create_block.call
|
|
220
232
|
@created += 1
|
|
@@ -2,20 +2,20 @@ class ConnectionPool
|
|
|
2
2
|
class Wrapper < ::BasicObject
|
|
3
3
|
METHODS = [:with, :pool_shutdown, :wrapped_pool]
|
|
4
4
|
|
|
5
|
-
def initialize(options
|
|
6
|
-
@pool = options.fetch(:pool) { ::ConnectionPool.new(options, &
|
|
5
|
+
def initialize(**options, &)
|
|
6
|
+
@pool = options.fetch(:pool) { ::ConnectionPool.new(**options, &) }
|
|
7
7
|
end
|
|
8
8
|
|
|
9
9
|
def wrapped_pool
|
|
10
10
|
@pool
|
|
11
11
|
end
|
|
12
12
|
|
|
13
|
-
def with(&
|
|
14
|
-
@pool.with(&
|
|
13
|
+
def with(**, &)
|
|
14
|
+
@pool.with(**, &)
|
|
15
15
|
end
|
|
16
16
|
|
|
17
|
-
def pool_shutdown(&
|
|
18
|
-
@pool.shutdown(&
|
|
17
|
+
def pool_shutdown(&)
|
|
18
|
+
@pool.shutdown(&)
|
|
19
19
|
end
|
|
20
20
|
|
|
21
21
|
def pool_size
|
|
@@ -26,31 +26,18 @@ class ConnectionPool
|
|
|
26
26
|
@pool.available
|
|
27
27
|
end
|
|
28
28
|
|
|
29
|
-
def respond_to?(id,
|
|
30
|
-
METHODS.include?(id) || with { |c| c.respond_to?(id,
|
|
29
|
+
def respond_to?(id, *, **)
|
|
30
|
+
METHODS.include?(id) || with { |c| c.respond_to?(id, *, **) }
|
|
31
31
|
end
|
|
32
32
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
elsif ::RUBY_VERSION >= "2.7.0"
|
|
41
|
-
ruby2_keywords def method_missing(name, *args, &block)
|
|
42
|
-
with do |connection|
|
|
43
|
-
connection.send(name, *args, &block)
|
|
44
|
-
end
|
|
45
|
-
end
|
|
46
|
-
else
|
|
47
|
-
def method_missing(name, *args, &block)
|
|
48
|
-
with do |connection|
|
|
49
|
-
connection.send(name, *args, &block)
|
|
50
|
-
end
|
|
33
|
+
def respond_to_missing?(id, *, **)
|
|
34
|
+
with { |c| c.respond_to?(id, *, **) }
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def method_missing(name, *, **, &)
|
|
38
|
+
with do |connection|
|
|
39
|
+
connection.send(name, *, **, &)
|
|
51
40
|
end
|
|
52
41
|
end
|
|
53
|
-
# rubocop:enable Style/MethodMissingSuper
|
|
54
|
-
# rubocop:enable Style/MissingRespondToMissing
|
|
55
42
|
end
|
|
56
43
|
end
|
data/lib/connection_pool.rb
CHANGED
|
@@ -39,72 +39,30 @@ end
|
|
|
39
39
|
# - :auto_reload_after_fork - automatically drop all connections after fork, defaults to true
|
|
40
40
|
#
|
|
41
41
|
class ConnectionPool
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
def self.wrap(options, &block)
|
|
45
|
-
Wrapper.new(options, &block)
|
|
46
|
-
end
|
|
47
|
-
|
|
48
|
-
if Process.respond_to?(:fork)
|
|
49
|
-
INSTANCES = ObjectSpace::WeakMap.new
|
|
50
|
-
private_constant :INSTANCES
|
|
51
|
-
|
|
52
|
-
def self.after_fork
|
|
53
|
-
INSTANCES.values.each do |pool|
|
|
54
|
-
next unless pool.auto_reload_after_fork
|
|
55
|
-
|
|
56
|
-
# We're on after fork, so we know all other threads are dead.
|
|
57
|
-
# All we need to do is to ensure the main thread doesn't have a
|
|
58
|
-
# checked out connection
|
|
59
|
-
pool.checkin(force: true)
|
|
60
|
-
pool.reload do |connection|
|
|
61
|
-
# Unfortunately we don't know what method to call to close the connection,
|
|
62
|
-
# so we try the most common one.
|
|
63
|
-
connection.close if connection.respond_to?(:close)
|
|
64
|
-
end
|
|
65
|
-
end
|
|
66
|
-
nil
|
|
67
|
-
end
|
|
68
|
-
|
|
69
|
-
if ::Process.respond_to?(:_fork) # MRI 3.1+
|
|
70
|
-
module ForkTracker
|
|
71
|
-
def _fork
|
|
72
|
-
pid = super
|
|
73
|
-
if pid == 0
|
|
74
|
-
ConnectionPool.after_fork
|
|
75
|
-
end
|
|
76
|
-
pid
|
|
77
|
-
end
|
|
78
|
-
end
|
|
79
|
-
Process.singleton_class.prepend(ForkTracker)
|
|
80
|
-
end
|
|
81
|
-
else
|
|
82
|
-
INSTANCES = nil
|
|
83
|
-
private_constant :INSTANCES
|
|
84
|
-
|
|
85
|
-
def self.after_fork
|
|
86
|
-
# noop
|
|
87
|
-
end
|
|
42
|
+
def self.wrap(**, &)
|
|
43
|
+
Wrapper.new(**, &)
|
|
88
44
|
end
|
|
89
45
|
|
|
90
|
-
|
|
91
|
-
raise ArgumentError, "Connection pool requires a block" unless block
|
|
92
|
-
|
|
93
|
-
options = DEFAULTS.merge(options)
|
|
46
|
+
attr_reader :size
|
|
94
47
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
@auto_reload_after_fork = options.fetch(:auto_reload_after_fork)
|
|
48
|
+
def initialize(timeout: 5, size: 5, auto_reload_after_fork: true, name: nil, &)
|
|
49
|
+
raise ArgumentError, "Connection pool requires a block" unless block_given?
|
|
98
50
|
|
|
99
|
-
@
|
|
51
|
+
@size = Integer(size)
|
|
52
|
+
@timeout = Float(timeout)
|
|
53
|
+
@available = TimedStack.new(size: @size, &)
|
|
100
54
|
@key = :"pool-#{@available.object_id}"
|
|
101
55
|
@key_count = :"pool-#{@available.object_id}-count"
|
|
102
|
-
|
|
56
|
+
@discard_key = :"pool-#{@available.object_id}-discard"
|
|
57
|
+
INSTANCES[self] = self if auto_reload_after_fork && INSTANCES
|
|
103
58
|
end
|
|
104
59
|
|
|
105
|
-
def with(
|
|
60
|
+
def with(**)
|
|
61
|
+
# We need to manage exception handling manually here in order
|
|
62
|
+
# to work correctly with `Timeout.timeout` and `Thread#raise`.
|
|
63
|
+
# Otherwise an interrupted Thread can leak connections.
|
|
106
64
|
Thread.handle_interrupt(Exception => :never) do
|
|
107
|
-
conn = checkout(
|
|
65
|
+
conn = checkout(**)
|
|
108
66
|
begin
|
|
109
67
|
Thread.handle_interrupt(Exception => :immediate) do
|
|
110
68
|
yield conn
|
|
@@ -116,20 +74,67 @@ class ConnectionPool
|
|
|
116
74
|
end
|
|
117
75
|
alias_method :then, :with
|
|
118
76
|
|
|
119
|
-
|
|
77
|
+
##
|
|
78
|
+
# Marks the current thread's checked-out connection for discard.
|
|
79
|
+
#
|
|
80
|
+
# When a connection is marked for discard, it will not be returned to the pool
|
|
81
|
+
# when checked in. Instead, the connection will be discarded.
|
|
82
|
+
# This is useful when a connection has become invalid or corrupted
|
|
83
|
+
# and should not be reused.
|
|
84
|
+
#
|
|
85
|
+
# Takes an optional block that will be called with the connection to be discarded.
|
|
86
|
+
# The block should perform any necessary clean-up on the connection.
|
|
87
|
+
#
|
|
88
|
+
# @yield [conn]
|
|
89
|
+
# @yieldparam conn [Object] The connection to be discarded.
|
|
90
|
+
# @yieldreturn [void]
|
|
91
|
+
#
|
|
92
|
+
#
|
|
93
|
+
# Note: This only affects the connection currently checked out by the calling thread.
|
|
94
|
+
# The connection will be discarded when +checkin+ is called.
|
|
95
|
+
#
|
|
96
|
+
# @return [void]
|
|
97
|
+
#
|
|
98
|
+
# @example
|
|
99
|
+
# pool.with do |conn|
|
|
100
|
+
# begin
|
|
101
|
+
# conn.execute("SELECT 1")
|
|
102
|
+
# rescue SomeConnectionError
|
|
103
|
+
# pool.discard_current_connection # Mark connection as bad
|
|
104
|
+
# raise
|
|
105
|
+
# end
|
|
106
|
+
# end
|
|
107
|
+
def discard_current_connection(&block)
|
|
108
|
+
::Thread.current[@discard_key] = block || proc { |conn| conn }
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
def checkout(timeout: @timeout, **)
|
|
120
112
|
if ::Thread.current[@key]
|
|
121
113
|
::Thread.current[@key_count] += 1
|
|
122
114
|
::Thread.current[@key]
|
|
123
115
|
else
|
|
116
|
+
conn = @available.pop(timeout:, **)
|
|
117
|
+
::Thread.current[@key] = conn
|
|
124
118
|
::Thread.current[@key_count] = 1
|
|
125
|
-
|
|
119
|
+
conn
|
|
126
120
|
end
|
|
127
121
|
end
|
|
128
122
|
|
|
129
123
|
def checkin(force: false)
|
|
130
124
|
if ::Thread.current[@key]
|
|
131
125
|
if ::Thread.current[@key_count] == 1 || force
|
|
132
|
-
|
|
126
|
+
if ::Thread.current[@discard_key]
|
|
127
|
+
begin
|
|
128
|
+
@available.decrement_created
|
|
129
|
+
::Thread.current[@discard_key].call(::Thread.current[@key])
|
|
130
|
+
rescue
|
|
131
|
+
nil
|
|
132
|
+
ensure
|
|
133
|
+
::Thread.current[@discard_key] = nil
|
|
134
|
+
end
|
|
135
|
+
else
|
|
136
|
+
@available.push(::Thread.current[@key])
|
|
137
|
+
end
|
|
133
138
|
::Thread.current[@key] = nil
|
|
134
139
|
::Thread.current[@key_count] = nil
|
|
135
140
|
else
|
|
@@ -146,31 +151,24 @@ class ConnectionPool
|
|
|
146
151
|
# Shuts down the ConnectionPool by passing each connection to +block+ and
|
|
147
152
|
# then removing it from the pool. Attempting to checkout a connection after
|
|
148
153
|
# shutdown will raise +ConnectionPool::PoolShuttingDownError+.
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
@available.shutdown(&block)
|
|
154
|
+
def shutdown(&)
|
|
155
|
+
@available.shutdown(&)
|
|
152
156
|
end
|
|
153
157
|
|
|
154
158
|
##
|
|
155
159
|
# Reloads the ConnectionPool by passing each connection to +block+ and then
|
|
156
160
|
# removing it the pool. Subsequent checkouts will create new connections as
|
|
157
161
|
# needed.
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
@available.shutdown(reload: true, &block)
|
|
162
|
+
def reload(&)
|
|
163
|
+
@available.shutdown(reload: true, &)
|
|
161
164
|
end
|
|
162
165
|
|
|
163
166
|
## Reaps idle connections that have been idle for over +idle_seconds+.
|
|
164
167
|
# +idle_seconds+ defaults to 60.
|
|
165
|
-
def reap(idle_seconds
|
|
166
|
-
@available.reap(idle_seconds
|
|
168
|
+
def reap(idle_seconds: 60, &)
|
|
169
|
+
@available.reap(idle_seconds:, &)
|
|
167
170
|
end
|
|
168
171
|
|
|
169
|
-
# Size of this connection pool
|
|
170
|
-
attr_reader :size
|
|
171
|
-
# Automatically drop all connections after fork
|
|
172
|
-
attr_reader :auto_reload_after_fork
|
|
173
|
-
|
|
174
172
|
# Number of pool entries available for checkout at this instant.
|
|
175
173
|
def available
|
|
176
174
|
@available.length
|
|
@@ -184,3 +182,4 @@ end
|
|
|
184
182
|
|
|
185
183
|
require_relative "connection_pool/timed_stack"
|
|
186
184
|
require_relative "connection_pool/wrapper"
|
|
185
|
+
require_relative "connection_pool/fork"
|
metadata
CHANGED
|
@@ -1,15 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: connection_pool
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version:
|
|
4
|
+
version: 3.0.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Mike Perham
|
|
8
8
|
- Damian Janowski
|
|
9
|
-
autorequire:
|
|
10
9
|
bindir: bin
|
|
11
10
|
cert_chain: []
|
|
12
|
-
date:
|
|
11
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
13
12
|
dependencies:
|
|
14
13
|
- !ruby/object:Gem::Dependency
|
|
15
14
|
name: bundler
|
|
@@ -26,19 +25,19 @@ dependencies:
|
|
|
26
25
|
- !ruby/object:Gem::Version
|
|
27
26
|
version: '0'
|
|
28
27
|
- !ruby/object:Gem::Dependency
|
|
29
|
-
name:
|
|
28
|
+
name: maxitest
|
|
30
29
|
requirement: !ruby/object:Gem::Requirement
|
|
31
30
|
requirements:
|
|
32
31
|
- - ">="
|
|
33
32
|
- !ruby/object:Gem::Version
|
|
34
|
-
version:
|
|
33
|
+
version: '0'
|
|
35
34
|
type: :development
|
|
36
35
|
prerelease: false
|
|
37
36
|
version_requirements: !ruby/object:Gem::Requirement
|
|
38
37
|
requirements:
|
|
39
38
|
- - ">="
|
|
40
39
|
- !ruby/object:Gem::Version
|
|
41
|
-
version:
|
|
40
|
+
version: '0'
|
|
42
41
|
- !ruby/object:Gem::Dependency
|
|
43
42
|
name: rake
|
|
44
43
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -66,6 +65,7 @@ files:
|
|
|
66
65
|
- README.md
|
|
67
66
|
- connection_pool.gemspec
|
|
68
67
|
- lib/connection_pool.rb
|
|
68
|
+
- lib/connection_pool/fork.rb
|
|
69
69
|
- lib/connection_pool/timed_stack.rb
|
|
70
70
|
- lib/connection_pool/version.rb
|
|
71
71
|
- lib/connection_pool/wrapper.rb
|
|
@@ -73,9 +73,12 @@ homepage: https://github.com/mperham/connection_pool
|
|
|
73
73
|
licenses:
|
|
74
74
|
- MIT
|
|
75
75
|
metadata:
|
|
76
|
+
bug_tracker_uri: https://github.com/mperham/connection_pool/issues
|
|
77
|
+
documentation_uri: https://github.com/mperham/connection_pool/wiki
|
|
76
78
|
changelog_uri: https://github.com/mperham/connection_pool/blob/main/Changes.md
|
|
79
|
+
source_code_uri: https://github.com/mperham/connection_pool
|
|
80
|
+
homepage_uri: https://github.com/mperham/connection_pool
|
|
77
81
|
rubygems_mfa_required: 'true'
|
|
78
|
-
post_install_message:
|
|
79
82
|
rdoc_options: []
|
|
80
83
|
require_paths:
|
|
81
84
|
- lib
|
|
@@ -83,15 +86,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
83
86
|
requirements:
|
|
84
87
|
- - ">="
|
|
85
88
|
- !ruby/object:Gem::Version
|
|
86
|
-
version: 2.
|
|
89
|
+
version: 3.2.0
|
|
87
90
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
88
91
|
requirements:
|
|
89
92
|
- - ">="
|
|
90
93
|
- !ruby/object:Gem::Version
|
|
91
94
|
version: '0'
|
|
92
95
|
requirements: []
|
|
93
|
-
rubygems_version: 3.
|
|
94
|
-
signing_key:
|
|
96
|
+
rubygems_version: 3.6.9
|
|
95
97
|
specification_version: 4
|
|
96
98
|
summary: Generic connection pool for Ruby
|
|
97
99
|
test_files: []
|