connection_pool 2.4.1 → 3.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Changes.md +63 -0
- data/README.md +68 -21
- data/connection_pool.gemspec +16 -5
- data/lib/connection_pool/fork.rb +40 -0
- data/lib/connection_pool/timed_stack.rb +103 -41
- data/lib/connection_pool/version.rb +1 -1
- data/lib/connection_pool/wrapper.rb +15 -28
- data/lib/connection_pool.rb +80 -70
- metadata +12 -10
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a064d41333b8b92fcb23617701011f1b34e6348b324048ca16e5cb758d31f05f
|
|
4
|
+
data.tar.gz: 7ca1cc56ff7d020f2b7f2cee01b2008502faadf563dad010c2b8eee4a0b83dcd
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 1c7554d540f6aefcd356154246a1cdcab5e4ddea745e9a580261fba0635e6b711a9e0781691503f20aca7faaf982e2ba8f5f48cc71ff2a4f67053039e947ba3d
|
|
7
|
+
data.tar.gz: 431dcf74c39f6a9db1503290a227f5780c4b03e392ef25709245e63d0ca46a2f0f6cc0792a5241e615eafd92663aa38c1165aae99f6dffb0783f5b442846582f
|
data/Changes.md
CHANGED
|
@@ -1,5 +1,68 @@
|
|
|
1
1
|
# connection_pool Changelog
|
|
2
2
|
|
|
3
|
+
3.0.2
|
|
4
|
+
------
|
|
5
|
+
|
|
6
|
+
- Support :name keyword for backwards compatibility [#210]
|
|
7
|
+
|
|
8
|
+
3.0.1
|
|
9
|
+
------
|
|
10
|
+
|
|
11
|
+
- Add missing `fork.rb` to gemspec.
|
|
12
|
+
|
|
13
|
+
3.0.0
|
|
14
|
+
------
|
|
15
|
+
|
|
16
|
+
- **BREAKING CHANGES** `ConnectionPool` and `ConnectionPool::TimedStack` now
|
|
17
|
+
use keyword arguments rather than positional arguments everywhere. Expected impact is minimal as most people use the `with` API, which is unchanged.
|
|
18
|
+
```ruby
|
|
19
|
+
pool = ConnectionPool.new(size: 5, timeout: 5)
|
|
20
|
+
pool.checkout(1) # 2.x
|
|
21
|
+
pool.reap(30) # 2.x
|
|
22
|
+
pool.checkout(timeout: 1) # 3.x
|
|
23
|
+
pool.reap(idle_seconds: 30) # 3.x
|
|
24
|
+
```
|
|
25
|
+
- Dropped support for Ruby <3.2.0
|
|
26
|
+
|
|
27
|
+
2.5.5
|
|
28
|
+
------
|
|
29
|
+
|
|
30
|
+
- Support `ConnectionPool::TimedStack#pop(exception: false)` [#207]
|
|
31
|
+
to avoid using exceptions as control flow.
|
|
32
|
+
|
|
33
|
+
2.5.4
|
|
34
|
+
------
|
|
35
|
+
|
|
36
|
+
- Add ability to remove a broken connection from the pool [#204, womblep]
|
|
37
|
+
|
|
38
|
+
2.5.3
|
|
39
|
+
------
|
|
40
|
+
|
|
41
|
+
- Fix TruffleRuby/JRuby crash [#201]
|
|
42
|
+
|
|
43
|
+
2.5.2
|
|
44
|
+
------
|
|
45
|
+
|
|
46
|
+
- Rollback inadvertant change to `auto_reload_after_fork` default. [#200]
|
|
47
|
+
|
|
48
|
+
2.5.1
|
|
49
|
+
------
|
|
50
|
+
|
|
51
|
+
- Pass options to TimedStack in `checkout` [#195]
|
|
52
|
+
- Optimize connection lookup [#196]
|
|
53
|
+
- Fixes for use with Ractors
|
|
54
|
+
|
|
55
|
+
2.5.0
|
|
56
|
+
------
|
|
57
|
+
|
|
58
|
+
- Reap idle connections [#187]
|
|
59
|
+
```ruby
|
|
60
|
+
idle_timeout = 60
|
|
61
|
+
pool = ConnectionPool.new ...
|
|
62
|
+
pool.reap(idle_timeout, &:close)
|
|
63
|
+
```
|
|
64
|
+
- `ConnectionPool#idle` returns the count of connections not in use [#187]
|
|
65
|
+
|
|
3
66
|
2.4.1
|
|
4
67
|
------
|
|
5
68
|
|
data/README.md
CHANGED
|
@@ -38,7 +38,7 @@ connection pool and a raw client.
|
|
|
38
38
|
$redis.then { |r| r.set 'foo' 'bar' }
|
|
39
39
|
```
|
|
40
40
|
|
|
41
|
-
Optionally, you can specify a timeout override
|
|
41
|
+
Optionally, you can specify a timeout override:
|
|
42
42
|
|
|
43
43
|
``` ruby
|
|
44
44
|
$memcached.with(timeout: 2.0) do |conn|
|
|
@@ -46,11 +46,9 @@ $memcached.with(timeout: 2.0) do |conn|
|
|
|
46
46
|
end
|
|
47
47
|
```
|
|
48
48
|
|
|
49
|
-
This will only modify the
|
|
50
|
-
invocation.
|
|
49
|
+
This will only modify the timeout for this particular invocation.
|
|
51
50
|
This is useful if you want to fail-fast on certain non-critical
|
|
52
51
|
sections when a resource is not available, or conversely if you are comfortable blocking longer on a particular resource.
|
|
53
|
-
This is not implemented in the `ConnectionPool::Wrapper` class.
|
|
54
52
|
|
|
55
53
|
## Migrating to a Connection Pool
|
|
56
54
|
|
|
@@ -72,7 +70,7 @@ $redis.with do |conn|
|
|
|
72
70
|
end
|
|
73
71
|
```
|
|
74
72
|
|
|
75
|
-
Once you've ported your entire system to use `with`, you can
|
|
73
|
+
Once you've ported your entire system to use `with`, you can remove `::Wrapper` and use `ConnectionPool` directly.
|
|
76
74
|
|
|
77
75
|
|
|
78
76
|
## Shutdown
|
|
@@ -90,16 +88,53 @@ Shutting down a connection pool will block until all connections are checked in
|
|
|
90
88
|
|
|
91
89
|
## Reload
|
|
92
90
|
|
|
93
|
-
You can reload a ConnectionPool instance
|
|
94
|
-
|
|
91
|
+
You can reload a ConnectionPool instance if it is necessary to close all existing connections and continue to use the pool.
|
|
92
|
+
ConnectionPool will automatically reload if the process is forked.
|
|
93
|
+
Use `auto_reload_after_fork: false` if you don't want this behavior.
|
|
95
94
|
|
|
96
95
|
```ruby
|
|
97
|
-
cp = ConnectionPool.new { Redis.new }
|
|
98
|
-
cp.reload { |conn| conn.quit }
|
|
96
|
+
cp = ConnectionPool.new(auto_reload_after_fork: false) { Redis.new }
|
|
97
|
+
cp.reload { |conn| conn.quit } # reload manually
|
|
99
98
|
cp.with { |conn| conn.get('some-count') }
|
|
100
99
|
```
|
|
101
100
|
|
|
102
|
-
Like `shutdown`,
|
|
101
|
+
Like `shutdown`, `reload` will block until all connections are checked in and closed.
|
|
102
|
+
|
|
103
|
+
## Reap
|
|
104
|
+
|
|
105
|
+
You can call `reap` periodically on the ConnectionPool instance to close connections that were created but have not been used for a certain amount of time. This can be useful in environments where connections are expensive.
|
|
106
|
+
|
|
107
|
+
You can specify how many seconds the connections have to be idle for them to be reaped, defaulting to 60 seconds.
|
|
108
|
+
|
|
109
|
+
```ruby
|
|
110
|
+
cp = ConnectionPool.new { Redis.new }
|
|
111
|
+
|
|
112
|
+
# Start a reaper thread to reap connections that have been
|
|
113
|
+
# idle more than 300 seconds (5 minutes)
|
|
114
|
+
Thread.new do
|
|
115
|
+
loop do
|
|
116
|
+
cp.reap(idle_seconds: 300, &:close)
|
|
117
|
+
sleep 30
|
|
118
|
+
end
|
|
119
|
+
end
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
## Discarding Connections
|
|
123
|
+
|
|
124
|
+
You can discard connections in the ConnectionPool instance to remove connections that are broken and can't be repaired.
|
|
125
|
+
It can only be done inside the block passed to `with`.
|
|
126
|
+
Takes an optional block that will be executed with the connection.
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
pool.with do |conn|
|
|
130
|
+
begin
|
|
131
|
+
conn.execute("SELECT 1")
|
|
132
|
+
rescue SomeConnectionError
|
|
133
|
+
pool.discard_current_connection(&:close) # remove the connection from the pool
|
|
134
|
+
raise
|
|
135
|
+
end
|
|
136
|
+
end
|
|
137
|
+
```
|
|
103
138
|
|
|
104
139
|
## Current State
|
|
105
140
|
|
|
@@ -109,27 +144,39 @@ There are several methods that return information about a pool.
|
|
|
109
144
|
cp = ConnectionPool.new(size: 10) { Redis.new }
|
|
110
145
|
cp.size # => 10
|
|
111
146
|
cp.available # => 10
|
|
147
|
+
cp.idle # => 0
|
|
112
148
|
|
|
113
149
|
cp.with do |conn|
|
|
114
150
|
cp.size # => 10
|
|
115
151
|
cp.available # => 9
|
|
152
|
+
cp.idle # => 0
|
|
116
153
|
end
|
|
154
|
+
|
|
155
|
+
cp.idle # => 1
|
|
117
156
|
```
|
|
118
157
|
|
|
119
|
-
|
|
120
|
-
|
|
158
|
+
## Upgrading from ConnectionPool 2
|
|
159
|
+
|
|
160
|
+
* Support for Ruby <3.2 has been removed.
|
|
161
|
+
* ConnectionPool's APIs now consistently use keyword arguments everywhere.
|
|
162
|
+
Positional arguments must be converted to keywords:
|
|
163
|
+
```ruby
|
|
164
|
+
pool = ConnectionPool.new(size: 5, timeout: 5)
|
|
165
|
+
pool.checkout(1) # 2.x
|
|
166
|
+
pool.reap(30) # 2.x
|
|
167
|
+
pool.checkout(timeout: 1) # 3.x
|
|
168
|
+
pool.reap(idle_seconds: 30) # 3.x
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
## Notes
|
|
121
172
|
|
|
122
173
|
- Connections are lazily created as needed.
|
|
123
|
-
-
|
|
124
|
-
connections should be self-repairing. This is true of the Dalli and Redis
|
|
125
|
-
clients.
|
|
126
|
-
- **WARNING**: Don't ever use `Timeout.timeout` in your Ruby code or you will see
|
|
174
|
+
- **WARNING**: Avoid `Timeout.timeout` in your Ruby code or you can see
|
|
127
175
|
occasional silent corruption and mysterious errors. The Timeout API is unsafe
|
|
128
|
-
and
|
|
129
|
-
|
|
176
|
+
and dangerous to use. Use proper socket timeout options as exposed by
|
|
177
|
+
Net::HTTP, Redis, Dalli, etc.
|
|
130
178
|
|
|
131
179
|
|
|
132
|
-
Author
|
|
133
|
-
------
|
|
180
|
+
## Author
|
|
134
181
|
|
|
135
|
-
Mike Perham, [@getajobmike](https://
|
|
182
|
+
Mike Perham, [@getajobmike](https://ruby.social/@getajobmike), <https://www.mikeperham.com>
|
data/connection_pool.gemspec
CHANGED
|
@@ -10,15 +10,26 @@ Gem::Specification.new do |s|
|
|
|
10
10
|
s.description = s.summary = "Generic connection pool for Ruby"
|
|
11
11
|
|
|
12
12
|
s.files = ["Changes.md", "LICENSE", "README.md", "connection_pool.gemspec",
|
|
13
|
-
"lib/connection_pool.rb",
|
|
14
|
-
"lib/connection_pool/
|
|
13
|
+
"lib/connection_pool.rb",
|
|
14
|
+
"lib/connection_pool/timed_stack.rb",
|
|
15
|
+
"lib/connection_pool/version.rb",
|
|
16
|
+
"lib/connection_pool/fork.rb",
|
|
17
|
+
"lib/connection_pool/wrapper.rb"]
|
|
15
18
|
s.executables = []
|
|
16
19
|
s.require_paths = ["lib"]
|
|
17
20
|
s.license = "MIT"
|
|
21
|
+
|
|
22
|
+
s.required_ruby_version = ">= 3.2.0"
|
|
18
23
|
s.add_development_dependency "bundler"
|
|
19
|
-
s.add_development_dependency "
|
|
24
|
+
s.add_development_dependency "maxitest"
|
|
20
25
|
s.add_development_dependency "rake"
|
|
21
|
-
s.required_ruby_version = ">= 2.5.0"
|
|
22
26
|
|
|
23
|
-
s.metadata = {
|
|
27
|
+
s.metadata = {
|
|
28
|
+
"bug_tracker_uri" => "https://github.com/mperham/connection_pool/issues",
|
|
29
|
+
"documentation_uri" => "https://github.com/mperham/connection_pool/wiki",
|
|
30
|
+
"changelog_uri" => "https://github.com/mperham/connection_pool/blob/main/Changes.md",
|
|
31
|
+
"source_code_uri" => "https://github.com/mperham/connection_pool",
|
|
32
|
+
"homepage_uri" => "https://github.com/mperham/connection_pool",
|
|
33
|
+
"rubygems_mfa_required" => "true"
|
|
34
|
+
}
|
|
24
35
|
end
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
class ConnectionPool
|
|
2
|
+
if Process.respond_to?(:fork)
|
|
3
|
+
INSTANCES = ObjectSpace::WeakMap.new
|
|
4
|
+
private_constant :INSTANCES
|
|
5
|
+
|
|
6
|
+
def self.after_fork
|
|
7
|
+
INSTANCES.each_value do |pool|
|
|
8
|
+
# We're in after_fork, so we know all other threads are dead.
|
|
9
|
+
# All we need to do is ensure the main thread doesn't have a
|
|
10
|
+
# checked out connection
|
|
11
|
+
pool.checkin(force: true)
|
|
12
|
+
pool.reload do |connection|
|
|
13
|
+
# Unfortunately we don't know what method to call to close the connection,
|
|
14
|
+
# so we try the most common one.
|
|
15
|
+
connection.close if connection.respond_to?(:close)
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
nil
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
module ForkTracker
|
|
22
|
+
def _fork
|
|
23
|
+
pid = super
|
|
24
|
+
if pid == 0
|
|
25
|
+
ConnectionPool.after_fork
|
|
26
|
+
end
|
|
27
|
+
pid
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
Process.singleton_class.prepend(ForkTracker)
|
|
31
|
+
else
|
|
32
|
+
# JRuby, et al
|
|
33
|
+
INSTANCES = nil
|
|
34
|
+
private_constant :INSTANCES
|
|
35
|
+
|
|
36
|
+
def self.after_fork
|
|
37
|
+
# noop
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
end
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
##
|
|
2
2
|
# The TimedStack manages a pool of homogeneous connections (or any resource
|
|
3
|
-
# you wish to manage).
|
|
3
|
+
# you wish to manage). Connections are created lazily up to a given maximum
|
|
4
4
|
# number.
|
|
5
|
-
|
|
5
|
+
#
|
|
6
6
|
# Examples:
|
|
7
7
|
#
|
|
8
|
-
# ts = TimedStack.new(1) { MyConnection.new }
|
|
8
|
+
# ts = TimedStack.new(size: 1) { MyConnection.new }
|
|
9
9
|
#
|
|
10
10
|
# # fetch a connection
|
|
11
11
|
# conn = ts.pop
|
|
@@ -16,15 +16,13 @@
|
|
|
16
16
|
# conn = ts.pop
|
|
17
17
|
# ts.pop timeout: 5
|
|
18
18
|
# #=> raises ConnectionPool::TimeoutError after 5 seconds
|
|
19
|
-
|
|
20
19
|
class ConnectionPool::TimedStack
|
|
21
20
|
attr_reader :max
|
|
22
21
|
|
|
23
22
|
##
|
|
24
23
|
# Creates a new pool with +size+ connections that are created from the given
|
|
25
24
|
# +block+.
|
|
26
|
-
|
|
27
|
-
def initialize(size = 0, &block)
|
|
25
|
+
def initialize(size: 0, &block)
|
|
28
26
|
@create_block = block
|
|
29
27
|
@created = 0
|
|
30
28
|
@que = []
|
|
@@ -35,15 +33,15 @@ class ConnectionPool::TimedStack
|
|
|
35
33
|
end
|
|
36
34
|
|
|
37
35
|
##
|
|
38
|
-
# Returns +obj+ to the stack.
|
|
36
|
+
# Returns +obj+ to the stack. Additional kwargs are ignored in TimedStack but may be
|
|
39
37
|
# used by subclasses that extend TimedStack.
|
|
40
|
-
|
|
41
|
-
def push(obj, options = {})
|
|
38
|
+
def push(obj, **)
|
|
42
39
|
@mutex.synchronize do
|
|
43
40
|
if @shutdown_block
|
|
41
|
+
@created -= 1 unless @created == 0
|
|
44
42
|
@shutdown_block.call(obj)
|
|
45
43
|
else
|
|
46
|
-
store_connection obj,
|
|
44
|
+
store_connection obj, **
|
|
47
45
|
end
|
|
48
46
|
|
|
49
47
|
@resource.broadcast
|
|
@@ -52,29 +50,35 @@ class ConnectionPool::TimedStack
|
|
|
52
50
|
alias_method :<<, :push
|
|
53
51
|
|
|
54
52
|
##
|
|
55
|
-
# Retrieves a connection from the stack.
|
|
56
|
-
# immediately returned.
|
|
53
|
+
# Retrieves a connection from the stack. If a connection is available it is
|
|
54
|
+
# immediately returned. If no connection is available within the given
|
|
57
55
|
# timeout a ConnectionPool::TimeoutError is raised.
|
|
58
56
|
#
|
|
59
|
-
#
|
|
60
|
-
#
|
|
61
|
-
#
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
timeout = options.fetch :timeout, timeout
|
|
66
|
-
|
|
57
|
+
# @option options [Float] :timeout (0.5) Wait this many seconds for an available entry
|
|
58
|
+
# @option options [Class] :exception (ConnectionPool::TimeoutError) Exception class to raise
|
|
59
|
+
# if an entry was not available within the timeout period. Use `exception: false` to return nil.
|
|
60
|
+
#
|
|
61
|
+
# Other options may be used by subclasses that extend TimedStack.
|
|
62
|
+
def pop(timeout: 0.5, exception: ConnectionPool::TimeoutError, **)
|
|
67
63
|
deadline = current_time + timeout
|
|
68
64
|
@mutex.synchronize do
|
|
69
65
|
loop do
|
|
70
66
|
raise ConnectionPool::PoolShuttingDownError if @shutdown_block
|
|
71
|
-
|
|
67
|
+
if (conn = try_fetch_connection(**))
|
|
68
|
+
return conn
|
|
69
|
+
end
|
|
72
70
|
|
|
73
|
-
connection = try_create(
|
|
71
|
+
connection = try_create(**)
|
|
74
72
|
return connection if connection
|
|
75
73
|
|
|
76
74
|
to_wait = deadline - current_time
|
|
77
|
-
|
|
75
|
+
if to_wait <= 0
|
|
76
|
+
if exception
|
|
77
|
+
raise exception, "Waited #{timeout} sec, #{length}/#{@max} available"
|
|
78
|
+
else
|
|
79
|
+
return nil
|
|
80
|
+
end
|
|
81
|
+
end
|
|
78
82
|
@resource.wait(@mutex, to_wait)
|
|
79
83
|
end
|
|
80
84
|
end
|
|
@@ -85,7 +89,6 @@ class ConnectionPool::TimedStack
|
|
|
85
89
|
# removing it from the pool. Attempting to checkout a connection after
|
|
86
90
|
# shutdown will raise +ConnectionPool::PoolShuttingDownError+ unless
|
|
87
91
|
# +:reload+ is +true+.
|
|
88
|
-
|
|
89
92
|
def shutdown(reload: false, &block)
|
|
90
93
|
raise ArgumentError, "shutdown must receive a block" unless block
|
|
91
94
|
|
|
@@ -99,19 +102,48 @@ class ConnectionPool::TimedStack
|
|
|
99
102
|
end
|
|
100
103
|
|
|
101
104
|
##
|
|
102
|
-
#
|
|
105
|
+
# Reaps connections that were checked in more than +idle_seconds+ ago.
|
|
106
|
+
def reap(idle_seconds:)
|
|
107
|
+
raise ArgumentError, "reap must receive a block" unless block_given?
|
|
108
|
+
raise ArgumentError, "idle_seconds must be a number" unless idle_seconds.is_a?(Numeric)
|
|
109
|
+
raise ConnectionPool::PoolShuttingDownError if @shutdown_block
|
|
110
|
+
|
|
111
|
+
count = idle
|
|
112
|
+
count.times do
|
|
113
|
+
conn = @mutex.synchronize do
|
|
114
|
+
raise ConnectionPool::PoolShuttingDownError if @shutdown_block
|
|
115
|
+
reserve_idle_connection(idle_seconds)
|
|
116
|
+
end
|
|
117
|
+
break unless conn
|
|
103
118
|
|
|
119
|
+
yield conn
|
|
120
|
+
end
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
##
|
|
124
|
+
# Returns +true+ if there are no available connections.
|
|
104
125
|
def empty?
|
|
105
126
|
(@created - @que.length) >= @max
|
|
106
127
|
end
|
|
107
128
|
|
|
108
129
|
##
|
|
109
130
|
# The number of connections available on the stack.
|
|
110
|
-
|
|
111
131
|
def length
|
|
112
132
|
@max - @created + @que.length
|
|
113
133
|
end
|
|
114
134
|
|
|
135
|
+
##
|
|
136
|
+
# The number of connections created and available on the stack.
|
|
137
|
+
def idle
|
|
138
|
+
@que.length
|
|
139
|
+
end
|
|
140
|
+
|
|
141
|
+
##
|
|
142
|
+
# Reduce the created count
|
|
143
|
+
def decrement_created
|
|
144
|
+
@created -= 1 unless @created == 0
|
|
145
|
+
end
|
|
146
|
+
|
|
115
147
|
private
|
|
116
148
|
|
|
117
149
|
def current_time
|
|
@@ -121,9 +153,18 @@ class ConnectionPool::TimedStack
|
|
|
121
153
|
##
|
|
122
154
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
123
155
|
#
|
|
124
|
-
# This method must returns
|
|
156
|
+
# This method must returns a connection from the stack if one exists. Allows
|
|
157
|
+
# subclasses with expensive match/search algorithms to avoid double-handling
|
|
158
|
+
# their stack.
|
|
159
|
+
def try_fetch_connection(**)
|
|
160
|
+
connection_stored?(**) && fetch_connection(**)
|
|
161
|
+
end
|
|
125
162
|
|
|
126
|
-
|
|
163
|
+
##
|
|
164
|
+
# This is an extension point for TimedStack and is called with a mutex.
|
|
165
|
+
#
|
|
166
|
+
# This method must returns true if a connection is available on the stack.
|
|
167
|
+
def connection_stored?(**)
|
|
127
168
|
!@que.empty?
|
|
128
169
|
end
|
|
129
170
|
|
|
@@ -131,31 +172,53 @@ class ConnectionPool::TimedStack
|
|
|
131
172
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
132
173
|
#
|
|
133
174
|
# This method must return a connection from the stack.
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
@que.pop
|
|
175
|
+
def fetch_connection(**)
|
|
176
|
+
@que.pop&.first
|
|
137
177
|
end
|
|
138
178
|
|
|
139
179
|
##
|
|
140
180
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
141
181
|
#
|
|
142
182
|
# This method must shut down all connections on the stack.
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
conn = fetch_connection(options)
|
|
183
|
+
def shutdown_connections(**)
|
|
184
|
+
while (conn = try_fetch_connection(**))
|
|
185
|
+
@created -= 1 unless @created == 0
|
|
147
186
|
@shutdown_block.call(conn)
|
|
148
187
|
end
|
|
149
|
-
@created = 0
|
|
150
188
|
end
|
|
151
189
|
|
|
152
190
|
##
|
|
153
191
|
# This is an extension point for TimedStack and is called with a mutex.
|
|
154
192
|
#
|
|
155
|
-
# This method
|
|
193
|
+
# This method returns the oldest idle connection if it has been idle for more than idle_seconds.
|
|
194
|
+
# This requires that the stack is kept in order of checked in time (oldest first).
|
|
195
|
+
def reserve_idle_connection(idle_seconds)
|
|
196
|
+
return unless idle_connections?(idle_seconds)
|
|
197
|
+
|
|
198
|
+
@created -= 1 unless @created == 0
|
|
156
199
|
|
|
157
|
-
|
|
158
|
-
|
|
200
|
+
# Most active elements are at the tail of the array.
|
|
201
|
+
# Most idle will be at the head so `shift` rather than `pop`.
|
|
202
|
+
@que.shift.first
|
|
203
|
+
end
|
|
204
|
+
|
|
205
|
+
##
|
|
206
|
+
# This is an extension point for TimedStack and is called with a mutex.
|
|
207
|
+
#
|
|
208
|
+
# Returns true if the first connection in the stack has been idle for more than idle_seconds
|
|
209
|
+
def idle_connections?(idle_seconds)
|
|
210
|
+
return unless connection_stored?
|
|
211
|
+
# Most idle will be at the head so `first`
|
|
212
|
+
age = (current_time - @que.first.last)
|
|
213
|
+
age > idle_seconds
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
##
|
|
217
|
+
# This is an extension point for TimedStack and is called with a mutex.
|
|
218
|
+
#
|
|
219
|
+
# This method must return +obj+ to the stack.
|
|
220
|
+
def store_connection(obj, **)
|
|
221
|
+
@que.push [obj, current_time]
|
|
159
222
|
end
|
|
160
223
|
|
|
161
224
|
##
|
|
@@ -163,8 +226,7 @@ class ConnectionPool::TimedStack
|
|
|
163
226
|
#
|
|
164
227
|
# This method must create a connection if and only if the total number of
|
|
165
228
|
# connections allowed has not been met.
|
|
166
|
-
|
|
167
|
-
def try_create(options = nil)
|
|
229
|
+
def try_create(**)
|
|
168
230
|
unless @created == @max
|
|
169
231
|
object = @create_block.call
|
|
170
232
|
@created += 1
|
|
@@ -2,20 +2,20 @@ class ConnectionPool
|
|
|
2
2
|
class Wrapper < ::BasicObject
|
|
3
3
|
METHODS = [:with, :pool_shutdown, :wrapped_pool]
|
|
4
4
|
|
|
5
|
-
def initialize(options
|
|
6
|
-
@pool = options.fetch(:pool) { ::ConnectionPool.new(options, &
|
|
5
|
+
def initialize(**options, &)
|
|
6
|
+
@pool = options.fetch(:pool) { ::ConnectionPool.new(**options, &) }
|
|
7
7
|
end
|
|
8
8
|
|
|
9
9
|
def wrapped_pool
|
|
10
10
|
@pool
|
|
11
11
|
end
|
|
12
12
|
|
|
13
|
-
def with(&
|
|
14
|
-
@pool.with(&
|
|
13
|
+
def with(**, &)
|
|
14
|
+
@pool.with(**, &)
|
|
15
15
|
end
|
|
16
16
|
|
|
17
|
-
def pool_shutdown(&
|
|
18
|
-
@pool.shutdown(&
|
|
17
|
+
def pool_shutdown(&)
|
|
18
|
+
@pool.shutdown(&)
|
|
19
19
|
end
|
|
20
20
|
|
|
21
21
|
def pool_size
|
|
@@ -26,31 +26,18 @@ class ConnectionPool
|
|
|
26
26
|
@pool.available
|
|
27
27
|
end
|
|
28
28
|
|
|
29
|
-
def respond_to?(id,
|
|
30
|
-
METHODS.include?(id) || with { |c| c.respond_to?(id,
|
|
29
|
+
def respond_to?(id, *, **)
|
|
30
|
+
METHODS.include?(id) || with { |c| c.respond_to?(id, *, **) }
|
|
31
31
|
end
|
|
32
32
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
elsif ::RUBY_VERSION >= "2.7.0"
|
|
41
|
-
ruby2_keywords def method_missing(name, *args, &block)
|
|
42
|
-
with do |connection|
|
|
43
|
-
connection.send(name, *args, &block)
|
|
44
|
-
end
|
|
45
|
-
end
|
|
46
|
-
else
|
|
47
|
-
def method_missing(name, *args, &block)
|
|
48
|
-
with do |connection|
|
|
49
|
-
connection.send(name, *args, &block)
|
|
50
|
-
end
|
|
33
|
+
def respond_to_missing?(id, *, **)
|
|
34
|
+
with { |c| c.respond_to?(id, *, **) }
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def method_missing(name, *, **, &)
|
|
38
|
+
with do |connection|
|
|
39
|
+
connection.send(name, *, **, &)
|
|
51
40
|
end
|
|
52
41
|
end
|
|
53
|
-
# rubocop:enable Style/MethodMissingSuper
|
|
54
|
-
# rubocop:enable Style/MissingRespondToMissing
|
|
55
42
|
end
|
|
56
43
|
end
|
data/lib/connection_pool.rb
CHANGED
|
@@ -39,72 +39,30 @@ end
|
|
|
39
39
|
# - :auto_reload_after_fork - automatically drop all connections after fork, defaults to true
|
|
40
40
|
#
|
|
41
41
|
class ConnectionPool
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
def self.wrap(options, &block)
|
|
45
|
-
Wrapper.new(options, &block)
|
|
46
|
-
end
|
|
47
|
-
|
|
48
|
-
if Process.respond_to?(:fork)
|
|
49
|
-
INSTANCES = ObjectSpace::WeakMap.new
|
|
50
|
-
private_constant :INSTANCES
|
|
51
|
-
|
|
52
|
-
def self.after_fork
|
|
53
|
-
INSTANCES.values.each do |pool|
|
|
54
|
-
next unless pool.auto_reload_after_fork
|
|
55
|
-
|
|
56
|
-
# We're on after fork, so we know all other threads are dead.
|
|
57
|
-
# All we need to do is to ensure the main thread doesn't have a
|
|
58
|
-
# checked out connection
|
|
59
|
-
pool.checkin(force: true)
|
|
60
|
-
pool.reload do |connection|
|
|
61
|
-
# Unfortunately we don't know what method to call to close the connection,
|
|
62
|
-
# so we try the most common one.
|
|
63
|
-
connection.close if connection.respond_to?(:close)
|
|
64
|
-
end
|
|
65
|
-
end
|
|
66
|
-
nil
|
|
67
|
-
end
|
|
68
|
-
|
|
69
|
-
if ::Process.respond_to?(:_fork) # MRI 3.1+
|
|
70
|
-
module ForkTracker
|
|
71
|
-
def _fork
|
|
72
|
-
pid = super
|
|
73
|
-
if pid == 0
|
|
74
|
-
ConnectionPool.after_fork
|
|
75
|
-
end
|
|
76
|
-
pid
|
|
77
|
-
end
|
|
78
|
-
end
|
|
79
|
-
Process.singleton_class.prepend(ForkTracker)
|
|
80
|
-
end
|
|
81
|
-
else
|
|
82
|
-
INSTANCES = nil
|
|
83
|
-
private_constant :INSTANCES
|
|
84
|
-
|
|
85
|
-
def self.after_fork
|
|
86
|
-
# noop
|
|
87
|
-
end
|
|
42
|
+
def self.wrap(**, &)
|
|
43
|
+
Wrapper.new(**, &)
|
|
88
44
|
end
|
|
89
45
|
|
|
90
|
-
|
|
91
|
-
raise ArgumentError, "Connection pool requires a block" unless block
|
|
92
|
-
|
|
93
|
-
options = DEFAULTS.merge(options)
|
|
46
|
+
attr_reader :size
|
|
94
47
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
@auto_reload_after_fork = options.fetch(:auto_reload_after_fork)
|
|
48
|
+
def initialize(timeout: 5, size: 5, auto_reload_after_fork: true, name: nil, &)
|
|
49
|
+
raise ArgumentError, "Connection pool requires a block" unless block_given?
|
|
98
50
|
|
|
99
|
-
@
|
|
51
|
+
@size = Integer(size)
|
|
52
|
+
@timeout = Float(timeout)
|
|
53
|
+
@available = TimedStack.new(size: @size, &)
|
|
100
54
|
@key = :"pool-#{@available.object_id}"
|
|
101
55
|
@key_count = :"pool-#{@available.object_id}-count"
|
|
102
|
-
|
|
56
|
+
@discard_key = :"pool-#{@available.object_id}-discard"
|
|
57
|
+
INSTANCES[self] = self if auto_reload_after_fork && INSTANCES
|
|
103
58
|
end
|
|
104
59
|
|
|
105
|
-
def with(
|
|
60
|
+
def with(**)
|
|
61
|
+
# We need to manage exception handling manually here in order
|
|
62
|
+
# to work correctly with `Timeout.timeout` and `Thread#raise`.
|
|
63
|
+
# Otherwise an interrupted Thread can leak connections.
|
|
106
64
|
Thread.handle_interrupt(Exception => :never) do
|
|
107
|
-
conn = checkout(
|
|
65
|
+
conn = checkout(**)
|
|
108
66
|
begin
|
|
109
67
|
Thread.handle_interrupt(Exception => :immediate) do
|
|
110
68
|
yield conn
|
|
@@ -116,20 +74,67 @@ class ConnectionPool
|
|
|
116
74
|
end
|
|
117
75
|
alias_method :then, :with
|
|
118
76
|
|
|
119
|
-
|
|
77
|
+
##
|
|
78
|
+
# Marks the current thread's checked-out connection for discard.
|
|
79
|
+
#
|
|
80
|
+
# When a connection is marked for discard, it will not be returned to the pool
|
|
81
|
+
# when checked in. Instead, the connection will be discarded.
|
|
82
|
+
# This is useful when a connection has become invalid or corrupted
|
|
83
|
+
# and should not be reused.
|
|
84
|
+
#
|
|
85
|
+
# Takes an optional block that will be called with the connection to be discarded.
|
|
86
|
+
# The block should perform any necessary clean-up on the connection.
|
|
87
|
+
#
|
|
88
|
+
# @yield [conn]
|
|
89
|
+
# @yieldparam conn [Object] The connection to be discarded.
|
|
90
|
+
# @yieldreturn [void]
|
|
91
|
+
#
|
|
92
|
+
#
|
|
93
|
+
# Note: This only affects the connection currently checked out by the calling thread.
|
|
94
|
+
# The connection will be discarded when +checkin+ is called.
|
|
95
|
+
#
|
|
96
|
+
# @return [void]
|
|
97
|
+
#
|
|
98
|
+
# @example
|
|
99
|
+
# pool.with do |conn|
|
|
100
|
+
# begin
|
|
101
|
+
# conn.execute("SELECT 1")
|
|
102
|
+
# rescue SomeConnectionError
|
|
103
|
+
# pool.discard_current_connection # Mark connection as bad
|
|
104
|
+
# raise
|
|
105
|
+
# end
|
|
106
|
+
# end
|
|
107
|
+
def discard_current_connection(&block)
|
|
108
|
+
::Thread.current[@discard_key] = block || proc { |conn| conn }
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
def checkout(timeout: @timeout, **)
|
|
120
112
|
if ::Thread.current[@key]
|
|
121
113
|
::Thread.current[@key_count] += 1
|
|
122
114
|
::Thread.current[@key]
|
|
123
115
|
else
|
|
116
|
+
conn = @available.pop(timeout:, **)
|
|
117
|
+
::Thread.current[@key] = conn
|
|
124
118
|
::Thread.current[@key_count] = 1
|
|
125
|
-
|
|
119
|
+
conn
|
|
126
120
|
end
|
|
127
121
|
end
|
|
128
122
|
|
|
129
123
|
def checkin(force: false)
|
|
130
124
|
if ::Thread.current[@key]
|
|
131
125
|
if ::Thread.current[@key_count] == 1 || force
|
|
132
|
-
|
|
126
|
+
if ::Thread.current[@discard_key]
|
|
127
|
+
begin
|
|
128
|
+
@available.decrement_created
|
|
129
|
+
::Thread.current[@discard_key].call(::Thread.current[@key])
|
|
130
|
+
rescue
|
|
131
|
+
nil
|
|
132
|
+
ensure
|
|
133
|
+
::Thread.current[@discard_key] = nil
|
|
134
|
+
end
|
|
135
|
+
else
|
|
136
|
+
@available.push(::Thread.current[@key])
|
|
137
|
+
end
|
|
133
138
|
::Thread.current[@key] = nil
|
|
134
139
|
::Thread.current[@key_count] = nil
|
|
135
140
|
else
|
|
@@ -146,30 +151,35 @@ class ConnectionPool
|
|
|
146
151
|
# Shuts down the ConnectionPool by passing each connection to +block+ and
|
|
147
152
|
# then removing it from the pool. Attempting to checkout a connection after
|
|
148
153
|
# shutdown will raise +ConnectionPool::PoolShuttingDownError+.
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
@available.shutdown(&block)
|
|
154
|
+
def shutdown(&)
|
|
155
|
+
@available.shutdown(&)
|
|
152
156
|
end
|
|
153
157
|
|
|
154
158
|
##
|
|
155
159
|
# Reloads the ConnectionPool by passing each connection to +block+ and then
|
|
156
160
|
# removing it the pool. Subsequent checkouts will create new connections as
|
|
157
161
|
# needed.
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
@available.shutdown(reload: true, &block)
|
|
162
|
+
def reload(&)
|
|
163
|
+
@available.shutdown(reload: true, &)
|
|
161
164
|
end
|
|
162
165
|
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
166
|
+
## Reaps idle connections that have been idle for over +idle_seconds+.
|
|
167
|
+
# +idle_seconds+ defaults to 60.
|
|
168
|
+
def reap(idle_seconds: 60, &)
|
|
169
|
+
@available.reap(idle_seconds:, &)
|
|
170
|
+
end
|
|
167
171
|
|
|
168
172
|
# Number of pool entries available for checkout at this instant.
|
|
169
173
|
def available
|
|
170
174
|
@available.length
|
|
171
175
|
end
|
|
176
|
+
|
|
177
|
+
# Number of pool entries created and idle in the pool.
|
|
178
|
+
def idle
|
|
179
|
+
@available.idle
|
|
180
|
+
end
|
|
172
181
|
end
|
|
173
182
|
|
|
174
183
|
require_relative "connection_pool/timed_stack"
|
|
175
184
|
require_relative "connection_pool/wrapper"
|
|
185
|
+
require_relative "connection_pool/fork"
|
metadata
CHANGED
|
@@ -1,15 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: connection_pool
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version:
|
|
4
|
+
version: 3.0.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Mike Perham
|
|
8
8
|
- Damian Janowski
|
|
9
|
-
autorequire:
|
|
10
9
|
bindir: bin
|
|
11
10
|
cert_chain: []
|
|
12
|
-
date:
|
|
11
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
13
12
|
dependencies:
|
|
14
13
|
- !ruby/object:Gem::Dependency
|
|
15
14
|
name: bundler
|
|
@@ -26,19 +25,19 @@ dependencies:
|
|
|
26
25
|
- !ruby/object:Gem::Version
|
|
27
26
|
version: '0'
|
|
28
27
|
- !ruby/object:Gem::Dependency
|
|
29
|
-
name:
|
|
28
|
+
name: maxitest
|
|
30
29
|
requirement: !ruby/object:Gem::Requirement
|
|
31
30
|
requirements:
|
|
32
31
|
- - ">="
|
|
33
32
|
- !ruby/object:Gem::Version
|
|
34
|
-
version:
|
|
33
|
+
version: '0'
|
|
35
34
|
type: :development
|
|
36
35
|
prerelease: false
|
|
37
36
|
version_requirements: !ruby/object:Gem::Requirement
|
|
38
37
|
requirements:
|
|
39
38
|
- - ">="
|
|
40
39
|
- !ruby/object:Gem::Version
|
|
41
|
-
version:
|
|
40
|
+
version: '0'
|
|
42
41
|
- !ruby/object:Gem::Dependency
|
|
43
42
|
name: rake
|
|
44
43
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -66,6 +65,7 @@ files:
|
|
|
66
65
|
- README.md
|
|
67
66
|
- connection_pool.gemspec
|
|
68
67
|
- lib/connection_pool.rb
|
|
68
|
+
- lib/connection_pool/fork.rb
|
|
69
69
|
- lib/connection_pool/timed_stack.rb
|
|
70
70
|
- lib/connection_pool/version.rb
|
|
71
71
|
- lib/connection_pool/wrapper.rb
|
|
@@ -73,9 +73,12 @@ homepage: https://github.com/mperham/connection_pool
|
|
|
73
73
|
licenses:
|
|
74
74
|
- MIT
|
|
75
75
|
metadata:
|
|
76
|
+
bug_tracker_uri: https://github.com/mperham/connection_pool/issues
|
|
77
|
+
documentation_uri: https://github.com/mperham/connection_pool/wiki
|
|
76
78
|
changelog_uri: https://github.com/mperham/connection_pool/blob/main/Changes.md
|
|
79
|
+
source_code_uri: https://github.com/mperham/connection_pool
|
|
80
|
+
homepage_uri: https://github.com/mperham/connection_pool
|
|
77
81
|
rubygems_mfa_required: 'true'
|
|
78
|
-
post_install_message:
|
|
79
82
|
rdoc_options: []
|
|
80
83
|
require_paths:
|
|
81
84
|
- lib
|
|
@@ -83,15 +86,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
83
86
|
requirements:
|
|
84
87
|
- - ">="
|
|
85
88
|
- !ruby/object:Gem::Version
|
|
86
|
-
version: 2.
|
|
89
|
+
version: 3.2.0
|
|
87
90
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
88
91
|
requirements:
|
|
89
92
|
- - ">="
|
|
90
93
|
- !ruby/object:Gem::Version
|
|
91
94
|
version: '0'
|
|
92
95
|
requirements: []
|
|
93
|
-
rubygems_version: 3.
|
|
94
|
-
signing_key:
|
|
96
|
+
rubygems_version: 3.6.9
|
|
95
97
|
specification_version: 4
|
|
96
98
|
summary: Generic connection pool for Ruby
|
|
97
99
|
test_files: []
|