connection_pool 2.4.1 → 3.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ea0776fcb09a3cc48ef4ca03774399e20b09e51039d0c47c1e4cb3bac621c52b
4
- data.tar.gz: b955d6b4e984259f20ae8cf6414f59692f9a51848424231363643e0c16dd2a3f
3
+ metadata.gz: a064d41333b8b92fcb23617701011f1b34e6348b324048ca16e5cb758d31f05f
4
+ data.tar.gz: 7ca1cc56ff7d020f2b7f2cee01b2008502faadf563dad010c2b8eee4a0b83dcd
5
5
  SHA512:
6
- metadata.gz: bf57d8b5547502d91f5550ca6ea0be16905604c90e61efb6741e5ec3ce607c7a65f0b31e1673c96c60a06a2f64f5239cab6e94d3a50095fb822ea9b1c1bb2f0a
7
- data.tar.gz: 4b42aa5aa67b0e45bbbc8a9f29ca3a969efd8ade3b6dfca6cff082f526ec65a2a2e5c8fa17f512d33470b82535af8b675fc80903ccd75db26748e9845dd9a612
6
+ metadata.gz: 1c7554d540f6aefcd356154246a1cdcab5e4ddea745e9a580261fba0635e6b711a9e0781691503f20aca7faaf982e2ba8f5f48cc71ff2a4f67053039e947ba3d
7
+ data.tar.gz: 431dcf74c39f6a9db1503290a227f5780c4b03e392ef25709245e63d0ca46a2f0f6cc0792a5241e615eafd92663aa38c1165aae99f6dffb0783f5b442846582f
data/Changes.md CHANGED
@@ -1,5 +1,68 @@
1
1
  # connection_pool Changelog
2
2
 
3
+ 3.0.2
4
+ ------
5
+
6
+ - Support :name keyword for backwards compatibility [#210]
7
+
8
+ 3.0.1
9
+ ------
10
+
11
+ - Add missing `fork.rb` to gemspec.
12
+
13
+ 3.0.0
14
+ ------
15
+
16
+ - **BREAKING CHANGES** `ConnectionPool` and `ConnectionPool::TimedStack` now
17
+ use keyword arguments rather than positional arguments everywhere. Expected impact is minimal as most people use the `with` API, which is unchanged.
18
+ ```ruby
19
+ pool = ConnectionPool.new(size: 5, timeout: 5)
20
+ pool.checkout(1) # 2.x
21
+ pool.reap(30) # 2.x
22
+ pool.checkout(timeout: 1) # 3.x
23
+ pool.reap(idle_seconds: 30) # 3.x
24
+ ```
25
+ - Dropped support for Ruby <3.2.0
26
+
27
+ 2.5.5
28
+ ------
29
+
30
+ - Support `ConnectionPool::TimedStack#pop(exception: false)` [#207]
31
+ to avoid using exceptions as control flow.
32
+
33
+ 2.5.4
34
+ ------
35
+
36
+ - Add ability to remove a broken connection from the pool [#204, womblep]
37
+
38
+ 2.5.3
39
+ ------
40
+
41
+ - Fix TruffleRuby/JRuby crash [#201]
42
+
43
+ 2.5.2
44
+ ------
45
+
46
+ - Rollback inadvertant change to `auto_reload_after_fork` default. [#200]
47
+
48
+ 2.5.1
49
+ ------
50
+
51
+ - Pass options to TimedStack in `checkout` [#195]
52
+ - Optimize connection lookup [#196]
53
+ - Fixes for use with Ractors
54
+
55
+ 2.5.0
56
+ ------
57
+
58
+ - Reap idle connections [#187]
59
+ ```ruby
60
+ idle_timeout = 60
61
+ pool = ConnectionPool.new ...
62
+ pool.reap(idle_timeout, &:close)
63
+ ```
64
+ - `ConnectionPool#idle` returns the count of connections not in use [#187]
65
+
3
66
  2.4.1
4
67
  ------
5
68
 
data/README.md CHANGED
@@ -38,7 +38,7 @@ connection pool and a raw client.
38
38
  $redis.then { |r| r.set 'foo' 'bar' }
39
39
  ```
40
40
 
41
- Optionally, you can specify a timeout override using the with-block semantics:
41
+ Optionally, you can specify a timeout override:
42
42
 
43
43
  ``` ruby
44
44
  $memcached.with(timeout: 2.0) do |conn|
@@ -46,11 +46,9 @@ $memcached.with(timeout: 2.0) do |conn|
46
46
  end
47
47
  ```
48
48
 
49
- This will only modify the resource-get timeout for this particular
50
- invocation.
49
+ This will only modify the timeout for this particular invocation.
51
50
  This is useful if you want to fail-fast on certain non-critical
52
51
  sections when a resource is not available, or conversely if you are comfortable blocking longer on a particular resource.
53
- This is not implemented in the `ConnectionPool::Wrapper` class.
54
52
 
55
53
  ## Migrating to a Connection Pool
56
54
 
@@ -72,7 +70,7 @@ $redis.with do |conn|
72
70
  end
73
71
  ```
74
72
 
75
- Once you've ported your entire system to use `with`, you can simply remove `Wrapper` and use the simpler and faster `ConnectionPool`.
73
+ Once you've ported your entire system to use `with`, you can remove `::Wrapper` and use `ConnectionPool` directly.
76
74
 
77
75
 
78
76
  ## Shutdown
@@ -90,16 +88,53 @@ Shutting down a connection pool will block until all connections are checked in
90
88
 
91
89
  ## Reload
92
90
 
93
- You can reload a ConnectionPool instance in the case it is desired to close all connections to the pool and, unlike `shutdown`, afterwards recreate connections so the pool may continue to be used.
94
- Reloading may be useful after forking the process.
91
+ You can reload a ConnectionPool instance if it is necessary to close all existing connections and continue to use the pool.
92
+ ConnectionPool will automatically reload if the process is forked.
93
+ Use `auto_reload_after_fork: false` if you don't want this behavior.
95
94
 
96
95
  ```ruby
97
- cp = ConnectionPool.new { Redis.new }
98
- cp.reload { |conn| conn.quit }
96
+ cp = ConnectionPool.new(auto_reload_after_fork: false) { Redis.new }
97
+ cp.reload { |conn| conn.quit } # reload manually
99
98
  cp.with { |conn| conn.get('some-count') }
100
99
  ```
101
100
 
102
- Like `shutdown`, this will block until all connections are checked in and closed.
101
+ Like `shutdown`, `reload` will block until all connections are checked in and closed.
102
+
103
+ ## Reap
104
+
105
+ You can call `reap` periodically on the ConnectionPool instance to close connections that were created but have not been used for a certain amount of time. This can be useful in environments where connections are expensive.
106
+
107
+ You can specify how many seconds the connections have to be idle for them to be reaped, defaulting to 60 seconds.
108
+
109
+ ```ruby
110
+ cp = ConnectionPool.new { Redis.new }
111
+
112
+ # Start a reaper thread to reap connections that have been
113
+ # idle more than 300 seconds (5 minutes)
114
+ Thread.new do
115
+ loop do
116
+ cp.reap(idle_seconds: 300, &:close)
117
+ sleep 30
118
+ end
119
+ end
120
+ ```
121
+
122
+ ## Discarding Connections
123
+
124
+ You can discard connections in the ConnectionPool instance to remove connections that are broken and can't be repaired.
125
+ It can only be done inside the block passed to `with`.
126
+ Takes an optional block that will be executed with the connection.
127
+
128
+ ```ruby
129
+ pool.with do |conn|
130
+ begin
131
+ conn.execute("SELECT 1")
132
+ rescue SomeConnectionError
133
+ pool.discard_current_connection(&:close) # remove the connection from the pool
134
+ raise
135
+ end
136
+ end
137
+ ```
103
138
 
104
139
  ## Current State
105
140
 
@@ -109,27 +144,39 @@ There are several methods that return information about a pool.
109
144
  cp = ConnectionPool.new(size: 10) { Redis.new }
110
145
  cp.size # => 10
111
146
  cp.available # => 10
147
+ cp.idle # => 0
112
148
 
113
149
  cp.with do |conn|
114
150
  cp.size # => 10
115
151
  cp.available # => 9
152
+ cp.idle # => 0
116
153
  end
154
+
155
+ cp.idle # => 1
117
156
  ```
118
157
 
119
- Notes
120
- -----
158
+ ## Upgrading from ConnectionPool 2
159
+
160
+ * Support for Ruby <3.2 has been removed.
161
+ * ConnectionPool's APIs now consistently use keyword arguments everywhere.
162
+ Positional arguments must be converted to keywords:
163
+ ```ruby
164
+ pool = ConnectionPool.new(size: 5, timeout: 5)
165
+ pool.checkout(1) # 2.x
166
+ pool.reap(30) # 2.x
167
+ pool.checkout(timeout: 1) # 3.x
168
+ pool.reap(idle_seconds: 30) # 3.x
169
+ ```
170
+
171
+ ## Notes
121
172
 
122
173
  - Connections are lazily created as needed.
123
- - There is no provision for repairing or checking the health of a connection;
124
- connections should be self-repairing. This is true of the Dalli and Redis
125
- clients.
126
- - **WARNING**: Don't ever use `Timeout.timeout` in your Ruby code or you will see
174
+ - **WARNING**: Avoid `Timeout.timeout` in your Ruby code or you can see
127
175
  occasional silent corruption and mysterious errors. The Timeout API is unsafe
128
- and cannot be used correctly, ever. Use proper socket timeout options as
129
- exposed by Net::HTTP, Redis, Dalli, etc.
176
+ and dangerous to use. Use proper socket timeout options as exposed by
177
+ Net::HTTP, Redis, Dalli, etc.
130
178
 
131
179
 
132
- Author
133
- ------
180
+ ## Author
134
181
 
135
- Mike Perham, [@getajobmike](https://twitter.com/getajobmike), <https://www.mikeperham.com>
182
+ Mike Perham, [@getajobmike](https://ruby.social/@getajobmike), <https://www.mikeperham.com>
@@ -10,15 +10,26 @@ Gem::Specification.new do |s|
10
10
  s.description = s.summary = "Generic connection pool for Ruby"
11
11
 
12
12
  s.files = ["Changes.md", "LICENSE", "README.md", "connection_pool.gemspec",
13
- "lib/connection_pool.rb", "lib/connection_pool/timed_stack.rb",
14
- "lib/connection_pool/version.rb", "lib/connection_pool/wrapper.rb"]
13
+ "lib/connection_pool.rb",
14
+ "lib/connection_pool/timed_stack.rb",
15
+ "lib/connection_pool/version.rb",
16
+ "lib/connection_pool/fork.rb",
17
+ "lib/connection_pool/wrapper.rb"]
15
18
  s.executables = []
16
19
  s.require_paths = ["lib"]
17
20
  s.license = "MIT"
21
+
22
+ s.required_ruby_version = ">= 3.2.0"
18
23
  s.add_development_dependency "bundler"
19
- s.add_development_dependency "minitest", ">= 5.0.0"
24
+ s.add_development_dependency "maxitest"
20
25
  s.add_development_dependency "rake"
21
- s.required_ruby_version = ">= 2.5.0"
22
26
 
23
- s.metadata = {"changelog_uri" => "https://github.com/mperham/connection_pool/blob/main/Changes.md", "rubygems_mfa_required" => "true"}
27
+ s.metadata = {
28
+ "bug_tracker_uri" => "https://github.com/mperham/connection_pool/issues",
29
+ "documentation_uri" => "https://github.com/mperham/connection_pool/wiki",
30
+ "changelog_uri" => "https://github.com/mperham/connection_pool/blob/main/Changes.md",
31
+ "source_code_uri" => "https://github.com/mperham/connection_pool",
32
+ "homepage_uri" => "https://github.com/mperham/connection_pool",
33
+ "rubygems_mfa_required" => "true"
34
+ }
24
35
  end
@@ -0,0 +1,40 @@
1
+ class ConnectionPool
2
+ if Process.respond_to?(:fork)
3
+ INSTANCES = ObjectSpace::WeakMap.new
4
+ private_constant :INSTANCES
5
+
6
+ def self.after_fork
7
+ INSTANCES.each_value do |pool|
8
+ # We're in after_fork, so we know all other threads are dead.
9
+ # All we need to do is ensure the main thread doesn't have a
10
+ # checked out connection
11
+ pool.checkin(force: true)
12
+ pool.reload do |connection|
13
+ # Unfortunately we don't know what method to call to close the connection,
14
+ # so we try the most common one.
15
+ connection.close if connection.respond_to?(:close)
16
+ end
17
+ end
18
+ nil
19
+ end
20
+
21
+ module ForkTracker
22
+ def _fork
23
+ pid = super
24
+ if pid == 0
25
+ ConnectionPool.after_fork
26
+ end
27
+ pid
28
+ end
29
+ end
30
+ Process.singleton_class.prepend(ForkTracker)
31
+ else
32
+ # JRuby, et al
33
+ INSTANCES = nil
34
+ private_constant :INSTANCES
35
+
36
+ def self.after_fork
37
+ # noop
38
+ end
39
+ end
40
+ end
@@ -1,11 +1,11 @@
1
1
  ##
2
2
  # The TimedStack manages a pool of homogeneous connections (or any resource
3
- # you wish to manage). Connections are created lazily up to a given maximum
3
+ # you wish to manage). Connections are created lazily up to a given maximum
4
4
  # number.
5
-
5
+ #
6
6
  # Examples:
7
7
  #
8
- # ts = TimedStack.new(1) { MyConnection.new }
8
+ # ts = TimedStack.new(size: 1) { MyConnection.new }
9
9
  #
10
10
  # # fetch a connection
11
11
  # conn = ts.pop
@@ -16,15 +16,13 @@
16
16
  # conn = ts.pop
17
17
  # ts.pop timeout: 5
18
18
  # #=> raises ConnectionPool::TimeoutError after 5 seconds
19
-
20
19
  class ConnectionPool::TimedStack
21
20
  attr_reader :max
22
21
 
23
22
  ##
24
23
  # Creates a new pool with +size+ connections that are created from the given
25
24
  # +block+.
26
-
27
- def initialize(size = 0, &block)
25
+ def initialize(size: 0, &block)
28
26
  @create_block = block
29
27
  @created = 0
30
28
  @que = []
@@ -35,15 +33,15 @@ class ConnectionPool::TimedStack
35
33
  end
36
34
 
37
35
  ##
38
- # Returns +obj+ to the stack. +options+ is ignored in TimedStack but may be
36
+ # Returns +obj+ to the stack. Additional kwargs are ignored in TimedStack but may be
39
37
  # used by subclasses that extend TimedStack.
40
-
41
- def push(obj, options = {})
38
+ def push(obj, **)
42
39
  @mutex.synchronize do
43
40
  if @shutdown_block
41
+ @created -= 1 unless @created == 0
44
42
  @shutdown_block.call(obj)
45
43
  else
46
- store_connection obj, options
44
+ store_connection obj, **
47
45
  end
48
46
 
49
47
  @resource.broadcast
@@ -52,29 +50,35 @@ class ConnectionPool::TimedStack
52
50
  alias_method :<<, :push
53
51
 
54
52
  ##
55
- # Retrieves a connection from the stack. If a connection is available it is
56
- # immediately returned. If no connection is available within the given
53
+ # Retrieves a connection from the stack. If a connection is available it is
54
+ # immediately returned. If no connection is available within the given
57
55
  # timeout a ConnectionPool::TimeoutError is raised.
58
56
  #
59
- # +:timeout+ is the only checked entry in +options+ and is preferred over
60
- # the +timeout+ argument (which will be removed in a future release). Other
61
- # options may be used by subclasses that extend TimedStack.
62
-
63
- def pop(timeout = 0.5, options = {})
64
- options, timeout = timeout, 0.5 if Hash === timeout
65
- timeout = options.fetch :timeout, timeout
66
-
57
+ # @option options [Float] :timeout (0.5) Wait this many seconds for an available entry
58
+ # @option options [Class] :exception (ConnectionPool::TimeoutError) Exception class to raise
59
+ # if an entry was not available within the timeout period. Use `exception: false` to return nil.
60
+ #
61
+ # Other options may be used by subclasses that extend TimedStack.
62
+ def pop(timeout: 0.5, exception: ConnectionPool::TimeoutError, **)
67
63
  deadline = current_time + timeout
68
64
  @mutex.synchronize do
69
65
  loop do
70
66
  raise ConnectionPool::PoolShuttingDownError if @shutdown_block
71
- return fetch_connection(options) if connection_stored?(options)
67
+ if (conn = try_fetch_connection(**))
68
+ return conn
69
+ end
72
70
 
73
- connection = try_create(options)
71
+ connection = try_create(**)
74
72
  return connection if connection
75
73
 
76
74
  to_wait = deadline - current_time
77
- raise ConnectionPool::TimeoutError, "Waited #{timeout} sec, #{length}/#{@max} available" if to_wait <= 0
75
+ if to_wait <= 0
76
+ if exception
77
+ raise exception, "Waited #{timeout} sec, #{length}/#{@max} available"
78
+ else
79
+ return nil
80
+ end
81
+ end
78
82
  @resource.wait(@mutex, to_wait)
79
83
  end
80
84
  end
@@ -85,7 +89,6 @@ class ConnectionPool::TimedStack
85
89
  # removing it from the pool. Attempting to checkout a connection after
86
90
  # shutdown will raise +ConnectionPool::PoolShuttingDownError+ unless
87
91
  # +:reload+ is +true+.
88
-
89
92
  def shutdown(reload: false, &block)
90
93
  raise ArgumentError, "shutdown must receive a block" unless block
91
94
 
@@ -99,19 +102,48 @@ class ConnectionPool::TimedStack
99
102
  end
100
103
 
101
104
  ##
102
- # Returns +true+ if there are no available connections.
105
+ # Reaps connections that were checked in more than +idle_seconds+ ago.
106
+ def reap(idle_seconds:)
107
+ raise ArgumentError, "reap must receive a block" unless block_given?
108
+ raise ArgumentError, "idle_seconds must be a number" unless idle_seconds.is_a?(Numeric)
109
+ raise ConnectionPool::PoolShuttingDownError if @shutdown_block
110
+
111
+ count = idle
112
+ count.times do
113
+ conn = @mutex.synchronize do
114
+ raise ConnectionPool::PoolShuttingDownError if @shutdown_block
115
+ reserve_idle_connection(idle_seconds)
116
+ end
117
+ break unless conn
103
118
 
119
+ yield conn
120
+ end
121
+ end
122
+
123
+ ##
124
+ # Returns +true+ if there are no available connections.
104
125
  def empty?
105
126
  (@created - @que.length) >= @max
106
127
  end
107
128
 
108
129
  ##
109
130
  # The number of connections available on the stack.
110
-
111
131
  def length
112
132
  @max - @created + @que.length
113
133
  end
114
134
 
135
+ ##
136
+ # The number of connections created and available on the stack.
137
+ def idle
138
+ @que.length
139
+ end
140
+
141
+ ##
142
+ # Reduce the created count
143
+ def decrement_created
144
+ @created -= 1 unless @created == 0
145
+ end
146
+
115
147
  private
116
148
 
117
149
  def current_time
@@ -121,9 +153,18 @@ class ConnectionPool::TimedStack
121
153
  ##
122
154
  # This is an extension point for TimedStack and is called with a mutex.
123
155
  #
124
- # This method must returns true if a connection is available on the stack.
156
+ # This method must returns a connection from the stack if one exists. Allows
157
+ # subclasses with expensive match/search algorithms to avoid double-handling
158
+ # their stack.
159
+ def try_fetch_connection(**)
160
+ connection_stored?(**) && fetch_connection(**)
161
+ end
125
162
 
126
- def connection_stored?(options = nil)
163
+ ##
164
+ # This is an extension point for TimedStack and is called with a mutex.
165
+ #
166
+ # This method must returns true if a connection is available on the stack.
167
+ def connection_stored?(**)
127
168
  !@que.empty?
128
169
  end
129
170
 
@@ -131,31 +172,53 @@ class ConnectionPool::TimedStack
131
172
  # This is an extension point for TimedStack and is called with a mutex.
132
173
  #
133
174
  # This method must return a connection from the stack.
134
-
135
- def fetch_connection(options = nil)
136
- @que.pop
175
+ def fetch_connection(**)
176
+ @que.pop&.first
137
177
  end
138
178
 
139
179
  ##
140
180
  # This is an extension point for TimedStack and is called with a mutex.
141
181
  #
142
182
  # This method must shut down all connections on the stack.
143
-
144
- def shutdown_connections(options = nil)
145
- while connection_stored?(options)
146
- conn = fetch_connection(options)
183
+ def shutdown_connections(**)
184
+ while (conn = try_fetch_connection(**))
185
+ @created -= 1 unless @created == 0
147
186
  @shutdown_block.call(conn)
148
187
  end
149
- @created = 0
150
188
  end
151
189
 
152
190
  ##
153
191
  # This is an extension point for TimedStack and is called with a mutex.
154
192
  #
155
- # This method must return +obj+ to the stack.
193
+ # This method returns the oldest idle connection if it has been idle for more than idle_seconds.
194
+ # This requires that the stack is kept in order of checked in time (oldest first).
195
+ def reserve_idle_connection(idle_seconds)
196
+ return unless idle_connections?(idle_seconds)
197
+
198
+ @created -= 1 unless @created == 0
156
199
 
157
- def store_connection(obj, options = nil)
158
- @que.push obj
200
+ # Most active elements are at the tail of the array.
201
+ # Most idle will be at the head so `shift` rather than `pop`.
202
+ @que.shift.first
203
+ end
204
+
205
+ ##
206
+ # This is an extension point for TimedStack and is called with a mutex.
207
+ #
208
+ # Returns true if the first connection in the stack has been idle for more than idle_seconds
209
+ def idle_connections?(idle_seconds)
210
+ return unless connection_stored?
211
+ # Most idle will be at the head so `first`
212
+ age = (current_time - @que.first.last)
213
+ age > idle_seconds
214
+ end
215
+
216
+ ##
217
+ # This is an extension point for TimedStack and is called with a mutex.
218
+ #
219
+ # This method must return +obj+ to the stack.
220
+ def store_connection(obj, **)
221
+ @que.push [obj, current_time]
159
222
  end
160
223
 
161
224
  ##
@@ -163,8 +226,7 @@ class ConnectionPool::TimedStack
163
226
  #
164
227
  # This method must create a connection if and only if the total number of
165
228
  # connections allowed has not been met.
166
-
167
- def try_create(options = nil)
229
+ def try_create(**)
168
230
  unless @created == @max
169
231
  object = @create_block.call
170
232
  @created += 1
@@ -1,3 +1,3 @@
1
1
  class ConnectionPool
2
- VERSION = "2.4.1"
2
+ VERSION = "3.0.2"
3
3
  end
@@ -2,20 +2,20 @@ class ConnectionPool
2
2
  class Wrapper < ::BasicObject
3
3
  METHODS = [:with, :pool_shutdown, :wrapped_pool]
4
4
 
5
- def initialize(options = {}, &block)
6
- @pool = options.fetch(:pool) { ::ConnectionPool.new(options, &block) }
5
+ def initialize(**options, &)
6
+ @pool = options.fetch(:pool) { ::ConnectionPool.new(**options, &) }
7
7
  end
8
8
 
9
9
  def wrapped_pool
10
10
  @pool
11
11
  end
12
12
 
13
- def with(&block)
14
- @pool.with(&block)
13
+ def with(**, &)
14
+ @pool.with(**, &)
15
15
  end
16
16
 
17
- def pool_shutdown(&block)
18
- @pool.shutdown(&block)
17
+ def pool_shutdown(&)
18
+ @pool.shutdown(&)
19
19
  end
20
20
 
21
21
  def pool_size
@@ -26,31 +26,18 @@ class ConnectionPool
26
26
  @pool.available
27
27
  end
28
28
 
29
- def respond_to?(id, *args)
30
- METHODS.include?(id) || with { |c| c.respond_to?(id, *args) }
29
+ def respond_to?(id, *, **)
30
+ METHODS.include?(id) || with { |c| c.respond_to?(id, *, **) }
31
31
  end
32
32
 
33
- # rubocop:disable Style/MissingRespondToMissing
34
- if ::RUBY_VERSION >= "3.0.0"
35
- def method_missing(name, *args, **kwargs, &block)
36
- with do |connection|
37
- connection.send(name, *args, **kwargs, &block)
38
- end
39
- end
40
- elsif ::RUBY_VERSION >= "2.7.0"
41
- ruby2_keywords def method_missing(name, *args, &block)
42
- with do |connection|
43
- connection.send(name, *args, &block)
44
- end
45
- end
46
- else
47
- def method_missing(name, *args, &block)
48
- with do |connection|
49
- connection.send(name, *args, &block)
50
- end
33
+ def respond_to_missing?(id, *, **)
34
+ with { |c| c.respond_to?(id, *, **) }
35
+ end
36
+
37
+ def method_missing(name, *, **, &)
38
+ with do |connection|
39
+ connection.send(name, *, **, &)
51
40
  end
52
41
  end
53
- # rubocop:enable Style/MethodMissingSuper
54
- # rubocop:enable Style/MissingRespondToMissing
55
42
  end
56
43
  end
@@ -39,72 +39,30 @@ end
39
39
  # - :auto_reload_after_fork - automatically drop all connections after fork, defaults to true
40
40
  #
41
41
  class ConnectionPool
42
- DEFAULTS = {size: 5, timeout: 5, auto_reload_after_fork: true}
43
-
44
- def self.wrap(options, &block)
45
- Wrapper.new(options, &block)
46
- end
47
-
48
- if Process.respond_to?(:fork)
49
- INSTANCES = ObjectSpace::WeakMap.new
50
- private_constant :INSTANCES
51
-
52
- def self.after_fork
53
- INSTANCES.values.each do |pool|
54
- next unless pool.auto_reload_after_fork
55
-
56
- # We're on after fork, so we know all other threads are dead.
57
- # All we need to do is to ensure the main thread doesn't have a
58
- # checked out connection
59
- pool.checkin(force: true)
60
- pool.reload do |connection|
61
- # Unfortunately we don't know what method to call to close the connection,
62
- # so we try the most common one.
63
- connection.close if connection.respond_to?(:close)
64
- end
65
- end
66
- nil
67
- end
68
-
69
- if ::Process.respond_to?(:_fork) # MRI 3.1+
70
- module ForkTracker
71
- def _fork
72
- pid = super
73
- if pid == 0
74
- ConnectionPool.after_fork
75
- end
76
- pid
77
- end
78
- end
79
- Process.singleton_class.prepend(ForkTracker)
80
- end
81
- else
82
- INSTANCES = nil
83
- private_constant :INSTANCES
84
-
85
- def self.after_fork
86
- # noop
87
- end
42
+ def self.wrap(**, &)
43
+ Wrapper.new(**, &)
88
44
  end
89
45
 
90
- def initialize(options = {}, &block)
91
- raise ArgumentError, "Connection pool requires a block" unless block
92
-
93
- options = DEFAULTS.merge(options)
46
+ attr_reader :size
94
47
 
95
- @size = Integer(options.fetch(:size))
96
- @timeout = options.fetch(:timeout)
97
- @auto_reload_after_fork = options.fetch(:auto_reload_after_fork)
48
+ def initialize(timeout: 5, size: 5, auto_reload_after_fork: true, name: nil, &)
49
+ raise ArgumentError, "Connection pool requires a block" unless block_given?
98
50
 
99
- @available = TimedStack.new(@size, &block)
51
+ @size = Integer(size)
52
+ @timeout = Float(timeout)
53
+ @available = TimedStack.new(size: @size, &)
100
54
  @key = :"pool-#{@available.object_id}"
101
55
  @key_count = :"pool-#{@available.object_id}-count"
102
- INSTANCES[self] = self if INSTANCES
56
+ @discard_key = :"pool-#{@available.object_id}-discard"
57
+ INSTANCES[self] = self if auto_reload_after_fork && INSTANCES
103
58
  end
104
59
 
105
- def with(options = {})
60
+ def with(**)
61
+ # We need to manage exception handling manually here in order
62
+ # to work correctly with `Timeout.timeout` and `Thread#raise`.
63
+ # Otherwise an interrupted Thread can leak connections.
106
64
  Thread.handle_interrupt(Exception => :never) do
107
- conn = checkout(options)
65
+ conn = checkout(**)
108
66
  begin
109
67
  Thread.handle_interrupt(Exception => :immediate) do
110
68
  yield conn
@@ -116,20 +74,67 @@ class ConnectionPool
116
74
  end
117
75
  alias_method :then, :with
118
76
 
119
- def checkout(options = {})
77
+ ##
78
+ # Marks the current thread's checked-out connection for discard.
79
+ #
80
+ # When a connection is marked for discard, it will not be returned to the pool
81
+ # when checked in. Instead, the connection will be discarded.
82
+ # This is useful when a connection has become invalid or corrupted
83
+ # and should not be reused.
84
+ #
85
+ # Takes an optional block that will be called with the connection to be discarded.
86
+ # The block should perform any necessary clean-up on the connection.
87
+ #
88
+ # @yield [conn]
89
+ # @yieldparam conn [Object] The connection to be discarded.
90
+ # @yieldreturn [void]
91
+ #
92
+ #
93
+ # Note: This only affects the connection currently checked out by the calling thread.
94
+ # The connection will be discarded when +checkin+ is called.
95
+ #
96
+ # @return [void]
97
+ #
98
+ # @example
99
+ # pool.with do |conn|
100
+ # begin
101
+ # conn.execute("SELECT 1")
102
+ # rescue SomeConnectionError
103
+ # pool.discard_current_connection # Mark connection as bad
104
+ # raise
105
+ # end
106
+ # end
107
+ def discard_current_connection(&block)
108
+ ::Thread.current[@discard_key] = block || proc { |conn| conn }
109
+ end
110
+
111
+ def checkout(timeout: @timeout, **)
120
112
  if ::Thread.current[@key]
121
113
  ::Thread.current[@key_count] += 1
122
114
  ::Thread.current[@key]
123
115
  else
116
+ conn = @available.pop(timeout:, **)
117
+ ::Thread.current[@key] = conn
124
118
  ::Thread.current[@key_count] = 1
125
- ::Thread.current[@key] = @available.pop(options[:timeout] || @timeout)
119
+ conn
126
120
  end
127
121
  end
128
122
 
129
123
  def checkin(force: false)
130
124
  if ::Thread.current[@key]
131
125
  if ::Thread.current[@key_count] == 1 || force
132
- @available.push(::Thread.current[@key])
126
+ if ::Thread.current[@discard_key]
127
+ begin
128
+ @available.decrement_created
129
+ ::Thread.current[@discard_key].call(::Thread.current[@key])
130
+ rescue
131
+ nil
132
+ ensure
133
+ ::Thread.current[@discard_key] = nil
134
+ end
135
+ else
136
+ @available.push(::Thread.current[@key])
137
+ end
133
138
  ::Thread.current[@key] = nil
134
139
  ::Thread.current[@key_count] = nil
135
140
  else
@@ -146,30 +151,35 @@ class ConnectionPool
146
151
  # Shuts down the ConnectionPool by passing each connection to +block+ and
147
152
  # then removing it from the pool. Attempting to checkout a connection after
148
153
  # shutdown will raise +ConnectionPool::PoolShuttingDownError+.
149
-
150
- def shutdown(&block)
151
- @available.shutdown(&block)
154
+ def shutdown(&)
155
+ @available.shutdown(&)
152
156
  end
153
157
 
154
158
  ##
155
159
  # Reloads the ConnectionPool by passing each connection to +block+ and then
156
160
  # removing it the pool. Subsequent checkouts will create new connections as
157
161
  # needed.
158
-
159
- def reload(&block)
160
- @available.shutdown(reload: true, &block)
162
+ def reload(&)
163
+ @available.shutdown(reload: true, &)
161
164
  end
162
165
 
163
- # Size of this connection pool
164
- attr_reader :size
165
- # Automatically drop all connections after fork
166
- attr_reader :auto_reload_after_fork
166
+ ## Reaps idle connections that have been idle for over +idle_seconds+.
167
+ # +idle_seconds+ defaults to 60.
168
+ def reap(idle_seconds: 60, &)
169
+ @available.reap(idle_seconds:, &)
170
+ end
167
171
 
168
172
  # Number of pool entries available for checkout at this instant.
169
173
  def available
170
174
  @available.length
171
175
  end
176
+
177
+ # Number of pool entries created and idle in the pool.
178
+ def idle
179
+ @available.idle
180
+ end
172
181
  end
173
182
 
174
183
  require_relative "connection_pool/timed_stack"
175
184
  require_relative "connection_pool/wrapper"
185
+ require_relative "connection_pool/fork"
metadata CHANGED
@@ -1,15 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: connection_pool
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.4.1
4
+ version: 3.0.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mike Perham
8
8
  - Damian Janowski
9
- autorequire:
10
9
  bindir: bin
11
10
  cert_chain: []
12
- date: 2023-05-19 00:00:00.000000000 Z
11
+ date: 1980-01-02 00:00:00.000000000 Z
13
12
  dependencies:
14
13
  - !ruby/object:Gem::Dependency
15
14
  name: bundler
@@ -26,19 +25,19 @@ dependencies:
26
25
  - !ruby/object:Gem::Version
27
26
  version: '0'
28
27
  - !ruby/object:Gem::Dependency
29
- name: minitest
28
+ name: maxitest
30
29
  requirement: !ruby/object:Gem::Requirement
31
30
  requirements:
32
31
  - - ">="
33
32
  - !ruby/object:Gem::Version
34
- version: 5.0.0
33
+ version: '0'
35
34
  type: :development
36
35
  prerelease: false
37
36
  version_requirements: !ruby/object:Gem::Requirement
38
37
  requirements:
39
38
  - - ">="
40
39
  - !ruby/object:Gem::Version
41
- version: 5.0.0
40
+ version: '0'
42
41
  - !ruby/object:Gem::Dependency
43
42
  name: rake
44
43
  requirement: !ruby/object:Gem::Requirement
@@ -66,6 +65,7 @@ files:
66
65
  - README.md
67
66
  - connection_pool.gemspec
68
67
  - lib/connection_pool.rb
68
+ - lib/connection_pool/fork.rb
69
69
  - lib/connection_pool/timed_stack.rb
70
70
  - lib/connection_pool/version.rb
71
71
  - lib/connection_pool/wrapper.rb
@@ -73,9 +73,12 @@ homepage: https://github.com/mperham/connection_pool
73
73
  licenses:
74
74
  - MIT
75
75
  metadata:
76
+ bug_tracker_uri: https://github.com/mperham/connection_pool/issues
77
+ documentation_uri: https://github.com/mperham/connection_pool/wiki
76
78
  changelog_uri: https://github.com/mperham/connection_pool/blob/main/Changes.md
79
+ source_code_uri: https://github.com/mperham/connection_pool
80
+ homepage_uri: https://github.com/mperham/connection_pool
77
81
  rubygems_mfa_required: 'true'
78
- post_install_message:
79
82
  rdoc_options: []
80
83
  require_paths:
81
84
  - lib
@@ -83,15 +86,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
83
86
  requirements:
84
87
  - - ">="
85
88
  - !ruby/object:Gem::Version
86
- version: 2.5.0
89
+ version: 3.2.0
87
90
  required_rubygems_version: !ruby/object:Gem::Requirement
88
91
  requirements:
89
92
  - - ">="
90
93
  - !ruby/object:Gem::Version
91
94
  version: '0'
92
95
  requirements: []
93
- rubygems_version: 3.4.7
94
- signing_key:
96
+ rubygems_version: 3.6.9
95
97
  specification_version: 4
96
98
  summary: Generic connection pool for Ruby
97
99
  test_files: []