connection_pool 2.2.5 → 2.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Changes.md +11 -0
- data/README.md +22 -32
- data/connection_pool.gemspec +5 -4
- data/lib/connection_pool/timed_stack.rb +5 -5
- data/lib/connection_pool/version.rb +1 -1
- data/lib/connection_pool/wrapper.rb +0 -1
- data/lib/connection_pool.rb +51 -7
- metadata +5 -15
- data/.github/workflows/ci.yml +0 -26
- data/.gitignore +0 -4
- data/Gemfile +0 -3
- data/Rakefile +0 -6
- data/test/helper.rb +0 -8
- data/test/test_connection_pool.rb +0 -567
- data/test/test_connection_pool_timed_stack.rb +0 -150
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e87682a6e57e8b0214de9c0713a7466a2b06399dc52d700fbf182cdf9e5b6606
|
4
|
+
data.tar.gz: d12337513b62d4677663403c512afad165275bdf848987182ce03637a1de6482
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: eca2c1f8ebe52039f00df70ddfe5525cc0408acd8ae9849a8d1412bec670e7d3a05d3609db933e3e906cd83307d39f754dd5f1d7b48b50b847090b5bf485b55a
|
7
|
+
data.tar.gz: fc9a62b4b0ba5a406543e8f6399bd566f2f31c6c9a6ce2772e3550c1c2a75945c8a34defeda10c83e23cbfe5fe50d0e8861f57ea23935d7c97540aa97b04a635
|
data/Changes.md
CHANGED
data/README.md
CHANGED
@@ -4,16 +4,14 @@ connection\_pool
|
|
4
4
|
|
5
5
|
Generic connection pooling for Ruby.
|
6
6
|
|
7
|
-
MongoDB has its own connection pool.
|
8
|
-
|
9
|
-
Dalli and other Ruby network clients.
|
10
|
-
|
7
|
+
MongoDB has its own connection pool.
|
8
|
+
ActiveRecord has its own connection pool.
|
9
|
+
This is a generic connection pool that can be used with anything, e.g. Redis, Dalli and other Ruby network clients.
|
11
10
|
|
12
11
|
Usage
|
13
12
|
-----
|
14
13
|
|
15
|
-
Create a pool of objects to share amongst the fibers or threads in your Ruby
|
16
|
-
application:
|
14
|
+
Create a pool of objects to share amongst the fibers or threads in your Ruby application:
|
17
15
|
|
18
16
|
``` ruby
|
19
17
|
$memcached = ConnectionPool.new(size: 5, timeout: 5) { Dalli::Client.new }
|
@@ -28,11 +26,12 @@ end
|
|
28
26
|
```
|
29
27
|
|
30
28
|
If all the objects in the connection pool are in use, `with` will block
|
31
|
-
until one becomes available.
|
32
|
-
|
29
|
+
until one becomes available.
|
30
|
+
If no object is available within `:timeout` seconds,
|
31
|
+
`with` will raise a `ConnectionPool::TimeoutError` (a subclass of `Timeout::Error`).
|
33
32
|
|
34
33
|
You can also use `ConnectionPool#then` to support _both_ a
|
35
|
-
connection pool and a raw client
|
34
|
+
connection pool and a raw client.
|
36
35
|
|
37
36
|
```ruby
|
38
37
|
# Compatible with a raw Redis::Client, and ConnectionPool Redis
|
@@ -48,15 +47,14 @@ end
|
|
48
47
|
```
|
49
48
|
|
50
49
|
This will only modify the resource-get timeout for this particular
|
51
|
-
invocation.
|
52
|
-
|
53
|
-
|
54
|
-
`ConnectionPool::Wrapper` class.
|
50
|
+
invocation.
|
51
|
+
This is useful if you want to fail-fast on certain non-critical
|
52
|
+
sections when a resource is not available, or conversely if you are comfortable blocking longer on a particular resource.
|
53
|
+
This is not implemented in the `ConnectionPool::Wrapper` class.
|
55
54
|
|
56
55
|
## Migrating to a Connection Pool
|
57
56
|
|
58
|
-
You can use `ConnectionPool::Wrapper` to wrap a single global connection,
|
59
|
-
making it easier to migrate existing connection code over time:
|
57
|
+
You can use `ConnectionPool::Wrapper` to wrap a single global connection, making it easier to migrate existing connection code over time:
|
60
58
|
|
61
59
|
``` ruby
|
62
60
|
$redis = ConnectionPool::Wrapper.new(size: 5, timeout: 3) { Redis.new }
|
@@ -64,10 +62,8 @@ $redis.sadd('foo', 1)
|
|
64
62
|
$redis.smembers('foo')
|
65
63
|
```
|
66
64
|
|
67
|
-
The wrapper uses `method_missing` to checkout a connection, run the requested
|
68
|
-
|
69
|
-
**not** high-performance so you'll want to port your performance sensitive code
|
70
|
-
to use `with` as soon as possible.
|
65
|
+
The wrapper uses `method_missing` to checkout a connection, run the requested method and then immediately check the connection back into the pool.
|
66
|
+
It's **not** high-performance so you'll want to port your performance sensitive code to use `with` as soon as possible.
|
71
67
|
|
72
68
|
``` ruby
|
73
69
|
$redis.with do |conn|
|
@@ -76,31 +72,26 @@ $redis.with do |conn|
|
|
76
72
|
end
|
77
73
|
```
|
78
74
|
|
79
|
-
Once you've ported your entire system to use `with`, you can simply remove
|
80
|
-
`Wrapper` and use the simpler and faster `ConnectionPool`.
|
75
|
+
Once you've ported your entire system to use `with`, you can simply remove `Wrapper` and use the simpler and faster `ConnectionPool`.
|
81
76
|
|
82
77
|
|
83
78
|
## Shutdown
|
84
79
|
|
85
80
|
You can shut down a ConnectionPool instance once it should no longer be used.
|
86
|
-
Further checkout attempts will immediately raise an error but existing checkouts
|
87
|
-
will work.
|
81
|
+
Further checkout attempts will immediately raise an error but existing checkouts will work.
|
88
82
|
|
89
83
|
```ruby
|
90
84
|
cp = ConnectionPool.new { Redis.new }
|
91
|
-
cp.shutdown { |
|
85
|
+
cp.shutdown { |c| c.close }
|
92
86
|
```
|
93
87
|
|
94
88
|
Shutting down a connection pool will block until all connections are checked in and closed.
|
95
|
-
**Note that shutting down is completely optional**; Ruby's garbage collector will reclaim
|
96
|
-
unreferenced pools under normal circumstances.
|
89
|
+
**Note that shutting down is completely optional**; Ruby's garbage collector will reclaim unreferenced pools under normal circumstances.
|
97
90
|
|
98
91
|
## Reload
|
99
92
|
|
100
|
-
You can reload a ConnectionPool instance in the case it is desired to close all
|
101
|
-
|
102
|
-
so the pool may continue to be used. Reloading may be useful after forking the
|
103
|
-
process.
|
93
|
+
You can reload a ConnectionPool instance in the case it is desired to close all connections to the pool and, unlike `shutdown`, afterwards recreate connections so the pool may continue to be used.
|
94
|
+
Reloading may be useful after forking the process.
|
104
95
|
|
105
96
|
```ruby
|
106
97
|
cp = ConnectionPool.new { Redis.new }
|
@@ -108,8 +99,7 @@ cp.reload { |conn| conn.quit }
|
|
108
99
|
cp.with { |conn| conn.get('some-count') }
|
109
100
|
```
|
110
101
|
|
111
|
-
Like `shutdown`, this will block until all connections are checked in and
|
112
|
-
closed.
|
102
|
+
Like `shutdown`, this will block until all connections are checked in and closed.
|
113
103
|
|
114
104
|
## Current State
|
115
105
|
|
data/connection_pool.gemspec
CHANGED
@@ -9,13 +9,14 @@ Gem::Specification.new do |s|
|
|
9
9
|
s.homepage = "https://github.com/mperham/connection_pool"
|
10
10
|
s.description = s.summary = "Generic connection pool for Ruby"
|
11
11
|
|
12
|
-
s.files =
|
13
|
-
|
14
|
-
|
12
|
+
s.files = ["Changes.md", "LICENSE", "README.md", "connection_pool.gemspec",
|
13
|
+
"lib/connection_pool.rb", "lib/connection_pool/timed_stack.rb",
|
14
|
+
"lib/connection_pool/version.rb", "lib/connection_pool/wrapper.rb"]
|
15
|
+
s.executables = []
|
15
16
|
s.require_paths = ["lib"]
|
16
17
|
s.license = "MIT"
|
17
18
|
s.add_development_dependency "bundler"
|
18
19
|
s.add_development_dependency "minitest", ">= 5.0.0"
|
19
20
|
s.add_development_dependency "rake"
|
20
|
-
s.required_ruby_version = ">= 2.
|
21
|
+
s.required_ruby_version = ">= 2.5.0"
|
21
22
|
end
|
@@ -29,8 +29,8 @@ class ConnectionPool::TimedStack
|
|
29
29
|
@created = 0
|
30
30
|
@que = []
|
31
31
|
@max = size
|
32
|
-
@mutex = Mutex.new
|
33
|
-
@resource = ConditionVariable.new
|
32
|
+
@mutex = Thread::Mutex.new
|
33
|
+
@resource = Thread::ConditionVariable.new
|
34
34
|
@shutdown_block = nil
|
35
35
|
end
|
36
36
|
|
@@ -49,7 +49,7 @@ class ConnectionPool::TimedStack
|
|
49
49
|
@resource.broadcast
|
50
50
|
end
|
51
51
|
end
|
52
|
-
|
52
|
+
alias_method :<<, :push
|
53
53
|
|
54
54
|
##
|
55
55
|
# Retrieves a connection from the stack. If a connection is available it is
|
@@ -74,7 +74,7 @@ class ConnectionPool::TimedStack
|
|
74
74
|
return connection if connection
|
75
75
|
|
76
76
|
to_wait = deadline - current_time
|
77
|
-
raise ConnectionPool::TimeoutError, "Waited #{timeout} sec" if to_wait <= 0
|
77
|
+
raise ConnectionPool::TimeoutError, "Waited #{timeout} sec, #{length}/#{@max} available" if to_wait <= 0
|
78
78
|
@resource.wait(@mutex, to_wait)
|
79
79
|
end
|
80
80
|
end
|
@@ -87,7 +87,7 @@ class ConnectionPool::TimedStack
|
|
87
87
|
# +:reload+ is +true+.
|
88
88
|
|
89
89
|
def shutdown(reload: false, &block)
|
90
|
-
raise ArgumentError, "shutdown must receive a block" unless
|
90
|
+
raise ArgumentError, "shutdown must receive a block" unless block
|
91
91
|
|
92
92
|
@mutex.synchronize do
|
93
93
|
@shutdown_block = block
|
@@ -30,7 +30,6 @@ class ConnectionPool
|
|
30
30
|
METHODS.include?(id) || with { |c| c.respond_to?(id, *args) }
|
31
31
|
end
|
32
32
|
|
33
|
-
# rubocop:disable Style/MethodMissingSuper
|
34
33
|
# rubocop:disable Style/MissingRespondToMissing
|
35
34
|
if ::RUBY_VERSION >= "3.0.0"
|
36
35
|
def method_missing(name, *args, **kwargs, &block)
|
data/lib/connection_pool.rb
CHANGED
@@ -1,9 +1,11 @@
|
|
1
1
|
require "timeout"
|
2
|
-
|
2
|
+
require_relative "connection_pool/version"
|
3
3
|
|
4
4
|
class ConnectionPool
|
5
5
|
class Error < ::RuntimeError; end
|
6
|
+
|
6
7
|
class PoolShuttingDownError < ::ConnectionPool::Error; end
|
8
|
+
|
7
9
|
class TimeoutError < ::Timeout::Error; end
|
8
10
|
end
|
9
11
|
|
@@ -42,6 +44,47 @@ class ConnectionPool
|
|
42
44
|
Wrapper.new(options, &block)
|
43
45
|
end
|
44
46
|
|
47
|
+
if Process.respond_to?(:fork)
|
48
|
+
INSTANCES = ObjectSpace::WeakMap.new
|
49
|
+
private_constant :INSTANCES
|
50
|
+
|
51
|
+
def self.after_fork
|
52
|
+
INSTANCES.values.each do |pool|
|
53
|
+
# We're on after fork, so we know all other threads are dead.
|
54
|
+
# All we need to do is to ensure the main thread doesn't have a
|
55
|
+
# checked out connection
|
56
|
+
pool.checkin(force: true)
|
57
|
+
|
58
|
+
pool.reload do |connection|
|
59
|
+
# Unfortunately we don't know what method to call to close the connection,
|
60
|
+
# so we try the most common one.
|
61
|
+
connection.close if connection.respond_to?(:close)
|
62
|
+
end
|
63
|
+
end
|
64
|
+
nil
|
65
|
+
end
|
66
|
+
|
67
|
+
if ::Process.respond_to?(:_fork) # MRI 3.1+
|
68
|
+
module ForkTracker
|
69
|
+
def _fork
|
70
|
+
pid = super
|
71
|
+
if pid == 0
|
72
|
+
ConnectionPool.after_fork
|
73
|
+
end
|
74
|
+
pid
|
75
|
+
end
|
76
|
+
end
|
77
|
+
Process.singleton_class.prepend(ForkTracker)
|
78
|
+
end
|
79
|
+
else
|
80
|
+
INSTANCES = nil
|
81
|
+
private_constant :INSTANCES
|
82
|
+
|
83
|
+
def self.after_fork
|
84
|
+
# noop
|
85
|
+
end
|
86
|
+
end
|
87
|
+
|
45
88
|
def initialize(options = {}, &block)
|
46
89
|
raise ArgumentError, "Connection pool requires a block" unless block
|
47
90
|
|
@@ -53,6 +96,7 @@ class ConnectionPool
|
|
53
96
|
@available = TimedStack.new(@size, &block)
|
54
97
|
@key = :"pool-#{@available.object_id}"
|
55
98
|
@key_count = :"pool-#{@available.object_id}-count"
|
99
|
+
INSTANCES[self] = self if INSTANCES
|
56
100
|
end
|
57
101
|
|
58
102
|
def with(options = {})
|
@@ -67,7 +111,7 @@ class ConnectionPool
|
|
67
111
|
end
|
68
112
|
end
|
69
113
|
end
|
70
|
-
|
114
|
+
alias_method :then, :with
|
71
115
|
|
72
116
|
def checkout(options = {})
|
73
117
|
if ::Thread.current[@key]
|
@@ -79,16 +123,16 @@ class ConnectionPool
|
|
79
123
|
end
|
80
124
|
end
|
81
125
|
|
82
|
-
def checkin
|
126
|
+
def checkin(force: false)
|
83
127
|
if ::Thread.current[@key]
|
84
|
-
if ::Thread.current[@key_count] == 1
|
128
|
+
if ::Thread.current[@key_count] == 1 || force
|
85
129
|
@available.push(::Thread.current[@key])
|
86
130
|
::Thread.current[@key] = nil
|
87
131
|
::Thread.current[@key_count] = nil
|
88
132
|
else
|
89
133
|
::Thread.current[@key_count] -= 1
|
90
134
|
end
|
91
|
-
|
135
|
+
elsif !force
|
92
136
|
raise ConnectionPool::Error, "no connections are checked out"
|
93
137
|
end
|
94
138
|
|
@@ -122,5 +166,5 @@ class ConnectionPool
|
|
122
166
|
end
|
123
167
|
end
|
124
168
|
|
125
|
-
|
126
|
-
|
169
|
+
require_relative "connection_pool/timed_stack"
|
170
|
+
require_relative "connection_pool/wrapper"
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: connection_pool
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.
|
4
|
+
version: 2.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Perham
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2023-03-24 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: bundler
|
@@ -61,21 +61,14 @@ executables: []
|
|
61
61
|
extensions: []
|
62
62
|
extra_rdoc_files: []
|
63
63
|
files:
|
64
|
-
- ".github/workflows/ci.yml"
|
65
|
-
- ".gitignore"
|
66
64
|
- Changes.md
|
67
|
-
- Gemfile
|
68
65
|
- LICENSE
|
69
66
|
- README.md
|
70
|
-
- Rakefile
|
71
67
|
- connection_pool.gemspec
|
72
68
|
- lib/connection_pool.rb
|
73
69
|
- lib/connection_pool/timed_stack.rb
|
74
70
|
- lib/connection_pool/version.rb
|
75
71
|
- lib/connection_pool/wrapper.rb
|
76
|
-
- test/helper.rb
|
77
|
-
- test/test_connection_pool.rb
|
78
|
-
- test/test_connection_pool_timed_stack.rb
|
79
72
|
homepage: https://github.com/mperham/connection_pool
|
80
73
|
licenses:
|
81
74
|
- MIT
|
@@ -88,18 +81,15 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
88
81
|
requirements:
|
89
82
|
- - ">="
|
90
83
|
- !ruby/object:Gem::Version
|
91
|
-
version: 2.
|
84
|
+
version: 2.5.0
|
92
85
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
93
86
|
requirements:
|
94
87
|
- - ">="
|
95
88
|
- !ruby/object:Gem::Version
|
96
89
|
version: '0'
|
97
90
|
requirements: []
|
98
|
-
rubygems_version: 3.
|
91
|
+
rubygems_version: 3.4.7
|
99
92
|
signing_key:
|
100
93
|
specification_version: 4
|
101
94
|
summary: Generic connection pool for Ruby
|
102
|
-
test_files:
|
103
|
-
- test/helper.rb
|
104
|
-
- test/test_connection_pool.rb
|
105
|
-
- test/test_connection_pool_timed_stack.rb
|
95
|
+
test_files: []
|
data/.github/workflows/ci.yml
DELETED
@@ -1,26 +0,0 @@
|
|
1
|
-
name: CI
|
2
|
-
|
3
|
-
on: [push, pull_request]
|
4
|
-
|
5
|
-
jobs:
|
6
|
-
test:
|
7
|
-
runs-on: ubuntu-latest
|
8
|
-
continue-on-error: ${{ matrix.experimental }}
|
9
|
-
strategy:
|
10
|
-
fail-fast: false
|
11
|
-
matrix:
|
12
|
-
ruby: ["2.4", "2.5", "2.6", "2.7", "3.0", "jruby"]
|
13
|
-
experimental: [false]
|
14
|
-
include:
|
15
|
-
- ruby: "truffleruby"
|
16
|
-
experimental: true
|
17
|
-
steps:
|
18
|
-
- uses: actions/checkout@v2
|
19
|
-
- uses: ruby/setup-ruby@v1
|
20
|
-
with:
|
21
|
-
ruby-version: ${{matrix.ruby}}
|
22
|
-
bundler-cache: true
|
23
|
-
|
24
|
-
- name: Run tests
|
25
|
-
timeout-minutes: 5
|
26
|
-
run: ${{matrix.env}} bundle exec rake
|
data/.gitignore
DELETED
data/Gemfile
DELETED
data/Rakefile
DELETED
data/test/helper.rb
DELETED
@@ -1,567 +0,0 @@
|
|
1
|
-
require_relative "helper"
|
2
|
-
|
3
|
-
class TestConnectionPool < Minitest::Test
|
4
|
-
class NetworkConnection
|
5
|
-
SLEEP_TIME = 0.1
|
6
|
-
|
7
|
-
def initialize
|
8
|
-
@x = 0
|
9
|
-
end
|
10
|
-
|
11
|
-
def do_something(*_args, increment: 1)
|
12
|
-
@x += increment
|
13
|
-
sleep SLEEP_TIME
|
14
|
-
@x
|
15
|
-
end
|
16
|
-
|
17
|
-
def do_something_with_positional_hash(options)
|
18
|
-
@x += options[:increment] || 1
|
19
|
-
sleep SLEEP_TIME
|
20
|
-
@x
|
21
|
-
end
|
22
|
-
|
23
|
-
def fast
|
24
|
-
@x += 1
|
25
|
-
end
|
26
|
-
|
27
|
-
def do_something_with_block
|
28
|
-
@x += yield
|
29
|
-
sleep SLEEP_TIME
|
30
|
-
@x
|
31
|
-
end
|
32
|
-
|
33
|
-
def respond_to?(method_id, *args)
|
34
|
-
method_id == :do_magic || super(method_id, *args)
|
35
|
-
end
|
36
|
-
end
|
37
|
-
|
38
|
-
class Recorder
|
39
|
-
def initialize
|
40
|
-
@calls = []
|
41
|
-
end
|
42
|
-
|
43
|
-
attr_reader :calls
|
44
|
-
|
45
|
-
def do_work(label)
|
46
|
-
@calls << label
|
47
|
-
end
|
48
|
-
end
|
49
|
-
|
50
|
-
def use_pool(pool, size)
|
51
|
-
Array.new(size) {
|
52
|
-
Thread.new do
|
53
|
-
pool.with { sleep }
|
54
|
-
end
|
55
|
-
}.each do |thread|
|
56
|
-
Thread.pass until thread.status == "sleep"
|
57
|
-
end
|
58
|
-
end
|
59
|
-
|
60
|
-
def kill_threads(threads)
|
61
|
-
threads.each do |thread|
|
62
|
-
thread.kill
|
63
|
-
thread.join
|
64
|
-
end
|
65
|
-
end
|
66
|
-
|
67
|
-
def test_basic_multithreaded_usage
|
68
|
-
pool_size = 5
|
69
|
-
pool = ConnectionPool.new(size: pool_size) { NetworkConnection.new }
|
70
|
-
|
71
|
-
start = Time.new
|
72
|
-
|
73
|
-
generations = 3
|
74
|
-
|
75
|
-
result = Array.new(pool_size * generations) {
|
76
|
-
Thread.new do
|
77
|
-
pool.with do |net|
|
78
|
-
net.do_something
|
79
|
-
end
|
80
|
-
end
|
81
|
-
}.map(&:value)
|
82
|
-
|
83
|
-
finish = Time.new
|
84
|
-
|
85
|
-
assert_equal((1..generations).cycle(pool_size).sort, result.sort)
|
86
|
-
|
87
|
-
assert_operator(finish - start, :>, generations * NetworkConnection::SLEEP_TIME)
|
88
|
-
end
|
89
|
-
|
90
|
-
def test_timeout
|
91
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { NetworkConnection.new }
|
92
|
-
thread = Thread.new {
|
93
|
-
pool.with do |net|
|
94
|
-
net.do_something
|
95
|
-
sleep 0.01
|
96
|
-
end
|
97
|
-
}
|
98
|
-
|
99
|
-
Thread.pass while thread.status == "run"
|
100
|
-
|
101
|
-
assert_raises Timeout::Error do
|
102
|
-
pool.with { |net| net.do_something }
|
103
|
-
end
|
104
|
-
|
105
|
-
thread.join
|
106
|
-
|
107
|
-
pool.with do |conn|
|
108
|
-
refute_nil conn
|
109
|
-
end
|
110
|
-
end
|
111
|
-
|
112
|
-
def test_with
|
113
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
114
|
-
|
115
|
-
pool.with do
|
116
|
-
Thread.new {
|
117
|
-
assert_raises Timeout::Error do
|
118
|
-
pool.checkout
|
119
|
-
end
|
120
|
-
}.join
|
121
|
-
end
|
122
|
-
|
123
|
-
assert Thread.new { pool.checkout }.join
|
124
|
-
end
|
125
|
-
|
126
|
-
def test_then
|
127
|
-
pool = ConnectionPool.new { Object.new }
|
128
|
-
|
129
|
-
assert_equal pool.method(:then), pool.method(:with)
|
130
|
-
end
|
131
|
-
|
132
|
-
def test_with_timeout
|
133
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
134
|
-
|
135
|
-
assert_raises Timeout::Error do
|
136
|
-
Timeout.timeout(0.01) do
|
137
|
-
pool.with do |obj|
|
138
|
-
assert_equal 0, pool.available
|
139
|
-
sleep 0.015
|
140
|
-
end
|
141
|
-
end
|
142
|
-
end
|
143
|
-
assert_equal 1, pool.available
|
144
|
-
end
|
145
|
-
|
146
|
-
def test_invalid_size
|
147
|
-
assert_raises ArgumentError, TypeError do
|
148
|
-
ConnectionPool.new(timeout: 0, size: nil) { Object.new }
|
149
|
-
end
|
150
|
-
assert_raises ArgumentError, TypeError do
|
151
|
-
ConnectionPool.new(timeout: 0, size: "") { Object.new }
|
152
|
-
end
|
153
|
-
end
|
154
|
-
|
155
|
-
def test_handle_interrupt_ensures_checkin
|
156
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
157
|
-
def pool.checkout(options)
|
158
|
-
sleep 0.015
|
159
|
-
super
|
160
|
-
end
|
161
|
-
|
162
|
-
did_something = false
|
163
|
-
|
164
|
-
action = lambda do
|
165
|
-
Timeout.timeout(0.01) do
|
166
|
-
pool.with do |obj|
|
167
|
-
did_something = true
|
168
|
-
# Timeout::Error will be triggered by any non-trivial Ruby code
|
169
|
-
# executed here since it couldn't be raised during checkout.
|
170
|
-
# It looks like setting the local variable above does not trigger
|
171
|
-
# the Timeout check in MRI 2.2.1.
|
172
|
-
obj.tap { obj.hash }
|
173
|
-
end
|
174
|
-
end
|
175
|
-
end
|
176
|
-
|
177
|
-
if RUBY_ENGINE == "ruby"
|
178
|
-
# These asserts rely on the Ruby implementation reaching `did_something =
|
179
|
-
# true` before the interrupt is detected by the thread. Interrupt
|
180
|
-
# detection timing is implementation-specific in practice, with JRuby,
|
181
|
-
# Rubinius, and TruffleRuby all having different interrupt timings to MRI.
|
182
|
-
# In fact they generally detect interrupts more quickly than MRI, so they
|
183
|
-
# may not reach `did_something = true` before detecting the interrupt.
|
184
|
-
|
185
|
-
assert_raises Timeout::Error, &action
|
186
|
-
|
187
|
-
assert did_something
|
188
|
-
else
|
189
|
-
action.call
|
190
|
-
end
|
191
|
-
|
192
|
-
assert_equal 1, pool.available
|
193
|
-
end
|
194
|
-
|
195
|
-
def test_explicit_return
|
196
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) {
|
197
|
-
mock = Minitest::Mock.new
|
198
|
-
def mock.disconnect!
|
199
|
-
raise "should not disconnect upon explicit return"
|
200
|
-
end
|
201
|
-
mock
|
202
|
-
}
|
203
|
-
|
204
|
-
pool.with do |conn|
|
205
|
-
return true
|
206
|
-
end
|
207
|
-
end
|
208
|
-
|
209
|
-
def test_with_timeout_override
|
210
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { NetworkConnection.new }
|
211
|
-
|
212
|
-
t = Thread.new {
|
213
|
-
pool.with do |net|
|
214
|
-
net.do_something
|
215
|
-
sleep 0.01
|
216
|
-
end
|
217
|
-
}
|
218
|
-
|
219
|
-
Thread.pass while t.status == "run"
|
220
|
-
|
221
|
-
assert_raises Timeout::Error do
|
222
|
-
pool.with { |net| net.do_something }
|
223
|
-
end
|
224
|
-
|
225
|
-
pool.with(timeout: 2 * NetworkConnection::SLEEP_TIME) do |conn|
|
226
|
-
refute_nil conn
|
227
|
-
end
|
228
|
-
end
|
229
|
-
|
230
|
-
def test_checkin
|
231
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { NetworkConnection.new }
|
232
|
-
conn = pool.checkout
|
233
|
-
|
234
|
-
Thread.new {
|
235
|
-
assert_raises Timeout::Error do
|
236
|
-
pool.checkout
|
237
|
-
end
|
238
|
-
}.join
|
239
|
-
|
240
|
-
pool.checkin
|
241
|
-
|
242
|
-
assert_same conn, Thread.new { pool.checkout }.value
|
243
|
-
end
|
244
|
-
|
245
|
-
def test_returns_value
|
246
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
247
|
-
assert_equal 1, pool.with { |o| 1 }
|
248
|
-
end
|
249
|
-
|
250
|
-
def test_checkin_never_checkout
|
251
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
252
|
-
|
253
|
-
e = assert_raises(ConnectionPool::Error) { pool.checkin }
|
254
|
-
assert_equal "no connections are checked out", e.message
|
255
|
-
end
|
256
|
-
|
257
|
-
def test_checkin_no_current_checkout
|
258
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
259
|
-
|
260
|
-
pool.checkout
|
261
|
-
pool.checkin
|
262
|
-
|
263
|
-
assert_raises ConnectionPool::Error do
|
264
|
-
pool.checkin
|
265
|
-
end
|
266
|
-
end
|
267
|
-
|
268
|
-
def test_checkin_twice
|
269
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
270
|
-
|
271
|
-
pool.checkout
|
272
|
-
pool.checkout
|
273
|
-
|
274
|
-
pool.checkin
|
275
|
-
|
276
|
-
Thread.new {
|
277
|
-
assert_raises Timeout::Error do
|
278
|
-
pool.checkout
|
279
|
-
end
|
280
|
-
}.join
|
281
|
-
|
282
|
-
pool.checkin
|
283
|
-
|
284
|
-
assert Thread.new { pool.checkout }.join
|
285
|
-
end
|
286
|
-
|
287
|
-
def test_checkout
|
288
|
-
pool = ConnectionPool.new(size: 1) { NetworkConnection.new }
|
289
|
-
|
290
|
-
conn = pool.checkout
|
291
|
-
|
292
|
-
assert_kind_of NetworkConnection, conn
|
293
|
-
|
294
|
-
assert_same conn, pool.checkout
|
295
|
-
end
|
296
|
-
|
297
|
-
def test_checkout_multithread
|
298
|
-
pool = ConnectionPool.new(size: 2) { NetworkConnection.new }
|
299
|
-
conn = pool.checkout
|
300
|
-
|
301
|
-
t = Thread.new {
|
302
|
-
pool.checkout
|
303
|
-
}
|
304
|
-
|
305
|
-
refute_same conn, t.value
|
306
|
-
end
|
307
|
-
|
308
|
-
def test_checkout_timeout
|
309
|
-
pool = ConnectionPool.new(timeout: 0, size: 0) { Object.new }
|
310
|
-
|
311
|
-
assert_raises Timeout::Error do
|
312
|
-
pool.checkout
|
313
|
-
end
|
314
|
-
end
|
315
|
-
|
316
|
-
def test_checkout_timeout_override
|
317
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { NetworkConnection.new }
|
318
|
-
|
319
|
-
thread = Thread.new {
|
320
|
-
pool.with do |net|
|
321
|
-
net.do_something
|
322
|
-
sleep 0.01
|
323
|
-
end
|
324
|
-
}
|
325
|
-
|
326
|
-
Thread.pass while thread.status == "run"
|
327
|
-
|
328
|
-
assert_raises Timeout::Error do
|
329
|
-
pool.checkout
|
330
|
-
end
|
331
|
-
|
332
|
-
assert pool.checkout timeout: 2 * NetworkConnection::SLEEP_TIME
|
333
|
-
end
|
334
|
-
|
335
|
-
def test_passthru
|
336
|
-
pool = ConnectionPool.wrap(timeout: 2 * NetworkConnection::SLEEP_TIME, size: 1) { NetworkConnection.new }
|
337
|
-
assert_equal 1, pool.do_something
|
338
|
-
assert_equal 2, pool.do_something
|
339
|
-
assert_equal 5, pool.do_something_with_block { 3 }
|
340
|
-
assert_equal 6, pool.with { |net| net.fast }
|
341
|
-
assert_equal 8, pool.do_something(increment: 2)
|
342
|
-
assert_equal 10, pool.do_something_with_positional_hash({ increment: 2, symbol_key: 3, "string_key" => 4 })
|
343
|
-
end
|
344
|
-
|
345
|
-
def test_passthru_respond_to
|
346
|
-
pool = ConnectionPool.wrap(timeout: 2 * NetworkConnection::SLEEP_TIME, size: 1) { NetworkConnection.new }
|
347
|
-
assert pool.respond_to?(:with)
|
348
|
-
assert pool.respond_to?(:do_something)
|
349
|
-
assert pool.respond_to?(:do_magic)
|
350
|
-
refute pool.respond_to?(:do_lots_of_magic)
|
351
|
-
end
|
352
|
-
|
353
|
-
def test_return_value
|
354
|
-
pool = ConnectionPool.new(timeout: 2 * NetworkConnection::SLEEP_TIME, size: 1) { NetworkConnection.new }
|
355
|
-
result = pool.with { |net|
|
356
|
-
net.fast
|
357
|
-
}
|
358
|
-
assert_equal 1, result
|
359
|
-
end
|
360
|
-
|
361
|
-
def test_heavy_threading
|
362
|
-
pool = ConnectionPool.new(timeout: 0.5, size: 3) { NetworkConnection.new }
|
363
|
-
|
364
|
-
threads = Array.new(20) {
|
365
|
-
Thread.new do
|
366
|
-
pool.with do |net|
|
367
|
-
sleep 0.01
|
368
|
-
end
|
369
|
-
end
|
370
|
-
}
|
371
|
-
|
372
|
-
threads.map { |thread| thread.join }
|
373
|
-
end
|
374
|
-
|
375
|
-
def test_reuses_objects_when_pool_not_saturated
|
376
|
-
pool = ConnectionPool.new(size: 5) { NetworkConnection.new }
|
377
|
-
|
378
|
-
ids = 10.times.map {
|
379
|
-
pool.with { |c| c.object_id }
|
380
|
-
}
|
381
|
-
|
382
|
-
assert_equal 1, ids.uniq.size
|
383
|
-
end
|
384
|
-
|
385
|
-
def test_nested_checkout
|
386
|
-
recorder = Recorder.new
|
387
|
-
pool = ConnectionPool.new(size: 1) { recorder }
|
388
|
-
pool.with do |r_outer|
|
389
|
-
@other = Thread.new { |t|
|
390
|
-
pool.with do |r_other|
|
391
|
-
r_other.do_work("other")
|
392
|
-
end
|
393
|
-
}
|
394
|
-
|
395
|
-
pool.with do |r_inner|
|
396
|
-
r_inner.do_work("inner")
|
397
|
-
end
|
398
|
-
|
399
|
-
Thread.pass
|
400
|
-
|
401
|
-
r_outer.do_work("outer")
|
402
|
-
end
|
403
|
-
|
404
|
-
@other.join
|
405
|
-
|
406
|
-
assert_equal ["inner", "outer", "other"], recorder.calls
|
407
|
-
end
|
408
|
-
|
409
|
-
def test_shutdown_is_executed_for_all_connections
|
410
|
-
recorders = []
|
411
|
-
|
412
|
-
pool = ConnectionPool.new(size: 3) {
|
413
|
-
Recorder.new.tap { |r| recorders << r }
|
414
|
-
}
|
415
|
-
|
416
|
-
threads = use_pool pool, 3
|
417
|
-
|
418
|
-
pool.shutdown do |recorder|
|
419
|
-
recorder.do_work("shutdown")
|
420
|
-
end
|
421
|
-
|
422
|
-
kill_threads(threads)
|
423
|
-
|
424
|
-
assert_equal [["shutdown"]] * 3, recorders.map { |r| r.calls }
|
425
|
-
end
|
426
|
-
|
427
|
-
def test_raises_error_after_shutting_down
|
428
|
-
pool = ConnectionPool.new(size: 1) { true }
|
429
|
-
|
430
|
-
pool.shutdown {}
|
431
|
-
|
432
|
-
assert_raises ConnectionPool::PoolShuttingDownError do
|
433
|
-
pool.checkout
|
434
|
-
end
|
435
|
-
end
|
436
|
-
|
437
|
-
def test_runs_shutdown_block_asynchronously_if_connection_was_in_use
|
438
|
-
recorders = []
|
439
|
-
|
440
|
-
pool = ConnectionPool.new(size: 3) {
|
441
|
-
Recorder.new.tap { |r| recorders << r }
|
442
|
-
}
|
443
|
-
|
444
|
-
threads = use_pool pool, 2
|
445
|
-
|
446
|
-
pool.checkout
|
447
|
-
|
448
|
-
pool.shutdown do |recorder|
|
449
|
-
recorder.do_work("shutdown")
|
450
|
-
end
|
451
|
-
|
452
|
-
kill_threads(threads)
|
453
|
-
|
454
|
-
assert_equal [["shutdown"], ["shutdown"], []], recorders.map { |r| r.calls }
|
455
|
-
|
456
|
-
pool.checkin
|
457
|
-
|
458
|
-
assert_equal [["shutdown"], ["shutdown"], ["shutdown"]], recorders.map { |r| r.calls }
|
459
|
-
end
|
460
|
-
|
461
|
-
def test_raises_an_error_if_shutdown_is_called_without_a_block
|
462
|
-
pool = ConnectionPool.new(size: 1) {}
|
463
|
-
|
464
|
-
assert_raises ArgumentError do
|
465
|
-
pool.shutdown
|
466
|
-
end
|
467
|
-
end
|
468
|
-
|
469
|
-
def test_shutdown_is_executed_for_all_connections_in_wrapped_pool
|
470
|
-
recorders = []
|
471
|
-
|
472
|
-
wrapper = ConnectionPool::Wrapper.new(size: 3) {
|
473
|
-
Recorder.new.tap { |r| recorders << r }
|
474
|
-
}
|
475
|
-
|
476
|
-
threads = use_pool wrapper, 3
|
477
|
-
|
478
|
-
wrapper.pool_shutdown do |recorder|
|
479
|
-
recorder.do_work("shutdown")
|
480
|
-
end
|
481
|
-
|
482
|
-
kill_threads(threads)
|
483
|
-
|
484
|
-
assert_equal [["shutdown"]] * 3, recorders.map { |r| r.calls }
|
485
|
-
end
|
486
|
-
|
487
|
-
def test_wrapper_wrapped_pool
|
488
|
-
wrapper = ConnectionPool::Wrapper.new { NetworkConnection.new }
|
489
|
-
assert_equal ConnectionPool, wrapper.wrapped_pool.class
|
490
|
-
end
|
491
|
-
|
492
|
-
def test_wrapper_method_missing
|
493
|
-
wrapper = ConnectionPool::Wrapper.new { NetworkConnection.new }
|
494
|
-
|
495
|
-
assert_equal 1, wrapper.fast
|
496
|
-
end
|
497
|
-
|
498
|
-
def test_wrapper_respond_to_eh
|
499
|
-
wrapper = ConnectionPool::Wrapper.new { NetworkConnection.new }
|
500
|
-
|
501
|
-
assert_respond_to wrapper, :with
|
502
|
-
|
503
|
-
assert_respond_to wrapper, :fast
|
504
|
-
refute_respond_to wrapper, :"nonexistent method"
|
505
|
-
end
|
506
|
-
|
507
|
-
def test_wrapper_with
|
508
|
-
wrapper = ConnectionPool::Wrapper.new(timeout: 0, size: 1) { Object.new }
|
509
|
-
|
510
|
-
wrapper.with do
|
511
|
-
Thread.new {
|
512
|
-
assert_raises Timeout::Error do
|
513
|
-
wrapper.with { flunk "connection checked out :(" }
|
514
|
-
end
|
515
|
-
}.join
|
516
|
-
end
|
517
|
-
|
518
|
-
assert Thread.new { wrapper.with {} }.join
|
519
|
-
end
|
520
|
-
|
521
|
-
class ConnWithEval
|
522
|
-
def eval(arg)
|
523
|
-
"eval'ed #{arg}"
|
524
|
-
end
|
525
|
-
end
|
526
|
-
|
527
|
-
def test_wrapper_kernel_methods
|
528
|
-
wrapper = ConnectionPool::Wrapper.new(timeout: 0, size: 1) { ConnWithEval.new }
|
529
|
-
|
530
|
-
assert_equal "eval'ed 1", wrapper.eval(1)
|
531
|
-
end
|
532
|
-
|
533
|
-
def test_wrapper_with_connection_pool
|
534
|
-
recorder = Recorder.new
|
535
|
-
pool = ConnectionPool.new(size: 1) { recorder }
|
536
|
-
wrapper = ConnectionPool::Wrapper.new(pool: pool)
|
537
|
-
|
538
|
-
pool.with { |r| r.do_work("with") }
|
539
|
-
wrapper.do_work("wrapped")
|
540
|
-
|
541
|
-
assert_equal ["with", "wrapped"], recorder.calls
|
542
|
-
end
|
543
|
-
|
544
|
-
def test_stats_without_active_connection
|
545
|
-
pool = ConnectionPool.new(size: 2) { NetworkConnection.new }
|
546
|
-
|
547
|
-
assert_equal(2, pool.size)
|
548
|
-
assert_equal(2, pool.available)
|
549
|
-
end
|
550
|
-
|
551
|
-
def test_stats_with_active_connection
|
552
|
-
pool = ConnectionPool.new(size: 2) { NetworkConnection.new }
|
553
|
-
|
554
|
-
pool.with do
|
555
|
-
assert_equal(1, pool.available)
|
556
|
-
end
|
557
|
-
end
|
558
|
-
|
559
|
-
def test_stats_with_string_size
|
560
|
-
pool = ConnectionPool.new(size: "2") { NetworkConnection.new }
|
561
|
-
|
562
|
-
pool.with do
|
563
|
-
assert_equal(2, pool.size)
|
564
|
-
assert_equal(1, pool.available)
|
565
|
-
end
|
566
|
-
end
|
567
|
-
end
|
@@ -1,150 +0,0 @@
|
|
1
|
-
require_relative "helper"
|
2
|
-
|
3
|
-
class TestConnectionPoolTimedStack < Minitest::Test
|
4
|
-
def setup
|
5
|
-
@stack = ConnectionPool::TimedStack.new { Object.new }
|
6
|
-
end
|
7
|
-
|
8
|
-
def test_empty_eh
|
9
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
10
|
-
|
11
|
-
refute_empty stack
|
12
|
-
|
13
|
-
popped = stack.pop
|
14
|
-
|
15
|
-
assert_empty stack
|
16
|
-
|
17
|
-
stack.push popped
|
18
|
-
|
19
|
-
refute_empty stack
|
20
|
-
end
|
21
|
-
|
22
|
-
def test_length
|
23
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
24
|
-
|
25
|
-
assert_equal 1, stack.length
|
26
|
-
|
27
|
-
popped = stack.pop
|
28
|
-
|
29
|
-
assert_equal 0, stack.length
|
30
|
-
|
31
|
-
stack.push popped
|
32
|
-
|
33
|
-
assert_equal 1, stack.length
|
34
|
-
end
|
35
|
-
|
36
|
-
def test_object_creation_fails
|
37
|
-
stack = ConnectionPool::TimedStack.new(2) { raise "failure" }
|
38
|
-
|
39
|
-
begin
|
40
|
-
stack.pop
|
41
|
-
rescue => error
|
42
|
-
assert_equal "failure", error.message
|
43
|
-
end
|
44
|
-
|
45
|
-
begin
|
46
|
-
stack.pop
|
47
|
-
rescue => error
|
48
|
-
assert_equal "failure", error.message
|
49
|
-
end
|
50
|
-
|
51
|
-
refute_empty stack
|
52
|
-
assert_equal 2, stack.length
|
53
|
-
end
|
54
|
-
|
55
|
-
def test_pop
|
56
|
-
object = Object.new
|
57
|
-
@stack.push object
|
58
|
-
|
59
|
-
popped = @stack.pop
|
60
|
-
|
61
|
-
assert_same object, popped
|
62
|
-
end
|
63
|
-
|
64
|
-
def test_pop_empty
|
65
|
-
e = assert_raises(ConnectionPool::TimeoutError) { @stack.pop timeout: 0 }
|
66
|
-
assert_equal "Waited 0 sec", e.message
|
67
|
-
end
|
68
|
-
|
69
|
-
def test_pop_empty_2_0_compatibility
|
70
|
-
e = assert_raises(Timeout::Error) { @stack.pop 0 }
|
71
|
-
assert_equal "Waited 0 sec", e.message
|
72
|
-
end
|
73
|
-
|
74
|
-
def test_pop_full
|
75
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
76
|
-
|
77
|
-
popped = stack.pop
|
78
|
-
|
79
|
-
refute_nil popped
|
80
|
-
assert_empty stack
|
81
|
-
end
|
82
|
-
|
83
|
-
def test_pop_wait
|
84
|
-
thread = Thread.start {
|
85
|
-
@stack.pop
|
86
|
-
}
|
87
|
-
|
88
|
-
Thread.pass while thread.status == "run"
|
89
|
-
|
90
|
-
object = Object.new
|
91
|
-
|
92
|
-
@stack.push object
|
93
|
-
|
94
|
-
assert_same object, thread.value
|
95
|
-
end
|
96
|
-
|
97
|
-
def test_pop_shutdown
|
98
|
-
@stack.shutdown {}
|
99
|
-
|
100
|
-
assert_raises ConnectionPool::PoolShuttingDownError do
|
101
|
-
@stack.pop
|
102
|
-
end
|
103
|
-
end
|
104
|
-
|
105
|
-
def test_pop_shutdown_reload
|
106
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
107
|
-
object = stack.pop
|
108
|
-
stack.push(object)
|
109
|
-
|
110
|
-
stack.shutdown(reload: true) {}
|
111
|
-
|
112
|
-
refute_equal object, stack.pop
|
113
|
-
end
|
114
|
-
|
115
|
-
def test_push
|
116
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
117
|
-
|
118
|
-
conn = stack.pop
|
119
|
-
|
120
|
-
stack.push conn
|
121
|
-
|
122
|
-
refute_empty stack
|
123
|
-
end
|
124
|
-
|
125
|
-
def test_push_shutdown
|
126
|
-
called = []
|
127
|
-
|
128
|
-
@stack.shutdown do |object|
|
129
|
-
called << object
|
130
|
-
end
|
131
|
-
|
132
|
-
@stack.push Object.new
|
133
|
-
|
134
|
-
refute_empty called
|
135
|
-
assert_empty @stack
|
136
|
-
end
|
137
|
-
|
138
|
-
def test_shutdown
|
139
|
-
@stack.push Object.new
|
140
|
-
|
141
|
-
called = []
|
142
|
-
|
143
|
-
@stack.shutdown do |object|
|
144
|
-
called << object
|
145
|
-
end
|
146
|
-
|
147
|
-
refute_empty called
|
148
|
-
assert_empty @stack
|
149
|
-
end
|
150
|
-
end
|