connection_pool 2.2.5 → 2.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Changes.md +11 -0
- data/README.md +22 -32
- data/connection_pool.gemspec +5 -4
- data/lib/connection_pool/timed_stack.rb +5 -5
- data/lib/connection_pool/version.rb +1 -1
- data/lib/connection_pool/wrapper.rb +0 -1
- data/lib/connection_pool.rb +51 -7
- metadata +5 -15
- data/.github/workflows/ci.yml +0 -26
- data/.gitignore +0 -4
- data/Gemfile +0 -3
- data/Rakefile +0 -6
- data/test/helper.rb +0 -8
- data/test/test_connection_pool.rb +0 -567
- data/test/test_connection_pool_timed_stack.rb +0 -150
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e87682a6e57e8b0214de9c0713a7466a2b06399dc52d700fbf182cdf9e5b6606
|
4
|
+
data.tar.gz: d12337513b62d4677663403c512afad165275bdf848987182ce03637a1de6482
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: eca2c1f8ebe52039f00df70ddfe5525cc0408acd8ae9849a8d1412bec670e7d3a05d3609db933e3e906cd83307d39f754dd5f1d7b48b50b847090b5bf485b55a
|
7
|
+
data.tar.gz: fc9a62b4b0ba5a406543e8f6399bd566f2f31c6c9a6ce2772e3550c1c2a75945c8a34defeda10c83e23cbfe5fe50d0e8861f57ea23935d7c97540aa97b04a635
|
data/Changes.md
CHANGED
data/README.md
CHANGED
@@ -4,16 +4,14 @@ connection\_pool
|
|
4
4
|
|
5
5
|
Generic connection pooling for Ruby.
|
6
6
|
|
7
|
-
MongoDB has its own connection pool.
|
8
|
-
|
9
|
-
Dalli and other Ruby network clients.
|
10
|
-
|
7
|
+
MongoDB has its own connection pool.
|
8
|
+
ActiveRecord has its own connection pool.
|
9
|
+
This is a generic connection pool that can be used with anything, e.g. Redis, Dalli and other Ruby network clients.
|
11
10
|
|
12
11
|
Usage
|
13
12
|
-----
|
14
13
|
|
15
|
-
Create a pool of objects to share amongst the fibers or threads in your Ruby
|
16
|
-
application:
|
14
|
+
Create a pool of objects to share amongst the fibers or threads in your Ruby application:
|
17
15
|
|
18
16
|
``` ruby
|
19
17
|
$memcached = ConnectionPool.new(size: 5, timeout: 5) { Dalli::Client.new }
|
@@ -28,11 +26,12 @@ end
|
|
28
26
|
```
|
29
27
|
|
30
28
|
If all the objects in the connection pool are in use, `with` will block
|
31
|
-
until one becomes available.
|
32
|
-
|
29
|
+
until one becomes available.
|
30
|
+
If no object is available within `:timeout` seconds,
|
31
|
+
`with` will raise a `ConnectionPool::TimeoutError` (a subclass of `Timeout::Error`).
|
33
32
|
|
34
33
|
You can also use `ConnectionPool#then` to support _both_ a
|
35
|
-
connection pool and a raw client
|
34
|
+
connection pool and a raw client.
|
36
35
|
|
37
36
|
```ruby
|
38
37
|
# Compatible with a raw Redis::Client, and ConnectionPool Redis
|
@@ -48,15 +47,14 @@ end
|
|
48
47
|
```
|
49
48
|
|
50
49
|
This will only modify the resource-get timeout for this particular
|
51
|
-
invocation.
|
52
|
-
|
53
|
-
|
54
|
-
`ConnectionPool::Wrapper` class.
|
50
|
+
invocation.
|
51
|
+
This is useful if you want to fail-fast on certain non-critical
|
52
|
+
sections when a resource is not available, or conversely if you are comfortable blocking longer on a particular resource.
|
53
|
+
This is not implemented in the `ConnectionPool::Wrapper` class.
|
55
54
|
|
56
55
|
## Migrating to a Connection Pool
|
57
56
|
|
58
|
-
You can use `ConnectionPool::Wrapper` to wrap a single global connection,
|
59
|
-
making it easier to migrate existing connection code over time:
|
57
|
+
You can use `ConnectionPool::Wrapper` to wrap a single global connection, making it easier to migrate existing connection code over time:
|
60
58
|
|
61
59
|
``` ruby
|
62
60
|
$redis = ConnectionPool::Wrapper.new(size: 5, timeout: 3) { Redis.new }
|
@@ -64,10 +62,8 @@ $redis.sadd('foo', 1)
|
|
64
62
|
$redis.smembers('foo')
|
65
63
|
```
|
66
64
|
|
67
|
-
The wrapper uses `method_missing` to checkout a connection, run the requested
|
68
|
-
|
69
|
-
**not** high-performance so you'll want to port your performance sensitive code
|
70
|
-
to use `with` as soon as possible.
|
65
|
+
The wrapper uses `method_missing` to checkout a connection, run the requested method and then immediately check the connection back into the pool.
|
66
|
+
It's **not** high-performance so you'll want to port your performance sensitive code to use `with` as soon as possible.
|
71
67
|
|
72
68
|
``` ruby
|
73
69
|
$redis.with do |conn|
|
@@ -76,31 +72,26 @@ $redis.with do |conn|
|
|
76
72
|
end
|
77
73
|
```
|
78
74
|
|
79
|
-
Once you've ported your entire system to use `with`, you can simply remove
|
80
|
-
`Wrapper` and use the simpler and faster `ConnectionPool`.
|
75
|
+
Once you've ported your entire system to use `with`, you can simply remove `Wrapper` and use the simpler and faster `ConnectionPool`.
|
81
76
|
|
82
77
|
|
83
78
|
## Shutdown
|
84
79
|
|
85
80
|
You can shut down a ConnectionPool instance once it should no longer be used.
|
86
|
-
Further checkout attempts will immediately raise an error but existing checkouts
|
87
|
-
will work.
|
81
|
+
Further checkout attempts will immediately raise an error but existing checkouts will work.
|
88
82
|
|
89
83
|
```ruby
|
90
84
|
cp = ConnectionPool.new { Redis.new }
|
91
|
-
cp.shutdown { |
|
85
|
+
cp.shutdown { |c| c.close }
|
92
86
|
```
|
93
87
|
|
94
88
|
Shutting down a connection pool will block until all connections are checked in and closed.
|
95
|
-
**Note that shutting down is completely optional**; Ruby's garbage collector will reclaim
|
96
|
-
unreferenced pools under normal circumstances.
|
89
|
+
**Note that shutting down is completely optional**; Ruby's garbage collector will reclaim unreferenced pools under normal circumstances.
|
97
90
|
|
98
91
|
## Reload
|
99
92
|
|
100
|
-
You can reload a ConnectionPool instance in the case it is desired to close all
|
101
|
-
|
102
|
-
so the pool may continue to be used. Reloading may be useful after forking the
|
103
|
-
process.
|
93
|
+
You can reload a ConnectionPool instance in the case it is desired to close all connections to the pool and, unlike `shutdown`, afterwards recreate connections so the pool may continue to be used.
|
94
|
+
Reloading may be useful after forking the process.
|
104
95
|
|
105
96
|
```ruby
|
106
97
|
cp = ConnectionPool.new { Redis.new }
|
@@ -108,8 +99,7 @@ cp.reload { |conn| conn.quit }
|
|
108
99
|
cp.with { |conn| conn.get('some-count') }
|
109
100
|
```
|
110
101
|
|
111
|
-
Like `shutdown`, this will block until all connections are checked in and
|
112
|
-
closed.
|
102
|
+
Like `shutdown`, this will block until all connections are checked in and closed.
|
113
103
|
|
114
104
|
## Current State
|
115
105
|
|
data/connection_pool.gemspec
CHANGED
@@ -9,13 +9,14 @@ Gem::Specification.new do |s|
|
|
9
9
|
s.homepage = "https://github.com/mperham/connection_pool"
|
10
10
|
s.description = s.summary = "Generic connection pool for Ruby"
|
11
11
|
|
12
|
-
s.files =
|
13
|
-
|
14
|
-
|
12
|
+
s.files = ["Changes.md", "LICENSE", "README.md", "connection_pool.gemspec",
|
13
|
+
"lib/connection_pool.rb", "lib/connection_pool/timed_stack.rb",
|
14
|
+
"lib/connection_pool/version.rb", "lib/connection_pool/wrapper.rb"]
|
15
|
+
s.executables = []
|
15
16
|
s.require_paths = ["lib"]
|
16
17
|
s.license = "MIT"
|
17
18
|
s.add_development_dependency "bundler"
|
18
19
|
s.add_development_dependency "minitest", ">= 5.0.0"
|
19
20
|
s.add_development_dependency "rake"
|
20
|
-
s.required_ruby_version = ">= 2.
|
21
|
+
s.required_ruby_version = ">= 2.5.0"
|
21
22
|
end
|
@@ -29,8 +29,8 @@ class ConnectionPool::TimedStack
|
|
29
29
|
@created = 0
|
30
30
|
@que = []
|
31
31
|
@max = size
|
32
|
-
@mutex = Mutex.new
|
33
|
-
@resource = ConditionVariable.new
|
32
|
+
@mutex = Thread::Mutex.new
|
33
|
+
@resource = Thread::ConditionVariable.new
|
34
34
|
@shutdown_block = nil
|
35
35
|
end
|
36
36
|
|
@@ -49,7 +49,7 @@ class ConnectionPool::TimedStack
|
|
49
49
|
@resource.broadcast
|
50
50
|
end
|
51
51
|
end
|
52
|
-
|
52
|
+
alias_method :<<, :push
|
53
53
|
|
54
54
|
##
|
55
55
|
# Retrieves a connection from the stack. If a connection is available it is
|
@@ -74,7 +74,7 @@ class ConnectionPool::TimedStack
|
|
74
74
|
return connection if connection
|
75
75
|
|
76
76
|
to_wait = deadline - current_time
|
77
|
-
raise ConnectionPool::TimeoutError, "Waited #{timeout} sec" if to_wait <= 0
|
77
|
+
raise ConnectionPool::TimeoutError, "Waited #{timeout} sec, #{length}/#{@max} available" if to_wait <= 0
|
78
78
|
@resource.wait(@mutex, to_wait)
|
79
79
|
end
|
80
80
|
end
|
@@ -87,7 +87,7 @@ class ConnectionPool::TimedStack
|
|
87
87
|
# +:reload+ is +true+.
|
88
88
|
|
89
89
|
def shutdown(reload: false, &block)
|
90
|
-
raise ArgumentError, "shutdown must receive a block" unless
|
90
|
+
raise ArgumentError, "shutdown must receive a block" unless block
|
91
91
|
|
92
92
|
@mutex.synchronize do
|
93
93
|
@shutdown_block = block
|
@@ -30,7 +30,6 @@ class ConnectionPool
|
|
30
30
|
METHODS.include?(id) || with { |c| c.respond_to?(id, *args) }
|
31
31
|
end
|
32
32
|
|
33
|
-
# rubocop:disable Style/MethodMissingSuper
|
34
33
|
# rubocop:disable Style/MissingRespondToMissing
|
35
34
|
if ::RUBY_VERSION >= "3.0.0"
|
36
35
|
def method_missing(name, *args, **kwargs, &block)
|
data/lib/connection_pool.rb
CHANGED
@@ -1,9 +1,11 @@
|
|
1
1
|
require "timeout"
|
2
|
-
|
2
|
+
require_relative "connection_pool/version"
|
3
3
|
|
4
4
|
class ConnectionPool
|
5
5
|
class Error < ::RuntimeError; end
|
6
|
+
|
6
7
|
class PoolShuttingDownError < ::ConnectionPool::Error; end
|
8
|
+
|
7
9
|
class TimeoutError < ::Timeout::Error; end
|
8
10
|
end
|
9
11
|
|
@@ -42,6 +44,47 @@ class ConnectionPool
|
|
42
44
|
Wrapper.new(options, &block)
|
43
45
|
end
|
44
46
|
|
47
|
+
if Process.respond_to?(:fork)
|
48
|
+
INSTANCES = ObjectSpace::WeakMap.new
|
49
|
+
private_constant :INSTANCES
|
50
|
+
|
51
|
+
def self.after_fork
|
52
|
+
INSTANCES.values.each do |pool|
|
53
|
+
# We're on after fork, so we know all other threads are dead.
|
54
|
+
# All we need to do is to ensure the main thread doesn't have a
|
55
|
+
# checked out connection
|
56
|
+
pool.checkin(force: true)
|
57
|
+
|
58
|
+
pool.reload do |connection|
|
59
|
+
# Unfortunately we don't know what method to call to close the connection,
|
60
|
+
# so we try the most common one.
|
61
|
+
connection.close if connection.respond_to?(:close)
|
62
|
+
end
|
63
|
+
end
|
64
|
+
nil
|
65
|
+
end
|
66
|
+
|
67
|
+
if ::Process.respond_to?(:_fork) # MRI 3.1+
|
68
|
+
module ForkTracker
|
69
|
+
def _fork
|
70
|
+
pid = super
|
71
|
+
if pid == 0
|
72
|
+
ConnectionPool.after_fork
|
73
|
+
end
|
74
|
+
pid
|
75
|
+
end
|
76
|
+
end
|
77
|
+
Process.singleton_class.prepend(ForkTracker)
|
78
|
+
end
|
79
|
+
else
|
80
|
+
INSTANCES = nil
|
81
|
+
private_constant :INSTANCES
|
82
|
+
|
83
|
+
def self.after_fork
|
84
|
+
# noop
|
85
|
+
end
|
86
|
+
end
|
87
|
+
|
45
88
|
def initialize(options = {}, &block)
|
46
89
|
raise ArgumentError, "Connection pool requires a block" unless block
|
47
90
|
|
@@ -53,6 +96,7 @@ class ConnectionPool
|
|
53
96
|
@available = TimedStack.new(@size, &block)
|
54
97
|
@key = :"pool-#{@available.object_id}"
|
55
98
|
@key_count = :"pool-#{@available.object_id}-count"
|
99
|
+
INSTANCES[self] = self if INSTANCES
|
56
100
|
end
|
57
101
|
|
58
102
|
def with(options = {})
|
@@ -67,7 +111,7 @@ class ConnectionPool
|
|
67
111
|
end
|
68
112
|
end
|
69
113
|
end
|
70
|
-
|
114
|
+
alias_method :then, :with
|
71
115
|
|
72
116
|
def checkout(options = {})
|
73
117
|
if ::Thread.current[@key]
|
@@ -79,16 +123,16 @@ class ConnectionPool
|
|
79
123
|
end
|
80
124
|
end
|
81
125
|
|
82
|
-
def checkin
|
126
|
+
def checkin(force: false)
|
83
127
|
if ::Thread.current[@key]
|
84
|
-
if ::Thread.current[@key_count] == 1
|
128
|
+
if ::Thread.current[@key_count] == 1 || force
|
85
129
|
@available.push(::Thread.current[@key])
|
86
130
|
::Thread.current[@key] = nil
|
87
131
|
::Thread.current[@key_count] = nil
|
88
132
|
else
|
89
133
|
::Thread.current[@key_count] -= 1
|
90
134
|
end
|
91
|
-
|
135
|
+
elsif !force
|
92
136
|
raise ConnectionPool::Error, "no connections are checked out"
|
93
137
|
end
|
94
138
|
|
@@ -122,5 +166,5 @@ class ConnectionPool
|
|
122
166
|
end
|
123
167
|
end
|
124
168
|
|
125
|
-
|
126
|
-
|
169
|
+
require_relative "connection_pool/timed_stack"
|
170
|
+
require_relative "connection_pool/wrapper"
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: connection_pool
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.
|
4
|
+
version: 2.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Perham
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2023-03-24 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: bundler
|
@@ -61,21 +61,14 @@ executables: []
|
|
61
61
|
extensions: []
|
62
62
|
extra_rdoc_files: []
|
63
63
|
files:
|
64
|
-
- ".github/workflows/ci.yml"
|
65
|
-
- ".gitignore"
|
66
64
|
- Changes.md
|
67
|
-
- Gemfile
|
68
65
|
- LICENSE
|
69
66
|
- README.md
|
70
|
-
- Rakefile
|
71
67
|
- connection_pool.gemspec
|
72
68
|
- lib/connection_pool.rb
|
73
69
|
- lib/connection_pool/timed_stack.rb
|
74
70
|
- lib/connection_pool/version.rb
|
75
71
|
- lib/connection_pool/wrapper.rb
|
76
|
-
- test/helper.rb
|
77
|
-
- test/test_connection_pool.rb
|
78
|
-
- test/test_connection_pool_timed_stack.rb
|
79
72
|
homepage: https://github.com/mperham/connection_pool
|
80
73
|
licenses:
|
81
74
|
- MIT
|
@@ -88,18 +81,15 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
88
81
|
requirements:
|
89
82
|
- - ">="
|
90
83
|
- !ruby/object:Gem::Version
|
91
|
-
version: 2.
|
84
|
+
version: 2.5.0
|
92
85
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
93
86
|
requirements:
|
94
87
|
- - ">="
|
95
88
|
- !ruby/object:Gem::Version
|
96
89
|
version: '0'
|
97
90
|
requirements: []
|
98
|
-
rubygems_version: 3.
|
91
|
+
rubygems_version: 3.4.7
|
99
92
|
signing_key:
|
100
93
|
specification_version: 4
|
101
94
|
summary: Generic connection pool for Ruby
|
102
|
-
test_files:
|
103
|
-
- test/helper.rb
|
104
|
-
- test/test_connection_pool.rb
|
105
|
-
- test/test_connection_pool_timed_stack.rb
|
95
|
+
test_files: []
|
data/.github/workflows/ci.yml
DELETED
@@ -1,26 +0,0 @@
|
|
1
|
-
name: CI
|
2
|
-
|
3
|
-
on: [push, pull_request]
|
4
|
-
|
5
|
-
jobs:
|
6
|
-
test:
|
7
|
-
runs-on: ubuntu-latest
|
8
|
-
continue-on-error: ${{ matrix.experimental }}
|
9
|
-
strategy:
|
10
|
-
fail-fast: false
|
11
|
-
matrix:
|
12
|
-
ruby: ["2.4", "2.5", "2.6", "2.7", "3.0", "jruby"]
|
13
|
-
experimental: [false]
|
14
|
-
include:
|
15
|
-
- ruby: "truffleruby"
|
16
|
-
experimental: true
|
17
|
-
steps:
|
18
|
-
- uses: actions/checkout@v2
|
19
|
-
- uses: ruby/setup-ruby@v1
|
20
|
-
with:
|
21
|
-
ruby-version: ${{matrix.ruby}}
|
22
|
-
bundler-cache: true
|
23
|
-
|
24
|
-
- name: Run tests
|
25
|
-
timeout-minutes: 5
|
26
|
-
run: ${{matrix.env}} bundle exec rake
|
data/.gitignore
DELETED
data/Gemfile
DELETED
data/Rakefile
DELETED
data/test/helper.rb
DELETED
@@ -1,567 +0,0 @@
|
|
1
|
-
require_relative "helper"
|
2
|
-
|
3
|
-
class TestConnectionPool < Minitest::Test
|
4
|
-
class NetworkConnection
|
5
|
-
SLEEP_TIME = 0.1
|
6
|
-
|
7
|
-
def initialize
|
8
|
-
@x = 0
|
9
|
-
end
|
10
|
-
|
11
|
-
def do_something(*_args, increment: 1)
|
12
|
-
@x += increment
|
13
|
-
sleep SLEEP_TIME
|
14
|
-
@x
|
15
|
-
end
|
16
|
-
|
17
|
-
def do_something_with_positional_hash(options)
|
18
|
-
@x += options[:increment] || 1
|
19
|
-
sleep SLEEP_TIME
|
20
|
-
@x
|
21
|
-
end
|
22
|
-
|
23
|
-
def fast
|
24
|
-
@x += 1
|
25
|
-
end
|
26
|
-
|
27
|
-
def do_something_with_block
|
28
|
-
@x += yield
|
29
|
-
sleep SLEEP_TIME
|
30
|
-
@x
|
31
|
-
end
|
32
|
-
|
33
|
-
def respond_to?(method_id, *args)
|
34
|
-
method_id == :do_magic || super(method_id, *args)
|
35
|
-
end
|
36
|
-
end
|
37
|
-
|
38
|
-
class Recorder
|
39
|
-
def initialize
|
40
|
-
@calls = []
|
41
|
-
end
|
42
|
-
|
43
|
-
attr_reader :calls
|
44
|
-
|
45
|
-
def do_work(label)
|
46
|
-
@calls << label
|
47
|
-
end
|
48
|
-
end
|
49
|
-
|
50
|
-
def use_pool(pool, size)
|
51
|
-
Array.new(size) {
|
52
|
-
Thread.new do
|
53
|
-
pool.with { sleep }
|
54
|
-
end
|
55
|
-
}.each do |thread|
|
56
|
-
Thread.pass until thread.status == "sleep"
|
57
|
-
end
|
58
|
-
end
|
59
|
-
|
60
|
-
def kill_threads(threads)
|
61
|
-
threads.each do |thread|
|
62
|
-
thread.kill
|
63
|
-
thread.join
|
64
|
-
end
|
65
|
-
end
|
66
|
-
|
67
|
-
def test_basic_multithreaded_usage
|
68
|
-
pool_size = 5
|
69
|
-
pool = ConnectionPool.new(size: pool_size) { NetworkConnection.new }
|
70
|
-
|
71
|
-
start = Time.new
|
72
|
-
|
73
|
-
generations = 3
|
74
|
-
|
75
|
-
result = Array.new(pool_size * generations) {
|
76
|
-
Thread.new do
|
77
|
-
pool.with do |net|
|
78
|
-
net.do_something
|
79
|
-
end
|
80
|
-
end
|
81
|
-
}.map(&:value)
|
82
|
-
|
83
|
-
finish = Time.new
|
84
|
-
|
85
|
-
assert_equal((1..generations).cycle(pool_size).sort, result.sort)
|
86
|
-
|
87
|
-
assert_operator(finish - start, :>, generations * NetworkConnection::SLEEP_TIME)
|
88
|
-
end
|
89
|
-
|
90
|
-
def test_timeout
|
91
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { NetworkConnection.new }
|
92
|
-
thread = Thread.new {
|
93
|
-
pool.with do |net|
|
94
|
-
net.do_something
|
95
|
-
sleep 0.01
|
96
|
-
end
|
97
|
-
}
|
98
|
-
|
99
|
-
Thread.pass while thread.status == "run"
|
100
|
-
|
101
|
-
assert_raises Timeout::Error do
|
102
|
-
pool.with { |net| net.do_something }
|
103
|
-
end
|
104
|
-
|
105
|
-
thread.join
|
106
|
-
|
107
|
-
pool.with do |conn|
|
108
|
-
refute_nil conn
|
109
|
-
end
|
110
|
-
end
|
111
|
-
|
112
|
-
def test_with
|
113
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
114
|
-
|
115
|
-
pool.with do
|
116
|
-
Thread.new {
|
117
|
-
assert_raises Timeout::Error do
|
118
|
-
pool.checkout
|
119
|
-
end
|
120
|
-
}.join
|
121
|
-
end
|
122
|
-
|
123
|
-
assert Thread.new { pool.checkout }.join
|
124
|
-
end
|
125
|
-
|
126
|
-
def test_then
|
127
|
-
pool = ConnectionPool.new { Object.new }
|
128
|
-
|
129
|
-
assert_equal pool.method(:then), pool.method(:with)
|
130
|
-
end
|
131
|
-
|
132
|
-
def test_with_timeout
|
133
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
134
|
-
|
135
|
-
assert_raises Timeout::Error do
|
136
|
-
Timeout.timeout(0.01) do
|
137
|
-
pool.with do |obj|
|
138
|
-
assert_equal 0, pool.available
|
139
|
-
sleep 0.015
|
140
|
-
end
|
141
|
-
end
|
142
|
-
end
|
143
|
-
assert_equal 1, pool.available
|
144
|
-
end
|
145
|
-
|
146
|
-
def test_invalid_size
|
147
|
-
assert_raises ArgumentError, TypeError do
|
148
|
-
ConnectionPool.new(timeout: 0, size: nil) { Object.new }
|
149
|
-
end
|
150
|
-
assert_raises ArgumentError, TypeError do
|
151
|
-
ConnectionPool.new(timeout: 0, size: "") { Object.new }
|
152
|
-
end
|
153
|
-
end
|
154
|
-
|
155
|
-
def test_handle_interrupt_ensures_checkin
|
156
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
157
|
-
def pool.checkout(options)
|
158
|
-
sleep 0.015
|
159
|
-
super
|
160
|
-
end
|
161
|
-
|
162
|
-
did_something = false
|
163
|
-
|
164
|
-
action = lambda do
|
165
|
-
Timeout.timeout(0.01) do
|
166
|
-
pool.with do |obj|
|
167
|
-
did_something = true
|
168
|
-
# Timeout::Error will be triggered by any non-trivial Ruby code
|
169
|
-
# executed here since it couldn't be raised during checkout.
|
170
|
-
# It looks like setting the local variable above does not trigger
|
171
|
-
# the Timeout check in MRI 2.2.1.
|
172
|
-
obj.tap { obj.hash }
|
173
|
-
end
|
174
|
-
end
|
175
|
-
end
|
176
|
-
|
177
|
-
if RUBY_ENGINE == "ruby"
|
178
|
-
# These asserts rely on the Ruby implementation reaching `did_something =
|
179
|
-
# true` before the interrupt is detected by the thread. Interrupt
|
180
|
-
# detection timing is implementation-specific in practice, with JRuby,
|
181
|
-
# Rubinius, and TruffleRuby all having different interrupt timings to MRI.
|
182
|
-
# In fact they generally detect interrupts more quickly than MRI, so they
|
183
|
-
# may not reach `did_something = true` before detecting the interrupt.
|
184
|
-
|
185
|
-
assert_raises Timeout::Error, &action
|
186
|
-
|
187
|
-
assert did_something
|
188
|
-
else
|
189
|
-
action.call
|
190
|
-
end
|
191
|
-
|
192
|
-
assert_equal 1, pool.available
|
193
|
-
end
|
194
|
-
|
195
|
-
def test_explicit_return
|
196
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) {
|
197
|
-
mock = Minitest::Mock.new
|
198
|
-
def mock.disconnect!
|
199
|
-
raise "should not disconnect upon explicit return"
|
200
|
-
end
|
201
|
-
mock
|
202
|
-
}
|
203
|
-
|
204
|
-
pool.with do |conn|
|
205
|
-
return true
|
206
|
-
end
|
207
|
-
end
|
208
|
-
|
209
|
-
def test_with_timeout_override
|
210
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { NetworkConnection.new }
|
211
|
-
|
212
|
-
t = Thread.new {
|
213
|
-
pool.with do |net|
|
214
|
-
net.do_something
|
215
|
-
sleep 0.01
|
216
|
-
end
|
217
|
-
}
|
218
|
-
|
219
|
-
Thread.pass while t.status == "run"
|
220
|
-
|
221
|
-
assert_raises Timeout::Error do
|
222
|
-
pool.with { |net| net.do_something }
|
223
|
-
end
|
224
|
-
|
225
|
-
pool.with(timeout: 2 * NetworkConnection::SLEEP_TIME) do |conn|
|
226
|
-
refute_nil conn
|
227
|
-
end
|
228
|
-
end
|
229
|
-
|
230
|
-
def test_checkin
|
231
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { NetworkConnection.new }
|
232
|
-
conn = pool.checkout
|
233
|
-
|
234
|
-
Thread.new {
|
235
|
-
assert_raises Timeout::Error do
|
236
|
-
pool.checkout
|
237
|
-
end
|
238
|
-
}.join
|
239
|
-
|
240
|
-
pool.checkin
|
241
|
-
|
242
|
-
assert_same conn, Thread.new { pool.checkout }.value
|
243
|
-
end
|
244
|
-
|
245
|
-
def test_returns_value
|
246
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
247
|
-
assert_equal 1, pool.with { |o| 1 }
|
248
|
-
end
|
249
|
-
|
250
|
-
def test_checkin_never_checkout
|
251
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
252
|
-
|
253
|
-
e = assert_raises(ConnectionPool::Error) { pool.checkin }
|
254
|
-
assert_equal "no connections are checked out", e.message
|
255
|
-
end
|
256
|
-
|
257
|
-
def test_checkin_no_current_checkout
|
258
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
259
|
-
|
260
|
-
pool.checkout
|
261
|
-
pool.checkin
|
262
|
-
|
263
|
-
assert_raises ConnectionPool::Error do
|
264
|
-
pool.checkin
|
265
|
-
end
|
266
|
-
end
|
267
|
-
|
268
|
-
def test_checkin_twice
|
269
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { Object.new }
|
270
|
-
|
271
|
-
pool.checkout
|
272
|
-
pool.checkout
|
273
|
-
|
274
|
-
pool.checkin
|
275
|
-
|
276
|
-
Thread.new {
|
277
|
-
assert_raises Timeout::Error do
|
278
|
-
pool.checkout
|
279
|
-
end
|
280
|
-
}.join
|
281
|
-
|
282
|
-
pool.checkin
|
283
|
-
|
284
|
-
assert Thread.new { pool.checkout }.join
|
285
|
-
end
|
286
|
-
|
287
|
-
def test_checkout
|
288
|
-
pool = ConnectionPool.new(size: 1) { NetworkConnection.new }
|
289
|
-
|
290
|
-
conn = pool.checkout
|
291
|
-
|
292
|
-
assert_kind_of NetworkConnection, conn
|
293
|
-
|
294
|
-
assert_same conn, pool.checkout
|
295
|
-
end
|
296
|
-
|
297
|
-
def test_checkout_multithread
|
298
|
-
pool = ConnectionPool.new(size: 2) { NetworkConnection.new }
|
299
|
-
conn = pool.checkout
|
300
|
-
|
301
|
-
t = Thread.new {
|
302
|
-
pool.checkout
|
303
|
-
}
|
304
|
-
|
305
|
-
refute_same conn, t.value
|
306
|
-
end
|
307
|
-
|
308
|
-
def test_checkout_timeout
|
309
|
-
pool = ConnectionPool.new(timeout: 0, size: 0) { Object.new }
|
310
|
-
|
311
|
-
assert_raises Timeout::Error do
|
312
|
-
pool.checkout
|
313
|
-
end
|
314
|
-
end
|
315
|
-
|
316
|
-
def test_checkout_timeout_override
|
317
|
-
pool = ConnectionPool.new(timeout: 0, size: 1) { NetworkConnection.new }
|
318
|
-
|
319
|
-
thread = Thread.new {
|
320
|
-
pool.with do |net|
|
321
|
-
net.do_something
|
322
|
-
sleep 0.01
|
323
|
-
end
|
324
|
-
}
|
325
|
-
|
326
|
-
Thread.pass while thread.status == "run"
|
327
|
-
|
328
|
-
assert_raises Timeout::Error do
|
329
|
-
pool.checkout
|
330
|
-
end
|
331
|
-
|
332
|
-
assert pool.checkout timeout: 2 * NetworkConnection::SLEEP_TIME
|
333
|
-
end
|
334
|
-
|
335
|
-
def test_passthru
|
336
|
-
pool = ConnectionPool.wrap(timeout: 2 * NetworkConnection::SLEEP_TIME, size: 1) { NetworkConnection.new }
|
337
|
-
assert_equal 1, pool.do_something
|
338
|
-
assert_equal 2, pool.do_something
|
339
|
-
assert_equal 5, pool.do_something_with_block { 3 }
|
340
|
-
assert_equal 6, pool.with { |net| net.fast }
|
341
|
-
assert_equal 8, pool.do_something(increment: 2)
|
342
|
-
assert_equal 10, pool.do_something_with_positional_hash({ increment: 2, symbol_key: 3, "string_key" => 4 })
|
343
|
-
end
|
344
|
-
|
345
|
-
def test_passthru_respond_to
|
346
|
-
pool = ConnectionPool.wrap(timeout: 2 * NetworkConnection::SLEEP_TIME, size: 1) { NetworkConnection.new }
|
347
|
-
assert pool.respond_to?(:with)
|
348
|
-
assert pool.respond_to?(:do_something)
|
349
|
-
assert pool.respond_to?(:do_magic)
|
350
|
-
refute pool.respond_to?(:do_lots_of_magic)
|
351
|
-
end
|
352
|
-
|
353
|
-
def test_return_value
|
354
|
-
pool = ConnectionPool.new(timeout: 2 * NetworkConnection::SLEEP_TIME, size: 1) { NetworkConnection.new }
|
355
|
-
result = pool.with { |net|
|
356
|
-
net.fast
|
357
|
-
}
|
358
|
-
assert_equal 1, result
|
359
|
-
end
|
360
|
-
|
361
|
-
def test_heavy_threading
|
362
|
-
pool = ConnectionPool.new(timeout: 0.5, size: 3) { NetworkConnection.new }
|
363
|
-
|
364
|
-
threads = Array.new(20) {
|
365
|
-
Thread.new do
|
366
|
-
pool.with do |net|
|
367
|
-
sleep 0.01
|
368
|
-
end
|
369
|
-
end
|
370
|
-
}
|
371
|
-
|
372
|
-
threads.map { |thread| thread.join }
|
373
|
-
end
|
374
|
-
|
375
|
-
def test_reuses_objects_when_pool_not_saturated
|
376
|
-
pool = ConnectionPool.new(size: 5) { NetworkConnection.new }
|
377
|
-
|
378
|
-
ids = 10.times.map {
|
379
|
-
pool.with { |c| c.object_id }
|
380
|
-
}
|
381
|
-
|
382
|
-
assert_equal 1, ids.uniq.size
|
383
|
-
end
|
384
|
-
|
385
|
-
def test_nested_checkout
|
386
|
-
recorder = Recorder.new
|
387
|
-
pool = ConnectionPool.new(size: 1) { recorder }
|
388
|
-
pool.with do |r_outer|
|
389
|
-
@other = Thread.new { |t|
|
390
|
-
pool.with do |r_other|
|
391
|
-
r_other.do_work("other")
|
392
|
-
end
|
393
|
-
}
|
394
|
-
|
395
|
-
pool.with do |r_inner|
|
396
|
-
r_inner.do_work("inner")
|
397
|
-
end
|
398
|
-
|
399
|
-
Thread.pass
|
400
|
-
|
401
|
-
r_outer.do_work("outer")
|
402
|
-
end
|
403
|
-
|
404
|
-
@other.join
|
405
|
-
|
406
|
-
assert_equal ["inner", "outer", "other"], recorder.calls
|
407
|
-
end
|
408
|
-
|
409
|
-
def test_shutdown_is_executed_for_all_connections
|
410
|
-
recorders = []
|
411
|
-
|
412
|
-
pool = ConnectionPool.new(size: 3) {
|
413
|
-
Recorder.new.tap { |r| recorders << r }
|
414
|
-
}
|
415
|
-
|
416
|
-
threads = use_pool pool, 3
|
417
|
-
|
418
|
-
pool.shutdown do |recorder|
|
419
|
-
recorder.do_work("shutdown")
|
420
|
-
end
|
421
|
-
|
422
|
-
kill_threads(threads)
|
423
|
-
|
424
|
-
assert_equal [["shutdown"]] * 3, recorders.map { |r| r.calls }
|
425
|
-
end
|
426
|
-
|
427
|
-
def test_raises_error_after_shutting_down
|
428
|
-
pool = ConnectionPool.new(size: 1) { true }
|
429
|
-
|
430
|
-
pool.shutdown {}
|
431
|
-
|
432
|
-
assert_raises ConnectionPool::PoolShuttingDownError do
|
433
|
-
pool.checkout
|
434
|
-
end
|
435
|
-
end
|
436
|
-
|
437
|
-
def test_runs_shutdown_block_asynchronously_if_connection_was_in_use
|
438
|
-
recorders = []
|
439
|
-
|
440
|
-
pool = ConnectionPool.new(size: 3) {
|
441
|
-
Recorder.new.tap { |r| recorders << r }
|
442
|
-
}
|
443
|
-
|
444
|
-
threads = use_pool pool, 2
|
445
|
-
|
446
|
-
pool.checkout
|
447
|
-
|
448
|
-
pool.shutdown do |recorder|
|
449
|
-
recorder.do_work("shutdown")
|
450
|
-
end
|
451
|
-
|
452
|
-
kill_threads(threads)
|
453
|
-
|
454
|
-
assert_equal [["shutdown"], ["shutdown"], []], recorders.map { |r| r.calls }
|
455
|
-
|
456
|
-
pool.checkin
|
457
|
-
|
458
|
-
assert_equal [["shutdown"], ["shutdown"], ["shutdown"]], recorders.map { |r| r.calls }
|
459
|
-
end
|
460
|
-
|
461
|
-
def test_raises_an_error_if_shutdown_is_called_without_a_block
|
462
|
-
pool = ConnectionPool.new(size: 1) {}
|
463
|
-
|
464
|
-
assert_raises ArgumentError do
|
465
|
-
pool.shutdown
|
466
|
-
end
|
467
|
-
end
|
468
|
-
|
469
|
-
def test_shutdown_is_executed_for_all_connections_in_wrapped_pool
|
470
|
-
recorders = []
|
471
|
-
|
472
|
-
wrapper = ConnectionPool::Wrapper.new(size: 3) {
|
473
|
-
Recorder.new.tap { |r| recorders << r }
|
474
|
-
}
|
475
|
-
|
476
|
-
threads = use_pool wrapper, 3
|
477
|
-
|
478
|
-
wrapper.pool_shutdown do |recorder|
|
479
|
-
recorder.do_work("shutdown")
|
480
|
-
end
|
481
|
-
|
482
|
-
kill_threads(threads)
|
483
|
-
|
484
|
-
assert_equal [["shutdown"]] * 3, recorders.map { |r| r.calls }
|
485
|
-
end
|
486
|
-
|
487
|
-
def test_wrapper_wrapped_pool
|
488
|
-
wrapper = ConnectionPool::Wrapper.new { NetworkConnection.new }
|
489
|
-
assert_equal ConnectionPool, wrapper.wrapped_pool.class
|
490
|
-
end
|
491
|
-
|
492
|
-
def test_wrapper_method_missing
|
493
|
-
wrapper = ConnectionPool::Wrapper.new { NetworkConnection.new }
|
494
|
-
|
495
|
-
assert_equal 1, wrapper.fast
|
496
|
-
end
|
497
|
-
|
498
|
-
def test_wrapper_respond_to_eh
|
499
|
-
wrapper = ConnectionPool::Wrapper.new { NetworkConnection.new }
|
500
|
-
|
501
|
-
assert_respond_to wrapper, :with
|
502
|
-
|
503
|
-
assert_respond_to wrapper, :fast
|
504
|
-
refute_respond_to wrapper, :"nonexistent method"
|
505
|
-
end
|
506
|
-
|
507
|
-
def test_wrapper_with
|
508
|
-
wrapper = ConnectionPool::Wrapper.new(timeout: 0, size: 1) { Object.new }
|
509
|
-
|
510
|
-
wrapper.with do
|
511
|
-
Thread.new {
|
512
|
-
assert_raises Timeout::Error do
|
513
|
-
wrapper.with { flunk "connection checked out :(" }
|
514
|
-
end
|
515
|
-
}.join
|
516
|
-
end
|
517
|
-
|
518
|
-
assert Thread.new { wrapper.with {} }.join
|
519
|
-
end
|
520
|
-
|
521
|
-
class ConnWithEval
|
522
|
-
def eval(arg)
|
523
|
-
"eval'ed #{arg}"
|
524
|
-
end
|
525
|
-
end
|
526
|
-
|
527
|
-
def test_wrapper_kernel_methods
|
528
|
-
wrapper = ConnectionPool::Wrapper.new(timeout: 0, size: 1) { ConnWithEval.new }
|
529
|
-
|
530
|
-
assert_equal "eval'ed 1", wrapper.eval(1)
|
531
|
-
end
|
532
|
-
|
533
|
-
def test_wrapper_with_connection_pool
|
534
|
-
recorder = Recorder.new
|
535
|
-
pool = ConnectionPool.new(size: 1) { recorder }
|
536
|
-
wrapper = ConnectionPool::Wrapper.new(pool: pool)
|
537
|
-
|
538
|
-
pool.with { |r| r.do_work("with") }
|
539
|
-
wrapper.do_work("wrapped")
|
540
|
-
|
541
|
-
assert_equal ["with", "wrapped"], recorder.calls
|
542
|
-
end
|
543
|
-
|
544
|
-
def test_stats_without_active_connection
|
545
|
-
pool = ConnectionPool.new(size: 2) { NetworkConnection.new }
|
546
|
-
|
547
|
-
assert_equal(2, pool.size)
|
548
|
-
assert_equal(2, pool.available)
|
549
|
-
end
|
550
|
-
|
551
|
-
def test_stats_with_active_connection
|
552
|
-
pool = ConnectionPool.new(size: 2) { NetworkConnection.new }
|
553
|
-
|
554
|
-
pool.with do
|
555
|
-
assert_equal(1, pool.available)
|
556
|
-
end
|
557
|
-
end
|
558
|
-
|
559
|
-
def test_stats_with_string_size
|
560
|
-
pool = ConnectionPool.new(size: "2") { NetworkConnection.new }
|
561
|
-
|
562
|
-
pool.with do
|
563
|
-
assert_equal(2, pool.size)
|
564
|
-
assert_equal(1, pool.available)
|
565
|
-
end
|
566
|
-
end
|
567
|
-
end
|
@@ -1,150 +0,0 @@
|
|
1
|
-
require_relative "helper"
|
2
|
-
|
3
|
-
class TestConnectionPoolTimedStack < Minitest::Test
|
4
|
-
def setup
|
5
|
-
@stack = ConnectionPool::TimedStack.new { Object.new }
|
6
|
-
end
|
7
|
-
|
8
|
-
def test_empty_eh
|
9
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
10
|
-
|
11
|
-
refute_empty stack
|
12
|
-
|
13
|
-
popped = stack.pop
|
14
|
-
|
15
|
-
assert_empty stack
|
16
|
-
|
17
|
-
stack.push popped
|
18
|
-
|
19
|
-
refute_empty stack
|
20
|
-
end
|
21
|
-
|
22
|
-
def test_length
|
23
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
24
|
-
|
25
|
-
assert_equal 1, stack.length
|
26
|
-
|
27
|
-
popped = stack.pop
|
28
|
-
|
29
|
-
assert_equal 0, stack.length
|
30
|
-
|
31
|
-
stack.push popped
|
32
|
-
|
33
|
-
assert_equal 1, stack.length
|
34
|
-
end
|
35
|
-
|
36
|
-
def test_object_creation_fails
|
37
|
-
stack = ConnectionPool::TimedStack.new(2) { raise "failure" }
|
38
|
-
|
39
|
-
begin
|
40
|
-
stack.pop
|
41
|
-
rescue => error
|
42
|
-
assert_equal "failure", error.message
|
43
|
-
end
|
44
|
-
|
45
|
-
begin
|
46
|
-
stack.pop
|
47
|
-
rescue => error
|
48
|
-
assert_equal "failure", error.message
|
49
|
-
end
|
50
|
-
|
51
|
-
refute_empty stack
|
52
|
-
assert_equal 2, stack.length
|
53
|
-
end
|
54
|
-
|
55
|
-
def test_pop
|
56
|
-
object = Object.new
|
57
|
-
@stack.push object
|
58
|
-
|
59
|
-
popped = @stack.pop
|
60
|
-
|
61
|
-
assert_same object, popped
|
62
|
-
end
|
63
|
-
|
64
|
-
def test_pop_empty
|
65
|
-
e = assert_raises(ConnectionPool::TimeoutError) { @stack.pop timeout: 0 }
|
66
|
-
assert_equal "Waited 0 sec", e.message
|
67
|
-
end
|
68
|
-
|
69
|
-
def test_pop_empty_2_0_compatibility
|
70
|
-
e = assert_raises(Timeout::Error) { @stack.pop 0 }
|
71
|
-
assert_equal "Waited 0 sec", e.message
|
72
|
-
end
|
73
|
-
|
74
|
-
def test_pop_full
|
75
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
76
|
-
|
77
|
-
popped = stack.pop
|
78
|
-
|
79
|
-
refute_nil popped
|
80
|
-
assert_empty stack
|
81
|
-
end
|
82
|
-
|
83
|
-
def test_pop_wait
|
84
|
-
thread = Thread.start {
|
85
|
-
@stack.pop
|
86
|
-
}
|
87
|
-
|
88
|
-
Thread.pass while thread.status == "run"
|
89
|
-
|
90
|
-
object = Object.new
|
91
|
-
|
92
|
-
@stack.push object
|
93
|
-
|
94
|
-
assert_same object, thread.value
|
95
|
-
end
|
96
|
-
|
97
|
-
def test_pop_shutdown
|
98
|
-
@stack.shutdown {}
|
99
|
-
|
100
|
-
assert_raises ConnectionPool::PoolShuttingDownError do
|
101
|
-
@stack.pop
|
102
|
-
end
|
103
|
-
end
|
104
|
-
|
105
|
-
def test_pop_shutdown_reload
|
106
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
107
|
-
object = stack.pop
|
108
|
-
stack.push(object)
|
109
|
-
|
110
|
-
stack.shutdown(reload: true) {}
|
111
|
-
|
112
|
-
refute_equal object, stack.pop
|
113
|
-
end
|
114
|
-
|
115
|
-
def test_push
|
116
|
-
stack = ConnectionPool::TimedStack.new(1) { Object.new }
|
117
|
-
|
118
|
-
conn = stack.pop
|
119
|
-
|
120
|
-
stack.push conn
|
121
|
-
|
122
|
-
refute_empty stack
|
123
|
-
end
|
124
|
-
|
125
|
-
def test_push_shutdown
|
126
|
-
called = []
|
127
|
-
|
128
|
-
@stack.shutdown do |object|
|
129
|
-
called << object
|
130
|
-
end
|
131
|
-
|
132
|
-
@stack.push Object.new
|
133
|
-
|
134
|
-
refute_empty called
|
135
|
-
assert_empty @stack
|
136
|
-
end
|
137
|
-
|
138
|
-
def test_shutdown
|
139
|
-
@stack.push Object.new
|
140
|
-
|
141
|
-
called = []
|
142
|
-
|
143
|
-
@stack.shutdown do |object|
|
144
|
-
called << object
|
145
|
-
end
|
146
|
-
|
147
|
-
refute_empty called
|
148
|
-
assert_empty @stack
|
149
|
-
end
|
150
|
-
end
|