pond 0.1.0 → 0.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
- ---
2
- SHA1:
3
- metadata.gz: 60082a3f35a388cf665c609388ee4025e58006d1
4
- data.tar.gz: 1e0ed085f1a3f3943c85ce1f00976f2d892c1e33
5
- SHA512:
6
- metadata.gz: 7e1b8250177dbd7b8ee56d1750debade92eabb7ad22b9b778d72f8465083d3c0071f1674713a0d44a06aa3d6df18ac549e7bfade50c3a24293a68168dbf0dde7
7
- data.tar.gz: 24b0bd2e914f6932bdb4ea2e4c532543a3a7928e8c01d11326ed44b5254c261b1ee3032408c0f42092a5a8a6bd48e6d8406b6f1ddaee3dce5ddf4fd5202b2529
1
+ ---
2
+ SHA1:
3
+ metadata.gz: c9254999a6eb11a5873947aa1fee8bc5ad326be3
4
+ data.tar.gz: fc080fdb56ada8713ffff8bbe2b82a535a4125c3
5
+ SHA512:
6
+ metadata.gz: aecc96803e996c38b4168fa287cd3f4f39eb0d8b2d448b9a6c36d9fa63fcb2cba14e2fc06f4377238d7bf6632410a1ebff38e460d8200c975249b77b1b9f7eae
7
+ data.tar.gz: ded459d09b79d4017db0e878a0cbccc44c99c47998a6224ebdb34ec8bd91dddf15763cda57d435a9f79f299fb0f316c1b2ed66de5b7223c25fb2bae1d91638b5
@@ -1,3 +1,8 @@
1
+ ### 0.2.0 (2016-02-05)
2
+
3
+ * Add an option for a detach_if callable, which can contain logic to
4
+ determine whether to remove an object from the pool.
5
+
1
6
  ### 0.1.0 (2014-02-14)
2
7
 
3
8
  * Initial release.
data/README.md CHANGED
@@ -1,6 +1,11 @@
1
1
  # Pond
2
2
 
3
- Pond is a gem that offers thread-safe object pooling. It can wrap anything that is costly to instantiate, but is usually used for connections. It is intentionally very similar to the `connection_pool` gem, but is intended to be more efficient and flexible. It instantiates objects lazily by default, which is important for things with high overhead like Postgres connections. It can also be dynamically resized.
3
+ Pond is a gem that offers thread-safe object pooling. It can wrap anything
4
+ that is costly to instantiate, but is usually used for connections. It is
5
+ intentionally very similar to the `connection_pool` gem, but is intended to be
6
+ more efficient and flexible. It instantiates objects lazily by default, which
7
+ is important for things with high overhead like Postgres connections. It can
8
+ also be dynamically resized, and does not block on object instantiation.
4
9
 
5
10
  Also, it was pretty fun to write.
6
11
 
@@ -20,36 +25,78 @@ Or install it yourself as:
20
25
 
21
26
  ## Usage
22
27
 
23
- require 'pond'
24
- require 'redis'
28
+ ```ruby
29
+ require 'pond'
30
+ require 'redis'
25
31
 
26
- $redis_pond = Pond.new(:maximum_size => 5, :timeout => 0.5) { Redis.new }
32
+ $redis_pond = Pond.new(:maximum_size => 5, :timeout => 0.5) { Redis.new }
27
33
 
28
- # No connections are established until we need one:
29
- $redis_pond.checkout do |redis|
30
- redis.incr 'my_counter'
31
- redis.lpush 'my_list', 'item'
32
- end
34
+ # No connections are established until we need one:
35
+ $redis_pond.checkout do |redis|
36
+ redis.incr 'my_counter'
37
+ redis.lpush 'my_list', 'item'
38
+ end
33
39
 
34
- # Alternatively, wrap it:
35
- $redis = Pond.wrap(:maximum_size => 5, :timeout => 0.5) { Redis.new }
40
+ # Alternatively, wrap it:
41
+ $redis = Pond.wrap(:maximum_size => 5, :timeout => 0.5) { Redis.new }
36
42
 
37
- # You can now use $redis as you normally would.
38
- $redis.incr 'my_counter'
39
- $redis.lpush 'my_list', 'item'
43
+ # You can now use $redis as you normally would.
44
+ $redis.incr 'my_counter'
45
+ $redis.lpush 'my_list', 'item'
40
46
 
41
- $redis.pipelined do
42
- # All these commands go to the same Redis connection, and so are pipelined correctly.
43
- $redis.incr 'my_counter'
44
- $redis.lpush 'my_list', 'item'
45
- end
47
+ $redis.pipelined do
48
+ # All these commands go to the same Redis connection, and so are pipelined correctly.
49
+ $redis.incr 'my_counter'
50
+ $redis.lpush 'my_list', 'item'
51
+ end
52
+ ```
46
53
 
47
54
  Options:
48
- * :maximum_size - The maximum number of objects you want the pool to contain. The default is 10.
49
- * :timeout - When attempting to check out an object but none are available, how many seconds to wait before raising a `Pond::Timeout` error. The default is 1.
50
- * :collection - How to manage the objects in the pool. The default is :queue, meaning that pond.checkout will yield the object that hasn't been used in the longest period of time. This is to prevent connections from becoming 'stale'. The alternative is :stack, so checkout will yield the object that has most recently been returned to the pool. This would be preferable if you're using connections that have their own logic for becoming idle in periods of low activity.
51
- * :eager - Set this to true to fill the pool with instantiated objects when it is created, similar to how `connection_pool` works.
55
+
56
+ * :maximum_size - The maximum number of objects you want the pool to contain.
57
+ The default is 10.
58
+ * :timeout - When attempting to check out an object but none are available,
59
+ how many seconds to wait before raising a `Pond::Timeout` error. The
60
+ default is 1. Integers or floats are both accepted.
61
+ * :collection - How to manage the objects in the pool. The default is :queue,
62
+ meaning that pond.checkout will yield the object that hasn't been used in
63
+ the longest period of time. This is to prevent connections from becoming
64
+ 'stale'. The alternative is :stack, so checkout will yield the object that
65
+ has most recently been returned to the pool. This would be preferable if
66
+ you're using connections that have their own logic for becoming idle in
67
+ periods of low activity.
68
+ * :eager - Set this to true to fill the pool with instantiated objects (up to
69
+ the maximum size) when it is created, similar to how the `connection_pool`
70
+ gem works.
71
+ * :detach_if - Set this to a callable object that can determine whether
72
+ objects should be returned to the pool or not. See the following example for
73
+ more information.
74
+
75
+ ### Detaching Objects
76
+
77
+ Sometimes objects in the pool outlive their usefulness (connections may fail)
78
+ and it becomes necessary to remove them. Pond's detach_if option is useful for
79
+ this - you can pass it any callable object, and Pond will pass it objects from
80
+ the pool that have been checked out before they are checked back in. For
81
+ example, when using Pond with PostgreSQL connections:
82
+
83
+ ```ruby
84
+ require 'pond'
85
+ require 'pg'
86
+
87
+ $pg_pond = Pond.new(:detach_if => lambda {|c| c.finished?}) do
88
+ PG.connect(:dbname => "pond_test")
89
+ end
90
+ ```
91
+
92
+ Now, after a PostgreSQL connection has been used, but before it is returned to
93
+ the pool, it will be passed to that lambda to see if it should be detached or
94
+ not. If the lambda returns truthy, the connection will be detached (and made
95
+ available for garbage collection), and a new one will be instantiated to
96
+ replace it as necessary (until the pool returns to its maximum size).
52
97
 
53
98
  ## Contributing
54
99
 
55
- I don't plan on adding too many more features to Pond, since I want to keep its design simple. If there's something you'd like to see it do, open an issue so we can discuss it before going to the trouble of creating a pull request.
100
+ I don't plan on adding too many more features to Pond, since I want to keep
101
+ its design simple. If there's something you'd like to see it do, open an issue
102
+ so we can discuss it before going to the trouble of creating a pull request.
data/Rakefile CHANGED
@@ -5,3 +5,5 @@ require 'rspec/core/rake_task'
5
5
  RSpec::Core::RakeTask.new :default do |spec|
6
6
  spec.pattern = './spec/**/*_spec.rb'
7
7
  end
8
+
9
+ Dir[File.dirname(__FILE__) + '/tasks/**/*.rake'].sort.each &method(:load)
@@ -5,7 +5,7 @@ require 'pond/version'
5
5
  class Pond
6
6
  class Timeout < StandardError; end
7
7
 
8
- attr_reader :allocated, :available, :timeout, :collection, :maximum_size
8
+ attr_reader :allocated, :available, :timeout, :collection, :maximum_size, :detach_if
9
9
 
10
10
  def initialize(options = {}, &block)
11
11
  @block = block
@@ -19,6 +19,7 @@ class Pond
19
19
 
20
20
  self.timeout = options.fetch :timeout, 1
21
21
  self.collection = options.fetch :collection, :queue
22
+ self.detach_if = options.fetch :detach_if, lambda { |_| false }
22
23
  self.maximum_size = maximum_size
23
24
  end
24
25
 
@@ -52,6 +53,11 @@ class Pond
52
53
  end
53
54
  end
54
55
 
56
+ def detach_if=(callable)
57
+ raise "Object given for Pond detach_if must respond to #call" unless callable.respond_to?(:call)
58
+ sync { @detach_if = callable }
59
+ end
60
+
55
61
  private
56
62
 
57
63
  def checkout_object
@@ -83,11 +89,23 @@ class Pond
83
89
  end
84
90
 
85
91
  def unlock_object
92
+ object = nil
93
+ detach_if = nil
94
+ should_return_object = nil
95
+
86
96
  sync do
87
- object = @allocated.delete(Thread.current)
97
+ object = current_object
98
+ detach_if = self.detach_if
99
+ should_return_object = object && object != @block && size <= maximum_size
100
+ end
88
101
 
89
- if object && object != @block && size < maximum_size
90
- @available << object
102
+ begin
103
+ should_return_object = !detach_if.call(object) if should_return_object
104
+ detach_check_finished = true
105
+ ensure
106
+ sync do
107
+ @available << object if detach_check_finished && should_return_object
108
+ @allocated.delete(Thread.current)
91
109
  @cv.signal
92
110
  end
93
111
  end
@@ -1,3 +1,3 @@
1
1
  class Pond
2
- VERSION = '0.1.0'
2
+ VERSION = '0.2.0'
3
3
  end
@@ -233,7 +233,7 @@ describe Pond, "#checkout" do
233
233
 
234
234
  it "should not block other threads if the object instantiation takes a long time" do
235
235
  t = nil
236
- q1, q2, q3 = Queue.new, Queue.new, Queue.new
236
+ q1, q2 = Queue.new, Queue.new
237
237
  pond = Pond.new do
238
238
  q1.push nil
239
239
  q2.pop
@@ -289,5 +289,109 @@ describe Pond, "#checkout" do
289
289
  pond.size.should == 1
290
290
  pond.allocated.should == {}
291
291
  pond.available.should == [1]
292
+
293
+ error = false
294
+
295
+ pond.checkout do |i|
296
+ i.should == 1
297
+
298
+ t = Thread.new do
299
+ pond.checkout { |j| j.should == 2 }
300
+ end
301
+
302
+ t.join
303
+ end
304
+
305
+ pond.size.should == 2
306
+ pond.allocated.should == {}
307
+ pond.available.should == [2, 1]
308
+ end
309
+
310
+ it "removes the object from the pool if the detach_if block returns true" do
311
+ int = 0
312
+ pond = Pond.new(detach_if: lambda { |obj| obj < 2 }) { int += 1 }
313
+ pond.available.should == []
314
+
315
+ # allocate 1, should not check back in
316
+ pond.checkout {|i| i.should == 1}
317
+ pond.available.should == []
318
+
319
+ # allocate 2, should be nothing else in the pond
320
+ pond.checkout do |i|
321
+ i.should == 2
322
+ pond.available.should == []
323
+ end
324
+
325
+ # 2 should still be in the pond
326
+ pond.available.should == [2]
327
+ end
328
+
329
+ it "should not block other threads if the detach_if block takes a long time" do
330
+ i = 0
331
+ q1, q2 = Queue.new, Queue.new
332
+
333
+ detach_if = proc do
334
+ q1.push nil
335
+ q2.pop
336
+ end
337
+
338
+ pond = Pond.new(:detach_if => detach_if) do
339
+ i += 1
340
+ end
341
+
342
+ t = Thread.new do
343
+ pond.checkout do |i|
344
+ i.should == 1
345
+ end
346
+ end
347
+
348
+ q1.pop
349
+ pond.available.should == []
350
+ pond.allocated.should == {t => 1}
351
+
352
+ # t is in the middle of invoking detach_if, we should still be able to
353
+ # instantiate new objects and check them out.
354
+ pond.checkout do |i|
355
+ i.should == 2
356
+ q2.push nil
357
+ q2.push nil
358
+ end
359
+
360
+ t.join
361
+ end
362
+
363
+ it "should not leave the Pond in a bad state if the detach_if block fails" do
364
+ i = 0
365
+ error = false
366
+ detach_if = proc do |obj|
367
+ raise "Detach Error!" if error
368
+ false
369
+ end
370
+
371
+ pond = Pond.new(:eager => true, :maximum_size => 5, :detach_if => detach_if) do
372
+ i += 1
373
+ end
374
+
375
+ pond.size.should == 5
376
+ pond.allocated.should == {}
377
+ pond.available.should == [1, 2, 3, 4, 5]
378
+
379
+ pond.checkout {}
380
+
381
+ pond.available.should == [2, 3, 4, 5, 1]
382
+
383
+ error = true
384
+
385
+ checked_out = nil
386
+ proc {
387
+ pond.checkout do |i|
388
+ checked_out = i
389
+ end
390
+ }.should raise_error RuntimeError, "Detach Error!"
391
+
392
+ checked_out.should == 2
393
+
394
+ pond.available.should == [3, 4, 5, 1]
395
+ pond.allocated.should == {}
292
396
  end
293
397
  end
@@ -1 +1,5 @@
1
1
  require 'pond'
2
+
3
+ RSpec.configure do |config|
4
+ config.expect_with(:rspec) { |c| c.syntax = [:expect, :should] }
5
+ end
@@ -2,6 +2,11 @@ require 'spec_helper'
2
2
 
3
3
  describe Pond::Wrapper do
4
4
  class Wrapped
5
+ # JRuby implements BasicObject#object_id, so we need a minor workaround.
6
+ def id
7
+ object_id
8
+ end
9
+
5
10
  def pipelined(&block)
6
11
  yield
7
12
  end
@@ -17,34 +22,34 @@ describe Pond::Wrapper do
17
22
 
18
23
  @wrapper.class.should == Wrapped
19
24
  @wrapper.respond_to?(:pipelined).should == true
20
- object_id = @wrapper.object_id
25
+ id = @wrapper.id
21
26
 
22
27
  @pond.size.should == 1
23
28
  @pond.allocated.should == {}
24
- @pond.available.map(&:object_id).should == [object_id]
29
+ @pond.available.map(&:id).should == [id]
25
30
  end
26
31
 
27
32
  it "should return the same object within a block passed to one of its methods" do
28
33
  q1, q2 = Queue.new, Queue.new
29
- oid1, oid2 = nil, nil
34
+ id1, id2 = nil, nil
30
35
 
31
36
  @wrapper.pipelined do
32
- oid1 = @wrapper.object_id
37
+ id1 = @wrapper.id
33
38
 
34
39
  t = Thread.new do
35
40
  @wrapper.pipelined do
36
41
  q1.push nil
37
42
  q2.pop
38
43
 
39
- oid2 = @wrapper.object_id
40
- oid2.should == @wrapper.object_id
44
+ id2 = @wrapper.id
45
+ id2.should == @wrapper.id
41
46
  @wrapper
42
47
  end
43
48
  end
44
49
 
45
50
  q1.pop
46
51
 
47
- @wrapper.object_id.should == oid1
52
+ @wrapper.id.should == id1
48
53
 
49
54
  @pond.allocated.keys.should == [Thread.current, t]
50
55
  @pond.available.should == []
@@ -52,11 +57,11 @@ describe Pond::Wrapper do
52
57
  q2.push nil
53
58
  t.join
54
59
 
55
- @wrapper.object_id.should == oid1
56
- @wrapper.object_id.should == oid1
60
+ @wrapper.id.should == id1
61
+ @wrapper.id.should == id1
57
62
  end
58
63
 
59
64
  @pond.allocated.should == {}
60
- @pond.available.map(&:object_id).should == [oid2, oid1]
65
+ @pond.available.map(&:id).should == [id2, id1]
61
66
  end
62
67
  end
@@ -0,0 +1,34 @@
1
+ require 'pond'
2
+
3
+ desc "Stress test the Pond gem to check for concurrency issues."
4
+ task :stress do
5
+ detach_if = proc do |obj|
6
+ raise "Bad Detach!" if rand < 0.05
7
+ obj != "Good!"
8
+ end
9
+
10
+ pond = Pond.new(detach_if: detach_if) do
11
+ raise "Bad Instantiation!" if rand < 0.05
12
+ "Good!"
13
+ end
14
+
15
+ threads =
16
+ 20.times.map do
17
+ Thread.new do
18
+ 10_000.times do
19
+ begin
20
+ pond.checkout do |o|
21
+ raise "Uh-oh!" unless o == "Good!"
22
+ o.replace "Bad!" if rand < 0.05
23
+ end
24
+ rescue => e
25
+ raise e unless ["Bad Detach!", "Bad Instantiation!"].include?(e.message)
26
+ end
27
+ end
28
+ end
29
+ end
30
+
31
+ threads.each(&:join)
32
+
33
+ puts "Stress test succeeded!"
34
+ end
metadata CHANGED
@@ -1,59 +1,66 @@
1
- --- !ruby/object:Gem::Specification
1
+ --- !ruby/object:Gem::Specification
2
2
  name: pond
3
- version: !ruby/object:Gem::Version
4
- version: 0.1.0
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.2.0
5
5
  platform: ruby
6
- authors:
6
+ authors:
7
7
  - Chris Hanks
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
-
12
- date: 2014-02-15 00:00:00 Z
13
- dependencies:
14
- - !ruby/object:Gem::Dependency
11
+ date: 2016-02-05 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
15
14
  name: bundler
16
- prerelease: false
17
- requirement: &id001 !ruby/object:Gem::Requirement
18
- requirements:
19
- - - ~>
20
- - !ruby/object:Gem::Version
21
- version: "1.3"
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '1.3'
22
20
  type: :development
23
- version_requirements: *id001
24
- - !ruby/object:Gem::Dependency
25
- name: rspec
26
21
  prerelease: false
27
- requirement: &id002 !ruby/object:Gem::Requirement
28
- requirements:
29
- - - ~>
30
- - !ruby/object:Gem::Version
31
- version: "2.14"
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '1.3'
27
+ - !ruby/object:Gem::Dependency
28
+ name: rspec
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '2.14'
32
34
  type: :development
33
- version_requirements: *id002
34
- - !ruby/object:Gem::Dependency
35
- name: rake
36
35
  prerelease: false
37
- requirement: &id003 !ruby/object:Gem::Requirement
38
- requirements:
39
- - &id004
40
- - ">="
41
- - !ruby/object:Gem::Version
42
- version: "0"
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '2.14'
41
+ - !ruby/object:Gem::Dependency
42
+ name: rake
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
43
48
  type: :development
44
- version_requirements: *id003
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
45
55
  description: A simple, generic, thread-safe pool for connections or whatever else
46
- email:
56
+ email:
47
57
  - christopher.m.hanks@gmail.com
48
58
  executables: []
49
-
50
59
  extensions: []
51
-
52
60
  extra_rdoc_files: []
53
-
54
- files:
55
- - .gitignore
56
- - .rspec
61
+ files:
62
+ - ".gitignore"
63
+ - ".rspec"
57
64
  - CHANGELOG.md
58
65
  - Gemfile
59
66
  - LICENSE.txt
@@ -66,30 +73,32 @@ files:
66
73
  - spec/config_spec.rb
67
74
  - spec/spec_helper.rb
68
75
  - spec/wrapper_spec.rb
76
+ - tasks/stress.rake
69
77
  homepage: https://github.com/chanks/pond
70
- licenses:
78
+ licenses:
71
79
  - MIT
72
80
  metadata: {}
73
-
74
81
  post_install_message:
75
82
  rdoc_options: []
76
-
77
- require_paths:
83
+ require_paths:
78
84
  - lib
79
- required_ruby_version: !ruby/object:Gem::Requirement
80
- requirements:
81
- - *id004
82
- required_rubygems_version: !ruby/object:Gem::Requirement
83
- requirements:
84
- - *id004
85
+ required_ruby_version: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ required_rubygems_version: !ruby/object:Gem::Requirement
91
+ requirements:
92
+ - - ">="
93
+ - !ruby/object:Gem::Version
94
+ version: '0'
85
95
  requirements: []
86
-
87
96
  rubyforge_project:
88
- rubygems_version: 2.2.2
97
+ rubygems_version: 2.5.1
89
98
  signing_key:
90
99
  specification_version: 4
91
100
  summary: A simple, generic, thread-safe pool
92
- test_files:
101
+ test_files:
93
102
  - spec/checkout_spec.rb
94
103
  - spec/config_spec.rb
95
104
  - spec/spec_helper.rb