listener 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.gitignore +17 -0
- data/.rspec +2 -0
- data/Gemfile +13 -0
- data/LICENSE.txt +22 -0
- data/README.md +61 -0
- data/Rakefile +7 -0
- data/lib/listener.rb +77 -0
- data/lib/listener/version.rb +3 -0
- data/listener.gemspec +25 -0
- data/spec/listener_spec.rb +79 -0
- data/spec/spec_helper.rb +16 -0
- metadata +89 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 59f0eff865f641d0db604060e0dec2a1ec67816d
|
4
|
+
data.tar.gz: d9e23c653ae8798fe9ba924dd461326f0d442a52
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: d92fa9ebb25cc34d1ba293f4d4ddf07600afa1ddb75a15eb2da621796e572fb303f7a13641c1e847a3628b8994ab01b358f6211d3be3c4255454bcbcf4b45745
|
7
|
+
data.tar.gz: c6b5fe3356b5df4d02273ddb3697df074e3c0bbcd66395b4c685cfda6f30ee9316995e1c2f6c147eb19cb88064a6108cc33e783dc8d9ef2efec49b796f0d2b4d
|
data/.gitignore
ADDED
data/.rspec
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2014 Chris Hanks
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,61 @@
|
|
1
|
+
# Listener
|
2
|
+
|
3
|
+
Postgres' LISTEN/NOTIFY system is awesome and all, but PG connections are expensive. Instead of establishing a new one for every place in your application that needs to LISTEN for something, use Listener.
|
4
|
+
|
5
|
+
## Installation
|
6
|
+
|
7
|
+
Add this line to your application's Gemfile:
|
8
|
+
|
9
|
+
gem 'listener'
|
10
|
+
|
11
|
+
And then execute:
|
12
|
+
|
13
|
+
$ bundle
|
14
|
+
|
15
|
+
Or install it yourself as:
|
16
|
+
|
17
|
+
$ gem install listener
|
18
|
+
|
19
|
+
## Usage
|
20
|
+
|
21
|
+
``` ruby
|
22
|
+
pg = PG::Connection.open(...)
|
23
|
+
|
24
|
+
listener = Listener.new(pg)
|
25
|
+
|
26
|
+
listener.listen :channel1 do |channel, pid, payload|
|
27
|
+
# This block will run on a NOTIFY to channel1.
|
28
|
+
# The return value will be discarded, and errors it raises will be ignored.
|
29
|
+
puts [channel, pid, payload].inspect
|
30
|
+
end
|
31
|
+
|
32
|
+
listener.listen :channel1, :channel2 do |channel, pid, payload|
|
33
|
+
# This block will run on a NOTIFY to either channel1 OR channel2. When
|
34
|
+
# channel1, it will run after the one above, since it was declared second.
|
35
|
+
end
|
36
|
+
|
37
|
+
# From another connection: NOTIFY channel1, 'payload string'
|
38
|
+
|
39
|
+
#=> ["channel1", 29923, "payload string"]
|
40
|
+
|
41
|
+
# Stop listening to all channels, and make the connection safe for other code to use:
|
42
|
+
listener.stop
|
43
|
+
```
|
44
|
+
|
45
|
+
### Caveats
|
46
|
+
|
47
|
+
1. It's up to you to make sure that no other threads try to use the PG connection that you pass to Listener. If you steal a connection from your ORM, make sure you've checked it out from the connection pool properly so that nothing else will try to use it.
|
48
|
+
2. Do **NOT** pass untrusted input as channel names. It is not currently escaped, since Postgres and/or the PG gem apparently don't support the use of placeholders in a LISTEN statement, so I'm using plain old string interpolation, which is an SQL injection risk. If anyone knows a way around this, please let me know.
|
49
|
+
3. Each listener you instantiate has a dedicated thread that runs all the blocks for all the notifications the connection receives. If you're not comfortable with multi-threading, you probably shouldn't be using this gem. Also, try to avoid doing anything time-consuming like I/O in your blocks - if you keep the listener thread busy, notifications may pile up and not be serviced in a timely manner. If you need to do something heavy duty, pass it off to another thread.
|
50
|
+
|
51
|
+
You can also pass options to Listener.new:
|
52
|
+
* **:timeout** is the amount of time passed to wait_for_notify each time the listener calls it. The default is 0.1, or 100 milliseconds. Higher values mean that there may be more of a delay before new blocks can be added or before the listener stops, while lower values may decrease efficiency by spending more time going back-and-forth with Postgres.
|
53
|
+
* **:priority** is the priority of the Listener thread - Ruby uses this to decide which threads get CPU time when there's not enough to go around. The default value is 1, which makes the Listener thread somewhat more important than your other Ruby threads (Ruby thread priority defaults to zero).
|
54
|
+
|
55
|
+
## Contributing
|
56
|
+
|
57
|
+
1. Fork it
|
58
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
59
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
60
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
61
|
+
5. Create new Pull Request
|
data/Rakefile
ADDED
data/lib/listener.rb
ADDED
@@ -0,0 +1,77 @@
|
|
1
|
+
require 'listener/version'
|
2
|
+
|
3
|
+
class Listener
|
4
|
+
attr_reader :connection
|
5
|
+
|
6
|
+
def initialize(connection, options = {})
|
7
|
+
@timeout = options[:timeout] || 0.1
|
8
|
+
|
9
|
+
@blocks = {}
|
10
|
+
@block_mutex = Mutex.new
|
11
|
+
|
12
|
+
@connection = connection
|
13
|
+
@connection_mutex = Mutex.new
|
14
|
+
|
15
|
+
@thread = Thread.new { listen_loop }
|
16
|
+
@thread.priority = options[:priority] || 1
|
17
|
+
end
|
18
|
+
|
19
|
+
def listen(*channels, &block)
|
20
|
+
to_listen = []
|
21
|
+
|
22
|
+
@block_mutex.synchronize do
|
23
|
+
channels.each do |channel|
|
24
|
+
channel = channel.to_s
|
25
|
+
|
26
|
+
if blocks = @blocks[channel]
|
27
|
+
blocks << block
|
28
|
+
else
|
29
|
+
@blocks[channel] = [block]
|
30
|
+
to_listen << channel
|
31
|
+
end
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
if to_listen.any?
|
36
|
+
@connection_mutex.synchronize do
|
37
|
+
connection.async_exec to_listen.map{|c| "LISTEN #{c}"}.join('; ')
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
41
|
+
|
42
|
+
def stop
|
43
|
+
@stop = true
|
44
|
+
@thread.join
|
45
|
+
end
|
46
|
+
|
47
|
+
private
|
48
|
+
|
49
|
+
def listen_loop
|
50
|
+
loop do
|
51
|
+
if notification = retrieve_notification
|
52
|
+
blocks_for_channel(notification.first).each do |block|
|
53
|
+
block.call(*notification) rescue nil
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
if @stop
|
58
|
+
@connection_mutex.synchronize do
|
59
|
+
connection.async_exec "UNLISTEN *"
|
60
|
+
{} while connection.notifies
|
61
|
+
end
|
62
|
+
|
63
|
+
break
|
64
|
+
end
|
65
|
+
end
|
66
|
+
end
|
67
|
+
|
68
|
+
def blocks_for_channel(channel)
|
69
|
+
@block_mutex.synchronize { @blocks[channel].dup }
|
70
|
+
end
|
71
|
+
|
72
|
+
def retrieve_notification
|
73
|
+
@connection_mutex.synchronize do
|
74
|
+
connection.wait_for_notify(@timeout) { |*args| return args }
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|
data/listener.gemspec
ADDED
@@ -0,0 +1,25 @@
|
|
1
|
+
# coding: utf-8
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require 'listener/version'
|
5
|
+
|
6
|
+
Gem::Specification.new do |spec|
|
7
|
+
spec.name = "listener"
|
8
|
+
spec.version = Listener::VERSION
|
9
|
+
spec.authors = ["Chris Hanks"]
|
10
|
+
spec.email = ["christopher.m.hanks@gmail.com"]
|
11
|
+
spec.description = %q{Share the LISTEN functionality of a single PG connection}
|
12
|
+
spec.summary = %q{Utility to allow multiple codebases to share a single listening Postgres connection}
|
13
|
+
spec.homepage = 'https://github.com/chanks/listener'
|
14
|
+
spec.license = 'MIT'
|
15
|
+
|
16
|
+
spec.files = `git ls-files`.split($/)
|
17
|
+
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
18
|
+
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
19
|
+
spec.require_paths = ["lib"]
|
20
|
+
|
21
|
+
spec.add_development_dependency 'bundler', '~> 1.3'
|
22
|
+
spec.add_development_dependency 'rake'
|
23
|
+
|
24
|
+
spec.add_dependency 'pg'
|
25
|
+
end
|
@@ -0,0 +1,79 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Listener do
|
4
|
+
it "should allow a block to be assigned to run when a notification is received on a channel" do
|
5
|
+
q = Queue.new
|
6
|
+
l = Listener.new($conn1, timeout: 0.001)
|
7
|
+
l.listen('my_channel') { |*output| q.push output }
|
8
|
+
|
9
|
+
pid = $conn2.async_exec("select pg_backend_pid()").first['pg_backend_pid']
|
10
|
+
$conn2.async_exec("notify my_channel, 'my_payload'")
|
11
|
+
|
12
|
+
q.pop.should == ['my_channel', pid.to_i, 'my_payload']
|
13
|
+
l.stop
|
14
|
+
end
|
15
|
+
|
16
|
+
it "should allow the same block to be assigned for multiple channels" do
|
17
|
+
q = Queue.new
|
18
|
+
l = Listener.new($conn1, timeout: 0.001)
|
19
|
+
l.listen('channel1', 'channel2') { |channel, pid, payload| q.push channel }
|
20
|
+
|
21
|
+
%w(channel1 channel2 channel1).each { |c| $conn2.async_exec("notify #{c}") }
|
22
|
+
|
23
|
+
3.times.map{q.pop}.should == %w(channel1 channel2 channel1)
|
24
|
+
l.stop
|
25
|
+
end
|
26
|
+
|
27
|
+
it "should allow multiple blocks to be assigned for the same channel" do
|
28
|
+
q = Queue.new
|
29
|
+
l = Listener.new($conn1, timeout: 0.001)
|
30
|
+
l.listen('my_channel') { q.push 1 }
|
31
|
+
l.listen('my_channel') { q.push 2 }
|
32
|
+
|
33
|
+
2.times { $conn2.async_exec("notify my_channel") }
|
34
|
+
|
35
|
+
4.times.map{q.pop}.should == [1, 2, 1, 2]
|
36
|
+
l.stop
|
37
|
+
end
|
38
|
+
|
39
|
+
it "should accept symbols as channel names" do
|
40
|
+
q = Queue.new
|
41
|
+
l = Listener.new($conn1, timeout: 0.001)
|
42
|
+
l.listen(:my_channel) { q.push 1 }
|
43
|
+
l.listen(:my_channel) { q.push 2 }
|
44
|
+
|
45
|
+
2.times { $conn2.async_exec("notify my_channel") }
|
46
|
+
|
47
|
+
4.times.map{q.pop}.should == [1, 2, 1, 2]
|
48
|
+
l.stop
|
49
|
+
end
|
50
|
+
|
51
|
+
it "should recover from errors in blocks" do
|
52
|
+
q = Queue.new
|
53
|
+
l = Listener.new($conn1, timeout: 0.001)
|
54
|
+
l.listen('my_channel') { raise }
|
55
|
+
l.listen('my_channel') { q.push 1 }
|
56
|
+
|
57
|
+
2.times { $conn2.async_exec("notify my_channel") }
|
58
|
+
|
59
|
+
2.times.map{q.pop}.should == [1, 1]
|
60
|
+
l.stop
|
61
|
+
end
|
62
|
+
|
63
|
+
it "when told to stop should unlisten to all channels and drain existing notifications before returning" do
|
64
|
+
q1, q2 = Queue.new, Queue.new
|
65
|
+
l = Listener.new($conn1, timeout: 0.001)
|
66
|
+
l.listen('my_channel') { q1.push nil; q2.pop }
|
67
|
+
|
68
|
+
2.times { $conn2.async_exec("notify my_channel") }
|
69
|
+
q1.pop
|
70
|
+
|
71
|
+
t = Thread.new { l.stop }
|
72
|
+
$conn2.async_exec("notify my_channel")
|
73
|
+
q2.push nil
|
74
|
+
t.join
|
75
|
+
|
76
|
+
$conn1.async_exec("select pg_listening_channels()").to_a.length.should == 0
|
77
|
+
$conn1.wait_for_notify(0.001).should be nil
|
78
|
+
end
|
79
|
+
end
|
data/spec/spec_helper.rb
ADDED
@@ -0,0 +1,16 @@
|
|
1
|
+
require 'uri'
|
2
|
+
require 'pg'
|
3
|
+
require 'listener'
|
4
|
+
require 'pry'
|
5
|
+
require 'thread'
|
6
|
+
|
7
|
+
url = ENV['DATABASE_URL'] || 'postgres://postgres:@localhost/listener-test'
|
8
|
+
|
9
|
+
$conn1, $conn2 = 2.times.map do
|
10
|
+
uri = URI.parse(url)
|
11
|
+
PG::Connection.open :host => uri.host,
|
12
|
+
:user => uri.user,
|
13
|
+
:password => uri.password,
|
14
|
+
:port => uri.port || 5432,
|
15
|
+
:dbname => uri.path[1..-1]
|
16
|
+
end
|
metadata
ADDED
@@ -0,0 +1,89 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: listener
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Chris Hanks
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
|
12
|
+
date: 2014-05-11 00:00:00 Z
|
13
|
+
dependencies:
|
14
|
+
- !ruby/object:Gem::Dependency
|
15
|
+
name: bundler
|
16
|
+
prerelease: false
|
17
|
+
requirement: &id001 !ruby/object:Gem::Requirement
|
18
|
+
requirements:
|
19
|
+
- - ~>
|
20
|
+
- !ruby/object:Gem::Version
|
21
|
+
version: "1.3"
|
22
|
+
type: :development
|
23
|
+
version_requirements: *id001
|
24
|
+
- !ruby/object:Gem::Dependency
|
25
|
+
name: rake
|
26
|
+
prerelease: false
|
27
|
+
requirement: &id002 !ruby/object:Gem::Requirement
|
28
|
+
requirements:
|
29
|
+
- &id003
|
30
|
+
- ">="
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: "0"
|
33
|
+
type: :development
|
34
|
+
version_requirements: *id002
|
35
|
+
- !ruby/object:Gem::Dependency
|
36
|
+
name: pg
|
37
|
+
prerelease: false
|
38
|
+
requirement: &id004 !ruby/object:Gem::Requirement
|
39
|
+
requirements:
|
40
|
+
- *id003
|
41
|
+
type: :runtime
|
42
|
+
version_requirements: *id004
|
43
|
+
description: Share the LISTEN functionality of a single PG connection
|
44
|
+
email:
|
45
|
+
- christopher.m.hanks@gmail.com
|
46
|
+
executables: []
|
47
|
+
|
48
|
+
extensions: []
|
49
|
+
|
50
|
+
extra_rdoc_files: []
|
51
|
+
|
52
|
+
files:
|
53
|
+
- .gitignore
|
54
|
+
- .rspec
|
55
|
+
- Gemfile
|
56
|
+
- LICENSE.txt
|
57
|
+
- README.md
|
58
|
+
- Rakefile
|
59
|
+
- lib/listener.rb
|
60
|
+
- lib/listener/version.rb
|
61
|
+
- listener.gemspec
|
62
|
+
- spec/listener_spec.rb
|
63
|
+
- spec/spec_helper.rb
|
64
|
+
homepage: https://github.com/chanks/listener
|
65
|
+
licenses:
|
66
|
+
- MIT
|
67
|
+
metadata: {}
|
68
|
+
|
69
|
+
post_install_message:
|
70
|
+
rdoc_options: []
|
71
|
+
|
72
|
+
require_paths:
|
73
|
+
- lib
|
74
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
75
|
+
requirements:
|
76
|
+
- *id003
|
77
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
78
|
+
requirements:
|
79
|
+
- *id003
|
80
|
+
requirements: []
|
81
|
+
|
82
|
+
rubyforge_project:
|
83
|
+
rubygems_version: 2.2.2
|
84
|
+
signing_key:
|
85
|
+
specification_version: 4
|
86
|
+
summary: Utility to allow multiple codebases to share a single listening Postgres connection
|
87
|
+
test_files:
|
88
|
+
- spec/listener_spec.rb
|
89
|
+
- spec/spec_helper.rb
|