klomp 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.gitignore +1 -0
- data/Gemfile +9 -0
- data/Gemfile.lock +20 -0
- data/Procfile +2 -0
- data/README.md +87 -0
- data/Rakefile +7 -0
- data/klomp.gemspec +20 -0
- data/lib/klomp.rb +9 -0
- data/lib/klomp/client.rb +88 -0
- data/tasks/test_failover.rake +35 -0
- data/test/test_client.rb +91 -0
- metadata +90 -0
data/.gitignore
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
pkg/
|
data/Gemfile
ADDED
data/Gemfile.lock
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
GEM
|
2
|
+
remote: http://rubygems.org/
|
3
|
+
specs:
|
4
|
+
ZenTest (4.8.0)
|
5
|
+
foreman (0.46.0)
|
6
|
+
thor (>= 0.13.6)
|
7
|
+
json (1.7.1)
|
8
|
+
onstomp (1.0.6)
|
9
|
+
rake (0.9.2.2)
|
10
|
+
thor (0.15.2)
|
11
|
+
|
12
|
+
PLATFORMS
|
13
|
+
ruby
|
14
|
+
|
15
|
+
DEPENDENCIES
|
16
|
+
ZenTest
|
17
|
+
foreman
|
18
|
+
json
|
19
|
+
onstomp
|
20
|
+
rake
|
data/Procfile
ADDED
data/README.md
ADDED
@@ -0,0 +1,87 @@
|
|
1
|
+
# Klomp
|
2
|
+
|
3
|
+
Klomp is a simple wrapper around the [OnStomp](https://github.com/meadvillerb/onstomp/)
|
4
|
+
library with some additional HA and usability features:
|
5
|
+
|
6
|
+
* When initialized with multiple broker URIs, Klomp will publish messages to
|
7
|
+
one broker at a time, but will consume from all brokers simultaneously. This is
|
8
|
+
a slight improvement over the regular [OnStomp::Failover::Client](http://mdvlrb.com/onstomp/OnStomp/Failover/Client.html)
|
9
|
+
which handles all publishing and subscribing through a single "active" broker.
|
10
|
+
This traditional one-broker-at-a-time technique can lead to a split-brain
|
11
|
+
scenario in which messages are only received by a subset of your STOMP clients.
|
12
|
+
By consuming from all brokers simultaneously, Klomp ensures that no message is
|
13
|
+
left behind.
|
14
|
+
|
15
|
+
* Where applicable, message bodies are automatically translated between native
|
16
|
+
Ruby and JSON objects.
|
17
|
+
|
18
|
+
* If a reply-to header is found in a message, a response is automatically
|
19
|
+
sent to the reply-to destination.
|
20
|
+
|
21
|
+
## Installation
|
22
|
+
|
23
|
+
gem install klomp
|
24
|
+
|
25
|
+
## Example usage
|
26
|
+
|
27
|
+
The goal is that you should be able to use most (if not all) of the standard
|
28
|
+
OnStomp API (see [OnStomp's UserNarrative](https://github.com/meadvillerb/onstomp/blob/master/extra_doc/UserNarrative.md))
|
29
|
+
via a `Klomp::Client`:
|
30
|
+
|
31
|
+
client = Klomp::Client.new([ ... ])
|
32
|
+
|
33
|
+
However, there will be some differences in the API due to how `Klomp::Client`
|
34
|
+
manages connections. For example, while the `connected?` method normally
|
35
|
+
returns a boolean value, Klomp's `connected?` will return an array of booleans
|
36
|
+
(i.e. one result for each broker).
|
37
|
+
|
38
|
+
## Developers
|
39
|
+
|
40
|
+
Set up the environment using `bundle install`. Note that the tests currently
|
41
|
+
assume a specific Apollo configuration which can be created on OSX using the
|
42
|
+
following commands:
|
43
|
+
|
44
|
+
brew install apollo
|
45
|
+
apollo create /usr/local/var/apollo-primary
|
46
|
+
apollo create /usr/local/var/apollo-secondary
|
47
|
+
sed -i -e 's/616/626/' /usr/local/var/apollo-secondary/etc/apollo.xml
|
48
|
+
|
49
|
+
Once Apollo is configured, the brokers can be started via `foreman start`. Now
|
50
|
+
you can run the test suite via `rake test` or `autotest`.
|
51
|
+
|
52
|
+
In addition to the regular test suite, there is a rake task called
|
53
|
+
"test_failover" that will start an infinite publish/subscribe loop. Once this
|
54
|
+
task is running, you can randomly kill Apollo brokers to test STOMP client
|
55
|
+
failover.
|
56
|
+
|
57
|
+
## Change Log
|
58
|
+
|
59
|
+
### 0.0.1
|
60
|
+
|
61
|
+
* Initial release
|
62
|
+
|
63
|
+
## License
|
64
|
+
|
65
|
+
Copyright (C) 2012 LivingSocial
|
66
|
+
|
67
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of
|
68
|
+
this software and associated documentation files (the "Software"), to deal in
|
69
|
+
the Software without restriction, including without limitation the rights to
|
70
|
+
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
|
71
|
+
of the Software, and to permit persons to whom the Software is furnished to do
|
72
|
+
so, subject to the following conditions:
|
73
|
+
|
74
|
+
The above copyright notice and this permission notice shall be included in all
|
75
|
+
copies or substantial portions of the Software.
|
76
|
+
|
77
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
78
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
79
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
80
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
81
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
82
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
83
|
+
SOFTWARE.
|
84
|
+
|
85
|
+
## Credits
|
86
|
+
|
87
|
+
* [Michael Paul Thomas Conigliaro](http://conigliaro.org): Original author
|
data/Rakefile
ADDED
data/klomp.gemspec
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
$:.push File.expand_path(File.join(File.dirname(__FILE__), "lib"))
|
2
|
+
require 'klomp'
|
3
|
+
|
4
|
+
Gem::Specification.new do |gem|
|
5
|
+
gem.authors = ["LivingSocial"]
|
6
|
+
gem.email = ["dev.happiness@livingsocial.com"]
|
7
|
+
gem.description = "A simple wrapper around the OnStomp library with additional features"
|
8
|
+
gem.summary = "A simple wrapper around the OnStomp library with additional features"
|
9
|
+
gem.homepage = "https://github.com/livingsocial/klomp"
|
10
|
+
|
11
|
+
gem.files = `git ls-files`.split($\)
|
12
|
+
gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
|
13
|
+
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
14
|
+
gem.name = "klomp"
|
15
|
+
gem.require_paths = ["lib"]
|
16
|
+
gem.version = Klomp::VERSION
|
17
|
+
|
18
|
+
gem.add_dependency("onstomp")
|
19
|
+
gem.add_dependency("json")
|
20
|
+
end
|
data/lib/klomp.rb
ADDED
data/lib/klomp/client.rb
ADDED
@@ -0,0 +1,88 @@
|
|
1
|
+
module Klomp
|
2
|
+
|
3
|
+
class Client
|
4
|
+
attr_reader :options, :read_conn, :write_conn
|
5
|
+
|
6
|
+
def initialize(uri, options={})
|
7
|
+
@options ||= {
|
8
|
+
:translate_json => true,
|
9
|
+
:auto_reply_to => true
|
10
|
+
}
|
11
|
+
|
12
|
+
ofc_options = options.inject({
|
13
|
+
:retry_attempts => -1,
|
14
|
+
:retry_delay => 1
|
15
|
+
}) { |memo,(k,v)| memo.merge({k => v}) if memo.has_key?(k) }
|
16
|
+
|
17
|
+
if uri.is_a?(Array)
|
18
|
+
@write_conn = OnStomp::Failover::Client.new(uri, ofc_options)
|
19
|
+
@read_conn = uri.inject([]) { |memo,obj| memo + [OnStomp::Failover::Client.new([obj], ofc_options)] }
|
20
|
+
else
|
21
|
+
@write_conn = OnStomp::Failover::Client.new([uri], ofc_options)
|
22
|
+
@read_conn = [@write_conn]
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
def send(*args, &block)
|
27
|
+
if @options[:translate_json] && [Array, Hash].any? { |type| args[1].kind_of?(type) }
|
28
|
+
args[1] = args[1].to_json
|
29
|
+
args[2] = {} if args[2].nil?
|
30
|
+
args[2][:'content-type'] = 'application/json'
|
31
|
+
else
|
32
|
+
args[1] = args[1].to_s
|
33
|
+
end
|
34
|
+
@write_conn.send(*args, &block)
|
35
|
+
end
|
36
|
+
|
37
|
+
def subscribe(*args, &block)
|
38
|
+
frames = []
|
39
|
+
@read_conn.each do |c|
|
40
|
+
frames << c.subscribe(*args) do |msg|
|
41
|
+
if @options[:translate_json]
|
42
|
+
msg.body = begin
|
43
|
+
JSON.parse(msg.body)
|
44
|
+
rescue JSON::ParserError
|
45
|
+
msg.body
|
46
|
+
end
|
47
|
+
end
|
48
|
+
reply_args = yield msg
|
49
|
+
if @options[:auto_reply_to] && !msg.headers[:'reply-to'].nil?
|
50
|
+
if reply_args.is_a?(Array)
|
51
|
+
send(msg.headers[:'reply-to'], *reply_args)
|
52
|
+
else
|
53
|
+
send(msg.headers[:'reply-to'], reply_args)
|
54
|
+
end
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
58
|
+
frames
|
59
|
+
end
|
60
|
+
|
61
|
+
def method_missing(method, *args, &block)
|
62
|
+
write_only_methods = [
|
63
|
+
:abort,
|
64
|
+
:begin,
|
65
|
+
:commit,
|
66
|
+
]
|
67
|
+
read_only_methods = [
|
68
|
+
:ack,
|
69
|
+
:nack,
|
70
|
+
:unsubscribe
|
71
|
+
]
|
72
|
+
returns = {
|
73
|
+
:connect => self
|
74
|
+
}
|
75
|
+
|
76
|
+
result = if write_only_methods.include?(method)
|
77
|
+
@write_conn.send(method, *args, &block)
|
78
|
+
elsif read_only_methods.include?(method)
|
79
|
+
@read_conn.map { |c| c.__send__(method, *args, &block) }
|
80
|
+
else
|
81
|
+
([@write_conn] + @read_conn).uniq.map { |c| c.__send__(method, *args) }
|
82
|
+
end
|
83
|
+
returns.include?(method) ? returns[method] : result
|
84
|
+
end
|
85
|
+
|
86
|
+
end
|
87
|
+
|
88
|
+
end
|
@@ -0,0 +1,35 @@
|
|
1
|
+
desc "Start an infinite publish/subscribe loop to test STOMP client failover"
|
2
|
+
task :test_failover do
|
3
|
+
require 'klomp'
|
4
|
+
|
5
|
+
# Set the delay between publish events. If this is too small, the consumer
|
6
|
+
# will never be able to catch up to the producer, giving the false impression
|
7
|
+
# of lost messages.
|
8
|
+
publish_interval = 0.01
|
9
|
+
|
10
|
+
client = Klomp::Client.new([
|
11
|
+
'stomp://admin:password@localhost:61613',
|
12
|
+
'stomp://admin:password@127.0.0.1:62613'
|
13
|
+
]).connect
|
14
|
+
|
15
|
+
last_i = nil
|
16
|
+
client.subscribe("/queue/test") do |msg|
|
17
|
+
print "-"
|
18
|
+
last_i = msg.body.to_i
|
19
|
+
end
|
20
|
+
|
21
|
+
begin
|
22
|
+
i = 0
|
23
|
+
loop do
|
24
|
+
i += 1
|
25
|
+
client.send("/queue/test", i.to_s) do |r|
|
26
|
+
print "+"
|
27
|
+
end
|
28
|
+
sleep publish_interval
|
29
|
+
end
|
30
|
+
rescue SignalException
|
31
|
+
client.disconnect
|
32
|
+
puts
|
33
|
+
puts "Sent #{i}; Received #{last_i}; Lost #{i - last_i}"
|
34
|
+
end
|
35
|
+
end
|
data/test/test_client.rb
ADDED
@@ -0,0 +1,91 @@
|
|
1
|
+
require 'minitest/autorun'
|
2
|
+
require 'minitest/pride'
|
3
|
+
|
4
|
+
require 'klomp'
|
5
|
+
|
6
|
+
describe Klomp::Client do
|
7
|
+
|
8
|
+
before do
|
9
|
+
@uris = [
|
10
|
+
'stomp://admin:password@localhost:61613',
|
11
|
+
'stomp://admin:password@127.0.0.1:62613'
|
12
|
+
]
|
13
|
+
@destination = '/queue/test_component.test_event'
|
14
|
+
end
|
15
|
+
|
16
|
+
it 'accepts a single uri and establishes separate failover connections for writes and reads' do
|
17
|
+
client = Klomp::Client.new(@uris.first).connect
|
18
|
+
|
19
|
+
assert_equal [client.write_conn], client.read_conn
|
20
|
+
assert client.write_conn.connected?
|
21
|
+
|
22
|
+
client.disconnect
|
23
|
+
end
|
24
|
+
|
25
|
+
it 'accepts an array of uris and establishes separate failover connections for writes and reads' do
|
26
|
+
client = Klomp::Client.new(@uris).connect
|
27
|
+
|
28
|
+
assert client.write_conn.connected?
|
29
|
+
refute_empty client.read_conn
|
30
|
+
client.read_conn.each do |obj|
|
31
|
+
assert obj.connected?
|
32
|
+
end
|
33
|
+
|
34
|
+
client.disconnect
|
35
|
+
end
|
36
|
+
|
37
|
+
it 'disconnnects' do
|
38
|
+
client = Klomp::Client.new(@uris.first).connect
|
39
|
+
assert client.write_conn.connected?
|
40
|
+
client.disconnect
|
41
|
+
refute client.write_conn.connected?
|
42
|
+
end
|
43
|
+
|
44
|
+
it 'sends heartbeat' do
|
45
|
+
client = Klomp::Client.new(@uris).connect.beat
|
46
|
+
end
|
47
|
+
|
48
|
+
it 'sends requests and gets responses' do
|
49
|
+
client = Klomp::Client.new(@uris).connect
|
50
|
+
body = { 'body' => rand(36**128).to_s(36) }
|
51
|
+
|
52
|
+
client.send(@destination, body, :ack=>'client')
|
53
|
+
|
54
|
+
got_message = false
|
55
|
+
client.subscribe(@destination) do |msg|
|
56
|
+
got_message = true if msg.body == body
|
57
|
+
client.ack(msg)
|
58
|
+
end
|
59
|
+
sleep 1
|
60
|
+
assert got_message
|
61
|
+
|
62
|
+
client.disconnect
|
63
|
+
end
|
64
|
+
|
65
|
+
it 'automatically publishes responses to the reply-to destination' do
|
66
|
+
client = Klomp::Client.new(@uris).connect
|
67
|
+
reply_to_body = { 'reply_to_body' => rand(36**128).to_s(36) }
|
68
|
+
|
69
|
+
client.send(@destination, nil, { 'reply-to' => @destination })
|
70
|
+
|
71
|
+
got_message = false
|
72
|
+
client.subscribe(@destination) do |msg|
|
73
|
+
got_message = true if msg.body == reply_to_body
|
74
|
+
reply_to_body
|
75
|
+
end
|
76
|
+
sleep 1
|
77
|
+
assert got_message
|
78
|
+
|
79
|
+
client.disconnect
|
80
|
+
end
|
81
|
+
|
82
|
+
it 'unsubscribes' do
|
83
|
+
client = Klomp::Client.new(@uris).connect
|
84
|
+
|
85
|
+
subscribe_frames = client.subscribe(@destination) { |msg| }
|
86
|
+
client.unsubscribe(subscribe_frames)
|
87
|
+
|
88
|
+
client.disconnect
|
89
|
+
end
|
90
|
+
|
91
|
+
end
|
metadata
ADDED
@@ -0,0 +1,90 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: klomp
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.0.1
|
5
|
+
prerelease:
|
6
|
+
platform: ruby
|
7
|
+
authors:
|
8
|
+
- LivingSocial
|
9
|
+
autorequire:
|
10
|
+
bindir: bin
|
11
|
+
cert_chain: []
|
12
|
+
date: 2012-06-11 00:00:00.000000000 Z
|
13
|
+
dependencies:
|
14
|
+
- !ruby/object:Gem::Dependency
|
15
|
+
name: onstomp
|
16
|
+
requirement: !ruby/object:Gem::Requirement
|
17
|
+
none: false
|
18
|
+
requirements:
|
19
|
+
- - ! '>='
|
20
|
+
- !ruby/object:Gem::Version
|
21
|
+
version: '0'
|
22
|
+
type: :runtime
|
23
|
+
prerelease: false
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
25
|
+
none: false
|
26
|
+
requirements:
|
27
|
+
- - ! '>='
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: '0'
|
30
|
+
- !ruby/object:Gem::Dependency
|
31
|
+
name: json
|
32
|
+
requirement: !ruby/object:Gem::Requirement
|
33
|
+
none: false
|
34
|
+
requirements:
|
35
|
+
- - ! '>='
|
36
|
+
- !ruby/object:Gem::Version
|
37
|
+
version: '0'
|
38
|
+
type: :runtime
|
39
|
+
prerelease: false
|
40
|
+
version_requirements: !ruby/object:Gem::Requirement
|
41
|
+
none: false
|
42
|
+
requirements:
|
43
|
+
- - ! '>='
|
44
|
+
- !ruby/object:Gem::Version
|
45
|
+
version: '0'
|
46
|
+
description: A simple wrapper around the OnStomp library with additional features
|
47
|
+
email:
|
48
|
+
- dev.happiness@livingsocial.com
|
49
|
+
executables: []
|
50
|
+
extensions: []
|
51
|
+
extra_rdoc_files: []
|
52
|
+
files:
|
53
|
+
- .gitignore
|
54
|
+
- Gemfile
|
55
|
+
- Gemfile.lock
|
56
|
+
- Procfile
|
57
|
+
- README.md
|
58
|
+
- Rakefile
|
59
|
+
- klomp.gemspec
|
60
|
+
- lib/klomp.rb
|
61
|
+
- lib/klomp/client.rb
|
62
|
+
- tasks/test_failover.rake
|
63
|
+
- test/test_client.rb
|
64
|
+
homepage: https://github.com/livingsocial/klomp
|
65
|
+
licenses: []
|
66
|
+
post_install_message:
|
67
|
+
rdoc_options: []
|
68
|
+
require_paths:
|
69
|
+
- lib
|
70
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
71
|
+
none: false
|
72
|
+
requirements:
|
73
|
+
- - ! '>='
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '0'
|
76
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
77
|
+
none: false
|
78
|
+
requirements:
|
79
|
+
- - ! '>='
|
80
|
+
- !ruby/object:Gem::Version
|
81
|
+
version: '0'
|
82
|
+
requirements: []
|
83
|
+
rubyforge_project:
|
84
|
+
rubygems_version: 1.8.23
|
85
|
+
signing_key:
|
86
|
+
specification_version: 3
|
87
|
+
summary: A simple wrapper around the OnStomp library with additional features
|
88
|
+
test_files:
|
89
|
+
- test/test_client.rb
|
90
|
+
has_rdoc:
|