multiple_man 0.2.6 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +97 -2
- data/lib/multiple_man/attribute_extractor.rb +3 -6
- data/lib/multiple_man/configuration.rb +6 -1
- data/lib/multiple_man/connection.rb +6 -4
- data/lib/multiple_man/listener.rb +1 -0
- data/lib/multiple_man/model_publisher.rb +7 -1
- data/lib/multiple_man/routing_key.rb +2 -6
- data/lib/multiple_man/version.rb +1 -1
- data/lib/multiple_man.rb +11 -3
- data/multiple_man.gemspec +5 -4
- data/spec/connection_spec.rb +1 -1
- data/spec/model_publisher_spec.rb +9 -7
- data/spec/routing_key_spec.rb +2 -2
- metadata +28 -14
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9ca5356191de47f8a4e08e52e9a568eeebdf9048
|
4
|
+
data.tar.gz: 79a7e0d2c6079170cc2a81541980c603f595a366
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 4064bbb0ab486a17f5315302a5edbbc79e191724bc65ad74298f4b7a1edbcaac992f5f7862979eaebf8f7a5b711e3b1da5173fc6ddcd285bcf4e5cdf86bc62e1
|
7
|
+
data.tar.gz: 56fcab0f9ab729ff37240d419bb030aad4e8275a3626cdee62e494d506b542bb6c9276bbb014077c4a712789f0ec35d8006d4f1fc24f04ffc3bfeb8f740cf432
|
data/README.md
CHANGED
@@ -1,6 +1,19 @@
|
|
1
1
|
# MultipleMan
|
2
2
|
|
3
|
-
|
3
|
+
[](https://codeclimate.com/github/influitive/multiple_man)
|
4
|
+
|
5
|
+
[](https://circleci.com/gh/influitive/multiple_man)
|
6
|
+
|
7
|
+
MultipleMan synchronizes your ActiveRecord models between Rails
|
8
|
+
apps, using RabbitMQ to send messages between your applications.
|
9
|
+
It's heavily inspired by Promiscuous, but differs in a few ways:
|
10
|
+
|
11
|
+
- MultipleMan makes a hard assumption that you're using
|
12
|
+
ActiveRecord for your models. This simplifies how models
|
13
|
+
are sychronized, which offers a few benefits like:
|
14
|
+
- Transactions are fully supported in MultipleMan. Records
|
15
|
+
which aren't committed fully to the database won't be sent
|
16
|
+
to your message queue.
|
4
17
|
|
5
18
|
## Installation
|
6
19
|
|
@@ -18,7 +31,65 @@ Or install it yourself as:
|
|
18
31
|
|
19
32
|
## Usage
|
20
33
|
|
21
|
-
|
34
|
+
### Configuration
|
35
|
+
|
36
|
+
MultipleMan can be configured (ideally inside an initializer) by
|
37
|
+
calling MultipleMan.configure like so:
|
38
|
+
|
39
|
+
MultipleMan.configure do |config|
|
40
|
+
# A connection string to your local server. Defaults to localhost.
|
41
|
+
config.connection = "amqp://example.com"
|
42
|
+
|
43
|
+
# The topic name to push to. If you have multiple
|
44
|
+
# multiple man apps, this should be unique per application. Publishers
|
45
|
+
# and subscribers should always use the same topic.
|
46
|
+
config.topic_name = "multiple_man"
|
47
|
+
|
48
|
+
# The application name (used for subscribers) - defaults to your
|
49
|
+
# Rails application name if you're using rails
|
50
|
+
config.app_name = "MyApp"
|
51
|
+
|
52
|
+
# Where you want to log errors to. Should be an instance of Logger
|
53
|
+
# Defaults to the Rails logger (for Rails) or STDOUT otherwise.
|
54
|
+
config.logger = Logger.new(STDOUT)
|
55
|
+
end
|
56
|
+
|
57
|
+
### Publishing models
|
58
|
+
|
59
|
+
To publish a model, include the following in your model definition:
|
60
|
+
|
61
|
+
class Widget < ActiveRecord::Base
|
62
|
+
include MultipleMan::Publisher
|
63
|
+
publish fields: [:id, :name, :type], identifier: :code
|
64
|
+
end
|
65
|
+
|
66
|
+
You can use the following options when publishing:
|
67
|
+
|
68
|
+
- `fields` - An array of all the fields you want to send to the message queue. These
|
69
|
+
can be either ActiveRecord attributes or methods on your model.
|
70
|
+
- `identifier` - Either a symbol or a proc used by MultipleMan to identify your model.
|
71
|
+
`id` is used by default and is generally fine, unless you're working in a multi-tenant
|
72
|
+
environment where ids may be shared between two different models of the same class.
|
73
|
+
|
74
|
+
### Subscribing to models
|
75
|
+
|
76
|
+
You can subscribe to a model as follows (in a seperate consumer app):
|
77
|
+
|
78
|
+
class Widget < ActiveRecord::Base
|
79
|
+
include MultipleMan::Subscriber
|
80
|
+
end
|
81
|
+
|
82
|
+
Currently, there's a few assumptions made of subscriber models:
|
83
|
+
|
84
|
+
- They need to have all of the fields that are published.
|
85
|
+
- They need an additional field, called `multiple_man_identifier`, that is the
|
86
|
+
same type as what's passed from the publisher side (just an integer if you're using
|
87
|
+
id)
|
88
|
+
|
89
|
+
Once you've set up your subscribers, you'll need to run a background worker to manage
|
90
|
+
the subscription process. Just run the following:
|
91
|
+
|
92
|
+
rake multiple_man:worker
|
22
93
|
|
23
94
|
## Contributing
|
24
95
|
|
@@ -27,3 +98,27 @@ TODO: Write usage instructions here
|
|
27
98
|
3. Commit your changes (`git commit -am 'Add some feature'`)
|
28
99
|
4. Push to the branch (`git push origin my-new-feature`)
|
29
100
|
5. Create new Pull Request
|
101
|
+
|
102
|
+
## License
|
103
|
+
|
104
|
+
The MIT License (MIT)
|
105
|
+
|
106
|
+
Copyright (c) 2014 Influitive Corporation
|
107
|
+
|
108
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
109
|
+
of this software and associated documentation files (the "Software"), to deal
|
110
|
+
in the Software without restriction, including without limitation the rights
|
111
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
112
|
+
copies of the Software, and to permit persons to whom the Software is
|
113
|
+
furnished to do so, subject to the following conditions:
|
114
|
+
|
115
|
+
The above copyright notice and this permission notice shall be included in
|
116
|
+
all copies or substantial portions of the Software.
|
117
|
+
|
118
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
119
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
120
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
121
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
122
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
123
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
124
|
+
THE SOFTWARE.
|
@@ -8,7 +8,7 @@ module MultipleMan
|
|
8
8
|
|
9
9
|
self.record = record
|
10
10
|
self.fields = fields
|
11
|
-
self.identifier = identifier
|
11
|
+
self.identifier = identifier || :id
|
12
12
|
end
|
13
13
|
|
14
14
|
def data
|
@@ -21,13 +21,10 @@ module MultipleMan
|
|
21
21
|
end
|
22
22
|
|
23
23
|
def identifier_for_record
|
24
|
-
|
25
|
-
if identifier.class == Symbol
|
26
|
-
record.send(identifier)
|
27
|
-
elsif identifier.class == Proc
|
24
|
+
if identifier.class == Proc
|
28
25
|
identifier.call(record)
|
29
26
|
else
|
30
|
-
record.
|
27
|
+
record.send(identifier)
|
31
28
|
end
|
32
29
|
end
|
33
30
|
|
@@ -5,13 +5,18 @@ module MultipleMan
|
|
5
5
|
self.topic_name = "multiple_man"
|
6
6
|
self.app_name = Rails.application.class.parent.to_s if defined?(Rails)
|
7
7
|
self.enabled = true
|
8
|
+
self.channel_pool_size = 5
|
8
9
|
end
|
9
10
|
|
10
11
|
def logger
|
11
12
|
@logger ||= defined?(Rails) ? Rails.logger : Logger.new(STDOUT)
|
12
13
|
end
|
13
14
|
|
14
|
-
|
15
|
+
def on_error(&block)
|
16
|
+
@error_handler = block
|
17
|
+
end
|
18
|
+
|
19
|
+
attr_accessor :topic_name, :app_name, :connection, :enabled, :channel_pool_size, :error_handler
|
15
20
|
attr_writer :logger
|
16
21
|
end
|
17
22
|
|
@@ -1,4 +1,5 @@
|
|
1
1
|
require 'bunny'
|
2
|
+
require 'connection_pool'
|
2
3
|
|
3
4
|
module MultipleMan
|
4
5
|
class Connection
|
@@ -11,13 +12,14 @@ module MultipleMan
|
|
11
12
|
end
|
12
13
|
end
|
13
14
|
|
14
|
-
def self.
|
15
|
-
|
15
|
+
def self.channel_pool
|
16
|
+
@channel_pool ||= ConnectionPool.new(size: 25, timeout: 5) { connection.create_channel }
|
16
17
|
end
|
17
18
|
|
18
19
|
def self.connect
|
19
|
-
|
20
|
-
|
20
|
+
channel_pool.with do |channel|
|
21
|
+
yield new(channel) if block_given?
|
22
|
+
end
|
21
23
|
end
|
22
24
|
|
23
25
|
def initialize(channel)
|
@@ -36,6 +36,7 @@ module MultipleMan
|
|
36
36
|
subscription.send(operation, JSON.parse(payload).with_indifferent_access)
|
37
37
|
rescue Exception => ex
|
38
38
|
MultipleMan.logger.error " Error - #{ex.message}"
|
39
|
+
MultipleMan.error(ex)
|
39
40
|
else
|
40
41
|
MultipleMan.logger.info " Successfully processed!"
|
41
42
|
end
|
@@ -21,9 +21,15 @@ module MultipleMan
|
|
21
21
|
|
22
22
|
def push_record(connection, record)
|
23
23
|
data = record_data(record)
|
24
|
-
routing_key = RoutingKey.new(record, operation).to_s
|
24
|
+
routing_key = RoutingKey.new(record_type(record), operation).to_s
|
25
25
|
MultipleMan.logger.info(" Record Data: #{data} | Routing Key: #{routing_key}")
|
26
26
|
connection.topic.publish(data, routing_key: routing_key)
|
27
|
+
rescue Exception => ex
|
28
|
+
MultipleMan.error(ex)
|
29
|
+
end
|
30
|
+
|
31
|
+
def record_type(record)
|
32
|
+
options[:as] || record.class.name
|
27
33
|
end
|
28
34
|
|
29
35
|
def record_data(record)
|
@@ -8,21 +8,17 @@ module MultipleMan
|
|
8
8
|
end
|
9
9
|
|
10
10
|
def to_s
|
11
|
-
"#{topic_name}.#{klass
|
11
|
+
"#{topic_name}.#{klass}.#{operation}"
|
12
12
|
end
|
13
13
|
|
14
14
|
attr_reader :operation
|
15
|
-
|
15
|
+
attr_accessor :klass
|
16
16
|
|
17
17
|
def operation=(value)
|
18
18
|
raise "Operation #{value} is not recognized" unless ALLOWED_OPERATIONS.include?(value)
|
19
19
|
@operation = value
|
20
20
|
end
|
21
21
|
|
22
|
-
def klass=(value)
|
23
|
-
@klass = (value.is_a?(Class) ? value : value.class)
|
24
|
-
end
|
25
|
-
|
26
22
|
private
|
27
23
|
def topic_name
|
28
24
|
MultipleMan.configuration.topic_name
|
data/lib/multiple_man/version.rb
CHANGED
data/lib/multiple_man.rb
CHANGED
@@ -14,14 +14,22 @@ module MultipleMan
|
|
14
14
|
require 'multiple_man/listener'
|
15
15
|
|
16
16
|
def self.logger
|
17
|
-
|
17
|
+
configuration.logger
|
18
18
|
end
|
19
19
|
|
20
20
|
def self.disable!
|
21
|
-
|
21
|
+
configuration.enabled = false
|
22
22
|
end
|
23
23
|
|
24
24
|
def self.enable!
|
25
|
-
|
25
|
+
configuration.enabled = true
|
26
|
+
end
|
27
|
+
|
28
|
+
def self.error(ex)
|
29
|
+
if configuration.error_handler
|
30
|
+
configuration.error_handler.call(ex)
|
31
|
+
else
|
32
|
+
raise ex
|
33
|
+
end
|
26
34
|
end
|
27
35
|
end
|
data/multiple_man.gemspec
CHANGED
@@ -19,8 +19,9 @@ Gem::Specification.new do |spec|
|
|
19
19
|
spec.require_paths = ["lib"]
|
20
20
|
|
21
21
|
spec.add_development_dependency "bundler", "~> 1.3"
|
22
|
-
spec.add_development_dependency "rake"
|
23
|
-
spec.add_development_dependency "rspec"
|
24
|
-
spec.add_runtime_dependency "bunny"
|
25
|
-
spec.add_runtime_dependency "activesupport"
|
22
|
+
spec.add_development_dependency "rake", '~> 10.1.0'
|
23
|
+
spec.add_development_dependency "rspec", '~> 2.14.1'
|
24
|
+
spec.add_runtime_dependency "bunny", '>= 1.0'
|
25
|
+
spec.add_runtime_dependency "activesupport", '>= 3.0'
|
26
|
+
spec.add_runtime_dependency "connection_pool", ">= 1.2"
|
26
27
|
end
|
data/spec/connection_spec.rb
CHANGED
@@ -9,7 +9,7 @@ describe MultipleMan::Connection do
|
|
9
9
|
it "should open a connection and a channel" do
|
10
10
|
mock_bunny.should_receive(:start)
|
11
11
|
Bunny.should_receive(:new).and_return(mock_bunny)
|
12
|
-
mock_bunny.should_receive(:create_channel).
|
12
|
+
mock_bunny.should_receive(:create_channel).exactly(25).times
|
13
13
|
|
14
14
|
described_class.connect
|
15
15
|
end
|
@@ -1,15 +1,14 @@
|
|
1
1
|
require 'spec_helper'
|
2
2
|
|
3
3
|
describe MultipleMan::ModelPublisher do
|
4
|
-
let(:channel_stub) { double(Bunny::Channel, topic: topic_stub
|
4
|
+
let(:channel_stub) { double(Bunny::Channel, topic: topic_stub)}
|
5
5
|
let(:topic_stub) { double(Bunny::Exchange, publish: nil) }
|
6
6
|
|
7
7
|
before {
|
8
|
+
MultipleMan::Connection.stub(:connect).and_yield(channel_stub)
|
8
9
|
MultipleMan.configure do |config|
|
9
10
|
config.topic_name = "app"
|
10
11
|
end
|
11
|
-
|
12
|
-
MultipleMan::Connection.stub(:open_channel).and_return(channel_stub)
|
13
12
|
}
|
14
13
|
|
15
14
|
class MockObject
|
@@ -29,15 +28,18 @@ describe MultipleMan::ModelPublisher do
|
|
29
28
|
subject { described_class.new(:create, fields: [:foo]) }
|
30
29
|
|
31
30
|
describe "after_commit" do
|
32
|
-
it "should queue the update in the correct topic" do
|
33
|
-
channel_stub.should_receive(:topic).with("app")
|
34
|
-
described_class.new(:create, fields: [:foo]).after_commit(MockObject.new)
|
35
|
-
end
|
36
31
|
it "should send the jsonified version of the model to the correct routing key" do
|
37
32
|
MultipleMan::AttributeExtractor.any_instance.should_receive(:to_json).and_return('{"id":10,"data":{"foo": "bar"}}')
|
38
33
|
topic_stub.should_receive(:publish).with('{"id":10,"data":{"foo": "bar"}}', routing_key: "app.MockObject.create")
|
39
34
|
described_class.new(:create, fields: [:foo]).after_commit(MockObject.new)
|
40
35
|
end
|
36
|
+
|
37
|
+
it "should call the error handler on error" do
|
38
|
+
ex = Exception.new("Bad stuff happened")
|
39
|
+
topic_stub.stub(:publish) { raise ex }
|
40
|
+
MultipleMan.should_receive(:error).with(ex)
|
41
|
+
described_class.new(:create, fields: [:foo]).after_commit(MockObject.new)
|
42
|
+
end
|
41
43
|
end
|
42
44
|
|
43
45
|
end
|
data/spec/routing_key_spec.rb
CHANGED
@@ -11,7 +11,7 @@ describe MultipleMan::RoutingKey do
|
|
11
11
|
end
|
12
12
|
|
13
13
|
describe "to_s" do
|
14
|
-
subject { described_class.new(MockObject
|
14
|
+
subject { described_class.new("MockObject", operation).to_s }
|
15
15
|
|
16
16
|
context "creating" do
|
17
17
|
let(:operation) { :create }
|
@@ -29,7 +29,7 @@ describe MultipleMan::RoutingKey do
|
|
29
29
|
end
|
30
30
|
|
31
31
|
context "not specified" do
|
32
|
-
subject { described_class.new(MockObject
|
32
|
+
subject { described_class.new("MockObject").to_s }
|
33
33
|
it { should == "app.MockObject.#" }
|
34
34
|
end
|
35
35
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: multiple_man
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
4
|
+
version: 0.2.7
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ryan Brunner
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-01-
|
11
|
+
date: 2014-01-09 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -28,58 +28,72 @@ dependencies:
|
|
28
28
|
name: rake
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
|
-
- -
|
31
|
+
- - ~>
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version:
|
33
|
+
version: 10.1.0
|
34
34
|
type: :development
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
|
-
- -
|
38
|
+
- - ~>
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version:
|
40
|
+
version: 10.1.0
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
42
|
name: rspec
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
44
44
|
requirements:
|
45
|
-
- -
|
45
|
+
- - ~>
|
46
46
|
- !ruby/object:Gem::Version
|
47
|
-
version:
|
47
|
+
version: 2.14.1
|
48
48
|
type: :development
|
49
49
|
prerelease: false
|
50
50
|
version_requirements: !ruby/object:Gem::Requirement
|
51
51
|
requirements:
|
52
|
-
- -
|
52
|
+
- - ~>
|
53
53
|
- !ruby/object:Gem::Version
|
54
|
-
version:
|
54
|
+
version: 2.14.1
|
55
55
|
- !ruby/object:Gem::Dependency
|
56
56
|
name: bunny
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
58
58
|
requirements:
|
59
59
|
- - '>='
|
60
60
|
- !ruby/object:Gem::Version
|
61
|
-
version: '0'
|
61
|
+
version: '1.0'
|
62
62
|
type: :runtime
|
63
63
|
prerelease: false
|
64
64
|
version_requirements: !ruby/object:Gem::Requirement
|
65
65
|
requirements:
|
66
66
|
- - '>='
|
67
67
|
- !ruby/object:Gem::Version
|
68
|
-
version: '0'
|
68
|
+
version: '1.0'
|
69
69
|
- !ruby/object:Gem::Dependency
|
70
70
|
name: activesupport
|
71
71
|
requirement: !ruby/object:Gem::Requirement
|
72
72
|
requirements:
|
73
73
|
- - '>='
|
74
74
|
- !ruby/object:Gem::Version
|
75
|
-
version: '0'
|
75
|
+
version: '3.0'
|
76
|
+
type: :runtime
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - '>='
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '3.0'
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: connection_pool
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - '>='
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: '1.2'
|
76
90
|
type: :runtime
|
77
91
|
prerelease: false
|
78
92
|
version_requirements: !ruby/object:Gem::Requirement
|
79
93
|
requirements:
|
80
94
|
- - '>='
|
81
95
|
- !ruby/object:Gem::Version
|
82
|
-
version: '
|
96
|
+
version: '1.2'
|
83
97
|
description: MultipleMan syncs changes to ActiveRecord models via AMQP
|
84
98
|
email:
|
85
99
|
- ryan@influitive.com
|