dispatch-rider 0.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. data/LICENSE.txt +22 -0
  2. data/README.md +270 -0
  3. data/bin/dispatch_rider +17 -0
  4. data/lib/dispatch-rider.rb +21 -0
  5. data/lib/dispatch-rider/demultiplexer.rb +36 -0
  6. data/lib/dispatch-rider/dispatcher.rb +21 -0
  7. data/lib/dispatch-rider/errors.rb +34 -0
  8. data/lib/dispatch-rider/message.rb +33 -0
  9. data/lib/dispatch-rider/notification_services.rb +9 -0
  10. data/lib/dispatch-rider/notification_services/aws_sns.rb +23 -0
  11. data/lib/dispatch-rider/notification_services/base.rb +57 -0
  12. data/lib/dispatch-rider/notification_services/file_system.rb +22 -0
  13. data/lib/dispatch-rider/notification_services/file_system/channel.rb +16 -0
  14. data/lib/dispatch-rider/notification_services/file_system/notifier.rb +17 -0
  15. data/lib/dispatch-rider/publisher.rb +65 -0
  16. data/lib/dispatch-rider/queue_services.rb +10 -0
  17. data/lib/dispatch-rider/queue_services/aws_sqs.rb +39 -0
  18. data/lib/dispatch-rider/queue_services/aws_sqs/message_body_extractor.rb +17 -0
  19. data/lib/dispatch-rider/queue_services/base.rb +74 -0
  20. data/lib/dispatch-rider/queue_services/file_system.rb +39 -0
  21. data/lib/dispatch-rider/queue_services/file_system/queue.rb +42 -0
  22. data/lib/dispatch-rider/queue_services/simple.rb +30 -0
  23. data/lib/dispatch-rider/registrars.rb +13 -0
  24. data/lib/dispatch-rider/registrars/base.rb +39 -0
  25. data/lib/dispatch-rider/registrars/file_system_channel.rb +11 -0
  26. data/lib/dispatch-rider/registrars/handler.rb +10 -0
  27. data/lib/dispatch-rider/registrars/notification_service.rb +10 -0
  28. data/lib/dispatch-rider/registrars/publishing_destination.rb +9 -0
  29. data/lib/dispatch-rider/registrars/queue_service.rb +10 -0
  30. data/lib/dispatch-rider/registrars/sns_channel.rb +10 -0
  31. data/lib/dispatch-rider/subscriber.rb +41 -0
  32. data/lib/dispatch-rider/version.rb +4 -0
  33. metadata +236 -0
data/LICENSE.txt ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2013 Suman Mukherjee
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,270 @@
1
+ # Dispatch::Rider
2
+
3
+ Dispatch rider is a pub/sub kind of library that allows you to publish a
4
+ message to a notification system (like Amazon SNS) and then you
5
+ can subscribe to the channels that you subscribed to and start
6
+ handling the messages.
7
+
8
+ ### Build status
9
+
10
+ [![Build Status](https://travis-ci.org/payrollhero/dispatch-rider.png)](https://travis-ci.org/payrollhero/dispatch-rider)
11
+
12
+ ## Installation
13
+
14
+ Add this line to your application's Gemfile:
15
+
16
+ gem 'dispatch-rider'
17
+
18
+ And then execute:
19
+
20
+ $ bundle
21
+
22
+ Or install it yourself as:
23
+
24
+ $ gem install dispatch-rider
25
+
26
+ ## Usage
27
+
28
+ ### Publisher
29
+
30
+ Setting up a publisher is simple.
31
+
32
+ To publish using the filesystem register the path where to publish the message files.
33
+
34
+ ```ruby
35
+
36
+ publisher = DispatchRider::Publisher.new
37
+
38
+ publisher.register_notification_service(:file_system)
39
+ publisher.register_destination(:local_message_queue, :file_system, :dev_channel, :path => "tmp/news-updates")
40
+
41
+ publisher.publish(:destinations => :local_message_queue, :message => {
42
+ :subject => "read_news",
43
+ :body => {"headlines" => [
44
+ "April 29, 2013: Rails 4.0.0.rc1 is released.",
45
+ "May 14, 2013: Ruby 2.0.0-p195 is released"
46
+ ]
47
+ }})
48
+
49
+ ```
50
+
51
+ To publish using ```AWS::SNS``` make sure ```AWS.config``` has been setup.
52
+ It's then as easy as providing the configuration details of the topic to the publisher.
53
+
54
+ ```ruby
55
+
56
+ publisher = DispatchRider::Publisher.new
57
+
58
+ publisher.register_notification_service(:aws_sns)
59
+ publisher.register_destination(:sns_message_queue, :aws_sns, :dev_channel, {
60
+ :account => 777,
61
+ :region => 'us-east-1',
62
+ :topic => 'RoR'
63
+ })
64
+
65
+ publisher.publish(:destinations => :sns_message_queue, :message => {
66
+ :subject => "read_news",
67
+ :body => {"headlines" => [
68
+ "April 29, 2013: Rails 4.0.0.rc1 is released.",
69
+ "May 14, 2013: Ruby 2.0.0-p195 is released"
70
+ ]
71
+ }})
72
+
73
+ ```
74
+
75
+ To publish to multiple destinations:
76
+
77
+ ```ruby
78
+
79
+ publisher.publish(:destinations => [:local_message_queue, :sns_message_queue], :message => {
80
+ :subject => "read_news",
81
+ :body => {"headlines" => [
82
+ "April 29, 2013: Rails 4.0.0.rc1 is released.",
83
+ "May 14, 2013: Ruby 2.0.0-p195 is released"
84
+ ]
85
+ }})
86
+
87
+ ```
88
+
89
+ Sample Rails publisher:
90
+
91
+ ```ruby
92
+
93
+ # app/publishers/news_update
94
+ class NewsPublisher
95
+ @publisher = DispatchRider::Publisher.new
96
+
97
+ amazon_config = YAML.load_file("#{Rails.root}/config/amazon.yml")
98
+
99
+ @publisher.register_notification_service(:aws_sns)
100
+ @publisher.register_destination(:sns_message_queue, :aws_sns, :dev_channel, {
101
+ :account => amazon_config[:account],
102
+ :region => amazon_config[:region],
103
+ :topic => "news-updates-#{Rails.env}"
104
+ })
105
+
106
+ @destinations = [:sns_message_queue]
107
+
108
+ class << self
109
+ attr_reader :publisher
110
+ attr_accessor :destinations
111
+ end
112
+
113
+ delegate :publisher, :destinations, :to => :"self.class"
114
+
115
+ def initialize(news)
116
+ @news = news
117
+ end
118
+
119
+ def publish
120
+ publisher.publish(:destinations => destinations, :message => {
121
+ :subject => "read_news",
122
+ :body => {"headlines" => @news.headlines}
123
+ })
124
+ end
125
+ end
126
+
127
+ # app/models/news
128
+ class News
129
+ serialize :headlines, Array
130
+
131
+ after_create :publish
132
+
133
+ def publish
134
+ NewsPublisher.new(self).publish
135
+ end
136
+ end
137
+
138
+ News.create!(:headlines => [
139
+ "April 29, 2013: Rails 4.0.0.rc1 is released.",
140
+ "May 14, 2013: Ruby 2.0.0-p195 is released"
141
+ ])
142
+ ```
143
+
144
+ ### Subscriber
145
+
146
+ To setup a subscriber you'll need message handlers. The handlers are named the same as the message subjects.
147
+
148
+ Sample message handler:
149
+ ```ruby
150
+
151
+ # app/handlers/bar_handler
152
+ module ReadNews
153
+ class << self
154
+ def process(message_body)
155
+ message_body["headlines"].each do |headline|
156
+ puts headline
157
+ end
158
+ end
159
+ end
160
+ end
161
+
162
+ ```
163
+
164
+ Sample subscriber setup:
165
+
166
+ ```ruby
167
+
168
+ subscriber = DispatchRider::Subscriber.new
169
+
170
+ subscriber.register_queue(:aws_sqs, :name => "news-updates")
171
+ subscriber.register_handler(:read_news)
172
+ subscriber.setup_demultiplexer(:aws_sqs)
173
+
174
+ subscriber.process
175
+
176
+ ```
177
+
178
+ Sample Rails application rake task:
179
+
180
+ ```ruby
181
+
182
+ # lib/tasks/dispatch-rider
183
+ namespace :"dispatch-rider" do
184
+ desc "Tells DispatchRider to start running"
185
+ task :run => [:"run:remote"]
186
+
187
+ namespace :run do
188
+ desc "Tells DispatchRider to start running"
189
+ task :remote => [:ready, :set_aws_sqs_queue, :process]
190
+
191
+ desc "Tells DispatchRider to start running (using the filesystem as the queue)"
192
+ task :local => [:ready, :set_local_queue, :process]
193
+
194
+ task :ready => :environment do
195
+ puts "Creating subscriber..."
196
+ @subscriber = DispatchRider::Subscriber.new
197
+
198
+ [ # list of message handlers
199
+ :read_news
200
+ ].each do |handler_name|
201
+ puts "Registering #{handler_name} handler..."
202
+ @subscriber.register_handler(handler_name)
203
+ end
204
+ end
205
+
206
+ task :set_aws_sqs_queue do
207
+ queue_name = "news-updates-#{Rails.env}"
208
+ puts "Setting AWS::SQS #{queue_name} queue..."
209
+ @subscriber.register_queue(:aws_sqs, :name => queue_name)
210
+ @subscriber.setup_demultiplexer(:aws_sqs)
211
+ end
212
+
213
+ task :set_local_queue do
214
+ queue_path = "tmp/news-updates-#{Rails.env}"
215
+ puts "Setting local filesystem queue @ #{queue_path.inspect}..."
216
+ @subscriber.register_queue(:file_system, :path => queue_path)
217
+ @subscriber.setup_demultiplexer(:file_system)
218
+ end
219
+
220
+ task :process do
221
+ puts "Running..."
222
+ @subscriber.process
223
+ end
224
+ end
225
+ end
226
+
227
+ ```
228
+
229
+ To run the subscriber simply execute following:
230
+
231
+ $ bundle exec rake dispatch-rider:run
232
+
233
+ To run locally:
234
+
235
+ $ bundle exec rake dispatch-rider:run:local
236
+
237
+ ## Contributing
238
+
239
+ ### Process
240
+
241
+ 1. Fork it
242
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
243
+ 3. Commit your changes (`git commit -am 'Add some feature'`)
244
+ 4. Push to the branch (`git push origin my-new-feature`)
245
+ 5. Create new Pull Request
246
+
247
+ ### Licence
248
+
249
+ Copyright (c) 2013 Suman Mukherjee
250
+
251
+ MIT License
252
+
253
+ Permission is hereby granted, free of charge, to any person obtaining
254
+ a copy of this software and associated documentation files (the
255
+ "Software"), to deal in the Software without restriction, including
256
+ without limitation the rights to use, copy, modify, merge, publish,
257
+ distribute, sublicense, and/or sell copies of the Software, and to
258
+ permit persons to whom the Software is furnished to do so, subject to
259
+ the following conditions:
260
+
261
+ The above copyright notice and this permission notice shall be
262
+ included in all copies or substantial portions of the Software.
263
+
264
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
265
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
266
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
267
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
268
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
269
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
270
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,17 @@
1
+ #!/bin/env ruby
2
+
3
+ raise "this script is meant to be ran from inside of a Rails.root" unless File.exist?("config/environment.rb")
4
+
5
+ require 'rubygems'
6
+ require 'daemons'
7
+ require 'pathname'
8
+
9
+ gem_home = Pathname.new(__FILE__) + "../.."
10
+ app_home = Pathname.new(Dir.getwd)
11
+
12
+ Daemons.run_proc("dispatch_rider", :log_output => true, :dir_mode => :normal, :log_dir => (app_home + "log").to_s, :dir => (app_home + "log").to_s) do
13
+ Dir.chdir(app_home.to_s) do
14
+ require "./config/environment"
15
+ Rails.application.config.dispatch_rider.process
16
+ end
17
+ end
@@ -0,0 +1,21 @@
1
+ # Top level namespace of the gem
2
+ require "dispatch-rider/version"
3
+
4
+ require "active_support/hash_with_indifferent_access"
5
+ require "active_support/inflector"
6
+ require "active_support/json"
7
+ require "active_support/core_ext/array/conversions"
8
+ require "active_model"
9
+
10
+ module DispatchRider
11
+ end
12
+
13
+ require "dispatch-rider/errors"
14
+ require "dispatch-rider/message"
15
+ require "dispatch-rider/registrars"
16
+ require "dispatch-rider/notification_services"
17
+ require "dispatch-rider/queue_services"
18
+ require "dispatch-rider/dispatcher"
19
+ require "dispatch-rider/demultiplexer"
20
+ require "dispatch-rider/publisher"
21
+ require "dispatch-rider/subscriber"
@@ -0,0 +1,36 @@
1
+ # The demultiplexer in the reactor pattern is implemented in this class.
2
+ # The object needs to be initiated with a queue and a dispatcher.
3
+ # Demultiplexer#start defines an event loop which pops items from the queue
4
+ # and passes it on to the dispatcher for dispatching to the appropriate message handler.
5
+ # The demultiplexer can be stopped by calling the Demultiplexer#stop method.
6
+ module DispatchRider
7
+ class Demultiplexer
8
+ attr_reader :queue, :dispatcher
9
+
10
+ def initialize(queue, dispatcher)
11
+ @queue = queue
12
+ @dispatcher = dispatcher
13
+ @continue = true
14
+ end
15
+
16
+ def start
17
+ catch(:done) do
18
+ loop do
19
+ throw :done unless @continue
20
+ queue.pop do |message|
21
+ dispatch_message(message)
22
+ end
23
+ end
24
+ end
25
+ self
26
+ end
27
+
28
+ def stop
29
+ @continue = false
30
+ end
31
+
32
+ def dispatch_message(message)
33
+ dispatcher.dispatch(message)
34
+ end
35
+ end
36
+ end
@@ -0,0 +1,21 @@
1
+ # This class is responsible for dispatching the messages to the appropriate handler.
2
+ # The handlers must be registered with the dispatcher.
3
+ # Tha handlers need to be modules that implement the process method.
4
+ # What handler to dispatch the message to is figured out from the subject of the message.
5
+ module DispatchRider
6
+ class Dispatcher
7
+ extend Forwardable
8
+
9
+ attr_reader :handler_registrar
10
+
11
+ def_delegators :handler_registrar, :register, :fetch, :unregister
12
+
13
+ def initialize
14
+ @handler_registrar = Registrars::Handler.new
15
+ end
16
+
17
+ def dispatch(message)
18
+ handler_registrar.fetch(message.subject).process(message.body)
19
+ end
20
+ end
21
+ end
@@ -0,0 +1,34 @@
1
+ # This file contains all the error classes for this gem.
2
+ module DispatchRider
3
+ # The base error class of the gem
4
+ class DispatchRiderError < StandardError
5
+ end
6
+
7
+ # The error class for objects not being found
8
+ class NotFound < DispatchRiderError
9
+ def initialize(name)
10
+ super("#{name.to_s} could not be found")
11
+ end
12
+ end
13
+
14
+ # The error class for keys not registered in a registrar
15
+ class NotRegistered < DispatchRiderError
16
+ def initialize(name)
17
+ super("#{name.to_s} has not been registered")
18
+ end
19
+ end
20
+
21
+ # This error is raised when a queue service depends on an external library, but that is not present
22
+ class AdapterNotFoundError < DispatchRiderError
23
+ def initialize(lib_name, gem_name)
24
+ super("Constant #{lib_name} wasn't found. Please install the #{gem_name} gem")
25
+ end
26
+ end
27
+
28
+ # This error is raised when validation fails on an object
29
+ class RecordInvalid < DispatchRiderError
30
+ def initialize(object, error_messages)
31
+ super("#{object.class.name} is not valid because of the following errors : #{error_messages.to_sentence}")
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,33 @@
1
+ # This class represents a message. All other objects dela with passing around instances of this class.
2
+ # A message must have a subject and a body. The subject represents the handlers name and the body represents
3
+ # the payload of the process method in the handler.
4
+ # When messages are stored in the queues, they are serialized.
5
+ module DispatchRider
6
+ class Message
7
+ include ActiveModel::Validations
8
+
9
+ attr_accessor :subject, :body
10
+
11
+ validates :subject, :presence => true
12
+
13
+ def initialize(options)
14
+ attrs = options.symbolize_keys
15
+ @subject = attrs[:subject]
16
+ @body = attrs[:body] || {}
17
+ raise RecordInvalid.new(self, errors.full_messages) unless valid?
18
+ end
19
+
20
+ def attributes
21
+ {:subject => subject, :body => body}
22
+ end
23
+
24
+ def to_json
25
+ attributes.to_json
26
+ end
27
+
28
+ def ==(other)
29
+ return false unless other.respond_to? :attributes
30
+ attributes == other.attributes
31
+ end
32
+ end
33
+ end