haredo 0.0.1 → 0.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 1585745834668ad5a1db7ceade69f55cad742453
4
- data.tar.gz: adf26761a2a995d47f3e3d7d5c11453fc78b65cd
3
+ metadata.gz: 85222fb487a3ac64174d10d323211ba37ddf427b
4
+ data.tar.gz: 758c7117f150c96b882132d14d38e2e0495ea080
5
5
  SHA512:
6
- metadata.gz: 20b36a13ae81dc1c33636d786da3c3248e3071faf9d5a55cede56b9c63ba7bf9165be0d2f13d0cc2e3d32126c2f6c5cb53e6109eb41dd74b9195003cbc895141
7
- data.tar.gz: dbe8eff27818b3ef38fa3687bfc20df928ba4e2e88edc85c7c789c0eeef4b7e8d1ec8b34cf03315c8d806ae8eb43c261943df9134b51d6ac31463c11f04c4d1b
6
+ metadata.gz: ac949da33da127bdf7b1b587c346349a5abdb8e5ccf30c4bc18c57613fa1729c50b2776bad73357ae3852f326e906870daecacacc745ed0006f42f4bab6f9141
7
+ data.tar.gz: 2a5d9416907828acbcdf9d6a4e8259b56b06d82025eb4901e2acecbd39488a41f0ef4a32af4b20258c54e45b8ab17e3503fe56d9da8e0f10681d83e3e3987f83
data/README.md CHANGED
@@ -2,22 +2,31 @@
2
2
 
3
3
  ## About
4
4
 
5
- An easy-to-use framework for quickly creating client/server applications in Ruby
6
- over RabbitMQ. The RabbitMQ terminology uses metaophors of publishers and
7
- consumers. This framework -- which is a very thin wrapper on top of
8
- [Bunny](http://rubybunny.info/) -- slightly modifies the semantics and provides
9
- an intuitive framework for easily and quickly implementing network services and
10
- clients to use them.
5
+ This is an easy-to-use framework for creating peer-to-peer applications in Ruby
6
+ via [RabbitMQ](http://www.rabbitmq.com/). While RabbitMQ is a robust, feature
7
+ rich message broker, it (like all AMQP message brokers) uses a somewhat complex
8
+ terminology that can be a little bewildering to to newcomers of AMQP.
9
+
10
+ This gem -- built atop the [Bunny AMQP client](http://rubybunny.info/) -- uses a
11
+ small subset of RabbitMQ's capabilities to provide an intuitive client/server
12
+ framework for easily and quickly implementing network services and simple peer
13
+ to peer applications. It specifically designed for those who are not familiar
14
+ with the ins and outs of the AMQP protocol.
11
15
 
12
16
  ## How It Works
13
17
 
14
- Here is a complete example -- the following plugin/program (included in
15
- <tt>src/examples/service.rb</tt>) create a simple network service that takes a
16
- number and add one, along with a client that uses it:
18
+ Here is a complete example -- the following program creates a simple network
19
+ service that takes a number and add one, along with a client that uses it. They
20
+ both run simultaneously in the same script.
17
21
 
18
22
  ```ruby
19
23
  #!/usr/bin/env ruby
20
24
 
25
+ $rabbitmq_host = 'localhost'
26
+ $rabbitmq_username = 'guest'
27
+ $rabbitmq_password = 'guest'
28
+ $haredo_test_queue = 'HareDo'
29
+
21
30
  # Simple service
22
31
  class Service < HareDo::Service
23
32
 
@@ -27,11 +36,8 @@ class Service < HareDo::Service
27
36
  end
28
37
 
29
38
  def serve(msg)
30
- headers = msg.properties.headers
31
- reply_to = msg.properties.reply_to
32
-
33
- data = headers['i'].to_i + 1
34
- send(reply_to, data.to_s, { :rc => 1 })
39
+ data = msg.headers['i'].to_i + 1
40
+ send(msg.properties.reply_to, :data => data.to_s)
35
41
  end
36
42
 
37
43
  end # class Server
@@ -49,10 +55,10 @@ client = HareDo::Client.new()
49
55
  client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
50
56
  service = Service.new($haredo_test_queue)
51
57
 
52
- service.run(false)
58
+ service.run(:blocking => false)
53
59
 
54
60
  1.upto(10) do |i|
55
- client.send($haredo_test_queue, 'data', { :i => i })
61
+ client.send($haredo_test_queue, :headers => { :i => i })
56
62
  msg = @client.receive()
57
63
 
58
64
  dump_message msg
@@ -61,9 +67,166 @@ service.run(false)
61
67
  end
62
68
  ```
63
69
 
64
- The server runs in non-blocking mode. The client makes 10 calls sending 10
65
- monotonically increasing integer values, waits for the response (using a
66
- blocking timeout) after each call and checks the results.
70
+ The service takes a single argument -- the queue name. This is like an email
71
+ address or an IP address. It's an identifier you use to direct messages to the
72
+ service.
73
+
74
+ The client sends 10 messages to the service, each containing monotonically
75
+ increasing integer values. It waits for the response (using a blocking timeout)
76
+ after each call and checks the results.
77
+
78
+ ### Request/Response Pattern
79
+
80
+ While it is not required that a service reply, the requests/response pattern is
81
+ the main function of this library. We are often interested in sending a request
82
+ and getting one (or more responses) back. Implementing this is only a slight
83
+ change from the above example. The following implements a service that responds
84
+ with a single message.
85
+
86
+ The server code is as follows:
87
+
88
+ ```ruby
89
+ #!/usr/bin/env ruby
90
+
91
+ require 'haredo/peer'
92
+
93
+ $rabbitmq_host = 'localhost'
94
+ $rabbitmq_username = 'guest'
95
+ $rabbitmq_password = 'guest'
96
+ $haredo_test_queue = 'HareDo'
97
+
98
+ class Service < HareDo::Service
99
+
100
+ def initialize(name)
101
+ super
102
+ connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
103
+ end
104
+
105
+ def serve(msg)
106
+ reply(msg, :data => 'Reply')
107
+ end
108
+ end
109
+
110
+ Service.new($haredo_test_queue).run()
111
+ ```
112
+
113
+ The client code is a follows:
114
+
115
+ ```ruby
116
+ #!/usr/bin/env ruby
117
+
118
+ require 'haredo/peer'
119
+
120
+ $rabbitmq_host = 'localhost'
121
+ $rabbitmq_username = 'guest'
122
+ $rabbitmq_password = 'guest'
123
+ $haredo_test_queue = 'HareDo'
124
+
125
+ client = HareDo::Client.new()
126
+ client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
127
+ response = @client.call($haredo_test_queue, :data => 'jujifruit')
128
+ ```
129
+
130
+ You can run both in the same script by setting the service code to non-blocking
131
+ (passing <tt>:blocking => false</tt> to the <tt>Service::run()</tt> method):
132
+
133
+ ```ruby
134
+ #!/usr/bin/env ruby
135
+
136
+ require 'haredo/peer'
137
+
138
+ $rabbitmq_host = 'localhost'
139
+ $rabbitmq_username = 'guest'
140
+ $rabbitmq_password = 'guest'
141
+ $haredo_test_queue = 'HareDo'
142
+
143
+ class Service < HareDo::Service
144
+
145
+ def initialize(name)
146
+ super
147
+ connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
148
+ end
149
+
150
+ def serve(msg)
151
+ reply(msg, :data => 'Reply')
152
+ end
153
+ end
154
+
155
+ service = Service.new($haredo_test_queue)
156
+ service.run(:blocking => false)
157
+
158
+ client = HareDo::Client.new()
159
+ client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
160
+ response = @client.call($haredo_test_queue, :data => 'jujifruit')
161
+ ```
162
+
163
+ The client uses the <tt>call()</tt> method to send a message and wait for a
164
+ response. This is just a convenience method that wraps <tt>send()</tt> and
165
+ <tt>receive()</tt>. Interally, a unique message ID is assigned to the request
166
+ message, and the service will send back a reply with the message ID set in the
167
+ message headers.
168
+
169
+ The client will block waiting for the reponse -- up to <tt>client.timeout</tt>
170
+ seconds (floating point value). The default is 1 second. You can change this to
171
+ whatever value you like. If a timeout occurs, the <tt>call()</tt> will return
172
+ <tt>nil</tt>. Subsequent calls will fail to get the response, even if it comes
173
+ in later (it will be discarded).
174
+
175
+ The service uses the <tt>reply()</tt> method rather than the <tt>send()</tt>
176
+ method to return a response. This method takes the original request message,
177
+ extracts the <tt>to</tt> address along with message ID and addresses a message
178
+ back to the client. It adds a header value of <tt>reply=true</tt>. This lets the
179
+ client know that the message coming in is a reply. It needs this because what if
180
+ another peer sent it a message with the same message id? It might interpret that
181
+ message as the reply from the service. Therefore this flag lets the client know
182
+ that the message is a reply to a previous request.
183
+
184
+ Using this simple pattern, you can easily create heterogeneous network services
185
+ in just a few lines of code.
186
+
187
+ ### Roud-Robin Services
188
+
189
+ Say you want to scale your service to three nodes distributed across three
190
+ different machines. The only thing you need to do differently is redefine one
191
+ method -- <tt>Service::createQueue()</tt>. The following is updates the above
192
+ example and makes is available to run in this way:
193
+
194
+ ```ruby
195
+ #!/usr/bin/env ruby
196
+
197
+ require 'haredo/peer'
198
+
199
+ $rabbitmq_host = 'localhost'
200
+ $rabbitmq_username = 'guest'
201
+ $rabbitmq_password = 'guest'
202
+ $haredo_test_queue = 'HareDo'
203
+
204
+ class Service < HareDo::Service
205
+
206
+ def initialize(name)
207
+ super
208
+ connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
209
+ end
210
+
211
+ def createQueue()
212
+ return @channel.queue(@queue_name, :auto_delete => true)
213
+ end
214
+
215
+
216
+ def serve(msg)
217
+ reply(msg, :data => 'Reply')
218
+ end
219
+ end
220
+
221
+ Service.new($haredo_test_queue).run()
222
+ ```
223
+
224
+ The only thing we did is remove the <tt>exclusive</tt> attribute which is by
225
+ default in the <tt>HareDO::Service</tt> base class. By making the queue this
226
+ services uses non-exclusive, mulitple service instances can now connect and
227
+ process requests. RabbitMQ will automatically distribute messages equally across
228
+ all the running services, dividing up the load. You can now scale your service
229
+ to as many machines and processes as you like.
67
230
 
68
231
  ## Installation
69
232
 
@@ -3,18 +3,35 @@ require 'haredo/version'
3
3
 
4
4
  module HareDo
5
5
 
6
+ # This is a simple class that represents a message delivered from a queue. The
7
+ # attributes are as follows:
8
+ #
9
+ # * @info member contains the delivery information
10
+ # * @properties contains the message properties
11
+ # * @headers the message headers
12
+ # * @data the message data
13
+
6
14
  class Message
7
15
 
8
- attr_reader :info, :properties, :data
16
+ attr_reader :info, :properties, :headers, :data
9
17
 
10
18
  def initialize(info=nil, properties=nil, data=nil)
11
19
  @info = info
12
20
  @properties = properties
13
- @data = data
21
+
22
+ if properties
23
+ @headers = properties[:headers]
24
+ else
25
+ @headers = {}
26
+ end
27
+
28
+ @data = data
14
29
  end
15
30
 
16
31
  end
17
32
 
33
+ # This is an abstract base class used for both Client and Service.
34
+
18
35
  class Node
19
36
 
20
37
  attr_reader :queue
@@ -23,40 +40,80 @@ class Node
23
40
 
24
41
  end
25
42
 
43
+ # Connect to RabbitMQ
44
+ #
45
+ # @param user The RabbitMQ username
46
+ # @param password The RabbitMQ password
47
+ # @param host The RabbitMQ host
48
+ # @param port The RabbitMQ port
49
+ # @return Returns true if connection was successful, false othewise.
50
+
26
51
  def connect(user, password, host='localhost', port='5672')
27
- @cnx = Bunny.new("amqp://#{user}:#{password}@#{host}:#{port}")
28
52
 
29
- @cnx.start()
53
+ begin
54
+ @cnx = Bunny.new("amqp://#{user}:#{password}@#{host}:#{port}")
55
+ @cnx.start()
56
+ rescue Bunny::TCPConnectionFailed => e
57
+ return false
58
+ end
30
59
 
31
60
  @channel = @cnx.create_channel()
32
61
 
33
62
  return true
34
63
  end
35
64
 
65
+ # Disconnect from RabbitMQ
36
66
  def disconnect()
37
67
  @cnx.close()
38
68
  end
39
69
 
40
- def send(to, data, headers = {}, from=nil)
70
+ # Sends a message.
71
+ # @param to The to address
72
+ # @param :data Message data
73
+ # @param :headers Message headers
74
+ # @param :headers Message from address
75
+ # @param :properties Message properties
76
+
77
+ def send(to, args)
78
+
79
+ data = args[:data]
80
+ from = args[:from]
81
+ headers = args[:headers] || {}
82
+ properties = args[:properties] || {}
41
83
 
42
- if from.nil?
43
- from = @queue.name if @queue
84
+ properties[:routing_key] = to
85
+ properties[:headers] = headers
86
+
87
+ if not from.nil?
88
+ properties[:reply_to] = from
89
+ else
90
+ properties[:reply_to] = @queue.name if @queue
44
91
  end
45
92
 
46
- @exchange.publish( data,
47
- :routing_key => to,
48
- :headers => headers,
49
- :reply_to => from )
93
+ properties[:message_id] = @mid
94
+
95
+ @exchange.publish(data, properties)
50
96
  end
51
97
 
52
98
  end # class Node
53
99
 
100
+ # Implements a basic client designed for sending messages asynchronously and
101
+ # blocking while receiving replies.
102
+
54
103
  class Client < Node
55
104
 
56
105
  attr_reader :queue
106
+ attr_accessor :timeout, :sleep_interval
57
107
 
58
108
  def initialize()
59
109
  super
110
+
111
+ @timeout = 1.0
112
+ @sleep_interval = 0.001
113
+ @receive_queue = {}
114
+
115
+ # Message identifier
116
+ @mid = 0
60
117
  end
61
118
 
62
119
  def connect(user, password, host='localhost', port='5672')
@@ -68,31 +125,114 @@ class Client < Node
68
125
  @exchange = @channel.default_exchange()
69
126
  end
70
127
 
128
+ # Disconnect from RabbitMQ
71
129
  def disconnect()
72
130
  @queue.delete() if @queue
73
131
  super
74
132
  end
75
133
 
76
- def receive(timeout = 1.0)
77
- now = Time::now.to_f
134
+ # Sends a message. Adds the @mid as message_id in message properties.
135
+ # @param to The to address
136
+ # @param :data Message data
137
+ # @param :headers Message headers
138
+ # @param :headers Message from address
139
+ # @param :properties Message properties
140
+ #
141
+ # @return Returns the message ID of sent message
142
+
143
+ def send(to, args)
144
+
145
+ data = args[:data]
146
+ from = args[:from]
147
+ headers = args[:headers] || {}
148
+ properties = args[:properties] || {}
149
+
150
+ properties[:message_id] = @mid
151
+
152
+ super to, :data => data, :headers => headers, :properties => properties, :from => from
153
+
154
+ rc = @mid
155
+ @mid += 1
156
+
157
+ return rc
158
+ end
78
159
 
160
+ # Sends a message and waits for response
161
+ #
162
+ # @param to The to address
163
+ # @param :data Message data
164
+ # @param :headers Message headers
165
+ # @param :headers Message from address
166
+ # @param :properties Message properties
167
+ #
168
+ # @return Returns the response message if successful, nil otherwise. Will
169
+ # block until @timeout seconds elapse. Sleeps in busywait. Sleep interval is
170
+ # given by @sleep_interval.
171
+
172
+ def call(to, data: '', headers: {}, properties: {}, from: nil)
173
+ # Get message id of sent message
174
+ id = send(to, data: data, headers: headers, properties: properties, from: from)
175
+
176
+ # Put blank entry in queue to indicate we are expecting a response with that
177
+ # message id.
178
+ @receive_queue[id] = nil
179
+
180
+ return receive(id)
181
+ end
182
+
183
+ def receive(mid=nil)
184
+
185
+ if mid != nil
186
+ if @receive_queue.has_key?(mid)
187
+ msg = @receive_queue[mid]
188
+
189
+ if msg != nil
190
+ @receive_queue.delete(mid)
191
+ return msg
192
+ end
193
+ end
194
+ end
195
+
196
+ now = Time::now.to_f
197
+
79
198
  while true
80
199
  delivery_info, properties, payload = @queue.pop()
81
200
 
82
201
  if delivery_info != nil
83
- return Message.new(delivery_info, properties, payload)
202
+ msg = Message.new(delivery_info, properties, payload)
203
+
204
+ if msg.headers.has_key?('id')
205
+ if mid != nil
206
+ if msg.headers['id'] == mid
207
+ # Reply flag must be set
208
+ if msg.headers['reply'] == 1
209
+ return msg
210
+ end
211
+ end
212
+ end
213
+
214
+ # Only add to receive queue if we are expecting it
215
+ if @receive_queue.has_key?
216
+ @receive_queue[msg.headers['id']] = msg
217
+ end
218
+ end
84
219
  end
85
-
86
- if (Time::now.to_f - now) > timeout
87
- return Message.new()
220
+
221
+ if (Time::now.to_f - now) > @timeout
222
+ # Delete entry from receive queue
223
+ @receive_queue.delete mid
224
+
225
+ return nil
88
226
  end
89
227
 
90
- sleep 0.001
228
+ sleep @sleep_interval
91
229
  end
92
230
  end
93
231
 
94
232
  end # class Client
95
233
 
234
+ # Represents a basic service class.
235
+
96
236
  class Service < Node
97
237
 
98
238
  def initialize(name)
@@ -101,19 +241,50 @@ class Service < Node
101
241
  @queue_name = name
102
242
  end
103
243
 
244
+ # Defined the queue this service will listen on. Assumes a single-instance
245
+ # service therefore declares queue as exclusive.
104
246
  def createQueue()
105
247
  return @channel.queue(@queue_name, :auto_delete => true, :exclusive => true)
106
248
  end
107
249
 
108
- def run(block = true)
250
+ # Causes the service to listen for incoming messages.
251
+ #
252
+ # @param :blocking If this is set to true, will go into indefinite blocking
253
+ # loop processing incoming messages.
254
+
255
+ # @returns Returns nil if non-blocking. Never returns if blocking.
256
+
257
+ def run(args)
109
258
  @queue = createQueue()
110
259
  @exchange = @channel.default_exchange()
111
260
 
261
+ block = args[:blocking] || false
262
+
112
263
  queue.subscribe(:block => block) do |info, props, data|
113
264
  serve Message.new(info, props, data)
114
265
  end
115
266
  end
116
267
 
268
+ # You should use this method to reply back to a peer. It sets the reply header
269
+ # which tells the remote that this message is a response (as opposed to a
270
+ # message originating from another source which just happens to have the same
271
+ # message_id).
272
+ def reply(msg, args)
273
+
274
+ data = args[:data]
275
+ headers = args[:headers] || {}
276
+
277
+ id = msg.properties.message_id.to_i
278
+ to = msg.properties.reply_to
279
+
280
+ # Set the reply flag to indicate that this is a response to a message
281
+ # sent. The message_id should already be set in the headers.
282
+ headers[:reply] = 1
283
+ headers[:id] = id.to_i
284
+
285
+ send(to, :headers => headers, :data => data)
286
+ end
287
+
117
288
  end # class Service
118
289
 
119
290
  end # module HareDo
@@ -1,5 +1,5 @@
1
1
  module HareDo
2
2
 
3
- VERSION = '0.0.1-1'
3
+ VERSION = '0.0.2-1'
4
4
 
5
5
  end # module HareDo
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: haredo
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
4
+ version: 0.0.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mike Owens
@@ -9,7 +9,21 @@ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
11
  date: 2013-09-19 00:00:00.000000000 Z
12
- dependencies: []
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: bunny
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - '>='
18
+ - !ruby/object:Gem::Version
19
+ version: '0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - '>='
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
13
27
  description:
14
28
  email: mikeowens@gmail.com
15
29
  executables: []