haredo 0.0.1 → 0.0.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 1585745834668ad5a1db7ceade69f55cad742453
4
- data.tar.gz: adf26761a2a995d47f3e3d7d5c11453fc78b65cd
3
+ metadata.gz: 85222fb487a3ac64174d10d323211ba37ddf427b
4
+ data.tar.gz: 758c7117f150c96b882132d14d38e2e0495ea080
5
5
  SHA512:
6
- metadata.gz: 20b36a13ae81dc1c33636d786da3c3248e3071faf9d5a55cede56b9c63ba7bf9165be0d2f13d0cc2e3d32126c2f6c5cb53e6109eb41dd74b9195003cbc895141
7
- data.tar.gz: dbe8eff27818b3ef38fa3687bfc20df928ba4e2e88edc85c7c789c0eeef4b7e8d1ec8b34cf03315c8d806ae8eb43c261943df9134b51d6ac31463c11f04c4d1b
6
+ metadata.gz: ac949da33da127bdf7b1b587c346349a5abdb8e5ccf30c4bc18c57613fa1729c50b2776bad73357ae3852f326e906870daecacacc745ed0006f42f4bab6f9141
7
+ data.tar.gz: 2a5d9416907828acbcdf9d6a4e8259b56b06d82025eb4901e2acecbd39488a41f0ef4a32af4b20258c54e45b8ab17e3503fe56d9da8e0f10681d83e3e3987f83
data/README.md CHANGED
@@ -2,22 +2,31 @@
2
2
 
3
3
  ## About
4
4
 
5
- An easy-to-use framework for quickly creating client/server applications in Ruby
6
- over RabbitMQ. The RabbitMQ terminology uses metaophors of publishers and
7
- consumers. This framework -- which is a very thin wrapper on top of
8
- [Bunny](http://rubybunny.info/) -- slightly modifies the semantics and provides
9
- an intuitive framework for easily and quickly implementing network services and
10
- clients to use them.
5
+ This is an easy-to-use framework for creating peer-to-peer applications in Ruby
6
+ via [RabbitMQ](http://www.rabbitmq.com/). While RabbitMQ is a robust, feature
7
+ rich message broker, it (like all AMQP message brokers) uses a somewhat complex
8
+ terminology that can be a little bewildering to to newcomers of AMQP.
9
+
10
+ This gem -- built atop the [Bunny AMQP client](http://rubybunny.info/) -- uses a
11
+ small subset of RabbitMQ's capabilities to provide an intuitive client/server
12
+ framework for easily and quickly implementing network services and simple peer
13
+ to peer applications. It specifically designed for those who are not familiar
14
+ with the ins and outs of the AMQP protocol.
11
15
 
12
16
  ## How It Works
13
17
 
14
- Here is a complete example -- the following plugin/program (included in
15
- <tt>src/examples/service.rb</tt>) create a simple network service that takes a
16
- number and add one, along with a client that uses it:
18
+ Here is a complete example -- the following program creates a simple network
19
+ service that takes a number and add one, along with a client that uses it. They
20
+ both run simultaneously in the same script.
17
21
 
18
22
  ```ruby
19
23
  #!/usr/bin/env ruby
20
24
 
25
+ $rabbitmq_host = 'localhost'
26
+ $rabbitmq_username = 'guest'
27
+ $rabbitmq_password = 'guest'
28
+ $haredo_test_queue = 'HareDo'
29
+
21
30
  # Simple service
22
31
  class Service < HareDo::Service
23
32
 
@@ -27,11 +36,8 @@ class Service < HareDo::Service
27
36
  end
28
37
 
29
38
  def serve(msg)
30
- headers = msg.properties.headers
31
- reply_to = msg.properties.reply_to
32
-
33
- data = headers['i'].to_i + 1
34
- send(reply_to, data.to_s, { :rc => 1 })
39
+ data = msg.headers['i'].to_i + 1
40
+ send(msg.properties.reply_to, :data => data.to_s)
35
41
  end
36
42
 
37
43
  end # class Server
@@ -49,10 +55,10 @@ client = HareDo::Client.new()
49
55
  client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
50
56
  service = Service.new($haredo_test_queue)
51
57
 
52
- service.run(false)
58
+ service.run(:blocking => false)
53
59
 
54
60
  1.upto(10) do |i|
55
- client.send($haredo_test_queue, 'data', { :i => i })
61
+ client.send($haredo_test_queue, :headers => { :i => i })
56
62
  msg = @client.receive()
57
63
 
58
64
  dump_message msg
@@ -61,9 +67,166 @@ service.run(false)
61
67
  end
62
68
  ```
63
69
 
64
- The server runs in non-blocking mode. The client makes 10 calls sending 10
65
- monotonically increasing integer values, waits for the response (using a
66
- blocking timeout) after each call and checks the results.
70
+ The service takes a single argument -- the queue name. This is like an email
71
+ address or an IP address. It's an identifier you use to direct messages to the
72
+ service.
73
+
74
+ The client sends 10 messages to the service, each containing monotonically
75
+ increasing integer values. It waits for the response (using a blocking timeout)
76
+ after each call and checks the results.
77
+
78
+ ### Request/Response Pattern
79
+
80
+ While it is not required that a service reply, the requests/response pattern is
81
+ the main function of this library. We are often interested in sending a request
82
+ and getting one (or more responses) back. Implementing this is only a slight
83
+ change from the above example. The following implements a service that responds
84
+ with a single message.
85
+
86
+ The server code is as follows:
87
+
88
+ ```ruby
89
+ #!/usr/bin/env ruby
90
+
91
+ require 'haredo/peer'
92
+
93
+ $rabbitmq_host = 'localhost'
94
+ $rabbitmq_username = 'guest'
95
+ $rabbitmq_password = 'guest'
96
+ $haredo_test_queue = 'HareDo'
97
+
98
+ class Service < HareDo::Service
99
+
100
+ def initialize(name)
101
+ super
102
+ connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
103
+ end
104
+
105
+ def serve(msg)
106
+ reply(msg, :data => 'Reply')
107
+ end
108
+ end
109
+
110
+ Service.new($haredo_test_queue).run()
111
+ ```
112
+
113
+ The client code is a follows:
114
+
115
+ ```ruby
116
+ #!/usr/bin/env ruby
117
+
118
+ require 'haredo/peer'
119
+
120
+ $rabbitmq_host = 'localhost'
121
+ $rabbitmq_username = 'guest'
122
+ $rabbitmq_password = 'guest'
123
+ $haredo_test_queue = 'HareDo'
124
+
125
+ client = HareDo::Client.new()
126
+ client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
127
+ response = @client.call($haredo_test_queue, :data => 'jujifruit')
128
+ ```
129
+
130
+ You can run both in the same script by setting the service code to non-blocking
131
+ (passing <tt>:blocking => false</tt> to the <tt>Service::run()</tt> method):
132
+
133
+ ```ruby
134
+ #!/usr/bin/env ruby
135
+
136
+ require 'haredo/peer'
137
+
138
+ $rabbitmq_host = 'localhost'
139
+ $rabbitmq_username = 'guest'
140
+ $rabbitmq_password = 'guest'
141
+ $haredo_test_queue = 'HareDo'
142
+
143
+ class Service < HareDo::Service
144
+
145
+ def initialize(name)
146
+ super
147
+ connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
148
+ end
149
+
150
+ def serve(msg)
151
+ reply(msg, :data => 'Reply')
152
+ end
153
+ end
154
+
155
+ service = Service.new($haredo_test_queue)
156
+ service.run(:blocking => false)
157
+
158
+ client = HareDo::Client.new()
159
+ client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
160
+ response = @client.call($haredo_test_queue, :data => 'jujifruit')
161
+ ```
162
+
163
+ The client uses the <tt>call()</tt> method to send a message and wait for a
164
+ response. This is just a convenience method that wraps <tt>send()</tt> and
165
+ <tt>receive()</tt>. Interally, a unique message ID is assigned to the request
166
+ message, and the service will send back a reply with the message ID set in the
167
+ message headers.
168
+
169
+ The client will block waiting for the reponse -- up to <tt>client.timeout</tt>
170
+ seconds (floating point value). The default is 1 second. You can change this to
171
+ whatever value you like. If a timeout occurs, the <tt>call()</tt> will return
172
+ <tt>nil</tt>. Subsequent calls will fail to get the response, even if it comes
173
+ in later (it will be discarded).
174
+
175
+ The service uses the <tt>reply()</tt> method rather than the <tt>send()</tt>
176
+ method to return a response. This method takes the original request message,
177
+ extracts the <tt>to</tt> address along with message ID and addresses a message
178
+ back to the client. It adds a header value of <tt>reply=true</tt>. This lets the
179
+ client know that the message coming in is a reply. It needs this because what if
180
+ another peer sent it a message with the same message id? It might interpret that
181
+ message as the reply from the service. Therefore this flag lets the client know
182
+ that the message is a reply to a previous request.
183
+
184
+ Using this simple pattern, you can easily create heterogeneous network services
185
+ in just a few lines of code.
186
+
187
+ ### Roud-Robin Services
188
+
189
+ Say you want to scale your service to three nodes distributed across three
190
+ different machines. The only thing you need to do differently is redefine one
191
+ method -- <tt>Service::createQueue()</tt>. The following is updates the above
192
+ example and makes is available to run in this way:
193
+
194
+ ```ruby
195
+ #!/usr/bin/env ruby
196
+
197
+ require 'haredo/peer'
198
+
199
+ $rabbitmq_host = 'localhost'
200
+ $rabbitmq_username = 'guest'
201
+ $rabbitmq_password = 'guest'
202
+ $haredo_test_queue = 'HareDo'
203
+
204
+ class Service < HareDo::Service
205
+
206
+ def initialize(name)
207
+ super
208
+ connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
209
+ end
210
+
211
+ def createQueue()
212
+ return @channel.queue(@queue_name, :auto_delete => true)
213
+ end
214
+
215
+
216
+ def serve(msg)
217
+ reply(msg, :data => 'Reply')
218
+ end
219
+ end
220
+
221
+ Service.new($haredo_test_queue).run()
222
+ ```
223
+
224
+ The only thing we did is remove the <tt>exclusive</tt> attribute which is by
225
+ default in the <tt>HareDO::Service</tt> base class. By making the queue this
226
+ services uses non-exclusive, mulitple service instances can now connect and
227
+ process requests. RabbitMQ will automatically distribute messages equally across
228
+ all the running services, dividing up the load. You can now scale your service
229
+ to as many machines and processes as you like.
67
230
 
68
231
  ## Installation
69
232
 
@@ -3,18 +3,35 @@ require 'haredo/version'
3
3
 
4
4
  module HareDo
5
5
 
6
+ # This is a simple class that represents a message delivered from a queue. The
7
+ # attributes are as follows:
8
+ #
9
+ # * @info member contains the delivery information
10
+ # * @properties contains the message properties
11
+ # * @headers the message headers
12
+ # * @data the message data
13
+
6
14
  class Message
7
15
 
8
- attr_reader :info, :properties, :data
16
+ attr_reader :info, :properties, :headers, :data
9
17
 
10
18
  def initialize(info=nil, properties=nil, data=nil)
11
19
  @info = info
12
20
  @properties = properties
13
- @data = data
21
+
22
+ if properties
23
+ @headers = properties[:headers]
24
+ else
25
+ @headers = {}
26
+ end
27
+
28
+ @data = data
14
29
  end
15
30
 
16
31
  end
17
32
 
33
+ # This is an abstract base class used for both Client and Service.
34
+
18
35
  class Node
19
36
 
20
37
  attr_reader :queue
@@ -23,40 +40,80 @@ class Node
23
40
 
24
41
  end
25
42
 
43
+ # Connect to RabbitMQ
44
+ #
45
+ # @param user The RabbitMQ username
46
+ # @param password The RabbitMQ password
47
+ # @param host The RabbitMQ host
48
+ # @param port The RabbitMQ port
49
+ # @return Returns true if connection was successful, false othewise.
50
+
26
51
  def connect(user, password, host='localhost', port='5672')
27
- @cnx = Bunny.new("amqp://#{user}:#{password}@#{host}:#{port}")
28
52
 
29
- @cnx.start()
53
+ begin
54
+ @cnx = Bunny.new("amqp://#{user}:#{password}@#{host}:#{port}")
55
+ @cnx.start()
56
+ rescue Bunny::TCPConnectionFailed => e
57
+ return false
58
+ end
30
59
 
31
60
  @channel = @cnx.create_channel()
32
61
 
33
62
  return true
34
63
  end
35
64
 
65
+ # Disconnect from RabbitMQ
36
66
  def disconnect()
37
67
  @cnx.close()
38
68
  end
39
69
 
40
- def send(to, data, headers = {}, from=nil)
70
+ # Sends a message.
71
+ # @param to The to address
72
+ # @param :data Message data
73
+ # @param :headers Message headers
74
+ # @param :headers Message from address
75
+ # @param :properties Message properties
76
+
77
+ def send(to, args)
78
+
79
+ data = args[:data]
80
+ from = args[:from]
81
+ headers = args[:headers] || {}
82
+ properties = args[:properties] || {}
41
83
 
42
- if from.nil?
43
- from = @queue.name if @queue
84
+ properties[:routing_key] = to
85
+ properties[:headers] = headers
86
+
87
+ if not from.nil?
88
+ properties[:reply_to] = from
89
+ else
90
+ properties[:reply_to] = @queue.name if @queue
44
91
  end
45
92
 
46
- @exchange.publish( data,
47
- :routing_key => to,
48
- :headers => headers,
49
- :reply_to => from )
93
+ properties[:message_id] = @mid
94
+
95
+ @exchange.publish(data, properties)
50
96
  end
51
97
 
52
98
  end # class Node
53
99
 
100
+ # Implements a basic client designed for sending messages asynchronously and
101
+ # blocking while receiving replies.
102
+
54
103
  class Client < Node
55
104
 
56
105
  attr_reader :queue
106
+ attr_accessor :timeout, :sleep_interval
57
107
 
58
108
  def initialize()
59
109
  super
110
+
111
+ @timeout = 1.0
112
+ @sleep_interval = 0.001
113
+ @receive_queue = {}
114
+
115
+ # Message identifier
116
+ @mid = 0
60
117
  end
61
118
 
62
119
  def connect(user, password, host='localhost', port='5672')
@@ -68,31 +125,114 @@ class Client < Node
68
125
  @exchange = @channel.default_exchange()
69
126
  end
70
127
 
128
+ # Disconnect from RabbitMQ
71
129
  def disconnect()
72
130
  @queue.delete() if @queue
73
131
  super
74
132
  end
75
133
 
76
- def receive(timeout = 1.0)
77
- now = Time::now.to_f
134
+ # Sends a message. Adds the @mid as message_id in message properties.
135
+ # @param to The to address
136
+ # @param :data Message data
137
+ # @param :headers Message headers
138
+ # @param :headers Message from address
139
+ # @param :properties Message properties
140
+ #
141
+ # @return Returns the message ID of sent message
142
+
143
+ def send(to, args)
144
+
145
+ data = args[:data]
146
+ from = args[:from]
147
+ headers = args[:headers] || {}
148
+ properties = args[:properties] || {}
149
+
150
+ properties[:message_id] = @mid
151
+
152
+ super to, :data => data, :headers => headers, :properties => properties, :from => from
153
+
154
+ rc = @mid
155
+ @mid += 1
156
+
157
+ return rc
158
+ end
78
159
 
160
+ # Sends a message and waits for response
161
+ #
162
+ # @param to The to address
163
+ # @param :data Message data
164
+ # @param :headers Message headers
165
+ # @param :headers Message from address
166
+ # @param :properties Message properties
167
+ #
168
+ # @return Returns the response message if successful, nil otherwise. Will
169
+ # block until @timeout seconds elapse. Sleeps in busywait. Sleep interval is
170
+ # given by @sleep_interval.
171
+
172
+ def call(to, data: '', headers: {}, properties: {}, from: nil)
173
+ # Get message id of sent message
174
+ id = send(to, data: data, headers: headers, properties: properties, from: from)
175
+
176
+ # Put blank entry in queue to indicate we are expecting a response with that
177
+ # message id.
178
+ @receive_queue[id] = nil
179
+
180
+ return receive(id)
181
+ end
182
+
183
+ def receive(mid=nil)
184
+
185
+ if mid != nil
186
+ if @receive_queue.has_key?(mid)
187
+ msg = @receive_queue[mid]
188
+
189
+ if msg != nil
190
+ @receive_queue.delete(mid)
191
+ return msg
192
+ end
193
+ end
194
+ end
195
+
196
+ now = Time::now.to_f
197
+
79
198
  while true
80
199
  delivery_info, properties, payload = @queue.pop()
81
200
 
82
201
  if delivery_info != nil
83
- return Message.new(delivery_info, properties, payload)
202
+ msg = Message.new(delivery_info, properties, payload)
203
+
204
+ if msg.headers.has_key?('id')
205
+ if mid != nil
206
+ if msg.headers['id'] == mid
207
+ # Reply flag must be set
208
+ if msg.headers['reply'] == 1
209
+ return msg
210
+ end
211
+ end
212
+ end
213
+
214
+ # Only add to receive queue if we are expecting it
215
+ if @receive_queue.has_key?
216
+ @receive_queue[msg.headers['id']] = msg
217
+ end
218
+ end
84
219
  end
85
-
86
- if (Time::now.to_f - now) > timeout
87
- return Message.new()
220
+
221
+ if (Time::now.to_f - now) > @timeout
222
+ # Delete entry from receive queue
223
+ @receive_queue.delete mid
224
+
225
+ return nil
88
226
  end
89
227
 
90
- sleep 0.001
228
+ sleep @sleep_interval
91
229
  end
92
230
  end
93
231
 
94
232
  end # class Client
95
233
 
234
+ # Represents a basic service class.
235
+
96
236
  class Service < Node
97
237
 
98
238
  def initialize(name)
@@ -101,19 +241,50 @@ class Service < Node
101
241
  @queue_name = name
102
242
  end
103
243
 
244
+ # Defined the queue this service will listen on. Assumes a single-instance
245
+ # service therefore declares queue as exclusive.
104
246
  def createQueue()
105
247
  return @channel.queue(@queue_name, :auto_delete => true, :exclusive => true)
106
248
  end
107
249
 
108
- def run(block = true)
250
+ # Causes the service to listen for incoming messages.
251
+ #
252
+ # @param :blocking If this is set to true, will go into indefinite blocking
253
+ # loop processing incoming messages.
254
+
255
+ # @returns Returns nil if non-blocking. Never returns if blocking.
256
+
257
+ def run(args)
109
258
  @queue = createQueue()
110
259
  @exchange = @channel.default_exchange()
111
260
 
261
+ block = args[:blocking] || false
262
+
112
263
  queue.subscribe(:block => block) do |info, props, data|
113
264
  serve Message.new(info, props, data)
114
265
  end
115
266
  end
116
267
 
268
+ # You should use this method to reply back to a peer. It sets the reply header
269
+ # which tells the remote that this message is a response (as opposed to a
270
+ # message originating from another source which just happens to have the same
271
+ # message_id).
272
+ def reply(msg, args)
273
+
274
+ data = args[:data]
275
+ headers = args[:headers] || {}
276
+
277
+ id = msg.properties.message_id.to_i
278
+ to = msg.properties.reply_to
279
+
280
+ # Set the reply flag to indicate that this is a response to a message
281
+ # sent. The message_id should already be set in the headers.
282
+ headers[:reply] = 1
283
+ headers[:id] = id.to_i
284
+
285
+ send(to, :headers => headers, :data => data)
286
+ end
287
+
117
288
  end # class Service
118
289
 
119
290
  end # module HareDo
@@ -1,5 +1,5 @@
1
1
  module HareDo
2
2
 
3
- VERSION = '0.0.1-1'
3
+ VERSION = '0.0.2-1'
4
4
 
5
5
  end # module HareDo
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: haredo
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
4
+ version: 0.0.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mike Owens
@@ -9,7 +9,21 @@ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
11
  date: 2013-09-19 00:00:00.000000000 Z
12
- dependencies: []
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: bunny
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - '>='
18
+ - !ruby/object:Gem::Version
19
+ version: '0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - '>='
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
13
27
  description:
14
28
  email: mikeowens@gmail.com
15
29
  executables: []