haredo 0.0.2 → 0.0.3

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 85222fb487a3ac64174d10d323211ba37ddf427b
4
- data.tar.gz: 758c7117f150c96b882132d14d38e2e0495ea080
3
+ metadata.gz: 3a240cc2da3eb83e870575dfd20ddeb87f7fe7eb
4
+ data.tar.gz: b0285cbf74adfa83800c08c3cd06b7d841c52a07
5
5
  SHA512:
6
- metadata.gz: ac949da33da127bdf7b1b587c346349a5abdb8e5ccf30c4bc18c57613fa1729c50b2776bad73357ae3852f326e906870daecacacc745ed0006f42f4bab6f9141
7
- data.tar.gz: 2a5d9416907828acbcdf9d6a4e8259b56b06d82025eb4901e2acecbd39488a41f0ef4a32af4b20258c54e45b8ab17e3503fe56d9da8e0f10681d83e3e3987f83
6
+ metadata.gz: 192aaeeaa0eb4e7e22f881c29fe60b793b8b1769565b18a5311a13baaab7826f4f4123777c90711315fcb068c94ddc1a35cf57341ec41687cd4daf9c164d8d5d
7
+ data.tar.gz: 38dbeacc3a6073722226b37c2d98b62c1e9a66a48f30b2c58c07b45805b4d4f68d7dd73f4a3f5447f41731a492150c22ec0a3e9d8a5a085b64bfb2f82d1295b3
data/README.md CHANGED
@@ -3,36 +3,53 @@
3
3
  ## About
4
4
 
5
5
  This is an easy-to-use framework for creating peer-to-peer applications in Ruby
6
- via [RabbitMQ](http://www.rabbitmq.com/). While RabbitMQ is a robust, feature
7
- rich message broker, it (like all AMQP message brokers) uses a somewhat complex
8
- terminology that can be a little bewildering to to newcomers of AMQP.
9
-
10
- This gem -- built atop the [Bunny AMQP client](http://rubybunny.info/) -- uses a
11
- small subset of RabbitMQ's capabilities to provide an intuitive client/server
12
- framework for easily and quickly implementing network services and simple peer
13
- to peer applications. It specifically designed for those who are not familiar
14
- with the ins and outs of the AMQP protocol.
6
+ via [RabbitMQ](http://www.rabbitmq.com/). Built atop the [Bunny AMQP
7
+ client](http://rubybunny.info/), it uses a small subset of RabbitMQ's
8
+ capabilities to provide an intuitive client/server framework for implementing
9
+ network services and simple peer to peer applications. It is specifically
10
+ designed for those who are not familiar with the ins and outs of the AMQP
11
+ protocol and employs RabbitMQ's default AMQP exchange to implement direct peer
12
+ to peer communication.
13
+
14
+ This framework's design targets a common case within web applications and other
15
+ miscellaneous contexts (logging, event handling, data retrieval, etc.), when you
16
+ just need to send a request somewhere and get a reply back. In this case
17
+ specifically:
18
+
19
+ * Clients send messages asynchronously (to one or more peers) and can receive
20
+ synchronously (blocking with a configurable timeout).
21
+ * Services run indefinitely either within the same process (in separate
22
+ threads), or separate processes.
23
+
24
+ In summary, this gem employs a small, practical and useful subset of the vast
25
+ array of features available within RabbitMQ to make it easy to whip up network
26
+ services quickly and easily without having to master all aspects of RabbitMQ or
27
+ the AMQP protocol.
15
28
 
16
29
  ## How It Works
17
30
 
18
- Here is a complete example -- the following program creates a simple network
19
- service that takes a number and add one, along with a client that uses it. They
20
- both run simultaneously in the same script.
31
+ The easiest way to explain how it works is by example. The following program
32
+ illustrates creating a network service and a client to talk to it. The service
33
+ simply takes a number from the message headers, adds one to it and sends the
34
+ result back.
35
+
36
+ In this example, both client and service run simultaneously in the same
37
+ script. However, services can run in separate scripts just as easily (more on
38
+ this below).
21
39
 
22
40
  ```ruby
23
41
  #!/usr/bin/env ruby
24
42
 
25
- $rabbitmq_host = 'localhost'
26
- $rabbitmq_username = 'guest'
27
- $rabbitmq_password = 'guest'
28
- $haredo_test_queue = 'HareDo'
43
+ $mq_host = 'localhost'
44
+ $mq_username = 'guest'
45
+ $mq_password = 'guest'
46
+ $queue = 'HareDo'
29
47
 
30
- # Simple service
31
48
  class Service < HareDo::Service
32
49
 
33
50
  def initialize(name)
34
51
  super
35
- connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
52
+ connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
36
53
  end
37
54
 
38
55
  def serve(msg)
@@ -40,74 +57,85 @@ class Service < HareDo::Service
40
57
  send(msg.properties.reply_to, :data => data.to_s)
41
58
  end
42
59
 
43
- end # class Server
44
-
45
- def dump_message(msg)
46
- puts 'Headers:'
47
- msg.properties.each do |k,v|
48
- puts " #{k}: #{v}"
49
- end
50
-
51
- puts "Data: #{msg.data}"
52
60
  end
53
61
 
54
- client = HareDo::Client.new()
55
- client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
56
- service = Service.new($haredo_test_queue)
57
-
62
+ service = Service.new($queue)
58
63
  service.run(:blocking => false)
59
64
 
65
+ client = HareDo::Client.new()
66
+ client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
67
+
60
68
  1.upto(10) do |i|
61
- client.send($haredo_test_queue, :headers => { :i => i })
69
+ client.send($queue, :headers => { :i => i })
62
70
  msg = @client.receive()
63
-
64
- dump_message msg
65
71
 
66
72
  puts msg.data.to_i == i + 1
67
73
  end
68
74
  ```
69
75
 
70
- The service takes a single argument -- the queue name. This is like an email
71
- address or an IP address. It's an identifier you use to direct messages to the
72
- service.
76
+ The service takes a single argument -- the a name. This name is the name of the
77
+ AMQP queue used to send messages to the service. A queue is like an email
78
+ address, phone number or an IP address -- it's just a unique identifier. In this
79
+ case, its identifer you use to direct messages to the service. The run method
80
+ then causes the servers to listen on the queue.
73
81
 
74
82
  The client sends 10 messages to the service, each containing monotonically
75
83
  increasing integer values. It waits for the response (using a blocking timeout)
76
84
  after each call and checks the results.
77
85
 
78
- ### Request/Response Pattern
79
-
80
- While it is not required that a service reply, the requests/response pattern is
81
- the main function of this library. We are often interested in sending a request
82
- and getting one (or more responses) back. Implementing this is only a slight
83
- change from the above example. The following implements a service that responds
84
- with a single message.
85
-
86
- The server code is as follows:
86
+ The client is under no obligation to wait for a response. This is just how we've
87
+ coded the service in this example -- to send something back. The interaction
88
+ between the two forms an ad-hoc protocol. Hence the point of this project --
89
+ quick and easy network protocols with minimal code (thanks to Rabbit and Bunny).
90
+
91
+ ## Clients
92
+
93
+ There are three basic use-cases the client addresses:
94
+
95
+ * Event Dispatch: Client sends one or messages expecting a response(s). This is
96
+ the case when you want to log events -- you don't really care about getting a
97
+ response back. You just want to send out the event and be done with it.
98
+ * Send/Receive: Client sends one or more messages expecting response and
99
+ processes those responses in no particular order. That is, there is no need to
100
+ correlate a response with its request, it can be processed in its own
101
+ right. This could be like loading a set of images or documents from a remote
102
+ service. When the document arrives, you don't really care about the request
103
+ used to obtain it. All you care about is the document itself. In this case, a
104
+ client could send out 10 messages asynchronously and then wait in a loop for
105
+ each to come back.
106
+ * Remote Procedure Call (RPC): Client sends one or more messages expecting
107
+ response and needs to processes those responses in a specific order. That is,
108
+ it needs to correlate the response to its associated request. This is like
109
+ calling a function. You need to know the specific like a return value for the
110
+ specific function you called.
111
+
112
+ Event dispatch is simple -- you don't send back a reply from the service, nor do
113
+ you expect a response on the client-side. Take a simple logging service for
114
+ example. Here is the server code:
87
115
 
88
116
  ```ruby
89
117
  #!/usr/bin/env ruby
90
118
 
91
119
  require 'haredo/peer'
92
120
 
93
- $rabbitmq_host = 'localhost'
94
- $rabbitmq_username = 'guest'
95
- $rabbitmq_password = 'guest'
96
- $haredo_test_queue = 'HareDo'
121
+ $mq_host = 'localhost'
122
+ $mq_username = 'guest'
123
+ $mq_password = 'guest'
97
124
 
98
- class Service < HareDo::Service
125
+ class Logger < HareDo::Service
99
126
 
100
127
  def initialize(name)
101
128
  super
102
- connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
129
+ connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
103
130
  end
104
131
 
105
132
  def serve(msg)
106
- reply(msg, :data => 'Reply')
133
+ # Log the message somewhere
107
134
  end
135
+
108
136
  end
109
137
 
110
- Service.new($haredo_test_queue).run()
138
+ Service.new('logger').run()
111
139
  ```
112
140
 
113
141
  The client code is a follows:
@@ -117,34 +145,82 @@ The client code is a follows:
117
145
 
118
146
  require 'haredo/peer'
119
147
 
120
- $rabbitmq_host = 'localhost'
121
- $rabbitmq_username = 'guest'
122
- $rabbitmq_password = 'guest'
123
- $haredo_test_queue = 'HareDo'
148
+ $mq_host = 'localhost'
149
+ $mq_username = 'guest'
150
+ $mq_password = 'guest'
124
151
 
125
152
  client = HareDo::Client.new()
126
- client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
127
- response = @client.call($haredo_test_queue, :data => 'jujifruit')
153
+ client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
154
+
155
+ client.send('logger', :data => 'Backup script failed')
128
156
  ```
129
157
 
130
158
  You can run both in the same script by setting the service code to non-blocking
131
- (passing <tt>:blocking => false</tt> to the <tt>Service::run()</tt> method):
159
+ (passing <tt>:blocking => false</tt> to the <tt>Service::run()</tt>
160
+ method). Services in either blocking or non-blocking mode, enabling you to run
161
+ them either in the same script as clients (non-blocking), or in separate scripts
162
+ where they idle and wait for incoming messages (blocking). By default, services
163
+ run in blocking mode, as it is assumed that they will be be run in their own
164
+ independent scripts/processes.
165
+
166
+ ```ruby
167
+ #!/usr/bin/env ruby
168
+
169
+ require 'haredo/peer'
170
+
171
+ $mq_host = 'localhost'
172
+ $mq_username = 'guest'
173
+ $mq_password = 'guest'
174
+
175
+ class Logger < HareDo::Service
176
+
177
+ def initialize(name)
178
+ super
179
+ connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
180
+ end
181
+
182
+ def serve(msg)
183
+ # Log the data somewhere
184
+ end
185
+
186
+ end
187
+
188
+ Service.new('logger').run(:blocking=>false)
189
+
190
+ client = HareDo::Client.new()
191
+ client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
192
+ client.send('logger', :data => 'Backup script failed')
193
+ ```
194
+
195
+ ### Send/Receive
196
+
197
+ The send/receive pattern is illustrated in the initial example -- the client
198
+ sends out ten integers and wait for the response. While the responses come back
199
+ in the correct order, there is no guarantee that this will happen every time. So
200
+ technically, the client example could break here. What the client needs here is
201
+ to correlate the response with the request. This is the RPC pattern.
202
+
203
+ ### Remote Procedure Call
204
+
205
+ This is the purpose of the <tt>Client::call()</tt> method. It assigns a specific
206
+ ID to the outgoing message and waits for a response back containing that message
207
+ ID. So <tt>call()</tt> acts like a synchronous function call. It sends out a
208
+ request and blocks waiting for a specific reply. Here is an example:
132
209
 
133
210
  ```ruby
134
211
  #!/usr/bin/env ruby
135
212
 
136
213
  require 'haredo/peer'
137
214
 
138
- $rabbitmq_host = 'localhost'
139
- $rabbitmq_username = 'guest'
140
- $rabbitmq_password = 'guest'
141
- $haredo_test_queue = 'HareDo'
215
+ $mq_host = 'localhost'
216
+ $mq_username = 'guest'
217
+ $mq_password = 'guest'
142
218
 
143
219
  class Service < HareDo::Service
144
220
 
145
221
  def initialize(name)
146
222
  super
147
- connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
223
+ connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
148
224
  end
149
225
 
150
226
  def serve(msg)
@@ -152,73 +228,91 @@ class Service < HareDo::Service
152
228
  end
153
229
  end
154
230
 
155
- service = Service.new($haredo_test_queue)
231
+ service = Service.new('rpc')
156
232
  service.run(:blocking => false)
157
233
 
158
234
  client = HareDo::Client.new()
159
- client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
160
- response = @client.call($haredo_test_queue, :data => 'jujifruit')
235
+ client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
236
+ response = @client.call('rpc', :data => 'jujifruit')
161
237
  ```
162
238
 
163
- The client uses the <tt>call()</tt> method to send a message and wait for a
164
- response. This is just a convenience method that wraps <tt>send()</tt> and
165
- <tt>receive()</tt>. Interally, a unique message ID is assigned to the request
166
- message, and the service will send back a reply with the message ID set in the
167
- message headers.
168
-
169
- The client will block waiting for the reponse -- up to <tt>client.timeout</tt>
170
- seconds (floating point value). The default is 1 second. You can change this to
171
- whatever value you like. If a timeout occurs, the <tt>call()</tt> will return
172
- <tt>nil</tt>. Subsequent calls will fail to get the response, even if it comes
173
- in later (it will be discarded).
174
-
175
- The service uses the <tt>reply()</tt> method rather than the <tt>send()</tt>
176
- method to return a response. This method takes the original request message,
177
- extracts the <tt>to</tt> address along with message ID and addresses a message
178
- back to the client. It adds a header value of <tt>reply=true</tt>. This lets the
179
- client know that the message coming in is a reply. It needs this because what if
180
- another peer sent it a message with the same message id? It might interpret that
181
- message as the reply from the service. Therefore this flag lets the client know
182
- that the message is a reply to a previous request.
183
-
184
- Using this simple pattern, you can easily create heterogeneous network services
185
- in just a few lines of code.
186
-
187
- ### Roud-Robin Services
188
-
189
- Say you want to scale your service to three nodes distributed across three
190
- different machines. The only thing you need to do differently is redefine one
191
- method -- <tt>Service::createQueue()</tt>. The following is updates the above
192
- example and makes is available to run in this way:
239
+ The <tt>call()</tt> method will block waiting for the reponse up to
240
+ <tt>client.timeout</tt> seconds (floating point value). The default is 1
241
+ second. You can change this to whatever value you like. If a timeout occurs,
242
+ <tt>call()</tt> will return <tt>nil</tt>.
243
+
244
+ The <tt>call()</tt> method is just a convenience method that wraps
245
+ <tt>send()</tt> and <tt>receive()</tt>. Interally, <tt>send()</tt> always
246
+ assigns a unique message ID to each outgoing message. It passes this ID back as
247
+ the return value. You can use this ID to fetch specific messages from
248
+ <tt>receive()</tt>.
249
+
250
+ Normally <tt>receive()</tt> just returns the oldest message to
251
+ arrive. Internally is keeps a queue of messages as they are recieved. When you
252
+ call it with no arguments, it just pops the oldest message off the queue (if
253
+ there are any), otherwise it blocks waiting for a message to come in until the
254
+ timeout interval expires. However, if you pass an ID to <tt>receive()</tt>, it
255
+ will look specifically for a message with that ID. If none is found, again, it
256
+ will block for the timeout interval waiting for it to come it. If none does, it
257
+ returns <tt>nil</tt>.
258
+
259
+ If <tt>call()</tt> times-out, subsequent calls will fail to get the response,
260
+ even if it comes in later as it will be discarded. This keeps dilatory messages
261
+ from filling up the receive queue. Once you've given up on a message, it makes
262
+ no sense to keep it around should it arrive sometime in the future.
263
+
264
+ ### The Service Side
265
+
266
+ Notice that the service uses the <tt>Service::reply()</tt> method rather than
267
+ the <tt>send()</tt> method to return a response. This method takes the original
268
+ request message, extracts the <tt>to</tt> address along with message ID and
269
+ addresses a message back to the client. It also adds a header value of
270
+ <tt>reply=true</tt>. This header lets the client know that the message coming in
271
+ is specifically a reply. This is needed this because what if another peer sent
272
+ the client a message which happened to have the same message ID? The client
273
+ would confuse that message as the reply from the service and that would mess
274
+ things up fast. Therefore the <tt>reply<tt> flag lets the client know that the
275
+ message is a reply to a previous request and can then use the message ID to
276
+ correlate it with the original request.
277
+
278
+ ## Services
279
+
280
+ Most of what needs to be said about services has be covered. However there is
281
+ one important feature RabbitMQ accords you which we need to address.
282
+
283
+ Say you want to scale your service from just a single thread or process and
284
+ distribute accross multiple machines which are perhaps even on different
285
+ networks around the world. You can enable this by modifying a single field. The
286
+ only thing you need to do differently is redefine one method --
287
+ <tt>Service::createQueue()</tt>. The following is updates the above example and
288
+ makes is available to run in this way:
193
289
 
194
290
  ```ruby
195
291
  #!/usr/bin/env ruby
196
292
 
197
293
  require 'haredo/peer'
198
294
 
199
- $rabbitmq_host = 'localhost'
200
- $rabbitmq_username = 'guest'
201
- $rabbitmq_password = 'guest'
202
- $haredo_test_queue = 'HareDo'
295
+ $mq_host = 'localhost'
296
+ $mq_username = 'guest'
297
+ $mq_password = 'guest'
203
298
 
204
299
  class Service < HareDo::Service
205
300
 
206
301
  def initialize(name)
207
302
  super
208
- connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
303
+ connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
209
304
  end
210
305
 
211
306
  def createQueue()
212
307
  return @channel.queue(@queue_name, :auto_delete => true)
213
308
  end
214
309
 
215
-
216
310
  def serve(msg)
217
311
  reply(msg, :data => 'Reply')
218
312
  end
219
313
  end
220
314
 
221
- Service.new($haredo_test_queue).run()
315
+ Service.new('rpc').run()
222
316
  ```
223
317
 
224
318
  The only thing we did is remove the <tt>exclusive</tt> attribute which is by
@@ -42,22 +42,35 @@ class Node
42
42
 
43
43
  # Connect to RabbitMQ
44
44
  #
45
- # @param user The RabbitMQ username
46
- # @param password The RabbitMQ password
47
- # @param host The RabbitMQ host
48
- # @param port The RabbitMQ port
45
+ # @param user The RabbitMQ username (default guest)
46
+ # @param password The RabbitMQ password (default guest)
47
+ # @param host The RabbitMQ host (default localhost)
48
+ # @param port The RabbitMQ port (default 5672)
49
+ # @param vhost The RabbitMQ vhost (default /)
50
+ # @param ssl Use SSL (default false)
49
51
  # @return Returns true if connection was successful, false othewise.
50
52
 
51
- def connect(user, password, host='localhost', port='5672')
53
+ def connect(args)
54
+
55
+ user = args[:user] || 'guest'
56
+ password = args[:password] || 'guest'
57
+ host = args[:host] || 'localhost'
58
+ port = args[:post] || '5672'
59
+ vhost = args[:vhost] || ''
60
+ ssl = ''
61
+
62
+ if args[:ssl] == true
63
+ ssl = 's'
64
+ end
52
65
 
53
66
  begin
54
- @cnx = Bunny.new("amqp://#{user}:#{password}@#{host}:#{port}")
67
+ @cnx = Bunny.new("amqp#{ssl}://#{user}:#{password}@#{host}:#{port}#{vhost}")
55
68
  @cnx.start()
56
69
  rescue Bunny::TCPConnectionFailed => e
57
70
  return false
58
71
  end
59
72
 
60
- @channel = @cnx.create_channel()
73
+ @channel = @cnx.create_channel()
61
74
 
62
75
  return true
63
76
  end
@@ -116,13 +129,17 @@ class Client < Node
116
129
  @mid = 0
117
130
  end
118
131
 
119
- def connect(user, password, host='localhost', port='5672')
120
- super
132
+ def connect(args)
133
+ ret = super
121
134
 
122
- @queue = @channel.queue( '', :auto_delete => true,
123
- :arguments => { "x-message-ttl" => 1000 } )
135
+ if ret == true
136
+ @queue = @channel.queue( '', :auto_delete => true,
137
+ :arguments => { "x-message-ttl" => 1000 } )
124
138
 
125
- @exchange = @channel.default_exchange()
139
+ @exchange = @channel.default_exchange()
140
+ end
141
+
142
+ return ret
126
143
  end
127
144
 
128
145
  # Disconnect from RabbitMQ
@@ -235,10 +252,14 @@ end # class Client
235
252
 
236
253
  class Service < Node
237
254
 
255
+ # @param name The name of the queue the service will attempt to bind to.
238
256
  def initialize(name)
239
257
  super()
240
258
 
241
259
  @queue_name = name
260
+
261
+ # The number of messages to prefecth from Rabbit
262
+ @prefetch = 10
242
263
  end
243
264
 
244
265
  # Defined the queue this service will listen on. Assumes a single-instance
@@ -260,7 +281,10 @@ class Service < Node
260
281
 
261
282
  block = args[:blocking] || false
262
283
 
263
- queue.subscribe(:block => block) do |info, props, data|
284
+ @channel.prefetch(@prefetch)
285
+
286
+ queue.subscribe(:block => block, :ack => true) do |info, props, data|
287
+ @channel.acknowledge(info.delivery_tag, false)
264
288
  serve Message.new(info, props, data)
265
289
  end
266
290
  end
@@ -1,5 +1,5 @@
1
1
  module HareDo
2
2
 
3
- VERSION = '0.0.2-1'
3
+ VERSION = '0.0.3-1'
4
4
 
5
5
  end # module HareDo
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: haredo
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.2
4
+ version: 0.0.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mike Owens
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2013-09-19 00:00:00.000000000 Z
11
+ date: 2013-09-23 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bunny