haredo 0.0.2 → 0.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +200 -106
- data/src/lib/haredo/peer.rb +37 -13
- data/src/lib/haredo/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3a240cc2da3eb83e870575dfd20ddeb87f7fe7eb
|
4
|
+
data.tar.gz: b0285cbf74adfa83800c08c3cd06b7d841c52a07
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 192aaeeaa0eb4e7e22f881c29fe60b793b8b1769565b18a5311a13baaab7826f4f4123777c90711315fcb068c94ddc1a35cf57341ec41687cd4daf9c164d8d5d
|
7
|
+
data.tar.gz: 38dbeacc3a6073722226b37c2d98b62c1e9a66a48f30b2c58c07b45805b4d4f68d7dd73f4a3f5447f41731a492150c22ec0a3e9d8a5a085b64bfb2f82d1295b3
|
data/README.md
CHANGED
@@ -3,36 +3,53 @@
|
|
3
3
|
## About
|
4
4
|
|
5
5
|
This is an easy-to-use framework for creating peer-to-peer applications in Ruby
|
6
|
-
via [RabbitMQ](http://www.rabbitmq.com/).
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
6
|
+
via [RabbitMQ](http://www.rabbitmq.com/). Built atop the [Bunny AMQP
|
7
|
+
client](http://rubybunny.info/), it uses a small subset of RabbitMQ's
|
8
|
+
capabilities to provide an intuitive client/server framework for implementing
|
9
|
+
network services and simple peer to peer applications. It is specifically
|
10
|
+
designed for those who are not familiar with the ins and outs of the AMQP
|
11
|
+
protocol and employs RabbitMQ's default AMQP exchange to implement direct peer
|
12
|
+
to peer communication.
|
13
|
+
|
14
|
+
This framework's design targets a common case within web applications and other
|
15
|
+
miscellaneous contexts (logging, event handling, data retrieval, etc.), when you
|
16
|
+
just need to send a request somewhere and get a reply back. In this case
|
17
|
+
specifically:
|
18
|
+
|
19
|
+
* Clients send messages asynchronously (to one or more peers) and can receive
|
20
|
+
synchronously (blocking with a configurable timeout).
|
21
|
+
* Services run indefinitely either within the same process (in separate
|
22
|
+
threads), or separate processes.
|
23
|
+
|
24
|
+
In summary, this gem employs a small, practical and useful subset of the vast
|
25
|
+
array of features available within RabbitMQ to make it easy to whip up network
|
26
|
+
services quickly and easily without having to master all aspects of RabbitMQ or
|
27
|
+
the AMQP protocol.
|
15
28
|
|
16
29
|
## How It Works
|
17
30
|
|
18
|
-
|
19
|
-
|
20
|
-
|
31
|
+
The easiest way to explain how it works is by example. The following program
|
32
|
+
illustrates creating a network service and a client to talk to it. The service
|
33
|
+
simply takes a number from the message headers, adds one to it and sends the
|
34
|
+
result back.
|
35
|
+
|
36
|
+
In this example, both client and service run simultaneously in the same
|
37
|
+
script. However, services can run in separate scripts just as easily (more on
|
38
|
+
this below).
|
21
39
|
|
22
40
|
```ruby
|
23
41
|
#!/usr/bin/env ruby
|
24
42
|
|
25
|
-
$
|
26
|
-
$
|
27
|
-
$
|
28
|
-
$
|
43
|
+
$mq_host = 'localhost'
|
44
|
+
$mq_username = 'guest'
|
45
|
+
$mq_password = 'guest'
|
46
|
+
$queue = 'HareDo'
|
29
47
|
|
30
|
-
# Simple service
|
31
48
|
class Service < HareDo::Service
|
32
49
|
|
33
50
|
def initialize(name)
|
34
51
|
super
|
35
|
-
connect(
|
52
|
+
connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
36
53
|
end
|
37
54
|
|
38
55
|
def serve(msg)
|
@@ -40,74 +57,85 @@ class Service < HareDo::Service
|
|
40
57
|
send(msg.properties.reply_to, :data => data.to_s)
|
41
58
|
end
|
42
59
|
|
43
|
-
end # class Server
|
44
|
-
|
45
|
-
def dump_message(msg)
|
46
|
-
puts 'Headers:'
|
47
|
-
msg.properties.each do |k,v|
|
48
|
-
puts " #{k}: #{v}"
|
49
|
-
end
|
50
|
-
|
51
|
-
puts "Data: #{msg.data}"
|
52
60
|
end
|
53
61
|
|
54
|
-
|
55
|
-
client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
|
56
|
-
service = Service.new($haredo_test_queue)
|
57
|
-
|
62
|
+
service = Service.new($queue)
|
58
63
|
service.run(:blocking => false)
|
59
64
|
|
65
|
+
client = HareDo::Client.new()
|
66
|
+
client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
67
|
+
|
60
68
|
1.upto(10) do |i|
|
61
|
-
client.send($
|
69
|
+
client.send($queue, :headers => { :i => i })
|
62
70
|
msg = @client.receive()
|
63
|
-
|
64
|
-
dump_message msg
|
65
71
|
|
66
72
|
puts msg.data.to_i == i + 1
|
67
73
|
end
|
68
74
|
```
|
69
75
|
|
70
|
-
The service takes a single argument -- the
|
71
|
-
|
72
|
-
|
76
|
+
The service takes a single argument -- the a name. This name is the name of the
|
77
|
+
AMQP queue used to send messages to the service. A queue is like an email
|
78
|
+
address, phone number or an IP address -- it's just a unique identifier. In this
|
79
|
+
case, its identifer you use to direct messages to the service. The run method
|
80
|
+
then causes the servers to listen on the queue.
|
73
81
|
|
74
82
|
The client sends 10 messages to the service, each containing monotonically
|
75
83
|
increasing integer values. It waits for the response (using a blocking timeout)
|
76
84
|
after each call and checks the results.
|
77
85
|
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
|
86
|
+
The client is under no obligation to wait for a response. This is just how we've
|
87
|
+
coded the service in this example -- to send something back. The interaction
|
88
|
+
between the two forms an ad-hoc protocol. Hence the point of this project --
|
89
|
+
quick and easy network protocols with minimal code (thanks to Rabbit and Bunny).
|
90
|
+
|
91
|
+
## Clients
|
92
|
+
|
93
|
+
There are three basic use-cases the client addresses:
|
94
|
+
|
95
|
+
* Event Dispatch: Client sends one or messages expecting a response(s). This is
|
96
|
+
the case when you want to log events -- you don't really care about getting a
|
97
|
+
response back. You just want to send out the event and be done with it.
|
98
|
+
* Send/Receive: Client sends one or more messages expecting response and
|
99
|
+
processes those responses in no particular order. That is, there is no need to
|
100
|
+
correlate a response with its request, it can be processed in its own
|
101
|
+
right. This could be like loading a set of images or documents from a remote
|
102
|
+
service. When the document arrives, you don't really care about the request
|
103
|
+
used to obtain it. All you care about is the document itself. In this case, a
|
104
|
+
client could send out 10 messages asynchronously and then wait in a loop for
|
105
|
+
each to come back.
|
106
|
+
* Remote Procedure Call (RPC): Client sends one or more messages expecting
|
107
|
+
response and needs to processes those responses in a specific order. That is,
|
108
|
+
it needs to correlate the response to its associated request. This is like
|
109
|
+
calling a function. You need to know the specific like a return value for the
|
110
|
+
specific function you called.
|
111
|
+
|
112
|
+
Event dispatch is simple -- you don't send back a reply from the service, nor do
|
113
|
+
you expect a response on the client-side. Take a simple logging service for
|
114
|
+
example. Here is the server code:
|
87
115
|
|
88
116
|
```ruby
|
89
117
|
#!/usr/bin/env ruby
|
90
118
|
|
91
119
|
require 'haredo/peer'
|
92
120
|
|
93
|
-
$
|
94
|
-
$
|
95
|
-
$
|
96
|
-
$haredo_test_queue = 'HareDo'
|
121
|
+
$mq_host = 'localhost'
|
122
|
+
$mq_username = 'guest'
|
123
|
+
$mq_password = 'guest'
|
97
124
|
|
98
|
-
class
|
125
|
+
class Logger < HareDo::Service
|
99
126
|
|
100
127
|
def initialize(name)
|
101
128
|
super
|
102
|
-
connect(
|
129
|
+
connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
103
130
|
end
|
104
131
|
|
105
132
|
def serve(msg)
|
106
|
-
|
133
|
+
# Log the message somewhere
|
107
134
|
end
|
135
|
+
|
108
136
|
end
|
109
137
|
|
110
|
-
Service.new(
|
138
|
+
Service.new('logger').run()
|
111
139
|
```
|
112
140
|
|
113
141
|
The client code is a follows:
|
@@ -117,34 +145,82 @@ The client code is a follows:
|
|
117
145
|
|
118
146
|
require 'haredo/peer'
|
119
147
|
|
120
|
-
$
|
121
|
-
$
|
122
|
-
$
|
123
|
-
$haredo_test_queue = 'HareDo'
|
148
|
+
$mq_host = 'localhost'
|
149
|
+
$mq_username = 'guest'
|
150
|
+
$mq_password = 'guest'
|
124
151
|
|
125
152
|
client = HareDo::Client.new()
|
126
|
-
client.connect(
|
127
|
-
|
153
|
+
client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
154
|
+
|
155
|
+
client.send('logger', :data => 'Backup script failed')
|
128
156
|
```
|
129
157
|
|
130
158
|
You can run both in the same script by setting the service code to non-blocking
|
131
|
-
(passing <tt>:blocking => false</tt> to the <tt>Service::run()</tt>
|
159
|
+
(passing <tt>:blocking => false</tt> to the <tt>Service::run()</tt>
|
160
|
+
method). Services in either blocking or non-blocking mode, enabling you to run
|
161
|
+
them either in the same script as clients (non-blocking), or in separate scripts
|
162
|
+
where they idle and wait for incoming messages (blocking). By default, services
|
163
|
+
run in blocking mode, as it is assumed that they will be be run in their own
|
164
|
+
independent scripts/processes.
|
165
|
+
|
166
|
+
```ruby
|
167
|
+
#!/usr/bin/env ruby
|
168
|
+
|
169
|
+
require 'haredo/peer'
|
170
|
+
|
171
|
+
$mq_host = 'localhost'
|
172
|
+
$mq_username = 'guest'
|
173
|
+
$mq_password = 'guest'
|
174
|
+
|
175
|
+
class Logger < HareDo::Service
|
176
|
+
|
177
|
+
def initialize(name)
|
178
|
+
super
|
179
|
+
connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
180
|
+
end
|
181
|
+
|
182
|
+
def serve(msg)
|
183
|
+
# Log the data somewhere
|
184
|
+
end
|
185
|
+
|
186
|
+
end
|
187
|
+
|
188
|
+
Service.new('logger').run(:blocking=>false)
|
189
|
+
|
190
|
+
client = HareDo::Client.new()
|
191
|
+
client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
192
|
+
client.send('logger', :data => 'Backup script failed')
|
193
|
+
```
|
194
|
+
|
195
|
+
### Send/Receive
|
196
|
+
|
197
|
+
The send/receive pattern is illustrated in the initial example -- the client
|
198
|
+
sends out ten integers and wait for the response. While the responses come back
|
199
|
+
in the correct order, there is no guarantee that this will happen every time. So
|
200
|
+
technically, the client example could break here. What the client needs here is
|
201
|
+
to correlate the response with the request. This is the RPC pattern.
|
202
|
+
|
203
|
+
### Remote Procedure Call
|
204
|
+
|
205
|
+
This is the purpose of the <tt>Client::call()</tt> method. It assigns a specific
|
206
|
+
ID to the outgoing message and waits for a response back containing that message
|
207
|
+
ID. So <tt>call()</tt> acts like a synchronous function call. It sends out a
|
208
|
+
request and blocks waiting for a specific reply. Here is an example:
|
132
209
|
|
133
210
|
```ruby
|
134
211
|
#!/usr/bin/env ruby
|
135
212
|
|
136
213
|
require 'haredo/peer'
|
137
214
|
|
138
|
-
$
|
139
|
-
$
|
140
|
-
$
|
141
|
-
$haredo_test_queue = 'HareDo'
|
215
|
+
$mq_host = 'localhost'
|
216
|
+
$mq_username = 'guest'
|
217
|
+
$mq_password = 'guest'
|
142
218
|
|
143
219
|
class Service < HareDo::Service
|
144
220
|
|
145
221
|
def initialize(name)
|
146
222
|
super
|
147
|
-
connect(
|
223
|
+
connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
148
224
|
end
|
149
225
|
|
150
226
|
def serve(msg)
|
@@ -152,73 +228,91 @@ class Service < HareDo::Service
|
|
152
228
|
end
|
153
229
|
end
|
154
230
|
|
155
|
-
service = Service.new(
|
231
|
+
service = Service.new('rpc')
|
156
232
|
service.run(:blocking => false)
|
157
233
|
|
158
234
|
client = HareDo::Client.new()
|
159
|
-
client.connect(
|
160
|
-
response = @client.call(
|
235
|
+
client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
236
|
+
response = @client.call('rpc', :data => 'jujifruit')
|
161
237
|
```
|
162
238
|
|
163
|
-
The
|
164
|
-
|
165
|
-
|
166
|
-
|
167
|
-
|
168
|
-
|
169
|
-
|
170
|
-
|
171
|
-
|
172
|
-
<tt>
|
173
|
-
|
174
|
-
|
175
|
-
|
176
|
-
|
177
|
-
|
178
|
-
|
179
|
-
|
180
|
-
|
181
|
-
|
182
|
-
|
183
|
-
|
184
|
-
|
185
|
-
|
186
|
-
|
187
|
-
|
188
|
-
|
189
|
-
|
190
|
-
|
191
|
-
|
192
|
-
|
239
|
+
The <tt>call()</tt> method will block waiting for the reponse up to
|
240
|
+
<tt>client.timeout</tt> seconds (floating point value). The default is 1
|
241
|
+
second. You can change this to whatever value you like. If a timeout occurs,
|
242
|
+
<tt>call()</tt> will return <tt>nil</tt>.
|
243
|
+
|
244
|
+
The <tt>call()</tt> method is just a convenience method that wraps
|
245
|
+
<tt>send()</tt> and <tt>receive()</tt>. Interally, <tt>send()</tt> always
|
246
|
+
assigns a unique message ID to each outgoing message. It passes this ID back as
|
247
|
+
the return value. You can use this ID to fetch specific messages from
|
248
|
+
<tt>receive()</tt>.
|
249
|
+
|
250
|
+
Normally <tt>receive()</tt> just returns the oldest message to
|
251
|
+
arrive. Internally is keeps a queue of messages as they are recieved. When you
|
252
|
+
call it with no arguments, it just pops the oldest message off the queue (if
|
253
|
+
there are any), otherwise it blocks waiting for a message to come in until the
|
254
|
+
timeout interval expires. However, if you pass an ID to <tt>receive()</tt>, it
|
255
|
+
will look specifically for a message with that ID. If none is found, again, it
|
256
|
+
will block for the timeout interval waiting for it to come it. If none does, it
|
257
|
+
returns <tt>nil</tt>.
|
258
|
+
|
259
|
+
If <tt>call()</tt> times-out, subsequent calls will fail to get the response,
|
260
|
+
even if it comes in later as it will be discarded. This keeps dilatory messages
|
261
|
+
from filling up the receive queue. Once you've given up on a message, it makes
|
262
|
+
no sense to keep it around should it arrive sometime in the future.
|
263
|
+
|
264
|
+
### The Service Side
|
265
|
+
|
266
|
+
Notice that the service uses the <tt>Service::reply()</tt> method rather than
|
267
|
+
the <tt>send()</tt> method to return a response. This method takes the original
|
268
|
+
request message, extracts the <tt>to</tt> address along with message ID and
|
269
|
+
addresses a message back to the client. It also adds a header value of
|
270
|
+
<tt>reply=true</tt>. This header lets the client know that the message coming in
|
271
|
+
is specifically a reply. This is needed this because what if another peer sent
|
272
|
+
the client a message which happened to have the same message ID? The client
|
273
|
+
would confuse that message as the reply from the service and that would mess
|
274
|
+
things up fast. Therefore the <tt>reply<tt> flag lets the client know that the
|
275
|
+
message is a reply to a previous request and can then use the message ID to
|
276
|
+
correlate it with the original request.
|
277
|
+
|
278
|
+
## Services
|
279
|
+
|
280
|
+
Most of what needs to be said about services has be covered. However there is
|
281
|
+
one important feature RabbitMQ accords you which we need to address.
|
282
|
+
|
283
|
+
Say you want to scale your service from just a single thread or process and
|
284
|
+
distribute accross multiple machines which are perhaps even on different
|
285
|
+
networks around the world. You can enable this by modifying a single field. The
|
286
|
+
only thing you need to do differently is redefine one method --
|
287
|
+
<tt>Service::createQueue()</tt>. The following is updates the above example and
|
288
|
+
makes is available to run in this way:
|
193
289
|
|
194
290
|
```ruby
|
195
291
|
#!/usr/bin/env ruby
|
196
292
|
|
197
293
|
require 'haredo/peer'
|
198
294
|
|
199
|
-
$
|
200
|
-
$
|
201
|
-
$
|
202
|
-
$haredo_test_queue = 'HareDo'
|
295
|
+
$mq_host = 'localhost'
|
296
|
+
$mq_username = 'guest'
|
297
|
+
$mq_password = 'guest'
|
203
298
|
|
204
299
|
class Service < HareDo::Service
|
205
300
|
|
206
301
|
def initialize(name)
|
207
302
|
super
|
208
|
-
connect(
|
303
|
+
connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
|
209
304
|
end
|
210
305
|
|
211
306
|
def createQueue()
|
212
307
|
return @channel.queue(@queue_name, :auto_delete => true)
|
213
308
|
end
|
214
309
|
|
215
|
-
|
216
310
|
def serve(msg)
|
217
311
|
reply(msg, :data => 'Reply')
|
218
312
|
end
|
219
313
|
end
|
220
314
|
|
221
|
-
Service.new(
|
315
|
+
Service.new('rpc').run()
|
222
316
|
```
|
223
317
|
|
224
318
|
The only thing we did is remove the <tt>exclusive</tt> attribute which is by
|
data/src/lib/haredo/peer.rb
CHANGED
@@ -42,22 +42,35 @@ class Node
|
|
42
42
|
|
43
43
|
# Connect to RabbitMQ
|
44
44
|
#
|
45
|
-
# @param user The RabbitMQ username
|
46
|
-
# @param password The RabbitMQ password
|
47
|
-
# @param host The RabbitMQ host
|
48
|
-
# @param port The RabbitMQ port
|
45
|
+
# @param user The RabbitMQ username (default guest)
|
46
|
+
# @param password The RabbitMQ password (default guest)
|
47
|
+
# @param host The RabbitMQ host (default localhost)
|
48
|
+
# @param port The RabbitMQ port (default 5672)
|
49
|
+
# @param vhost The RabbitMQ vhost (default /)
|
50
|
+
# @param ssl Use SSL (default false)
|
49
51
|
# @return Returns true if connection was successful, false othewise.
|
50
52
|
|
51
|
-
def connect(
|
53
|
+
def connect(args)
|
54
|
+
|
55
|
+
user = args[:user] || 'guest'
|
56
|
+
password = args[:password] || 'guest'
|
57
|
+
host = args[:host] || 'localhost'
|
58
|
+
port = args[:post] || '5672'
|
59
|
+
vhost = args[:vhost] || ''
|
60
|
+
ssl = ''
|
61
|
+
|
62
|
+
if args[:ssl] == true
|
63
|
+
ssl = 's'
|
64
|
+
end
|
52
65
|
|
53
66
|
begin
|
54
|
-
@cnx = Bunny.new("amqp://#{user}:#{password}@#{host}:#{port}")
|
67
|
+
@cnx = Bunny.new("amqp#{ssl}://#{user}:#{password}@#{host}:#{port}#{vhost}")
|
55
68
|
@cnx.start()
|
56
69
|
rescue Bunny::TCPConnectionFailed => e
|
57
70
|
return false
|
58
71
|
end
|
59
72
|
|
60
|
-
@channel
|
73
|
+
@channel = @cnx.create_channel()
|
61
74
|
|
62
75
|
return true
|
63
76
|
end
|
@@ -116,13 +129,17 @@ class Client < Node
|
|
116
129
|
@mid = 0
|
117
130
|
end
|
118
131
|
|
119
|
-
def connect(
|
120
|
-
super
|
132
|
+
def connect(args)
|
133
|
+
ret = super
|
121
134
|
|
122
|
-
|
123
|
-
|
135
|
+
if ret == true
|
136
|
+
@queue = @channel.queue( '', :auto_delete => true,
|
137
|
+
:arguments => { "x-message-ttl" => 1000 } )
|
124
138
|
|
125
|
-
|
139
|
+
@exchange = @channel.default_exchange()
|
140
|
+
end
|
141
|
+
|
142
|
+
return ret
|
126
143
|
end
|
127
144
|
|
128
145
|
# Disconnect from RabbitMQ
|
@@ -235,10 +252,14 @@ end # class Client
|
|
235
252
|
|
236
253
|
class Service < Node
|
237
254
|
|
255
|
+
# @param name The name of the queue the service will attempt to bind to.
|
238
256
|
def initialize(name)
|
239
257
|
super()
|
240
258
|
|
241
259
|
@queue_name = name
|
260
|
+
|
261
|
+
# The number of messages to prefecth from Rabbit
|
262
|
+
@prefetch = 10
|
242
263
|
end
|
243
264
|
|
244
265
|
# Defined the queue this service will listen on. Assumes a single-instance
|
@@ -260,7 +281,10 @@ class Service < Node
|
|
260
281
|
|
261
282
|
block = args[:blocking] || false
|
262
283
|
|
263
|
-
|
284
|
+
@channel.prefetch(@prefetch)
|
285
|
+
|
286
|
+
queue.subscribe(:block => block, :ack => true) do |info, props, data|
|
287
|
+
@channel.acknowledge(info.delivery_tag, false)
|
264
288
|
serve Message.new(info, props, data)
|
265
289
|
end
|
266
290
|
end
|
data/src/lib/haredo/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: haredo
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Owens
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2013-09-
|
11
|
+
date: 2013-09-23 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bunny
|