haredo 0.0.5 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 46d06a4070b9f99459fa6b9e316c696fcbac6324
4
- data.tar.gz: fb724322cd7195b7dd05850c88704abb1ba3da22
3
+ metadata.gz: ff09def9c61a951c40c71dd2ce8873f6883e84e4
4
+ data.tar.gz: 9e2a10e846685afb1b5b238117aaecf15f9b4116
5
5
  SHA512:
6
- metadata.gz: a76e1578cbcff7f49c8a949350ae4f30fcbe42816cbc64cb6a65e319b95c00018b18e021bc06cfbfe9697c8f1ca56ab0c522b817d33ae2bd0d96219daed1eeb2
7
- data.tar.gz: a8b7b6a30d7445519145c61721f2068afb3681a9a948169b2533b691562af4d31ba1577b335496037cccd3a5c6f9de953259dac16823e8441243ced2b7c80cc8
6
+ metadata.gz: 4bea5b65ac4a075d7a317186163e222b7eeab387f163d6c56c3b363d90f29047fdf46c87a11f10758b2ed766c4b3425c6998c16bf538ba355d890d98fab04107
7
+ data.tar.gz: 13fa22657bfe7fa540c30a5570a90cfe185f17fbf6580ab94532d85b1d873853d3bdc94f64682302f5cd025e5fbf1a65bc3ca55fe45d28be4d1d075f6ef94c8a
data/README.md CHANGED
@@ -45,7 +45,7 @@ $mq_username = 'guest'
45
45
  $mq_password = 'guest'
46
46
  $queue = 'HareDo'
47
47
 
48
- class Service < HareDo::Service
48
+ class Service < HareDo::Peer
49
49
 
50
50
  def initialize(name)
51
51
  super
@@ -60,9 +60,9 @@ class Service < HareDo::Service
60
60
  end
61
61
 
62
62
  service = Service.new($queue)
63
- service.run(:blocking => false)
63
+ service.listen(:blocking => false)
64
64
 
65
- client = HareDo::Client.new()
65
+ client = HareDo::Peer.new()
66
66
  client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
67
67
 
68
68
  1.upto(10) do |i|
@@ -71,6 +71,10 @@ client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
71
71
 
72
72
  puts msg.data.to_i == i + 1
73
73
  end
74
+
75
+ # Very important -- you will leave queues open on RabbitMQ if you don't do this
76
+ service.disconnect()
77
+ client.disconnect()
74
78
  ```
75
79
 
76
80
  The service takes a single argument -- the a name. This name is the name of the
@@ -92,7 +96,7 @@ quick and easy network protocols with minimal code (thanks to Rabbit and Bunny).
92
96
 
93
97
  There are three basic use-cases the client addresses:
94
98
 
95
- * Event Dispatch: Client sends one or messages expecting a response(s). This is
99
+ * Event Dispatch: Client sends one or messages expecting no response. This is
96
100
  the case when you want to log events -- you don't really care about getting a
97
101
  response back. You just want to send out the event and be done with it.
98
102
  * Send/Receive: Client sends one or more messages expecting response and
@@ -122,7 +126,7 @@ $mq_host = 'localhost'
122
126
  $mq_username = 'guest'
123
127
  $mq_password = 'guest'
124
128
 
125
- class Logger < HareDo::Service
129
+ class Service < HareDo::Peer
126
130
 
127
131
  def initialize(name)
128
132
  super
@@ -135,7 +139,8 @@ class Logger < HareDo::Service
135
139
 
136
140
  end
137
141
 
138
- Service.new('logger').run()
142
+ Service.new('logger').listen()
143
+ service.disconnect()
139
144
  ```
140
145
 
141
146
  The client code is a follows:
@@ -149,10 +154,11 @@ $mq_host = 'localhost'
149
154
  $mq_username = 'guest'
150
155
  $mq_password = 'guest'
151
156
 
152
- client = HareDo::Client.new()
157
+ client = HareDo::Peer.new()
153
158
  client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
154
159
 
155
160
  client.send('logger', :data => 'Backup script failed')
161
+ client.disconnect()
156
162
  ```
157
163
 
158
164
  You can run both in the same script by setting the service code to non-blocking
@@ -172,7 +178,7 @@ $mq_host = 'localhost'
172
178
  $mq_username = 'guest'
173
179
  $mq_password = 'guest'
174
180
 
175
- class Logger < HareDo::Service
181
+ class Service < HareDo::Peer
176
182
 
177
183
  def initialize(name)
178
184
  super
@@ -185,11 +191,14 @@ class Logger < HareDo::Service
185
191
 
186
192
  end
187
193
 
188
- Service.new('logger').run(:blocking=>false)
194
+ Service.new('logger').listen(:blocking=>false)
189
195
 
190
- client = HareDo::Client.new()
196
+ client = HareDo::Peer.new()
191
197
  client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
192
198
  client.send('logger', :data => 'Backup script failed')
199
+
200
+ service.disconnect()
201
+ client.disconnect()
193
202
  ```
194
203
 
195
204
  ### Send/Receive
@@ -216,7 +225,7 @@ $mq_host = 'localhost'
216
225
  $mq_username = 'guest'
217
226
  $mq_password = 'guest'
218
227
 
219
- class Service < HareDo::Service
228
+ class Service < HareDo::Peer
220
229
 
221
230
  def initialize(name)
222
231
  super
@@ -229,11 +238,14 @@ class Service < HareDo::Service
229
238
  end
230
239
 
231
240
  service = Service.new('rpc')
232
- service.run(:blocking => false)
241
+ service.listen(:blocking => false)
233
242
 
234
- client = HareDo::Client.new()
243
+ client = HareDo::Peer.new()
235
244
  client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
236
245
  response = @client.call('rpc', :data => 'jujifruit')
246
+
247
+ client.disconnect()
248
+ service.disconnect()
237
249
  ```
238
250
 
239
251
  The <tt>call()</tt> method will block waiting for the reponse up to
@@ -271,7 +283,7 @@ addresses a message back to the client. It also adds a header value of
271
283
  is specifically a reply. This is needed this because what if another peer sent
272
284
  the client a message which happened to have the same message ID? The client
273
285
  would confuse that message as the reply from the service and that would mess
274
- things up fast. Therefore the <tt>reply<tt> flag lets the client know that the
286
+ things up fast. Therefore the <tt>reply</tt> flag lets the client know that the
275
287
  message is a reply to a previous request and can then use the message ID to
276
288
  correlate it with the original request.
277
289
 
@@ -296,7 +308,7 @@ $mq_host = 'localhost'
296
308
  $mq_username = 'guest'
297
309
  $mq_password = 'guest'
298
310
 
299
- class Service < HareDo::Service
311
+ class Service < HareDo::Peer
300
312
 
301
313
  def initialize(name)
302
314
  super
@@ -312,7 +324,8 @@ class Service < HareDo::Service
312
324
  end
313
325
  end
314
326
 
315
- Service.new('rpc').run()
327
+ Service.new('rpc').listen()
328
+ service.disconnect()
316
329
  ```
317
330
 
318
331
  The only thing we did is remove the <tt>exclusive</tt> attribute which is by
@@ -322,6 +335,332 @@ process requests. RabbitMQ will automatically distribute messages equally across
322
335
  all the running services, dividing up the load. You can now scale your service
323
336
  to as many machines and processes as you like.
324
337
 
338
+ ## Peers
339
+
340
+ In case you haven't noticed by now, there is no distinction in code between a
341
+ service and client. Everything is implemented in the same class:
342
+ <tt>HareDo::Peer</tt>. So the only difference between a client and service is
343
+ how you use the <tt>Peer</tt> class. To create custom services, you just derive
344
+ from <tt>Peer</tt> and implement a <tt>serve()</tt> method. Or, you can get away
345
+ with not even doing that -- you could just write a plugin.
346
+
347
+ ## Plugins
348
+
349
+ The library includes a simple plugin architecture which is included in the
350
+ <tt>Peer</tt> class. This allows message processing to be delegated to external
351
+ modules written outside the library.
352
+
353
+ For example, I could write a simple echo plugin like this:
354
+
355
+ ```ruby
356
+ require 'haredo/plugin'
357
+
358
+ class Plugin < HareDo::Plugin
359
+
360
+ UUID = 'echo'
361
+
362
+ def initialize(service, config)
363
+ super UUID, service, config
364
+ end
365
+
366
+ def process(msg)
367
+ @peer.reply(msg, :data=>msg.data)
368
+ end
369
+
370
+ end # class Plugin
371
+ ```
372
+
373
+ Plugins are loaded in anonymous modules at runtime, each having its own private
374
+ namespaces. The <tt>Peer</tt> class then looks inside the anonymous module and
375
+ extracts the <tt>Plugin</tt> class defined within, instantiates it, and adds it
376
+ to the set of loaded modules.
377
+
378
+ Plugins are identified by their UUID. To send a message to a plugin, you include
379
+ the UUID of the plugin in the message headers.
380
+
381
+ Each plugin is passed a reference to the peer that uses them on
382
+ construction. This is stored in the <tt>@peer</tt> member. This is the message
383
+ gateway that your plugin uses to send/reply with.
384
+
385
+ Now I can implement my service and call it as follows:
386
+
387
+ ```ruby
388
+ #!/usr/bin/env ruby
389
+
390
+ service = HareDo::Peer.new('myservice')
391
+ service.plugins.load('echo')
392
+ service.listen(:blocking => false)
393
+
394
+ client = HareDo::Peer.new()
395
+ client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
396
+ response = @client.call('myservice', headers=>{:uuid=>'echo'}, :data => 'jujifruit')
397
+
398
+ client.disconnect()
399
+ service.disconnect()
400
+ ```
401
+
402
+ In this case, I didn't even need to implement a custom service. I just loaded my
403
+ plugin. By default the base class implementation of
404
+ <tt>HareDo::Peer::serve()</tt> passes all messages into the plugin system. If no
405
+ plugin is found to service the message, it is simply dropped.
406
+
407
+ By default the <tt>Peer::plugins::load()</tt> method iterates over each entry in
408
+ Ruby <tt>PATH</tt> and looks within the <tt>haredo/plugins</tt> directly of each
409
+ entry. This value is given by the <tt>@module_path_prefix</tt> member of
410
+ <tt>Peer</tt>, which you can modify to suit your needs. Therefore, if you stored
411
+ your plugins in <tt>/opt/share/haredo/plugins</tt> and you wanted to load from
412
+ there, you could do it a couple different ways:
413
+
414
+ Add <tt>/opt/share</tt> to your Ruby path:
415
+
416
+ ```ruby
417
+ $:.unshift '/opt/share'
418
+ service.plugins.load('echo')
419
+ ```
420
+
421
+ Add <tt>/opt/share</tt> to your Ruby path and modify the service's
422
+ <tt>@module_path_prefix</tt>:
423
+
424
+ ```ruby
425
+ $:.unshift '/opt/share/haredo/plugins'
426
+
427
+ # Remove the module prefix.
428
+ service.module_path_prefix = ''
429
+
430
+ service.plugins.load('echo')
431
+ ```
432
+
433
+ In this case, every module in the Ruby path that has the name <tt>echo.rb</tt>
434
+ will be a candidate. The first file to match will be loaded as the module. You
435
+ can see that in this case, namespace collisions are a very real danger. That's
436
+ why the <tt>@module_path_prefix</tt> member exists -- there is a very low
437
+ probability that there will be another module named located in the Ruby path
438
+ which has a relative directory and name of <tt>haredo/plugins/echo.rb</tt>.
439
+
440
+ ### Configuration
441
+
442
+ You can pass configuration data into a plugin when you load it using the
443
+ <tt>Peer::plugins.loadConfig()</tt> method. This takes a HASH whose keys are the
444
+ names of the plugins to load and the values are Hashes passed into each plugin
445
+ as the configuration. The contents of the configuration are complete up to you
446
+ as the plugin author.
447
+
448
+ For example:
449
+
450
+ ```ruby
451
+ #!/usr/bin/env ruby
452
+
453
+ service = HareDo::Peer.new('myservice')
454
+
455
+ # Load modules from configuration
456
+ config = {
457
+ # module name => module configuration
458
+ 'example' => { 'x' => '1' },
459
+ 'echo' => { 'y' => '2' }
460
+ }
461
+
462
+ service.plugins.loadConfig(config)
463
+ service.listen(:blocking => false)
464
+
465
+ client = HareDo::Peer.new()
466
+ client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
467
+ response = @client.call('myservice', headers=>{:uuid=>'example'}, :data => 'jujifruit')
468
+
469
+ service.disconnect()
470
+ client.disconnect()
471
+ ```
472
+
473
+ By using this method to load plugins, you can store all your plugin
474
+ configuration in a YAML file and load it at runtime to load/configure all of
475
+ your plugins for your service(s).
476
+
477
+ There is nothing stopping you from using plugins within a client context as
478
+ well. If you get a message in from a peer, you can simply check for a UUID in
479
+ the message headers and pass it to <tt>@plugins.process()</tt> if you so choose.
480
+
481
+ The plugin architecture is a powerful way to implement and add an endless
482
+ variety of services with minimal code.
483
+
484
+ A complete example of using plugins is illustrated in the <tt>basic.rb</tt> unit
485
+ test with the source distribution.
486
+
487
+ ## Daemons
488
+
489
+ Say you have a service written and now you want an easy way to start/stop/manage
490
+ it. The <tt>haredo</tt> command-line utility is included just for that. You can
491
+ add your service to a simple configuration file, run <tt>haredo start</tt> and
492
+ your service is online. The default configuration file is in
493
+ <tt>/etc/haredo/haredo.conf</tt> and looks something like this:
494
+
495
+ ```yaml
496
+ ---
497
+ system:
498
+ broker:
499
+ host: localhost
500
+ user: guest
501
+ password: guest
502
+ port: '5672'
503
+ daemon:
504
+ queue:
505
+ name: 'jujifruit'
506
+ properties:
507
+ exclusive: false
508
+ auto_delete: true
509
+ key: 'b8d96b66-8cae-4e16-b5ec-d607483b061e'
510
+ modules:
511
+ - sonar/service/config
512
+ services:
513
+ haredo:
514
+ path_prefix: haredo/plugins
515
+ plugins:
516
+ echo:
517
+ juji: fruit
518
+ myplugins:
519
+ path_prefix: mylib/plugins
520
+ plugins:
521
+ log:
522
+ db:
523
+ host: localhost
524
+ db: sonar
525
+ user: root
526
+ ```
527
+
528
+ ### The Listen Queue
529
+
530
+ The daemon creates a queue using the <tt>queue</tt> entries. The name is given
531
+ by the <tt>name</tt> attribute and the other optional queue properties are given
532
+ by the <tt>properties</tt> attributes. The will be applied to the daemon's
533
+ listen queue.
534
+
535
+ ### Services
536
+
537
+ Servces are sets of plugins that are loaded from different places. Usually these
538
+ are delineated by the need to change the <tt>path_prefix</tt> value to load a
539
+ given set of modules. At load time, the daemon iterates over each entry in
540
+ services, modified the <tt>path_prefix</tt> according to the given value and
541
+ then attempts to load all the plugins listed. It prints the results for each
542
+ plugin to standard error. If a plugin fails to load, it will also log this in
543
+ syslog.
544
+
545
+ ### Modules
546
+
547
+ Modules are additional modules you want to daemon to load. This is done before
548
+ services are processed. This is for when you want to modify the daemon's code in
549
+ some way (adding or overriding methods), or also including some custom
550
+ initialization. For example, say I have a module <tt>sonar/service/config</tt>
551
+ that adds database connection functionality to <tt>HareDo::Service::Config</tt>
552
+ mixin as follows:
553
+
554
+ ```ruby
555
+ require 'jw/dbi'
556
+
557
+ module HareDo
558
+ module Service
559
+
560
+ module Config
561
+
562
+ # Connect to the Database
563
+ #
564
+ # @return Return a client instance if connection was successful, nil otherwise
565
+ def connectDb(params)
566
+
567
+ vars = {
568
+ host: params['host'],
569
+ db: params['db'],
570
+ username: params['user'],
571
+ password: params['password'],
572
+ driver: 'QPSQL',
573
+ }
574
+
575
+ if params.has_key? 'port'
576
+ vars[:port] = params['port']
577
+ end
578
+
579
+ db = JW::DBI::Database.new()
580
+
581
+ if db.open(vars) == false
582
+ raise "Failed to connect: #{db.error()}"
583
+ end
584
+
585
+ return db
586
+ end
587
+
588
+ end # class Context
589
+
590
+ end # module HareDo
591
+ end # module Sonar
592
+ ```
593
+
594
+ This uses my custom database abstraction extensions which I want to add to
595
+ <tt>HareDo::Daemon</tt>. To incorporate it, I simply add it to the modules
596
+ section on the config file. The daemon will load this on startup, the module
597
+ will add the custom method, and now my plugins defined in the <tt>services</tt>
598
+ section of the configuration file will have it available to them on service
599
+ load. For example:
600
+
601
+ ```ruby
602
+ require 'haredo/plugin'
603
+
604
+ class Plugin < HareDo::Plugin
605
+
606
+ UUID = 'log'
607
+
608
+ attr_reader :db, :logrecord
609
+
610
+ def initialize(peer, config)
611
+ super UUID, peer, config
612
+
613
+ @db = @peer.connectDB(config['db'])
614
+ end
615
+
616
+ def finalize()
617
+ super
618
+
619
+ @db.close()
620
+ end
621
+
622
+ .
623
+ .
624
+ .
625
+ end
626
+ ```
627
+
628
+ The <tt>peer</tt> member is a reference to the <tt>Daemon</tt> instance which
629
+ has been modified to include the <tt>connectDB()</tt> method.
630
+
631
+ This gives you an easy way to customize the daemon without having to derive a
632
+ new class and instantiate it (and thus having to create a new binary and/or
633
+ modify the command-line utility).
634
+
635
+ ### Command Line
636
+
637
+ The command line utility is simple. It includes <tt>start</tt>, <tt>stop</tt>
638
+ and <tt>status</tt> commands. It also has a help (<tt>-h</tt>) switch, along
639
+ with each command. So you can check the status using the <tt>haredo status</tt>
640
+ command which detects if the daemon is running. If it is, it sends it a message
641
+ and get back basic information in JSON, including loaded plugins:
642
+
643
+ ```bash
644
+ bash $ haredo status
645
+ {"time":"2013-10-01 12:12:02 -0500","pid":"25290","plugins":["echo","dbi","log","status"]}
646
+ ```
647
+
648
+ There is also a subcommand called <tt>config</tt>. This is used to test and set
649
+ values in the configuration file for broker and (otionally) a database. It
650
+ exists mainly as a utility for installers. You can test a broker connection as
651
+ follows:
652
+
653
+ ```bash
654
+ bash $ PRBPASSWORD=guest haredo config broker -t -u guest -s localhost
655
+ {:user=>"guest", :password=>nil, :port=>nil, :vhost=>nil, :host=>"localhost", :ssl=>false}
656
+ succeeded
657
+ ```
658
+
659
+ The RabbitMQ user password is passed in via environmental variables.
660
+
661
+ Most likely you will never need to use this as it was designed specifically for
662
+ use in the Debian installer.
663
+
325
664
  ## Installation
326
665
 
327
666
  You can install directly from the command line via Ruby gems as follows: