haredo 0.0.5 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 46d06a4070b9f99459fa6b9e316c696fcbac6324
-  data.tar.gz: fb724322cd7195b7dd05850c88704abb1ba3da22
+  metadata.gz: ff09def9c61a951c40c71dd2ce8873f6883e84e4
+  data.tar.gz: 9e2a10e846685afb1b5b238117aaecf15f9b4116
 SHA512:
-  metadata.gz: a76e1578cbcff7f49c8a949350ae4f30fcbe42816cbc64cb6a65e319b95c00018b18e021bc06cfbfe9697c8f1ca56ab0c522b817d33ae2bd0d96219daed1eeb2
-  data.tar.gz: a8b7b6a30d7445519145c61721f2068afb3681a9a948169b2533b691562af4d31ba1577b335496037cccd3a5c6f9de953259dac16823e8441243ced2b7c80cc8
+  metadata.gz: 4bea5b65ac4a075d7a317186163e222b7eeab387f163d6c56c3b363d90f29047fdf46c87a11f10758b2ed766c4b3425c6998c16bf538ba355d890d98fab04107
+  data.tar.gz: 13fa22657bfe7fa540c30a5570a90cfe185f17fbf6580ab94532d85b1d873853d3bdc94f64682302f5cd025e5fbf1a65bc3ca55fe45d28be4d1d075f6ef94c8a

data/README.md

@@ -45,7 +45,7 @@ $mq_username = 'guest'
 $mq_password = 'guest'
 $queue       = 'HareDo'
 
-class Service < HareDo::Service
+class Service < HareDo::Peer
 
   def initialize(name)
     super
@@ -60,9 +60,9 @@ class Service < HareDo::Service
 end
 
 service = Service.new($queue)
-service.run(:blocking => false)
+service.listen(:blocking => false)
 
-client = HareDo::Client.new()
+client = HareDo::Peer.new()
 client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
 
 1.upto(10) do |i|
@@ -71,6 +71,10 @@ client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
       
   puts msg.data.to_i == i + 1
 end
+
+# Very important -- you will leave queues open on RabbitMQ if you don't do this
+service.disconnect()
+client.disconnect()
 ```
 
 The service takes a single argument -- the a name. This name is the name of the
@@ -92,7 +96,7 @@ quick and easy network protocols with minimal code (thanks to Rabbit and Bunny).
 
 There are three basic use-cases the client addresses:
 
-* Event Dispatch: Client sends one or messages expecting a response(s). This is
+* Event Dispatch: Client sends one or messages expecting no response. This is
   the case when you want to log events -- you don't really care about getting a
   response back. You just want to send out the event and be done with it.
 * Send/Receive: Client sends one or more messages expecting response and
@@ -122,7 +126,7 @@ $mq_host     = 'localhost'
 $mq_username = 'guest'
 $mq_password = 'guest'
 
-class Logger < HareDo::Service
+class Service < HareDo::Peer
 
   def initialize(name)
     super
@@ -135,7 +139,8 @@ class Logger < HareDo::Service
 
 end
 
-Service.new('logger').run()
+Service.new('logger').listen()
+service.disconnect()
 ```
 
 The client code is a follows:
@@ -149,10 +154,11 @@ $mq_host     = 'localhost'
 $mq_username = 'guest'
 $mq_password = 'guest'
 
-client = HareDo::Client.new()
+client = HareDo::Peer.new()
 client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
 
 client.send('logger', :data => 'Backup script failed')
+client.disconnect()
 ```
 
 You can run both in the same script by setting the service code to non-blocking
@@ -172,7 +178,7 @@ $mq_host     = 'localhost'
 $mq_username = 'guest'
 $mq_password = 'guest'
 
-class Logger < HareDo::Service
+class Service < HareDo::Peer
 
   def initialize(name)
     super
@@ -185,11 +191,14 @@ class Logger < HareDo::Service
 
 end
 
-Service.new('logger').run(:blocking=>false)
+Service.new('logger').listen(:blocking=>false)
 
-client = HareDo::Client.new()
+client = HareDo::Peer.new()
 client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
 client.send('logger', :data => 'Backup script failed')
+
+service.disconnect()
+client.disconnect()
 ```
 
 ### Send/Receive
@@ -216,7 +225,7 @@ $mq_host     = 'localhost'
 $mq_username = 'guest'
 $mq_password = 'guest'
 
-class Service < HareDo::Service
+class Service < HareDo::Peer
 
   def initialize(name)
     super
@@ -229,11 +238,14 @@ class Service < HareDo::Service
 end
 
 service = Service.new('rpc')
-service.run(:blocking => false)
+service.listen(:blocking => false)
 
-client = HareDo::Client.new()
+client = HareDo::Peer.new()
 client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
 response = @client.call('rpc', :data => 'jujifruit')
+
+client.disconnect()
+service.disconnect()
 ```
 
 The <tt>call()</tt> method will block waiting for the reponse up to
@@ -271,7 +283,7 @@ addresses a message back to the client. It also adds a header value of
 is specifically a reply. This is needed this because what if another peer sent
 the client a message which happened to have the same message ID? The client
 would confuse that message as the reply from the service and that would mess
-things up fast. Therefore the <tt>reply<tt> flag lets the client know that the
+things up fast. Therefore the <tt>reply</tt> flag lets the client know that the
 message is a reply to a previous request and can then use the message ID to
 correlate it with the original request.
 
@@ -296,7 +308,7 @@ $mq_host     = 'localhost'
 $mq_username = 'guest'
 $mq_password = 'guest'
 
-class Service < HareDo::Service
+class Service < HareDo::Peer
 
   def initialize(name)
     super
@@ -312,7 +324,8 @@ class Service < HareDo::Service
   end
 end
 
-Service.new('rpc').run()
+Service.new('rpc').listen()
+service.disconnect()
 ```
 
 The only thing we did is remove the <tt>exclusive</tt> attribute which is by
@@ -322,6 +335,332 @@ process requests. RabbitMQ will automatically distribute messages equally across
 all the running services, dividing up the load. You can now scale your service
 to as many machines and processes as you like.
 
+## Peers
+
+In case you haven't noticed by now, there is no distinction in code between a
+service and client. Everything is implemented in the same class:
+<tt>HareDo::Peer</tt>. So the only difference between a client and service is
+how you use the <tt>Peer</tt> class. To create custom services, you just derive
+from <tt>Peer</tt> and implement a <tt>serve()</tt> method. Or, you can get away
+with not even doing that -- you could just write a plugin.
+
+## Plugins
+
+The library includes a simple plugin architecture which is included in the
+<tt>Peer</tt> class. This allows message processing to be delegated to external
+modules written outside the library.
+
+For example, I could write a simple echo plugin like this:
+
+```ruby
+require 'haredo/plugin'
+
+class Plugin < HareDo::Plugin
+
+  UUID = 'echo'
+
+  def initialize(service, config)
+    super UUID, service, config 
+  end
+
+  def process(msg)
+    @peer.reply(msg, :data=>msg.data)
+  end
+
+end # class Plugin
+```
+
+Plugins are loaded in anonymous modules at runtime, each having its own private
+namespaces. The <tt>Peer</tt> class then looks inside the anonymous module and
+extracts the <tt>Plugin</tt> class defined within, instantiates it, and adds it
+to the set of loaded modules.
+
+Plugins are identified by their UUID. To send a message to a plugin, you include
+the UUID of the plugin in the message headers.
+
+Each plugin is passed a reference to the peer that uses them on
+construction. This is stored in the <tt>@peer</tt> member. This is the message
+gateway that your plugin uses to send/reply with.
+
+Now I can implement my service and call it as follows:
+
+```ruby
+#!/usr/bin/env ruby
+
+service = HareDo::Peer.new('myservice')
+service.plugins.load('echo')
+service.listen(:blocking => false)
+
+client = HareDo::Peer.new()
+client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
+response = @client.call('myservice', headers=>{:uuid=>'echo'}, :data => 'jujifruit')
+
+client.disconnect()
+service.disconnect()
+```
+
+In this case, I didn't even need to implement a custom service. I just loaded my
+plugin. By default the base class implementation of
+<tt>HareDo::Peer::serve()</tt> passes all messages into the plugin system. If no
+plugin is found to service the message, it is simply dropped.
+
+By default the <tt>Peer::plugins::load()</tt> method iterates over each entry in
+Ruby <tt>PATH</tt> and looks within the <tt>haredo/plugins</tt> directly of each
+entry. This value is given by the <tt>@module_path_prefix</tt> member of
+<tt>Peer</tt>, which you can modify to suit your needs. Therefore, if you stored
+your plugins in <tt>/opt/share/haredo/plugins</tt> and you wanted to load from
+there, you could do it a couple different ways:
+
+Add <tt>/opt/share</tt> to your Ruby path:
+
+```ruby
+$:.unshift '/opt/share'
+service.plugins.load('echo')
+```
+
+Add <tt>/opt/share</tt> to your Ruby path and modify the service's
+<tt>@module_path_prefix</tt>:
+
+```ruby
+$:.unshift '/opt/share/haredo/plugins'
+
+# Remove the module prefix.
+service.module_path_prefix = ''
+
+service.plugins.load('echo')
+```
+
+In this case, every module in the Ruby path that has the name <tt>echo.rb</tt>
+will be a candidate. The first file to match will be loaded as the module. You
+can see that in this case, namespace collisions are a very real danger. That's
+why the <tt>@module_path_prefix</tt> member exists -- there is a very low
+probability that there will be another module named located in the Ruby path
+which has a relative directory and name of <tt>haredo/plugins/echo.rb</tt>.
+
+### Configuration
+
+You can pass configuration data into a plugin when you load it using the
+<tt>Peer::plugins.loadConfig()</tt> method. This takes a HASH whose keys are the
+names of the plugins to load and the values are Hashes passed into each plugin
+as the configuration. The contents of the configuration are complete up to you
+as the plugin author.
+
+For example:
+
+```ruby
+#!/usr/bin/env ruby
+
+service = HareDo::Peer.new('myservice')
+
+# Load modules from configuration
+config = {
+   # module name  => module configuration
+   'example' => { 'x' => '1' },
+   'echo'    => { 'y' => '2' }
+}
+
+service.plugins.loadConfig(config)
+service.listen(:blocking => false)
+
+client = HareDo::Peer.new()
+client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
+response = @client.call('myservice', headers=>{:uuid=>'example'}, :data => 'jujifruit')
+
+service.disconnect()
+client.disconnect()
+```
+
+By using this method to load plugins, you can store all your plugin
+configuration in a YAML file and load it at runtime to load/configure all of
+your plugins for your service(s).
+
+There is nothing stopping you from using plugins within a client context as
+well. If you get a message in from a peer, you can simply check for a UUID in
+the message headers and pass it to <tt>@plugins.process()</tt> if you so choose.
+
+The plugin architecture is a powerful way to implement and add an endless
+variety of services with minimal code.
+
+A complete example of using plugins is illustrated in the <tt>basic.rb</tt> unit
+test with the source distribution.
+
+## Daemons
+
+Say you have a service written and now you want an easy way to start/stop/manage
+it. The <tt>haredo</tt> command-line utility is included just for that. You can
+add your service to a simple configuration file, run <tt>haredo start</tt> and
+your service is online. The default configuration file is in
+<tt>/etc/haredo/haredo.conf</tt> and looks something like this:
+
+```yaml
+---
+system:
+  broker:
+    host: localhost
+    user: guest
+    password: guest
+    port: '5672'
+daemon:
+  queue: 
+    name: 'jujifruit'
+    properties:
+      exclusive: false
+      auto_delete: true
+  key: 'b8d96b66-8cae-4e16-b5ec-d607483b061e'
+  modules:
+    - sonar/service/config
+  services:
+    haredo:
+      path_prefix: haredo/plugins
+      plugins:
+        echo:
+          juji: fruit
+    myplugins:
+      path_prefix: mylib/plugins
+      plugins:
+        log:
+          db:
+            host: localhost
+            db: sonar
+            user: root
+```
+
+### The Listen Queue
+
+The daemon creates a queue using the <tt>queue</tt> entries. The name is given
+by the <tt>name</tt> attribute and the other optional queue properties are given
+by the <tt>properties</tt> attributes. The will be applied to the daemon's
+listen queue.
+
+### Services
+
+Servces are sets of plugins that are loaded from different places. Usually these
+are delineated by the need to change the <tt>path_prefix</tt> value to load a
+given set of modules. At load time, the daemon iterates over each entry in
+services, modified the <tt>path_prefix</tt> according to the given value and
+then attempts to load all the plugins listed. It prints the results for each
+plugin to standard error. If a plugin fails to load, it will also log this in
+syslog.
+
+### Modules
+
+Modules are additional modules you want to daemon to load. This is done before
+services are processed. This is for when you want to modify the daemon's code in
+some way (adding or overriding methods), or also including some custom
+initialization. For example, say I have a module <tt>sonar/service/config</tt>
+that adds database connection functionality to <tt>HareDo::Service::Config</tt>
+mixin as follows:
+
+```ruby
+require 'jw/dbi'
+
+module HareDo
+module Service
+
+module Config 
+
+  # Connect to the Database
+  #
+  # @return Return a client instance if connection was successful, nil otherwise
+  def connectDb(params)
+
+    vars = { 
+      host: params['host'],
+      db: params['db'], 
+      username: params['user'],
+      password: params['password'],
+      driver: 'QPSQL',
+    }
+
+    if params.has_key? 'port'
+      vars[:port] = params['port']
+    end
+
+    db = JW::DBI::Database.new()
+
+    if db.open(vars) == false
+      raise "Failed to connect: #{db.error()}"
+    end
+
+    return db
+  end
+
+end # class Context
+
+end # module HareDo
+end # module Sonar
+```
+
+This uses my custom database abstraction extensions which I want to add to
+<tt>HareDo::Daemon</tt>. To incorporate it, I simply add it to the modules
+section on the config file. The daemon will load this on startup, the module
+will add the custom method, and now my plugins defined in the <tt>services</tt>
+section of the configuration file will have it available to them on service
+load. For example:
+
+```ruby
+require 'haredo/plugin'
+
+class Plugin < HareDo::Plugin
+
+  UUID = 'log'
+
+  attr_reader :db, :logrecord
+
+  def initialize(peer, config)
+    super UUID, peer, config 
+
+    @db = @peer.connectDB(config['db'])
+  end
+
+  def finalize()
+    super
+
+    @db.close()
+  end
+
+  .
+  .
+  .
+end
+```
+
+The <tt>peer</tt> member is a reference to the <tt>Daemon</tt> instance which
+has been modified to include the <tt>connectDB()</tt> method.
+
+This gives you an easy way to customize the daemon without having to derive a
+new class and instantiate it (and thus having to create a new binary and/or
+modify the command-line utility).
+
+### Command Line
+
+The command line utility is simple. It includes <tt>start</tt>, <tt>stop</tt>
+and <tt>status</tt> commands. It also has a help (<tt>-h</tt>) switch, along
+with each command. So you can check the status using the <tt>haredo status</tt>
+command which detects if the daemon is running. If it is, it sends it a message
+and get back basic information in JSON, including loaded plugins:
+
+```bash
+bash $ haredo status
+{"time":"2013-10-01 12:12:02 -0500","pid":"25290","plugins":["echo","dbi","log","status"]}
+```
+
+There is also a subcommand called <tt>config</tt>. This is used to test and set
+values in the configuration file for broker and (otionally) a database. It
+exists mainly as a utility for installers. You can test a broker connection as
+follows:
+
+```bash
+bash $ PRBPASSWORD=guest haredo config broker -t -u guest -s localhost
+{:user=>"guest", :password=>nil, :port=>nil, :vhost=>nil, :host=>"localhost", :ssl=>false}
+succeeded
+```
+
+The RabbitMQ user password is passed in via environmental variables.
+
+Most likely you will never need to use this as it was designed specifically for
+use in the Debian installer.
+
 ## Installation
 
 You can install directly from the command line via Ruby gems as follows: