jerbil 1.3.3 → 1.4.5

checksums.yaml

@@ -1,15 +1,15 @@
 ---
 !binary "U0hBMQ==":
   metadata.gz: !binary |-
-    YjBlMmZlMjU5N2FhNDY2NzIwOTJlYTNlMWUzMzUwZmI0MjIzMjZiOA==
+    OThlODQwNmFhOWZlOWUwMjYxYWU0MjY5ZGRhYTBjNGI1YzBiYjU2NQ==
   data.tar.gz: !binary |-
-    MzBjYmIzY2Q1M2EzMjhjYTZlN2FmMDcxNzMzYjJhOGMyYWMwY2I0MA==
-!binary "U0hBNTEy":
+    ZGU5MTgzZTgzYzNlY2Y4NjQ1OTcwMjJmZTY1MzllOTVhYzAyYjc0MA==
+SHA512:
   metadata.gz: !binary |-
-    OGIzZjRhY2IxZDdkMGYyNTI1MDE0ZjRjMjc5ZGQ2OTU2ZGU5ZjEyNWQwNDY3
-    ZjdjYmVmMDU5ZDIzOTdkYTc5MjczMGM0MmY3MmVjZWI3YmRjNDI1ZTY4NGE0
-    YzQ1ZDAyNGFjODNiZWNhMzM4MTAzNTNkMGExY2U2OTZiMjYxYjQ=
+    M2QxMWI2ZjUyMzk4ZDZiMTNkMTk5N2Y4Mzk3YjhhNGY2YjQ1NWY2MWQ0NGMw
+    YWM0NzgxZDg4NjVjYmE1ZjMzNzFlMzQ3ZjIyODdiYjhmNWJiMTU4Y2U3YzRi
+    ZWYyOTlhODMxNTJjOGQ1MGExYTUxN2RmOTczNGZhMTgyY2NiMTU=
   data.tar.gz: !binary |-
-    NDkyYmIxZWNlZDVhM2Y2NzgxOGNkYjQ5M2JkN2Y2ZDE5M2YxMDUyZDk1OTdm
-    YjhiMDM2MzMzZjQ4MmYyNGM2OWI5NmY3NThjMTc3NDMwZTdiNGE4ZWM2Mzg2
-    OGMyY2VkZWQyODJiMTI5OTFmODc5M2EyYWZkYmE2Y2ZmODYxOTk=
+    NTJjNzU5ZmYzMzA5ZGNlNzI3ZGViYTQxMjUyYmI0MGE0OTcyYTFhNjU4YjYw
+    ZDE2NmFlNjNiZDg5MmZlNTdhNWJjMzU1MmM1N2Q5NmY1MGU0ZDBlYzFkZjA2
+    ZDJmMTNjZmFkYjg1MjE5NDQwYjMwNzlkYjE4ZjQ0YTYxOTM0YWM=

data/Bugs.rdoc

@@ -1,11 +1,18 @@
 == Bugs
 
+[jerbil-1.4.4 16-Oct-2014]
+
+  Started up a service with a previous version still running and it did not detect or
+  notify the problem!
+
 [jerbil-1.3.1 29-Aug-2013]
 
   Need to update jerbil command to get rid of thor and improve what it does.
   Consider Network Dependencies - where a service on one machine does not start
   without a service on another or when a service is taken down on which others depend
   then they are notified first, through Jerbil?
+  
+  [Thor removed in 2014]
 
 [jerbil-1.2.3 06-June-2013]
 

data/Gemfile

@@ -1,14 +1,13 @@
 # @markup ruby
 #
 # general utilities
-gem 'netaddr'
-gem 'term-ansicolor'
-gem 'thor'
-gem 'daemons'
+gem 'netaddr', '~> 1.0'
+gem 'term-ansicolor', '~> 1.0'
+gem 'daemons', '~> 1.0'
 
 # jerbil-related utilities
-gem 'jellog', '>=1.0.14' # for logging
-gem 'jeckyl', '>=0.2.7' # for config files
-gem 'jeni', '>=0.2.5' # for installation
+gem 'jellog', '~> 1.0', '>=1.0.14' # for logging
+gem 'jeckyl', '~> 0.2', '>=0.2.7' # for config files
+gem 'jeni', '~> 0.2', '>=0.2.5' # for installation
 
-gem 'optplus', '>=0.0.2'
+gem 'optplus', '~> 0.0', '>=0.0.2'

data/History.txt

@@ -4,6 +4,34 @@
 
 ==History
 
+[jerbil-1.4.5 17-Oct-2014]
+
+  Add server version to 'jerbil remotes -V' and add system logging for start and
+  end of monitor process.
+
+[jerbil-1.4.4 13-Oct-2014]
+
+  Fix Gemfile dependencies to please rubygems and remove thor as a dependency.
+
+[jerbil-1.4.3 13-Oct-2014]
+
+  Ensure monitor raises the alarm if the secret key is rejected by another server.
+
+[jerbil-1.4.2 09-Oct-2014]
+
+  Fix jeni installer error.
+
+[jerbil-1.4.1 03-Aug-2014]
+
+  Relocate server discovery to Jerbil::Monitor. Tidy up tests and readmes. Allow for 
+  gethostname not to return fqdn and use gethostbyname if this happens.
+
+[jerbil-1.4.0 19-Apr-2014]
+
+  Ensure that jerbild init does not complete until the server has completed
+  discovery. This prevents services from starting when there is apparently
+  a server but it is not yet ready for them. 
+
 [jerbil-1.3.3 03-Sep-2013]
 
   Ensure jerbil methods are called on the server and not the server record.

data/Intro.txt

@@ -1,5 +1,9 @@
-Provides an Object Request Broker and service framework for ruby-based services.
+Provides a reliable Object Request Broker and a simple framework for ruby-based services.
 Jerbil servers run on each machine in the system and share information on registering
-services. A parent class for services hides all of the jerbil interactions and client
-and supervisor modules hide interactions from scripts or applications interacting
-with these services.
+services. This ensures no single point of failure - machines can come and go (orderly or disorderly)
+and the network extends or heals as they do. For services there is a parent class that hides 
+all of the jerbil server interactions so that new services can be written without having
+to write any distributing code. Clients can also be written using an interface that
+can find one or more services on the network and connect to each or the first. Finally, there
+are scripts to start and stop the Jerbil server and any Services so that the whole thing
+can be quickly installed and integrated with your system.

data/README.md

@@ -1,10 +1,18 @@
 # Jerbil
 
-**(a.k.a Jumpin Ermin's Reliable Broker for Integrated Linux services)**
+A "Framework" for writing services in Ruby that provides a reliable way to deploy
+services across a network. Reliability is a key objective and with a server
+on each machine you can rely on Jerbil to connect you effortlessly to your services
+wherever they are running. 
 
-An Object Broker for ruby that provides reliable access to services across the LAN
-and comes complete with a service class that makes writing services easy, hiding all
-of the broker interactions.
+Jerbil comes complete with a basic service class to make writing a new service very
+quick and easy. It also has helpers to connect clients to services through Jerbil
+without having to worry about where the service is running. You can find single or 
+multiple services and you can run services at development, test or production status 
+all on the same LAN or same machine.
+
+Overall, Jerbil should save anyone looking to develop a service in Ruby from a lot of
+tedious work and should avoid a lot of the headaches of alternatives such as Rinda.
 
 **GitHub:** [https://github.com/osburn-sharp/jerbil](https://github.com/osburn-sharp/jerbil)
 
@@ -22,7 +30,8 @@ uses the Jerbil Server
 
 ### Note
 
-Jerbil will work on any Linux distro (and possibly other systems too) but it comes with
+Jerbil **(a.k.a Jumpin Ermin's Reliable Broker for Integrated Linux services)** will 
+work on any Linux distro (and possibly other systems too) but it comes with
 all the gear necessary to work on Gentoo out of the box - e.g. runscripts, /etc/conf.d files
 and the like.
 
@@ -71,13 +80,19 @@ the defaults, you need to generate a secret. A random secret can be generated us
 
     # jerbil server secret
 
-although any long string will do. Full details of the configuration parameters for Jerbil are in {Jerbil::Config}.
+although any long string will do. 
+
+You also need to set some parameters for your network. This is to help the server
+discover any other Jerbil servers on the network without having to hard-code
+this information on all computers. Further details on Server Discovery are provided below.
+Full details of the configuration parameters for Jerbil are in {Jerbil::Config}.
 
 ### Editting /etc/services
 
-Jerbil uses /etc/services to find its services (unsurprisingly) so this file needs to be updated. There must
-be an entry for the server itself, as well as the services that use jerbil. Because Jerbil supports three
-operating environments (dev, test and prod), these services must be assigned in triplets. The following is an example
+Jerbil uses /etc/services (Socket.getservbyname) to find the ports for its services (unsurprisingly) so this 
+file needs to be updated. There must be an entry for the Jerbil server itself, as well as the 
+services that use Jerbil. Because Jerbil supports three operating environments (dev, test and prod), 
+these services must be assigned in triplets. The following is an example
 
     # starts from 49195
     #
@@ -109,31 +124,33 @@ and offsets it if either test or dev is selected. They are included here as a re
 Once you have generated a secret and set up some services in /etc/services, it is now possible to
 start the Jerbil Server. This can be done using the jerbild script:
 
-    # /usr/sbin/jerbild
+    # /usr/local/sbin/jerbild
 
-which will start the server, get the default config file (/etc/jermine/jerbil.rb), daemonise, and say very little.
+which will start the server, get the default config file (/etc/jerbil/jerbil.rb), daemonise, and say very little.
 If you add the -h or --help switch you can see how to overide this default behaviour.
 You can then test that the server is running with:
 
-    # jerbil server
+    # jerbil status
 
 If this does not report the server as up, go to the troubleshooting section.
 
-If you can support init scripts, then jerbil is ready to go. Edit /etc/conf.d/jerbild if you need to change settings such as
-where the config file is and then run:
+If you can support init scripts, then jerbil is ready to go. Edit /etc/conf.d/jerbild if 
+you need to change settings such as where the config file is and then run:
 
     # /etc/init.d/jerbild start
+    or alternatively
+    # rc-service jerbild start
+    and to set the service to run at startup:
     # rc-update add jerbild default
 
-the latter command works on Gentoo at least.
 
 You can also check if the server is working using:
 
-    # /etc/init.d/jerbild status
+    # rc-service jerbild status
 
 To stop the server:
 
-    # /etc/init.d/jerbild stop
+    # rc-service jerbild stop
     
 If your distro does not support runscripts like this, then you can start, stop and test status
 manually using:
@@ -147,7 +164,7 @@ Further details of all these commands can be obtained with the -h or --help opti
 
 ## General Description
 
-The function of Jerbil is to make it easy to write linux services in ruby that can be deployed anywhere on
+The aim of Jerbil is to make it easier to write Linux services in Ruby that can be deployed anywhere on
 a network, once or multiple times, and enabling clients to access these services regardless of where they are. Its
 a wrapper around DRb and a replacement for Rinda.
 
@@ -160,7 +177,7 @@ single point of failure. If it goes down nobody can find anything and all servic
 server is restored.
 
 Part of the self-healing in Jerbil is dealing with services that become unavailable. When this happens, the server
-checks if the services local server is running. If it is, this local server is asked to deal with the missing service
+checks if the service's local server is running. If it is, this local server is asked to deal with the missing service
 and update the database accordingly. If it is not, then the original server takes responsibility for the missing
 service and purges it from all the other servers.
 
@@ -175,8 +192,8 @@ Jerbil with only a couple of lines of code.
 
 Controlling a service is also made easy by the {JerbilService::Supervisor} class. This hides all of the actions needed
 to start a service. However, service control is made even easier by /usr/bin/jserviced, which starts any service
-given that services name, provided the files conform to certain protocols. The /usr/sbin/jservice-status and
-/usr/sbin/jservice-stop commands work in a similar manner.
+given that services name, provided the files conform to certain protocols. The /usr/local/sbin/jservice-status and
+/usr/local/sbin/jservice-stop commands work in a similar manner.
 
 Writing a client is similarly made easy by the {JerbilService::Client} class, which finds one or more services ready
 to be connected to and acted upon.
@@ -194,7 +211,7 @@ Jerbil is divided into two groups: the server and the services that use the serv
 
 The server consists of one main class: {Jerbil::Broker} and two data-type classes: 
 {Jerbil::Servers} and {Jerbil::ServiceRecord}. The Broker contains the main server code, 
-finding and registering with other servers, accepting and recording services and
+accepting and recording services and
 responding to queries about registered services. When a service registers with 
 the broker, the broker will also inform all of the other servers of that service. 
 The {Jerbil::Servers} class is used by the broker to record information
@@ -202,15 +219,6 @@ about a server and it provides convenience methods to connect to a server and a
 method to find the local server. The {Jerbil::ServiceRecord} class fulfils a similar 
 role for services.
 
-When a server starts up it uses the {Jerbil::Servers.find_servers} class method to
-search the network for any other servers. This method uses the NetAddr gem to create a
-list of network addresses from the parameters defined by {Jerbil::Config}, allowing
-users to limit jerbil addresses, both to control which machines get polled and to speed
-up the search. The jerbil port is polled using the standard library TCPSocket interface
-with a timeout (standard library again). When the server finds another server it first 
-does a minor security check (see below) and then obtains all of the services registered with
-that server and adds it to its service database.
-
 Jerbil is intended to be as reliable as possible - to survive any of the servers and 
 their services leaving the network unexpectedly. If a local client attempts to connect 
 to a remote service and fails, then the server will be asked to check with {Jerbil::Broker#service_missing?}. 
@@ -228,6 +236,30 @@ but they keep track of their own pids instead of relying on daemon. To stop the
 an attempt is made to call the stop method, which cleans up with all the other servers, 
 but failing that the pid is used to kill the server.
 
+### Server Discovery
+
+Jerbil servers automatically connect to each other on a LAN to make their management
+as easy as possible. They do this using brute force - polling each address on the network
+until they find a server and then registering with it. This all takes time and to avoid
+delaying the server itself, the discovery task is delegated to {Jerbil::Monitor}, which
+is run in a sub-process by the sbin/jerbild script.
+
+The {Jerbil::Monitor} class uses the {Jerbil::Servers.find_servers} class method to
+search the network for any other servers. This method uses the NetAddr gem to create a
+list of network addresses from the parameters defined by {Jerbil::Config}, allowing
+users to limit jerbil addresses, both to control which machines get polled and to speed
+up the search. The Jerbil port is polled using the standard library TCPSocket interface
+with a timeout (standard library again). When the server finds another server it first 
+does a minor security check (see below) and then obtains all of the services registered with
+that server and adds it to its service database.
+
+To avoid missing any servers that may have been busy during the short period waited on
+each port, the monitor process can loop round the discovery task multiple times (see
+defaults in {Jerbil::Config}. There is also a delay between each cycle that can be 
+set in {Jeckyl::Config}. Overall, the parameters for the monitor task can be tuned
+to ensure success on anything from a small subnet to a large network. The only thing
+it cannot do is monitor more than one network.
+
 ### Services
 
 Services are created by inheriting the {JerbilService::Base} class:
@@ -303,7 +335,7 @@ event that an exception is raised during the daemonization process.
 
 ## Dependencies
 
-A ruby compiler - works with 1.8.7.
+Tested on Ruby 1.9.3 and 2.0.0.
 
 Check the {file:Gemfile} for other dependencies.
 
@@ -326,7 +358,7 @@ See {file:History.txt} for a summary change history
 
 ## Copyright and Licence
 
-Copyright (c) 2012 Robert Sharp
+Copyright (c) 2014 Robert Sharp
 
 This software is licensed under the terms defined in {file:LICENCE.rdoc}
 

data/README_TESTING.md

@@ -1,6 +1,7 @@
 # Jerbil Testing
 
-Testing is generally through rspec, development/test versions of the Jerbil server, and service mocks.
+Testing is generally through rspec, development/test versions of the Jerbil server, and service mocks. 
+Rspec 3.1.0 has been used for all tests
 
 ## Basic tests
 

data/bin/jerbil

@@ -19,6 +19,7 @@
 #
 
 require 'optplus'
+require 'optplus/IO'
 require 'jerbil'
 require 'jellog/proxy'
 require 'colored'
@@ -26,6 +27,8 @@ require 'socket'
 
 class Jerbs < Optplus::Parser
   
+  include Optplus::IO
+  
   usage "[options] actions"
   
   description "Provide information on Jerbil servers and services"
@@ -45,7 +48,7 @@ class Jerbs < Optplus::Parser
       if FileTest.exists?(c) then
         set_option :config, c
       else
-        message :fatal, "the config file given does not exist"
+        alert "the config file given does not exist"
       end
     end
   
@@ -67,79 +70,104 @@ class Jerbs < Optplus::Parser
     puts "Error: No config file #{config_file ? config_file : '/etc/jerbil/jerbil.rb'}".red.bold
   end
   
+  
   describe :status, "show the status of the local server"
+  help :status, "  jerbil status",
+    "",
+    "Display status information about the local Jerbil server, such as its version",
+    "and the time it was started"
+    
   def status
-    puts "Checking for local Jerbil server running in env: #{@config[:environment]}"
+    say "Checking for local Jerbil server running in env: #{@config[:environment]}"
     jerbs = @server.connect
     started = jerbs.started
-    puts "  Jerbil server found, version: #{jerbs.version}".green
-    puts "  Server has been up since #{started.strftime('%d %b %Y at %H:%M')}"
-    puts "  and has had #{jerbs.registrations.to_s} registrations"
-  rescue Exception => err
-    puts "Server did not respond: #{err.message}".red.bold
+    say_ok "Jerbil server found, version: #{jerbs.version}".green
+    say "  Server has been up since #{started.strftime('%d %b %Y at %H:%M')}"
+    say "  and has had #{jerbs.registrations.to_s} registrations."
+  rescue Jerbil::JerbilError
+    show_err "Jerbil Server Error"
+  rescue DRb::DRbError
+    show_err "Server did not respond"
   end
   
+  
   describe :remotes, "list the remote servers on the network"
+  help :remotes, "  jerbil remotes [-vV]",
+    "List remote servers and optionally verify the server is running.",
+    "Verbose listing does little atm but display the Ruby version."
+    
   def remotes
-    puts "Checking for remote Jerbil servers running in env: #{@config[:environment]}"
-    begin
-      jerbs = @server.connect
-      remotes = jerbs.remote_servers
-      remotes.each do |remote|
-        if option?(:verify) then
-          begin
-            remote.connect.verify
-            puts "  #{remote.ident}".green
-          rescue
-            puts "  #{remote.ident}".red
-          end
-        else
-          puts "  #{remote.ident}".cyan
-        end #
-        if option? :verbose then
-          begin
-            puts "    Running under Ruby version: #{remote.connect.ruby_version}"
-          rescue
-            puts "    No info on ruby version from this server"
-          end
+    say "Checking for remote Jerbil servers running in env: #{@config[:environment]}"
+    jerbs = @server.connect
+    remotes = jerbs.remote_servers
+    remotes.each do |remote|
+      if option?(:verify) then
+        begin
+          remote.connect.verify
+          say_ok "  #{remote.ident}"
+        rescue
+          say "  #{remote.ident}".red
+        end
+      else
+        say "  #{remote.ident}".cyan
+      end #
+      if option? :verbose then
+        begin
+          say "    Jerbil server version: #{remote.version}"
+          say "    Running under Ruby version: #{remote.connect.ruby_version}"
+          say "    Remote server has been up since #{remote.started.strftime('%d %b %Y at %H:%M')}"
+        rescue
+          say "    No info on ruby version from this server"
         end
       end
-    rescue Exception => err
-      puts "  Server did not respond: #{err.message}".red.bold
     end
+    unless remotes && remotes.length > 0
+      # no remotes
+      warn "There are no remote servers registered at this time"
+    end
+      
+  rescue Jerbil::JerbilError
+    show_err "Jerbil Server Error"
+  rescue DRb::DRbError
+    show_err "Jerbil Server did not respond"
+    
   end
   
   describe :services, "list the services registered with Jerbil"
+  help :services, "  jerbil services [-avV -h <hostname>]",
+    "Display the services registered with the local Jerbil Server.",
+    "By default, these are just the local services. To see services",
+    "on remote servers use the -a switch. You can also see just those",
+    "services on a given host with the -h switch. To verify that the services",
+    "are operating, use the -v switch."
+    
   def services
     jerbs = @server.connect
     local_host = Socket.gethostname
     if option?(:all) then
-      services = jerbs.get_all(true)
+      servers = jerbs.get_all_by_server
+      servers.each_pair do |name, services|
+        puts "Services for #{name}:"
+        display_services(services, jerbs)
+      end
+      unless servers && servers.length > 0
+        warn "There are no servers and therefore services registered with Jerbil"
+      end
+      return
     elsif option?(:host) then
       services = jerbs.find(host: get_option(:host), ignore_access: true)
     else
       services = jerbs.find(host: local_host, ignore_access: true)
     end
-    services.each do |s|
-      puts "  #{s.name}[:#{s.env}]@#{s.host}:#{s.port}".cyan
-      if option? :verbose then
-        puts "    started at: #{s.registered_at.strftime('%d/%m/%y %H:%M')}"
-        puts "    accessed #{s.access_count.to_s} times, last time at: #{s.accessed_at.strftime('%d/%m/%y %H:%M')}"
-      end
-      if option? :verify then
-        if jerbs.service_missing?(s) then
-          puts "  #{s.ident} has failed and should be removed".red
-        else
-          puts "  #{s.ident} responded".green
-        end
-      end
-    end
-      
-  rescue Jerbil::JerbilServerError => err
-    puts "Jerbil Server did not respond: #{err.message}".red.bold
+    display_services(services, jerbs)
+          
+  rescue Jerbil::JerbilError
+    show_err "Jerbil Server Error"
+  rescue DRb::DRbError
+    show_err "Jerbil Server did not respond"
   end
   
-  describe :secret, "generate a secret key for the Jerbil Servers"
+  describe :secret, "generate a random secret key for the Jerbil Servers"
   def secret
     puts "Paste the following into the config files for each server"
     key = Digest::SHA1.hexdigest(Time.now.to_s + rand(12341234).to_s)
@@ -157,6 +185,7 @@ class Jerbs < Optplus::Parser
     end    
   end
   
+  # if hostname is not FQDN then get the FQDN
   def resolve_hostname(host)
     if host.split('.').length > 1 then
       return host
@@ -167,6 +196,36 @@ class Jerbs < Optplus::Parser
     end
   end
   
+  # helper method to display service information
+  def display_services(services, jerbs)
+    unless services && services.length > 0
+      warn "There are no services registered with Jerbil"
+      return false
+    end
+
+    services.each do |s|
+      puts "  #{s.name}[:#{s.env}]@#{s.host}:#{s.port}".cyan
+      if option? :verbose then
+        puts "    started at: #{s.registered_at.strftime('%d/%m/%y %H:%M')}"
+        puts "    accessed #{s.access_count.to_s} times, last time at: #{s.accessed_at.strftime('%d/%m/%y %H:%M')}"
+      end
+      if option? :verify then
+        if jerbs.service_missing?(s) then
+          puts "  #{s.ident} has failed and should be removed".red
+        else
+          puts "  #{s.ident} responded".green
+        end
+      end
+    end
+    
+  end # display_services
+  
+  # helper method to show errors
+  def show_err(msg)
+    alert msg + ':'
+    puts "  #{$!.message}"
+  end
+  
 end
 
 Jerbs.run!