kennethkalmer-daemon-kit 0.1.7.9 → 0.1.7.10

data/Configuration.txt

@@ -57,3 +57,54 @@ instance call also be modified from the command line using the special
 
 This happens after <em>config/environment.rb</em> is processed, so all
 command line arguments will overwrite those values.
+
+=== Daemon umask
+
+By default daemon processes run with a umask of 022, but this can be changed
+on the command line or in +config/environment.rb+.
+
+To set a more restrictive umask via command line arguments, you can start your
+daemon like this:
+
+  $ ./bin/daemon start --config umask=0077
+
+Or the same in +config/environment.rb+
+
+  DaemonKit::Initializer.run do |config|
+    # ...
+
+    # restrictive umask
+    config.umask = 0077
+
+    # ...
+  end
+
+=== Privilege Separation
+
+By default daemon processes run as the user that starts them, inheriting all
+their privileges (or lack thereof). Getting daemon-kit to drop privileges
+can currently only be done using command-line parameters, and only works
+reliable on *nix (OSX seemed cranky at the time of testing).
+
+  $ ./bin/daemon start --config user=nobody --config group=nobody
+
+Privileges are dropped at the earliest possible phase of starting the daemon.
+
+Things to note on privilege separation:
+
+* You generally have to be root to be able to perform this
+* File system permissions for +log/+ needs to be correct
+* Daemon-kit will only shed privileges on the +start+ command, not on +run+
+* Make sure your code is secure if accepting stuff from the outside world
+* The daemon will continue to run if it failed, this is because the feature is experimental and could change in the future.
+* The damon logs the reduced privileges in the log file shortly after booting, please check it carefully
+
+The implementation stems from the advice given by Joe Damato on his blog post
+http://timetobleed.com/tag/privilege-escalation/
+
+IMPORTANT NOTE FOR OSX USERS:
+
+Testing on my iBook with OSX 10.5.8 using Ruby 1.8.6-p287 failed to drop
+privileges correctly because of the 'nobody' user's UID being too large
+(Bignum), however testing with Ruby 1.9.1-p129 on OSX 10.5.8 did work as
+expected.

data/History.txt

@@ -1,3 +1,11 @@
+== 0.1.7.10 (2009-08-12)
+
+* Ruote remote participants
+* Allow process umask to be configured, defaults to 022
+* Updates to DaemonKit::Config hashes
+* Fixed argument parsing bug (reported by Mathijs Kwik (bluescreen303)
+* Support for privilege separation (See Configuration.txt)
+
 == 0.1.7.9 2009-06-22
 
 * Backtraces only logged on unclean shutdown

data/Manifest.txt

@@ -6,6 +6,7 @@ Manifest.txt
 PostInstall.txt
 README.rdoc
 Rakefile
+RuoteParticipants.txt
 TODO.txt
 app_generators/daemon_kit/USAGE
 app_generators/daemon_kit/daemon_kit_generator.rb
@@ -65,6 +66,14 @@ daemon_generators/rspec/templates/spec.rb
 daemon_generators/rspec/templates/spec/spec.opts
 daemon_generators/rspec/templates/spec/spec_helper.rb
 daemon_generators/rspec/templates/tasks/rspec.rake
+daemon_generators/ruote/USAGE
+daemon_generators/ruote/ruote_generator.rb
+daemon_generators/ruote/templates/config/amqp.yml
+daemon_generators/ruote/templates/config/initializers/ruote.rb
+daemon_generators/ruote/templates/config/ruote.yml
+daemon_generators/ruote/templates/lib/daemon.rb
+daemon_generators/ruote/templates/lib/sample.rb
+daemon_generators/ruote/templates/libexec/daemon.rb
 lib/daemon_kit.rb
 lib/daemon_kit/abstract_logger.rb
 lib/daemon_kit/amqp.rb
@@ -83,11 +92,15 @@ lib/daemon_kit/em.rb
 lib/daemon_kit/error_handlers/base.rb
 lib/daemon_kit/error_handlers/hoptoad.rb
 lib/daemon_kit/error_handlers/mail.rb
+lib/daemon_kit/exceptions.rb
 lib/daemon_kit/initializer.rb
 lib/daemon_kit/jabber.rb
 lib/daemon_kit/nanite.rb
 lib/daemon_kit/nanite/agent.rb
 lib/daemon_kit/pid_file.rb
+lib/daemon_kit/ruote_participants.rb
+lib/daemon_kit/ruote_pseudo_participant.rb
+lib/daemon_kit/ruote_workitem.rb
 lib/daemon_kit/safety.rb
 lib/daemon_kit/tasks.rb
 lib/daemon_kit/tasks/environment.rake
@@ -122,6 +135,7 @@ test/test_generator_helper.rb
 test/test_helper.rb
 test/test_jabber_generator.rb
 test/test_nanite_agent_generator.rb
+test/test_ruote_generator.rb
 vendor/tmail-1.2.3/tmail.rb
 vendor/tmail-1.2.3/tmail/address.rb
 vendor/tmail-1.2.3/tmail/attachments.rb

data/README.rdoc

@@ -13,9 +13,11 @@ Using simple built-in generators it is easy to created evented and non-evented d
 
 Supported generators:
 
-* Evented and non-evented Jabber Bot (coming next)
-* Evented and non-evented loops (coming soon)
-* Queue poller (SQS, AMQP, etc) (coming soon)
+* XMPP bot (non-evented)
+* AMQP consumer (evented)
+* Nanite agent
+* Cron-style daemon
+* ruote remote participants
 
 == Features/Problems
 
@@ -24,17 +26,21 @@ Supported generators:
 
 == Synopsis
 
-  $ daemon-kit [/path/to/your/daemon] [options]
+  $ daemon_kit -h
+
+Get some help
+
+  $ daemon_kit [/path/to/your/daemon] [options]
 
 The above command generates a skeleton daemon environment for you to adapt.
 
-  $ daemon-kit [/path/to/your/daemon] -i jabber
+  $ daemon_kit [/path/to/your/daemon] -i jabber
 
 Use the 'jabber' generator instead of the default one.
 
 == Generators
 
-Currently only two generators exist, a 'default' generator and a 'jabber' generator.
+Currently six generators exist: default, jabber, amqp, cron, nanite & ruote
 
 The default generator creates a simple daemon with an infinite loop inside that you can adapt.
 
@@ -50,6 +56,14 @@ The cron generator creates a simple daemon that leverages the "rufus-scheduler":
 
 The AMQP generator creates a simple daemon that has all the stub code and configuration in place to help you write AMQP consumers quickly and effectively. The generated daemon relies on the presence of the "amqp":http://github.com/tmm1/amqp gem.
 
+=== Nanite Agent Generator
+
+The "nanite":http://github.com/ezmobius/nanite agent generator gets you up and running with nanite agents very quickly.
+
+=== ruote Remote Participants
+
+The "ruote":http://openwfe.rubyforge.org remote participant generator speeds up the development of workflow participants that run outside of the Ruby process that houses the engine. Daemon-kit handles all the communication and delegation logic, allowing you to focus purely on your participant's activities.
+
 == Requirements
 
 * Ruby 1.8.6
@@ -61,8 +75,10 @@ The AMQP generator creates a simple daemon that has all the stub code and config
 Depending on the generator you choose for your daemon, it might require additional gems to run.
 
 * jabber - xmpp4r-simple[http://xmpp4r-simple.rubyforge.org]
-* cron - rufus-scheduler[http://github.com/jmettraux/rufus-scheduler]
+* cron - rufus-scheduler[http://github.com/jmettraux/rufus-scheduler] (at least version 2.0.0)
 * amqp - amqp[http://github.com/tmm1/amqp]
+* nanite - nanite[http://github.com/ezmobius/nanite]
+* ruote - none, although ruote[http://openwfe.rubyforge.org] should probably be running somewhere
 
 == Install
 
@@ -84,6 +100,9 @@ Stable versions, when released are available directly from Rubyforge:
 
 * Configuration.txt
 * Deployment.txt
+* Logging.txt
+* RuoteParticipants.txt
+* http://www.opensourcery.co.za/tag/daemon-kit/
 
 == License
 

data/Rakefile

@@ -1,9 +1,16 @@
-%w[rubygems rake rake/clean fileutils newgem rubigen].each { |f| require f }
+require 'rubygems'
+gem 'hoe', '>= 2.1.0'
+require 'hoe'
+require 'fileutils'
 require File.dirname(__FILE__) + '/lib/daemon_kit'
 
+Hoe.plugin :newgem
+Hoe.plugin :website
+# Hoe.plugin :cucumberfeatures
+
 # Generate all the Rake tasks
 # Run 'rake -T' to see list of generated tasks (from gem root directory)
-$hoe = Hoe.new('daemon-kit', DaemonKit::VERSION) do |p|
+$hoe = Hoe.spec('daemon-kit') do |p|
   p.summary = 'Daemon Kit aims to simplify creating Ruby daemons by providing a sound application skeleton (through a generator), task specific generators (jabber bot, etc) and robust environment management code.'
   p.developer('Kenneth Kalmer', 'kenneth.kalmer@gmail.com')
   p.changes              = p.paragraphs_of("History.txt", 0..1).join("\n\n")

data/RuoteParticipants.txt

@@ -0,0 +1,113 @@
+= Writing remote ruote participants with daemon-kit
+
+daemon-kit is an ideal housing for remote ruote participants, providing a lot
+of convenience in terms of receiving and sending workitems, delegating work to
+pseudo-participant classes, handling configuration of the communication channel
+between ruote and the remote participant, and much more.
+
+== What is ruote?
+
+Ruote is a Ruby workflow engine. It is a powerful tool for defining, running and
+orchestrating business processes.
+
+* http://openwferu.rubyforge.org/
+* http://www.opensourcery.co.za/2009/03/04/ruote-in-20-minutes/
+* http://www.opensourcery.co.za/2009/07/06/driving-business-processes-in-ruby/
+
+== What are remote participants?
+
+Remote participants are participants that perform their work in a different
+Ruby processes from the one running the engine. This is useful in two cases,
+possibily many more, that involves autonomous participants.
+
+* Autonomous participants located on remote servers, driven by identity
+* Clustering autonomous participants to process workitems from a queue
+
+To learn more about the differences between local and remote participants
+please see http://openwferu.rubyforge.org/part.html
+
+Currently on the AMQP components are in place in daemon-kit, with XMPP coming
+soon.
+
+== Creating a remote participant with daemon-kit
+
+Generate your daemon using the 'ruote' generator:
+
+  $ daemon_kit partd -i ruote
+
+Make sure you have the JSON gem install, and the AMQP gem as well.
+
+== Configuring the daemon
+
+You need to review +config/ruote.yml+ to specify the AMQP queues that the daemon
+will subscribe to for receiving workitems. You'll also need to configure the 
+AMQP gem be updating +config/amqp.yml+
+
+The generated daemon in +libexec/+ already defaults to using AMQP as a transport
+for workitems.
+
+== Writing pseudo-participants
+
+Pseudo-participants in daemon-kit are pure Ruby classes. Implement your classes
+in +lib/+ and require them from +lib/<daemon_name>.rb+.
+
+Register your classes as pseudo-participants by registering them in the daemon
+file in +libexec+, just as the Sample class is registered in the generated
+code. Your class will be instantiated upon registration, and will be re-used
+for every incoming workitem passed to it.
+
+All your public methods in the pseudo-participant classes should be accept
+a single parameter, which is a ruote workitem in pure Hash form.
+
+== Wiring up the remote participant in ruote
+
+See the complete code here: http://gist.github.com/144861
+
+A sample process definition might look something like this:
+
+  class QuoteProcess < OpenWFE::ProcessDefinition
+    sequence do
+      kit :command => '/sample/quote', :queue => 'work1'
+
+      console
+    end
+  end
+
++kit+ in the above process definition is registered with the ruote engine as an
+AMQPParticipant. The AMQPParticipant delivers workitems to the specified AMQP
+queue.
+
+Based on the values in +config/ruote.yml+, your daemon will be subscribed to
+those queues.
+
+The second part is delegating the workitem inside the daemon to the correct
+pseudo-participant. This is handled by the +:command+ parameter in the process
+definition. DaemonKit::Workitem looks at the command parameter of the incoming
+workitem, and then finds a registered pseudo-participant instance and calls the
+requested method on that class.
+
+The +:command+ parameter follows the following convention (stolen shamelessly
+from Nanite):
+
+  :command => '/class_name/method_name'
+
+When classes are registered, the name of the class is downcased and camel-case
+words are separated by underscores. Method names are not changed, but methods
+are required to be public.
+
+== Processing workitems and replying to the engine
+
+The methods called in the pseudo-participants receive a single parameter, a
+ruote workitem as a hash. The participant is then free to analyze the hash
+and perform the appropriate actions required. The return value of the method
+is discarded, and the workitem is returned back to the engine. If the method
+modified the workitem, these changes will be sent along as well.
+
+== Random other notes
+
+Apart from configuring the AMPQ client (or XMPP in future) and the ruote.yml
+file, daemon developers don't need to worry about anything related to receiving
+workitems or sending replies.
+
+Our aim is to allow you to swap between participants on both sides of the
+transport without changing any of your code.

data/app_generators/daemon_kit/daemon_kit_generator.rb

@@ -3,7 +3,7 @@ class DaemonKitGenerator < RubiGen::Base
   DEFAULT_SHEBANG = File.join(Config::CONFIG['bindir'],
                               Config::CONFIG['ruby_install_name'])
 
-  VALID_GENERATORS = ['default', 'jabber', 'cron', 'amqp', 'nanite_agent']
+  VALID_GENERATORS = ['default', 'jabber', 'cron', 'amqp', 'nanite_agent', 'ruote']
 
   DEPLOYERS = ['none', 'capistrano']
 
@@ -76,13 +76,13 @@ class DaemonKitGenerator < RubiGen::Base
       m.directory "config/post-daemonize"
       m.file      "config/post-daemonize/readme", "config/post-daemonize/readme"
       m.directory "script"
-      m.file      "script/destroy", "script/destroy"
-      m.file      "script/console", "script/console"
-      m.file      "script/generate", "script/generate"
+      m.file      "script/destroy", "script/destroy", script_options
+      m.file      "script/console", "script/console", script_options
+      m.file      "script/generate", "script/generate", script_options
 
       # Libraries
       m.directory "lib"
-      m.file "lib/daemon.rb", "lib/#{daemon_name}.rb"
+      m.file "lib/daemon.rb", "lib/#{daemon_name}.rb", :collision => :skip
 
       # Tasks
       m.directory "tasks"

data/daemon_generators/ruote/USAGE

@@ -0,0 +1,5 @@
+Description:
+  
+  
+Usage:
+  

data/daemon_generators/ruote/ruote_generator.rb

@@ -0,0 +1,67 @@
+class RuoteGenerator < RubiGen::Base
+
+  default_options :author => nil
+
+  attr_reader :name
+
+  def initialize(runtime_args, runtime_options = {})
+    super
+    usage if args.empty?
+    @name = args.shift
+    extract_options
+  end
+
+  def manifest
+    record do |m|
+      # Ensure appropriate folder(s) exists
+      m.directory ''
+
+      # Create stubs
+      # m.template           "template.rb.erb", "some_file_after_erb.rb"
+      # m.template_copy_each ["template.rb", "template2.rb"]
+      # m.template_copy_each ["template.rb", "template2.rb"], "some/path"
+      # m.file           "file", "some_file_copied"
+      # m.file_copy_each ["path/to/file", "path/to/file2"]
+      # m.file_copy_each ["path/to/file", "path/to/file2"], "some/path"
+
+      m.directory 'config'
+      m.template  'config/amqp.yml', 'config/amqp.yml'
+      m.template  'config/ruote.yml', 'config/ruote.yml'
+      m.directory 'config/initializers'
+      m.template  'config/initializers/ruote.rb', "config/initializers/#{name}.rb"
+
+      m.directory 'lib'
+      m.template  'lib/daemon.rb', "lib/#{name}.rb"
+      m.template  'lib/sample.rb', 'lib/sample.rb'
+      m.directory 'libexec'
+      m.template 'libexec/daemon.rb', "libexec/#{name}-daemon.rb"
+    end
+  end
+
+  protected
+    def banner
+      <<-EOS
+Creates a ...
+
+USAGE: #{$0} #{spec.name} name
+EOS
+    end
+
+    def add_options!(opts)
+      # opts.separator ''
+      # opts.separator 'Options:'
+      # For each option below, place the default
+      # at the top of the file next to "default_options"
+      # opts.on("-a", "--author=\"Your Name\"", String,
+      #         "Some comment about this option",
+      #         "Default: none") { |o| options[:author] = o }
+      # opts.on("-v", "--version", "Show the #{File.basename($0)} version number and quit.")
+    end
+
+    def extract_options
+      # for each option, extract it into a local variable (and create an "attr_reader :author" at the top)
+      # Templates can access these value via the attr_reader-generated methods, but not the
+      # raw instance variable value.
+      # @author = options[:author]
+    end
+end

data/daemon_generators/ruote/templates/config/amqp.yml

@@ -0,0 +1,30 @@
+# AMQP client configuration file for ruote remote participants. If you are not
+# planning to use the AMQP participant/listener pair in ruote, you can safely
+# delete this file.
+
+# These values will be used to configure the ampq gem, any values
+# omitted will let the gem use it's own defaults.
+
+# The configuration specifies the following keys:
+# * user - Username for the broker
+# * pass - Password for the broker
+# * host     - Hostname where the broker is running
+# * vhost    - Vhost to connect to
+# * port     - Port where the broker is running
+# * ssl      - Use ssl or not
+# * timeout  - Timeout
+
+defaults: &defaults
+  user: guest
+  pass: guest
+  host: localhost
+  vhost: /
+
+development:
+  <<: *defaults
+
+test:
+  <<: *defaults
+
+production:
+  <<: *defaults