traut 0.1.2 → 0.2.0

data/.gitignore

@@ -2,3 +2,4 @@
 .bundle
 Gemfile.lock
 pkg/*
+logs/

data/README.md

@@ -1,28 +1,14 @@
-Traut -- a real-time notification automaton
-===========================================
-
-It's not uncommon that an event will happen in a computer cluster for
-which a system administrator is needed to execute a program or two: a
-database goes down and it's time to switch to a hot-standby or there's
-a vital configuration fix that needs to go out faster than the ususal
-Puppet update interval. Maybe you have a specific human process that's
-kicked off when a 5xx HTTP error gets recorded by your
-httpd. Heretofore someone scans the logs periodically, though it could
-be done by a machine. In fact, all of this could be done by a machine.
-
-Traut is a program which listens to an AMQP queue and executes scripts
-found in local `/usr/local/bin/traut/` based on the message route. It
-is presumed that traut will be supported by a small legion of log
-watchers, daemon prodders and otherwise. Here at CarePilot, we have a
-daemon hooked up to Gerrit's ssh stream-events so that we might turn
-'change-merged' events into immediate project deployments, for
-example. See `samples/kili`. All payloads are delivered to the
-scripts' stdin.
-
-Traut cannot daemonize itself. We use
-[supervisord](http://supervisord.org/) to daemonize Traut; the code
-needed to achieve self-daemonization is outside of the core focus of
-this program.
+Traut -- a cron-like for AMQP
+=============================
+
+Unix cron's an ancient program that runs a script dependent on the passage
+time. Traut's the same way, except it keys off AMQP events instead. Use it to
+update your Puppet machines or log-rotate if nagios sends a disk-full
+warning. Significant number of possibilities.
+
+Traut cannot daemonize itself. Use [supervisord](http://supervisord.org/) or
+similar to daemonize Traut; the code needed to achieve self-daemonization is
+outside of the core focus of this program.
 
 Installation
 ------------
@@ -37,9 +23,31 @@ files. See the contents of etc/ for an example traut.conf. Note that
 response to `com.carepilot.event.code.review.app` events, making its
 full path `/tmp/traut/scripts/deploy/app`.
 
-Known Issues
-------------
+Configuration
+-------------
+
+The source distribution has an [etc/](etc/) directory with example
+configuration.
+
+Use
+---
+
+Run traut from the root of the project like this:
+
+    bundle exec bin/traut -C etc/traut.conf
+
+Now, using [hare](https://github.com/blt/hare):
+
+    $ hare --exchange_name traut --exchange_type topic --route_key whatthesum
+    --producer "that wasn't so bad"
+
+You should see nothing. Open up another terminal and
+
+    $ hare --exchange_name traut --exchange_type topic --route_key 'whatthesum.exited'
+
+Note that the route key is the same as before, save that '.exited' has been
+appended to it. Listen to this channel if you need notification of
+success. Run both hare commands in the order presented, notice the second print
+a json hash.
 
-* traut has no ability to reload its configuration or restart. It must
-  be killed and started.
-* traut doesn't do log rotation. Run logrotate on your system.
+See the sample configuration file for more details.

data/bin/traut

@@ -1,6 +1,33 @@
 #!/usr/bin/env ruby
-# Traut command line interface script.
-# Run <tt>traut -h</tt> to get more usage.
+
+require 'optparse'
+require 'logger'
 
 require 'traut'
-Traut::Runner.new(ARGV).run!
+
+# Default options
+options = Traut.defaults
+
+p = OptionParser.new do |opts|
+  opts.banner = "Usage: traut [options]"
+  opts.on("-C", "--config FILE", "Load options from config file") {
+    |file| options['config'] = file
+  }
+  opts.on_tail("-h", "--help", "Show this message.") do
+    puts opts; exit
+  end
+  opts.on_tail("-v", "--version", "Show version") {
+    puts Traut::VERSION; exit
+  }
+end
+p.parse! ARGV
+
+@immortal = true
+trap('INT') { @immortal = false; EM.stop }
+trap('HUP') { EM.stop }
+
+while @immortal do
+  EventMachine.run do
+    Traut::Application.new(:options => options).run
+  end
+end

data/etc/events.d/summer.yml

@@ -0,0 +1,7 @@
+- event: whatthesum
+  command: /usr/bin/sha512sum
+  user: blt
+  group: blt
+
+- event: passiton
+  command: cat > /tmp/bleh.txt

data/etc/traut.conf

@@ -0,0 +1,16 @@
+amqp:
+  host: localhost
+  port: 5672
+  vhost: '/'
+  username: 'guest'
+  password: 'guest'
+  exchange: 'traut'
+  queue: ''
+
+debug: true
+logdir: './logs/'
+include: './events.d'
+
+events:
+  - event: need.listing
+    command: /bin/ls

data/lib/traut/application.rb

@@ -0,0 +1,65 @@
+require 'yaml'
+require 'amqp'
+
+module Traut
+
+  class Application
+    def initialize(params)
+      @options = params[:options] || raise('parameter :options required')
+    end
+
+    # Parse the arguments and run the program. Exit on error.
+    def run
+      load_options
+
+      @logger = Logger.new File.join( File.absolute_path(@options['logdir']), 'traut.log')
+      @logger.level = boolean(@options['debug']) ? Logger::DEBUG : Logger::INFO
+
+      ## NOTE: Have to start AMQP connection out here.
+      amqp = @options['amqp']
+
+      AMQP.connect(:host => amqp['host'], :port => amqp['port'], :vhost => amqp['vhost'],
+        :username => amqp['username'], :password => amqp['password']) do |connection|
+        @logger.info "Traut #{Traut::VERSION} started"
+        channel  = AMQP::Channel.new(connection)
+        exchange = channel.topic(amqp['exchange'] || 'traut')
+
+        Traut::Server.new(:channel => channel, :exchange => exchange,
+          :events => @options['events'], :log => @logger).run
+      end
+    end
+
+    private
+    def boolean(string)
+      return true if string== true || string =~ (/(true|t|yes|y|1)$/i)
+      return false
+    end
+
+    def abs(p)
+      File.absolute_path(p)
+    end
+
+    def abs?(p)
+      abs(p) == p
+    end
+
+    def mung_config_path(includedir, config)
+      # if includedir is absolute do nothing else
+      includedir if abs?(includedir)
+      # else take abs-dirname of config and append includedir
+      File.join( abs(File.dirname(config)), includedir )
+    end
+
+    def load_options
+      YAML.load_file(@options['config']).each {
+        |key, value| @options[key] = value
+      }
+      includedir =  mung_config_path(@options['include'], @options['config'])
+      Dir.open(includedir).each do |f|
+        ff = File.join(includedir, f)
+        @options['events'] += YAML.load_file(ff) if File.file?(ff)
+      end
+    end
+
+  end
+end

data/lib/traut/server.rb

@@ -1,41 +1,54 @@
-require 'amqp'
+require 'json'
 require 'systemu'
 
 module Traut
 
   class Server
-    def initialize(amqp, actions, log)
-      @amqp = amqp
-      @actions = actions
-      @log  = log
+    def initialize(params)
+      @channel = params[:channel] || raise('parameter :channel required')
+      @exchange = params[:exchange] || raise('parameter :exchange required')
+      @events = params[:events] || raise('parameter :events required')
+      @log = params[:log] || raise('parameter :log required')
     end
 
-    def loop
-      EventMachine.run do
-        AMQP.connect(:host => @amqp[:host], :port => @amqp[:port]) do |connection|
-          @log.info "Connected to AMQP at #{@amqp[:host]}:#{@amqp[:port]}"
-          channel  = AMQP::Channel.new(connection)
-          exchange = channel.topic('traut')
-
-          @actions.each { |route, script|
-            @log.info("Registering #{script} for route #{route}")
-            channel.queue("").bind(exchange, :routing_key => route).subscribe do |headers, payload|
-              status, stdout, stderr = systemu script, 'stdin' => payload
-              if status.exitstatus != 0
-                @log.error("[#{script}] exit status: #{status.exitstatus}")
-                @log.error("[#{script}] stdout: #{stdout.strip}")
-                @log.error("[#{script}] stderr: #{stderr.strip}")
-              else
-                @log.info("[#{script}] exit status: #{status.exitstatus}")
-                @log.info("[#{script}] stdout: #{stdout.strip}")
-                @log.info("[#{script}] stderr: #{stderr.strip}")
-              end
-            end
-          }
-
-        end
+    # :: () -> ()
+    def run
+      subscribe('#') do |headers, payload|
+        @log.debug("Noted the reception of message with route '#{headers.routing_key}'.")
       end
+
+      @events.each do |event|
+        route, script, user, group = event['event'], event['command'], event['user'], event['group']
+        @log.debug("Registering #{script} to run as #{user}:#{group} for event #{route}")
+
+        subscribe(route) do |headers, payload|
+          Traut.spawn(:user => user, :group => group, :command => script,
+            :payload => payload, :logger => @log) do |status, stdout, stderr|
+            condition = 0 == status.exitstatus ? :debug : :error
+            result = {:exitstatus => status.exitstatus, :stdout => stdout.strip, :stderr => stderr.strip}
+            @log.send(condition, "[#{script}] #{result}")
+            publish(result.to_json, headers.routing_key)
+          end
+        end # channel.queue
+
+      end # eventmap.each
+    end # run
+
+    private
+
+    # :: string -> string
+    def finished_route(key)
+      [key, 'exited'].join('.')
     end
 
-  end
+    def publish(msg, route)
+      @exchange.publish msg, :routing_key => finished_route(route)
+    end
+
+    def subscribe(route, &block)
+      @channel.queue('').bind(@exchange, :routing_key => route).subscribe(&block)
+    end
+
+  end # Server
+
 end

data/lib/traut/spawn.rb

@@ -0,0 +1,55 @@
+require 'systemu'
+
+module Traut
+  def self.spawn(params, &block)
+    uid = params[:user].nil? ? Process::UID.eid : Etc::getpwnam(params[:user])[:uid]
+    gid = params[:group].nil? ? Process::GID.eid : Etc::getgrnam(params[:group])[:gid]
+    command = params[:command] || require('parameter :command is required')
+    payload = params[:payload]
+
+    s = Spawn.new(params[:logger])
+    s.spawn(uid, gid, command, payload, block)
+  end
+
+  class Spawn
+    def initialize(log)
+      @log = log
+    end
+
+    def spawn(uid, gid, command, payload, block)
+      runas(uid, gid) do
+        # Why do I use systemu? Have a look at this:
+        ## http://stackoverflow.com/questions/8998097/how-do-i-close-eventmachine-systems-side-of-an-stdin-pipe
+        status, stdout, stderr = systemu command, 0=>payload
+        block.call(status, stdout, stderr)
+
+        # If you have an answer for that, consider enabling the following code,
+        # keeping in mind that you need to figure out a way to get stderr back
+        # as well.
+
+        # EM.system command, proc{ |p| msg(p, payload) } do |stdout,status|
+        #     @log.debug("#{stdout} :: #{status}")
+        # end
+      end
+    end
+
+    private
+    def msg(process, m)
+      process.send_data(m + "\n")
+    end
+
+    def runas(uid, gid, &block)
+      cur_uid = Process::UID.eid
+      cur_gid = Process::GID.eid
+
+      begin
+        Process::UID.change_privilege(uid)
+        Process::GID.change_privilege(gid)
+        block.call
+      ensure
+        Process::UID.change_privilege(cur_uid)
+        Process::GID.change_privilege(cur_gid)
+      end
+    end
+  end
+end

data/lib/traut/version.rb

@@ -1,3 +1,3 @@
 module Traut
-  VERSION = "0.1.2"
+  VERSION = "0.2.0"
 end

data/lib/traut.rb

@@ -1,11 +1,23 @@
-require "traut/version"
+require 'traut/version'
+require 'ostruct'
+require 'eventmachine'
 
 module Traut
   ROOT = File.expand_path(File.dirname(__FILE__))
 
-  autoload :Runner,             "#{ROOT}/traut/runner"
-  autoload :Server,             "#{ROOT}/traut/server"
-  autoload :Daemon,             "#{ROOT}/traut/daemon"
+  autoload :Server,      "#{ROOT}/traut/server"
+  autoload :Application, "#{ROOT}/traut/application"
+
+  # Provide the base option sets for all Textme daemons and their
+  # defaults.
+  def self.defaults
+    {
+      'config' => './traut.conf',
+      'logdir' => './logs/',
+      'debug'  => true
+    }
+  end
 end
 
 require "#{Traut::ROOT}/traut/version"
+require "#{Traut::ROOT}/traut/spawn"

data/traut.gemspec

@@ -6,10 +6,10 @@ Gem::Specification.new do |s|
   s.name        = "traut"
   s.version     = Traut::VERSION
   s.authors     = ["Brian L. Troutwine"]
-  s.email       = ["brian.troutwine@carepilot.com"]
-  s.homepage    = "https://github.com/CarePilot/Traut"
-  s.summary     = %q{Turns AMQP events to system command execution}
-  s.description = %q{Traut is a configurable daemon for running localhost commands in response to events generated elsewhere. AMQP is used as the interchange. Traut can make application deployments in response to code checkins, automate database failover and anything else that can be scripted. It needs only companions to pump events through the 'traut' exchange.}
+  s.email       = ["brian@troutwine.us"]
+  s.homepage    = "https://github.com/blt/traut"
+  s.summary     = %q{Traut is like cron for AMQP events.}
+  s.description = %q{Unix cron is a venerable program that turns the passage of time into program invokation. Traut does the same, but using AMQP events to trigger execution. AMQP message payloads are written to the stdin of invoked commands.}
 
   s.rubyforge_project = "traut"
 
@@ -18,8 +18,14 @@ Gem::Specification.new do |s|
   s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   s.require_paths = ["lib"]
 
-  s.add_runtime_dependency "daemons", '~> 1.1'
+  ## Someday...
+  # s.add_development_dependency "rspec", '~> 2.8.0'
+  # s.add_development_dependency 'guard'
+  # s.add_development_dependency 'guard-rspec'
+  # s.add_development_dependency 'simplecov'
+
   s.add_runtime_dependency "amqp", '>= 0.8.0'
   s.add_runtime_dependency "systemu", '~> 2.4'
+  s.add_runtime_dependency 'json', '~> 1.6.5'
 
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: traut
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.2.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,48 +9,46 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2011-11-14 00:00:00.000000000Z
+date: 2012-01-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
-  name: daemons
-  requirement: &71562050 !ruby/object:Gem::Requirement
+  name: amqp
+  requirement: &75597390 !ruby/object:Gem::Requirement
     none: false
     requirements:
-    - - ~>
+    - - ! '>='
       - !ruby/object:Gem::Version
-        version: '1.1'
+        version: 0.8.0
   type: :runtime
   prerelease: false
-  version_requirements: *71562050
+  version_requirements: *75597390
 - !ruby/object:Gem::Dependency
-  name: amqp
-  requirement: &71561500 !ruby/object:Gem::Requirement
+  name: systemu
+  requirement: &75597140 !ruby/object:Gem::Requirement
     none: false
     requirements:
-    - - ! '>='
+    - - ~>
       - !ruby/object:Gem::Version
-        version: 0.8.0
+        version: '2.4'
   type: :runtime
   prerelease: false
-  version_requirements: *71561500
+  version_requirements: *75597140
 - !ruby/object:Gem::Dependency
-  name: systemu
-  requirement: &71560720 !ruby/object:Gem::Requirement
+  name: json
+  requirement: &75596790 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: '2.4'
+        version: 1.6.5
   type: :runtime
   prerelease: false
-  version_requirements: *71560720
-description: Traut is a configurable daemon for running localhost commands in response
-  to events generated elsewhere. AMQP is used as the interchange. Traut can make application
-  deployments in response to code checkins, automate database failover and anything
-  else that can be scripted. It needs only companions to pump events through the 'traut'
-  exchange.
+  version_requirements: *75596790
+description: Unix cron is a venerable program that turns the passage of time into
+  program invokation. Traut does the same, but using AMQP events to trigger execution.
+  AMQP message payloads are written to the stdin of invoked commands.
 email:
-- brian.troutwine@carepilot.com
+- brian@troutwine.us
 executables:
 - traut
 extensions: []
@@ -62,15 +60,16 @@ files:
 - README.md
 - Rakefile
 - bin/traut
-- etc/actions/app.action
-- etc/traut.conf.sample
+- etc/events.d/summer.yml
+- etc/traut.conf
 - lib/traut.rb
-- lib/traut/runner.rb
+- lib/traut/application.rb
 - lib/traut/server.rb
+- lib/traut/spawn.rb
 - lib/traut/version.rb
 - samples/kili
 - traut.gemspec
-homepage: https://github.com/CarePilot/Traut
+homepage: https://github.com/blt/traut
 licenses: []
 post_install_message: 
 rdoc_options: []
@@ -93,5 +92,5 @@ rubyforge_project: traut
 rubygems_version: 1.8.10
 signing_key: 
 specification_version: 3
-summary: Turns AMQP events to system command execution
+summary: Traut is like cron for AMQP events.
 test_files: []

data/etc/actions/app.action

@@ -1 +0,0 @@
-com.carepilot.event.code.review.app: deploy/app

data/etc/traut.conf.sample

@@ -1,7 +0,0 @@
-amqp:
-    :host: localhost
-    :port: 5672
-logs: /tmp/traut.log
-pid:  /tmp/traut.pid
-scripts: /tmp/traut/scripts
-action_dir: /tmp/traut/actions

data/lib/traut/runner.rb

@@ -1,124 +0,0 @@
-require 'optparse'
-require 'yaml'
-require 'logger'
-
-module Traut
-  trap(:INT) { puts; exit }
-
-  # CLI runner. Parse options, run program.
-  class Runner
-    COMMANDS = %w(start stop restart status)
-
-    attr_accessor :options   # parsed options
-
-    def initialize(argv)
-      @argv = argv
-
-      # Default options
-      @options = {
-        :amqp => {
-          :host => 'localhost',
-          :port => '5672'
-        },
-        :logs       => '/var/log/traut.log',
-        :config     => '/etc/traut/traut.conf',
-        :action_dir => '/etc/traut/actions/',
-        :scripts    => '/usr/local/bin/traut',
-        :debug      => false,
-        :actions    => nil,
-      }
-
-      parse!
-    end
-
-    def parser
-      @parser ||= OptionParser.new do |opts|
-        opts.banner = "Usage: traut [options]"
-        opts.separator ""
-        opts.on("-A", "--amqp HOST", "The AMQP server host") {
-          |host| @options[:amqp][:host] = host
-        }
-        opts.on("-P", "--amqp_port PORT", "The AMQP server host port") {
-          |port| @options[:amqp][:port] = port
-        }
-        opts.on("-C", "--config FILE", "Load options from config file") {
-          |file| @options[:config] = file
-        }
-        opts.on("-s", "--scripts DIR", "Location of traut scripts directory"){
-          |scripts| @options[:scripts] = scripts
-        }
-        opts.on("-a", "--actions DIR", "Location of traut actions directory"){
-          |acts| @options[:action_dir] = acts
-        }
-        opts.on('-l', '--logs LOG', "Location of log directory location") {
-          |log| @options[:logs] = log
-        }
-        opts.on('--debug', 'Enable debug logging') {
-          |debug| @options[:debug] = true
-        }
-        opts.on_tail("-h", "--help", "Show this message.") do
-          puts opts
-          exit
-        end
-        opts.on_tail("-v", "--version", "Show version") {
-          puts Traut::VERSION; exit
-        }
-      end
-    end
-
-    # Parse command options out of @argv
-    def parse!
-      parser.parse! @argv
-      @command   = @argv.shift
-      @arguments = @argv
-    end
-
-    # Parse the arguments and run the program. Exit on error.
-    def run!
-      load_options_from_config_file!
-
-      log = Logger.new(@options[:logs])
-      log.level = @options[:debug] ? Logger::DEBUG : Logger::INFO
-
-      actions = @options[:actions]
-      actions = actions.merge(actions) { |k,v|
-        File.join(@options[:scripts], v)
-      }
-
-      actions.each { |route, script|
-        if ! File.exists?(script)
-          log.error("#{script} does not exist on disk")
-          exit 1
-        elsif ! File.executable?(script)
-          log.error("#{script} exists on disk but is not executable.")
-          exit 1
-        else
-          log.info("#{script} recognized on disk and executable.")
-        end
-      }
-
-      server = Server.new(@options[:amqp], actions, log)
-
-      server.loop
-    end
-
-    private
-
-    def load_options_from_config_file!
-      if file = @options.delete(:config)
-        YAML.load_file(file).each {
-          |key, value| @options[key.to_sym] = value
-        }
-        dir = Dir.open(@options[:action_dir])
-        @options[:actions] = {}
-        dir.each do |f|
-          af = File.join(@options[:action_dir], f)
-          if ! File.directory? af
-            YAML.load_file(af).each { |k,v| @options[:actions][k] = v }
-          end
-        end
-      end
-    end
-
-  end
-end