termite 0.0.10 → 0.0.20

data/README.md

@@ -20,9 +20,10 @@ your Gemfile or on your gem command line.
 Logging Dynamically
 ===================
 
-Create a logger with something like:
+Create a logger with something like one of these:
 
   @logger = Termite::Logger.new
+  @logger = Termite::Logger.new :component => "MyLibrary"
 
 Then use it like a regular logger, possibly with options:
 
@@ -34,38 +35,54 @@ You can also pass in JSON data after your message and before your options:
   @logger.fatal("Pain and misery!", { :where_it_hurts => "elbow" }, :component => "WhinyLib")
   @logger.info("I ache", {:where_it_hurts => "kidney"}, :application => "NotMe", :component => "StoicLib")
 
-Termite also supports full Ruby Logger initialize parameters for backward compatibility:
+Termite also supports full Ruby Logger initialize parameters for
+backward compatibility:
 
   @logger = Termite::Logger.new("/var/lib/daily_termite_logs", "daily")
   @logger = Termite::Logger.new("/tmp/rotatable.txt", 15, 1024000)  # Up to 15 logs of size 1024000
 
+You can also use all the standard methods of Ruby logging:
+
+  @logger.log(Logger::INFO, "I feel Ruby-compatible")
+  @logger << "This message gets logged as INFO"
+
+Similarly, when using a file logger, the output should be very similar
+to what you'd get from a Ruby logger.  So a Termite logger is nearly a
+drop-in replacement for the Ruby logger, plus you get Syslog output
+and console output *in addition* to your existing to-file output.
+
 Log Level
 =========
 
-Termite loggers, like Ruby loggers, allow you to set the logger's level.  All events below that
-level will be silently discarded, and will also not be sent to non-syslog loggers.  You can change
-the level with "logger.level = Logger::WARN" and similar.
+Termite loggers, like Ruby loggers, allow you to set the logger's
+level.  All events below that level will be silently discarded, and
+will also not be sent to non-syslog loggers.  You can change the level
+with "logger.level = Logger::WARN" and similar.
 
-By default, Termite will log events of all severities to standard output, and events of at least
-severity :error to standard error.  You can set .stdout_level and .stderr_level just like setting
-.level above.  Level takes precedence over the other two and stderr_level takes precedence over
-stdout_level.
+By default, Termite will log events of all severities to standard
+output, and events of at least severity :error to standard error.  You
+can set .stdout_level and .stderr_level just like setting .level
+above.  Level takes precedence over the other two and stderr_level
+takes precedence over stdout_level.
 
 Non-Syslog Logging
 ==================
 
-If a filename or handle is specified in the constructor, Termite will instantiate a Ruby Logger
-with the same parameters and mirror all events to it.
+If a filename or handle is specified in the constructor, Termite will
+instantiate a Ruby Logger with the same parameters and mirror all
+events to it.
 
-You can pass in objects with an :add method to a Termite logger's "add_extra_logger" method, and
-from then on all events will be mirrored to that object.  This can be useful for chaining loggers
-if you need to.  Events below the Termite logger's level (see above) won't be mirrored.
+You can pass in objects with an :add method to a Termite logger's
+"add_extra_logger" method, and from then on all events will be
+mirrored to that object.  This can be useful for chaining loggers if
+you need to.  Events below the Termite logger's level (see above)
+won't be mirrored.
 
 Translating to SysLog
 =====================
 
-When writing to SysLog, Termite translates Ruby Logger severities into SysLog severities.  By
-default, this is the mapping:
+When writing to SysLog, Termite translates Ruby Logger severities into
+SysLog severities.  By default, this is the mapping:
 
 Logger   => SysLog
 :unknown => :alert
@@ -78,8 +95,9 @@ Logger   => SysLog
 Configuring with an Ecology
 ===========================
 
-Termite supports a standard Ecology file.  By default, it will look at the location of the current
-executable ($0) with extension .ecology.  So "bob.rb" would have "bob.ecology" next to it.
+Termite supports a standard Ecology file.  By default, it will look at
+the location of the current executable ($0) with extension .ecology.
+So "bob.rb" would have "bob.ecology" next to it.
 
 An Ecology is a JSON file of roughly this structure:
 
@@ -91,41 +109,77 @@ An Ecology is a JSON file of roughly this structure:
       "app_group": "SuperSpiffyGroup",
       "precedence": 7
     },
-    "console_print": "off",
-    "filename": "/tmp/bobo.txt",
-    "shift_age": 10,
-    "shift_size": 1024000,
-    "stderr_level": "fatal"
+    "sinks": [
+      {
+        "type": "stdout",
+        "color": "green",
+        "min_level": "debug",
+        "max_level": "warning"
+      },
+      {
+        "type": "stderr",
+        "color": "red",
+        "min_level": "error"
+      },
+      {
+        "type": "file",
+        "filename": "/tmp/bobo.txt",
+        "min_level": "warning",
+        "shift_age": 10,
+        "shift_size": 1024000
+      },
+      {
+        "type": "syslog",
+        "transport": "UDP"
+      },
+      {
+        "type": "hastur",
+        "udp_port": 9199,
+        "labels": {
+          "app_flavor": "vanilla",
+          "track_for": "jbhat"
+        }
+      }
+    ]
   }
 }
 
-Absolutely every part of it is optional, including the presence of the file at all.
+Absolutely every part of it is optional, including the presence of the
+file at all.
 
-You can override the application name, as shown above.  Other than the application name,
-all Termite-specific parameters are under the "logging" field, as above.
+You can override the application name, as shown above.  Other than the
+application name, all Termite-specific parameters are under the
+"logging" field, as above.
 
-The default_component is what application component is given for an add() call by default.
-If set, it can be removed with ":component => nil" for a given add() call.
+The default_component is what application component is given for an
+add() call by default.  If set, it can be removed with ":component =>
+nil" for a given add() call.
 
 Extra JSON fields are added to the JSON data of every add() call.
 
-Console_print can be set to "off" (or "no" or "0") to turn off Termite printing to stderr
-and stdout by default at different log levels.
+Console_print can be set to "off" (or "no" or "0") to turn off Termite
+printing to stderr and stdout by default at different log levels.
+
+Filename, shift_age and shift_size are the same as Ruby Logger's
+normal initialize parameters.  The first is the filename to log to,
+the second is how many total log files to keep, and the third is how
+large each log file can grow.  Or the second can be set to a value
+like "daily" or "monthly", and then the third is irrelevant.
 
-Filename, shift_age and shift_size are the same as Ruby Logger's normal initialize
-parameters.  The first is the filename to log to, the second is how many total log files
-to keep, and the third is how large each log file can grow.  Or the second can be set to
-a value like "daily" or "monthly", and then the third is irrelevant.
+You can also set level, stdout_level and stderr_level explained above.
+We allow numerical (syslog) values, or standard names of Ruby Logger
+severities.
 
-You can also set level, stdout_level and stderr_level explained above.  We allow numerical
-(syslog) values, or standard names of Ruby Logger severities.
+If you reset your Ecology (usually used for testing), you should
+recreate all your Termite::Logger objects.  The old ones will still
+have stale settings from the old Ecology's defaults and data.
 
 Releasing within Ooyala
 =======================
 
 Ooyalans, to release Termite to gems.sv2, use the following:
 
-  gem build
+  rake build
   rake _0.8.7_ -f ../ooyala_gems.rake gem:push termite-0.0.1.gem
 
 Change the version to the actual version you'd like to push.

data/TODO

@@ -1,3 +1,9 @@
 Ecology configuration:
-  * Extra loggers from Ecology
-  * Transport protocol from Ecology
+  * Load extra loggers from Ecology
+  * Load which transport protocol from Ecology
+
+When logging:
+  * Send call-site with data (configurable depth?)
+
+Environment variables:
+  * Add variable(s) to set console output when debugging

data/lib/termite.rb

@@ -1,8 +1,11 @@
 require "rubygems"
+require "termite/hastur_logger"
+require "termite/syslog_logger"
 require "termite/version"
 
 require "ecology"
 require "multi_json"
+require "rainbow"
 
 require "thread"
 require "syslog"
@@ -14,7 +17,7 @@ module Termite
     ##
     # Maps Logger severities to syslog(3) severities.
 
-    LOGGER_MAP = {
+    LOGGER_SYSLOG_MAP = {
       :unknown => :alert,
       :fatal   => :crit,
       :error   => :err,
@@ -23,27 +26,33 @@ module Termite
       :debug   => :debug,
     }
 
+    # Get Ruby Logger labels for Logger-compatible output
+    RUBY_LOGGER_SEV_LABELS = ::Logger::SEV_LABEL
+
     ##
-    # Maps Ruby Logger log levels to their values so we can silence.
+    # Maps Ruby Logger log levels to their numerical values.
 
     LOGGER_LEVEL_MAP = {}
 
-    LOGGER_MAP.each_key do |key|
+    LOGGER_SYSLOG_MAP.each_key do |key|
       LOGGER_LEVEL_MAP[key] = ::Logger.const_get key.to_s.upcase
     end
 
     ##
-    # Maps Logger numerical log level values to syslog log levels.
+    # Maps Logger numerical log level values to syslog level names.
 
-    LEVEL_LOGGER_MAP = {}
+    LEVEL_SYSLOG_MAP = {}
 
     LOGGER_LEVEL_MAP.invert.each do |level, severity|
-      LEVEL_LOGGER_MAP[level] = LOGGER_MAP[severity]
+      LEVEL_SYSLOG_MAP[level] = LOGGER_SYSLOG_MAP[severity]
     end
 
+    ##
+    # Maps Syslog level names to their numerical severity levels
+
     SYSLOG_SEVERITY_MAP = {}
 
-    LOGGER_MAP.values.each do |syslog_severity|
+    LOGGER_SYSLOG_MAP.values.each do |syslog_severity|
       SYSLOG_SEVERITY_MAP[syslog_severity] = ::Syslog.const_get("LOG_" + syslog_severity.to_s.upcase)
     end
 
@@ -63,7 +72,7 @@ module Termite
       EOM
     end
 
-    LOGGER_MAP.each_key do |level|
+    LOGGER_SYSLOG_MAP.each_key do |level|
       make_methods level
     end
 
@@ -73,32 +82,156 @@ module Termite
     attr_accessor :level
     attr_reader :stdout_level
     attr_reader :stderr_level
-    attr_reader :file_logger
 
-    def initialize(logdev = nil, shift_age = 0, shift_size = 1048576, options = {})
-      options = logdev if(logdev.is_a?(Hash))
+    def initialize(logdev = nil, shift_age = nil, shift_size = nil, options = {})
+      if logdev.is_a?(Hash)
+        options = logdev
+        logdev = nil
+      end
+
+      @loggers ||= []
+      @log_filename ||= logdev
+      @shift_age ||= shift_age
+      @shift_size ||= shift_size
 
       Ecology.read
+      read_ecology_data(options)
+    end
 
-      read_ecology_data
-      @log_filename ||= logdev
+    private
 
-      @file_logger = ::Logger.new(@log_filename, @shift_age || shift_age, @shift_size || shift_size) if @log_filename
-      @extra_loggers << @file_logger if @file_logger
+    def read_ecology_data(options = {})
+      @application = Ecology.application
+      @default_component = options[:component] || Ecology.property("logging::default_component") ||
+        options[:default_component]
+      # @console_print defaults to "yes", but can be nil if "no", "off" or "0" is specified
+      @console_print = Ecology.property("logging::console_print") || options[:console_print] || "yes"
+      @console_print = nil if ["no", "off", "0"].include?(@console_print)
+      @default_fields = Ecology.property("logging::extra_json_fields") || options[:extra_json_fields] || {}
+      @use_logger_prefix = Ecology.property("logging::use_logger_prefix") || options[:use_logger_prefix]
+      @level = Ecology.property("logging::level") || options[:level]
+      @level = string_to_severity(@level) if @level
+      @level = ::Logger::DEBUG if @level.nil? || ENV["TERMITE_DEBUG"]
 
-      # For UDP socket
-      @server_addr = options[:address] || "0.0.0.0"
-      @server_port = options[:port] ? options[:port].to_i : 514
+      sinks = Ecology.property("logging::sinks")
+      sinks ? instantiate_sinks(sinks, options) : read_unsinked_ecology(options)
+    end
 
-      @@sockets ||= {}
-      key = "#{@server_addr}:#{@server_port}"
-      @@sockets[key] ||= UDPSocket.new
-      @socket = @@sockets[key]
+    # This is the codepath for ecology files that do not have a 'sinks' section defined
+    def read_unsinked_ecology(options)
+      sinks = []
+      # File Sink
+      file_sink = {"type" => "file", "min_level" => @level}
+      file_sink["filename"] = Ecology.property("logging::filename", :as => :path) ||
+        options[:logging_filename]
+      file_sink["shift_age"] = Ecology.property("logging::shift_age") || options[:shift_age]
+      file_sink["shift_size"] = Ecology.property("logging::shift_size") || options[:shift_size]
+      file_sink["logger_prefix?"] = @use_logger_prefix if @use_logger_prefix
+      sinks << file_sink if file_sink["filename"]
+
+      # STDERR Sink
+      stderr_sink = {"type" => "file", "filename" => STDERR}
+      @stderr_level = Ecology.property("logging::stderr_level") || options[:stderr_level] || ::Logger::ERROR
+      stderr_sink["min_level"] = string_to_severity(@stderr_level) if @stderr_level
+      if @use_logger_prefix
+        stderr_sink["logger_prefix?"] = @use_logger_prefix
+      else
+        stderr_prefix = Ecology.property("logging::stderr_logger_prefix") || options[:stderr_logger_prefix]
+        stderr_sink["logger_prefix?"] = stderr_prefix if stderr_prefix
+      end
+      stderr_sink["color"] = Ecology.property("logging::stderr_color") || options[:stderr_color]
+      stderr_sink["color"] ||= "red" if STDERR.tty?
+      sinks << stderr_sink if @console_print
+
+      # STDOUT Sink
+      stdout_sink = {"type" => "file", "filename" => STDOUT}
+      @stdout_level = Ecology.property("logging::stdout_level") || options[:stdout_level] || ::Logger::INFO
+      stdout_sink["min_level"] = string_to_severity(@stdout_level) if @stdout_level
+      stdout_sink["max_level"] = string_to_severity(@stderr_level) - 1
+      if @use_logger_prefix
+        stdout_sink["logger_prefix?"] = @use_logger_prefix
+      else
+        stdout_prefix = Ecology.property("logging::stdout_logger_prefix") || options[:stdout_logger_prefix]
+        stdout_sink["logger_prefix?"] = stdout_prefix if stdout_prefix
+      end
+      stdout_sink["color"] = Ecology.property("logging:stdout_color") || options[:stdout_color]
+      stdout_sink["color"] ||= "blue" if STDOUT.tty?
+      sinks << stdout_sink if @console_print
+
+      # Syslog Sink
+      syslog_sink = {"type" => "syslog"}
+      syslog_sink["transport"] = Ecology.property("logging::transport") || options[:transport]
+      sinks << syslog_sink
+
+
+      # Set up sinks
+      instantiate_sinks(sinks, options)
     end
 
-    private
+    # For each sink, and to loggers
+    def instantiate_sinks(sinks, options)
+      sinks = sinks.dup
+      syslog = false
+      # Maximum Level
+      min_level = 5
+      sinks.each do |sink|
+        # Set min level if lower than current (@level if not defined)
+        if sink["min_level"]
+          sink_level = string_to_severity(sink["min_level"])
+          min_level = sink_level if sink_level < min_level
+        else min_level = @level
+        end
+        cur_logger = case sink["type"]
+          when "file"
+            sink["newline?"] = true unless sink.has_key? "newline?"
+            case sink["filename"]
+              when STDOUT
+                ::Logger.new(STDOUT) if @console_print
+              when STDERR
+                ::Logger.new(STDERR) if @console_print
+              else
+                ::Logger.new(sink["filename"], sink["shift_age"] || 0, sink["shift_size"] || 1048576)
+            end
+          when "stdout"
+            sink["newline?"] = true unless sink.has_key? "newline?"
+            ::Logger.new(STDOUT) if @console_print
+          when "stderr"
+            sink["newline?"] = true unless sink.has_key? "newline?"
+            ::Logger.new(STDERR) if @console_print
+          when "syslog"
+            syslog = true
+            setup_syslog_logger(sink, options)
+          when "hastur"
+            setup_hastur_logger(sink, options)
+        end
+        sink["logger"] = cur_logger
+      end
+      @loggers = sinks
+
+      # Create syslog logger if not defined in sinks
+      unless syslog
+        min_level = @level
+        add_logger(setup_syslog_logger(options), "type" => "syslog", "min_level" => min_level)
+      end
+
+      # Constructor params logger
+      if @log_filename
+        min_level = @level
+        add_logger(::Logger.new(@log_filename, @shift_age || 0, @shift_size || 1048576), "type" => "file",
+          "filename" => @log_filename,
+          "shift_age" => @shift_age || 0,
+          "shift_size" => @shift_size || 1048576,
+          "logger_prefix?" => @use_logger_prefix,
+          "min_level" => @level,
+          "newline?" => true
+        )
+      end
+      # If the min level of all loggers is greater than @level, use that
+      @level = [@level, min_level].max
+    end
 
     def string_to_severity(str)
+      return str if str.is_a? Numeric
       orig_string = str
       str = str.strip.downcase
       return str.to_i if str =~ /\d+/
@@ -107,61 +240,60 @@ module Termite
       ret
     end
 
-    def read_ecology_data
-      @application = Ecology.application
-
-      @extra_loggers = []
-
-      # @console_print defaults to "yes", but can be nil if "no", "off" or "0" is specified
-      @console_print = Ecology.property("logging::console_print") || "yes"
-      @console_print = nil if ["no", "off", "0"].include?(@console_print)
-
-      @log_filename = Ecology.property("logging::filename", :as => :path)
-      @shift_age = Ecology.property("logging::shift_age")
-      @shift_size = Ecology.property("logging::shift_size")
-      @default_component = Ecology.property("logging::default_component")
-      @level = Ecology.property("logging::level")
-      @level = string_to_severity(@level) if @level
-      @stderr_level = Ecology.property("logging::stderr_level")
-      @stderr_level = string_to_severity(@stderr_level) if @stderr_level
-      @stdout_level = Ecology.property("logging::stdout_level")
-      @stdout_level = string_to_severity(@stdout_level) if @stdout_level
+    def setup_syslog_logger(sink, options)
+      # For UDP socket
+      @server_addr = options[:address] || "0.0.0.0"
+      @server_port = options[:port] ? options[:port].to_i : 514
+      @socket = find_or_create_socket(@server_addr, @server_port)
+      SyslogLogger.new(@socket, @server_addr, @server_port, sink["transport"])
+    end
 
-      @default_fields = Ecology.property("logging::extra_json_fields") || {}
+    def setup_hastur_logger(sink, options)
+      @hastur_addr = options[:hastur_address] || "127.0.0.1"
+      @hastur_port = sink["udp_port"] || options[:hastur_port] || 8125
+      @hastur_socket = find_or_create_socket(@hastur_addr, @hastur_port)
+      HasturLogger.new(@hastur_socket, @hastur_addr, @hastur_port, sink["labels"])
+    end
 
-      @level ||= ::Logger::DEBUG
-      @stdout_level ||= ::Logger::INFO
-      @stderr_level ||= ::Logger::ERROR
+    def find_or_create_socket(addr, port)
+      @@sockets ||= {}
+      key = "#{addr}:#{port}"
+      @@sockets[key] ||= UDPSocket.new
     end
 
     public
 
-    def add_logger(logger)
-      @extra_loggers << logger
+    def add_logger(logger, options={})
+      if logger.is_a? Hash
+        @loggers << logger
+      else
+        options["logger"] = logger
+        @loggers << options
+      end
     end
 
     def socket
       @socket
     end
 
-    def raw_add(severity, message = nil, data = nil, options = {}, &block)
-      raw_message = message
+    COLORS = [:black, :red, :green, :yellow, :blue, :magenta, :cyan, :white, :default]
+
+    def raw_add(severity, raw_message = nil, data = nil, options = {}, &block)
       # Severity is a numerical severity using Ruby Logger's scale
       severity ||= ::Logger::UNKNOWN
       return true if severity < @level
 
-      application = options[:application] || @application
-
-      component ||= @default_component
+      application =  options[:application] || @application
+      component = @default_component
       component = options[:component] if options.has_key?(:component)
-      if component
-        application += ":" + component
-      end
+
+      combined_app = application + ":" + component if component
+      app_data = {:combined => combined_app, :app => application, :component => component}
 
       data ||= {}
       if data.is_a?(Hash)
         data = @default_fields.merge(data)
-        data = MultiJson.encode(data)
+        data = MultiJson.dump(data)
       elsif data.is_a?(String)
         # Can't merge a JSON string with default data
         raise "Can't merge a JSON string with extra fields!" unless @default_fields.empty?
@@ -169,42 +301,37 @@ module Termite
         raise "Unknown data object passed as JSON!"
       end
 
-      tid = Ecology.thread_id(Thread.current)
       time = Time.now
-      day = time.strftime("%b %d").sub(/0(\d)/, ' \\1')
-      time_of_day = time.strftime("%T")
-      hostname = Socket.gethostname
-      tag = Syslog::LOG_LOCAL6 + SYSLOG_SEVERITY_MAP[LEVEL_LOGGER_MAP[severity]]
-
-      syslog_string = "<#{tag}>#{day} #{time_of_day} #{hostname} #{application} [#{Process.pid}]: [#{tid}] "
-      full_message = clean(message || block.call)
-
-      # ruby_severity is the Ruby Logger severity as a symbol
-      ruby_severity = LOGGER_LEVEL_MAP.invert[severity]
-
-      full_message.split("\n").each do |line|
-        message = syslog_string + "#{line} #{data}"
-
-        begin
-          @socket.send(message, 0, @server_addr, @server_port)
-        rescue Exception
-          # Didn't work.  Try built-in Ruby syslog
-          require "syslog"
-          Syslog.open(application, Syslog::LOG_PID | Syslog::LOG_CONS) do |s|
-            s.error("UDP syslog failed!  Falling back to libc syslog!") rescue nil
-            s.send(LEVEL_LOGGER_MAP[severity], "#{line} #{data}") rescue nil
-          end
-        end
-      end
-
-      if @console_print && severity >= @stderr_level
-        STDERR.puts raw_message
-      elsif @console_print && severity >= @stdout_level
-        STDOUT.puts raw_message
+      full_message = if raw_message
+        raw_message
+      elsif block
+        block.call
+      else
+        "Error! Logger called with no message or block to execute! Trace: #{caller.join(", ")}"
       end
-
-      @extra_loggers.each do |logger|
-        logger.send(ruby_severity, raw_message) rescue nil
+      full_message = clean(full_message)
+      # Lifted from Logger::Formatter
+      ruby_logger_severity = RUBY_LOGGER_SEV_LABELS[severity]
+      ruby_logger_message = "%s, [%s#%d] %5s -- %s: %s" % [ruby_logger_severity[0..0],
+          (time.strftime("%Y-%m-%dT%H:%M:%S.") << "%06d " % time.usec),
+          $$, ruby_logger_severity, "", full_message]
+
+      @loggers.each do |sink|
+        next if (sink["min_level"] && severity < string_to_severity(sink["min_level"])) ||
+          (sink["max_level"] && severity > string_to_severity(sink["max_level"])) ||
+          sink["logger"].nil?
+        message = sink["logger_prefix?"] ? ruby_logger_message : full_message
+        message += " #{data}" if sink["logger_data?"]
+        if sink["logger"].respond_to?(:send_message)
+          sink["logger"].send_message(severity, message, app_data, time, data)
+        else
+          message += "\n" if sink["newline?"] && sink["newline?"] != "false"
+          if sink["color"]
+            color = (COLORS.include? sink["color"].to_sym) ? sink["color"].to_sym : sink["color"]
+            message = message.color(color)
+          end
+          sink["logger"] << message
+        end rescue nil
       end
 
       true
@@ -256,4 +383,52 @@ module Termite
       return message
     end
   end
+
+  def FakeLogger
+    Termite::LOGGER_SYSLOG_MAP.each_key do |key|
+      define_method(LOGGER_LEVEL_MAP[key]) do
+        # Do nothing
+      end
+    end
+
+    # Alias the other methods to a do-nothing method
+    alias :add :error
+    alias :<< :error
+    alias :log :error
+    alias :silence :error
+    alias :add_logger :error
+
+    # Read and write level
+    attr_accessor :level
+
+    # For now, don't read an Ecology, just mock out these accessors.
+
+    def stdout_level
+      4
+    end
+
+    def stderr_level
+      2
+    end
+  end
+end
+
+module Termite
+  module Thread
+    def self.new(*args, &block)
+      ::Thread.new do
+        begin
+          block.call
+        rescue ::Exception
+          if args[0].respond_to?(:warn)
+            logger = args[0]
+          else
+            logger = ::Termite::Logger.new(*args)
+          end
+          logger.warn "Exception in thread: #{$!.message}"
+          logger.warn "  Backtrace:\n#{$!.backtrace.join("\n")}"
+        end
+      end
+    end
+  end
 end