recmon 1.0.0

data/.yardopts

@@ -0,0 +1,2 @@
+--title "REC Monitor" -m rdoc --main README lib/**/*.rb - README CHANGELOG
+

data/CHANGELOG

@@ -0,0 +1,2 @@
+== Version 1.0.0
+- Initial publication

data/README

@@ -0,0 +1,143 @@
+= Recmon
+Recmon is a host-based system monitor. It complements REC, the Ruby Event Correlator.
+
+== Installation
+
+  $ sudo gem install recmon
+
+== Usage
+Require the recmon gem:
+
+  require 'rubygems'
+  require 'recmon'
+
+Create a monitor to obtain readings from the sensors at the right time:
+
+  s = Recmon::Monitor.new()
+
+Now add sensors for each aspect you want to monitor:
+
+  s.web("Website", "http://www.example.com/index.html")
+  s.diskspace("Database", "/var/postgres/data/")
+  s.filesize("Messages", "/var/log/messages")
+  s.ping("earth", "206.125.172.58")
+
+Then start the monitor running:
+
+  s.start()
+
+and it will periodically write entries to the log file (default = +/var/log/recmon.log+):
+
+  2012-09-13T16:08:59+10:00 Recmon is monitoring.
+  ...
+  2012-09-13T16:08:59+10:00 site=Website status=down
+  2012-09-13T16:08:59+10:00 Database usage=247 MB
+  2012-09-13T16:08:59+10:00 Messages filesize=83 KB
+  2012-09-13T16:08:59+10:00 ping host=earth status=up
+  ...
+  2012-09-13T16:09:16+10:00 Recmon is exiting.
+
+== Sensors
+There are several sensors:
+
+- *web*: ensure a website is responding
+- *diskspace*: track the diskspace used by a folder
+- *filesize*: monitor the size of a file
+- *ping*: check if a server is alive
+- *proc*: look for a named process
+- *ssh*: ensure the SSH service is running
+- *command*: run an arbitrary command and report success
+
+=== Common characteristics
+All sensors have a name which is used to distinguish it from others of the same type.
+For example, you can monitor the +earth+ server and the +terra+ server.
+The name is used in composing the log entry.
+
+All sensors have a sane default frequency, so it is not necessary to
+specify a frequency unless you want to override the default.
+
+Sensors compose log messages that is designed for further processing. Although they are
+readable, it is more important that they are parsable, so they always start with an ISO8601 date time
+and then several +name+=+value+ pairs.
+
+=== web: Ensure a website is responding
+To periodically check if a website is reachable, add a WebSensor, specifying the title, the URL, and optionally a frequency (default = 60 seconds).
+
+  s.web(name, url, freq=60)
+  s.web("Main", "http://www.finalstep.com.au/heartbeat.png")
+    # log entry => "2012-09-13T16:08:59+10:00 site=Main status=down"
+
+  s.web("Google", "http://www.google.com/jsapi", 120)
+    # log entry => "2012-09-13T16:09:02+10:00 site=Google status=up"
+
+=== diskspace: track the diskspace used by a folder
+A DiskspaceSensor tracks how much disk space is being consumed by a set of files (log files or database files) in a folder:
+
+  s.diskspace(name, folderPath, freq=1200)
+  s.diskspace("Database", "/var/postgres/data/")
+    # log entry => "2012-09-13T16:09:02+10:00 Database usage=45 MB"
+  s.diskspace("Logs", "/var/log/", 86400)     # daily frequency
+    # log entry => "2012-09-13T20:00:00+10:00 Logs usage=16 MB"
+
+=== filesize: monitor the size of a file
+A FilesizeSensor tracks the size of a given file every 10 minutes:
+
+  s.filesize(name, path, freq=1200)
+  s.filesize("Messages", "/var/log/messages")
+    # log entry => "2012-09-13T16:09:02+10:00 Messages filesize=78 KB"
+
+=== ping: check if a server is alive
+The PingSensor pings a server to ensure it is accessible through the network
+
+  s.ping(name, ipaddr, freq=300)
+  s.ping("earth", "206.125.172.58")
+    # log entry => "2012-09-13T16:09:02+10:00 ping host=earth status=up"
+
+=== proc: look for a named process
+It is often useful to directly check that a process is still running.
+The ProcSensor finds processes that match a pattern.
+
+  s.proc(name, pattern, freq=60)
+  s.proc("Webserver", "httpd")
+    # log entry => "2012-09-13T16:09:02+10:00 proc=Webserver status=running"
+    # log entry => "2012-09-13T16:10:30+10:00 proc=Webserver status=stopped"
+  s.proc("Postgres server", "postgres: writer process")
+    # log entry => "2012-09-13T16:10:30+10:00 proc=Postgres server status=running"
+
+=== ssh: ensure the SSH service is running
+Check that a server is accepting SSH connections. The SSHSensor actually runs a harmless
+command (+pwd+) on the target server to confirm SSH is working. Obviously to make this
+work in batch mode, the recmon user needs to have a private key in <code>~/.ssh/</code> and add it
+to <code>~/.ssh/authorized_keys</code> on the target server.
+
+  s.ssh(hostname, user="_recmon", port=22, freq=300)
+  s.ssh("earth")
+    # log entry => "2012-09-13T16:09:02+10:00 SSH at host=earth for user=_recmon status=up"
+  s.ssh("moon", "richard", 922, 600)
+    # log entry => "2012-09-13T16:09:02+10:00 SSH at host=moon for user=richard status=up"
+
+=== command: run an arbitrary command and report success
+The *command* sensor simply executes the command and reports success (exit code = 0)
+or failure.
+
+  s.command("MySQL server", "/usr/local/bin/mysqladmin ping")
+    # log entry => "2012-09-13T16:09:02+10:00 MySQL server status=up"
+
+
+== Why Recmon?
+=== 1. Recmon is lightweight
+There are several wonderful system monitoring tools (nagios, splunk)
+but they require a considerable investment in configuration, not to mention
+dependencies that may be undesired.
+If you just want to keep an eye on a few key metrics then Recmon is much
+faster and easier to set up.
+
+=== 2. Recmon complements REC
+Recmon generates log entries for a range of events that are not typically logged.
+Once they are logged, they can be analysed to generate alerts.
+
+REC (Ruby Event Correlation) is the tool that correlates events across time to determine
+if a situation is abnormal, and so generates fewer, more meaningful alerts
+by email or instant message.
+
+So Recmon + REC = lightweight Nagios

data/lib/recmon.rb

@@ -0,0 +1,22 @@
+require 'recmon/monitor'
+require 'recmon/sensor'
+require 'recmon/command-sensor'
+require 'recmon/diskspace-sensor'
+require 'recmon/filesize-sensor'
+require 'recmon/ping-sensor'
+require 'recmon/proc-sensor'
+require 'recmon/ssh-sensor'
+require 'recmon/web-sensor'
+
+# The Recmon Module includes:
+# - Recmon::Monitor
+# - Recmon::Sensor
+# - Recmon::CommandSensor
+# - Recmon::DiskspaceSensor
+# - Recmon::FilesizeSensor
+# - Recmon::PingSensor
+# - Recmon::ProcSensor
+# - Recmon::SSHSensor
+# - Recmon::WebSensor
+module Recmon
+end

data/lib/recmon/command-sensor.rb

@@ -0,0 +1,26 @@
+require 'recmon/sensor'
+require 'ipaddr'
+
+module Recmon
+
+# Periodically execute a command and report success
+class CommandSensor < Sensor
+
+  # Create a new CommandSensor. The name is descriptive and the command
+  # should be runnable by the +_recmon+ user
+  def initialize(name, command, freq=600)
+    super(name, freq)
+    @command = command
+  end
+
+  # Ensure user _recmon has permission to run the command.
+  # An exit status of zero is required for success
+  def sense()
+    `#{@command}`
+    up = $?.exitstatus == 0
+    status = up ? "up" : "down"
+    return("#{@name} status=#{status}")
+  end
+
+end
+end

data/lib/recmon/diskspace-sensor.rb

@@ -0,0 +1,28 @@
+require 'recmon/sensor'
+
+module Recmon
+
+# Periodically check the diskspace usage for a folder
+class DiskspaceSensor < Sensor
+
+  # create a DiskspaceSensor to check the diskspace usage of a folder.
+  # * +name+ is a descriptive name for the folder
+  # * +path+ should be absolute
+  def initialize(name, path, freq=1200)
+    super(name, freq)
+    @path = path.strip()
+    raise "Path not found: #{path}" unless File.directory?(@path)
+  end
+
+  # Called by Monitor. Executes du(1) to determine the disk usage in Megabytes.
+  def sense()
+    begin
+      size = `sudo /usr/bin/du -sm #{@path}`.to_i().to_s()
+    rescue
+      size = "unknown"
+    end
+    return("#{@name} usage=#{size} MB")
+  end
+
+end
+end

data/lib/recmon/monitor.rb

@@ -0,0 +1,105 @@
+require 'time'
+
+module Recmon
+
+# The Monitor manages the list of sensors, and calls on them periodically
+# to report their results.
+class Monitor
+
+  # Returns an array of sensors
+  attr_reader :sensors
+
+  # Creates a new monitor, writing to the given log file.
+  # The default frequency of 10 seconds is frequent enough to be near-real-time
+  # without any burden on the processor. This is the frequency on which the
+  # monitor checks if any sensors are due - not the frequency for each sensor.
+  def initialize(log="/var/log/recmon.log", freq=10)
+    @logfile = log
+    @freq = freq  # seconds
+    @log = ""
+    @sensors = []
+  end
+
+  # Starts the monitor running until an INT signal is received.
+  def start()
+    trap("HUP") {  # defer monitoring for a minute and reopen log files
+      @log.close() unless @log.nil? or @log.closed?
+      sleep(60)
+      open_log()
+    }
+    trap("INT") {
+      stop()
+    }
+    open_log()
+    log("Recmon is monitoring.")
+    monitor()
+    while sleep(@freq)
+      monitor()
+    end
+  end
+
+  # opens the log file - occurs on starting, and if HUP signal is received
+  def open_log()
+    begin
+      @log = File.new(@logfile, 'a')
+    rescue
+      $stderr.puts("Unable to open log file #{@logfile}.}")
+      exit 1
+    end
+  end
+
+  # Stops the monitor
+  def stop()
+    log("Recmon is exiting.")
+    @log.close() unless @log.nil? or @log.closed?
+    exit 0
+  end
+
+  # Checks the list of sensors to see if any are due to report
+  def monitor()
+    @sensors.each { |s| log(s.sense()) if s.check()  }
+    @log.flush()
+  end
+
+  # Logs a message to the log file
+  def log(message)
+    @log.puts("#{Time.now.iso8601} #{message}")
+  end
+
+  # Adds a CommandSensor
+  def command(name, command, freq=120)
+    @sensors << Recmon::CommandSensor.new(name, command, freq)
+  end
+
+  # Adds a DiskspaceSensor
+  def diskspace(name, path, freq=1200)
+    @sensors << Recmon::DiskspaceSensor.new(name, path, freq)
+  end
+
+  # Adds a FilesizeSensor
+  def filesize(name, path, freq=1200)
+    @sensors << Recmon::FilesizeSensor.new(name, path, freq)
+  end
+
+  # Adds a PingSensor
+  def ping(name, ip, freq=120)
+    @sensors << Recmon::PingSensor.new(name, ip, freq)
+  end
+
+  # Adds a ProcSensor
+  def proc(name, pattern, freq=60)
+    @sensors << Recmon::ProcSensor.new(name, pattern, freq)
+  end
+
+  # Add an SSHSensor
+  def ssh(name, user, port=22, freq=120)
+    @sensors << Recmon::SSHSensor.new(name, user, port, freq)
+  end
+
+  # Adds a WebSensor
+  def web(name, url, freq=120)
+    @sensors << Recmon::WebSensor.new(name, url, freq)
+  end
+
+end
+end

data/lib/recmon/ping-sensor.rb

@@ -0,0 +1,25 @@
+require 'recmon/sensor'
+require 'ipaddr'
+
+module Recmon
+
+# Periodically pings a host to check it is alive
+class PingSensor < Sensor
+
+  # Create a new PingSensor to ping the given IP every 2 minutes
+  def initialize(name, ip, freq=120)
+    super(name, freq)
+    @ip = IPAddr.new(ip)
+  end
+
+  # Called by Monitor. Sends a single ICMP ping to the IP.
+  # Of course, there is no point doing this if any firewall in between drops ICMP pings.
+  def sense()
+    `/sbin/ping -c1 #{@ip}`
+    up = $?.exitstatus == 0
+    status = up ? "up" : "down"
+    return("ping host=#{@name} status=#{status}")
+  end
+
+end
+end

data/lib/recmon/proc-sensor.rb

@@ -0,0 +1,23 @@
+require 'recmon/sensor'
+
+module Recmon
+
+# ProcSensor checks periodically to ensure a certain process is running
+class ProcSensor < Sensor
+
+  # Create a new ProcSensor. The pattern is a string as specified in pgrep(1).
+  def initialize(name, pattern, freq=60)
+    super(name, freq)
+    @pattern = pattern
+  end
+
+  # Called by Monitor. This method executes pgrep(1) to check if one or more
+  # processes match
+  def sense()
+    down = `sudo /usr/bin/pgrep -f "#{@pattern}"` == ""
+    status = down ? "stopped" : "running"
+    return("proc=#{@name} status=#{status}")
+  end
+
+end
+end

data/lib/recmon/sensor.rb

@@ -0,0 +1,34 @@
+
+module Recmon
+
+# Sensor is the abstract parent class for all sensors.
+class Sensor
+
+  # should only be called by the Monitor
+  def initialize(name, freq)
+    @name = name
+    @freq = freq.to_i
+    @due = Time.now
+    raise "A name must be specified for the sensor" if name.nil? or name.length == 0
+    raise "Frequency must be positive seconds, #{@freq} is not valid" if freq < 1
+  end
+
+  # check if the sensor is due to take another reading
+  def check()
+    now = Time.now
+    if @due < now
+      @due = now + @freq
+      return(true)
+    end
+    return(false)
+  end
+
+  # *Must* be implemented by the subclass.
+  #
+  # Called by the Monitor if #check returns true
+  def sense()
+    raise("#sense method should be overridden by the sub-class")
+  end
+
+end
+end

data/lib/recmon/ssh-sensor.rb

@@ -0,0 +1,26 @@
+require 'recmon/sensor'
+
+module Recmon
+
+# SSHSensor periodically checks that an SSH service is available
+class SSHSensor < Sensor
+
+  # Create a new SSHSensor to periodically test SSH at the named host
+  def initialize(name, user="_recmon", port=22, freq=300)
+    super(name, freq)
+    @user = user
+    @port = port
+  end
+
+  # Called by Monitor. sense() attempts to log in to the host in batch mode
+  # so you *MUST* set up a private key authentication for this user or
+  # this sensor will generate false negatives.
+  def sense()
+    `/usr/bin/ssh -p#{@port} -o "BatchMode yes" #{@user}@#{@name} pwd`
+    up = $?.exitstatus == 0
+    status = up ? "up" : "down"
+    return("SSH at host=#{@name} for user=#{@user} status=#{status}")
+  end
+
+end
+end

data/lib/recmon/web-sensor.rb

@@ -0,0 +1,36 @@
+require 'recmon/sensor'
+require 'uri'
+require 'net/http'
+
+module Recmon
+
+# WebSensor periodically checks that a website is accessible
+class WebSensor < Sensor
+
+  # Create a new WebSensor with the given name, to check the url periodically
+  # to check if the website is available
+  def initialize(name, url, freq=60)
+    super(name, freq)
+    @url = URI.parse(url)
+  end
+
+  # Called by the Monitor, #sense checks for an <code>HTTP 200 OK</code> response from the URL.
+  def sense()
+    begin
+      q = @url.query || ""
+      q += (q.length > 0 ? "&" : "?")
+      q += "ts=" + Time.now().to_i().to_s()    # make request unique to prevent caching
+        req = Net::HTTP::Get.new(@url.path + q)
+        res = Net::HTTP.start(@url.host, @url.port) {|http|
+          http.request(req)
+        }
+      up = (res.code == '200')
+    rescue
+      up = false
+    end
+    status = up ? "up" : "down"
+    return("site=#{@name} status=#{status}")
+  end
+
+end
+end

metadata

@@ -0,0 +1,77 @@
+--- !ruby/object:Gem::Specification 
+name: recmon
+version: !ruby/object:Gem::Version 
+  hash: 23
+  prerelease: 
+  segments: 
+  - 1
+  - 0
+  - 0
+  version: 1.0.0
+platform: ruby
+authors: 
+- Richard Kernahan
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2012-09-28 00:00:00 Z
+dependencies: []
+
+description: "\tRecmon generates log entries for a range of events that are not typically logged.\n    Once they are logged, they can be analysed to generate alerts.\n\n    REC (Ruby Event Correlation) is the tool that correlates events across time to determine\n    if a situation is abnormal, and so generates fewer, more meaningful alerts\n    by email or instant message.\n\n    So Recmon + REC = lightweight Nagios\n"
+email: dev.recmon@finalstep.com.au
+executables: []
+
+extensions: []
+
+extra_rdoc_files: []
+
+files: 
+- lib/recmon.rb
+- lib/recmon/monitor.rb
+- lib/recmon/sensor.rb
+- lib/recmon/command-sensor.rb
+- lib/recmon/diskspace-sensor.rb
+- lib/recmon/ping-sensor.rb
+- lib/recmon/proc-sensor.rb
+- lib/recmon/ssh-sensor.rb
+- lib/recmon/web-sensor.rb
+- .yardopts
+- CHANGELOG
+- README
+homepage: http://rubygems.org/gems/recmon
+licenses: []
+
+post_install_message: 
+rdoc_options: []
+
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 3
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 3
+      segments: 
+      - 0
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.8.24
+signing_key: 
+specification_version: 3
+summary: Recmon is a host-based system monitor. It complements REC, the Ruby Event Correlator.
+test_files: []
+
+has_rdoc: