cognizant 0.0.1 → 0.0.2

data/.gitignore

@@ -15,4 +15,5 @@ spec/reports
 test/tmp
 test/version_tmp
 tmp
-.DS_Store
+.DS_Store
+.rbenv*

data/README.md

@@ -1,31 +1,32 @@
-# Cognizant
+# Cognizant: system utility to supervise your processes
 
-Cognizant is a system utility that supervises your processes, ensuring their
-state based on a flexible criteria.
+Let us say that you have a program that is critical for your application. Any
+downtime for the processes of this program would hurt your business. You want
+to make sure these processes are always up and working. If anything unexpected
+happens to their state, you want to be notified about it.
 
-In simpler terms, cognizant keeps your processes up and running. It ensures
+Enter Cognizant. Cognizant is a system utility that supervises your processes,
+monitoring and ensuring their state based on a flexible criteria.
+
+In simpler terms, it keeps your processes up and running. It ensures
 that something productive (like restart) is done when the process being
 monitored matches a certain condition (like RAM, CPU usage or a custom
 condition).
 
-PS: Although the core works efficiently, yet the command interface to the
-utility, documentation and some other features need a lot of work.
-Contributions will be overwhelmingly welcomed!
-
-PS2: This README is written as a roadmap for the features cognizant would
-ideally provide. Some of them might not yet be implemented (e.g. conditions).
+And if it matters, cognizant means "having knowledge or being aware of",
+according to Apple's Dictionary.
 
 Cognizant can be used to monitor any long running process, including the
-following examples:
+following:
 
 - Web servers (Nginx, Apache httpd, WebSocket, etc.)
 - Databases (Redis, MongoDB, MySQL, PostgreSQL, etc.)
-- Job workers (Resque, Sidekiq, etc.)
+- Job workers (Resque, Sidekiq, Qless, etc.)
 - Message queue applications (RabbitMQ, Beanstalkd, etc.)
-- Logs collection daemon
+- Logs collection daemons
 - Or any other program that needs to keep running
 
-## Monitoring
+### Monitoring
 
 Cognizant can be used to monitor an application process' state so that it
 is running at all times. When cognizant is instructed to monitor a process,
@@ -33,13 +34,23 @@ by default, the process will automatically be started. These processes are
 automatically monitored for their state. i.e. a process is automatically
 started again if it stops unexpectedly.
 
-## Conditions
+### Conditions
 
 Conditions provide a way to monitor and act on more than just the state of a
 process. For example, conditions can monitor the resource utilization (RAM,
 CPU, etc.) of the application process and restart it if it matches a
 condition.
 
+See examples for conditions usage.
+
+### Notifications
+
+PS: Notifications currently need work.
+
+Notifications provide a way to alert system administrator of unexpected events
+happening with the process. Notifications can use multiple gateways, including
+email and twitter [direct message].
+
 ## Getting started
 
 Cognizant is written in the Ruby programming language, and hence depends on
@@ -73,17 +84,44 @@ The daemon, with it's default options, requires superuser access to certain syst
 
 To start without superuser access, specify these file and directory config variables to where the user starting it has write access:
 
-    $ cognizantd ~/.cognizant/cognizantd.yml
+    $ cognizantd ./examples/cognizantd.yml
     
     # assuming that:
     
-    $ cat ~/.cognizant/cognizantd.yml
+    $ cat ~/.examples/cognizantd.yml # find this file in source code for latest version
     ---
     socket:   ~/.cognizant/cognizantd.sock
     pidfile:  ~/.cognizant/cognizantd.pid
     logfile:  ~/.cognizant/cognizantd.log
     pids_dir: ~/.cognizant/pids/
     logs_dir: ~/.cognizant/logs/
+    
+    monitor: {
+      redis-server-1: {
+        group: redis,
+        start_command: /usr/local/bin/redis-server -,
+        start_with_input: "daemonize no\nport 6666",
+        ping_command: redis-cli -p 6666 PING,
+        stop_signals: [TERM, INT]
+      },
+      redis-server-2: {
+        group: redis,
+        start_command: /usr/local/bin/redis-server -,
+        start_with_input: "daemonize no\nport 7777",
+        ping_command: redis-cli -p 7777 PING,
+        stop_command: redis-cli -p 7777 SHUTDOWN
+      },
+      sleep: {
+        start_command: sleep 3,
+        checks: {
+          flapping: {
+            times: 4,
+            within: 15, # Seconds.
+            retry_after: 30 # Seconds.
+          }
+        }
+      }
+    }
 
 or
 
@@ -91,56 +129,76 @@ or
     $ echo <<EOF | cognizantd -
     ---
     socket:   ~/.cognizant/cognizantd.sock
-    pidfile:  ~/.cognizant/cognizantd.pid
-    logfile:  ~/.cognizant/cognizantd.log
-    pids_dir: ~/.cognizant/pids/
-    logs_dir: ~/.cognizant/logs/
+    ...
+    monitor: {
+      ...
+      ...
+    }
     EOF
 
+Process information can also be provided via Ruby code. See `load` command below.
+
 ## Using the administration utility
 
 Cognizant can be administered using the `cognizant` command line utility. This is an application for performing administration tasks like monitoring, starting, stopping processes or loading configuration and processes' information.
 
-Here's how you tell cognizant to start monitoring a new process:
+Here are some basic operational commands:
 
-    $ cognizant monitor --name          resque-worker-1                \
-                        --group         resque                         \
-                        --chdir         /apps/example/current/         \
-                        --start_command "bundle exec rake resque:work"
+    $ cognizant monitor redis # by group name
+    $ cognizant restart redis-server-1 # by process name
 
-Now check it's status:
+To get cognizant to ignore a particular process' state:
 
-    $ cognizant status resque-worker-1
-    ---
-    resque-worker-1: {
-      state: running,
-      uptime: 3600 # 1 hour
-    }
+    $ cognizant unmonitor sleep-10
 
-Or check status of all processes:
+Now check status of all managed processes:
 
-    $ cognizant status resque-worker-1
-    ---
-    redis-server: {
-      state: running,
-      uptime: 5400 # 1 hour 30 minutes
-    },
-    resque-worker-1: {
-      group: resque,
-      state: running,
-      uptime: 3600 # 1 hour
-    },
-    resque-worker-2: {
-      group: resque,
-      state: stoppped,
-      uptime: 0
-    }
+    $ cognizant status
+    +----------------+-------+--------------------------------------+
+    | Process        | Group | State                                |
+    +----------------+-------+--------------------------------------+
+    | redis-server-1 | redis | running since 1 minute               |
+    +----------------+-------+--------------------------------------+
+    | redis-server-2 | redis | running since 2 minutes              |
+    +----------------+-------+--------------------------------------+
+    | sleep-10       |       | unmonitored since less than a minute |
+    +----------------+-------+--------------------------------------+
+    2012-11-23 01:16:18 +0530
+
+Here's how you tell cognizant to start monitoring new processes:
+
+    $ cognizant load ./examples/redis-server.rb # find this file in source code
+
+All of the available commands are:
+
+- `help`               - Shows a list of commands or help for one command
+- `status    [name]`   - Display status of managed process(es) or group(s)
+- `load      ./cog.rb` - Loads the process information from specified Ruby file
+- `monitor   name`     - Monitor the specified process or group
+- `unmonitor name`     - Unmonitor the specified process or group
+- `start     name`     - Start the specified process or group
+- `stop      name`     - Stop the specified process or group
+- `restart   name`     - Restart the specified process or group
+- `shutdown`           - Stop the monitoring daemon without affecting processes
+
+See `cognizant help` for options.
+
+## Contributing
+
+Contributions are definitely welcome. To contribute, just follow the usual
+workflow:
+
+1. Fork Cognizant
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Added some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Github pull request
 
-### Works anywhere
+## Compatibility
 
-Cognizant can be used on any operating system where Ruby 1.9+ works.
+Cognizant was developed and tested under Ruby 1.9.2-p290.
 
-### What are the other programs similar to cognizant?
+## Similar programs
 
 - Monit
 - God (Ruby)
@@ -151,10 +209,13 @@ Cognizant can be used on any operating system where Ruby 1.9+ works.
 - Systemd
 - Launchd
 
-### Which one of these should I be using?
+### Links
 
-The one that gets your job done efficiently.
+- Documentation: http://rubydoc.info/github/Gurpartap/cognizant/frames
+- Source: https://github.com/Gurpartap/cognizant
+- Rubygems: https://rubygems.org/gems/cognizant
 
-### What does the term "cognizant" mean?
+## About
 
-If it matters, cognizant means "having knowledge or being aware of", according to Apple's Dictionary.
+Cognizant is a project of [Gurpartap Singh](http://gurpartap.com/). Feel free
+to get in touch.

data/bin/cognizant

@@ -1,58 +1,178 @@
 #!/usr/bin/env ruby
 
+require "cognizant/version"
+require "gli"
+require "formatador"
+require "socket"
+require "json"
+
+include GLI::App
+
 # Set the process name.
 $0 = File.basename(__FILE__)
 
-# Flush standard output/error immediately.
+# Set to flush standard output/error immediately.
 $stdout.sync = true
 $stderr.sync = true
 
-begin
-  require 'optparse'
+program_desc "administration utility for cognizantd"
+@version = Cognizant::VERSION
 
-  options = {
-    socket: "/var/run/cognizant/cognizantd.sock"
-  }
+sort_help :manually
 
-  OptionParser.new do |opts|
-    opts.banner = <<-EOF
-  Usage:
-    cognizant [OPTIONS] command
+desc "The socket lock file of the daemon server"
+arg_name "FILE"
+flag :socket, :type => String, :default_value => "/var/run/cognizant/cognizantd.sock"
 
-  Options:
-    EOF
+desc "The server address of the daemon server"
+arg_name "ADDRESS"
+flag "bind-address", :type => String
 
-    opts.on("-s FILE", "--socket FILE", String, "The socket lock file for the server") do |value|
-      options[:socket] = value
-    end
+desc "The server port of the daemon server"
+arg_name "PORT"
+flag :port, :type => Integer
 
-    opts.on("-b ADDRESS", "--bind-address ADDRESS", String, "The interface to bind the TCP server to.") do |value|
-      options[:bind_address] = value
-    end
+desc "Username to use for authentication with server"
+arg_name "USERNAME"
+flag :username, :type => String
 
-    opts.on("-p PORT", "--port PORT", Integer, "The TCP port to start the server with.") do |value|
-      options[:port] = value
-    end
+desc "Password to use for authentication with server"
+arg_name "PASSWORD"
+flag :password, :type => String
 
-    opts.on("--username USERNAME", String, "Username for securing server access.") do |value|
-      options[:username] = value
-    end
+desc "Print the version number and exit"
+switch :version, :negatable => false
 
-    opts.on("--password PASSWORD", String, "Password to accompany the username.") do |value|
-      options[:password] = value
-    end
+desc "Turn on tracing, enabling full backtrace"
+switch :trace, :negatable => false
+
+desc "Display status of managed process(es) or group(s)"
+arg_name "process_name", :optional
+command :status do |c|
+  c.action do |global_options, options, args|
+    connect_and_perform(global_options, :status, args) do |output|
+      begin  
+        JSON.parse(output)
 
-    opts.on("-t", "--trace", "Turn on tracing, enable full backtrace.") do
-      options[:trace] = true
+        status_data = JSON.parse(output)
+        status_data.each_with_index do |process, index|
+          status_data[index]["State"]  = "[green]#{process["State"]}[/] since #{distance_of_time_in_words(Time.now.to_i - process["Since"])}"
+          status_data[index]["Memory"] = number_to_human_size(process["Memory"] * 1024) # Convert kilobytes to bytes.
+        end
+
+        formatador = Formatador.new
+        formatador.display_line
+        formatador.display_table(status_data, ["Process", "Group", "State", "PID", "% CPU", "Memory"])
+        formatador.display_line(Time.now)
+        formatador.display_line
+      rescue => e
+        Formatador.display_line("[red]#{output}[/]")
+      end
     end
+  end
+end
+
+desc "Loads the process information from specified Ruby file"
+arg_name "file"
+command :load do |c|
+  c.action do |global_options, options, args|
+    connect_and_perform(global_options, :load, args)
+  end
+end
+
+desc "Monitor the specified process or group"
+arg_name "process_name"
+command :monitor do |c|
+  c.action do |global_options, options, args|
+    connect_and_perform(global_options, :monitor, args)
+  end
+end
+
+desc "Unmonitor the specified process or group"
+arg_name "process_name"
+command :unmonitor do |c|
+  c.action do |global_options, options, args|
+    connect_and_perform(global_options, :unmonitor, args)
+  end
+end
 
-    opts.on("-v", "--version", "Print the version number and exit.") do
-      require 'cognizant/version'
-      $stdout.puts Cognizant::VERSION
-      exit(0)
+desc "Start the specified process or group"
+arg_name "process_name"
+command :start do |c|
+  c.action do |global_options, options, args|
+    connect_and_perform(global_options, :start, args)
+  end
+end
+
+desc "Stop the specified process or group"
+arg_name "process_name"
+command :stop do |c|
+  c.action do |global_options, options, args|
+    connect_and_perform(global_options, :stop, args)
+  end
+end
+
+desc "Restart the specified process or group"
+arg_name "process_name"
+command :restart do |c|
+  c.action do |global_options, options, args|
+    connect_and_perform(global_options, :restart, args)
+  end
+end
+
+desc "Stop the monitoring daemon without affecting processes"
+command :shutdown do |c|
+  c.action do |global_options, options, args|
+    connect_and_perform(global_options, :shutdown, args)
+  end
+end
+
+def connect_and_perform(global_options, task, args, &block)
+  output_handler = Proc.new do |socket|
+    socket.write("#{task.to_s} #{args.join(' ')}")
+    socket.flush
+    output = socket.gets
+    if block
+      block.call(output)
+    else
+      Formatador.display_line(output)
     end
-  end.parse!
+  end
 
-  require 'cognizant'
-  Cognizant.add_process(options)
+  if global_options[:socket]
+    output_handler.call(UNIXSocket.new(global_options[:socket]))
+  else
+    output_handler.call(TCPSocket.new(global_options["bind-address"], global_options["port"]))
+  end
 end
+
+def distance_of_time_in_words(seconds)
+  minutes = seconds / 60
+  return nil if minutes < 0
+  case minutes
+    when 0               then "less than a minute"
+    when 1..59           then pluralize((minutes/1).floor, "minute", "minutes")
+    when 60..1439        then pluralize((minutes/60).floor, "hour", "hours")
+    when 1440..11519     then pluralize((minutes/1440).floor, "day", "days")
+    when 11520..43199    then pluralize((minutes/11520).floor, "week", "weeks")
+    when 43200..525599   then pluralize((minutes/43200).floor, "month", "months")  
+    else                      pluralize((minutes/525600).floor, "year", "years")
+  end
+end
+
+SIZE_PREFIX = %w(TiB GiB MiB KiB Bytes).freeze
+def number_to_human_size(size)
+  size = size.to_f
+  i = SIZE_PREFIX.length - 1
+  while size > 512 && i > 0
+    i -= 1
+    size /= 1024
+  end
+  ((size > 9 || size.modulo(1) < 0.1 ? "%d" : "%.1f") % size) + " " + SIZE_PREFIX[i]
+end
+
+def pluralize(count, singular, plural)
+  "#{count || 0} " + ((count == 1 || count =~ /^1(\.0+)?$/) ? singular : plural)
+end
+
+exit run(ARGV)

data/bin/cognizantd

@@ -8,27 +8,34 @@ $stdout.sync = true
 $stderr.sync = true
 
 begin
-  require 'optparse'
+  require "optparse"
+  require "cognizant/version"
 
   options = {}
 
   OptionParser.new do |opts|
     opts.banner = <<-EOF
-  Usage:
-    cognizantd [OPTIONS] [CONFIG | -]
+NAME
+    cognizantd - system utility daemon to supervise your processes
 
-  Options:
+SYNOPSIS:
+    cognizantd [global options] [config file | -]
+
+VERSION
+    #{Cognizant::VERSION}
+
+GLOBAL OPTIONS:
     EOF
 
-    opts.on("--[no-]daemonize", "Whether or not to daemonize cognizant into background.") do |value|
+    opts.on("--[no-]daemonize", "Whether or not to daemonize cognizantd into background.") do |value|
       options[:daemonize] = value
     end
 
-    opts.on("--pidfile FILE", String, "The pid lock file for the daemon.") do |value|
+    opts.on("--pidfile FILE", String, "The pid (process identifier) lock file for the daemon.") do |value|
       options[:pidfile] = value
     end
 
-    opts.on("--logfile FILE", String, "The file to log the daemon's operations information into.") do |value|
+    opts.on("--logfile FILE", String, "The file to log the daemon's operational information into.") do |value|
       options[:logfile] = value
     end
 
@@ -89,23 +96,26 @@ begin
     end
 
     opts.on("-v", "--version", "Print the version number and exit.") do
-      require 'cognizant/version'
+      require "cognizant/version"
       $stdout.puts Cognizant::VERSION
       exit(0)
     end
   end.parse!
 
   if config_file = ARGV.shift
+    require "cognizant/util/symbolize_hash_keys"
+
     if config_file.eql?("-")
       config = YAML.load(ARGF.read)
     else
       config = YAML.load_file(config_file)
     end
     config = config.inject({}) { |c,(k,v)| c[k.gsub("-", "_").to_sym] = v; c }
+    config.deep_symbolize_keys!
     options = config.merge(options)
   end
 
-  require 'cognizant/server'
+  require "cognizant/server"
   Cognizant::Server.start(options)
 rescue => exception
   if exception.instance_of?(SystemExit)

data/cognizant.gemspec

@@ -4,9 +4,9 @@ require File.expand_path('../lib/cognizant/version', __FILE__)
 Gem::Specification.new do |gem|
   gem.authors       = ["Gurpartap Singh"]
   gem.email         = ["contact@gurpartap.com"]
-  gem.description   = "Process monitoring and management framework"
-  gem.summary       = "Cognizant is an advanced and efficient process monitoring and management framework."
-  gem.homepage      = "http://gurpartap.github.com/cognizant/"
+  gem.description   = "Cognizant is a process management system utility that supervises your processes, ensuring their state based on a flexible criteria."
+  gem.summary       = "Cognizant is a process management system utility that supervises your processes, ensuring their state based on a flexible criteria."
+  gem.homepage      = "http://github.com/Gurpartap/cognizant"
 
   gem.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   gem.files         = `git ls-files`.split("\n")
@@ -19,6 +19,12 @@ Gem::Specification.new do |gem|
   gem.add_development_dependency "redcarpet"
   gem.add_development_dependency "yard"
 
+  # cognizantd
   gem.add_dependency "eventmachine"
   gem.add_dependency "state_machine"
+  gem.add_dependency "activesupport"
+
+  # cognizant
+  gem.add_dependency "gli"
+  gem.add_dependency "formatador"
 end

data/examples/cognizantd.yml

@@ -7,17 +7,44 @@ pids_dir: ~/.cognizant/pids/
 logs_dir: ~/.cognizant/logs/
 
 monitor: {
-  redis-server: {
+  redis-server-1: {
+    autostart: false,
+    group: redis,
     start_command: /usr/local/bin/redis-server -,
-    start_with_input: daemonize no,
-    ping_command: redis-cli PING,
-    autostart: true,
-    stop_timeout: 5,
-    start_timeout: 2,
-    restart_timeout: 2
+    start_with_input: "daemonize no\nport 6666",
+    ping_command: redis-cli -p 6666 PING,
+    stop_signals: [TERM, INT],
+    checks: {
+      cpu_usage: {
+        every: 3,
+        above: 60,
+        times: 3,
+        do: restart
+      },
+      memory_usage: {
+        every: 5, # Seconds.
+        above: 1220608, # Bytes.
+        times: [3, 5], # Three out of five times.
+        do: stop # Defaults to restart.
+      }
+    }
   },
-  sleep-10: {
-    start_command: sleep 10,
-    autostart: true
+  redis-server-2: {
+    autostart: false,
+    group: redis,
+    start_command: /usr/local/bin/redis-server -,
+    start_with_input: "daemonize no\nport 7777",
+    ping_command: redis-cli -p 7777 PING,
+    stop_command: redis-cli -p 7777 SHUTDOWN
+  },
+  sleep: {
+    start_command: sleep 3,
+    checks: {
+      flapping: {
+        times: 4,
+        within: 15, # Seconds.
+        retry_after: 30 # Seconds.
+      }
+    }
   }
 }

data/examples/redis-server.rb

@@ -1,11 +1,11 @@
-# 4 slave instances to master at port 6000.
-5.times do |i|
-  Cognizant.monitor "redis-server-600#{i}" do |process|
-    process.group            = "redis"
-    process.uid              = "redis"
-    process.gid              = "redis"
-    process.start_command    = "redis-server -"
-    process.ping_command     = "redis-cli -p 600#{i} PING"
+# 2 slave instances to master at port 6000.
+3.times do |i|
+  Cognizant.monitor("redis-server-600#{i}") do |process|
+    process.group         = "redis"
+    # process.uid           = "redis"
+    # process.gid           = "redis"
+    process.start_command = "redis-server -"
+    process.ping_command  = "redis-cli -p 600#{i} PING"
 
     slaveof = i == 0 ? "" : "slaveof 127.0.0.1 6000"
     process.start_with_input = <<-heredoc
@@ -13,5 +13,16 @@
       port 600#{i}
       #{slaveof}
     heredoc
+
+    # process.on :always_true, :every => 2.seconds, :times => 3 do |p|
+    #   `say "Boom!"`
+    # end
+
+    process.check(:flapping, :times => 2, :within => 30.seconds, :retry_after => 7.seconds)
+    process.check(:cpu_usage, :every => 3.seconds, :above => 60, :times => 3, :do => :restart)
+
+    process.check(:memory_usage, :every => 5.seconds, :above => 100.megabytes, :times => [3, 5]) do |p|
+      p.restart # Restart is the default anyways.
+    end
   end
 end