flapjack 0.6.38 → 0.6.39

data/.gitmodules

@@ -0,0 +1,3 @@
+[submodule "doc"]
+	path = doc
+	url = https://github.com/flpjck/flapjack.wiki.git

data/.travis.yml

@@ -7,4 +7,6 @@ services:
   - redis-server
 before_script:
   - mkdir -p ./log
+before_install:
+  - git submodule update --init --recursive
 script: bundle exec rspec spec && bundle exec cucumber features

data/README.md

@@ -1,5 +1,4 @@
-Flapjack
-========
+# Flapjack
 
 [![Travis CI Status][id_travis_img]][id_travis_link]
 
@@ -10,203 +9,33 @@ Flapjack is a highly scalable and distributed monitoring notification system.
 
 Flapjack provides a scalable method for dealing with events representing changes in system state (OK -> WARNING -> CRITICAL transitions) and alerting appropriate people as necessary.
 
-At its core, flapjack process events received from external check execution engines, such as Nagios. Nagios provides a 'perfdata' event output channel, which writes to a named pipe. `flapjack-nagios-receiver` then reads from this named pipe, converts each line to JSON and adds them to the events queue. `executive` picks up the events and processes them - deciding when and who to notifify about problems, recoveries, acknowledgements etc. Additional check engines can be supported by adding additional receiver processes similar to the nagios receiver.
+At its core, flapjack process events received from external check execution engines, such as Nagios. Nagios provides a 'perfdata' event output channel, which writes to a named pipe. `flapjack-nagios-receiver` then reads from this named pipe, converts each line to JSON and adds them to the events queue. Flapjack's `executive` component picks up the events and processes them -- deciding when and who to notifify about problems, recoveries, acknowledgements etc. Additional check engines can be supported by adding additional receiver processes similar to the nagios receiver.
 
 
-What things do
---------------
+## Using Flapjack
 
-Executables:
+### Quickstart
 
-  * `flapjack` => starts multiple components ('pikelets') within the one ruby process as specified in the configuration file.
-  * `flapjack-nagios-receiver` => reads nagios check output on standard input and places them on the events queue in redis as JSON blobs. Currently unable to be run in-process with `flapjack`
+TODO numbered list for simplest possible Flapjack run
 
-There are more fun executables in the bin directory.
+For more information, including full specification of the configuration file and the data import formats, please refer to the [Flapjack Wiki](https://github.com/ali-graham/flapjack/wiki/USING).
 
-Pikelets:
+## Developing Flapjack
 
-  * `executive` => processes events off a queue (a redis list) and decides what actions to take (alert, record state changes, etc)
-  * `email_notifier` => generates email notifications (resque, mail)
-  * `sms_notifier` => generates sms notifications (resque)
-  * `jabber_gateway` => connects to an XMPP (jabber) server, sends notifications (to rooms and individuals), handles acknowledgements from jabber users and other commands (blather)
-  * `web` => web UI (sinatra, thin)
-  * `api` => HTTP API server (sinatra, thin)
+Information on developing more Flapjack components or contributing to core Flapjack development can be found in the [Flapjack Wiki](https://github.com/ali-graham/flapjack/wiki/DEVELOPING).
 
-Pikelets are flapjack components which can be run within the same ruby process, or as separate processes.
+## Documentation Submodule
 
-The simplest configuration will have one `flapjack` process running executive, web, and some notification gateways, and one `flapjack-nagios-receiver` process receiving events from Nagios and placing them on the events queue for processing by executive.
-
-
-Developing
-----------
-
-Clone the repository:
-
-    git clone https://auxesis@github.com/auxesis/flapjack.git
-
-Install development dependencies:
-
-    gem install bundler
-    bundle install
-
-Run the tests:
-
-    cucumber features/
-    rspec spec
-
-Testing
--------
-
-Feature tests live in `features/`.
-
-To run feature tests:
-
-    $ cucumber features
-
-There's some rspec unit tests as well, run these like so:
-
-    $ rspec spec
-
-NB, if the cucumber tests fail with a [spurious lexing error](https://github.com/cucumber/gherkin/issues/182) on line 2 of events.feature, then try this:
-
-    $ cucumber -f fuubar features
-
-
-
-Releasing
----------
-
-Gem releases are handled with [Bundler](http://gembundler.com/rubygems.html).
-
-To build the gem, run:
-
-    gem build flapjack.gemspec
-
-Configuring
-===========
+We have the documentation for this project on a github wiki and also referenced as a submodule at /doc in this project. Run the following commands to populate the local doc/ directory:
 
 ```
-cp etc/flapjack-config.yaml.example etc/flapjack-config.yaml
+git submodule init
+git submodule update
 ```
 
-Edit the configuration to suit (redis connection, smtp server, jabber server, sms gateway, etc)
-
-The default FLAPJACK_ENV is "development" if there is no environment variable set.
-
-Nagios config
--------------
-
-You need a Nagios prior to version 3.3 as this breaks perfdata output for checks which don't generate performance data (stuff after a | in the check output). We are developing and running against Nagios version 3.2.3 with success.
-
-nagios.cfg config file changes:
-
-```
-# modified lines:
-enable_notifications=0
-host_perfdata_file=/var/cache/nagios3/event_stream.fifo
-service_perfdata_file=/var/cache/nagios3/event_stream.fifo
-host_perfdata_file_template=[HOSTPERFDATA]\t$TIMET$\t$HOSTNAME$\tHOST\t$HOSTSTATE$\t$HOSTEXECUTIONTIME$\t$HOSTLATENCY$\t$HOSTOUTPUT$\t$HOSTPERFDATA$
-service_perfdata_file_template=[SERVICEPERFDATA]\t$TIMET$\t$HOSTNAME$\t$SERVICEDESC$\t$SERVICESTATE$\t$SERVICEEXECUTIONTIME$\t$SERVICELATENCY$\t$SERVICEOUTPUT$\t$SERVICEPERFDATA$
-host_perfdata_file_mode=p
-service_perfdata_file_mode=p
-```
-
-What we're doing here is telling Nagios to generate a line of output for every host and service check into a named pipe. The template lines must be as above so that `flapjack-nagios-receiver` knows what to expect.
-
-All hosts and services (or templates that they use) will need to have process_perf_data enabled on them. (This is a real misnomer, it doesn't mean the performance data will be processed, just that it will be fed to the perfdata output channel, a named pipe in our case.)
-
-Create the named pipe if it doesn't already exist:
-
-    mkfifo -m 0666 /var/cache/nagios3/event_stream.fifo
-
-
-Running
-=======
-
-    bin/flapjack [options]
-
-    $ flapjack --help
-    Usage: flapjack [options]
-        -c, --config [PATH]              PATH to the config file to use
-        -d, --[no-]daemonize             Daemonize?
-
-The command line option for daemonize overrides whatever is set in the config file.
-
-flapjack-nagios-receiver
-------------------------
-
-There is a control script that uses [Daemons](http://daemons.rubyforge.org/) to start, stop, restart, show status etc of the flapjack-nagios-receiver process. Options after -- are passed through to flapjack-nagios-receiver. Run it like so:
-
-    flapjack-nagios-receiver-control start -- --config /etc/flapjack/flapjack-config.yaml --fifo /path/to/nagios/perfdata.fifo
-    flapjack-nagios-receiver-control status
-    flapjack-nagios-receiver-control restart -- --config /etc/flapjack/flapjack-config.yaml --fifo /path/to/nagios/perfdata.fifo
-    flapjack-nagios-receiver-control stop
-
-* Bypassing daemons control script: *
-
-Specify the path to the nagios named pipe (fifo):
-
-    flapjack-nagios-receiver --fifo /var/cache/nagios3/event_stream.fifo
-
-Now as nagios feeds check execution results into the perfdata named pipe, flapjack-nagios-receiver will convert them to JSON encoded ruby objects and insert them into the *events* queue.
-
-Importing contacts and entities
--------------------------------
-
-The `flapjack-populator` script provides a mechanism for importing contacts and entities from JSON formatted import files.
-
-    bin/flapjack-populator import-contacts --from tmp/dummy_contacts.json --config /etc/flapjack/flapjack-config.yml
-    bin/flapjack-populator import-entities --from tmp/dummy_entities.json --config /etc/flapjack/flapjack-config.yml
-
-TODO: convert the format documentation into markdown and include in the project
-
-In lieu of actual documentation of the formats there are example files, and example ruby code which generated them, in the tmp directory.
-
-executive
----------
-
-Flapjack executive processes events from the 'events' list in redis. It does a blocking read on redis so new events are picked off the events list and processed as soon as they created.
-
-When executive decides somebody ought to be notified (for a problem, recovery, or acknowledgement or what-have-you) it looks up contact information and then creates a notification job on one of the notification queues (eg sms_notifications) in rescue, or via the jabber_notifications redis list which is processed by the jabber_gateway.
-
-Resque Workers
---------------
-
-We're using [Resque](https://github.com/defunkt/resque) to queue email and sms notifications generated by flapjack-executive. The queues are named as follows:
-- email_notifications
-- sms_notifications
-
-There'll be more of these as we add more notification mediums.
-
-Note that using the flapjack-config.yaml file you can have flapjack start the resque workers in-process. Or you can run them standalone with the `rake resque:work` command as follows.
-
-One resque worker that processes both queues (but prioritises SMS above email) can be started as follows:
-
-    QUEUE=sms_notifications,email_notifications VERBOSE=1 be rake resque:work
-
-resque sets the command name so grep'ing ps output for `rake` or `ruby` will NOT find resque processes. Search instead for `resque`. (and remember the 'q').
-
-To background it you can add `BACKGROUND=yes`. Excellent documentation is available in [Resque's README](https://github.com/defunkt/resque/blob/master/README.markdown)
-
-Redis Database Instances
-------------------------
-
-We use the following redis database numbers by convention:
-
-* 0 => production
-* 13 => development
-* 14 => testing
-
-Architecture
-------------
-
-TODO: insert architecture diagram and notes here
-
-Dependencies
-------------
+If you make changes to the documentation locally, here's how to publish them:
 
-Apart from a bundle of gems (see flapjack.gemspec for runtime gems, and Gemfile for additional gems required for testing):
-- Ruby >= 1.9
-- Redis >= 2.4.15
+* git add, commit and push from inside the doc subdir
+* add, commit and push the doc dir from the root (this updates the pointer in the main git repo to the correct ref in the doc repo, we think...)
 
 

data/Rakefile

@@ -13,15 +13,14 @@ Dir['tasks/**/*.rake'].each { |t| load t }
 
 require 'cucumber'
 require 'cucumber/rake/task'
-require 'colorize'
 require 'rake/clean'
 require 'bundler'
 Bundler::GemHelper.install_tasks
 
 Cucumber::Rake::Task.new(:features) do |t|
-  #t.cucumber_opts = "features --format pretty"
-  #t.cucumber_opts = "--format progress"
-  t.cucumber_opts = "--format fuubar"
+  #t.cucumber_opts = 'features --format pretty'
+  #t.cucumber_opts = '--format progress'
+  t.cucumber_opts = '--format fuubar'
 end
 
 require 'rspec/core/rake_task'
@@ -30,12 +29,18 @@ RSpec::Core::RakeTask.new(:spec)
 task :default => :tests
 
 namespace :verify do
+
   task :uncommitted do
+
+    def red(text)
+      "\e[0;31;49m#{text}\e[0m"
+    end
+
     uncommitted = `git ls-files -m`.split("\n")
     if uncommitted.size > 0
-      puts "The following files are uncommitted:".red
+      puts red('The following files are uncommitted:')
       uncommitted.each do |filename|
-        puts " - #{filename}".red
+        puts red(" - #{filename}")
       end
       exit 1
     end
@@ -44,6 +49,5 @@ namespace :verify do
   task :all => [ :uncommitted ]
 end
 
-# FIXME: getting that intermittent gherken lexing error so removing :features from verify list
 task :verify => [ 'verify:all', :spec, :features]
 task :tests  => [ :spec, :features]

data/bin/flapjack

@@ -1,9 +1,15 @@
 #!/usr/bin/env ruby
 
-require 'yaml'
 require 'optparse'
 require 'ostruct'
 
+# add lib to the default include path
+unless $:.include?(File.dirname(__FILE__) + '/../lib/')
+  $: << File.dirname(__FILE__) + '/../lib'
+end
+
+require 'flapjack/configuration'
+
 options = OpenStruct.new
 options.config    = File.join('etc', 'flapjack_config.yaml')
 options.daemonize = nil
@@ -27,16 +33,7 @@ end.parse!(ARGV)
 
 FLAPJACK_ENV = ENV['FLAPJACK_ENV'] || 'development'
 
-# load the config hash for the current environment
-
-if File.file?(options.config)
-  config = YAML::load_file(options.config)
-else
-  puts "Could not find config file at '#{options.config}', you may want to specify one with the --config option"
-  exit(false)
-end
-
-config_env = config[FLAPJACK_ENV]
+config_env = Flapjack::Configuration.new.load(options.config)
 
 if config_env.nil? || config_env.empty?
   puts "No config data for environment '#{FLAPJACK_ENV}' found in '#{options.config}'"
@@ -49,11 +46,6 @@ else
   pid_file = options.pidfile
 end
 
-# add lib to the default include path
-unless $:.include?(File.dirname(__FILE__) + '/../lib/')
-  $: << File.dirname(__FILE__) + '/../lib'
-end
-
 # TODO Flapjack falls over when Redis restarted -- trap errors and attempt reconnect
 
 require 'flapjack/coordinator'
@@ -69,4 +61,4 @@ else
 end
 
 puts "Daemonising ... " if daemonize
-coordinator.start(:daemonize => daemonize)
+coordinator.start(:daemonize => daemonize, :signals => true)

data/bin/flapjack-nagios-receiver

@@ -1,11 +1,16 @@
 #!/usr/bin/env ruby
 
-require 'redis'
-require 'json'
-require 'yaml'
 require 'optparse'
 require 'ostruct'
-require 'flapjack/data/entity_check'
+require 'redis'
+require 'yajl/json_gem'
+
+# add lib to the default include path
+unless $:.include?(File.dirname(__FILE__) + '/../lib/')
+  $: << File.dirname(__FILE__) + '/../lib'
+end
+
+require 'flapjack/configuration'
 
 options = OpenStruct.new
 options.config    = File.join('etc', 'flapjack_config.yaml')
@@ -26,18 +31,9 @@ end.parse!(ARGV)
 
 FLAPJACK_ENV = ENV['FLAPJACK_ENV'] || 'development'
 
-# load the config hash for the current environment
-
-if File.file?(options.config)
-  config = YAML::load(File.open(options.config))
-else
-  puts "Could not find config file at '#{options.config}', you may want to specify one with the --config option"
-  exit(false)
-end
+@config_env = Flapjack::Configuration.new.load(options.config)
 
-config_env = config[FLAPJACK_ENV]
-
-if config_env.nil? || config_env.empty?
+if @config_env.nil? || @config_env.empty?
   puts "No config data for environment '#{FLAPJACK_ENV}' found in '#{options.config}'"
   exit(false)
 end
@@ -55,24 +51,13 @@ end
 # host_perfdata_file_template=[HOSTPERFDATA]\t$TIMET$\t$HOSTNAME$\tHOST\t$HOSTSTATE$\t$HOSTEXECUTIONTIME$\t$HOSTLATENCY$\t$HOSTOUTPUT$\t$HOSTPERFDATA$
 # service_perfdata_file_template=[SERVICEPERFDATA]\t$TIMET$\t$HOSTNAME$\t$SERVICEDESC$\t$SERVICESTATE$\t$SERVICEEXECUTIONTIME$\t$SERVICELATENCY$\t$SERVICEOUTPUT$\t$SERVICEPERFDATA$
 
-@redis_host     = config_env['redis']['host'] || 'localhost'
-@redis_port     = config_env['redis']['port'] || '6379'
-@redis_path     = config_env['redis']['path'] || nil
-@redis_db       = config_env['redis']['db']   || 0
-
-if config_env.nil? || config_env.empty?
-  puts "No config data for environment '#{FLAPJACK_ENV}'"
-  exit(false)
+def get_redis_connection(cfg)
+  opts = cfg['path'] ? {:path => cfg['path']} :
+                       {:host => cfg['host'], :port => cfg['port']}
+  Redis.new(opts.merge(:db => cfg['db']))
 end
 
-def process_input
-
-  if @redis_path
-    redis = Redis.new(:db => @redis_db, :path => @redis_path)
-  else
-    redis = Redis.new(:db => @redis_db, :host => @redis_host, :port => @redis_port)
-  end
-
+def process_input(redis)
   begin
     while line = @fifo.gets
       skip unless line
@@ -112,8 +97,10 @@ def process_input
 end
 
 def main
+  redis = get_redis_connection(@config_env['redis'])
+
   while true
-    process_input
+    process_input(redis)
     puts "Whoops, restarting main loop in 10 seconds"
     sleep 10
   end

data/bin/flapjack-populator

@@ -1,9 +1,9 @@
 #!/usr/bin/env ruby
 
-require 'yajl/json_gem'
-require 'redis'
 require 'optparse'
 require 'ostruct'
+require 'yajl/json_gem'
+require 'redis'
 
 # add lib to the default include path
 unless $:.include?(File.dirname(__FILE__) + '/../lib/')
@@ -45,6 +45,11 @@ FLAPJACK_ENV = ENV['FLAPJACK_ENV'] || 'development'
 
 config_env = Flapjack::Configuration.new.load(options.config)
 
+if config_env.nil? || config_env.empty?
+  puts "No config data for environment '#{FLAPJACK_ENV}' found in '#{options.config}'"
+  exit(false)
+end
+
 if options.from
   filename = options.from
   file     = File.new(filename)

data/etc/flapjack_config.yaml.example

@@ -1,5 +1,25 @@
 ---
 
+quickstart:
+  redis:
+    host: 127.0.0.1
+    port: 6379
+    db: 6
+  executive:
+    enabled: yes
+    email_queue: email_notifications
+    notification_log_file: log/flapjack-notification.log
+  email_notifier:
+    enabled: yes
+    queue: email_notifications
+    smtp_config:
+      address: "localhost"
+      domain: 'localhost.localdomain'
+      port: 25
+  web:
+    enabled: yes
+    port: 5080
+
 development:
   pid_file: tmp/pids/flapjack.pid
   log_file: log/flapjack.log