flapjack 0.7.28 → 0.7.29

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: a8a6bbb199b138fb0da3af9765f5f0fe7b2ad552
+  data.tar.gz: 3dbd00e9e7e63978726cb0f1e78fc96639f1762b
+SHA512:
+  metadata.gz: 981b9061652d0a75e5e32cb82590c20eeca4a7b2d624aad076097c8a39c799a0850110e67fe080afa6ffac5dbe16d39184d06014db3d3cdbf1c469353425019f
+  data.tar.gz: cad93481bc1dfd36c636ea84a0e7d59a9e5b1c926e046cf15b35a58cf4d4d06545bdd600b0cfe4d3e343ab6298944b40efc6ea89747355da04c456db56f46058

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 ## Flapjack Changelog
 
+# 0.7.29 - 2013-11-01
+- Feature: Per contact media rollup complete gh-291 (@jessereynolds)
+- Feature: Rollup - show alerting checks and rollup status for each media on contact page gh-326 (@jessereynolds)
+- Feature: Make message templating more flexible and consistent gh-329 (@jessereynolds)
+- Feature: Remove misleading internal metrics gh-344 (@jessereynolds)
+- Bug: clear up pagerduty details API code gh-338 (@ali-graham)
+- Bug: non-numeric contact ID not properly supported - thanks @avishai-ish-shalom gh-338 (@jessereynolds)
+- Bug: GET /contacts gives you more than you bargained for gh-339 (@jessereynolds)
+- Bug: SMS alerts being sent from separate environment gh-135 (@jessereynolds)
+
 # 0.7.28 - 2013-10-21
 - Feature: Per contact media rollup (summary) notifications - foundation (data, api, web UI etc) gh-291 (@jessereynolds)
 - Feature: flapper takes an IP to bind to gh-98 (@ali-graham)

data/features/notification_rules.feature

@@ -4,51 +4,51 @@ Feature: Notification rules on a per contact basis
   Background:
     Given the following users exist:
       | id  | first_name | last_name | email             | sms          | timezone         |
-      | 1   | Malak      | Al-Musawi | malak@example.com | +61400000001 | Asia/Baghdad     |
-      | 2   | Imani      | Farooq    | imani@example.com | +61400000002 | Europe/Moscow    |
-      | 3   | Vera       | Дурейко   | vera@example.com  | +61400000003 | Europe/Paris     |
-      | 4   | Lucia      | Moretti   | lucia@example.com | +61400000004 | Europe/Rome      |
-      | 5   | Wang Fang  | Wong      | fang@example.com  | +61400000005 | Asia/Shanghai    |
+      | c1  | Malak      | Al-Musawi | malak@example.com | +61400000001 | Asia/Baghdad     |
+      | c2  | Imani      | Farooq    | imani@example.com | +61400000002 | Europe/Moscow    |
+      | c3  | Vera       | Дурейко   | vera@example.com  | +61400000003 | Europe/Paris     |
+      | c4  | Lucia      | Moretti   | lucia@example.com | +61400000004 | Europe/Rome      |
+      | c5  | Wang Fang  | Wong      | fang@example.com  | +61400000005 | Asia/Shanghai    |
 
     And the following entities exist:
       | id  | name           | contacts |
-      | 1   | foo            | 1        |
-      | 2   | bar            | 1,2,3    |
-      | 3   | baz            | 1,3      |
-      | 4   | buf            | 1,2,3    |
-      | 5   | foo-app-01.xyz | 4        |
+      | 1   | foo            | c1       |
+      | 2   | bar            | c1,c2,c3 |
+      | 3   | baz            | c1,c3    |
+      | 4   | buf            | c1,c2,c3 |
+      | 5   | foo-app-01.xyz | c4       |
 
-    And user 1 has the following notification intervals:
+    And user c1 has the following notification intervals:
       | email | sms |
       | 15    | 60  |
 
-    And user 2 has the following notification intervals:
+    And user c2 has the following notification intervals:
       | email | sms |
       | 15    | 60  |
 
-    And user 3 has the following notification intervals:
+    And user c3 has the following notification intervals:
       | email | sms |
       | 15    | 60  |
 
-    And user 4 has the following notification intervals:
+    And user c4 has the following notification intervals:
       | email | sms |
       | 15    | 60  |
 
-    And user 1 has the following notification rules:
+    And user c1 has the following notification rules:
       | entities | unknown_media | warning_media | critical_media   | warning_blackhole | critical_blackhole | time_restrictions |
       |          |               | email         | sms,email        | true              | true               |                   |
       | foo      |               | email         | sms,email        |                   |                    | 8-18 weekdays     |
       | bar      | email         |               | sms,email        | true              |                    |                   |
       | baz      |               | email         | sms,email        |                   |                    |                   |
 
-    And user 2 has the following notification rules:
+    And user c2 has the following notification rules:
       | entities | tags | warning_media | critical_media   | warning_blackhole | critical_blackhole |
       |          |      | email         | email            |                   |                    |
       |          |      | sms           | sms              |                   |                    |
       | bar      |      | email         | email,sms        |                   |                    |
       | bar      | wags |               |                  | true              | true               |
 
-    And user 3 has the following notification rules:
+    And user c3 has the following notification rules:
       | entities | warning_media | critical_media   | warning_blackhole | critical_blackhole |
       |          | email         | email            |                   |                    |
       | baz      | sms           | sms              |                   |                    |
@@ -56,13 +56,13 @@ Feature: Notification rules on a per contact basis
       | buf      | sms           | sms              |                   |                    |
       | bar      | email         | email            | true              | true               |
 
-    And user 4 has the following notification rules:
+    And user c4 has the following notification rules:
       | tags            | warning_media | critical_media   | time_restrictions |
       |                 |               |                  |                   |
       | xyz, disk, util | sms           | sms              |                   |
       | xyz, ping       | sms,email     | sms,email        | 8-18 weekdays     |
 
-    And user 5 has the following notification rules:
+    And user c5 has the following notification rules:
       | unknown_media | critical_media |
       | email         | email, sms     |
 
@@ -81,15 +81,15 @@ Feature: Notification rules on a per contact basis
     And   a critical event is received
     Then  1 email alert should be queued for malak@example.com
     When  the time is February 1 2013 12:00
-    Then  all alert dropping keys for user 1 should have expired
+    Then  all alert dropping keys for user c1 should have expired
     When  a critical event is received
     Then  2 email alerts should be queued for malak@example.com
     When  the time is February 1 2013 17:59
-    Then  all alert dropping keys for user 1 should have expired
+    Then  all alert dropping keys for user c1 should have expired
     When  a critical event is received
     Then  3 email alerts should be queued for malak@example.com
     When  the time is February 1 2013 18:01
-    Then  all alert dropping keys for user 1 should have expired
+    Then  all alert dropping keys for user c1 should have expired
     When  a critical event is received
     Then  3 email alerts should be queued for malak@example.com
 
@@ -395,15 +395,15 @@ Feature: Notification rules on a per contact basis
     And   a critical event is received
     Then  1 sms alert should be queued for +61400000004
     When  the time is February 1 2013 12:00
-    Then  all alert dropping keys for user 1 should have expired
+    Then  all alert dropping keys for user c1 should have expired
     When  a critical event is received
     Then  2 sms alerts should be queued for +61400000004
     When  the time is February 1 2013 17:59
-    Then  all alert dropping keys for user 1 should have expired
+    Then  all alert dropping keys for user c1 should have expired
     When  a critical event is received
     Then  3 sms alerts should be queued for +61400000004
     When  the time is February 1 2013 18:01
-    Then  all alert dropping keys for user 1 should have expired
+    Then  all alert dropping keys for user c1 should have expired
     When  a critical event is received
     Then  3 sms alerts should be queued for +61400000004
 

data/features/rollup.feature

@@ -177,22 +177,42 @@ Feature: Rollup on a per contact, per media basis
     Then  1 sms alert of type problem and rollup problem should be queued for +61400000001
     And   2 sms alerts should be queued for +61400000001
 
-#  @time
-#  Scenario: Contact ceases to be a contact on an entity that they were being alerted for
-#    Given check 'ping' for entity 'foo' is in an ok state
-#    And   check 'ping' for entity 'baz' is in an ok state
-#    When  a critical event is received for check 'ping' on entity 'foo'
-#    And   1 minute passes
-#    And   a critical event is received for check 'ping' on entity 'foo'
-#    Then  1 sms alerts of type problem and rollup none should be queued for +61400000001
-#    When  5 minutes passes
-#    And   a critical event is received for check 'ping' on entity 'baz'
-#    And   1 minute passes
-#    And   a critical event is received for check 'ping' on entity 'baz'
-#    Then  1 sms alert of type problem and rollup problem should be queued for +61400000001
-#    And   2 sms alerts should be queued for +61400000001
-#    When  1 minute passes
-#    And   user 1 ceases to be a contact of entity 'foo'
-#    And   a critical event is received for check 'ping' on entity 'baz'
-#    Then  1 sms alert of rollup recovery should be queued for +61400000001
+  @time
+  Scenario: Contact ceases to be a contact on an entity that they were being alerted for
+    Given check 'ping' for entity 'foo' is in an ok state
+    And   check 'ping' for entity 'baz' is in an ok state
+    When  a critical event is received for check 'ping' on entity 'foo'
+    And   1 minute passes
+    And   a critical event is received for check 'ping' on entity 'foo'
+    Then  1 sms alerts of type problem and rollup none should be queued for +61400000001
+    When  5 minutes passes
+    And   a critical event is received for check 'ping' on entity 'baz'
+    And   1 minute passes
+    And   a critical event is received for check 'ping' on entity 'baz'
+    Then  1 sms alert of type problem and rollup problem should be queued for +61400000001
+    And   1 sms alerts of type problem and rollup none should be queued for +61400000001
+    And   2 sms alerts should be queued for +61400000001
+    When  20 minute passes
+    And   user 1 ceases to be a contact of entity 'foo'
+    And   a critical event is received for check 'ping' on entity 'baz'
+    Then  1 sms alert of rollup recovery should be queued for +61400000001
+
+  @time
+  Scenario: Test notification to not contribute to rollup
+    Given check 'ping' for entity 'foo' is in an ok state
+    And   check 'ping' for entity 'baz' is in an ok state
+    When  a critical event is received for check 'ping' on entity 'foo'
+    And   1 minute passes
+    And   a critical event is received for check 'ping' on entity 'foo'
+    Then  1 sms alert of type problem and rollup none should be queued for +61400000001
+    And   1 sms alert should be queued for +61400000001
+    When  1 minute passes
+    And   a test event is received for check 'sausage' on entity 'foo'
+    Then  1 sms alert of type problem and rollup none should be queued for +61400000001
+    And   1 sms alert of type test and rollup none should be queued for +61400000001
+    And   2 sms alerts should be queued for +61400000001
+    When  20 minutes passes
+    And   a critical event is received for check 'ping' on entity 'foo'
+    Then  2 sms alerts of type problem and rollup none should be queued for +61400000001
+    And   3 sms alerts should be queued for +61400000001
 

data/features/steps/events_steps.rb

@@ -301,7 +301,7 @@ Then /^show me the (\w+ )*log$/ do |adjective|
   puts @logger.messages.join("\n")
 end
 
-Then /^dump notification rules for user (\d+)$/ do |contact|
+Then /^dump notification rules for user (\S+)$/ do |contact|
   rule_ids = @redis.smembers("contact_notification_rules:#{contact}")
   puts "There #{(rule_ids.length == 1) ? 'is' : 'are'} #{rule_ids.length} notification rule#{(rule_ids.length == 1) ? '' : 's'} for user #{contact}:"
   rule_ids.each {|rule_id|
@@ -338,7 +338,7 @@ Given /^the following users exist:$/ do |contacts|
   end
 end
 
-Given /^user (\d+) has the following notification intervals:$/ do |contact_id, intervals|
+Given /^user (\S+) has the following notification intervals:$/ do |contact_id, intervals|
   contact = Flapjack::Data::Contact.find_by_id(contact_id, :redis => @redis)
   intervals.hashes.each do |interval|
     contact.set_interval_for_media('email', interval['email'].to_i * 60)
@@ -346,7 +346,7 @@ Given /^user (\d+) has the following notification intervals:$/ do |contact_id, i
   end
 end
 
-Given /^user (\d+) has the following notification rollup thresholds:$/ do |contact_id, rollup_thresholds|
+Given /^user (\S+) has the following notification rollup thresholds:$/ do |contact_id, rollup_thresholds|
   contact = Flapjack::Data::Contact.find_by_id(contact_id, :redis => @redis)
   rollup_thresholds.hashes.each do |rollup_threshold|
     contact.set_rollup_threshold_for_media('email', rollup_threshold['email'].to_i)
@@ -354,7 +354,7 @@ Given /^user (\d+) has the following notification rollup thresholds:$/ do |conta
   end
 end
 
-Given /^user (\d+) has the following notification rules:$/ do |contact_id, rules|
+Given /^user (\S+) has the following notification rules:$/ do |contact_id, rules|
   contact = Flapjack::Data::Contact.find_by_id(contact_id, :redis => @redis)
   timezone = contact.timezone
 
@@ -400,7 +400,7 @@ Given /^user (\d+) has the following notification rules:$/ do |contact_id, rules
   end
 end
 
-Then /^all alert dropping keys for user (\d+) should have expired$/ do |contact_id|
+Then /^all alert dropping keys for user (\S+) should have expired$/ do |contact_id|
   @redis.keys("drop_alerts_for_contact:#{contact_id}*").should be_empty
 end
 
@@ -426,3 +426,8 @@ Then /^(\w+) (\w+) alert(?:s)?(?: of)?(?: type (\w+))?(?: and)?(?: rollup (\w+))
   }.length.should == num_queued.to_i
 end
 
+When(/^user (\S+) ceases to be a contact of entity '(.*)'$/) do |contact_id, entity|
+  entity = Flapjack::Data::Entity.find_by_name(entity, :redis => @redis)
+  @redis.srem("contacts_for:#{entity.id}", contact_id)
+end
+

data/features/steps/notifications_steps.rb

@@ -109,12 +109,14 @@ Given /^a user SMS notification has been queued for entity '([\w\.\-]+)'$/ do |e
   @sms_notification = {'notification_type'  => 'problem',
                        'contact_first_name' => 'John',
                        'contact_last_name'  => 'Smith',
-                       'state'              => 'CRITICAL',
+                       'state'              => 'critical',
                        'summary'            => 'Socket timeout after 10 seconds',
                        'time'               => Time.now.to_i,
                        'event_id'           => "#{entity}:ping",
                        'address'            => '+61412345678',
-                       'id'                 => 1}
+                       'id'                 => 1,
+                       'state_duration'     => 30,
+                       'duration'           => 45}
 end
 
 Given /^a user email notification has been queued for entity '([\w\.\-]+)'$/ do |entity|
@@ -124,12 +126,14 @@ Given /^a user email notification has been queued for entity '([\w\.\-]+)'$/ do
   @email_notification = {'notification_type'  => 'problem',
                          'contact_first_name' => 'John',
                          'contact_last_name'  => 'Smith',
-                         'state'              => 'CRITICAL',
+                         'state'              => 'critical',
                          'summary'            => 'Socket timeout after 10 seconds',
                          'time'               => Time.now.to_i,
                          'event_id'           => "#{entity}:ping",
                          'address'            => 'johns@example.dom',
-                         'id'                 => 2}
+                         'id'                 => 2,
+                         'state_duration'     => 30,
+                         'duration'           => 3600}
 end
 
 # NB using perform, the notifiers were accessing the wrong Redis DB number

data/lib/flapjack/data/alert.rb

@@ -0,0 +1,207 @@
+#!/usr/bin/env ruby
+
+require 'active_support/inflector'
+require 'flapjack/utility'
+
+# Alert is the object ready to send to someone, complete with an address and all
+# the data with which to render the text of the alert in the appropriate gateway
+#
+# It should possibly be renamed AlertPresenter
+
+module Flapjack
+  module Data
+    class Alert
+
+      # from Flapjack::Data::Notification
+      attr_reader :event_id,
+                  :state,
+                  :summary,
+                  :acknowledgement_duration,
+                  :last_state,
+                  :last_summary,
+                  :state_duration,
+                  :details,
+                  :time,
+                  :notification_type,
+                  :event_count,
+                  :tags
+
+      # from Flapjack::Data::Message
+                # :id,
+      attr_reader :media,
+                  :address,
+                  :rollup,
+                  :contact_id,
+                  :contact_first_name,
+                  :contact_last_name
+
+      # from Flapjack::Notifier
+      attr_reader :rollup_threshold,
+                  :rollup_alerts,
+                  :in_scheduled_maintenance,
+                  :in_unscheduled_maintenance
+
+      # from self
+      attr_reader :entity,
+                  :check,
+                  :notification_id
+
+      include Flapjack::Utility
+
+      def initialize(contents, opts)
+        raise "no logger supplied" unless @logger = opts[:logger]
+
+        @event_id                   = contents['event_id']
+        @state                      = contents['state']
+        @summary                    = contents['summary']
+        @acknowledgement_duration   = contents['duration'] # SMELLY
+        @last_state                 = contents['last_state']
+        @last_summary               = contents['last_summary']
+        @state_duration             = contents['state_duration']
+        @details                    = contents['details']
+        @time                       = contents['time']
+        @notification_type          = contents['notification_type']
+        @event_count                = contents['event_count']
+        @tags                       = contents['tags']
+
+        @media                      = contents['media']
+        @address                    = contents['address']
+        @rollup                     = contents['rollup']
+        @contact_id                 = contents['contact_id']
+        @contact_first_name         = contents['contact_first_name']
+        @contact_last_name          = contents['contact_last_name']
+
+        @rollup_threshold           = contents['rollup_threshold']
+        @rollup_alerts              = contents['rollup_alerts']
+        @in_scheduled_maintenance   = contents['in_scheduled_maintenance']
+        @in_unscheduled_maintenance = contents['in_unscheduled_maintenance']
+
+        @entity, @check             = @event_id.split(':', 2)
+        @notification_id            = contents['id'] || SecureRandom.uuid
+
+        allowed_states        = ['ok', 'critical', 'warning', 'unknown', 'test_notifications', 'acknowledgement']
+        allowed_rollup_states = ['critical', 'warning', 'unknown']
+        raise "state #{@state.inspect} is invalid" unless
+          allowed_states.include?(@state)
+
+        raise "state_duration #{@state_duration.inspect} is invalid" unless
+          @state_duration && @state_duration.is_a?(Integer) && @state_duration >= 0
+
+        if @rollup_alerts
+          raise "rollup_alerts should be nil or a hash" unless @rollup_alerts.is_a?(Hash)
+          @rollup_alerts.each_pair do |check, details|
+            raise "duration of rollup_alerts['#{check}'] must be an integer" unless
+              details['duration'] && details['duration'].is_a?(Integer)
+            raise "state of rollup_alerts['#{check}'] is invalid" unless
+              details['state'] && allowed_rollup_states.include?(details['state'])
+          end
+        end
+
+      end
+
+      def type
+        case @rollup
+        when "problem"
+          "rollup_problem"
+        when "recovery"
+          "rollup_recovery"
+        else
+          @notification_type
+        end
+      end
+
+      def type_sentence_case
+        case type
+        when "rollup_problem"
+          "Problem summary"
+        when "rollup_recovery"
+          "Problem summaries finishing"
+        else
+          type.titleize
+        end
+      end
+
+      def state_title_case
+        ['ok'].include?(@state) ? @state.upcase : @state.titleize
+      end
+
+      def last_state_title_case
+        ['ok'].include?(@last_state) ? @last_state.upcase : @last_state.titleize
+      end
+
+      def rollup_alerts_by_state
+        ['critical', 'warning', 'unknown'].inject({}) do |memo, state|
+          alerts = rollup_alerts.find_all {|alert| alert[1]['state'] == state}
+          memo[state] = alerts
+          memo
+        end
+      end
+
+      def rollup_state_counts
+        rollup_alerts.inject({}) do |memo, alert|
+          memo[alert[1]['state']] = (memo[alert[1]['state']] || 0) + 1
+          memo
+        end
+      end
+
+      def rollup_states_summary
+        state_counts = rollup_state_counts
+        ['critical', 'warning', 'unknown'].inject([]) do |memo, state|
+          next memo unless rollup_state_counts[state]
+          memo << "#{state.titleize}: #{state_counts[state]}"
+          memo
+        end.join(', ')
+      end
+
+      # produces a textual list of checks that are failing broken down by state, eg:
+      # Critical: 'PING' on 'foo-app-01.example.com', 'SSH' on 'foo-app-01.example.com';
+      #   Warning: 'Disk / Utilisation' on 'foo-app-02.example.com'
+      def rollup_states_detail_text(opts)
+        max_checks = opts[:max_checks_per_state]
+        rollup_alerts_by_state.inject([]) do |memo, state|
+          state_titleized = state[0].titleize
+          alerts = max_checks && max_checks > 0 ? state[1][0..(max_checks - 1)] : state[1]
+          next memo if alerts.empty?
+          checks = alerts.map {|alert| alert[0]}
+          checks << '...' if checks.length < rollup_state_counts[state[0]]
+          memo << "#{state[0].titleize}: #{checks.join(', ')}"
+          memo
+        end.join('; ')
+      end
+
+      def to_s
+        msg = "Alert via #{media}:#{address} to contact #{contact_id} (#{contact_first_name} #{contact_last_name}): "
+        msg += type_sentence_case
+        if rollup
+          msg += " - #{rollup_states_summary} (#{rollup_states_detail_text(:max_checks_per_state => 3)})"
+        else
+          msg += " - '#{check}' on #{entity}"
+          unless ['acknowledgement', 'test'].include?(type)
+            msg += " is #{state_title_case}"
+          end
+          if ['acknowledgement'].include?(type)
+            msg += " has been acknowledged, unscheduled maintenance created for "
+            msg += time_period_in_words(acknowledgement_duration)
+          end
+          if summary && summary.length > 0
+            msg += " - #{summary}"
+          end
+        end
+      end
+
+      def record_send_success!
+        @logger.info "Sent alert successfully: #{to_s}"
+      end
+
+      # TODO: perhaps move message send failure porting to this method
+      # to avoid duplication in the gateways, and to more easily allow
+      # better error reporting on message generation / send failure
+      #def record_send_failure!(opts)
+      #  exception = opts[:exception]
+      #  message   = opts[:message]
+      #  @logger.error "Error sending an alert! #{alert}"
+      #end
+
+    end
+  end
+end