kennel 1.53.0 → 1.56.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 95c82242166781e34d9132c511942a71c45c480951cb4b986702479cca60e271
-  data.tar.gz: 44ab3de5ad5709b6be941bfc8706ae4b712f94a3d1b9ae75a2f028a4e2c42926
+  metadata.gz: 491e174273f8af94504a4e81d9a2ff7da5820aafe287ea9cb1e78df93d07f0f0
+  data.tar.gz: bf918b4be464d408259e0f8204541a0619d1719670a9843a6e564cc4edc968d1
 SHA512:
-  metadata.gz: eb2bf40bccca3072b58c6eb7e7342de8ed51f6a716eba29eb5d717aeb013da18e082fdc0b4565fa666d6a7146d8dd77bc085f343b06d08fb81a285da7d64d520
-  data.tar.gz: be449f1b2c32d1f5996d16d17baa3e215c4446ae2ce4c865bd3cb160baece6bd8f56049cc7e29a56f025339a660ba474b6258fc042d808222a156cc2eb90f8e5
+  metadata.gz: 6ff9000e02764618a7e08c7349f315737778ff7afadf8ca303b7dda8db5648d2bd966412a255e7987ebd2416f8bfd264c3c2979bf143ed0d1f6fdd1a759fc85f
+  data.tar.gz: daf144b7fc436cfb317c4cc298dbbd62d61d252b554261746ed00bcaf5279422bf1a4ab185d45c57974665bc38ebbbf61d0b8e28d8269225c011beb11a451d67

data/Readme.md

@@ -42,33 +42,38 @@ Keep datadog monitors/dashboards/etc in version control, avoid chaotic managemen
  - `gem install bundler && bundle install`
  - `cp .env.example .env`
  - open [Datadog API Settings](https://app.datadoghq.com/account/settings#api)
- - find or create your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=` (will be on the last page if new)
  - copy any `API Key` and add it to `.env` as `DATADOG_API_KEY`
+ - find or create (check last page) your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=`
+ - change the `DATADOG_SUBDOMAIN=app` in `.env` to your companies subdomain if you have one
 -->
 
 ### Adding a team
 
+ - `mention` is used for all team monitors via `super()`
+ - `renotify_interval` is used for all team monitors (defaults to `0` / off)
+ - `tags` is used for all team monitors/dashboards (defaults to `team:<team-name>`)
+
 ```Ruby
 # teams/my_team.rb
 module Teams
   class MyTeam < Kennel::Models::Team
     defaults(
-      slack: -> { "my-alerts" },
-      email: -> { "my-team@example.com" }
+      mention: -> { "@slack-my-team" }
     )
   end
 end
 ```
 
 ### Adding a new monitor
- - use [datadog monitor UI](https://app.datadoghq.com/monitors#create/metric) to create a monitor
- - get the `id` from the url
- - `RESOURCE=monitor ID=12345 bundle exec rake kennel:import`
+ - use [datadog monitor UI](https://app.datadoghq.com/monitors#create) to create a monitor
  - see below
 
 ### Updating an existing monitor
+ - use [datadog monitor UI](https://app.datadoghq.com/monitors/manage) to find a monitor
+ - get the `id` from the url
+ - run `RESOURCE=monitor ID=12345 bundle exec rake kennel:import` and copy the output
  - find or create a project in `projects/`
- - add a monitor to `parts: [` list
+ - add the monitor to `parts: [` list, for example:
   ```Ruby
   # projects/my_project.rb
   class MyProject < Kennel::Models::Project
@@ -83,7 +88,8 @@ end
             kennel_id: -> { "load-too-high" }, # make up a unique name
             name: -> { "Foobar Load too high" }, # nice descriptive name that will show up in alerts and emails
             message: -> {
-              # Explain what behavior to expect and how to fix the cause. Use #{super()} to add team notifications.
+              # Explain what behavior to expect and how to fix the cause
+              # Use #{super()} to add team notifications.
               <<~TEXT
                 Foobar will be slow and that could cause Barfoo to go down.
                 Add capacity or debug why it is suddenly slow.
@@ -98,21 +104,22 @@ end
     )
   end
   ```
- - `bundle exec rake plan` update to existing should be shown (not Create / Delete)
- - alternatively: `bundle exec rake generate` to only update the generated `json` files
+ - run `PROJECT=my_project bundle exec rake plan`, an Update to the existing monitor should be shown (not Create / Delete)
+ - alternatively: `bundle exec rake generate` to only locally update the generated `json` files
  - review changes then `git commit`
  - make a PR ... get reviewed ... merge
  - datadog is updated by travis
 
 ### Adding a new dashboard
  - go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to create a dashboard
- - get the `id` from the url
- - `RESOURCE=dashboard ID=abc-def-ghi bundle exec rake kennel:import`
  - see below
 
 ### Updating an existing dashboard
+ - go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to find a dashboard
+ - get the `id` from the url
+ - run `RESOURCE=dashboard ID=abc-def-ghi bundle exec rake kennel:import` and copy the output
  - find or create a project in `projects/`
- - add a dashboard to `parts: [` list
+ - add a dashboard to `parts: [` list, for example:
   ```Ruby
   class MyProject < Kennel::Models::Project
     defaults(
@@ -154,12 +161,6 @@ end
 Some validations might be too strict for your usecase or just wrong, please [open an issue](https://github.com/grosser/kennel/issues) and
 to unblock use the `validate: -> { false }` option.
 
-### Monitor re-notification
-
-Monitors inherit the re-notification setting from their `project.team`.
-Set this to for example `renotify_interval: -> { 120 }` minutes,
-to make alerts not get ignored by popping back up if they are still alerting.
-
 ### Linking with kennel_ids
 
 To link to existing monitors via their kennel_id
@@ -178,6 +179,10 @@ To link to existing monitors via their kennel_id
 
 Run `rake kennel:alerts TAG=service:my-service` to see all un-muted alerts for a given datadog monitor tag.
 
+### Validating mentions work
+
+`rake kennel:validate_mentions` should run as part of CI
+
 ## Examples
 
 ### Reusable monitors/dashes/etc
@@ -202,7 +207,7 @@ Reuse it in multiple projects.
 ```Ruby
 class Database < Kennel::Models::Project
   defaults(
-    team: -> { Kennel::Models::Team.new(slack: -> { 'foo' }, kennel_id: -> { 'foo' }) },
+    team: -> { Kennel::Models::Team.new(mention: -> { '@slack-foo' }, kennel_id: -> { 'foo' }) },
     parts: -> { [Monitors::LoadTooHigh.new(self, critical: -> { 13 })] }
   )
 end

data/lib/kennel.rb

@@ -10,14 +10,16 @@ require "kennel/syncer"
 require "kennel/api"
 require "kennel/github_reporter"
 require "kennel/subclass_tracking"
+require "kennel/settings_as_methods"
 require "kennel/file_cache"
 require "kennel/template_variables"
 require "kennel/optional_validations"
 require "kennel/unmuted_alerts"
 
 require "kennel/models/base"
+require "kennel/models/record"
 
-# parts
+# records
 require "kennel/models/monitor"
 require "kennel/models/dashboard"
 

data/lib/kennel/importer.rb

@@ -28,7 +28,7 @@ module Kennel
 
       title_field = TITLES.detect { |f| data[f] }
       title = data.fetch(title_field)
-      title.tr!(Kennel::Models::Base::LOCK, "") # avoid double lock icon
+      title.tr!(Kennel::Models::Record::LOCK, "") # avoid double lock icon
 
       # calculate or reuse kennel_id
       # TODO: this is copy-pasted from syncer, need to find a nice way to reuse it
@@ -53,6 +53,8 @@ module Kennel
         if query && critical
           query.sub!(/([><=]) (#{Regexp.escape(critical.to_f.to_s)}|#{Regexp.escape(critical.to_i.to_s)})$/, "\\1 \#{critical}")
         end
+
+        data[:type] = "query alert" if data[:type] == "metric alert"
       elsif resource == "dashboard"
         widgets = data[:widgets]&.flat_map { |widget| widget.dig(:definition, :widgets) || [widget] }
         widgets&.each { |widget| dry_up_query!(widget) }

data/lib/kennel/models/base.rb

@@ -4,115 +4,10 @@ require "hashdiff"
 module Kennel
   module Models
     class Base
-      LOCK = "\u{1F512}"
-      READONLY_ATTRIBUTES = [
-        :deleted, :matching_downtimes, :id, :created, :created_at, :creator, :org_id, :modified,
-        :overall_state_modified, :overall_state, :api_resource
-      ].freeze
-      REQUEST_DEFAULTS = {
-        style: { width: "normal", palette: "dog_classic", type: "solid" },
-        conditional_formats: [],
-        aggregator: "avg"
-      }.freeze
-      OVERRIDABLE_METHODS = [:name, :kennel_id].freeze
+      extend SubclassTracking
+      include SettingsAsMethods
 
-      class ValidationError < RuntimeError
-      end
-
-      class << self
-        include SubclassTracking
-
-        def settings(*names)
-          duplicates = (@set & names)
-          if duplicates.any?
-            raise ArgumentError, "Settings #{duplicates.map(&:inspect).join(", ")} are already defined"
-          end
-
-          overrides = ((instance_methods - OVERRIDABLE_METHODS) & names)
-          if overrides.any?
-            raise ArgumentError, "Settings #{overrides.map(&:inspect).join(", ")} are already used as methods"
-          end
-
-          @set.concat names
-          names.each do |name|
-            next if method_defined?(name)
-            define_method name do
-              message = "Trying to call #{name} for #{self.class} but it was never set or passed as option"
-              raise_with_location ArgumentError, message
-            end
-          end
-        end
-
-        def defaults(options)
-          options.each do |name, block|
-            validate_setting_exists name
-            define_method name, &block
-          end
-        end
-
-        def inherited(child)
-          super
-          child.instance_variable_set(:@set, (@set || []).dup)
-        end
-
-        def validate_setting_exists(name)
-          return if !@set || @set.include?(name)
-          supported = @set.map(&:inspect)
-          raise ArgumentError, "Unsupported setting #{name.inspect}, supported settings are #{supported.join(", ")}"
-        end
-
-        private
-
-        def normalize(_expected, actual)
-          self::READONLY_ATTRIBUTES.each { |k| actual.delete k }
-        end
-
-        # discard styles/conditional_formats/aggregator if nothing would change when we applied (both are default or nil)
-        def ignore_request_defaults(expected, actual, level1, level2)
-          actual = actual[level1] || {}
-          expected = expected[level1] || {}
-          [expected.size.to_i, actual.size.to_i].max.times do |i|
-            a_r = actual.dig(i, level2, :requests) || []
-            e_r = expected.dig(i, level2, :requests) || []
-            ignore_defaults e_r, a_r, self::REQUEST_DEFAULTS
-          end
-        end
-
-        def ignore_defaults(expected, actual, defaults)
-          [expected&.size.to_i, actual&.size.to_i].max.times do |i|
-            e = expected[i] || {}
-            a = actual[i] || {}
-            ignore_default(e, a, defaults)
-          end
-        end
-
-        def ignore_default(expected, actual, defaults)
-          definitions = [actual, expected]
-          defaults.each do |key, default|
-            if definitions.all? { |r| !r.key?(key) || r[key] == default }
-              actual.delete(key)
-              expected.delete(key)
-            end
-          end
-        end
-      end
-
-      def initialize(options = {})
-        validate_options(options)
-
-        options.each do |name, block|
-          self.class.validate_setting_exists name
-          define_singleton_method name, &block
-        end
-
-        # need expand_path so it works wih rake and when run individually
-        pwd = /^#{Regexp.escape(Dir.pwd)}\//
-        @invocation_location = caller.detect do |l|
-          if found = File.expand_path(l).sub!(pwd, "")
-            break found
-          end
-        end
-      end
+      SETTING_OVERRIDABLE_METHODS = [:name, :kennel_id].freeze
 
       def kennel_id
         name = self.class.name
@@ -126,56 +21,9 @@ module Kennel
         self.class.name
       end
 
-      def diff(actual)
-        expected = as_json
-        expected.delete(:id)
-
-        self.class.send(:normalize, expected, actual)
-
-        HashDiff.diff(actual, expected, use_lcs: false)
-      end
-
-      def tracking_id
-        "#{project.kennel_id}:#{kennel_id}"
-      end
-
       def to_json
         raise NotImplementedError, "Use as_json"
       end
-
-      def resolve_linked_tracking_ids(*)
-      end
-
-      private
-
-      def resolve_link(id, id_map, force:)
-        id_map[id] || begin
-          message = "Unable to find #{id} in existing monitors (they need to be created first to link them)"
-          force ? invalid!(message) : Kennel.err.puts(message)
-        end
-      end
-
-      # let users know which project/resource failed when something happens during diffing where the backtrace is hidden
-      def invalid!(message)
-        raise ValidationError, "#{tracking_id} #{message}"
-      end
-
-      def raise_with_location(error, message)
-        message = message.dup
-        message << " for project #{project.kennel_id}" if defined?(project)
-        message << " on #{@invocation_location}" if @invocation_location
-        raise error, message
-      end
-
-      def validate_options(options)
-        unless options.is_a?(Hash)
-          raise ArgumentError, "Expected #{self.class.name}.new options to be a Hash, got a #{options.class}"
-        end
-        options.each do |k, v|
-          next if v.class == Proc
-          raise ArgumentError, "Expected #{self.class.name}.new option :#{k} to be Proc, for example `#{k}: -> { 12 }`"
-        end
-      end
     end
   end
 end

data/lib/kennel/models/dashboard.rb

@@ -1,13 +1,13 @@
 # frozen_string_literal: true
 module Kennel
   module Models
-    class Dashboard < Base
+    class Dashboard < Record
       include TemplateVariables
       include OptionalValidations
 
       API_LIST_INCOMPLETE = true
       DASHBOARD_DEFAULTS = { template_variables: [] }.freeze
-      READONLY_ATTRIBUTES = Base::READONLY_ATTRIBUTES + [
+      READONLY_ATTRIBUTES = superclass::READONLY_ATTRIBUTES + [
         :author_handle, :author_name, :modified_at, :url, :is_read_only, :notify_list
       ]
       REQUEST_DEFAULTS = {
@@ -32,8 +32,6 @@ module Kennel
         def normalize(expected, actual)
           super
 
-          ignore_default expected, actual, DASHBOARD_DEFAULTS
-
           base_pairs(expected, actual).each do |pair|
             # conditional_formats ordering is randomly changed by datadog, compare a stable ordering
             pair.each do |b|
@@ -64,13 +62,6 @@ module Kennel
         end
       end
 
-      attr_reader :project
-
-      def initialize(project, *args)
-        @project = project
-        super(*args)
-      end
-
       def as_json
         return @json if @json
         all_widgets = render_definitions + widgets

data/lib/kennel/models/monitor.rb

@@ -1,14 +1,14 @@
 # frozen_string_literal: true
 module Kennel
   module Models
-    class Monitor < Base
+    class Monitor < Record
       include OptionalValidations
 
       API_LIST_INCOMPLETE = false
       RENOTIFY_INTERVALS = [0, 10, 20, 30, 40, 50, 60, 90, 120, 180, 240, 300, 360, 720, 1440].freeze # minutes
       QUERY_INTERVALS = ["1m", "5m", "10m", "15m", "30m", "1h", "2h", "4h", "1d"].freeze
       OPTIONAL_SERVICE_CHECK_THRESHOLDS = [:ok, :warning].freeze
-      READONLY_ATTRIBUTES = Base::READONLY_ATTRIBUTES + [:multi]
+      READONLY_ATTRIBUTES = superclass::READONLY_ATTRIBUTES + [:multi]
 
       # defaults that datadog uses when options are not sent, so safe to leave out if our values match their defaults
       MONITOR_OPTION_DEFAULTS = {
@@ -27,7 +27,7 @@ module Kennel
       )
 
       defaults(
-        message: -> { "\n\n@slack-#{project.slack}" },
+        message: -> { "\n\n#{project.mention}" },
         escalation_message: -> { DEFAULT_ESCALATION_MESSAGE.first },
         renotify_interval: -> { project.team.renotify_interval },
         warning: -> { nil },
@@ -45,13 +45,6 @@ module Kennel
         threshold_windows: -> { nil }
       )
 
-      attr_reader :project
-
-      def initialize(project, *args)
-        @project = project
-        super(*args)
-      end
-
       def as_json
         return @as_json if @as_json
         data = {

data/lib/kennel/models/project.rb

@@ -2,10 +2,10 @@
 module Kennel
   module Models
     class Project < Base
-      settings :team, :parts, :tags, :slack
+      settings :team, :parts, :tags, :mention
       defaults(
         tags: -> { ["service:#{kennel_id}"] + team.tags },
-        slack: -> { team.slack }
+        mention: -> { team.mention }
       )
 
       def self.file_location

data/lib/kennel/models/record.rb

@@ -0,0 +1,98 @@
+# frozen_string_literal: true
+module Kennel
+  module Models
+    class Record < Base
+      LOCK = "\u{1F512}"
+      READONLY_ATTRIBUTES = [
+        :deleted, :matching_downtimes, :id, :created, :created_at, :creator, :org_id, :modified,
+        :overall_state_modified, :overall_state, :api_resource
+      ].freeze
+      REQUEST_DEFAULTS = {
+        style: { width: "normal", palette: "dog_classic", type: "solid" },
+        conditional_formats: [],
+        aggregator: "avg"
+      }.freeze
+
+      class ValidationError < RuntimeError
+      end
+
+      class << self
+        private
+
+        def normalize(_expected, actual)
+          self::READONLY_ATTRIBUTES.each { |k| actual.delete k }
+        end
+
+        # discard styles/conditional_formats/aggregator if nothing would change when we applied (both are default or nil)
+        def ignore_request_defaults(expected, actual, level1, level2)
+          actual = actual[level1] || {}
+          expected = expected[level1] || {}
+          [expected.size.to_i, actual.size.to_i].max.times do |i|
+            a_r = actual.dig(i, level2, :requests) || []
+            e_r = expected.dig(i, level2, :requests) || []
+            ignore_defaults e_r, a_r, self::REQUEST_DEFAULTS
+          end
+        end
+
+        def ignore_defaults(expected, actual, defaults)
+          [expected&.size.to_i, actual&.size.to_i].max.times do |i|
+            e = expected[i] || {}
+            a = actual[i] || {}
+            ignore_default(e, a, defaults)
+          end
+        end
+
+        def ignore_default(expected, actual, defaults)
+          definitions = [actual, expected]
+          defaults.each do |key, default|
+            if definitions.all? { |r| !r.key?(key) || r[key] == default }
+              actual.delete(key)
+              expected.delete(key)
+            end
+          end
+        end
+      end
+
+      attr_reader :project
+
+      def initialize(project, *args)
+        @project = project
+        super(*args)
+      end
+
+      def diff(actual)
+        expected = as_json
+        expected.delete(:id)
+
+        self.class.send(:normalize, expected, actual)
+
+        HashDiff.diff(actual, expected, use_lcs: false)
+      end
+
+      def tracking_id
+        "#{project.kennel_id}:#{kennel_id}"
+      end
+
+      def resolve_linked_tracking_ids(*)
+      end
+
+      private
+
+      def resolve_link(id, id_map, force:)
+        id_map[id] || begin
+          message = "Unable to find #{id} in existing monitors (they need to be created first to link them)"
+          force ? invalid!(message) : Kennel.err.puts(message)
+        end
+      end
+
+      # let users know which project/resource failed when something happens during diffing where the backtrace is hidden
+      def invalid!(message)
+        raise ValidationError, "#{tracking_id} #{message}"
+      end
+
+      def raise_with_location(error, message)
+        super error, "#{message} for project #{project.kennel_id}"
+      end
+    end
+  end
+end

data/lib/kennel/models/team.rb

@@ -2,20 +2,11 @@
 module Kennel
   module Models
     class Team < Base
-      settings :slack, :email, :tags, :renotify_interval, :kennel_id
+      settings :mention, :tags, :renotify_interval, :kennel_id
       defaults(
         tags: -> { ["team:#{kennel_id.sub(/^teams_/, "")}"] },
         renotify_interval: -> { 0 }
       )
-
-      def initialize(*)
-        super
-        invalid! "remove leading # from slack" if slack.to_s.start_with?("#")
-      end
-
-      def tracking_id
-        kennel_id
-      end
     end
   end
 end

data/lib/kennel/optional_validations.rb

@@ -6,19 +6,10 @@ module Kennel
       base.defaults(validate: -> { true })
     end
 
-    # https://stackoverflow.com/questions/20235206/ruby-get-all-keys-in-a-hash-including-sub-keys/53876255#53876255
-    def self.all_keys(items)
-      case items
-      when Hash then items.keys + items.values.flat_map { |v| all_keys(v) }
-      when Array then items.flat_map { |i| all_keys(i) }
-      else []
-      end
-    end
-
     private
 
     def validate_json(data)
-      bad = OptionalValidations.all_keys(data).grep_v(Symbol)
+      bad = Kennel::Utils.all_keys(data).grep_v(Symbol)
       return if bad.empty?
       invalid!(
         "Only use Symbols as hash keys to avoid permanent diffs when updating.\n" \

data/lib/kennel/settings_as_methods.rb

@@ -0,0 +1,87 @@
+# frozen_string_literal: true
+module Kennel
+  module SettingsAsMethods
+    SETTING_OVERRIDABLE_METHODS = [].freeze
+
+    def self.included(base)
+      base.extend ClassMethods
+      base.instance_variable_set(:@settings, [])
+    end
+
+    module ClassMethods
+      def settings(*names)
+        duplicates = (@settings & names)
+        if duplicates.any?
+          raise ArgumentError, "Settings #{duplicates.map(&:inspect).join(", ")} are already defined"
+        end
+
+        overrides = ((instance_methods - self::SETTING_OVERRIDABLE_METHODS) & names)
+        if overrides.any?
+          raise ArgumentError, "Settings #{overrides.map(&:inspect).join(", ")} are already used as methods"
+        end
+
+        @settings.concat names
+
+        names.each do |name|
+          next if method_defined?(name)
+          define_method name do
+            message = "Trying to call #{name} for #{self.class} but it was never set or passed as option"
+            raise_with_location ArgumentError, message
+          end
+        end
+      end
+
+      def defaults(options)
+        options.each do |name, block|
+          validate_setting_exist name
+          define_method name, &block
+        end
+      end
+
+      private
+
+      def validate_setting_exist(name)
+        return if !@settings || @settings.include?(name)
+        supported = @settings.map(&:inspect)
+        raise ArgumentError, "Unsupported setting #{name.inspect}, supported settings are #{supported.join(", ")}"
+      end
+
+      def inherited(child)
+        super
+        child.instance_variable_set(:@settings, (@settings || []).dup)
+      end
+    end
+
+    def initialize(options = {})
+      super()
+
+      unless options.is_a?(Hash)
+        raise ArgumentError, "Expected #{self.class.name}.new options to be a Hash, got a #{options.class}"
+      end
+
+      options.each do |k, v|
+        next if v.class == Proc
+        raise ArgumentError, "Expected #{self.class.name}.new option :#{k} to be Proc, for example `#{k}: -> { 12 }`"
+      end
+
+      options.each do |name, block|
+        self.class.send :validate_setting_exist, name
+        define_singleton_method name, &block
+      end
+
+      # need expand_path so it works wih rake and when run individually
+      pwd = /^#{Regexp.escape(Dir.pwd)}\//
+      @invocation_location = caller.detect do |l|
+        if found = File.expand_path(l).sub!(pwd, "")
+          break found
+        end
+      end
+    end
+
+    def raise_with_location(error, message)
+      message = message.dup
+      message << " on #{@invocation_location}" if @invocation_location
+      raise error, message
+    end
+  end
+end

data/lib/kennel/syncer.rb

@@ -122,12 +122,7 @@ module Kennel
     end
 
     def download_definitions
-      api_resources = Models::Base.subclasses.map do |m|
-        next unless m.respond_to?(:api_resource)
-        m.api_resource
-      end
-
-      Utils.parallel(api_resources.compact.uniq) do |api_resource|
+      Utils.parallel(Models::Record.subclasses.map(&:api_resource)) do |api_resource|
         results = @api.list(api_resource, with_downtimes: false) # lookup monitors without adding unnecessary downtime information
         results = results[results.keys.first] if results.is_a?(Hash) # dashes/screens are nested in {dash: {}}
         results.each { |c| c[:api_resource] = api_resource } # store api resource for later diffing

data/lib/kennel/tasks.rb

@@ -12,6 +12,36 @@ namespace :kennel do
     abort "Error during diffing" unless $CHILD_STATUS.success?
   end
 
+  # ideally do this on every run, but it's slow (~1.5s) and brittle (might not find all + might find false-positives)
+  # https://help.datadoghq.com/hc/en-us/requests/254114 for automatic validation
+  desc "Verify that all used monitor  mentions are valid"
+  task validate_mentions: :environment do
+    known = Kennel.send(:api)
+      .send(:request, :get, "/monitor/notifications")
+      .fetch(:handles)
+      .values
+      .flatten(1)
+      .map { |v| v.fetch(:value) }
+
+    known += ENV["KNOWN"].to_s.split(",")
+
+    bad = []
+    Dir["generated/**/*.json"].each do |f|
+      next unless message = JSON.parse(File.read(f))["message"]
+      used = message.scan(/\s(@[^\s{,'"]+)/).flatten(1)
+        .grep(/^@.*@|^@.*-/) # ignore @here etc handles ... datadog uses @foo@bar.com for emails and @foo-bar for integrations
+      (used - known).each { |v| bad << [f, v] }
+    end
+
+    if bad.any?
+      subdomain = ENV["DATADOG_SUBDOMAIN"]
+      url = (subdomain ? "https://zendesk.datadoghq.com" : "") + "/account/settings"
+      puts "Invalid mentions found, either ignore them by adding to `KNOWN` env var or add them via #{url}"
+      bad.each { |f, v| puts "Invalid mention #{v} in monitor message of #{f}" }
+      abort
+    end
+  end
+
   desc "generate local definitions"
   task generate: :environment do
     Kennel.generate

data/lib/kennel/utils.rb

@@ -133,6 +133,15 @@ module Kennel
         Kennel.err.puts "Error #{e}, #{times} retries left"
         retry
       end
+
+      # https://stackoverflow.com/questions/20235206/ruby-get-all-keys-in-a-hash-including-sub-keys/53876255#53876255
+      def all_keys(items)
+        case items
+        when Hash then items.keys + items.values.flat_map { |v| all_keys(v) }
+        when Array then items.flat_map { |i| all_keys(i) }
+        else []
+        end
+      end
     end
   end
 end

data/lib/kennel/version.rb

@@ -1,4 +1,4 @@
 # frozen_string_literal: true
 module Kennel
-  VERSION = "1.53.0"
+  VERSION = "1.56.0"
 end

data/template/Readme.md

@@ -25,32 +25,37 @@ Keep datadog monitors/dashboards/etc in version control, avoid chaotic managemen
  - `gem install bundler && bundle install`
  - `cp .env.example .env`
  - open [Datadog API Settings](https://app.datadoghq.com/account/settings#api)
- - find or create your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=` (will be on the last page if new)
  - copy any `API Key` and add it to `.env` as `DATADOG_API_KEY`
+ - find or create (check last page) your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=`
+ - change the `DATADOG_SUBDOMAIN=app` in `.env` to your companies subdomain if you have one
 
 ### Adding a team
 
+ - `mention` is used for all team monitors via `super()`
+ - `renotify_interval` is used for all team monitors (defaults to `0` / off)
+ - `tags` is used for all team monitors/dashboards (defaults to `team:<team-name>`)
+
 ```Ruby
 # teams/my_team.rb
 module Teams
   class MyTeam < Kennel::Models::Team
     defaults(
-      slack: -> { "my-alerts" },
-      email: -> { "my-team@example.com" }
+      mention: -> { "@slack-my-team" }
     )
   end
 end
 ```
 
 ### Adding a new monitor
- - use [datadog monitor UI](https://app.datadoghq.com/monitors#create/metric) to create a monitor
- - get the `id` from the url
- - `RESOURCE=monitor ID=12345 bundle exec rake kennel:import`
+ - use [datadog monitor UI](https://app.datadoghq.com/monitors#create) to create a monitor
  - see below
 
 ### Updating an existing monitor
+ - use [datadog monitor UI](https://app.datadoghq.com/monitors/manage) to find a monitor
+ - get the `id` from the url
+ - run `RESOURCE=monitor ID=12345 bundle exec rake kennel:import` and copy the output
  - find or create a project in `projects/`
- - add a monitor to `parts: [` list
+ - add the monitor to `parts: [` list, for example:
   ```Ruby
   # projects/my_project.rb
   class MyProject < Kennel::Models::Project
@@ -65,7 +70,8 @@ end
             kennel_id: -> { "load-too-high" }, # make up a unique name
             name: -> { "Foobar Load too high" }, # nice descriptive name that will show up in alerts and emails
             message: -> {
-              # Explain what behavior to expect and how to fix the cause. Use #{super()} to add team notifications.
+              # Explain what behavior to expect and how to fix the cause
+              # Use #{super()} to add team notifications.
               <<~TEXT
                 Foobar will be slow and that could cause Barfoo to go down.
                 Add capacity or debug why it is suddenly slow.
@@ -80,21 +86,22 @@ end
     )
   end
   ```
- - `bundle exec rake plan` update to existing should be shown (not Create / Delete)
- - alternatively: `bundle exec rake generate` to only update the generated `json` files
+ - run `PROJECT=my_project bundle exec rake plan`, an Update to the existing monitor should be shown (not Create / Delete)
+ - alternatively: `bundle exec rake generate` to only locally update the generated `json` files
  - review changes then `git commit`
  - make a PR ... get reviewed ... merge
  - datadog is updated by travis
 
 ### Adding a new dashboard
  - go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to create a dashboard
- - get the `id` from the url
- - `RESOURCE=dashboard ID=abc-def-ghi bundle exec rake kennel:import`
  - see below
 
 ### Updating an existing dashboard
+ - go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to find a dashboard
+ - get the `id` from the url
+ - run `RESOURCE=dashboard ID=abc-def-ghi bundle exec rake kennel:import` and copy the output
  - find or create a project in `projects/`
- - add a dashboard to `parts: [` list
+ - add a dashboard to `parts: [` list, for example:
   ```Ruby
   class MyProject < Kennel::Models::Project
     defaults(
@@ -136,11 +143,6 @@ end
 Some validations might be too strict for your usecase or just wrong, please [open an issue](https://github.com/grosser/kennel/issues) and
 to unblock use the `validate: -> { false }` option.
 
-### Monitor re-notification
-
-Monitors inherit the re-notification setting from their projects team.
-By default this is `renotify_interval: -> { 120 }` minutes, which will make alerts not get ignored by popping back up.
-
 ### Linking with kennel_ids
 
 To link to existing monitors via their kennel_id
@@ -159,6 +161,10 @@ To link to existing monitors via their kennel_id
 
 Run `rake kennel:alerts TAG=service:my-service` to see all un-muted alerts for a given datadog monitor tag.
 
+### Validating mentions work
+
+`rake kennel:validate_mentions` should run as part of CI
+
 ## Examples
 
 ### Reusable monitors/dashes/etc
@@ -183,7 +189,7 @@ Reuse it in multiple projects.
 ```Ruby
 class Database < Kennel::Models::Project
   defaults(
-    team: -> { Kennel::Models::Team.new(slack: -> { 'foo' }, kennel_id: -> { 'foo' }) },
+    team: -> { Kennel::Models::Team.new(mention: -> { '@slack-foo' }, kennel_id: -> { 'foo' }) },
     parts: -> { [Monitors::LoadTooHigh.new(self, critical: -> { 13 })] }
   )
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: kennel
 version: !ruby/object:Gem::Version
-  version: 1.53.0
+  version: 1.56.0
 platform: ruby
 authors:
 - Michael Grosser
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-09-12 00:00:00.000000000 Z
+date: 2019-10-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: faraday
@@ -68,9 +68,11 @@ files:
 - lib/kennel/models/dashboard.rb
 - lib/kennel/models/monitor.rb
 - lib/kennel/models/project.rb
+- lib/kennel/models/record.rb
 - lib/kennel/models/team.rb
 - lib/kennel/optional_validations.rb
 - lib/kennel/progress.rb
+- lib/kennel/settings_as_methods.rb
 - lib/kennel/subclass_tracking.rb
 - lib/kennel/syncer.rb
 - lib/kennel/tasks.rb