kennel 1.50.0 → 1.53.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Readme.md +23 -12
- data/lib/kennel/importer.rb +18 -0
- data/lib/kennel/models/base.rb +7 -0
- data/lib/kennel/models/dashboard.rb +10 -14
- data/lib/kennel/models/monitor.rb +12 -2
- data/lib/kennel/models/team.rb +3 -2
- data/lib/kennel/syncer.rb +14 -3
- data/lib/kennel/version.rb +1 -1
- data/template/Readme.md +23 -12
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 6a39dd7d2e69ff27625b450457e54c3e57ff2e0950e10f239f6b14d28dc0826b
|
4
|
+
data.tar.gz: 4effd0e4f0428a9e065e7b1e272115356aa352749a7dc11b1aaadf657a6cd81e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: '0930a5440d6c4d7ced1f838d8c5489f8899064b4a7ba8b12dba2a3c7daea71eab51e797667ee8ede2e496200316fe1244f230ed95f9aa12bbcc1ceb5d361ecad'
|
7
|
+
data.tar.gz: 56e685b04a6fb73cfb273d6af73fab31f8daf901266acd7934892fff2dc17113c99cf0f4b70e5d9cca9f4f208fce36bc98f6702b3d027fdb059fe058c40d78d0
|
data/Readme.md
CHANGED
@@ -42,8 +42,9 @@ Keep datadog monitors/dashboards/etc in version control, avoid chaotic managemen
|
|
42
42
|
- `gem install bundler && bundle install`
|
43
43
|
- `cp .env.example .env`
|
44
44
|
- open [Datadog API Settings](https://app.datadoghq.com/account/settings#api)
|
45
|
-
- find or create your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=` (will be on the last page if new)
|
46
45
|
- copy any `API Key` and add it to `.env` as `DATADOG_API_KEY`
|
46
|
+
- find or create your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=` (will be on the last page if new)
|
47
|
+
- change the `DATADOG_SUBDOMAIN=app` in `.env` to your companies subdomain if you have one
|
47
48
|
-->
|
48
49
|
|
49
50
|
### Adding a team
|
@@ -53,8 +54,8 @@ Keep datadog monitors/dashboards/etc in version control, avoid chaotic managemen
|
|
53
54
|
module Teams
|
54
55
|
class MyTeam < Kennel::Models::Team
|
55
56
|
defaults(
|
56
|
-
slack: -> { "my-alerts" },
|
57
|
-
email: -> { "my-team@example.com" }
|
57
|
+
slack: -> { "my-alerts" }, # where to send alerts
|
58
|
+
email: -> { "my-team@example.com" } # optional
|
58
59
|
)
|
59
60
|
end
|
60
61
|
end
|
@@ -62,13 +63,14 @@ end
|
|
62
63
|
|
63
64
|
### Adding a new monitor
|
64
65
|
- use [datadog monitor UI](https://app.datadoghq.com/monitors#create/metric) to create a monitor
|
65
|
-
- get the `id` from the url
|
66
|
-
- `RESOURCE=monitor ID=12345 bundle exec rake kennel:import`
|
67
66
|
- see below
|
68
67
|
|
69
68
|
### Updating an existing monitor
|
69
|
+
- use [datadog monitor UI](https://app.datadoghq.com/monitors#create/metric) to find a monitor
|
70
|
+
- get the `id` from the url
|
71
|
+
- run `RESOURCE=monitor ID=12345 bundle exec rake kennel:import` and copy the output
|
70
72
|
- find or create a project in `projects/`
|
71
|
-
- add
|
73
|
+
- add the monitor to `parts: [` list, for example:
|
72
74
|
```Ruby
|
73
75
|
# projects/my_project.rb
|
74
76
|
class MyProject < Kennel::Models::Project
|
@@ -83,7 +85,8 @@ end
|
|
83
85
|
kennel_id: -> { "load-too-high" }, # make up a unique name
|
84
86
|
name: -> { "Foobar Load too high" }, # nice descriptive name that will show up in alerts and emails
|
85
87
|
message: -> {
|
86
|
-
# Explain what behavior to expect and how to fix the cause
|
88
|
+
# Explain what behavior to expect and how to fix the cause
|
89
|
+
# Use #{super()} to add team notifications.
|
87
90
|
<<~TEXT
|
88
91
|
Foobar will be slow and that could cause Barfoo to go down.
|
89
92
|
Add capacity or debug why it is suddenly slow.
|
@@ -98,21 +101,22 @@ end
|
|
98
101
|
)
|
99
102
|
end
|
100
103
|
```
|
101
|
-
- `bundle exec rake plan
|
102
|
-
- alternatively: `bundle exec rake generate` to only update the generated `json` files
|
104
|
+
- run `PROJECT=my_project bundle exec rake plan`, an Update to the existing monitor should be shown (not Create / Delete)
|
105
|
+
- alternatively: `bundle exec rake generate` to only locally update the generated `json` files
|
103
106
|
- review changes then `git commit`
|
104
107
|
- make a PR ... get reviewed ... merge
|
105
108
|
- datadog is updated by travis
|
106
109
|
|
107
110
|
### Adding a new dashboard
|
108
111
|
- go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to create a dashboard
|
109
|
-
- get the `id` from the url
|
110
|
-
- `RESOURCE=dashboard ID=abc-def-ghi bundle exec rake kennel:import`
|
111
112
|
- see below
|
112
113
|
|
113
114
|
### Updating an existing dashboard
|
115
|
+
- go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to find a dashboard
|
116
|
+
- get the `id` from the url
|
117
|
+
- run `RESOURCE=dashboard ID=abc-def-ghi bundle exec rake kennel:import` and copy the output
|
114
118
|
- find or create a project in `projects/`
|
115
|
-
- add a dashboard to `parts: [` list
|
119
|
+
- add a dashboard to `parts: [` list, for example:
|
116
120
|
```Ruby
|
117
121
|
class MyProject < Kennel::Models::Project
|
118
122
|
defaults(
|
@@ -154,12 +158,19 @@ end
|
|
154
158
|
Some validations might be too strict for your usecase or just wrong, please [open an issue](https://github.com/grosser/kennel/issues) and
|
155
159
|
to unblock use the `validate: -> { false }` option.
|
156
160
|
|
161
|
+
### Monitor re-notification
|
162
|
+
|
163
|
+
Monitors inherit the re-notification setting from their `project.team`.
|
164
|
+
Set this to for example `renotify_interval: -> { 120 }` minutes,
|
165
|
+
to make alerts not get ignored by popping back up if they are still alerting.
|
166
|
+
|
157
167
|
### Linking with kennel_ids
|
158
168
|
|
159
169
|
To link to existing monitors via their kennel_id
|
160
170
|
|
161
171
|
- Screens `uptime` widgets can use `monitor: {id: "foo:bar"}`
|
162
172
|
- Screens `alert_graph` widgets can use `alert_id: "foo:bar"`
|
173
|
+
- Monitors `composite` can use `query: -> { "%{foo:bar} || %{foo:baz}" }`
|
163
174
|
|
164
175
|
### Debugging changes locally
|
165
176
|
|
data/lib/kennel/importer.rb
CHANGED
@@ -53,6 +53,9 @@ module Kennel
|
|
53
53
|
if query && critical
|
54
54
|
query.sub!(/([><=]) (#{Regexp.escape(critical.to_f.to_s)}|#{Regexp.escape(critical.to_i.to_s)})$/, "\\1 \#{critical}")
|
55
55
|
end
|
56
|
+
elsif resource == "dashboard"
|
57
|
+
widgets = data[:widgets]&.flat_map { |widget| widget.dig(:definition, :widgets) || [widget] }
|
58
|
+
widgets&.each { |widget| dry_up_query!(widget) }
|
56
59
|
end
|
57
60
|
|
58
61
|
# simplify template_variables to array of string when possible
|
@@ -71,6 +74,20 @@ module Kennel
|
|
71
74
|
|
72
75
|
private
|
73
76
|
|
77
|
+
# reduce duplication in imports by using dry `q: :metadata` when possible
|
78
|
+
def dry_up_query!(widget)
|
79
|
+
(widget.dig(:definition, :requests) || []).each do |request|
|
80
|
+
next unless request.is_a?(Hash)
|
81
|
+
next unless metadata = request[:metadata]
|
82
|
+
next unless query = request[:q]&.dup
|
83
|
+
metadata.each do |m|
|
84
|
+
next unless exp = m[:expression]
|
85
|
+
query.sub!(exp, "")
|
86
|
+
end
|
87
|
+
request[:q] = :metadata if query.delete(", ") == ""
|
88
|
+
end
|
89
|
+
end
|
90
|
+
|
74
91
|
def pretty_print(hash)
|
75
92
|
sort_widgets hash
|
76
93
|
|
@@ -84,6 +101,7 @@ module Kennel
|
|
84
101
|
.gsub(/(^\s*)"([a-zA-Z][a-zA-Z\d_]*)":/, "\\1\\2:") # "foo": 1 -> foo: 1
|
85
102
|
.gsub(/: \[\n\s+\]/, ": []") # empty arrays on a single line
|
86
103
|
.gsub(/^/, " ") # indent
|
104
|
+
.gsub('q: "metadata"', "q: :metadata") # bring symbols back
|
87
105
|
|
88
106
|
"\n#{pretty}\n "
|
89
107
|
elsif k == :message
|
data/lib/kennel/models/base.rb
CHANGED
@@ -148,6 +148,13 @@ module Kennel
|
|
148
148
|
|
149
149
|
private
|
150
150
|
|
151
|
+
def resolve_link(id, id_map, force:)
|
152
|
+
id_map[id] || begin
|
153
|
+
message = "Unable to find #{id} in existing monitors (they need to be created first to link them)"
|
154
|
+
force ? invalid!(message) : Kennel.err.puts(message)
|
155
|
+
end
|
156
|
+
end
|
157
|
+
|
151
158
|
# let users know which project/resource failed when something happens during diffing where the backtrace is hidden
|
152
159
|
def invalid!(message)
|
153
160
|
raise ValidationError, "#{tracking_id} #{message}"
|
@@ -39,7 +39,7 @@ module Kennel
|
|
39
39
|
pair.each do |b|
|
40
40
|
b[:widgets]&.each do |w|
|
41
41
|
if formats = w.dig(:definition, :conditional_formats)
|
42
|
-
w[:definition][:conditional_formats] = formats.sort_by
|
42
|
+
w[:definition][:conditional_formats] = formats.sort_by(&:hash)
|
43
43
|
end
|
44
44
|
end
|
45
45
|
end
|
@@ -101,11 +101,13 @@ module Kennel
|
|
101
101
|
case definition[:type]
|
102
102
|
when "uptime"
|
103
103
|
if ids = definition[:monitor_ids]
|
104
|
-
definition[:monitor_ids] = ids.map
|
104
|
+
definition[:monitor_ids] = ids.map do |id|
|
105
|
+
tracking_id?(id) ? resolve_link(id, id_map, force: false) : id
|
106
|
+
end
|
105
107
|
end
|
106
108
|
when "alert_graph"
|
107
|
-
if id = definition[:alert_id]
|
108
|
-
definition[:alert_id] = resolve_link(id, id_map).to_s
|
109
|
+
if (id = definition[:alert_id]) && tracking_id?(id)
|
110
|
+
definition[:alert_id] = resolve_link(id, id_map, force: false).to_s
|
109
111
|
end
|
110
112
|
end
|
111
113
|
end
|
@@ -113,6 +115,10 @@ module Kennel
|
|
113
115
|
|
114
116
|
private
|
115
117
|
|
118
|
+
def tracking_id?(id)
|
119
|
+
id.is_a?(String) && !id.match?(/\A\d+\z/)
|
120
|
+
end
|
121
|
+
|
116
122
|
# creates queries from metadata to avoid having to keep q and expression in sync
|
117
123
|
#
|
118
124
|
# {q: :metadata, metadata: [{expression: "sum:bar", alias_name: "foo"}, ...], }
|
@@ -127,16 +133,6 @@ module Kennel
|
|
127
133
|
end
|
128
134
|
end
|
129
135
|
|
130
|
-
def resolve_link(id, id_map)
|
131
|
-
return id unless tracking_id?(id)
|
132
|
-
id_map[id] ||
|
133
|
-
Kennel.err.puts("Unable to find #{id} in existing monitors (they need to be created first to link them)")
|
134
|
-
end
|
135
|
-
|
136
|
-
def tracking_id?(id)
|
137
|
-
id.is_a?(String) && !id.match?(/\A\d+\z/)
|
138
|
-
end
|
139
|
-
|
140
136
|
def validate_json(data)
|
141
137
|
super
|
142
138
|
|
@@ -15,7 +15,7 @@ module Kennel
|
|
15
15
|
evaluation_delay: nil,
|
16
16
|
new_host_delay: 300,
|
17
17
|
timeout_h: 0,
|
18
|
-
renotify_interval:
|
18
|
+
renotify_interval: 0,
|
19
19
|
no_data_timeframe: nil # this works out ok since if notify_no_data is on, it would never be nil
|
20
20
|
}.freeze
|
21
21
|
DEFAULT_ESCALATION_MESSAGE = ["", nil].freeze
|
@@ -29,7 +29,7 @@ module Kennel
|
|
29
29
|
defaults(
|
30
30
|
message: -> { "\n\n@slack-#{project.slack}" },
|
31
31
|
escalation_message: -> { DEFAULT_ESCALATION_MESSAGE.first },
|
32
|
-
renotify_interval: -> {
|
32
|
+
renotify_interval: -> { project.team.renotify_interval },
|
33
33
|
warning: -> { nil },
|
34
34
|
ok: -> { nil },
|
35
35
|
id: -> { nil },
|
@@ -108,6 +108,16 @@ module Kennel
|
|
108
108
|
@as_json = data
|
109
109
|
end
|
110
110
|
|
111
|
+
# resolve composite monitors ... only works when referenced monitors already exist
|
112
|
+
# since leaving names or bad ids in the query breaks the monitor update
|
113
|
+
def resolve_linked_tracking_ids(id_map)
|
114
|
+
if as_json[:type] == "composite"
|
115
|
+
as_json[:query] = as_json[:query].gsub(/%\{(.*?)\}/) do
|
116
|
+
resolve_link($1, id_map, force: true)
|
117
|
+
end
|
118
|
+
end
|
119
|
+
end
|
120
|
+
|
111
121
|
def self.api_resource
|
112
122
|
"monitor"
|
113
123
|
end
|
data/lib/kennel/models/team.rb
CHANGED
@@ -2,9 +2,10 @@
|
|
2
2
|
module Kennel
|
3
3
|
module Models
|
4
4
|
class Team < Base
|
5
|
-
settings :slack, :email, :tags, :kennel_id
|
5
|
+
settings :slack, :email, :tags, :renotify_interval, :kennel_id
|
6
6
|
defaults(
|
7
|
-
tags: -> { ["team:#{kennel_id.sub(/^teams_/, "")}"] }
|
7
|
+
tags: -> { ["team:#{kennel_id.sub(/^teams_/, "")}"] },
|
8
|
+
renotify_interval: -> { 0 }
|
8
9
|
)
|
9
10
|
|
10
11
|
def initialize(*)
|
data/lib/kennel/syncer.rb
CHANGED
@@ -72,11 +72,22 @@ module Kennel
|
|
72
72
|
filter_by_project! actual
|
73
73
|
|
74
74
|
details_cache do |cache|
|
75
|
-
actual.
|
76
|
-
id = a.fetch(:id)
|
75
|
+
items = actual.map do |a|
|
77
76
|
e = matching_expected(a)
|
78
77
|
if e && @expected.delete(e)
|
79
|
-
|
78
|
+
[e, a]
|
79
|
+
else
|
80
|
+
[nil, a]
|
81
|
+
end
|
82
|
+
end
|
83
|
+
|
84
|
+
# fill details of things we need to compare (only do this part in parallel for safety & balancing)
|
85
|
+
Utils.parallel(items.select { |e, _| e && e.class::API_LIST_INCOMPLETE }) { |_, a| fill_details(a, cache) }
|
86
|
+
|
87
|
+
# pick out things to update or delete
|
88
|
+
items.each do |e, a|
|
89
|
+
id = a.fetch(:id)
|
90
|
+
if e
|
80
91
|
diff = e.diff(a)
|
81
92
|
@update << [id, e, a, diff] if diff.any?
|
82
93
|
elsif tracking_id(a) # was previously managed
|
data/lib/kennel/version.rb
CHANGED
data/template/Readme.md
CHANGED
@@ -25,8 +25,9 @@ Keep datadog monitors/dashboards/etc in version control, avoid chaotic managemen
|
|
25
25
|
- `gem install bundler && bundle install`
|
26
26
|
- `cp .env.example .env`
|
27
27
|
- open [Datadog API Settings](https://app.datadoghq.com/account/settings#api)
|
28
|
-
- find or create your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=` (will be on the last page if new)
|
29
28
|
- copy any `API Key` and add it to `.env` as `DATADOG_API_KEY`
|
29
|
+
- find or create your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=` (will be on the last page if new)
|
30
|
+
- change the `DATADOG_SUBDOMAIN=app` in `.env` to your companies subdomain if you have one
|
30
31
|
|
31
32
|
### Adding a team
|
32
33
|
|
@@ -35,8 +36,8 @@ Keep datadog monitors/dashboards/etc in version control, avoid chaotic managemen
|
|
35
36
|
module Teams
|
36
37
|
class MyTeam < Kennel::Models::Team
|
37
38
|
defaults(
|
38
|
-
slack: -> { "my-alerts" },
|
39
|
-
email: -> { "my-team@example.com" }
|
39
|
+
slack: -> { "my-alerts" }, # where to send alerts
|
40
|
+
email: -> { "my-team@example.com" } # optional
|
40
41
|
)
|
41
42
|
end
|
42
43
|
end
|
@@ -44,13 +45,14 @@ end
|
|
44
45
|
|
45
46
|
### Adding a new monitor
|
46
47
|
- use [datadog monitor UI](https://app.datadoghq.com/monitors#create/metric) to create a monitor
|
47
|
-
- get the `id` from the url
|
48
|
-
- `RESOURCE=monitor ID=12345 bundle exec rake kennel:import`
|
49
48
|
- see below
|
50
49
|
|
51
50
|
### Updating an existing monitor
|
51
|
+
- use [datadog monitor UI](https://app.datadoghq.com/monitors#create/metric) to find a monitor
|
52
|
+
- get the `id` from the url
|
53
|
+
- run `RESOURCE=monitor ID=12345 bundle exec rake kennel:import` and copy the output
|
52
54
|
- find or create a project in `projects/`
|
53
|
-
- add
|
55
|
+
- add the monitor to `parts: [` list, for example:
|
54
56
|
```Ruby
|
55
57
|
# projects/my_project.rb
|
56
58
|
class MyProject < Kennel::Models::Project
|
@@ -65,7 +67,8 @@ end
|
|
65
67
|
kennel_id: -> { "load-too-high" }, # make up a unique name
|
66
68
|
name: -> { "Foobar Load too high" }, # nice descriptive name that will show up in alerts and emails
|
67
69
|
message: -> {
|
68
|
-
# Explain what behavior to expect and how to fix the cause
|
70
|
+
# Explain what behavior to expect and how to fix the cause
|
71
|
+
# Use #{super()} to add team notifications.
|
69
72
|
<<~TEXT
|
70
73
|
Foobar will be slow and that could cause Barfoo to go down.
|
71
74
|
Add capacity or debug why it is suddenly slow.
|
@@ -80,21 +83,22 @@ end
|
|
80
83
|
)
|
81
84
|
end
|
82
85
|
```
|
83
|
-
- `bundle exec rake plan
|
84
|
-
- alternatively: `bundle exec rake generate` to only update the generated `json` files
|
86
|
+
- run `PROJECT=my_project bundle exec rake plan`, an Update to the existing monitor should be shown (not Create / Delete)
|
87
|
+
- alternatively: `bundle exec rake generate` to only locally update the generated `json` files
|
85
88
|
- review changes then `git commit`
|
86
89
|
- make a PR ... get reviewed ... merge
|
87
90
|
- datadog is updated by travis
|
88
91
|
|
89
92
|
### Adding a new dashboard
|
90
93
|
- go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to create a dashboard
|
91
|
-
- get the `id` from the url
|
92
|
-
- `RESOURCE=dashboard ID=abc-def-ghi bundle exec rake kennel:import`
|
93
94
|
- see below
|
94
95
|
|
95
96
|
### Updating an existing dashboard
|
97
|
+
- go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to find a dashboard
|
98
|
+
- get the `id` from the url
|
99
|
+
- run `RESOURCE=dashboard ID=abc-def-ghi bundle exec rake kennel:import` and copy the output
|
96
100
|
- find or create a project in `projects/`
|
97
|
-
- add a dashboard to `parts: [` list
|
101
|
+
- add a dashboard to `parts: [` list, for example:
|
98
102
|
```Ruby
|
99
103
|
class MyProject < Kennel::Models::Project
|
100
104
|
defaults(
|
@@ -136,12 +140,19 @@ end
|
|
136
140
|
Some validations might be too strict for your usecase or just wrong, please [open an issue](https://github.com/grosser/kennel/issues) and
|
137
141
|
to unblock use the `validate: -> { false }` option.
|
138
142
|
|
143
|
+
### Monitor re-notification
|
144
|
+
|
145
|
+
Monitors inherit the re-notification setting from their `project.team`.
|
146
|
+
Set this to for example `renotify_interval: -> { 120 }` minutes,
|
147
|
+
to make alerts not get ignored by popping back up if they are still alerting.
|
148
|
+
|
139
149
|
### Linking with kennel_ids
|
140
150
|
|
141
151
|
To link to existing monitors via their kennel_id
|
142
152
|
|
143
153
|
- Screens `uptime` widgets can use `monitor: {id: "foo:bar"}`
|
144
154
|
- Screens `alert_graph` widgets can use `alert_id: "foo:bar"`
|
155
|
+
- Monitors `composite` can use `query: -> { "%{foo:bar} || %{foo:baz}" }`
|
145
156
|
|
146
157
|
### Debugging changes locally
|
147
158
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: kennel
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.53.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Michael Grosser
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-
|
11
|
+
date: 2019-09-12 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: faraday
|