kennel 1.74.0 → 1.76.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Readme.md +76 -30
- data/lib/kennel/github_reporter.rb +4 -2
- data/lib/kennel/importer.rb +9 -2
- data/lib/kennel/models/monitor.rb +6 -4
- data/lib/kennel/tasks.rb +17 -8
- data/lib/kennel/version.rb +1 -1
- data/template/Readme.md +70 -27
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 950dfe31db52b682fe8158995404f60035aeee8be018884db0a9f8bb33900ce3
|
4
|
+
data.tar.gz: 17300f8517cdc04eee3f8a037e77e6f9b3a15f1646b09e2a1f2250a6e565fa85
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 1356476dbc82439b5fb79f6041e3bf3fdb43b1696fb21da903fb25f9e43c6c33d4cdfb25e30042029f5751744bd0a8da43e8f330d62a3bd033f3cddb07a0ccb2
|
7
|
+
data.tar.gz: e6a7666696af4c04938f79d824442a4315efaf591d8a57be32ae065b528ce09a4fe65aad02650cce5ee422526a57713e3c8f887097964eb3c2e4f3bf9b042f94
|
data/Readme.md
CHANGED
@@ -1,17 +1,58 @@
|
|
1
|
-
# Kennel
|
2
|
-
|
3
1
|
![](template/github/cage.jpg?raw=true)
|
4
2
|
|
5
|
-
Manage
|
3
|
+
Manage Datadog Monitors / Dashboards / Slos as code
|
6
4
|
|
7
|
-
-
|
8
|
-
- Changes are PR reviewed and
|
5
|
+
- DRY, searchable, audited, documented
|
6
|
+
- Changes are PR reviewed and applied on merge
|
9
7
|
- Updating shows diff before applying
|
10
|
-
- Automated import of existing
|
8
|
+
- Automated import of existing resources
|
9
|
+
- Resources are grouped into projects that belong to teams and inherit tags
|
10
|
+
- No copy-pasting of ids to create new resources
|
11
|
+
- Automated cleanup when removing code
|
12
|
+
- [Helpers](#helpers) for automating common tasks
|
13
|
+
|
14
|
+
### Applying changes
|
11
15
|
|
12
16
|
![](template/github/screen.png?raw=true)
|
17
|
+
|
18
|
+
### Example code
|
19
|
+
|
20
|
+
```Ruby
|
21
|
+
# teams/foo.rb
|
22
|
+
module Teams
|
23
|
+
class Foo < Kennel::Models::Team
|
24
|
+
defaults(mention: -> { "@slack-my-team" })
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
# projects/bar.rb
|
29
|
+
class Bar < Kennel::Models::Project
|
30
|
+
defaults(
|
31
|
+
team: -> { Teams::Foo.new }, # use mention and tags from the team
|
32
|
+
parts: -> {
|
33
|
+
[
|
34
|
+
Kennel::Models::Monitor.new(
|
35
|
+
self, # the current project
|
36
|
+
type: -> { "query alert" },
|
37
|
+
kennel_id: -> { "load-too-high" }, # pick a unique name
|
38
|
+
name: -> { "Foobar Load too high" }, # nice descriptive name that will show up in alerts and emails
|
39
|
+
message: -> {
|
40
|
+
<<~TEXT
|
41
|
+
This is bad!
|
42
|
+
#{super()} # inserts mention from team
|
43
|
+
TEXT
|
44
|
+
},
|
45
|
+
query: -> { "avg(last_5m):avg:system.load.5{hostgroup:api} by {pod} > #{critical}" },
|
46
|
+
critical: -> { 20 }
|
47
|
+
)
|
48
|
+
]
|
49
|
+
}
|
50
|
+
)
|
51
|
+
end
|
52
|
+
```
|
53
|
+
|
13
54
|
<!-- NOT IN template/Readme.md -->
|
14
|
-
##
|
55
|
+
## Installation
|
15
56
|
|
16
57
|
- create a new private `kennel` repo for your organization (do not fork this repo)
|
17
58
|
- use the template folder as starting point:
|
@@ -22,8 +63,8 @@ Manage datadog monitors/dashboards/slos as code
|
|
22
63
|
cd kennel && git add . && git commit -m 'initial'
|
23
64
|
```
|
24
65
|
- add a basic projects and teams so others can copy-paste to get started
|
25
|
-
- setup
|
26
|
-
- uncomment `.travis.yml` section for
|
66
|
+
- setup CI build for your repo (travis and Github Actions supported)
|
67
|
+
- uncomment `.travis.yml` section for datadog updates on merge (TODO: example setup for Github Actions)
|
27
68
|
- follow `Setup` in your repos Readme.md
|
28
69
|
<!-- NOT IN -->
|
29
70
|
|
@@ -109,7 +150,7 @@ end
|
|
109
150
|
- alternatively: `bundle exec rake generate` to only locally update the generated `json` files
|
110
151
|
- review changes then `git commit`
|
111
152
|
- make a PR ... get reviewed ... merge
|
112
|
-
- datadog is updated by
|
153
|
+
- datadog is updated by CI
|
113
154
|
|
114
155
|
### Adding a new dashboard
|
115
156
|
- go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to create a dashboard
|
@@ -164,7 +205,7 @@ to unblock use the `validate: -> { false }` option.
|
|
164
205
|
|
165
206
|
### Linking with kennel_ids
|
166
207
|
|
167
|
-
To link to existing monitors via their kennel_id
|
208
|
+
To link to existing monitors via their kennel_id `projects kennel_id` + `:` + `monitors kennel id`
|
168
209
|
|
169
210
|
- Screens `uptime` widgets can use `monitor: {id: "foo:bar"}`
|
170
211
|
- Screens `alert_graph` widgets can use `alert_id: "foo:bar"`
|
@@ -176,25 +217,7 @@ To link to existing monitors via their kennel_id
|
|
176
217
|
- figure out project name by converting the class name to snake-case
|
177
218
|
- run `PROJECT=foo bundle exec rake kennel:update_datadog` to test changes for a single project
|
178
219
|
|
179
|
-
###
|
180
|
-
|
181
|
-
Run `rake kennel:alerts TAG=service:my-service` to see all un-muted alerts for a given datadog monitor tag.
|
182
|
-
|
183
|
-
### Validating mentions work
|
184
|
-
|
185
|
-
`rake kennel:validate_mentions` should run as part of CI
|
186
|
-
|
187
|
-
### Grepping through all of datadog
|
188
|
-
|
189
|
-
`TYPE=monitor rake kennel:dump`
|
190
|
-
|
191
|
-
### Find all monitors with No-Data
|
192
|
-
|
193
|
-
`rake kennel:nodata TAG=team:foo`
|
194
|
-
|
195
|
-
## Examples
|
196
|
-
|
197
|
-
### Reusable monitors/dashes/etc
|
220
|
+
### Reuse
|
198
221
|
|
199
222
|
Add to `parts/<folder>`.
|
200
223
|
|
@@ -221,8 +244,31 @@ class Database < Kennel::Models::Project
|
|
221
244
|
)
|
222
245
|
end
|
223
246
|
```
|
247
|
+
|
248
|
+
## Helpers
|
249
|
+
|
250
|
+
### Listing un-muted alerts
|
251
|
+
|
252
|
+
Run `rake kennel:alerts TAG=service:my-service` to see all un-muted alerts for a given datadog monitor tag.
|
253
|
+
|
254
|
+
### Validating mentions work
|
255
|
+
|
256
|
+
`rake kennel:validate_mentions` should run as part of CI
|
257
|
+
|
258
|
+
### Grepping through all of datadog
|
259
|
+
|
260
|
+
`rake kennel:dump`
|
261
|
+
focus on a single type: `TYPE=monitors`
|
262
|
+
|
263
|
+
### Find all monitors with No-Data
|
264
|
+
|
265
|
+
`rake kennel:nodata TAG=team:foo`
|
266
|
+
|
224
267
|
<!-- NOT IN template/Readme.md -->
|
225
268
|
|
269
|
+
|
270
|
+
## Development
|
271
|
+
|
226
272
|
### Integration testing
|
227
273
|
|
228
274
|
```Bash
|
@@ -15,8 +15,10 @@ module Kennel
|
|
15
15
|
def initialize(token, git_sha)
|
16
16
|
@token = token
|
17
17
|
@git_sha = git_sha
|
18
|
-
|
19
|
-
|
18
|
+
@repo_part = ENV["GITHUB_REPOSITORY"] || begin
|
19
|
+
origin = ENV["PROJECT_REPOSITORY"] || Utils.capture_sh("git remote -v").split("\n").first
|
20
|
+
origin[%r{github\.com[:/](\S+?)(\.git|$)}, 1] || raise("no origin found in #{origin}")
|
21
|
+
end
|
20
22
|
end
|
21
23
|
|
22
24
|
def report(&block)
|
data/lib/kennel/importer.rb
CHANGED
@@ -42,10 +42,15 @@ module Kennel
|
|
42
42
|
|
43
43
|
case resource
|
44
44
|
when "monitor"
|
45
|
-
# flatten monitor options so they are all on the base
|
45
|
+
# flatten monitor options so they are all on the base which is how Monitor builds them
|
46
46
|
data.merge!(data.delete(:options))
|
47
47
|
data.merge!(data.delete(:thresholds) || {})
|
48
|
-
|
48
|
+
|
49
|
+
# clean up values that are the default
|
50
|
+
data.delete(:notify_no_data) if data[:notify_no_data] # Monitor uses true by default
|
51
|
+
data.delete(:notify_audit) unless data[:notify_audit] # Monitor uses false by default
|
52
|
+
|
53
|
+
# keep all values that are settable
|
49
54
|
data = data.slice(*model.instance_methods)
|
50
55
|
|
51
56
|
# make query use critical method if it matches
|
@@ -61,6 +66,8 @@ module Kennel
|
|
61
66
|
widgets&.each { |widget| dry_up_query!(widget) }
|
62
67
|
end
|
63
68
|
|
69
|
+
data.delete(:tags) if data[:tags] == [] # do not create super + [] call
|
70
|
+
|
64
71
|
# simplify template_variables to array of string when possible
|
65
72
|
if vars = data[:template_variables]
|
66
73
|
vars.map! { |v| v[:default] == "*" && v[:prefix] == v[:name] ? v[:name] : v }
|
@@ -18,7 +18,9 @@ module Kennel
|
|
18
18
|
new_host_delay: 300,
|
19
19
|
timeout_h: 0,
|
20
20
|
renotify_interval: 0,
|
21
|
-
|
21
|
+
notify_audit: false,
|
22
|
+
no_data_timeframe: nil, # this works out ok since if notify_no_data is on, it would never be nil
|
23
|
+
groupby_simple_monitor: false
|
22
24
|
}.freeze
|
23
25
|
DEFAULT_ESCALATION_MESSAGE = ["", nil].freeze
|
24
26
|
|
@@ -35,9 +37,9 @@ module Kennel
|
|
35
37
|
warning: -> { nil },
|
36
38
|
ok: -> { nil },
|
37
39
|
id: -> { nil },
|
38
|
-
notify_no_data: -> { true },
|
40
|
+
notify_no_data: -> { true }, # datadog sets this to false by default, but true is the safer
|
39
41
|
no_data_timeframe: -> { 60 },
|
40
|
-
notify_audit: -> {
|
42
|
+
notify_audit: -> { MONITOR_OPTION_DEFAULTS.fetch(:notify_audit) },
|
41
43
|
new_host_delay: -> { MONITOR_OPTION_DEFAULTS.fetch(:new_host_delay) },
|
42
44
|
tags: -> { @project.tags },
|
43
45
|
timeout_h: -> { MONITOR_OPTION_DEFAULTS.fetch(:timeout_h) },
|
@@ -119,7 +121,7 @@ module Kennel
|
|
119
121
|
Utils.path_to_url "/monitors##{id}/edit"
|
120
122
|
end
|
121
123
|
|
122
|
-
# datadog uses
|
124
|
+
# datadog uses / for show and # for edit as separator in it's links
|
123
125
|
def self.parse_url(url)
|
124
126
|
return unless id = url[/\/monitors[\/#](\d+)/, 1]
|
125
127
|
Integer(id)
|
data/lib/kennel/tasks.rb
CHANGED
@@ -68,15 +68,16 @@ namespace :kennel do
|
|
68
68
|
Kennel.update
|
69
69
|
end
|
70
70
|
|
71
|
-
desc "update
|
72
|
-
task :
|
73
|
-
|
74
|
-
|
71
|
+
desc "update on push to the default branch, otherwise show plan"
|
72
|
+
task :ci do
|
73
|
+
branch = (ENV["TRAVIS_BRANCH"] || ENV["GITHUB_REF"]).to_s.sub(/^refs\/heads\//, "")
|
74
|
+
on_default_branch = (branch == (ENV["DEFAULT_BRANCH"] || "master"))
|
75
|
+
is_push = (ENV["TRAVIS_PULL_REQUEST"] == "false" || ENV["GITHUB_EVENT_NAME"] == "push")
|
75
76
|
task_name =
|
76
77
|
if on_default_branch && is_push
|
77
78
|
"kennel:update_datadog"
|
78
79
|
else
|
79
|
-
"kennel:plan" # show plan in
|
80
|
+
"kennel:plan" # show plan in CI logs
|
80
81
|
end
|
81
82
|
|
82
83
|
Rake::Task[task_name].invoke
|
@@ -123,10 +124,18 @@ namespace :kennel do
|
|
123
124
|
Kennel.out.puts Kennel::Importer.new(Kennel.send(:api)).import(resource, id)
|
124
125
|
end
|
125
126
|
|
126
|
-
desc "Dump ALL of datadog config as raw json ... useful for grep/search TYPE=slo|monitor|dashboard"
|
127
|
+
desc "Dump ALL of datadog config as raw json ... useful for grep/search [TYPE=slo|monitor|dashboard]"
|
127
128
|
task dump: :environment do
|
128
|
-
|
129
|
-
|
129
|
+
resources =
|
130
|
+
if type = ENV["TYPE"]
|
131
|
+
[type]
|
132
|
+
else
|
133
|
+
Kennel::Models::Record.subclasses.map(&:api_resource)
|
134
|
+
end
|
135
|
+
resources.each do |resource|
|
136
|
+
Kennel.send(:api).list(resource).each do |r|
|
137
|
+
Kennel.out.puts JSON.pretty_generate(r)
|
138
|
+
end
|
130
139
|
end
|
131
140
|
end
|
132
141
|
|
data/lib/kennel/version.rb
CHANGED
data/template/Readme.md
CHANGED
@@ -1,16 +1,57 @@
|
|
1
|
-
# Kennel
|
2
|
-
|
3
1
|
![](github/cage.jpg?raw=true)
|
4
2
|
|
5
|
-
Manage
|
3
|
+
Manage Datadog Monitors / Dashboards / Slos as code
|
6
4
|
|
7
|
-
-
|
8
|
-
- Changes are PR reviewed and
|
5
|
+
- DRY, searchable, audited, documented
|
6
|
+
- Changes are PR reviewed and applied on merge
|
9
7
|
- Updating shows diff before applying
|
10
|
-
- Automated import of existing
|
8
|
+
- Automated import of existing resources
|
9
|
+
- Resources are grouped into projects that belong to teams and inherit tags
|
10
|
+
- No copy-pasting of ids to create new resources
|
11
|
+
- Automated cleanup when removing code
|
12
|
+
- [Helpers](#helpers) for automating common tasks
|
13
|
+
|
14
|
+
### Applying changes
|
11
15
|
|
12
16
|
![](github/screen.png?raw=true)
|
13
17
|
|
18
|
+
### Example code
|
19
|
+
|
20
|
+
```Ruby
|
21
|
+
# teams/foo.rb
|
22
|
+
module Teams
|
23
|
+
class Foo < Kennel::Models::Team
|
24
|
+
defaults(mention: -> { "@slack-my-team" })
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
# projects/bar.rb
|
29
|
+
class Bar < Kennel::Models::Project
|
30
|
+
defaults(
|
31
|
+
team: -> { Teams::Foo.new }, # use mention and tags from the team
|
32
|
+
parts: -> {
|
33
|
+
[
|
34
|
+
Kennel::Models::Monitor.new(
|
35
|
+
self, # the current project
|
36
|
+
type: -> { "query alert" },
|
37
|
+
kennel_id: -> { "load-too-high" }, # pick a unique name
|
38
|
+
name: -> { "Foobar Load too high" }, # nice descriptive name that will show up in alerts and emails
|
39
|
+
message: -> {
|
40
|
+
<<~TEXT
|
41
|
+
This is bad!
|
42
|
+
#{super()} # inserts mention from team
|
43
|
+
TEXT
|
44
|
+
},
|
45
|
+
query: -> { "avg(last_5m):avg:system.load.5{hostgroup:api} by {pod} > #{critical}" },
|
46
|
+
critical: -> { 20 }
|
47
|
+
)
|
48
|
+
]
|
49
|
+
}
|
50
|
+
)
|
51
|
+
end
|
52
|
+
```
|
53
|
+
|
54
|
+
|
14
55
|
## Structure
|
15
56
|
|
16
57
|
- `projects/` monitors/dashboards/etc scoped by project
|
@@ -91,7 +132,7 @@ end
|
|
91
132
|
- alternatively: `bundle exec rake generate` to only locally update the generated `json` files
|
92
133
|
- review changes then `git commit`
|
93
134
|
- make a PR ... get reviewed ... merge
|
94
|
-
- datadog is updated by
|
135
|
+
- datadog is updated by CI
|
95
136
|
|
96
137
|
### Adding a new dashboard
|
97
138
|
- go to [datadog dashboard UI](https://app.datadoghq.com/dashboard/lists) and click on _New Dashboard_ to create a dashboard
|
@@ -146,7 +187,7 @@ to unblock use the `validate: -> { false }` option.
|
|
146
187
|
|
147
188
|
### Linking with kennel_ids
|
148
189
|
|
149
|
-
To link to existing monitors via their kennel_id
|
190
|
+
To link to existing monitors via their kennel_id `projects kennel_id` + `:` + `monitors kennel id`
|
150
191
|
|
151
192
|
- Screens `uptime` widgets can use `monitor: {id: "foo:bar"}`
|
152
193
|
- Screens `alert_graph` widgets can use `alert_id: "foo:bar"`
|
@@ -158,25 +199,7 @@ To link to existing monitors via their kennel_id
|
|
158
199
|
- figure out project name by converting the class name to snake-case
|
159
200
|
- run `PROJECT=foo bundle exec rake kennel:update_datadog` to test changes for a single project
|
160
201
|
|
161
|
-
###
|
162
|
-
|
163
|
-
Run `rake kennel:alerts TAG=service:my-service` to see all un-muted alerts for a given datadog monitor tag.
|
164
|
-
|
165
|
-
### Validating mentions work
|
166
|
-
|
167
|
-
`rake kennel:validate_mentions` should run as part of CI
|
168
|
-
|
169
|
-
### Grepping through all of datadog
|
170
|
-
|
171
|
-
`TYPE=monitor rake kennel:dump`
|
172
|
-
|
173
|
-
### Find all monitors with No-Data
|
174
|
-
|
175
|
-
`rake kennel:nodata TAG=team:foo`
|
176
|
-
|
177
|
-
## Examples
|
178
|
-
|
179
|
-
### Reusable monitors/dashes/etc
|
202
|
+
### Reuse
|
180
203
|
|
181
204
|
Add to `parts/<folder>`.
|
182
205
|
|
@@ -203,3 +226,23 @@ class Database < Kennel::Models::Project
|
|
203
226
|
)
|
204
227
|
end
|
205
228
|
```
|
229
|
+
|
230
|
+
## Helpers
|
231
|
+
|
232
|
+
### Listing un-muted alerts
|
233
|
+
|
234
|
+
Run `rake kennel:alerts TAG=service:my-service` to see all un-muted alerts for a given datadog monitor tag.
|
235
|
+
|
236
|
+
### Validating mentions work
|
237
|
+
|
238
|
+
`rake kennel:validate_mentions` should run as part of CI
|
239
|
+
|
240
|
+
### Grepping through all of datadog
|
241
|
+
|
242
|
+
`rake kennel:dump`
|
243
|
+
focus on a single type: `TYPE=monitors`
|
244
|
+
|
245
|
+
### Find all monitors with No-Data
|
246
|
+
|
247
|
+
`rake kennel:nodata TAG=team:foo`
|
248
|
+
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: kennel
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.76.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Michael Grosser
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-
|
11
|
+
date: 2020-08-26 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: faraday
|