kennel 1.16.1 → 1.16.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/lib/kennel/version.rb +1 -1
- data/template/Readme.md +188 -0
- metadata +2 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 617e550ae31d28ef32f0e46948246f78156fddde356a0c42bd6dfcacf63eac8d
|
4
|
+
data.tar.gz: d7e3b335b847c8bfe3d9a1cf2a4d967160539137ebe64704a5d71c7b5ac2ccda
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 723d7464968287acb6157a59f3cebc30322c17c92dfad47a9e1a49c63a4c5ab3a490b1a13bc179591c30c424b57ac28410526c493bc80ca34b5084ec8f3b65ba
|
7
|
+
data.tar.gz: 62cf68ab5ef460ece823a3ac014542692afabb8980c84f584af1b5de73230508039ab634bfcfa480f1608b6a7b9039734de5ef98b337e82a596540d82092d3b5
|
data/lib/kennel/version.rb
CHANGED
data/template/Readme.md
ADDED
@@ -0,0 +1,188 @@
|
|
1
|
+
# Kennel
|
2
|
+
|
3
|
+
![](github/cage.jpg?raw=true)
|
4
|
+
|
5
|
+
Keep datadog monitors/dashboards/etc in version control, avoid chaotic management via UI.
|
6
|
+
|
7
|
+
- Documented, reusable, automated, and searchable configuration
|
8
|
+
- Changes are PR reviewed and auditable
|
9
|
+
- Good defaults like no-data / re-notify are preselected
|
10
|
+
- Reliable cleanup with automated deletion
|
11
|
+
|
12
|
+
![](github/screen.png?raw=true)
|
13
|
+
|
14
|
+
## Structure
|
15
|
+
|
16
|
+
- `projects/` monitors/dashboards/etc scoped by project
|
17
|
+
- `teams/` team definitions
|
18
|
+
- `parts/` monitors/dashes/etc that are used by multiple projects
|
19
|
+
- `generated/` projects as json, to show current state and proposed changes in PRs
|
20
|
+
|
21
|
+
## Workflows
|
22
|
+
|
23
|
+
### Setup
|
24
|
+
- clone the repo
|
25
|
+
- `gem install bundler && bundle install`
|
26
|
+
- go to [Datadog API Settings](https://app.datadoghq.com/account/settings#api)
|
27
|
+
- find or create your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=` (will be on the last page if new)
|
28
|
+
- copy any `API Key` and add it to `.env` as `DATADOG_API_KEY`
|
29
|
+
|
30
|
+
### Adding a team
|
31
|
+
|
32
|
+
```ruby
|
33
|
+
# teams/my_team.rb
|
34
|
+
module Teams
|
35
|
+
class MyTeam < Kennel::Models::Team
|
36
|
+
defaults(
|
37
|
+
slack: -> { "my-alerts" },
|
38
|
+
email: -> { "my-team@exmaple.com" }
|
39
|
+
)
|
40
|
+
end
|
41
|
+
end
|
42
|
+
```
|
43
|
+
|
44
|
+
### Adding a new monitor
|
45
|
+
- use [datadog monitor UI](https://app.datadoghq.com/monitors#create/metric) to create a monitor
|
46
|
+
- get the `id` from the url, click "Export Monitor" on the monitors edit tab to get the `query` and `type`
|
47
|
+
- see below
|
48
|
+
|
49
|
+
### Updating an existing monitor
|
50
|
+
- find or create a project in `projects/`
|
51
|
+
- add a monitor to `parts: [` list
|
52
|
+
```Ruby
|
53
|
+
# projects/my_project.rb
|
54
|
+
class MyProject < Kennel::Models::Project
|
55
|
+
defaults(
|
56
|
+
team: -> { Teams::MyTeam.new }, # use existing team or create new one in teams/
|
57
|
+
parts: -> {
|
58
|
+
[
|
59
|
+
Kennel::Models::Monitor.new(
|
60
|
+
self,
|
61
|
+
id: -> { 123456 }, # id from datadog url
|
62
|
+
type: -> { "query alert" },
|
63
|
+
kennel_id: -> { "load-too-high" }, # make up a unique name
|
64
|
+
name: -> { "Foobar Load too high" }, # nice descriptive name that will show up in alerts and emails
|
65
|
+
message: -> {
|
66
|
+
# Explain what behavior to expect and how to fix the cause. Use #{super()} to add team notifications.
|
67
|
+
<<~TEXT
|
68
|
+
Foobar will be slow and that could cause Barfoo to go down.
|
69
|
+
Add capacity or debug why it is suddenly slow.
|
70
|
+
#{super()}
|
71
|
+
TEXT
|
72
|
+
},
|
73
|
+
query: -> { "avg(last_5m):avg:system.load.5{hostgroup:api} by {pod} > #{critical}" }, # replace actual value with #{critical} to keep them in sync
|
74
|
+
critical: -> { 20 }
|
75
|
+
)
|
76
|
+
]
|
77
|
+
}
|
78
|
+
)
|
79
|
+
end
|
80
|
+
```
|
81
|
+
- `bundle exec rake plan` update to existing should be shown (not Create / Delete)
|
82
|
+
- alternatively: `bundle exec rake generate` to only update the generated `json` files
|
83
|
+
- review changes then `git commit`
|
84
|
+
- make a PR ... get reviewed ... merge
|
85
|
+
- datadog is updated by travis
|
86
|
+
|
87
|
+
### Adding a new dashboard
|
88
|
+
- go to [datadog dashboard UI](https://app.datadoghq.com/dash/list) and click on _New Dashboard_ to create a dashboard
|
89
|
+
- get the `id` from the url
|
90
|
+
- see below
|
91
|
+
|
92
|
+
### Updating an existing dashboard
|
93
|
+
- find or create a project in `projects/`
|
94
|
+
- add a dashboard to `parts: [` list
|
95
|
+
```Ruby
|
96
|
+
class MyProject < Kennel::Models::Project
|
97
|
+
defaults(
|
98
|
+
team: -> { Teams::MyTeam.new }, # use existing team or create new one in teams/
|
99
|
+
parts: -> {
|
100
|
+
[
|
101
|
+
Kennel::Models::Dash.new(
|
102
|
+
self,
|
103
|
+
id: -> { 123457 }, # id from datadog url
|
104
|
+
title: -> { "My Dashboard" },
|
105
|
+
description: -> { "Overview of foobar" },
|
106
|
+
template_variables: -> { ["environment"] }, # see https://docs.datadoghq.com/api/?lang=ruby#timeboards
|
107
|
+
kennel_id: -> { "overview-dashboard" }, # make up a unique name
|
108
|
+
definitions: -> {
|
109
|
+
[ # An array or arrays, each one is a graph in the dashboard, alternatively a hash for finer control
|
110
|
+
[
|
111
|
+
# title, viz, type, query, edit an existing graph and see the json definition
|
112
|
+
"Graph name", "timeseries", "area", "sum:mystats.foobar{$environment}"
|
113
|
+
],
|
114
|
+
[
|
115
|
+
# queries can be an Array as well, this will generate multiple requests
|
116
|
+
# for a single graph
|
117
|
+
"Graph name", "timeseries", "area", ["sum:mystats.foobar{$environment}", "sum:mystats.success{$environment}"],
|
118
|
+
# add events too ...
|
119
|
+
events: [{q: "tags:foobar,deploy", tags_execution: "and"}]
|
120
|
+
]
|
121
|
+
]
|
122
|
+
}
|
123
|
+
)
|
124
|
+
]
|
125
|
+
}
|
126
|
+
)
|
127
|
+
end
|
128
|
+
```
|
129
|
+
|
130
|
+
### Adding a new screenboard
|
131
|
+
- similar to `dash.rb`
|
132
|
+
- add to `parts:` list
|
133
|
+
```Ruby
|
134
|
+
Kennel::Models::Screen.new(
|
135
|
+
self,
|
136
|
+
board_title: -> { "test-board" },
|
137
|
+
kennel_id: -> { "test-screen" },
|
138
|
+
widgets: -> {
|
139
|
+
[
|
140
|
+
{text: "Hello World", height: 6, width: 24, x: 0, y: 0, type: "free_text"},
|
141
|
+
{title_text: "CPU", height: 12, width: 36, timeframe: "1mo", x: 0, y: 6, type: "timeseries", tile_def: {viz: "timeseries", requests: [{q: "avg:system.cpu.user{*}", type: "line"}]}}
|
142
|
+
]
|
143
|
+
}
|
144
|
+
)
|
145
|
+
```
|
146
|
+
|
147
|
+
### Skipping validations
|
148
|
+
|
149
|
+
Some validations might be too strict for your usecase or just wrong, please open an issue and
|
150
|
+
to unblock use the `validate: -> { false }` option.
|
151
|
+
|
152
|
+
### Debugging locally
|
153
|
+
|
154
|
+
- make sure to be on update `master` to not undo other changes
|
155
|
+
- run `PROJECT=foo bundle exec rake kennel:update_datadog` to test changes for a single project
|
156
|
+
|
157
|
+
### Listing umuted alerts
|
158
|
+
|
159
|
+
Run `rake kennel:alerts TAG=service:my-service` to see all un-muted alerts for a given datadog monitor tag.
|
160
|
+
|
161
|
+
## Examples
|
162
|
+
|
163
|
+
### Reusable monitors/dashes/etc
|
164
|
+
|
165
|
+
Add to `parts/<folder>`.
|
166
|
+
|
167
|
+
```Ruby
|
168
|
+
module Monitors
|
169
|
+
class LoadTooHigh < Kennel::Models::Monitor
|
170
|
+
defaults(
|
171
|
+
name: -> { "#{project.name} load too high" },
|
172
|
+
message: -> { "Shut it down!" },
|
173
|
+
query: -> { "avg(last_5m):avg:system.load.5{hostgroup:#{project.kennel_id}} by {pod} > #{critical}" }
|
174
|
+
)
|
175
|
+
end
|
176
|
+
end
|
177
|
+
```
|
178
|
+
|
179
|
+
Reuse it in multiple projects.
|
180
|
+
|
181
|
+
```Ruby
|
182
|
+
class Database < Kennel::Models::Project
|
183
|
+
defaults(
|
184
|
+
team: -> { Kennel::Models::Team.new(slack: -> { 'foo' }, kennel_id: -> { 'foo' }) },
|
185
|
+
parts: -> { [Monitors::LoadTooHigh.new(self, critical: -> { 13 })] }
|
186
|
+
)
|
187
|
+
end
|
188
|
+
```
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: kennel
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.16.
|
4
|
+
version: 1.16.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Michael Grosser
|
@@ -64,6 +64,7 @@ files:
|
|
64
64
|
- lib/kennel/unmuted_alerts.rb
|
65
65
|
- lib/kennel/utils.rb
|
66
66
|
- lib/kennel/version.rb
|
67
|
+
- template/Readme.md
|
67
68
|
homepage: https://github.com/grosser/kennel
|
68
69
|
licenses:
|
69
70
|
- MIT
|