kennel 1.16.0 → 1.16.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Readme.md +227 -0
- data/lib/kennel/version.rb +1 -1
- metadata +3 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d67a0d2b5d6d6b4d4862d7a5b99710e6ec4e775c281b11dff4495400a6f2cb4b
|
|
4
|
+
data.tar.gz: cb1835599773e4a9eb78b00469f5964b4f1490b6dcb583e8fe30c39d39f5ce42
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 935b9f68937e7fe8b26174c78ed2a5bcc9bbe282917c0da212d44a160dffc3dbfd4ca34ec6c25ae8a86dcdfd9e774b1712bb24895eaa8d2ce2f1cd364a8cbcc9
|
|
7
|
+
data.tar.gz: 1b6e28c77a09bcc7dc5af9409e85437b64212411ef029f8eaf7f60e41c24d7fedef797f3f0685c214cb16da868e63f44c4caab968e87d5ec3754d033a29531bc
|
data/Readme.md
ADDED
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
# Kennel
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
|
|
5
|
+
Keep datadog monitors/dashboards/etc in version control, avoid chaotic management via UI.
|
|
6
|
+
|
|
7
|
+
- Documented, reusable, automated, and searchable configuration
|
|
8
|
+
- Changes are PR reviewed and auditable
|
|
9
|
+
- Good defaults like no-data / re-notify are preselected
|
|
10
|
+
- Reliable cleanup with automated deletion
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
<!-- NOT IN template/Readme.md -->
|
|
14
|
+
## Install
|
|
15
|
+
|
|
16
|
+
- create a new private `kennel` repo for your organization (do not fork this repo)
|
|
17
|
+
- use the template folder as starting point:
|
|
18
|
+
```
|
|
19
|
+
git clone git@github.com:your-org/kennel.git
|
|
20
|
+
git clone git@github.com:grosser/kennel.git seed
|
|
21
|
+
mv seed/teamplate/* kennel/
|
|
22
|
+
cd kennel && git add . && git commit -m 'initial'
|
|
23
|
+
```
|
|
24
|
+
- add a basic projects and teams so others can copy-paste to get started
|
|
25
|
+
- setup travis build for your repo
|
|
26
|
+
- uncomment `.travis.yml` section for automated github PR feedback and datadog updates on merge
|
|
27
|
+
- follow `Setup` in your repos Readme.md
|
|
28
|
+
<!-- NOT IN -->
|
|
29
|
+
|
|
30
|
+
## Structure
|
|
31
|
+
|
|
32
|
+
- `projects/` monitors/dashboards/etc scoped by project
|
|
33
|
+
- `teams/` team definitions
|
|
34
|
+
- `parts/` monitors/dashes/etc that are used by multiple projects
|
|
35
|
+
- `generated/` projects as json, to show current state and proposed changes in PRs
|
|
36
|
+
|
|
37
|
+
## Workflows
|
|
38
|
+
|
|
39
|
+
<!-- ONLY IN template/Readme.md
|
|
40
|
+
### Setup
|
|
41
|
+
- clone the repo
|
|
42
|
+
- `gem install bundler && bundle install`
|
|
43
|
+
- go to [Datadog API Settings](https://app.datadoghq.com/account/settings#api)
|
|
44
|
+
- find or create your personal "Application Key" and add it to `.env` as `DATADOG_APP_KEY=` (will be on the last page if new)
|
|
45
|
+
- copy any `API Key` and add it to `.env` as `DATADOG_API_KEY`
|
|
46
|
+
-->
|
|
47
|
+
|
|
48
|
+
### Adding a team
|
|
49
|
+
|
|
50
|
+
```ruby
|
|
51
|
+
# teams/my_team.rb
|
|
52
|
+
module Teams
|
|
53
|
+
class MyTeam < Kennel::Models::Team
|
|
54
|
+
defaults(
|
|
55
|
+
slack: -> { "my-alerts" },
|
|
56
|
+
email: -> { "my-team@exmaple.com" }
|
|
57
|
+
)
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### Adding a new monitor
|
|
63
|
+
- use [datadog monitor UI](https://app.datadoghq.com/monitors#create/metric) to create a monitor
|
|
64
|
+
- get the `id` from the url, click "Export Monitor" on the monitors edit tab to get the `query` and `type`
|
|
65
|
+
- see below
|
|
66
|
+
|
|
67
|
+
### Updating an existing monitor
|
|
68
|
+
- find or create a project in `projects/`
|
|
69
|
+
- add a monitor to `parts: [` list
|
|
70
|
+
```Ruby
|
|
71
|
+
# projects/my_project.rb
|
|
72
|
+
class MyProject < Kennel::Models::Project
|
|
73
|
+
defaults(
|
|
74
|
+
team: -> { Teams::MyTeam.new }, # use existing team or create new one in teams/
|
|
75
|
+
parts: -> {
|
|
76
|
+
[
|
|
77
|
+
Kennel::Models::Monitor.new(
|
|
78
|
+
self,
|
|
79
|
+
id: -> { 123456 }, # id from datadog url
|
|
80
|
+
type: -> { "query alert" },
|
|
81
|
+
kennel_id: -> { "load-too-high" }, # make up a unique name
|
|
82
|
+
name: -> { "Foobar Load too high" }, # nice descriptive name that will show up in alerts and emails
|
|
83
|
+
message: -> {
|
|
84
|
+
# Explain what behavior to expect and how to fix the cause. Use #{super()} to add team notifications.
|
|
85
|
+
<<~TEXT
|
|
86
|
+
Foobar will be slow and that could cause Barfoo to go down.
|
|
87
|
+
Add capacity or debug why it is suddenly slow.
|
|
88
|
+
#{super()}
|
|
89
|
+
TEXT
|
|
90
|
+
},
|
|
91
|
+
query: -> { "avg(last_5m):avg:system.load.5{hostgroup:api} by {pod} > #{critical}" }, # replace actual value with #{critical} to keep them in sync
|
|
92
|
+
critical: -> { 20 }
|
|
93
|
+
)
|
|
94
|
+
]
|
|
95
|
+
}
|
|
96
|
+
)
|
|
97
|
+
end
|
|
98
|
+
```
|
|
99
|
+
- `bundle exec rake plan` update to existing should be shown (not Create / Delete)
|
|
100
|
+
- alternatively: `bundle exec rake generate` to only update the generated `json` files
|
|
101
|
+
- review changes then `git commit`
|
|
102
|
+
- make a PR ... get reviewed ... merge
|
|
103
|
+
- datadog is updated by travis
|
|
104
|
+
|
|
105
|
+
### Adding a new dashboard
|
|
106
|
+
- go to [datadog dashboard UI](https://app.datadoghq.com/dash/list) and click on _New Dashboard_ to create a dashboard
|
|
107
|
+
- get the `id` from the url
|
|
108
|
+
- see below
|
|
109
|
+
|
|
110
|
+
### Updating an existing dashboard
|
|
111
|
+
- find or create a project in `projects/`
|
|
112
|
+
- add a dashboard to `parts: [` list
|
|
113
|
+
```Ruby
|
|
114
|
+
class MyProject < Kennel::Models::Project
|
|
115
|
+
defaults(
|
|
116
|
+
team: -> { Teams::MyTeam.new }, # use existing team or create new one in teams/
|
|
117
|
+
parts: -> {
|
|
118
|
+
[
|
|
119
|
+
Kennel::Models::Dash.new(
|
|
120
|
+
self,
|
|
121
|
+
id: -> { 123457 }, # id from datadog url
|
|
122
|
+
title: -> { "My Dashboard" },
|
|
123
|
+
description: -> { "Overview of foobar" },
|
|
124
|
+
template_variables: -> { ["environment"] }, # see https://docs.datadoghq.com/api/?lang=ruby#timeboards
|
|
125
|
+
kennel_id: -> { "overview-dashboard" }, # make up a unique name
|
|
126
|
+
definitions: -> {
|
|
127
|
+
[ # An array or arrays, each one is a graph in the dashboard, alternatively a hash for finer control
|
|
128
|
+
[
|
|
129
|
+
# title, viz, type, query, edit an existing graph and see the json definition
|
|
130
|
+
"Graph name", "timeseries", "area", "sum:mystats.foobar{$environment}"
|
|
131
|
+
],
|
|
132
|
+
[
|
|
133
|
+
# queries can be an Array as well, this will generate multiple requests
|
|
134
|
+
# for a single graph
|
|
135
|
+
"Graph name", "timeseries", "area", ["sum:mystats.foobar{$environment}", "sum:mystats.success{$environment}"],
|
|
136
|
+
# add events too ...
|
|
137
|
+
events: [{q: "tags:foobar,deploy", tags_execution: "and"}]
|
|
138
|
+
]
|
|
139
|
+
]
|
|
140
|
+
}
|
|
141
|
+
)
|
|
142
|
+
]
|
|
143
|
+
}
|
|
144
|
+
)
|
|
145
|
+
end
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### Adding a new screenboard
|
|
149
|
+
- similar to `dash.rb`
|
|
150
|
+
- add to `parts:` list
|
|
151
|
+
```Ruby
|
|
152
|
+
Kennel::Models::Screen.new(
|
|
153
|
+
self,
|
|
154
|
+
board_title: -> { "test-board" },
|
|
155
|
+
kennel_id: -> { "test-screen" },
|
|
156
|
+
widgets: -> {
|
|
157
|
+
[
|
|
158
|
+
{text: "Hello World", height: 6, width: 24, x: 0, y: 0, type: "free_text"},
|
|
159
|
+
{title_text: "CPU", height: 12, width: 36, timeframe: "1mo", x: 0, y: 6, type: "timeseries", tile_def: {viz: "timeseries", requests: [{q: "avg:system.cpu.user{*}", type: "line"}]}}
|
|
160
|
+
]
|
|
161
|
+
}
|
|
162
|
+
)
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
### Skipping validations
|
|
166
|
+
|
|
167
|
+
Some validations might be too strict for your usecase or just wrong, please open an issue and
|
|
168
|
+
to unblock use the `validate: -> { false }` option.
|
|
169
|
+
|
|
170
|
+
### Debugging locally
|
|
171
|
+
|
|
172
|
+
- make sure to be on update `master` to not undo other changes
|
|
173
|
+
- run `PROJECT=foo bundle exec rake kennel:update_datadog` to test changes for a single project
|
|
174
|
+
|
|
175
|
+
### Listing umuted alerts
|
|
176
|
+
|
|
177
|
+
Run `rake kennel:alerts TAG=service:my-service` to see all un-muted alerts for a given datadog monitor tag.
|
|
178
|
+
|
|
179
|
+
## Examples
|
|
180
|
+
|
|
181
|
+
### Reusable monitors/dashes/etc
|
|
182
|
+
|
|
183
|
+
Add to `parts/<folder>`.
|
|
184
|
+
|
|
185
|
+
```Ruby
|
|
186
|
+
module Monitors
|
|
187
|
+
class LoadTooHigh < Kennel::Models::Monitor
|
|
188
|
+
defaults(
|
|
189
|
+
name: -> { "#{project.name} load too high" },
|
|
190
|
+
message: -> { "Shut it down!" },
|
|
191
|
+
query: -> { "avg(last_5m):avg:system.load.5{hostgroup:#{project.kennel_id}} by {pod} > #{critical}" }
|
|
192
|
+
)
|
|
193
|
+
end
|
|
194
|
+
end
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
Reuse it in multiple projects.
|
|
198
|
+
|
|
199
|
+
```Ruby
|
|
200
|
+
class Database < Kennel::Models::Project
|
|
201
|
+
defaults(
|
|
202
|
+
team: -> { Kennel::Models::Team.new(slack: -> { 'foo' }, kennel_id: -> { 'foo' }) },
|
|
203
|
+
parts: -> { [Monitors::LoadTooHigh.new(self, critical: -> { 13 })] }
|
|
204
|
+
)
|
|
205
|
+
end
|
|
206
|
+
```
|
|
207
|
+
<!-- NOT IN template/Readme.md -->
|
|
208
|
+
|
|
209
|
+
### Integration testing
|
|
210
|
+
|
|
211
|
+
```
|
|
212
|
+
rake play
|
|
213
|
+
cd template
|
|
214
|
+
rake kennel:plan
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
Then make changes to play around, do not commit changes and make sure to revert with a `rake kennel:update` after deleting everything.
|
|
218
|
+
|
|
219
|
+
To make changes via the UI, make a new free datadog account and use it's credentaisl instead.
|
|
220
|
+
|
|
221
|
+
Author
|
|
222
|
+
======
|
|
223
|
+
[Michael Grosser](http://grosser.it)<br/>
|
|
224
|
+
michael@grosser.it<br/>
|
|
225
|
+
License: MIT<br/>
|
|
226
|
+
[](https://travis-ci.org/grosser/kennel)
|
|
227
|
+
<!-- NOT IN -->
|
data/lib/kennel/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kennel
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.16.
|
|
4
|
+
version: 1.16.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Michael Grosser
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2018-10-
|
|
11
|
+
date: 2018-10-09 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: faraday
|
|
@@ -44,6 +44,7 @@ executables: []
|
|
|
44
44
|
extensions: []
|
|
45
45
|
extra_rdoc_files: []
|
|
46
46
|
files:
|
|
47
|
+
- Readme.md
|
|
47
48
|
- lib/kennel.rb
|
|
48
49
|
- lib/kennel/api.rb
|
|
49
50
|
- lib/kennel/file_cache.rb
|