sensu-extensions-influxdb2 0.0.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/LICENSE +15 -0
- data/README.md +258 -0
- data/lib/sensu/extensions/history.rb +85 -0
- data/lib/sensu/extensions/influxdb2.rb +228 -0
- data/lib/sensu/extensions/influxdb2/influx_relay.rb +77 -0
- data/lib/sensu/extensions/influxdb2/version.rb +10 -0
- metadata +219 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 0b0f5ae119ad47adabd692247f89da5e49cb47de
|
4
|
+
data.tar.gz: 0ae41118ccc5cc0e25601dcb74100f962bf00e3b
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 467a5cdf430eeace8fa6fd22031a6298414bccc34e4060d5dca52f300b9a2132f2d6d95754d0a8623b0e52ec3acc953261772a6862b70123bb7b62cab8efc518
|
7
|
+
data.tar.gz: a9f5a6f2640344f25b1ad23e7a7a34e90405fa5c5241992fc50be12520988f584b613dbdd05e4c55a5d02515190b3ca5cf21797926c2a6cf48ee89867932fe31
|
data/LICENSE
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
Copyright 2014 John E. Vincent and contributors.
|
2
|
+
|
3
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
+
you may not use this file except in compliance with the License.
|
5
|
+
You may obtain a copy of the License at
|
6
|
+
|
7
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
8
|
+
|
9
|
+
Unless required by applicable law or agreed to in writing, software
|
10
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
+
See the License for the specific language governing permissions and
|
13
|
+
limitations under the License.
|
14
|
+
|
15
|
+
|
data/README.md
ADDED
@@ -0,0 +1,258 @@
|
|
1
|
+
__NOTE:__ running `sensu-install -e sensu-extensions-influxdb` will __NOT__ install this extension, as a project with the same name already exists on Rubygems.org. Before reporting any issues here, make sure you are using this version of the extension. (_hint:_ check the version number on the logs)
|
2
|
+
|
3
|
+
# Requirements
|
4
|
+
|
5
|
+
This extension uses InfluxDB [Line Protocol](https://influxdb.com/docs/v0.9/write_protocols/line.html) over HTTP to send metrics.
|
6
|
+
|
7
|
+
Since Sensu already uses [eventmachine](https://github.com/eventmachine/eventmachine), you just have to ensure that em-http-request gem is present inside Sensu's embedded Ruby :
|
8
|
+
* `em-http-request` ruby gem
|
9
|
+
|
10
|
+
By default, this extension uses in memory metrics caching : it makes InfluxDB writes batched.
|
11
|
+
This cache is flushed every __500 items__ or __6 seconds__ (this values can be changed in conf).
|
12
|
+
|
13
|
+
## Caveat
|
14
|
+
* SSL/TLS connection to InfluxDB is still broken in Sensu `0.20.6`. We're waiting for a Ruby / EM upgrade in Sensu project :
|
15
|
+
* https://github.com/sensu/sensu/issues/1084
|
16
|
+
|
17
|
+
# Usage
|
18
|
+
The configuration options are pretty straight forward.
|
19
|
+
|
20
|
+
Metrics are inserted into the database using the check's key name as _measurement_ name. So if you're using the `sensu-plugins-load-checks` community plugin :
|
21
|
+
```
|
22
|
+
my-host-01.load_avg.one 0.02 1444824197
|
23
|
+
my-host-01.load_avg.five 0.04 1444824197
|
24
|
+
my-host-01.load_avg.fifteen 0.09 1444824197
|
25
|
+
```
|
26
|
+
In this example, you'll have 3 differents _measurements_ in your database :
|
27
|
+
```
|
28
|
+
> show measurements
|
29
|
+
name: measurements
|
30
|
+
------------------
|
31
|
+
name
|
32
|
+
my-host-01.load_avg.fifteen
|
33
|
+
my-host-01.load_avg.five
|
34
|
+
my-host-01.load_avg.one
|
35
|
+
```
|
36
|
+
|
37
|
+
```
|
38
|
+
> select * from "my-host-01.load_avg.one";
|
39
|
+
name: my-host-01.load_avg.one
|
40
|
+
------------------
|
41
|
+
time host value duration
|
42
|
+
2015-10-14T13:53:22Z my-host-01 0.34 0.399
|
43
|
+
2015-10-14T13:53:32Z my-host-01 0.29 0.419
|
44
|
+
2015-10-14T13:53:42Z my-host-01 0.39 0.392
|
45
|
+
2015-10-14T13:53:52Z my-host-01 0.41 0.398
|
46
|
+
[...]
|
47
|
+
```
|
48
|
+
|
49
|
+
Additionally a `duration` value will be present based on the time it took the check to run (this is gleaned from the sensu event data).
|
50
|
+
|
51
|
+
The name of the _measurement_ is based on the value of `strip_metric` and/or a template, as described below.
|
52
|
+
The name of the key ```host``` is grabbed from sensu event client name.
|
53
|
+
|
54
|
+
## Extension not a handler
|
55
|
+
Note that the first push of this was a handler that could be called via `pipe`. This is now an actual extension that's more performant since it's actually in the sensu-server runtime. Additionally it's now using batch submission to InfluxDB by writing all the points for a given series at once.
|
56
|
+
|
57
|
+
To [load the extension](https://sensuapp.org/docs/latest/reference/extensions.html), you will need to install the gem into the sensu ruby, and then add the following config file
|
58
|
+
```json
|
59
|
+
{
|
60
|
+
"extensions": {
|
61
|
+
"influxdb": {
|
62
|
+
"gem": "sensu-extensions-influxdb",
|
63
|
+
},
|
64
|
+
"history": {
|
65
|
+
"gem": "sensu-extensions-history",
|
66
|
+
}
|
67
|
+
}
|
68
|
+
}
|
69
|
+
```
|
70
|
+
|
71
|
+
**NOTE**: Even though this gem is called `sensu-extensions-influxdb`, the way sensu loads the extension is by parsing the name of the gem, and loading the library. This is why we have to specify `sensu-extensions-history`, even though that gem doesn't exist.
|
72
|
+
|
73
|
+
Create a set to wrap this extension into a callable handler. In this example, we created a ```metrics``` handler wrapping a debug output and this Influx extension :
|
74
|
+
|
75
|
+
_/etc/sensu/conf.d/handlers/metrics.json_ :
|
76
|
+
```json
|
77
|
+
{
|
78
|
+
"handlers": {
|
79
|
+
"metrics": {
|
80
|
+
"type": "set",
|
81
|
+
"handlers": [ "debug", "influxdb"]
|
82
|
+
}
|
83
|
+
}
|
84
|
+
}
|
85
|
+
}
|
86
|
+
```
|
87
|
+
|
88
|
+
_Note :_ Since Sensu 0.17.1 you can also use extension name directly :
|
89
|
+
```
|
90
|
+
Check definitions can now specify a Sensu check extension to run,
|
91
|
+
"extension", instead of a command.
|
92
|
+
```
|
93
|
+
|
94
|
+
## Handler config
|
95
|
+
|
96
|
+
`/etc/sensu/conf.d/influxdb.json`
|
97
|
+
```json
|
98
|
+
{
|
99
|
+
"influxdb": {
|
100
|
+
"database": "stats",
|
101
|
+
"host": "localhost",
|
102
|
+
"port": "8086",
|
103
|
+
"username": "stats",
|
104
|
+
"password": "stats",
|
105
|
+
"use_ssl": false,
|
106
|
+
"strip_metric": "host",
|
107
|
+
"tags": {
|
108
|
+
"region": "my-dc-01",
|
109
|
+
"stage": "prod"
|
110
|
+
},
|
111
|
+
"templates": [
|
112
|
+
{"^sensu\\.checks\\..*":"source.measurement.field*"},
|
113
|
+
{".*\\.cgroup\\..*": "host.path.component"}
|
114
|
+
]
|
115
|
+
}
|
116
|
+
}
|
117
|
+
```
|
118
|
+
|
119
|
+
### Config attributes
|
120
|
+
|
121
|
+
* `host`, `port`, `username`, `password` and `database` are pretty straight forward. If `use_ssl` is set to true, the connection to the influxdb server will be made using https instead of http.
|
122
|
+
|
123
|
+
* `use_basic_auth` if your InfluxDB is behind a proxy that requires authentication you can set this to `true`, in that case you also have to define `basic_user` and `basic_pass`.
|
124
|
+
|
125
|
+
* `filters` allow's you to "find and replace" words/patterns in the key, it's a Hash where the key is a regex pattern and the value is a string that should replace it. (these are passed directly to the gsub method). Filters are applyed before any other transformations like templates.
|
126
|
+
|
127
|
+
* `templates` like with the InfluxDB Graphite plugin, you can specify patterns and formats, if the metric name matches the pattern, the template will be applied. Templates can be defined per check, in the check configuration, or globaly in the handler's configuration. Parts of the metric name will be converted into tags, reducing the measurement name in InfluxDB whilst keeping all the information as tags or fields.
|
128
|
+
You can specifty multiple templates, if your metric matches more than one the first one will be used.
|
129
|
+
For example, if you have a metric named:
|
130
|
+
```
|
131
|
+
ip-10-0-1-32.host_stats.load_avg 3
|
132
|
+
```
|
133
|
+
You can set a template like:
|
134
|
+
```json
|
135
|
+
"templates":{
|
136
|
+
".*\\.host_stats\\..*": "host.type"
|
137
|
+
}
|
138
|
+
```
|
139
|
+
And as a result the following will be sent to InfluxDB:
|
140
|
+
```
|
141
|
+
load_avg,host=ip-10-0-1-32,type=host_stats value=3
|
142
|
+
```
|
143
|
+
You can also use three special keywords in your templates, `void`, `measurement` and `field` and these can be used with a `*` at the end.
|
144
|
+
Example, considereing the following metrics:
|
145
|
+
```
|
146
|
+
ip-10-0-1-32.host_stats.load_avg.one 3
|
147
|
+
ip-10-0-1-32.host_stats.load_avg.five 9
|
148
|
+
ip-10-0-1-32.host_stats.load_avg.fifteen 13
|
149
|
+
```
|
150
|
+
You could create a template like this:
|
151
|
+
```json
|
152
|
+
"templates":{
|
153
|
+
".*\\.host_stats\\..*": "host.measurement.field*"
|
154
|
+
}
|
155
|
+
```
|
156
|
+
And you would get:
|
157
|
+
```
|
158
|
+
host_stats,host=ip-10-0-1-32 load_avg.one=3
|
159
|
+
host_stats,host=ip-10-0-1-32 load_avg.five=9
|
160
|
+
host_stats,host=ip-10-0-1-32 load_avg.fifteen=13
|
161
|
+
```
|
162
|
+
Or, a template like this:
|
163
|
+
```json
|
164
|
+
"templates":{
|
165
|
+
".*\\.host_stats\\..*": "host.measurement.field.aggregation"
|
166
|
+
}
|
167
|
+
```
|
168
|
+
Would get you:
|
169
|
+
```
|
170
|
+
host_stats,host=ip-10-0-1-32,aggregation=one load_avg=3
|
171
|
+
host_stats,host=ip-10-0-1-32,aggregation=five load_avg=9
|
172
|
+
host_stats,host=ip-10-0-1-32,aggregation=fifteen load_avg=13
|
173
|
+
```
|
174
|
+
the `void` keyword will make the handler ignore that part of the metric name.
|
175
|
+
|
176
|
+
* `tags` hash is also pretty straight forward. Just list here in a flat-hash design as many influxdb tags you wish to be added in your measures. This can also be set both in the handler configuration or as part of the check configuration.
|
177
|
+
|
178
|
+
* `strip_metric` however might not be. This is used to "clean up" the data sent to influxdb. Normally everything sent to handlers is akin to the `graphite`/`stats` style:
|
179
|
+
```
|
180
|
+
something.host.metrictype.foo.bar
|
181
|
+
```
|
182
|
+
or
|
183
|
+
```
|
184
|
+
host.stats.something.foo.bar
|
185
|
+
```
|
186
|
+
|
187
|
+
Really the pattern is irrelevant. People have different tastes. Adding much of that data to the column name in InfluxDB is rather silly so `strip_metric` provides you with a chance to add a value that strips off everything up to (and including that value). This allows you to continue sending to graphite or statsd or whatever and still use this handler.
|
188
|
+
|
189
|
+
Using the examples above, if you set the `strip_metric` to `host`, then the measurement in InfluxDB would be called `metrictype.foo.bar` or `stats.something.foo.bar`. If you set the value to `foo` then the measurement would simply be called `foo`
|
190
|
+
|
191
|
+
Note that :
|
192
|
+
* `strip_metric` isn't required.
|
193
|
+
* `strip_metric` can be set either in the handler configuration or as part of the check configuration, check configuration takes precedence.
|
194
|
+
* you can cleanup an arbitrary string from your keyname or use `host` as special value to cleanup the sensu event client name from your key.
|
195
|
+
|
196
|
+
#### Other attributes
|
197
|
+
|
198
|
+
* `time_precision` : global checks time precision (default is `'s'`)
|
199
|
+
* `buffer_max_size` : buffer size limit before flush - This is the amount of points in the InfluxDB batch - (default is `500`)
|
200
|
+
* `buffer_max_age` : buffer maximum age - Flush will be forced after this amount of time - (default is `6` seconds)
|
201
|
+
* `Proxy mode` : If the extension is configured to be in proxy mode, it will skip the transformation step and assume that the data is valid [line protocol](https://docs.influxdata.com/influxdb/latest/write_protocols/line_protocol_reference). It will not take into account any tags defined in the sensu-configuration.
|
202
|
+
|
203
|
+
## Check options
|
204
|
+
|
205
|
+
In the check config, an optional `influxdb` section can be added, containing a `database` option and `tags`.
|
206
|
+
You can also set `proxy_mode` in this section on the checks to override the default configuration set on the handler. This way checks that use the InfluxDB line protocol can coexsist with checks that use the Graphite format and use the same handler.
|
207
|
+
As mentioned above you can also specify `strip_metric`, `templates` and `filters`, in the check configuration.
|
208
|
+
If specified, this overrides the default `database` and `strip_metric` options in the handler config and adds (or overrides) influxdb tags and templates.
|
209
|
+
|
210
|
+
This allows events to be written to different influxdb databases and modify key indexes on a check-by-check basis.
|
211
|
+
|
212
|
+
You can also specify the time precision of your check script in the check config with the `time_precision` attribute.
|
213
|
+
|
214
|
+
### Example check config
|
215
|
+
|
216
|
+
`/etc/sensu/conf.d/checks/metrics-load.json`
|
217
|
+
```json
|
218
|
+
{
|
219
|
+
"checks": {
|
220
|
+
"metrics-load": {
|
221
|
+
"type": "metric",
|
222
|
+
"command": "metrics-load.rb",
|
223
|
+
"standalone": true,
|
224
|
+
"handlers": [
|
225
|
+
"metrics"
|
226
|
+
],
|
227
|
+
"interval": 60,
|
228
|
+
"time_precision": "s",
|
229
|
+
"influxdb": {
|
230
|
+
"database": "custom-db",
|
231
|
+
"tags": {
|
232
|
+
"stage": "prod",
|
233
|
+
"region": "eu-west-1"
|
234
|
+
},
|
235
|
+
"templates": {
|
236
|
+
"^load_avg\\..*": "measurement.field"
|
237
|
+
}
|
238
|
+
}
|
239
|
+
}
|
240
|
+
}
|
241
|
+
}
|
242
|
+
```
|
243
|
+
|
244
|
+
_Result_ :
|
245
|
+
```
|
246
|
+
load_avg,stage:prod,region:eu-west-1,host:iprint-test-sa-01.photobox.com one=1.04,duration=0.402 1444816792147
|
247
|
+
load_avg,stage:prod,region:eu-west-1,host:iprint-test-sa-01.photobox.com five=0.86,duration=0.398 1444816792147
|
248
|
+
load_avg,stage:prod,region:eu-west-1,host:iprint-test-sa-01.photobox.com fifteen=0.84,duration=0.375 1444816792147
|
249
|
+
|
250
|
+
* will be sent to -> http://my-influx09.company.com:8086/db/custom-db/series?time_precision=s&u=sensu&p=sensu
|
251
|
+
```
|
252
|
+
|
253
|
+
Without the tags and templates you would get:
|
254
|
+
```
|
255
|
+
load_avg.one,host:iprint-test-sa-01.photobox.com value=1.04,duration=0.402 1444816792147
|
256
|
+
load_avg.five,host:iprint-test-sa-01.photobox.com value=0.86,duration=0.398 1444816792147
|
257
|
+
load_avg.fifteen,host:iprint-test-sa-01.photobox.com value=0.84,duration=0.375 1444816792147
|
258
|
+
```
|
@@ -0,0 +1,85 @@
|
|
1
|
+
require 'rubygems' if RUBY_VERSION < '1.9.0'
|
2
|
+
require 'em-http-request'
|
3
|
+
require 'eventmachine'
|
4
|
+
require 'multi_json'
|
5
|
+
require 'sensu/extension'
|
6
|
+
require 'sensu/extensions/influxdb2/influx_relay'
|
7
|
+
|
8
|
+
module Sensu
|
9
|
+
module Extension
|
10
|
+
class History < Bridge
|
11
|
+
def name
|
12
|
+
definition[:name]
|
13
|
+
end
|
14
|
+
|
15
|
+
def definition
|
16
|
+
{
|
17
|
+
type: 'extension',
|
18
|
+
name: 'history'
|
19
|
+
}
|
20
|
+
end
|
21
|
+
|
22
|
+
def description
|
23
|
+
'Sends check result data to influxdb'
|
24
|
+
end
|
25
|
+
|
26
|
+
def post_init
|
27
|
+
@influx_conf = parse_settings
|
28
|
+
logger.info("InfluxDB history extension initialiazed using #{@influx_conf['base_url']} - Defaults : db=#{@influx_conf['database']} precision=#{@influx_conf['time_precision']}")
|
29
|
+
|
30
|
+
@relay = InfluxRelay.new
|
31
|
+
@relay.init(@influx_conf)
|
32
|
+
|
33
|
+
logger.info("History write buffer initiliazed : buffer flushed every #{@influx_conf['buffer_max_size']} points OR every #{@influx_conf['buffer_max_age']} seconds) ")
|
34
|
+
end
|
35
|
+
|
36
|
+
def run(event_data)
|
37
|
+
if event_data[:check][:type] != 'standard' && event_data[:check][:type] != 'check'
|
38
|
+
yield '', 0
|
39
|
+
return
|
40
|
+
end
|
41
|
+
|
42
|
+
host = event_data[:client][:name]
|
43
|
+
metric = event_data[:check][:name]
|
44
|
+
timestamp = event_data[:check][:executed]
|
45
|
+
value = event_data[:check][:status]
|
46
|
+
output = "#{@influx_conf['scheme']}.checks.#{metric},type=history,host=#{host} value=#{value} #{timestamp}"
|
47
|
+
|
48
|
+
@relay.push(@influx_conf['database'], @influx_conf['time_precision'], output)
|
49
|
+
yield output, 0
|
50
|
+
end
|
51
|
+
|
52
|
+
def stop
|
53
|
+
logger.info('Flushing history buffer before exiting')
|
54
|
+
@relay.flush_buffer
|
55
|
+
true
|
56
|
+
end
|
57
|
+
|
58
|
+
private
|
59
|
+
|
60
|
+
def parse_settings
|
61
|
+
settings = @settings['influxdb']
|
62
|
+
if @settings.include?('history')
|
63
|
+
settings.merge!(@settings['history'])
|
64
|
+
end
|
65
|
+
# default values
|
66
|
+
settings['tags'] ||= {}
|
67
|
+
settings['use_ssl'] ||= false
|
68
|
+
settings['time_precision'] ||= 's'
|
69
|
+
settings['protocol'] = settings['use_ssl'] ? 'https' : 'http'
|
70
|
+
settings['buffer_max_size'] ||= 500
|
71
|
+
settings['buffer_max_age'] ||= 6 # seconds
|
72
|
+
settings['port'] ||= 8086
|
73
|
+
settings['scheme'] ||= 'sensu'
|
74
|
+
settings['base_url'] = "#{settings['protocol']}://#{settings['host']}:#{settings['port']}"
|
75
|
+
return settings
|
76
|
+
rescue => e
|
77
|
+
logger.error("Failed to parse History settings #{e}")
|
78
|
+
end
|
79
|
+
|
80
|
+
def logger
|
81
|
+
Sensu::Logger.get
|
82
|
+
end
|
83
|
+
end
|
84
|
+
end
|
85
|
+
end
|
@@ -0,0 +1,228 @@
|
|
1
|
+
require 'rubygems' if RUBY_VERSION < '1.9.0'
|
2
|
+
require 'em-http-request'
|
3
|
+
require 'eventmachine'
|
4
|
+
require 'multi_json'
|
5
|
+
require 'sensu/extension'
|
6
|
+
require 'sensu/extensions/influxdb2/influx_relay'
|
7
|
+
|
8
|
+
require 'sensu/extensions/influxdb2/version'
|
9
|
+
|
10
|
+
module Sensu
|
11
|
+
module Extension
|
12
|
+
class InfluxDB2 < Handler
|
13
|
+
def name
|
14
|
+
definition[:name]
|
15
|
+
end
|
16
|
+
|
17
|
+
def definition
|
18
|
+
{
|
19
|
+
type: 'extension',
|
20
|
+
name: 'influxdb'
|
21
|
+
}
|
22
|
+
end
|
23
|
+
|
24
|
+
def description
|
25
|
+
'Outputs metrics to InfluxDB'
|
26
|
+
end
|
27
|
+
|
28
|
+
def post_init
|
29
|
+
@influx_conf = parse_settings
|
30
|
+
logger.info("InfluxDB extension initialiazed using #{@influx_conf['base_url']} - Defaults : db=#{@influx_conf['database']} precision=#{@influx_conf['time_precision']}")
|
31
|
+
|
32
|
+
@relay = InfluxRelay.new
|
33
|
+
@relay.init(@influx_conf)
|
34
|
+
|
35
|
+
logger.info("InfluxDB write buffer initiliazed : buffer flushed every #{@influx_conf['buffer_max_size']} points OR every #{@influx_conf['buffer_max_age']} seconds) ")
|
36
|
+
end
|
37
|
+
|
38
|
+
def run(event_data)
|
39
|
+
event = parse_event(event_data)
|
40
|
+
if event[:check][:status] != 0
|
41
|
+
logger.error('Check status is not OK!')
|
42
|
+
yield 'error', event[:check][:status]
|
43
|
+
return
|
44
|
+
end
|
45
|
+
data = {}
|
46
|
+
# init event and check data
|
47
|
+
data[:client] = event[:client][:name]
|
48
|
+
# This will merge : default conf tags < check embedded tags < sensu client/host tag
|
49
|
+
data[:tags] = @influx_conf['tags'].merge(event[:check][:influxdb][:tags]).merge('host' => data[:client])
|
50
|
+
client_tags = event[:client][:tags] || {}
|
51
|
+
check_tags = event[:check][:tags] || {}
|
52
|
+
data[:tags].merge!(client_tags.merge(check_tags))
|
53
|
+
# This will merge : check embedded templaes < default conf templates (check embedded templates will take precedence)
|
54
|
+
data[:templates] = event[:check][:influxdb][:templates] + @influx_conf['templates']
|
55
|
+
data[:filters] = event[:check][:influxdb][:filters].merge(@influx_conf['filters'])
|
56
|
+
event[:check][:influxdb][:database] ||= @influx_conf['database']
|
57
|
+
event[:check][:time_precision] ||= @influx_conf['time_precision']
|
58
|
+
event[:check][:influxdb][:strip_metric] ||= @influx_conf['strip_metric']
|
59
|
+
data[:strip_metric] = event[:check][:influxdb][:strip_metric]
|
60
|
+
data[:duration] = event[:check][:duration]
|
61
|
+
event[:check][:output].split(/\r\n|\n/).each do |line|
|
62
|
+
unless @influx_conf['proxy_mode'] || event[:check][:influxdb][:proxy_mode]
|
63
|
+
data[:line] = line
|
64
|
+
line = parse_line(data)
|
65
|
+
end
|
66
|
+
@relay.push(event[:check][:influxdb][:database], event[:check][:time_precision], line)
|
67
|
+
end
|
68
|
+
yield 'ok', 0
|
69
|
+
end
|
70
|
+
|
71
|
+
def stop
|
72
|
+
logger.info('Flushing InfluxDB buffer before exiting')
|
73
|
+
@relay.flush_buffer
|
74
|
+
true
|
75
|
+
end
|
76
|
+
|
77
|
+
private
|
78
|
+
|
79
|
+
def parse_event(event_data)
|
80
|
+
event = MultiJson.load(event_data, symbolize_keys: true)
|
81
|
+
|
82
|
+
# default values
|
83
|
+
# n, u, ms, s, m, and h (default community plugins use standard epoch date)
|
84
|
+
event[:check][:time_precision] ||= nil
|
85
|
+
event[:check][:influxdb] ||= {}
|
86
|
+
event[:check][:influxdb][:tags] ||= {}
|
87
|
+
event[:check][:influxdb][:templates] ||= {}
|
88
|
+
event[:check][:influxdb][:templates] = h2a(event[:check][:influxdb][:templates])
|
89
|
+
event[:check][:influxdb][:filters] ||= {}
|
90
|
+
event[:check][:influxdb][:database] ||= nil
|
91
|
+
event[:check][:influxdb][:proxy_mode] ||= false
|
92
|
+
return event
|
93
|
+
rescue => e
|
94
|
+
logger.error("Failed to parse event data: #{e}")
|
95
|
+
end
|
96
|
+
|
97
|
+
def h2a(h)
|
98
|
+
return h unless h.is_a?(Hash)
|
99
|
+
x = []
|
100
|
+
h.each do |k, v|
|
101
|
+
x << { k => v }
|
102
|
+
end
|
103
|
+
x
|
104
|
+
end
|
105
|
+
|
106
|
+
def parse_settings
|
107
|
+
settings = @settings['influxdb']
|
108
|
+
|
109
|
+
# default values
|
110
|
+
settings['tags'] ||= {}
|
111
|
+
settings['templates'] ||= {}
|
112
|
+
settings['templates'] = h2a(settings['templates'])
|
113
|
+
settings['filters'] ||= {}
|
114
|
+
settings['strip_metric'] ||= nil
|
115
|
+
settings['use_ssl'] ||= false
|
116
|
+
settings['use_basic_auth'] ||= false
|
117
|
+
settings['proxy_mode'] ||= false
|
118
|
+
settings['debug_relay'] ||= false
|
119
|
+
settings['time_precision'] ||= 's'
|
120
|
+
settings['protocol'] = settings['use_ssl'] ? 'https' : 'http'
|
121
|
+
settings['buffer_max_size'] ||= 500
|
122
|
+
settings['buffer_max_age'] ||= 6 # seconds
|
123
|
+
settings['port'] ||= 8086
|
124
|
+
settings['base_url'] = "#{settings['protocol']}://#{settings['host']}:#{settings['port']}"
|
125
|
+
return settings
|
126
|
+
rescue => e
|
127
|
+
logger.error("Failed to parse InfluxDB settings #{e}")
|
128
|
+
end
|
129
|
+
|
130
|
+
def strip_key(key, strip_metric, hostname)
|
131
|
+
return key if strip_metric.to_s.empty?
|
132
|
+
if strip_metric == 'host'
|
133
|
+
slice_host(key, hostname)
|
134
|
+
elsif strip_metric
|
135
|
+
key.gsub(/^.*#{strip_metric}\.(.*$)/, '\1')
|
136
|
+
end
|
137
|
+
end
|
138
|
+
|
139
|
+
def slice_host(slice, prefix)
|
140
|
+
prefix.chars.zip(slice.chars).each do |char1, char2|
|
141
|
+
break if char1 != char2
|
142
|
+
slice.slice!(char1)
|
143
|
+
end
|
144
|
+
slice.slice!('.') if slice.chars.first == '.'
|
145
|
+
slice
|
146
|
+
end
|
147
|
+
|
148
|
+
def get_name(arr1, arr2, pattern)
|
149
|
+
pos = arr1.index { |s| s =~ /#{pattern}/ }
|
150
|
+
if arr1[pos] =~ /\*$/
|
151
|
+
arr2[pos...arr2.length].join('.')
|
152
|
+
elsif arr1[pos] =~ /\d$/
|
153
|
+
arr2[pos...arr1[pos].scan(/\d/).join.to_i + pos].join('.')
|
154
|
+
else
|
155
|
+
arr2[pos]
|
156
|
+
end
|
157
|
+
end
|
158
|
+
|
159
|
+
def sanitize(str)
|
160
|
+
str.gsub(',', '\,').gsub(/\s/, '\ ').gsub('"', '\"').gsub('\\') { '\\\\' }.delete('*').squeeze('.')
|
161
|
+
end
|
162
|
+
|
163
|
+
def parse_line(event)
|
164
|
+
field_name = 'value'
|
165
|
+
key, value, time = event[:line].split(/\s+/)
|
166
|
+
|
167
|
+
# Apply filters
|
168
|
+
event[:filters].each do |pattern, replacement|
|
169
|
+
key.gsub!(/#{pattern}/, replacement)
|
170
|
+
end
|
171
|
+
|
172
|
+
# Strip metric name
|
173
|
+
key = strip_key(key, event[:strip_metric], event[:client]) unless event[:strip_metric].nil?
|
174
|
+
|
175
|
+
# Sanitize key name
|
176
|
+
key = sanitize(key)
|
177
|
+
|
178
|
+
tags = {}.merge(event[:tags])
|
179
|
+
event[:templates].each do |k, v|
|
180
|
+
pattern = k
|
181
|
+
template = v
|
182
|
+
if k.is_a?(Hash) # Array of Hashes
|
183
|
+
k = k.to_a.flatten
|
184
|
+
pattern = k[0]
|
185
|
+
template = k[1]
|
186
|
+
end
|
187
|
+
next unless key =~ /#{pattern}/
|
188
|
+
template = template.split('.')
|
189
|
+
key = key.split('.')
|
190
|
+
key_tags = if template.last =~ /\*$/ && !(template.last =~ /field/) && !(template.last =~ /measurement/)
|
191
|
+
key[0...template.length - 1] << key[template.length - 1...key.length].join('.')
|
192
|
+
else
|
193
|
+
key[0...template.length]
|
194
|
+
end
|
195
|
+
|
196
|
+
field_name = get_name(template, key, 'field') if template.index { |s| s =~ /field/ }
|
197
|
+
|
198
|
+
key = if template.index { |s| s =~ /measurement/ }
|
199
|
+
get_name(template, key, 'measurement')
|
200
|
+
else
|
201
|
+
key[key_tags.length...key.length]
|
202
|
+
end
|
203
|
+
|
204
|
+
template.each_with_index do |tag, i|
|
205
|
+
unless i >= key_tags.length || tag =~ /field/ || tag =~ /measurement/ || tag == 'void' || tag == 'null' || tag == 'nil'
|
206
|
+
tags.merge!(sanitize(tag) => key_tags[i])
|
207
|
+
end
|
208
|
+
end
|
209
|
+
break
|
210
|
+
end
|
211
|
+
|
212
|
+
# Append tags to measurement
|
213
|
+
tags.each do |tag, val|
|
214
|
+
next if val.to_s.empty? # skips tags without values
|
215
|
+
key += ",#{sanitize(tag.to_s)}=#{sanitize(val.to_s)}"
|
216
|
+
end
|
217
|
+
|
218
|
+
values = "#{field_name}=#{value.to_f}"
|
219
|
+
values += ",duration=#{event[:duration].to_f}" if event[:duration]
|
220
|
+
[key, values, time.to_i].join(' ')
|
221
|
+
end
|
222
|
+
|
223
|
+
def logger
|
224
|
+
Sensu::Logger.get
|
225
|
+
end
|
226
|
+
end
|
227
|
+
end
|
228
|
+
end
|
@@ -0,0 +1,77 @@
|
|
1
|
+
require 'rubygems' if RUBY_VERSION < '1.9.0'
|
2
|
+
require 'em-http-request'
|
3
|
+
require 'eventmachine'
|
4
|
+
|
5
|
+
module Sensu
|
6
|
+
module Extension
|
7
|
+
class InfluxRelay
|
8
|
+
def init(config)
|
9
|
+
@influx_conf = config
|
10
|
+
@buffer = {}
|
11
|
+
@flush_timer = EventMachine::PeriodicTimer.new(@influx_conf['buffer_max_age'].to_i) do
|
12
|
+
unless buffer_size.zero?
|
13
|
+
flush_buffer
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
def flush_buffer
|
19
|
+
logger.info('Flushing Buffer')
|
20
|
+
@buffer.each do |db, tp|
|
21
|
+
tp.each do |p, points|
|
22
|
+
influxdb = EventMachine::HttpRequest.new("#{@influx_conf['base_url']}/write")
|
23
|
+
post_data = {}
|
24
|
+
post_data[:query] = { 'db' => db, 'precision' => p, 'u' => @influx_conf['username'], 'p' => @influx_conf['password'] }
|
25
|
+
post_data[:body] = points.join(" \n")
|
26
|
+
if @influx_conf['use_basic_auth']
|
27
|
+
post_data[:head] = { 'authorization' => [@influx_conf['basic_user'], @influx_conf['basic_pass']] }
|
28
|
+
end
|
29
|
+
result = influxdb.post(post_data)
|
30
|
+
next if @influx_conf.key?(db) && @influx_conf['debug_relay'] == false # this is to avoid the performance impact of checking the response everytime
|
31
|
+
result.callback do
|
32
|
+
if result.response =~ /.*error.*/
|
33
|
+
logger.error("InfluxDB response: #{result.response}")
|
34
|
+
if result.response =~ /.*database not found.*/
|
35
|
+
post_data = {}
|
36
|
+
post_data[:body] = ''
|
37
|
+
post_data[:query] = {
|
38
|
+
'db' => db,
|
39
|
+
'precision' => p,
|
40
|
+
'u' => @influx_conf['username'],
|
41
|
+
'p' => @influx_conf['password'],
|
42
|
+
'q' => "create database #{db}"
|
43
|
+
}
|
44
|
+
if @influx_conf['use_basic_auth']
|
45
|
+
post_data[:head] = { 'authorization' => [@influx_conf['basic_user'], @influx_conf['basic_pass']] }
|
46
|
+
end
|
47
|
+
EventMachine::HttpRequest.new("#{@influx_conf['base_url']}/query").post(post_data)
|
48
|
+
@influx_conf[db] = true
|
49
|
+
end
|
50
|
+
else
|
51
|
+
logger.debug("Written: #{post_data[:body]}")
|
52
|
+
@influx_conf[db] = true
|
53
|
+
end
|
54
|
+
end
|
55
|
+
end
|
56
|
+
@buffer[db] = {}
|
57
|
+
end
|
58
|
+
end
|
59
|
+
|
60
|
+
def buffer_size
|
61
|
+
@buffer.map { |_db, tp| tp.map { |_p, points| points.length }.inject(:+) }.inject(:+) || 0
|
62
|
+
end
|
63
|
+
|
64
|
+
def push(database, time_precision, data)
|
65
|
+
@buffer[database] ||= {}
|
66
|
+
@buffer[database][time_precision] ||= []
|
67
|
+
|
68
|
+
@buffer[database][time_precision].push(data)
|
69
|
+
flush_buffer if buffer_size >= @influx_conf['buffer_max_size']
|
70
|
+
end
|
71
|
+
|
72
|
+
def logger
|
73
|
+
Sensu::Logger.get
|
74
|
+
end
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|
metadata
ADDED
@@ -0,0 +1,219 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: sensu-extensions-influxdb2
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.0.2
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Critical Media
|
8
|
+
- Sensu-Plugins and contributors
|
9
|
+
autorequire:
|
10
|
+
bindir: bin
|
11
|
+
cert_chain: []
|
12
|
+
date: 2017-11-18 00:00:00.000000000 Z
|
13
|
+
dependencies:
|
14
|
+
- !ruby/object:Gem::Dependency
|
15
|
+
name: sensu-extension
|
16
|
+
requirement: !ruby/object:Gem::Requirement
|
17
|
+
requirements:
|
18
|
+
- - ">="
|
19
|
+
- !ruby/object:Gem::Version
|
20
|
+
version: '0'
|
21
|
+
type: :runtime
|
22
|
+
prerelease: false
|
23
|
+
version_requirements: !ruby/object:Gem::Requirement
|
24
|
+
requirements:
|
25
|
+
- - ">="
|
26
|
+
- !ruby/object:Gem::Version
|
27
|
+
version: '0'
|
28
|
+
- !ruby/object:Gem::Dependency
|
29
|
+
name: em-http-request
|
30
|
+
requirement: !ruby/object:Gem::Requirement
|
31
|
+
requirements:
|
32
|
+
- - "~>"
|
33
|
+
- !ruby/object:Gem::Version
|
34
|
+
version: '1.1'
|
35
|
+
type: :runtime
|
36
|
+
prerelease: false
|
37
|
+
version_requirements: !ruby/object:Gem::Requirement
|
38
|
+
requirements:
|
39
|
+
- - "~>"
|
40
|
+
- !ruby/object:Gem::Version
|
41
|
+
version: '1.1'
|
42
|
+
- !ruby/object:Gem::Dependency
|
43
|
+
name: multi_json
|
44
|
+
requirement: !ruby/object:Gem::Requirement
|
45
|
+
requirements:
|
46
|
+
- - ">="
|
47
|
+
- !ruby/object:Gem::Version
|
48
|
+
version: '0'
|
49
|
+
type: :runtime
|
50
|
+
prerelease: false
|
51
|
+
version_requirements: !ruby/object:Gem::Requirement
|
52
|
+
requirements:
|
53
|
+
- - ">="
|
54
|
+
- !ruby/object:Gem::Version
|
55
|
+
version: '0'
|
56
|
+
- !ruby/object:Gem::Dependency
|
57
|
+
name: bundler
|
58
|
+
requirement: !ruby/object:Gem::Requirement
|
59
|
+
requirements:
|
60
|
+
- - "~>"
|
61
|
+
- !ruby/object:Gem::Version
|
62
|
+
version: '1.6'
|
63
|
+
type: :development
|
64
|
+
prerelease: false
|
65
|
+
version_requirements: !ruby/object:Gem::Requirement
|
66
|
+
requirements:
|
67
|
+
- - "~>"
|
68
|
+
- !ruby/object:Gem::Version
|
69
|
+
version: '1.6'
|
70
|
+
- !ruby/object:Gem::Dependency
|
71
|
+
name: rake
|
72
|
+
requirement: !ruby/object:Gem::Requirement
|
73
|
+
requirements:
|
74
|
+
- - ">="
|
75
|
+
- !ruby/object:Gem::Version
|
76
|
+
version: '0'
|
77
|
+
type: :development
|
78
|
+
prerelease: false
|
79
|
+
version_requirements: !ruby/object:Gem::Requirement
|
80
|
+
requirements:
|
81
|
+
- - ">="
|
82
|
+
- !ruby/object:Gem::Version
|
83
|
+
version: '0'
|
84
|
+
- !ruby/object:Gem::Dependency
|
85
|
+
name: rspec
|
86
|
+
requirement: !ruby/object:Gem::Requirement
|
87
|
+
requirements:
|
88
|
+
- - ">="
|
89
|
+
- !ruby/object:Gem::Version
|
90
|
+
version: '0'
|
91
|
+
type: :development
|
92
|
+
prerelease: false
|
93
|
+
version_requirements: !ruby/object:Gem::Requirement
|
94
|
+
requirements:
|
95
|
+
- - ">="
|
96
|
+
- !ruby/object:Gem::Version
|
97
|
+
version: '0'
|
98
|
+
- !ruby/object:Gem::Dependency
|
99
|
+
name: redcarpet
|
100
|
+
requirement: !ruby/object:Gem::Requirement
|
101
|
+
requirements:
|
102
|
+
- - "~>"
|
103
|
+
- !ruby/object:Gem::Version
|
104
|
+
version: '3.2'
|
105
|
+
type: :development
|
106
|
+
prerelease: false
|
107
|
+
version_requirements: !ruby/object:Gem::Requirement
|
108
|
+
requirements:
|
109
|
+
- - "~>"
|
110
|
+
- !ruby/object:Gem::Version
|
111
|
+
version: '3.2'
|
112
|
+
- !ruby/object:Gem::Dependency
|
113
|
+
name: rubocop
|
114
|
+
requirement: !ruby/object:Gem::Requirement
|
115
|
+
requirements:
|
116
|
+
- - "~>"
|
117
|
+
- !ruby/object:Gem::Version
|
118
|
+
version: 0.40.0
|
119
|
+
type: :development
|
120
|
+
prerelease: false
|
121
|
+
version_requirements: !ruby/object:Gem::Requirement
|
122
|
+
requirements:
|
123
|
+
- - "~>"
|
124
|
+
- !ruby/object:Gem::Version
|
125
|
+
version: 0.40.0
|
126
|
+
- !ruby/object:Gem::Dependency
|
127
|
+
name: sensu-logger
|
128
|
+
requirement: !ruby/object:Gem::Requirement
|
129
|
+
requirements:
|
130
|
+
- - ">="
|
131
|
+
- !ruby/object:Gem::Version
|
132
|
+
version: '0'
|
133
|
+
type: :development
|
134
|
+
prerelease: false
|
135
|
+
version_requirements: !ruby/object:Gem::Requirement
|
136
|
+
requirements:
|
137
|
+
- - ">="
|
138
|
+
- !ruby/object:Gem::Version
|
139
|
+
version: '0'
|
140
|
+
- !ruby/object:Gem::Dependency
|
141
|
+
name: sensu-settings
|
142
|
+
requirement: !ruby/object:Gem::Requirement
|
143
|
+
requirements:
|
144
|
+
- - ">="
|
145
|
+
- !ruby/object:Gem::Version
|
146
|
+
version: '0'
|
147
|
+
type: :development
|
148
|
+
prerelease: false
|
149
|
+
version_requirements: !ruby/object:Gem::Requirement
|
150
|
+
requirements:
|
151
|
+
- - ">="
|
152
|
+
- !ruby/object:Gem::Version
|
153
|
+
version: '0'
|
154
|
+
- !ruby/object:Gem::Dependency
|
155
|
+
name: github-markup
|
156
|
+
requirement: !ruby/object:Gem::Requirement
|
157
|
+
requirements:
|
158
|
+
- - "~>"
|
159
|
+
- !ruby/object:Gem::Version
|
160
|
+
version: '1.3'
|
161
|
+
type: :development
|
162
|
+
prerelease: false
|
163
|
+
version_requirements: !ruby/object:Gem::Requirement
|
164
|
+
requirements:
|
165
|
+
- - "~>"
|
166
|
+
- !ruby/object:Gem::Version
|
167
|
+
version: '1.3'
|
168
|
+
- !ruby/object:Gem::Dependency
|
169
|
+
name: yard
|
170
|
+
requirement: !ruby/object:Gem::Requirement
|
171
|
+
requirements:
|
172
|
+
- - "~>"
|
173
|
+
- !ruby/object:Gem::Version
|
174
|
+
version: '0.8'
|
175
|
+
type: :development
|
176
|
+
prerelease: false
|
177
|
+
version_requirements: !ruby/object:Gem::Requirement
|
178
|
+
requirements:
|
179
|
+
- - "~>"
|
180
|
+
- !ruby/object:Gem::Version
|
181
|
+
version: '0.8'
|
182
|
+
description: Extension to get metrics and checks results into InfluxDB
|
183
|
+
email:
|
184
|
+
- "<steve.viola@criticalmedia.com>"
|
185
|
+
- "<sensu-users@googlegroups.com>"
|
186
|
+
executables: []
|
187
|
+
extensions: []
|
188
|
+
extra_rdoc_files: []
|
189
|
+
files:
|
190
|
+
- LICENSE
|
191
|
+
- README.md
|
192
|
+
- lib/sensu/extensions/history.rb
|
193
|
+
- lib/sensu/extensions/influxdb2.rb
|
194
|
+
- lib/sensu/extensions/influxdb2/influx_relay.rb
|
195
|
+
- lib/sensu/extensions/influxdb2/version.rb
|
196
|
+
homepage: https://github.com/criticalmedia/sensu-extensions-influxdb
|
197
|
+
licenses: []
|
198
|
+
metadata: {}
|
199
|
+
post_install_message:
|
200
|
+
rdoc_options: []
|
201
|
+
require_paths:
|
202
|
+
- lib
|
203
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
204
|
+
requirements:
|
205
|
+
- - ">="
|
206
|
+
- !ruby/object:Gem::Version
|
207
|
+
version: '0'
|
208
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
209
|
+
requirements:
|
210
|
+
- - ">="
|
211
|
+
- !ruby/object:Gem::Version
|
212
|
+
version: '0'
|
213
|
+
requirements: []
|
214
|
+
rubyforge_project:
|
215
|
+
rubygems_version: 2.5.2.1
|
216
|
+
signing_key:
|
217
|
+
specification_version: 4
|
218
|
+
summary: Extension to get metrics and checks results into InfluxDB
|
219
|
+
test_files: []
|