sensu-extensions-influxdb2 0.0.2

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 0b0f5ae119ad47adabd692247f89da5e49cb47de
4
+ data.tar.gz: 0ae41118ccc5cc0e25601dcb74100f962bf00e3b
5
+ SHA512:
6
+ metadata.gz: 467a5cdf430eeace8fa6fd22031a6298414bccc34e4060d5dca52f300b9a2132f2d6d95754d0a8623b0e52ec3acc953261772a6862b70123bb7b62cab8efc518
7
+ data.tar.gz: a9f5a6f2640344f25b1ad23e7a7a34e90405fa5c5241992fc50be12520988f584b613dbdd05e4c55a5d02515190b3ca5cf21797926c2a6cf48ee89867932fe31
data/LICENSE ADDED
@@ -0,0 +1,15 @@
1
+ Copyright 2014 John E. Vincent and contributors.
2
+
3
+ Licensed under the Apache License, Version 2.0 (the "License");
4
+ you may not use this file except in compliance with the License.
5
+ You may obtain a copy of the License at
6
+
7
+ http://www.apache.org/licenses/LICENSE-2.0
8
+
9
+ Unless required by applicable law or agreed to in writing, software
10
+ distributed under the License is distributed on an "AS IS" BASIS,
11
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
+ See the License for the specific language governing permissions and
13
+ limitations under the License.
14
+
15
+
@@ -0,0 +1,258 @@
1
+ __NOTE:__ running `sensu-install -e sensu-extensions-influxdb` will __NOT__ install this extension, as a project with the same name already exists on Rubygems.org. Before reporting any issues here, make sure you are using this version of the extension. (_hint:_ check the version number on the logs)
2
+
3
+ # Requirements
4
+
5
+ This extension uses InfluxDB [Line Protocol](https://influxdb.com/docs/v0.9/write_protocols/line.html) over HTTP to send metrics.
6
+
7
+ Since Sensu already uses [eventmachine](https://github.com/eventmachine/eventmachine), you just have to ensure that em-http-request gem is present inside Sensu's embedded Ruby :
8
+ * `em-http-request` ruby gem
9
+
10
+ By default, this extension uses in memory metrics caching : it makes InfluxDB writes batched.
11
+ This cache is flushed every __500 items__ or __6 seconds__ (this values can be changed in conf).
12
+
13
+ ## Caveat
14
+ * SSL/TLS connection to InfluxDB is still broken in Sensu `0.20.6`. We're waiting for a Ruby / EM upgrade in Sensu project :
15
+ * https://github.com/sensu/sensu/issues/1084
16
+
17
+ # Usage
18
+ The configuration options are pretty straight forward.
19
+
20
+ Metrics are inserted into the database using the check's key name as _measurement_ name. So if you're using the `sensu-plugins-load-checks` community plugin :
21
+ ```
22
+ my-host-01.load_avg.one 0.02 1444824197
23
+ my-host-01.load_avg.five 0.04 1444824197
24
+ my-host-01.load_avg.fifteen 0.09 1444824197
25
+ ```
26
+ In this example, you'll have 3 differents _measurements_ in your database :
27
+ ```
28
+ > show measurements
29
+ name: measurements
30
+ ------------------
31
+ name
32
+ my-host-01.load_avg.fifteen
33
+ my-host-01.load_avg.five
34
+ my-host-01.load_avg.one
35
+ ```
36
+
37
+ ```
38
+ > select * from "my-host-01.load_avg.one";
39
+ name: my-host-01.load_avg.one
40
+ ------------------
41
+ time host value duration
42
+ 2015-10-14T13:53:22Z my-host-01 0.34 0.399
43
+ 2015-10-14T13:53:32Z my-host-01 0.29 0.419
44
+ 2015-10-14T13:53:42Z my-host-01 0.39 0.392
45
+ 2015-10-14T13:53:52Z my-host-01 0.41 0.398
46
+ [...]
47
+ ```
48
+
49
+ Additionally a `duration` value will be present based on the time it took the check to run (this is gleaned from the sensu event data).
50
+
51
+ The name of the _measurement_ is based on the value of `strip_metric` and/or a template, as described below.
52
+ The name of the key ```host``` is grabbed from sensu event client name.
53
+
54
+ ## Extension not a handler
55
+ Note that the first push of this was a handler that could be called via `pipe`. This is now an actual extension that's more performant since it's actually in the sensu-server runtime. Additionally it's now using batch submission to InfluxDB by writing all the points for a given series at once.
56
+
57
+ To [load the extension](https://sensuapp.org/docs/latest/reference/extensions.html), you will need to install the gem into the sensu ruby, and then add the following config file
58
+ ```json
59
+ {
60
+ "extensions": {
61
+ "influxdb": {
62
+ "gem": "sensu-extensions-influxdb",
63
+ },
64
+ "history": {
65
+ "gem": "sensu-extensions-history",
66
+ }
67
+ }
68
+ }
69
+ ```
70
+
71
+ **NOTE**: Even though this gem is called `sensu-extensions-influxdb`, the way sensu loads the extension is by parsing the name of the gem, and loading the library. This is why we have to specify `sensu-extensions-history`, even though that gem doesn't exist.
72
+
73
+ Create a set to wrap this extension into a callable handler. In this example, we created a ```metrics``` handler wrapping a debug output and this Influx extension :
74
+
75
+ _/etc/sensu/conf.d/handlers/metrics.json_ :
76
+ ```json
77
+ {
78
+ "handlers": {
79
+ "metrics": {
80
+ "type": "set",
81
+ "handlers": [ "debug", "influxdb"]
82
+ }
83
+ }
84
+ }
85
+ }
86
+ ```
87
+
88
+ _Note :_ Since Sensu 0.17.1 you can also use extension name directly :
89
+ ```
90
+ Check definitions can now specify a Sensu check extension to run,
91
+ "extension", instead of a command.
92
+ ```
93
+
94
+ ## Handler config
95
+
96
+ `/etc/sensu/conf.d/influxdb.json`
97
+ ```json
98
+ {
99
+ "influxdb": {
100
+ "database": "stats",
101
+ "host": "localhost",
102
+ "port": "8086",
103
+ "username": "stats",
104
+ "password": "stats",
105
+ "use_ssl": false,
106
+ "strip_metric": "host",
107
+ "tags": {
108
+ "region": "my-dc-01",
109
+ "stage": "prod"
110
+ },
111
+ "templates": [
112
+ {"^sensu\\.checks\\..*":"source.measurement.field*"},
113
+ {".*\\.cgroup\\..*": "host.path.component"}
114
+ ]
115
+ }
116
+ }
117
+ ```
118
+
119
+ ### Config attributes
120
+
121
+ * `host`, `port`, `username`, `password` and `database` are pretty straight forward. If `use_ssl` is set to true, the connection to the influxdb server will be made using https instead of http.
122
+
123
+ * `use_basic_auth` if your InfluxDB is behind a proxy that requires authentication you can set this to `true`, in that case you also have to define `basic_user` and `basic_pass`.
124
+
125
+ * `filters` allow's you to "find and replace" words/patterns in the key, it's a Hash where the key is a regex pattern and the value is a string that should replace it. (these are passed directly to the gsub method). Filters are applyed before any other transformations like templates.
126
+
127
+ * `templates` like with the InfluxDB Graphite plugin, you can specify patterns and formats, if the metric name matches the pattern, the template will be applied. Templates can be defined per check, in the check configuration, or globaly in the handler's configuration. Parts of the metric name will be converted into tags, reducing the measurement name in InfluxDB whilst keeping all the information as tags or fields.
128
+ You can specifty multiple templates, if your metric matches more than one the first one will be used.
129
+ For example, if you have a metric named:
130
+ ```
131
+ ip-10-0-1-32.host_stats.load_avg 3
132
+ ```
133
+ You can set a template like:
134
+ ```json
135
+ "templates":{
136
+ ".*\\.host_stats\\..*": "host.type"
137
+ }
138
+ ```
139
+ And as a result the following will be sent to InfluxDB:
140
+ ```
141
+ load_avg,host=ip-10-0-1-32,type=host_stats value=3
142
+ ```
143
+ You can also use three special keywords in your templates, `void`, `measurement` and `field` and these can be used with a `*` at the end.
144
+ Example, considereing the following metrics:
145
+ ```
146
+ ip-10-0-1-32.host_stats.load_avg.one 3
147
+ ip-10-0-1-32.host_stats.load_avg.five 9
148
+ ip-10-0-1-32.host_stats.load_avg.fifteen 13
149
+ ```
150
+ You could create a template like this:
151
+ ```json
152
+ "templates":{
153
+ ".*\\.host_stats\\..*": "host.measurement.field*"
154
+ }
155
+ ```
156
+ And you would get:
157
+ ```
158
+ host_stats,host=ip-10-0-1-32 load_avg.one=3
159
+ host_stats,host=ip-10-0-1-32 load_avg.five=9
160
+ host_stats,host=ip-10-0-1-32 load_avg.fifteen=13
161
+ ```
162
+ Or, a template like this:
163
+ ```json
164
+ "templates":{
165
+ ".*\\.host_stats\\..*": "host.measurement.field.aggregation"
166
+ }
167
+ ```
168
+ Would get you:
169
+ ```
170
+ host_stats,host=ip-10-0-1-32,aggregation=one load_avg=3
171
+ host_stats,host=ip-10-0-1-32,aggregation=five load_avg=9
172
+ host_stats,host=ip-10-0-1-32,aggregation=fifteen load_avg=13
173
+ ```
174
+ the `void` keyword will make the handler ignore that part of the metric name.
175
+
176
+ * `tags` hash is also pretty straight forward. Just list here in a flat-hash design as many influxdb tags you wish to be added in your measures. This can also be set both in the handler configuration or as part of the check configuration.
177
+
178
+ * `strip_metric` however might not be. This is used to "clean up" the data sent to influxdb. Normally everything sent to handlers is akin to the `graphite`/`stats` style:
179
+ ```
180
+ something.host.metrictype.foo.bar
181
+ ```
182
+ or
183
+ ```
184
+ host.stats.something.foo.bar
185
+ ```
186
+
187
+ Really the pattern is irrelevant. People have different tastes. Adding much of that data to the column name in InfluxDB is rather silly so `strip_metric` provides you with a chance to add a value that strips off everything up to (and including that value). This allows you to continue sending to graphite or statsd or whatever and still use this handler.
188
+
189
+ Using the examples above, if you set the `strip_metric` to `host`, then the measurement in InfluxDB would be called `metrictype.foo.bar` or `stats.something.foo.bar`. If you set the value to `foo` then the measurement would simply be called `foo`
190
+
191
+ Note that :
192
+ * `strip_metric` isn't required.
193
+ * `strip_metric` can be set either in the handler configuration or as part of the check configuration, check configuration takes precedence.
194
+ * you can cleanup an arbitrary string from your keyname or use `host` as special value to cleanup the sensu event client name from your key.
195
+
196
+ #### Other attributes
197
+
198
+ * `time_precision` : global checks time precision (default is `'s'`)
199
+ * `buffer_max_size` : buffer size limit before flush - This is the amount of points in the InfluxDB batch - (default is `500`)
200
+ * `buffer_max_age` : buffer maximum age - Flush will be forced after this amount of time - (default is `6` seconds)
201
+ * `Proxy mode` : If the extension is configured to be in proxy mode, it will skip the transformation step and assume that the data is valid [line protocol](https://docs.influxdata.com/influxdb/latest/write_protocols/line_protocol_reference). It will not take into account any tags defined in the sensu-configuration.
202
+
203
+ ## Check options
204
+
205
+ In the check config, an optional `influxdb` section can be added, containing a `database` option and `tags`.
206
+ You can also set `proxy_mode` in this section on the checks to override the default configuration set on the handler. This way checks that use the InfluxDB line protocol can coexsist with checks that use the Graphite format and use the same handler.
207
+ As mentioned above you can also specify `strip_metric`, `templates` and `filters`, in the check configuration.
208
+ If specified, this overrides the default `database` and `strip_metric` options in the handler config and adds (or overrides) influxdb tags and templates.
209
+
210
+ This allows events to be written to different influxdb databases and modify key indexes on a check-by-check basis.
211
+
212
+ You can also specify the time precision of your check script in the check config with the `time_precision` attribute.
213
+
214
+ ### Example check config
215
+
216
+ `/etc/sensu/conf.d/checks/metrics-load.json`
217
+ ```json
218
+ {
219
+ "checks": {
220
+ "metrics-load": {
221
+ "type": "metric",
222
+ "command": "metrics-load.rb",
223
+ "standalone": true,
224
+ "handlers": [
225
+ "metrics"
226
+ ],
227
+ "interval": 60,
228
+ "time_precision": "s",
229
+ "influxdb": {
230
+ "database": "custom-db",
231
+ "tags": {
232
+ "stage": "prod",
233
+ "region": "eu-west-1"
234
+ },
235
+ "templates": {
236
+ "^load_avg\\..*": "measurement.field"
237
+ }
238
+ }
239
+ }
240
+ }
241
+ }
242
+ ```
243
+
244
+ _Result_ :
245
+ ```
246
+ load_avg,stage:prod,region:eu-west-1,host:iprint-test-sa-01.photobox.com one=1.04,duration=0.402 1444816792147
247
+ load_avg,stage:prod,region:eu-west-1,host:iprint-test-sa-01.photobox.com five=0.86,duration=0.398 1444816792147
248
+ load_avg,stage:prod,region:eu-west-1,host:iprint-test-sa-01.photobox.com fifteen=0.84,duration=0.375 1444816792147
249
+
250
+ * will be sent to -> http://my-influx09.company.com:8086/db/custom-db/series?time_precision=s&u=sensu&p=sensu
251
+ ```
252
+
253
+ Without the tags and templates you would get:
254
+ ```
255
+ load_avg.one,host:iprint-test-sa-01.photobox.com value=1.04,duration=0.402 1444816792147
256
+ load_avg.five,host:iprint-test-sa-01.photobox.com value=0.86,duration=0.398 1444816792147
257
+ load_avg.fifteen,host:iprint-test-sa-01.photobox.com value=0.84,duration=0.375 1444816792147
258
+ ```
@@ -0,0 +1,85 @@
1
+ require 'rubygems' if RUBY_VERSION < '1.9.0'
2
+ require 'em-http-request'
3
+ require 'eventmachine'
4
+ require 'multi_json'
5
+ require 'sensu/extension'
6
+ require 'sensu/extensions/influxdb2/influx_relay'
7
+
8
+ module Sensu
9
+ module Extension
10
+ class History < Bridge
11
+ def name
12
+ definition[:name]
13
+ end
14
+
15
+ def definition
16
+ {
17
+ type: 'extension',
18
+ name: 'history'
19
+ }
20
+ end
21
+
22
+ def description
23
+ 'Sends check result data to influxdb'
24
+ end
25
+
26
+ def post_init
27
+ @influx_conf = parse_settings
28
+ logger.info("InfluxDB history extension initialiazed using #{@influx_conf['base_url']} - Defaults : db=#{@influx_conf['database']} precision=#{@influx_conf['time_precision']}")
29
+
30
+ @relay = InfluxRelay.new
31
+ @relay.init(@influx_conf)
32
+
33
+ logger.info("History write buffer initiliazed : buffer flushed every #{@influx_conf['buffer_max_size']} points OR every #{@influx_conf['buffer_max_age']} seconds) ")
34
+ end
35
+
36
+ def run(event_data)
37
+ if event_data[:check][:type] != 'standard' && event_data[:check][:type] != 'check'
38
+ yield '', 0
39
+ return
40
+ end
41
+
42
+ host = event_data[:client][:name]
43
+ metric = event_data[:check][:name]
44
+ timestamp = event_data[:check][:executed]
45
+ value = event_data[:check][:status]
46
+ output = "#{@influx_conf['scheme']}.checks.#{metric},type=history,host=#{host} value=#{value} #{timestamp}"
47
+
48
+ @relay.push(@influx_conf['database'], @influx_conf['time_precision'], output)
49
+ yield output, 0
50
+ end
51
+
52
+ def stop
53
+ logger.info('Flushing history buffer before exiting')
54
+ @relay.flush_buffer
55
+ true
56
+ end
57
+
58
+ private
59
+
60
+ def parse_settings
61
+ settings = @settings['influxdb']
62
+ if @settings.include?('history')
63
+ settings.merge!(@settings['history'])
64
+ end
65
+ # default values
66
+ settings['tags'] ||= {}
67
+ settings['use_ssl'] ||= false
68
+ settings['time_precision'] ||= 's'
69
+ settings['protocol'] = settings['use_ssl'] ? 'https' : 'http'
70
+ settings['buffer_max_size'] ||= 500
71
+ settings['buffer_max_age'] ||= 6 # seconds
72
+ settings['port'] ||= 8086
73
+ settings['scheme'] ||= 'sensu'
74
+ settings['base_url'] = "#{settings['protocol']}://#{settings['host']}:#{settings['port']}"
75
+ return settings
76
+ rescue => e
77
+ logger.error("Failed to parse History settings #{e}")
78
+ end
79
+
80
+ def logger
81
+ Sensu::Logger.get
82
+ end
83
+ end
84
+ end
85
+ end
@@ -0,0 +1,228 @@
1
+ require 'rubygems' if RUBY_VERSION < '1.9.0'
2
+ require 'em-http-request'
3
+ require 'eventmachine'
4
+ require 'multi_json'
5
+ require 'sensu/extension'
6
+ require 'sensu/extensions/influxdb2/influx_relay'
7
+
8
+ require 'sensu/extensions/influxdb2/version'
9
+
10
+ module Sensu
11
+ module Extension
12
+ class InfluxDB2 < Handler
13
+ def name
14
+ definition[:name]
15
+ end
16
+
17
+ def definition
18
+ {
19
+ type: 'extension',
20
+ name: 'influxdb'
21
+ }
22
+ end
23
+
24
+ def description
25
+ 'Outputs metrics to InfluxDB'
26
+ end
27
+
28
+ def post_init
29
+ @influx_conf = parse_settings
30
+ logger.info("InfluxDB extension initialiazed using #{@influx_conf['base_url']} - Defaults : db=#{@influx_conf['database']} precision=#{@influx_conf['time_precision']}")
31
+
32
+ @relay = InfluxRelay.new
33
+ @relay.init(@influx_conf)
34
+
35
+ logger.info("InfluxDB write buffer initiliazed : buffer flushed every #{@influx_conf['buffer_max_size']} points OR every #{@influx_conf['buffer_max_age']} seconds) ")
36
+ end
37
+
38
+ def run(event_data)
39
+ event = parse_event(event_data)
40
+ if event[:check][:status] != 0
41
+ logger.error('Check status is not OK!')
42
+ yield 'error', event[:check][:status]
43
+ return
44
+ end
45
+ data = {}
46
+ # init event and check data
47
+ data[:client] = event[:client][:name]
48
+ # This will merge : default conf tags < check embedded tags < sensu client/host tag
49
+ data[:tags] = @influx_conf['tags'].merge(event[:check][:influxdb][:tags]).merge('host' => data[:client])
50
+ client_tags = event[:client][:tags] || {}
51
+ check_tags = event[:check][:tags] || {}
52
+ data[:tags].merge!(client_tags.merge(check_tags))
53
+ # This will merge : check embedded templaes < default conf templates (check embedded templates will take precedence)
54
+ data[:templates] = event[:check][:influxdb][:templates] + @influx_conf['templates']
55
+ data[:filters] = event[:check][:influxdb][:filters].merge(@influx_conf['filters'])
56
+ event[:check][:influxdb][:database] ||= @influx_conf['database']
57
+ event[:check][:time_precision] ||= @influx_conf['time_precision']
58
+ event[:check][:influxdb][:strip_metric] ||= @influx_conf['strip_metric']
59
+ data[:strip_metric] = event[:check][:influxdb][:strip_metric]
60
+ data[:duration] = event[:check][:duration]
61
+ event[:check][:output].split(/\r\n|\n/).each do |line|
62
+ unless @influx_conf['proxy_mode'] || event[:check][:influxdb][:proxy_mode]
63
+ data[:line] = line
64
+ line = parse_line(data)
65
+ end
66
+ @relay.push(event[:check][:influxdb][:database], event[:check][:time_precision], line)
67
+ end
68
+ yield 'ok', 0
69
+ end
70
+
71
+ def stop
72
+ logger.info('Flushing InfluxDB buffer before exiting')
73
+ @relay.flush_buffer
74
+ true
75
+ end
76
+
77
+ private
78
+
79
+ def parse_event(event_data)
80
+ event = MultiJson.load(event_data, symbolize_keys: true)
81
+
82
+ # default values
83
+ # n, u, ms, s, m, and h (default community plugins use standard epoch date)
84
+ event[:check][:time_precision] ||= nil
85
+ event[:check][:influxdb] ||= {}
86
+ event[:check][:influxdb][:tags] ||= {}
87
+ event[:check][:influxdb][:templates] ||= {}
88
+ event[:check][:influxdb][:templates] = h2a(event[:check][:influxdb][:templates])
89
+ event[:check][:influxdb][:filters] ||= {}
90
+ event[:check][:influxdb][:database] ||= nil
91
+ event[:check][:influxdb][:proxy_mode] ||= false
92
+ return event
93
+ rescue => e
94
+ logger.error("Failed to parse event data: #{e}")
95
+ end
96
+
97
+ def h2a(h)
98
+ return h unless h.is_a?(Hash)
99
+ x = []
100
+ h.each do |k, v|
101
+ x << { k => v }
102
+ end
103
+ x
104
+ end
105
+
106
+ def parse_settings
107
+ settings = @settings['influxdb']
108
+
109
+ # default values
110
+ settings['tags'] ||= {}
111
+ settings['templates'] ||= {}
112
+ settings['templates'] = h2a(settings['templates'])
113
+ settings['filters'] ||= {}
114
+ settings['strip_metric'] ||= nil
115
+ settings['use_ssl'] ||= false
116
+ settings['use_basic_auth'] ||= false
117
+ settings['proxy_mode'] ||= false
118
+ settings['debug_relay'] ||= false
119
+ settings['time_precision'] ||= 's'
120
+ settings['protocol'] = settings['use_ssl'] ? 'https' : 'http'
121
+ settings['buffer_max_size'] ||= 500
122
+ settings['buffer_max_age'] ||= 6 # seconds
123
+ settings['port'] ||= 8086
124
+ settings['base_url'] = "#{settings['protocol']}://#{settings['host']}:#{settings['port']}"
125
+ return settings
126
+ rescue => e
127
+ logger.error("Failed to parse InfluxDB settings #{e}")
128
+ end
129
+
130
+ def strip_key(key, strip_metric, hostname)
131
+ return key if strip_metric.to_s.empty?
132
+ if strip_metric == 'host'
133
+ slice_host(key, hostname)
134
+ elsif strip_metric
135
+ key.gsub(/^.*#{strip_metric}\.(.*$)/, '\1')
136
+ end
137
+ end
138
+
139
+ def slice_host(slice, prefix)
140
+ prefix.chars.zip(slice.chars).each do |char1, char2|
141
+ break if char1 != char2
142
+ slice.slice!(char1)
143
+ end
144
+ slice.slice!('.') if slice.chars.first == '.'
145
+ slice
146
+ end
147
+
148
+ def get_name(arr1, arr2, pattern)
149
+ pos = arr1.index { |s| s =~ /#{pattern}/ }
150
+ if arr1[pos] =~ /\*$/
151
+ arr2[pos...arr2.length].join('.')
152
+ elsif arr1[pos] =~ /\d$/
153
+ arr2[pos...arr1[pos].scan(/\d/).join.to_i + pos].join('.')
154
+ else
155
+ arr2[pos]
156
+ end
157
+ end
158
+
159
+ def sanitize(str)
160
+ str.gsub(',', '\,').gsub(/\s/, '\ ').gsub('"', '\"').gsub('\\') { '\\\\' }.delete('*').squeeze('.')
161
+ end
162
+
163
+ def parse_line(event)
164
+ field_name = 'value'
165
+ key, value, time = event[:line].split(/\s+/)
166
+
167
+ # Apply filters
168
+ event[:filters].each do |pattern, replacement|
169
+ key.gsub!(/#{pattern}/, replacement)
170
+ end
171
+
172
+ # Strip metric name
173
+ key = strip_key(key, event[:strip_metric], event[:client]) unless event[:strip_metric].nil?
174
+
175
+ # Sanitize key name
176
+ key = sanitize(key)
177
+
178
+ tags = {}.merge(event[:tags])
179
+ event[:templates].each do |k, v|
180
+ pattern = k
181
+ template = v
182
+ if k.is_a?(Hash) # Array of Hashes
183
+ k = k.to_a.flatten
184
+ pattern = k[0]
185
+ template = k[1]
186
+ end
187
+ next unless key =~ /#{pattern}/
188
+ template = template.split('.')
189
+ key = key.split('.')
190
+ key_tags = if template.last =~ /\*$/ && !(template.last =~ /field/) && !(template.last =~ /measurement/)
191
+ key[0...template.length - 1] << key[template.length - 1...key.length].join('.')
192
+ else
193
+ key[0...template.length]
194
+ end
195
+
196
+ field_name = get_name(template, key, 'field') if template.index { |s| s =~ /field/ }
197
+
198
+ key = if template.index { |s| s =~ /measurement/ }
199
+ get_name(template, key, 'measurement')
200
+ else
201
+ key[key_tags.length...key.length]
202
+ end
203
+
204
+ template.each_with_index do |tag, i|
205
+ unless i >= key_tags.length || tag =~ /field/ || tag =~ /measurement/ || tag == 'void' || tag == 'null' || tag == 'nil'
206
+ tags.merge!(sanitize(tag) => key_tags[i])
207
+ end
208
+ end
209
+ break
210
+ end
211
+
212
+ # Append tags to measurement
213
+ tags.each do |tag, val|
214
+ next if val.to_s.empty? # skips tags without values
215
+ key += ",#{sanitize(tag.to_s)}=#{sanitize(val.to_s)}"
216
+ end
217
+
218
+ values = "#{field_name}=#{value.to_f}"
219
+ values += ",duration=#{event[:duration].to_f}" if event[:duration]
220
+ [key, values, time.to_i].join(' ')
221
+ end
222
+
223
+ def logger
224
+ Sensu::Logger.get
225
+ end
226
+ end
227
+ end
228
+ end
@@ -0,0 +1,77 @@
1
+ require 'rubygems' if RUBY_VERSION < '1.9.0'
2
+ require 'em-http-request'
3
+ require 'eventmachine'
4
+
5
+ module Sensu
6
+ module Extension
7
+ class InfluxRelay
8
+ def init(config)
9
+ @influx_conf = config
10
+ @buffer = {}
11
+ @flush_timer = EventMachine::PeriodicTimer.new(@influx_conf['buffer_max_age'].to_i) do
12
+ unless buffer_size.zero?
13
+ flush_buffer
14
+ end
15
+ end
16
+ end
17
+
18
+ def flush_buffer
19
+ logger.info('Flushing Buffer')
20
+ @buffer.each do |db, tp|
21
+ tp.each do |p, points|
22
+ influxdb = EventMachine::HttpRequest.new("#{@influx_conf['base_url']}/write")
23
+ post_data = {}
24
+ post_data[:query] = { 'db' => db, 'precision' => p, 'u' => @influx_conf['username'], 'p' => @influx_conf['password'] }
25
+ post_data[:body] = points.join(" \n")
26
+ if @influx_conf['use_basic_auth']
27
+ post_data[:head] = { 'authorization' => [@influx_conf['basic_user'], @influx_conf['basic_pass']] }
28
+ end
29
+ result = influxdb.post(post_data)
30
+ next if @influx_conf.key?(db) && @influx_conf['debug_relay'] == false # this is to avoid the performance impact of checking the response everytime
31
+ result.callback do
32
+ if result.response =~ /.*error.*/
33
+ logger.error("InfluxDB response: #{result.response}")
34
+ if result.response =~ /.*database not found.*/
35
+ post_data = {}
36
+ post_data[:body] = ''
37
+ post_data[:query] = {
38
+ 'db' => db,
39
+ 'precision' => p,
40
+ 'u' => @influx_conf['username'],
41
+ 'p' => @influx_conf['password'],
42
+ 'q' => "create database #{db}"
43
+ }
44
+ if @influx_conf['use_basic_auth']
45
+ post_data[:head] = { 'authorization' => [@influx_conf['basic_user'], @influx_conf['basic_pass']] }
46
+ end
47
+ EventMachine::HttpRequest.new("#{@influx_conf['base_url']}/query").post(post_data)
48
+ @influx_conf[db] = true
49
+ end
50
+ else
51
+ logger.debug("Written: #{post_data[:body]}")
52
+ @influx_conf[db] = true
53
+ end
54
+ end
55
+ end
56
+ @buffer[db] = {}
57
+ end
58
+ end
59
+
60
+ def buffer_size
61
+ @buffer.map { |_db, tp| tp.map { |_p, points| points.length }.inject(:+) }.inject(:+) || 0
62
+ end
63
+
64
+ def push(database, time_precision, data)
65
+ @buffer[database] ||= {}
66
+ @buffer[database][time_precision] ||= []
67
+
68
+ @buffer[database][time_precision].push(data)
69
+ flush_buffer if buffer_size >= @influx_conf['buffer_max_size']
70
+ end
71
+
72
+ def logger
73
+ Sensu::Logger.get
74
+ end
75
+ end
76
+ end
77
+ end
@@ -0,0 +1,10 @@
1
+
2
+ module SensuExtensionsInfluxDB2
3
+ module Version
4
+ MAJOR = 0
5
+ MINOR = 0
6
+ PATCH = 2
7
+
8
+ VER_STRING = [MAJOR, MINOR, PATCH].compact.join('.')
9
+ end
10
+ end
metadata ADDED
@@ -0,0 +1,219 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: sensu-extensions-influxdb2
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.2
5
+ platform: ruby
6
+ authors:
7
+ - Critical Media
8
+ - Sensu-Plugins and contributors
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2017-11-18 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: sensu-extension
16
+ requirement: !ruby/object:Gem::Requirement
17
+ requirements:
18
+ - - ">="
19
+ - !ruby/object:Gem::Version
20
+ version: '0'
21
+ type: :runtime
22
+ prerelease: false
23
+ version_requirements: !ruby/object:Gem::Requirement
24
+ requirements:
25
+ - - ">="
26
+ - !ruby/object:Gem::Version
27
+ version: '0'
28
+ - !ruby/object:Gem::Dependency
29
+ name: em-http-request
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - "~>"
33
+ - !ruby/object:Gem::Version
34
+ version: '1.1'
35
+ type: :runtime
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - "~>"
40
+ - !ruby/object:Gem::Version
41
+ version: '1.1'
42
+ - !ruby/object:Gem::Dependency
43
+ name: multi_json
44
+ requirement: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - ">="
47
+ - !ruby/object:Gem::Version
48
+ version: '0'
49
+ type: :runtime
50
+ prerelease: false
51
+ version_requirements: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - ">="
54
+ - !ruby/object:Gem::Version
55
+ version: '0'
56
+ - !ruby/object:Gem::Dependency
57
+ name: bundler
58
+ requirement: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - "~>"
61
+ - !ruby/object:Gem::Version
62
+ version: '1.6'
63
+ type: :development
64
+ prerelease: false
65
+ version_requirements: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - "~>"
68
+ - !ruby/object:Gem::Version
69
+ version: '1.6'
70
+ - !ruby/object:Gem::Dependency
71
+ name: rake
72
+ requirement: !ruby/object:Gem::Requirement
73
+ requirements:
74
+ - - ">="
75
+ - !ruby/object:Gem::Version
76
+ version: '0'
77
+ type: :development
78
+ prerelease: false
79
+ version_requirements: !ruby/object:Gem::Requirement
80
+ requirements:
81
+ - - ">="
82
+ - !ruby/object:Gem::Version
83
+ version: '0'
84
+ - !ruby/object:Gem::Dependency
85
+ name: rspec
86
+ requirement: !ruby/object:Gem::Requirement
87
+ requirements:
88
+ - - ">="
89
+ - !ruby/object:Gem::Version
90
+ version: '0'
91
+ type: :development
92
+ prerelease: false
93
+ version_requirements: !ruby/object:Gem::Requirement
94
+ requirements:
95
+ - - ">="
96
+ - !ruby/object:Gem::Version
97
+ version: '0'
98
+ - !ruby/object:Gem::Dependency
99
+ name: redcarpet
100
+ requirement: !ruby/object:Gem::Requirement
101
+ requirements:
102
+ - - "~>"
103
+ - !ruby/object:Gem::Version
104
+ version: '3.2'
105
+ type: :development
106
+ prerelease: false
107
+ version_requirements: !ruby/object:Gem::Requirement
108
+ requirements:
109
+ - - "~>"
110
+ - !ruby/object:Gem::Version
111
+ version: '3.2'
112
+ - !ruby/object:Gem::Dependency
113
+ name: rubocop
114
+ requirement: !ruby/object:Gem::Requirement
115
+ requirements:
116
+ - - "~>"
117
+ - !ruby/object:Gem::Version
118
+ version: 0.40.0
119
+ type: :development
120
+ prerelease: false
121
+ version_requirements: !ruby/object:Gem::Requirement
122
+ requirements:
123
+ - - "~>"
124
+ - !ruby/object:Gem::Version
125
+ version: 0.40.0
126
+ - !ruby/object:Gem::Dependency
127
+ name: sensu-logger
128
+ requirement: !ruby/object:Gem::Requirement
129
+ requirements:
130
+ - - ">="
131
+ - !ruby/object:Gem::Version
132
+ version: '0'
133
+ type: :development
134
+ prerelease: false
135
+ version_requirements: !ruby/object:Gem::Requirement
136
+ requirements:
137
+ - - ">="
138
+ - !ruby/object:Gem::Version
139
+ version: '0'
140
+ - !ruby/object:Gem::Dependency
141
+ name: sensu-settings
142
+ requirement: !ruby/object:Gem::Requirement
143
+ requirements:
144
+ - - ">="
145
+ - !ruby/object:Gem::Version
146
+ version: '0'
147
+ type: :development
148
+ prerelease: false
149
+ version_requirements: !ruby/object:Gem::Requirement
150
+ requirements:
151
+ - - ">="
152
+ - !ruby/object:Gem::Version
153
+ version: '0'
154
+ - !ruby/object:Gem::Dependency
155
+ name: github-markup
156
+ requirement: !ruby/object:Gem::Requirement
157
+ requirements:
158
+ - - "~>"
159
+ - !ruby/object:Gem::Version
160
+ version: '1.3'
161
+ type: :development
162
+ prerelease: false
163
+ version_requirements: !ruby/object:Gem::Requirement
164
+ requirements:
165
+ - - "~>"
166
+ - !ruby/object:Gem::Version
167
+ version: '1.3'
168
+ - !ruby/object:Gem::Dependency
169
+ name: yard
170
+ requirement: !ruby/object:Gem::Requirement
171
+ requirements:
172
+ - - "~>"
173
+ - !ruby/object:Gem::Version
174
+ version: '0.8'
175
+ type: :development
176
+ prerelease: false
177
+ version_requirements: !ruby/object:Gem::Requirement
178
+ requirements:
179
+ - - "~>"
180
+ - !ruby/object:Gem::Version
181
+ version: '0.8'
182
+ description: Extension to get metrics and checks results into InfluxDB
183
+ email:
184
+ - "<steve.viola@criticalmedia.com>"
185
+ - "<sensu-users@googlegroups.com>"
186
+ executables: []
187
+ extensions: []
188
+ extra_rdoc_files: []
189
+ files:
190
+ - LICENSE
191
+ - README.md
192
+ - lib/sensu/extensions/history.rb
193
+ - lib/sensu/extensions/influxdb2.rb
194
+ - lib/sensu/extensions/influxdb2/influx_relay.rb
195
+ - lib/sensu/extensions/influxdb2/version.rb
196
+ homepage: https://github.com/criticalmedia/sensu-extensions-influxdb
197
+ licenses: []
198
+ metadata: {}
199
+ post_install_message:
200
+ rdoc_options: []
201
+ require_paths:
202
+ - lib
203
+ required_ruby_version: !ruby/object:Gem::Requirement
204
+ requirements:
205
+ - - ">="
206
+ - !ruby/object:Gem::Version
207
+ version: '0'
208
+ required_rubygems_version: !ruby/object:Gem::Requirement
209
+ requirements:
210
+ - - ">="
211
+ - !ruby/object:Gem::Version
212
+ version: '0'
213
+ requirements: []
214
+ rubyforge_project:
215
+ rubygems_version: 2.5.2.1
216
+ signing_key:
217
+ specification_version: 4
218
+ summary: Extension to get metrics and checks results into InfluxDB
219
+ test_files: []