fluent-plugin-sumologic_output 1.0.3 → 1.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +57 -0
- data/Gemfile +6 -1
- data/README.md +29 -1
- data/fluent-plugin-sumologic_output.gemspec +1 -2
- data/lib/fluent/plugin/out_sumologic.rb +58 -16
- data/test/helper.rb +8 -2
- data/test/plugin/test_out_sumologic.rb +256 -28
- metadata +3 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 8fd43e4a8259784985b70eaac09c43346525f61d08c0383912f6434887a9aa03
|
4
|
+
data.tar.gz: 7f8a58eee2a297ca40cb7d599371040bf7225632de7dbc561710050e01d692a0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e6436be0d5af2de4521319e7bee494cae41fc4b2052a54256317268ca02c2873bfc48060755ca7b361e21e69f633cca1a38aa255449ff72b21c0dfe2451b50ef
|
7
|
+
data.tar.gz: 5ee85c23b49b9608754e76342508ce9d4a7df0f1a35b4a82f8686b9eafb7264f8c09d1c736c8975b5fb61fa420b9efcb016bb6790aeb6f30f54c8b5e12a73500
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,57 @@
|
|
1
|
+
# Change Log
|
2
|
+
|
3
|
+
All notable changes to this project will be documented in this file. Tracking did not begin until version 1.10.
|
4
|
+
|
5
|
+
<a name="1.1.0"></a>
|
6
|
+
# [1.1.0] (2018-06-29)
|
7
|
+
|
8
|
+
* Add support for sending metrics.
|
9
|
+
|
10
|
+
<a name="1.0.3"></a>
|
11
|
+
# [1.0.3] (2018-05-07)
|
12
|
+
|
13
|
+
* [Fix #26 -- Don't chomp if the message is not chomp-able](https://github.com/SumoLogic/fluentd-output-sumologic/pull/29)
|
14
|
+
|
15
|
+
<a name="1.0.2"></a>
|
16
|
+
# [1.0.2] (2018-04-09)
|
17
|
+
|
18
|
+
* [add option to turn off adding timestamp to logs](https://github.com/SumoLogic/fluentd-output-sumologic/pull/27)
|
19
|
+
|
20
|
+
<a name="1.0.1"></a>
|
21
|
+
# [1.0.1] (2017-12-19)
|
22
|
+
|
23
|
+
* [Add client header for fluentd output](https://github.com/SumoLogic/fluentd-output-sumologic/pull/22)
|
24
|
+
|
25
|
+
<a name="1.0.0"></a>
|
26
|
+
# [1.0.0] (2017-11-06)
|
27
|
+
|
28
|
+
* [Upgrade to 0.14 API, send with ms precision](https://github.com/SumoLogic/fluentd-output-sumologic/pull/12)
|
29
|
+
* [Switch to httpclient](https://github.com/SumoLogic/fluentd-output-sumologic/pull/16)
|
30
|
+
* [Fix missing variable and improve config example](https://github.com/SumoLogic/fluentd-output-sumologic/pull/17)
|
31
|
+
|
32
|
+
<a name="0.0.7"></a>
|
33
|
+
# [0.0.7] (2017-10-26)
|
34
|
+
|
35
|
+
* [Expand parameters in the output configuration](https://github.com/SumoLogic/fluentd-output-sumologic/pull/14)
|
36
|
+
* [add open_timeout option](https://github.com/SumoLogic/fluentd-output-sumologic/pull/15)
|
37
|
+
|
38
|
+
<a name="0.0.6"></a>
|
39
|
+
# [0.0.6] (2017-08-23)
|
40
|
+
|
41
|
+
* Fix 0.0.5
|
42
|
+
|
43
|
+
<a name="0.0.5"></a>
|
44
|
+
# [0.0.5] (2017-08-18)
|
45
|
+
|
46
|
+
* [Ignore garbage records. Fix inspired by other plugins](https://github.com/SumoLogic/fluentd-output-sumologic/pull/7)
|
47
|
+
* [Extract the source_name from FluentD's buffer](https://github.com/SumoLogic/fluentd-output-sumologic/pull/8)
|
48
|
+
|
49
|
+
<a name="0.0.4"></a>
|
50
|
+
# [0.0.4] (2017-07-05)
|
51
|
+
|
52
|
+
* [Raise an exception for all non HTTP Success response codes](https://github.com/SumoLogic/fluentd-output-sumologic/pull/5)
|
53
|
+
|
54
|
+
<a name="0.0.3"></a>
|
55
|
+
# [0.0.3] (2016-12-10)
|
56
|
+
|
57
|
+
* Initial Release
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# fluent-plugin-sumologic_output, a plugin for [Fluentd](http://fluentd.org)
|
2
2
|
|
3
|
-
This plugin has been designed to output logs to [SumoLogic](http://www.sumologic.com) via a [HTTP collector endpoint](http://help.sumologic.com/Send_Data/Sources/02Sources_for_Hosted_Collectors/HTTP_Source)
|
3
|
+
This plugin has been designed to output logs or metrics to [SumoLogic](http://www.sumologic.com) via a [HTTP collector endpoint](http://help.sumologic.com/Send_Data/Sources/02Sources_for_Hosted_Collectors/HTTP_Source)
|
4
4
|
|
5
5
|
| TLS Deprecation Notice |
|
6
6
|
| --- |
|
@@ -17,6 +17,7 @@ The code in this repository has been contributed by the Sumo Logic community and
|
|
17
17
|
|
18
18
|
Configuration options for fluent.conf are:
|
19
19
|
|
20
|
+
* `data_type` - The type of data that will be sent to Sumo Logic, either `logs` or `metrics` (Default is `logs `)
|
20
21
|
* `endpoint` - SumoLogic HTTP Collector URL
|
21
22
|
* `verify_ssl` - Verify ssl certificate. (default is `true`)
|
22
23
|
* `source_category` - Set _sourceCategory metadata field within SumoLogic (default is `nil`)
|
@@ -31,7 +32,9 @@ Configuration options for fluent.conf are:
|
|
31
32
|
* `open_timeout` - Set timeout seconds to wait until connection is opened.
|
32
33
|
* `add_timestamp` - Add `timestamp` field to logs before sending to sumologic (default `true`)
|
33
34
|
* `proxy_uri` - Add the `uri` of the `proxy` environment if present.
|
35
|
+
* `metric_data_format` - The format of metrics you will be sending, either `graphite` or `carbon2` (Default is `graphite `)
|
34
36
|
|
37
|
+
### Example Configuration
|
35
38
|
Reading from the JSON formatted log files with `in_tail` and wildcard filenames:
|
36
39
|
```
|
37
40
|
<source>
|
@@ -55,6 +58,31 @@ Reading from the JSON formatted log files with `in_tail` and wildcard filenames:
|
|
55
58
|
</match>
|
56
59
|
```
|
57
60
|
|
61
|
+
Sending metrics to Sumo Logic using `in_http`:
|
62
|
+
```
|
63
|
+
<source>
|
64
|
+
@type http
|
65
|
+
port 8888
|
66
|
+
bind 0.0.0.0
|
67
|
+
</source>
|
68
|
+
|
69
|
+
<match test.carbon2>
|
70
|
+
@type sumologic
|
71
|
+
endpoint https://endpoint3.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV1hYfCAiqSH-PDY6gUOIgZvO60U_-y8SPQfK0Ks-ht7owrbk1AkX_ACp0uUxuLZOCw5QjBg1ndVPZ5TOJCFgNGRtFDoTDuQ2hzs3sn6FlfBSw==
|
72
|
+
data_type metrics
|
73
|
+
metric_data_format carbon2
|
74
|
+
flush_interval 1s
|
75
|
+
</match>
|
76
|
+
|
77
|
+
<match test.graphite>
|
78
|
+
@type sumologic
|
79
|
+
endpoint https://endpoint3.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV1hYfCAiqSH-PDY6gUOIgZvO60U_-y8SPQfK0Ks-ht7owrbk1AkX_ACp0uUxuLZOCw5QjBg1ndVPZ5TOJCFgNGRtFDoTDuQ2hzs3sn6FlfBSw==
|
80
|
+
data_type metrics
|
81
|
+
metric_data_format graphite
|
82
|
+
flush_interval 1s
|
83
|
+
</match>
|
84
|
+
```
|
85
|
+
|
58
86
|
## Example input/output
|
59
87
|
|
60
88
|
Assuming following inputs are coming from a log file named `/var/log/appa_webserver.log`
|
@@ -4,7 +4,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
4
4
|
|
5
5
|
Gem::Specification.new do |gem|
|
6
6
|
gem.name = "fluent-plugin-sumologic_output"
|
7
|
-
gem.version = "1.0
|
7
|
+
gem.version = "1.1.0"
|
8
8
|
gem.authors = ["Steven Adams", "Frank Reno"]
|
9
9
|
gem.email = ["stevezau@gmail.com", "frank.reno@me.com"]
|
10
10
|
gem.description = %q{Output plugin to SumoLogic HTTP Endpoint}
|
@@ -24,6 +24,5 @@ Gem::Specification.new do |gem|
|
|
24
24
|
gem.add_development_dependency "rake"
|
25
25
|
gem.add_runtime_dependency "fluentd", ">= 0.14.0"
|
26
26
|
gem.add_development_dependency 'test-unit', '~> 3.1.0'
|
27
|
-
|
28
27
|
gem.add_runtime_dependency 'httpclient', '~> 2.8.0'
|
29
28
|
end
|
@@ -1,4 +1,3 @@
|
|
1
|
-
|
2
1
|
require 'fluent/plugin/output'
|
3
2
|
require 'net/https'
|
4
3
|
require 'yajl'
|
@@ -13,22 +12,31 @@ class SumologicConnection
|
|
13
12
|
create_http_client(verify_ssl, connect_timeout, proxy_uri)
|
14
13
|
end
|
15
14
|
|
16
|
-
def publish(raw_data, source_host=nil, source_category=nil, source_name=nil)
|
17
|
-
response = http.post(@endpoint, raw_data, request_headers(source_host, source_category, source_name))
|
15
|
+
def publish(raw_data, source_host=nil, source_category=nil, source_name=nil, data_type, metric_data_type)
|
16
|
+
response = http.post(@endpoint, raw_data, request_headers(source_host, source_category, source_name, data_type, metric_data_type))
|
18
17
|
unless response.ok?
|
19
|
-
raise "Failed to send data to HTTP Source. #{response.code} - #{response.body}"
|
18
|
+
raise RuntimeError, "Failed to send data to HTTP Source. #{response.code} - #{response.body}"
|
20
19
|
end
|
21
20
|
end
|
22
21
|
|
23
|
-
|
24
|
-
|
25
|
-
def request_headers(source_host, source_category, source_name)
|
26
|
-
{
|
22
|
+
def request_headers(source_host, source_category, source_name, data_type, metric_data_format)
|
23
|
+
headers = {
|
27
24
|
'X-Sumo-Name' => source_name,
|
28
25
|
'X-Sumo-Category' => source_category,
|
29
26
|
'X-Sumo-Host' => source_host,
|
30
27
|
'X-Sumo-Client' => 'fluentd-output'
|
31
28
|
}
|
29
|
+
if data_type == 'metrics'
|
30
|
+
case metric_data_format
|
31
|
+
when 'graphite'
|
32
|
+
headers['Content-Type'] = 'application/vnd.sumologic.graphite'
|
33
|
+
when 'carbon2'
|
34
|
+
headers['Content-Type'] = 'application/vnd.sumologic.carbon2'
|
35
|
+
else
|
36
|
+
raise RuntimeError, "Invalid #{metric_data_format}, must be graphite or carbon2"
|
37
|
+
end
|
38
|
+
end
|
39
|
+
return headers
|
32
40
|
end
|
33
41
|
|
34
42
|
def ssl_options(verify_ssl)
|
@@ -49,7 +57,15 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
49
57
|
|
50
58
|
helpers :compat_parameters
|
51
59
|
DEFAULT_BUFFER_TYPE = "memory"
|
52
|
-
|
60
|
+
LOGS_DATA_TYPE = "logs"
|
61
|
+
METRICS_DATA_TYPE = "metrics"
|
62
|
+
DEFAULT_DATA_TYPE = LOGS_DATA_TYPE
|
63
|
+
GRAPHITE_METRIC_FORMAT_TYPE = "graphite"
|
64
|
+
CARBON2_METRIC_FORMAT_TYPE = "carbon2"
|
65
|
+
DEFAULT_METRIC_FORMAT_TYPE = CARBON2_METRIC_FORMAT_TYPE
|
66
|
+
|
67
|
+
config_param :data_type, :string, :default => DEFAULT_DATA_TYPE
|
68
|
+
config_param :metric_data_format, :default => DEFAULT_METRIC_FORMAT_TYPE
|
53
69
|
config_param :endpoint, :string
|
54
70
|
config_param :log_format, :string, :default => 'json'
|
55
71
|
config_param :log_key, :string, :default => 'message'
|
@@ -81,8 +97,24 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
81
97
|
raise Fluent::ConfigError, "Invalid SumoLogic endpoint url: #{conf['endpoint']}"
|
82
98
|
end
|
83
99
|
|
84
|
-
unless conf['
|
85
|
-
|
100
|
+
unless conf['data_type'].nil?
|
101
|
+
unless conf['data_type'] =~ /\A(?:logs|metrics)\z/
|
102
|
+
raise Fluent::ConfigError, "Invalid data_type #{conf['data_type']} must be logs or metrics"
|
103
|
+
end
|
104
|
+
end
|
105
|
+
|
106
|
+
if conf['data_type'].nil? || conf['data_type'] == LOGS_DATA_TYPE
|
107
|
+
unless conf['log_format'].nil?
|
108
|
+
unless conf['log_format'] =~ /\A(?:json|text|json_merge)\z/
|
109
|
+
raise Fluent::ConfigError, "Invalid log_format #{conf['log_format']} must be text, json or json_merge"
|
110
|
+
end
|
111
|
+
end
|
112
|
+
end
|
113
|
+
|
114
|
+
if conf['data_type'] == METRICS_DATA_TYPE && ! conf['metrics_data_type'].nil?
|
115
|
+
unless conf['metrics_data_type'] =~ /\A(?:graphite|carbon2)\z/
|
116
|
+
raise Fluent::ConfigError, "Invalid metrics_data_type #{conf['metrics_data_type']} must be graphite or carbon2"
|
117
|
+
end
|
86
118
|
end
|
87
119
|
|
88
120
|
@sumo_conn = SumologicConnection.new(conf['endpoint'], conf['verify_ssl'], conf['open_timeout'].to_i, conf['proxy_uri'])
|
@@ -191,14 +223,16 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
191
223
|
# plugin dies randomly
|
192
224
|
# https://github.com/uken/fluent-plugin-elasticsearch/commit/8597b5d1faf34dd1f1523bfec45852d380b26601#diff-ae62a005780cc730c558e3e4f47cc544R94
|
193
225
|
next unless record.is_a? Hash
|
194
|
-
sumo_metadata = record.fetch('_sumo_metadata', {
|
226
|
+
sumo_metadata = record.fetch('_sumo_metadata', {:source => record[@source_name_key] })
|
195
227
|
key = sumo_key(sumo_metadata, record, tag)
|
196
228
|
log_format = sumo_metadata['log_format'] || @log_format
|
197
229
|
|
198
230
|
# Strip any unwanted newlines
|
199
231
|
record[@log_key].chomp! if record[@log_key] && record[@log_key].respond_to?(:chomp!)
|
200
232
|
|
201
|
-
case
|
233
|
+
case @data_type
|
234
|
+
when 'logs'
|
235
|
+
case log_format
|
202
236
|
when 'text'
|
203
237
|
log = record[@log_key]
|
204
238
|
unless log.nil?
|
@@ -214,6 +248,12 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
214
248
|
record = { :timestamp => sumo_timestamp(time) }.merge(record)
|
215
249
|
end
|
216
250
|
log = dump_log(record)
|
251
|
+
end
|
252
|
+
when 'metrics'
|
253
|
+
log = record[@log_key]
|
254
|
+
unless log.nil?
|
255
|
+
log.strip!
|
256
|
+
end
|
217
257
|
end
|
218
258
|
|
219
259
|
unless log.nil?
|
@@ -231,9 +271,11 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
231
271
|
source_name, source_category, source_host = key.split(':')
|
232
272
|
@sumo_conn.publish(
|
233
273
|
messages.join("\n"),
|
234
|
-
source_host
|
235
|
-
source_category=source_category,
|
236
|
-
source_name
|
274
|
+
source_host =source_host,
|
275
|
+
source_category =source_category,
|
276
|
+
source_name =source_name,
|
277
|
+
data_type =@data_type,
|
278
|
+
metric_data_format =@metric_data_format
|
237
279
|
)
|
238
280
|
end
|
239
281
|
|
data/test/helper.rb
CHANGED
@@ -1,2 +1,8 @@
|
|
1
|
-
|
2
|
-
require
|
1
|
+
$LOAD_PATH.unshift(File.expand_path("../../", __FILE__))
|
2
|
+
require "test-unit"
|
3
|
+
require "fluent/test"
|
4
|
+
require "fluent/test/driver/output"
|
5
|
+
require "fluent/test/helpers"
|
6
|
+
|
7
|
+
Test::Unit::TestCase.include(Fluent::Test::Helpers)
|
8
|
+
Test::Unit::TestCase.extend(Fluent::Test::Helpers)
|
@@ -1,41 +1,269 @@
|
|
1
|
-
require
|
2
|
-
require
|
1
|
+
require "test-unit"
|
2
|
+
require "fluent/test"
|
3
|
+
require "fluent/test/driver/output"
|
4
|
+
require "fluent/test/helpers"
|
5
|
+
require 'webmock/test_unit'
|
6
|
+
require 'fluent/plugin/out_sumologic'
|
3
7
|
|
4
8
|
class SumologicOutput < Test::Unit::TestCase
|
9
|
+
include Fluent::Test::Helpers
|
10
|
+
|
5
11
|
def setup
|
6
12
|
Fluent::Test.setup
|
7
|
-
require 'fluent/plugin/out_sumologic'
|
8
|
-
@driver = nil
|
9
|
-
log = Fluent::Engine.log
|
10
|
-
log.out.logs.slice!(0, log.out.logs.length)
|
11
13
|
end
|
12
14
|
|
13
|
-
def
|
14
|
-
|
15
|
+
def create_driver(conf = CONFIG)
|
16
|
+
Fluent::Test::Driver::Output.new(Fluent::Plugin::Sumologic).configure(conf)
|
17
|
+
end
|
18
|
+
|
19
|
+
def test_no_endpoint_configure
|
20
|
+
config = %{}
|
21
|
+
exception = assert_raise(Fluent::ConfigError) {create_driver(config)}
|
22
|
+
assert_equal("Invalid SumoLogic endpoint url: ", exception.message)
|
23
|
+
end
|
24
|
+
|
25
|
+
def test_invalid_data_type_configure
|
26
|
+
config = %{
|
27
|
+
endpoint https://SUMOLOGIC_URL
|
28
|
+
data_type 'foo'
|
29
|
+
}
|
30
|
+
exception = assert_raise(Fluent::ConfigError) {create_driver(config)}
|
31
|
+
assert_equal("Invalid data_type foo must be logs or metrics", exception.message)
|
32
|
+
end
|
33
|
+
|
34
|
+
def test_invalid_log_format_configure
|
35
|
+
config = %{
|
36
|
+
endpoint https://SUMOLOGIC_URL
|
37
|
+
log_format foo
|
38
|
+
}
|
39
|
+
exception = assert_raise(Fluent::ConfigError) {create_driver(config)}
|
40
|
+
assert_equal("Invalid log_format foo must be text, json or json_merge", exception.message)
|
41
|
+
end
|
42
|
+
|
43
|
+
def test_invalid_metrics_data_type
|
44
|
+
config = %{
|
45
|
+
endpoint https://SUMOLOGIC_URL
|
46
|
+
data_type metrics
|
47
|
+
metrics_data_type foo
|
48
|
+
}
|
49
|
+
exception = assert_raise(Fluent::ConfigError) {create_driver(config)}
|
50
|
+
assert_equal("Invalid metrics_data_type foo must be graphite or carbon2", exception.message)
|
15
51
|
end
|
16
52
|
|
17
|
-
def
|
53
|
+
def test_default_configure
|
18
54
|
config = %{
|
19
55
|
endpoint https://SUMOLOGIC_URL
|
20
|
-
log_format text
|
21
|
-
log_key LOG_KEY
|
22
|
-
source_category SOURCE_CATEGORY
|
23
|
-
source_name SOURCE_NAME
|
24
|
-
source_name_key SOURCE_NAME_KEY
|
25
|
-
source_host SOURCE_HOST
|
26
|
-
verify_ssl false
|
27
|
-
open_timeout 10
|
28
56
|
}
|
29
|
-
instance =
|
57
|
+
instance = create_driver(config).instance
|
30
58
|
|
59
|
+
assert_equal instance.data_type, 'logs'
|
60
|
+
assert_equal instance.metric_data_format, 'carbon2'
|
31
61
|
assert_equal instance.endpoint, 'https://SUMOLOGIC_URL'
|
32
|
-
assert_equal instance.log_format, '
|
33
|
-
assert_equal instance.log_key, '
|
34
|
-
assert_equal instance.source_category,
|
35
|
-
assert_equal instance.source_name,
|
36
|
-
assert_equal instance.source_name_key, '
|
37
|
-
assert_equal instance.source_host,
|
38
|
-
assert_equal instance.verify_ssl,
|
39
|
-
assert_equal instance.
|
40
|
-
|
41
|
-
|
62
|
+
assert_equal instance.log_format, 'json'
|
63
|
+
assert_equal instance.log_key, 'message'
|
64
|
+
assert_equal instance.source_category, nil
|
65
|
+
assert_equal instance.source_name, nil
|
66
|
+
assert_equal instance.source_name_key, 'source_name'
|
67
|
+
assert_equal instance.source_host, nil
|
68
|
+
assert_equal instance.verify_ssl, true
|
69
|
+
assert_equal instance.delimiter, '.'
|
70
|
+
assert_equal instance.open_timeout, 60
|
71
|
+
assert_equal instance.add_timestamp, true
|
72
|
+
assert_equal instance.proxy_uri, nil
|
73
|
+
end
|
74
|
+
|
75
|
+
def test_emit_invalid_url_json
|
76
|
+
config = %{
|
77
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
78
|
+
}
|
79
|
+
driver = create_driver(config)
|
80
|
+
time = event_time
|
81
|
+
WebMock.allow_net_connect!
|
82
|
+
assert_raise RuntimeError do
|
83
|
+
driver.run do
|
84
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => "test"})
|
85
|
+
end
|
86
|
+
end
|
87
|
+
WebMock.disable_net_connect!
|
88
|
+
end
|
89
|
+
|
90
|
+
def test_emit_invalid_url_json_merge
|
91
|
+
config = %{
|
92
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
93
|
+
log_format json_merge
|
94
|
+
}
|
95
|
+
driver = create_driver(config)
|
96
|
+
time = event_time
|
97
|
+
WebMock.allow_net_connect!
|
98
|
+
assert_raise RuntimeError do
|
99
|
+
driver.run do
|
100
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
101
|
+
end
|
102
|
+
end
|
103
|
+
WebMock.disable_net_connect!
|
104
|
+
end
|
105
|
+
|
106
|
+
def test_emit_invalid_url_text
|
107
|
+
config = %{
|
108
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
109
|
+
log_format text
|
110
|
+
}
|
111
|
+
driver = create_driver(config)
|
112
|
+
time = event_time
|
113
|
+
WebMock.allow_net_connect!
|
114
|
+
assert_raise RuntimeError do
|
115
|
+
driver.run do
|
116
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
117
|
+
end
|
118
|
+
end
|
119
|
+
WebMock.disable_net_connect!
|
120
|
+
end
|
121
|
+
|
122
|
+
def test_emit_text
|
123
|
+
config = %{
|
124
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
125
|
+
log_format text
|
126
|
+
source_category test
|
127
|
+
source_host test
|
128
|
+
source_name test
|
129
|
+
|
130
|
+
}
|
131
|
+
driver = create_driver(config)
|
132
|
+
time = event_time
|
133
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
134
|
+
driver.run do
|
135
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
136
|
+
end
|
137
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
138
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test'},
|
139
|
+
body: "test",
|
140
|
+
times:1
|
141
|
+
end
|
142
|
+
|
143
|
+
def test_emit_json
|
144
|
+
config = %{
|
145
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
146
|
+
log_format json
|
147
|
+
source_category test
|
148
|
+
source_host test
|
149
|
+
source_name test
|
150
|
+
|
151
|
+
}
|
152
|
+
driver = create_driver(config)
|
153
|
+
time = event_time
|
154
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
155
|
+
driver.run do
|
156
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
157
|
+
end
|
158
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
159
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test'},
|
160
|
+
body: /\A{"timestamp":\d+.,"foo":"bar","message":"test"}\z/,
|
161
|
+
times:1
|
162
|
+
end
|
163
|
+
|
164
|
+
def test_emit_json_merge
|
165
|
+
config = %{
|
166
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
167
|
+
log_format json_merge
|
168
|
+
source_category test
|
169
|
+
source_host test
|
170
|
+
source_name test
|
171
|
+
|
172
|
+
}
|
173
|
+
driver = create_driver(config)
|
174
|
+
time = event_time
|
175
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
176
|
+
driver.run do
|
177
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => '{"foo2":"bar2"}'})
|
178
|
+
end
|
179
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
180
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test'},
|
181
|
+
body: /\A{"foo2":"bar2","timestamp":\d+,"foo":"bar"}\z/,
|
182
|
+
times:1
|
183
|
+
end
|
184
|
+
|
185
|
+
def test_emit_with_sumo_metadata
|
186
|
+
config = %{
|
187
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
188
|
+
log_format json
|
189
|
+
}
|
190
|
+
driver = create_driver(config)
|
191
|
+
time = event_time
|
192
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
193
|
+
driver.run do
|
194
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test', '_sumo_metadata' => {
|
195
|
+
"host": "#{ENV['HOST']}",
|
196
|
+
"source": "${tag}",
|
197
|
+
"category": "test"
|
198
|
+
}})
|
199
|
+
end
|
200
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
201
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'', 'X-Sumo-Name'=>'output.test'},
|
202
|
+
body: /\A{"timestamp":\d+.,"foo":"bar","message":"test"}\z/,
|
203
|
+
times:1
|
204
|
+
end
|
205
|
+
|
206
|
+
def test_emit_json_no_timestamp
|
207
|
+
config = %{
|
208
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
209
|
+
log_format json
|
210
|
+
source_category test
|
211
|
+
source_host test
|
212
|
+
source_name test
|
213
|
+
add_timestamp false
|
214
|
+
}
|
215
|
+
driver = create_driver(config)
|
216
|
+
time = event_time
|
217
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
218
|
+
driver.run do
|
219
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
220
|
+
end
|
221
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
222
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test'},
|
223
|
+
body: /\A{"foo":"bar","message":"test"}\z/,
|
224
|
+
times:1
|
225
|
+
end
|
226
|
+
|
227
|
+
def test_emit_graphite
|
228
|
+
config = %{
|
229
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
230
|
+
data_type metrics
|
231
|
+
metric_data_format graphite
|
232
|
+
source_category test
|
233
|
+
source_host test
|
234
|
+
source_name test
|
235
|
+
}
|
236
|
+
driver = create_driver(config)
|
237
|
+
time = event_time
|
238
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
239
|
+
driver.run do
|
240
|
+
driver.feed("output.test", time, {'message' =>'prod.lb-1.cpu 87.2 1501753030'})
|
241
|
+
end
|
242
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
243
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test', 'Content-Type'=>'application/vnd.sumologic.graphite'},
|
244
|
+
body: /\Aprod.lb-1.cpu 87.2 1501753030\z/,
|
245
|
+
times:1
|
246
|
+
end
|
247
|
+
|
248
|
+
def test_emit_carbon
|
249
|
+
config = %{
|
250
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
251
|
+
data_type metrics
|
252
|
+
metric_data_format carbon2
|
253
|
+
source_category test
|
254
|
+
source_host test
|
255
|
+
source_name test
|
256
|
+
}
|
257
|
+
driver = create_driver(config)
|
258
|
+
time = event_time
|
259
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
260
|
+
driver.run do
|
261
|
+
driver.feed("output.test", time, {'message' =>'cluster=prod node=lb-1 metric=cpu ip=2.2.3.4 team=infra 87.2 1501753030'})
|
262
|
+
end
|
263
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
264
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test', 'Content-Type'=>'application/vnd.sumologic.carbon2'},
|
265
|
+
body: /\Acluster=prod node=lb-1 metric=cpu ip=2.2.3.4 team=infra 87.2 1501753030\z/,
|
266
|
+
times:1
|
267
|
+
end
|
268
|
+
|
269
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: fluent-plugin-sumologic_output
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Steven Adams
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2018-
|
12
|
+
date: 2018-06-29 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: bundler
|
@@ -90,6 +90,7 @@ extensions: []
|
|
90
90
|
extra_rdoc_files: []
|
91
91
|
files:
|
92
92
|
- ".gitignore"
|
93
|
+
- CHANGELOG.md
|
93
94
|
- Gemfile
|
94
95
|
- LICENSE
|
95
96
|
- README.md
|