fluent-plugin-sumologic_output 1.0.3 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +57 -0
- data/Gemfile +6 -1
- data/README.md +29 -1
- data/fluent-plugin-sumologic_output.gemspec +1 -2
- data/lib/fluent/plugin/out_sumologic.rb +58 -16
- data/test/helper.rb +8 -2
- data/test/plugin/test_out_sumologic.rb +256 -28
- metadata +3 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 8fd43e4a8259784985b70eaac09c43346525f61d08c0383912f6434887a9aa03
|
4
|
+
data.tar.gz: 7f8a58eee2a297ca40cb7d599371040bf7225632de7dbc561710050e01d692a0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e6436be0d5af2de4521319e7bee494cae41fc4b2052a54256317268ca02c2873bfc48060755ca7b361e21e69f633cca1a38aa255449ff72b21c0dfe2451b50ef
|
7
|
+
data.tar.gz: 5ee85c23b49b9608754e76342508ce9d4a7df0f1a35b4a82f8686b9eafb7264f8c09d1c736c8975b5fb61fa420b9efcb016bb6790aeb6f30f54c8b5e12a73500
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,57 @@
|
|
1
|
+
# Change Log
|
2
|
+
|
3
|
+
All notable changes to this project will be documented in this file. Tracking did not begin until version 1.10.
|
4
|
+
|
5
|
+
<a name="1.1.0"></a>
|
6
|
+
# [1.1.0] (2018-06-29)
|
7
|
+
|
8
|
+
* Add support for sending metrics.
|
9
|
+
|
10
|
+
<a name="1.0.3"></a>
|
11
|
+
# [1.0.3] (2018-05-07)
|
12
|
+
|
13
|
+
* [Fix #26 -- Don't chomp if the message is not chomp-able](https://github.com/SumoLogic/fluentd-output-sumologic/pull/29)
|
14
|
+
|
15
|
+
<a name="1.0.2"></a>
|
16
|
+
# [1.0.2] (2018-04-09)
|
17
|
+
|
18
|
+
* [add option to turn off adding timestamp to logs](https://github.com/SumoLogic/fluentd-output-sumologic/pull/27)
|
19
|
+
|
20
|
+
<a name="1.0.1"></a>
|
21
|
+
# [1.0.1] (2017-12-19)
|
22
|
+
|
23
|
+
* [Add client header for fluentd output](https://github.com/SumoLogic/fluentd-output-sumologic/pull/22)
|
24
|
+
|
25
|
+
<a name="1.0.0"></a>
|
26
|
+
# [1.0.0] (2017-11-06)
|
27
|
+
|
28
|
+
* [Upgrade to 0.14 API, send with ms precision](https://github.com/SumoLogic/fluentd-output-sumologic/pull/12)
|
29
|
+
* [Switch to httpclient](https://github.com/SumoLogic/fluentd-output-sumologic/pull/16)
|
30
|
+
* [Fix missing variable and improve config example](https://github.com/SumoLogic/fluentd-output-sumologic/pull/17)
|
31
|
+
|
32
|
+
<a name="0.0.7"></a>
|
33
|
+
# [0.0.7] (2017-10-26)
|
34
|
+
|
35
|
+
* [Expand parameters in the output configuration](https://github.com/SumoLogic/fluentd-output-sumologic/pull/14)
|
36
|
+
* [add open_timeout option](https://github.com/SumoLogic/fluentd-output-sumologic/pull/15)
|
37
|
+
|
38
|
+
<a name="0.0.6"></a>
|
39
|
+
# [0.0.6] (2017-08-23)
|
40
|
+
|
41
|
+
* Fix 0.0.5
|
42
|
+
|
43
|
+
<a name="0.0.5"></a>
|
44
|
+
# [0.0.5] (2017-08-18)
|
45
|
+
|
46
|
+
* [Ignore garbage records. Fix inspired by other plugins](https://github.com/SumoLogic/fluentd-output-sumologic/pull/7)
|
47
|
+
* [Extract the source_name from FluentD's buffer](https://github.com/SumoLogic/fluentd-output-sumologic/pull/8)
|
48
|
+
|
49
|
+
<a name="0.0.4"></a>
|
50
|
+
# [0.0.4] (2017-07-05)
|
51
|
+
|
52
|
+
* [Raise an exception for all non HTTP Success response codes](https://github.com/SumoLogic/fluentd-output-sumologic/pull/5)
|
53
|
+
|
54
|
+
<a name="0.0.3"></a>
|
55
|
+
# [0.0.3] (2016-12-10)
|
56
|
+
|
57
|
+
* Initial Release
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# fluent-plugin-sumologic_output, a plugin for [Fluentd](http://fluentd.org)
|
2
2
|
|
3
|
-
This plugin has been designed to output logs to [SumoLogic](http://www.sumologic.com) via a [HTTP collector endpoint](http://help.sumologic.com/Send_Data/Sources/02Sources_for_Hosted_Collectors/HTTP_Source)
|
3
|
+
This plugin has been designed to output logs or metrics to [SumoLogic](http://www.sumologic.com) via a [HTTP collector endpoint](http://help.sumologic.com/Send_Data/Sources/02Sources_for_Hosted_Collectors/HTTP_Source)
|
4
4
|
|
5
5
|
| TLS Deprecation Notice |
|
6
6
|
| --- |
|
@@ -17,6 +17,7 @@ The code in this repository has been contributed by the Sumo Logic community and
|
|
17
17
|
|
18
18
|
Configuration options for fluent.conf are:
|
19
19
|
|
20
|
+
* `data_type` - The type of data that will be sent to Sumo Logic, either `logs` or `metrics` (Default is `logs `)
|
20
21
|
* `endpoint` - SumoLogic HTTP Collector URL
|
21
22
|
* `verify_ssl` - Verify ssl certificate. (default is `true`)
|
22
23
|
* `source_category` - Set _sourceCategory metadata field within SumoLogic (default is `nil`)
|
@@ -31,7 +32,9 @@ Configuration options for fluent.conf are:
|
|
31
32
|
* `open_timeout` - Set timeout seconds to wait until connection is opened.
|
32
33
|
* `add_timestamp` - Add `timestamp` field to logs before sending to sumologic (default `true`)
|
33
34
|
* `proxy_uri` - Add the `uri` of the `proxy` environment if present.
|
35
|
+
* `metric_data_format` - The format of metrics you will be sending, either `graphite` or `carbon2` (Default is `graphite `)
|
34
36
|
|
37
|
+
### Example Configuration
|
35
38
|
Reading from the JSON formatted log files with `in_tail` and wildcard filenames:
|
36
39
|
```
|
37
40
|
<source>
|
@@ -55,6 +58,31 @@ Reading from the JSON formatted log files with `in_tail` and wildcard filenames:
|
|
55
58
|
</match>
|
56
59
|
```
|
57
60
|
|
61
|
+
Sending metrics to Sumo Logic using `in_http`:
|
62
|
+
```
|
63
|
+
<source>
|
64
|
+
@type http
|
65
|
+
port 8888
|
66
|
+
bind 0.0.0.0
|
67
|
+
</source>
|
68
|
+
|
69
|
+
<match test.carbon2>
|
70
|
+
@type sumologic
|
71
|
+
endpoint https://endpoint3.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV1hYfCAiqSH-PDY6gUOIgZvO60U_-y8SPQfK0Ks-ht7owrbk1AkX_ACp0uUxuLZOCw5QjBg1ndVPZ5TOJCFgNGRtFDoTDuQ2hzs3sn6FlfBSw==
|
72
|
+
data_type metrics
|
73
|
+
metric_data_format carbon2
|
74
|
+
flush_interval 1s
|
75
|
+
</match>
|
76
|
+
|
77
|
+
<match test.graphite>
|
78
|
+
@type sumologic
|
79
|
+
endpoint https://endpoint3.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV1hYfCAiqSH-PDY6gUOIgZvO60U_-y8SPQfK0Ks-ht7owrbk1AkX_ACp0uUxuLZOCw5QjBg1ndVPZ5TOJCFgNGRtFDoTDuQ2hzs3sn6FlfBSw==
|
80
|
+
data_type metrics
|
81
|
+
metric_data_format graphite
|
82
|
+
flush_interval 1s
|
83
|
+
</match>
|
84
|
+
```
|
85
|
+
|
58
86
|
## Example input/output
|
59
87
|
|
60
88
|
Assuming following inputs are coming from a log file named `/var/log/appa_webserver.log`
|
@@ -4,7 +4,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
4
4
|
|
5
5
|
Gem::Specification.new do |gem|
|
6
6
|
gem.name = "fluent-plugin-sumologic_output"
|
7
|
-
gem.version = "1.0
|
7
|
+
gem.version = "1.1.0"
|
8
8
|
gem.authors = ["Steven Adams", "Frank Reno"]
|
9
9
|
gem.email = ["stevezau@gmail.com", "frank.reno@me.com"]
|
10
10
|
gem.description = %q{Output plugin to SumoLogic HTTP Endpoint}
|
@@ -24,6 +24,5 @@ Gem::Specification.new do |gem|
|
|
24
24
|
gem.add_development_dependency "rake"
|
25
25
|
gem.add_runtime_dependency "fluentd", ">= 0.14.0"
|
26
26
|
gem.add_development_dependency 'test-unit', '~> 3.1.0'
|
27
|
-
|
28
27
|
gem.add_runtime_dependency 'httpclient', '~> 2.8.0'
|
29
28
|
end
|
@@ -1,4 +1,3 @@
|
|
1
|
-
|
2
1
|
require 'fluent/plugin/output'
|
3
2
|
require 'net/https'
|
4
3
|
require 'yajl'
|
@@ -13,22 +12,31 @@ class SumologicConnection
|
|
13
12
|
create_http_client(verify_ssl, connect_timeout, proxy_uri)
|
14
13
|
end
|
15
14
|
|
16
|
-
def publish(raw_data, source_host=nil, source_category=nil, source_name=nil)
|
17
|
-
response = http.post(@endpoint, raw_data, request_headers(source_host, source_category, source_name))
|
15
|
+
def publish(raw_data, source_host=nil, source_category=nil, source_name=nil, data_type, metric_data_type)
|
16
|
+
response = http.post(@endpoint, raw_data, request_headers(source_host, source_category, source_name, data_type, metric_data_type))
|
18
17
|
unless response.ok?
|
19
|
-
raise "Failed to send data to HTTP Source. #{response.code} - #{response.body}"
|
18
|
+
raise RuntimeError, "Failed to send data to HTTP Source. #{response.code} - #{response.body}"
|
20
19
|
end
|
21
20
|
end
|
22
21
|
|
23
|
-
|
24
|
-
|
25
|
-
def request_headers(source_host, source_category, source_name)
|
26
|
-
{
|
22
|
+
def request_headers(source_host, source_category, source_name, data_type, metric_data_format)
|
23
|
+
headers = {
|
27
24
|
'X-Sumo-Name' => source_name,
|
28
25
|
'X-Sumo-Category' => source_category,
|
29
26
|
'X-Sumo-Host' => source_host,
|
30
27
|
'X-Sumo-Client' => 'fluentd-output'
|
31
28
|
}
|
29
|
+
if data_type == 'metrics'
|
30
|
+
case metric_data_format
|
31
|
+
when 'graphite'
|
32
|
+
headers['Content-Type'] = 'application/vnd.sumologic.graphite'
|
33
|
+
when 'carbon2'
|
34
|
+
headers['Content-Type'] = 'application/vnd.sumologic.carbon2'
|
35
|
+
else
|
36
|
+
raise RuntimeError, "Invalid #{metric_data_format}, must be graphite or carbon2"
|
37
|
+
end
|
38
|
+
end
|
39
|
+
return headers
|
32
40
|
end
|
33
41
|
|
34
42
|
def ssl_options(verify_ssl)
|
@@ -49,7 +57,15 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
49
57
|
|
50
58
|
helpers :compat_parameters
|
51
59
|
DEFAULT_BUFFER_TYPE = "memory"
|
52
|
-
|
60
|
+
LOGS_DATA_TYPE = "logs"
|
61
|
+
METRICS_DATA_TYPE = "metrics"
|
62
|
+
DEFAULT_DATA_TYPE = LOGS_DATA_TYPE
|
63
|
+
GRAPHITE_METRIC_FORMAT_TYPE = "graphite"
|
64
|
+
CARBON2_METRIC_FORMAT_TYPE = "carbon2"
|
65
|
+
DEFAULT_METRIC_FORMAT_TYPE = CARBON2_METRIC_FORMAT_TYPE
|
66
|
+
|
67
|
+
config_param :data_type, :string, :default => DEFAULT_DATA_TYPE
|
68
|
+
config_param :metric_data_format, :default => DEFAULT_METRIC_FORMAT_TYPE
|
53
69
|
config_param :endpoint, :string
|
54
70
|
config_param :log_format, :string, :default => 'json'
|
55
71
|
config_param :log_key, :string, :default => 'message'
|
@@ -81,8 +97,24 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
81
97
|
raise Fluent::ConfigError, "Invalid SumoLogic endpoint url: #{conf['endpoint']}"
|
82
98
|
end
|
83
99
|
|
84
|
-
unless conf['
|
85
|
-
|
100
|
+
unless conf['data_type'].nil?
|
101
|
+
unless conf['data_type'] =~ /\A(?:logs|metrics)\z/
|
102
|
+
raise Fluent::ConfigError, "Invalid data_type #{conf['data_type']} must be logs or metrics"
|
103
|
+
end
|
104
|
+
end
|
105
|
+
|
106
|
+
if conf['data_type'].nil? || conf['data_type'] == LOGS_DATA_TYPE
|
107
|
+
unless conf['log_format'].nil?
|
108
|
+
unless conf['log_format'] =~ /\A(?:json|text|json_merge)\z/
|
109
|
+
raise Fluent::ConfigError, "Invalid log_format #{conf['log_format']} must be text, json or json_merge"
|
110
|
+
end
|
111
|
+
end
|
112
|
+
end
|
113
|
+
|
114
|
+
if conf['data_type'] == METRICS_DATA_TYPE && ! conf['metrics_data_type'].nil?
|
115
|
+
unless conf['metrics_data_type'] =~ /\A(?:graphite|carbon2)\z/
|
116
|
+
raise Fluent::ConfigError, "Invalid metrics_data_type #{conf['metrics_data_type']} must be graphite or carbon2"
|
117
|
+
end
|
86
118
|
end
|
87
119
|
|
88
120
|
@sumo_conn = SumologicConnection.new(conf['endpoint'], conf['verify_ssl'], conf['open_timeout'].to_i, conf['proxy_uri'])
|
@@ -191,14 +223,16 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
191
223
|
# plugin dies randomly
|
192
224
|
# https://github.com/uken/fluent-plugin-elasticsearch/commit/8597b5d1faf34dd1f1523bfec45852d380b26601#diff-ae62a005780cc730c558e3e4f47cc544R94
|
193
225
|
next unless record.is_a? Hash
|
194
|
-
sumo_metadata = record.fetch('_sumo_metadata', {
|
226
|
+
sumo_metadata = record.fetch('_sumo_metadata', {:source => record[@source_name_key] })
|
195
227
|
key = sumo_key(sumo_metadata, record, tag)
|
196
228
|
log_format = sumo_metadata['log_format'] || @log_format
|
197
229
|
|
198
230
|
# Strip any unwanted newlines
|
199
231
|
record[@log_key].chomp! if record[@log_key] && record[@log_key].respond_to?(:chomp!)
|
200
232
|
|
201
|
-
case
|
233
|
+
case @data_type
|
234
|
+
when 'logs'
|
235
|
+
case log_format
|
202
236
|
when 'text'
|
203
237
|
log = record[@log_key]
|
204
238
|
unless log.nil?
|
@@ -214,6 +248,12 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
214
248
|
record = { :timestamp => sumo_timestamp(time) }.merge(record)
|
215
249
|
end
|
216
250
|
log = dump_log(record)
|
251
|
+
end
|
252
|
+
when 'metrics'
|
253
|
+
log = record[@log_key]
|
254
|
+
unless log.nil?
|
255
|
+
log.strip!
|
256
|
+
end
|
217
257
|
end
|
218
258
|
|
219
259
|
unless log.nil?
|
@@ -231,9 +271,11 @@ class Fluent::Plugin::Sumologic < Fluent::Plugin::Output
|
|
231
271
|
source_name, source_category, source_host = key.split(':')
|
232
272
|
@sumo_conn.publish(
|
233
273
|
messages.join("\n"),
|
234
|
-
source_host
|
235
|
-
source_category=source_category,
|
236
|
-
source_name
|
274
|
+
source_host =source_host,
|
275
|
+
source_category =source_category,
|
276
|
+
source_name =source_name,
|
277
|
+
data_type =@data_type,
|
278
|
+
metric_data_format =@metric_data_format
|
237
279
|
)
|
238
280
|
end
|
239
281
|
|
data/test/helper.rb
CHANGED
@@ -1,2 +1,8 @@
|
|
1
|
-
|
2
|
-
require
|
1
|
+
$LOAD_PATH.unshift(File.expand_path("../../", __FILE__))
|
2
|
+
require "test-unit"
|
3
|
+
require "fluent/test"
|
4
|
+
require "fluent/test/driver/output"
|
5
|
+
require "fluent/test/helpers"
|
6
|
+
|
7
|
+
Test::Unit::TestCase.include(Fluent::Test::Helpers)
|
8
|
+
Test::Unit::TestCase.extend(Fluent::Test::Helpers)
|
@@ -1,41 +1,269 @@
|
|
1
|
-
require
|
2
|
-
require
|
1
|
+
require "test-unit"
|
2
|
+
require "fluent/test"
|
3
|
+
require "fluent/test/driver/output"
|
4
|
+
require "fluent/test/helpers"
|
5
|
+
require 'webmock/test_unit'
|
6
|
+
require 'fluent/plugin/out_sumologic'
|
3
7
|
|
4
8
|
class SumologicOutput < Test::Unit::TestCase
|
9
|
+
include Fluent::Test::Helpers
|
10
|
+
|
5
11
|
def setup
|
6
12
|
Fluent::Test.setup
|
7
|
-
require 'fluent/plugin/out_sumologic'
|
8
|
-
@driver = nil
|
9
|
-
log = Fluent::Engine.log
|
10
|
-
log.out.logs.slice!(0, log.out.logs.length)
|
11
13
|
end
|
12
14
|
|
13
|
-
def
|
14
|
-
|
15
|
+
def create_driver(conf = CONFIG)
|
16
|
+
Fluent::Test::Driver::Output.new(Fluent::Plugin::Sumologic).configure(conf)
|
17
|
+
end
|
18
|
+
|
19
|
+
def test_no_endpoint_configure
|
20
|
+
config = %{}
|
21
|
+
exception = assert_raise(Fluent::ConfigError) {create_driver(config)}
|
22
|
+
assert_equal("Invalid SumoLogic endpoint url: ", exception.message)
|
23
|
+
end
|
24
|
+
|
25
|
+
def test_invalid_data_type_configure
|
26
|
+
config = %{
|
27
|
+
endpoint https://SUMOLOGIC_URL
|
28
|
+
data_type 'foo'
|
29
|
+
}
|
30
|
+
exception = assert_raise(Fluent::ConfigError) {create_driver(config)}
|
31
|
+
assert_equal("Invalid data_type foo must be logs or metrics", exception.message)
|
32
|
+
end
|
33
|
+
|
34
|
+
def test_invalid_log_format_configure
|
35
|
+
config = %{
|
36
|
+
endpoint https://SUMOLOGIC_URL
|
37
|
+
log_format foo
|
38
|
+
}
|
39
|
+
exception = assert_raise(Fluent::ConfigError) {create_driver(config)}
|
40
|
+
assert_equal("Invalid log_format foo must be text, json or json_merge", exception.message)
|
41
|
+
end
|
42
|
+
|
43
|
+
def test_invalid_metrics_data_type
|
44
|
+
config = %{
|
45
|
+
endpoint https://SUMOLOGIC_URL
|
46
|
+
data_type metrics
|
47
|
+
metrics_data_type foo
|
48
|
+
}
|
49
|
+
exception = assert_raise(Fluent::ConfigError) {create_driver(config)}
|
50
|
+
assert_equal("Invalid metrics_data_type foo must be graphite or carbon2", exception.message)
|
15
51
|
end
|
16
52
|
|
17
|
-
def
|
53
|
+
def test_default_configure
|
18
54
|
config = %{
|
19
55
|
endpoint https://SUMOLOGIC_URL
|
20
|
-
log_format text
|
21
|
-
log_key LOG_KEY
|
22
|
-
source_category SOURCE_CATEGORY
|
23
|
-
source_name SOURCE_NAME
|
24
|
-
source_name_key SOURCE_NAME_KEY
|
25
|
-
source_host SOURCE_HOST
|
26
|
-
verify_ssl false
|
27
|
-
open_timeout 10
|
28
56
|
}
|
29
|
-
instance =
|
57
|
+
instance = create_driver(config).instance
|
30
58
|
|
59
|
+
assert_equal instance.data_type, 'logs'
|
60
|
+
assert_equal instance.metric_data_format, 'carbon2'
|
31
61
|
assert_equal instance.endpoint, 'https://SUMOLOGIC_URL'
|
32
|
-
assert_equal instance.log_format, '
|
33
|
-
assert_equal instance.log_key, '
|
34
|
-
assert_equal instance.source_category,
|
35
|
-
assert_equal instance.source_name,
|
36
|
-
assert_equal instance.source_name_key, '
|
37
|
-
assert_equal instance.source_host,
|
38
|
-
assert_equal instance.verify_ssl,
|
39
|
-
assert_equal instance.
|
40
|
-
|
41
|
-
|
62
|
+
assert_equal instance.log_format, 'json'
|
63
|
+
assert_equal instance.log_key, 'message'
|
64
|
+
assert_equal instance.source_category, nil
|
65
|
+
assert_equal instance.source_name, nil
|
66
|
+
assert_equal instance.source_name_key, 'source_name'
|
67
|
+
assert_equal instance.source_host, nil
|
68
|
+
assert_equal instance.verify_ssl, true
|
69
|
+
assert_equal instance.delimiter, '.'
|
70
|
+
assert_equal instance.open_timeout, 60
|
71
|
+
assert_equal instance.add_timestamp, true
|
72
|
+
assert_equal instance.proxy_uri, nil
|
73
|
+
end
|
74
|
+
|
75
|
+
def test_emit_invalid_url_json
|
76
|
+
config = %{
|
77
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
78
|
+
}
|
79
|
+
driver = create_driver(config)
|
80
|
+
time = event_time
|
81
|
+
WebMock.allow_net_connect!
|
82
|
+
assert_raise RuntimeError do
|
83
|
+
driver.run do
|
84
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => "test"})
|
85
|
+
end
|
86
|
+
end
|
87
|
+
WebMock.disable_net_connect!
|
88
|
+
end
|
89
|
+
|
90
|
+
def test_emit_invalid_url_json_merge
|
91
|
+
config = %{
|
92
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
93
|
+
log_format json_merge
|
94
|
+
}
|
95
|
+
driver = create_driver(config)
|
96
|
+
time = event_time
|
97
|
+
WebMock.allow_net_connect!
|
98
|
+
assert_raise RuntimeError do
|
99
|
+
driver.run do
|
100
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
101
|
+
end
|
102
|
+
end
|
103
|
+
WebMock.disable_net_connect!
|
104
|
+
end
|
105
|
+
|
106
|
+
def test_emit_invalid_url_text
|
107
|
+
config = %{
|
108
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
109
|
+
log_format text
|
110
|
+
}
|
111
|
+
driver = create_driver(config)
|
112
|
+
time = event_time
|
113
|
+
WebMock.allow_net_connect!
|
114
|
+
assert_raise RuntimeError do
|
115
|
+
driver.run do
|
116
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
117
|
+
end
|
118
|
+
end
|
119
|
+
WebMock.disable_net_connect!
|
120
|
+
end
|
121
|
+
|
122
|
+
def test_emit_text
|
123
|
+
config = %{
|
124
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
125
|
+
log_format text
|
126
|
+
source_category test
|
127
|
+
source_host test
|
128
|
+
source_name test
|
129
|
+
|
130
|
+
}
|
131
|
+
driver = create_driver(config)
|
132
|
+
time = event_time
|
133
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
134
|
+
driver.run do
|
135
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
136
|
+
end
|
137
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
138
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test'},
|
139
|
+
body: "test",
|
140
|
+
times:1
|
141
|
+
end
|
142
|
+
|
143
|
+
def test_emit_json
|
144
|
+
config = %{
|
145
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
146
|
+
log_format json
|
147
|
+
source_category test
|
148
|
+
source_host test
|
149
|
+
source_name test
|
150
|
+
|
151
|
+
}
|
152
|
+
driver = create_driver(config)
|
153
|
+
time = event_time
|
154
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
155
|
+
driver.run do
|
156
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
157
|
+
end
|
158
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
159
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test'},
|
160
|
+
body: /\A{"timestamp":\d+.,"foo":"bar","message":"test"}\z/,
|
161
|
+
times:1
|
162
|
+
end
|
163
|
+
|
164
|
+
def test_emit_json_merge
|
165
|
+
config = %{
|
166
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
167
|
+
log_format json_merge
|
168
|
+
source_category test
|
169
|
+
source_host test
|
170
|
+
source_name test
|
171
|
+
|
172
|
+
}
|
173
|
+
driver = create_driver(config)
|
174
|
+
time = event_time
|
175
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
176
|
+
driver.run do
|
177
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => '{"foo2":"bar2"}'})
|
178
|
+
end
|
179
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
180
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test'},
|
181
|
+
body: /\A{"foo2":"bar2","timestamp":\d+,"foo":"bar"}\z/,
|
182
|
+
times:1
|
183
|
+
end
|
184
|
+
|
185
|
+
def test_emit_with_sumo_metadata
|
186
|
+
config = %{
|
187
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
188
|
+
log_format json
|
189
|
+
}
|
190
|
+
driver = create_driver(config)
|
191
|
+
time = event_time
|
192
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
193
|
+
driver.run do
|
194
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test', '_sumo_metadata' => {
|
195
|
+
"host": "#{ENV['HOST']}",
|
196
|
+
"source": "${tag}",
|
197
|
+
"category": "test"
|
198
|
+
}})
|
199
|
+
end
|
200
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
201
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'', 'X-Sumo-Name'=>'output.test'},
|
202
|
+
body: /\A{"timestamp":\d+.,"foo":"bar","message":"test"}\z/,
|
203
|
+
times:1
|
204
|
+
end
|
205
|
+
|
206
|
+
def test_emit_json_no_timestamp
|
207
|
+
config = %{
|
208
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
209
|
+
log_format json
|
210
|
+
source_category test
|
211
|
+
source_host test
|
212
|
+
source_name test
|
213
|
+
add_timestamp false
|
214
|
+
}
|
215
|
+
driver = create_driver(config)
|
216
|
+
time = event_time
|
217
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
218
|
+
driver.run do
|
219
|
+
driver.feed("output.test", time, {'foo' => 'bar', 'message' => 'test'})
|
220
|
+
end
|
221
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
222
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test'},
|
223
|
+
body: /\A{"foo":"bar","message":"test"}\z/,
|
224
|
+
times:1
|
225
|
+
end
|
226
|
+
|
227
|
+
def test_emit_graphite
|
228
|
+
config = %{
|
229
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
230
|
+
data_type metrics
|
231
|
+
metric_data_format graphite
|
232
|
+
source_category test
|
233
|
+
source_host test
|
234
|
+
source_name test
|
235
|
+
}
|
236
|
+
driver = create_driver(config)
|
237
|
+
time = event_time
|
238
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
239
|
+
driver.run do
|
240
|
+
driver.feed("output.test", time, {'message' =>'prod.lb-1.cpu 87.2 1501753030'})
|
241
|
+
end
|
242
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
243
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test', 'Content-Type'=>'application/vnd.sumologic.graphite'},
|
244
|
+
body: /\Aprod.lb-1.cpu 87.2 1501753030\z/,
|
245
|
+
times:1
|
246
|
+
end
|
247
|
+
|
248
|
+
def test_emit_carbon
|
249
|
+
config = %{
|
250
|
+
endpoint https://collectors.sumologic.com/v1/receivers/http/1234
|
251
|
+
data_type metrics
|
252
|
+
metric_data_format carbon2
|
253
|
+
source_category test
|
254
|
+
source_host test
|
255
|
+
source_name test
|
256
|
+
}
|
257
|
+
driver = create_driver(config)
|
258
|
+
time = event_time
|
259
|
+
stub_request(:post, 'https://collectors.sumologic.com/v1/receivers/http/1234')
|
260
|
+
driver.run do
|
261
|
+
driver.feed("output.test", time, {'message' =>'cluster=prod node=lb-1 metric=cpu ip=2.2.3.4 team=infra 87.2 1501753030'})
|
262
|
+
end
|
263
|
+
assert_requested :post, "https://collectors.sumologic.com/v1/receivers/http/1234",
|
264
|
+
headers: {'X-Sumo-Category'=>'test', 'X-Sumo-Client'=>'fluentd-output', 'X-Sumo-Host'=>'test', 'X-Sumo-Name'=>'test', 'Content-Type'=>'application/vnd.sumologic.carbon2'},
|
265
|
+
body: /\Acluster=prod node=lb-1 metric=cpu ip=2.2.3.4 team=infra 87.2 1501753030\z/,
|
266
|
+
times:1
|
267
|
+
end
|
268
|
+
|
269
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: fluent-plugin-sumologic_output
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Steven Adams
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2018-
|
12
|
+
date: 2018-06-29 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: bundler
|
@@ -90,6 +90,7 @@ extensions: []
|
|
90
90
|
extra_rdoc_files: []
|
91
91
|
files:
|
92
92
|
- ".gitignore"
|
93
|
+
- CHANGELOG.md
|
93
94
|
- Gemfile
|
94
95
|
- LICENSE
|
95
96
|
- README.md
|