fluent-plugin-calyptia-monitoring 0.1.0.rc8 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 071d15721bdabcee712ce74be5d1f7125458e97f79f88c8e1b30927d353a9f04
4
- data.tar.gz: 8704372a766aeb87d7993cf7e42ff0b2a07b6927398a25ee125bda03d035315d
3
+ metadata.gz: c8a47c5ddcd660a66dcec0b732510d97f65d5df3c732faf3f6575828948076de
4
+ data.tar.gz: d4796efe64fe5cc8bdd6c531a3f3b27de8c759e3fed3fadb5d7e79e5345f76fd
5
5
  SHA512:
6
- metadata.gz: 5c8c50b693112a46959ee857d356a53957957fd1823eb4d98773765e7f3141535e0c3aac3e17e4cba80d930aea0f8b33e5174105955dfef0cb2d01a341ad05d1
7
- data.tar.gz: 59a2cb53277e5bc524f5e18a534764252a523d94c3ce5b9e3fdd46214f5a61bb191398255fbcd99f858dc335ebd22f3537ba4b23fc76734bfb2720381dc7aa79
6
+ metadata.gz: 3af292be5c20644444af230895df1b6a1f918513d899736e2f8d97cedec6b9d2335f25469e557cb7b8a14d78b6cdabe5bf3e59e2c5863e19f131a066876bce57
7
+ data.tar.gz: fbc3e6d065c9fe6ed75cfe31e58f81367d7b1fdc02c5c93d0e5148fe789ccab2cfdcf4cae42948a309a5cf6ea206c4b129da157ca489540884bcf623ca98a125
data/README.md CHANGED
@@ -40,10 +40,11 @@ $ bundle
40
40
 
41
41
  |parameter|type|description|default|
42
42
  |---|---|---|---|
43
- |endpoint|string (optional)|The endpoint for Monitoring API HTTP request, e.g. http://example.com/api|`TBD`|
43
+ |endpoint|string (optional)|The endpoint for Monitoring API HTTP request, e.g. http://example.com/api|`"https://cloud-api.calyptia.com"`|
44
44
  |api_key|string (required)|The API KEY for Monitoring API HTTP request||
45
45
  |rate|time (optional)|Emit monitoring values interval. (minimum interval is 30 seconds.)|`30`|
46
46
  |pending_metrics_size|size (optional)|Setup pending metrics capacity size|`100`|
47
+ |fluentd_conf_path|string (optional)|Specify Fluentd config file path for RPC not to be available case|`nil`|
47
48
 
48
49
  ### Example
49
50
 
@@ -84,6 +85,44 @@ And enabling RPC and configDump endpoint is required if sending Fluentd configur
84
85
  </source>
85
86
  ```
86
87
 
88
+ And also retrieving configuration from actual file is also supported:
89
+
90
+ ```aconf
91
+ <system>
92
+ # If users want to use multi workers feature which corresponds to logical number of CPUs, please comment out this line.
93
+ # workers "#{require 'etc'; Etc.nprocessors}"
94
+ enable_input_metrics true
95
+ # This record size measuring settings might impact for performance.
96
+ # Please be careful for high loaded environment to turn on.
97
+ enable_size_metrics true
98
+ <metrics>
99
+ @type cmetrics
100
+ </metrics>
101
+ </system>
102
+ # And other configurations....
103
+
104
+ ## Fill YOUR_API_KEY with your Calyptia API KEY
105
+ <source>
106
+ @type calyptia_monitoring
107
+ @id input_caplyptia_moniroting
108
+ <cloud_monitoring>
109
+ # endpoint http://development-environment-or-production.fqdn:5000
110
+ api_key YOUR_API_KEY
111
+ rate 30
112
+ pending_metrics_size 100 # Specify capacity for pending metrics
113
+ fluentd_conf_path /path/to/fluent.conf
114
+ </cloud_monitoring>
115
+ <storage>
116
+ @type local
117
+ path /path/to/agent/accessible/directories/agent_states
118
+ </storage>
119
+ </source>
120
+ ```
121
+
122
+ **Note:** We recommend to use RPC version due to some circumstances should differ between a loaded configuration and a saved Fluentd configuration.
123
+ This is because calling dumping config RPC feature can obtain from configuration contents which are loaded on memory. But retrieving configuration from the specified file is just read from the file contents and it cannot handle/retrieve loaded configurations on Fluentd.
124
+ When users just update their Fluentd configurations and forgot to restart/reload their Fluentd instances, loaded configurations differ from just edited ones.
125
+
87
126
  ## Calyptia Monitoring API config generator
88
127
 
89
128
  Usage:
@@ -91,7 +130,7 @@ Usage:
91
130
  ```
92
131
  Usage: calyptia-config-generator api_key [options]
93
132
 
94
- Output plugin config definitions
133
+ Generate Calyptia monitoring plugin config definitions
95
134
 
96
135
  Arguments:
97
136
  api_key: Specify your API_KEY
@@ -104,10 +143,12 @@ Options:
104
143
  --disable-get-dump Disable RPC getDump procedure. getDump is enabled by default.
105
144
  --storage-agent-token-dir DIR
106
145
  Specify accesible storage token dir. (default: /path/to/accesible/dir)
146
+ --fluentd-conf-path PATH Specify fluentd configuration file path. (default: nil)
107
147
  ```
108
148
 
109
149
  ## Copyright
110
150
 
111
- * Copyright(c) 2021- Hiroshi Hatake
151
+ * Copyright(c) 2021- Calyptia Inc.
152
+ * Maintainer: Hiroshi Hatake <hatake@calyptia.com>
112
153
  * License
113
154
  * Apache License, Version 2.0
@@ -3,7 +3,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
3
3
 
4
4
  Gem::Specification.new do |spec|
5
5
  spec.name = "fluent-plugin-calyptia-monitoring"
6
- spec.version = "0.1.0.rc8"
6
+ spec.version = "0.1.2"
7
7
  spec.authors = ["Hiroshi Hatake"]
8
8
  spec.email = ["hatake@calyptia.com"]
9
9
 
@@ -23,6 +23,6 @@ Gem::Specification.new do |spec|
23
23
  spec.add_development_dependency "bundler", "~> 2.2.15"
24
24
  spec.add_development_dependency "rake", "~> 13.0"
25
25
  spec.add_development_dependency "test-unit", "~> 3.3"
26
- spec.add_runtime_dependency "fluentd", [">= 1.13.0", "< 2"]
27
- spec.add_runtime_dependency "fluent-plugin-metrics-cmetrics", ">= 0.1.0.rc3"
26
+ spec.add_runtime_dependency "fluentd", [">= 1.14.0", "< 2"]
27
+ spec.add_runtime_dependency "fluent-plugin-metrics-cmetrics", ">= 0.1.1"
28
28
  end
@@ -33,6 +33,8 @@ class CalyptiaConfigGenerator
33
33
  @enable_get_dump = true
34
34
  @rpc_endpoint = "127.0.0.1:24444"
35
35
  @storage_agent_token_dir = default_storage_dir
36
+ @fluentd_conf_path = nil
37
+ @disable_rpc = false
36
38
 
37
39
  prepare_option_parser
38
40
  end
@@ -96,6 +98,10 @@ BANNER
96
98
  @parser.on("--storage-agent-token-dir DIR", "Specify accesible storage token dir. (default: #{default_storage_dir})") do |s|
97
99
  @storage_agent_token_dir = s
98
100
  end
101
+ @parser.on("--fluentd-conf-path PATH", "Specify fluentd configuration file path. (default: nil)") do |s|
102
+ @fluentd_conf_path = s
103
+ @disable_rpc = true
104
+ end
99
105
  end
100
106
 
101
107
  def usage(message = nil)
@@ -125,6 +131,8 @@ BANNER
125
131
  size_metrics: @enable_size_metrics,
126
132
  enable_get_dump: @enable_get_dump,
127
133
  storage_agent_token_dir: @storage_agent_token_dir,
134
+ fluentd_conf_path: @fluentd_conf_path,
135
+ disable_rpc: @disable_rpc,
128
136
  }
129
137
  rescue => e
130
138
  usage(e)
@@ -0,0 +1,36 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'fluent/env'
4
+ require 'fluent/engine'
5
+ require 'fluent/log'
6
+ require 'fluent/config'
7
+ require 'fluent/configurable'
8
+ require 'fluent/system_config'
9
+ require 'fluent/config/element'
10
+ require 'serverengine'
11
+ require 'stringio'
12
+
13
+ include Fluent::Configurable
14
+
15
+ def init_log
16
+ dl_opts = {}
17
+ dl_opts[:log_level] = ServerEngine::DaemonLogger::WARN
18
+ @sio = StringIO.new('', 'r+')
19
+ logger = ServerEngine::DaemonLogger.new(@sio, dl_opts)
20
+ $log = Fluent::Log.new(logger)
21
+ end
22
+
23
+ init_log
24
+
25
+ File.open(ENV["FLUENT_CONFIG_PATH"], "r") do |f|
26
+ config = Fluent::Config.parse(f.read, '(supervisor)', '(readFromFile)', true)
27
+ system_config = Fluent::SystemConfig.create(config)
28
+ Fluent::Engine.init(system_config, supervisor_mode: true)
29
+ Fluent::Engine.run_configure(config, dry_run: true)
30
+ confs = []
31
+ masked_element = config.to_masked_element
32
+ masked_element.elements.each{|e|
33
+ confs << e.to_s
34
+ }
35
+ puts confs.join
36
+ end
@@ -0,0 +1,87 @@
1
+ #
2
+ # fluent-plugin-calyptia-monitoring
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+ require 'fluent/plugin/in_monitor_agent'
17
+
18
+ module Fluent::Plugin
19
+ class CalyptiaMonitoringBufferExtInput < MonitorAgentInput
20
+ CALYPTIA_PLUGIN_BUFFER_METRIC_INFO = {
21
+ 'buffer_total_queued_size' => ->() {
22
+ throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
23
+ @buffer.total_queued_size_metrics.cmetrics.to_msgpack
24
+ },
25
+ 'buffer_stage_length' => ->() {
26
+ throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
27
+ @buffer.stage_length_metrics.cmetrics.to_msgpack
28
+ },
29
+ 'buffer_stage_byte_size' => ->() {
30
+ throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
31
+ @buffer.stage_size_metrics.cmetrics.to_msgpack
32
+ },
33
+ 'buffer_queue_length' => ->() {
34
+ throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
35
+ @buffer.queue_length_metrics.cmetrics.to_msgpack
36
+ },
37
+ 'buffer_queue_byte_size' => ->() {
38
+ throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
39
+ @buffer.queue_size_metrics.cmetrics.to_msgpack
40
+ },
41
+ 'available_buffer_space_ratios' => ->() {
42
+ throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
43
+ @buffer.available_buffer_space_ratios_metrics.cmetrics.to_msgpack
44
+ },
45
+ 'newest_timekey' => ->() {
46
+ throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
47
+ @buffer.newest_timekey_metrics.cmetrics.to_msgpack
48
+ },
49
+ 'oldest_timekey' => ->() {
50
+ throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
51
+ @buffer.oldest_timekey_metrics.cmetrics.to_msgpack
52
+ },
53
+ }
54
+
55
+ def get_monitor_info(pe, opts = {})
56
+ obj = {}
57
+
58
+ obj['metrics'] = get_plugin_metric(pe)
59
+
60
+ obj
61
+ end
62
+
63
+ def get_plugin_metric(pe)
64
+ # Nop for non output plugin
65
+ return {} if plugin_category(pe) != "output"
66
+
67
+ metrics = {}
68
+
69
+ if pe.respond_to?(:statistics)
70
+ # Force to update buffers' metrics values
71
+ pe.statistics
72
+ end
73
+
74
+ CALYPTIA_PLUGIN_BUFFER_METRIC_INFO.each_pair { |key, code|
75
+ begin
76
+ v = pe.instance_exec(&code)
77
+ unless v.nil?
78
+ metrics[key] = v
79
+ end
80
+ rescue
81
+ end
82
+ }
83
+
84
+ metrics
85
+ end
86
+ end
87
+ end
@@ -37,12 +37,17 @@ module Fluent::Plugin
37
37
  ENV['HTTPS_PROXY'] || ENV['HTTP_PROXY'] || ENV['http_proxy'] || ENV['https_proxy']
38
38
  end
39
39
 
40
+ def create_go_semver(version)
41
+ version.gsub(/.(?<prever>(rc|alpha|beta|pre))/,
42
+ '-\k<prever>')
43
+ end
44
+
40
45
  def agent_metadata(current_config)
41
46
  metadata = {
42
47
  "name" => Socket.gethostname,
43
48
  "type" => "fluentd",
44
49
  "rawConfig" => current_config,
45
- "version" => Fluent::VERSION,
50
+ "version" => create_go_semver(Fluent::VERSION),
46
51
  "edition" => "community".freeze,
47
52
  }
48
53
  if system_config.workers.to_i > 1
@@ -27,9 +27,25 @@ module Fluent::Plugin
27
27
  @emit_records_metrics.cmetrics.to_msgpack
28
28
  },
29
29
  'retry_count' => ->(){
30
- throw(:skip) unless instance_variable_defined?(:@buffer) && !@buffer.nil? && @buffer.is_a?(::Fluent::Plugin::Buffer)
30
+ throw(:skip) if @num_errors_metrics.get.nil?
31
31
  @num_errors_metrics.cmetrics.to_msgpack
32
32
  },
33
+ 'write_count' => ->(){
34
+ throw(:skip) if @write_count_metrics.get.nil?
35
+ @write_count_metrics.cmetrics.to_msgpack
36
+ },
37
+ 'rollback_count' => ->(){
38
+ throw(:skip) if @rollback_count_metrics.get.nil?
39
+ @rollback_count_metrics.cmetrics.to_msgpack
40
+ },
41
+ 'flush_time_count' => ->(){
42
+ throw(:skip) if @flush_time_count_metrics.get.nil?
43
+ @flush_time_count_metrics.cmetrics.to_msgpack
44
+ },
45
+ 'slow_flush_count' => ->(){
46
+ throw(:skip) if @slow_flush_count_metrics.get.nil?
47
+ @slow_flush_count_metrics.cmetrics.to_msgpack
48
+ },
33
49
  }
34
50
 
35
51
  def get_monitor_info(pe, opts = {})
@@ -0,0 +1,68 @@
1
+ #
2
+ # fluent-plugin-calyptia-monitoring
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+ require 'fluent/plugin/in_monitor_agent'
17
+
18
+ module Fluent::Plugin
19
+ class CalyptiaMonitoringInTailInput < MonitorAgentInput
20
+ CALYPTIA_PLUGIN_METRIC_INFO = {
21
+ 'files_opened_total' => ->(){
22
+ throw(:skip) unless instance_variable_defined?(:@metrics) && !@metrics.nil? && !@metrics.opened.get.nil?
23
+ @metrics.opened.cmetrics.to_msgpack
24
+ },
25
+ 'files_closed_total' => ->(){
26
+ throw(:skip) unless instance_variable_defined?(:@metrics) && !@metrics.nil? && !@metrics.closed.get.nil?
27
+ @metrics.closed.cmetrics.to_msgpack
28
+ },
29
+ 'files_rotated_total' => ->(){
30
+ throw(:skip) unless instance_variable_defined?(:@metrics) && !@metrics.nil? && !@metrics.rotated.get.nil?
31
+ @metrics.rotated.cmetrics.to_msgpack
32
+ },
33
+ }
34
+
35
+ def plugins_info_all(opts={})
36
+ agent_info = all_plugins.select {|pe|
37
+ pe.config['@type'] == "tail".freeze rescue nil
38
+ }
39
+
40
+ agent_info.map do |pe|
41
+ get_monitor_info(pe, opts)
42
+ end
43
+ end
44
+
45
+ def get_monitor_info(pe, opts = {})
46
+ obj = {}
47
+
48
+ obj['metrics'] = get_plugin_metric(pe)
49
+
50
+ obj
51
+ end
52
+
53
+ def get_plugin_metric(pe)
54
+ metrics = {}
55
+ CALYPTIA_PLUGIN_METRIC_INFO.each_pair { |key, code|
56
+ begin
57
+ v = pe.instance_exec(&code)
58
+ unless v.nil?
59
+ metrics[key] = v
60
+ end
61
+ rescue
62
+ end
63
+ }
64
+
65
+ metrics
66
+ end
67
+ end
68
+ end
@@ -17,10 +17,12 @@ require 'net/http'
17
17
  require 'monitor'
18
18
  require 'time'
19
19
  require 'fluent/version'
20
- require 'fluent/config/element'
21
- require 'fluent/plugin/metrics'
20
+ require 'fluent/env'
22
21
  require "fluent/plugin/input"
22
+ require "serverengine"
23
23
  require_relative "calyptia_monitoring_ext"
24
+ require_relative "calyptia_monitoring_buffer_ext"
25
+ require_relative "calyptia_monitoring_in_tail"
24
26
  require_relative "calyptia_monitoring_calyptia_api_requester"
25
27
 
26
28
  module Fluent
@@ -31,7 +33,7 @@ module Fluent
31
33
  class CreateAgentError < Fluent::ConfigError; end
32
34
  class UpdateAgentError < Fluent::ConfigError; end
33
35
 
34
- helpers :timer, :storage
36
+ helpers :timer, :storage, :child_process
35
37
 
36
38
  RPC_CONFIG_DUMP_ENDPOINT = "/api/config.getDump".freeze
37
39
  DEFAULT_STORAGE_TYPE = 'local'
@@ -52,6 +54,8 @@ module Fluent
52
54
  config_param :rate, :time, default: 30
53
55
  desc 'Setup pending metrics capacity size'
54
56
  config_param :pending_metrics_size, :size, default: DEFAULT_PENDING_METRICS_SIZE
57
+ desc 'Specify Fluentd config file path for RPC not to be available case'
58
+ config_param :fluentd_conf_path, :string, default: nil
55
59
  end
56
60
 
57
61
  def multi_workers_ready?
@@ -67,6 +71,7 @@ module Fluent
67
71
 
68
72
  def configure(conf)
69
73
  super
74
+
70
75
  config = conf.elements.select{|e| e.name == 'storage' }.first
71
76
  @storage_agent = storage_create(usage: 'calyptia_monitoring_agent', conf: config, default_type: DEFAULT_STORAGE_TYPE)
72
77
  end
@@ -83,6 +88,43 @@ module Fluent
83
88
  confs.join
84
89
  end
85
90
 
91
+ def get_masked_conf_from_conf_file
92
+ return "" unless File.exist?(@cloud_monitoring.fluentd_conf_path) # check file existence.
93
+
94
+ conf = ""
95
+ callback = ->(status) {
96
+ if status && status.success?
97
+ #nop
98
+ elsif status
99
+ log.warn "config dumper exits with error code", prog: prog, status: status.exitstatus, signal: status.termsig
100
+ else
101
+ log.warn "config dumper unexpectedly exits without exit status", prog: prog
102
+ end
103
+ }
104
+ spawn_command, arguments = if Fluent.windows?
105
+ [::ServerEngine.ruby_bin_path, File.join(File.dirname(__FILE__), "calyptia_config_dumper.rb")]
106
+ else
107
+ [File.join(File.dirname(__FILE__), "calyptia_config_dumper.rb")]
108
+ end
109
+
110
+ retval = child_process_execute(:exec_calyptia_config_dumper, spawn_command, arguments: arguments, immediate: true,
111
+ env: {"FLUENT_CONFIG_PATH" => @cloud_monitoring.fluentd_conf_path}, parallel: true, mode: [:read_with_stderr],
112
+ on_exit_callback: callback) do |io|
113
+ io.set_encoding(Encoding::ASCII_8BIT)
114
+ conf = io.read
115
+ end
116
+ unless retval.nil?
117
+ begin
118
+ Timeout.timeout(10) do
119
+ sleep 0.1 until !conf.empty?
120
+ end
121
+ rescue Timeout::Error
122
+ log.warn "cannot retrive configuration contents on #{@cloud_monitoring.fluentd_conf_path} within 10 seconds."
123
+ end
124
+ end
125
+ conf
126
+ end
127
+
86
128
  def start
87
129
  super
88
130
 
@@ -93,13 +135,17 @@ module Fluent
93
135
  end
94
136
  raise Fluent::ConfigError, "cmetrics plugin should be used to collect metrics on Calyptia Cloud" unless enabled_cmetrics
95
137
  @monitor_agent = Fluent::Plugin::CalyptiaMonitoringExtInput.new
138
+ @monitor_agent_buffer = Fluent::Plugin::CalyptiaMonitoringBufferExtInput.new
139
+ @monitor_agent_in_tail = Fluent::Plugin::CalyptiaMonitoringInTailInput.new
96
140
  @api_requester = Fluent::Plugin::CalyptiaAPI::Requester.new(@cloud_monitoring.endpoint,
97
141
  @cloud_monitoring.api_key,
98
142
  log,
99
143
  fluentd_worker_id)
100
- if check_config_sending_usability
101
- @current_config = get_current_config_from_rpc
102
- end
144
+ @current_config = if !@cloud_monitoring.fluentd_conf_path.nil?
145
+ get_masked_conf_from_conf_file
146
+ elsif check_config_sending_usability
147
+ get_current_config_from_rpc
148
+ end
103
149
 
104
150
  if @cloud_monitoring.rate < 30
105
151
  log.warn "High frequency events ingestion is not supported. Set up 30s as ingestion interval"
@@ -222,6 +268,18 @@ module Fluent
222
268
  buffer += v
223
269
  end
224
270
  }
271
+ @monitor_agent_buffer.plugins_info_all(opts).each {|record|
272
+ metrics = record["metrics"]
273
+ metrics.each_pair do |k, v|
274
+ buffer += v
275
+ end
276
+ }
277
+ @monitor_agent_in_tail.plugins_info_all(opts).each {|record|
278
+ metrics = record["metrics"]
279
+ metrics.each_pair do |k, v|
280
+ buffer += v
281
+ end
282
+ }
225
283
  if buffer.empty?
226
284
  log.debug "No initialized metrics is found. Trying to send cmetrics on the next tick."
227
285
  else
@@ -4,8 +4,10 @@
4
4
  </metrics>
5
5
  enable_input_metrics <%= @enable_input_metrics %>
6
6
  enable_size_metrics <%= @enable_size_metrics %>
7
+ <%- unless @disable_rpc -%>
7
8
  rpc_endpoint <%= @rpc_endpoint %>
8
9
  enable_get_dump <%= @enable_get_dump %>
10
+ <%- end -%>
9
11
  <system>
10
12
  <source>
11
13
  @type calyptia_monitoring
@@ -15,6 +17,9 @@
15
17
  endpoint <%= @endpoint %>
16
18
  <%- end -%>
17
19
  api_key <%= @api_key %>
20
+ <%- if @fluentd_conf_path -%>
21
+ fluentd_conf_path <%= @fluentd_conf_path %>
22
+ <%- end -%>
18
23
  </cloud_monitoring>
19
24
  <storage>
20
25
  @type local
@@ -192,6 +192,37 @@ TEXT
192
192
  path #{storage_dir}/agent_state
193
193
  </storage>
194
194
  </source>
195
+ TEXT
196
+ assert_equal(expected, dumped_config)
197
+ end
198
+
199
+ test "with api_key and fluentd_config_path" do
200
+ storage_dir = Dir.tmpdir
201
+ conf_dir = Dir.tmpdir
202
+ conf_path = File.join(conf_dir, "fluent.conf")
203
+ dumped_config = capture_stdout do
204
+ CalyptiaConfigGenerator.new(["YOUR_API_KEY", "--storage_agent_token_dir", storage_dir, "--fluentd-conf-path", conf_path]).call
205
+ end
206
+ expected =<<TEXT
207
+ <system>
208
+ <metrics>
209
+ @type cmetrics
210
+ </metrics>
211
+ enable_input_metrics true
212
+ enable_size_metrics false
213
+ <system>
214
+ <source>
215
+ @type calyptia_monitoring
216
+ @id input_caplyptia_monitoring
217
+ <cloud_monitoring>
218
+ api_key YOUR_API_KEY
219
+ fluentd_conf_path #{conf_path}
220
+ </cloud_monitoring>
221
+ <storage>
222
+ @type local
223
+ path #{storage_dir}/agent_state
224
+ </storage>
225
+ </source>
195
226
  TEXT
196
227
  assert_equal(expected, dumped_config)
197
228
  end
@@ -0,0 +1,29 @@
1
+ require "helper"
2
+ require "logger"
3
+ require "fluent/plugin/calyptia_monitoring_calyptia_api_requester"
4
+
5
+ class CalyptiaMonitoringMachineIdTest < Test::Unit::TestCase
6
+ API_KEY = 'YOUR_API_KEY'.freeze
7
+ API_ENDPOINT = "https://cloud-api.calyptia.com".freeze
8
+
9
+ setup do
10
+ @log = Logger.new($stdout)
11
+ worker_id = [1,2,3,4,5,6,7,8,9,10,11].sample
12
+ @api_requester = Fluent::Plugin::CalyptiaAPI::Requester.new(API_ENDPOINT,
13
+ API_KEY,
14
+ @log,
15
+ worker_id)
16
+ end
17
+
18
+ data("rc" => ["1.14.0.rc", "1.14.0-rc"],
19
+ "rc2" => ["1.14.0.rc2", "1.14.0-rc2"],
20
+ "alpha" => ["1.14.0.alpha", "1.14.0-alpha"],
21
+ "beta" => ["1.14.0.beta", "1.14.0-beta"],
22
+ "pre" => ["1.14.0.pre", "1.14.0-pre"],
23
+ )
24
+ def test_create_go_semver(data)
25
+ version, expected = data
26
+ actual = @api_requester.create_go_semver(version)
27
+ assert_equal expected, actual
28
+ end
29
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-calyptia-monitoring
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0.rc8
4
+ version: 0.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Hiroshi Hatake
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-08-13 00:00:00.000000000 Z
11
+ date: 2021-09-29 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -58,7 +58,7 @@ dependencies:
58
58
  requirements:
59
59
  - - ">="
60
60
  - !ruby/object:Gem::Version
61
- version: 1.13.0
61
+ version: 1.14.0
62
62
  - - "<"
63
63
  - !ruby/object:Gem::Version
64
64
  version: '2'
@@ -68,7 +68,7 @@ dependencies:
68
68
  requirements:
69
69
  - - ">="
70
70
  - !ruby/object:Gem::Version
71
- version: 1.13.0
71
+ version: 1.14.0
72
72
  - - "<"
73
73
  - !ruby/object:Gem::Version
74
74
  version: '2'
@@ -78,14 +78,14 @@ dependencies:
78
78
  requirements:
79
79
  - - ">="
80
80
  - !ruby/object:Gem::Version
81
- version: 0.1.0.rc3
81
+ version: 0.1.1
82
82
  type: :runtime
83
83
  prerelease: false
84
84
  version_requirements: !ruby/object:Gem::Requirement
85
85
  requirements:
86
86
  - - ">="
87
87
  - !ruby/object:Gem::Version
88
- version: 0.1.0.rc3
88
+ version: 0.1.1
89
89
  description: Monitoring Fluentd via Calyptia Cloud
90
90
  email:
91
91
  - hatake@calyptia.com
@@ -102,13 +102,17 @@ files:
102
102
  - bin/calyptia-config-generator
103
103
  - fluent-plugin-calyptia-monitoring.gemspec
104
104
  - lib/fluent/command/config_generator.rb
105
+ - lib/fluent/plugin/calyptia_config_dumper.rb
106
+ - lib/fluent/plugin/calyptia_monitoring_buffer_ext.rb
105
107
  - lib/fluent/plugin/calyptia_monitoring_calyptia_api_requester.rb
106
108
  - lib/fluent/plugin/calyptia_monitoring_ext.rb
109
+ - lib/fluent/plugin/calyptia_monitoring_in_tail.rb
107
110
  - lib/fluent/plugin/calyptia_monitoring_machine_id.rb
108
111
  - lib/fluent/plugin/in_calyptia_monitoring.rb
109
112
  - templates/calyptia-conf.erb
110
113
  - test/command/test_config_generator.rb
111
114
  - test/helper.rb
115
+ - test/plugin/test_calyptia_monitoring_calyptia_api_requester.rb
112
116
  - test/plugin/test_calyptia_montoring_machine_id.rb
113
117
  - test/plugin/test_in_calyptia_monitoring.rb
114
118
  homepage: https://github.com/calyptia/fluent-plugin-calyptia-monitoring
@@ -126,9 +130,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
126
130
  version: '0'
127
131
  required_rubygems_version: !ruby/object:Gem::Requirement
128
132
  requirements:
129
- - - ">"
133
+ - - ">="
130
134
  - !ruby/object:Gem::Version
131
- version: 1.3.1
135
+ version: '0'
132
136
  requirements: []
133
137
  rubygems_version: 3.2.22
134
138
  signing_key:
@@ -137,5 +141,6 @@ summary: Monitoring Fluentd via Calyptia Cloud
137
141
  test_files:
138
142
  - test/command/test_config_generator.rb
139
143
  - test/helper.rb
144
+ - test/plugin/test_calyptia_monitoring_calyptia_api_requester.rb
140
145
  - test/plugin/test_calyptia_montoring_machine_id.rb
141
146
  - test/plugin/test_in_calyptia_monitoring.rb