fluentd 1.6.0 → 1.6.1

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of fluentd might be problematic. Click here for more details.

checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8d97f061ed5f540026a4cae384b2508ee503be4dcb0ef836d37e118a541477df
4
- data.tar.gz: 82edeaa9350871f8ba1b38d966cfc331f56c5ba97464c3dd14fb2bf7005a1bb5
3
+ metadata.gz: 1fef23fb6e94b2100154a4ef1618ad628435118b13fc9c1f6884e69ecc221368
4
+ data.tar.gz: 6d1eb5b941ce9df4987fcf19f0345d46965534d858205ed07c71c67e7b8c2966
5
5
  SHA512:
6
- metadata.gz: 7966763ba1215bde1a571f500dedaf5a100b6e72f14fb8b8121e09ca7d3fe697e4c6910b3dd7f844bd3d86484f0ea656997baff81fd9332aab94a898ff2e74d7
7
- data.tar.gz: be65ac015fc43ac71d52ad3a2bb628e904a285918588b31102ecdf51e84af689ef77007028b30c72a9ecc25aa01e50a88091f136829396bf7cacaa608312b591
6
+ metadata.gz: b426456c884540d5bd9aafbd0c7e3f5dec5a64db50ce631a09a93d2dfee1adeff8f96f051ffc4702ac464ea91d05a485866838f0ea383380d397a3073363da67
7
+ data.tar.gz: 34791ef0f1498fc68e27935ed3d9f1393d1aaa28f39ee1718f679f5aef2c8c0094301d49edbde5b793651a8226efe7f070ab4f422dffe7da2a6bc833f297cf6a
@@ -1,5 +1,21 @@
1
1
  # v1.6
2
2
 
3
+ ## Release v1.6.1 - 2019/07/10
4
+
5
+ ### Enhancement
6
+
7
+ * socket/cert: Support all private keys OpenSSL supports, not only RSA.
8
+ https://github.com/fluent/fluentd/pull/2487
9
+ * output/buffer: Improve statistics method performance
10
+ https://github.com/fluent/fluentd/pull/2491
11
+
12
+ ### Bug fixes
13
+
14
+ * plugin_config_formatter: update new doc URL
15
+ https://github.com/fluent/fluentd/pull/2481
16
+ * out_forward: Avoid zero division error when there are no available nodes
17
+ https://github.com/fluent/fluentd/pull/2482
18
+
3
19
  ## Release v1.6.0 - 2019/07/01
4
20
 
5
21
  ### New feature
data/README.md CHANGED
@@ -13,7 +13,7 @@ GitLab CI:
13
13
  [Fluentd](https://www.fluentd.org/) collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure (Learn more about the [Unified Logging Layer](https://www.fluentd.org/blog/unified-logging-layer)).
14
14
 
15
15
  <p align="center">
16
- <img src="https://docs.fluentd.org/images/fluentd-architecture.png" width="500px"/>
16
+ <img src="https://raw.githubusercontent.com/fluent/fluentd-docs-gitbook/1.0/images/fluentd-architecture.png" width="500px"/>
17
17
  </p>
18
18
 
19
19
  An event consists of *tag*, *time* and *record*. Tag is a string separated with '.' (e.g. myapp.access). It is used to categorize events. Time is a UNIX time recorded at occurrence of an event. Record is a JSON object.
@@ -91,7 +91,7 @@ HELP
91
91
  def self.generate_server_pair(opts={})
92
92
  key = OpenSSL::PKey::RSA.generate(opts[:private_key_length])
93
93
 
94
- ca_key = OpenSSL::PKey::RSA.new(File.read(opts[:ca_key_path]), opts[:ca_key_passphrase])
94
+ ca_key = OpenSSL::PKey::read(File.read(opts[:ca_key_path]), opts[:ca_key_passphrase])
95
95
  ca_cert = OpenSSL::X509::Certificate.new(File.read(opts[:ca_cert_path]))
96
96
  issuer = ca_cert.issuer
97
97
 
@@ -32,8 +32,8 @@ class FluentPluginConfigFormatter
32
32
  "buffer", "parser", "formatter", "storage"
33
33
  ]
34
34
 
35
- DOCS_BASE_URL = "https://docs.fluentd.org/v1.0/articles/quickstart"
36
- DOCS_ARTICLE_BASE_URL = "https://docs.fluentd.org/v1.0/articles/"
35
+ DOCS_BASE_URL = "https://docs.fluentd.org/v/1.0"
36
+ DOCS_PLUGIN_HELPER_BASE_URL = "#{DOCS_BASE_URL}/plugin-helper-overview/"
37
37
 
38
38
  def initialize(argv = ARGV)
39
39
  @argv = argv
@@ -198,7 +198,7 @@ class FluentPluginConfigFormatter
198
198
  end
199
199
 
200
200
  def plugin_helper_url(plugin_helper)
201
- "#{DOCS_ARTICLE_BASE_URL}api-plugin-helper-#{plugin_helper}"
201
+ "#{DOCS_PLUGIN_HELPER_BASE_URL}api-plugin-helper-#{plugin_helper}"
202
202
  end
203
203
 
204
204
  def plugin_helper_markdown_link(plugin_helper)
@@ -207,7 +207,7 @@ class FluentPluginConfigFormatter
207
207
 
208
208
  def plugin_overview_url(class_name)
209
209
  plugin_type = class_name.slice(/::(\w+)\z/, 1).downcase
210
- "#{DOCS_ARTICLE_BASE_URL}#{plugin_type}-plugin-overview"
210
+ "#{DOCS_BASE_URL}/#{plugin_type}#overview"
211
211
  end
212
212
 
213
213
  def plugin_overview_markdown_link(class_name)
@@ -741,15 +741,27 @@ module Fluent
741
741
  retry
742
742
  end
743
743
 
744
+ STATS_KEYS = [
745
+ 'stage_length',
746
+ 'stage_byte_size',
747
+ 'queue_length',
748
+ 'queue_byte_size',
749
+ 'available_buffer_space_ratios',
750
+ 'total_queued_size',
751
+ 'oldest_timekey',
752
+ 'newest_timekey'
753
+ ]
754
+
744
755
  def statistics
745
- buffer_space = 1.0 - ((@stage_size + @queue_size * 1.0) / @total_limit_size).round
756
+ stage_size, queue_size = @stage_size, @queue_size
757
+ buffer_space = 1.0 - ((stage_size + queue_size * 1.0) / @total_limit_size).round
746
758
  stats = {
747
759
  'stage_length' => @stage.size,
748
- 'stage_byte_size' => @stage_size,
760
+ 'stage_byte_size' => stage_size,
749
761
  'queue_length' => @queue.size,
750
- 'queue_byte_size' => @queue_size,
762
+ 'queue_byte_size' => queue_size,
751
763
  'available_buffer_space_ratios' => buffer_space * 100,
752
- 'total_queued_size' => @stage_size + @queue_size,
764
+ 'total_queued_size' => stage_size + queue_size,
753
765
  }
754
766
 
755
767
  if (m = timekeys.min)
@@ -428,6 +428,12 @@ module Fluent::Plugin
428
428
  end
429
429
 
430
430
  weight_array = []
431
+ if regular_nodes.empty?
432
+ log.warn('No nodes are available')
433
+ @weight_array = weight_array
434
+ return @weight_array
435
+ end
436
+
431
437
  gcd = regular_nodes.map {|n| n.weight }.inject(0) {|r,w| r.gcd(w) }
432
438
  regular_nodes.each {|n|
433
439
  (n.weight / gcd).times {
@@ -16,6 +16,7 @@
16
16
 
17
17
  require 'fluent/error'
18
18
  require 'fluent/plugin/base'
19
+ require 'fluent/plugin/buffer'
19
20
  require 'fluent/plugin_helper/record_accessor'
20
21
  require 'fluent/log'
21
22
  require 'fluent/plugin_id'
@@ -1466,6 +1467,11 @@ module Fluent
1466
1467
  end
1467
1468
  end
1468
1469
 
1470
+ BUFFER_STATS_KEYS = {}
1471
+ Fluent::Plugin::Buffer::STATS_KEYS.each { |key|
1472
+ BUFFER_STATS_KEYS[key] = "buffer_#{key}"
1473
+ }
1474
+
1469
1475
  def statistics
1470
1476
  stats = {
1471
1477
  'emit_records' => @emit_records,
@@ -1481,7 +1487,7 @@ module Fluent
1481
1487
 
1482
1488
  if @buffer && @buffer.respond_to?(:statistics)
1483
1489
  (@buffer.statistics['buffer'] || {}).each do |k, v|
1484
- stats["buffer_#{k}"] = v
1490
+ stats[BUFFER_STATS_KEYS[k]] = v
1485
1491
  end
1486
1492
  end
1487
1493
 
@@ -76,7 +76,7 @@ module Fluent
76
76
  end
77
77
 
78
78
  def cert_option_load(cert_path, private_key_path, private_key_passphrase)
79
- key = OpenSSL::PKey::RSA.new(File.read(private_key_path), private_key_passphrase)
79
+ key = OpenSSL::PKey::read(File.read(private_key_path), private_key_passphrase)
80
80
  certs = cert_option_certificates_from_file(cert_path)
81
81
  cert = certs.shift
82
82
  return cert, key, certs
@@ -137,7 +137,7 @@ module Fluent
137
137
  end
138
138
 
139
139
  def cert_option_generate_server_pair_by_ca(ca_cert_path, ca_key_path, ca_key_passphrase, generate_opts)
140
- ca_key = OpenSSL::PKey::RSA.new(File.read(ca_key_path), ca_key_passphrase)
140
+ ca_key = OpenSSL::PKey::read(File.read(ca_key_path), ca_key_passphrase)
141
141
  ca_cert = OpenSSL::X509::Certificate.new(File.read(ca_cert_path))
142
142
  cert, key = cert_option_generate_pair(generate_opts, ca_cert.subject)
143
143
  raise "BUG: certificate digest algorithm not set" unless generate_opts[:digest]
@@ -139,7 +139,7 @@ module Fluent
139
139
  context.cert_store = cert_store
140
140
  context.verify_hostname = true if verify_fqdn && fqdn && context.respond_to?(:verify_hostname=)
141
141
  context.cert = OpenSSL::X509::Certificate.new(File.read(cert_path)) if cert_path
142
- context.key = OpenSSL::PKey::RSA.new(File.read(private_key_path), private_key_passphrase) if private_key_path
142
+ context.key = OpenSSL::PKey::read(File.read(private_key_path), private_key_passphrase) if private_key_path
143
143
  end
144
144
 
145
145
  tcpsock = socket_create_tcp(host, port, **kwargs)
@@ -16,6 +16,6 @@
16
16
 
17
17
  module Fluent
18
18
 
19
- VERSION = '1.6.0'
19
+ VERSION = '1.6.1'
20
20
 
21
21
  end
@@ -180,10 +180,10 @@ TEXT
180
180
  expected = <<TEXT
181
181
  ## Plugin helpers
182
182
 
183
- * [inject](https://docs.fluentd.org/v1.0/articles/api-plugin-helper-inject)
184
- * [compat_parameters](https://docs.fluentd.org/v1.0/articles/api-plugin-helper-compat_parameters)
183
+ * [inject](https://docs.fluentd.org/v/1.0/plugin-helper-overview/api-plugin-helper-inject)
184
+ * [compat_parameters](https://docs.fluentd.org/v/1.0/plugin-helper-overview/api-plugin-helper-compat_parameters)
185
185
 
186
- * See also: [Input Plugin Overview](https://docs.fluentd.org/v1.0/articles/input-plugin-overview)
186
+ * See also: [Input Plugin Overview](https://docs.fluentd.org/v/1.0/input#overview)
187
187
 
188
188
  ## TestFluentPluginConfigFormatter::SimpleInput
189
189
 
@@ -203,10 +203,10 @@ TEXT
203
203
  expected = <<TEXT
204
204
  ## Plugin helpers
205
205
 
206
- * [inject](https://docs.fluentd.org/v1.0/articles/api-plugin-helper-inject)
207
- * [compat_parameters](https://docs.fluentd.org/v1.0/articles/api-plugin-helper-compat_parameters)
206
+ * [inject](https://docs.fluentd.org/v/1.0/plugin-helper-overview/api-plugin-helper-inject)
207
+ * [compat_parameters](https://docs.fluentd.org/v/1.0/plugin-helper-overview/api-plugin-helper-compat_parameters)
208
208
 
209
- * See also: [Output Plugin Overview](https://docs.fluentd.org/v1.0/articles/output-plugin-overview)
209
+ * See also: [Output Plugin Overview](https://docs.fluentd.org/v/1.0/output#overview)
210
210
 
211
211
  ## TestFluentPluginConfigFormatter::ComplexOutput
212
212
 
@@ -1010,6 +1010,20 @@ EOL
1010
1010
  end
1011
1011
  end
1012
1012
 
1013
+ test 'if no available node' do
1014
+ # do not create output driver
1015
+ d = create_driver(%[
1016
+ <server>
1017
+ name test
1018
+ standby
1019
+ host #{TARGET_HOST}
1020
+ port #{TARGET_PORT}
1021
+ </server>
1022
+ ])
1023
+ d.instance_start
1024
+ assert_nothing_raised { d.run }
1025
+ end
1026
+
1013
1027
  sub_test_case 'keepalive' do
1014
1028
  test 'Do not create connection per send_data' do
1015
1029
  target_input_driver = create_target_input_driver(conf: TARGET_CONFIG)
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluentd
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.6.0
4
+ version: 1.6.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Sadayuki Furuhashi
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-07-02 00:00:00.000000000 Z
11
+ date: 2019-07-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: msgpack