fluentd 1.12.0.rc2 → 1.12.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of fluentd might be problematic. Click here for more details.

checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7233372dc2b1c0ab6ae93682dc0a3ecd8ba2eb4956dfecc2456173cf30729090
4
- data.tar.gz: e61aa30c17b3f27322a10fbc4b2e81a219eeeb0f061c4ccfd0b6c8c10777a607
3
+ metadata.gz: e1d896832b5f3e1f9de589d63240f7e181d95d3c1b6f425bb200ba28f6eb837a
4
+ data.tar.gz: b5298a76eb700a0a717d474f9b2cd55a063be27dec4ce7117d60fee2c8c17bb2
5
5
  SHA512:
6
- metadata.gz: 737b0727cd390374168ed176a19a73b94d3010a7e93d5951a5580cfbca5bdd697c082afbde07ba1b266edd77be057ec7ccef6ea4383f626e7a11bcf4ad9d9c40
7
- data.tar.gz: 94f055d9d2d88359a49caf597bc9076cee741d61e80d84aa3cd1cefe8b01e7ee78721502b5b0dd4e329cb0fceeec952b43b51efd666ee7c9742fc763d1c7cb11
6
+ metadata.gz: c5f1df166ee4dd7b53913f8e042a82ed001f414f841442d683ad9db74b990923a9fdb0886a3858357b8e9be278aa335974d8dd4aaffeba5531b97c179eea0117
7
+ data.tar.gz: 2338b8ec10282dd47a5641f003a49bfce19184fb10d5df949993828b8ba1d19f6c21028ed0afab3c997c95b1a3c01fea49b231438ee29e3b6424f908a9acf227
@@ -1,3 +1,40 @@
1
+ # v1.12
2
+
3
+ ## Release v1.12.0 - 2020/01/05
4
+
5
+ ### New feature
6
+
7
+ * in_tail: Add `follow_inode` to support log rotation with wild card
8
+ https://github.com/fluent/fluentd/pull/2992
9
+ * in_tail: Handle linux capability
10
+ https://github.com/fluent/fluentd/pull/3155
11
+ https://github.com/fluent/fluentd/pull/3162
12
+ * windows: Add win32 events alternative to unix signals
13
+ https://github.com/fluent/fluentd/pull/3131
14
+
15
+ ### Enhancement
16
+
17
+ * buffer: Enable metadata comparison optimization on all platform
18
+ https://github.com/fluent/fluentd/pull/3095
19
+ * fluent-plugin-config-formatter: Handle `service_discovery` type
20
+ https://github.com/fluent/fluentd/pull/3178
21
+ * in_http: Add `add_query_params` parameter to add query params to event record
22
+ https://github.com/fluent/fluentd/pull/3197
23
+ * inject: Support `unixtime_micros` and `unixtime_nanos` in `time_type`
24
+ https://github.com/fluent/fluentd/pull/3220
25
+ * Refactoring code
26
+ https://github.com/fluent/fluentd/pull/3167
27
+ https://github.com/fluent/fluentd/pull/3170
28
+ https://github.com/fluent/fluentd/pull/3180
29
+ https://github.com/fluent/fluentd/pull/3196
30
+ https://github.com/fluent/fluentd/pull/3213
31
+ https://github.com/fluent/fluentd/pull/3222
32
+
33
+ ### Bug fix
34
+
35
+ * output: Prevent retry.step from being called too many times in a short time
36
+ https://github.com/fluent/fluentd/pull/3203
37
+
1
38
  # v1.11
2
39
 
3
40
  ## Release v1.11.5 - 2020/11/06
data/README.md CHANGED
@@ -13,7 +13,7 @@ Drone CI for Arm64:
13
13
  [Fluentd](https://www.fluentd.org/) collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure (Learn more about the [Unified Logging Layer](https://www.fluentd.org/blog/unified-logging-layer)).
14
14
 
15
15
  <p align="center">
16
- <img src="https://raw.githubusercontent.com/fluent/fluentd-docs-gitbook/1.0/images/fluentd-architecture.png" width="500px"/>
16
+ <img src="https://www.fluentd.org/images/fluentd-architecture.png" width="500px"/>
17
17
  </p>
18
18
 
19
19
  An event consists of *tag*, *time* and *record*. Tag is a string separated with '.' (e.g. myapp.access). It is used to categorize events. Time is a UNIX time recorded at occurrence of an event. Record is a JSON object.
@@ -98,6 +98,7 @@ module Fluent::Plugin
98
98
  def start
99
99
  super
100
100
 
101
+ log.info "listening tcp socket", bind: @bind, port: @port
101
102
  del_size = @delimiter.length
102
103
  if @_extract_enabled && @_extract_tag_key
103
104
  server_create(:in_tcp_server_single_emit, @port, bind: @bind, resolve_name: !!@source_hostname_key) do |data, conn|
@@ -76,7 +76,7 @@ module Fluent
76
76
  config_param :time_key, :string, default: nil
77
77
 
78
78
  # To avoid defining :time_type twice
79
- config_param :time_type, :enum, list: [:float, :unixtime, :unixtime_millis, :string], default: :float
79
+ config_param :time_type, :enum, list: [:float, :unixtime, :unixtime_millis, :unixtime_micros, :unixtime_nanos, :string], default: :float
80
80
 
81
81
  Fluent::TimeMixin::TIME_PARAMETERS.each do |name, type, opts|
82
82
  config_param(name, type, **opts)
@@ -132,7 +132,9 @@ module Fluent
132
132
  if @_inject_time_key
133
133
  @_inject_time_formatter = case @inject_config.time_type
134
134
  when :float then ->(time){ time.to_r.truncate(+6).to_f } # microsecond floating point value
135
- when :unixtime_millis then ->(time) { time.to_f.floor(3) * 1000 }
135
+ when :unixtime_millis then ->(time) { time.respond_to?(:nsec) ? time.to_i * 1_000 + time.nsec / 1_000_000 : (time * 1_000).floor }
136
+ when :unixtime_micros then ->(time) { time.respond_to?(:nsec) ? time.to_i * 1_000_000 + time.nsec / 1_000 : (time * 1_000_000).floor }
137
+ when :unixtime_nanos then ->(time) { time.respond_to?(:nsec) ? time.to_i * 1_000_000_000 + time.nsec : (time * 1_000_000_000).floor }
136
138
  when :unixtime then ->(time){ time.to_i }
137
139
  else
138
140
  localtime = @inject_config.localtime && !@inject_config.utc
@@ -50,6 +50,7 @@ module Fluent
50
50
  def to_int
51
51
  @sec
52
52
  end
53
+ alias :to_i :to_int
53
54
 
54
55
  def to_f
55
56
  @sec + @nsec / 1_000_000_000.0
@@ -16,6 +16,6 @@
16
16
 
17
17
  module Fluent
18
18
 
19
- VERSION = '1.12.0.rc2'
19
+ VERSION = '1.12.0'
20
20
 
21
21
  end
@@ -38,7 +38,7 @@ class SyslogParserTest < ::Test::Unit::TestCase
38
38
  data('regexp' => 'regexp', 'string' => 'string')
39
39
  def test_parse_with_time_format2(param)
40
40
  @parser.configure('time_format' => '%Y-%m-%dT%H:%M:%SZ', 'parser_type' => param)
41
- @parser.instance.parse('2020-03-03T10:14:29Z 192.168.0.1 fluentd[11111]: [error] Syslog test') { |time, record|
41
+ @parser.instance.parse("#{Time.now.year}-03-03T10:14:29Z 192.168.0.1 fluentd[11111]: [error] Syslog test") { |time, record|
42
42
  assert_equal(event_time('Mar 03 10:14:29', format: '%b %d %H:%M:%S'), time)
43
43
  assert_equal(@expected, record)
44
44
  }
@@ -47,7 +47,7 @@ class SyslogParserTest < ::Test::Unit::TestCase
47
47
 
48
48
  def test_parse_with_time_format_rfc5424
49
49
  @parser.configure('time_format' => '%Y-%m-%dT%H:%M:%SZ', 'message_format' => 'rfc5424')
50
- @parser.instance.parse('2020-03-03T10:14:29Z 192.168.0.1 fluentd 11111 - - [error] Syslog test') { |time, record|
50
+ @parser.instance.parse("#{Time.now.year}-03-03T10:14:29Z 192.168.0.1 fluentd 11111 - - [error] Syslog test") { |time, record|
51
51
  assert_equal(event_time('Mar 03 10:14:29', format: '%b %d %H:%M:%S'), time)
52
52
  assert_equal(@expected.merge('host' => '192.168.0.1', 'msgid' => '-', 'extradata' => '-'), record)
53
53
  }
@@ -187,6 +187,35 @@ class InjectHelperTest < Test::Unit::TestCase
187
187
 
188
188
  record = {"key1" => "value1", "key2" => 2}
189
189
  assert_equal record.merge({"timedata" => unixtime_millis}), @d.inject_values_to_record('tag', time, record)
190
+ assert_equal record.merge({"timedata" => time_in_unix * 1_000}), @d.inject_values_to_record('tag', time_in_unix, record)
191
+ end
192
+
193
+ test 'injects time as unix time micros into specified key' do
194
+ time_in_unix = Time.parse("2016-06-21 08:10:11 +0900").to_i
195
+ time_subsecond = 320_101_224
196
+ time = Fluent::EventTime.new(time_in_unix, time_subsecond)
197
+ unixtime_micros = 1466464211320101
198
+
199
+ @d.configure(config_inject_section("time_key" => "timedata", "time_type" => "unixtime_micros"))
200
+ @d.start
201
+
202
+ record = {"key1" => "value1", "key2" => 2}
203
+ assert_equal record.merge({"timedata" => unixtime_micros}), @d.inject_values_to_record('tag', time, record)
204
+ assert_equal record.merge({"timedata" => time_in_unix * 1_000_000}), @d.inject_values_to_record('tag', time_in_unix, record)
205
+ end
206
+
207
+ test 'injects time as unix time nanos into specified key' do
208
+ time_in_unix = Time.parse("2016-06-21 08:10:11 +0900").to_i
209
+ time_subsecond = 320_101_224
210
+ time = Fluent::EventTime.new(time_in_unix, time_subsecond)
211
+ unixtime_nanos = 1466464211320101224
212
+
213
+ @d.configure(config_inject_section("time_key" => "timedata", "time_type" => "unixtime_nanos"))
214
+ @d.start
215
+
216
+ record = {"key1" => "value1", "key2" => 2}
217
+ assert_equal record.merge({"timedata" => unixtime_nanos}), @d.inject_values_to_record('tag', time, record)
218
+ assert_equal record.merge({"timedata" => time_in_unix * 1_000_000_000}), @d.inject_values_to_record('tag', time_in_unix, record)
190
219
  end
191
220
 
192
221
  test 'injects time as unix time into specified key' do
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluentd
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.12.0.rc2
4
+ version: 1.12.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Sadayuki Furuhashi
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-12-22 00:00:00.000000000 Z
11
+ date: 2021-01-05 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -924,9 +924,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
924
924
  version: '2.4'
925
925
  required_rubygems_version: !ruby/object:Gem::Requirement
926
926
  requirements:
927
- - - ">"
927
+ - - ">="
928
928
  - !ruby/object:Gem::Version
929
- version: 1.3.1
929
+ version: '0'
930
930
  requirements: []
931
931
  rubygems_version: 3.0.3
932
932
  signing_key: