fluentd 1.12.0.rc1 → 1.12.3

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of fluentd might be problematic. Click here for more details.

Files changed (85) hide show
  1. checksums.yaml +4 -4
  2. data/.deepsource.toml +13 -0
  3. data/.github/ISSUE_TEMPLATE/bug_report.md +1 -1
  4. data/.github/ISSUE_TEMPLATE/config.yml +5 -0
  5. data/.github/workflows/linux-test.yaml +36 -0
  6. data/.github/workflows/macos-test.yaml +30 -0
  7. data/.github/workflows/stale-actions.yml +22 -0
  8. data/.github/workflows/windows-test.yaml +35 -0
  9. data/.gitlab-ci.yml +41 -19
  10. data/CHANGELOG.md +151 -0
  11. data/MAINTAINERS.md +5 -2
  12. data/README.md +7 -4
  13. data/bin/fluent-ctl +7 -0
  14. data/fluentd.gemspec +5 -4
  15. data/lib/fluent/command/bundler_injection.rb +1 -1
  16. data/lib/fluent/command/ca_generate.rb +6 -3
  17. data/lib/fluent/command/cat.rb +0 -1
  18. data/lib/fluent/command/ctl.rb +177 -0
  19. data/lib/fluent/command/fluentd.rb +4 -0
  20. data/lib/fluent/command/plugin_config_formatter.rb +18 -2
  21. data/lib/fluent/command/plugin_generator.rb +31 -1
  22. data/lib/fluent/compat/parser.rb +2 -2
  23. data/lib/fluent/config/section.rb +2 -2
  24. data/lib/fluent/config/types.rb +2 -2
  25. data/lib/fluent/event.rb +3 -13
  26. data/lib/fluent/load.rb +0 -1
  27. data/lib/fluent/plugin.rb +5 -0
  28. data/lib/fluent/plugin/file_wrapper.rb +39 -3
  29. data/lib/fluent/plugin/formatter_ltsv.rb +2 -2
  30. data/lib/fluent/plugin/in_http.rb +24 -3
  31. data/lib/fluent/plugin/in_monitor_agent.rb +1 -1
  32. data/lib/fluent/plugin/in_tail.rb +121 -40
  33. data/lib/fluent/plugin/in_tail/position_file.rb +39 -14
  34. data/lib/fluent/plugin/in_tcp.rb +1 -0
  35. data/lib/fluent/plugin/out_copy.rb +18 -5
  36. data/lib/fluent/plugin/out_exec_filter.rb +3 -3
  37. data/lib/fluent/plugin/out_forward.rb +61 -28
  38. data/lib/fluent/plugin/out_http.rb +9 -2
  39. data/lib/fluent/plugin/output.rb +18 -10
  40. data/lib/fluent/plugin/parser_csv.rb +2 -2
  41. data/lib/fluent/plugin/parser_syslog.rb +2 -2
  42. data/lib/fluent/plugin/storage_local.rb +4 -4
  43. data/lib/fluent/plugin_helper/http_server/compat/server.rb +1 -1
  44. data/lib/fluent/plugin_helper/inject.rb +4 -2
  45. data/lib/fluent/plugin_helper/retry_state.rb +4 -0
  46. data/lib/fluent/plugin_helper/server.rb +4 -2
  47. data/lib/fluent/plugin_helper/socket_option.rb +2 -2
  48. data/lib/fluent/supervisor.rb +153 -47
  49. data/lib/fluent/system_config.rb +2 -1
  50. data/lib/fluent/time.rb +58 -1
  51. data/lib/fluent/version.rb +1 -1
  52. data/lib/fluent/winsvc.rb +22 -4
  53. data/templates/new_gem/fluent-plugin.gemspec.erb +3 -3
  54. data/templates/plugin_config_formatter/param.md-table.erb +10 -0
  55. data/test/command/test_ctl.rb +57 -0
  56. data/test/command/test_fluentd.rb +38 -0
  57. data/test/command/test_plugin_config_formatter.rb +124 -2
  58. data/test/config/test_configurable.rb +1 -1
  59. data/test/plugin/in_tail/test_position_file.rb +46 -26
  60. data/test/plugin/test_file_wrapper.rb +105 -0
  61. data/test/plugin/test_in_exec.rb +1 -1
  62. data/test/plugin/test_in_http.rb +25 -0
  63. data/test/plugin/test_in_tail.rb +456 -41
  64. data/test/plugin/test_out_copy.rb +87 -0
  65. data/test/plugin/test_out_forward.rb +94 -6
  66. data/test/plugin/test_out_http.rb +20 -1
  67. data/test/plugin/test_output.rb +15 -3
  68. data/test/plugin/test_output_as_buffered_backup.rb +2 -0
  69. data/test/plugin/test_parser_csv.rb +14 -0
  70. data/test/plugin/test_parser_syslog.rb +16 -2
  71. data/test/plugin/test_sd_file.rb +1 -1
  72. data/test/plugin_helper/service_discovery/test_manager.rb +1 -1
  73. data/test/plugin_helper/test_child_process.rb +5 -2
  74. data/test/plugin_helper/test_http_server_helper.rb +4 -2
  75. data/test/plugin_helper/test_inject.rb +29 -0
  76. data/test/plugin_helper/test_server.rb +26 -7
  77. data/test/test_event.rb +16 -0
  78. data/test/test_formatter.rb +30 -0
  79. data/test/test_output.rb +2 -2
  80. data/test/test_supervisor.rb +133 -10
  81. data/test/test_time_parser.rb +109 -0
  82. metadata +43 -32
  83. data/.github/stale.yml +0 -22
  84. data/.travis.yml +0 -77
  85. data/appveyor.yml +0 -31
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ea33aa9fbbca10538b329676df8dc024f38836b8c0a3bd78ff8afe9b21e59a27
4
- data.tar.gz: d3500418626d7f49305b4dee07ab6dfa10c84687ee754fd967aee1200a815dc2
3
+ metadata.gz: 63664b249be827ef09d4b9b96807e455481255742203898de8d5dd83b232707c
4
+ data.tar.gz: fe6020cec7c43a62d770b9e7a72040dee8220fc64c109365efafe6dbb3c0ea33
5
5
  SHA512:
6
- metadata.gz: 7a43d215a3a026042028643ca627f81411379efe8eb5090c4c98f8886c22efe33595362b82273889b97ecc8a95af8c93d0d2dbf5d8494c3c8e680942dd8c0c67
7
- data.tar.gz: a17be6c1b92193649412e223ce7bba96d94f1122b309e459dc72a6e7f0a5339874e2a9d9d352a88612c3d7e603c2276b8109af97b8b26cc3967d81cae7925071
6
+ metadata.gz: 3507b87e4b8ab5744474b6ff2142f84aa5a7a889e5088ad18b6e436db68a385942b851f49c7de855495cddced9cc6b6128ff84003b7ba194a1011e98f4358ef2
7
+ data.tar.gz: db3bf1ba907a491d82b7b0dc136921d2950726264f38b4419d961f198aad2603a87f7b63a110fe730b312bc9950c6e2a61ed816ef5e653d80da84c0808fb36b3
data/.deepsource.toml ADDED
@@ -0,0 +1,13 @@
1
+ version = 1
2
+
3
+ test_patterns = ["test/**/test_*.rb"]
4
+
5
+ exclude_patterns = [
6
+ "bin/**",
7
+ "docs/**",
8
+ "example/**"
9
+ ]
10
+
11
+ [[analyzers]]
12
+ name = "ruby"
13
+ enabled = true
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: Bug Report
3
- about: Create a report to help us improve. If you have questions about Fluentd and plugins, please direct these to https://groups.google.com/forum/#!forum/fluentd
3
+ about: Create a report with a procedure for reproducing the bug
4
4
 
5
5
  ---
6
6
 
@@ -0,0 +1,5 @@
1
+ blank_issues_enabled: false
2
+ contact_links:
3
+ - name: Ask a Question
4
+ url: https://discuss.fluentd.org/
5
+ about: I have questions about Fluentd and plugins. Please ask and answer questions at https://discuss.fluentd.org/.
@@ -0,0 +1,36 @@
1
+ name: Testing on Ubuntu
2
+
3
+ on:
4
+ push:
5
+ branches: [master]
6
+ pull_request:
7
+ branches: [master]
8
+
9
+ jobs:
10
+ test:
11
+ runs-on: ${{ matrix.os }}
12
+ continue-on-error: ${{ matrix.experimental }}
13
+ strategy:
14
+ fail-fast: false
15
+ matrix:
16
+ ruby-version: ['3.0', '2.7', '2.6', '2.5']
17
+ os: [ubuntu-latest]
18
+ experimental: [false]
19
+ include:
20
+ - ruby-version: head
21
+ os: ubuntu-latest
22
+ experimental: true
23
+
24
+ name: Unit testing with Ruby ${{ matrix.ruby-version }} on ${{ matrix.os }}
25
+ steps:
26
+ - uses: actions/checkout@v2
27
+ - name: Set up Ruby
28
+ uses: ruby/setup-ruby@v1
29
+ with:
30
+ ruby-version: ${{ matrix.ruby-version }}
31
+ - name: Install addons
32
+ run: sudo apt-get install libgmp3-dev libcap-ng-dev
33
+ - name: Install dependencies
34
+ run: bundle install
35
+ - name: Run tests
36
+ run: bundle exec rake test
@@ -0,0 +1,30 @@
1
+ name: Testing on macOS
2
+
3
+ on:
4
+ push:
5
+ branches: [master]
6
+ pull_request:
7
+ branches: [master]
8
+
9
+ jobs:
10
+ test:
11
+ runs-on: ${{ matrix.os }}
12
+ continue-on-error: ${{ matrix.experimental }}
13
+ strategy:
14
+ fail-fast: false
15
+ matrix:
16
+ ruby-version: ['head', '2.7']
17
+ os: [macos-latest]
18
+ experimental: [true]
19
+
20
+ name: Unit testing with Ruby ${{ matrix.ruby-version }} on ${{ matrix.os }}
21
+ steps:
22
+ - uses: actions/checkout@v2
23
+ - name: Set up Ruby
24
+ uses: ruby/setup-ruby@v1
25
+ with:
26
+ ruby-version: ${{ matrix.ruby-version }}
27
+ - name: Install dependencies
28
+ run: bundle install
29
+ - name: Run tests
30
+ run: bundle exec rake test
@@ -0,0 +1,22 @@
1
+ name: "Mark or close stale issues and PRs"
2
+ on:
3
+ schedule:
4
+ - cron: "00 10 * * *"
5
+
6
+ jobs:
7
+ stale:
8
+ runs-on: ubuntu-latest
9
+ steps:
10
+ - uses: actions/stale@v3
11
+ with:
12
+ repo-token: ${{ secrets.GITHUB_TOKEN }}
13
+ days-before-stale: 90
14
+ days-before-close: 30
15
+ stale-issue-message: "This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days"
16
+ stale-pr-message: "This PR has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this PR will be closed in 30 days"
17
+ close-issue-message: "This issue was automatically closed because of stale in 30 days"
18
+ close-pr-message: "This PR was automatically closed because of stale in 30 days"
19
+ stale-pr-label: "stale"
20
+ stale-issue-label: "stale"
21
+ exempt-issue-labels: "bug,enhancement,feature request,pending,work_in_progress,v1,v2"
22
+ exempt-pr-labels: "bug,enhancement,feature request,pending,work_in_progress,v1,v2"
@@ -0,0 +1,35 @@
1
+ name: Testing on Windows
2
+
3
+ on:
4
+ push:
5
+ branches: [master]
6
+ pull_request:
7
+ branches: [master]
8
+
9
+ jobs:
10
+ test:
11
+ runs-on: ${{ matrix.os }}
12
+ continue-on-error: ${{ matrix.experimental }}
13
+ strategy:
14
+ fail-fast: false
15
+ matrix:
16
+ ruby-version: ['2.7', '2.6', '2.5']
17
+ os:
18
+ - windows-latest
19
+ experimental: [false]
20
+ include:
21
+ - ruby-version: '3.0'
22
+ os: windows-latest
23
+ experimental: true
24
+
25
+ name: Unit testing with Ruby ${{ matrix.ruby-version }} on ${{ matrix.os }}
26
+ steps:
27
+ - uses: actions/checkout@v2
28
+ - name: Set up Ruby
29
+ uses: ruby/setup-ruby@v1
30
+ with:
31
+ ruby-version: ${{ matrix.ruby-version }}
32
+ - name: Install dependencies
33
+ run: ridk exec bundle install
34
+ - name: Run tests
35
+ run: bundle exec rake test TESTOPTS=-v
data/.gitlab-ci.yml CHANGED
@@ -1,6 +1,6 @@
1
1
  .install-template:
2
2
  before_script:
3
- - gem update --system=2.7.8
3
+ - gem update --system
4
4
  - ruby -v
5
5
 
6
6
  .test-template:
@@ -13,34 +13,44 @@ stages:
13
13
  - build
14
14
  - tests
15
15
 
16
- 2-4-5:
17
- image: "ruby:2.4.5"
16
+ 2-5-8:
17
+ image: "ruby:2.5.8"
18
18
  stage: build
19
19
  extends: .install-template
20
20
  script:
21
21
  - bundle install --jobs=3 --retry=3
22
22
  cache:
23
- key: "$CI_COMMIT_REF_SLUG 2-4-5"
23
+ key: "$CI_COMMIT_REF_SLUG 2-5-8"
24
24
  paths:
25
25
 
26
- 2-5-5:
27
- image: "ruby:2.5.5"
26
+ 2-6-6:
27
+ image: "ruby:2.6.6"
28
28
  stage: build
29
29
  extends: .install-template
30
30
  script:
31
31
  - bundle install --jobs=3 --retry=3
32
32
  cache:
33
- key: "$CI_COMMIT_REF_SLUG 2-5-5"
33
+ key: "$CI_COMMIT_REF_SLUG 2-6-6"
34
34
  paths:
35
35
 
36
- 2-6-3:
37
- image: "ruby:2.6.3"
36
+ 2-7-2:
37
+ image: "ruby:2.7.2"
38
38
  stage: build
39
39
  extends: .install-template
40
40
  script:
41
41
  - bundle install --jobs=3 --retry=3
42
42
  cache:
43
- key: "$CI_COMMIT_REF_SLUG 2-6-3"
43
+ key: "$CI_COMMIT_REF_SLUG 2-7-2"
44
+ paths:
45
+
46
+ 3-0-0:
47
+ image: "ruby:3.0.0"
48
+ stage: build
49
+ extends: .install-template
50
+ script:
51
+ - bundle install --jobs=3 --retry=3
52
+ cache:
53
+ key: "$CI_COMMIT_REF_SLUG 3-0-0"
44
54
  paths:
45
55
 
46
56
  rubyhead:
@@ -54,39 +64,51 @@ rubyhead:
54
64
  paths:
55
65
  - ./*
56
66
 
57
- 2-4-5-test:
58
- image: "ruby:2.4.5"
67
+ 2-5-8-test:
68
+ image: "ruby:2.5.8"
69
+ stage: tests
70
+ allow_failure: true
71
+ extends: .test-template
72
+ script:
73
+ - bundle exec rake test
74
+ cache:
75
+ key: "$CI_COMMIT_REF_SLUG 2-5-8"
76
+ paths:
77
+ - ./*
78
+
79
+ 2-6-6-test:
80
+ image: "ruby:2.6.6"
59
81
  stage: tests
60
82
  allow_failure: true
61
83
  extends: .test-template
62
84
  script:
63
85
  - bundle exec rake test
64
86
  cache:
65
- key: "$CI_COMMIT_REF_SLUG 2-4-5"
87
+ key: "$CI_COMMIT_REF_SLUG 2-6-6"
66
88
  paths:
67
89
  - ./*
68
90
 
69
- 2-5-5-test:
70
- image: "ruby:2.5.5"
91
+ 2-7-2-test:
92
+ image: "ruby:2.7.2"
71
93
  stage: tests
72
94
  allow_failure: true
73
95
  extends: .test-template
74
96
  script:
75
97
  - bundle exec rake test
76
98
  cache:
77
- key: "$CI_COMMIT_REF_SLUG 2-5-5"
99
+ key: "$CI_COMMIT_REF_SLUG 2-7-2"
78
100
  paths:
79
101
  - ./*
80
102
 
81
- 2-6-3-test:
82
- image: "ruby:2.6.3"
103
+ 3-0-0-test:
104
+ image: "ruby:3.0.0"
83
105
  stage: tests
84
106
  allow_failure: true
85
107
  extends: .test-template
86
108
  script:
87
109
  - bundle exec rake test
88
110
  cache:
89
- key: "$CI_COMMIT_REF_SLUG 2-6-3"
111
+ key: "$CI_COMMIT_REF_SLUG 3-0-0"
90
112
  paths:
91
113
  - ./*
92
114
 
data/CHANGELOG.md CHANGED
@@ -1,3 +1,154 @@
1
+ # v1.12
2
+
3
+ ## Release v1.12.3 - 2021/04/23
4
+
5
+ ### Enhancement
6
+
7
+ * plugin_helper: Allow TLS to use keep-alive socket option
8
+ https://github.com/fluent/fluentd/pull/3308
9
+
10
+ ### Bug fix
11
+
12
+ * parser_csv, parser_syslog: Fix a naming conflict on parser_type
13
+ https://github.com/fluent/fluentd/pull/3302
14
+ * in_tail: Fix incorrect error code & message on Windows
15
+ https://github.com/fluent/fluentd/pull/3325
16
+ https://github.com/fluent/fluentd/pull/3329
17
+ https://github.com/fluent/fluentd/pull/3331
18
+ https://github.com/fluent/fluentd/pull/3337
19
+ * in_tail: Fix a crash bug on catching a short-lived log
20
+ https://github.com/fluent/fluentd/pull/3328
21
+ * storage_local: Fix position file corruption issue on concurrent gracefulReloads
22
+ https://github.com/fluent/fluentd/pull/3335
23
+ * Fix incorrect warnings about ${chunk_id} with out_s3
24
+ https://github.com/fluent/fluentd/pull/3339
25
+ * TLS Server: Add peer information to error log message
26
+ https://github.com/fluent/fluentd/pull/3330
27
+
28
+ ### Misc
29
+
30
+ * fluent-plugin-generate: add note about plugin name
31
+ https://github.com/fluent/fluentd/pull/3303
32
+ * fluent-plugin-generate: Use same depended gem version with fluentd
33
+ https://github.com/fluent/fluentd/pull/3305
34
+ * Fix some broken unit tests and improve CI's stability
35
+ https://github.com/fluent/fluentd/pull/3304
36
+ https://github.com/fluent/fluentd/pull/3307
37
+ https://github.com/fluent/fluentd/pull/3312
38
+ https://github.com/fluent/fluentd/pull/3313
39
+ https://github.com/fluent/fluentd/pull/3314
40
+ https://github.com/fluent/fluentd/pull/3316
41
+ https://github.com/fluent/fluentd/pull/3336
42
+ * Permit to install with win32-service 2.2.0 on Windows
43
+ https://github.com/fluent/fluentd/pull/3343
44
+
45
+ ## Release v1.12.2 - 2021/03/29
46
+
47
+ ### Enhancement
48
+
49
+ * out_copy: Add ignore_if_prev_successes
50
+ https://github.com/fluent/fluentd/pull/3190
51
+ https://github.com/fluent/fluentd/pull/3287
52
+ * Support multiple kind of timestamp format
53
+ https://github.com/fluent/fluentd/pull/3252
54
+ * formatter_ltsv: suppress delimiters in output
55
+ https://github.com/fluent/fluentd/pull/1666
56
+ https://github.com/fluent/fluentd/pull/3288
57
+ https://github.com/fluent/fluentd/pull/3289
58
+
59
+ ### Bug fix
60
+
61
+ * in_tail: Expect ENOENT during stat
62
+ https://github.com/fluent/fluentd/pull/3275
63
+ * out_forward: Prevent transferring duplicate logs on restart
64
+ https://github.com/fluent/fluentd/pull/3267
65
+ https://github.com/fluent/fluentd/pull/3285
66
+ * in_tail: Handle to send rotated logs when mv is used for rotating
67
+ https://github.com/fluent/fluentd/pull/3294
68
+ * fluent-plugin-config-format: Fill an uninitialized instance variable
69
+ https://github.com/fluent/fluentd/pull/3297
70
+ * Fix MessagePackEventStream issue with Enumerable methods
71
+ https://github.com/fluent/fluentd/pull/2116
72
+
73
+ ### Misc
74
+
75
+ * Add webrick to support Ruby 3.0
76
+ https://github.com/fluent/fluentd/pull/3257
77
+ * Suggest Discource instead of Google Groups
78
+ https://github.com/fluent/fluentd/pull/3261
79
+ * Update MAINTAINERS.md
80
+ https://github.com/fluent/fluentd/pull/3282
81
+ * Introduce DeepSource to check code quality
82
+ https://github.com/fluent/fluentd/pull/3286
83
+ https://github.com/fluent/fluentd/pull/3259
84
+ https://github.com/fluent/fluentd/pull/3291
85
+ * Migrate to GitHub Actions and stabilize tests
86
+ https://github.com/fluent/fluentd/pull/3266
87
+ https://github.com/fluent/fluentd/pull/3268
88
+ https://github.com/fluent/fluentd/pull/3281
89
+ https://github.com/fluent/fluentd/pull/3283
90
+ https://github.com/fluent/fluentd/pull/3290
91
+
92
+ ## Release v1.12.1 - 2021/02/18
93
+
94
+ ### Enhancement
95
+
96
+ * out_http: Add `headers_from_placeholders` parameter
97
+ https://github.com/fluent/fluentd/pull/3241
98
+ * fluent-plugin-config-format: Add `--table` option to use markdown table
99
+ https://github.com/fluent/fluentd/pull/3240
100
+ * Add `--disable-shared-socket`/`disable_shared_socket` to disable ServerEngine's shared socket setup
101
+ https://github.com/fluent/fluentd/pull/3250
102
+
103
+ ### Bug fix
104
+
105
+ * ca_generate: Fix creating TLS certification files which include broken extensions
106
+ https://github.com/fluent/fluentd/pull/3246
107
+ * test: Drop TLS 1.1 tests
108
+ https://github.com/fluent/fluentd/pull/3256
109
+ * Remove old gem constraints to support Ruby 3
110
+
111
+ ### Misc
112
+
113
+ * Use GitHub Actions
114
+ https://github.com/fluent/fluentd/pull/3233
115
+ https://github.com/fluent/fluentd/pull/3255
116
+
117
+ ## Release v1.12.0 - 2021/01/05
118
+
119
+ ### New feature
120
+
121
+ * in_tail: Add `follow_inode` to support log rotation with wild card
122
+ https://github.com/fluent/fluentd/pull/2992
123
+ * in_tail: Handle linux capability
124
+ https://github.com/fluent/fluentd/pull/3155
125
+ https://github.com/fluent/fluentd/pull/3162
126
+ * windows: Add win32 events alternative to unix signals
127
+ https://github.com/fluent/fluentd/pull/3131
128
+
129
+ ### Enhancement
130
+
131
+ * buffer: Enable metadata comparison optimization on all platforms
132
+ https://github.com/fluent/fluentd/pull/3095
133
+ * fluent-plugin-config-formatter: Handle `service_discovery` type
134
+ https://github.com/fluent/fluentd/pull/3178
135
+ * in_http: Add `add_query_params` parameter to add query params to event record
136
+ https://github.com/fluent/fluentd/pull/3197
137
+ * inject: Support `unixtime_micros` and `unixtime_nanos` in `time_type`
138
+ https://github.com/fluent/fluentd/pull/3220
139
+ * Refactoring code
140
+ https://github.com/fluent/fluentd/pull/3167
141
+ https://github.com/fluent/fluentd/pull/3170
142
+ https://github.com/fluent/fluentd/pull/3180
143
+ https://github.com/fluent/fluentd/pull/3196
144
+ https://github.com/fluent/fluentd/pull/3213
145
+ https://github.com/fluent/fluentd/pull/3222
146
+
147
+ ### Bug fix
148
+
149
+ * output: Prevent retry.step from being called too many times in a short time
150
+ https://github.com/fluent/fluentd/pull/3203
151
+
1
152
  # v1.11
2
153
 
3
154
  ## Release v1.11.5 - 2020/11/06