fluent-plugin-elasticsearch-dext 5.0.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (58) hide show
  1. checksums.yaml +7 -0
  2. data/.coveralls.yml +2 -0
  3. data/.editorconfig +9 -0
  4. data/.github/ISSUE_TEMPLATE/bug_report.md +37 -0
  5. data/.github/ISSUE_TEMPLATE/feature_request.md +24 -0
  6. data/.github/workflows/issue-auto-closer.yml +12 -0
  7. data/.github/workflows/linux.yml +26 -0
  8. data/.github/workflows/macos.yml +26 -0
  9. data/.github/workflows/windows.yml +26 -0
  10. data/.gitignore +18 -0
  11. data/.travis.yml +40 -0
  12. data/CONTRIBUTING.md +24 -0
  13. data/Gemfile +11 -0
  14. data/History.md +553 -0
  15. data/ISSUE_TEMPLATE.md +30 -0
  16. data/LICENSE.txt +201 -0
  17. data/PULL_REQUEST_TEMPLATE.md +10 -0
  18. data/README.ElasticsearchGenID.md +116 -0
  19. data/README.ElasticsearchInput.md +293 -0
  20. data/README.Troubleshooting.md +601 -0
  21. data/README.md +1467 -0
  22. data/Rakefile +11 -0
  23. data/appveyor.yml +20 -0
  24. data/fluent-plugin-elasticsearch.gemspec +35 -0
  25. data/gemfiles/Gemfile.elasticsearch.v6 +12 -0
  26. data/lib/fluent/log-ext.rb +38 -0
  27. data/lib/fluent/plugin/default-ilm-policy.json +14 -0
  28. data/lib/fluent/plugin/elasticsearch_constants.rb +13 -0
  29. data/lib/fluent/plugin/elasticsearch_error.rb +5 -0
  30. data/lib/fluent/plugin/elasticsearch_error_handler.rb +129 -0
  31. data/lib/fluent/plugin/elasticsearch_fallback_selector.rb +9 -0
  32. data/lib/fluent/plugin/elasticsearch_index_lifecycle_management.rb +67 -0
  33. data/lib/fluent/plugin/elasticsearch_index_template.rb +211 -0
  34. data/lib/fluent/plugin/elasticsearch_simple_sniffer.rb +10 -0
  35. data/lib/fluent/plugin/elasticsearch_tls.rb +70 -0
  36. data/lib/fluent/plugin/filter_elasticsearch_genid.rb +77 -0
  37. data/lib/fluent/plugin/in_elasticsearch.rb +325 -0
  38. data/lib/fluent/plugin/oj_serializer.rb +22 -0
  39. data/lib/fluent/plugin/out_elasticsearch.rb +1108 -0
  40. data/lib/fluent/plugin/out_elasticsearch_data_stream.rb +218 -0
  41. data/lib/fluent/plugin/out_elasticsearch_dynamic.rb +282 -0
  42. data/test/helper.rb +24 -0
  43. data/test/plugin/test_alias_template.json +9 -0
  44. data/test/plugin/test_elasticsearch_error_handler.rb +646 -0
  45. data/test/plugin/test_elasticsearch_fallback_selector.rb +74 -0
  46. data/test/plugin/test_elasticsearch_index_lifecycle_management.rb +66 -0
  47. data/test/plugin/test_elasticsearch_tls.rb +145 -0
  48. data/test/plugin/test_filter_elasticsearch_genid.rb +215 -0
  49. data/test/plugin/test_in_elasticsearch.rb +459 -0
  50. data/test/plugin/test_index_alias_template.json +11 -0
  51. data/test/plugin/test_index_template.json +25 -0
  52. data/test/plugin/test_oj_serializer.rb +19 -0
  53. data/test/plugin/test_out_elasticsearch.rb +5688 -0
  54. data/test/plugin/test_out_elasticsearch_data_stream.rb +337 -0
  55. data/test/plugin/test_out_elasticsearch_dynamic.rb +1134 -0
  56. data/test/plugin/test_template.json +23 -0
  57. data/test/test_log-ext.rb +35 -0
  58. metadata +236 -0
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: c74941c01afec4650824402b6ce4ac56240c98d9d9de1b59297c98c05feabbef
4
+ data.tar.gz: '02387f67defa1064b98607dd52bebaddfae3bf96ce5266532887593d42b53499'
5
+ SHA512:
6
+ metadata.gz: 986621286e073ce8476de2f21e6edd0a0ded7eb61209cc88bf305fd5b96ef995674f0fcab47b452e022700fd63b250519243463dc0dae9a7f149d48d7fe2ae3e
7
+ data.tar.gz: 1d1e3c081abf91b4528d0affd9d278a227bc018cc3ea44106bebd379aacaa5fd2cdf35c79459c5a20197e3ec9cad500b7d1bc595d89e4cc079bf71d74550a752
data/.coveralls.yml ADDED
@@ -0,0 +1,2 @@
1
+ service_name: travis-ci
2
+
data/.editorconfig ADDED
@@ -0,0 +1,9 @@
1
+ # EditorConfig: http://EditorConfig.org
2
+
3
+ root = true
4
+
5
+ [*]
6
+ end_of_line = lf
7
+ insert_final_newline = true
8
+ indent_style = space
9
+ indent_size = 2
@@ -0,0 +1,37 @@
1
+ ---
2
+ name: Bug Report
3
+ about: Create a report to help us improve. If you have questions about ES plugin on kubernetes, please direct these to https://discuss.kubernetes.io/ before sumbit kubernetes related issue.
4
+
5
+ ---
6
+
7
+ (check apply)
8
+ - [ ] read [the contribution guideline](https://github.com/uken/fluent-plugin-elasticsearch/blob/master/CONTRIBUTING.md)
9
+ - [ ] (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.
10
+
11
+ #### Problem
12
+
13
+ ...
14
+
15
+ #### Steps to replicate
16
+
17
+ Either clone and modify https://gist.github.com/pitr/9a518e840db58f435911
18
+
19
+ **OR**
20
+
21
+ Provide example config and message
22
+
23
+ #### Expected Behavior or What you need to ask
24
+
25
+ ...
26
+
27
+ #### Using Fluentd and ES plugin versions
28
+
29
+ * OS version
30
+ * Bare Metal or within Docker or Kubernetes or others?
31
+ * Fluentd v0.12 or v0.14/v1.0
32
+ * paste result of ``fluentd --version`` or ``td-agent --version``
33
+ * ES plugin 3.x.y/2.x.y or 1.x.y
34
+ * paste boot log of fluentd or td-agent
35
+ * paste result of ``fluent-gem list``, ``td-agent-gem list`` or your Gemfile.lock
36
+ * ES version (optional)
37
+ * ES template(s) (optional)
@@ -0,0 +1,24 @@
1
+ ---
2
+ name: Feature request
3
+ about: Suggest an idea for this project
4
+
5
+ ---
6
+
7
+ (check apply)
8
+ - [ ] read [the contribution guideline](https://github.com/uken/fluent-plugin-elasticsearch/blob/master/CONTRIBUTING.md)
9
+
10
+ **Is your feature request related to a problem? Please describe.**
11
+
12
+ <!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
13
+
14
+ **Describe the solution you'd like**
15
+
16
+ <!-- A clear and concise description of what you want to happen. -->
17
+
18
+ **Describe alternatives you've considered**
19
+
20
+ <!-- A clear and concise description of any alternative solutions or features you've considered. -->
21
+
22
+ **Additional context**
23
+
24
+ <!-- Add any other context or screenshots about the feature request here. -->
@@ -0,0 +1,12 @@
1
+ name: Autocloser
2
+ on: [issues]
3
+ jobs:
4
+ autoclose:
5
+ runs-on: ubuntu-latest
6
+ steps:
7
+ - name: Autoclose issues that did not follow issue template
8
+ uses: roots/issue-closer-action@v1.1
9
+ with:
10
+ repo-token: ${{ secrets.GITHUB_TOKEN }}
11
+ issue-close-message: "@${issue.user.login} this issue was automatically closed because it did not follow the issue template."
12
+ issue-pattern: "(.*Problem.*)|(.*Expected Behavior or What you need to ask.*)|(.*Using Fluentd and ES plugin versions.*)|(.*Is your feature request related to a problem? Please describe.*)|(.*Describe the solution you'd like.*)|(.*Describe alternatives you've considered.*)"
@@ -0,0 +1,26 @@
1
+ name: Testing on Ubuntu
2
+ on:
3
+ - push
4
+ - pull_request
5
+ jobs:
6
+ build:
7
+ runs-on: ${{ matrix.os }}
8
+ strategy:
9
+ fail-fast: false
10
+ matrix:
11
+ ruby: [ '2.4', '2.5', '2.6', '2.7' ]
12
+ os:
13
+ - ubuntu-latest
14
+ name: Ruby ${{ matrix.ruby }} unit testing on ${{ matrix.os }}
15
+ steps:
16
+ - uses: actions/checkout@v2
17
+ - uses: ruby/setup-ruby@v1
18
+ with:
19
+ ruby-version: ${{ matrix.ruby }}
20
+ - name: unit testing
21
+ env:
22
+ CI: true
23
+ run: |
24
+ gem install bundler rake
25
+ bundle install --jobs 4 --retry 3
26
+ bundle exec rake test
@@ -0,0 +1,26 @@
1
+ name: Testing on macOS
2
+ on:
3
+ - push
4
+ - pull_request
5
+ jobs:
6
+ build:
7
+ runs-on: ${{ matrix.os }}
8
+ strategy:
9
+ fail-fast: false
10
+ matrix:
11
+ ruby: [ '2.4', '2.5', '2.6', '2.7' ]
12
+ os:
13
+ - macOS-latest
14
+ name: Ruby ${{ matrix.ruby }} unit testing on ${{ matrix.os }}
15
+ steps:
16
+ - uses: actions/checkout@v2
17
+ - uses: ruby/setup-ruby@v1
18
+ with:
19
+ ruby-version: ${{ matrix.ruby }}
20
+ - name: unit testing
21
+ env:
22
+ CI: true
23
+ run: |
24
+ gem install bundler rake
25
+ bundle install --jobs 4 --retry 3
26
+ bundle exec rake test
@@ -0,0 +1,26 @@
1
+ name: Testing on Windows
2
+ on:
3
+ - push
4
+ - pull_request
5
+ jobs:
6
+ build:
7
+ runs-on: ${{ matrix.os }}
8
+ strategy:
9
+ fail-fast: false
10
+ matrix:
11
+ ruby: [ '2.4', '2.5', '2.6', '2.7' ]
12
+ os:
13
+ - windows-latest
14
+ name: Ruby ${{ matrix.ruby }} unit testing on ${{ matrix.os }}
15
+ steps:
16
+ - uses: actions/checkout@v2
17
+ - uses: ruby/setup-ruby@v1
18
+ with:
19
+ ruby-version: ${{ matrix.ruby }}
20
+ - name: unit testing
21
+ env:
22
+ CI: true
23
+ run: |
24
+ gem install bundler rake
25
+ bundle install --jobs 4 --retry 3
26
+ bundle exec rake test
data/.gitignore ADDED
@@ -0,0 +1,18 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
18
+ .DS_Store
data/.travis.yml ADDED
@@ -0,0 +1,40 @@
1
+ language: ruby
2
+
3
+ jobs:
4
+ include:
5
+ - rvm: 2.4.6
6
+ gemfile: Gemfile
7
+ os: linux
8
+ arch: amd64
9
+ - rvm: 2.5.5
10
+ gemfile: Gemfile
11
+ os: linux
12
+ arch: amd64
13
+ - rvm: 2.6.3
14
+ gemfile: gemfiles/Gemfile.elasticsearch.v6
15
+ os: linux
16
+ arch: amd64
17
+ - rvm: 2.6.3
18
+ gemfile: Gemfile
19
+ os: linux
20
+ arch: amd64
21
+ - rvm: 2.6.3
22
+ gemfile: Gemfile
23
+ os: linux
24
+ arch: arm64
25
+ - rvm: 2.6.3
26
+ gemfile: Gemfile
27
+ os: osx
28
+ osx_image: xcode11.3
29
+ - rvm: 2.7.0
30
+ gemfile: Gemfile
31
+ os: linux
32
+ arch: amd64
33
+
34
+ gemfile:
35
+ - Gemfile
36
+
37
+ before_install:
38
+ - gem update --system=2.7.8
39
+
40
+ script: bundle exec rake test
data/CONTRIBUTING.md ADDED
@@ -0,0 +1,24 @@
1
+ Contributing to fluent-plugin-elasticsearch
2
+ ===
3
+
4
+ We'd love your contribution. Here are the guidelines!
5
+
6
+ ## Got a question or problem?
7
+
8
+ RESOURCES of [Official site's documentation](https://docs.fluentd.org/output/elasticsearch) and [ES plugin documentation](https://github.com/uken/fluent-plugin-elasticsearch/blob/master/README.md) may help you.
9
+
10
+ If you have further questions about ES plugin, feel free to [open an Issue](https://github.com/uken/fluent-plugin-elasticsearch/issues).
11
+
12
+ ## Found a bug?
13
+
14
+ If you find a bug of ES plugin or a mistake in the documentation, you can help us by submitting an issue to ES plugin. Even better you can submit a pull request with a fix.
15
+
16
+ Note: Please follow the issue template/pull request template when sending your issue. Otherwise, your issue _might going to be closed with no response_.
17
+
18
+ ## Patch Guidelines
19
+
20
+ Here are some things that would increase a chance that your patch is accepted:
21
+
22
+ * Write tests
23
+ * Run tests before sending pull request with `bundle exec rake test`
24
+ * Write [a good commit message](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html)
data/Gemfile ADDED
@@ -0,0 +1,11 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in fluent-plugin-elasticsearch.gemspec
4
+ gemspec
5
+
6
+ gem 'simplecov', require: false
7
+ gem 'coveralls', ">= 0.8.0", require: false
8
+ gem 'strptime', require: false if RUBY_ENGINE == "ruby" && RUBY_VERSION =~ /^2/
9
+ gem "irb" if RUBY_ENGINE == "ruby" && RUBY_VERSION >= "2.6"
10
+ gem "elasticsearch-xpack"
11
+ gem "oj"
data/History.md ADDED
@@ -0,0 +1,553 @@
1
+ ## Changelog [[tags]](https://github.com/uken/fluent-plugin-elasticsearch/tags)
2
+
3
+ ### [Unreleased]
4
+
5
+ ### 5.0.1
6
+ - Use elasticsearch/api instead of elasticsearch/xpack (#870)
7
+
8
+ ### 5.0.0
9
+ - Support #retry_operate on data stream (#863)
10
+ - Support placeholder in @data\_stream\_name for @type elasticsearch\_data\_stream (#862)
11
+ - Extract troubleshooting section (#861)
12
+ - Fix unmatched `<source>` close tag (#860)
13
+ - Initial support for Elasticsearch Data Stream (#859)
14
+
15
+ ### 4.3.3
16
+ - Handle invalid Elasticsearch::Client#info response (#855)
17
+
18
+ ### 4.3.2
19
+ - Use log.debug to dump ES 8.x _type information (#852)
20
+
21
+ ### 4.3.1
22
+ - Unsplit huge records by default (#851)
23
+
24
+ ### 4.3.0
25
+ - Add cloud_id & cloud_auth settings (#850)
26
+ - Fix failing tests on es6 client (#848)
27
+ - feat: retry on transport errors (#846)
28
+ - fix typo (#838)
29
+ - Remove unnecessary nil check (#826)
30
+
31
+ ### 4.2.2
32
+ - Remove unnecessary nil check (#826)
33
+ - Fix ILM rollover index template pattern to apply index_separator (#825)
34
+ - Fix ILM rollover alias creation when a placeholder is used in index_name (#823)
35
+ - Add a note about the pitfalls of per-date indexes used with ILM (#822)
36
+
37
+ ### 4.2.1
38
+ - Update a broken link (#821)
39
+ - Include chunk_id in records (#820)
40
+ - Fix a failing testcase (#818)
41
+
42
+ ### 4.2.0
43
+ - ci: Add Ruby 2.7 jobs (#812)
44
+ - Support Elasticsearch new style template (#810)
45
+
46
+ ### 4.1.4
47
+ - ElasticsearchGenID update docs for hash_type (#809)
48
+ - Handle api key header (#808)
49
+ - index_template: Handle error object entirely on index creation failure (#807)
50
+
51
+ ### 4.1.3
52
+ - Load multiple templates even if template_name and template_file given (#799)
53
+ - Handle elasticsearch-ruby 7.9.0 using HTTP method changes (#795)
54
+
55
+ ### 4.1.2
56
+ - Use Hash#dig instead of Hash#[] on retrieving version information from Elasticsearch info API (#793)
57
+
58
+ ### 4.1.1
59
+ - Correct ILM explain on logstash_format case (#786)
60
+
61
+ ### 4.1.0
62
+ - Implement Fallback selector and configurable selector class (#782)
63
+
64
+ ### 4.0.11
65
+ - Add custom and time placeholders combination testcase for ILM (#781)
66
+ - Really support ILM on `logstash_format` enabled environment (#779)
67
+
68
+ ### 4.0.10
69
+ - filter_elasticsearch_genid: Use entire record as hash seed (#777)
70
+ - Suppress type in meta with suppress_type_name parameter (#774)
71
+ - filter\_elasticsearch\_genid: Add hash generation mechanism from events (#773)
72
+ - Clean up error text (#772)
73
+ - Use GitHub Actions badges instead of Travis' (#760)
74
+ - Add issue auto closer workflow (#759)
75
+ - Document required permissions (#757)
76
+
77
+ ### 4.0.9
78
+ - Add possibility to configure multiple ILM policies (#753)
79
+ - Document required permissions (#757)
80
+
81
+ ### 4.0.8
82
+ - Handle compressable connection usable state (#743)
83
+ - Use newer tls protocol versions (#739)
84
+ - Add GitHub Actions file (#740)
85
+
86
+ ### 4.0.7
87
+ - Added http_backend_excon_nonblock config in out_elasticsearch (#733)
88
+
89
+ ### 4.0.6
90
+ - Add fallback mechanism for handling to detect es version (#730)
91
+ - Remove needless section (#728)
92
+ - Handle exception if index already exists (#727)
93
+ - Tweak test cases (#726)
94
+
95
+ ### 4.0.5
96
+ - add logstash_dateformat as placeholder (#718)
97
+ - Tweak travis.yml for suppressing validator warnings and add CI for Linux Arm64 architecture and macOS 10.14 (#724)
98
+ - Elasticsearch ruby v7.5 (#723)
99
+ - Add Oj serializer testcases for all job (#722)
100
+ - Update documentation for ILM (#721)
101
+
102
+ ### 4.0.4
103
+ - Provide clearing caches timer (#719)
104
+
105
+ ### 4.0.3
106
+ - Use http.scheme settings for persisent scheme setup (#713)
107
+
108
+ ### 4.0.2
109
+ - Support TLSv1.3 (#710)
110
+
111
+ ### 4.0.1
112
+ - Placeholders for template name and customize template (#708)
113
+ - Add overwriting ilm policy config parameter (#707)
114
+ - Fix a failing ILM config testcase (#706)
115
+
116
+ ### 4.0.0
117
+ - Restructuring ILM related features (#701)
118
+ - Extract placeholders in pipeline parameter (#695)
119
+ - fix typo in `README.md` (#698)
120
+ - Reduce log noise when not using rollover_index (#692)
121
+
122
+ ### 3.8.0
123
+ - Add FAQ for specifying index.codec (#679)
124
+ - Add FAQ for connect_write timeout reached error (#687)
125
+ - Unblocking buffer overflow with block action (#688)
126
+
127
+ ### 3.7.1
128
+ - Make conpatible for Fluentd v1.8 (#677)
129
+ - Handle flatten_hashes in elasticsearch_dynamic (#675)
130
+ - Handle empty index_date_pattern parameter (#674)
131
+
132
+ ### 3.7.0
133
+ - Tweak for cosmetic change (#671)
134
+ - Fix access to Elasticsearch::Transport::VERSION with explicit top level class path (#670)
135
+ - Implement Elasticsearch Input plugin (#669)
136
+
137
+ ### 3.6.1
138
+ - retry upsert on recoverable error. (#667)
139
+ - Allow `_index` in chunk_keys (#665)
140
+ - Support compression feature (#664)
141
+
142
+ ### 3.6.0
143
+ - Set order in newly created templates (#660)
144
+ - Merge Support index lifecycle management into master (#659)
145
+ - Support template installation with host placeholder (#654)
146
+ - Support index lifecycle management (#651)
147
+
148
+ ### 3.5.6
149
+ - Support elasticsearch8 removal of mapping types (#656)
150
+ - Upgrade webmock to 3 (#652)
151
+ - Suppress `ruby -c` warnings (#649)
152
+ - Add tips for sniffer class (#644)
153
+ - Make `client_key_pass` secret (#637)
154
+ - Validate `user` and `password` early (#636)
155
+
156
+ ### 3.5.5
157
+ - Fix arguments order of `assert_equal` (#635)
158
+ - Add description for `port` option (#634)
159
+ - Fix description position and add examples for `hosts` option (#633)
160
+ - Use upper-case to compare before and after conversion (#630)
161
+ - Validate `max_retry_get_es_version` (#629)
162
+ - Remove unused getting plugin instance (#628)
163
+ - Fix error message for `max_retry_putting_template` (#627)
164
+ - Fix DST-breaking unit test (#623)
165
+ - Handle ClusterBlockException (#621)
166
+ - Add contribution guideline document (#618)
167
+
168
+ ### 3.5.4
169
+ - Add FAQ for Fluentd seems to hang if it unable to connect Elasticsearch (#617)
170
+ - Check bulk_message size before appending (#616)
171
+ - Add FAQ for Elasticsearch index mapping glitch (#614)
172
+ - Display retry counts and interval (#613)
173
+
174
+ ### 3.5.3
175
+ - Handle nil items response (#611)
176
+
177
+ ### 3.5.2
178
+ - Fix `@meta_config_map` creation timing (#592)
179
+
180
+ ### 3.5.1
181
+ - Configurable split request size threshold (#586)
182
+
183
+ ### 3.5.0
184
+ - Adopt Elasticsearch ruby client v7 loggable class (#583)
185
+
186
+ ### 3.4.3
187
+ - Add fail_on_putting_template_retry_exceed config (#579)
188
+
189
+ ### 3.4.2
190
+ - Comparing DEFAULT_TYPE_NAME_ES_7x to target_type instead of type_name (#573)
191
+
192
+ ### 3.4.1
193
+ - Handle non-String value on parse_time (#570)
194
+
195
+ ### 3.4.0
196
+ - Check exclusive feature on #configure (#569)
197
+ - Modify FAQ for highly load k8s EFK stack (#566)
198
+ - Add FAQ for high load k8s EFK stack (#564)
199
+
200
+ ### 3.3.3
201
+ - Add unit test for exception message (#563)
202
+ - Add ignore_exceptions config (#562)
203
+
204
+ ### 3.3.2
205
+ - Fix support for host, hosts placeholders (#560)
206
+ - typo fixes in README.md (#559)
207
+
208
+ ### 3.3.1
209
+ - add new option to suppress doc wrapping (#557)
210
+ - Include 2 (#555)
211
+
212
+ ### 3.3.0
213
+ - Support builtin placeholders for host and hosts parameter (#554)
214
+
215
+ ### 3.2.4
216
+ - Pass chunk for built in placeholders (#553)
217
+
218
+ ### 3.2.3
219
+ - Expose exception backtrace for typhoeus gem loading error (#550)
220
+
221
+ ### 3.2.2
222
+ - Don't validate ES cliuent version under dry-run mode (#547)
223
+
224
+ ### 3.2.1
225
+ - Don't attempt to connect to Elasticsearch in dry run mode (#543)
226
+ - Add FAQ for typhoeus gem installation (#544)
227
+
228
+ ### 3.2.0
229
+ - Split huge record requests (#539)
230
+
231
+ ### 3.1.1
232
+ - Add document for custom_headers (#538)
233
+ - out_elasticsearch: Add custom_headers parameter (#529)
234
+ - Bundle irb on Ruby 2.6 or later (#537)
235
+
236
+ ### 3.1.0
237
+ - Retry obtaining Elasticsearch version (#532)
238
+ - Fix broken id links (#530)
239
+
240
+ ### 3.0.2
241
+ - appveyor: Remove Ruby 2.1 CI targets on AppVeyor (#524)
242
+ - Follow removal of _routing field change on recent Elasticsearch (#523)
243
+ - Travis: Tweak to use Ruby versions (#522)
244
+
245
+ ### 3.0.1
246
+ - Remove needless Elasticsearch version detection (#520)
247
+
248
+ ### 3.0.0
249
+ - Use fluentd core retry mechanism (#519)
250
+ - Depends on builtin retrying mechanism (#518)
251
+ - Loosen ConnectionRetryFailure condition when flush_thread_count > 1 and depends on Fluentd core retrying mechanism (#516)
252
+
253
+ ### 2.12.5
254
+ - Ensure sniffer class constants definition before calling #client (#515)
255
+
256
+ ### 2.12.4
257
+ - #506 Rollover index will be in effect in case of template overwrite also. (#513)
258
+
259
+ ### 2.12.3
260
+ - Added log_es_400_reason configuration item (#511)
261
+ - Allow a user to specify the rollover index date pattern (#510)
262
+
263
+ ### 2.12.2
264
+ - Verify connection at startup (#504)
265
+ - Add faq for glob pattern tag routing (#502)
266
+
267
+ ### 2.12.1
268
+ - Make configurable unrecoverable types (#501)
269
+ - Add FAQ for TLS enabled nginx proxy TLS version incompatibility trouble (#496)
270
+ - Add FAQs (#492)
271
+ - Remove issuestats.com badges (#489)
272
+
273
+ ### 2.12.0
274
+ - Decoupling the custom template and rollover index creation #485 (#486)
275
+
276
+ ### 2.11.11
277
+ - Handle error not to acquire version information (#479)
278
+
279
+ ### 2.11.10
280
+ - Verbose error reason output (#469)
281
+
282
+ ### 2.11.9
283
+ - Use ConnectionRetryFailure in plugin specific retrying for consistency (#468)
284
+ - Remove outdated generating hash_id_key code (#466)
285
+ - Tweak behavior for UnrecoverableError and #detect_es_major_version (#465)
286
+
287
+ ### 2.11.8
288
+ - Serialize requests with Oj (#464)
289
+
290
+ ### 2.11.7
291
+ - Add mechanism to detect ES and its client version mismatch (#463)
292
+
293
+ ### 2.11.6
294
+ - 355 customize template (#431)
295
+
296
+ ### 2.11.5
297
+ - Uplift Merge pull request #459 from richm/v0.12-simple-sniffer (#461)
298
+
299
+ ### 2.11.4
300
+ - Persistent backend (#456)
301
+
302
+ ### 2.11.3
303
+ - Implement the `include_index_in_url` option for out_elasticsearch (#451)
304
+ - Add an option `include_index_in_url` to allow URL-based conrtrols (#450)
305
+
306
+ ### 2.11.2
307
+ - Strictness scheme (#445)
308
+
309
+ ### 2.11.0
310
+ - Uplift Merge pull request #437 from jcantrill/fix_bulk_count (#438)
311
+
312
+ ### 2.10.5
313
+ - Uplift Merge pull request #435 from jcantrill/add_trace_logging (#436)
314
+
315
+ ### 2.10.4
316
+ - Use Fluent::UnrecoverableError as unrecoverable error class ancestors (#433)
317
+ - Add parameter validation for retrying template installation (#429)
318
+
319
+ ### 2.10.3
320
+ - Add retry mechanism for template installation (#428)
321
+
322
+ ### 2.10.2
323
+ - Use logstash_prefix_separator on elasticsearch_dynamic (#427)
324
+
325
+ ### 2.10.1
326
+ - Uplift Merge pull request #419 from jcantrill/retry_prefix (#421)
327
+ - Uplift Merge pull request #418 from jcantrill/emit_exception (#420)
328
+
329
+ ### 2.10.0
330
+ - Uplift Merge pull request #405 from jcantrill/sanitize_bulk (#414)
331
+
332
+ ### 2.9.2
333
+ - Uplift Merge pull request #410 from richm/v0.12-consistent-errors-and-tests (#411)
334
+ - Add correct link for include_timestamp (#408)
335
+
336
+ ### 2.9.1
337
+ - Uplift Merge pull request #406 from richm/v0.12-successes-duplicates-no-retry (#407)
338
+
339
+ ### 2.9.0
340
+ - DLQ revisited v1 uplifted #398, #402 (#404)
341
+
342
+ ### 2.8.6
343
+ - auth: Fix missing auth tokens after reloading connections (#394)
344
+
345
+ ### 2.8.5
346
+ - Add deprecated option into content_type parameter (#391)
347
+
348
+ ### 2.8.4
349
+ - Use nanosecond precision in elasticsearch_dynamic (#387)
350
+
351
+ ### 2.8.3
352
+ - Specify SSL/TLS version in out_elasticsearch_dynamic (#385)
353
+
354
+ ### 2.8.2
355
+ - Revert content type header default value (#383)
356
+
357
+ ### 2.8.1
358
+ - Restore default value of type name #(377)
359
+
360
+ ### 2.8.0
361
+ - Remove outdated generating hash id support module (#373)
362
+ - Check Elasticsearch major version (#371)
363
+
364
+ ### 2.7.0
365
+ - Configureable content type (#367)
366
+
367
+ ### 2.6.1
368
+ - Prevent error when using template in elasticsearch_dynamic for elementally use case (#363)
369
+
370
+ ### 2.6.0
371
+ - Handle high precision time format when using custom time_key (#360)
372
+
373
+ ### 2.5.0
374
+ - Using nested record in `id_key`, `parent_key`, and `routing_key` (#351)
375
+ - Fix inverted case of a proper noun "Elasticsearch" (#349)
376
+
377
+ ### 2.4.1
378
+ - Add config parameter to enable elasticsearch-ruby's transporter logging (#342)
379
+
380
+ ### 2.4.0
381
+ - Add built-in placeholders support against type_name parameter (#338)
382
+
383
+ ### 2.3.0
384
+ - Allow overwriting existing index template (#239)
385
+
386
+ ### 2.2.0
387
+ - GA release 2.2.0.
388
+
389
+ ### 2.2.0.rc.1
390
+ - Separate generate hash id module and bundled new plugin for generating unique hash id (#331)
391
+
392
+ ### 2.1.1
393
+ - Raise ConfigError when specifying different @hash_config.hash_id_key and id_key configration (#327)
394
+ - Small typo fix in README.md (#325)
395
+
396
+ ### 2.1.0
397
+ - Retry on certain errors from Elasticsearch (#322)
398
+
399
+ ### 2.0.1
400
+ - Releasing generating hash id mechanism to avoid records duplication feature.
401
+
402
+ ### 2.0.1.rc.1
403
+ - Add generating hash id mechanism to avoid records duplication (#318)
404
+
405
+ ### 2.0.0
406
+ - Release for Fluentd v0.14 stable.
407
+
408
+ ### 2.0.0.rc.7
409
+ - Add `include_timestamp` option (#310)
410
+
411
+ ### 2.0.0.rc.6
412
+ - Improve documentation (#304)
413
+ - Handle dynamic_config misconfigurations (#305)
414
+ - Escape basic authentication user information placeholders (#306)
415
+
416
+ ### 2.0.0.rc.5
417
+ - make configurable with `ssl_version` parameter (#299)
418
+ - add `logstash_prefix_separator` config parameter (#297)
419
+
420
+ ### 2.0.0.rc.4
421
+ - fix license identifier in gemspec (#294)
422
+
423
+ ### 2.0.0.rc.3
424
+ - add built-in placeholders support (#288, #293)
425
+ - permit multi workers feature (#291)
426
+
427
+ ### 2.0.0.rc.2
428
+ - add pipeline parameter (#290)
429
+
430
+ ### 2.0.0.rc.1
431
+ - Use v0.14 API to support nanosecond precision (#223)
432
+
433
+ ### 1.9.5
434
+ - sub-second time precision [(#249)](https://github.com/uken/fluent-plugin-elasticsearch/pull/249)
435
+
436
+ ### 1.9.4
437
+ - Include 'Content-Type' header in `transport_options`
438
+
439
+ ### 1.9.3
440
+ - Use latest elasticsearch-ruby (#240)
441
+ - Log ES response errors (#230)
442
+
443
+ ### 1.9.2
444
+ - Fix elasticsearch_dynamic for v0.14 (#224)
445
+
446
+ ### 1.9.1
447
+ - Cast `reload_*` configs in out_elasticsearch_dynamic to bool (#220)
448
+
449
+ ### 1.9.0
450
+ - add `time_parse_error_tag` (#211)
451
+ - add `reconnect_on_error` (#214)
452
+
453
+ ### 1.9.0.rc.1
454
+ - Optimize output plugins (#203)
455
+
456
+ ### 1.8.0
457
+ - fix typo in defaults for ssl_verify on elasticsearch_dynamic (#202)
458
+ - add support for `templates` (#196)
459
+ - rename `send` method to `send_bulk` (#206)
460
+
461
+ ### 1.7.0
462
+ - add support for `template_name` and `template_file` (#194)
463
+
464
+ ### 1.6.0
465
+ - add support for dot separated `target_index_key` and `target_type_key` (#175)
466
+ - add `remove_keys_on_update` and `remove_keys_on_update_key` (#189)
467
+ - fix support for fluentd v0.14 (#191)
468
+ - remove support for elasticsearch v2 for now (#177)
469
+
470
+ ### 1.5.0
471
+ - add `routing_key` (#158)
472
+ - add `time_key_exclude_timestamp` to exclude `@timestamp` (#161)
473
+ - convert index names to lowercase (#163)
474
+ - add `remove_keys` (#164)
475
+ - add `flatten_hashes` (#168)
476
+ - add `target_type_key` (#169)
477
+
478
+ ### 1.4.0
479
+ - add `target_index_key` to specify target index (#153)
480
+ - add `time_key_format` for faster time format parsing (#154)
481
+
482
+ ### 1.3.0
483
+ - add `write_operation`
484
+
485
+ ### 1.2.1
486
+ - fix `resurrect_after` in out_elasticsearch_dynamic
487
+
488
+ ### 1.2.0
489
+ - out_elasticsearch_dynamic get memory improvement and fix for race condition (#133)
490
+ - Add `resurrect_after` option (#136)
491
+
492
+ ### 1.1.0
493
+ - Support SSL client verification and custom CA file (#123)
494
+ - Release experimental `type elasticsearch_dynamic` (#127)
495
+
496
+ ### 1.0.0
497
+ - password config param is now marked as secret and won't be displayed in logs.
498
+
499
+ ### 0.9.0
500
+ - Add `ssl_verify` option (#108)
501
+
502
+ ### 0.8.0
503
+ - Replace Patron with Excon HTTP client (#93)
504
+
505
+ ### 0.7.0
506
+ - Add new option `time_key` (#85)
507
+
508
+ ### 0.6.1
509
+ - 0.10.43 is minimum version required of fluentd (#79)
510
+
511
+ ### 0.6.0
512
+ - added `reload_on_failure` and `reload_connections` flags (#78)
513
+
514
+ ### 0.5.1
515
+ - fix legacy hosts option, port should be optional (#75)
516
+
517
+ ### 0.5.0
518
+ - add full connection URI support (#65)
519
+ - use `@timestamp` for index (#41)
520
+ - add support for elasticsearch gem version 1 (#71)
521
+ - fix connection reset & retry when connection is lost (#67)
522
+
523
+ ### 0.4.0
524
+ - add `request_timeout` config (#59)
525
+ - fix lockup when non-hash values are sent (#52)
526
+
527
+ ### 0.3.1
528
+ - force using patron (#46)
529
+ - do not generate @timestamp if already part of message (#35)
530
+
531
+ ### 0.3.0
532
+ - add `parent_key` option (#28)
533
+ - have travis-ci build on multiple rubies (#30)
534
+ - add `utc_index` and `hosts` options, switch to using `elasticsearch` gem (#26, #29)
535
+
536
+ ### 0.2.0
537
+ - fix encoding issues with JSON conversion and again when sending to elasticsearch (#19, #21)
538
+ - add logstash_dateformat option (#20)
539
+
540
+ ### 0.1.4
541
+ - add logstash_prefix option
542
+
543
+ ### 0.1.3
544
+ - raising an exception on non-success response from elasticsearch
545
+
546
+ ### 0.1.2
547
+ - add id_key option
548
+
549
+ ### 0.1.1
550
+ - fix timezone in logstash key
551
+
552
+ ### 0.1.0
553
+ - Initial gem release.