fluent-plugin-elasticsearch 1.9.4 → 5.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/.github/ISSUE_TEMPLATE/bug_report.md +37 -0
- data/.github/ISSUE_TEMPLATE/feature_request.md +24 -0
- data/.github/workflows/issue-auto-closer.yml +12 -0
- data/.github/workflows/linux.yml +26 -0
- data/.github/workflows/macos.yml +26 -0
- data/.github/workflows/windows.yml +26 -0
- data/.travis.yml +33 -6
- data/CONTRIBUTING.md +24 -0
- data/Gemfile +4 -1
- data/History.md +445 -1
- data/ISSUE_TEMPLATE.md +19 -0
- data/README.ElasticsearchGenID.md +116 -0
- data/README.ElasticsearchInput.md +293 -0
- data/README.Troubleshooting.md +692 -0
- data/README.md +1013 -38
- data/appveyor.yml +20 -0
- data/fluent-plugin-elasticsearch.gemspec +15 -9
- data/{Gemfile.v0.12 → gemfiles/Gemfile.elasticsearch.v6} +6 -5
- data/lib/fluent/log-ext.rb +38 -0
- data/lib/fluent/plugin/default-ilm-policy.json +14 -0
- data/lib/fluent/plugin/elasticsearch_constants.rb +13 -0
- data/lib/fluent/plugin/elasticsearch_error.rb +5 -0
- data/lib/fluent/plugin/elasticsearch_error_handler.rb +129 -0
- data/lib/fluent/plugin/elasticsearch_fallback_selector.rb +9 -0
- data/lib/fluent/plugin/elasticsearch_index_lifecycle_management.rb +67 -0
- data/lib/fluent/plugin/elasticsearch_index_template.rb +186 -12
- data/lib/fluent/plugin/elasticsearch_simple_sniffer.rb +10 -0
- data/lib/fluent/plugin/elasticsearch_tls.rb +70 -0
- data/lib/fluent/plugin/filter_elasticsearch_genid.rb +77 -0
- data/lib/fluent/plugin/in_elasticsearch.rb +325 -0
- data/lib/fluent/plugin/oj_serializer.rb +22 -0
- data/lib/fluent/plugin/out_elasticsearch.rb +1008 -267
- data/lib/fluent/plugin/out_elasticsearch_data_stream.rb +218 -0
- data/lib/fluent/plugin/out_elasticsearch_dynamic.rb +232 -214
- data/test/plugin/test_alias_template.json +9 -0
- data/test/plugin/test_elasticsearch_error_handler.rb +646 -0
- data/test/plugin/test_elasticsearch_fallback_selector.rb +74 -0
- data/test/plugin/test_elasticsearch_index_lifecycle_management.rb +66 -0
- data/test/plugin/test_elasticsearch_tls.rb +145 -0
- data/test/plugin/test_filter_elasticsearch_genid.rb +215 -0
- data/test/plugin/test_in_elasticsearch.rb +459 -0
- data/test/plugin/test_index_alias_template.json +11 -0
- data/test/plugin/test_index_template.json +25 -0
- data/test/plugin/test_oj_serializer.rb +19 -0
- data/test/plugin/test_out_elasticsearch.rb +5029 -387
- data/test/plugin/test_out_elasticsearch_data_stream.rb +337 -0
- data/test/plugin/test_out_elasticsearch_dynamic.rb +681 -208
- data/test/test_log-ext.rb +35 -0
- metadata +97 -19
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
|
-
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 6eb418d889b91bf79c37c1cd72789981eb4133bcfc34e21b26e79bb919462272
|
|
4
|
+
data.tar.gz: 168ddc77fb73216da63f1ce65ed673cf12ab13d6db549fc3e2da81956c938990
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5c4a2a8f63b25ea8785e0d4ef291043a93a86fea8f42638caaf46dd3efe29fd0a12919dd339d8a28c97aef65494ba054a3ff3312a122ab2cd89cb54cb069055c
|
|
7
|
+
data.tar.gz: 6b13851ce29b2a6f2083ebc57ae733a9ca00d79a9fd09ac4c7630d99a9c6d2da302c48f2e0cd901810e0a851911e14f8355fc336e21afd97d3f33fe205b0c78a
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Bug Report
|
|
3
|
+
about: Create a report to help us improve. If you have questions about ES plugin on kubernetes, please direct these to https://discuss.kubernetes.io/ before sumbit kubernetes related issue.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
(check apply)
|
|
8
|
+
- [ ] read [the contribution guideline](https://github.com/uken/fluent-plugin-elasticsearch/blob/master/CONTRIBUTING.md)
|
|
9
|
+
- [ ] (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.
|
|
10
|
+
|
|
11
|
+
#### Problem
|
|
12
|
+
|
|
13
|
+
...
|
|
14
|
+
|
|
15
|
+
#### Steps to replicate
|
|
16
|
+
|
|
17
|
+
Either clone and modify https://gist.github.com/pitr/9a518e840db58f435911
|
|
18
|
+
|
|
19
|
+
**OR**
|
|
20
|
+
|
|
21
|
+
Provide example config and message
|
|
22
|
+
|
|
23
|
+
#### Expected Behavior or What you need to ask
|
|
24
|
+
|
|
25
|
+
...
|
|
26
|
+
|
|
27
|
+
#### Using Fluentd and ES plugin versions
|
|
28
|
+
|
|
29
|
+
* OS version
|
|
30
|
+
* Bare Metal or within Docker or Kubernetes or others?
|
|
31
|
+
* Fluentd v0.12 or v0.14/v1.0
|
|
32
|
+
* paste result of ``fluentd --version`` or ``td-agent --version``
|
|
33
|
+
* ES plugin 3.x.y/2.x.y or 1.x.y
|
|
34
|
+
* paste boot log of fluentd or td-agent
|
|
35
|
+
* paste result of ``fluent-gem list``, ``td-agent-gem list`` or your Gemfile.lock
|
|
36
|
+
* ES version (optional)
|
|
37
|
+
* ES template(s) (optional)
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Feature request
|
|
3
|
+
about: Suggest an idea for this project
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
(check apply)
|
|
8
|
+
- [ ] read [the contribution guideline](https://github.com/uken/fluent-plugin-elasticsearch/blob/master/CONTRIBUTING.md)
|
|
9
|
+
|
|
10
|
+
**Is your feature request related to a problem? Please describe.**
|
|
11
|
+
|
|
12
|
+
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
|
|
13
|
+
|
|
14
|
+
**Describe the solution you'd like**
|
|
15
|
+
|
|
16
|
+
<!-- A clear and concise description of what you want to happen. -->
|
|
17
|
+
|
|
18
|
+
**Describe alternatives you've considered**
|
|
19
|
+
|
|
20
|
+
<!-- A clear and concise description of any alternative solutions or features you've considered. -->
|
|
21
|
+
|
|
22
|
+
**Additional context**
|
|
23
|
+
|
|
24
|
+
<!-- Add any other context or screenshots about the feature request here. -->
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
name: Autocloser
|
|
2
|
+
on: [issues]
|
|
3
|
+
jobs:
|
|
4
|
+
autoclose:
|
|
5
|
+
runs-on: ubuntu-latest
|
|
6
|
+
steps:
|
|
7
|
+
- name: Autoclose issues that did not follow issue template
|
|
8
|
+
uses: roots/issue-closer-action@v1.1
|
|
9
|
+
with:
|
|
10
|
+
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
|
11
|
+
issue-close-message: "@${issue.user.login} this issue was automatically closed because it did not follow the issue template."
|
|
12
|
+
issue-pattern: "(.*Problem.*)|(.*Expected Behavior or What you need to ask.*)|(.*Using Fluentd and ES plugin versions.*)|(.*Is your feature request related to a problem? Please describe.*)|(.*Describe the solution you'd like.*)|(.*Describe alternatives you've considered.*)"
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
name: Testing on Ubuntu
|
|
2
|
+
on:
|
|
3
|
+
- push
|
|
4
|
+
- pull_request
|
|
5
|
+
jobs:
|
|
6
|
+
build:
|
|
7
|
+
runs-on: ${{ matrix.os }}
|
|
8
|
+
strategy:
|
|
9
|
+
fail-fast: false
|
|
10
|
+
matrix:
|
|
11
|
+
ruby: [ '2.5', '2.6', '2.7', '3.0' ]
|
|
12
|
+
os:
|
|
13
|
+
- ubuntu-latest
|
|
14
|
+
name: Ruby ${{ matrix.ruby }} unit testing on ${{ matrix.os }}
|
|
15
|
+
steps:
|
|
16
|
+
- uses: actions/checkout@v2
|
|
17
|
+
- uses: ruby/setup-ruby@v1
|
|
18
|
+
with:
|
|
19
|
+
ruby-version: ${{ matrix.ruby }}
|
|
20
|
+
- name: unit testing
|
|
21
|
+
env:
|
|
22
|
+
CI: true
|
|
23
|
+
run: |
|
|
24
|
+
gem install bundler rake
|
|
25
|
+
bundle install --jobs 4 --retry 3
|
|
26
|
+
bundle exec rake test
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
name: Testing on macOS
|
|
2
|
+
on:
|
|
3
|
+
- push
|
|
4
|
+
- pull_request
|
|
5
|
+
jobs:
|
|
6
|
+
build:
|
|
7
|
+
runs-on: ${{ matrix.os }}
|
|
8
|
+
strategy:
|
|
9
|
+
fail-fast: false
|
|
10
|
+
matrix:
|
|
11
|
+
ruby: [ '2.5', '2.6', '2.7', '3.0' ]
|
|
12
|
+
os:
|
|
13
|
+
- macOS-latest
|
|
14
|
+
name: Ruby ${{ matrix.ruby }} unit testing on ${{ matrix.os }}
|
|
15
|
+
steps:
|
|
16
|
+
- uses: actions/checkout@v2
|
|
17
|
+
- uses: ruby/setup-ruby@v1
|
|
18
|
+
with:
|
|
19
|
+
ruby-version: ${{ matrix.ruby }}
|
|
20
|
+
- name: unit testing
|
|
21
|
+
env:
|
|
22
|
+
CI: true
|
|
23
|
+
run: |
|
|
24
|
+
gem install bundler rake
|
|
25
|
+
bundle install --jobs 4 --retry 3
|
|
26
|
+
bundle exec rake test
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
name: Testing on Windows
|
|
2
|
+
on:
|
|
3
|
+
- push
|
|
4
|
+
- pull_request
|
|
5
|
+
jobs:
|
|
6
|
+
build:
|
|
7
|
+
runs-on: ${{ matrix.os }}
|
|
8
|
+
strategy:
|
|
9
|
+
fail-fast: false
|
|
10
|
+
matrix:
|
|
11
|
+
ruby: [ '2.5', '2.6', '2.7', '3.0' ]
|
|
12
|
+
os:
|
|
13
|
+
- windows-latest
|
|
14
|
+
name: Ruby ${{ matrix.ruby }} unit testing on ${{ matrix.os }}
|
|
15
|
+
steps:
|
|
16
|
+
- uses: actions/checkout@v2
|
|
17
|
+
- uses: ruby/setup-ruby@v1
|
|
18
|
+
with:
|
|
19
|
+
ruby-version: ${{ matrix.ruby }}
|
|
20
|
+
- name: unit testing
|
|
21
|
+
env:
|
|
22
|
+
CI: true
|
|
23
|
+
run: |
|
|
24
|
+
gem install bundler rake
|
|
25
|
+
bundle install --jobs 4 --retry 3
|
|
26
|
+
bundle exec rake test
|
data/.travis.yml
CHANGED
|
@@ -1,13 +1,40 @@
|
|
|
1
1
|
language: ruby
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
3
|
+
jobs:
|
|
4
|
+
include:
|
|
5
|
+
- rvm: 2.4.6
|
|
6
|
+
gemfile: Gemfile
|
|
7
|
+
os: linux
|
|
8
|
+
arch: amd64
|
|
9
|
+
- rvm: 2.5.5
|
|
10
|
+
gemfile: Gemfile
|
|
11
|
+
os: linux
|
|
12
|
+
arch: amd64
|
|
13
|
+
- rvm: 2.6.3
|
|
14
|
+
gemfile: gemfiles/Gemfile.elasticsearch.v6
|
|
15
|
+
os: linux
|
|
16
|
+
arch: amd64
|
|
17
|
+
- rvm: 2.6.3
|
|
18
|
+
gemfile: Gemfile
|
|
19
|
+
os: linux
|
|
20
|
+
arch: amd64
|
|
21
|
+
- rvm: 2.6.3
|
|
22
|
+
gemfile: Gemfile
|
|
23
|
+
os: linux
|
|
24
|
+
arch: arm64
|
|
25
|
+
- rvm: 2.6.3
|
|
26
|
+
gemfile: Gemfile
|
|
27
|
+
os: osx
|
|
28
|
+
osx_image: xcode11.3
|
|
29
|
+
- rvm: 2.7.0
|
|
30
|
+
gemfile: Gemfile
|
|
31
|
+
os: linux
|
|
32
|
+
arch: amd64
|
|
7
33
|
|
|
8
34
|
gemfile:
|
|
9
35
|
- Gemfile
|
|
10
|
-
|
|
36
|
+
|
|
37
|
+
before_install:
|
|
38
|
+
- gem update --system=2.7.8
|
|
11
39
|
|
|
12
40
|
script: bundle exec rake test
|
|
13
|
-
sudo: false
|
data/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
Contributing to fluent-plugin-elasticsearch
|
|
2
|
+
===
|
|
3
|
+
|
|
4
|
+
We'd love your contribution. Here are the guidelines!
|
|
5
|
+
|
|
6
|
+
## Got a question or problem?
|
|
7
|
+
|
|
8
|
+
RESOURCES of [Official site's documentation](https://docs.fluentd.org/output/elasticsearch) and [ES plugin documentation](https://github.com/uken/fluent-plugin-elasticsearch/blob/master/README.md) may help you.
|
|
9
|
+
|
|
10
|
+
If you have further questions about ES plugin, feel free to [open an Issue](https://github.com/uken/fluent-plugin-elasticsearch/issues).
|
|
11
|
+
|
|
12
|
+
## Found a bug?
|
|
13
|
+
|
|
14
|
+
If you find a bug of ES plugin or a mistake in the documentation, you can help us by submitting an issue to ES plugin. Even better you can submit a pull request with a fix.
|
|
15
|
+
|
|
16
|
+
Note: Please follow the issue template/pull request template when sending your issue. Otherwise, your issue _might going to be closed with no response_.
|
|
17
|
+
|
|
18
|
+
## Patch Guidelines
|
|
19
|
+
|
|
20
|
+
Here are some things that would increase a chance that your patch is accepted:
|
|
21
|
+
|
|
22
|
+
* Write tests
|
|
23
|
+
* Run tests before sending pull request with `bundle exec rake test`
|
|
24
|
+
* Write [a good commit message](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html)
|
data/Gemfile
CHANGED
|
@@ -4,5 +4,8 @@ source 'https://rubygems.org'
|
|
|
4
4
|
gemspec
|
|
5
5
|
|
|
6
6
|
gem 'simplecov', require: false
|
|
7
|
-
gem 'coveralls', require: false
|
|
7
|
+
gem 'coveralls', ">= 0.8.0", require: false
|
|
8
8
|
gem 'strptime', require: false if RUBY_ENGINE == "ruby" && RUBY_VERSION =~ /^2/
|
|
9
|
+
gem "irb" if RUBY_ENGINE == "ruby" && RUBY_VERSION >= "2.6"
|
|
10
|
+
gem "elasticsearch-xpack"
|
|
11
|
+
gem "oj"
|