logstash-output-opensearch 2.0.2-java → 2.1.0-java
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/DEVELOPER_GUIDE.md +6 -1
- data/MAINTAINERS.md +13 -12
- data/README.md +14 -6
- data/RELEASING.md +7 -10
- data/docs/building_custom_docker_images.md +79 -0
- data/lib/logstash/outputs/opensearch/http_client/manticore_adapter.rb +4 -2
- data/lib/logstash/plugin_mixins/opensearch/common.rb +1 -0
- data/logstash-output-opensearch.gemspec +2 -2
- data/spec/integration/outputs/index_spec.rb +3 -2
- data/spec/opensearch_spec_helper.rb +9 -0
- data/spec/unit/outputs/opensearch_spec.rb +24 -0
- data.tar.gz.sig +0 -0
- metadata +33 -35
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9e28db2587cf1e6799c6c3e120ae645c32731c4ac2deb133b56657991ccecf45
|
|
4
|
+
data.tar.gz: 79ff25a9dbbe8404d02a73839ad8b8fccadb48b050e0d04296e5727402d6a82c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: caffc9d8fe50149b4608b4c1317394459a333485d6f3c06aa1d0388eb02b2b0e74433a0f34df89bcd8079767911ab7a71f88a38043dc38f3516c40bf73572f1a
|
|
7
|
+
data.tar.gz: 29dbc4775ddc406eec95d85e04d31e2d002ebfc0bc327dd15377f43bf433c1cb6acbaeb7c0fe9f5befccf4ced94766b62401b3c7d71b78fde6f06a81a663a395
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
data/DEVELOPER_GUIDE.md
CHANGED
|
@@ -9,6 +9,7 @@
|
|
|
9
9
|
- [Configuration for Logstash Output OpenSearch Plugin](#configuration-for-logstash-output-opensearch-plugin)
|
|
10
10
|
- [Submitting Changes](#submitting-changes)
|
|
11
11
|
- [Backports](#backports)
|
|
12
|
+
- [Building Custom Docker Images](docs/building_custom_docker_images.md)
|
|
12
13
|
|
|
13
14
|
# Developer Guide
|
|
14
15
|
|
|
@@ -185,13 +186,14 @@ Build the gem locally and install it using:
|
|
|
185
186
|
## Configuration for Logstash Output OpenSearch Plugin
|
|
186
187
|
|
|
187
188
|
To run the Logstash Output Opensearch plugin, add following configuration in your logstash.conf file.
|
|
189
|
+
Note: For logstash running with OpenSearch 2.12.0 and higher the admin password needs to be a custom strong password supplied during cluster setup.
|
|
188
190
|
|
|
189
191
|
```
|
|
190
192
|
output {
|
|
191
193
|
opensearch {
|
|
192
194
|
hosts => ["hostname:port"]
|
|
193
195
|
user => "admin"
|
|
194
|
-
password => "admin"
|
|
196
|
+
password => "<your-admin-password>"
|
|
195
197
|
index => "logstash-logs-%{+YYYY.MM.dd}"
|
|
196
198
|
}
|
|
197
199
|
}
|
|
@@ -215,3 +217,6 @@ original PR with an appropriate label `backport <backport-branch-name>` is merge
|
|
|
215
217
|
run successfully on the PR. For example, if a PR on main needs to be backported to `1.x` branch, add a label
|
|
216
218
|
`backport 1.x` to the PR and make sure the backport workflow runs on the PR along with other checks. Once this PR is
|
|
217
219
|
merged to main, the workflow will create a backport PR to the `1.x` branch.
|
|
220
|
+
|
|
221
|
+
# [Building Custom Docker Images](docs/building_custom_docker_images.md)
|
|
222
|
+
|
data/MAINTAINERS.md
CHANGED
|
@@ -5,20 +5,21 @@ This document contains a list of maintainers in this repo. See [opensearch-proje
|
|
|
5
5
|
## Current Maintainers
|
|
6
6
|
|
|
7
7
|
|
|
8
|
-
| Maintainer
|
|
9
|
-
|
|
|
10
|
-
| Asif Sohail Mohammed | [asifsmohammed](https://github.com/asifsmohammed)
|
|
11
|
-
| David Venable | [dlvenable](https://github.com/dlvenable)
|
|
12
|
-
| Hai Yan | [oeyh](https://github.com/oeyh)
|
|
8
|
+
| Maintainer | GitHub ID | Affiliation |
|
|
9
|
+
| -------------------- | ------------------------------------------------- | ----------- |
|
|
10
|
+
| Asif Sohail Mohammed | [asifsmohammed](https://github.com/asifsmohammed) | Amazon |
|
|
11
|
+
| David Venable | [dlvenable](https://github.com/dlvenable) | Amazon |
|
|
12
|
+
| Hai Yan | [oeyh](https://github.com/oeyh) | Amazon |
|
|
13
13
|
|
|
14
14
|
|
|
15
15
|
|
|
16
16
|
## Emeritus
|
|
17
17
|
|
|
18
|
-
| Maintainer
|
|
19
|
-
|
|
|
20
|
-
|
|
|
21
|
-
|
|
|
22
|
-
|
|
|
23
|
-
|
|
|
24
|
-
|
|
|
18
|
+
| Maintainer | GitHub ID | Affiliation |
|
|
19
|
+
| ------------------------ | ------------------------------------------- | ----------- |
|
|
20
|
+
| Daniel "dB." Doubrovkine | [dblock](https://github.com/dblock) | Independent |
|
|
21
|
+
| Jack Mazanec | [jmazanec15](https://github.com/jmazanec15) | Amazon |
|
|
22
|
+
| Vamshi Vijay Nakkirtha | [vamshin](https://github.com/vamshin) | Amazon |
|
|
23
|
+
| Vijayan Balasubramanian | [VijayanB](https://github.com/VijayanB) | Amazon |
|
|
24
|
+
| Deep Datta | [deepdatta](https://github.com/deepdatta) | Amazon |
|
|
25
|
+
| Shivani Shukla | [sshivanii](https://github.com/sshivanii) | Amazon |
|
data/README.md
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
|
|
12
12
|
## Welcome!
|
|
13
13
|
|
|
14
|
-
**logstash-output-opensearch** is a community-driven, open source fork logstash-output-elasticsearch licensed under the [Apache v2.0 License](LICENSE). For more information, see [opensearch.org](https://opensearch.org/).
|
|
14
|
+
**logstash-output-opensearch** is a community-driven, open source fork of logstash-output-elasticsearch licensed under the [Apache v2.0 License](LICENSE). For more information, see [opensearch.org](https://opensearch.org/).
|
|
15
15
|
|
|
16
16
|
The logstash-output-opensearch plugin helps to ship events from Logstash to OpenSearch cluster.
|
|
17
17
|
|
|
@@ -32,12 +32,14 @@ The logstash-output-opensearch plugin helps to ship events from Logstash to Open
|
|
|
32
32
|
## Configuration for Logstash Output Opensearch Plugin
|
|
33
33
|
|
|
34
34
|
To run the Logstash Output Opensearch plugin, add following configuration in your logstash.conf file.
|
|
35
|
+
Note: For logstash running with OpenSearch 2.12.0 and higher the admin password needs to be a custom strong password supplied during cluster setup.
|
|
36
|
+
|
|
35
37
|
```
|
|
36
38
|
output {
|
|
37
39
|
opensearch {
|
|
38
40
|
hosts => ["hostname:port"]
|
|
39
41
|
user => "admin"
|
|
40
|
-
password => "admin"
|
|
42
|
+
password => "<your-admin-password>"
|
|
41
43
|
index => "logstash-logs-%{+YYYY.MM.dd}"
|
|
42
44
|
}
|
|
43
45
|
}
|
|
@@ -62,6 +64,8 @@ output {
|
|
|
62
64
|
In addition to the existing authentication mechanisms, if we want to add new authentication then we will be adding them in the configuration by using auth_type.
|
|
63
65
|
|
|
64
66
|
Example Configuration for basic authentication:
|
|
67
|
+
Note: For logstash running with OpenSearch 2.12.0 and higher the admin password needs to be a custom strong password supplied during cluster setup.
|
|
68
|
+
|
|
65
69
|
```
|
|
66
70
|
output {
|
|
67
71
|
opensearch {
|
|
@@ -69,7 +73,7 @@ output {
|
|
|
69
73
|
auth_type => {
|
|
70
74
|
type => 'basic'
|
|
71
75
|
user => 'admin'
|
|
72
|
-
password => 'admin'
|
|
76
|
+
password => '<your-admin-password>'
|
|
73
77
|
}
|
|
74
78
|
index => "logstash-logs-%{+YYYY.MM.dd}"
|
|
75
79
|
}
|
|
@@ -77,6 +81,7 @@ output {
|
|
|
77
81
|
```
|
|
78
82
|
|
|
79
83
|
To ingest data into a `data stream` through logstash, we need to create the data stream and specify the name of data stream and the `op_type` of `create` in the output configuration. The sample configuration is shown below:
|
|
84
|
+
Note: For logstash running with OpenSearch 2.12.0 and higher the admin password needs to be a custom strong password supplied during cluster setup.
|
|
80
85
|
|
|
81
86
|
```yml
|
|
82
87
|
output {
|
|
@@ -85,7 +90,7 @@ output {
|
|
|
85
90
|
auth_type => {
|
|
86
91
|
type => 'basic'
|
|
87
92
|
user => 'admin'
|
|
88
|
-
password => 'admin'
|
|
93
|
+
password => '<your-admin-password>'
|
|
89
94
|
}
|
|
90
95
|
index => "my-data-stream"
|
|
91
96
|
action => "create"
|
|
@@ -105,10 +110,13 @@ Starting in 2.0.0, the aws sdk version is bumped to v3. In order for all other A
|
|
|
105
110
|
/usr/share/logstash/bin/logstash-plugin remove logstash-output-cloudwatch
|
|
106
111
|
|
|
107
112
|
/usr/share/logstash/bin/logstash-plugin install --version 0.1.0.pre logstash-integration-aws
|
|
108
|
-
bin/logstash-plugin install --version 2.0.0 logstash-output-opensearch
|
|
113
|
+
/usr/share/logstash/bin/logstash-plugin install --version 2.0.0 logstash-output-opensearch
|
|
109
114
|
```
|
|
115
|
+
|
|
116
|
+
For all other plugin configuration parameters, please inspect [api_configs.rb](lib/logstash/plugin_mixins/opensearch/api_configs.rb).
|
|
117
|
+
|
|
110
118
|
## ECS Compatibility
|
|
111
|
-
[Elastic Common Schema(ECS)](https://www.elastic.co/guide/en/ecs/current/index.html
|
|
119
|
+
[Elastic Common Schema(ECS)](https://www.elastic.co/guide/en/ecs/current/index.html) compatibility for V8 was added in 1.3.0. For more details on ECS support refer to this [documentation](docs/ecs_compatibility.md).
|
|
112
120
|
|
|
113
121
|
|
|
114
122
|
## Code of Conduct
|
data/RELEASING.md
CHANGED
|
@@ -13,13 +13,7 @@ This document explains the release strategy for artifacts in this organization.
|
|
|
13
13
|
|
|
14
14
|
### Release Branching
|
|
15
15
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
* **main**: The next _major_ release. This is the branch where all merges take place and code moves fast.
|
|
19
|
-
* **1.x**: The next _minor_ release. Once a change is merged into `main`, decide whether to backport it to `1.x`.
|
|
20
|
-
* **1.0**: The _current_ release. In between minor releases, only hotfixes (e.g. security) are backported to `1.0`.
|
|
21
|
-
|
|
22
|
-
Label PRs with the next major version label (e.g. `2.0.0`) and merge changes into `main`. Label PRs that you believe need to be backported as `1.x` and `1.0`. Backport PRs by checking out the versioned branch, cherry-pick changes and open a PR against each target backport branch.
|
|
16
|
+
This project currently releases only from `main`.
|
|
23
17
|
|
|
24
18
|
### Feature Branches
|
|
25
19
|
|
|
@@ -27,14 +21,17 @@ Do not creating branches in the upstream repo, use your fork, for the exception
|
|
|
27
21
|
|
|
28
22
|
## Release Labels
|
|
29
23
|
|
|
30
|
-
Repositories create consistent release labels, such as `
|
|
24
|
+
Repositories create consistent release labels, such as `1.0.0`, `1.1.0` and `2.0.0`, as well as `backport`. Use release labels to target an issue or a PR for a given release. See [MAINTAINERS](MAINTAINERS.md#triage-open-issues) for more information on triaging issues.
|
|
31
25
|
|
|
32
26
|
## Releasing
|
|
33
27
|
|
|
34
28
|
The release process is standard across repositories in this org and is run by a release manager volunteering from amongst [MAINTAINERS](MAINTAINERS.md).
|
|
35
29
|
|
|
36
|
-
1. Create a tag, e.g. 1.0.0
|
|
30
|
+
1. Create a tag, e.g. `1.0.0`, and push it to this GitHub repository. You can do this from your local fork:
|
|
31
|
+
1. `git fetch upstream --tags`
|
|
32
|
+
1. `git tag 2.0.3`
|
|
33
|
+
1. `git push --tags upstream`
|
|
37
34
|
1. The [release-drafter.yml](.github/workflows/release-drafter.yml) will be automatically kicked off and a draft release will be created.
|
|
38
|
-
1. This draft release triggers the [jenkins release workflow](https://build.ci.opensearch.org/job/logstash-
|
|
35
|
+
1. This draft release triggers the [jenkins release workflow](https://build.ci.opensearch.org/job/logstash-output-opensearch-release) as a As a result of which the logstash-output-plugin is released on [rubygems.org](https://rubygems.org/gems/logstash-output-opensearch). Please note that the release workflow is triggered only if created release is in draft state.
|
|
39
36
|
1. Once the above release workflow is successful, the drafted release on GitHub is published automatically.
|
|
40
37
|
1. Increment "version" in [logstash-output-opensearch.gemspec](./logstash-output-opensearch.gemspec) to the next iteration, e.g. 1.0.1.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
- [Building Custom Docker Images](#building-custom-docker-images)
|
|
2
|
+
- [Logstash 8.x](#logstash-8x)
|
|
3
|
+
- [Logstash 7.x](#logstash-7x)
|
|
4
|
+
- [Build Logstash Output OpenSearch Plugin Gem](#build-logstash-output-opensearch-plugin-gem)
|
|
5
|
+
- [Dockerfile](#dockerfile)
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
# Building Custom Docker Images
|
|
9
|
+
|
|
10
|
+
To build an image that is not available in the [official Docker repository tags](https://hub.docker.com/r/opensearchproject/logstash-oss-with-opensearch-output-plugin/tags), or to add specific plugins to your image, you can create a custom Dockerfile.
|
|
11
|
+
|
|
12
|
+
The process varies depending on whether you want to build an image with `Logstash 8.x` versions or `Logstash 7.x` versions
|
|
13
|
+
|
|
14
|
+
## Logstash 8.x
|
|
15
|
+
|
|
16
|
+
Create this `Dockerfile` to build an image with `OpenSearch 2.0.2` for **`Logstash 8.x`**
|
|
17
|
+
|
|
18
|
+
``` Dockerfile
|
|
19
|
+
ARG APP_VERSION
|
|
20
|
+
|
|
21
|
+
FROM docker.elastic.co/logstash/logstash-oss:${APP_VERSION}
|
|
22
|
+
RUN logstash-plugin install --version 7.1.1 logstash-integration-aws
|
|
23
|
+
RUN logstash-plugin install --version 2.0.2 logstash-output-opensearch
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Logstash 7.x
|
|
27
|
+
|
|
28
|
+
For **`Logstash 7.x`** , the Logstash output OpenSearch gem needs to be build.
|
|
29
|
+
|
|
30
|
+
### Build Logstash Output OpenSearch Plugin Gem
|
|
31
|
+
|
|
32
|
+
1. Clone `logstash-output-opensearch repo`
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
git clone https://github.com/opensearch-project/logstash-output-opensearch.git
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
2. Checkout the tag for the plugin version you want to build. For the version [2.0.2](https://github.com/opensearch-project/logstash-output-opensearch/tree/2.0.2) for example
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
git checkout 2.0.2
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
3. Remove [this line that adds the json version spec](https://github.com/opensearch-project/logstash-output-opensearch/blob/2.0.2/logstash-output-opensearch.gemspec#L49). This version of the JSON gem is incompatible with `Logstash version 7.x`.
|
|
46
|
+
|
|
47
|
+
4. Build the gem by running the following command:
|
|
48
|
+
|
|
49
|
+
```sh
|
|
50
|
+
gem build logstash-output-opensearch.gemspec
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
The Gemfile `logstash-output-opensearch-2.0.2-x86_64-linux.gem` will be generated.
|
|
54
|
+
|
|
55
|
+
### Dockerfile
|
|
56
|
+
|
|
57
|
+
Create this Dockerfile to build an image with `logstash version 7.x` and the previously generated `Gemfile` :
|
|
58
|
+
|
|
59
|
+
```Dockerfile
|
|
60
|
+
ARG APP_VERSION
|
|
61
|
+
|
|
62
|
+
FROM docker.elastic.co/logstash/logstash-oss:${APP_VERSION}
|
|
63
|
+
|
|
64
|
+
USER logstash
|
|
65
|
+
# Remove existing logstash aws plugins and install logstash-integration-aws to keep sdk dependency the same
|
|
66
|
+
# https://github.com/logstash-plugins/logstash-mixin-aws/issues/38
|
|
67
|
+
# https://github.com/opensearch-project/logstash-output-opensearch#configuration-for-logstash-output-opensearch-plugin
|
|
68
|
+
RUN logstash-plugin remove logstash-input-s3
|
|
69
|
+
RUN logstash-plugin remove logstash-input-sqs
|
|
70
|
+
RUN logstash-plugin remove logstash-output-s3
|
|
71
|
+
RUN logstash-plugin remove logstash-output-sns
|
|
72
|
+
RUN logstash-plugin remove logstash-output-sqs
|
|
73
|
+
RUN logstash-plugin remove logstash-output-cloudwatch
|
|
74
|
+
|
|
75
|
+
RUN logstash-plugin install --version 7.1.1 logstash-integration-aws
|
|
76
|
+
|
|
77
|
+
COPY logstash-output-opensearch-2.0.2-x86_64-linux.gem /usr/share
|
|
78
|
+
RUN logstash-plugin install /usr/share/logstash-output-opensearch-2.0.2-x86_64-linux.gem
|
|
79
|
+
```
|
|
@@ -33,7 +33,8 @@ module LogStash; module Outputs; class OpenSearch; class HttpClient;
|
|
|
33
33
|
:profile,
|
|
34
34
|
:instance_profile_credentials_retries,
|
|
35
35
|
:instance_profile_credentials_timeout,
|
|
36
|
-
:region
|
|
36
|
+
:region,
|
|
37
|
+
:account_id)
|
|
37
38
|
|
|
38
39
|
class ManticoreAdapter
|
|
39
40
|
attr_reader :manticore, :logger
|
|
@@ -78,10 +79,11 @@ module LogStash; module Outputs; class OpenSearch; class HttpClient;
|
|
|
78
79
|
instance_cred_retries = options[:auth_type]["instance_profile_credentials_retries"] || AWS_DEFAULT_PROFILE_CREDENTIAL_RETRY
|
|
79
80
|
instance_cred_timeout = options[:auth_type]["instance_profile_credentials_timeout"] || AWS_DEFAULT_PROFILE_CREDENTIAL_TIMEOUT
|
|
80
81
|
region = options[:auth_type]["region"] || AWS_DEFAULT_REGION
|
|
82
|
+
account_id = nil
|
|
81
83
|
set_aws_region(region)
|
|
82
84
|
set_service_name(options[:auth_type]["service_name"] || AWS_SERVICE)
|
|
83
85
|
|
|
84
|
-
credential_config = AWSIAMCredential.new(aws_access_key_id, aws_secret_access_key, session_token, profile, instance_cred_retries, instance_cred_timeout, region)
|
|
86
|
+
credential_config = AWSIAMCredential.new(aws_access_key_id, aws_secret_access_key, session_token, profile, instance_cred_retries, instance_cred_timeout, region, account_id)
|
|
85
87
|
@credentials = Aws::CredentialProviderChain.new(credential_config).resolve
|
|
86
88
|
end
|
|
87
89
|
|
|
@@ -87,6 +87,7 @@ module LogStash; module PluginMixins; module OpenSearch
|
|
|
87
87
|
|
|
88
88
|
def discover_cluster_uuid
|
|
89
89
|
return unless defined?(plugin_metadata)
|
|
90
|
+
return if params && params['auth_type'] && params['auth_type']['service_name'] == "aoss" # AOSS doesn't support GET /
|
|
90
91
|
cluster_info = client.get('/')
|
|
91
92
|
plugin_metadata.set(:cluster_uuid, cluster_info['cluster_uuid'])
|
|
92
93
|
rescue => e
|
|
@@ -11,7 +11,7 @@ signing_key_path = "gem-private_key.pem"
|
|
|
11
11
|
|
|
12
12
|
Gem::Specification.new do |s|
|
|
13
13
|
s.name = 'logstash-output-opensearch'
|
|
14
|
-
s.version = '2.0
|
|
14
|
+
s.version = '2.1.0'
|
|
15
15
|
|
|
16
16
|
s.licenses = ['Apache-2.0']
|
|
17
17
|
s.summary = "Stores logs in OpenSearch"
|
|
@@ -45,7 +45,7 @@ Gem::Specification.new do |s|
|
|
|
45
45
|
s.add_runtime_dependency 'stud', ['>= 0.0.17', '~> 0.0']
|
|
46
46
|
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
|
|
47
47
|
s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.0'
|
|
48
|
-
s.add_runtime_dependency 'aws-sdk', '~> 3'
|
|
48
|
+
s.add_runtime_dependency 'aws-sdk-core', '~> 3'
|
|
49
49
|
s.add_runtime_dependency 'json', '>= 2.3.0', '~> 2'
|
|
50
50
|
|
|
51
51
|
s.add_development_dependency 'logstash-codec-plain'
|
|
@@ -140,7 +140,7 @@ describe "indexing" do
|
|
|
140
140
|
end
|
|
141
141
|
describe "a secured indexer", :secure_integration => true do
|
|
142
142
|
let(:user) { "admin" }
|
|
143
|
-
let(:password) {
|
|
143
|
+
let(:password) { OpenSearchHelper.admin_password }
|
|
144
144
|
let(:opensearch_url) {"https://integration:9200"}
|
|
145
145
|
let(:config) do
|
|
146
146
|
{
|
|
@@ -172,7 +172,8 @@ describe "indexing" do
|
|
|
172
172
|
:auth_type => {
|
|
173
173
|
"type"=>"basic",
|
|
174
174
|
"user" => "admin",
|
|
175
|
-
"password" =>
|
|
175
|
+
"password" => OpenSearchHelper.admin_password
|
|
176
|
+
}
|
|
176
177
|
} }
|
|
177
178
|
let(:user) {options[:auth_type]["user"]}
|
|
178
179
|
let(:password) {options[:auth_type]["password"]}
|
|
@@ -72,6 +72,15 @@ module OpenSearchHelper
|
|
|
72
72
|
end
|
|
73
73
|
end
|
|
74
74
|
|
|
75
|
+
# set admin password based on version
|
|
76
|
+
def self.admin_password
|
|
77
|
+
if check_version?('< 2.12.0') || check_version?('> 7')
|
|
78
|
+
"admin"
|
|
79
|
+
else
|
|
80
|
+
"myStrongPassword123!"
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
75
84
|
def clean(client)
|
|
76
85
|
client.indices.delete_template(:name => "*")
|
|
77
86
|
client.indices.delete_index_template(:name => "logstash*") rescue nil
|
|
@@ -776,6 +776,30 @@ describe LogStash::Outputs::OpenSearch do
|
|
|
776
776
|
expect(logger).to have_received(:error).with(/Unable to retrieve OpenSearch cluster uuid/i, anything)
|
|
777
777
|
end
|
|
778
778
|
|
|
779
|
+
context 'with iam auth' do
|
|
780
|
+
context 'es' do
|
|
781
|
+
let(:options) { { 'hosts' => '127.0.0.1:9999', 'auth_type' => { 'service_name' => 'es' } } }
|
|
782
|
+
it "logs inability to retrieve uuid" do
|
|
783
|
+
allow(subject).to receive(:install_template)
|
|
784
|
+
subject.register
|
|
785
|
+
subject.send :wait_for_successful_connection
|
|
786
|
+
|
|
787
|
+
expect(logger).to have_received(:error).with(/Unable to retrieve OpenSearch cluster uuid/i, anything)
|
|
788
|
+
end
|
|
789
|
+
end
|
|
790
|
+
|
|
791
|
+
context 'aoss' do
|
|
792
|
+
let(:options) { { 'hosts' => '127.0.0.1:9999', 'auth_type' => { 'service_name' => 'aoss' } } }
|
|
793
|
+
it "does not attempt to retrieve uuid" do
|
|
794
|
+
allow(subject).to receive(:install_template)
|
|
795
|
+
subject.register
|
|
796
|
+
subject.send :wait_for_successful_connection
|
|
797
|
+
|
|
798
|
+
expect(logger).to_not have_received(:error)
|
|
799
|
+
end
|
|
800
|
+
end
|
|
801
|
+
end
|
|
802
|
+
|
|
779
803
|
it "logs template install failure" do
|
|
780
804
|
allow(subject).to receive(:discover_cluster_uuid)
|
|
781
805
|
subject.register
|
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,19 +1,18 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: logstash-output-opensearch
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 2.0
|
|
4
|
+
version: 2.1.0
|
|
5
5
|
platform: java
|
|
6
6
|
authors:
|
|
7
7
|
- Elastic
|
|
8
8
|
- OpenSearch Contributors
|
|
9
|
-
autorequire:
|
|
10
9
|
bindir: bin
|
|
11
10
|
cert_chain:
|
|
12
11
|
- |
|
|
13
12
|
-----BEGIN CERTIFICATE-----
|
|
14
13
|
MIIDfDCCAmSgAwIBAgIBATANBgkqhkiG9w0BAQsFADBCMRMwEQYDVQQDDApvcGVu
|
|
15
14
|
c2VhcmNoMRYwFAYKCZImiZPyLGQBGRYGYW1hem9uMRMwEQYKCZImiZPyLGQBGRYD
|
|
16
|
-
|
|
15
|
+
Y29tMB4XDTI1MDcwNzIwMDIyNVoXDTI2MDcwNzIwMDIyNVowQjETMBEGA1UEAwwK
|
|
17
16
|
b3BlbnNlYXJjaDEWMBQGCgmSJomT8ixkARkWBmFtYXpvbjETMBEGCgmSJomT8ixk
|
|
18
17
|
ARkWA2NvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAM1z3/jcitjV
|
|
19
18
|
umXwRFn+JSBBd36qZB54Dtucf6+E2fmNPzBRhgYN5XJy/+clQJ9NIJV7C8H52P3V
|
|
@@ -24,16 +23,17 @@ cert_chain:
|
|
|
24
23
|
zfR37/NQFkECAwEAAaN9MHswCQYDVR0TBAIwADALBgNVHQ8EBAMCBLAwHQYDVR0O
|
|
25
24
|
BBYEFJJ2myhLXK742btavNbG0IWrMNBIMCAGA1UdEQQZMBeBFW9wZW5zZWFyY2hA
|
|
26
25
|
YW1hem9uLmNvbTAgBgNVHRIEGTAXgRVvcGVuc2VhcmNoQGFtYXpvbi5jb20wDQYJ
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
26
|
+
KoZIhvcNAQELBQADggEBABoyE+vT8JDWYpSyDbMTDS7Am/wRsxzRPN9elTTTLB8a
|
|
27
|
+
YlIn33WwOjg4OZeSmGqTOrlr3ml2YRFr7AuBvk2QrPCSwa25Ta7D39wmy+DMU6RZ
|
|
28
|
+
TkgPE/0/WH1HH72XS+gyQ+VypItQRAWuXw773uXAw7nxUibhBFnvvugfiv1vDD3W
|
|
29
|
+
T54ektIq3Zge7DJ+jldYHsgkRSpDNGDKE1FMT15DqIbC9/Zncu5OAcyvP0MAsIu6
|
|
30
|
+
o3OY7TEtlEaoDIDrA4z7IYumFy9TDivmILkTJzWsxSloQBapMYO7NBc340n2avwn
|
|
31
|
+
4EIvCSrKF9/Pk18ts8FuoqT89wCNnJBrRqMy/ylgHgY=
|
|
33
32
|
-----END CERTIFICATE-----
|
|
34
|
-
date:
|
|
33
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
35
34
|
dependencies:
|
|
36
35
|
- !ruby/object:Gem::Dependency
|
|
36
|
+
name: manticore
|
|
37
37
|
requirement: !ruby/object:Gem::Requirement
|
|
38
38
|
requirements:
|
|
39
39
|
- - ">="
|
|
@@ -42,9 +42,8 @@ dependencies:
|
|
|
42
42
|
- - "<"
|
|
43
43
|
- !ruby/object:Gem::Version
|
|
44
44
|
version: 1.0.0
|
|
45
|
-
name: manticore
|
|
46
|
-
prerelease: false
|
|
47
45
|
type: :runtime
|
|
46
|
+
prerelease: false
|
|
48
47
|
version_requirements: !ruby/object:Gem::Requirement
|
|
49
48
|
requirements:
|
|
50
49
|
- - ">="
|
|
@@ -54,6 +53,7 @@ dependencies:
|
|
|
54
53
|
- !ruby/object:Gem::Version
|
|
55
54
|
version: 1.0.0
|
|
56
55
|
- !ruby/object:Gem::Dependency
|
|
56
|
+
name: stud
|
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
|
58
58
|
requirements:
|
|
59
59
|
- - ">="
|
|
@@ -62,9 +62,8 @@ dependencies:
|
|
|
62
62
|
- - "~>"
|
|
63
63
|
- !ruby/object:Gem::Version
|
|
64
64
|
version: '0.0'
|
|
65
|
-
name: stud
|
|
66
|
-
prerelease: false
|
|
67
65
|
type: :runtime
|
|
66
|
+
prerelease: false
|
|
68
67
|
version_requirements: !ruby/object:Gem::Requirement
|
|
69
68
|
requirements:
|
|
70
69
|
- - ">="
|
|
@@ -74,6 +73,7 @@ dependencies:
|
|
|
74
73
|
- !ruby/object:Gem::Version
|
|
75
74
|
version: '0.0'
|
|
76
75
|
- !ruby/object:Gem::Dependency
|
|
76
|
+
name: logstash-core-plugin-api
|
|
77
77
|
requirement: !ruby/object:Gem::Requirement
|
|
78
78
|
requirements:
|
|
79
79
|
- - ">="
|
|
@@ -82,9 +82,8 @@ dependencies:
|
|
|
82
82
|
- - "<="
|
|
83
83
|
- !ruby/object:Gem::Version
|
|
84
84
|
version: '2.99'
|
|
85
|
-
name: logstash-core-plugin-api
|
|
86
|
-
prerelease: false
|
|
87
85
|
type: :runtime
|
|
86
|
+
prerelease: false
|
|
88
87
|
version_requirements: !ruby/object:Gem::Requirement
|
|
89
88
|
requirements:
|
|
90
89
|
- - ">="
|
|
@@ -94,34 +93,35 @@ dependencies:
|
|
|
94
93
|
- !ruby/object:Gem::Version
|
|
95
94
|
version: '2.99'
|
|
96
95
|
- !ruby/object:Gem::Dependency
|
|
96
|
+
name: logstash-mixin-ecs_compatibility_support
|
|
97
97
|
requirement: !ruby/object:Gem::Requirement
|
|
98
98
|
requirements:
|
|
99
99
|
- - "~>"
|
|
100
100
|
- !ruby/object:Gem::Version
|
|
101
101
|
version: '1.0'
|
|
102
|
-
name: logstash-mixin-ecs_compatibility_support
|
|
103
|
-
prerelease: false
|
|
104
102
|
type: :runtime
|
|
103
|
+
prerelease: false
|
|
105
104
|
version_requirements: !ruby/object:Gem::Requirement
|
|
106
105
|
requirements:
|
|
107
106
|
- - "~>"
|
|
108
107
|
- !ruby/object:Gem::Version
|
|
109
108
|
version: '1.0'
|
|
110
109
|
- !ruby/object:Gem::Dependency
|
|
110
|
+
name: aws-sdk-core
|
|
111
111
|
requirement: !ruby/object:Gem::Requirement
|
|
112
112
|
requirements:
|
|
113
113
|
- - "~>"
|
|
114
114
|
- !ruby/object:Gem::Version
|
|
115
115
|
version: '3'
|
|
116
|
-
name: aws-sdk
|
|
117
|
-
prerelease: false
|
|
118
116
|
type: :runtime
|
|
117
|
+
prerelease: false
|
|
119
118
|
version_requirements: !ruby/object:Gem::Requirement
|
|
120
119
|
requirements:
|
|
121
120
|
- - "~>"
|
|
122
121
|
- !ruby/object:Gem::Version
|
|
123
122
|
version: '3'
|
|
124
123
|
- !ruby/object:Gem::Dependency
|
|
124
|
+
name: json
|
|
125
125
|
requirement: !ruby/object:Gem::Requirement
|
|
126
126
|
requirements:
|
|
127
127
|
- - ">="
|
|
@@ -130,9 +130,8 @@ dependencies:
|
|
|
130
130
|
- - "~>"
|
|
131
131
|
- !ruby/object:Gem::Version
|
|
132
132
|
version: '2'
|
|
133
|
-
name: json
|
|
134
|
-
prerelease: false
|
|
135
133
|
type: :runtime
|
|
134
|
+
prerelease: false
|
|
136
135
|
version_requirements: !ruby/object:Gem::Requirement
|
|
137
136
|
requirements:
|
|
138
137
|
- - ">="
|
|
@@ -142,70 +141,70 @@ dependencies:
|
|
|
142
141
|
- !ruby/object:Gem::Version
|
|
143
142
|
version: '2'
|
|
144
143
|
- !ruby/object:Gem::Dependency
|
|
144
|
+
name: logstash-codec-plain
|
|
145
145
|
requirement: !ruby/object:Gem::Requirement
|
|
146
146
|
requirements:
|
|
147
147
|
- - ">="
|
|
148
148
|
- !ruby/object:Gem::Version
|
|
149
149
|
version: '0'
|
|
150
|
-
name: logstash-codec-plain
|
|
151
|
-
prerelease: false
|
|
152
150
|
type: :development
|
|
151
|
+
prerelease: false
|
|
153
152
|
version_requirements: !ruby/object:Gem::Requirement
|
|
154
153
|
requirements:
|
|
155
154
|
- - ">="
|
|
156
155
|
- !ruby/object:Gem::Version
|
|
157
156
|
version: '0'
|
|
158
157
|
- !ruby/object:Gem::Dependency
|
|
158
|
+
name: logstash-devutils
|
|
159
159
|
requirement: !ruby/object:Gem::Requirement
|
|
160
160
|
requirements:
|
|
161
161
|
- - ">="
|
|
162
162
|
- !ruby/object:Gem::Version
|
|
163
163
|
version: '0'
|
|
164
|
-
name: logstash-devutils
|
|
165
|
-
prerelease: false
|
|
166
164
|
type: :development
|
|
165
|
+
prerelease: false
|
|
167
166
|
version_requirements: !ruby/object:Gem::Requirement
|
|
168
167
|
requirements:
|
|
169
168
|
- - ">="
|
|
170
169
|
- !ruby/object:Gem::Version
|
|
171
170
|
version: '0'
|
|
172
171
|
- !ruby/object:Gem::Dependency
|
|
172
|
+
name: flores
|
|
173
173
|
requirement: !ruby/object:Gem::Requirement
|
|
174
174
|
requirements:
|
|
175
175
|
- - ">="
|
|
176
176
|
- !ruby/object:Gem::Version
|
|
177
177
|
version: '0'
|
|
178
|
-
name: flores
|
|
179
|
-
prerelease: false
|
|
180
178
|
type: :development
|
|
179
|
+
prerelease: false
|
|
181
180
|
version_requirements: !ruby/object:Gem::Requirement
|
|
182
181
|
requirements:
|
|
183
182
|
- - ">="
|
|
184
183
|
- !ruby/object:Gem::Version
|
|
185
184
|
version: '0'
|
|
186
185
|
- !ruby/object:Gem::Dependency
|
|
186
|
+
name: cabin
|
|
187
187
|
requirement: !ruby/object:Gem::Requirement
|
|
188
188
|
requirements:
|
|
189
189
|
- - "~>"
|
|
190
190
|
- !ruby/object:Gem::Version
|
|
191
191
|
version: '0.6'
|
|
192
|
-
name: cabin
|
|
193
|
-
prerelease: false
|
|
194
192
|
type: :development
|
|
193
|
+
prerelease: false
|
|
195
194
|
version_requirements: !ruby/object:Gem::Requirement
|
|
196
195
|
requirements:
|
|
197
196
|
- - "~>"
|
|
198
197
|
- !ruby/object:Gem::Version
|
|
199
198
|
version: '0.6'
|
|
200
199
|
- !ruby/object:Gem::Dependency
|
|
200
|
+
name: opensearch-ruby
|
|
201
201
|
requirement: !ruby/object:Gem::Requirement
|
|
202
202
|
requirements:
|
|
203
203
|
- - "~>"
|
|
204
204
|
- !ruby/object:Gem::Version
|
|
205
205
|
version: '1'
|
|
206
|
-
name: opensearch-ruby
|
|
207
|
-
prerelease: false
|
|
208
206
|
type: :development
|
|
207
|
+
prerelease: false
|
|
209
208
|
version_requirements: !ruby/object:Gem::Requirement
|
|
210
209
|
requirements:
|
|
211
210
|
- - "~>"
|
|
@@ -231,6 +230,7 @@ files:
|
|
|
231
230
|
- README.md
|
|
232
231
|
- RELEASING.md
|
|
233
232
|
- SECURITY.md
|
|
233
|
+
- docs/building_custom_docker_images.md
|
|
234
234
|
- docs/ecs_compatibility.md
|
|
235
235
|
- lib/logstash/outputs/opensearch.rb
|
|
236
236
|
- lib/logstash/outputs/opensearch/http_client.rb
|
|
@@ -292,7 +292,6 @@ metadata:
|
|
|
292
292
|
logstash_plugin: 'true'
|
|
293
293
|
logstash_group: output
|
|
294
294
|
source_code_uri: https://github.com/opensearch-project/logstash-output-opensearch
|
|
295
|
-
post_install_message:
|
|
296
295
|
rdoc_options: []
|
|
297
296
|
require_paths:
|
|
298
297
|
- lib
|
|
@@ -307,8 +306,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
307
306
|
- !ruby/object:Gem::Version
|
|
308
307
|
version: '0'
|
|
309
308
|
requirements: []
|
|
310
|
-
rubygems_version: 3.
|
|
311
|
-
signing_key:
|
|
309
|
+
rubygems_version: 3.7.2
|
|
312
310
|
specification_version: 4
|
|
313
311
|
summary: Stores logs in OpenSearch
|
|
314
312
|
test_files:
|
metadata.gz.sig
CHANGED
|
Binary file
|