logstash-output-amazon_es 7.0.1-java → 7.1.0-java
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile +4 -0
- data/NOTICE.TXT +10 -10
- data/README.md +66 -33
- data/lib/logstash/outputs/amazon_es/common.rb +11 -3
- data/lib/logstash/outputs/amazon_es/common_configs.rb +4 -0
- data/lib/logstash/outputs/amazon_es/http_client/manticore_adapter.rb +6 -1
- data/lib/logstash/outputs/amazon_es/http_client/pool.rb +28 -17
- data/lib/logstash/outputs/amazon_es/http_client.rb +9 -2
- data/lib/logstash/outputs/amazon_es/http_client_builder.rb +7 -1
- data/lib/logstash/outputs/amazon_es/template_manager.rb +4 -0
- data/lib/logstash/outputs/amazon_es.rb +14 -1
- data/logstash-output-amazon_es.gemspec +5 -1
- data/spec/es_spec_helper.rb +4 -0
- data/spec/unit/http_client_builder_spec.rb +4 -0
- data/spec/unit/outputs/elasticsearch/http_client/manticore_adapter_spec.rb +4 -0
- data/spec/unit/outputs/elasticsearch/http_client/pool_spec.rb +4 -0
- data/spec/unit/outputs/elasticsearch/http_client_spec.rb +4 -0
- data/spec/unit/outputs/elasticsearch/template_manager_spec.rb +4 -0
- data/spec/unit/outputs/elasticsearch_spec.rb +4 -0
- data/spec/unit/outputs/error_whitelist_spec.rb +4 -0
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 827216d23ab630ee15e9ab6fd7324ef2dd506ff3ca7296ada51afa09e78b08e3
|
4
|
+
data.tar.gz: 67471b7f10a922559cd3247a370f80b5989c30947bd3981ea2233c1b0f310a18
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 7ce4a46376dce1d4f0cd684086b28d94c30fd1fb841e59562713b2c674b3ed01df89e6257a78612df7cad1b88569b771e2e513581981366994563fecd7d23edc
|
7
|
+
data.tar.gz: b7c513e943a05a6e78cf602d66cf6139d42a36881acd23b7d7fbebb1319d61c9ceff7753b2c1303f3c29b0afcf621f80d741e013bf8e79a35cdc2561bbe76694
|
data/Gemfile
CHANGED
data/NOTICE.TXT
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
Logstash-output-amazon_es Plugin
|
2
|
-
Copyright
|
2
|
+
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
3
3
|
|
4
|
-
Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
-
you may not use this file except in compliance with the License.
|
6
|
-
You may obtain a copy of the License at
|
7
4
|
|
8
|
-
|
5
|
+
This product includes software from the Apache-2.0 licensed Elasticsearch project.
|
9
6
|
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
7
|
+
It requires the following notice:
|
8
|
+
|
9
|
+
Elasticsearch
|
10
|
+
Copyright 2012-2015 Elasticsearch
|
11
|
+
Copyright 2020 Elastic and contributors
|
12
|
+
|
13
|
+
This product includes software developed by The Apache Software
|
14
|
+
Foundation (http://www.apache.org/).
|
data/README.md
CHANGED
@@ -1,6 +1,9 @@
|
|
1
1
|
# Logstash Plugin
|
2
2
|
|
3
|
-
This is a plugin for [Logstash](https://github.com/elastic/logstash)
|
3
|
+
This is a plugin for [Logstash](https://github.com/elastic/logstash) which outputs
|
4
|
+
to [Amazon OpenSearch Service](https://aws.amazon.com/opensearch-service/)
|
5
|
+
(successor to Amazon Elasticsearch Service) using
|
6
|
+
[SigV4 signing](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html).
|
4
7
|
|
5
8
|
## License
|
6
9
|
|
@@ -8,14 +11,39 @@ This library is licensed under Apache License 2.0.
|
|
8
11
|
|
9
12
|
## Compatibility
|
10
13
|
|
11
|
-
The following table shows the versions of logstash and logstash-output-amazon_es
|
14
|
+
The following table shows the versions of logstash and logstash-output-amazon_es plugin was built with.
|
15
|
+
|
16
|
+
| logstash-output-amazon_es | Logstash |
|
17
|
+
|---------------------------|----------|
|
18
|
+
| 6.0.0 | < 6.0.0 |
|
19
|
+
| 6.4.2 | >= 6.0.0 |
|
20
|
+
| 7.0.1 | >= 7.0.0 |
|
21
|
+
|
22
|
+
|
23
|
+
Also, logstash-output-amazon_es plugin versions 6.4.0 and newer are tested to be compatible with Elasticsearch 6.5 and greater.
|
24
|
+
|
25
|
+
| logstash-output-amazon_es | Elasticsearch |
|
26
|
+
| ------------- |----------|
|
27
|
+
| 6.4.0+ | 6.5+ |
|
28
|
+
|
29
|
+
|
30
|
+
## Installation
|
31
|
+
|
32
|
+
To install the latest version, use the normal Logstash plugin script.
|
33
|
+
|
34
|
+
```sh
|
35
|
+
bin/logstash-plugin install logstash-output-amazon_es
|
36
|
+
```
|
37
|
+
|
38
|
+
If you want to use old version of logstash-output-amazon_es, you can use the `--version`
|
39
|
+
flag to specify the version. For example:
|
40
|
+
|
41
|
+
```sh
|
42
|
+
bin/logstash-plugin install --version 6.4.2 logstash-output-amazon_es
|
43
|
+
```
|
44
|
+
|
12
45
|
|
13
|
-
| logstash-output-amazon_es | Logstash |
|
14
|
-
| ------------- | ------------- |
|
15
|
-
| 6.0.0 | <6.0.0 |
|
16
|
-
| 6.4.0 | >6.0.0 |
|
17
46
|
|
18
|
-
logstash-output-amazon_es plugin versions 6.4.0 and newer are tested to be compatible with Elasticsearch 6.5 and greater.
|
19
47
|
|
20
48
|
## Configuration for Amazon Elasticsearch Service Output Plugin
|
21
49
|
|
@@ -62,17 +90,34 @@ output {
|
|
62
90
|
- max_bulk_bytes - The max size for a bulk request in bytes. Default is 20MB. It is recommended not to change this value unless needed. For guidance on changing this value, please consult the table for network limits for your instance type: https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html#network-limits
|
63
91
|
|
64
92
|
After 6.4.0, users can't set batch size in this output plugin config. However, users can still set batch size in logstash.yml file.
|
93
|
+
|
94
|
+
### Advanced Optional Parameters
|
95
|
+
|
96
|
+
Starting logstash-output-amazon_es v7.1.0, we have introduced the following optional parameters to resolve specific use cases:
|
97
|
+
|
98
|
+
- service_name (string, default => "es") - Users can define any service name to which the plugin will send a SigV4 signed request
|
99
|
+
- skip_healthcheck (boolean, default => false) - Boolean to skip healthcheck API and set the major ES version to 7
|
100
|
+
- skip_template_installation (boolean, default => false) - Boolean to allow users to skip installing templates in usecases that don't require them
|
101
|
+
|
65
102
|
## Developing
|
66
103
|
|
67
|
-
### 1.
|
104
|
+
### 1. Prerequisites
|
105
|
+
To get started, you can install JRuby with the Bundler gem using [RVM](https://rvm.io/rvm/install)
|
106
|
+
|
107
|
+
rvm install jruby
|
108
|
+
|
109
|
+
### 2. Plugin Development and Testing
|
68
110
|
|
69
111
|
#### Code
|
70
112
|
|
71
|
-
1.
|
113
|
+
1. Verify JRuby is already installed
|
114
|
+
|
115
|
+
```sh
|
116
|
+
jruby -v
|
117
|
+
```
|
72
118
|
|
73
|
-
2. Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. [Example plugins](https://github.com/logstash-plugins?query=example) exist.
|
74
119
|
|
75
|
-
|
120
|
+
2. Install dependencies:
|
76
121
|
|
77
122
|
```sh
|
78
123
|
bundle install
|
@@ -92,14 +137,14 @@ After 6.4.0, users can't set batch size in this output plugin config. However, u
|
|
92
137
|
bundle exec rspec
|
93
138
|
```
|
94
139
|
|
95
|
-
###
|
140
|
+
### 3. Running your unpublished plugin in Logstash
|
96
141
|
|
97
|
-
####
|
142
|
+
#### 3.1 Run in a local Logstash clone
|
98
143
|
|
99
144
|
1. Edit Logstash `Gemfile` and add the local plugin path, for example:
|
100
145
|
|
101
146
|
```ruby
|
102
|
-
gem "logstash-
|
147
|
+
gem "logstash-output-amazon_es", :path => "/your/local/logstash-output-amazon_es"
|
103
148
|
```
|
104
149
|
|
105
150
|
2. Install the plugin:
|
@@ -115,12 +160,12 @@ After 6.4.0, users can't set batch size in this output plugin config. However, u
|
|
115
160
|
3. Run Logstash with your plugin:
|
116
161
|
|
117
162
|
```sh
|
118
|
-
bin/logstash -e '
|
163
|
+
bin/logstash -e 'output {amazon_es {}}'
|
119
164
|
```
|
120
165
|
|
121
166
|
At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply re-run Logstash.
|
122
167
|
|
123
|
-
####
|
168
|
+
#### 3.2 Run in an installed Logstash
|
124
169
|
|
125
170
|
Before build your `Gemfile`, please make sure use JRuby. Here is how you can know your local Ruby version:
|
126
171
|
|
@@ -131,37 +176,25 @@ rvm list
|
|
131
176
|
Please make sure you current using JRuby. Here is how you can change to JRuby
|
132
177
|
|
133
178
|
```sh
|
134
|
-
rvm jruby
|
179
|
+
rvm jruby-9.2.5.0
|
135
180
|
```
|
136
181
|
|
137
|
-
You can use the same **
|
182
|
+
You can use the same **3.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory. You can also build the gem and install it using:
|
138
183
|
|
139
184
|
1. Build your plugin gem:
|
140
185
|
|
141
186
|
```sh
|
142
|
-
gem build logstash-
|
187
|
+
gem build logstash-output-amazon_es.gemspec
|
143
188
|
```
|
144
189
|
|
145
|
-
2. Install the plugin from the Logstash home:
|
190
|
+
2. Install the plugin from the Logstash home. Please be sure to check the version number against the actual Gem file. Run:
|
146
191
|
|
147
192
|
```sh
|
148
|
-
|
149
|
-
bin/logstash-plugin install --no-verify
|
150
|
-
|
151
|
-
# Prior to Logstash 2.3
|
152
|
-
bin/plugin install --no-verify
|
193
|
+
bin/logstash-plugin install /your/local/logstash-output-amazon_es/logstash-output-amazon_es-7.0.1-java.gem
|
153
194
|
```
|
154
195
|
|
155
196
|
3. Start Logstash and test the plugin.
|
156
197
|
|
157
|
-
## Old version support
|
158
|
-
|
159
|
-
If you want to use old version of logstash-output-amazon_es, you can install with this:
|
160
|
-
```sh
|
161
|
-
bin/logstash-plugin install logstash-output-amazon_es -v 2.0.0
|
162
|
-
```
|
163
|
-
|
164
|
-
|
165
198
|
|
166
199
|
## Contributing
|
167
200
|
|
@@ -1,8 +1,12 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require "logstash/outputs/amazon_es/template_manager"
|
2
6
|
|
3
7
|
module LogStash; module Outputs; class AmazonElasticSearch;
|
4
8
|
module Common
|
5
|
-
attr_reader :client, :hosts
|
9
|
+
attr_reader :client, :hosts, :skip_template_installation
|
6
10
|
|
7
11
|
# These codes apply to documents, not at the request level
|
8
12
|
DOC_DLQ_CODES = [400, 404]
|
@@ -115,8 +119,12 @@ module LogStash; module Outputs; class AmazonElasticSearch;
|
|
115
119
|
end
|
116
120
|
|
117
121
|
def install_template
|
118
|
-
|
119
|
-
|
122
|
+
if skip_template_installation == false
|
123
|
+
TemplateManager.install_template(self)
|
124
|
+
@template_installed.make_true
|
125
|
+
elsif skip_template_installation == true
|
126
|
+
@template_installed.make_true
|
127
|
+
end
|
120
128
|
end
|
121
129
|
|
122
130
|
def check_action_validity
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require 'forwardable' # Needed for logstash core SafeURI. We need to patch this in core: https://github.com/elastic/logstash/pull/5978
|
2
6
|
|
3
7
|
module LogStash; module Outputs; class AmazonElasticSearch
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require 'manticore'
|
2
6
|
require 'cgi'
|
3
7
|
require 'aws-sdk-core'
|
@@ -33,6 +37,7 @@ module LogStash; module Outputs; class AmazonElasticSearch; class HttpClient;
|
|
33
37
|
@port = options[:port] || 9200
|
34
38
|
@protocol = options[:protocol] || 'http'
|
35
39
|
@region = options[:region] || 'us-east-1'
|
40
|
+
@service_name = options[:service_name] || 'es'
|
36
41
|
aws_access_key_id = options[:aws_access_key_id] || nil
|
37
42
|
aws_secret_access_key = options[:aws_secret_access_key] || nil
|
38
43
|
session_token = options[:session_token] || nil
|
@@ -105,7 +110,7 @@ module LogStash; module Outputs; class AmazonElasticSearch; class HttpClient;
|
|
105
110
|
key = Seahorse::Client::Http::Request.new(options={:endpoint=>url, :http_method => method.to_s.upcase,
|
106
111
|
:headers => params[:headers],:body => params[:body]})
|
107
112
|
|
108
|
-
aws_signer = Aws::Signers::V4.new(@credentials,
|
113
|
+
aws_signer = Aws::Signers::V4.new(@credentials, @service_name, @region )
|
109
114
|
|
110
115
|
|
111
116
|
signed_key = aws_signer.sign(key)
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
module LogStash; module Outputs; class AmazonElasticSearch; class HttpClient;
|
2
6
|
class Pool
|
3
7
|
class NoConnectionAvailableError < Error; end
|
@@ -28,7 +32,7 @@ module LogStash; module Outputs; class AmazonElasticSearch; class HttpClient;
|
|
28
32
|
end
|
29
33
|
end
|
30
34
|
|
31
|
-
attr_reader :logger, :adapter, :sniffing, :sniffer_delay, :resurrect_delay, :healthcheck_path, :sniffing_path, :bulk_path
|
35
|
+
attr_reader :logger, :adapter, :sniffing, :sniffer_delay, :resurrect_delay, :healthcheck_path, :sniffing_path, :bulk_path, :skip_healthcheck
|
32
36
|
|
33
37
|
ROOT_URI_PATH = '/'.freeze
|
34
38
|
|
@@ -40,6 +44,7 @@ module LogStash; module Outputs; class AmazonElasticSearch; class HttpClient;
|
|
40
44
|
:resurrect_delay => 5,
|
41
45
|
:sniffing => false,
|
42
46
|
:sniffer_delay => 10,
|
47
|
+
:skip_healthcheck => false,
|
43
48
|
}.freeze
|
44
49
|
|
45
50
|
def initialize(logger, adapter, initial_urls=[], options={})
|
@@ -57,6 +62,7 @@ module LogStash; module Outputs; class AmazonElasticSearch; class HttpClient;
|
|
57
62
|
@resurrect_delay = merged[:resurrect_delay]
|
58
63
|
@sniffing = merged[:sniffing]
|
59
64
|
@sniffer_delay = merged[:sniffer_delay]
|
65
|
+
@skip_healthcheck = merged[:skip_healthcheck]
|
60
66
|
end
|
61
67
|
|
62
68
|
# Used for all concurrent operations in this class
|
@@ -238,25 +244,30 @@ module LogStash; module Outputs; class AmazonElasticSearch; class HttpClient;
|
|
238
244
|
|
239
245
|
def healthcheck!
|
240
246
|
# Try to keep locking granularity low such that we don't affect IO...
|
241
|
-
@state_mutex.synchronize { @url_info.select {|url,meta| meta[:state] != :alive } }.each do |url,meta|
|
247
|
+
@state_mutex.synchronize { @url_info.select { |url, meta| meta[:state] != :alive } }.each do |url, meta|
|
242
248
|
begin
|
243
|
-
|
249
|
+
if skip_healthcheck == false
|
250
|
+
logger.info("Running health check to see if an Elasticsearch connection is working",
|
244
251
|
:healthcheck_url => url, :path => @healthcheck_path)
|
245
|
-
|
246
|
-
|
247
|
-
|
248
|
-
|
249
|
-
|
250
|
-
|
251
|
-
|
252
|
-
|
253
|
-
|
254
|
-
|
255
|
-
|
256
|
-
|
257
|
-
|
258
|
-
|
252
|
+
response = perform_request_to_url(url, :head, @healthcheck_path)
|
253
|
+
# If no exception was raised it must have succeeded!
|
254
|
+
logger.warn("Restored connection to ES instance", :url => url.sanitized.to_s)
|
255
|
+
# We reconnected to this node, check its ES version
|
256
|
+
es_version = get_es_version(url)
|
257
|
+
@state_mutex.synchronize do
|
258
|
+
meta[:version] = es_version
|
259
|
+
major = major_version(es_version)
|
260
|
+
if !@maximum_seen_major_version
|
261
|
+
@logger.info("ES Output version determined", :es_version => major)
|
262
|
+
set_new_major_version(major)
|
263
|
+
elsif major > @maximum_seen_major_version
|
264
|
+
@logger.warn("Detected a node with a higher major version than previously observed. This could be the result of an amazon_es cluster upgrade.", :previous_major => @maximum_seen_major_version, :new_major => major, :node_url => url)
|
265
|
+
set_new_major_version(major)
|
266
|
+
end
|
267
|
+
meta[:state] = :alive
|
259
268
|
end
|
269
|
+
elsif skip_healthcheck == true
|
270
|
+
set_new_major_version(7)
|
260
271
|
meta[:state] = :alive
|
261
272
|
end
|
262
273
|
rescue HostUnreachableError, BadResponseCodeError => e
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require "logstash/outputs/amazon_es"
|
2
6
|
require "cabin"
|
3
7
|
require "base64"
|
@@ -25,7 +29,7 @@ module LogStash; module Outputs; class AmazonElasticSearch;
|
|
25
29
|
# * `:user` - String. The user to use for authentication.
|
26
30
|
# * `:password` - String. The password to use for authentication.
|
27
31
|
# * `:timeout` - Float. A duration value, in seconds, after which a socket
|
28
|
-
# operation or request will be aborted if not yet
|
32
|
+
# operation or request will be aborted if not yet successful
|
29
33
|
# * `:client_settings` - a hash; see below for keys.
|
30
34
|
#
|
31
35
|
# The `client_settings` key is a has that can contain other settings:
|
@@ -272,6 +276,8 @@ module LogStash; module Outputs; class AmazonElasticSearch;
|
|
272
276
|
|
273
277
|
adapter_options[:region] = options[:region]
|
274
278
|
|
279
|
+
adapter_options[:service_name] = options[:service_name]
|
280
|
+
|
275
281
|
adapter_options[:port] = options[:port]
|
276
282
|
|
277
283
|
adapter_options[:protocol] = options[:protocol]
|
@@ -294,7 +300,8 @@ module LogStash; module Outputs; class AmazonElasticSearch;
|
|
294
300
|
:healthcheck_path => options[:healthcheck_path],
|
295
301
|
:resurrect_delay => options[:resurrect_delay],
|
296
302
|
:url_normalizer => self.method(:host_to_url),
|
297
|
-
:metric => options[:metric]
|
303
|
+
:metric => options[:metric],
|
304
|
+
:skip_healthcheck => options[:skip_healthcheck],
|
298
305
|
}
|
299
306
|
pool_options[:scheme] = self.scheme if self.scheme
|
300
307
|
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require 'cgi'
|
2
6
|
|
3
7
|
module LogStash; module Outputs; class AmazonElasticSearch;
|
@@ -100,9 +104,11 @@ module LogStash; module Outputs; class AmazonElasticSearch;
|
|
100
104
|
:protocol => params["protocol"],
|
101
105
|
:port => params["port"],
|
102
106
|
:region => params["region"],
|
107
|
+
:service_name => params["service_name"],
|
103
108
|
:aws_access_key_id => params["aws_access_key_id"],
|
104
109
|
:aws_secret_access_key => params["aws_secret_access_key"],
|
105
|
-
:max_bulk_bytes => params["max_bulk_bytes"]
|
110
|
+
:max_bulk_bytes => params["max_bulk_bytes"],
|
111
|
+
:skip_healthcheck => params["skip_healthcheck"])
|
106
112
|
)
|
107
113
|
end
|
108
114
|
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
module LogStash; module Outputs; class AmazonElasticSearch
|
2
6
|
class TemplateManager
|
3
7
|
# To be mixed into the amazon_es plugin base
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
# encoding: utf-8
|
2
6
|
require "logstash/namespace"
|
3
7
|
require "logstash/environment"
|
@@ -121,11 +125,14 @@ class LogStash::Outputs::AmazonElasticSearch < LogStash::Outputs::Base
|
|
121
125
|
# You can set the remote port as part of the host, or explicitly here as well
|
122
126
|
config :port, :validate => :number, :default => 443
|
123
127
|
|
124
|
-
# Sets the protocol thats used to connect to
|
128
|
+
# Sets the protocol thats used to connect to elasticsearch
|
125
129
|
config :protocol, :validate => :string, :default => "https"
|
126
130
|
|
127
131
|
#Signing specific details
|
128
132
|
config :region, :validate => :string, :default => "us-east-1"
|
133
|
+
|
134
|
+
#Service name, default is `es`
|
135
|
+
config :service_name, :validate => :string, :default => "es"
|
129
136
|
# Credential resolution logic works as follows:
|
130
137
|
#
|
131
138
|
# - User passed aws_access_key_id and aws_secret_access_key in aes configuration
|
@@ -251,6 +258,12 @@ class LogStash::Outputs::AmazonElasticSearch < LogStash::Outputs::Base
|
|
251
258
|
#Max bulk size in bytes
|
252
259
|
config :max_bulk_bytes, :validate => :number, :default => 20 * 1024 * 1024
|
253
260
|
|
261
|
+
#Option for user to skip Healthcheck API for a host when set to True
|
262
|
+
config :skip_healthcheck, :validate => :boolean, :default => false
|
263
|
+
|
264
|
+
#Allow user to skip installing template when set to True
|
265
|
+
config :skip_template_installation, :validate => :boolean, :default => false
|
266
|
+
|
254
267
|
def build_client
|
255
268
|
params["metric"] = metric
|
256
269
|
@client ||= ::LogStash::Outputs::AmazonElasticSearch::HttpClientBuilder.build(@logger, @hosts, params)
|
@@ -1,6 +1,10 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
Gem::Specification.new do |s|
|
2
6
|
s.name = 'logstash-output-amazon_es'
|
3
|
-
s.version = '7.0
|
7
|
+
s.version = '7.1.0'
|
4
8
|
s.licenses = ['Apache-2.0']
|
5
9
|
s.summary = "Logstash Output to Amazon Elasticsearch Service"
|
6
10
|
s.description = "Output events to Amazon Elasticsearch Service with V4 signing"
|
data/spec/es_spec_helper.rb
CHANGED
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require "logstash/devutils/rspec/spec_helper"
|
2
6
|
require "logstash/outputs/amazon_es"
|
3
7
|
require "logstash/outputs/amazon_es/http_client"
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require "logstash/devutils/rspec/spec_helper"
|
2
6
|
require "logstash/outputs/amazon_es/http_client"
|
3
7
|
require "json"
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require "logstash/devutils/rspec/spec_helper"
|
2
6
|
require "logstash/outputs/amazon_es/http_client"
|
3
7
|
require "java"
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require "logstash/devutils/rspec/spec_helper"
|
2
6
|
require "logstash/outputs/amazon_es/http_client"
|
3
7
|
require "java"
|
@@ -1,3 +1,7 @@
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
2
|
+
# See NOTICE for attribution details.
|
3
|
+
# Modifications Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
4
|
+
|
1
5
|
require_relative "../../../spec/es_spec_helper"
|
2
6
|
require "flores/random"
|
3
7
|
require "logstash/outputs/amazon_es"
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: logstash-output-amazon_es
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 7.0
|
4
|
+
version: 7.1.0
|
5
5
|
platform: java
|
6
6
|
authors:
|
7
7
|
- Amazon
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2022-01-26 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
requirement: !ruby/object:Gem::Requirement
|