logstash-input-elastic_serverless_forwarder 0.1.1-java → 0.1.3-java
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: '08bce725c7626a1e3cc794ab1679b2ca61de4f10fb58a4c6efb895af4a4c2a5f'
|
4
|
+
data.tar.gz: b997c8f5c4a7011f219ea2cf338d6813c2cd58350918e9f2fe1d58ece1c43e11
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9f69616a38002781b3f896f67c09cf780f708419bceef0789ef3e7f93f88bafafdea34539468bbb4de793c1231434282912b15dc6dad69da0aabdd63d29d0fd8
|
7
|
+
data.tar.gz: 3b306b08747a7f86611f785ba3b3d477707172b15eb096e2a7854a2bbc6937e2c376a65eb23c8f3e7758a3b6aae81bbb79dc31d73986246ce9264ced232aa698
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,10 @@
|
|
1
|
+
## 0.1.3
|
2
|
+
- Deprecates the `ssl` option in favor of `ssl_enabled` [#6](https://github.com/logstash-plugins/logstash-input-elastic_serverless_forwarder/pull/6)
|
3
|
+
- Bumps `logstash-input-http` gem version to `>= 3.7.2` (SSL-normalized)
|
4
|
+
|
5
|
+
## 0.1.2
|
6
|
+
- [DOC] Adds "Technical Preview" call-out to documentation [#4](https://github.com/logstash-plugins/logstash-input-elastic_serverless_forwarder/pull/4)
|
7
|
+
|
1
8
|
## 0.1.1
|
2
9
|
- Fixes an issue that prevents this prototype from being instantiated in an actual Logstash pipeline [#3](https://github.com/logstash-plugins/logstash-input-elastic_serverless_forwarder/pull/3)
|
3
10
|
|
data/docs/index.asciidoc
CHANGED
@@ -26,7 +26,7 @@ include::{include_path}/plugin_header.asciidoc[]
|
|
26
26
|
Using this input you can receive events from {esf-name} over http(s) connections to the configured <<plugins-{type}s-{plugin}-port>>.
|
27
27
|
|
28
28
|
[id="plugins-{type}s-{plugin}-ext-field"]
|
29
|
-
|
29
|
+
===== Minimum Configuration
|
30
30
|
[cols="3a,2a"]
|
31
31
|
|=======================================================================================================================
|
32
32
|
|SSL Enabled |SSL Disabled
|
@@ -51,20 +51,33 @@ input {
|
|
51
51
|
input {
|
52
52
|
elastic_serverless_forwarder {
|
53
53
|
port => 8080
|
54
|
-
|
54
|
+
ssl_enabled => false
|
55
55
|
}
|
56
56
|
}
|
57
57
|
----
|
58
58
|
|
59
59
|
|=======================================================================================================================
|
60
60
|
|
61
|
+
.Technical Preview
|
62
|
+
****
|
63
|
+
This {esf-name} input plugin is part of a _Technical Preview_, which means that both configuration options and implementation details are subject to change in minor releases without being preceded by deprecation warnings.
|
64
|
+
|
65
|
+
Before upgrading this plugin or Logstash itself, please pay special attention to this plugin's https://github.com/logstash-plugins/logstash-input-elastic_serverless_forwarder/blob/main/CHANGELOG.md[CHANGELOG.md] to avoid being caught by surprise.
|
66
|
+
****
|
67
|
+
|
68
|
+
|
61
69
|
[id="plugins-{type}s-{plugin}-enrichment"]
|
62
70
|
==== Enrichment
|
63
71
|
|
64
72
|
This input provides _minimal enrichment_ on events, and avoids including information about itself, the client from which it received the data, or about the original event as-decoded from the request.
|
65
|
-
If the decoded event has a valid ISO8601-encoded `@timestamp`, it will be used. Otherwise this required field will be populated with the current time.
|
66
73
|
|
67
74
|
NOTE: Senders are advised to use care with respect to fields that are {logstash-ref}/processing.html#reserved-fields[reserved in Logstash].
|
75
|
+
ESF sends the Logstash-required `@timestamp` field by default, but if this value is missing it will be populated with the current time.
|
76
|
+
|
77
|
+
|
78
|
+
////
|
79
|
+
// BEGIN: Elastic-internal implementation details
|
80
|
+
//
|
68
81
|
|
69
82
|
[id="plugins-{type}s-{plugin}-blocking"]
|
70
83
|
==== Blocking Behavior
|
@@ -76,6 +89,10 @@ A client that receives an HTTP request timeout is expected to retry the entire r
|
|
76
89
|
When this plugin is blocked, it will reject _new_ requests with HTTP `429 Too Many Requests`.
|
77
90
|
Clients that receive `429`-s are expected to wait a moment before retrying the request — exponential back-off is encouraged to ease flood recovery.
|
78
91
|
|
92
|
+
//
|
93
|
+
// END: Elastic-internal implementation details
|
94
|
+
////
|
95
|
+
|
79
96
|
[id="plugins-{type}s-{plugin}-security"]
|
80
97
|
==== Security
|
81
98
|
|
@@ -87,7 +104,7 @@ Additionally, you may wish to authenticate clients using SSL client authenticati
|
|
87
104
|
|
88
105
|
===== SSL Identity
|
89
106
|
|
90
|
-
In order to establish SSL with a client, this input plugin will need to present an SSL certificate that the client trusts and have access to the associated key.
|
107
|
+
In order to establish SSL with a client, this input plugin will need to present an SSL certificate that the client trusts, and have access to the associated key.
|
91
108
|
These are configurable with <<plugins-{type}s-{plugin}-ssl_certificate>>, <<plugins-{type}s-{plugin}-ssl_key>>, and optionally <<plugins-{type}s-{plugin}-ssl_key_passphrase>>.
|
92
109
|
|
93
110
|
===== SSL Client Authentication
|
@@ -98,6 +115,8 @@ It can be configured to either request or require client certificates using <<pl
|
|
98
115
|
which often also requires configuring it with a list of <<plugins-{type}s-{plugin}-ssl_certificate_authorities>> to trust.
|
99
116
|
When validating a certificate that is presented, <<plugins-{type}s-{plugin}-ssl_verification_mode>> controls how certificates are verified.
|
100
117
|
|
118
|
+
NOTE: ESF does not currently support _presenting_ client certificates, so requesting or requiring clients to present identity is only useful when combined with an SSL-terminating proxy.
|
119
|
+
|
101
120
|
===== SSL Advanced Configuration
|
102
121
|
|
103
122
|
This plugin exposes several advanced SSL configurations:
|
@@ -125,11 +144,12 @@ This plugin supports the following configuration options plus the <<plugins-{typ
|
|
125
144
|
| <<plugins-{type}s-{plugin}-auth_basic_password>> |<<password,password>>|No
|
126
145
|
| <<plugins-{type}s-{plugin}-host>> |<<string,string>>|No
|
127
146
|
| <<plugins-{type}s-{plugin}-port>> |<<number,number>>|No
|
128
|
-
| <<plugins-{type}s-{plugin}-ssl>> |<<boolean,boolean>>|
|
147
|
+
| <<plugins-{type}s-{plugin}-ssl>> |<<boolean,boolean>>|__Deprecated__
|
129
148
|
| <<plugins-{type}s-{plugin}-ssl_certificate>> |a valid filesystem path|No
|
130
149
|
| <<plugins-{type}s-{plugin}-ssl_certificate_authorities>> |<<array,array>>|No
|
131
150
|
| <<plugins-{type}s-{plugin}-ssl_client_authentication>> |<<string,string>>, one of `["none", "optional", "required"]`|No
|
132
151
|
| <<plugins-{type}s-{plugin}-ssl_cipher_suites>> |<<array,array>>|No
|
152
|
+
| <<plugins-{type}s-{plugin}-ssl_enabled>> |<<boolean,boolean>>|No
|
133
153
|
| <<plugins-{type}s-{plugin}-ssl_handshake_timeout>> |<<number,number>>|No
|
134
154
|
| <<plugins-{type}s-{plugin}-ssl_key>> |a valid filesystem path|No
|
135
155
|
| <<plugins-{type}s-{plugin}-ssl_key_passphrase>> |<<password,password>>|No
|
@@ -178,6 +198,7 @@ The TCP port to bind to
|
|
178
198
|
|
179
199
|
[id="plugins-{type}s-{plugin}-ssl"]
|
180
200
|
===== `ssl`
|
201
|
+
deprecated[0.1.3, Replaced by <<plugins-{type}s-{plugin}-ssl_enabled>>]
|
181
202
|
|
182
203
|
* Value type is <<boolean,boolean>>
|
183
204
|
* Default value is `true`
|
@@ -233,9 +254,20 @@ For example, the ChaCha20 family of ciphers is not supported in older versions.
|
|
233
254
|
* Default value is `"none"`
|
234
255
|
|
235
256
|
By default the server doesn't do any client authentication.
|
236
|
-
This means that connections from clients are
|
257
|
+
This means that connections from clients are _private_ when SSL is enabled, but that this input will allow SSL connections from _any_ client.
|
237
258
|
If you wish to configure this plugin to reject connections from untrusted hosts, you will need to configure this plugin to authenticate clients, and may also need to configure it with a list of `ssl_certificate_authorities`.
|
238
259
|
|
260
|
+
|
261
|
+
[id="plugins-{type}s-{plugin}-ssl_enabled"]
|
262
|
+
===== `ssl_enabled`
|
263
|
+
|
264
|
+
* Value type is <<boolean,boolean>>
|
265
|
+
* Default value is `true`
|
266
|
+
|
267
|
+
Events are, by default, sent over SSL, which requires configuring this plugin to present an identity certificate using <<plugins-{type}s-{plugin}-ssl_certificate>> and key using <<plugins-{type}s-{plugin}-ssl_key>>.
|
268
|
+
|
269
|
+
You can disable SSL with `+ssl_enabled => false+`.
|
270
|
+
|
239
271
|
[id="plugins-{type}s-{plugin}-ssl_handshake_timeout"]
|
240
272
|
===== `ssl_handshake_timeout`
|
241
273
|
|
@@ -3,12 +3,14 @@ require "logstash/inputs/base"
|
|
3
3
|
require "logstash/namespace"
|
4
4
|
|
5
5
|
require "logstash/plugin_mixins/plugin_factory_support"
|
6
|
+
require "logstash/plugin_mixins/normalize_config_support"
|
6
7
|
|
7
8
|
require 'logstash/inputs/http'
|
8
9
|
require 'logstash/codecs/json_lines'
|
9
10
|
|
10
11
|
class LogStash::Inputs::ElasticServerlessForwarder < LogStash::Inputs::Base
|
11
12
|
include LogStash::PluginMixins::PluginFactorySupport
|
13
|
+
include LogStash::PluginMixins::NormalizeConfigSupport
|
12
14
|
|
13
15
|
config_name "elastic_serverless_forwarder"
|
14
16
|
|
@@ -21,7 +23,8 @@ class LogStash::Inputs::ElasticServerlessForwarder < LogStash::Inputs::Base
|
|
21
23
|
config :auth_basic_password, :validate => :password
|
22
24
|
|
23
25
|
# ssl-config
|
24
|
-
config :ssl, :validate => :boolean, :default => true
|
26
|
+
config :ssl, :validate => :boolean, :default => true, :deprecated => "Use 'ssl_enabled' instead."
|
27
|
+
config :ssl_enabled, :validate => :boolean, :default => true
|
25
28
|
|
26
29
|
# ssl-identity
|
27
30
|
config :ssl_certificate, :validate => :path
|
@@ -38,20 +41,11 @@ class LogStash::Inputs::ElasticServerlessForwarder < LogStash::Inputs::Base
|
|
38
41
|
config :ssl_supported_protocols, :validate => :string, :list => true
|
39
42
|
config :ssl_handshake_timeout, :validate => :number, :default => 10_000
|
40
43
|
|
41
|
-
# we present the ES-like ssl_certificate_authorities, but our
|
42
|
-
# internal http input plugin uses ssl_verify_mode to describe
|
43
|
-
# the same behaviour.
|
44
|
-
SSL_CLIENT_AUTHENTICATION_TO_VERIFY_MODE_MAP = {
|
45
|
-
'none' => 'none',
|
46
|
-
'optional' => 'peer',
|
47
|
-
'required' => 'force_peer',
|
48
|
-
}.each_value(&:freeze).freeze # deep freeze
|
49
|
-
private_constant :SSL_CLIENT_AUTHENTICATION_TO_VERIFY_MODE_MAP
|
50
|
-
|
51
|
-
|
52
44
|
def initialize(*a)
|
53
45
|
super
|
54
46
|
|
47
|
+
normalize_ssl_configs!
|
48
|
+
|
55
49
|
if original_params.include?('codec')
|
56
50
|
fail LogStash::ConfigurationError, 'The `elastic_serverless_forwarder` input does not have an externally-configurable `codec`'
|
57
51
|
end
|
@@ -109,14 +103,14 @@ class LogStash::Inputs::ElasticServerlessForwarder < LogStash::Inputs::Base
|
|
109
103
|
if @auth_basic_username
|
110
104
|
http_options['user'] = @auth_basic_username
|
111
105
|
http_options['password'] = @auth_basic_password || fail(LogStash::ConfigurationError, '`auth_basic_password` is REQUIRED when `auth_basic_username` is provided')
|
112
|
-
logger.warn("HTTP Basic Auth over non-secured connection") if @
|
106
|
+
logger.warn("HTTP Basic Auth over non-secured connection") if @ssl_enabled == false
|
113
107
|
end
|
114
108
|
|
115
|
-
if @
|
109
|
+
if @ssl_enabled == false
|
116
110
|
ignored_ssl_settings = @original_params.keys.grep('ssl_')
|
117
|
-
logger.warn("Explicit SSL-related settings are ignored because `
|
111
|
+
logger.warn("Explicit SSL-related settings are ignored because `ssl_enabled => false`: #{ignored_ssl_settings.keys}") if ignored_ssl_settings.any?
|
118
112
|
else
|
119
|
-
http_options['
|
113
|
+
http_options['ssl_enabled'] = true
|
120
114
|
|
121
115
|
http_options['ssl_cipher_suites'] = @ssl_cipher_suites if @original_params.include?('ssl_cipher_suites')
|
122
116
|
http_options['ssl_supported_protocols'] = @ssl_supported_protocols if @original_params.include?('ssl_supported_protocols')
|
@@ -131,9 +125,10 @@ class LogStash::Inputs::ElasticServerlessForwarder < LogStash::Inputs::Base
|
|
131
125
|
end
|
132
126
|
|
133
127
|
def ssl_identity_options
|
128
|
+
ssl_enabled_config = @original_params.include?('ssl') ? 'ssl' : 'ssl_enabled'
|
134
129
|
identity_options = {
|
135
|
-
'ssl_certificate' => @ssl_certificate || fail(LogStash::ConfigurationError,
|
136
|
-
'ssl_key' => @ssl_key || fail(LogStash::ConfigurationError,
|
130
|
+
'ssl_certificate' => @ssl_certificate || fail(LogStash::ConfigurationError, "`ssl_certificate` is REQUIRED when `#{ssl_enabled_config} => true`"),
|
131
|
+
'ssl_key' => @ssl_key || fail(LogStash::ConfigurationError, "`ssl_key` is REQUIRED when `#{ssl_enabled_config} => true`")
|
137
132
|
}
|
138
133
|
identity_options['ssl_key_passphrase'] = @ssl_key_passphrase if @original_params.include?('ssl_key_passphrase')
|
139
134
|
|
@@ -142,7 +137,7 @@ class LogStash::Inputs::ElasticServerlessForwarder < LogStash::Inputs::Base
|
|
142
137
|
|
143
138
|
def ssl_trust_options
|
144
139
|
trust_options = {
|
145
|
-
'
|
140
|
+
'ssl_client_authentication' => @ssl_client_authentication
|
146
141
|
}
|
147
142
|
if @ssl_client_authentication == 'none'
|
148
143
|
logger.warn("Explicit `ssl_certificate_authorities` is ignored because `ssl_client_authentication => #{@ssl_client_authentication}`")
|
@@ -160,6 +155,12 @@ class LogStash::Inputs::ElasticServerlessForwarder < LogStash::Inputs::Base
|
|
160
155
|
}
|
161
156
|
end
|
162
157
|
|
158
|
+
def normalize_ssl_configs!
|
159
|
+
@ssl_enabled = normalize_config(:ssl_enabled) do |normalizer|
|
160
|
+
normalizer.with_deprecated_alias(:ssl)
|
161
|
+
end
|
162
|
+
end
|
163
|
+
|
163
164
|
class QueueWrapper
|
164
165
|
def initialize(wrapped_queue)
|
165
166
|
@wrapped_queue = wrapped_queue
|
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
Gem::Specification.new do |s|
|
4
4
|
s.name = 'logstash-input-elastic_serverless_forwarder'
|
5
|
-
s.version = '0.1.
|
5
|
+
s.version = '0.1.3'
|
6
6
|
s.licenses = ['Apache License (2.0)']
|
7
7
|
s.summary = "Receives events from Elastic Serverless Forwarder over HTTP or HTTPS"
|
8
8
|
s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"
|
@@ -23,8 +23,9 @@ Gem::Specification.new do |s|
|
|
23
23
|
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
|
24
24
|
s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.2'
|
25
25
|
s.add_runtime_dependency 'logstash-mixin-plugin_factory_support'
|
26
|
-
s.add_runtime_dependency 'logstash-input-http'
|
26
|
+
s.add_runtime_dependency 'logstash-input-http', '>= 3.7.2'
|
27
27
|
s.add_runtime_dependency 'logstash-codec-json_lines'
|
28
|
+
s.add_runtime_dependency 'logstash-mixin-normalize_config_support', '~>1.0'
|
28
29
|
|
29
30
|
s.add_development_dependency 'logstash-devutils'
|
30
31
|
|
@@ -28,7 +28,7 @@ describe LogStash::Inputs::ElasticServerlessForwarder do
|
|
28
28
|
let!(:queue) { Queue.new }
|
29
29
|
|
30
30
|
context 'baseline' do
|
31
|
-
let(:config) { super().merge('
|
31
|
+
let(:config) { super().merge('ssl_enabled' => false) }
|
32
32
|
let(:scheme) { 'http' }
|
33
33
|
|
34
34
|
it_behaves_like "an interruptible input plugin" do
|
@@ -45,7 +45,7 @@ describe LogStash::Inputs::ElasticServerlessForwarder do
|
|
45
45
|
end
|
46
46
|
|
47
47
|
context 'no user-defined codec' do
|
48
|
-
let(:config) { super().merge('
|
48
|
+
let(:config) { super().merge('ssl_enabled' => false) } # minimal config
|
49
49
|
|
50
50
|
##
|
51
51
|
# @codec ivar is required PENDING https://github.com/elastic/logstash/issues/14828
|
@@ -185,7 +185,7 @@ describe LogStash::Inputs::ElasticServerlessForwarder do
|
|
185
185
|
end
|
186
186
|
|
187
187
|
describe 'unsecured HTTP' do
|
188
|
-
let(:config) { super().merge('
|
188
|
+
let(:config) { super().merge('ssl_enabled' => false) }
|
189
189
|
let(:scheme) { 'http' }
|
190
190
|
|
191
191
|
include_examples 'successful request handling'
|
@@ -321,4 +321,23 @@ describe LogStash::Inputs::ElasticServerlessForwarder do
|
|
321
321
|
end
|
322
322
|
end
|
323
323
|
end
|
324
|
+
|
325
|
+
describe 'deprecated SSL options' do
|
326
|
+
let(:config) do
|
327
|
+
super().merge({
|
328
|
+
'ssl_certificate' => generated_certs_directory.join('server_from_root.crt').to_path,
|
329
|
+
'ssl_key' => generated_certs_directory.join('server_from_root.key.pkcs8').to_path,
|
330
|
+
})
|
331
|
+
end
|
332
|
+
|
333
|
+
[true, false].each do |enabled|
|
334
|
+
context "when `ssl => #{enabled}`" do
|
335
|
+
let(:config) { super().merge('ssl' => enabled) }
|
336
|
+
|
337
|
+
it "sets @ssl_enabled to `#{enabled}`" do
|
338
|
+
expect(esf_input.instance_variable_get(:@ssl_enabled)).to be enabled
|
339
|
+
end
|
340
|
+
end
|
341
|
+
end
|
342
|
+
end
|
324
343
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: logstash-input-elastic_serverless_forwarder
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.3
|
5
5
|
platform: java
|
6
6
|
authors:
|
7
7
|
- Elastic
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-
|
11
|
+
date: 2023-09-14 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
requirement: !ruby/object:Gem::Requirement
|
@@ -63,7 +63,7 @@ dependencies:
|
|
63
63
|
requirements:
|
64
64
|
- - ">="
|
65
65
|
- !ruby/object:Gem::Version
|
66
|
-
version:
|
66
|
+
version: 3.7.2
|
67
67
|
name: logstash-input-http
|
68
68
|
prerelease: false
|
69
69
|
type: :runtime
|
@@ -71,7 +71,7 @@ dependencies:
|
|
71
71
|
requirements:
|
72
72
|
- - ">="
|
73
73
|
- !ruby/object:Gem::Version
|
74
|
-
version:
|
74
|
+
version: 3.7.2
|
75
75
|
- !ruby/object:Gem::Dependency
|
76
76
|
requirement: !ruby/object:Gem::Requirement
|
77
77
|
requirements:
|
@@ -86,6 +86,20 @@ dependencies:
|
|
86
86
|
- - ">="
|
87
87
|
- !ruby/object:Gem::Version
|
88
88
|
version: '0'
|
89
|
+
- !ruby/object:Gem::Dependency
|
90
|
+
requirement: !ruby/object:Gem::Requirement
|
91
|
+
requirements:
|
92
|
+
- - "~>"
|
93
|
+
- !ruby/object:Gem::Version
|
94
|
+
version: '1.0'
|
95
|
+
name: logstash-mixin-normalize_config_support
|
96
|
+
prerelease: false
|
97
|
+
type: :runtime
|
98
|
+
version_requirements: !ruby/object:Gem::Requirement
|
99
|
+
requirements:
|
100
|
+
- - "~>"
|
101
|
+
- !ruby/object:Gem::Version
|
102
|
+
version: '1.0'
|
89
103
|
- !ruby/object:Gem::Dependency
|
90
104
|
requirement: !ruby/object:Gem::Requirement
|
91
105
|
requirements:
|
@@ -160,7 +174,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
160
174
|
- !ruby/object:Gem::Version
|
161
175
|
version: '0'
|
162
176
|
requirements: []
|
163
|
-
rubygems_version: 3.
|
177
|
+
rubygems_version: 3.2.33
|
164
178
|
signing_key:
|
165
179
|
specification_version: 4
|
166
180
|
summary: Receives events from Elastic Serverless Forwarder over HTTP or HTTPS
|