logstash-input-beats 6.1.4-java → 6.2.1-java

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 01075715620de7282a22c602c6a3b22c337f485ddea79a83090b889ef8d1027e
4
- data.tar.gz: e8b716cc2641203ef9ebc0b7335e2435e8b15c4b1a3a04ae49e28e567f75d1fc
3
+ metadata.gz: 53b66be9656d02212b1449645c83e128eb4fb32ceaa26ec76bc725a8cdd06a3a
4
+ data.tar.gz: 7ed11fe837aa9c6bf439ca0d7e16fdebec26e359f658410e18b6ec793a472ca5
5
5
  SHA512:
6
- metadata.gz: 2b693326cae6afba65c266e1dbcbb492db74f1742a3b6aed01b2b4eda21fd1e32c6f70e819d98f3dea34b9f6df632aae2a73f5b027c2d721b63330d6ffa20533
7
- data.tar.gz: 1fdf0f6111f9014e5c460b131bbbe3a29e6dc76eb59bd95e6de147bb6ba4e1d39749273a8f2d26a498a9c978b78732381513b638ad74e5185ba0f4d2ea2dff2f
6
+ metadata.gz: 6c75faeb9ae359e700f2feffeab892a200c9e011102ee4d21211813647749630776a9f4e88336dddc7873ae223623cc84852c28e06e3f6e3788a3aa1ba57e4a7
7
+ data.tar.gz: af9985a8fd2fed373f455d8980721f05a2473603a578593476af13f009a18ea1527bb7164db20ae0c364f98e49703fee95c7e21706729c2e0c6fd2dad411e4cb
data/CHANGELOG.md CHANGED
@@ -1,3 +1,19 @@
1
+ ## 6.2.1
2
+ - Fix: LS failing with `ssl_peer_metadata => true` [#431](https://github.com/logstash-plugins/logstash-input-beats/pull/431)
3
+ - [DOC] described `executor_threads` configuration parameter [#421](https://github.com/logstash-plugins/logstash-input-beats/pull/421)
4
+
5
+ ## 6.2.0
6
+ - ECS compatibility enablement: Adds alias to support upcoming ECS v8 with the existing ECS v1 implementation
7
+
8
+ ## 6.1.7
9
+ - [DOC] Remove limitations topic and link [#428](https://github.com/logstash-plugins/logstash-input-beats/pull/428)
10
+
11
+ ## 6.1.6
12
+ - [DOC] Applied more attributes to manage plugin name in doc content, and implemented conditional text processing. [#423](https://github.com/logstash-plugins/logstash-input-http/pull/423)
13
+
14
+ ## 6.1.5
15
+ - Changed jar dependencies to reflect newer versions [#425](https://github.com/logstash-plugins/logstash-input-beats/pull/425)
16
+
1
17
  ## 6.1.4
2
18
  - Fix: reduce error logging on connection resets [#424](https://github.com/logstash-plugins/logstash-input-beats/pull/424)
3
19
 
data/VERSION CHANGED
@@ -1 +1 @@
1
- 6.1.4
1
+ 6.2.1
data/docs/index.asciidoc CHANGED
@@ -2,6 +2,7 @@
2
2
  :type: input
3
3
  :default_codec: plain
4
4
  :plugin-uc: Beats
5
+ :plugin-singular: Beat
5
6
 
6
7
  ///////////////////////////////////////////
7
8
  START - GENERATED VARIABLES, DO NOT EDIT!
@@ -19,18 +20,21 @@ END - GENERATED VARIABLES, DO NOT EDIT!
19
20
  === {plugin-uc} input plugin
20
21
 
21
22
  NOTE: The `input-elastic_agent` plugin is the next generation of the
22
- `input-beats` plugin. They currently share a common codebase.
23
+ `input-beats` plugin.
24
+ They currently share code and a https://github.com/logstash-plugins/logstash-input-beats[common codebase].
23
25
 
24
26
  include::{include_path}/plugin_header.asciidoc[]
25
27
 
26
28
  ==== Description
27
29
 
28
30
  This input plugin enables Logstash to receive events from the
29
- https://www.elastic.co/products/beats[Elastic Beats] framework.
31
+ {plugin-uc} framework.
30
32
 
31
33
  The following example shows how to configure Logstash to listen on port
32
34
  5044 for incoming {plugin-uc} connections and to index into Elasticsearch.
33
35
 
36
+ //Example for Beats
37
+ ifeval::["{plugin}"=="beats"]
34
38
  ["source","sh",subs="attributes"]
35
39
  -----
36
40
 
@@ -48,9 +52,8 @@ output {
48
52
  }
49
53
  -----
50
54
  <1> `%{[@metadata][beat]}` sets the first part of the index name to the value
51
- of the `beat` metadata field and `%{[@metadata][version]}` sets the second part to
52
- the {plugin-uc} version. For example:
53
- metricbeat-7.4.0.
55
+ of the metadata field and `%{[@metadata][version]}` sets the second part to
56
+ the {plugin-singular} version. For example: metricbeat-6.1.6.
54
57
 
55
58
  Events indexed into Elasticsearch with the Logstash configuration shown here
56
59
  will be similar to events directly indexed by {plugin-uc} into Elasticsearch.
@@ -59,14 +62,47 @@ NOTE: If ILM is not being used, set `index` to
59
62
  `%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}` instead so
60
63
  Logstash creates an index per day, based on the `@timestamp` value of the events
61
64
  coming from {plugin-uc}.
65
+ endif::[]
62
66
 
63
- IMPORTANT: If you are shipping events that span multiple lines, you need to use
67
+ //Example for Elastic Agent
68
+ ifeval::["{plugin}"!="beats"]
69
+ ["source","sh",subs="attributes"]
70
+ -----
71
+
72
+ input {
73
+ {plugin} {
74
+ port => 5044
75
+ }
76
+ }
77
+
78
+ output {
79
+ elasticsearch {
80
+ hosts => ["http://localhost:9200"]
81
+ data_stream => "true"
82
+ }
83
+ }
84
+ -----
85
+
86
+ Events indexed into Elasticsearch with the Logstash configuration shown here
87
+ will be similar to events directly indexed by {plugin-uc} into Elasticsearch.
88
+ endif::[]
89
+
90
+
91
+ //Content for Beats
92
+ ifeval::["{plugin}"=="beats"]
93
+ [id="plugins-{type}s-{plugin}-multiline"]
94
+ ===== Multi-line events
95
+
96
+ If you are shipping events that span multiple lines, you need to use
64
97
  the {filebeat-ref}/multiline-examples.html[configuration options available in
65
98
  Filebeat] to handle multiline events before sending the event data to Logstash.
66
99
  You cannot use the {logstash-ref}/plugins-codecs-multiline.html[Multiline codec
67
100
  plugin] to handle multiline events. Doing so will result in the failure to start
68
101
  Logstash.
102
+ endif::[]
69
103
 
104
+ //Content for Beats
105
+ ifeval::["{plugin}"=="beats"]
70
106
  [id="plugins-{type}s-{plugin}-versioned-indexes"]
71
107
  ==== Versioned indices
72
108
 
@@ -89,6 +125,7 @@ Logstash `@timestamp` field.
89
125
 
90
126
  This configuration results in daily index names like
91
127
  +filebeat-{logstash_version}-{localdate}+.
128
+ endif::[]
92
129
 
93
130
 
94
131
  [id="plugins-{type}s-{plugin}-ecs_metadata"]
@@ -104,18 +141,18 @@ output.
104
141
 
105
142
  [cols="<l,<l,e,<e"]
106
143
  |=======================================================================
107
- |ECS disabled |ECS v1 |Availability |Description
144
+ |ECS `disabled` |ECS `v1`, `v8` |Availability |Description
108
145
 
109
- |[host] |[@metadata][input][beats][host][name] |Always |Name or address of the beat host
110
- |[@metadata][ip_address] |[@metadata][input][beats][host][ip] |Always |IP address of the Beats client
111
- |[@metadata][tls_peer][status] | [@metadata][tls_peer][status] | When SSL related fields are populated | Contains "verified"/"unverified" labels in `disabled`, `true`/`false` in `v1`
146
+ |[host] |[@metadata][input][beats][host][name] |Always |Name or address of the {plugin-singular} host
147
+ |[@metadata][ip_address] |[@metadata][input][beats][host][ip] |Always |IP address of the {plugin-uc} client
148
+ |[@metadata][tls_peer][status] | [@metadata][tls_peer][status] | When SSL related fields are populated | Contains "verified"/"unverified" labels in `disabled`, `true`/`false` in `v1`/`v8`
112
149
  |[@metadata][tls_peer][protocol] | [@metadata][input][beats][tls][version_protocol] | When SSL status is "verified" | Contains the TLS version used (e.g. `TLSv1.2`)
113
150
  |[@metadata][tls_peer][subject] | [@metadata][input][beats][tls][client][subject] | When SSL status is "verified" | Contains the identity name of the remote end (e.g. `CN=artifacts-no-kpi.elastic.co`)
114
151
  |[@metadata][tls_peer][cipher_suite] | [@metadata][input][beats][tls][cipher] | When SSL status is "verified" | Contains the name of cipher suite used (e.g. `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256`)
115
152
  |=======================================================================
116
153
 
117
154
  [id="plugins-{type}s-{plugin}-options"]
118
- ==== {plugin-uc} Input Configuration Options
155
+ ==== {plugin-uc} input configuration options
119
156
 
120
157
  This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> described later.
121
158
 
@@ -126,6 +163,7 @@ This plugin supports the following configuration options plus the <<plugins-{typ
126
163
  | <<plugins-{type}s-{plugin}-cipher_suites>> |<<array,array>>|No
127
164
  | <<plugins-{type}s-{plugin}-client_inactivity_timeout>> |<<number,number>>|No
128
165
  | <<plugins-{type}s-{plugin}-ecs_compatibility>> | <<string,string>>|No
166
+ | <<plugins-{type}s-{plugin}-executor_threads>> |<<number,number>>|No
129
167
  | <<plugins-{type}s-{plugin}-host>> |<<string,string>>|No
130
168
  | <<plugins-{type}s-{plugin}-include_codec_tag>> |<<boolean,boolean>>|No
131
169
  | <<plugins-{type}s-{plugin}-port>> |<<number,number>>|Yes
@@ -154,7 +192,7 @@ input plugins.
154
192
  * Value type is <<boolean,boolean>>
155
193
  * Default value is `false`
156
194
 
157
- Flag to determine whether to add `host` field to event using the value supplied by the beat in the `hostname` field.
195
+ Flag to determine whether to add `host` field to event using the value supplied by the {plugin-singular} in the `hostname` field.
158
196
 
159
197
 
160
198
  [id="plugins-{type}s-{plugin}-cipher_suites"]
@@ -179,13 +217,32 @@ Close Idle clients after X seconds of inactivity.
179
217
  * Value type is <<string,string>>
180
218
  * Supported values are:
181
219
  ** `disabled`: unstructured connection metadata added at root level
182
- ** `v1`: structured connection metadata added under ECS compliant namespaces
220
+ ** `v1`: structured connection metadata added under ECS v1 compliant namespaces
221
+ ** `v8`: structured connection metadata added under ECS v8 compliant namespaces
183
222
  * Default value depends on which version of Logstash is running:
184
223
  ** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
185
224
  ** Otherwise, the default value is `disabled`.
186
225
 
187
226
  Refer to <<plugins-{type}s-{plugin}-ecs_metadata,ECS mapping>> for detailed information.
188
227
 
228
+ [id="plugins-{type}s-{plugin}-executor_threads"]
229
+ ===== `executor_threads`
230
+
231
+ * Value type is <<number,number>>
232
+ * Default value is 1 executor thread per CPU core
233
+
234
+ The number of threads to be used to process incoming beats requests.
235
+ By default Beats input will create a number of threads equals to 2*CPU cores to handle incoming connections,
236
+ reading from the established sockets and execute most of the tasks related to network connection managements,
237
+ except the parsing of Lumberjack protocol that's offloaded to a dedicated thread pool.
238
+
239
+ Generally you don't need to touch this setting.
240
+ In case you are sending very large events and observing "OutOfDirectMemory" exceptions,
241
+ you may want to reduce this number to half or 1/4 of the CPU cores.
242
+ This will reduce the number of threads decompressing batches of data into direct memory.
243
+ However, this will only be a mitigating tweak, as the proper solution may require resizing your Logstash deployment,
244
+ either by increasing number of Logstash nodes or increasing the JVM's Direct Memory.
245
+
189
246
  [id="plugins-{type}s-{plugin}-host"]
190
247
  ===== `host`
191
248
 
@@ -317,3 +374,4 @@ The minimum TLS version allowed for the encrypted connections. The value must be
317
374
  include::{include_path}/{type}.asciidoc[]
318
375
 
319
376
  :default_codec!:
377
+
@@ -15,6 +15,8 @@ module LogStash module Inputs class Beats
15
15
 
16
16
  attr_reader :logger, :input, :connections_list
17
17
 
18
+ attr_reader :event_factory
19
+
18
20
  def initialize(queue, input)
19
21
  @connections_list = ThreadSafe::Hash.new
20
22
  @queue = queue
@@ -25,6 +27,7 @@ module LogStash module Inputs class Beats
25
27
 
26
28
  @nocodec_transformer = RawEventTransform.new(@input)
27
29
  @codec_transformer = DecodedEventTransform.new(@input)
30
+ @event_factory = input.event_factory
28
31
  end
29
32
 
30
33
  def onNewMessage(ctx, message)
@@ -39,7 +42,7 @@ module LogStash module Inputs class Beats
39
42
  extract_tls_peer(hash, ctx)
40
43
 
41
44
  if target_field.nil?
42
- event = LogStash::Event.new(hash)
45
+ event = event_factory.new_event(hash)
43
46
  @nocodec_transformer.transform(event)
44
47
  @queue << event
45
48
  else
@@ -129,7 +132,7 @@ module LogStash module Inputs class Beats
129
132
  tls_session = ctx.channel().pipeline().get("ssl-handler").engine().getSession()
130
133
  tls_verified = true
131
134
 
132
- if not @input.client_authentication_required?
135
+ unless @input.client_authentication_required?
133
136
  # throws SSLPeerUnverifiedException if unverified
134
137
  begin
135
138
  tls_session.getPeerCertificates()
@@ -141,18 +144,16 @@ module LogStash module Inputs class Beats
141
144
  end
142
145
  end
143
146
 
147
+ meta_data = hash['@metadata'] ||= {}
148
+
144
149
  if tls_verified
145
- set_nested(hash, @field_tls_protocol_version, tls_session.getProtocol())
146
- set_nested(hash, @field_tls_peer_subject, tls_session.getPeerPrincipal().getName())
147
- set_nested(hash, @field_tls_cipher, tls_session.getCipherSuite())
150
+ meta_data['tls_peer'] = { :status => "verified" }
148
151
 
149
- hash['@metadata']['tls_peer'] = {
150
- :status => "verified"
151
- }
152
+ set_nested(hash, input.field_tls_protocol_version, tls_session.getProtocol())
153
+ set_nested(hash, input.field_tls_peer_subject, tls_session.getPeerPrincipal().getName())
154
+ set_nested(hash, input.field_tls_cipher, tls_session.getCipherSuite())
152
155
  else
153
- hash['@metadata']['tls_peer'] = {
154
- :status => "unverified"
155
- }
156
+ meta_data['tls_peer'] = { :status => "unverified" }
156
157
  end
157
158
  end
158
159
  end
@@ -163,9 +164,6 @@ module LogStash module Inputs class Beats
163
164
  field_ref = Java::OrgLogstash::FieldReference.from(field_name)
164
165
  # create @metadata sub-hash if needed
165
166
  if field_ref.type == Java::OrgLogstash::FieldReference::META_CHILD
166
- unless hash.key?("@metadata")
167
- hash["@metadata"] = {}
168
- end
169
167
  nesting_hash = hash["@metadata"]
170
168
  else
171
169
  nesting_hash = hash
@@ -6,6 +6,7 @@ require "logstash/codecs/multiline"
6
6
  require "logstash/util"
7
7
  require "logstash-input-beats_jars"
8
8
  require "logstash/plugin_mixins/ecs_compatibility_support"
9
+ require 'logstash/plugin_mixins/event_support/event_factory_adapter'
9
10
  require_relative "beats/patch"
10
11
 
11
12
  # This input plugin enables Logstash to receive events from the
@@ -51,7 +52,9 @@ class LogStash::Inputs::Beats < LogStash::Inputs::Base
51
52
  require "logstash/inputs/beats/tls"
52
53
 
53
54
  # adds ecs_compatibility config which could be :disabled or :v1
54
- include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled,:v1)
55
+ include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled,:v1, :v8 => :v1)
56
+
57
+ include LogStash::PluginMixins::EventSupport::EventFactoryAdapter
55
58
 
56
59
  config_name "beats"
57
60
 
@@ -126,6 +129,7 @@ class LogStash::Inputs::Beats < LogStash::Inputs::Base
126
129
  config :executor_threads, :validate => :number, :default => LogStash::Config::CpuCoreStrategy.maximum
127
130
 
128
131
  attr_reader :field_hostname, :field_hostip
132
+ attr_reader :field_tls_protocol_version, :field_tls_peer_subject, :field_tls_cipher
129
133
 
130
134
  def register
131
135
  # For Logstash 2.4 we need to make sure that the logger is correctly set for the
@@ -164,10 +168,10 @@ class LogStash::Inputs::Beats < LogStash::Inputs::Base
164
168
 
165
169
  # define ecs name mapping
166
170
  @field_hostname = ecs_select[disabled: "host", v1: "[@metadata][input][beats][host][name]"]
167
- @field_hostip = ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][beats][host][ip]"]
168
- @field_tls_protocol_version = ecs_select[disabled: "[@metadata][tls_peer][protocol]", v1: "[@metadata][input][beats][tls][version_protocol]"]
169
- @field_tls_peer_subject = ecs_select[disabled: "[@metadata][tls_peer][subject]", v1: "[@metadata][input][beats][tls][client][subject]"]
170
- @field_tls_cipher = ecs_select[disabled: "[@metadata][tls_peer][cipher_suite]", v1: "[@metadata][input][beats][tls][cipher]"]
171
+ @field_hostip = ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][beats][host][ip]"]
172
+ @field_tls_protocol_version = ecs_select[disabled: "[@metadata][tls_peer][protocol]", v1: "[@metadata][input][beats][tls][version_protocol]"]
173
+ @field_tls_peer_subject = ecs_select[disabled: "[@metadata][tls_peer][subject]", v1: "[@metadata][input][beats][tls][client][subject]"]
174
+ @field_tls_cipher = ecs_select[disabled: "[@metadata][tls_peer][cipher_suite]", v1: "[@metadata][input][beats][tls][cipher]"]
171
175
 
172
176
  @logger.info("Starting input listener", :address => "#{@host}:#{@port}")
173
177
 
@@ -1,11 +1,11 @@
1
1
  # AUTOGENERATED BY THE GRADLE SCRIPT. DO NOT EDIT.
2
2
 
3
3
  require 'jar_dependencies'
4
- require_jar('io.netty', 'netty-all', '4.1.49.Final')
4
+ require_jar('io.netty', 'netty-all', '4.1.65.Final')
5
5
  require_jar('org.javassist', 'javassist', '3.24.0-GA')
6
6
  require_jar('com.fasterxml.jackson.core', 'jackson-core', '2.9.10')
7
7
  require_jar('com.fasterxml.jackson.core', 'jackson-annotations', '2.9.10')
8
8
  require_jar('com.fasterxml.jackson.core', 'jackson-databind', '2.9.10.8')
9
9
  require_jar('com.fasterxml.jackson.module', 'jackson-module-afterburner', '2.9.10')
10
10
  require_jar('org.apache.logging.log4j', 'log4j-api', '2.11.1')
11
- require_jar('org.logstash.beats', 'logstash-input-beats', '6.1.4')
11
+ require_jar('org.logstash.beats', 'logstash-input-beats', '6.2.1')
@@ -27,7 +27,8 @@ Gem::Specification.new do |s|
27
27
  s.add_runtime_dependency "thread_safe", "~> 0.3.5"
28
28
  s.add_runtime_dependency "logstash-codec-multiline", ">= 2.0.5"
29
29
  s.add_runtime_dependency 'jar-dependencies', '~> 0.3', '>= 0.3.4'
30
- s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.1'
30
+ s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.3'
31
+ s.add_runtime_dependency 'logstash-mixin-event_support', '~>1.0'
31
32
 
32
33
  s.add_development_dependency "flores", "~>0.0.6"
33
34
  s.add_development_dependency "rspec"
@@ -31,6 +31,7 @@ describe LogStash::Inputs::Beats::DecodedEventTransform do
31
31
 
32
32
  include_examples "Common Event Transformation", :disabled, "host"
33
33
  include_examples "Common Event Transformation", :v1, "[@metadata][input][beats][host][name]"
34
+ include_examples "Common Event Transformation", :v8, "[@metadata][input][beats][host][name]"
34
35
 
35
36
  it "tags the event" do
36
37
  expect(subject.get("tags")).to include("beats_input_codec_plain_applied")
@@ -9,4 +9,5 @@ describe LogStash::Inputs::Beats::EventTransformCommon do
9
9
 
10
10
  include_examples "Common Event Transformation", :disabled, "host"
11
11
  include_examples "Common Event Transformation", :v1, "[@metadata][input][beats][host][name]"
12
+ include_examples "Common Event Transformation", :v8, "[@metadata][input][beats][host][name]"
12
13
  end
@@ -211,6 +211,7 @@ describe LogStash::Inputs::Beats::MessageListener do
211
211
 
212
212
  it_behaves_like "when the message is from any libbeat", :disabled, "[@metadata][ip_address]"
213
213
  it_behaves_like "when the message is from any libbeat", :v1, "[@metadata][input][beats][host][ip]"
214
+ it_behaves_like "when the message is from any libbeat", :v8, "[@metadata][input][beats][host][ip]"
214
215
  end
215
216
 
216
217
  context "onException" do
@@ -20,6 +20,7 @@ describe LogStash::Inputs::Beats::RawEventTransform do
20
20
 
21
21
  include_examples "Common Event Transformation", :disabled, "host"
22
22
  include_examples "Common Event Transformation", :v1, "[@metadata][input][beats][host][name]"
23
+ include_examples "Common Event Transformation", :v8, "[@metadata][input][beats][host][name]"
23
24
 
24
25
  it "tags the event" do
25
26
  expect(subject.get("tags")).to include("beats_input_raw_event")
@@ -12,26 +12,28 @@ describe LogStash::Inputs::Beats do
12
12
  let(:connection) { double("connection") }
13
13
  let(:certificate) { BeatsInputTest.certificate }
14
14
  let(:port) { BeatsInputTest.random_port }
15
+ let(:client_inactivity_timeout) { 400 }
16
+ let(:threads) { 1 + rand(9) }
15
17
  let(:queue) { Queue.new }
16
18
  let(:config) do
17
19
  {
18
- "port" => 0,
20
+ "port" => port,
19
21
  "ssl_certificate" => certificate.ssl_cert,
20
22
  "ssl_key" => certificate.ssl_key,
23
+ "client_inactivity_timeout" => client_inactivity_timeout,
24
+ "executor_threads" => threads,
21
25
  "type" => "example",
22
26
  "tags" => "beats"
23
27
  }
24
28
  end
25
29
 
30
+ subject(:plugin) { LogStash::Inputs::Beats.new(config) }
31
+
26
32
  context "#register" do
27
33
  context "host related configuration" do
28
- let(:config) { super().merge("host" => host, "port" => port, "client_inactivity_timeout" => client_inactivity_timeout, "executor_threads" => threads) }
34
+ let(:config) { super().merge("host" => host, "port" => port) }
29
35
  let(:host) { "192.168.1.20" }
30
- let(:port) { 9000 }
31
- let(:client_inactivity_timeout) { 400 }
32
- let(:threads) { 10 }
33
-
34
- subject(:plugin) { LogStash::Inputs::Beats.new(config) }
36
+ let(:port) { 9001 }
35
37
 
36
38
  it "sends the required options to the server" do
37
39
  expect(org.logstash.beats.Server).to receive(:new).with(host, port, client_inactivity_timeout, threads)
@@ -158,9 +160,80 @@ describe LogStash::Inputs::Beats do
158
160
 
159
161
  it "raise a ConfigurationError when multiline codec is set" do
160
162
  plugin = LogStash::Inputs::Beats.new(config)
161
- expect {plugin.register}.to raise_error(LogStash::ConfigurationError, "Multiline codec with beats input is not supported. Please refer to the beats documentation for how to best manage multiline data. See https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html")
163
+ expect { plugin.register }.to raise_error(LogStash::ConfigurationError, "Multiline codec with beats input is not supported. Please refer to the beats documentation for how to best manage multiline data. See https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html")
164
+ end
165
+ end
166
+ end
167
+
168
+ context "tls meta-data" do
169
+ let(:config) { super().merge("host" => host, "ssl_peer_metadata" => true, "ssl_certificate_authorities" => [ certificate.ssl_cert ]) }
170
+ let(:host) { "192.168.1.20" }
171
+ let(:port) { 9002 }
172
+
173
+ let(:queue) { Queue.new }
174
+ let(:event) { LogStash::Event.new }
175
+
176
+ subject(:plugin) { LogStash::Inputs::Beats.new(config) }
177
+
178
+ before do
179
+ @server = org.logstash.beats.Server.new(host, port, client_inactivity_timeout, threads)
180
+ expect( org.logstash.beats.Server ).to receive(:new).with(host, port, client_inactivity_timeout, threads).and_return @server
181
+ expect( @server ).to receive(:listen)
182
+
183
+ subject.register
184
+ subject.run(queue) # listen does nothing
185
+ @message_listener = @server.getMessageListener
186
+
187
+ allow( ssl_engine = double('ssl_engine') ).to receive(:getSession).and_return ssl_session
188
+ allow( ssl_handler = double('ssl-handler') ).to receive(:engine).and_return ssl_engine
189
+ allow( pipeline = double('pipeline') ).to receive(:get).and_return ssl_handler
190
+ allow( @channel = double('channel') ).to receive(:pipeline).and_return pipeline
191
+ end
192
+
193
+ let(:ctx) do
194
+ Java::io.netty.channel.ChannelHandlerContext.impl do |method, *args|
195
+ fail("unexpected #{method}( #{args} )") unless method.eql?(:channel)
196
+ @channel
162
197
  end
163
198
  end
199
+
200
+ let(:ssl_session) do
201
+ Java::javax.net.ssl.SSLSession.impl do |method, *args|
202
+ case method
203
+ when :getPeerCertificates
204
+ [].to_java(java.security.cert.Certificate)
205
+ when :getProtocol
206
+ 'TLS-Mock'
207
+ when :getCipherSuite
208
+ 'SSL_NULL_WITH_TEST_SPEC'
209
+ when :getPeerPrincipal
210
+ javax.security.auth.x500.X500Principal.new('CN=TEST, OU=RSpec, O=Logstash, C=NL', {})
211
+ else
212
+ fail("unexpected #{method}( #{args} )")
213
+ end
214
+ end
215
+ end
216
+
217
+ let(:ssl_session_peer_principal) do
218
+ javax.security.auth.x500.X500Principal
219
+ end
220
+
221
+ let(:message) do
222
+ org.logstash.beats.Message.new(0, java.util.HashMap.new('foo' => 'bar'))
223
+ end
224
+
225
+ it 'sets tls fields' do
226
+ @message_listener.onNewMessage(ctx, message)
227
+
228
+ expect( queue.size ).to be 1
229
+ expect( event = queue.pop ).to be_a LogStash::Event
230
+
231
+ expect( event.get('[@metadata][tls_peer][status]') ).to eql 'verified'
232
+
233
+ expect( event.get('[@metadata][tls_peer][protocol]') ).to eql 'TLS-Mock'
234
+ expect( event.get('[@metadata][tls_peer][cipher_suite]') ).to eql 'SSL_NULL_WITH_TEST_SPEC'
235
+ expect( event.get('[@metadata][tls_peer][subject]') ).to eql 'CN=TEST,OU=RSpec,O=Logstash,C=NL'
236
+ end
164
237
  end
165
238
 
166
239
  context "when interrupting the plugin" do
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-input-beats
3
3
  version: !ruby/object:Gem::Version
4
- version: 6.1.4
4
+ version: 6.2.1
5
5
  platform: java
6
6
  authors:
7
7
  - Elastic
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-06-17 00:00:00.000000000 Z
11
+ date: 2021-10-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  requirement: !ruby/object:Gem::Requirement
@@ -111,7 +111,7 @@ dependencies:
111
111
  requirements:
112
112
  - - "~>"
113
113
  - !ruby/object:Gem::Version
114
- version: '1.1'
114
+ version: '1.3'
115
115
  name: logstash-mixin-ecs_compatibility_support
116
116
  prerelease: false
117
117
  type: :runtime
@@ -119,7 +119,21 @@ dependencies:
119
119
  requirements:
120
120
  - - "~>"
121
121
  - !ruby/object:Gem::Version
122
- version: '1.1'
122
+ version: '1.3'
123
+ - !ruby/object:Gem::Dependency
124
+ requirement: !ruby/object:Gem::Requirement
125
+ requirements:
126
+ - - "~>"
127
+ - !ruby/object:Gem::Version
128
+ version: '1.0'
129
+ name: logstash-mixin-event_support
130
+ prerelease: false
131
+ type: :runtime
132
+ version_requirements: !ruby/object:Gem::Requirement
133
+ requirements:
134
+ - - "~>"
135
+ - !ruby/object:Gem::Version
136
+ version: '1.0'
123
137
  - !ruby/object:Gem::Dependency
124
138
  requirement: !ruby/object:Gem::Requirement
125
139
  requirements:
@@ -282,10 +296,10 @@ files:
282
296
  - vendor/jar-dependencies/com/fasterxml/jackson/core/jackson-core/2.9.10/jackson-core-2.9.10.jar
283
297
  - vendor/jar-dependencies/com/fasterxml/jackson/core/jackson-databind/2.9.10.8/jackson-databind-2.9.10.8.jar
284
298
  - vendor/jar-dependencies/com/fasterxml/jackson/module/jackson-module-afterburner/2.9.10/jackson-module-afterburner-2.9.10.jar
285
- - vendor/jar-dependencies/io/netty/netty-all/4.1.49.Final/netty-all-4.1.49.Final.jar
299
+ - vendor/jar-dependencies/io/netty/netty-all/4.1.65.Final/netty-all-4.1.65.Final.jar
286
300
  - vendor/jar-dependencies/org/apache/logging/log4j/log4j-api/2.11.1/log4j-api-2.11.1.jar
287
301
  - vendor/jar-dependencies/org/javassist/javassist/3.24.0-GA/javassist-3.24.0-GA.jar
288
- - vendor/jar-dependencies/org/logstash/beats/logstash-input-beats/6.1.4/logstash-input-beats-6.1.4.jar
302
+ - vendor/jar-dependencies/org/logstash/beats/logstash-input-beats/6.2.1/logstash-input-beats-6.2.1.jar
289
303
  homepage: http://www.elastic.co/guide/en/logstash/current/index.html
290
304
  licenses:
291
305
  - Apache License (2.0)
@@ -308,8 +322,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
308
322
  - !ruby/object:Gem::Version
309
323
  version: '0'
310
324
  requirements: []
311
- rubyforge_project:
312
- rubygems_version: 2.6.13
325
+ rubygems_version: 3.1.6
313
326
  signing_key:
314
327
  specification_version: 4
315
328
  summary: Receives events from the Elastic Beats framework