logstash-input-beats 6.1.5-java → 6.2.2-java

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: dfd025acbb07379ea0d9fa66d381cd0aa88e92b7e3f0d162efff407cd27763b5
4
- data.tar.gz: f4071900f9e42da024156f3215699ecda37027ec137c6f5bb01d47daf9153562
3
+ metadata.gz: 187ef06d14ac3b4ccab2dffde80f1f54e0dc0c08ddcf5383022d1181200626c1
4
+ data.tar.gz: e4d0d2f120146186225a3e239e589691b7a051ca5b67b476e34c1b4a997950eb
5
5
  SHA512:
6
- metadata.gz: 902a418a13e1cf18e8c8125ed04a4f77f945786dcb9af0cfabb581ea5afe96199614bd3f240eba758897e677adc34948866394673550c061f19cf5b6bac5b4be
7
- data.tar.gz: aa57f6663812fece5e775878d10904d003654c25cd4931fcf02cb692ab6fcb235d9cbbf3925450df8078abc71c613b1b95d8d251d65f9e2d3fd4011ac4c3a161
6
+ metadata.gz: fb532b95ec66cf1fa7a7df02f2b7f9286298974deef2ac75633a6bdb12abd7a1ef7c51f72b3b90e64f822e96bac20d5280ade15fabcb1e3734e8e9e4218d9623
7
+ data.tar.gz: 7d1bdd0693e3b00e6401efbfee45728566eef845630dfbb69ba0f5224d254747a159890b186187675e6bb29cc62b74c5e99b20a9439cd06806a4b067c3842504
data/CHANGELOG.md CHANGED
@@ -1,5 +1,22 @@
1
+ ## 6.2.2
2
+ - Fix: update to Gradle 7 [#432](https://github.com/logstash-plugins/logstash-input-beats/pull/432)
3
+ - [DOC] Edit documentation for `executor_threads` [#435](https://github.com/logstash-plugins/logstash-input-beats/pull/435)
4
+
5
+ ## 6.2.1
6
+ - Fix: LS failing with `ssl_peer_metadata => true` [#431](https://github.com/logstash-plugins/logstash-input-beats/pull/431)
7
+ - [DOC] described `executor_threads` configuration parameter [#421](https://github.com/logstash-plugins/logstash-input-beats/pull/421)
8
+
9
+ ## 6.2.0
10
+ - ECS compatibility enablement: Adds alias to support upcoming ECS v8 with the existing ECS v1 implementation
11
+
12
+ ## 6.1.7
13
+ - [DOC] Remove limitations topic and link [#428](https://github.com/logstash-plugins/logstash-input-beats/pull/428)
14
+
15
+ ## 6.1.6
16
+ - [DOC] Applied more attributes to manage plugin name in doc content, and implemented conditional text processing. [#423](https://github.com/logstash-plugins/logstash-input-http/pull/423)
17
+
1
18
  ## 6.1.5
2
- - Changed jar dependencies to reflect newer versions [#425](https://github.com/logstash-plugins/logstash-input-http/pull/425)
19
+ - Changed jar dependencies to reflect newer versions [#425](https://github.com/logstash-plugins/logstash-input-beats/pull/425)
3
20
 
4
21
  ## 6.1.4
5
22
  - Fix: reduce error logging on connection resets [#424](https://github.com/logstash-plugins/logstash-input-beats/pull/424)
data/VERSION CHANGED
@@ -1 +1 @@
1
- 6.1.5
1
+ 6.2.2
data/docs/index.asciidoc CHANGED
@@ -2,6 +2,7 @@
2
2
  :type: input
3
3
  :default_codec: plain
4
4
  :plugin-uc: Beats
5
+ :plugin-singular: Beat
5
6
 
6
7
  ///////////////////////////////////////////
7
8
  START - GENERATED VARIABLES, DO NOT EDIT!
@@ -19,18 +20,21 @@ END - GENERATED VARIABLES, DO NOT EDIT!
19
20
  === {plugin-uc} input plugin
20
21
 
21
22
  NOTE: The `input-elastic_agent` plugin is the next generation of the
22
- `input-beats` plugin. They currently share a common codebase.
23
+ `input-beats` plugin.
24
+ They currently share code and a https://github.com/logstash-plugins/logstash-input-beats[common codebase].
23
25
 
24
26
  include::{include_path}/plugin_header.asciidoc[]
25
27
 
26
28
  ==== Description
27
29
 
28
30
  This input plugin enables Logstash to receive events from the
29
- https://www.elastic.co/products/beats[Elastic Beats] framework.
31
+ {plugin-uc} framework.
30
32
 
31
33
  The following example shows how to configure Logstash to listen on port
32
34
  5044 for incoming {plugin-uc} connections and to index into Elasticsearch.
33
35
 
36
+ //Example for Beats
37
+ ifeval::["{plugin}"=="beats"]
34
38
  ["source","sh",subs="attributes"]
35
39
  -----
36
40
 
@@ -48,9 +52,8 @@ output {
48
52
  }
49
53
  -----
50
54
  <1> `%{[@metadata][beat]}` sets the first part of the index name to the value
51
- of the `beat` metadata field and `%{[@metadata][version]}` sets the second part to
52
- the {plugin-uc} version. For example:
53
- metricbeat-7.4.0.
55
+ of the metadata field and `%{[@metadata][version]}` sets the second part to
56
+ the {plugin-singular} version. For example: metricbeat-6.1.6.
54
57
 
55
58
  Events indexed into Elasticsearch with the Logstash configuration shown here
56
59
  will be similar to events directly indexed by {plugin-uc} into Elasticsearch.
@@ -59,14 +62,47 @@ NOTE: If ILM is not being used, set `index` to
59
62
  `%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}` instead so
60
63
  Logstash creates an index per day, based on the `@timestamp` value of the events
61
64
  coming from {plugin-uc}.
65
+ endif::[]
62
66
 
63
- IMPORTANT: If you are shipping events that span multiple lines, you need to use
67
+ //Example for Elastic Agent
68
+ ifeval::["{plugin}"!="beats"]
69
+ ["source","sh",subs="attributes"]
70
+ -----
71
+
72
+ input {
73
+ {plugin} {
74
+ port => 5044
75
+ }
76
+ }
77
+
78
+ output {
79
+ elasticsearch {
80
+ hosts => ["http://localhost:9200"]
81
+ data_stream => "true"
82
+ }
83
+ }
84
+ -----
85
+
86
+ Events indexed into Elasticsearch with the Logstash configuration shown here
87
+ will be similar to events directly indexed by {plugin-uc} into Elasticsearch.
88
+ endif::[]
89
+
90
+
91
+ //Content for Beats
92
+ ifeval::["{plugin}"=="beats"]
93
+ [id="plugins-{type}s-{plugin}-multiline"]
94
+ ===== Multi-line events
95
+
96
+ If you are shipping events that span multiple lines, you need to use
64
97
  the {filebeat-ref}/multiline-examples.html[configuration options available in
65
98
  Filebeat] to handle multiline events before sending the event data to Logstash.
66
99
  You cannot use the {logstash-ref}/plugins-codecs-multiline.html[Multiline codec
67
100
  plugin] to handle multiline events. Doing so will result in the failure to start
68
101
  Logstash.
102
+ endif::[]
69
103
 
104
+ //Content for Beats
105
+ ifeval::["{plugin}"=="beats"]
70
106
  [id="plugins-{type}s-{plugin}-versioned-indexes"]
71
107
  ==== Versioned indices
72
108
 
@@ -89,6 +125,7 @@ Logstash `@timestamp` field.
89
125
 
90
126
  This configuration results in daily index names like
91
127
  +filebeat-{logstash_version}-{localdate}+.
128
+ endif::[]
92
129
 
93
130
 
94
131
  [id="plugins-{type}s-{plugin}-ecs_metadata"]
@@ -104,18 +141,18 @@ output.
104
141
 
105
142
  [cols="<l,<l,e,<e"]
106
143
  |=======================================================================
107
- |ECS disabled |ECS v1 |Availability |Description
144
+ |ECS `disabled` |ECS `v1`, `v8` |Availability |Description
108
145
 
109
- |[host] |[@metadata][input][beats][host][name] |Always |Name or address of the beat host
110
- |[@metadata][ip_address] |[@metadata][input][beats][host][ip] |Always |IP address of the Beats client
111
- |[@metadata][tls_peer][status] | [@metadata][tls_peer][status] | When SSL related fields are populated | Contains "verified"/"unverified" labels in `disabled`, `true`/`false` in `v1`
146
+ |[host] |[@metadata][input][beats][host][name] |Always |Name or address of the {plugin-singular} host
147
+ |[@metadata][ip_address] |[@metadata][input][beats][host][ip] |Always |IP address of the {plugin-uc} client
148
+ |[@metadata][tls_peer][status] | [@metadata][tls_peer][status] | When SSL related fields are populated | Contains "verified"/"unverified" labels in `disabled`, `true`/`false` in `v1`/`v8`
112
149
  |[@metadata][tls_peer][protocol] | [@metadata][input][beats][tls][version_protocol] | When SSL status is "verified" | Contains the TLS version used (e.g. `TLSv1.2`)
113
150
  |[@metadata][tls_peer][subject] | [@metadata][input][beats][tls][client][subject] | When SSL status is "verified" | Contains the identity name of the remote end (e.g. `CN=artifacts-no-kpi.elastic.co`)
114
151
  |[@metadata][tls_peer][cipher_suite] | [@metadata][input][beats][tls][cipher] | When SSL status is "verified" | Contains the name of cipher suite used (e.g. `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256`)
115
152
  |=======================================================================
116
153
 
117
154
  [id="plugins-{type}s-{plugin}-options"]
118
- ==== {plugin-uc} Input Configuration Options
155
+ ==== {plugin-uc} input configuration options
119
156
 
120
157
  This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> described later.
121
158
 
@@ -126,6 +163,7 @@ This plugin supports the following configuration options plus the <<plugins-{typ
126
163
  | <<plugins-{type}s-{plugin}-cipher_suites>> |<<array,array>>|No
127
164
  | <<plugins-{type}s-{plugin}-client_inactivity_timeout>> |<<number,number>>|No
128
165
  | <<plugins-{type}s-{plugin}-ecs_compatibility>> | <<string,string>>|No
166
+ | <<plugins-{type}s-{plugin}-executor_threads>> |<<number,number>>|No
129
167
  | <<plugins-{type}s-{plugin}-host>> |<<string,string>>|No
130
168
  | <<plugins-{type}s-{plugin}-include_codec_tag>> |<<boolean,boolean>>|No
131
169
  | <<plugins-{type}s-{plugin}-port>> |<<number,number>>|Yes
@@ -154,7 +192,7 @@ input plugins.
154
192
  * Value type is <<boolean,boolean>>
155
193
  * Default value is `false`
156
194
 
157
- Flag to determine whether to add `host` field to event using the value supplied by the beat in the `hostname` field.
195
+ Flag to determine whether to add `host` field to event using the value supplied by the {plugin-singular} in the `hostname` field.
158
196
 
159
197
 
160
198
  [id="plugins-{type}s-{plugin}-cipher_suites"]
@@ -179,13 +217,32 @@ Close Idle clients after X seconds of inactivity.
179
217
  * Value type is <<string,string>>
180
218
  * Supported values are:
181
219
  ** `disabled`: unstructured connection metadata added at root level
182
- ** `v1`: structured connection metadata added under ECS compliant namespaces
220
+ ** `v1`: structured connection metadata added under ECS v1 compliant namespaces
221
+ ** `v8`: structured connection metadata added under ECS v8 compliant namespaces
183
222
  * Default value depends on which version of Logstash is running:
184
223
  ** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
185
224
  ** Otherwise, the default value is `disabled`.
186
225
 
187
226
  Refer to <<plugins-{type}s-{plugin}-ecs_metadata,ECS mapping>> for detailed information.
188
227
 
228
+ [id="plugins-{type}s-{plugin}-executor_threads"]
229
+ ===== `executor_threads`
230
+
231
+ * Value type is <<number,number>>
232
+ * Default value is 1 executor thread per CPU core
233
+
234
+ The number of threads to be used to process incoming beats requests.
235
+ By default the Beats input creates a number of threads equal to 2*CPU cores.
236
+ These threads handle incoming connections, reading from established sockets, and executing most of the tasks related to network connection management.
237
+ Parsing the Lumberjack protocol is offloaded to a dedicated thread pool.
238
+
239
+ Generally you don't need to touch this setting.
240
+ In case you are sending very large events and observing "OutOfDirectMemory" exceptions,
241
+ you may want to reduce this number to half or 1/4 of the CPU cores.
242
+ This change reduces the number of threads decompressing batches of data into direct memory.
243
+ However, this will only be a mitigating tweak, as the proper solution may require resizing your Logstash deployment,
244
+ either by increasing number of Logstash nodes or increasing the JVM's Direct Memory.
245
+
189
246
  [id="plugins-{type}s-{plugin}-host"]
190
247
  ===== `host`
191
248
 
@@ -317,3 +374,4 @@ The minimum TLS version allowed for the encrypted connections. The value must be
317
374
  include::{include_path}/{type}.asciidoc[]
318
375
 
319
376
  :default_codec!:
377
+
@@ -15,6 +15,8 @@ module LogStash module Inputs class Beats
15
15
 
16
16
  attr_reader :logger, :input, :connections_list
17
17
 
18
+ attr_reader :event_factory
19
+
18
20
  def initialize(queue, input)
19
21
  @connections_list = ThreadSafe::Hash.new
20
22
  @queue = queue
@@ -25,6 +27,7 @@ module LogStash module Inputs class Beats
25
27
 
26
28
  @nocodec_transformer = RawEventTransform.new(@input)
27
29
  @codec_transformer = DecodedEventTransform.new(@input)
30
+ @event_factory = input.event_factory
28
31
  end
29
32
 
30
33
  def onNewMessage(ctx, message)
@@ -39,7 +42,7 @@ module LogStash module Inputs class Beats
39
42
  extract_tls_peer(hash, ctx)
40
43
 
41
44
  if target_field.nil?
42
- event = LogStash::Event.new(hash)
45
+ event = event_factory.new_event(hash)
43
46
  @nocodec_transformer.transform(event)
44
47
  @queue << event
45
48
  else
@@ -129,7 +132,7 @@ module LogStash module Inputs class Beats
129
132
  tls_session = ctx.channel().pipeline().get("ssl-handler").engine().getSession()
130
133
  tls_verified = true
131
134
 
132
- if not @input.client_authentication_required?
135
+ unless @input.client_authentication_required?
133
136
  # throws SSLPeerUnverifiedException if unverified
134
137
  begin
135
138
  tls_session.getPeerCertificates()
@@ -141,18 +144,16 @@ module LogStash module Inputs class Beats
141
144
  end
142
145
  end
143
146
 
147
+ meta_data = hash['@metadata'] ||= {}
148
+
144
149
  if tls_verified
145
- set_nested(hash, @field_tls_protocol_version, tls_session.getProtocol())
146
- set_nested(hash, @field_tls_peer_subject, tls_session.getPeerPrincipal().getName())
147
- set_nested(hash, @field_tls_cipher, tls_session.getCipherSuite())
150
+ meta_data['tls_peer'] = { :status => "verified" }
148
151
 
149
- hash['@metadata']['tls_peer'] = {
150
- :status => "verified"
151
- }
152
+ set_nested(hash, input.field_tls_protocol_version, tls_session.getProtocol())
153
+ set_nested(hash, input.field_tls_peer_subject, tls_session.getPeerPrincipal().getName())
154
+ set_nested(hash, input.field_tls_cipher, tls_session.getCipherSuite())
152
155
  else
153
- hash['@metadata']['tls_peer'] = {
154
- :status => "unverified"
155
- }
156
+ meta_data['tls_peer'] = { :status => "unverified" }
156
157
  end
157
158
  end
158
159
  end
@@ -163,9 +164,6 @@ module LogStash module Inputs class Beats
163
164
  field_ref = Java::OrgLogstash::FieldReference.from(field_name)
164
165
  # create @metadata sub-hash if needed
165
166
  if field_ref.type == Java::OrgLogstash::FieldReference::META_CHILD
166
- unless hash.key?("@metadata")
167
- hash["@metadata"] = {}
168
- end
169
167
  nesting_hash = hash["@metadata"]
170
168
  else
171
169
  nesting_hash = hash
@@ -6,6 +6,7 @@ require "logstash/codecs/multiline"
6
6
  require "logstash/util"
7
7
  require "logstash-input-beats_jars"
8
8
  require "logstash/plugin_mixins/ecs_compatibility_support"
9
+ require 'logstash/plugin_mixins/event_support/event_factory_adapter'
9
10
  require_relative "beats/patch"
10
11
 
11
12
  # This input plugin enables Logstash to receive events from the
@@ -51,7 +52,9 @@ class LogStash::Inputs::Beats < LogStash::Inputs::Base
51
52
  require "logstash/inputs/beats/tls"
52
53
 
53
54
  # adds ecs_compatibility config which could be :disabled or :v1
54
- include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled,:v1)
55
+ include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled,:v1, :v8 => :v1)
56
+
57
+ include LogStash::PluginMixins::EventSupport::EventFactoryAdapter
55
58
 
56
59
  config_name "beats"
57
60
 
@@ -126,6 +129,7 @@ class LogStash::Inputs::Beats < LogStash::Inputs::Base
126
129
  config :executor_threads, :validate => :number, :default => LogStash::Config::CpuCoreStrategy.maximum
127
130
 
128
131
  attr_reader :field_hostname, :field_hostip
132
+ attr_reader :field_tls_protocol_version, :field_tls_peer_subject, :field_tls_cipher
129
133
 
130
134
  def register
131
135
  # For Logstash 2.4 we need to make sure that the logger is correctly set for the
@@ -164,10 +168,10 @@ class LogStash::Inputs::Beats < LogStash::Inputs::Base
164
168
 
165
169
  # define ecs name mapping
166
170
  @field_hostname = ecs_select[disabled: "host", v1: "[@metadata][input][beats][host][name]"]
167
- @field_hostip = ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][beats][host][ip]"]
168
- @field_tls_protocol_version = ecs_select[disabled: "[@metadata][tls_peer][protocol]", v1: "[@metadata][input][beats][tls][version_protocol]"]
169
- @field_tls_peer_subject = ecs_select[disabled: "[@metadata][tls_peer][subject]", v1: "[@metadata][input][beats][tls][client][subject]"]
170
- @field_tls_cipher = ecs_select[disabled: "[@metadata][tls_peer][cipher_suite]", v1: "[@metadata][input][beats][tls][cipher]"]
171
+ @field_hostip = ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][beats][host][ip]"]
172
+ @field_tls_protocol_version = ecs_select[disabled: "[@metadata][tls_peer][protocol]", v1: "[@metadata][input][beats][tls][version_protocol]"]
173
+ @field_tls_peer_subject = ecs_select[disabled: "[@metadata][tls_peer][subject]", v1: "[@metadata][input][beats][tls][client][subject]"]
174
+ @field_tls_cipher = ecs_select[disabled: "[@metadata][tls_peer][cipher_suite]", v1: "[@metadata][input][beats][tls][cipher]"]
171
175
 
172
176
  @logger.info("Starting input listener", :address => "#{@host}:#{@port}")
173
177
 
@@ -8,4 +8,4 @@ require_jar('com.fasterxml.jackson.core', 'jackson-annotations', '2.9.10')
8
8
  require_jar('com.fasterxml.jackson.core', 'jackson-databind', '2.9.10.8')
9
9
  require_jar('com.fasterxml.jackson.module', 'jackson-module-afterburner', '2.9.10')
10
10
  require_jar('org.apache.logging.log4j', 'log4j-api', '2.11.1')
11
- require_jar('org.logstash.beats', 'logstash-input-beats', '6.1.5')
11
+ require_jar('org.logstash.beats', 'logstash-input-beats', '6.2.2')
@@ -27,7 +27,8 @@ Gem::Specification.new do |s|
27
27
  s.add_runtime_dependency "thread_safe", "~> 0.3.5"
28
28
  s.add_runtime_dependency "logstash-codec-multiline", ">= 2.0.5"
29
29
  s.add_runtime_dependency 'jar-dependencies', '~> 0.3', '>= 0.3.4'
30
- s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.1'
30
+ s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.3'
31
+ s.add_runtime_dependency 'logstash-mixin-event_support', '~>1.0'
31
32
 
32
33
  s.add_development_dependency "flores", "~>0.0.6"
33
34
  s.add_development_dependency "rspec"
@@ -31,6 +31,7 @@ describe LogStash::Inputs::Beats::DecodedEventTransform do
31
31
 
32
32
  include_examples "Common Event Transformation", :disabled, "host"
33
33
  include_examples "Common Event Transformation", :v1, "[@metadata][input][beats][host][name]"
34
+ include_examples "Common Event Transformation", :v8, "[@metadata][input][beats][host][name]"
34
35
 
35
36
  it "tags the event" do
36
37
  expect(subject.get("tags")).to include("beats_input_codec_plain_applied")
@@ -9,4 +9,5 @@ describe LogStash::Inputs::Beats::EventTransformCommon do
9
9
 
10
10
  include_examples "Common Event Transformation", :disabled, "host"
11
11
  include_examples "Common Event Transformation", :v1, "[@metadata][input][beats][host][name]"
12
+ include_examples "Common Event Transformation", :v8, "[@metadata][input][beats][host][name]"
12
13
  end
@@ -211,6 +211,7 @@ describe LogStash::Inputs::Beats::MessageListener do
211
211
 
212
212
  it_behaves_like "when the message is from any libbeat", :disabled, "[@metadata][ip_address]"
213
213
  it_behaves_like "when the message is from any libbeat", :v1, "[@metadata][input][beats][host][ip]"
214
+ it_behaves_like "when the message is from any libbeat", :v8, "[@metadata][input][beats][host][ip]"
214
215
  end
215
216
 
216
217
  context "onException" do
@@ -20,6 +20,7 @@ describe LogStash::Inputs::Beats::RawEventTransform do
20
20
 
21
21
  include_examples "Common Event Transformation", :disabled, "host"
22
22
  include_examples "Common Event Transformation", :v1, "[@metadata][input][beats][host][name]"
23
+ include_examples "Common Event Transformation", :v8, "[@metadata][input][beats][host][name]"
23
24
 
24
25
  it "tags the event" do
25
26
  expect(subject.get("tags")).to include("beats_input_raw_event")
@@ -12,26 +12,28 @@ describe LogStash::Inputs::Beats do
12
12
  let(:connection) { double("connection") }
13
13
  let(:certificate) { BeatsInputTest.certificate }
14
14
  let(:port) { BeatsInputTest.random_port }
15
+ let(:client_inactivity_timeout) { 400 }
16
+ let(:threads) { 1 + rand(9) }
15
17
  let(:queue) { Queue.new }
16
18
  let(:config) do
17
19
  {
18
- "port" => 0,
20
+ "port" => port,
19
21
  "ssl_certificate" => certificate.ssl_cert,
20
22
  "ssl_key" => certificate.ssl_key,
23
+ "client_inactivity_timeout" => client_inactivity_timeout,
24
+ "executor_threads" => threads,
21
25
  "type" => "example",
22
26
  "tags" => "beats"
23
27
  }
24
28
  end
25
29
 
30
+ subject(:plugin) { LogStash::Inputs::Beats.new(config) }
31
+
26
32
  context "#register" do
27
33
  context "host related configuration" do
28
- let(:config) { super().merge("host" => host, "port" => port, "client_inactivity_timeout" => client_inactivity_timeout, "executor_threads" => threads) }
34
+ let(:config) { super().merge("host" => host, "port" => port) }
29
35
  let(:host) { "192.168.1.20" }
30
- let(:port) { 9000 }
31
- let(:client_inactivity_timeout) { 400 }
32
- let(:threads) { 10 }
33
-
34
- subject(:plugin) { LogStash::Inputs::Beats.new(config) }
36
+ let(:port) { 9001 }
35
37
 
36
38
  it "sends the required options to the server" do
37
39
  expect(org.logstash.beats.Server).to receive(:new).with(host, port, client_inactivity_timeout, threads)
@@ -158,9 +160,80 @@ describe LogStash::Inputs::Beats do
158
160
 
159
161
  it "raise a ConfigurationError when multiline codec is set" do
160
162
  plugin = LogStash::Inputs::Beats.new(config)
161
- expect {plugin.register}.to raise_error(LogStash::ConfigurationError, "Multiline codec with beats input is not supported. Please refer to the beats documentation for how to best manage multiline data. See https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html")
163
+ expect { plugin.register }.to raise_error(LogStash::ConfigurationError, "Multiline codec with beats input is not supported. Please refer to the beats documentation for how to best manage multiline data. See https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html")
164
+ end
165
+ end
166
+ end
167
+
168
+ context "tls meta-data" do
169
+ let(:config) { super().merge("host" => host, "ssl_peer_metadata" => true, "ssl_certificate_authorities" => [ certificate.ssl_cert ]) }
170
+ let(:host) { "192.168.1.20" }
171
+ let(:port) { 9002 }
172
+
173
+ let(:queue) { Queue.new }
174
+ let(:event) { LogStash::Event.new }
175
+
176
+ subject(:plugin) { LogStash::Inputs::Beats.new(config) }
177
+
178
+ before do
179
+ @server = org.logstash.beats.Server.new(host, port, client_inactivity_timeout, threads)
180
+ expect( org.logstash.beats.Server ).to receive(:new).with(host, port, client_inactivity_timeout, threads).and_return @server
181
+ expect( @server ).to receive(:listen)
182
+
183
+ subject.register
184
+ subject.run(queue) # listen does nothing
185
+ @message_listener = @server.getMessageListener
186
+
187
+ allow( ssl_engine = double('ssl_engine') ).to receive(:getSession).and_return ssl_session
188
+ allow( ssl_handler = double('ssl-handler') ).to receive(:engine).and_return ssl_engine
189
+ allow( pipeline = double('pipeline') ).to receive(:get).and_return ssl_handler
190
+ allow( @channel = double('channel') ).to receive(:pipeline).and_return pipeline
191
+ end
192
+
193
+ let(:ctx) do
194
+ Java::io.netty.channel.ChannelHandlerContext.impl do |method, *args|
195
+ fail("unexpected #{method}( #{args} )") unless method.eql?(:channel)
196
+ @channel
162
197
  end
163
198
  end
199
+
200
+ let(:ssl_session) do
201
+ Java::javax.net.ssl.SSLSession.impl do |method, *args|
202
+ case method
203
+ when :getPeerCertificates
204
+ [].to_java(java.security.cert.Certificate)
205
+ when :getProtocol
206
+ 'TLS-Mock'
207
+ when :getCipherSuite
208
+ 'SSL_NULL_WITH_TEST_SPEC'
209
+ when :getPeerPrincipal
210
+ javax.security.auth.x500.X500Principal.new('CN=TEST, OU=RSpec, O=Logstash, C=NL', {})
211
+ else
212
+ fail("unexpected #{method}( #{args} )")
213
+ end
214
+ end
215
+ end
216
+
217
+ let(:ssl_session_peer_principal) do
218
+ javax.security.auth.x500.X500Principal
219
+ end
220
+
221
+ let(:message) do
222
+ org.logstash.beats.Message.new(0, java.util.HashMap.new('foo' => 'bar'))
223
+ end
224
+
225
+ it 'sets tls fields' do
226
+ @message_listener.onNewMessage(ctx, message)
227
+
228
+ expect( queue.size ).to be 1
229
+ expect( event = queue.pop ).to be_a LogStash::Event
230
+
231
+ expect( event.get('[@metadata][tls_peer][status]') ).to eql 'verified'
232
+
233
+ expect( event.get('[@metadata][tls_peer][protocol]') ).to eql 'TLS-Mock'
234
+ expect( event.get('[@metadata][tls_peer][cipher_suite]') ).to eql 'SSL_NULL_WITH_TEST_SPEC'
235
+ expect( event.get('[@metadata][tls_peer][subject]') ).to eql 'CN=TEST,OU=RSpec,O=Logstash,C=NL'
236
+ end
164
237
  end
165
238
 
166
239
  context "when interrupting the plugin" do
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-input-beats
3
3
  version: !ruby/object:Gem::Version
4
- version: 6.1.5
4
+ version: 6.2.2
5
5
  platform: java
6
6
  authors:
7
7
  - Elastic
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-06-21 00:00:00.000000000 Z
11
+ date: 2021-11-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  requirement: !ruby/object:Gem::Requirement
@@ -111,7 +111,7 @@ dependencies:
111
111
  requirements:
112
112
  - - "~>"
113
113
  - !ruby/object:Gem::Version
114
- version: '1.1'
114
+ version: '1.3'
115
115
  name: logstash-mixin-ecs_compatibility_support
116
116
  prerelease: false
117
117
  type: :runtime
@@ -119,7 +119,21 @@ dependencies:
119
119
  requirements:
120
120
  - - "~>"
121
121
  - !ruby/object:Gem::Version
122
- version: '1.1'
122
+ version: '1.3'
123
+ - !ruby/object:Gem::Dependency
124
+ requirement: !ruby/object:Gem::Requirement
125
+ requirements:
126
+ - - "~>"
127
+ - !ruby/object:Gem::Version
128
+ version: '1.0'
129
+ name: logstash-mixin-event_support
130
+ prerelease: false
131
+ type: :runtime
132
+ version_requirements: !ruby/object:Gem::Requirement
133
+ requirements:
134
+ - - "~>"
135
+ - !ruby/object:Gem::Version
136
+ version: '1.0'
123
137
  - !ruby/object:Gem::Dependency
124
138
  requirement: !ruby/object:Gem::Requirement
125
139
  requirements:
@@ -285,7 +299,7 @@ files:
285
299
  - vendor/jar-dependencies/io/netty/netty-all/4.1.65.Final/netty-all-4.1.65.Final.jar
286
300
  - vendor/jar-dependencies/org/apache/logging/log4j/log4j-api/2.11.1/log4j-api-2.11.1.jar
287
301
  - vendor/jar-dependencies/org/javassist/javassist/3.24.0-GA/javassist-3.24.0-GA.jar
288
- - vendor/jar-dependencies/org/logstash/beats/logstash-input-beats/6.1.5/logstash-input-beats-6.1.5.jar
302
+ - vendor/jar-dependencies/org/logstash/beats/logstash-input-beats/6.2.2/logstash-input-beats-6.2.2.jar
289
303
  homepage: http://www.elastic.co/guide/en/logstash/current/index.html
290
304
  licenses:
291
305
  - Apache License (2.0)
@@ -308,8 +322,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
308
322
  - !ruby/object:Gem::Version
309
323
  version: '0'
310
324
  requirements: []
311
- rubyforge_project:
312
- rubygems_version: 2.6.13
325
+ rubygems_version: 3.1.6
313
326
  signing_key:
314
327
  specification_version: 4
315
328
  summary: Receives events from the Elastic Beats framework