logstash-input-tcp 6.1.1-java → 6.2.0-java

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9b37d806eedc118d3a553f449a68e90d9366e258b0484c5e1aad54e0e6939883
4
- data.tar.gz: cae6d73f16e144241b588a3eb9b2c7703ac42ccb0d26a266736ad3d5d5d3f4bb
3
+ metadata.gz: 29b44274cfe25c623273407b63c29125143a6ec704c274732b73e9a3fc39f085
4
+ data.tar.gz: 4bced499147e26e9f8257d2c7a489a7dae93ec2a1c06faf2b45b3d9af801d222
5
5
  SHA512:
6
- metadata.gz: bf8d08b9b60d6268b2e0625b175048c0def7aed442661c89415797cbbcf6409e54d25bad0b4d7cd1d6dc8816fa3b4d9b79ebb78e60a8344fca7f62451637b283
7
- data.tar.gz: fe74219396c110829a71fbdfcbb36d07d1c8e4d06a58af00093e43741b62aad3d34638bbb3d19e9888e8d5edb642edd59110865b4a39c2a1ef624797c4f66225
6
+ metadata.gz: d45ef4273d94e4dc3dbcc94749d0e8d907ea96acb20ee4aeff94251a5390bb26b51105fc2f6104e09b885dc195f9fbe94261047035f905884d8306649e8aec49
7
+ data.tar.gz: f00ff1165183ce006a12541b7836bb3a09496eb4370bca185104ea81c8fc8e2360aae8ed09eeb170ab93ee9e64e7233af9360e4cbb73fc5d9b49adf27d26534b
data/CHANGELOG.md CHANGED
@@ -1,3 +1,8 @@
1
+ ## 6.2.0
2
+ - Added ECS Compatibility Mode [#165](https://github.com/logstash-plugins/logstash-input-tcp/pull/165)
3
+ - When operating in an ECS Compatibility mode, metadata about the connection on which we are receiving data is nested in well-named fields under `[@metadata][input][tcp]` instead of at the root level.
4
+ - Fix: source address is no longer missing when a proxy is present
5
+
1
6
  ## 6.1.1
2
7
  - Changed jar dependencies to reflect newer versions [#179](https://github.com/logstash-plugins/logstash-input-http/pull/179)
3
8
 
data/docs/index.asciidoc CHANGED
@@ -70,6 +70,52 @@ event timestamp
70
70
  }
71
71
  }
72
72
 
73
+ [id="plugins-{type}s-{plugin}-ecs_metadata"]
74
+ ==== Event Metadata and the Elastic Common Schema (ECS)
75
+
76
+ In addition to decoding the events, this input will add metadata about the TCP connection itself to each event.
77
+ This can be helpful when applications are configured to send events directly to this input's TCP listener without including information about themselves.
78
+
79
+ Historically, this metadata was added to a variety of non-standard top-level fields, which had the potential to create confusion and schema conflicts downstream.
80
+ With ECS compatibility mode, we can ensure a pipeline still has access to this metadata throughout the event's lifecycle without polluting the top-level namespace.
81
+
82
+ [cols="3,7,5"]
83
+ |=======================================================================
84
+ | Metadata Group | ecs: `v1`, `v8` | ecs: `disabled`
85
+
86
+ .3+|Source Metadata from the TCP connection
87
+ on which events are being received, including
88
+ the sender's name, ip, and outbound port. l|[@metadata][input][tcp][source][name] l|[host]
89
+ l|[@metadata][input][tcp][source][ip] l|[@metadata][ip_address]
90
+ l|[@metadata][input][tcp][source][port] l|[port]
91
+
92
+ .2+|Proxy Metadata from a proxied TCP connection.
93
+ Available when receiving events by proxy and
94
+ `proxy_protocol => true` l|[@metadata][input][tcp][proxy][ip] l|[proxy_host]
95
+ l|[@metadata][input][tcp][proxy][port] l|[proxy_port]
96
+
97
+ .1+|SSL Subject Metadata from a secured TCP
98
+ connection. Available when `ssl_enable => true`
99
+ AND `ssl_verify => true` l|[@metadata][input][tcp][ssl][subject] l|[sslsubject]
100
+ |=======================================================================
101
+
102
+ For example, the Elastic Common Schema reserves the https://www.elastic.co/guide/en/ecs/current/ecs-host.html[top-level `host` field] for information about the host on which the event happened.
103
+ If an event is missing this metadata, it can be copied into place from the source TCP connection metadata that has been added to the event:
104
+
105
+ [source,txt]
106
+ -----
107
+ filter {
108
+ if [@metadata][input][tcp][source] and not [host] {
109
+ mutate {
110
+ copy {
111
+ "[@metadata][input][tcp][source][name]" => "[host][name]"
112
+ "[@metadata][input][tcp][source][ip]" => "[host][ip]"
113
+ }
114
+ }
115
+ }
116
+ }
117
+ -----
118
+
73
119
  [id="plugins-{type}s-{plugin}-options"]
74
120
  ==== Tcp Input Configuration Options
75
121
 
@@ -79,6 +125,7 @@ This plugin supports the following configuration options plus the <<plugins-{typ
79
125
  |=======================================================================
80
126
  |Setting |Input type|Required
81
127
  | <<plugins-{type}s-{plugin}-dns_reverse_lookup_enabled>> |<<boolean,boolean>>|No
128
+ | <<plugins-{type}s-{plugin}-ecs_compatibility>> | <<string,string>>|No
82
129
  | <<plugins-{type}s-{plugin}-host>> |<<string,string>>|No
83
130
  | <<plugins-{type}s-{plugin}-mode>> |<<string,string>>, one of `["server", "client"]`|No
84
131
  | <<plugins-{type}s-{plugin}-port>> |<<number,number>>|Yes
@@ -108,6 +155,20 @@ It is possible to avoid DNS reverse-lookups by disabling this setting. If disabl
108
155
  the address metadata that is added to events will contain the source address as-specified
109
156
  at the TCP layer and IPs will not be resolved to hostnames.
110
157
 
158
+ [id="plugins-{type}s-{plugin}-ecs_compatibility"]
159
+ ===== `ecs_compatibility`
160
+
161
+ * Value type is <<string,string>>
162
+ * Supported values are:
163
+ ** `disabled`: unstructured connection metadata added at root level
164
+ ** `v1`,`v8`: structured connection metadata added under `[@metadata][input][tcp]`
165
+ * Default value depends on which version of Logstash is running:
166
+ ** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
167
+ ** Otherwise, the default value is `disabled`.
168
+
169
+ Controls this plugin's compatibility with the https://www.elastic.co/guide/en/ecs/current/index.html[Elastic Common Schema (ECS)].
170
+ The value of this setting affects the <<plugins-{type}s-{plugin}-ecs_metadata,placement of a TCP connection's metadata>> on events.
171
+
111
172
  [id="plugins-{type}s-{plugin}-host"]
112
173
  ===== `host`
113
174
 
@@ -5,7 +5,7 @@ require "java"
5
5
  require "logstash/inputs/base"
6
6
  require "logstash/util/socket_peer"
7
7
  require "logstash-input-tcp_jars"
8
- require "logstash/inputs/tcp/decoder_impl"
8
+ require 'logstash/plugin_mixins/ecs_compatibility_support'
9
9
 
10
10
  require "socket"
11
11
  require "openssl"
@@ -63,6 +63,11 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
63
63
  java_import 'org.logstash.tcp.InputLoop'
64
64
  java_import 'org.logstash.tcp.SslContextBuilder'
65
65
 
66
+ require_relative "tcp/decoder_impl"
67
+
68
+ # ecs_compatibility option, provided by Logstash core or the support adapter.
69
+ include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled, :v1, :v8 => :v1)
70
+
66
71
  config_name "tcp"
67
72
 
68
73
  default :codec, "line"
@@ -113,13 +118,6 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
113
118
  # Option to allow users to avoid DNS Reverse Lookup.
114
119
  config :dns_reverse_lookup_enabled, :validate => :boolean, :default => true
115
120
 
116
- HOST_FIELD = "host".freeze
117
- HOST_IP_FIELD = "[@metadata][ip_address]".freeze
118
- PORT_FIELD = "port".freeze
119
- PROXY_HOST_FIELD = "proxy_host".freeze
120
- PROXY_PORT_FIELD = "proxy_port".freeze
121
- SSLSUBJECT_FIELD = "sslsubject".freeze
122
-
123
121
  # Monkey patch TCPSocket and SSLSocket to include socket peer
124
122
  # @private
125
123
  def self.patch_socket_peer!
@@ -134,6 +132,8 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
134
132
  def initialize(*args)
135
133
  super(*args)
136
134
 
135
+ setup_fields!
136
+
137
137
  self.class.patch_socket_peer!
138
138
 
139
139
  # threadsafe socket bookkeeping
@@ -186,8 +186,8 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
186
186
  proxy_port, tbuf, socket)
187
187
  codec.decode(tbuf) do |event|
188
188
  if @proxy_protocol
189
- event.set(PROXY_HOST_FIELD, proxy_address) unless event.get(PROXY_HOST_FIELD)
190
- event.set(PROXY_PORT_FIELD, proxy_port) unless event.get(PROXY_PORT_FIELD)
189
+ event.set(@field_proxy_host, proxy_address) unless event.get(@field_proxy_host)
190
+ event.set(@field_proxy_port, proxy_port) unless event.get(@field_proxy_port)
191
191
  end
192
192
  enqueue_decorated(event, client_ip_address, client_address, client_port, socket)
193
193
  end
@@ -260,14 +260,24 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
260
260
  end
261
261
 
262
262
  def enqueue_decorated(event, client_ip_address, client_address, client_port, socket)
263
- event.set(HOST_FIELD, client_address) unless event.get(HOST_FIELD)
264
- event.set(HOST_IP_FIELD, client_ip_address) unless event.get(HOST_IP_FIELD)
265
- event.set(PORT_FIELD, client_port) unless event.get(PORT_FIELD)
266
- event.set(SSLSUBJECT_FIELD, socket.peer_cert.subject.to_s) if socket && @ssl_enable && @ssl_verify && event.get(SSLSUBJECT_FIELD).nil?
263
+ event.set(@field_host, client_address) unless event.get(@field_host)
264
+ event.set(@field_host_ip, client_ip_address) unless event.get(@field_host_ip)
265
+ event.set(@field_port, client_port) unless event.get(@field_port)
266
+ event.set(@field_sslsubject, socket.peer_cert.subject.to_s) if socket && @ssl_enable && @ssl_verify && event.get(@field_sslsubject).nil?
267
267
  decorate(event)
268
268
  @output_queue << event
269
269
  end
270
270
 
271
+ # setup the field names, with respect to ECS compatibility.
272
+ def setup_fields!
273
+ @field_host = ecs_select[disabled: "host", v1: "[@metadata][input][tcp][source][name]" ].freeze
274
+ @field_host_ip = ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][tcp][source][ip]" ].freeze
275
+ @field_port = ecs_select[disabled: "port", v1: "[@metadata][input][tcp][source][port]" ].freeze
276
+ @field_proxy_host = ecs_select[disabled: "proxy_host", v1: "[@metadata][input][tcp][proxy][ip]" ].freeze
277
+ @field_proxy_port = ecs_select[disabled: "proxy_port", v1: "[@metadata][input][tcp][proxy][port]" ].freeze
278
+ @field_sslsubject = ecs_select[disabled: "sslsubject", v1: "[@metadata][input][tcp][tls][client][subject]"].freeze
279
+ end
280
+
271
281
  def server?
272
282
  @mode == "server"
273
283
  end
@@ -1,7 +1,7 @@
1
1
  # encoding: utf-8
2
2
  require 'java'
3
3
 
4
- class DecoderImpl
4
+ class LogStash::Inputs::Tcp::DecoderImpl
5
5
 
6
6
  include org.logstash.tcp.Decoder
7
7
 
@@ -24,7 +24,7 @@ class DecoderImpl
24
24
  end
25
25
 
26
26
  def copy
27
- DecoderImpl.new(@codec.clone, @tcp)
27
+ self.class.new(@codec.clone, @tcp)
28
28
  end
29
29
 
30
30
  def flush
@@ -41,16 +41,17 @@ class DecoderImpl
41
41
  @tcp.logger.error("Invalid proxy protocol header label", :header => pp_hdr)
42
42
  raise IOError.new("Invalid proxy protocol header label #{pp_hdr.inspect}")
43
43
  else
44
- @proxy_address = pp_info[3]
45
- @proxy_port = pp_info[5]
46
- @address = pp_info[2]
47
- @port = pp_info[4]
44
+ @proxy_address = pp_info[3] # layer 3 destination address (proxy's receiving address)
45
+ @proxy_port = pp_info[5] # TCP destination port (proxy's receiving port)
46
+ @ip_address = pp_info[2] # layer 3 source address (outgoing ip of sender)
47
+ @address = extract_host_name(@ip_address)
48
+ @port = pp_info[4] # TCP source port (outgoing port on sender [probably random])
48
49
  end
49
50
  else
50
51
  filtered = received
51
- @ip_address = channel_addr.get_address.get_host_address
52
- @address = extract_host_name(channel_addr)
53
- @port = channel_addr.get_port
52
+ @ip_address = channel_addr.get_address.get_host_address # ip address of sender
53
+ @address = extract_host_name(channel_addr) # name _or_ address of sender
54
+ @port = channel_addr.get_port # outgoing port of sender (probably random)
54
55
  end
55
56
  @first_read = false
56
57
  filtered
@@ -58,6 +59,8 @@ class DecoderImpl
58
59
 
59
60
  private
60
61
  def extract_host_name(channel_addr)
62
+ channel_addr = java.net.InetSocketAddress.new(channel_addr, 0) if channel_addr.kind_of?(String)
63
+
61
64
  return channel_addr.get_host_string unless @tcp.dns_reverse_lookup_enabled?
62
65
 
63
66
  channel_addr.get_host_name
@@ -21,6 +21,7 @@ Gem::Specification.new do |s|
21
21
 
22
22
  # Gem dependencies
23
23
  s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
24
+ s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.2'
24
25
 
25
26
  s.add_runtime_dependency 'logstash-core', '>= 6.7.0'
26
27
 
@@ -15,9 +15,11 @@ java_import "io.netty.handler.ssl.util.SelfSignedCertificate"
15
15
 
16
16
  require_relative "../spec_helper"
17
17
 
18
+ require 'logstash/plugin_mixins/ecs_compatibility_support/spec_helper'
19
+
18
20
  #Cabin::Channel.get(LogStash).subscribe(STDOUT)
19
21
  #Cabin::Channel.get(LogStash).level = :debug
20
- describe LogStash::Inputs::Tcp do
22
+ describe LogStash::Inputs::Tcp, :ecs_compatibility_support do
21
23
 
22
24
  def get_port
23
25
  begin
@@ -52,160 +54,176 @@ describe LogStash::Inputs::Tcp do
52
54
  end
53
55
  end
54
56
 
55
- it "should read plain with unicode" do
56
- event_count = 10
57
- conf = <<-CONFIG
58
- input {
59
- tcp {
60
- port => #{port}
57
+ ecs_compatibility_matrix(:disabled,:v1, :v8 => :v1) do |ecs_select|
58
+ before(:each) do
59
+ allow_any_instance_of(described_class).to receive(:ecs_compatibility).and_return(ecs_compatibility)
60
+ end
61
+
62
+ it "should read plain with unicode" do
63
+ event_count = 10
64
+ conf = <<-CONFIG
65
+ input {
66
+ tcp {
67
+ port => #{port}
68
+ }
61
69
  }
62
- }
63
- CONFIG
70
+ CONFIG
71
+
72
+ host = 'localhost'
73
+ events = input(conf) do |pipeline, queue|
74
+ socket = Stud::try(5.times) { TCPSocket.new(host, port) }
75
+ event_count.times do |i|
76
+ # unicode smiley for testing unicode support!
77
+ socket.puts("#{i} ☹")
78
+ socket.flush
79
+ end
80
+ socket.close
64
81
 
65
- host = 'localhost'
66
- events = input(conf) do |pipeline, queue|
67
- socket = Stud::try(5.times) { TCPSocket.new(host, port) }
68
- event_count.times do |i|
69
- # unicode smiley for testing unicode support!
70
- socket.puts("#{i} ☹")
71
- socket.flush
82
+ event_count.times.collect {queue.pop}
72
83
  end
73
- socket.close
74
84
 
75
- event_count.times.collect {queue.pop}
76
- end
85
+ expect(events.length).to eq(event_count)
86
+ events = events.sort_by {|e| e.get("message")} # the ordering of events in the queue is highly timing-dependent
87
+ event_count.times do |i|
88
+ event = events[i]
77
89
 
78
- insist { events.length } == event_count
79
- events = events.sort_by {|e| e.get("message")} # the ordering of events in the queue is highly timing-dependent
80
- event_count.times do |i|
81
- event = events[i]
82
- insist { event.get("message") } == "#{i} ☹"
83
- insist { ["localhost","ip6-localhost"].includes? event.get("host") }
84
- insist { event.get("[@metadata][ip_address]") } == '127.0.0.1'
90
+ aggregate_failures("event #{i}") do
91
+ expect(event.get("message")).to eq("#{i} ☹")
92
+ expect(event.get(ecs_select[disabled: "host", v1: "[@metadata][input][tcp][source][name]"])).to eq("localhost").or eq("ip6-localhost")
93
+ expect(event.get(ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][tcp][source][ip]"])).to eq('127.0.0.1')
94
+ end
95
+ end
85
96
  end
86
- end
87
97
 
88
- it "should handle PROXY protocol v1 connections" do
89
- event_count = 10
90
- conf = <<-CONFIG
91
- input {
92
- tcp {
93
- proxy_protocol => true
94
- port => '#{port}'
98
+ it "should handle PROXY protocol v1 connections" do
99
+ event_count = 10
100
+ conf = <<-CONFIG
101
+ input {
102
+ tcp {
103
+ proxy_protocol => true
104
+ port => '#{port}'
105
+ }
95
106
  }
96
- }
97
- CONFIG
107
+ CONFIG
98
108
 
99
- events = input(conf) do |pipeline, queue|
100
- socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
101
- socket.puts("PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678\r");
102
- socket.flush
103
- event_count.times do |i|
104
- # unicode smiley for testing unicode support!
105
- socket.puts("#{i} ☹")
109
+ events = input(conf) do |pipeline, queue|
110
+ socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
111
+ socket.puts("PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678\r");
106
112
  socket.flush
107
- end
108
- socket.close
113
+ event_count.times do |i|
114
+ # unicode smiley for testing unicode support!
115
+ socket.puts("#{i} ☹")
116
+ socket.flush
117
+ end
118
+ socket.close
109
119
 
110
- event_count.times.collect {queue.pop}
111
- end
120
+ event_count.times.collect {queue.pop}
121
+ end
112
122
 
113
- insist { events.length } == event_count
114
- events = events.sort_by {|e| e.get("message")} # the ordering of events in the queue is highly timing-dependent
115
- event_count.times do |i|
116
- insist { events[i].get("message") } == "#{i} ☹"
117
- insist { events[i].get("host") } == "1.2.3.4"
118
- insist { events[i].get("port") } == "1234"
119
- insist { events[i].get("proxy_host") } == "5.6.7.8"
120
- insist { events[i].get("proxy_port") } == "5678"
123
+ expect(events.length).to eq(event_count)
124
+ events = events.sort_by {|e| e.get("message")} # the ordering of events in the queue is highly timing-dependent
125
+ events.each_with_index do |event, i|
126
+ aggregate_failures("event #{i}") do
127
+ expect(event.get("message")).to eq("#{i} ")
128
+ expect(event.get(ecs_select[disabled: "host", v1: "[@metadata][input][tcp][source][name]"])).to eq('1.2.3.4')
129
+ expect(event.get(ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][tcp][source][ip]" ])).to eq('1.2.3.4')
130
+ expect(event.get(ecs_select[disabled: "port", v1: "[@metadata][input][tcp][source][port]"])).to eq('1234')
131
+ expect(event.get(ecs_select[disabled: "proxy_host", v1: "[@metadata][input][tcp][proxy][ip]" ])).to eq('5.6.7.8')
132
+ expect(event.get(ecs_select[disabled: "proxy_port", v1: "[@metadata][input][tcp][proxy][port]" ])).to eq('5678')
133
+ end
134
+ end
121
135
  end
122
- end
123
136
 
124
- it "should read events with plain codec and ISO-8859-1 charset" do
125
- charset = "ISO-8859-1"
126
- conf = <<-CONFIG
127
- input {
128
- tcp {
129
- port => #{port}
130
- codec => plain { charset => "#{charset}" }
137
+ it "should read events with json codec" do
138
+ conf = <<-CONFIG
139
+ input {
140
+ tcp {
141
+ port => #{port}
142
+ codec => json
143
+ }
131
144
  }
145
+ CONFIG
146
+
147
+ data = {
148
+ "hello" => "world",
149
+ "foo" => [1,2,3],
150
+ "baz" => { "1" => "2" },
151
+ "host" => "example host"
132
152
  }
133
- CONFIG
134
153
 
135
- event = input(conf) do |pipeline, queue|
136
- socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
137
- text = "\xA3" # the £ symbol in ISO-8859-1 aka Latin-1
138
- text.force_encoding("ISO-8859-1")
139
- socket.puts(text)
140
- socket.close
154
+ event = input(conf) do |pipeline, queue|
155
+ socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
156
+ socket.puts(LogStash::Json.dump(data))
157
+ socket.close
141
158
 
142
- queue.pop
143
- end
159
+ queue.pop
160
+ end
144
161
 
145
- # Make sure the 0xA3 latin-1 code converts correctly to UTF-8.
146
- insist { event.get("message").size } == 1
147
- insist { event.get("message").bytesize } == 2
148
- insist { event.get("message") } == "£"
149
- end
162
+ insist { event.get("hello") } == data["hello"]
163
+ insist { event.get("foo").to_a } == data["foo"] # to_a to cast Java ArrayList produced by JrJackson
164
+ insist { event.get("baz") } == data["baz"]
150
165
 
151
- it "should read events with json codec" do
152
- conf = <<-CONFIG
153
- input {
154
- tcp {
155
- port => #{port}
156
- codec => json
166
+ # Make sure the tcp input, w/ json codec, uses the event's 'host' value,
167
+ # if present, instead of providing its own
168
+ insist { event.get("host") } == data["host"]
169
+ end
170
+
171
+ it "should read events with json codec (testing 'host' handling)" do
172
+ conf = <<-CONFIG
173
+ input {
174
+ tcp {
175
+ port => #{port}
176
+ codec => json
177
+ }
157
178
  }
179
+ CONFIG
180
+
181
+ data = {
182
+ "hello" => "world"
158
183
  }
159
- CONFIG
160
184
 
161
- data = {
162
- "hello" => "world",
163
- "foo" => [1,2,3],
164
- "baz" => { "1" => "2" },
165
- "host" => "example host"
166
- }
185
+ event = input(conf) do |pipeline, queue|
186
+ socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
187
+ socket.puts(LogStash::Json.dump(data))
188
+ socket.close
167
189
 
168
- event = input(conf) do |pipeline, queue|
169
- socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
170
- socket.puts(LogStash::Json.dump(data))
171
- socket.close
190
+ queue.pop
191
+ end
172
192
 
173
- queue.pop
193
+ aggregate_failures("event") do
194
+ expect(event.get("hello")).to eq(data["hello"])
195
+ expect(event).to include(ecs_select[disabled: "host", v1: "[@metadata][input][tcp][source][name]"])
196
+ expect(event).to include(ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][tcp][source][ip]" ])
197
+ end
174
198
  end
175
-
176
- insist { event.get("hello") } == data["hello"]
177
- insist { event.get("foo").to_a } == data["foo"] # to_a to cast Java ArrayList produced by JrJackson
178
- insist { event.get("baz") } == data["baz"]
179
-
180
- # Make sure the tcp input, w/ json codec, uses the event's 'host' value,
181
- # if present, instead of providing its own
182
- insist { event.get("host") } == data["host"]
183
199
  end
184
200
 
185
- it "should read events with json codec (testing 'host' handling)" do
201
+ it "should read events with plain codec and ISO-8859-1 charset" do
202
+ charset = "ISO-8859-1"
186
203
  conf = <<-CONFIG
187
- input {
188
- tcp {
189
- port => #{port}
190
- codec => json
204
+ input {
205
+ tcp {
206
+ port => #{port}
207
+ codec => plain { charset => "#{charset}" }
208
+ }
191
209
  }
192
- }
193
210
  CONFIG
194
211
 
195
- data = {
196
- "hello" => "world"
197
- }
198
-
199
212
  event = input(conf) do |pipeline, queue|
200
213
  socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
201
- socket.puts(LogStash::Json.dump(data))
214
+ text = "\xA3" # the £ symbol in ISO-8859-1 aka Latin-1
215
+ text.force_encoding("ISO-8859-1")
216
+ socket.puts(text)
202
217
  socket.close
203
218
 
204
219
  queue.pop
205
220
  end
206
221
 
207
- insist { event.get("hello") } == data["hello"]
208
- insist { event }.include?("host")
222
+ # Make sure the 0xA3 latin-1 code converts correctly to UTF-8.
223
+ aggregate_failures("event") do
224
+ expect(event.get("message")).to have_attributes(size: 1, bytesize: 2, encoding: Encoding.find("UTF-8"))
225
+ expect(event.get("message")).to eq("£")
226
+ end
209
227
  end
210
228
 
211
229
  it "should read events with json_lines codec" do
data/version CHANGED
@@ -1 +1 @@
1
- 6.1.1
1
+ 6.2.0
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-input-tcp
3
3
  version: !ruby/object:Gem::Version
4
- version: 6.1.1
4
+ version: 6.2.0
5
5
  platform: java
6
6
  authors:
7
7
  - Elastic
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-06-18 00:00:00.000000000 Z
11
+ date: 2021-06-28 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  requirement: !ruby/object:Gem::Requirement
@@ -30,6 +30,20 @@ dependencies:
30
30
  - - "<="
31
31
  - !ruby/object:Gem::Version
32
32
  version: '2.99'
33
+ - !ruby/object:Gem::Dependency
34
+ requirement: !ruby/object:Gem::Requirement
35
+ requirements:
36
+ - - "~>"
37
+ - !ruby/object:Gem::Version
38
+ version: '1.2'
39
+ name: logstash-mixin-ecs_compatibility_support
40
+ prerelease: false
41
+ type: :runtime
42
+ version_requirements: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - "~>"
45
+ - !ruby/object:Gem::Version
46
+ version: '1.2'
33
47
  - !ruby/object:Gem::Dependency
34
48
  requirement: !ruby/object:Gem::Requirement
35
49
  requirements:
@@ -211,7 +225,7 @@ files:
211
225
  - logstash-input-tcp.gemspec
212
226
  - spec/inputs/tcp_spec.rb
213
227
  - spec/spec_helper.rb
214
- - vendor/jar-dependencies/org/logstash/inputs/logstash-input-tcp/6.1.1/logstash-input-tcp-6.1.1.jar
228
+ - vendor/jar-dependencies/org/logstash/inputs/logstash-input-tcp/6.2.0/logstash-input-tcp-6.2.0.jar
215
229
  - version
216
230
  homepage: http://www.elastic.co/guide/en/logstash/current/index.html
217
231
  licenses: