logstash-input-tcp 6.1.1-java → 6.2.0-java

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9b37d806eedc118d3a553f449a68e90d9366e258b0484c5e1aad54e0e6939883
4
- data.tar.gz: cae6d73f16e144241b588a3eb9b2c7703ac42ccb0d26a266736ad3d5d5d3f4bb
3
+ metadata.gz: 29b44274cfe25c623273407b63c29125143a6ec704c274732b73e9a3fc39f085
4
+ data.tar.gz: 4bced499147e26e9f8257d2c7a489a7dae93ec2a1c06faf2b45b3d9af801d222
5
5
  SHA512:
6
- metadata.gz: bf8d08b9b60d6268b2e0625b175048c0def7aed442661c89415797cbbcf6409e54d25bad0b4d7cd1d6dc8816fa3b4d9b79ebb78e60a8344fca7f62451637b283
7
- data.tar.gz: fe74219396c110829a71fbdfcbb36d07d1c8e4d06a58af00093e43741b62aad3d34638bbb3d19e9888e8d5edb642edd59110865b4a39c2a1ef624797c4f66225
6
+ metadata.gz: d45ef4273d94e4dc3dbcc94749d0e8d907ea96acb20ee4aeff94251a5390bb26b51105fc2f6104e09b885dc195f9fbe94261047035f905884d8306649e8aec49
7
+ data.tar.gz: f00ff1165183ce006a12541b7836bb3a09496eb4370bca185104ea81c8fc8e2360aae8ed09eeb170ab93ee9e64e7233af9360e4cbb73fc5d9b49adf27d26534b
data/CHANGELOG.md CHANGED
@@ -1,3 +1,8 @@
1
+ ## 6.2.0
2
+ - Added ECS Compatibility Mode [#165](https://github.com/logstash-plugins/logstash-input-tcp/pull/165)
3
+ - When operating in an ECS Compatibility mode, metadata about the connection on which we are receiving data is nested in well-named fields under `[@metadata][input][tcp]` instead of at the root level.
4
+ - Fix: source address is no longer missing when a proxy is present
5
+
1
6
  ## 6.1.1
2
7
  - Changed jar dependencies to reflect newer versions [#179](https://github.com/logstash-plugins/logstash-input-http/pull/179)
3
8
 
data/docs/index.asciidoc CHANGED
@@ -70,6 +70,52 @@ event timestamp
70
70
  }
71
71
  }
72
72
 
73
+ [id="plugins-{type}s-{plugin}-ecs_metadata"]
74
+ ==== Event Metadata and the Elastic Common Schema (ECS)
75
+
76
+ In addition to decoding the events, this input will add metadata about the TCP connection itself to each event.
77
+ This can be helpful when applications are configured to send events directly to this input's TCP listener without including information about themselves.
78
+
79
+ Historically, this metadata was added to a variety of non-standard top-level fields, which had the potential to create confusion and schema conflicts downstream.
80
+ With ECS compatibility mode, we can ensure a pipeline still has access to this metadata throughout the event's lifecycle without polluting the top-level namespace.
81
+
82
+ [cols="3,7,5"]
83
+ |=======================================================================
84
+ | Metadata Group | ecs: `v1`, `v8` | ecs: `disabled`
85
+
86
+ .3+|Source Metadata from the TCP connection
87
+ on which events are being received, including
88
+ the sender's name, ip, and outbound port. l|[@metadata][input][tcp][source][name] l|[host]
89
+ l|[@metadata][input][tcp][source][ip] l|[@metadata][ip_address]
90
+ l|[@metadata][input][tcp][source][port] l|[port]
91
+
92
+ .2+|Proxy Metadata from a proxied TCP connection.
93
+ Available when receiving events by proxy and
94
+ `proxy_protocol => true` l|[@metadata][input][tcp][proxy][ip] l|[proxy_host]
95
+ l|[@metadata][input][tcp][proxy][port] l|[proxy_port]
96
+
97
+ .1+|SSL Subject Metadata from a secured TCP
98
+ connection. Available when `ssl_enable => true`
99
+ AND `ssl_verify => true` l|[@metadata][input][tcp][ssl][subject] l|[sslsubject]
100
+ |=======================================================================
101
+
102
+ For example, the Elastic Common Schema reserves the https://www.elastic.co/guide/en/ecs/current/ecs-host.html[top-level `host` field] for information about the host on which the event happened.
103
+ If an event is missing this metadata, it can be copied into place from the source TCP connection metadata that has been added to the event:
104
+
105
+ [source,txt]
106
+ -----
107
+ filter {
108
+ if [@metadata][input][tcp][source] and not [host] {
109
+ mutate {
110
+ copy {
111
+ "[@metadata][input][tcp][source][name]" => "[host][name]"
112
+ "[@metadata][input][tcp][source][ip]" => "[host][ip]"
113
+ }
114
+ }
115
+ }
116
+ }
117
+ -----
118
+
73
119
  [id="plugins-{type}s-{plugin}-options"]
74
120
  ==== Tcp Input Configuration Options
75
121
 
@@ -79,6 +125,7 @@ This plugin supports the following configuration options plus the <<plugins-{typ
79
125
  |=======================================================================
80
126
  |Setting |Input type|Required
81
127
  | <<plugins-{type}s-{plugin}-dns_reverse_lookup_enabled>> |<<boolean,boolean>>|No
128
+ | <<plugins-{type}s-{plugin}-ecs_compatibility>> | <<string,string>>|No
82
129
  | <<plugins-{type}s-{plugin}-host>> |<<string,string>>|No
83
130
  | <<plugins-{type}s-{plugin}-mode>> |<<string,string>>, one of `["server", "client"]`|No
84
131
  | <<plugins-{type}s-{plugin}-port>> |<<number,number>>|Yes
@@ -108,6 +155,20 @@ It is possible to avoid DNS reverse-lookups by disabling this setting. If disabl
108
155
  the address metadata that is added to events will contain the source address as-specified
109
156
  at the TCP layer and IPs will not be resolved to hostnames.
110
157
 
158
+ [id="plugins-{type}s-{plugin}-ecs_compatibility"]
159
+ ===== `ecs_compatibility`
160
+
161
+ * Value type is <<string,string>>
162
+ * Supported values are:
163
+ ** `disabled`: unstructured connection metadata added at root level
164
+ ** `v1`,`v8`: structured connection metadata added under `[@metadata][input][tcp]`
165
+ * Default value depends on which version of Logstash is running:
166
+ ** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
167
+ ** Otherwise, the default value is `disabled`.
168
+
169
+ Controls this plugin's compatibility with the https://www.elastic.co/guide/en/ecs/current/index.html[Elastic Common Schema (ECS)].
170
+ The value of this setting affects the <<plugins-{type}s-{plugin}-ecs_metadata,placement of a TCP connection's metadata>> on events.
171
+
111
172
  [id="plugins-{type}s-{plugin}-host"]
112
173
  ===== `host`
113
174
 
@@ -5,7 +5,7 @@ require "java"
5
5
  require "logstash/inputs/base"
6
6
  require "logstash/util/socket_peer"
7
7
  require "logstash-input-tcp_jars"
8
- require "logstash/inputs/tcp/decoder_impl"
8
+ require 'logstash/plugin_mixins/ecs_compatibility_support'
9
9
 
10
10
  require "socket"
11
11
  require "openssl"
@@ -63,6 +63,11 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
63
63
  java_import 'org.logstash.tcp.InputLoop'
64
64
  java_import 'org.logstash.tcp.SslContextBuilder'
65
65
 
66
+ require_relative "tcp/decoder_impl"
67
+
68
+ # ecs_compatibility option, provided by Logstash core or the support adapter.
69
+ include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled, :v1, :v8 => :v1)
70
+
66
71
  config_name "tcp"
67
72
 
68
73
  default :codec, "line"
@@ -113,13 +118,6 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
113
118
  # Option to allow users to avoid DNS Reverse Lookup.
114
119
  config :dns_reverse_lookup_enabled, :validate => :boolean, :default => true
115
120
 
116
- HOST_FIELD = "host".freeze
117
- HOST_IP_FIELD = "[@metadata][ip_address]".freeze
118
- PORT_FIELD = "port".freeze
119
- PROXY_HOST_FIELD = "proxy_host".freeze
120
- PROXY_PORT_FIELD = "proxy_port".freeze
121
- SSLSUBJECT_FIELD = "sslsubject".freeze
122
-
123
121
  # Monkey patch TCPSocket and SSLSocket to include socket peer
124
122
  # @private
125
123
  def self.patch_socket_peer!
@@ -134,6 +132,8 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
134
132
  def initialize(*args)
135
133
  super(*args)
136
134
 
135
+ setup_fields!
136
+
137
137
  self.class.patch_socket_peer!
138
138
 
139
139
  # threadsafe socket bookkeeping
@@ -186,8 +186,8 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
186
186
  proxy_port, tbuf, socket)
187
187
  codec.decode(tbuf) do |event|
188
188
  if @proxy_protocol
189
- event.set(PROXY_HOST_FIELD, proxy_address) unless event.get(PROXY_HOST_FIELD)
190
- event.set(PROXY_PORT_FIELD, proxy_port) unless event.get(PROXY_PORT_FIELD)
189
+ event.set(@field_proxy_host, proxy_address) unless event.get(@field_proxy_host)
190
+ event.set(@field_proxy_port, proxy_port) unless event.get(@field_proxy_port)
191
191
  end
192
192
  enqueue_decorated(event, client_ip_address, client_address, client_port, socket)
193
193
  end
@@ -260,14 +260,24 @@ class LogStash::Inputs::Tcp < LogStash::Inputs::Base
260
260
  end
261
261
 
262
262
  def enqueue_decorated(event, client_ip_address, client_address, client_port, socket)
263
- event.set(HOST_FIELD, client_address) unless event.get(HOST_FIELD)
264
- event.set(HOST_IP_FIELD, client_ip_address) unless event.get(HOST_IP_FIELD)
265
- event.set(PORT_FIELD, client_port) unless event.get(PORT_FIELD)
266
- event.set(SSLSUBJECT_FIELD, socket.peer_cert.subject.to_s) if socket && @ssl_enable && @ssl_verify && event.get(SSLSUBJECT_FIELD).nil?
263
+ event.set(@field_host, client_address) unless event.get(@field_host)
264
+ event.set(@field_host_ip, client_ip_address) unless event.get(@field_host_ip)
265
+ event.set(@field_port, client_port) unless event.get(@field_port)
266
+ event.set(@field_sslsubject, socket.peer_cert.subject.to_s) if socket && @ssl_enable && @ssl_verify && event.get(@field_sslsubject).nil?
267
267
  decorate(event)
268
268
  @output_queue << event
269
269
  end
270
270
 
271
+ # setup the field names, with respect to ECS compatibility.
272
+ def setup_fields!
273
+ @field_host = ecs_select[disabled: "host", v1: "[@metadata][input][tcp][source][name]" ].freeze
274
+ @field_host_ip = ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][tcp][source][ip]" ].freeze
275
+ @field_port = ecs_select[disabled: "port", v1: "[@metadata][input][tcp][source][port]" ].freeze
276
+ @field_proxy_host = ecs_select[disabled: "proxy_host", v1: "[@metadata][input][tcp][proxy][ip]" ].freeze
277
+ @field_proxy_port = ecs_select[disabled: "proxy_port", v1: "[@metadata][input][tcp][proxy][port]" ].freeze
278
+ @field_sslsubject = ecs_select[disabled: "sslsubject", v1: "[@metadata][input][tcp][tls][client][subject]"].freeze
279
+ end
280
+
271
281
  def server?
272
282
  @mode == "server"
273
283
  end
@@ -1,7 +1,7 @@
1
1
  # encoding: utf-8
2
2
  require 'java'
3
3
 
4
- class DecoderImpl
4
+ class LogStash::Inputs::Tcp::DecoderImpl
5
5
 
6
6
  include org.logstash.tcp.Decoder
7
7
 
@@ -24,7 +24,7 @@ class DecoderImpl
24
24
  end
25
25
 
26
26
  def copy
27
- DecoderImpl.new(@codec.clone, @tcp)
27
+ self.class.new(@codec.clone, @tcp)
28
28
  end
29
29
 
30
30
  def flush
@@ -41,16 +41,17 @@ class DecoderImpl
41
41
  @tcp.logger.error("Invalid proxy protocol header label", :header => pp_hdr)
42
42
  raise IOError.new("Invalid proxy protocol header label #{pp_hdr.inspect}")
43
43
  else
44
- @proxy_address = pp_info[3]
45
- @proxy_port = pp_info[5]
46
- @address = pp_info[2]
47
- @port = pp_info[4]
44
+ @proxy_address = pp_info[3] # layer 3 destination address (proxy's receiving address)
45
+ @proxy_port = pp_info[5] # TCP destination port (proxy's receiving port)
46
+ @ip_address = pp_info[2] # layer 3 source address (outgoing ip of sender)
47
+ @address = extract_host_name(@ip_address)
48
+ @port = pp_info[4] # TCP source port (outgoing port on sender [probably random])
48
49
  end
49
50
  else
50
51
  filtered = received
51
- @ip_address = channel_addr.get_address.get_host_address
52
- @address = extract_host_name(channel_addr)
53
- @port = channel_addr.get_port
52
+ @ip_address = channel_addr.get_address.get_host_address # ip address of sender
53
+ @address = extract_host_name(channel_addr) # name _or_ address of sender
54
+ @port = channel_addr.get_port # outgoing port of sender (probably random)
54
55
  end
55
56
  @first_read = false
56
57
  filtered
@@ -58,6 +59,8 @@ class DecoderImpl
58
59
 
59
60
  private
60
61
  def extract_host_name(channel_addr)
62
+ channel_addr = java.net.InetSocketAddress.new(channel_addr, 0) if channel_addr.kind_of?(String)
63
+
61
64
  return channel_addr.get_host_string unless @tcp.dns_reverse_lookup_enabled?
62
65
 
63
66
  channel_addr.get_host_name
@@ -21,6 +21,7 @@ Gem::Specification.new do |s|
21
21
 
22
22
  # Gem dependencies
23
23
  s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
24
+ s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.2'
24
25
 
25
26
  s.add_runtime_dependency 'logstash-core', '>= 6.7.0'
26
27
 
@@ -15,9 +15,11 @@ java_import "io.netty.handler.ssl.util.SelfSignedCertificate"
15
15
 
16
16
  require_relative "../spec_helper"
17
17
 
18
+ require 'logstash/plugin_mixins/ecs_compatibility_support/spec_helper'
19
+
18
20
  #Cabin::Channel.get(LogStash).subscribe(STDOUT)
19
21
  #Cabin::Channel.get(LogStash).level = :debug
20
- describe LogStash::Inputs::Tcp do
22
+ describe LogStash::Inputs::Tcp, :ecs_compatibility_support do
21
23
 
22
24
  def get_port
23
25
  begin
@@ -52,160 +54,176 @@ describe LogStash::Inputs::Tcp do
52
54
  end
53
55
  end
54
56
 
55
- it "should read plain with unicode" do
56
- event_count = 10
57
- conf = <<-CONFIG
58
- input {
59
- tcp {
60
- port => #{port}
57
+ ecs_compatibility_matrix(:disabled,:v1, :v8 => :v1) do |ecs_select|
58
+ before(:each) do
59
+ allow_any_instance_of(described_class).to receive(:ecs_compatibility).and_return(ecs_compatibility)
60
+ end
61
+
62
+ it "should read plain with unicode" do
63
+ event_count = 10
64
+ conf = <<-CONFIG
65
+ input {
66
+ tcp {
67
+ port => #{port}
68
+ }
61
69
  }
62
- }
63
- CONFIG
70
+ CONFIG
71
+
72
+ host = 'localhost'
73
+ events = input(conf) do |pipeline, queue|
74
+ socket = Stud::try(5.times) { TCPSocket.new(host, port) }
75
+ event_count.times do |i|
76
+ # unicode smiley for testing unicode support!
77
+ socket.puts("#{i} ☹")
78
+ socket.flush
79
+ end
80
+ socket.close
64
81
 
65
- host = 'localhost'
66
- events = input(conf) do |pipeline, queue|
67
- socket = Stud::try(5.times) { TCPSocket.new(host, port) }
68
- event_count.times do |i|
69
- # unicode smiley for testing unicode support!
70
- socket.puts("#{i} ☹")
71
- socket.flush
82
+ event_count.times.collect {queue.pop}
72
83
  end
73
- socket.close
74
84
 
75
- event_count.times.collect {queue.pop}
76
- end
85
+ expect(events.length).to eq(event_count)
86
+ events = events.sort_by {|e| e.get("message")} # the ordering of events in the queue is highly timing-dependent
87
+ event_count.times do |i|
88
+ event = events[i]
77
89
 
78
- insist { events.length } == event_count
79
- events = events.sort_by {|e| e.get("message")} # the ordering of events in the queue is highly timing-dependent
80
- event_count.times do |i|
81
- event = events[i]
82
- insist { event.get("message") } == "#{i} ☹"
83
- insist { ["localhost","ip6-localhost"].includes? event.get("host") }
84
- insist { event.get("[@metadata][ip_address]") } == '127.0.0.1'
90
+ aggregate_failures("event #{i}") do
91
+ expect(event.get("message")).to eq("#{i} ☹")
92
+ expect(event.get(ecs_select[disabled: "host", v1: "[@metadata][input][tcp][source][name]"])).to eq("localhost").or eq("ip6-localhost")
93
+ expect(event.get(ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][tcp][source][ip]"])).to eq('127.0.0.1')
94
+ end
95
+ end
85
96
  end
86
- end
87
97
 
88
- it "should handle PROXY protocol v1 connections" do
89
- event_count = 10
90
- conf = <<-CONFIG
91
- input {
92
- tcp {
93
- proxy_protocol => true
94
- port => '#{port}'
98
+ it "should handle PROXY protocol v1 connections" do
99
+ event_count = 10
100
+ conf = <<-CONFIG
101
+ input {
102
+ tcp {
103
+ proxy_protocol => true
104
+ port => '#{port}'
105
+ }
95
106
  }
96
- }
97
- CONFIG
107
+ CONFIG
98
108
 
99
- events = input(conf) do |pipeline, queue|
100
- socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
101
- socket.puts("PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678\r");
102
- socket.flush
103
- event_count.times do |i|
104
- # unicode smiley for testing unicode support!
105
- socket.puts("#{i} ☹")
109
+ events = input(conf) do |pipeline, queue|
110
+ socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
111
+ socket.puts("PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678\r");
106
112
  socket.flush
107
- end
108
- socket.close
113
+ event_count.times do |i|
114
+ # unicode smiley for testing unicode support!
115
+ socket.puts("#{i} ☹")
116
+ socket.flush
117
+ end
118
+ socket.close
109
119
 
110
- event_count.times.collect {queue.pop}
111
- end
120
+ event_count.times.collect {queue.pop}
121
+ end
112
122
 
113
- insist { events.length } == event_count
114
- events = events.sort_by {|e| e.get("message")} # the ordering of events in the queue is highly timing-dependent
115
- event_count.times do |i|
116
- insist { events[i].get("message") } == "#{i} ☹"
117
- insist { events[i].get("host") } == "1.2.3.4"
118
- insist { events[i].get("port") } == "1234"
119
- insist { events[i].get("proxy_host") } == "5.6.7.8"
120
- insist { events[i].get("proxy_port") } == "5678"
123
+ expect(events.length).to eq(event_count)
124
+ events = events.sort_by {|e| e.get("message")} # the ordering of events in the queue is highly timing-dependent
125
+ events.each_with_index do |event, i|
126
+ aggregate_failures("event #{i}") do
127
+ expect(event.get("message")).to eq("#{i} ")
128
+ expect(event.get(ecs_select[disabled: "host", v1: "[@metadata][input][tcp][source][name]"])).to eq('1.2.3.4')
129
+ expect(event.get(ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][tcp][source][ip]" ])).to eq('1.2.3.4')
130
+ expect(event.get(ecs_select[disabled: "port", v1: "[@metadata][input][tcp][source][port]"])).to eq('1234')
131
+ expect(event.get(ecs_select[disabled: "proxy_host", v1: "[@metadata][input][tcp][proxy][ip]" ])).to eq('5.6.7.8')
132
+ expect(event.get(ecs_select[disabled: "proxy_port", v1: "[@metadata][input][tcp][proxy][port]" ])).to eq('5678')
133
+ end
134
+ end
121
135
  end
122
- end
123
136
 
124
- it "should read events with plain codec and ISO-8859-1 charset" do
125
- charset = "ISO-8859-1"
126
- conf = <<-CONFIG
127
- input {
128
- tcp {
129
- port => #{port}
130
- codec => plain { charset => "#{charset}" }
137
+ it "should read events with json codec" do
138
+ conf = <<-CONFIG
139
+ input {
140
+ tcp {
141
+ port => #{port}
142
+ codec => json
143
+ }
131
144
  }
145
+ CONFIG
146
+
147
+ data = {
148
+ "hello" => "world",
149
+ "foo" => [1,2,3],
150
+ "baz" => { "1" => "2" },
151
+ "host" => "example host"
132
152
  }
133
- CONFIG
134
153
 
135
- event = input(conf) do |pipeline, queue|
136
- socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
137
- text = "\xA3" # the £ symbol in ISO-8859-1 aka Latin-1
138
- text.force_encoding("ISO-8859-1")
139
- socket.puts(text)
140
- socket.close
154
+ event = input(conf) do |pipeline, queue|
155
+ socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
156
+ socket.puts(LogStash::Json.dump(data))
157
+ socket.close
141
158
 
142
- queue.pop
143
- end
159
+ queue.pop
160
+ end
144
161
 
145
- # Make sure the 0xA3 latin-1 code converts correctly to UTF-8.
146
- insist { event.get("message").size } == 1
147
- insist { event.get("message").bytesize } == 2
148
- insist { event.get("message") } == "£"
149
- end
162
+ insist { event.get("hello") } == data["hello"]
163
+ insist { event.get("foo").to_a } == data["foo"] # to_a to cast Java ArrayList produced by JrJackson
164
+ insist { event.get("baz") } == data["baz"]
150
165
 
151
- it "should read events with json codec" do
152
- conf = <<-CONFIG
153
- input {
154
- tcp {
155
- port => #{port}
156
- codec => json
166
+ # Make sure the tcp input, w/ json codec, uses the event's 'host' value,
167
+ # if present, instead of providing its own
168
+ insist { event.get("host") } == data["host"]
169
+ end
170
+
171
+ it "should read events with json codec (testing 'host' handling)" do
172
+ conf = <<-CONFIG
173
+ input {
174
+ tcp {
175
+ port => #{port}
176
+ codec => json
177
+ }
157
178
  }
179
+ CONFIG
180
+
181
+ data = {
182
+ "hello" => "world"
158
183
  }
159
- CONFIG
160
184
 
161
- data = {
162
- "hello" => "world",
163
- "foo" => [1,2,3],
164
- "baz" => { "1" => "2" },
165
- "host" => "example host"
166
- }
185
+ event = input(conf) do |pipeline, queue|
186
+ socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
187
+ socket.puts(LogStash::Json.dump(data))
188
+ socket.close
167
189
 
168
- event = input(conf) do |pipeline, queue|
169
- socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
170
- socket.puts(LogStash::Json.dump(data))
171
- socket.close
190
+ queue.pop
191
+ end
172
192
 
173
- queue.pop
193
+ aggregate_failures("event") do
194
+ expect(event.get("hello")).to eq(data["hello"])
195
+ expect(event).to include(ecs_select[disabled: "host", v1: "[@metadata][input][tcp][source][name]"])
196
+ expect(event).to include(ecs_select[disabled: "[@metadata][ip_address]", v1: "[@metadata][input][tcp][source][ip]" ])
197
+ end
174
198
  end
175
-
176
- insist { event.get("hello") } == data["hello"]
177
- insist { event.get("foo").to_a } == data["foo"] # to_a to cast Java ArrayList produced by JrJackson
178
- insist { event.get("baz") } == data["baz"]
179
-
180
- # Make sure the tcp input, w/ json codec, uses the event's 'host' value,
181
- # if present, instead of providing its own
182
- insist { event.get("host") } == data["host"]
183
199
  end
184
200
 
185
- it "should read events with json codec (testing 'host' handling)" do
201
+ it "should read events with plain codec and ISO-8859-1 charset" do
202
+ charset = "ISO-8859-1"
186
203
  conf = <<-CONFIG
187
- input {
188
- tcp {
189
- port => #{port}
190
- codec => json
204
+ input {
205
+ tcp {
206
+ port => #{port}
207
+ codec => plain { charset => "#{charset}" }
208
+ }
191
209
  }
192
- }
193
210
  CONFIG
194
211
 
195
- data = {
196
- "hello" => "world"
197
- }
198
-
199
212
  event = input(conf) do |pipeline, queue|
200
213
  socket = Stud::try(5.times) { TCPSocket.new("127.0.0.1", port) }
201
- socket.puts(LogStash::Json.dump(data))
214
+ text = "\xA3" # the £ symbol in ISO-8859-1 aka Latin-1
215
+ text.force_encoding("ISO-8859-1")
216
+ socket.puts(text)
202
217
  socket.close
203
218
 
204
219
  queue.pop
205
220
  end
206
221
 
207
- insist { event.get("hello") } == data["hello"]
208
- insist { event }.include?("host")
222
+ # Make sure the 0xA3 latin-1 code converts correctly to UTF-8.
223
+ aggregate_failures("event") do
224
+ expect(event.get("message")).to have_attributes(size: 1, bytesize: 2, encoding: Encoding.find("UTF-8"))
225
+ expect(event.get("message")).to eq("£")
226
+ end
209
227
  end
210
228
 
211
229
  it "should read events with json_lines codec" do
data/version CHANGED
@@ -1 +1 @@
1
- 6.1.1
1
+ 6.2.0
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-input-tcp
3
3
  version: !ruby/object:Gem::Version
4
- version: 6.1.1
4
+ version: 6.2.0
5
5
  platform: java
6
6
  authors:
7
7
  - Elastic
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-06-18 00:00:00.000000000 Z
11
+ date: 2021-06-28 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  requirement: !ruby/object:Gem::Requirement
@@ -30,6 +30,20 @@ dependencies:
30
30
  - - "<="
31
31
  - !ruby/object:Gem::Version
32
32
  version: '2.99'
33
+ - !ruby/object:Gem::Dependency
34
+ requirement: !ruby/object:Gem::Requirement
35
+ requirements:
36
+ - - "~>"
37
+ - !ruby/object:Gem::Version
38
+ version: '1.2'
39
+ name: logstash-mixin-ecs_compatibility_support
40
+ prerelease: false
41
+ type: :runtime
42
+ version_requirements: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - "~>"
45
+ - !ruby/object:Gem::Version
46
+ version: '1.2'
33
47
  - !ruby/object:Gem::Dependency
34
48
  requirement: !ruby/object:Gem::Requirement
35
49
  requirements:
@@ -211,7 +225,7 @@ files:
211
225
  - logstash-input-tcp.gemspec
212
226
  - spec/inputs/tcp_spec.rb
213
227
  - spec/spec_helper.rb
214
- - vendor/jar-dependencies/org/logstash/inputs/logstash-input-tcp/6.1.1/logstash-input-tcp-6.1.1.jar
228
+ - vendor/jar-dependencies/org/logstash/inputs/logstash-input-tcp/6.2.0/logstash-input-tcp-6.2.0.jar
215
229
  - version
216
230
  homepage: http://www.elastic.co/guide/en/logstash/current/index.html
217
231
  licenses: