fluent-plugin-devo-radiant 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 11587ede4fb7a85c2e3d9994d80514e429ff24785fdde2e61c0ce4fd7621883b
4
- data.tar.gz: b20e719d8d884847df0aacb90fb6b93a0fabbbc04c3997e815b7ef06c6e9ceab
3
+ metadata.gz: e63e8c2d366e9bfb47562256b9c75c08fe93c9d319da00d6efc53708487466dc
4
+ data.tar.gz: 61ec5363d67d6ad9f223fc3e635aed2038bfd8857b798b780cb4fa5e6364c628
5
5
  SHA512:
6
- metadata.gz: 4f1a8ff173b12a657be041f9969df4bc22f69cd33e21ccb0ec72ed27f4cfadab5b9c0c7340ef924d4a2f0fdb6848b923750d98ca71952fd3b57ef16526678769
7
- data.tar.gz: 3b1f47135ecde3d48a20f335688c29c100fc39d58d290dbe39e50c1704dc48debc5d8bd85232cf8e4d9c3303e5f9d1d9e0d308c2e727853cc90bd12bd99675d7
6
+ metadata.gz: 2283adcd685777f9979284cc8e5c8460a0ec1dbb7dabf942a5de3cc43b0924ba56e581260a869f22d7f06fa609a7df9fbec47f66df6b0abc89aec4fe5b594216
7
+ data.tar.gz: ecb118345893239ce363f4d4266995c4a37227020e8adcdde0c4f114d0e052fb4c990a82c500c6f56ac7ea9385e5ffeb0e27e375a1cb86f65f2dbc850d13ed1d
data/CHANGELOG.md ADDED
@@ -0,0 +1,29 @@
1
+ # Changelog
2
+
3
+ ## 0.1.1 — 2026-11-05
4
+
5
+ - Detect severity from level fields, timestamp-prefixed text, and LDAP result
6
+ codes by default; add `auto_severity` to disable detection.
7
+ - Normalize severity aliases, case, and priorities 0–7; unknown values and NONE
8
+ fall back to INFO without failing a chunk.
9
+ - Scrub string keys and values recursively to UTF-8 before formatting, including
10
+ headers. Emit exactly one newline per event and escape embedded CR/LF for
11
+ non-JSON formatters.
12
+ - Preserve the legacy Devo header and inject UTC event time with milliseconds
13
+ into `@timestamp`; add `inject_timestamp` and `timestamp_key`. Preserve
14
+ existing payload timestamp fields and event-time precision through buffering.
15
+ - Add `sanitize_tag`, remove wire delimiters from headers, and warn once for
16
+ changed or out-of-range Devo tags.
17
+ - Close partially constructed TCP/TLS connections on setup, connect, or handshake
18
+ failure; try every resolved address in order with per-address readiness waits.
19
+ - Move connection creation outside the logger-cache lock, close concurrent
20
+ duplicate loggers, and prevent reconnection after shutdown or transport close.
21
+ - Support hashed CA directories without a trailing slash, encrypted client keys
22
+ through secret `client_key_passphrase`, and leaf-first PEM certificate bundles
23
+ with intermediate chains.
24
+ - Warn that `token`, `procid_key`, and `msgid_key` are ignored by this syslog
25
+ format while retaining their configuration compatibility.
26
+ - Add explicit `tls false` for plain TCP relays with an unencrypted-traffic warning.
27
+ - Document unacknowledged writes, partial-chunk duplicates, buffer-capacity drops,
28
+ and Fluentd's rejection of individual records exceeding the chunk limit.
29
+ - Add focused record, framing, TLS/mTLS, cache-race, and lifecycle regressions.
data/README.md CHANGED
@@ -1,23 +1,9 @@
1
1
  # fluent-plugin-devo-radiant
2
2
 
3
- A hardened Fluentd output plugin for sending records to Devo over TLS. This is
4
- a derivative of [DevoInc/fluent-plugin-devo 1.2.0](https://github.com/DevoInc/fluent-plugin-devo)
5
- under the Apache License 2.0.
6
-
7
- ## Changes from upstream
8
-
9
- - Fluentd buffering with retryable chunk failures instead of per-record error swallowing
10
- - Secret masking for `token` and `client_key`
11
- - Multi-worker support and synchronized, worker-aware logger caching
12
- - Peer verification with the system CA store by default
13
- - Explicit warnings if peer or hostname verification is disabled
14
- - Configurable minimum TLS version (`TLSv1_2` by default; `TLSv1_3` is supported)
15
- - Fork-specific namespace for the bundled syslog TLS implementation
16
- - Both `@type devo_radiant` and `@type devo` registrations
17
-
18
- Buffered delivery is at least once. If a chunk fails after some records were
19
- accepted by the remote endpoint, Fluentd retries the chunk and those earlier
20
- records can be delivered again.
3
+ A Fluentd output plugin for sending records to Devo over TLS or an explicitly
4
+ selected plain TCP relay. This is a derivative of
5
+ [DevoInc/fluent-plugin-devo 1.2.0](https://github.com/DevoInc/fluent-plugin-devo)
6
+ under the Apache License 2.0. Both `@type devo_radiant` and `@type devo` are supported.
21
7
 
22
8
  ## Installation
23
9
 
@@ -32,15 +18,23 @@ fluent-gem install fluent-plugin-devo-radiant
32
18
  @type devo_radiant
33
19
  host logs.example.invalid
34
20
  port 443
21
+ hostname fid-0
22
+ app_name_key devo_tag
35
23
 
36
- token YOUR_DEVO_TOKEN
24
+ tls true
37
25
  ca_cert system
38
26
  verify_cert_name true
39
27
  tls_min_version TLSv1_2
40
28
 
41
- # Optional mutual TLS credentials
29
+ # Optional mutual TLS credentials; certificate bundle is leaf first.
42
30
  client_cert /path/to/client.crt
43
31
  client_key /path/to/client.key
32
+ # Supply client_key_passphrase securely if the key is encrypted.
33
+
34
+ auto_severity true
35
+ inject_timestamp true
36
+ timestamp_key @timestamp
37
+ sanitize_tag true
44
38
 
45
39
  <format>
46
40
  @type json
@@ -55,13 +49,102 @@ fluent-gem install fluent-plugin-devo-radiant
55
49
  </match>
56
50
  ```
57
51
 
58
- The buffer chunk key defaults to `tag`, so the explicit `<buffer tag>` argument
59
- above is optional. A persistent file buffer and an intentional retry policy are
60
- recommended for production durability.
61
-
62
- Setting `ca_cert false` disables both certificate-chain and hostname
63
- verification and produces a warning. Setting `verify_cert_name false` disables
64
- hostname matching only and also produces a warning. Both weaken TLS security.
52
+ Set the record's `devo_tag` to a tag such as `my.app.fid`. Without `app_name_key`,
53
+ the Fluentd tag is used as the Devo tag. The buffer chunk key defaults to `tag`,
54
+ so the explicit `<buffer tag>` argument above is optional. A persistent file
55
+ buffer and an intentional retry policy are recommended for production.
56
+
57
+ | Parameter | Default | Behavior |
58
+ | --- | --- | --- |
59
+ | `host`, `port` | Required | Listener address and port. All resolved addresses are tried in order per connection attempt. |
60
+ | `tls` | `true` | TLS transport. `false` selects plain TCP and warns that traffic is unencrypted. |
61
+ | `ca_cert` | `system` | System CA store, CA file, or hashed CA directory (with or without a trailing slash). `false` disables peer and hostname verification and warns. |
62
+ | `verify_cert_name` | `true` | Check server hostname. `false` disables hostname matching and warns when TLS is enabled. |
63
+ | `tls_min_version` | `TLSv1_2` | Minimum TLS version; `TLSv1_3` is accepted when supported by the OpenSSL runtime. No maximum is pinned. |
64
+ | `client_cert` | Unset | PEM client certificate or bundle, with the leaf first followed by intermediate certificates. |
65
+ | `client_key` | Unset | PEM private key, including RSA and EC. Masked in Fluentd configuration output. |
66
+ | `client_key_passphrase` | Unset | Passphrase for an encrypted private key. Masked in configuration output. Missing or incorrect passphrases fail without an interactive prompt. |
67
+ | `idle_timeout` | Unset | Seconds of inactivity before reconnecting on the next write; no background idle sweeper. |
68
+ | `hostname` | Local short hostname | Default syslog hostname. |
69
+ | `facility` | `LOCAL0` | Default syslog facility. |
70
+ | `severity_key` | Unset | Record field with an explicit severity; a missing or null value allows automatic detection. |
71
+ | `auto_severity` | `true` | Detect severity when no explicit value is present. `false` uses INFO in that case. |
72
+ | `inject_timestamp` | `true` | Add the Fluentd event time to the outgoing record as ISO 8601 UTC with milliseconds. Existing keys, including null values, are preserved. |
73
+ | `timestamp_key` | `@timestamp` | Field for the injected timestamp. Non-JSON formatters must include this field to transmit it. |
74
+ | `sanitize_tag` | `true` | Remove characters outside `[A-Za-z0-9.]` and collapse empty tag levels. |
75
+ | `hostname_key`, `facility_key`, `app_name_key` | Unset | Per-record header mappings. |
76
+ | `token`, `procid_key`, `msgid_key` | Unset | Accepted for compatibility, but ignored by this wire format; configuring any of them produces one warning. |
77
+
78
+ `output_type` and the existing Fluentd formatter/inject compatibility settings
79
+ remain accepted. `token` is masked but is **never transmitted**. PROCID, MSGID,
80
+ and structured data are not serialized. Configure authentication for the selected
81
+ listener, such as mutual TLS; setting `token` does not authenticate a connection.
82
+
83
+ ## Severity and record handling
84
+
85
+ An explicit severity is normalized case-insensitively. Numeric priorities 0–7
86
+ and their strings are accepted. FATAL/CRITICAL map to CRIT, SEVERE/ERROR to ERR,
87
+ WARNING to WARN, TRACE/FINE/FINER/FINEST/VERBOSE to DEBUG,
88
+ INFORMATION/INFORMATIONAL to INFO, and EMERGENCY/PANIC to EMERG. Unknown values,
89
+ including NONE, become INFO and do not fail a chunk.
90
+
91
+ Automatic detection checks these record keys in order, ignoring key case:
92
+ `loglevel`, `status`, `severity`, `level`, `syslog.severity`, `log_level`,
93
+ `log.level`, `@l`, `severitytext`, `lvl`, `levelname`. If none has a non-null
94
+ value, it checks the timestamp-prefixed text level, using the same prefix pattern
95
+ as the Dynatrace output. Text fields are checked in priority order: `content`,
96
+ `message`, `payload`, `body`, `log`, `_raw`; the first non-null field is used.
97
+ Finally, LDAP `RESULTCODE` maps 0/5/6/10/14 to INFO, 1/51/52/53/54/80 to ERR,
98
+ and other integer codes to WARN. Unrecognized content defaults to INFO.
99
+
100
+ String keys and values are normalized recursively to UTF-8 before formatting.
101
+ Invalid bytes become U+FFFD; valid UTF-8 binary strings are preserved. The same
102
+ normalization applies to the wire header. Formatter terminal CR/LF is removed,
103
+ then one newline frames each event. Embedded CRLF, CR, and LF in non-JSON
104
+ formatter output become literal `\n`; JSON's own escaping is preserved.
105
+
106
+ The RFC 3164-style header remains `<PRI>Mon DD HH:MM:SS hostname app_name`.
107
+ It does not carry year, timezone, or fractional seconds. The injected payload
108
+ timestamp preserves that information from the **Fluentd event time**; it cannot
109
+ repair a source parser that assigned read time instead of parsing the log time.
110
+
111
+ ## Devo tags
112
+
113
+ [Devo's tag rules](https://devodocs.atlassian.net/wiki/spaces/latest/pages/95126204)
114
+ require 2–6 nonempty dot-separated levels and a maximum of 50 characters, with
115
+ alphanumeric characters and periods. This plugin's sanitizer uses ASCII letters
116
+ and digits. For example, `..my..app.vds_server_access/.` becomes
117
+ `my.app.vdsserveraccess`. Each distinct original tag that changes produces one
118
+ warning showing original and cleaned values. Each distinct tag still outside
119
+ the level/length rules produces one warning and is sent anyway.
120
+
121
+ Even with `sanitize_tag false`, hostname and app_name always have whitespace,
122
+ CR/LF, and control characters removed because they delimit the wire header.
123
+ Sanitizing can make different original tags identical. Prefer explicit valid
124
+ tags and verify that they select the intended Devo parser. Syntax cleanup does
125
+ not establish that a matching parser exists.
126
+
127
+ ## Delivery limits
128
+
129
+ - Syslog over TCP/TLS has no application acknowledgement. A successful socket
130
+ write is not proof that Devo received, stored, or parsed an event.
131
+ - After a partial-chunk failure, Fluentd retries the whole chunk. Previously
132
+ sent records can be duplicated. This is not an exactly-once delivery channel.
133
+ - A buffer configured with `overflow_action drop_oldest_chunk` discards old
134
+ chunks when capacity is reached. Size persistence and retention deliberately,
135
+ and monitor backlog and drops. This plugin does not change Fluentd's default
136
+ buffer overflow policy.
137
+ - A single record larger than `chunk_limit_size` is rejected by Fluentd before
138
+ transmission. The plugin does not split or truncate oversized events or
139
+ impose a separate receiver-specific event limit.
140
+ - Connect and TLS readiness waits are 10 seconds per address/wait; write
141
+ readiness waits are 5 seconds. Slow progress, retries, DNS, and waiting for a
142
+ transport's write lock can extend total elapsed time.
143
+ - Each worker caches a connection per Fluentd tag. High tag cardinality retains
144
+ more connections and warning-cache entries until shutdown.
145
+ - Shutdown prevents new connections and reconnects, and closes cached transports
146
+ and any concurrent logger-creation loser. Delivery errors remain visible to
147
+ Fluentd so buffered failures can follow its retry policy.
65
148
 
66
149
  ## Development
67
150
 
@@ -70,6 +153,12 @@ bundle install
70
153
  bundle exec rake test
71
154
  ```
72
155
 
156
+ The regression receiver uses connected local socket pairs for real TLS/mTLS
157
+ handshakes and wire assertions without a network listener. TCP dial failures,
158
+ address ordering, and readiness timeouts also use controlled boundary doubles.
159
+ Set `DEVO_TEST_TMPDIR` to choose where temporary test credentials and buffers
160
+ are created; the tests remove their own files and stop their own threads.
161
+
73
162
  ## Attribution and license
74
163
 
75
164
  Copyright notices from the upstream project are retained in derived source
@@ -8,7 +8,7 @@
8
8
  module Fluent
9
9
  module Plugin
10
10
  module DevoRadiantConstants
11
- VERSION = '0.1.0'
11
+ VERSION = '0.1.1'
12
12
  end
13
13
  end
14
14
  end
@@ -19,7 +19,10 @@
19
19
 
20
20
  require 'msgpack'
21
21
  require 'socket'
22
+ require 'time'
22
23
  require 'fluent/plugin/output'
24
+ require 'fluent/plugin/formatter_json'
25
+ require 'fluent/msgpack_factory'
23
26
  require 'fluent_plugin_devo_radiant/syslog_tls/logger'
24
27
  require_relative 'devo_radiant_constants'
25
28
 
@@ -33,6 +36,14 @@ module Fluent
33
36
 
34
37
  DEFAULT_FORMAT_TYPE = 'json'
35
38
  TLS_MIN_VERSIONS = FluentPluginDevoRadiant::SyslogTls::SSLTransport::TLS_MIN_VERSIONS.keys.freeze
39
+ Utf8 = FluentPluginDevoRadiant::SyslogTls::Utf8
40
+ Severity = FluentPluginDevoRadiant::SyslogTls::Severity
41
+ CONTENT_KEYS = %w[content message payload body log _raw].freeze
42
+ LEVEL_KEYS = %w[loglevel status severity level syslog.severity log_level log.level @l severitytext lvl levelname].freeze
43
+ TEXT_LEVEL_PATTERN = /\A\[?\d{4}-\d{1,2}-\d{1,2}[T ]
44
+ \d{1,2}:\d{2}(?::\d{2})?(?:[.,]\d+)?(?:Z|[+-]\d{2}:?\d{2})?\]?\s+
45
+ (?:\[[^\]\r\n]{0,128}\]\s+)?
46
+ (TRACE|DEBUG|INFO|NOTICE|WARN|WARNING|ERROR|SEVERE|FATAL|CRITICAL)\b/x.freeze
36
47
 
37
48
  config_param :host, :string
38
49
  config_param :port, :integer
@@ -42,9 +53,15 @@ module Fluent
42
53
  config_param :token, :string, default: nil, secret: true
43
54
  config_param :client_cert, :string, default: nil
44
55
  config_param :client_key, :string, default: nil, secret: true
56
+ config_param :client_key_passphrase, :string, default: nil, secret: true
45
57
  config_param :hostname, :string, default: nil
46
58
  config_param :facility, :string, default: 'LOCAL0'
47
59
  config_param :tls_min_version, :enum, list: TLS_MIN_VERSIONS, default: :'TLSv1_2'
60
+ config_param :tls, :bool, default: true
61
+ config_param :auto_severity, :bool, default: true
62
+ config_param :inject_timestamp, :bool, default: true
63
+ config_param :timestamp_key, :string, default: '@timestamp'
64
+ config_param :sanitize_tag, :bool, default: true
48
65
 
49
66
  SYSLOG_HEADERS = %i[severity facility hostname app_name procid msgid].freeze
50
67
 
@@ -66,6 +83,9 @@ module Fluent
66
83
  super
67
84
  @loggers = {}
68
85
  @loggers_mutex = Mutex.new
86
+ @closing = false
87
+ @tag_warnings = {}
88
+ @tag_warnings_mutex = Mutex.new
69
89
  end
70
90
 
71
91
  def configure(conf)
@@ -82,8 +102,14 @@ module Fluent
82
102
  end
83
103
 
84
104
  @formatter = formatter_create(conf: conf.elements('format').first, default_type: DEFAULT_FORMAT_TYPE)
105
+ @json_format = @formatter.is_a?(Fluent::Plugin::JSONFormatter)
85
106
 
86
- if peer_verification_disabled?
107
+ if token || procid_key || msgid_key
108
+ log.warn 'token, procid_key and msgid_key are ignored by this syslog format'
109
+ end
110
+ if !tls
111
+ log.warn 'TLS is disabled; traffic is unencrypted plain TCP'
112
+ elsif peer_verification_disabled?
87
113
  log.warn 'TLS peer and hostname verification are disabled because ca_cert is false'
88
114
  elsif !verify_cert_name
89
115
  log.warn 'TLS hostname verification is disabled because verify_cert_name is false'
@@ -91,13 +117,23 @@ module Fluent
91
117
  end
92
118
 
93
119
  def shutdown
120
+ close_loggers
121
+ super
122
+ end
123
+
124
+ def close
125
+ close_loggers
126
+ super
127
+ end
128
+
129
+ def close_loggers
94
130
  loggers = @loggers_mutex.synchronize do
131
+ @closing = true
95
132
  current = @loggers.values
96
133
  @loggers = {}
97
134
  current
98
135
  end
99
- loggers.each { |cached_logger| cached_logger.close unless cached_logger.closed? }
100
- super
136
+ loggers.each(&:close)
101
137
  end
102
138
 
103
139
  def prefer_buffered_processing
@@ -114,7 +150,7 @@ module Fluent
114
150
 
115
151
  def format(tag, time, record)
116
152
  record = inject_values_to_record(tag, time, record)
117
- [time, record].to_msgpack
153
+ Fluent::MessagePackFactory.msgpack_packer.pack([time, record]).full_pack
118
154
  end
119
155
 
120
156
  def write(chunk)
@@ -137,33 +173,55 @@ module Fluent
137
173
 
138
174
  def logger(tag)
139
175
  cache_key = [fluentd_worker_id, tag]
140
- @loggers_mutex.synchronize do
176
+ cached_logger = @loggers_mutex.synchronize do
177
+ raise IOError, 'Devo output is closing' if @closing
178
+
179
+ @loggers[cache_key]
180
+ end
181
+ return cached_logger if cached_logger && !cached_logger.closed?
182
+
183
+ candidate = new_logger(tag)
184
+ selected = @loggers_mutex.synchronize do
185
+ raise IOError, 'Devo output is closing' if @closing
186
+
141
187
  cached_logger = @loggers[cache_key]
142
188
  if cached_logger.nil? || cached_logger.closed?
143
- cached_logger = new_logger(tag)
144
- @loggers[cache_key] = cached_logger
189
+ @loggers[cache_key] = candidate
190
+ else
191
+ cached_logger
145
192
  end
146
- cached_logger
147
193
  end
194
+ selected
195
+ ensure
196
+ candidate.close if candidate && !candidate.equal?(selected)
148
197
  end
149
198
 
150
- def new_logger(tag)
151
- transport = FluentPluginDevoRadiant::SyslogTls::SSLTransport.new(
199
+ def new_transport(transport_class = FluentPluginDevoRadiant::SyslogTls::SSLTransport)
200
+ transport_class.new(
152
201
  host,
153
202
  port,
154
203
  idle_timeout: idle_timeout,
155
204
  ca_cert: ca_cert,
156
205
  client_cert: client_cert,
157
206
  client_key: client_key,
207
+ client_key_passphrase: client_key_passphrase,
208
+ tls: tls,
158
209
  verify_cert_name: verify_cert_name,
159
210
  tls_min_version: tls_min_version,
160
211
  max_retries: 3
161
212
  )
213
+ end
214
+
215
+ def new_logger(tag)
216
+ transport = new_transport
162
217
  syslog_logger = FluentPluginDevoRadiant::SyslogTls::Logger.new(transport, token)
163
218
  syslog_logger.facility(facility)
164
219
  syslog_logger.hostname(hostname)
165
220
  syslog_logger.app_name(tag)
166
221
  syslog_logger
222
+ rescue StandardError
223
+ transport&.close
224
+ raise
167
225
  end
168
226
 
169
227
  private
@@ -173,18 +231,79 @@ module Fluent
173
231
  end
174
232
 
175
233
  def send_record(tag, time, record)
176
- severity = if @mappings.key?(:severity)
177
- record[@mappings[:severity]] || 'INFO'
178
- else
179
- 'INFO'
180
- end
181
-
182
- message = @formatter.format(tag, time, record)
183
- logger(tag).log(severity, message, time: Time.at(time)) do |header|
234
+ record = Utf8.value(record)
235
+ event_time = time.is_a?(Fluent::EventTime) ? Time.at(time.sec, time.nsec, :nanosecond) : Time.at(time)
236
+ if inject_timestamp && !record.key?(timestamp_key)
237
+ record[timestamp_key] = event_time.getutc.iso8601(3)
238
+ end
239
+ severity = record_severity(record)
240
+ message = Utf8.string(@formatter.format(tag, time, record)).sub(/[\r\n]+\z/, '')
241
+ message = message.gsub(/\r\n|\r|\n/) { '\\n' } unless @json_format
242
+ logger(tag).log(severity, message, time: event_time) do |header|
184
243
  @mappings.each do |name, record_key|
244
+ next if %i[severity procid msgid].include?(name)
245
+
185
246
  header.public_send(:"#{name}=", record[record_key]) unless record[record_key].nil?
186
247
  end
248
+ header.hostname = Utf8.header_token(header.hostname || hostname)
249
+ header.app_name = devo_tag(record[app_name_key] || tag)
250
+ end
251
+ end
252
+
253
+ def record_severity(record)
254
+ explicit = record[severity_key] if severity_key
255
+ return severity_name(explicit) unless explicit.nil?
256
+ return 'INFO' unless auto_severity
257
+
258
+ present = {}
259
+ record.each do |key, value|
260
+ normalized_key = Utf8.string(key).downcase
261
+ present[normalized_key] = value unless value.nil? || present.key?(normalized_key)
262
+ end
263
+ level_key = LEVEL_KEYS.find { |key| present.key?(key) }
264
+ return severity_name(present[level_key]) if level_key
265
+
266
+ content_key = CONTENT_KEYS.find { |key| present.key?(key) }
267
+ if content_key
268
+ text = Utf8.string(present[content_key].to_s.byteslice(0, 256))
269
+ match = TEXT_LEVEL_PATTERN.match(text)
270
+ return severity_name(match[1]) if match
271
+ end
272
+ result = present['resultcode']
273
+ result = result.strip.to_i if result.is_a?(String) && /\A[0-9]+\z/.match?(result.strip)
274
+ return 'INFO' unless result.is_a?(Integer)
275
+
276
+ case result
277
+ when 0, 5, 6, 10, 14 then 'INFO'
278
+ when 1, 51, 52, 53, 54, 80 then 'ERR'
279
+ else 'WARN'
280
+ end
281
+ end
282
+
283
+ def severity_name(value)
284
+ %w[EMERG ALERT CRIT ERR WARN NOTICE INFO DEBUG][Severity.normalize(value)]
285
+ end
286
+
287
+ def devo_tag(value)
288
+ original = Utf8.string(value)
289
+ cleaned = Utf8.header_token(original)
290
+ if sanitize_tag
291
+ cleaned = cleaned.gsub(/[^A-Za-z0-9.]/, '').split('.').reject(&:empty?).join('.')
292
+ end
293
+ warnings = @tag_warnings_mutex.synchronize do
294
+ next [] if @tag_warnings.key?(original)
295
+
296
+ @tag_warnings[original] = true
297
+ messages = []
298
+ messages << "Devo tag changed: #{original.inspect} -> #{cleaned.inspect}" if original != cleaned
299
+ levels = cleaned.split('.', -1)
300
+ if !(2..6).cover?(levels.length) || levels.any?(&:empty?) || cleaned.length > 50
301
+ messages << "Devo tag outside 2-6 nonempty levels or 50 characters: #{cleaned.inspect}; sending anyway"
302
+ end
303
+ messages
187
304
  end
305
+ warnings.each { |warning| log.warn warning }
306
+ cleaned
188
307
  end
189
308
  end
190
309
  end
@@ -21,6 +21,7 @@ require 'date'
21
21
 
22
22
  require_relative 'facility'
23
23
  require_relative 'severity'
24
+ require_relative 'utf8'
24
25
 
25
26
  module FluentPluginDevoRadiant
26
27
  module SyslogTls
@@ -60,9 +61,7 @@ module FluentPluginDevoRadiant
60
61
  end
61
62
 
62
63
  def severity=(value)
63
- raise ArgumentError, "Invalid severity value: #{value.inspect}" unless SEVERITIES.key?(value)
64
-
65
- @severity = value
64
+ @severity = Severity.normalize(value)
66
65
  end
67
66
 
68
67
  def pri
@@ -72,8 +71,8 @@ module FluentPluginDevoRadiant
72
71
  def assemble
73
72
  [
74
73
  "<#{pri}>#{timestamp.to_datetime.strftime('%b %d %H:%M:%S')}",
75
- hostname,
76
- app_name
74
+ Utf8.header_token(hostname),
75
+ Utf8.header_token(app_name)
77
76
  ].join(' ')
78
77
  end
79
78
 
@@ -121,7 +120,8 @@ module FluentPluginDevoRadiant
121
120
 
122
121
  def assemble
123
122
  output = [header.to_s]
124
- output << msg if msg.length.positive?
123
+ message = Utf8.string(msg).sub(/[\r\n]+\z/, '')
124
+ output << message if message.length.positive?
125
125
  output.join(' ') + "\n"
126
126
  end
127
127
 
@@ -17,6 +17,7 @@
17
17
  # limitations under the License.
18
18
 
19
19
  require_relative 'lookup_from_const'
20
+ require_relative 'utf8'
20
21
 
21
22
  module FluentPluginDevoRadiant
22
23
  module SyslogTls
@@ -26,12 +27,28 @@ module FluentPluginDevoRadiant
26
27
  EMERG = PANIC = 0
27
28
  ALERT = 1
28
29
  CRIT = 2
30
+ FATAL = CRITICAL = CRIT
29
31
  ERR = ERROR = 3
32
+ SEVERE = ERR
30
33
  WARN = WARNING = 4
31
34
  NOTICE = 5
32
35
  INFO = 6
36
+ INFORMATION = INFORMATIONAL = INFO
33
37
  DEBUG = 7
34
- NONE = 10
38
+ TRACE = FINE = FINER = FINEST = VERBOSE = DEBUG
39
+ EMERGENCY = EMERG
40
+ NONE = INFO
41
+
42
+ def self.normalize(value)
43
+ if value.is_a?(Integer) || value.is_a?(Float)
44
+ return value.to_i if (0..7).cover?(value) && value == value.to_i
45
+ elsif value.is_a?(String) || value.is_a?(Symbol)
46
+ name = Utf8.string(value).strip.upcase
47
+ return name.to_i if /\A[0-7]\z/.match?(name)
48
+ return const_get(name, false) if /\A[A-Z_]+\z/.match?(name) && const_defined?(name, false)
49
+ end
50
+ INFO
51
+ end
35
52
  end
36
53
  end
37
54
  end
@@ -35,18 +35,21 @@ module FluentPluginDevoRadiant
35
35
 
36
36
  attr_accessor :socket
37
37
  attr_reader :host, :port, :idle_timeout, :ca_cert, :client_cert, :client_key,
38
- :verify_cert_name, :tls_min_version
38
+ :verify_cert_name, :tls_min_version, :tls
39
39
  attr_writer :retries
40
40
 
41
41
  def initialize(host, port, idle_timeout: nil, ca_cert: 'system', client_cert: nil,
42
42
  client_key: nil, verify_cert_name: true, tls_min_version: :'TLSv1_2',
43
- max_retries: 1)
43
+ max_retries: 1, client_key_passphrase: nil, tls: true)
44
44
  @host = host
45
45
  @port = port
46
46
  @idle_timeout = idle_timeout
47
47
  @ca_cert = ca_cert
48
48
  @client_cert = client_cert
49
49
  @client_key = client_key
50
+ @client_key_passphrase = client_key_passphrase
51
+ @tls = tls
52
+ @closed = false
50
53
  @verify_cert_name = verify_cert_name
51
54
  @tls_min_version = tls_min_version.to_sym
52
55
  @retries = max_retries
@@ -59,62 +62,95 @@ module FluentPluginDevoRadiant
59
62
  end
60
63
 
61
64
  def connect
62
- @socket = get_ssl_connection
63
- begin
65
+ raise IOError, 'Transport is closed' if @closed
66
+
67
+ last_error = nil
68
+ resolved_addresses.each do |address_info|
69
+ candidate = nil
64
70
  begin
65
- @socket.connect_nonblock
66
- rescue Errno::EAGAIN, Errno::EWOULDBLOCK, IO::WaitReadable
67
- select_with_timeout(@socket, :connect_read) && retry
68
- rescue IO::WaitWritable
69
- select_with_timeout(@socket, :connect_write) && retry
71
+ raise IOError, 'Transport is closed' if @closed
72
+
73
+ candidate = tls ? get_ssl_connection(address_info) : get_tcp_connection(address_info)
74
+ if tls
75
+ begin
76
+ candidate.connect_nonblock
77
+ rescue IO::WaitReadable
78
+ select_with_timeout(candidate, :connect_read)
79
+ retry
80
+ rescue IO::WaitWritable
81
+ select_with_timeout(candidate, :connect_write)
82
+ retry
83
+ end
84
+ end
85
+ raise IOError, 'Transport is closed' if @closed
86
+
87
+ @socket = candidate
88
+ @last_write = Time.now if idle_timeout
89
+ return
90
+ rescue StandardError => error
91
+ candidate.close if candidate && !candidate.closed?
92
+ raise if @closed
93
+
94
+ last_error = error
70
95
  end
71
- rescue Errno::ETIMEDOUT
72
- raise 'Socket timeout during connect'
73
96
  end
74
- @last_write = Time.now if idle_timeout
97
+ raise(last_error || SocketError.new('No addresses resolved'))
75
98
  end
76
99
 
77
- def get_tcp_connection
78
- tcp = nil
79
- family = Socket::Constants::AF_UNSPEC
80
- socket_type = Socket::Constants::SOCK_STREAM
81
- address_info = Socket.getaddrinfo(host, port, family, socket_type, nil, nil, false).first
82
- _, resolved_port, _, address, family, socket_type = address_info
100
+ def resolved_addresses
101
+ Socket.getaddrinfo(host, port, Socket::AF_UNSPEC, Socket::SOCK_STREAM, nil, nil, false)
102
+ end
83
103
 
84
- begin
85
- socket_address = Socket.sockaddr_in(resolved_port, address)
86
- tcp = Socket.new(family, socket_type, 0)
87
- tcp.setsockopt(Socket::SOL_SOCKET, Socket::Constants::SO_REUSEADDR, false)
88
- unless RbConfig::CONFIG['host_os'].match?(/mswin|mingw|cygwin/)
89
- tcp.setsockopt(Socket::SOL_SOCKET, Socket::Constants::SO_REUSEPORT, false)
90
- end
91
- tcp.connect_nonblock(socket_address)
92
- rescue Errno::EINPROGRESS
93
- select_with_timeout(tcp, :connect_write)
104
+ def get_tcp_connection(address_info = nil)
105
+ addresses = address_info ? [address_info] : resolved_addresses
106
+ last_error = nil
107
+ addresses.each do |address_info|
108
+ tcp = nil
94
109
  begin
95
- tcp.connect_nonblock(socket_address)
96
- rescue Errno::EISCONN
97
- # Connected.
98
- rescue SystemCallError
99
- tcp.close rescue nil
100
- raise
110
+ _, resolved_port, _, address, family, socket_type = address_info
111
+ socket_address = Socket.sockaddr_in(resolved_port, address)
112
+ tcp = Socket.new(family, socket_type, 0)
113
+ tcp.setsockopt(Socket::SOL_SOCKET, Socket::SO_REUSEADDR, false)
114
+ unless RbConfig::CONFIG['host_os'].match?(/mswin|mingw|cygwin/)
115
+ tcp.setsockopt(Socket::SOL_SOCKET, Socket::SO_REUSEPORT, false)
116
+ end
117
+ begin
118
+ tcp.connect_nonblock(socket_address)
119
+ rescue IO::WaitWritable
120
+ select_with_timeout(tcp, :connect_write)
121
+ error = tcp.getsockopt(Socket::SOL_SOCKET, Socket::SO_ERROR).int
122
+ raise SystemCallError.new('connect', error) unless error.zero?
123
+ end
124
+ tcp.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, true)
125
+ return tcp
126
+ rescue StandardError => error
127
+ tcp&.close
128
+ last_error = error
101
129
  end
102
- rescue SystemCallError
103
- tcp.close rescue nil
104
- raise
105
130
  end
106
-
107
- tcp.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, true)
108
- tcp
131
+ raise(last_error || SocketError.new('No addresses resolved'))
109
132
  end
110
133
 
111
- def get_ssl_connection
112
- tcp = get_tcp_connection
134
+ def get_ssl_connection(address_info = nil)
135
+ tcp = get_tcp_connection(address_info)
113
136
  context = build_ssl_context
114
137
  ssl_socket = OpenSSL::SSL::SSLSocket.new(tcp, context)
115
- ssl_socket.hostname = host
116
138
  ssl_socket.sync_close = true
139
+ ssl_socket.hostname = host
117
140
  ssl_socket
141
+ rescue StandardError
142
+ ssl_socket&.close
143
+ tcp.close if tcp && !tcp.closed?
144
+ raise
145
+ end
146
+
147
+ def closed?
148
+ @closed || @socket.nil? || @socket.closed?
149
+ end
150
+
151
+ def close
152
+ @closed = true
153
+ @write_mutex.synchronize { close_socket }
118
154
  end
119
155
 
120
156
  def write(string)
@@ -127,6 +163,8 @@ module FluentPluginDevoRadiant
127
163
  sent = 0
128
164
  begin
129
165
  sent = @socket.write_nonblock(data)
166
+ rescue IO::WaitReadable
167
+ select_with_timeout(@socket, :write_read) && retry
130
168
  rescue OpenSSL::SSL::SSLError, Errno::EAGAIN, Errno::EWOULDBLOCK, IO::WaitWritable => error
131
169
  if error.is_a?(OpenSSL::SSL::SSLError) && error.message !~ /write would block/
132
170
  raise error
@@ -149,6 +187,8 @@ module FluentPluginDevoRadiant
149
187
  [nil, [tcp], nil, CONNECT_TIMEOUT]
150
188
  when :write
151
189
  [nil, [tcp], nil, WRITE_TIMEOUT]
190
+ when :write_read
191
+ [[tcp], nil, nil, WRITE_TIMEOUT]
152
192
  else
153
193
  raise "Unknown select type #{type}"
154
194
  end
@@ -180,14 +220,24 @@ module FluentPluginDevoRadiant
180
220
  when false, 'false'
181
221
  context.verify_hostname = false
182
222
  context.verify_mode = OpenSSL::SSL::VERIFY_NONE
183
- when %r{/$}
184
- context.ca_path = ca_cert
185
223
  when String
186
- context.ca_file = ca_cert
224
+ if File.directory?(ca_cert)
225
+ context.ca_path = ca_cert
226
+ else
227
+ context.ca_file = ca_cert
228
+ end
187
229
  end
188
230
 
189
- context.cert = OpenSSL::X509::Certificate.new(File.read(client_cert)) if client_cert
190
- context.key = OpenSSL::PKey.read(File.read(client_key)) if client_key
231
+ if client_cert
232
+ certificates = File.read(client_cert).scan(/-----BEGIN CERTIFICATE-----.*?-----END CERTIFICATE-----/m)
233
+ raise OpenSSL::X509::CertificateError, 'Client certificate bundle is empty' if certificates.empty?
234
+
235
+ context.cert, *chain = certificates.map { |pem| OpenSSL::X509::Certificate.new(pem) }
236
+ context.extra_chain_cert = chain unless chain.empty?
237
+ end
238
+ if client_key
239
+ context.key = OpenSSL::PKey.read(File.read(client_key), @client_key_passphrase) { @client_key_passphrase }
240
+ end
191
241
  context
192
242
  end
193
243
 
@@ -195,26 +245,29 @@ module FluentPluginDevoRadiant
195
245
  context.min_version = TLS_MIN_VERSIONS.fetch(tls_min_version)
196
246
  end
197
247
 
248
+ def close_socket
249
+ @socket.close if @socket && !@socket.closed?
250
+ end
251
+
198
252
  def write_with_retries(string)
199
- if idle_timeout
200
- if (now = Time.now) > @last_write + idle_timeout
201
- @socket.close rescue nil
202
- connect
203
- else
204
- @last_write = now
205
- end
206
- end
253
+ raise IOError, 'Transport is closed' if @closed
207
254
 
255
+ retry_id = 0
208
256
  begin
209
- retry_id ||= 0
210
- do_write(string)
211
- rescue StandardError => error
212
- if (retry_id += 1) < @retries
213
- @socket.close rescue nil
257
+ raise IOError, 'Transport is closed' if @closed
258
+
259
+ if @socket.nil? || @socket.closed? || (idle_timeout && Time.now > @last_write + idle_timeout)
260
+ close_socket
214
261
  connect
215
- retry
216
262
  end
217
- raise error
263
+ do_write(string)
264
+ @last_write = Time.now if idle_timeout
265
+ rescue StandardError
266
+ close_socket
267
+ retry_id += 1
268
+ retry if !@closed && retry_id < @retries
269
+
270
+ raise
218
271
  end
219
272
  end
220
273
  end
@@ -0,0 +1,49 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2016 Acquia, Inc.
4
+ # Copyright 2016-2019 t.e.morgan.
5
+ # Copyright 2021 JMC
6
+ # Copyright 2026 Radiant Logic, Inc.
7
+ #
8
+ # Licensed under the Apache License, Version 2.0 (the "License");
9
+ # you may not use this file except in compliance with the License.
10
+ # You may obtain a copy of the License at
11
+ #
12
+ # http://www.apache.org/licenses/LICENSE-2.0
13
+ #
14
+ # Unless required by applicable law or agreed to in writing, software
15
+ # distributed under the License is distributed on an "AS IS" BASIS,
16
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
17
+ # See the License for the specific language governing permissions and
18
+ # limitations under the License.
19
+
20
+ module FluentPluginDevoRadiant
21
+ module SyslogTls
22
+ module Utf8
23
+ module_function
24
+
25
+ def string(value)
26
+ text = value.to_s
27
+ if text.encoding == Encoding::BINARY || text.encoding == Encoding::UTF_8
28
+ text.dup.force_encoding(Encoding::UTF_8).scrub("\uFFFD")
29
+ else
30
+ text.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "\uFFFD")
31
+ end
32
+ end
33
+
34
+ def value(item)
35
+ case item
36
+ when String then string(item)
37
+ when Hash
38
+ item.each_with_object({}) { |(key, entry), result| result[value(key)] = value(entry) }
39
+ when Array then item.map { |entry| value(entry) }
40
+ else item
41
+ end
42
+ end
43
+
44
+ def header_token(value)
45
+ string(value).gsub(/[[:space:][:cntrl:]]/, '')
46
+ end
47
+ end
48
+ end
49
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-devo-radiant
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Radiant Logic, Inc.
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-09-03 00:00:00.000000000 Z
11
+ date: 2026-11-05 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: fluentd
@@ -87,6 +87,7 @@ executables: []
87
87
  extensions: []
88
88
  extra_rdoc_files: []
89
89
  files:
90
+ - CHANGELOG.md
90
91
  - LICENSE
91
92
  - NOTICE
92
93
  - README.md
@@ -99,6 +100,7 @@ files:
99
100
  - lib/fluent_plugin_devo_radiant/syslog_tls/protocol.rb
100
101
  - lib/fluent_plugin_devo_radiant/syslog_tls/severity.rb
101
102
  - lib/fluent_plugin_devo_radiant/syslog_tls/ssl_transport.rb
103
+ - lib/fluent_plugin_devo_radiant/syslog_tls/utf8.rb
102
104
  - lib/fluent_plugin_devo_radiant/syslog_tls/version.rb
103
105
  homepage: https://github.com/gnanirahulnutakki/fluent-plugin-devo-radiant
104
106
  licenses: