semantic_logger 5.0.0 → 5.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +9 -0
- data/Rakefile +29 -0
- data/docs/api.md +480 -0
- data/docs/appenders.md +990 -0
- data/docs/config.md +605 -0
- data/docs/index.md +197 -0
- data/docs/log.md +108 -0
- data/docs/metrics.md +168 -0
- data/docs/operations.md +400 -0
- data/docs/rails.md +816 -0
- data/docs/security.md +119 -0
- data/docs/testing.md +285 -0
- data/docs/upgrading.md +409 -0
- data/lib/semantic_logger/appender/cloudwatch_logs.rb +4 -2
- data/lib/semantic_logger/appender/elasticsearch.rb +2 -5
- data/lib/semantic_logger/appender/elasticsearch_base.rb +17 -17
- data/lib/semantic_logger/appender/elasticsearch_http.rb +14 -7
- data/lib/semantic_logger/appender/file.rb +5 -0
- data/lib/semantic_logger/appender/new_relic.rb +6 -0
- data/lib/semantic_logger/appender/new_relic_logs.rb +2 -1
- data/lib/semantic_logger/appender/open_telemetry.rb +2 -1
- data/lib/semantic_logger/appender/opensearch.rb +2 -1
- data/lib/semantic_logger/appender/rabbitmq.rb +4 -3
- data/lib/semantic_logger/appender/sentry.rb +5 -0
- data/lib/semantic_logger/appender/syslog.rb +6 -3
- data/lib/semantic_logger/appender/tcp.rb +4 -3
- data/lib/semantic_logger/appender/wrapper.rb +2 -1
- data/lib/semantic_logger/appender.rb +6 -3
- data/lib/semantic_logger/appenders.rb +2 -1
- data/lib/semantic_logger/base.rb +0 -2
- data/lib/semantic_logger/formatters/color.rb +5 -2
- data/lib/semantic_logger/formatters/default.rb +7 -2
- data/lib/semantic_logger/formatters/logfmt.rb +4 -2
- data/lib/semantic_logger/formatters/pattern.rb +3 -1
- data/lib/semantic_logger/formatters/syslog.rb +2 -1
- data/lib/semantic_logger/formatters/syslog_cee.rb +2 -1
- data/lib/semantic_logger/metric/signalfx.rb +4 -2
- data/lib/semantic_logger/queue_processor.rb +5 -2
- data/lib/semantic_logger/reporters/minitest.rb +4 -4
- data/lib/semantic_logger/test/capture_log_events.rb +1 -1
- data/lib/semantic_logger/version.rb +1 -1
- metadata +14 -3
data/docs/upgrading.md
ADDED
|
@@ -0,0 +1,409 @@
|
|
|
1
|
+
---
|
|
2
|
+
layout: default
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
## Upgrading
|
|
6
|
+
{:.no_toc}
|
|
7
|
+
|
|
8
|
+
**Contents**
|
|
9
|
+
|
|
10
|
+
* TOC
|
|
11
|
+
{:toc}
|
|
12
|
+
|
|
13
|
+
### Upgrading to Semantic Logger v5.1
|
|
14
|
+
|
|
15
|
+
v5.1 deprecates several legacy appenders and one appender option. They all still work in v5.1, but
|
|
16
|
+
each now emits a Ruby deprecation warning and will be **removed in v6**. The warnings use
|
|
17
|
+
`category: :deprecated`, so they are silent by default and appear when deprecation warnings are
|
|
18
|
+
enabled (for example `ruby -W:deprecated`, or Rails development mode). Switch away now to be ready
|
|
19
|
+
for v6.
|
|
20
|
+
|
|
21
|
+
#### Replace the `:sentry` appender with `:sentry_ruby`
|
|
22
|
+
|
|
23
|
+
The `:sentry` appender depends on the end-of-life `sentry-raven` gem. Use the `:sentry_ruby`
|
|
24
|
+
appender, backed by the maintained `sentry-ruby` gem.
|
|
25
|
+
|
|
26
|
+
In your `Gemfile`:
|
|
27
|
+
|
|
28
|
+
~~~ruby
|
|
29
|
+
# Remove:
|
|
30
|
+
gem "sentry-raven"
|
|
31
|
+
# Add:
|
|
32
|
+
gem "sentry-ruby"
|
|
33
|
+
~~~
|
|
34
|
+
|
|
35
|
+
When adding the appender:
|
|
36
|
+
|
|
37
|
+
~~~ruby
|
|
38
|
+
# Before:
|
|
39
|
+
SemanticLogger.add_appender(appender: :sentry)
|
|
40
|
+
|
|
41
|
+
# After:
|
|
42
|
+
SemanticLogger.add_appender(appender: :sentry_ruby)
|
|
43
|
+
~~~
|
|
44
|
+
|
|
45
|
+
#### Replace the `:new_relic` appender with `:new_relic_logs`
|
|
46
|
+
|
|
47
|
+
The `:new_relic` appender reports `:error` and `:fatal` entries to New Relic as **error events** via
|
|
48
|
+
`newrelic_rpm`. It is superseded by `:new_relic_logs`, which forwards log entries to New Relic Logs:
|
|
49
|
+
|
|
50
|
+
~~~ruby
|
|
51
|
+
# Before:
|
|
52
|
+
SemanticLogger.add_appender(appender: :new_relic)
|
|
53
|
+
|
|
54
|
+
# After:
|
|
55
|
+
SemanticLogger.add_appender(appender: :new_relic_logs)
|
|
56
|
+
~~~
|
|
57
|
+
|
|
58
|
+
Note that the behaviour differs. `:new_relic` created New Relic *error events* (and defaulted to the
|
|
59
|
+
`:error` level), whereas `:new_relic_logs` forwards *log messages* to New Relic's log ingestion. Set
|
|
60
|
+
`level:` to match what you want forwarded, and update any New Relic dashboards or alerts that relied
|
|
61
|
+
on the old error events. Both appenders require the `newrelic_rpm` gem.
|
|
62
|
+
|
|
63
|
+
#### Remove the `type:` option from the Elasticsearch and OpenSearch appenders
|
|
64
|
+
|
|
65
|
+
Document `_type` has been unused since Elasticsearch 7, so the `type:` option (on `:elasticsearch`,
|
|
66
|
+
`:elasticsearch_http`, and `:opensearch`) is now ignored on all server versions: the bulk appenders
|
|
67
|
+
no longer send a `_type`, and the `:elasticsearch_http` appender posts to the typeless `_doc`
|
|
68
|
+
endpoint. Passing `type:` logs a deprecation warning. Remove it:
|
|
69
|
+
|
|
70
|
+
~~~ruby
|
|
71
|
+
# Before:
|
|
72
|
+
SemanticLogger.add_appender(appender: :elasticsearch, url: "http://localhost:9200", type: "log")
|
|
73
|
+
|
|
74
|
+
# After:
|
|
75
|
+
SemanticLogger.add_appender(appender: :elasticsearch, url: "http://localhost:9200")
|
|
76
|
+
~~~
|
|
77
|
+
|
|
78
|
+
### Upgrading to Semantic Logger v5.0
|
|
79
|
+
|
|
80
|
+
#### Minimum Ruby version is now 3.2
|
|
81
|
+
|
|
82
|
+
Semantic Logger v5 requires Ruby 3.2 or later. Earlier Ruby versions are end-of-life and are no
|
|
83
|
+
longer supported or tested.
|
|
84
|
+
|
|
85
|
+
If you are on an older Ruby, upgrade Ruby before upgrading Semantic Logger, or stay on the v4.x
|
|
86
|
+
series.
|
|
87
|
+
|
|
88
|
+
#### `SemanticLogger::Appender::AsyncBatch` has been removed
|
|
89
|
+
|
|
90
|
+
The internal asynchronous proxy classes have been consolidated. Batch processing now runs through
|
|
91
|
+
the same `SemanticLogger::Appender::Async` proxy (backed by an internal `QueueProcessor`), so the
|
|
92
|
+
separate `SemanticLogger::Appender::AsyncBatch` class no longer exists.
|
|
93
|
+
|
|
94
|
+
This only affects code that referenced the class directly, which is uncommon since appenders are
|
|
95
|
+
added through `SemanticLogger.add_appender`. The `batch:`, `batch_size:`, and `batch_seconds:`
|
|
96
|
+
options are unchanged:
|
|
97
|
+
|
|
98
|
+
~~~ruby
|
|
99
|
+
SemanticLogger.add_appender(appender: :http, url: "https://example.com/log", batch: true)
|
|
100
|
+
~~~
|
|
101
|
+
|
|
102
|
+
What changed:
|
|
103
|
+
|
|
104
|
+
- `SemanticLogger.add_appender(..., batch: true)` now returns a `SemanticLogger::Appender::Async`
|
|
105
|
+
(with `#batch?` returning `true`) instead of a `SemanticLogger::Appender::AsyncBatch`.
|
|
106
|
+
- Referencing the constant `SemanticLogger::Appender::AsyncBatch` now raises `NameError`.
|
|
107
|
+
|
|
108
|
+
If you have a custom appender or test asserting on the proxy class, change
|
|
109
|
+
`instance_of?`/`is_a?(SemanticLogger::Appender::AsyncBatch)` checks to
|
|
110
|
+
`SemanticLogger::Appender::Async` and, if needed, check `appender.batch?`.
|
|
111
|
+
|
|
112
|
+
#### Appenders are reopened automatically after fork
|
|
113
|
+
|
|
114
|
+
Previously, applications that fork (Puma, Unicorn, Spring, Resque, `Process.daemon`, etc.) had to
|
|
115
|
+
call `SemanticLogger.reopen` themselves in an after-fork hook, otherwise the child shared the
|
|
116
|
+
parent's file handles and background thread.
|
|
117
|
+
|
|
118
|
+
In v5 this happens automatically: a `Process._fork` / `Process.daemon` hook calls
|
|
119
|
+
`SemanticLogger.reopen` in the child process. The call is guarded to run once per process.
|
|
120
|
+
|
|
121
|
+
For most applications you can now **remove** the manual reopen hook, for example:
|
|
122
|
+
|
|
123
|
+
~~~ruby
|
|
124
|
+
# No longer needed in v5 — handled automatically:
|
|
125
|
+
before_fork { SemanticLogger.flush }
|
|
126
|
+
after_fork { SemanticLogger.reopen }
|
|
127
|
+
~~~
|
|
128
|
+
|
|
129
|
+
If you have a reason to manage this yourself (for example you reopen at a very specific point in
|
|
130
|
+
your boot sequence), disable the automatic behaviour:
|
|
131
|
+
|
|
132
|
+
~~~ruby
|
|
133
|
+
SemanticLogger.reopen_on_fork = false
|
|
134
|
+
~~~
|
|
135
|
+
|
|
136
|
+
#### Recommended: enable logger caching
|
|
137
|
+
|
|
138
|
+
By default `SemanticLogger[SomeClass]` returns a **new** `Logger` instance on every call. This means
|
|
139
|
+
that if one part of the code obtains a logger and later changes its `level` (or filter), other
|
|
140
|
+
holders of "the same" logger do not see the change, because they hold different instances.
|
|
141
|
+
|
|
142
|
+
v5 adds opt-in caching so that a `Class` or `Module` maps to a single shared `Logger` instance:
|
|
143
|
+
|
|
144
|
+
~~~ruby
|
|
145
|
+
SemanticLogger.cache_loggers = true
|
|
146
|
+
~~~
|
|
147
|
+
|
|
148
|
+
With caching enabled:
|
|
149
|
+
|
|
150
|
+
- `SemanticLogger[MyClass]` and the `Loggable` mixin's `MyClass.logger` return the **same** instance.
|
|
151
|
+
- Changing that logger's level or filter is visible to every holder.
|
|
152
|
+
- Strings (`SemanticLogger["Name"]`) always get a fresh instance, and anonymous classes (no `name`)
|
|
153
|
+
are never cached.
|
|
154
|
+
|
|
155
|
+
Enabling caching is recommended for most applications. It is opt-in (default `false`) to preserve
|
|
156
|
+
existing behaviour for code that relied on getting an independent logger per call. If you need to
|
|
157
|
+
discard cached loggers (for example in tests, or after redefining a class), call
|
|
158
|
+
`SemanticLogger.clear_logger_cache`.
|
|
159
|
+
|
|
160
|
+
#### Control characters are escaped in Syslog output
|
|
161
|
+
|
|
162
|
+
To prevent log-injection via embedded control characters (newlines, escape sequences, etc.), the
|
|
163
|
+
Syslog formatter now escapes control characters in records by default. If you relied on raw control
|
|
164
|
+
characters reaching syslog, this output now differs. The text formatters (default, color) also gain
|
|
165
|
+
an opt-in `escape_control_chars` option (default `false`) for the same protection. See the
|
|
166
|
+
[Security](security.html) page for details.
|
|
167
|
+
|
|
168
|
+
#### `Formatters::Base#cleanse` renamed
|
|
169
|
+
|
|
170
|
+
The internal `SemanticLogger::Formatters::Base#cleanse` method was renamed to
|
|
171
|
+
`#escape_control_characters`. This only affects custom formatters or subclasses that called the old
|
|
172
|
+
method name directly; update them to call `#escape_control_characters`.
|
|
173
|
+
|
|
174
|
+
#### Rails: appenders are now configured in one block
|
|
175
|
+
|
|
176
|
+
If you use [rails_semantic_logger](rails.html), the companion gem's v5 release moves appender
|
|
177
|
+
configuration into a single `config.rails_semantic_logger.appenders do |appenders| ... end` block.
|
|
178
|
+
The v4 options (`format`, `add_file_appender`, `ap_options`, `filter`, `console_logger`) still work
|
|
179
|
+
but are deprecated and will be removed in v6. See
|
|
180
|
+
[Migrating from v4 to v5](rails.html#migrating-from-v4-to-v5) for the full before/after mapping.
|
|
181
|
+
|
|
182
|
+
### Upgrading to Semantic Logger v4.18
|
|
183
|
+
|
|
184
|
+
#### Async queue is now bounded by default
|
|
185
|
+
|
|
186
|
+
The default `max_queue_size` for async appenders changed from `-1` (unbounded) to `10_000`.
|
|
187
|
+
Under sustained high log volume, messages that exceed the queue limit will be dropped rather than
|
|
188
|
+
causing memory to grow without bound.
|
|
189
|
+
|
|
190
|
+
If your application logs at very high throughput and you were relying on the unbounded default,
|
|
191
|
+
set the limit explicitly when adding each appender:
|
|
192
|
+
|
|
193
|
+
~~~ruby
|
|
194
|
+
SemanticLogger.add_appender(io: $stdout, async: true, max_queue_size: -1)
|
|
195
|
+
~~~
|
|
196
|
+
|
|
197
|
+
Alternatively, tune the value to match your application's tolerance for log loss vs. memory usage.
|
|
198
|
+
|
|
199
|
+
#### Invalid exception values are now wrapped
|
|
200
|
+
|
|
201
|
+
Previously, passing a non-`Exception` object to the `exception:` keyword argument would silently
|
|
202
|
+
misbehave. It is now wrapped in an `ArgumentError` with a description of the invalid value and a
|
|
203
|
+
backtrace pointing to the call site:
|
|
204
|
+
|
|
205
|
+
~~~ruby
|
|
206
|
+
# This previously caused undefined behaviour — now raises a descriptive ArgumentError
|
|
207
|
+
logger.error("Something went wrong", exception: "a plain string")
|
|
208
|
+
~~~
|
|
209
|
+
|
|
210
|
+
#### New Relic formatter — trace and span metadata
|
|
211
|
+
|
|
212
|
+
Previously, New Relic trace metadata (`trace.id`, `span.id`, `entity.name`, etc.) was captured at
|
|
213
|
+
format time, so it worked with any appender type. In v4.18, this metadata is captured via an
|
|
214
|
+
`on_log` callback that is only registered when `SemanticLogger::Appender::NewRelicLogs` is
|
|
215
|
+
instantiated.
|
|
216
|
+
|
|
217
|
+
If you use a plain IO or File appender with `SemanticLogger::Formatters::NewRelicLogs` as the
|
|
218
|
+
formatter (rather than the dedicated `NewRelicLogs` appender), you must register the callback
|
|
219
|
+
manually in your initializer, otherwise `trace.id`, `span.id`, and `entity.name` will be absent
|
|
220
|
+
from all logs:
|
|
221
|
+
|
|
222
|
+
~~~ruby
|
|
223
|
+
require "semantic_logger/appender/new_relic_logs"
|
|
224
|
+
SemanticLogger.on_log(SemanticLogger::Appender::NewRelicLogs::CAPTURE_CONTEXT)
|
|
225
|
+
~~~
|
|
226
|
+
|
|
227
|
+
This line should be added before any log messages are emitted.
|
|
228
|
+
|
|
229
|
+
---
|
|
230
|
+
|
|
231
|
+
### Upgrading to Semantic Logger v4.17
|
|
232
|
+
|
|
233
|
+
#### New Relic formatter (breaking changes for New Relic users)
|
|
234
|
+
|
|
235
|
+
The New Relic log formatter (`SemanticLogger::Formatters::NewRelicLogs`) has been significantly
|
|
236
|
+
updated to produce a structure that New Relic can natively parse and index as individual
|
|
237
|
+
attributes. Most users will benefit from richer log correlation in New Relic, but there are
|
|
238
|
+
several breaking changes to be aware of.
|
|
239
|
+
|
|
240
|
+
##### Log output structure
|
|
241
|
+
|
|
242
|
+
Previously, all semantic fields (`payload`, `tags`, `named_tags`, `metric`, etc.) were serialized
|
|
243
|
+
as a JSON string inside the `message` key. They are now emitted as top-level keys:
|
|
244
|
+
|
|
245
|
+
Before (v4.16):
|
|
246
|
+
~~~json
|
|
247
|
+
{
|
|
248
|
+
"message": "{\"message\":\"Order processed\",\"payload\":{\"order_id\":123},\"tags\":[],\"named_tags\":{}}",
|
|
249
|
+
"timestamp": 1234567890000,
|
|
250
|
+
"log.level": "INFO"
|
|
251
|
+
}
|
|
252
|
+
~~~
|
|
253
|
+
|
|
254
|
+
After (v4.17):
|
|
255
|
+
~~~json
|
|
256
|
+
{
|
|
257
|
+
"message": "Order processed",
|
|
258
|
+
"payload": { "order_id": 123 },
|
|
259
|
+
"timestamp": 1234567890000,
|
|
260
|
+
"logger": { "name": "OrderService" },
|
|
261
|
+
"thread": { "name": "main" }
|
|
262
|
+
}
|
|
263
|
+
~~~
|
|
264
|
+
|
|
265
|
+
**Impact**: Any New Relic alert queries, dashboards, or NRQL that references fields inside the
|
|
266
|
+
old `message` JSON string (e.g. `message.payload.order_id`) must be updated to reference the
|
|
267
|
+
new top-level keys (e.g. `payload.order_id`).
|
|
268
|
+
|
|
269
|
+
Note that New Relic's log ingestion flattens nested objects. A `payload` value that is itself a
|
|
270
|
+
nested Hash may not be fully indexed as individually searchable attributes. If you need payload
|
|
271
|
+
fields to be queryable in New Relic, promote them to `named_tags` instead:
|
|
272
|
+
|
|
273
|
+
~~~ruby
|
|
274
|
+
# Payload fields may not be individually queryable in New Relic:
|
|
275
|
+
logger.info "Order processed", order_id: 123, amount: 49.99
|
|
276
|
+
|
|
277
|
+
# Use named_tags for fields you want to query directly in New Relic:
|
|
278
|
+
SemanticLogger.tagged(order_id: 123, amount: 49.99) do
|
|
279
|
+
logger.info "Order processed"
|
|
280
|
+
end
|
|
281
|
+
~~~
|
|
282
|
+
|
|
283
|
+
##### named_tags are now merged to the top level
|
|
284
|
+
|
|
285
|
+
Previously `named_tags` were nested inside the `message` JSON string. They are now merged
|
|
286
|
+
directly into the top-level log entry, making them first-class attributes in New Relic.
|
|
287
|
+
|
|
288
|
+
If a named tag key conflicts with an existing top-level key (e.g. `message`, `timestamp`), it is
|
|
289
|
+
dropped and the conflicting key names are recorded under `named_tag_conflicts`.
|
|
290
|
+
|
|
291
|
+
##### Error, logger, and thread fields restructured
|
|
292
|
+
|
|
293
|
+
Before (v4.16):
|
|
294
|
+
~~~json
|
|
295
|
+
{
|
|
296
|
+
"error.message": "Something went wrong",
|
|
297
|
+
"error.class": "RuntimeError",
|
|
298
|
+
"error.stack": "...",
|
|
299
|
+
"log.level": "ERROR",
|
|
300
|
+
"logger.name": "OrderService",
|
|
301
|
+
"thread.name": "main"
|
|
302
|
+
}
|
|
303
|
+
~~~
|
|
304
|
+
|
|
305
|
+
After (v4.17):
|
|
306
|
+
~~~json
|
|
307
|
+
{
|
|
308
|
+
"error": { "message": "Something went wrong", "class": "RuntimeError", "stack": "..." },
|
|
309
|
+
"logger": { "name": "OrderService" },
|
|
310
|
+
"thread": { "name": "main" }
|
|
311
|
+
}
|
|
312
|
+
~~~
|
|
313
|
+
|
|
314
|
+
Update any New Relic alert conditions or NRQL queries that reference the old dot-notation keys.
|
|
315
|
+
|
|
316
|
+
#### Suppressing timestamps in formatters
|
|
317
|
+
|
|
318
|
+
v4.17 added `:notime` as an explicit `time_format` value to suppress timestamp output. If you
|
|
319
|
+
were previously passing `time_format: nil` expecting it to suppress output, use `:notime` instead:
|
|
320
|
+
|
|
321
|
+
~~~ruby
|
|
322
|
+
SemanticLogger.add_appender(io: $stdout, formatter: MyFormatter.new(time_format: :notime))
|
|
323
|
+
~~~
|
|
324
|
+
|
|
325
|
+
---
|
|
326
|
+
|
|
327
|
+
### Upgrading to Semantic Logger v4.9
|
|
328
|
+
|
|
329
|
+
These changes should not be noticeable by the majority of users of Semantic Logger, since
|
|
330
|
+
they are to the internal API. It is possible that advanced users may be using these internal
|
|
331
|
+
API's directly.
|
|
332
|
+
|
|
333
|
+
This does not affect any calls to the public api `SemanticLogger.add_appender`.
|
|
334
|
+
|
|
335
|
+
File and IO are now separate appenders. When creating the File appender explicitly, its arguments
|
|
336
|
+
have changed. For example, when requesting an IO stream, it needs to be changed from:
|
|
337
|
+
|
|
338
|
+
~~~ruby
|
|
339
|
+
SemanticLogger::Appender::File.new(io: $stderr)
|
|
340
|
+
~~~
|
|
341
|
+
to:
|
|
342
|
+
~~~ruby
|
|
343
|
+
SemanticLogger::Appender::IO.new($stderr)
|
|
344
|
+
~~~
|
|
345
|
+
|
|
346
|
+
Additionally, this needs to be changed from:
|
|
347
|
+
~~~ruby
|
|
348
|
+
SemanticLogger::Appender::File.new(file_name: "file.log")
|
|
349
|
+
~~~
|
|
350
|
+
to:
|
|
351
|
+
~~~ruby
|
|
352
|
+
SemanticLogger::Appender::File.new("file.log")
|
|
353
|
+
~~~
|
|
354
|
+
|
|
355
|
+
Rails Semantic Logger, if used, needs to be upgraded to v4.9 when upgrading to Semantic Logger v4.9.
|
|
356
|
+
|
|
357
|
+
---
|
|
358
|
+
|
|
359
|
+
### Upgrading to Semantic Logger v4.4
|
|
360
|
+
|
|
361
|
+
With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
|
|
362
|
+
workaround for Ruby 2.5 crashes is no longer needed.
|
|
363
|
+
I.e. Please remove the following line if being called anywhere:
|
|
364
|
+
|
|
365
|
+
~~~ruby
|
|
366
|
+
SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
|
|
367
|
+
~~~
|
|
368
|
+
|
|
369
|
+
---
|
|
370
|
+
|
|
371
|
+
### Upgrading to Semantic Logger v4.0
|
|
372
|
+
|
|
373
|
+
The following changes need to be made when upgrading to V4:
|
|
374
|
+
- Ruby V2.3 / JRuby V9.1 is now the minimum runtime version.
|
|
375
|
+
- Replace calls to `Logger#with_payload` with `SemanticLogger.named_tagged`.
|
|
376
|
+
- Replace calls to `Logger#payload` with `SemanticLogger.named_tags`.
|
|
377
|
+
- MongoDB Appender requires Mongo Ruby Client V2 or greater.
|
|
378
|
+
- Appenders now write payload data in a separate `:payload` tag instead of mixing them
|
|
379
|
+
directly into the root elements to avoid name clashes.
|
|
380
|
+
|
|
381
|
+
As a result any calls like the following:
|
|
382
|
+
|
|
383
|
+
~~~ruby
|
|
384
|
+
logger.debug foo: 'foo', bar: 'bar'
|
|
385
|
+
~~~
|
|
386
|
+
|
|
387
|
+
Must be replaced with the following in v4:
|
|
388
|
+
|
|
389
|
+
~~~ruby
|
|
390
|
+
logger.debug payload: {foo: 'foo', bar: 'bar'}
|
|
391
|
+
~~~
|
|
392
|
+
|
|
393
|
+
Similarly, for measure blocks:
|
|
394
|
+
|
|
395
|
+
~~~ruby
|
|
396
|
+
logger.measure_info('How long is the sleep', foo: 'foo', bar: 'bar') { sleep 1 }
|
|
397
|
+
~~~
|
|
398
|
+
|
|
399
|
+
Must be replaced with the following in v4:
|
|
400
|
+
|
|
401
|
+
~~~ruby
|
|
402
|
+
logger.measure_info('How long is the sleep', payload: {foo: 'foo', bar: 'bar'}) { sleep 1 }
|
|
403
|
+
~~~
|
|
404
|
+
|
|
405
|
+
The common log call has not changed, and the payload is still logged directly:
|
|
406
|
+
|
|
407
|
+
~~~ruby
|
|
408
|
+
logger.debug('log this', foo: 'foo', bar: 'bar')
|
|
409
|
+
~~~
|
|
@@ -2,7 +2,8 @@ begin
|
|
|
2
2
|
require "aws-sdk-cloudwatchlogs"
|
|
3
3
|
rescue LoadError
|
|
4
4
|
raise LoadError,
|
|
5
|
-
|
|
5
|
+
"Gem aws-sdk-cloudwatchlogs is required for logging to CloudWatch Logs. " \
|
|
6
|
+
'Please add the gem "aws-sdk-cloudwatchlogs" to your Gemfile.'
|
|
6
7
|
end
|
|
7
8
|
|
|
8
9
|
require "concurrent"
|
|
@@ -20,7 +21,8 @@ require "concurrent"
|
|
|
20
21
|
module SemanticLogger
|
|
21
22
|
module Appender
|
|
22
23
|
class CloudwatchLogs < SemanticLogger::Subscriber
|
|
23
|
-
attr_reader :client_kwargs, :group, :create_group, :create_stream,
|
|
24
|
+
attr_reader :client_kwargs, :group, :create_group, :create_stream,
|
|
25
|
+
:force_flush_interval_seconds, :max_buffered_events,
|
|
24
26
|
:task, :client, :buffered_logs
|
|
25
27
|
|
|
26
28
|
# Create CloudWatch Logs Appender
|
|
@@ -2,7 +2,8 @@ begin
|
|
|
2
2
|
require "elasticsearch"
|
|
3
3
|
rescue LoadError
|
|
4
4
|
raise LoadError,
|
|
5
|
-
|
|
5
|
+
"Gem elasticsearch is required for logging to Elasticsearch. " \
|
|
6
|
+
'Please add the gem "elasticsearch" to your Gemfile.'
|
|
6
7
|
end
|
|
7
8
|
|
|
8
9
|
require "semantic_logger/appender/elasticsearch_base"
|
|
@@ -26,10 +27,6 @@ module SemanticLogger
|
|
|
26
27
|
def client_class
|
|
27
28
|
::Elasticsearch::Client
|
|
28
29
|
end
|
|
29
|
-
|
|
30
|
-
def version_supports_type?
|
|
31
|
-
Gem::Version.new(::Elasticsearch::VERSION) < Gem::Version.new(7)
|
|
32
|
-
end
|
|
33
30
|
end
|
|
34
31
|
end
|
|
35
32
|
end
|
|
@@ -6,14 +6,13 @@ module SemanticLogger
|
|
|
6
6
|
#
|
|
7
7
|
# Implements the shared bulk-indexing pipeline used by both the
|
|
8
8
|
# {Elasticsearch} and {OpenSearch} appenders. Subclasses only need to
|
|
9
|
-
# supply the backing client class
|
|
10
|
-
# version still supports document `_type`).
|
|
9
|
+
# supply the backing client class.
|
|
11
10
|
#
|
|
12
11
|
# This class is internal: applications add an appender via
|
|
13
12
|
# `SemanticLogger.add_appender(appender: :elasticsearch, ...)` or
|
|
14
13
|
# `appender: :opensearch`, never by referencing this class directly.
|
|
15
14
|
class ElasticsearchBase < SemanticLogger::Subscriber
|
|
16
|
-
attr_accessor :url, :index, :date_pattern, :
|
|
15
|
+
attr_accessor :url, :index, :date_pattern, :client, :flush_interval, :timeout_interval, :batch_size,
|
|
17
16
|
:client_args
|
|
18
17
|
|
|
19
18
|
# Create an Elasticsearch-compatible appender over persistent HTTP(S).
|
|
@@ -31,9 +30,9 @@ module SemanticLogger
|
|
|
31
30
|
# Default: '%Y.%m.%d'
|
|
32
31
|
#
|
|
33
32
|
# type: [String]
|
|
34
|
-
#
|
|
35
|
-
#
|
|
36
|
-
#
|
|
33
|
+
# Deprecated and ignored. Document `_type` has been unused since
|
|
34
|
+
# Elasticsearch 7 and is ignored by OpenSearch. Passing it logs a
|
|
35
|
+
# deprecation warning; it will be removed in v6.
|
|
37
36
|
#
|
|
38
37
|
# level: [:trace | :debug | :info | :warn | :error | :fatal]
|
|
39
38
|
# Override the log level for this appender.
|
|
@@ -124,7 +123,7 @@ module SemanticLogger
|
|
|
124
123
|
def initialize(url: "http://localhost:9200",
|
|
125
124
|
index: "semantic_logger",
|
|
126
125
|
date_pattern: "%Y.%m.%d",
|
|
127
|
-
type:
|
|
126
|
+
type: nil,
|
|
128
127
|
level: nil,
|
|
129
128
|
formatter: nil,
|
|
130
129
|
filter: nil,
|
|
@@ -134,16 +133,24 @@ module SemanticLogger
|
|
|
134
133
|
data_stream: false,
|
|
135
134
|
**client_args,
|
|
136
135
|
&)
|
|
136
|
+
if type
|
|
137
|
+
Kernel.warn(
|
|
138
|
+
"The Elasticsearch/OpenSearch appender `type:` parameter is deprecated and will be removed in v6. " \
|
|
139
|
+
"Document `_type` has been unused since Elasticsearch 7 and is ignored by OpenSearch.",
|
|
140
|
+
category: :deprecated
|
|
141
|
+
)
|
|
142
|
+
end
|
|
143
|
+
|
|
137
144
|
@url = url
|
|
138
145
|
@index = index
|
|
139
146
|
@date_pattern = date_pattern
|
|
140
|
-
@type = type
|
|
141
147
|
@client_args = client_args.dup
|
|
142
148
|
@client_args[:url] = url if url && !client_args[:hosts]
|
|
143
149
|
@client_args[:logger] = logger
|
|
144
150
|
@data_stream = data_stream
|
|
145
151
|
|
|
146
|
-
super(level: level, formatter: formatter, filter: filter, application: application,
|
|
152
|
+
super(level: level, formatter: formatter, filter: filter, application: application,
|
|
153
|
+
environment: environment, host: host, metrics: false, &)
|
|
147
154
|
reopen
|
|
148
155
|
end
|
|
149
156
|
|
|
@@ -193,20 +200,13 @@ module SemanticLogger
|
|
|
193
200
|
expanded_index_name = log.time.strftime("#{index}-#{date_pattern}")
|
|
194
201
|
return {"create" => {}} if @data_stream
|
|
195
202
|
|
|
196
|
-
|
|
197
|
-
bulk_index["index"].merge!({"_type" => type}) if version_supports_type?
|
|
198
|
-
bulk_index
|
|
203
|
+
{"index" => {"_index" => expanded_index_name}}
|
|
199
204
|
end
|
|
200
205
|
|
|
201
206
|
def default_formatter
|
|
202
207
|
time_key = @data_stream ? "@timestamp" : :timestamp
|
|
203
208
|
SemanticLogger::Formatters::Raw.new(time_format: :iso_8601, time_key: time_key)
|
|
204
209
|
end
|
|
205
|
-
|
|
206
|
-
# Modern Elasticsearch (>= 7) and OpenSearch no longer support document `_type`.
|
|
207
|
-
def version_supports_type?
|
|
208
|
-
false
|
|
209
|
-
end
|
|
210
210
|
end
|
|
211
211
|
end
|
|
212
212
|
end
|
|
@@ -13,7 +13,7 @@ require "date"
|
|
|
13
13
|
module SemanticLogger
|
|
14
14
|
module Appender
|
|
15
15
|
class ElasticsearchHttp < SemanticLogger::Appender::Http
|
|
16
|
-
attr_accessor :index
|
|
16
|
+
attr_accessor :index
|
|
17
17
|
|
|
18
18
|
# Create Elasticsearch appender over persistent HTTP(S)
|
|
19
19
|
#
|
|
@@ -25,9 +25,9 @@ module SemanticLogger
|
|
|
25
25
|
# Default: 'semantic_logger'
|
|
26
26
|
#
|
|
27
27
|
# type: [String]
|
|
28
|
-
#
|
|
29
|
-
#
|
|
30
|
-
#
|
|
28
|
+
# Deprecated and ignored. Document `_type` has been unused since
|
|
29
|
+
# Elasticsearch 7. Passing it logs a deprecation warning; it will be
|
|
30
|
+
# removed in v6.
|
|
31
31
|
#
|
|
32
32
|
# level: [:trace | :debug | :info | :warn | :error | :fatal]
|
|
33
33
|
# Override the log level for this appender.
|
|
@@ -52,16 +52,23 @@ module SemanticLogger
|
|
|
52
52
|
# Name of this application to appear in log messages.
|
|
53
53
|
# Default: SemanticLogger.application
|
|
54
54
|
def initialize(index: "semantic_logger",
|
|
55
|
-
type:
|
|
55
|
+
type: nil,
|
|
56
56
|
url: "http://localhost:9200",
|
|
57
57
|
**http_args,
|
|
58
58
|
&)
|
|
59
|
+
if type
|
|
60
|
+
Kernel.warn(
|
|
61
|
+
"The Elasticsearch/OpenSearch appender `type:` parameter is deprecated and will be removed in v6. " \
|
|
62
|
+
"Document `_type` has been unused since Elasticsearch 7 and is ignored by OpenSearch.",
|
|
63
|
+
category: :deprecated
|
|
64
|
+
)
|
|
65
|
+
end
|
|
66
|
+
|
|
59
67
|
@index = index
|
|
60
|
-
@type = type
|
|
61
68
|
super(url: url, **http_args, &)
|
|
62
69
|
|
|
63
70
|
@request_path = "#{@path.end_with?('/') ? @path : "#{@path}/"}#{@index}-%Y.%m.%d"
|
|
64
|
-
@logging_path = "#{@request_path}
|
|
71
|
+
@logging_path = "#{@request_path}/_doc"
|
|
65
72
|
end
|
|
66
73
|
|
|
67
74
|
# Log to the index for today.
|
|
@@ -150,6 +150,11 @@ module SemanticLogger
|
|
|
150
150
|
@reopen_at = nil
|
|
151
151
|
|
|
152
152
|
super(**args, &)
|
|
153
|
+
|
|
154
|
+
# Open the file at creation time so that misconfiguration (bad directory,
|
|
155
|
+
# insufficient permissions, ...) raises immediately in SemanticLogger.add_appender,
|
|
156
|
+
# instead of surfacing later on the appender thread via the internal logger.
|
|
157
|
+
reopen
|
|
153
158
|
end
|
|
154
159
|
|
|
155
160
|
# After forking an active process call #reopen to re-open
|
|
@@ -33,6 +33,12 @@ module SemanticLogger
|
|
|
33
33
|
# Proc: Only include log messages where the supplied Proc returns true
|
|
34
34
|
# The Proc must return true or false.
|
|
35
35
|
def initialize(level: :error, **args, &block)
|
|
36
|
+
Kernel.warn(
|
|
37
|
+
"SemanticLogger::Appender::NewRelic (appender: :new_relic) is deprecated and will be removed in v6. " \
|
|
38
|
+
"It reports errors via newrelic_rpm and is superseded by the log-forwarding appender " \
|
|
39
|
+
"appender: :new_relic_logs.",
|
|
40
|
+
category: :deprecated
|
|
41
|
+
)
|
|
36
42
|
super
|
|
37
43
|
end
|
|
38
44
|
|
|
@@ -48,7 +48,8 @@ module SemanticLogger
|
|
|
48
48
|
def initialize(formatter: SemanticLogger::Formatters::NewRelicLogs.new, **args, &block)
|
|
49
49
|
super
|
|
50
50
|
|
|
51
|
-
# Record NewRelic's "trace.id"/"entity.name"/"hostname"/etc, so we can include them
|
|
51
|
+
# Record NewRelic's "trace.id"/"entity.name"/"hostname"/etc, so we can include them
|
|
52
|
+
# later in the formatted output.
|
|
52
53
|
# These are thread-local, so need to be captured as soon as the log-message is created.
|
|
53
54
|
# https://rubydoc.info/gems/newrelic_rpm/NewRelic/Agent#linking_metadata-instance_method
|
|
54
55
|
SemanticLogger.on_log(CAPTURE_CONTEXT)
|
|
@@ -2,7 +2,8 @@ begin
|
|
|
2
2
|
require "opentelemetry/logs"
|
|
3
3
|
rescue LoadError
|
|
4
4
|
raise LoadError,
|
|
5
|
-
|
|
5
|
+
"Gem opentelemetry-logs-sdk is required for logging to Open Telemetry. " \
|
|
6
|
+
'Please add the gem "opentelemetry-logs-sdk" to your Gemfile.'
|
|
6
7
|
end
|
|
7
8
|
|
|
8
9
|
# Open Telemetry Appender
|
|
@@ -2,7 +2,8 @@ begin
|
|
|
2
2
|
require "opensearch-ruby"
|
|
3
3
|
rescue LoadError
|
|
4
4
|
raise LoadError,
|
|
5
|
-
|
|
5
|
+
"Gem opensearch-ruby is required for logging to OpenSearch. " \
|
|
6
|
+
'Please add the gem "opensearch-ruby" to your Gemfile.'
|
|
6
7
|
end
|
|
7
8
|
|
|
8
9
|
require "semantic_logger/appender/elasticsearch_base"
|
|
@@ -81,14 +81,15 @@ module SemanticLogger
|
|
|
81
81
|
#
|
|
82
82
|
# more parameters supported by Bunny: http://rubybunny.info/articles/connecting.html
|
|
83
83
|
def initialize(queue_name: "semantic_logger", rabbitmq_host: nil,
|
|
84
|
-
level: nil, formatter: nil, filter: nil, application: nil, environment: nil,
|
|
85
|
-
**args, &)
|
|
84
|
+
level: nil, formatter: nil, filter: nil, application: nil, environment: nil,
|
|
85
|
+
host: nil, metrics: true, **args, &)
|
|
86
86
|
@queue_name = queue_name
|
|
87
87
|
@rabbitmq_args = args.dup
|
|
88
88
|
@rabbitmq_args[:host] = rabbitmq_host
|
|
89
89
|
@rabbitmq_args[:logger] = logger
|
|
90
90
|
|
|
91
|
-
super(level: level, formatter: formatter, filter: filter, application: application,
|
|
91
|
+
super(level: level, formatter: formatter, filter: filter, application: application,
|
|
92
|
+
environment: environment, host: host, metrics: metrics, &)
|
|
92
93
|
reopen
|
|
93
94
|
end
|
|
94
95
|
|
|
@@ -39,6 +39,11 @@ module SemanticLogger
|
|
|
39
39
|
# Name of this application to appear in log messages.
|
|
40
40
|
# Default: SemanticLogger.application
|
|
41
41
|
def initialize(level: :error, **args, &block)
|
|
42
|
+
Kernel.warn(
|
|
43
|
+
"SemanticLogger::Appender::Sentry (appender: :sentry) is deprecated and will be removed in v6. " \
|
|
44
|
+
"It depends on the end-of-life sentry-raven gem. Use appender: :sentry_ruby instead.",
|
|
45
|
+
category: :deprecated
|
|
46
|
+
)
|
|
42
47
|
# Replace the Sentry Raven logger so that we can identify its log messages and not forward them to Sentry
|
|
43
48
|
Raven.configure { |config| config.logger = SemanticLogger[Raven] }
|
|
44
49
|
super
|