semantic_logger 5.2.0 → 5.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/docs/api.md +160 -9
- data/docs/appenders.md +461 -34
- data/docs/config.md +543 -36
- data/docs/log.md +38 -7
- data/docs/metrics.md +139 -28
- data/docs/operations.md +293 -24
- data/docs/rails.md +319 -5
- data/docs/security.md +135 -107
- data/docs/testing.md +23 -0
- data/docs/upgrading.md +644 -1
- data/lib/semantic_logger/appender/async.rb +29 -11
- data/lib/semantic_logger/appender/bugsnag.rb +7 -2
- data/lib/semantic_logger/appender/cloudwatch_logs.rb +46 -6
- data/lib/semantic_logger/appender/elasticsearch_base.rb +197 -29
- data/lib/semantic_logger/appender/elasticsearch_http.rb +22 -5
- data/lib/semantic_logger/appender/file.rb +29 -14
- data/lib/semantic_logger/appender/graylog.rb +81 -7
- data/lib/semantic_logger/appender/honeybadger.rb +7 -1
- data/lib/semantic_logger/appender/honeybadger_insights.rb +8 -2
- data/lib/semantic_logger/appender/http.rb +244 -41
- data/lib/semantic_logger/appender/io.rb +11 -6
- data/lib/semantic_logger/appender/kafka.rb +192 -46
- data/lib/semantic_logger/appender/loki.rb +42 -6
- data/lib/semantic_logger/appender/mongodb.rb +94 -9
- data/lib/semantic_logger/appender/new_relic.rb +2 -3
- data/lib/semantic_logger/appender/new_relic_logs.rb +30 -8
- data/lib/semantic_logger/appender/open_telemetry.rb +37 -35
- data/lib/semantic_logger/appender/rabbitmq.rb +241 -17
- data/lib/semantic_logger/appender/sentry.rb +2 -3
- data/lib/semantic_logger/appender/sentry_ruby.rb +82 -15
- data/lib/semantic_logger/appender/splunk.rb +4 -4
- data/lib/semantic_logger/appender/splunk_http.rb +92 -1
- data/lib/semantic_logger/appender/syslog.rb +89 -10
- data/lib/semantic_logger/appender/tcp.rb +68 -98
- data/lib/semantic_logger/appender/text_output.rb +65 -0
- data/lib/semantic_logger/appender/udp.rb +130 -18
- data/lib/semantic_logger/appender/wrapper.rb +10 -7
- data/lib/semantic_logger/appender.rb +115 -10
- data/lib/semantic_logger/appenders.rb +136 -4
- data/lib/semantic_logger/backtrace_cleaning.rb +77 -0
- data/lib/semantic_logger/base.rb +53 -71
- data/lib/semantic_logger/call_site.rb +235 -0
- data/lib/semantic_logger/context.rb +222 -0
- data/lib/semantic_logger/core_ext/process.rb +20 -5
- data/lib/semantic_logger/core_ext/warning.rb +70 -0
- data/lib/semantic_logger/deprecation.rb +139 -0
- data/lib/semantic_logger/exception_details.rb +4 -3
- data/lib/semantic_logger/formatters/base.rb +139 -19
- data/lib/semantic_logger/formatters/color.rb +3 -4
- data/lib/semantic_logger/formatters/default.rb +35 -7
- data/lib/semantic_logger/formatters/ecs.rb +9 -4
- data/lib/semantic_logger/formatters/json.rb +9 -0
- data/lib/semantic_logger/formatters/logfmt.rb +9 -7
- data/lib/semantic_logger/formatters/loki/labels.rb +282 -0
- data/lib/semantic_logger/formatters/loki.rb +412 -24
- data/lib/semantic_logger/formatters/new_relic_logs.rb +15 -13
- data/lib/semantic_logger/formatters/one_line.rb +44 -10
- data/lib/semantic_logger/formatters/open_telemetry.rb +186 -29
- data/lib/semantic_logger/formatters/pattern.rb +60 -8
- data/lib/semantic_logger/formatters/raw.rb +38 -8
- data/lib/semantic_logger/formatters/signalfx.rb +55 -15
- data/lib/semantic_logger/formatters/syslog.rb +4 -2
- data/lib/semantic_logger/formatters/syslog_cee.rb +10 -4
- data/lib/semantic_logger/formatters/syslog_header.rb +49 -0
- data/lib/semantic_logger/formatters.rb +1 -0
- data/lib/semantic_logger/log.rb +236 -80
- data/lib/semantic_logger/loggable.rb +55 -95
- data/lib/semantic_logger/logger.rb +31 -5
- data/lib/semantic_logger/metric/new_relic.rb +32 -17
- data/lib/semantic_logger/metric/signalfx.rb +21 -12
- data/lib/semantic_logger/metric/statsd.rb +26 -8
- data/lib/semantic_logger/processor.rb +194 -5
- data/lib/semantic_logger/queue_processor/backoff.rb +140 -0
- data/lib/semantic_logger/queue_processor/batch_failure.rb +69 -0
- data/lib/semantic_logger/queue_processor/held_write.rb +62 -0
- data/lib/semantic_logger/queue_processor/inbox.rb +97 -0
- data/lib/semantic_logger/queue_processor/reopener.rb +56 -0
- data/lib/semantic_logger/queue_processor/reporter.rb +113 -0
- data/lib/semantic_logger/queue_processor/request.rb +48 -10
- data/lib/semantic_logger/queue_processor/throttled_counter.rb +60 -17
- data/lib/semantic_logger/queue_processor.rb +426 -176
- data/lib/semantic_logger/ractor_copy.rb +120 -0
- data/lib/semantic_logger/ractor_processor.rb +82 -0
- data/lib/semantic_logger/ractor_receiver.rb +282 -0
- data/lib/semantic_logger/ractor_support.rb +28 -0
- data/lib/semantic_logger/redactor.rb +310 -0
- data/lib/semantic_logger/semantic_logger.rb +313 -79
- data/lib/semantic_logger/subscriber.rb +169 -15
- data/lib/semantic_logger/sync_processor.rb +226 -26
- data/lib/semantic_logger/test/capture_log_events.rb +20 -2
- data/lib/semantic_logger/test/minitest.rb +2 -1
- data/lib/semantic_logger/test/rspec.rb +1 -1
- data/lib/semantic_logger/trap_safe.rb +113 -0
- data/lib/semantic_logger/utils.rb +266 -79
- data/lib/semantic_logger/version.rb +1 -1
- data/lib/semantic_logger.rb +35 -2
- metadata +22 -2
data/docs/appenders.md
CHANGED
|
@@ -40,6 +40,12 @@ SemanticLogger.add_appender(logger: Logger.new($stdout))
|
|
|
40
40
|
SemanticLogger.add_appender(metric: :statsd, url: "udp://localhost:8125")
|
|
41
41
|
~~~
|
|
42
42
|
|
|
43
|
+
Pass one of these keywords to each call, and call `add_appender` once for each destination, see
|
|
44
|
+
[Logging to several destinations at once](#logging-to-several-destinations-at-once). Given more than
|
|
45
|
+
one, it adds only the first of `file_name:`, `io:`, `logger:`, `appender:`, and `metric:`, and
|
|
46
|
+
reports a [deprecation warning](upgrading.html#seeing-deprecation-warnings); v6 raises an
|
|
47
|
+
`ArgumentError`.
|
|
48
|
+
|
|
43
49
|
In a Rails app using [rails_semantic_logger](rails.html), put `add_appender` calls in an initializer
|
|
44
50
|
such as `config/initializers/semantic_logger.rb`. Otherwise add them wherever you configure Semantic
|
|
45
51
|
Logger when the application boots.
|
|
@@ -51,10 +57,11 @@ destination can keep a different subset of the logs in a different format:
|
|
|
51
57
|
|
|
52
58
|
| Option | Description |
|
|
53
59
|
|--------|-------------|
|
|
54
|
-
| `level` | Only write entries at this level or higher to this appender
|
|
55
|
-
| `formatter` | How to format the output, for example `:default`, `:color`, or `:json`. See [Custom formatters](config.html#custom-formatters). |
|
|
60
|
+
| `level` | Only write entries at this level or higher to this appender; for what it defaults to, and how it works with each logger's own level, see [Per-appender settings](config.html#per-appender-settings). |
|
|
61
|
+
| `formatter` | How to format the output, for example `:default`, `:color`, or `:json`. A [block given to `add_appender`](config.html#formatter-as-a-block-or-proc) takes its place. See [Custom formatters](config.html#custom-formatters). |
|
|
56
62
|
| `filter` | A `Regexp` or `Proc` selecting which entries this appender accepts. See [Filtering](config.html#filtering). |
|
|
57
63
|
| `application`, `environment`, `host` | Override the global values for this appender only. |
|
|
64
|
+
| `metrics` | Whether to also write metric-only entries, which have a metric but no message and no exception. Defaults to `false`, except for `:kafka`, `:rabbitmq`, and `:udp`, which default to `true` until v6, and warn when it is not supplied. See [Metrics in log appenders](metrics.html#metrics-in-log-appenders). |
|
|
58
65
|
|
|
59
66
|
For example, write only warnings and above to a file, formatted as JSON:
|
|
60
67
|
|
|
@@ -231,6 +238,7 @@ The file, IO, and HTTP appenders can emit machine-readable output by choosing a
|
|
|
231
238
|
"thread": "Thread name or id",
|
|
232
239
|
"file": "filename",
|
|
233
240
|
"line": "line number",
|
|
241
|
+
"method": "Method that logged it, such as Order#submit",
|
|
234
242
|
|
|
235
243
|
"level": "trace|debug|info|warn|error|fatal",
|
|
236
244
|
"level_index": "0|1|2|3|4|5",
|
|
@@ -241,6 +249,7 @@ The file, IO, and HTTP appenders can emit machine-readable output by choosing a
|
|
|
241
249
|
"duration_ms": "Duration in milliseconds",
|
|
242
250
|
"metric": "Name of the metric",
|
|
243
251
|
"metric_amount": "Size of the metric, usually 1",
|
|
252
|
+
"dimensions": {"region": "The metric's dimensions, when it has any"},
|
|
244
253
|
|
|
245
254
|
"named_tags": {
|
|
246
255
|
"tag1": "any named tags will be inside this named_tags tag",
|
|
@@ -270,6 +279,12 @@ Notes:
|
|
|
270
279
|
* The layout above is formatted for readability. The real output is a single line terminated by one
|
|
271
280
|
newline, with no embedded newlines.
|
|
272
281
|
* A field with a `nil` value is excluded from the output.
|
|
282
|
+
* An exception, and each exception that caused it, is named as Ruby names its class in
|
|
283
|
+
`Exception#inspect`, as every formatter names it: by the constant the class was assigned to, even
|
|
284
|
+
when the class overrides `self.name`, or for an anonymous class by the class as Ruby writes it,
|
|
285
|
+
such as `#<Class:0x000...>`. Its `message` is always a String: when the exception's `#message`
|
|
286
|
+
raises, such as one that reads a record that is nil, the message it was raised with is written
|
|
287
|
+
instead.
|
|
273
288
|
* To also log the detail Ruby adds to an exception's message, such as did_you_mean suggestions, as
|
|
274
289
|
`exception.details`, enable `log_exception_details`: `formatter: {json: {log_exception_details: true}}`.
|
|
275
290
|
See [Exception details](config.html#exception-details).
|
|
@@ -288,7 +303,7 @@ Differences from `:json`:
|
|
|
288
303
|
* `level` / `level_index` become `severity` / `severity_index`.
|
|
289
304
|
* `host` is excluded by default (under Fluentd it is usually the not-very-useful container id). Pass
|
|
290
305
|
`log_host: true` to include it.
|
|
291
|
-
* The process fields `pid`, `thread`, `file`, and `
|
|
306
|
+
* The process fields `pid`, `thread`, `file`, `line`, and `method` are excluded by default. Pass
|
|
292
307
|
`need_process_info: true` to include them.
|
|
293
308
|
|
|
294
309
|
Construct the formatter explicitly to override these defaults:
|
|
@@ -319,6 +334,7 @@ Semantic Logger fields map to ECS as follows:
|
|
|
319
334
|
| `level` | `log.level` |
|
|
320
335
|
| `name` | `log.logger` |
|
|
321
336
|
| `file` / `line` | `log.origin.file.name` / `log.origin.file.line` |
|
|
337
|
+
| `method` | `log.origin.function` |
|
|
322
338
|
| `message` | `message` |
|
|
323
339
|
| `thread` | `process.thread.name` |
|
|
324
340
|
| `pid` | `process.pid` |
|
|
@@ -329,10 +345,10 @@ Semantic Logger fields map to ECS as follows:
|
|
|
329
345
|
| `duration` | `event.duration` (nanoseconds) |
|
|
330
346
|
| `tags` | `tags` |
|
|
331
347
|
| `named_tags` | `labels.*` |
|
|
332
|
-
| `payload`, `metric`, `metric_amount` | nested under a custom namespace (see below) |
|
|
348
|
+
| `payload`, `metric`, `metric_amount`, `dimensions` | nested under a custom namespace (see below) |
|
|
333
349
|
|
|
334
350
|
ECS reserves the top-level field names it defines, so Semantic Logger data with no native ECS home
|
|
335
|
-
(`payload`, `metric`, and `
|
|
351
|
+
(`payload`, `metric`, `metric_amount`, and `dimensions`) is nested under a custom top-level namespace,
|
|
336
352
|
`semantic_logger` by default. A proper-noun namespace is
|
|
337
353
|
[the approach ECS recommends](https://www.elastic.co/docs/reference/ecs/ecs-custom-fields-in-ecs) for
|
|
338
354
|
custom fields, since it never collides with a current or future ECS field. Rename it with
|
|
@@ -455,6 +471,15 @@ one document per element, such as the
|
|
|
455
471
|
SemanticLogger.add_appender(appender: :http, url: "http://localhost:8088/path", batch: true)
|
|
456
472
|
~~~
|
|
457
473
|
|
|
474
|
+
The formatter formats the whole batch with its `#batch(logs, logger)`, as the default `:json`
|
|
475
|
+
formatter does. `add_appender` raises an `ArgumentError` for a formatter that cannot, such as a
|
|
476
|
+
[formatter block](config.html#formatter-as-a-block-or-proc), given with `batch: true`.
|
|
477
|
+
|
|
478
|
+
`:elasticsearch_http` cannot be batched, since Elasticsearch indexes a batch of entries only through
|
|
479
|
+
its bulk API, and rejects a JSON array: `add_appender` raises an `ArgumentError` for it with
|
|
480
|
+
`batch: true`. To send the entries in batches, use the [Elasticsearch appender](#elasticsearch),
|
|
481
|
+
which sends them in bulk requests.
|
|
482
|
+
|
|
458
483
|
With batching the appender runs on its own thread and flushes once `batch_size` entries have
|
|
459
484
|
accumulated (default 300) or `batch_seconds` have elapsed (default 5), whichever comes first:
|
|
460
485
|
|
|
@@ -468,6 +493,79 @@ SemanticLogger.add_appender(
|
|
|
468
493
|
)
|
|
469
494
|
~~~
|
|
470
495
|
|
|
496
|
+
#### When the server does not accept a request
|
|
497
|
+
|
|
498
|
+
An entry is written once the server answers its request with a 2xx status. The HTTP appender, and
|
|
499
|
+
those built on it (`:elasticsearch_http`, `:splunk_http`, `:loki`, and the `:signalfx` metrics
|
|
500
|
+
appender), handle any other answer by its status:
|
|
501
|
+
|
|
502
|
+
* A 4xx other than 408 and 429, such as `400 Bad Request` or `401 Unauthorized`, rejects the request
|
|
503
|
+
itself, so sending it again cannot succeed. With `async: true` or `batch: true`, its entries are
|
|
504
|
+
discarded straight away. When the server rejects a batch, which is sent in one request, for the
|
|
505
|
+
entries it holds, with `400 Bad Request`, `413 Content Too Large`, or `422 Unprocessable Content`,
|
|
506
|
+
the batch is first split up and sent again in parts, so that only the entries in the parts the
|
|
507
|
+
server rejects are discarded. Any other such 4xx, such as `401 Unauthorized`, `403 Forbidden`, or
|
|
508
|
+
`404 Not Found`, rejects the request whatever it holds, so the whole batch is discarded at once.
|
|
509
|
+
* `408 Request Timeout`, `429 Too Many Requests`, or `503 Service Unavailable` asks to send the
|
|
510
|
+
request again later, and the server did not store the entries. With `async: true` or
|
|
511
|
+
`batch: true`, the worker thread [backs off](operations.html#retry-a-failing-appender), then sends
|
|
512
|
+
the same entry or batch again, so that it is written once the server is back, ahead of the entries
|
|
513
|
+
logged meanwhile, which wait on the queue. Once the server has answered it in this way after it
|
|
514
|
+
was sent again 3 times, it is discarded, and the worker thread carries on with the next one once
|
|
515
|
+
it has backed off, so that a request the server keeps turning away, such as one it answers with a
|
|
516
|
+
`429` that never clears, does not hold up the others.
|
|
517
|
+
* Any other status, such as `500 Internal Server Error`, `502 Bad Gateway`, `504 Gateway Timeout`,
|
|
518
|
+
or a redirect, is a problem with the destination too, but the entry or batch is not sent again: a
|
|
519
|
+
`500` often comes again for the same request, and a proxy can answer a `502` or `504` once the
|
|
520
|
+
server behind it has stored it. With `async: true` or `batch: true`, it is discarded straight
|
|
521
|
+
away, and the worker thread backs off before sending the next one.
|
|
522
|
+
|
|
523
|
+
A request that the appender could not send, since it could not connect to the server (the name does
|
|
524
|
+
not resolve, the server cannot be reached, or it refuses the connection or does not accept it within
|
|
525
|
+
`open_timeout`), is sent again in the same way as a `503`, but without counting towards the 3 times:
|
|
526
|
+
the server has not answered it, so an outage loses no entries until the worker thread gives up. A
|
|
527
|
+
request that failed once it was sent, such as one whose answer did not come within `read_timeout`,
|
|
528
|
+
or whose connection was reset, is discarded instead, after the same back-off, since the server may
|
|
529
|
+
have stored its entries. While a flush is waiting, such as the one at exit, a request that fails is
|
|
530
|
+
discarded rather than sent again, and so are the entries queued after it, ahead of the flush,
|
|
531
|
+
without being sent; see [Retry a failing appender](operations.html#retry-a-failing-appender).
|
|
532
|
+
|
|
533
|
+
The worker thread stops for good after `async_max_retries` (default `100`) failures in a row,
|
|
534
|
+
discarding the entry or batch it was sending; pass `async_max_retries: -1` to keep trying through a
|
|
535
|
+
long outage.
|
|
536
|
+
|
|
537
|
+
The discarded entries are counted as `failed` in `SemanticLogger.stats`, and reported to the internal
|
|
538
|
+
logger with the status code, the url without its credentials, and the start of the response body.
|
|
539
|
+
Without `async:` or `batch:`, each request that fails is reported in the same way, and while the
|
|
540
|
+
process exits, once a request fails other than by the server rejecting it (a 4xx status other than
|
|
541
|
+
408 and 429), the appender is given no more entries for the rest of the flush at exit, rather than
|
|
542
|
+
have the flush at exit wait for each one to fail in turn; see
|
|
543
|
+
[Retry a failing appender](operations.html#retry-a-failing-appender).
|
|
544
|
+
|
|
545
|
+
While the worker thread backs off, the entries logged meanwhile wait on the appender's own queue, as
|
|
546
|
+
they already did while the server could not be reached. Once that queue is full (`max_queue_size`,
|
|
547
|
+
default `10,000`), the thread that writes to every appender waits for room in it, so the other
|
|
548
|
+
appenders stop receiving entries as well, and once the main queue is full too, so does every logging
|
|
549
|
+
call, until the server accepts requests again. To keep logging while the server is unavailable, add
|
|
550
|
+
the appender with `non_blocking: true` as well, so that once its queue is full, its entries are
|
|
551
|
+
dropped and counted instead (see
|
|
552
|
+
[Drop messages instead of blocking](operations.html#drop-messages-instead-of-blocking)):
|
|
553
|
+
|
|
554
|
+
~~~ruby
|
|
555
|
+
SemanticLogger.add_appender(
|
|
556
|
+
appender: :http,
|
|
557
|
+
url: "http://localhost:8088/path",
|
|
558
|
+
batch: true,
|
|
559
|
+
non_blocking: true
|
|
560
|
+
)
|
|
561
|
+
~~~
|
|
562
|
+
|
|
563
|
+
When the server rejects every request with a `401`, `403`, or `404`, for example because the
|
|
564
|
+
password or the path is wrong, each batch is discarded with its first request, and reported once. A
|
|
565
|
+
server that answers `400` for a problem with its own configuration rejects every part of each batch
|
|
566
|
+
too, which takes about two requests for each entry in the batch. The `:splunk_http` appender tells
|
|
567
|
+
those apart for Splunk's HTTP Event Collector, see [Splunk HTTP](#splunk-http).
|
|
568
|
+
|
|
471
569
|
### TCP Appender (+SSL)
|
|
472
570
|
|
|
473
571
|
The TCP appender sends JSON or other formatted messages to services that accept log messages over TCP,
|
|
@@ -501,8 +599,20 @@ end
|
|
|
501
599
|
SemanticLogger.add_appender(appender: :tcp, server: "localhost:8088", formatter: formatter)
|
|
502
600
|
~~~
|
|
503
601
|
|
|
504
|
-
|
|
505
|
-
|
|
602
|
+
The `server:` is `"host:port"`. [Net::TCPClient](https://github.com/reidmorrison/net_tcp_client)
|
|
603
|
+
connects over IPv4 only, so give an IPv4 address, or a host name that has one. Pass several servers
|
|
604
|
+
as `servers:` to try the next when one cannot be connected to. Every host name is resolved before
|
|
605
|
+
the first of them is connected to, so a name that does not resolve fails at once, without trying the
|
|
606
|
+
others or retrying.
|
|
607
|
+
|
|
608
|
+
Net::TCPClient retries only after a network error, such as a refused, reset, or timed-out
|
|
609
|
+
connection: it connects again up to `connect_retry_count:` times (default 10), and writes an entry
|
|
610
|
+
again on a new connection up to `retry_count:` times (default 3). A write timeout, or an SSL error
|
|
611
|
+
such as a certificate that does not verify, fails without a retry. The appender passes these and its
|
|
612
|
+
other connection options, such as `connect_timeout:`, `write_timeout:`, and `ssl:`, to
|
|
613
|
+
`Net::TCPClient.new`, which raises `ArgumentError` for a keyword it does not take; see
|
|
614
|
+
[`Net::TCPClient#initialize`](https://github.com/reidmorrison/net_tcp_client/blob/master/lib/net/tcp_client/tcp_client.rb)
|
|
615
|
+
for them all and their defaults.
|
|
506
616
|
|
|
507
617
|
The TCP and UDP appenders separate records with a newline, so they default to the JSON formatter,
|
|
508
618
|
which escapes embedded newlines and is safe with untrusted data. If you switch to a text formatter
|
|
@@ -524,9 +634,25 @@ SemanticLogger.add_appender(
|
|
|
524
634
|
The UDP appender sends JSON or other formatted messages over UDP:
|
|
525
635
|
|
|
526
636
|
~~~ruby
|
|
527
|
-
SemanticLogger.add_appender(appender: :udp, server: "localhost:8088")
|
|
637
|
+
SemanticLogger.add_appender(appender: :udp, server: "localhost:8088", metrics: false)
|
|
528
638
|
~~~
|
|
529
639
|
|
|
640
|
+
The `server:` is `"host:port"`. Write an IPv6 address in brackets, as a URL does, such as
|
|
641
|
+
`server: "[::1]:8088"`. A host name with an IPv4 address is sent to over IPv4, even when the
|
|
642
|
+
resolver returns its IPv6 address first, as it does for `localhost` on macOS. It is sent to over
|
|
643
|
+
IPv6 when it has only IPv6 addresses, or when the socket cannot connect over IPv4, such as on a
|
|
644
|
+
machine with no IPv4 route. Its IPv6 addresses are looked up only when the resolver answers that it
|
|
645
|
+
has no IPv4 address, or when the socket cannot connect to any of them. When looking up its IPv4
|
|
646
|
+
addresses fails for another reason, such as a resolver that does not answer in time, adding or
|
|
647
|
+
reopening the appender raises that `SocketError`. On Linux with glibc, that includes a name whose
|
|
648
|
+
only addresses are IPv6 ones in `/etc/hosts`, while DNS cannot be reached: its IPv4 lookup then
|
|
649
|
+
fails, rather than finding no address. On Ruby 3.2 and JRuby, whose `SocketError` does not say why
|
|
650
|
+
a lookup failed, the IPv6 addresses are looked up after any failure.
|
|
651
|
+
|
|
652
|
+
Pass `metrics: true` to also send metric-only entries, which have a metric but no message and no
|
|
653
|
+
exception. Without `metrics:`, it sends them and reports a
|
|
654
|
+
[deprecation warning](upgrading.html#seeing-deprecation-warnings), since v6 leaves them out by default.
|
|
655
|
+
|
|
530
656
|
Customize the message with a formatter:
|
|
531
657
|
|
|
532
658
|
~~~ruby
|
|
@@ -536,7 +662,7 @@ formatter = Proc.new do |log, logger|
|
|
|
536
662
|
h.to_json
|
|
537
663
|
end
|
|
538
664
|
|
|
539
|
-
SemanticLogger.add_appender(appender: :udp, server: "localhost:8088", formatter: formatter)
|
|
665
|
+
SemanticLogger.add_appender(appender: :udp, server: "localhost:8088", formatter: formatter, metrics: false)
|
|
540
666
|
~~~
|
|
541
667
|
|
|
542
668
|
### Syslog
|
|
@@ -581,6 +707,13 @@ The `facility:`, `level_map:`, and `max_size:` options also apply when `formatte
|
|
|
581
707
|
as in `formatter: {syslog: {facility: ::Syslog::LOG_LOCAL1}}`, take precedence. A formatter passed
|
|
582
708
|
as an instance keeps its own settings.
|
|
583
709
|
|
|
710
|
+
Over TCP or UDP, each record's header holds the host name and the application name, as its tag. The
|
|
711
|
+
`syslog_protocol` gem that builds the record allows only printable ASCII characters other than the
|
|
712
|
+
space in either, and up to 32 in the tag, so any other character is left out, and the application
|
|
713
|
+
name is cut to 32 characters:
|
|
714
|
+
an application named `"Café Orders"` is written as `CafOrders`. A name with no character left is
|
|
715
|
+
written as `localhost` or `SemanticLogger`.
|
|
716
|
+
|
|
584
717
|
Add a filter to exclude noisy entries such as health checks:
|
|
585
718
|
|
|
586
719
|
~~~ruby
|
|
@@ -612,26 +745,51 @@ Note: `:trace` level messages are mapped to `:debug`.
|
|
|
612
745
|
### Elasticsearch
|
|
613
746
|
|
|
614
747
|
Forward all log entries to Elasticsearch (requires the `elasticsearch` gem). By default entries are
|
|
615
|
-
written to a daily index named `semantic_logger-YYYY.MM.DD
|
|
748
|
+
written to a daily index named `semantic_logger-YYYY.MM.DD`. `index:` sets the name before the date,
|
|
749
|
+
and `date_pattern:` the date, such as `"%Y.%m"` for a monthly index:
|
|
750
|
+
|
|
751
|
+
~~~ruby
|
|
752
|
+
SemanticLogger.add_appender(
|
|
753
|
+
appender: :elasticsearch,
|
|
754
|
+
url: "http://localhost:9200",
|
|
755
|
+
index: "my-index"
|
|
756
|
+
)
|
|
757
|
+
~~~
|
|
758
|
+
|
|
759
|
+
With `data_stream: true`, entries are written to the data stream that `index:` names, without a
|
|
760
|
+
date, and the default formatter writes the time under `@timestamp`, which a data stream requires.
|
|
761
|
+
Elasticsearch creates the data stream with the first entry when an index template that enables data
|
|
762
|
+
streams matches its name; otherwise it creates an ordinary index of that name:
|
|
616
763
|
|
|
617
764
|
~~~ruby
|
|
618
765
|
SemanticLogger.add_appender(
|
|
619
766
|
appender: :elasticsearch,
|
|
620
767
|
url: "http://localhost:9200",
|
|
621
|
-
index: "my-
|
|
768
|
+
index: "my-logs",
|
|
622
769
|
data_stream: true
|
|
623
770
|
)
|
|
624
771
|
~~~
|
|
625
772
|
|
|
773
|
+
The appender sends its entries in bulk requests, a batch at a time by default. When Elasticsearch
|
|
774
|
+
indexes only some of the entries of a request, the entries it indexed are not sent again. It rejects
|
|
775
|
+
an entry with status 429 when its write queue is full, so those entries alone are sent again, up to 3
|
|
776
|
+
times, after waiting 0.2, 0.4, then 0.8 seconds. That holds up the thread that writes the entries for
|
|
777
|
+
waits of at most 1.4 seconds for a request, plus up to 3 more bulk requests. It is the appender's own
|
|
778
|
+
thread when it writes in batches, as it does by default, or with `async: true`. With `batch: false`,
|
|
779
|
+
and without `async: true`, it is the thread that writes to every appender, so the other appenders
|
|
780
|
+
wait too; with `SemanticLogger.sync!`, that is the thread that logged, and every other thread that
|
|
781
|
+
logs waits for it. The entries still not indexed are discarded, and reported to the internal logger.
|
|
782
|
+
|
|
626
783
|
For an end-to-end walkthrough with Kibana, see [Centralized Logging](operations.html#centralized-logging).
|
|
627
784
|
|
|
628
785
|
### OpenSearch
|
|
629
786
|
|
|
630
787
|
Forward all log entries to OpenSearch, for example AWS OpenSearch (requires the `opensearch-ruby`
|
|
631
788
|
gem). OpenSearch is a fork of Elasticsearch and uses the same bulk indexing API, so this appender
|
|
632
|
-
accepts the same options as [Elasticsearch](#elasticsearch)
|
|
633
|
-
|
|
634
|
-
non-Elasticsearch servers with an
|
|
789
|
+
accepts the same options as [Elasticsearch](#elasticsearch), and sends again the entries rejected
|
|
790
|
+
with status 429 in the same way. Use it instead of the Elasticsearch appender when talking to an
|
|
791
|
+
OpenSearch server, since recent `elasticsearch` gems reject non-Elasticsearch servers with an
|
|
792
|
+
`Elasticsearch::UnsupportedProductError`:
|
|
635
793
|
|
|
636
794
|
~~~ruby
|
|
637
795
|
SemanticLogger.add_appender(
|
|
@@ -679,9 +837,43 @@ For HTTPS, change the URL scheme to `https`. Once entries have been sent, open t
|
|
|
679
837
|
interface, select Search, limit to the new host (`host=hostname`), switch to table view, and select
|
|
680
838
|
the interesting columns: `host`, `duration`, `name`, `level`, `message`.
|
|
681
839
|
|
|
840
|
+
To send many entries in each request, add it with `batch: true`. Each request then holds the entries
|
|
841
|
+
of a batch as event objects stacked one after the other, each one what the appender sends for that
|
|
842
|
+
entry on its own, the batch format of Splunk's
|
|
843
|
+
[Format events for HTTP Event Collector](https://help.splunk.com/en/splunk-enterprise/get-started/get-data-in/10.6/get-data-with-http-event-collector/format-events-for-http-event-collector):
|
|
844
|
+
|
|
845
|
+
~~~ruby
|
|
846
|
+
SemanticLogger.add_appender(
|
|
847
|
+
appender: :splunk_http,
|
|
848
|
+
url: "http://localhost:8088/services/collector/event",
|
|
849
|
+
token: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
|
|
850
|
+
batch: true
|
|
851
|
+
)
|
|
852
|
+
~~~
|
|
853
|
+
|
|
854
|
+
A batch needs a formatter that formats each entry as exactly one event object, with its `event`
|
|
855
|
+
field, as the default formatter does. The collector names an event it rejects by its position in the
|
|
856
|
+
request, which the appender takes as the position of the entry in the batch. So do not add it with
|
|
857
|
+
`batch: true` and a formatter such as `:json`, whose objects the collector rejects, or one that
|
|
858
|
+
formats an entry as more than one event.
|
|
859
|
+
|
|
860
|
+
The HTTP Event Collector answers `400` both for the entries a request holds and for its own
|
|
861
|
+
configuration, and says which with the `code` and `text` in its response. When it rejects a request
|
|
862
|
+
for its configuration (an incorrect index, a missing or invalid data channel, acknowledgement
|
|
863
|
+
disabled, query string authorization not enabled, or an invalid or disabled token), a batch is
|
|
864
|
+
discarded at once and reported once to the internal logger with that text, such as
|
|
865
|
+
`"Incorrect index"`, rather than split up to find the entries at fault. When it rejects a batch at
|
|
866
|
+
one of its entries, such as with `"Invalid data format"`, it says which one, and has already indexed
|
|
867
|
+
those before it: that entry is discarded, and only the entries after it are sent again, so that none
|
|
868
|
+
is indexed twice. A `400` that does not say which entry, such as one from a proxy, has the batch split
|
|
869
|
+
up, as for any [request rejected for its entries](#when-the-server-does-not-accept-a-request). See
|
|
870
|
+
Splunk's
|
|
871
|
+
[HTTP Event Collector error codes](https://help.splunk.com/en/splunk-enterprise/get-started/get-data-in/10.4/get-data-with-http-event-collector/troubleshoot-http-event-collector).
|
|
872
|
+
|
|
682
873
|
**Performance:** against a local Splunk instance, the HTTP collector handled only about 30 entries per
|
|
683
|
-
second.
|
|
684
|
-
instead, which reached about 1,400 entries per second (1,200
|
|
874
|
+
second, one entry for each request. To send more, add it with `batch: true`, or write to Splunk with
|
|
875
|
+
the [TCP appender](#tcp-appender-ssl) instead, which reached about 1,400 entries per second (1,200
|
|
876
|
+
with SSL).
|
|
685
877
|
|
|
686
878
|
### Splunk SDK
|
|
687
879
|
|
|
@@ -700,17 +892,141 @@ Send log entries to [Grafana Loki](https://grafana.com/docs/loki) via its
|
|
|
700
892
|
|
|
701
893
|
~~~ruby
|
|
702
894
|
SemanticLogger.add_appender(
|
|
703
|
-
appender:
|
|
704
|
-
url:
|
|
705
|
-
username:
|
|
706
|
-
password:
|
|
707
|
-
compress:
|
|
895
|
+
appender: :loki,
|
|
896
|
+
url: "https://logs-prod-001.grafana.net",
|
|
897
|
+
username: "grafana_username",
|
|
898
|
+
password: "grafana_token_here",
|
|
899
|
+
compress: true,
|
|
900
|
+
formatter: {loki: {structured_metadata: [:pid, :thread]}}
|
|
708
901
|
)
|
|
709
902
|
~~~
|
|
710
903
|
|
|
711
904
|
Set the URL, username, and password to match your Loki instance. Set `compress: true` to compress the
|
|
712
905
|
log messages. The ingestion path is appended to the url, so a query string on the url is kept and sent
|
|
713
|
-
with every push.
|
|
906
|
+
with every push. `structured_metadata:` writes the process id and the thread name as structured
|
|
907
|
+
metadata rather than as labels, see
|
|
908
|
+
[pid and thread as structured metadata](#pid-and-thread-as-structured-metadata).
|
|
909
|
+
|
|
910
|
+
Each entry is a stream whose labels are `pid`, `host`, `application`, `environment`, `level`,
|
|
911
|
+
`thread`, `tags` (as JSON), each named tag, and each value an `on_log` subscriber adds to the
|
|
912
|
+
context, except `pid` and `thread` when they are structured metadata. Its line is the message. Its
|
|
913
|
+
structured metadata holds each payload key, and these fields when the entry has the value:
|
|
914
|
+
|
|
915
|
+
| Field | Value |
|
|
916
|
+
|-------|-------|
|
|
917
|
+
| `metric`, `metric_value`, `dimensions` | The metric's name, its amount, and its dimensions as JSON. |
|
|
918
|
+
| `duration`, `duration_human` | The duration in milliseconds, from the `measure_*` methods, and as text. |
|
|
919
|
+
| `exception_name`, `exception_message` | The class and message of the logged exception. |
|
|
920
|
+
| `stack_trace` | Its backtrace, followed by each exception that caused it. |
|
|
921
|
+
| `pid`, `thread` | The process id and the thread name, instead of as labels, when [`structured_metadata:`](#pid-and-thread-as-structured-metadata) names them. |
|
|
922
|
+
|
|
923
|
+
Structured metadata holds only Strings, so a nested value in the payload is written as JSON, and any
|
|
924
|
+
other value as its text. A payload key named like one of these fields is written with a `payload_`
|
|
925
|
+
prefix, as in `payload_duration`, whether or not the entry has that field, so that logged data cannot
|
|
926
|
+
replace them. Loki stores a name with characters other than ASCII letters and digits with each run of
|
|
927
|
+
them replaced by `_`, so a key that it would store under one of these names, such as `metric.value`,
|
|
928
|
+
is prefixed too, and written as Loki stores it: `payload_metric_value`. When another payload key has
|
|
929
|
+
that name once Loki has stored it, the prefix is added again, as in `payload_payload_metric_value`,
|
|
930
|
+
so that Loki keeps the value of each prefixed key. A key that is exactly one of these names gets the
|
|
931
|
+
name with one prefix, so `payload_metric_value` always holds the payload's `metric_value`. Loki keeps
|
|
932
|
+
only one value of other payload keys that it stores alike, such as `a.b` and `a_b`. Loki rejects a
|
|
933
|
+
payload key without an ASCII letter or digit, such as an empty key or `-`, so it is prefixed too, and
|
|
934
|
+
written as `payload_`.
|
|
935
|
+
|
|
936
|
+
Loki also rejects a label name other than an ASCII letter or underscore followed by letters, digits,
|
|
937
|
+
and underscores. A named tag or context key that it would reject, such as `request-id` or `1st`, is
|
|
938
|
+
written under the name Loki would give it as structured metadata, such as `request_id` or `key_1st`,
|
|
939
|
+
and an empty key as `key_`. A named tag or context value that is nil is left out.
|
|
940
|
+
|
|
941
|
+
Loki keeps one value of a label name, so a label whose name another label already has, such as a
|
|
942
|
+
named tag `host`, is written with the first number added that no other label has, as in `host_2`,
|
|
943
|
+
so that both values are kept. When the label of that name, or of a numbered name before it, has the
|
|
944
|
+
same value, as Loki stores them, it is not written again. The labels come in this order: the
|
|
945
|
+
formatter's own (`pid`, `application`, `environment`, `host`, `level`, `thread`, and `tags`, without
|
|
946
|
+
`pid` or `thread` when it is structured metadata), the named tags, and the context, each under its
|
|
947
|
+
own name; then each named tag and context key whose name was replaced, under that name when no label
|
|
948
|
+
has it yet; and then the rest, numbered, so that every label keeps the name it was written under. So
|
|
949
|
+
the named tags `host: "db-1", host_2: "db-2"` are written as `host_2="db-2"` and `host_3="db-1"`, and
|
|
950
|
+
so are `host: "db-1", "host-2": "db-2"`, in either order.
|
|
951
|
+
|
|
952
|
+
#### pid and thread as structured metadata
|
|
953
|
+
|
|
954
|
+
Loki [advises against](https://grafana.com/docs/loki/latest/get-started/labels/bp-labels/) labels
|
|
955
|
+
whose values are many and keep changing, as `pid` and `thread` do: every process restart, and every
|
|
956
|
+
thread, adds a stream, which Loki indexes and stores on its own. Its documentation names process and
|
|
957
|
+
thread ids as data for
|
|
958
|
+
[structured metadata](https://grafana.com/docs/loki/latest/get-started/labels/structured-metadata/)
|
|
959
|
+
instead. The formatter's `structured_metadata:` option names the labels to write there instead,
|
|
960
|
+
`:pid`, `:thread`, or both, where a query can still filter on them, as in
|
|
961
|
+
`{application="shop"} | thread="worker-1"`, but a stream selector cannot:
|
|
962
|
+
|
|
963
|
+
~~~ruby
|
|
964
|
+
SemanticLogger.add_appender(
|
|
965
|
+
appender: :loki,
|
|
966
|
+
url: "https://logs-prod-001.grafana.net",
|
|
967
|
+
formatter: {loki: {structured_metadata: [:pid, :thread]}}
|
|
968
|
+
)
|
|
969
|
+
~~~
|
|
970
|
+
|
|
971
|
+
Structured metadata needs a Loki that accepts it: one with schema v13, and with
|
|
972
|
+
`allow_structured_metadata`, which is on by default since Loki 3.0. Any other Loki rejects each
|
|
973
|
+
entry that has structured metadata, which with `:pid` is every entry, rather than only those with a
|
|
974
|
+
payload, a metric, a duration, or an exception.
|
|
975
|
+
|
|
976
|
+
A payload key named `pid` or `thread` is then written with the `payload_` prefix, as for the other
|
|
977
|
+
fields above. A named tag or context key of that name is written as a label with a number added, as
|
|
978
|
+
in `thread_2`, since Loki would otherwise show the structured metadata as `thread_extracted`.
|
|
979
|
+
|
|
980
|
+
By default both are labels, as with `structured_metadata: []`. Semantic Logger v6 writes both as
|
|
981
|
+
structured metadata by default, so building the formatter without `structured_metadata:`, as
|
|
982
|
+
`appender: :loki` does unless it is given one with `formatter: {loki: {...}}`, reports a
|
|
983
|
+
[deprecation warning](upgrading.html#seeing-deprecation-warnings). Pass `structured_metadata: []` to
|
|
984
|
+
keep them as labels without it.
|
|
985
|
+
|
|
986
|
+
#### Loki's label limits
|
|
987
|
+
|
|
988
|
+
By default Loki rejects the whole push, with every entry in it, for a stream with more than 15
|
|
989
|
+
labels, a label name longer than 1024 bytes, or a label value longer than 2048 bytes. So a label past
|
|
990
|
+
these limits is written to the structured metadata instead, where a query can still filter on it, as
|
|
991
|
+
in `{application="shop"} | tenant="acme"`, but a stream selector cannot:
|
|
992
|
+
|
|
993
|
+
* each label after the 15th, in the order above;
|
|
994
|
+
* a label with a longer name;
|
|
995
|
+
* a label with a longer value, such as many `tags`.
|
|
996
|
+
|
|
997
|
+
A label with an empty value, which Loki leaves out, is never moved, whatever the length of its name.
|
|
998
|
+
As Loki does, it is not counted, nor is `service_name`, which Loki adds to each stream.
|
|
999
|
+
A moved label is written as Loki stores structured metadata: as a String, under its name with each
|
|
1000
|
+
run of underscores as one, and with the `payload_` prefix when a payload key with its name would
|
|
1001
|
+
have it, as in `payload_duration`. When the structured metadata already has that name, a number is
|
|
1002
|
+
added, as in `tenant_2`, unless the only payload key that Loki stores under that name, such as
|
|
1003
|
+
`a.b` for `a_b`, has the same value, when the label is written once. So a moved label is numbered,
|
|
1004
|
+
as a label is, while a payload key whose prefixed name is taken gets the prefix again, as in
|
|
1005
|
+
`payload_payload_duration`.
|
|
1006
|
+
|
|
1007
|
+
When your Loki raises these limits in its `limits_config`, set the formatter's options of the same
|
|
1008
|
+
names to match, each an Integer of 1 or more:
|
|
1009
|
+
|
|
1010
|
+
~~~ruby
|
|
1011
|
+
SemanticLogger.add_appender(
|
|
1012
|
+
appender: :loki,
|
|
1013
|
+
url: "https://logs-prod-001.grafana.net",
|
|
1014
|
+
formatter: {
|
|
1015
|
+
loki: {
|
|
1016
|
+
structured_metadata: [:pid, :thread],
|
|
1017
|
+
max_label_names_per_series: 30,
|
|
1018
|
+
max_label_name_length: 2048,
|
|
1019
|
+
max_label_value_length: 4096
|
|
1020
|
+
}
|
|
1021
|
+
}
|
|
1022
|
+
)
|
|
1023
|
+
~~~
|
|
1024
|
+
|
|
1025
|
+
| Option | Default | Loki's limit |
|
|
1026
|
+
|--------|---------|--------------|
|
|
1027
|
+
| `max_label_names_per_series` | `15` | The most labels a stream can have. |
|
|
1028
|
+
| `max_label_name_length` | `1024` | The longest label name, in bytes. |
|
|
1029
|
+
| `max_label_value_length` | `2048` | The longest label value, in bytes. |
|
|
714
1030
|
|
|
715
1031
|
### CloudWatch Logs
|
|
716
1032
|
|
|
@@ -753,15 +1069,72 @@ SemanticLogger.add_appender(appender: :open_telemetry)
|
|
|
753
1069
|
~~~
|
|
754
1070
|
|
|
755
1071
|
Each entry is emitted with its level mapped to the matching OpenTelemetry severity number, the message
|
|
756
|
-
as the record body, and the payload as record attributes. The appender registers a
|
|
1072
|
+
as the record body, and the named tags and payload as record attributes. The appender registers a
|
|
757
1073
|
`SemanticLogger.on_log` subscriber that captures the current OpenTelemetry context as each entry is
|
|
758
1074
|
logged, so log records are correlated with the active trace and span.
|
|
759
1075
|
|
|
1076
|
+
For example, inside a span:
|
|
1077
|
+
|
|
1078
|
+
~~~ruby
|
|
1079
|
+
SemanticLogger.named_tagged(request_id: "abc") do
|
|
1080
|
+
logger.info("Created user", user_id: 42)
|
|
1081
|
+
end
|
|
1082
|
+
~~~
|
|
1083
|
+
|
|
1084
|
+
is emitted as a record with the severity text `info` and number `9`, the trace and span ids of that
|
|
1085
|
+
span, and:
|
|
1086
|
+
|
|
1087
|
+
~~~
|
|
1088
|
+
body: "Created user"
|
|
1089
|
+
attributes: {"request_id" => "abc", "user_id" => 42, "semantic_logger.name" => "UserService",
|
|
1090
|
+
"thread.name" => "puma srv tp 001", "process.pid" => 12345}
|
|
1091
|
+
~~~
|
|
1092
|
+
|
|
1093
|
+
Each named tag and payload key is an attribute of its own; a payload key takes precedence over a
|
|
1094
|
+
named tag of the same name. An entry without a message, such as `logger.error(exception)`, has no
|
|
1095
|
+
body. Semantic Logger adds these attributes when the entry has the value:
|
|
1096
|
+
|
|
1097
|
+
| Attribute | Value |
|
|
1098
|
+
|-----------|-------|
|
|
1099
|
+
| `exception.type`, `exception.message` | The class and message of the logged exception. |
|
|
1100
|
+
| `exception.stacktrace` | Its backtrace, followed by each exception that caused it. |
|
|
1101
|
+
| `code.file.path`, `code.line.number` | Where the entry was logged, when it has a call site (see [`call_site_level`](config.html#recording-where-each-entry-was-logged)) or captured a backtrace (see `backtrace_level`). |
|
|
1102
|
+
| `code.function.name` | The method that logged the entry, such as `Order#submit`, when it has a call site. |
|
|
1103
|
+
| `thread.name` | The name of the thread that logged the entry, or its object id when it has none. |
|
|
1104
|
+
| `process.pid` | The id of the process that logged the entry. |
|
|
1105
|
+
| `semantic_logger.name` | The logger's name, usually the class name. |
|
|
1106
|
+
| `semantic_logger.tags` | The tags, as an array. |
|
|
1107
|
+
| `semantic_logger.duration_ms` | The duration in milliseconds, from the `measure_*` methods. |
|
|
1108
|
+
| `semantic_logger.metric`, `semantic_logger.metric_amount` | The metric name and amount. |
|
|
1109
|
+
| `semantic_logger.dimensions.<key>` | Each of the metric's dimensions, such as `semantic_logger.dimensions.region`. |
|
|
1110
|
+
|
|
1111
|
+
The first six rows use the names from the
|
|
1112
|
+
[OpenTelemetry semantic conventions](https://opentelemetry.io/docs/specs/semconv/). A named tag or
|
|
1113
|
+
payload key named like one of these attributes is prefixed with its source, as in
|
|
1114
|
+
`payload.exception.type`, so that logged data cannot replace or forge them.
|
|
1115
|
+
|
|
1116
|
+
OpenTelemetry attribute values are Strings, Integers, Floats, booleans, or arrays of one of those
|
|
1117
|
+
types, so a nested Hash, Array, `Struct`, or `Set` is sent as JSON, any other value, such as a Symbol
|
|
1118
|
+
or a Time, as its `#to_s`, every element of an array that mixes types, such as `[1, "two"]`, as a
|
|
1119
|
+
String, and a `nil` value is left out.
|
|
1120
|
+
|
|
1121
|
+
The host name, application, and environment are not attributes: OpenTelemetry reports them once per
|
|
1122
|
+
export, in its resource, rather than on every record. Set them with the standard environment
|
|
1123
|
+
variables:
|
|
1124
|
+
|
|
1125
|
+
~~~
|
|
1126
|
+
OTEL_SERVICE_NAME=my_app
|
|
1127
|
+
OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=production,host.name=web-1
|
|
1128
|
+
~~~
|
|
1129
|
+
|
|
1130
|
+
The process id is an attribute because the resource holds the id of the process that configured
|
|
1131
|
+
OpenTelemetry, which is wrong for a worker forked from it, for example by Puma or Unicorn.
|
|
1132
|
+
|
|
760
1133
|
| Option | Description |
|
|
761
1134
|
|--------|-------------|
|
|
762
1135
|
| `name` | Instrumentation scope name reported to OpenTelemetry. Defaults to `"SemanticLogger"`. |
|
|
763
1136
|
| `version` | Instrumentation scope version. Defaults to the Semantic Logger gem version. |
|
|
764
|
-
| `metrics` | Whether to
|
|
1137
|
+
| `metrics` | Whether to also send metric-only entries, as log records with no body. Defaults to `false`. |
|
|
765
1138
|
|
|
766
1139
|
### Logstash
|
|
767
1140
|
|
|
@@ -840,8 +1213,18 @@ for more.
|
|
|
840
1213
|
|
|
841
1214
|
## Error and exception monitoring
|
|
842
1215
|
|
|
843
|
-
> **Note:** These appenders forward the payload
|
|
844
|
-
>
|
|
1216
|
+
> **Note:** These appenders forward the payload and named tags. Name the keys whose values must not
|
|
1217
|
+
> be sent with [`SemanticLogger.sensitive_keys`](security.html#redacting-sensitive-data), and keep
|
|
1218
|
+
> sensitive information out of messages and tags, which are sent as they are.
|
|
1219
|
+
|
|
1220
|
+
The `:bugsnag`, `:sentry_ruby`, `:honeybadger`, and `:honeybadger_insights` appenders send the text
|
|
1221
|
+
of an entry, such as its message, payload, and named tags, converted to valid UTF-8, as the JSON
|
|
1222
|
+
formatters write it: binary text, such as a response body, is read as the UTF-8 it usually holds,
|
|
1223
|
+
and invalid bytes are dropped. A Hash or Array nested more than 32 levels deep, or that holds itself,
|
|
1224
|
+
is cut off there with a placeholder such as `"#<Hash nested too deeply>"`, as the JSON formatters cut
|
|
1225
|
+
it off. Any other object in the data is handed over unchanged, for the service to write. When an
|
|
1226
|
+
entry has an exception, `:bugsnag`, `:sentry_ruby`, and `:honeybadger` report that exception itself,
|
|
1227
|
+
whose message the service's gem cleans.
|
|
845
1228
|
|
|
846
1229
|
### Bugsnag
|
|
847
1230
|
|
|
@@ -872,12 +1255,21 @@ SemanticLogger.add_appender(appender: :sentry_ruby)
|
|
|
872
1255
|
> **Deprecated:** the older `appender: :sentry` (backed by the end-of-life `sentry-raven` gem) still
|
|
873
1256
|
> works but is deprecated and will be removed in v6. Switch to `appender: :sentry_ruby`.
|
|
874
1257
|
|
|
875
|
-
|
|
1258
|
+
Each entry is sent with the Sentry context of the thread (or fiber) that logged it: the user, tags,
|
|
1259
|
+
contexts, breadcrumbs, trace, and event processors that `sentry-ruby`, `sentry-rails`, or the
|
|
1260
|
+
application set for it, such as with `Sentry.set_user`. The request itself (its url, headers, and
|
|
1261
|
+
body) is left out, since Sentry reads the request body to send it, which the appender thread must not
|
|
1262
|
+
do while the request is using it; the errors that `sentry-rails` reports itself still include it. The appender sends the entry from its own thread, so it
|
|
1263
|
+
registers an `on_log` subscriber that keeps a copy of that Sentry scope with each entry it will send
|
|
1264
|
+
(with `Log#capture`, so that no formatter writes it), as the entry is logged.
|
|
1265
|
+
|
|
1266
|
+
Some logging context is also forwarded to Sentry, added to that of the thread:
|
|
876
1267
|
|
|
877
1268
|
* From named tags, `transaction_name`. See
|
|
878
1269
|
<https://docs.sentry.io/platforms/ruby/enriching-events/transaction-name/>.
|
|
879
1270
|
* From named tags, `user` is built from the `user_id`, `username`, `user_email`, and
|
|
880
|
-
`ip_address` keys, plus any `user` key that is itself a hash.
|
|
1271
|
+
`ip_address` keys, plus any `user` key that is itself a hash. Its fields replace the same fields
|
|
1272
|
+
of the thread's Sentry user. See
|
|
881
1273
|
<https://docs.sentry.io/platforms/ruby/enriching-events/identify-user/>.
|
|
882
1274
|
* From named tags, `fingerprint` (an Array of Strings, Symbols, or numbers) configures grouping
|
|
883
1275
|
granularity. See
|
|
@@ -908,7 +1300,7 @@ Forward errors to Honeybadger (requires the `honeybadger` gem):
|
|
|
908
1300
|
SemanticLogger.add_appender(appender: :honeybadger)
|
|
909
1301
|
~~~
|
|
910
1302
|
|
|
911
|
-
Or forward
|
|
1303
|
+
Or forward log entries at `:info` and above, its default level, to Honeybadger Insights as events:
|
|
912
1304
|
|
|
913
1305
|
~~~ruby
|
|
914
1306
|
SemanticLogger.add_appender(appender: :honeybadger_insights)
|
|
@@ -948,9 +1340,9 @@ SemanticLogger.on_log do |log|
|
|
|
948
1340
|
end
|
|
949
1341
|
~~~
|
|
950
1342
|
|
|
951
|
-
It sends the whole entry, payload included,
|
|
952
|
-
[
|
|
953
|
-
|
|
1343
|
+
It sends the whole entry, payload included, as it was logged: `on_log` subscribers run before
|
|
1344
|
+
[`SemanticLogger.sensitive_keys`](security.html#redacting-sensitive-data) redacts the entry, so its
|
|
1345
|
+
values are not filtered. Send only the fields Rollbar needs when entries can hold sensitive values.
|
|
954
1346
|
|
|
955
1347
|
## Databases and message queues
|
|
956
1348
|
|
|
@@ -985,6 +1377,12 @@ Each entry is stored as a document:
|
|
|
985
1377
|
}
|
|
986
1378
|
~~~
|
|
987
1379
|
|
|
1380
|
+
MongoDB stores only valid UTF-8 text, and keys without NUL bytes. An entry holding other text, such
|
|
1381
|
+
as a binary response body in its payload, or an exception message that quotes invalid input, is
|
|
1382
|
+
stored with that text converted, as the JSON formatters write it: binary text is read as the UTF-8 it
|
|
1383
|
+
usually holds, invalid bytes are dropped, and NUL bytes are removed from keys. Such an entry nested
|
|
1384
|
+
more than 32 levels deep is cut off there, as the JSON formatters cut it off.
|
|
1385
|
+
|
|
988
1386
|
### Apache Kafka
|
|
989
1387
|
|
|
990
1388
|
Publish log entries to an Apache Kafka broker (requires the `ruby-kafka` gem):
|
|
@@ -992,10 +1390,15 @@ Publish log entries to an Apache Kafka broker (requires the `ruby-kafka` gem):
|
|
|
992
1390
|
~~~ruby
|
|
993
1391
|
SemanticLogger.add_appender(
|
|
994
1392
|
appender: :kafka,
|
|
995
|
-
seed_brokers: ["kafka1:9092", "kafka2:9092"]
|
|
1393
|
+
seed_brokers: ["kafka1:9092", "kafka2:9092"],
|
|
1394
|
+
metrics: false
|
|
996
1395
|
)
|
|
997
1396
|
~~~
|
|
998
1397
|
|
|
1398
|
+
Pass `metrics: true` to also publish metric-only entries, which have a metric but no message and no
|
|
1399
|
+
exception. Without `metrics:`, it publishes them and reports a
|
|
1400
|
+
[deprecation warning](upgrading.html#seeing-deprecation-warnings), since v6 leaves them out by default.
|
|
1401
|
+
|
|
999
1402
|
### RabbitMQ (AMQP)
|
|
1000
1403
|
|
|
1001
1404
|
Stream log entries through a queue on a RabbitMQ broker (requires the `bunny` gem):
|
|
@@ -1006,10 +1409,34 @@ SemanticLogger.add_appender(
|
|
|
1006
1409
|
queue_name: "semantic_logger",
|
|
1007
1410
|
rabbitmq_host: "localhost",
|
|
1008
1411
|
username: "the-username",
|
|
1009
|
-
password: "the-password"
|
|
1412
|
+
password: "the-password",
|
|
1413
|
+
metrics: false
|
|
1010
1414
|
)
|
|
1011
1415
|
~~~
|
|
1012
1416
|
|
|
1417
|
+
Pass `metrics: true` to also send metric-only entries, which have a metric but no message and no
|
|
1418
|
+
exception. Without `metrics:`, it sends them and reports a
|
|
1419
|
+
[deprecation warning](upgrading.html#seeing-deprecation-warnings), since v6 leaves them out by default.
|
|
1420
|
+
|
|
1421
|
+
When it is reopened, such as with `SemanticLogger.reopen(force: true)`, it connects to RabbitMQ again,
|
|
1422
|
+
publishes the next entries on a new channel, and closes the connection it replaced. When it cannot
|
|
1423
|
+
connect, it keeps the connection it has. In a forked child, it leaves the connection it inherited to
|
|
1424
|
+
the parent (see [Process forking](operations.html#process-forking)).
|
|
1425
|
+
|
|
1426
|
+
RabbitMQ closes the channel when it refuses an entry, such as one larger than its `max_message_size`
|
|
1427
|
+
(`PRECONDITION_FAILED`). The appender then publishes the next entries on a new channel on the same
|
|
1428
|
+
connection, and reports the close to the internal logger. The refused entry is lost: RabbitMQ closes
|
|
1429
|
+
the channel after the entry was published, without answering the publish. While Bunny recovers the
|
|
1430
|
+
connection from a network failure, its channels are closed too: the appender then leaves them to
|
|
1431
|
+
Bunny, and the entries are written again once it has recovered. Once Bunny will not recover the
|
|
1432
|
+
connection, such as after its `recovery_attempts` have run out, or after a network failure with
|
|
1433
|
+
`automatic_recovery: false`, the appender closes it and publishes on a new connection. With
|
|
1434
|
+
`recover_from_connection_close: false` and automatic recovery on, Bunny does not recover a connection
|
|
1435
|
+
that RabbitMQ closed, and the appender cannot tell it from one about to be recovered: it raises until
|
|
1436
|
+
it is reopened, such as with `SemanticLogger.reopen(force: true)`. When Bunny itself fails to write
|
|
1437
|
+
an entry to the socket, such as with a `write_timeout:` set, it does not raise, so that entry is
|
|
1438
|
+
lost without a report.
|
|
1439
|
+
|
|
1013
1440
|
## Logging to several destinations at once
|
|
1014
1441
|
|
|
1015
1442
|
Add as many appenders as you like; every entry is written to all of them. Use a per-appender `level`
|