semantic_logger 5.2.0 → 5.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/docs/api.md +160 -9
- data/docs/appenders.md +461 -34
- data/docs/config.md +543 -36
- data/docs/log.md +38 -7
- data/docs/metrics.md +139 -28
- data/docs/operations.md +293 -24
- data/docs/rails.md +319 -5
- data/docs/security.md +135 -107
- data/docs/testing.md +23 -0
- data/docs/upgrading.md +644 -1
- data/lib/semantic_logger/appender/async.rb +29 -11
- data/lib/semantic_logger/appender/bugsnag.rb +7 -2
- data/lib/semantic_logger/appender/cloudwatch_logs.rb +46 -6
- data/lib/semantic_logger/appender/elasticsearch_base.rb +197 -29
- data/lib/semantic_logger/appender/elasticsearch_http.rb +22 -5
- data/lib/semantic_logger/appender/file.rb +29 -14
- data/lib/semantic_logger/appender/graylog.rb +81 -7
- data/lib/semantic_logger/appender/honeybadger.rb +7 -1
- data/lib/semantic_logger/appender/honeybadger_insights.rb +8 -2
- data/lib/semantic_logger/appender/http.rb +244 -41
- data/lib/semantic_logger/appender/io.rb +11 -6
- data/lib/semantic_logger/appender/kafka.rb +192 -46
- data/lib/semantic_logger/appender/loki.rb +42 -6
- data/lib/semantic_logger/appender/mongodb.rb +94 -9
- data/lib/semantic_logger/appender/new_relic.rb +2 -3
- data/lib/semantic_logger/appender/new_relic_logs.rb +30 -8
- data/lib/semantic_logger/appender/open_telemetry.rb +37 -35
- data/lib/semantic_logger/appender/rabbitmq.rb +241 -17
- data/lib/semantic_logger/appender/sentry.rb +2 -3
- data/lib/semantic_logger/appender/sentry_ruby.rb +82 -15
- data/lib/semantic_logger/appender/splunk.rb +4 -4
- data/lib/semantic_logger/appender/splunk_http.rb +92 -1
- data/lib/semantic_logger/appender/syslog.rb +89 -10
- data/lib/semantic_logger/appender/tcp.rb +68 -98
- data/lib/semantic_logger/appender/text_output.rb +65 -0
- data/lib/semantic_logger/appender/udp.rb +130 -18
- data/lib/semantic_logger/appender/wrapper.rb +10 -7
- data/lib/semantic_logger/appender.rb +115 -10
- data/lib/semantic_logger/appenders.rb +136 -4
- data/lib/semantic_logger/backtrace_cleaning.rb +77 -0
- data/lib/semantic_logger/base.rb +53 -71
- data/lib/semantic_logger/call_site.rb +235 -0
- data/lib/semantic_logger/context.rb +222 -0
- data/lib/semantic_logger/core_ext/process.rb +20 -5
- data/lib/semantic_logger/core_ext/warning.rb +70 -0
- data/lib/semantic_logger/deprecation.rb +139 -0
- data/lib/semantic_logger/exception_details.rb +4 -3
- data/lib/semantic_logger/formatters/base.rb +139 -19
- data/lib/semantic_logger/formatters/color.rb +3 -4
- data/lib/semantic_logger/formatters/default.rb +35 -7
- data/lib/semantic_logger/formatters/ecs.rb +9 -4
- data/lib/semantic_logger/formatters/json.rb +9 -0
- data/lib/semantic_logger/formatters/logfmt.rb +9 -7
- data/lib/semantic_logger/formatters/loki/labels.rb +282 -0
- data/lib/semantic_logger/formatters/loki.rb +412 -24
- data/lib/semantic_logger/formatters/new_relic_logs.rb +15 -13
- data/lib/semantic_logger/formatters/one_line.rb +44 -10
- data/lib/semantic_logger/formatters/open_telemetry.rb +186 -29
- data/lib/semantic_logger/formatters/pattern.rb +60 -8
- data/lib/semantic_logger/formatters/raw.rb +38 -8
- data/lib/semantic_logger/formatters/signalfx.rb +55 -15
- data/lib/semantic_logger/formatters/syslog.rb +4 -2
- data/lib/semantic_logger/formatters/syslog_cee.rb +10 -4
- data/lib/semantic_logger/formatters/syslog_header.rb +49 -0
- data/lib/semantic_logger/formatters.rb +1 -0
- data/lib/semantic_logger/log.rb +236 -80
- data/lib/semantic_logger/loggable.rb +55 -95
- data/lib/semantic_logger/logger.rb +31 -5
- data/lib/semantic_logger/metric/new_relic.rb +32 -17
- data/lib/semantic_logger/metric/signalfx.rb +21 -12
- data/lib/semantic_logger/metric/statsd.rb +26 -8
- data/lib/semantic_logger/processor.rb +194 -5
- data/lib/semantic_logger/queue_processor/backoff.rb +140 -0
- data/lib/semantic_logger/queue_processor/batch_failure.rb +69 -0
- data/lib/semantic_logger/queue_processor/held_write.rb +62 -0
- data/lib/semantic_logger/queue_processor/inbox.rb +97 -0
- data/lib/semantic_logger/queue_processor/reopener.rb +56 -0
- data/lib/semantic_logger/queue_processor/reporter.rb +113 -0
- data/lib/semantic_logger/queue_processor/request.rb +48 -10
- data/lib/semantic_logger/queue_processor/throttled_counter.rb +60 -17
- data/lib/semantic_logger/queue_processor.rb +426 -176
- data/lib/semantic_logger/ractor_copy.rb +120 -0
- data/lib/semantic_logger/ractor_processor.rb +82 -0
- data/lib/semantic_logger/ractor_receiver.rb +282 -0
- data/lib/semantic_logger/ractor_support.rb +28 -0
- data/lib/semantic_logger/redactor.rb +310 -0
- data/lib/semantic_logger/semantic_logger.rb +313 -79
- data/lib/semantic_logger/subscriber.rb +169 -15
- data/lib/semantic_logger/sync_processor.rb +226 -26
- data/lib/semantic_logger/test/capture_log_events.rb +20 -2
- data/lib/semantic_logger/test/minitest.rb +2 -1
- data/lib/semantic_logger/test/rspec.rb +1 -1
- data/lib/semantic_logger/trap_safe.rb +113 -0
- data/lib/semantic_logger/utils.rb +266 -79
- data/lib/semantic_logger/version.rb +1 -1
- data/lib/semantic_logger.rb +35 -2
- metadata +22 -2
data/docs/log.md
CHANGED
|
@@ -55,8 +55,9 @@ A handful of attributes are populated for **every** event:
|
|
|
55
55
|
|
|
56
56
|
The rest are **situational**: they are set only when relevant to that particular call. For example
|
|
57
57
|
`duration` is present only for `measure_*` calls, `exception` only when an exception was logged,
|
|
58
|
-
`metric`/`metric_amount`/`dimensions` only for metrics,
|
|
59
|
-
capture is enabled (`SemanticLogger.backtrace_level`)
|
|
58
|
+
`metric`/`metric_amount`/`dimensions` only for metrics, `backtrace` only when backtrace
|
|
59
|
+
capture is enabled (`SemanticLogger.backtrace_level`), and `call_site` only when call site recording
|
|
60
|
+
is enabled (`SemanticLogger.call_site_level`). Always guard against `nil` when reading a
|
|
60
61
|
situational field.
|
|
61
62
|
|
|
62
63
|
### Attributes
|
|
@@ -73,13 +74,39 @@ situational field.
|
|
|
73
74
|
|`tags`|`Array<String>`|Tags active on the thread when the call was made.|
|
|
74
75
|
|`named_tags`|`Hash<String, Object>`|Named tags active on the thread when the call was made.|
|
|
75
76
|
|`context`|`Hash`|Named contexts captured in-line at the point the event was created. `nil` when none.|
|
|
77
|
+
|`captures`|`Hash`|Objects an appender captured in-line for its own use, such as the OpenTelemetry context. Unlike `context`, formatters do not write them, except one built for that appender, as `:new_relic_logs` writes the linking metadata its appender captures. `nil` when none.|
|
|
76
78
|
|`duration`|`Float`|Time in milliseconds taken by a `measure_*` call. `nil` for ordinary log calls.|
|
|
77
79
|
|`exception`|`Exception`|The Ruby exception that was logged. `nil` when none. Use `each_exception` to walk a `cause` chain.|
|
|
78
80
|
|`backtrace`|`Array<String>`|Backtrace captured at the call site, present only when the level is at or above `SemanticLogger.backtrace_level`.|
|
|
81
|
+
|`call_site`|`SemanticLogger::CallSite`|Where the entry was logged, present only when the level is at or above `SemanticLogger.call_site_level`, and the log call did not pass `call_site: false`. See [The call site](#the-call-site).|
|
|
79
82
|
|`metric`|`String`|The metric name, present for metric and `measure_*` calls. See [Metrics](metrics.html).|
|
|
80
|
-
|`metric_amount`|`
|
|
83
|
+
|`metric_amount`|`Numeric`|Amount to add to a counter metric, for example the quantity purchased. A counter adds 1 when it is `nil`. A negative amount with a `metric` is deprecated, see [Negative metric amounts](upgrading.html#negative-metric-amounts).|
|
|
81
84
|
|`dimensions`|`Hash`|Dimensions (key/value labels) supplied for a metric.|
|
|
82
85
|
|
|
86
|
+
### The call site
|
|
87
|
+
|
|
88
|
+
`call_site` is a frozen value object that records where the entry was logged. See
|
|
89
|
+
[Recording where each entry was logged](config.html#recording-where-each-entry-was-logged) for how
|
|
90
|
+
it is found.
|
|
91
|
+
|
|
92
|
+
|Method|Returns|Description|
|
|
93
|
+
|------|-------|-----------|
|
|
94
|
+
|`path`|`String`|The file, as in a backtrace, for example `"/app/models/order.rb"`.|
|
|
95
|
+
|`line`|`Integer`|The line number in the file.|
|
|
96
|
+
|`label`|`String` or `nil`|The label Ruby gives the frame, usually the method: `"Order#submit"` on Ruby 3.4 and later, `"submit"` on Ruby 3.2, 3.3, and JRuby.|
|
|
97
|
+
|`file_name_and_line(short_name: false)`|`[String, Integer]`|The path and line. With `short_name: true` the file name has no directory, for example `"order.rb"`.|
|
|
98
|
+
|`to_s`|`String`|The call site as a backtrace line, for example `"/app/models/order.rb:42:in 'Order#submit'"`.|
|
|
99
|
+
|
|
100
|
+
~~~ruby
|
|
101
|
+
class MyFormatter
|
|
102
|
+
def call(log, _logger)
|
|
103
|
+
site = log.call_site
|
|
104
|
+
location = site ? " (#{site.label} at #{File.basename(site.path)}:#{site.line})" : ""
|
|
105
|
+
"#{log.level_to_s} #{log.name} -- #{log.message}#{location}"
|
|
106
|
+
end
|
|
107
|
+
end
|
|
108
|
+
~~~
|
|
109
|
+
|
|
83
110
|
### Helper methods
|
|
84
111
|
|
|
85
112
|
These read-only helpers derive convenient values from the attributes above:
|
|
@@ -87,18 +114,22 @@ These read-only helpers derive convenient values from the attributes above:
|
|
|
87
114
|
|Method|Returns|Description|
|
|
88
115
|
|------|-------|-----------|
|
|
89
116
|
|`payload?`|`true`/`false`|Whether the event carries a non-empty payload.|
|
|
117
|
+
|`dimensions?`|`true`/`false`|Whether the event has a metric with dimensions, a `Hash` that is not empty. The formatters write the dimensions only then.|
|
|
90
118
|
|`payload_to_s`|`String` or `nil`|The payload rendered with `inspect`, or `nil` when absent.|
|
|
91
|
-
|`metric_only?`|`true`/`false`|`true` when the event has a metric but no message and no exception.
|
|
119
|
+
|`metric_only?`|`true`/`false`|`true` when the event has a metric but no message and no exception. Metric subscribers receive these; log appenders leave them out unless their `metrics` option is true, the default only for `:kafka`, `:rabbitmq`, and `:udp` until v6. See [Metrics in log appenders](metrics.html#metrics-in-log-appenders).|
|
|
92
120
|
|`duration_to_s`|`String` or `nil`|The duration in milliseconds as a short string, for example `"12ms"`. `nil` when there is no duration.|
|
|
93
121
|
|`duration_human`|`String` or `nil`|The duration in human readable form, scaling up to seconds, minutes, hours, or days as needed.|
|
|
94
122
|
|`level_to_s`|`String`|The level as a single upper-case character, for example `"I"` for `:info`.|
|
|
95
123
|
|`cleansed_message`|`String`|The message with Rails/ANSI color escape codes stripped. Used to keep terminal codes out of structured output.|
|
|
96
124
|
|`each_exception`|Enumerator|Iterates the exception and its nested `cause` chain, yielding `(exception, depth)`.|
|
|
97
|
-
|`backtrace_to_s`|`String`|The exception backtrace as a string, including every exception in the `cause` chain. With a block, each cause's message and each backtrace line is replaced by the value the block returns, for example to escape it.|
|
|
125
|
+
|`backtrace_to_s(cleaner: nil)`|`String`|The exception backtrace as a string, including every exception in the `cause` chain. Lines are separated by a line break, with none before the first or after the last, so put your own line break before it. With a block, each cause's message and each backtrace line is replaced by the value the block returns, for example to escape it. With a `cleaner`, each backtrace is first cleaned as `backtrace_of` cleans it. Each cause is written as `Cause: Class: message`, with the class named as Ruby names it in `Exception#inspect`: the constant it was assigned to, even when the class overrides `self.name`, or for an anonymous class the class as Ruby writes it, such as `#<Class:0x000...>`. A cause's message, as written and as passed to the block, is always a `String`: when its `#message` raises, the message it was raised with is used instead, and a `nil` message is empty. Text in encodings that cannot be joined, such as a binary message and a UTF-8 backtrace line, is joined as UTF-8, and so are binary or US-ASCII backtrace lines that hold bytes outside ASCII, such as Ruby's own where no locale is set in a directory such as `café`, which would otherwise make the whole text binary or invalid US-ASCII.|
|
|
126
|
+
|`backtrace_of(exception, cleaner: nil)`|`Array<String>` or `nil`|The backtrace of `exception` (this event's exception or one of its causes). With a [backtrace cleaner](config.html#cleaning-backtraces) object, such as `Rails.backtrace_cleaner` (not `true`, as the formatter option accepts), the lines it keeps, or the full backtrace when it keeps none; the exception is not changed. A cleaned backtrace, and each of its lines, is frozen and cached on the event.|
|
|
98
127
|
|`exception_details(exception, source: false)`|`String` or `nil`|The detail Ruby's `Exception#detailed_message` adds to the message of `exception` (this event's exception or one of its causes), such as did_you_mean suggestions, without the class name. `source: true` also includes error_highlight's source snippet. `nil` when there is none. Cached on the event.|
|
|
99
|
-
|`file_name_and_line(short_name = false)`|`[String, Integer]` or `nil`|The file name and line number from the event's backtrace or exception. Pass `true` for just the base file name.|
|
|
128
|
+
|`file_name_and_line(short_name = false)`|`[String, Integer]` or `nil`|The file name and line number from the event's call site, or else its backtrace or exception. Pass `true` for just the base file name.|
|
|
100
129
|
|`to_h`|`Hash`|The event as a plain Hash (via the Raw formatter), including `host`, `application`, and `environment`. Handy for inspecting an event or building a custom formatter.|
|
|
101
|
-
|`set_context(key, value)`|`
|
|
130
|
+
|`set_context(key, value)`|`Object`|Lazily initializes `context`, stores a key/value pair on it, and returns the value.|
|
|
131
|
+
|`capture(key, value)`|`Object`|Lazily initializes `captures`, stores a key/value pair on it, and returns the value, for an appender that needs state of the logging thread, such as the current trace, that is not log data.|
|
|
132
|
+
|`captured(key)`|`Object` or `nil`|The value captured with `key`, or `nil` when there is none.|
|
|
102
133
|
|
|
103
134
|
### Inspecting an event
|
|
104
135
|
|
data/docs/metrics.md
CHANGED
|
@@ -27,8 +27,10 @@ There are two parts to using metrics:
|
|
|
27
27
|
1. **Emit** a metric by adding the `:metric` option to any log or `measure_` call. This is covered in
|
|
28
28
|
Steps 1 to 3 below.
|
|
29
29
|
2. **Subscribe** a destination (Statsd, New Relic, SignalFx, and so on) that receives those metrics.
|
|
30
|
-
Until a subscriber is registered, the `:metric` option is harmless
|
|
31
|
-
|
|
30
|
+
Until a subscriber is registered, the `:metric` option is harmless: the number reaches no metrics
|
|
31
|
+
backend, though the log appenders that write structured data still write it with the entry. See
|
|
32
|
+
[Send metrics to a backend](#send-metrics-to-a-backend) and
|
|
33
|
+
[Metrics in log appenders](#metrics-in-log-appenders).
|
|
32
34
|
|
|
33
35
|
Metric subscribers are notified asynchronously on the background log thread, so emitting a metric
|
|
34
36
|
never slows the calling thread. A metric rides along with its log entry, so it obeys the same level
|
|
@@ -49,12 +51,18 @@ and the metric is incremented by 1:
|
|
|
49
51
|
logger.info(message: "Signed up", metric: "users/signup")
|
|
50
52
|
~~~
|
|
51
53
|
|
|
52
|
-
Supply `:metric_amount` to change the amount.
|
|
54
|
+
Supply `:metric_amount` to change the amount. A metric counts upward, so the amount is zero or
|
|
55
|
+
more. To count something that goes down, such as items removed from a cart, give it a metric of its
|
|
56
|
+
own:
|
|
53
57
|
|
|
54
58
|
~~~ruby
|
|
55
|
-
logger.info(message: "Item removed", metric: "cart/
|
|
59
|
+
logger.info(message: "Item removed", metric: "cart/items_removed")
|
|
56
60
|
~~~
|
|
57
61
|
|
|
62
|
+
A negative `:metric_amount` with a `:metric` is deprecated: it is still logged and sent as before,
|
|
63
|
+
but reports a [deprecation warning](upgrading.html#seeing-deprecation-warnings), and v6 drops it. See
|
|
64
|
+
[Negative metric amounts](upgrading.html#negative-metric-amounts).
|
|
65
|
+
|
|
58
66
|
To add to a running total, such as the dollar amount of purchases per department:
|
|
59
67
|
|
|
60
68
|
~~~ruby
|
|
@@ -85,23 +93,70 @@ logger.info(message: "Called supplier", metric: "supplier/add_user", duration: 1
|
|
|
85
93
|
## Step 3: Break a metric down with dimensions
|
|
86
94
|
|
|
87
95
|
Dimensions are key/value labels attached to a metric, so a backend that supports them (such as
|
|
88
|
-
SignalFx) can
|
|
89
|
-
option:
|
|
96
|
+
SignalFx) can break it down, for example by region, plan, or status. Add them with the
|
|
97
|
+
`:dimensions` option:
|
|
90
98
|
|
|
91
99
|
~~~ruby
|
|
92
|
-
# A counter, broken down by
|
|
93
|
-
logger.info(metric: "filters.count", dimensions: {
|
|
100
|
+
# A counter, broken down by plan:
|
|
101
|
+
logger.info(metric: "filters.count", dimensions: {plan: "premium"})
|
|
102
|
+
|
|
103
|
+
# An amount, broken down by region:
|
|
104
|
+
logger.info(metric: "filters.applied", metric_amount: 3, dimensions: {region: "eu-west"})
|
|
94
105
|
|
|
95
|
-
# A
|
|
96
|
-
logger.
|
|
106
|
+
# A duration, broken down by supplier:
|
|
107
|
+
logger.measure_info("Called supplier", metric: "supplier/add_user", dimensions: {supplier: "acme"}) do
|
|
108
|
+
# Code to call the external service ...
|
|
109
|
+
end
|
|
97
110
|
~~~
|
|
98
111
|
|
|
99
|
-
Not every
|
|
100
|
-
dimensions
|
|
112
|
+
Not every subscriber supports dimensions. **Statsd** and **New Relic** record a metric with
|
|
113
|
+
dimensions under its name alone, leaving out the dimensions, which their clients do not accept;
|
|
114
|
+
**SignalFx** is built around them. See each subscriber below. To keep the metrics with dimensions
|
|
115
|
+
out of Statsd or New Relic, for example when they also go to SignalFx, add the subscriber with
|
|
116
|
+
`filter: ->(log) { !log.dimensions? }`. The formatters that write structured data also write the
|
|
117
|
+
dimensions with the entry, see [Metrics in log appenders](#metrics-in-log-appenders).
|
|
118
|
+
|
|
119
|
+
### Keep the values of each dimension bounded
|
|
120
|
+
|
|
121
|
+
Give each dimension a small, fixed set of values, such as a status, a region, a plan, an HTTP
|
|
122
|
+
method, or the route of a request (`/users/:id`, not `/users/42`). Never use a value that grows
|
|
123
|
+
with your data or your traffic, such as a user id, an email address, an order or request id, an IP
|
|
124
|
+
address, a session, a timestamp, or a URL with ids in it.
|
|
125
|
+
|
|
126
|
+
A metrics backend keeps a separate time series for every combination of dimension values that it
|
|
127
|
+
receives under one metric name, and most count each one as a metric of its own:
|
|
128
|
+
|
|
129
|
+
* [Datadog](https://docs.datadoghq.com/account_management/billing/custom_metrics/) identifies a
|
|
130
|
+
custom metric by its name together with its tag values, so each combination is billed as a
|
|
131
|
+
custom metric.
|
|
132
|
+
* [SignalFx](https://help.splunk.com/en/splunk-observability-cloud/manage-data/metrics-metadata-and-events/metrics-events-and-metadata/get-started-with-metrics)
|
|
133
|
+
(Splunk Observability Cloud) keeps a metric time series (MTS) for each combination of a metric
|
|
134
|
+
and its dimensions. The active ones count toward a
|
|
135
|
+
[limit](https://help.splunk.com/en/splunk-observability-cloud/administer/org-reference-info/per-product-system-limits-in-splunk-observability-cloud/infrastructure-system-limits-details)
|
|
136
|
+
that your subscription sets, beyond which it refuses new time series without an error, so new
|
|
137
|
+
values stop being recorded.
|
|
138
|
+
* [Google Cloud Monitoring](https://docs.cloud.google.com/monitoring/api/v3/metric-model#cardinality)
|
|
139
|
+
keeps a time series for each combination of label values, and asks that the values of every
|
|
140
|
+
label be constrained: no unique identifiers, user ids, IP addresses, timestamps, or URLs that
|
|
141
|
+
are not parameterized.
|
|
142
|
+
|
|
143
|
+
The combinations multiply: a metric with a `status` of 5 values, a `region` of 4, and a `plan` of 3
|
|
144
|
+
can make up to 60 time series. Add a `user_id` dimension for 10,000 users, and the same metric can
|
|
145
|
+
make up to 600,000, each one billed or counted against a limit.
|
|
146
|
+
|
|
147
|
+
The same holds for the metric name: never build it from an id, as in `"users/#{user.id}/fetch"`.
|
|
148
|
+
Statsd and New Relic then record a new metric for every user, and SignalFx turns the parts of the
|
|
149
|
+
name into `class` and `action` dimensions, a new time series for every user.
|
|
150
|
+
|
|
151
|
+
To find what happened to one user, order, or request, log its id in the
|
|
152
|
+
[payload](api.html#step-3-add-structured-data) of the entry instead, where a log appender stores
|
|
153
|
+
it with that entry rather than as a metric of its own. Named tags are written with the entry too,
|
|
154
|
+
except by the `:loki` appender, which makes each named tag a stream label: Loki keeps a stream for
|
|
155
|
+
every combination of label values, so an id there has the same effect as an id in a dimension.
|
|
101
156
|
|
|
102
157
|
## Send metrics to a backend
|
|
103
158
|
|
|
104
|
-
A metric
|
|
159
|
+
A metric reaches no metrics backend until a **metric subscriber** is registered. Add one with
|
|
105
160
|
`SemanticLogger.add_appender(metric: ...)`, usually when your application starts. A subscriber
|
|
106
161
|
receives every logged entry that has a `:metric`, asynchronously on the background thread.
|
|
107
162
|
|
|
@@ -115,7 +170,11 @@ forward them to [Graphite](https://graphiteapp.org), MongoDB, and others (requir
|
|
|
115
170
|
SemanticLogger.add_appender(metric: :statsd, url: "udp://localhost:8125")
|
|
116
171
|
~~~
|
|
117
172
|
|
|
118
|
-
|
|
173
|
+
The `url:` must be `udp://`, and its host must resolve when the subscriber is added, or `add_appender`
|
|
174
|
+
raises. A path in the url, as in `udp://localhost:8125/my_app`, is the namespace of every metric.
|
|
175
|
+
|
|
176
|
+
Counters are integers (float amounts are rounded). A metric is sent under its name alone, leaving out
|
|
177
|
+
its dimensions, since the `statsd-ruby` client has no tags.
|
|
119
178
|
|
|
120
179
|
### New Relic
|
|
121
180
|
|
|
@@ -126,7 +185,11 @@ Forward metrics to New Relic so they can be displayed on custom dashboards (requ
|
|
|
126
185
|
SemanticLogger.add_appender(metric: :new_relic)
|
|
127
186
|
~~~
|
|
128
187
|
|
|
129
|
-
|
|
188
|
+
New Relic charts custom metrics only under `Custom/`, so the subscriber records
|
|
189
|
+
`metric: "Orders/placed"` as `Custom/Orders/placed`, and a name that already starts with `Custom/`
|
|
190
|
+
as it is. To group an application's metrics, pass a `prefix:` that also starts with `Custom/`:
|
|
191
|
+
`prefix: "Custom/MyApp"` records `Orders/placed` as `Custom/MyApp/Orders/placed`. A metric is sent
|
|
192
|
+
under its name alone, leaving out its dimensions, since the agent's metric API takes none.
|
|
130
193
|
|
|
131
194
|
### SignalFx
|
|
132
195
|
|
|
@@ -137,27 +200,75 @@ which is built around dimensions:
|
|
|
137
200
|
SemanticLogger.add_appender(metric: :signalfx, token: "SIGNALFX_ORG_ACCESS_TOKEN")
|
|
138
201
|
~~~
|
|
139
202
|
|
|
140
|
-
`application` and `host` are always sent as dimensions
|
|
141
|
-
dimensions whenever they are present on a log
|
|
203
|
+
`application` and `host` are always sent as dimensions, and so is `environment` when it is set. To
|
|
204
|
+
also forward specific named tags as dimensions whenever they are present, and not blank, on a log
|
|
205
|
+
entry that has no dimensions of its own, list them:
|
|
142
206
|
|
|
143
207
|
~~~ruby
|
|
144
208
|
SemanticLogger.add_appender(
|
|
145
209
|
metric: :signalfx,
|
|
146
210
|
token: "SIGNALFX_ORG_ACCESS_TOKEN",
|
|
147
|
-
dimensions: [:
|
|
211
|
+
dimensions: [:region, :plan]
|
|
148
212
|
)
|
|
149
213
|
~~~
|
|
150
214
|
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
as
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
215
|
+
List only named tags whose values are bounded (see
|
|
216
|
+
[Keep the values of each dimension bounded](#keep-the-values-of-each-dimension-bounded)): a named
|
|
217
|
+
tag such as a request or user id would make a new metric time series for every request or user.
|
|
218
|
+
|
|
219
|
+
When a duration metric has no dimensions, SignalFx receives both its value and a count, so you can
|
|
220
|
+
chart both the timing and the number of occurrences. The value is the duration and the count is 1,
|
|
221
|
+
unless a `metric_amount` is given, which is sent as both. They are sent as `Application.average` and
|
|
222
|
+
`Application.counter`, with `class` and `action` dimensions taken from the metric name, such as
|
|
223
|
+
`class: "supplier"` and `action: "add_user"` for `supplier/add_user`. A count without dimensions is
|
|
224
|
+
sent as `Application.counter` alone, with the same `class` and `action` dimensions.
|
|
225
|
+
|
|
226
|
+
When dimensions are present, the metric is sent as-is: under its own name, without its leading `/`
|
|
227
|
+
characters and with each other `/` replaced by `.`, with its value (the `metric_amount` when given,
|
|
228
|
+
otherwise the duration, or 1 for a count), and, for a duration, without a count. Its dimensions are sent in place of the named tags
|
|
229
|
+
listed in `dimensions:` above, so add any of those values it needs to its own dimensions.
|
|
230
|
+
|
|
231
|
+
A dimension whose value is blank, `nil` or text that is empty or only whitespace, is left out, since
|
|
232
|
+
it carries nothing to break the metric down by. Dimensions that are empty, such as `{}`, or whose
|
|
233
|
+
values are all blank, such as `{state: ""}` or `{supplier: nil}`, count as none, so the metric is
|
|
234
|
+
sent as a metric without dimensions is.
|
|
235
|
+
|
|
236
|
+
## Metrics in log appenders
|
|
237
|
+
|
|
238
|
+
Metric subscribers are the destinations for metrics. Log [appenders](appenders.html) treat a metric
|
|
239
|
+
as part of the log entry that carries it:
|
|
240
|
+
|
|
241
|
+
* **A metric on an entry with a message or an exception** is written with the entry by the
|
|
242
|
+
formatters that write structured data, so a log store such as Elasticsearch or Splunk can chart
|
|
243
|
+
it without a metric subscriber. The text formatters do not write it; they show only the
|
|
244
|
+
entry's duration.
|
|
245
|
+
* **A metric-only entry**, one with a metric but no message and no exception, such as
|
|
246
|
+
`logger.info(metric: "users/signup")`, is meant for the metric subscribers. There can be many
|
|
247
|
+
such entries, so log appenders leave them out, unless one is added with `metrics: true`:
|
|
248
|
+
|
|
249
|
+
~~~ruby
|
|
250
|
+
# Also index metric-only entries in Elasticsearch:
|
|
251
|
+
SemanticLogger.add_appender(appender: :elasticsearch, url: "http://localhost:9200", metrics: true)
|
|
252
|
+
~~~
|
|
253
|
+
|
|
254
|
+
Until v6, the `:kafka`, `:rabbitmq`, and `:udp` appenders send them by default, and report a
|
|
255
|
+
[deprecation warning](upgrading.html#seeing-deprecation-warnings) when they are added without
|
|
256
|
+
`metrics:`. Pass `metrics: false` to leave them out, as v6 does by default, or `metrics: true` to
|
|
257
|
+
keep sending them.
|
|
258
|
+
|
|
259
|
+
What each formatter writes for a metric on an entry with a message:
|
|
260
|
+
|
|
261
|
+
| Formatter | Metric fields written |
|
|
262
|
+
|-----------|-----------------------|
|
|
263
|
+
| `:json`, `:raw`, `:fluentd`, `:syslog_cee`, `:new_relic_logs` | `metric`, `metric_amount`, and `dimensions` |
|
|
264
|
+
| `:ecs` | `metric`, `metric_amount`, and `dimensions`, under its namespace, `semantic_logger` by default, or in `labels` when it has none |
|
|
265
|
+
| `:loki` | `metric`, `metric_value`, and `dimensions` (as JSON), in the structured metadata |
|
|
266
|
+
| `:open_telemetry` | The `semantic_logger.metric` and `semantic_logger.metric_amount` attributes, and a `semantic_logger.dimensions.<key>` attribute for each dimension |
|
|
267
|
+
| `:default`, `:color`, `:one_line`, `:pattern`, `:syslog`, `:logfmt` | None; only the duration is shown |
|
|
268
|
+
|
|
269
|
+
An appender writes what its formatter writes. For example `:elasticsearch`, `:opensearch`,
|
|
270
|
+
`:splunk_http`, `:mongodb`, and `:graylog` use `:raw` by default, and `:http`, `:tcp`, `:udp`,
|
|
271
|
+
`:kafka`, `:rabbitmq`, and `:cloudwatch_logs` use `:json`.
|
|
161
272
|
|
|
162
273
|
## How metrics behave
|
|
163
274
|
|