fast-prometheus 0.3.2 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +106 -0
- data/README.md +3 -3
- data/lib/fast/prometheus/errors.rb +1 -0
- data/lib/fast/prometheus/exposition.rb +52 -4
- data/lib/fast/prometheus/formats/protobuf.rb +7 -3
- data/lib/fast/prometheus/formats/text.rb +5 -1
- data/lib/fast/prometheus/histogram.rb +8 -3
- data/lib/fast/prometheus/metric.rb +43 -9
- data/lib/fast/prometheus/middleware/exporter.rb +9 -1
- data/lib/fast/prometheus/native_histogram.rb +22 -1
- data/lib/fast/prometheus/otlp/mapper.rb +94 -16
- data/lib/fast/prometheus/request_metrics.rb +48 -6
- data/lib/fast/prometheus/summary.rb +6 -11
- data/lib/fast/prometheus/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 0421a8eb55a0265588296f0cdac33f1028cf6785c629a85f192f354bcec7d55a
|
|
4
|
+
data.tar.gz: bd02dcbb69a4a0f5bd5f592f36b9e984a3fc3cd74f2023fcb95c21f0d9662f10
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: bfb624426fc812ea430544cdc0f9d1690a2cde91d65949e77765730bde6ebc264cbb1e8c774c7faad6cb1da8202543f36f26e4bc987c7836aea1ec506bcc0763
|
|
7
|
+
data.tar.gz: 30dfdebab4860d83532092fa969298f8b38d79b18573980baaca8430b9a910e3aad4e541b6a71da0e41cca8a291d41ab0c28115e10abbaef0adc35caeca5241e
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,112 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.4.0] - 2026-09-19
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- **Breaking:** declared label names are normalized to `Symbol`s at
|
|
15
|
+
construction. `labels: ["method"]` and `labels: [:method]` now declare the
|
|
16
|
+
same metric and `Metric#labels` reports `[:method]` for both, so a `String`
|
|
17
|
+
key at observe time — `increment(labels: { "method" => "GET" })` — is an
|
|
18
|
+
unknown label and raises `InvalidLabelSet` where it used to resolve. Two
|
|
19
|
+
names that normalize to the same `Symbol` (`%i[x x]`, `[:x, "x"]`) are a
|
|
20
|
+
duplicate and raise `InvalidLabelName`, as do the reserved names now that
|
|
21
|
+
they are checked after normalization: `"le"` on a `Histogram` and
|
|
22
|
+
`"quantile"` on a `Summary` raise where only `:le` and `:quantile` did. The
|
|
23
|
+
`labels:` and `buckets:` Arrays are copied and frozen, so mutating the Array
|
|
24
|
+
you passed no longer reaches into the metric.
|
|
25
|
+
- Label values and docstrings are normalized to valid UTF-8 at the boundary
|
|
26
|
+
where they enter the store, so no reader, renderer or exporter downstream can
|
|
27
|
+
meet a mixed or an invalid encoding. An ASCII-only String is kept exactly as
|
|
28
|
+
it is whatever its encoding tag (`"GET".b` out of a Rack env, `US-ASCII` out
|
|
29
|
+
of `200.to_s`); invalid UTF-8 is scrubbed; `BINARY` is reinterpreted as UTF-8
|
|
30
|
+
bytes and scrubbed, never transcoded; any other encoding is transcoded with
|
|
31
|
+
replacement. A registry holding label values in two encodings no longer fails
|
|
32
|
+
its scrape with `Encoding::CompatibilityError`, and `Formats::Text.render`
|
|
33
|
+
builds its body in UTF-8 from the first byte, matching the `charset=utf-8` it
|
|
34
|
+
advertises. Bytes that are not valid UTF-8 are replaced, not preserved — see
|
|
35
|
+
Normalization in the metrics reference. The cost is on the labeled write
|
|
36
|
+
path: about 5–10% fewer iterations per second in `benchmark/observe.rb`
|
|
37
|
+
against 0.3.2 in a paired run, with bound and unlabeled calls and every
|
|
38
|
+
allocation budget unchanged.
|
|
39
|
+
- The `method` label on `Middleware::Instrumentation` and
|
|
40
|
+
`Rack::Instrumentation` is allowlisted to the RFC 9110 methods plus `PATCH`;
|
|
41
|
+
every other token is counted as `method="_OTHER"`, the value OpenTelemetry's
|
|
42
|
+
HTTP semantic conventions collapse an unrecognized method to. An HTTP method
|
|
43
|
+
is any token both `Protocol::HTTP` and Rack accept, and nothing reclaims a
|
|
44
|
+
series once it exists, so the two RED metrics now hold at most ten `method`
|
|
45
|
+
values however many a client invents. Matching is case-sensitive, so `get` is
|
|
46
|
+
`_OTHER`.
|
|
47
|
+
- `Accept` and `Accept-Encoding` are parsed as RFC 9110 lists rather than
|
|
48
|
+
searched for a substring: the token is matched whole and case-insensitively,
|
|
49
|
+
`q=0` is a refusal, the higher `q` wins between
|
|
50
|
+
`application/vnd.google.protobuf` and `text/plain`, and a tie is protobuf.
|
|
51
|
+
`Accept: text/plain;version=0.0.4` no longer selects protobuf because the
|
|
52
|
+
word appears in a parameter, and `gzip;q=0` is honoured as the refusal it is.
|
|
53
|
+
Text remains the fallback for a request accepting neither format.
|
|
54
|
+
- `Middleware::Instrumentation` and `Rack::Instrumentation` validate a metric
|
|
55
|
+
already registered under either of their names before reusing it: it must
|
|
56
|
+
declare `labels: %i[method status]` and be a kind `record` can drive — a
|
|
57
|
+
counter under the counter name, something observable under the duration name.
|
|
58
|
+
A mismatch raises `InvalidMetricType` (kind) or `InvalidLabelSet` (labels)
|
|
59
|
+
from the constructor, at boot where the mistaken declaration is, instead of
|
|
60
|
+
once per request from inside the app's request path.
|
|
61
|
+
- **Breaking:** `OTLP::Mapper.new(start_time:)` takes a `Time`, and both it and
|
|
62
|
+
`snapshot.taken_at` are carried as exact nanoseconds (`tv_sec`/`tv_nsec`)
|
|
63
|
+
rather than through a `Float`, which lost the low ~256 ns at epoch
|
|
64
|
+
magnitudes. A `Float` start time no longer works.
|
|
65
|
+
- OTLP resource attributes keep their Ruby type — `String` to `string_value`,
|
|
66
|
+
`Integer` to `int_value`, `Float` to `double_value`, `true`/`false` to
|
|
67
|
+
`bool_value`, anything else its `to_s` — and their keys may be `Symbol`s,
|
|
68
|
+
which makes the `resource_attributes: { service: "my-app" }` in the
|
|
69
|
+
export-otlp how-to run as written. Data point label attributes stay
|
|
70
|
+
`string_value`.
|
|
71
|
+
- `Formats::Protobuf.render` omits a label whose value is the empty String,
|
|
72
|
+
which is implicit presence for a proto3 `string` and what `google-protobuf`
|
|
73
|
+
writes: a series carrying an empty label value is now byte-identical to the
|
|
74
|
+
reference encoder's output instead of a byte longer.
|
|
75
|
+
- `NativeHistogram#observe` raises `ArgumentError` unless its value is
|
|
76
|
+
`Numeric`, as every other metric type does; `NaN` and `±Infinity` are still
|
|
77
|
+
accepted and still never raise.
|
|
78
|
+
- `Histogram.new` rejects a non-finite bucket bound with `ArgumentError`. The
|
|
79
|
+
`+Inf` bucket is implicit and always rendered, so an explicit
|
|
80
|
+
`Float::INFINITY` (or `Float::NAN`) bound was a silent mistake, not a second
|
|
81
|
+
`+Inf`.
|
|
82
|
+
- Benchmarks re-taken on this tree against fast-protowire 0.3.0 and
|
|
83
|
+
republished on the benchmarks page and in the README.
|
|
84
|
+
|
|
85
|
+
### Fixed
|
|
86
|
+
|
|
87
|
+
- `OTLP::Mapper` no longer sizes a bucket array by the distance between two
|
|
88
|
+
observations. A `:native_histogram` series clamps a `±Infinity` observation
|
|
89
|
+
to bucket index `2**31 - 1`, and OTLP carries one contiguous bucket-count
|
|
90
|
+
array per side, so exporting such a series allocated an Array of billions of
|
|
91
|
+
slots. An `ExponentialHistogram` data point now carries the series' finite
|
|
92
|
+
observations only — the clamp bucket and `NaN`s are left out of the bucket
|
|
93
|
+
arrays and out of `count`, and `sum` is omitted, as the field is `optional`,
|
|
94
|
+
once the accumulated sum is no longer finite, which is what the OTel SDKs do
|
|
95
|
+
— and a series needing more than 1024 dense slots on either side is merged to
|
|
96
|
+
a coarser exported `scale` until it fits. The Prometheus-side value is
|
|
97
|
+
untouched; `Formats::Protobuf` still exposes every observation.
|
|
98
|
+
- `NativeHistogram` puts a subnormal observation (under `Float::MIN`) in the
|
|
99
|
+
right bucket. The bucket bounds down there are subnormal too and the `2.0**x`
|
|
100
|
+
the index search compares against underflows to zero among them, so a
|
|
101
|
+
subnormal landed in a neighbouring bucket; it is now scaled into the normal
|
|
102
|
+
range by a power of two first, which is exact, and its index shifted back.
|
|
103
|
+
- `Middleware::Exporter` serves `/metrics?x=1`. A `Protocol::HTTP` request
|
|
104
|
+
target carries its query string, unlike Rack's `PATH_INFO`, so the exact
|
|
105
|
+
match on the path meant a Prometheus scrape config with `params:` fell
|
|
106
|
+
through to the wrapped app. The path component is now matched; `/metrics/`,
|
|
107
|
+
`/METRICS` and `/metricsx` still delegate.
|
|
108
|
+
- Content negotiation is a function of the request header's bytes, whatever
|
|
109
|
+
encoding the String is tagged with, and never raises. A UTF-8-tagged header
|
|
110
|
+
holding invalid bytes — what a client can send and a Rack harness hands over
|
|
111
|
+
as it arrived — used to raise `ArgumentError` out of `Exposition.render` and
|
|
112
|
+
fail the scrape; a header that cannot be made sense of now simply scores
|
|
113
|
+
nothing, which is text and no compression, and a well-formed member beside
|
|
114
|
+
garbage still selects.
|
|
115
|
+
|
|
10
116
|
## [0.3.2] - 2026-09-19
|
|
11
117
|
|
|
12
118
|
### Changed
|
data/README.md
CHANGED
|
@@ -98,10 +98,10 @@ comparisons against `prometheus-client` and `google-protobuf`; see
|
|
|
98
98
|
`google-protobuf` encoder needed 3.5 million Ruby objects and 504k native arenas, and
|
|
99
99
|
in a hundredth of the GC time; taking the snapshot it reads is 1 ms and one Hash per metric
|
|
100
100
|
- The text renderer allocates **66x fewer objects** than prometheus-client's formatter
|
|
101
|
-
for the same body, and renders it 3.
|
|
101
|
+
for the same body, and renders it 3.1x faster at 36,000 series
|
|
102
102
|
- Bound counter and gauge writes allocate nothing and are **1.45x** faster than
|
|
103
|
-
prometheus-client's; histogram observe is **
|
|
104
|
-
- Native histogram observe runs at **1.
|
|
103
|
+
prometheus-client's; histogram observe is **2.9x** faster and summary observe **3.1x**
|
|
104
|
+
- Native histogram observe runs at **1.2M i/s** with no prometheus-client equivalent
|
|
105
105
|
|
|
106
106
|
## Development
|
|
107
107
|
|
|
@@ -12,8 +12,17 @@ module Fast
|
|
|
12
12
|
# negotiating format (text vs protobuf) and gzip from Accept / Accept-Encoding
|
|
13
13
|
# header values. Shared by Middleware::Exporter and Rack::Exporter so
|
|
14
14
|
# negotiation and compression exist exactly once.
|
|
15
|
+
#
|
|
16
|
+
# Both headers are read as RFC 9110 lists over their bytes: comma-separated
|
|
17
|
+
# members, each a token with optional ";"-parameters. The token is matched
|
|
18
|
+
# whole and case-insensitively, parameters other than "q" are ignored, and
|
|
19
|
+
# "q=0" is a refusal. Nothing here raises: a header we cannot make sense of
|
|
20
|
+
# — invalid bytes included — simply scores nothing, which is the text
|
|
21
|
+
# format and no compression.
|
|
15
22
|
module Exposition
|
|
16
23
|
PROTOBUF_ACCEPT = "application/vnd.google.protobuf"
|
|
24
|
+
TEXT_ACCEPT = "text/plain"
|
|
25
|
+
GZIP_CODING = "gzip"
|
|
17
26
|
|
|
18
27
|
FORMAT_MAP = {
|
|
19
28
|
true => [Formats::Protobuf, Formats::Protobuf::CONTENT_TYPE],
|
|
@@ -22,11 +31,11 @@ module Fast
|
|
|
22
31
|
private_constant :FORMAT_MAP
|
|
23
32
|
|
|
24
33
|
def self.render(registry, accept:, accept_encoding:)
|
|
25
|
-
format, content_type = FORMAT_MAP[protobuf?(accept)]
|
|
34
|
+
format, content_type = FORMAT_MAP[protobuf?(byte_view(accept))]
|
|
26
35
|
body = format.render(registry.collect)
|
|
27
36
|
headers = { "content-type" => content_type }
|
|
28
37
|
|
|
29
|
-
if gzip?(accept_encoding)
|
|
38
|
+
if gzip?(byte_view(accept_encoding))
|
|
30
39
|
body = Zlib.gzip(body)
|
|
31
40
|
headers["content-encoding"] = "gzip"
|
|
32
41
|
end
|
|
@@ -34,12 +43,51 @@ module Fast
|
|
|
34
43
|
[body, headers]
|
|
35
44
|
end
|
|
36
45
|
|
|
46
|
+
# Protobuf wins when it is acceptable and no less acceptable than text —
|
|
47
|
+
# on a tie, the order Prometheus itself lists the two in.
|
|
37
48
|
private_class_method def self.protobuf?(accept)
|
|
38
|
-
accept
|
|
49
|
+
quality = quality_of(accept, PROTOBUF_ACCEPT)
|
|
50
|
+
quality.positive? && quality >= quality_of(accept, TEXT_ACCEPT)
|
|
39
51
|
end
|
|
40
52
|
|
|
41
53
|
private_class_method def self.gzip?(accept_encoding)
|
|
42
|
-
accept_encoding.
|
|
54
|
+
quality_of(accept_encoding, GZIP_CODING).positive?
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
# The header's bytes, whatever String the server handed us: negotiation
|
|
58
|
+
# answers the bytes and nothing else. #strip and #casecmp? raise on a
|
|
59
|
+
# UTF-8-tagged String holding invalid ones — and io-stream, Puma and a
|
|
60
|
+
# test harness tag the same bytes BINARY, BINARY and UTF-8 — while BINARY
|
|
61
|
+
# bytes are always valid, so nothing below can raise. One String per
|
|
62
|
+
# header on a path that renders the whole registry anyway.
|
|
63
|
+
private_class_method def self.byte_view(header)
|
|
64
|
+
header.to_s.b
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# The highest q value +token+ carries across the members of a list header:
|
|
68
|
+
# 0.0 when it is absent or refused with q=0, 1.0 when it is listed without
|
|
69
|
+
# one. #split with a block keeps a long header's members out of an Array.
|
|
70
|
+
private_class_method def self.quality_of(header, token)
|
|
71
|
+
best = 0.0
|
|
72
|
+
header.split(",") do |member|
|
|
73
|
+
name, _, parameters = member.partition(";")
|
|
74
|
+
next unless name.strip.casecmp?(token)
|
|
75
|
+
|
|
76
|
+
quality = quality_parameter(parameters)
|
|
77
|
+
best = quality if quality > best
|
|
78
|
+
end
|
|
79
|
+
best
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
# One member's q parameter, 1.0 when it has none. A quoted parameter value
|
|
83
|
+
# holding ";" or "=" can only hide a q from us, never fake one, and a
|
|
84
|
+
# hidden q is the default anyway.
|
|
85
|
+
private_class_method def self.quality_parameter(parameters)
|
|
86
|
+
parameters.split(";") do |parameter|
|
|
87
|
+
name, _, value = parameter.partition("=")
|
|
88
|
+
return value.strip.to_f if name.strip.casecmp?("q")
|
|
89
|
+
end
|
|
90
|
+
1.0
|
|
43
91
|
end
|
|
44
92
|
end
|
|
45
93
|
end
|
|
@@ -105,16 +105,20 @@ module Fast
|
|
|
105
105
|
end
|
|
106
106
|
end
|
|
107
107
|
|
|
108
|
-
# LabelPair { string name = 1; string value = 2 }.
|
|
108
|
+
# LabelPair { string name = 1; string value = 2 }. Implicit presence:
|
|
109
|
+
# an empty value is the field's default and is not sent, as the
|
|
110
|
+
# declared Proto::LabelPair (and google-protobuf) leave it out.
|
|
109
111
|
private_class_method def self.append_label_pair(out, name, value)
|
|
110
112
|
name = name.name if name.is_a?(Symbol)
|
|
111
113
|
value = value.to_s
|
|
112
114
|
name_size = name.bytesize
|
|
113
115
|
value_size = value.bytesize
|
|
116
|
+
size = 1 + varint_size(name_size) + name_size
|
|
117
|
+
size += 1 + varint_size(value_size) + value_size unless value.empty?
|
|
114
118
|
out << METRIC_LABEL
|
|
115
|
-
append_varint(out,
|
|
119
|
+
append_varint(out, size)
|
|
116
120
|
Fast::Protowire::Wire.append_length_delimited(out, LABEL_NAME, name)
|
|
117
|
-
Fast::Protowire::Wire.append_length_delimited(out, LABEL_VALUE, value)
|
|
121
|
+
Fast::Protowire::Wire.append_length_delimited(out, LABEL_VALUE, value) unless value.empty?
|
|
118
122
|
end
|
|
119
123
|
|
|
120
124
|
# Counter / Gauge { double value = 1 }: empty when the value is the
|
|
@@ -11,6 +11,10 @@ module Fast
|
|
|
11
11
|
# Every piece is appended straight to the output buffer: no line, label
|
|
12
12
|
# or name is built as its own String first, so a render allocates one
|
|
13
13
|
# String per sample (the value's to_s) and nothing per label.
|
|
14
|
+
#
|
|
15
|
+
# The buffer is UTF-8 from the start and the snapshot's docstrings and
|
|
16
|
+
# label values are already valid UTF-8 (see Metric), so the body is
|
|
17
|
+
# always a UTF-8-tagged, valid String whatever it holds.
|
|
14
18
|
module Text
|
|
15
19
|
CONTENT_TYPE = "text/plain; version=0.0.4; charset=utf-8"
|
|
16
20
|
|
|
@@ -21,7 +25,7 @@ module Fast
|
|
|
21
25
|
LABEL_REPLACE = { "\\" => "\\\\", '"' => '\\"', "\n" => "\\n" }.freeze
|
|
22
26
|
|
|
23
27
|
def self.render(snapshot)
|
|
24
|
-
snapshot.metrics.each_with_object(String.new) do |metric, output|
|
|
28
|
+
snapshot.metrics.each_with_object(String.new(encoding: Encoding::UTF_8)) do |metric, output|
|
|
25
29
|
next if metric.type == :native_histogram
|
|
26
30
|
|
|
27
31
|
name = metric.name.name
|
|
@@ -46,11 +46,10 @@ module Fast
|
|
|
46
46
|
def initialize(name, docstring:, labels: [], preset_labels: {}, buckets: DEFAULT_BUCKETS, store: nil)
|
|
47
47
|
raise ArgumentError, "buckets must be a non-empty Array" unless buckets.is_a?(Array) && !buckets.empty?
|
|
48
48
|
raise ArgumentError, "buckets must contain only Numeric values" unless buckets.all? { |b| b.is_a?(Numeric) }
|
|
49
|
+
raise ArgumentError, "buckets must contain only finite values" unless buckets.all?(&:finite?)
|
|
49
50
|
raise ArgumentError, "buckets must be strictly ascending" unless buckets.each_cons(2).all? { |a, b| a < b }
|
|
50
51
|
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
@buckets = buckets
|
|
52
|
+
@buckets = buckets.dup.freeze
|
|
54
53
|
super(name, docstring: docstring, labels: labels, preset_labels: preset_labels, store: store)
|
|
55
54
|
end
|
|
56
55
|
|
|
@@ -143,6 +142,12 @@ module Fast
|
|
|
143
142
|
hash["sum"] = slot.sum
|
|
144
143
|
hash
|
|
145
144
|
end
|
|
145
|
+
|
|
146
|
+
def validate_label_names(labels)
|
|
147
|
+
raise InvalidLabelName, "reserved label name: :le" if labels.include?(:le)
|
|
148
|
+
|
|
149
|
+
super
|
|
150
|
+
end
|
|
146
151
|
end
|
|
147
152
|
end
|
|
148
153
|
end
|
|
@@ -7,6 +7,10 @@ module Fast
|
|
|
7
7
|
module Prometheus
|
|
8
8
|
# Metric is the abstract base class for all metric types.
|
|
9
9
|
#
|
|
10
|
+
# Declared label names are normalized to Symbols and label values (and the
|
|
11
|
+
# docstring) to valid UTF-8 here, where they enter the store, so no reader
|
|
12
|
+
# or renderer downstream can meet a mixed encoding or an invalid one.
|
|
13
|
+
#
|
|
10
14
|
# Every metric's per-series storage is a Store, guarded by its own lock.
|
|
11
15
|
# Metrics produced by #with_labels share their parent's Store (and thus
|
|
12
16
|
# its lock), so a mutation on a bound metric and a mutation on its parent
|
|
@@ -21,13 +25,14 @@ module Fast
|
|
|
21
25
|
def initialize(name, docstring:, labels: [], preset_labels: {}, store: nil)
|
|
22
26
|
validate_metric_name(name)
|
|
23
27
|
validate_docstring(docstring)
|
|
24
|
-
validate_label_names(labels)
|
|
25
|
-
validate_preset_labels(labels, preset_labels)
|
|
26
|
-
|
|
27
28
|
@name = name.to_s.to_sym
|
|
28
|
-
@docstring = docstring
|
|
29
|
-
|
|
30
|
-
@
|
|
29
|
+
@docstring = normalize_text(docstring)
|
|
30
|
+
|
|
31
|
+
@labels = labels.map { |label| label.to_s.to_sym }.freeze
|
|
32
|
+
validate_label_names(@labels)
|
|
33
|
+
@preset_labels = preset_labels.to_h { |key, value| [key.to_s.to_sym, normalize_text(value).freeze] }
|
|
34
|
+
validate_preset_labels(@labels, @preset_labels)
|
|
35
|
+
|
|
31
36
|
@store = store || Store.new
|
|
32
37
|
|
|
33
38
|
return unless fully_bound?
|
|
@@ -59,7 +64,7 @@ module Fast
|
|
|
59
64
|
|
|
60
65
|
def with_labels(**labels)
|
|
61
66
|
validate_label_keys(labels)
|
|
62
|
-
merged = @preset_labels.merge(labels.transform_values { |v| v.
|
|
67
|
+
merged = @preset_labels.merge(labels.transform_values { |v| normalize_text(v).freeze })
|
|
63
68
|
self.class.new(
|
|
64
69
|
@name,
|
|
65
70
|
docstring: @docstring,
|
|
@@ -109,7 +114,7 @@ module Fast
|
|
|
109
114
|
|
|
110
115
|
@labels.map do |n|
|
|
111
116
|
if labels.key?(n)
|
|
112
|
-
labels[n]
|
|
117
|
+
normalize_text(labels[n])
|
|
113
118
|
else
|
|
114
119
|
value = @preset_labels[n]
|
|
115
120
|
raise InvalidLabelSet, "missing labels: #{n.inspect}" unless value
|
|
@@ -164,6 +169,29 @@ module Fast
|
|
|
164
169
|
@preset_labels.size == @labels.size
|
|
165
170
|
end
|
|
166
171
|
|
|
172
|
+
# Label values and docstrings are stored as valid UTF-8 bytes. An
|
|
173
|
+
# ASCII-only String already is one, whatever encoding it is tagged with
|
|
174
|
+
# (US-ASCII from Integer#to_s or Symbol#to_s, BINARY from a Rack env),
|
|
175
|
+
# so it is returned as is: #ascii_only? is coderange-cached, and the hot
|
|
176
|
+
# path allocates nothing per value. Only a non-ASCII String pays for the
|
|
177
|
+
# tag check, and valid UTF-8 is still returned as is — String#scrub
|
|
178
|
+
# would copy.
|
|
179
|
+
def normalize_text(value)
|
|
180
|
+
string = value.is_a?(String) ? value : value.to_s
|
|
181
|
+
return string if string.ascii_only?
|
|
182
|
+
|
|
183
|
+
case string.encoding
|
|
184
|
+
when Encoding::UTF_8
|
|
185
|
+
string.valid_encoding? ? string : string.scrub
|
|
186
|
+
when Encoding::BINARY
|
|
187
|
+
# Reinterpreted, never transcoded: BINARY is bytes, not a charset.
|
|
188
|
+
# #scrub! returns self, so the dup is the only object either way.
|
|
189
|
+
string.dup.force_encoding(Encoding::UTF_8).scrub!
|
|
190
|
+
else
|
|
191
|
+
string.encode(Encoding::UTF_8, invalid: :replace, undef: :replace)
|
|
192
|
+
end
|
|
193
|
+
end
|
|
194
|
+
|
|
167
195
|
def validate_metric_name(name)
|
|
168
196
|
return if name.to_s.match?(METRIC_NAME)
|
|
169
197
|
|
|
@@ -176,12 +204,18 @@ module Fast
|
|
|
176
204
|
raise ArgumentError, "docstring must be a non-empty string"
|
|
177
205
|
end
|
|
178
206
|
|
|
207
|
+
# Validates the already-normalized Symbol names as a set: each one a
|
|
208
|
+
# legal, unreserved label name, and no two of them the same.
|
|
179
209
|
def validate_label_names(labels)
|
|
180
210
|
labels.each do |label|
|
|
181
|
-
name = label.
|
|
211
|
+
name = label.name
|
|
182
212
|
raise InvalidLabelName, "label name must not start with __: #{label.inspect}" if name.start_with?("__")
|
|
183
213
|
raise InvalidLabelName, "invalid label name: #{label.inspect}" unless name.match?(LABEL_NAME)
|
|
184
214
|
end
|
|
215
|
+
|
|
216
|
+
return if labels.uniq.size == labels.size
|
|
217
|
+
|
|
218
|
+
raise InvalidLabelName, "duplicate label names: #{labels.inspect}"
|
|
185
219
|
end
|
|
186
220
|
|
|
187
221
|
def validate_preset_labels(labels, preset_labels)
|
|
@@ -14,16 +14,24 @@ module Fast
|
|
|
14
14
|
super(delegate)
|
|
15
15
|
@registry = registry
|
|
16
16
|
@path = path
|
|
17
|
+
@query_prefix = "#{path}?".freeze
|
|
17
18
|
end
|
|
18
19
|
|
|
19
20
|
def call(request)
|
|
20
|
-
return serve_metrics(request) if request.method == "GET" && request.path
|
|
21
|
+
return serve_metrics(request) if request.method == "GET" && metrics_target?(request.path)
|
|
21
22
|
|
|
22
23
|
super
|
|
23
24
|
end
|
|
24
25
|
|
|
25
26
|
private
|
|
26
27
|
|
|
28
|
+
# A Protocol::HTTP request target carries its query string ("/metrics?x=1"),
|
|
29
|
+
# unlike Rack's PATH_INFO, so match the path component: the path itself,
|
|
30
|
+
# or the path followed by any query.
|
|
31
|
+
def metrics_target?(target)
|
|
32
|
+
target == @path || target.start_with?(@query_prefix)
|
|
33
|
+
end
|
|
34
|
+
|
|
27
35
|
def serve_metrics(request)
|
|
28
36
|
body, headers = Exposition.render(
|
|
29
37
|
@registry,
|
|
@@ -39,6 +39,12 @@ module Fast
|
|
|
39
39
|
# client_golang math.MaxInt32 convention for clamped ±Inf bucket index.
|
|
40
40
|
MAX_BUCKET_INDEX = (2**31) - 1
|
|
41
41
|
|
|
42
|
+
# Powers of two a subnormal is scaled up by before its bucket is found —
|
|
43
|
+
# enough to bring the smallest of them (2**-1074) well into the normal
|
|
44
|
+
# range, and a multiple of 2**4 so the index shifts by a whole bucket at
|
|
45
|
+
# every schema.
|
|
46
|
+
SUBNORMAL_SHIFT = 64
|
|
47
|
+
|
|
42
48
|
attr_reader :schema, :zero_threshold, :max_buckets
|
|
43
49
|
|
|
44
50
|
def initialize(name, docstring:, labels: [], preset_labels: {}, schema: 3, zero_threshold: 2.0**-128,
|
|
@@ -63,8 +69,11 @@ module Fast
|
|
|
63
69
|
:native_histogram
|
|
64
70
|
end
|
|
65
71
|
|
|
66
|
-
# Record an observation.
|
|
72
|
+
# Record an observation. A non-Numeric is an ArgumentError, as it is on
|
|
73
|
+
# every other type; NaN and ±Inf are handled gracefully and never raise.
|
|
67
74
|
def observe(value, labels: NO_LABELS)
|
|
75
|
+
raise ArgumentError, "value must be a numeric" unless value.is_a?(Numeric)
|
|
76
|
+
|
|
68
77
|
v = value.to_f
|
|
69
78
|
key = resolve(labels)
|
|
70
79
|
|
|
@@ -135,7 +144,19 @@ module Fast
|
|
|
135
144
|
slot.positive_buckets, slot.negative_buckets).freeze
|
|
136
145
|
end
|
|
137
146
|
|
|
147
|
+
# ceil(log2(value) * 2**schema), the log2 estimate corrected against the
|
|
148
|
+
# bucket bounds either side of it.
|
|
149
|
+
#
|
|
150
|
+
# A subnormal value is scaled into the normal range first: its bounds
|
|
151
|
+
# are subnormal too, and the 2.0**x below underflows to zero among them,
|
|
152
|
+
# which puts the smallest values in the wrong bucket. Scaling by a power
|
|
153
|
+
# of two is exact, and moves every index by that power times the schema.
|
|
138
154
|
def index_for(value, schema)
|
|
155
|
+
if value < Float::MIN
|
|
156
|
+
shift = schema.positive? ? SUBNORMAL_SHIFT << schema : SUBNORMAL_SHIFT >> -schema
|
|
157
|
+
return index_for(Math.ldexp(value, SUBNORMAL_SHIFT), schema) - shift
|
|
158
|
+
end
|
|
159
|
+
|
|
139
160
|
factor = 2.0**schema
|
|
140
161
|
idx = (Math.log2(value) * factor).ceil
|
|
141
162
|
idx -= 1 while 2.0**((idx - 1) / factor) >= value
|
|
@@ -11,12 +11,20 @@ module Fast
|
|
|
11
11
|
CUMULATIVE = :AGGREGATION_TEMPORALITY_CUMULATIVE
|
|
12
12
|
private_constant :CUMULATIVE
|
|
13
13
|
|
|
14
|
+
# OTLP carries one contiguous bucket-count array per side, so an export
|
|
15
|
+
# must never size an Array by the distance between two observations:
|
|
16
|
+
# past this many slots a data point drops to a coarser scale instead.
|
|
17
|
+
MAX_DENSE_BUCKETS = 1024
|
|
18
|
+
private_constant :MAX_DENSE_BUCKETS
|
|
19
|
+
|
|
14
20
|
def initialize(resource_attributes: {}, start_time: Time.now)
|
|
15
|
-
@resource_attributes = resource_attributes.map { |
|
|
16
|
-
@start_time_unix_nano = (start_time
|
|
21
|
+
@resource_attributes = resource_attributes.map { |key, value| kv(key, any_value(value)) }
|
|
22
|
+
@start_time_unix_nano = unix_nano(start_time)
|
|
17
23
|
end
|
|
18
24
|
|
|
19
25
|
def request(snapshot)
|
|
26
|
+
time_unix_nano = unix_nano(snapshot.taken_at)
|
|
27
|
+
|
|
20
28
|
Proto::ExportMetricsServiceRequest.new(
|
|
21
29
|
resource_metrics: [
|
|
22
30
|
Proto::ResourceMetrics.new(
|
|
@@ -29,7 +37,7 @@ module Fast
|
|
|
29
37
|
name: "fast-prometheus",
|
|
30
38
|
version: Fast::Prometheus::VERSION
|
|
31
39
|
),
|
|
32
|
-
metrics: snapshot.metrics.map { |ms| build_metric(ms,
|
|
40
|
+
metrics: snapshot.metrics.map { |ms| build_metric(ms, time_unix_nano) }
|
|
33
41
|
)
|
|
34
42
|
]
|
|
35
43
|
)
|
|
@@ -39,9 +47,12 @@ module Fast
|
|
|
39
47
|
|
|
40
48
|
private
|
|
41
49
|
|
|
42
|
-
|
|
43
|
-
|
|
50
|
+
# Exact nanoseconds: at epoch magnitudes a Float loses the low ~256ns.
|
|
51
|
+
def unix_nano(time)
|
|
52
|
+
(time.tv_sec * 1_000_000_000) + time.tv_nsec
|
|
53
|
+
end
|
|
44
54
|
|
|
55
|
+
def build_metric(metric_snapshot, time_unix_nano)
|
|
45
56
|
Proto::Metric.new(
|
|
46
57
|
name: metric_snapshot.name.to_s,
|
|
47
58
|
description: metric_snapshot.docstring,
|
|
@@ -154,21 +165,79 @@ module Fast
|
|
|
154
165
|
)
|
|
155
166
|
end
|
|
156
167
|
|
|
168
|
+
# A data point carries only the series' finite observations: OTLP has no
|
|
169
|
+
# bucket for an infinity and no count for a NaN, so both are dropped —
|
|
170
|
+
# from the buckets, from +count+, and from +sum+, which is left absent
|
|
171
|
+
# when the series' accumulated sum is no longer finite.
|
|
157
172
|
def exponential_histogram_data_point(attributes, value, time_unix_nano)
|
|
173
|
+
positive = finite_buckets(value.positive_buckets, value.schema)
|
|
174
|
+
negative = finite_buckets(value.negative_buckets, value.schema)
|
|
175
|
+
reduction = scale_reduction(positive, negative)
|
|
176
|
+
positive = merge_buckets(positive, reduction)
|
|
177
|
+
negative = merge_buckets(negative, reduction)
|
|
178
|
+
|
|
158
179
|
Proto::ExponentialHistogramDataPoint.new(
|
|
159
180
|
attributes: attributes,
|
|
160
181
|
start_time_unix_nano: @start_time_unix_nano,
|
|
161
182
|
time_unix_nano: time_unix_nano,
|
|
162
|
-
count: value.
|
|
163
|
-
sum: value.sum,
|
|
164
|
-
scale: value.schema,
|
|
183
|
+
count: value.zero_count + bucket_total(positive) + bucket_total(negative),
|
|
184
|
+
sum: (value.sum if value.sum.finite?),
|
|
185
|
+
scale: value.schema - reduction,
|
|
165
186
|
zero_count: value.zero_count,
|
|
166
187
|
zero_threshold: value.zero_threshold,
|
|
167
|
-
positive: dense_buckets(
|
|
168
|
-
negative: dense_buckets(
|
|
188
|
+
positive: dense_buckets(positive),
|
|
189
|
+
negative: dense_buckets(negative)
|
|
169
190
|
)
|
|
170
191
|
end
|
|
171
192
|
|
|
193
|
+
# NativeHistogram clamps a ±Inf observation to MAX_BUCKET_INDEX, and
|
|
194
|
+
# downscaling halves that index along with every other one. Either way
|
|
195
|
+
# it stays above the index of any finite observation, which |value| <=
|
|
196
|
+
# Float::MAX puts at 1024 * 2**schema at the most, so it sorts last and
|
|
197
|
+
# is the only pair up there.
|
|
198
|
+
def finite_buckets(buckets, schema)
|
|
199
|
+
return buckets if buckets.empty? || buckets.last.first <= 1024 * (2.0**schema)
|
|
200
|
+
|
|
201
|
+
buckets[0..-2]
|
|
202
|
+
end
|
|
203
|
+
|
|
204
|
+
def bucket_total(buckets)
|
|
205
|
+
buckets.sum { |_, count| count }
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
# How many times both sides' resolution has to halve for each dense
|
|
209
|
+
# array to fit MAX_DENSE_BUCKETS. A data point carries one scale, so the
|
|
210
|
+
# two sides reduce together.
|
|
211
|
+
def scale_reduction(positive, negative)
|
|
212
|
+
reduction = 0
|
|
213
|
+
reduction += 1 while dense_length(positive, reduction) > MAX_DENSE_BUCKETS ||
|
|
214
|
+
dense_length(negative, reduction) > MAX_DENSE_BUCKETS
|
|
215
|
+
reduction
|
|
216
|
+
end
|
|
217
|
+
|
|
218
|
+
def dense_length(buckets, reduction)
|
|
219
|
+
return 0 if buckets.empty?
|
|
220
|
+
|
|
221
|
+
coarse_index(buckets.last.first, reduction) - coarse_index(buckets.first.first, reduction) + 1
|
|
222
|
+
end
|
|
223
|
+
|
|
224
|
+
# Merge each run of 2**reduction adjacent buckets into one, as
|
|
225
|
+
# NativeHistogram#downscale does for the schema it reports.
|
|
226
|
+
def merge_buckets(buckets, reduction)
|
|
227
|
+
return buckets if reduction.zero?
|
|
228
|
+
|
|
229
|
+
buckets.each_with_object({}) do |(index, count), merged|
|
|
230
|
+
key = coarse_index(index, reduction)
|
|
231
|
+
merged[key] = (merged[key] || 0) + count
|
|
232
|
+
end.to_a
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
# The bucket +index+ falls into after halving resolution +reduction+
|
|
236
|
+
# times: rounded toward the wider bucket, as downscaling rounds.
|
|
237
|
+
def coarse_index(index, reduction)
|
|
238
|
+
-(-index / (1 << reduction))
|
|
239
|
+
end
|
|
240
|
+
|
|
172
241
|
# Convert sparse [[prom_idx, count], ...] to OTLP Buckets{offset, bucket_counts}.
|
|
173
242
|
# OTLP index = prom index - 1. Offset = min OTLP index.
|
|
174
243
|
# Buckets are always sorted (from NativeHistogram::Slot#positive_buckets / #negative_buckets).
|
|
@@ -190,14 +259,23 @@ module Fast
|
|
|
190
259
|
end
|
|
191
260
|
|
|
192
261
|
def build_attributes(labels)
|
|
193
|
-
labels.map { |
|
|
262
|
+
labels.map { |name, value| kv(name, Proto::AnyValue.new(string_value: value.to_s)) }
|
|
194
263
|
end
|
|
195
264
|
|
|
196
|
-
def kv(key,
|
|
197
|
-
Proto::KeyValue.new(
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
265
|
+
def kv(key, any_value)
|
|
266
|
+
Proto::KeyValue.new(key: key.to_s, value: any_value)
|
|
267
|
+
end
|
|
268
|
+
|
|
269
|
+
# A resource attribute keeps its Ruby type where OTLP has a member for
|
|
270
|
+
# it; anything else goes over as its +to_s+.
|
|
271
|
+
def any_value(value)
|
|
272
|
+
case value
|
|
273
|
+
when String then Proto::AnyValue.new(string_value: value)
|
|
274
|
+
when Integer then Proto::AnyValue.new(int_value: value)
|
|
275
|
+
when Float then Proto::AnyValue.new(double_value: value)
|
|
276
|
+
when true, false then Proto::AnyValue.new(bool_value: value)
|
|
277
|
+
else Proto::AnyValue.new(string_value: value.to_s)
|
|
278
|
+
end
|
|
201
279
|
end
|
|
202
280
|
end
|
|
203
281
|
end
|
|
@@ -7,15 +7,41 @@ module Fast
|
|
|
7
7
|
# Owns the RED metrics (request count + duration) recorded for every HTTP
|
|
8
8
|
# request. Shared by Middleware::Instrumentation and Rack::Instrumentation
|
|
9
9
|
# so registration and recording exist exactly once.
|
|
10
|
+
#
|
|
11
|
+
# The method label is allowlisted, so the two metrics hold at most ten
|
|
12
|
+
# method values however many tokens clients invent — nothing reclaims a
|
|
13
|
+
# series once it exists.
|
|
10
14
|
class RequestMetrics
|
|
15
|
+
LABELS = %i[method status].freeze
|
|
16
|
+
|
|
17
|
+
# The RFC 9110 methods plus PATCH, each mapped to the frozen String the
|
|
18
|
+
# store keeps, so an allowlisted method costs a lookup and no object.
|
|
19
|
+
# Matching is case-sensitive (RFC 9110 section 9.1) and finds a
|
|
20
|
+
# BINARY-tagged token from a Rack env: ASCII-only Strings hash and
|
|
21
|
+
# compare equal whatever their encoding tag.
|
|
22
|
+
METHODS = %w[GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATCH].to_h { |name| [name, name] }.freeze
|
|
23
|
+
|
|
24
|
+
# What every other token is counted as, the value OpenTelemetry's HTTP
|
|
25
|
+
# semantic conventions collapse an unrecognized method to.
|
|
26
|
+
OTHER_METHOD = "_OTHER"
|
|
27
|
+
|
|
28
|
+
# The kinds #record can drive under each name, and that each name
|
|
29
|
+
# promises whoever scrapes it. A Gauge answers #increment, but a gauge
|
|
30
|
+
# under "_total" is not a counter to a scraper; everything that answers
|
|
31
|
+
# #observe is a duration metric.
|
|
32
|
+
COUNTER_TYPES = %i[counter].freeze
|
|
33
|
+
DURATION_TYPES = %i[histogram native_histogram summary].freeze
|
|
34
|
+
|
|
35
|
+
private_constant :LABELS, :METHODS, :OTHER_METHOD, :COUNTER_TYPES, :DURATION_TYPES
|
|
36
|
+
|
|
11
37
|
def initialize(registry:, native: false, prefix: "http_server")
|
|
12
38
|
@registry = registry
|
|
13
|
-
@counter = ensure_counter(prefix)
|
|
14
|
-
@histogram = ensure_histogram(prefix, native)
|
|
39
|
+
@counter = validate(ensure_counter(prefix), COUNTER_TYPES)
|
|
40
|
+
@histogram = validate(ensure_histogram(prefix, native), DURATION_TYPES)
|
|
15
41
|
end
|
|
16
42
|
|
|
17
43
|
def record(method, status, started_at)
|
|
18
|
-
labels = { method: method, status: status }
|
|
44
|
+
labels = { method: METHODS[method] || OTHER_METHOD, status: status }
|
|
19
45
|
duration = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at
|
|
20
46
|
@counter.increment(labels: labels)
|
|
21
47
|
@histogram.observe(duration, labels: labels)
|
|
@@ -26,7 +52,7 @@ module Fast
|
|
|
26
52
|
def ensure_counter(prefix)
|
|
27
53
|
name = :"#{prefix}_requests_total"
|
|
28
54
|
@registry.fetch_or_register(name) do
|
|
29
|
-
Counter.new(name, docstring: "Total HTTP requests", labels:
|
|
55
|
+
Counter.new(name, docstring: "Total HTTP requests", labels: LABELS)
|
|
30
56
|
end
|
|
31
57
|
end
|
|
32
58
|
|
|
@@ -34,12 +60,28 @@ module Fast
|
|
|
34
60
|
name = :"#{prefix}_request_duration_seconds"
|
|
35
61
|
@registry.fetch_or_register(name) do
|
|
36
62
|
if native
|
|
37
|
-
NativeHistogram.new(name, docstring: "HTTP request duration in seconds", labels:
|
|
63
|
+
NativeHistogram.new(name, docstring: "HTTP request duration in seconds", labels: LABELS)
|
|
38
64
|
else
|
|
39
|
-
Histogram.new(name, docstring: "HTTP request duration in seconds", labels:
|
|
65
|
+
Histogram.new(name, docstring: "HTTP request duration in seconds", labels: LABELS)
|
|
40
66
|
end
|
|
41
67
|
end
|
|
42
68
|
end
|
|
69
|
+
|
|
70
|
+
# A metric already registered under one of these names is reused only if
|
|
71
|
+
# it is one of +types+ and declares this label set. Otherwise every
|
|
72
|
+
# request would raise from inside the app's request path; raise here
|
|
73
|
+
# instead, once, where the mistaken declaration is.
|
|
74
|
+
def validate(metric, types)
|
|
75
|
+
unless types.include?(metric.type)
|
|
76
|
+
raise InvalidMetricType, "#{metric.name} declares type #{metric.type.inspect}, not #{types.inspect}"
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
unless metric.labels == LABELS
|
|
80
|
+
raise InvalidLabelSet, "#{metric.name} declares labels #{metric.labels.inspect}, not #{LABELS.inspect}"
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
metric
|
|
84
|
+
end
|
|
43
85
|
end
|
|
44
86
|
end
|
|
45
87
|
end
|
|
@@ -40,17 +40,6 @@ module Fast
|
|
|
40
40
|
series_map { |slot| hash_shape(slot) }
|
|
41
41
|
end
|
|
42
42
|
|
|
43
|
-
protected
|
|
44
|
-
|
|
45
|
-
def validate_label_names(labels)
|
|
46
|
-
labels.each do |label|
|
|
47
|
-
name = label.to_s
|
|
48
|
-
raise InvalidLabelName, "reserved label name: :quantile" if name == "quantile"
|
|
49
|
-
end
|
|
50
|
-
|
|
51
|
-
super
|
|
52
|
-
end
|
|
53
|
-
|
|
54
43
|
private
|
|
55
44
|
|
|
56
45
|
def zero_value
|
|
@@ -64,6 +53,12 @@ module Fast
|
|
|
64
53
|
def hash_shape(slot)
|
|
65
54
|
{ "count" => slot.count, "sum" => slot.sum }
|
|
66
55
|
end
|
|
56
|
+
|
|
57
|
+
def validate_label_names(labels)
|
|
58
|
+
raise InvalidLabelName, "reserved label name: :quantile" if labels.include?(:quantile)
|
|
59
|
+
|
|
60
|
+
super
|
|
61
|
+
end
|
|
67
62
|
end
|
|
68
63
|
end
|
|
69
64
|
end
|