fast-prometheus 0.3.2 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 25cba069a74bd1f00499ff8d2c3a3feb84e9646b1447de704d2dff860dc5cea4
4
- data.tar.gz: 533131b2461c719cb5525695a5abdc26b474bc6ce9fe35ba338855a698e887a3
3
+ metadata.gz: 0421a8eb55a0265588296f0cdac33f1028cf6785c629a85f192f354bcec7d55a
4
+ data.tar.gz: bd02dcbb69a4a0f5bd5f592f36b9e984a3fc3cd74f2023fcb95c21f0d9662f10
5
5
  SHA512:
6
- metadata.gz: c0a779b8563993d01400a7587852f2ec65c65621facd1bd3e9c44d39404e44eff0656a137a12f6a8da1b60b902d6bfe897b35ab82627064c8238f005171d0623
7
- data.tar.gz: 8b9c2ce522dc9360c40f1779b7e4240370e5a7dbe4e6b333f9c39583bf2ba5a43805aa4b523763322ca62716b23c619047291beff28cb5b33abe88ad7874f96c
6
+ metadata.gz: bfb624426fc812ea430544cdc0f9d1690a2cde91d65949e77765730bde6ebc264cbb1e8c774c7faad6cb1da8202543f36f26e4bc987c7836aea1ec506bcc0763
7
+ data.tar.gz: 30dfdebab4860d83532092fa969298f8b38d79b18573980baaca8430b9a910e3aad4e541b6a71da0e41cca8a291d41ab0c28115e10abbaef0adc35caeca5241e
data/CHANGELOG.md CHANGED
@@ -7,6 +7,112 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.4.0] - 2026-09-19
11
+
12
+ ### Changed
13
+
14
+ - **Breaking:** declared label names are normalized to `Symbol`s at
15
+ construction. `labels: ["method"]` and `labels: [:method]` now declare the
16
+ same metric and `Metric#labels` reports `[:method]` for both, so a `String`
17
+ key at observe time — `increment(labels: { "method" => "GET" })` — is an
18
+ unknown label and raises `InvalidLabelSet` where it used to resolve. Two
19
+ names that normalize to the same `Symbol` (`%i[x x]`, `[:x, "x"]`) are a
20
+ duplicate and raise `InvalidLabelName`, as do the reserved names now that
21
+ they are checked after normalization: `"le"` on a `Histogram` and
22
+ `"quantile"` on a `Summary` raise where only `:le` and `:quantile` did. The
23
+ `labels:` and `buckets:` Arrays are copied and frozen, so mutating the Array
24
+ you passed no longer reaches into the metric.
25
+ - Label values and docstrings are normalized to valid UTF-8 at the boundary
26
+ where they enter the store, so no reader, renderer or exporter downstream can
27
+ meet a mixed or an invalid encoding. An ASCII-only String is kept exactly as
28
+ it is whatever its encoding tag (`"GET".b` out of a Rack env, `US-ASCII` out
29
+ of `200.to_s`); invalid UTF-8 is scrubbed; `BINARY` is reinterpreted as UTF-8
30
+ bytes and scrubbed, never transcoded; any other encoding is transcoded with
31
+ replacement. A registry holding label values in two encodings no longer fails
32
+ its scrape with `Encoding::CompatibilityError`, and `Formats::Text.render`
33
+ builds its body in UTF-8 from the first byte, matching the `charset=utf-8` it
34
+ advertises. Bytes that are not valid UTF-8 are replaced, not preserved — see
35
+ Normalization in the metrics reference. The cost is on the labeled write
36
+ path: about 5–10% fewer iterations per second in `benchmark/observe.rb`
37
+ against 0.3.2 in a paired run, with bound and unlabeled calls and every
38
+ allocation budget unchanged.
39
+ - The `method` label on `Middleware::Instrumentation` and
40
+ `Rack::Instrumentation` is allowlisted to the RFC 9110 methods plus `PATCH`;
41
+ every other token is counted as `method="_OTHER"`, the value OpenTelemetry's
42
+ HTTP semantic conventions collapse an unrecognized method to. An HTTP method
43
+ is any token both `Protocol::HTTP` and Rack accept, and nothing reclaims a
44
+ series once it exists, so the two RED metrics now hold at most ten `method`
45
+ values however many a client invents. Matching is case-sensitive, so `get` is
46
+ `_OTHER`.
47
+ - `Accept` and `Accept-Encoding` are parsed as RFC 9110 lists rather than
48
+ searched for a substring: the token is matched whole and case-insensitively,
49
+ `q=0` is a refusal, the higher `q` wins between
50
+ `application/vnd.google.protobuf` and `text/plain`, and a tie is protobuf.
51
+ `Accept: text/plain;version=0.0.4` no longer selects protobuf because the
52
+ word appears in a parameter, and `gzip;q=0` is honoured as the refusal it is.
53
+ Text remains the fallback for a request accepting neither format.
54
+ - `Middleware::Instrumentation` and `Rack::Instrumentation` validate a metric
55
+ already registered under either of their names before reusing it: it must
56
+ declare `labels: %i[method status]` and be a kind `record` can drive — a
57
+ counter under the counter name, something observable under the duration name.
58
+ A mismatch raises `InvalidMetricType` (kind) or `InvalidLabelSet` (labels)
59
+ from the constructor, at boot where the mistaken declaration is, instead of
60
+ once per request from inside the app's request path.
61
+ - **Breaking:** `OTLP::Mapper.new(start_time:)` takes a `Time`, and both it and
62
+ `snapshot.taken_at` are carried as exact nanoseconds (`tv_sec`/`tv_nsec`)
63
+ rather than through a `Float`, which lost the low ~256 ns at epoch
64
+ magnitudes. A `Float` start time no longer works.
65
+ - OTLP resource attributes keep their Ruby type — `String` to `string_value`,
66
+ `Integer` to `int_value`, `Float` to `double_value`, `true`/`false` to
67
+ `bool_value`, anything else its `to_s` — and their keys may be `Symbol`s,
68
+ which makes the `resource_attributes: { service: "my-app" }` in the
69
+ export-otlp how-to run as written. Data point label attributes stay
70
+ `string_value`.
71
+ - `Formats::Protobuf.render` omits a label whose value is the empty String,
72
+ which is implicit presence for a proto3 `string` and what `google-protobuf`
73
+ writes: a series carrying an empty label value is now byte-identical to the
74
+ reference encoder's output instead of a byte longer.
75
+ - `NativeHistogram#observe` raises `ArgumentError` unless its value is
76
+ `Numeric`, as every other metric type does; `NaN` and `±Infinity` are still
77
+ accepted and still never raise.
78
+ - `Histogram.new` rejects a non-finite bucket bound with `ArgumentError`. The
79
+ `+Inf` bucket is implicit and always rendered, so an explicit
80
+ `Float::INFINITY` (or `Float::NAN`) bound was a silent mistake, not a second
81
+ `+Inf`.
82
+ - Benchmarks re-taken on this tree against fast-protowire 0.3.0 and
83
+ republished on the benchmarks page and in the README.
84
+
85
+ ### Fixed
86
+
87
+ - `OTLP::Mapper` no longer sizes a bucket array by the distance between two
88
+ observations. A `:native_histogram` series clamps a `±Infinity` observation
89
+ to bucket index `2**31 - 1`, and OTLP carries one contiguous bucket-count
90
+ array per side, so exporting such a series allocated an Array of billions of
91
+ slots. An `ExponentialHistogram` data point now carries the series' finite
92
+ observations only — the clamp bucket and `NaN`s are left out of the bucket
93
+ arrays and out of `count`, and `sum` is omitted, as the field is `optional`,
94
+ once the accumulated sum is no longer finite, which is what the OTel SDKs do
95
+ — and a series needing more than 1024 dense slots on either side is merged to
96
+ a coarser exported `scale` until it fits. The Prometheus-side value is
97
+ untouched; `Formats::Protobuf` still exposes every observation.
98
+ - `NativeHistogram` puts a subnormal observation (under `Float::MIN`) in the
99
+ right bucket. The bucket bounds down there are subnormal too and the `2.0**x`
100
+ the index search compares against underflows to zero among them, so a
101
+ subnormal landed in a neighbouring bucket; it is now scaled into the normal
102
+ range by a power of two first, which is exact, and its index shifted back.
103
+ - `Middleware::Exporter` serves `/metrics?x=1`. A `Protocol::HTTP` request
104
+ target carries its query string, unlike Rack's `PATH_INFO`, so the exact
105
+ match on the path meant a Prometheus scrape config with `params:` fell
106
+ through to the wrapped app. The path component is now matched; `/metrics/`,
107
+ `/METRICS` and `/metricsx` still delegate.
108
+ - Content negotiation is a function of the request header's bytes, whatever
109
+ encoding the String is tagged with, and never raises. A UTF-8-tagged header
110
+ holding invalid bytes — what a client can send and a Rack harness hands over
111
+ as it arrived — used to raise `ArgumentError` out of `Exposition.render` and
112
+ fail the scrape; a header that cannot be made sense of now simply scores
113
+ nothing, which is text and no compression, and a well-formed member beside
114
+ garbage still selects.
115
+
10
116
  ## [0.3.2] - 2026-09-19
11
117
 
12
118
  ### Changed
data/README.md CHANGED
@@ -98,10 +98,10 @@ comparisons against `prometheus-client` and `google-protobuf`; see
98
98
  `google-protobuf` encoder needed 3.5 million Ruby objects and 504k native arenas, and
99
99
  in a hundredth of the GC time; taking the snapshot it reads is 1 ms and one Hash per metric
100
100
  - The text renderer allocates **66x fewer objects** than prometheus-client's formatter
101
- for the same body, and renders it 3.2x faster at 36,000 series
101
+ for the same body, and renders it 3.1x faster at 36,000 series
102
102
  - Bound counter and gauge writes allocate nothing and are **1.45x** faster than
103
- prometheus-client's; histogram observe is **3.1x** faster and summary observe **3.3x**
104
- - Native histogram observe runs at **1.32M i/s** with no prometheus-client equivalent
103
+ prometheus-client's; histogram observe is **2.9x** faster and summary observe **3.1x**
104
+ - Native histogram observe runs at **1.2M i/s** with no prometheus-client equivalent
105
105
 
106
106
  ## Development
107
107
 
@@ -6,6 +6,7 @@ module Fast
6
6
  class InvalidMetricName < Error; end
7
7
  class InvalidLabelName < Error; end
8
8
  class InvalidLabelSet < Error; end
9
+ class InvalidMetricType < Error; end
9
10
  class DuplicateMetric < Error; end
10
11
  end
11
12
  end
@@ -12,8 +12,17 @@ module Fast
12
12
  # negotiating format (text vs protobuf) and gzip from Accept / Accept-Encoding
13
13
  # header values. Shared by Middleware::Exporter and Rack::Exporter so
14
14
  # negotiation and compression exist exactly once.
15
+ #
16
+ # Both headers are read as RFC 9110 lists over their bytes: comma-separated
17
+ # members, each a token with optional ";"-parameters. The token is matched
18
+ # whole and case-insensitively, parameters other than "q" are ignored, and
19
+ # "q=0" is a refusal. Nothing here raises: a header we cannot make sense of
20
+ # — invalid bytes included — simply scores nothing, which is the text
21
+ # format and no compression.
15
22
  module Exposition
16
23
  PROTOBUF_ACCEPT = "application/vnd.google.protobuf"
24
+ TEXT_ACCEPT = "text/plain"
25
+ GZIP_CODING = "gzip"
17
26
 
18
27
  FORMAT_MAP = {
19
28
  true => [Formats::Protobuf, Formats::Protobuf::CONTENT_TYPE],
@@ -22,11 +31,11 @@ module Fast
22
31
  private_constant :FORMAT_MAP
23
32
 
24
33
  def self.render(registry, accept:, accept_encoding:)
25
- format, content_type = FORMAT_MAP[protobuf?(accept)]
34
+ format, content_type = FORMAT_MAP[protobuf?(byte_view(accept))]
26
35
  body = format.render(registry.collect)
27
36
  headers = { "content-type" => content_type }
28
37
 
29
- if gzip?(accept_encoding)
38
+ if gzip?(byte_view(accept_encoding))
30
39
  body = Zlib.gzip(body)
31
40
  headers["content-encoding"] = "gzip"
32
41
  end
@@ -34,12 +43,51 @@ module Fast
34
43
  [body, headers]
35
44
  end
36
45
 
46
+ # Protobuf wins when it is acceptable and no less acceptable than text —
47
+ # on a tie, the order Prometheus itself lists the two in.
37
48
  private_class_method def self.protobuf?(accept)
38
- accept.to_s.include?(PROTOBUF_ACCEPT)
49
+ quality = quality_of(accept, PROTOBUF_ACCEPT)
50
+ quality.positive? && quality >= quality_of(accept, TEXT_ACCEPT)
39
51
  end
40
52
 
41
53
  private_class_method def self.gzip?(accept_encoding)
42
- accept_encoding.to_s.include?("gzip")
54
+ quality_of(accept_encoding, GZIP_CODING).positive?
55
+ end
56
+
57
+ # The header's bytes, whatever String the server handed us: negotiation
58
+ # answers the bytes and nothing else. #strip and #casecmp? raise on a
59
+ # UTF-8-tagged String holding invalid ones — and io-stream, Puma and a
60
+ # test harness tag the same bytes BINARY, BINARY and UTF-8 — while BINARY
61
+ # bytes are always valid, so nothing below can raise. One String per
62
+ # header on a path that renders the whole registry anyway.
63
+ private_class_method def self.byte_view(header)
64
+ header.to_s.b
65
+ end
66
+
67
+ # The highest q value +token+ carries across the members of a list header:
68
+ # 0.0 when it is absent or refused with q=0, 1.0 when it is listed without
69
+ # one. #split with a block keeps a long header's members out of an Array.
70
+ private_class_method def self.quality_of(header, token)
71
+ best = 0.0
72
+ header.split(",") do |member|
73
+ name, _, parameters = member.partition(";")
74
+ next unless name.strip.casecmp?(token)
75
+
76
+ quality = quality_parameter(parameters)
77
+ best = quality if quality > best
78
+ end
79
+ best
80
+ end
81
+
82
+ # One member's q parameter, 1.0 when it has none. A quoted parameter value
83
+ # holding ";" or "=" can only hide a q from us, never fake one, and a
84
+ # hidden q is the default anyway.
85
+ private_class_method def self.quality_parameter(parameters)
86
+ parameters.split(";") do |parameter|
87
+ name, _, value = parameter.partition("=")
88
+ return value.strip.to_f if name.strip.casecmp?("q")
89
+ end
90
+ 1.0
43
91
  end
44
92
  end
45
93
  end
@@ -105,16 +105,20 @@ module Fast
105
105
  end
106
106
  end
107
107
 
108
- # LabelPair { string name = 1; string value = 2 }.
108
+ # LabelPair { string name = 1; string value = 2 }. Implicit presence:
109
+ # an empty value is the field's default and is not sent, as the
110
+ # declared Proto::LabelPair (and google-protobuf) leave it out.
109
111
  private_class_method def self.append_label_pair(out, name, value)
110
112
  name = name.name if name.is_a?(Symbol)
111
113
  value = value.to_s
112
114
  name_size = name.bytesize
113
115
  value_size = value.bytesize
116
+ size = 1 + varint_size(name_size) + name_size
117
+ size += 1 + varint_size(value_size) + value_size unless value.empty?
114
118
  out << METRIC_LABEL
115
- append_varint(out, 2 + varint_size(name_size) + name_size + varint_size(value_size) + value_size)
119
+ append_varint(out, size)
116
120
  Fast::Protowire::Wire.append_length_delimited(out, LABEL_NAME, name)
117
- Fast::Protowire::Wire.append_length_delimited(out, LABEL_VALUE, value)
121
+ Fast::Protowire::Wire.append_length_delimited(out, LABEL_VALUE, value) unless value.empty?
118
122
  end
119
123
 
120
124
  # Counter / Gauge { double value = 1 }: empty when the value is the
@@ -11,6 +11,10 @@ module Fast
11
11
  # Every piece is appended straight to the output buffer: no line, label
12
12
  # or name is built as its own String first, so a render allocates one
13
13
  # String per sample (the value's to_s) and nothing per label.
14
+ #
15
+ # The buffer is UTF-8 from the start and the snapshot's docstrings and
16
+ # label values are already valid UTF-8 (see Metric), so the body is
17
+ # always a UTF-8-tagged, valid String whatever it holds.
14
18
  module Text
15
19
  CONTENT_TYPE = "text/plain; version=0.0.4; charset=utf-8"
16
20
 
@@ -21,7 +25,7 @@ module Fast
21
25
  LABEL_REPLACE = { "\\" => "\\\\", '"' => '\\"', "\n" => "\\n" }.freeze
22
26
 
23
27
  def self.render(snapshot)
24
- snapshot.metrics.each_with_object(String.new) do |metric, output|
28
+ snapshot.metrics.each_with_object(String.new(encoding: Encoding::UTF_8)) do |metric, output|
25
29
  next if metric.type == :native_histogram
26
30
 
27
31
  name = metric.name.name
@@ -46,11 +46,10 @@ module Fast
46
46
  def initialize(name, docstring:, labels: [], preset_labels: {}, buckets: DEFAULT_BUCKETS, store: nil)
47
47
  raise ArgumentError, "buckets must be a non-empty Array" unless buckets.is_a?(Array) && !buckets.empty?
48
48
  raise ArgumentError, "buckets must contain only Numeric values" unless buckets.all? { |b| b.is_a?(Numeric) }
49
+ raise ArgumentError, "buckets must contain only finite values" unless buckets.all?(&:finite?)
49
50
  raise ArgumentError, "buckets must be strictly ascending" unless buckets.each_cons(2).all? { |a, b| a < b }
50
51
 
51
- raise InvalidLabelName, "label :le is reserved" if labels.include?(:le)
52
-
53
- @buckets = buckets
52
+ @buckets = buckets.dup.freeze
54
53
  super(name, docstring: docstring, labels: labels, preset_labels: preset_labels, store: store)
55
54
  end
56
55
 
@@ -143,6 +142,12 @@ module Fast
143
142
  hash["sum"] = slot.sum
144
143
  hash
145
144
  end
145
+
146
+ def validate_label_names(labels)
147
+ raise InvalidLabelName, "reserved label name: :le" if labels.include?(:le)
148
+
149
+ super
150
+ end
146
151
  end
147
152
  end
148
153
  end
@@ -7,6 +7,10 @@ module Fast
7
7
  module Prometheus
8
8
  # Metric is the abstract base class for all metric types.
9
9
  #
10
+ # Declared label names are normalized to Symbols and label values (and the
11
+ # docstring) to valid UTF-8 here, where they enter the store, so no reader
12
+ # or renderer downstream can meet a mixed encoding or an invalid one.
13
+ #
10
14
  # Every metric's per-series storage is a Store, guarded by its own lock.
11
15
  # Metrics produced by #with_labels share their parent's Store (and thus
12
16
  # its lock), so a mutation on a bound metric and a mutation on its parent
@@ -21,13 +25,14 @@ module Fast
21
25
  def initialize(name, docstring:, labels: [], preset_labels: {}, store: nil)
22
26
  validate_metric_name(name)
23
27
  validate_docstring(docstring)
24
- validate_label_names(labels)
25
- validate_preset_labels(labels, preset_labels)
26
-
27
28
  @name = name.to_s.to_sym
28
- @docstring = docstring
29
- @labels = labels
30
- @preset_labels = preset_labels.transform_values { |v| v.to_s.freeze }
29
+ @docstring = normalize_text(docstring)
30
+
31
+ @labels = labels.map { |label| label.to_s.to_sym }.freeze
32
+ validate_label_names(@labels)
33
+ @preset_labels = preset_labels.to_h { |key, value| [key.to_s.to_sym, normalize_text(value).freeze] }
34
+ validate_preset_labels(@labels, @preset_labels)
35
+
31
36
  @store = store || Store.new
32
37
 
33
38
  return unless fully_bound?
@@ -59,7 +64,7 @@ module Fast
59
64
 
60
65
  def with_labels(**labels)
61
66
  validate_label_keys(labels)
62
- merged = @preset_labels.merge(labels.transform_values { |v| v.to_s.freeze })
67
+ merged = @preset_labels.merge(labels.transform_values { |v| normalize_text(v).freeze })
63
68
  self.class.new(
64
69
  @name,
65
70
  docstring: @docstring,
@@ -109,7 +114,7 @@ module Fast
109
114
 
110
115
  @labels.map do |n|
111
116
  if labels.key?(n)
112
- labels[n].to_s
117
+ normalize_text(labels[n])
113
118
  else
114
119
  value = @preset_labels[n]
115
120
  raise InvalidLabelSet, "missing labels: #{n.inspect}" unless value
@@ -164,6 +169,29 @@ module Fast
164
169
  @preset_labels.size == @labels.size
165
170
  end
166
171
 
172
+ # Label values and docstrings are stored as valid UTF-8 bytes. An
173
+ # ASCII-only String already is one, whatever encoding it is tagged with
174
+ # (US-ASCII from Integer#to_s or Symbol#to_s, BINARY from a Rack env),
175
+ # so it is returned as is: #ascii_only? is coderange-cached, and the hot
176
+ # path allocates nothing per value. Only a non-ASCII String pays for the
177
+ # tag check, and valid UTF-8 is still returned as is — String#scrub
178
+ # would copy.
179
+ def normalize_text(value)
180
+ string = value.is_a?(String) ? value : value.to_s
181
+ return string if string.ascii_only?
182
+
183
+ case string.encoding
184
+ when Encoding::UTF_8
185
+ string.valid_encoding? ? string : string.scrub
186
+ when Encoding::BINARY
187
+ # Reinterpreted, never transcoded: BINARY is bytes, not a charset.
188
+ # #scrub! returns self, so the dup is the only object either way.
189
+ string.dup.force_encoding(Encoding::UTF_8).scrub!
190
+ else
191
+ string.encode(Encoding::UTF_8, invalid: :replace, undef: :replace)
192
+ end
193
+ end
194
+
167
195
  def validate_metric_name(name)
168
196
  return if name.to_s.match?(METRIC_NAME)
169
197
 
@@ -176,12 +204,18 @@ module Fast
176
204
  raise ArgumentError, "docstring must be a non-empty string"
177
205
  end
178
206
 
207
+ # Validates the already-normalized Symbol names as a set: each one a
208
+ # legal, unreserved label name, and no two of them the same.
179
209
  def validate_label_names(labels)
180
210
  labels.each do |label|
181
- name = label.to_s
211
+ name = label.name
182
212
  raise InvalidLabelName, "label name must not start with __: #{label.inspect}" if name.start_with?("__")
183
213
  raise InvalidLabelName, "invalid label name: #{label.inspect}" unless name.match?(LABEL_NAME)
184
214
  end
215
+
216
+ return if labels.uniq.size == labels.size
217
+
218
+ raise InvalidLabelName, "duplicate label names: #{labels.inspect}"
185
219
  end
186
220
 
187
221
  def validate_preset_labels(labels, preset_labels)
@@ -14,16 +14,24 @@ module Fast
14
14
  super(delegate)
15
15
  @registry = registry
16
16
  @path = path
17
+ @query_prefix = "#{path}?".freeze
17
18
  end
18
19
 
19
20
  def call(request)
20
- return serve_metrics(request) if request.method == "GET" && request.path == @path
21
+ return serve_metrics(request) if request.method == "GET" && metrics_target?(request.path)
21
22
 
22
23
  super
23
24
  end
24
25
 
25
26
  private
26
27
 
28
+ # A Protocol::HTTP request target carries its query string ("/metrics?x=1"),
29
+ # unlike Rack's PATH_INFO, so match the path component: the path itself,
30
+ # or the path followed by any query.
31
+ def metrics_target?(target)
32
+ target == @path || target.start_with?(@query_prefix)
33
+ end
34
+
27
35
  def serve_metrics(request)
28
36
  body, headers = Exposition.render(
29
37
  @registry,
@@ -39,6 +39,12 @@ module Fast
39
39
  # client_golang math.MaxInt32 convention for clamped ±Inf bucket index.
40
40
  MAX_BUCKET_INDEX = (2**31) - 1
41
41
 
42
+ # Powers of two a subnormal is scaled up by before its bucket is found —
43
+ # enough to bring the smallest of them (2**-1074) well into the normal
44
+ # range, and a multiple of 2**4 so the index shifts by a whole bucket at
45
+ # every schema.
46
+ SUBNORMAL_SHIFT = 64
47
+
42
48
  attr_reader :schema, :zero_threshold, :max_buckets
43
49
 
44
50
  def initialize(name, docstring:, labels: [], preset_labels: {}, schema: 3, zero_threshold: 2.0**-128,
@@ -63,8 +69,11 @@ module Fast
63
69
  :native_histogram
64
70
  end
65
71
 
66
- # Record an observation. Never raises NaN/±Inf are handled gracefully.
72
+ # Record an observation. A non-Numeric is an ArgumentError, as it is on
73
+ # every other type; NaN and ±Inf are handled gracefully and never raise.
67
74
  def observe(value, labels: NO_LABELS)
75
+ raise ArgumentError, "value must be a numeric" unless value.is_a?(Numeric)
76
+
68
77
  v = value.to_f
69
78
  key = resolve(labels)
70
79
 
@@ -135,7 +144,19 @@ module Fast
135
144
  slot.positive_buckets, slot.negative_buckets).freeze
136
145
  end
137
146
 
147
+ # ceil(log2(value) * 2**schema), the log2 estimate corrected against the
148
+ # bucket bounds either side of it.
149
+ #
150
+ # A subnormal value is scaled into the normal range first: its bounds
151
+ # are subnormal too, and the 2.0**x below underflows to zero among them,
152
+ # which puts the smallest values in the wrong bucket. Scaling by a power
153
+ # of two is exact, and moves every index by that power times the schema.
138
154
  def index_for(value, schema)
155
+ if value < Float::MIN
156
+ shift = schema.positive? ? SUBNORMAL_SHIFT << schema : SUBNORMAL_SHIFT >> -schema
157
+ return index_for(Math.ldexp(value, SUBNORMAL_SHIFT), schema) - shift
158
+ end
159
+
139
160
  factor = 2.0**schema
140
161
  idx = (Math.log2(value) * factor).ceil
141
162
  idx -= 1 while 2.0**((idx - 1) / factor) >= value
@@ -11,12 +11,20 @@ module Fast
11
11
  CUMULATIVE = :AGGREGATION_TEMPORALITY_CUMULATIVE
12
12
  private_constant :CUMULATIVE
13
13
 
14
+ # OTLP carries one contiguous bucket-count array per side, so an export
15
+ # must never size an Array by the distance between two observations:
16
+ # past this many slots a data point drops to a coarser scale instead.
17
+ MAX_DENSE_BUCKETS = 1024
18
+ private_constant :MAX_DENSE_BUCKETS
19
+
14
20
  def initialize(resource_attributes: {}, start_time: Time.now)
15
- @resource_attributes = resource_attributes.map { |k, v| kv(k, v) }
16
- @start_time_unix_nano = (start_time.to_f * 1_000_000_000).to_i
21
+ @resource_attributes = resource_attributes.map { |key, value| kv(key, any_value(value)) }
22
+ @start_time_unix_nano = unix_nano(start_time)
17
23
  end
18
24
 
19
25
  def request(snapshot)
26
+ time_unix_nano = unix_nano(snapshot.taken_at)
27
+
20
28
  Proto::ExportMetricsServiceRequest.new(
21
29
  resource_metrics: [
22
30
  Proto::ResourceMetrics.new(
@@ -29,7 +37,7 @@ module Fast
29
37
  name: "fast-prometheus",
30
38
  version: Fast::Prometheus::VERSION
31
39
  ),
32
- metrics: snapshot.metrics.map { |ms| build_metric(ms, snapshot.taken_at) }
40
+ metrics: snapshot.metrics.map { |ms| build_metric(ms, time_unix_nano) }
33
41
  )
34
42
  ]
35
43
  )
@@ -39,9 +47,12 @@ module Fast
39
47
 
40
48
  private
41
49
 
42
- def build_metric(metric_snapshot, taken_at)
43
- time_unix_nano = (taken_at.to_f * 1_000_000_000).to_i
50
+ # Exact nanoseconds: at epoch magnitudes a Float loses the low ~256ns.
51
+ def unix_nano(time)
52
+ (time.tv_sec * 1_000_000_000) + time.tv_nsec
53
+ end
44
54
 
55
+ def build_metric(metric_snapshot, time_unix_nano)
45
56
  Proto::Metric.new(
46
57
  name: metric_snapshot.name.to_s,
47
58
  description: metric_snapshot.docstring,
@@ -154,21 +165,79 @@ module Fast
154
165
  )
155
166
  end
156
167
 
168
+ # A data point carries only the series' finite observations: OTLP has no
169
+ # bucket for an infinity and no count for a NaN, so both are dropped —
170
+ # from the buckets, from +count+, and from +sum+, which is left absent
171
+ # when the series' accumulated sum is no longer finite.
157
172
  def exponential_histogram_data_point(attributes, value, time_unix_nano)
173
+ positive = finite_buckets(value.positive_buckets, value.schema)
174
+ negative = finite_buckets(value.negative_buckets, value.schema)
175
+ reduction = scale_reduction(positive, negative)
176
+ positive = merge_buckets(positive, reduction)
177
+ negative = merge_buckets(negative, reduction)
178
+
158
179
  Proto::ExponentialHistogramDataPoint.new(
159
180
  attributes: attributes,
160
181
  start_time_unix_nano: @start_time_unix_nano,
161
182
  time_unix_nano: time_unix_nano,
162
- count: value.count,
163
- sum: value.sum,
164
- scale: value.schema,
183
+ count: value.zero_count + bucket_total(positive) + bucket_total(negative),
184
+ sum: (value.sum if value.sum.finite?),
185
+ scale: value.schema - reduction,
165
186
  zero_count: value.zero_count,
166
187
  zero_threshold: value.zero_threshold,
167
- positive: dense_buckets(value.positive_buckets),
168
- negative: dense_buckets(value.negative_buckets)
188
+ positive: dense_buckets(positive),
189
+ negative: dense_buckets(negative)
169
190
  )
170
191
  end
171
192
 
193
+ # NativeHistogram clamps a ±Inf observation to MAX_BUCKET_INDEX, and
194
+ # downscaling halves that index along with every other one. Either way
195
+ # it stays above the index of any finite observation, which |value| <=
196
+ # Float::MAX puts at 1024 * 2**schema at the most, so it sorts last and
197
+ # is the only pair up there.
198
+ def finite_buckets(buckets, schema)
199
+ return buckets if buckets.empty? || buckets.last.first <= 1024 * (2.0**schema)
200
+
201
+ buckets[0..-2]
202
+ end
203
+
204
+ def bucket_total(buckets)
205
+ buckets.sum { |_, count| count }
206
+ end
207
+
208
+ # How many times both sides' resolution has to halve for each dense
209
+ # array to fit MAX_DENSE_BUCKETS. A data point carries one scale, so the
210
+ # two sides reduce together.
211
+ def scale_reduction(positive, negative)
212
+ reduction = 0
213
+ reduction += 1 while dense_length(positive, reduction) > MAX_DENSE_BUCKETS ||
214
+ dense_length(negative, reduction) > MAX_DENSE_BUCKETS
215
+ reduction
216
+ end
217
+
218
+ def dense_length(buckets, reduction)
219
+ return 0 if buckets.empty?
220
+
221
+ coarse_index(buckets.last.first, reduction) - coarse_index(buckets.first.first, reduction) + 1
222
+ end
223
+
224
+ # Merge each run of 2**reduction adjacent buckets into one, as
225
+ # NativeHistogram#downscale does for the schema it reports.
226
+ def merge_buckets(buckets, reduction)
227
+ return buckets if reduction.zero?
228
+
229
+ buckets.each_with_object({}) do |(index, count), merged|
230
+ key = coarse_index(index, reduction)
231
+ merged[key] = (merged[key] || 0) + count
232
+ end.to_a
233
+ end
234
+
235
+ # The bucket +index+ falls into after halving resolution +reduction+
236
+ # times: rounded toward the wider bucket, as downscaling rounds.
237
+ def coarse_index(index, reduction)
238
+ -(-index / (1 << reduction))
239
+ end
240
+
172
241
  # Convert sparse [[prom_idx, count], ...] to OTLP Buckets{offset, bucket_counts}.
173
242
  # OTLP index = prom index - 1. Offset = min OTLP index.
174
243
  # Buckets are always sorted (from NativeHistogram::Slot#positive_buckets / #negative_buckets).
@@ -190,14 +259,23 @@ module Fast
190
259
  end
191
260
 
192
261
  def build_attributes(labels)
193
- labels.map { |k, v| kv(k.to_s, v.to_s) }
262
+ labels.map { |name, value| kv(name, Proto::AnyValue.new(string_value: value.to_s)) }
194
263
  end
195
264
 
196
- def kv(key, value)
197
- Proto::KeyValue.new(
198
- key: key,
199
- value: Proto::AnyValue.new(string_value: value)
200
- )
265
+ def kv(key, any_value)
266
+ Proto::KeyValue.new(key: key.to_s, value: any_value)
267
+ end
268
+
269
+ # A resource attribute keeps its Ruby type where OTLP has a member for
270
+ # it; anything else goes over as its +to_s+.
271
+ def any_value(value)
272
+ case value
273
+ when String then Proto::AnyValue.new(string_value: value)
274
+ when Integer then Proto::AnyValue.new(int_value: value)
275
+ when Float then Proto::AnyValue.new(double_value: value)
276
+ when true, false then Proto::AnyValue.new(bool_value: value)
277
+ else Proto::AnyValue.new(string_value: value.to_s)
278
+ end
201
279
  end
202
280
  end
203
281
  end
@@ -7,15 +7,41 @@ module Fast
7
7
  # Owns the RED metrics (request count + duration) recorded for every HTTP
8
8
  # request. Shared by Middleware::Instrumentation and Rack::Instrumentation
9
9
  # so registration and recording exist exactly once.
10
+ #
11
+ # The method label is allowlisted, so the two metrics hold at most ten
12
+ # method values however many tokens clients invent — nothing reclaims a
13
+ # series once it exists.
10
14
  class RequestMetrics
15
+ LABELS = %i[method status].freeze
16
+
17
+ # The RFC 9110 methods plus PATCH, each mapped to the frozen String the
18
+ # store keeps, so an allowlisted method costs a lookup and no object.
19
+ # Matching is case-sensitive (RFC 9110 section 9.1) and finds a
20
+ # BINARY-tagged token from a Rack env: ASCII-only Strings hash and
21
+ # compare equal whatever their encoding tag.
22
+ METHODS = %w[GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATCH].to_h { |name| [name, name] }.freeze
23
+
24
+ # What every other token is counted as, the value OpenTelemetry's HTTP
25
+ # semantic conventions collapse an unrecognized method to.
26
+ OTHER_METHOD = "_OTHER"
27
+
28
+ # The kinds #record can drive under each name, and that each name
29
+ # promises whoever scrapes it. A Gauge answers #increment, but a gauge
30
+ # under "_total" is not a counter to a scraper; everything that answers
31
+ # #observe is a duration metric.
32
+ COUNTER_TYPES = %i[counter].freeze
33
+ DURATION_TYPES = %i[histogram native_histogram summary].freeze
34
+
35
+ private_constant :LABELS, :METHODS, :OTHER_METHOD, :COUNTER_TYPES, :DURATION_TYPES
36
+
11
37
  def initialize(registry:, native: false, prefix: "http_server")
12
38
  @registry = registry
13
- @counter = ensure_counter(prefix)
14
- @histogram = ensure_histogram(prefix, native)
39
+ @counter = validate(ensure_counter(prefix), COUNTER_TYPES)
40
+ @histogram = validate(ensure_histogram(prefix, native), DURATION_TYPES)
15
41
  end
16
42
 
17
43
  def record(method, status, started_at)
18
- labels = { method: method, status: status }
44
+ labels = { method: METHODS[method] || OTHER_METHOD, status: status }
19
45
  duration = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at
20
46
  @counter.increment(labels: labels)
21
47
  @histogram.observe(duration, labels: labels)
@@ -26,7 +52,7 @@ module Fast
26
52
  def ensure_counter(prefix)
27
53
  name = :"#{prefix}_requests_total"
28
54
  @registry.fetch_or_register(name) do
29
- Counter.new(name, docstring: "Total HTTP requests", labels: %i[method status])
55
+ Counter.new(name, docstring: "Total HTTP requests", labels: LABELS)
30
56
  end
31
57
  end
32
58
 
@@ -34,12 +60,28 @@ module Fast
34
60
  name = :"#{prefix}_request_duration_seconds"
35
61
  @registry.fetch_or_register(name) do
36
62
  if native
37
- NativeHistogram.new(name, docstring: "HTTP request duration in seconds", labels: %i[method status])
63
+ NativeHistogram.new(name, docstring: "HTTP request duration in seconds", labels: LABELS)
38
64
  else
39
- Histogram.new(name, docstring: "HTTP request duration in seconds", labels: %i[method status])
65
+ Histogram.new(name, docstring: "HTTP request duration in seconds", labels: LABELS)
40
66
  end
41
67
  end
42
68
  end
69
+
70
+ # A metric already registered under one of these names is reused only if
71
+ # it is one of +types+ and declares this label set. Otherwise every
72
+ # request would raise from inside the app's request path; raise here
73
+ # instead, once, where the mistaken declaration is.
74
+ def validate(metric, types)
75
+ unless types.include?(metric.type)
76
+ raise InvalidMetricType, "#{metric.name} declares type #{metric.type.inspect}, not #{types.inspect}"
77
+ end
78
+
79
+ unless metric.labels == LABELS
80
+ raise InvalidLabelSet, "#{metric.name} declares labels #{metric.labels.inspect}, not #{LABELS.inspect}"
81
+ end
82
+
83
+ metric
84
+ end
43
85
  end
44
86
  end
45
87
  end
@@ -40,17 +40,6 @@ module Fast
40
40
  series_map { |slot| hash_shape(slot) }
41
41
  end
42
42
 
43
- protected
44
-
45
- def validate_label_names(labels)
46
- labels.each do |label|
47
- name = label.to_s
48
- raise InvalidLabelName, "reserved label name: :quantile" if name == "quantile"
49
- end
50
-
51
- super
52
- end
53
-
54
43
  private
55
44
 
56
45
  def zero_value
@@ -64,6 +53,12 @@ module Fast
64
53
  def hash_shape(slot)
65
54
  { "count" => slot.count, "sum" => slot.sum }
66
55
  end
56
+
57
+ def validate_label_names(labels)
58
+ raise InvalidLabelName, "reserved label name: :quantile" if labels.include?(:quantile)
59
+
60
+ super
61
+ end
67
62
  end
68
63
  end
69
64
  end
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Fast
4
4
  module Prometheus
5
- VERSION = "0.3.2"
5
+ VERSION = "0.4.0"
6
6
  end
7
7
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fast-prometheus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.2
4
+ version: 0.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Eric Jacobs