standard_health 0.4.0 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 73c1743f46e03601e44970d282ca02dd195b22e5f22fe502e20e8a45281e0d69
4
- data.tar.gz: 507f8e5fda5420690b7146b0f6a9136561c0a1b61d6c567c5be3dcd5c250be61
3
+ metadata.gz: 68d0908117b9ae70545445465e84f94b4c6ccc5ebecb2aa6ab05b3aafa5ba9f7
4
+ data.tar.gz: 52c3ffe85837e265283011db180de726957df2eecd09b971efc369e879ae4ec3
5
5
  SHA512:
6
- metadata.gz: 38b527cf8973dc2557b1b8c13e40ace60f73efb78f1aab34100b4a51cf4f53069700c7ac0047f038a9bcf5fdcb9a1f3223fa92ddc653e1293f6c5d4b51f320ed
7
- data.tar.gz: 3662477381f1149d26394ba63fc2380c2104d1cb78a65bf3d943681a907b591ad0e7e9a6f990c868cf48087306cf6cb845c399fd976a346858ba083647b6bc3e
6
+ metadata.gz: 379b80dbdf211c4c1a3e7639ec6cc926219949f9c352fbadddfada54bdea53f20df6ab52d5672baa0974bb6b2952b5f5e67b9069191d980baff1de0d3c77f972
7
+ data.tar.gz: feaba7a044e8e63e4dc60ad4d010edfbe197691596ea2520632c4d590f6e52907d9275a5825dea8deff42c716517cb569b54f2824460820327f5003e9e5b5157
data/CHANGELOG.md CHANGED
@@ -7,6 +7,112 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.4.1] - 2026-07-29
11
+
12
+ Observability release. Until now the gem emitted **nothing** — no events, no
13
+ logs, no metrics. It computed `latency_ms` for every check and discarded it
14
+ into the response body. A failing check was invisible unless somebody happened
15
+ to call `/ready` at the right moment, so there was no history, no alerting
16
+ hook, and no way to chart check latency.
17
+
18
+ Everything here is **additive**: no status code, roll-up rule, or response
19
+ field changes meaning, so this is a pure `bundle update` for consumers on
20
+ `~> 0.4`.
21
+
22
+ ### Added
23
+
24
+ - **Instrumentation.** Three events over whichever bus is live (`Rails.event`
25
+ on Rails 8.1+, `ActiveSupport::Notifications` otherwise), matching
26
+ `standard_circuit`'s `EventEmitter` idiom:
27
+ - `standard_health.check.completed` — `name`, `critical`, `status`,
28
+ `latency_ms`, `error_class`, `error_message`
29
+ - `standard_health.check.timed_out` — `name`, `critical`, `timeout_s`
30
+ - `standard_health.ready.evaluated` — `status`, `duration_ms`, `failed[]`
31
+ - **Three built-in subscribers**, registered by a new engine initializer
32
+ (`after: :load_config_initializers`, so host config is final):
33
+ - `Notifiers::Logger` — **silent while healthy.** `warn` on degraded,
34
+ `error` on unavailable. Health events are polls (~6/min/instance), not
35
+ transitions; a line per evaluation is ~8,640/day/instance of "everything
36
+ is fine".
37
+ - `Notifiers::Sentry` — **transition-only**, with a 60s repeat floor and
38
+ one `info` on recovery. Capturing every non-ok poll would turn a
39
+ five-minute outage into ~30 duplicate issues. Mutex-guarded (Puma is
40
+ threaded). Sentry stays a soft dependency.
41
+ - `Notifiers::Metrics` — per-poll counters and latency distributions. This
42
+ fires on every evaluation on purpose: it is what makes `latency_ms`
43
+ chartable.
44
+ - `config.add_notifier` for host-supplied `call(event_name, payload)`
45
+ subscribers, validated at add time.
46
+ - Config: `instrumentation_enabled`, `logger`, `sentry_enabled`,
47
+ `metric_prefix`.
48
+ - **Per-check timeout machinery** — `default_check_timeout`,
49
+ `total_check_budget`, and a `timeout:` option on `register_check`. Uses a
50
+ dedicated `StandardHealth::CheckTimeout` rather than bare `Timeout::Error`,
51
+ so a timeout the host raised for its own reasons is never mislabelled as
52
+ ours. **All default to `nil` (OFF)** — see below.
53
+ - `status` on `/diagnostics/env`: `:ok` or `:incomplete` (a `required` var is
54
+ missing). **Additive — the endpoint still returns 200 either way.**
55
+
56
+ ### Fixed
57
+
58
+ - `Check#with_timing` now records `error_class` alongside the message. All
59
+ three built-in checks route through it, so without this every real driver
60
+ failure reached the aggregator with no class and the redacted body fell back
61
+ to a generic `StandardError` — exactly the useless label redaction exists to
62
+ replace.
63
+ - Failure detail (`error_class` + `error_message`) now rides on
64
+ `ready.evaluated`, not only on `check.completed`. Both built-in subscribers
65
+ are driven by `ready.evaluated` (deliberately — it is the transition-gated
66
+ event), so the message would otherwise have been deleted from the response
67
+ without ever reaching logs or Sentry.
68
+
69
+ ### Security
70
+
71
+ - **`/ready` no longer returns raw exception messages.** The endpoint is
72
+ unauthenticated by design, and a driver error will happily tell an anonymous
73
+ caller the database host, port and username — during exactly the incident
74
+ you least want to be leaking. Failing rows now carry `error_class` and a
75
+ stable `error_code` instead. The full message is not lost: it goes to logs
76
+ and Sentry via the instrumentation above, which is why redaction ships in
77
+ the same release rather than separately.
78
+ - `config.expose_check_errors = true` restores the previous bodies.
79
+ - `config.detail_token` + an `X-Health-Token` header is a break-glass path
80
+ for on-call, compared in constant time.
81
+ - `status`, `name`, `critical`, `latency_ms` and `generated_at` are
82
+ unchanged, so dashboards and existing specs keep working.
83
+
84
+ ### Notes on what is deliberately NOT on
85
+
86
+ Timeouts and the total budget default to **off**, giving byte-identical
87
+ behaviour to 0.4.0. Turning them on is a semantic change: a check that has
88
+ always been slow-but-fine starts reporting `:fail`, and for a critical check
89
+ that pulls the instance out of rotation. Shipping that in a patch, to five
90
+ apps, on a `bundle update`, is how you cause the outage you were preventing.
91
+
92
+ The machinery ships now so apps can opt in per check, and so the events above
93
+ can reveal the real p99 latencies. Defaults get chosen from that data in
94
+ 0.5.0, which requires an explicit Gemfile edit.
95
+
96
+ One semantic guard is already in place for when they are enabled: a check
97
+ skipped because the budget ran out reports `:skipped` and floors the roll-up
98
+ at `:degraded` — **never `:unavailable`**, even when the skipped check is
99
+ critical. Otherwise a slow *non-critical* check could exhaust the budget,
100
+ leave the database check unrun, and pull a healthy instance out of rotation.
101
+
102
+ `total_check_budget` bounds **how many checks run**, not how long the probe
103
+ takes — it is evaluated before each check starts, not during one. Clamping
104
+ each check to the remaining budget would close that gap but would apply
105
+ `Timeout.timeout` to checks whose author never asked for one, so setting a
106
+ budget deliberately does not opt you into that. Asserted by spec, not just
107
+ documented.
108
+
109
+ A second reason to leave them off: `Timeout.timeout` raises into the running
110
+ thread at an arbitrary point, so firing one mid-connection-checkout can return
111
+ a broken connection to the pool. The dedicated `CheckTimeout` fixes
112
+ mislabelling, not this. The README now carries the full caveat — the right
113
+ answer for datastore checks is usually a driver-level timeout
114
+ (`connect_timeout` / `statement_timeout`), not this setting.
115
+
10
116
  ## [0.4.0] - 2026-05-05
11
117
 
12
118
  ### Added
@@ -1,4 +1,6 @@
1
- Copyright (c) 2026 Jaryl Sim
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2026 Rarebit One
2
4
 
3
5
  Permission is hereby granted, free of charge, to any person obtaining
4
6
  a copy of this software and associated documentation files (the
@@ -13,8 +15,8 @@ included in all copies or substantial portions of the Software.
13
15
 
14
16
  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
17
  EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
17
- IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
18
- CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
19
- TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
20
- SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md CHANGED
@@ -184,6 +184,23 @@ Audit rows for entries declared inside a `group` block carry a `group` key:
184
184
 
185
185
  Entries declared outside any `group` block omit the `group` key entirely.
186
186
 
187
+ ### Top-level status
188
+
189
+ Since 0.4.1 the response carries a `status` alongside the audit, so a caller
190
+ can gate on one field instead of re-implementing the roll-up:
191
+
192
+ ```json
193
+ { "mode": "production", "status": "incomplete", "audit": [ ... ] }
194
+ ```
195
+
196
+ `incomplete` means at least one `required` var is missing; `recommended` is
197
+ advisory and never affects it.
198
+
199
+ **The endpoint still returns 200 either way in 0.4.1.** That is deliberate: it
200
+ gives monitors a release to migrate onto the field before 0.5.0 makes
201
+ `incomplete` a 503. Flipping the status code without that window would
202
+ silently redden every existing caller.
203
+
187
204
  ### Backward compatibility
188
205
 
189
206
  All v0.2.0 specs continue to produce identical audit output in v0.3.0. The new fields (`description`, `consumed_by`, `group`, `reason`) appear only when the corresponding feature is used; the new `:not_applicable` status only appears when a predicate suppresses an entry.
@@ -283,6 +300,130 @@ When `diagnostics_parent_controller` is unset, `DiagnosticsController` falls bac
283
300
 
284
301
  The orchestrator should pull the instance out of rotation only on 503; degraded means "still serving, page someone."
285
302
 
303
+ `skipped` also exists, for a check the total budget never reached (see
304
+ Timeouts). A skip floors the roll-up at `degraded` and **never** produces
305
+ `unavailable`, even for a critical check.
306
+
307
+ ## Failure detail is redacted
308
+
309
+ Since 0.4.1, a failing check reports the exception **class** rather than its
310
+ message:
311
+
312
+ ```json
313
+ { "name": "database", "critical": true, "status": "fail",
314
+ "error_class": "PG::ConnectionBad", "error_code": "pg_connection_bad" }
315
+ ```
316
+
317
+ `/ready` is unauthenticated by design — probes carry no credentials — and a
318
+ raw driver message will happily tell an anonymous caller the database host,
319
+ port and username. The full message still reaches your logs and Sentry through
320
+ the instrumentation below; it just isn't in the public body.
321
+
322
+ ```ruby
323
+ c.expose_check_errors = true # restore pre-0.4.1 verbose bodies
324
+ c.detail_token = ENV["HEALTH_DETAIL_TOKEN"] # X-Health-Token unlocks detail
325
+ ```
326
+
327
+ `status`, `name`, `critical`, `latency_ms` and `generated_at` are unchanged.
328
+
329
+ ## Instrumentation
330
+
331
+ The gem emits events on whichever bus is live — `Rails.event` on Rails 8.1+,
332
+ `ActiveSupport::Notifications` otherwise:
333
+
334
+ | Event | Payload |
335
+ |---|---|
336
+ | `standard_health.check.completed` | `name`, `critical`, `status`, `latency_ms`, `error_class`, `error_message` |
337
+ | `standard_health.check.timed_out` | `name`, `critical`, `timeout_s` |
338
+ | `standard_health.ready.evaluated` | `status`, `duration_ms`, `failed[]` |
339
+
340
+ Three subscribers are registered automatically. Their noise profiles differ on
341
+ purpose, because health events are **polls**, not state transitions — a 10s
342
+ probe period means ~6 evaluations/minute/instance:
343
+
344
+ - **Logger** — silent while healthy. `warn` on degraded, `error` on
345
+ unavailable. A line per evaluation would be ~8,640/day/instance of
346
+ "everything is fine".
347
+ - **Sentry** — only on status **change**, plus a 60s repeat floor for a
348
+ sustained bad state, plus one `info` on recovery. Capturing every non-ok
349
+ poll would turn a five-minute outage into ~30 duplicate issues. Sentry is a
350
+ soft dependency; no gemspec entry.
351
+ - **Metrics** — every poll, deliberately. Counters plus latency
352
+ distributions are what make `latency_ms` chartable.
353
+
354
+ ```ruby
355
+ StandardHealth.configure do |c|
356
+ c.instrumentation_enabled = true # default
357
+ c.logger = Rails.logger # default: Rails.logger
358
+ c.sentry_enabled = true # default
359
+ c.metric_prefix = "health" # default
360
+
361
+ c.add_notifier(MyNotifier.new) # must respond to call(event_name, payload)
362
+ end
363
+ ```
364
+
365
+ ## Timeouts
366
+
367
+ **Off by default.** Both settings are `nil`, which is byte-identical to 0.4.0.
368
+
369
+ ```ruby
370
+ c.default_check_timeout = 2.0 # seconds, per check
371
+ c.total_check_budget = 5.0 # checked BEFORE each check starts
372
+ c.register_check :db, MyCheck, critical: true, timeout: 1.0 # per-check override
373
+ ```
374
+
375
+ **`total_check_budget` bounds how many checks RUN, not how long the probe
376
+ takes.** It is evaluated before each check starts, not during one. With a 1s
377
+ budget and a first check that blocks for 30s, `/ready` still takes 30s — only
378
+ the checks *after* it are skipped.
379
+
380
+ Clamping each check to the remaining budget would close that gap, and is
381
+ deliberately not done: it would apply `Timeout.timeout` to checks whose author
382
+ never asked for one, and that mechanism raises into the thread at an arbitrary
383
+ point (see below). Setting a budget must not silently opt you into that. To
384
+ bound wall-clock, put a `timeout:` on the checks that can safely take one.
385
+
386
+ Enabling them is a semantic change — a check that has always been
387
+ slow-but-fine starts reporting `:fail`, and for a critical check that pulls
388
+ the instance out of rotation. Pick values from observed `latency_ms` rather
389
+ than intuition; that is what the `check.completed` events are for.
390
+
391
+ Checks the total budget never reaches report `:skipped`, never silently `:ok`.
392
+ A skip alone floors the roll-up at `degraded` — otherwise a slow *non-critical*
393
+ check could exhaust the budget, leave the database check unrun, and pull a
394
+ healthy instance out of rotation.
395
+
396
+ Checks run **sequentially**. They are not parallelised on purpose: running
397
+ them in threads would mean connection checkouts from the health path, and a
398
+ health endpoint that can exhaust the connection pool is a worse problem than
399
+ the one it was added to solve.
400
+
401
+ ### ⚠ Know what `Timeout.timeout` actually does before enabling this
402
+
403
+ Ruby's `Timeout.timeout` interrupts by raising **into the running thread at an
404
+ arbitrary point**. It cannot wait for a safe boundary. If a timeout fires
405
+ while a check is mid-way through a non-atomic operation — say a connection
406
+ checkout in the `ActiveRecord` check — the connection can be returned to the
407
+ pool in a broken state. You would have traded a slow check for a corrupted
408
+ pool, on the health path, during an incident.
409
+
410
+ The dedicated `CheckTimeout` class solves the *mislabelling* problem (a host's
411
+ own `Timeout::Error` is never mistaken for ours). **It does not solve this
412
+ one.** Nothing can, at this layer.
413
+
414
+ So, before turning timeouts on:
415
+
416
+ - Prefer a driver-level timeout where one exists — `connect_timeout` /
417
+ `statement_timeout` on the database, a client timeout on an HTTP dependency.
418
+ Those abort at a safe point because the driver owns the operation.
419
+ - Reserve `timeout:` here for checks whose `run` is genuinely interruptible —
420
+ typically your own custom checks — rather than the built-in datastore ones.
421
+ - Set the value comfortably above observed p99 (see the `latency_ms` in
422
+ `check.completed`), so it fires on a hang rather than on a slow day.
423
+
424
+ This is why the defaults ship as `nil` and why choosing them is deferred: the
425
+ right answer is usually a driver timeout, not this.
426
+
286
427
  ## License
287
428
 
288
429
  MIT.
@@ -25,9 +25,30 @@ module StandardHealth
25
25
 
26
26
  render json: {
27
27
  mode: mode,
28
+ status: audit_status(audit),
28
29
  audit: audit,
29
30
  generated_at: Time.now.utc.iso8601
30
31
  }
31
32
  end
33
+
34
+ private
35
+
36
+ # Top-level verdict so a caller can gate on ONE field instead of
37
+ # re-implementing the roll-up over `audit`.
38
+ #
39
+ # :ok — nothing required is missing
40
+ # :incomplete — at least one `required` var is missing
41
+ #
42
+ # ADDITIVE ONLY in 0.4.1: the endpoint still returns 200 either way, so
43
+ # nothing that asserts on the status code breaks. That is the point —
44
+ # it gives monitors a release in which to migrate onto this field before
45
+ # 0.5.0 makes `:incomplete` a 503. Without the migration window, turning
46
+ # the code non-200 would silently redden every existing caller.
47
+ def audit_status(audit)
48
+ incomplete = Array(audit).any? do |row|
49
+ row[:level].to_s == "required" && row[:status].to_s == "missing"
50
+ end
51
+ incomplete ? :incomplete : :ok
52
+ end
32
53
  end
33
54
  end
@@ -12,10 +12,38 @@ module StandardHealth
12
12
  # Readiness probe. Runs every registered check and returns:
13
13
  # 200 if the rolled-up status is :ok or :degraded
14
14
  # 503 if any critical check failed (:unavailable)
15
+ #
16
+ # Failure detail is REDACTED by default — see StandardHealth::Redactor.
17
+ # This endpoint is unauthenticated so probes need no credentials, and a
18
+ # raw driver error will happily tell an anonymous caller the database
19
+ # host, port and username. The full message goes to logs and Sentry
20
+ # instead. Status codes and every other field are unchanged.
15
21
  def ready
16
22
  result = StandardHealth::Aggregator.call
17
23
  http_status = result[:status] == :unavailable ? :service_unavailable : :ok
18
- render json: result, status: http_status
24
+ render json: StandardHealth::Redactor.call(result, expose: expose_errors?),
25
+ status: http_status
26
+ end
27
+
28
+ private
29
+
30
+ # Detail is exposed when the host opted in globally, or when the caller
31
+ # presents the break-glass token. Compared in constant time: the token is
32
+ # a secret, and a naive == leaks its length and prefix to a patient
33
+ # attacker on an endpoint that is public by design.
34
+ def expose_errors?
35
+ config = StandardHealth.config
36
+ return true if config.expose_check_errors
37
+
38
+ token = config.detail_token
39
+ return false if token.nil? || token.empty?
40
+
41
+ presented = request.headers["X-Health-Token"].to_s
42
+ return false if presented.empty?
43
+
44
+ ActiveSupport::SecurityUtils.secure_compare(presented, token)
45
+ rescue StandardError
46
+ false
19
47
  end
20
48
  end
21
49
  end
@@ -1,48 +1,187 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "timeout"
4
+
3
5
  module StandardHealth
6
+ # Raised internally when a check exceeds its timeout.
7
+ #
8
+ # A DEDICATED subclass, never bare `Timeout::Error`: `Timeout.timeout` with
9
+ # the default class would also swallow a timeout the host app raised for its
10
+ # own reasons and mislabel it as ours. Passing our own class means we only
11
+ # ever catch the one we threw.
12
+ class CheckTimeout < ::Timeout::Error; end
13
+
4
14
  # Runs all registered checks and rolls them up into a single status.
5
15
  #
6
16
  # Status semantics:
7
17
  # :ok — every check returned :ok
8
- # :degraded — at least one non-critical check failed
18
+ # :degraded — at least one non-critical check failed, OR a check was
19
+ # skipped because the total budget ran out
9
20
  # :unavailable — at least one critical check failed
10
21
  #
11
- # The aggregator never raises. Each check is invoked through
12
- # `safe_run` which catches `StandardError` so a buggy custom check
13
- # cannot take down /ready.
22
+ # The aggregator never raises. Each check is invoked through `safe_run`
23
+ # which catches `StandardError` so a buggy custom check cannot take down
24
+ # /ready. Instrumentation is held to the same bar — every emit is wrapped
25
+ # so a broken subscriber cannot become a new way for /ready to 500.
14
26
  class Aggregator
15
27
  def self.call(checks: StandardHealth.config.checks, now: Time.now.utc)
16
- check_rows = checks.map { |reg| safe_run(reg) }
28
+ started = monotonic
29
+ budget = StandardHealth.config.total_check_budget
30
+
31
+ check_rows = checks.map do |reg|
32
+ if budget_exhausted?(budget, started)
33
+ row = skipped_row(reg, budget)
34
+ # Skips emit too. Without this a skip is invisible to the Metrics
35
+ # notifier — the only trace would be a name inside ready.evaluated's
36
+ # `failed[]`, with no per-check counter — so once a budget is
37
+ # enabled you could not answer "how often is check X getting
38
+ # skipped", which is exactly the question a budget creates.
39
+ emit_check(row)
40
+ row
41
+ else
42
+ safe_run(reg)
43
+ end
44
+ end
45
+
46
+ duration_ms = ((monotonic - started) * 1000).round
47
+ status = overall_status(check_rows)
48
+
49
+ failing = check_rows.reject { |r| r[:status] == :ok }
50
+
51
+ emit(
52
+ "standard_health.ready.evaluated",
53
+ status: status,
54
+ duration_ms: duration_ms,
55
+ failed: failing.map { |r| r[:name] },
56
+ # The full failure detail rides on THIS event, not only on the
57
+ # per-check one. Redaction removes the message from the HTTP body on
58
+ # the promise that it still reaches logs and Sentry — but both of
59
+ # those subscribers are driven by ready.evaluated (deliberately: it
60
+ # is the transition-gated event, so they don't fire per poll). If the
61
+ # message only existed on check.completed, redaction would delete the
62
+ # last copy instead of relocating it.
63
+ failures: failing.map do |r|
64
+ {
65
+ name: r[:name],
66
+ critical: r[:critical],
67
+ status: r[:status],
68
+ error_class: r[:error_class],
69
+ error_message: r[:error]
70
+ }.compact
71
+ end
72
+ )
73
+
17
74
  {
18
- status: overall_status(check_rows),
75
+ status: status,
19
76
  checks: check_rows,
20
77
  generated_at: now.iso8601
21
78
  }
22
79
  end
23
80
 
24
81
  def self.safe_run(reg)
25
- result = reg.klass.new(name: reg.name, critical: reg.critical).run
26
- result.merge(name: reg.name, critical: reg.critical)
82
+ timeout = reg.timeout || StandardHealth.config.default_check_timeout
83
+ instance = reg.klass.new(name: reg.name, critical: reg.critical)
84
+
85
+ result =
86
+ if timeout
87
+ Timeout.timeout(timeout, CheckTimeout) { instance.run }
88
+ else
89
+ instance.run
90
+ end
91
+
92
+ row = result.merge(name: reg.name, critical: reg.critical)
93
+ emit_check(row)
94
+ row
95
+ rescue CheckTimeout
96
+ emit("standard_health.check.timed_out",
97
+ name: reg.name, critical: reg.critical, timeout_s: timeout)
98
+ row = {
99
+ name: reg.name,
100
+ critical: reg.critical,
101
+ status: :fail,
102
+ error: "timed out after #{timeout}s",
103
+ error_class: "StandardHealth::CheckTimeout"
104
+ }
105
+ emit_check(row)
106
+ row
27
107
  rescue StandardError => e
28
- {
108
+ row = {
29
109
  name: reg.name,
30
110
  critical: reg.critical,
31
111
  status: :fail,
32
- error: e.message
112
+ error: e.message,
113
+ error_class: e.class.name
33
114
  }
115
+ emit_check(row)
116
+ row
34
117
  end
35
118
  private_class_method :safe_run
36
119
 
120
+ def self.budget_exhausted?(budget, started)
121
+ return false unless budget
122
+
123
+ (monotonic - started) >= budget
124
+ end
125
+ private_class_method :budget_exhausted?
126
+
127
+ # A check the total budget never reached. `:skipped`, never silently
128
+ # `:ok` — an unperformed check is not a healthy one.
129
+ def self.skipped_row(reg, budget)
130
+ {
131
+ name: reg.name,
132
+ critical: reg.critical,
133
+ status: :skipped,
134
+ error: "skipped — total check budget of #{budget}s exhausted"
135
+ }
136
+ end
137
+ private_class_method :skipped_row
138
+
37
139
  def self.overall_status(rows)
38
140
  return :ok if rows.empty?
39
141
 
40
142
  failures = rows.reject { |r| r[:status] == :ok }
41
143
  return :ok if failures.empty?
42
- return :unavailable if failures.any? { |r| r[:critical] }
144
+
145
+ # A SKIP MUST NEVER PRODUCE :unavailable, even for a critical check.
146
+ # Otherwise a slow *non-critical* check could exhaust the budget, leave
147
+ # the database check unrun, and pull a perfectly healthy instance out of
148
+ # rotation — a self-inflicted outage caused by the safety mechanism.
149
+ # Skips floor the result at :degraded; only a real critical FAILURE
150
+ # reaches :unavailable.
151
+ return :unavailable if failures.any? { |r| r[:critical] && r[:status] != :skipped }
43
152
 
44
153
  :degraded
45
154
  end
46
155
  private_class_method :overall_status
156
+
157
+ def self.emit_check(row)
158
+ emit(
159
+ "standard_health.check.completed",
160
+ name: row[:name],
161
+ critical: row[:critical],
162
+ status: row[:status],
163
+ latency_ms: row[:latency_ms],
164
+ error_class: row[:error_class],
165
+ error_message: row[:error]
166
+ )
167
+ end
168
+ private_class_method :emit_check
169
+
170
+ # Belt and braces on top of EventEmitter's own rescue. The never-raise
171
+ # rule names this file, and instrumentation is the newest way to violate
172
+ # it — so the call site guards too.
173
+ def self.emit(event_name, **payload)
174
+ return unless StandardHealth.config.instrumentation_enabled
175
+
176
+ EventEmitter.emit(event_name, payload)
177
+ rescue StandardError
178
+ nil
179
+ end
180
+ private_class_method :emit
181
+
182
+ def self.monotonic
183
+ Process.clock_gettime(Process::CLOCK_MONOTONIC)
184
+ end
185
+ private_class_method :monotonic
47
186
  end
48
187
  end
@@ -43,7 +43,12 @@ module StandardHealth
43
43
  latency_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000).round
44
44
  { status: :ok, latency_ms: latency_ms }
45
45
  rescue StandardError => e
46
- { status: :fail, error: e.message }
46
+ # `error_class` matters as much as the message. All three built-in
47
+ # checks route through here, so without it every real driver failure
48
+ # reaches the aggregator with no class — and the redacted response then
49
+ # falls back to a generic "StandardError", which is precisely the
50
+ # useless label redaction exists to replace with something groupable.
51
+ { status: :fail, error: e.message, error_class: e.class.name }
47
52
  end
48
53
  end
49
54
  end