standard_health 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 73c1743f46e03601e44970d282ca02dd195b22e5f22fe502e20e8a45281e0d69
4
- data.tar.gz: 507f8e5fda5420690b7146b0f6a9136561c0a1b61d6c567c5be3dcd5c250be61
3
+ metadata.gz: 3d6a6ea3266864aaa15232529241e2040a836d810c4898eb512746e78a582901
4
+ data.tar.gz: fc9575f2c3e425a6065e3e5285094d03ba5cd694848c386937251b0fea5d75af
5
5
  SHA512:
6
- metadata.gz: 38b527cf8973dc2557b1b8c13e40ace60f73efb78f1aab34100b4a51cf4f53069700c7ac0047f038a9bcf5fdcb9a1f3223fa92ddc653e1293f6c5d4b51f320ed
7
- data.tar.gz: 3662477381f1149d26394ba63fc2380c2104d1cb78a65bf3d943681a907b591ad0e7e9a6f990c868cf48087306cf6cb845c399fd976a346858ba083647b6bc3e
6
+ metadata.gz: 83a04f1025e9e8b6e62344ff53828a716c262d7779d30f64afeb69b704fb34289e9138f751a93fb46215ba24c71612e17e9cfd7a37c859ac67feca64bbb2e25a
7
+ data.tar.gz: 6cf0b8d3a4100e8d3737eb1bbaddfb7c1a40d6dd77ad4e9c097ad547308ff196fb012b46ebb7c1d267b30d784abc863f734df4bd8520971d99af9d705e2af51b
data/CHANGELOG.md CHANGED
@@ -7,6 +7,186 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.5.0] - 2026-07-30
11
+
12
+ Closes the gap that had four host apps writing their own checks. The env-spec
13
+ DSL could only audit **presence**, so "this toggle must not be set on
14
+ production" and "this value must be exactly `false`" were inexpressible — and
15
+ the workaround was a hand-written check carrying a duplicate list of variable
16
+ names, kept in sync with the spec by comment.
17
+
18
+ Everything here is **additive and opt-in**. No check is auto-registered and no
19
+ existing status changes meaning, so this is a pure `bundle update` for
20
+ consumers on `~> 0.4`.
21
+
22
+ ### Added
23
+
24
+ - **`forbidden` EnvSpec level.** Inverts the presence rule: absent is `:ok`,
25
+ present reports the new `:forbidden` status. For dangerous ops toggles —
26
+ demo modes, auth bypasses, bootstrap flags — that are legitimate on staging
27
+ and must never survive promotion. Composes with `in:`, `mode_alias`, groups
28
+ and the `if:`/`unless:` predicates like any other level.
29
+ - **`expected_value:` assertion** on `required` / `recommended`. Asserts the
30
+ value, not just presence; a present-but-wrong value reports the new
31
+ `:mismatch` status. Accepts a String, an Array (any-of), a Regexp (matched,
32
+ not compared), or any object comparable by its string form. The **actual
33
+ value is never surfaced** — env values are routinely secrets — only the
34
+ declared expectation.
35
+ - **`Checks::EnvSpecAudit`** (opt-in). Runs the configured `env_spec` and
36
+ fails on `:missing` / `:forbidden` / `:mismatch`, reporting offending
37
+ variable *names* grouped by status. Reads the spec directly, so there is no
38
+ second list to drift. Non-critical by default: config drift is visibility,
39
+ not a rotation signal. Skips `consumed_by` resolution — that is file IO per
40
+ entry, fine for an on-demand doctor endpoint and not on a polled tier.
41
+ - **`Checks::SolidCable`** (opt-in). Bounded read against
42
+ `solid_cable_messages`. Promoted from sidekick-web. Falls back to the
43
+ primary connection when SolidCable isn't pointed at its own database.
44
+ - **`standard_health:install` generator.** Writes the initializer and mounts
45
+ the engine, with the aggregate-route ordering requirement noted inline.
46
+ Idempotent; `--skip-initializer`, `--skip-routes`, `--force`.
47
+
48
+ ### Changed
49
+
50
+ - **`/diagnostics/env` top-level `status`** now reports `incomplete` for
51
+ `:forbidden` and `:mismatch` rows as well as `:missing`. Callers already
52
+ gating on that one field pick up the new assertions for free, which is why
53
+ they joined the existing roll-up instead of getting a verdict of their own.
54
+ The endpoint still returns 200 either way. `:should_set` remains advisory
55
+ and still never affects the roll-up.
56
+ - **Rails dependency relaxed** from `~> 8.0` to `>= 8.0`, matching the rest of
57
+ the `standard_*` family. `~> 8.0` was the only floor in the family that
58
+ would have blocked a Rails 9 host.
59
+
60
+ ### Not in this release
61
+
62
+ - **Timeout defaults remain unset.** 0.4.1 said sensible `default_check_timeout`
63
+ / budget values would be chosen "in 0.5.0". They are not: choosing them still
64
+ needs enough real p99 latency data from the instrumentation 0.4.1 added, and
65
+ guessing a timeout on a health check is how you cause the outage you were
66
+ trying to prevent. The machinery stays opt-in per check. The stale promise in
67
+ `Configuration` has been repointed (#48).
68
+ - **`/diagnostics/env` still returns 200** when `status` is `incomplete`.
69
+ Turning that into a 503 is a breaking contract change for any caller gating on
70
+ the response code, so it wants its own decision rather than riding along with
71
+ an otherwise additive release.
72
+
73
+ ### Documentation
74
+
75
+ - **The aggregate `GET /health` tier and its ordering requirement are now
76
+ documented.** The engine draws sub-paths only; the aggregate tier is the
77
+ host's job and must be drawn **before** `mount`. An app that mounts first
78
+ and relies on the engine to serve it silently has no aggregate tier — no
79
+ boot error, no failing route spec. Also documents what each of the four
80
+ tiers is for, and that readiness gates only on hard infra the app owns.
81
+ - New README section on opt-in checks, including why nothing here
82
+ auto-registers.
83
+
84
+ ## [0.4.1] - 2026-07-29
85
+
86
+ Observability release. Until now the gem emitted **nothing** — no events, no
87
+ logs, no metrics. It computed `latency_ms` for every check and discarded it
88
+ into the response body. A failing check was invisible unless somebody happened
89
+ to call `/ready` at the right moment, so there was no history, no alerting
90
+ hook, and no way to chart check latency.
91
+
92
+ Everything here is **additive**: no status code, roll-up rule, or response
93
+ field changes meaning, so this is a pure `bundle update` for consumers on
94
+ `~> 0.4`.
95
+
96
+ ### Added
97
+
98
+ - **Instrumentation.** Three events over whichever bus is live (`Rails.event`
99
+ on Rails 8.1+, `ActiveSupport::Notifications` otherwise), matching
100
+ `standard_circuit`'s `EventEmitter` idiom:
101
+ - `standard_health.check.completed` — `name`, `critical`, `status`,
102
+ `latency_ms`, `error_class`, `error_message`
103
+ - `standard_health.check.timed_out` — `name`, `critical`, `timeout_s`
104
+ - `standard_health.ready.evaluated` — `status`, `duration_ms`, `failed[]`
105
+ - **Three built-in subscribers**, registered by a new engine initializer
106
+ (`after: :load_config_initializers`, so host config is final):
107
+ - `Notifiers::Logger` — **silent while healthy.** `warn` on degraded,
108
+ `error` on unavailable. Health events are polls (~6/min/instance), not
109
+ transitions; a line per evaluation is ~8,640/day/instance of "everything
110
+ is fine".
111
+ - `Notifiers::Sentry` — **transition-only**, with a 60s repeat floor and
112
+ one `info` on recovery. Capturing every non-ok poll would turn a
113
+ five-minute outage into ~30 duplicate issues. Mutex-guarded (Puma is
114
+ threaded). Sentry stays a soft dependency.
115
+ - `Notifiers::Metrics` — per-poll counters and latency distributions. This
116
+ fires on every evaluation on purpose: it is what makes `latency_ms`
117
+ chartable.
118
+ - `config.add_notifier` for host-supplied `call(event_name, payload)`
119
+ subscribers, validated at add time.
120
+ - Config: `instrumentation_enabled`, `logger`, `sentry_enabled`,
121
+ `metric_prefix`.
122
+ - **Per-check timeout machinery** — `default_check_timeout`,
123
+ `total_check_budget`, and a `timeout:` option on `register_check`. Uses a
124
+ dedicated `StandardHealth::CheckTimeout` rather than bare `Timeout::Error`,
125
+ so a timeout the host raised for its own reasons is never mislabelled as
126
+ ours. **All default to `nil` (OFF)** — see below.
127
+ - `status` on `/diagnostics/env`: `:ok` or `:incomplete` (a `required` var is
128
+ missing). **Additive — the endpoint still returns 200 either way.**
129
+
130
+ ### Fixed
131
+
132
+ - `Check#with_timing` now records `error_class` alongside the message. All
133
+ three built-in checks route through it, so without this every real driver
134
+ failure reached the aggregator with no class and the redacted body fell back
135
+ to a generic `StandardError` — exactly the useless label redaction exists to
136
+ replace.
137
+ - Failure detail (`error_class` + `error_message`) now rides on
138
+ `ready.evaluated`, not only on `check.completed`. Both built-in subscribers
139
+ are driven by `ready.evaluated` (deliberately — it is the transition-gated
140
+ event), so the message would otherwise have been deleted from the response
141
+ without ever reaching logs or Sentry.
142
+
143
+ ### Security
144
+
145
+ - **`/ready` no longer returns raw exception messages.** The endpoint is
146
+ unauthenticated by design, and a driver error will happily tell an anonymous
147
+ caller the database host, port and username — during exactly the incident
148
+ you least want to be leaking. Failing rows now carry `error_class` and a
149
+ stable `error_code` instead. The full message is not lost: it goes to logs
150
+ and Sentry via the instrumentation above, which is why redaction ships in
151
+ the same release rather than separately.
152
+ - `config.expose_check_errors = true` restores the previous bodies.
153
+ - `config.detail_token` + an `X-Health-Token` header is a break-glass path
154
+ for on-call, compared in constant time.
155
+ - `status`, `name`, `critical`, `latency_ms` and `generated_at` are
156
+ unchanged, so dashboards and existing specs keep working.
157
+
158
+ ### Notes on what is deliberately NOT on
159
+
160
+ Timeouts and the total budget default to **off**, giving byte-identical
161
+ behaviour to 0.4.0. Turning them on is a semantic change: a check that has
162
+ always been slow-but-fine starts reporting `:fail`, and for a critical check
163
+ that pulls the instance out of rotation. Shipping that in a patch, to five
164
+ apps, on a `bundle update`, is how you cause the outage you were preventing.
165
+
166
+ The machinery ships now so apps can opt in per check, and so the events above
167
+ can reveal the real p99 latencies. Defaults get chosen from that data in
168
+ 0.5.0, which requires an explicit Gemfile edit.
169
+
170
+ One semantic guard is already in place for when they are enabled: a check
171
+ skipped because the budget ran out reports `:skipped` and floors the roll-up
172
+ at `:degraded` — **never `:unavailable`**, even when the skipped check is
173
+ critical. Otherwise a slow *non-critical* check could exhaust the budget,
174
+ leave the database check unrun, and pull a healthy instance out of rotation.
175
+
176
+ `total_check_budget` bounds **how many checks run**, not how long the probe
177
+ takes — it is evaluated before each check starts, not during one. Clamping
178
+ each check to the remaining budget would close that gap but would apply
179
+ `Timeout.timeout` to checks whose author never asked for one, so setting a
180
+ budget deliberately does not opt you into that. Asserted by spec, not just
181
+ documented.
182
+
183
+ A second reason to leave them off: `Timeout.timeout` raises into the running
184
+ thread at an arbitrary point, so firing one mid-connection-checkout can return
185
+ a broken connection to the pool. The dedicated `CheckTimeout` fixes
186
+ mislabelling, not this. The README now carries the full caveat — the right
187
+ answer for datastore checks is usually a driver-level timeout
188
+ (`connect_timeout` / `statement_timeout`), not this setting.
189
+
10
190
  ## [0.4.0] - 2026-05-05
11
191
 
12
192
  ### Added
@@ -1,4 +1,6 @@
1
- Copyright (c) 2026 Jaryl Sim
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2026 Rarebit One
2
4
 
3
5
  Permission is hereby granted, free of charge, to any person obtaining
4
6
  a copy of this software and associated documentation files (the
@@ -13,8 +15,8 @@ included in all copies or substantial portions of the Software.
13
15
 
14
16
  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
17
  EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
17
- IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
18
- CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
19
- TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
20
- SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md CHANGED
@@ -18,7 +18,16 @@ Add to your Gemfile:
18
18
  gem "standard_health"
19
19
  ```
20
20
 
21
- Then `bundle install`.
21
+ Then `bundle install`, and run the install generator:
22
+
23
+ ```bash
24
+ bin/rails generate standard_health:install
25
+ ```
26
+
27
+ It writes `config/initializers/standard_health.rb` and mounts the engine in
28
+ `config/routes.rb` with the ordering requirement below noted inline. It is
29
+ idempotent — re-running skips what is already installed. Flags:
30
+ `--skip-initializer`, `--skip-routes`, `--force` (overwrite the initializer).
22
31
 
23
32
  ## Mounting
24
33
 
@@ -34,9 +43,45 @@ This wires up:
34
43
  - `GET /health/ready`
35
44
  - `GET /health/diagnostics/env`
36
45
 
46
+ ### The aggregate `GET /health` is yours to draw — before the mount
47
+
48
+ **The engine draws sub-paths only.** There is no `GET /health` in
49
+ `config/routes.rb` here; the aggregate tier is the host's responsibility, and
50
+ the ordering is not optional:
51
+
52
+ ```ruby
53
+ get "/health", to: "health_aggregate#show" # aggregate — FIRST
54
+ mount StandardHealth::Engine => "/health", as: :standard_health
55
+ ```
56
+
57
+ An app that mounts the engine **first** and relies on it to serve the aggregate
58
+ tier **silently has no aggregate tier at all** — no boot error, no 404 at boot,
59
+ and no failing route spec unless one exists. The failure surfaces as a
60
+ dashboard that has been reading nothing for months. Add a route spec asserting
61
+ `GET /health` resolves.
62
+
63
+ Why this way round: `mount` claims the `/health` prefix, so a bare `GET /health`
64
+ drawn *after* it still resolves (the engine has no route for the bare path) —
65
+ but nothing about that is obvious from reading the file, and getting it
66
+ backwards fails silently either way. Draw it first and the ordering question
67
+ never arises.
68
+
69
+ The four tiers, and what each is for:
70
+
71
+ | Route | Tier | Consumer |
72
+ |---|---|---|
73
+ | `GET /health/alive` | liveness — process is up | restart probes, CI reachability gates |
74
+ | `GET /health/ready` | readiness — **rotation gate** | load balancer / platform health check |
75
+ | `GET /health` | aggregate — full picture | dashboards, humans |
76
+ | `GET /health/diagnostics/env` | doctor (authed) | on-call, config audits |
77
+
78
+ Readiness gates **only** on hard infra the app owns. A soft upstream that
79
+ degrades must not pull an instance out of rotation — put those on the aggregate
80
+ tier instead.
81
+
37
82
  ## Configuration
38
83
 
39
- Create `config/initializers/standard_health.rb`:
84
+ Create `config/initializers/standard_health.rb` (or let the generator write it):
40
85
 
41
86
  ```ruby
42
87
  StandardHealth.configure do |c|
@@ -62,18 +107,21 @@ end
62
107
 
63
108
  ## EnvSpec
64
109
 
65
- The DSL has two declarations:
110
+ The DSL has three declarations:
66
111
 
67
112
  - `required :NAME` — missing value reports `status: :missing`
68
113
  - `recommended :NAME` — missing value reports `status: :should_set`
114
+ - `forbidden :NAME` — **present** value reports `status: :forbidden`; absent is `:ok`
69
115
 
70
- Both accept:
116
+ All three accept:
71
117
 
72
118
  - `in: %w[staging production]` — restricts the entry to those `APP_ENVIRONMENT` values; ignored otherwise. May also be a Symbol resolved via `mode_alias` (see below).
73
119
  - `description: "..."` — surfaced verbatim in the audit JSON
74
120
  - `consumed_by: "config/initializers/sentry.rb"` — pointer (or `Array<String>`) to where the value is read; surfaced verbatim
75
121
  - `if: -> { ... }` / `unless: -> { ... }` — Proc predicates evaluated at audit time. When `unless:` returns truthy or `if:` returns falsy, the entry is reported with `status: :not_applicable`
76
122
 
123
+ `required` and `recommended` additionally accept `expected_value:` (see below). `forbidden` does not — its assertion *is* "absent" — and combining them raises an `ArgumentError` at `define` time rather than silently ignoring the option.
124
+
77
125
  Audit output (one row per applicable entry):
78
126
 
79
127
  ```json
@@ -85,7 +133,68 @@ Audit output (one row per applicable entry):
85
133
  }
86
134
  ```
87
135
 
88
- Possible `status` values are `ok`, `missing` (required + absent), `should_set` (recommended + absent), and `not_applicable` (suppressed by an `if:`/`unless:` predicate).
136
+ Possible `status` values:
137
+
138
+ | status | meaning |
139
+ |---|---|
140
+ | `ok` | nothing to report |
141
+ | `missing` | `required` + absent |
142
+ | `should_set` | `recommended` + absent (advisory) |
143
+ | `forbidden` | `forbidden` + **present** |
144
+ | `mismatch` | present, but `expected_value:` says otherwise |
145
+ | `not_applicable` | suppressed by an `if:`/`unless:` predicate |
146
+
147
+ `missing`, `forbidden` and `mismatch` are the **violation** statuses — they drive the top-level `status: "incomplete"` and the optional `EnvSpecAudit` check. `should_set` is advisory and never counts as a violation.
148
+
149
+ ### `forbidden`: vars that must NOT be set
150
+
151
+ For dangerous ops toggles — demo modes, auth bypasses, bootstrap flags — that are legitimate on staging and must never survive promotion to production:
152
+
153
+ ```ruby
154
+ mode_alias :live, %w[production]
155
+
156
+ group "Production-forbidden toggles" do
157
+ forbidden :DEMO_MODE_ENABLED, in: :live,
158
+ description: "Demo/ops dashboard surfaces; unset before promoting"
159
+ forbidden :STANDARD_ID_BYPASS_CODE, in: :live,
160
+ description: "Fixed E2E OTP bypass code; staging only"
161
+ end
162
+ ```
163
+
164
+ ```json
165
+ { "name": "DEMO_MODE_ENABLED", "level": "forbidden", "status": "forbidden", "mode": "production" }
166
+ ```
167
+
168
+ Before this level existed, hosts expressed "forbidden" by declaring the var `recommended` with an `if: -> { ENV[...].present? }` predicate so that a set toggle at least produced a row — but the row was a green `:ok`, so the signal had to be carried by a hand-written custom check with its own duplicate list of toggle names. `forbidden` replaces both halves.
169
+
170
+ ### `expected_value:`: assert the value, not just presence
171
+
172
+ Presence auditing misses the case where a var is set to the *wrong* thing — which for a security toggle is the failure mode that matters:
173
+
174
+ ```ruby
175
+ # CSP is only enforced when this is exactly the string "false";
176
+ # "true" passes a presence audit while leaving the CSP report-only.
177
+ required :CONTENT_SECURITY_POLICY_REPORT_ONLY, in: :live,
178
+ expected_value: "false",
179
+ description: "Any other value leaves the CSP report-only"
180
+
181
+ required :LOG_LEVEL, expected_value: %w[info warn] # any of
182
+ required :DATABASE_URL, expected_value: /\Apostgres:/ # matched, not compared
183
+ ```
184
+
185
+ A present-but-wrong value reports `status: :mismatch`. An absent value still reports `:missing` / `:should_set` — there is nothing to compare. Comparison is on the string form, so `expected_value: 3000` matches `"3000"`.
186
+
187
+ **The actual value is never surfaced.** Env values are routinely secrets, and the endpoint exists to report *that* something is wrong, not to echo it back. The row carries the declared `expected_value` (host config, safe) and nothing else:
188
+
189
+ ```json
190
+ {
191
+ "name": "CONTENT_SECURITY_POLICY_REPORT_ONLY",
192
+ "level": "required",
193
+ "status": "mismatch",
194
+ "expected_value": "false",
195
+ "mode": "production"
196
+ }
197
+ ```
89
198
 
90
199
  ### Predicates: `if:` and `unless:`
91
200
 
@@ -184,6 +293,41 @@ Audit rows for entries declared inside a `group` block carry a `group` key:
184
293
 
185
294
  Entries declared outside any `group` block omit the `group` key entirely.
186
295
 
296
+ ### Top-level status
297
+
298
+ Since 0.4.1 the response carries a `status` alongside the audit, so a caller
299
+ can gate on one field instead of re-implementing the roll-up:
300
+
301
+ ```json
302
+ { "mode": "production", "status": "incomplete", "audit": [ ... ] }
303
+ ```
304
+
305
+ `incomplete` means at least one row is a **violation** — `missing`,
306
+ `forbidden`, or `mismatch`. `should_set` is advisory and never affects it.
307
+
308
+ The level is deliberately not consulted for `mismatch`: a `recommended` var
309
+ declared with an `expected_value:` that does not hold is a failed assertion,
310
+ not advice.
311
+
312
+ **The endpoint still returns 200 either way.** Callers that already gate on
313
+ `status == "incomplete"` pick up the `forbidden` and `mismatch` assertions for
314
+ free — which is why they joined this roll-up rather than getting a verdict of
315
+ their own.
316
+
317
+ ### Surfacing the audit on a health tier
318
+
319
+ `/diagnostics/env` is authed and polled by nobody, so config drift declared in
320
+ the spec is invisible until someone looks. Register the opt-in `EnvSpecAudit`
321
+ check to put the same verdict on a health tier:
322
+
323
+ ```ruby
324
+ c.register_check :env_spec, StandardHealth::Checks::EnvSpecAudit
325
+ ```
326
+
327
+ It fails on `:missing` / `:forbidden` / `:mismatch` rows and reports the
328
+ offending **names** (never values). Non-critical by default — see
329
+ [Opt-in checks](#opt-in-checks).
330
+
187
331
  ### Backward compatibility
188
332
 
189
333
  All v0.2.0 specs continue to produce identical audit output in v0.3.0. The new fields (`description`, `consumed_by`, `group`, `reason`) appear only when the corresponding feature is used; the new `:not_applicable` status only appears when a predicate suppresses an entry.
@@ -206,6 +350,81 @@ end
206
350
 
207
351
  `with_timing` captures `latency_ms` on success and converts any `StandardError` into `{ status: :fail, error: <message> }`.
208
352
 
353
+ **A check must never raise.** `Aggregator` rescues `StandardError` per check, so a buggy check degrades to `:fail` rather than 500ing the endpoint — but don't rely on that as the only line of defence. Route fallible work through `with_timing`.
354
+
355
+ ## Opt-in checks
356
+
357
+ Every check in this gem must be registered explicitly; **none are registered
358
+ automatically**, including the ones below. That is a deliberate constraint on
359
+ the gem: auto-registering a check turns it on for every host on a `bundle
360
+ update`, and a host that has never had SolidCable installed — or whose env spec
361
+ has a pre-existing violation — would go from green to yellow across its estate
362
+ for a check nobody asked for. New checks are additive only when they are opt-in.
363
+
364
+ | Check | Probes | `critical:` default |
365
+ |---|---|---|
366
+ | `Checks::ActiveRecord` | `SELECT 1` on the primary connection | `true` |
367
+ | `Checks::SolidQueue` | SolidQueue's tables | `true` |
368
+ | `Checks::SolidCache` | read-only `Rails.cache` probe | `false` |
369
+ | `Checks::SolidCable` | `solid_cable_messages` store | `false` |
370
+ | `Checks::EnvSpecAudit` | the configured `env_spec` | `false` |
371
+
372
+ The `critical:` default is only a default — `register_check` overrides it, and
373
+ what belongs on the rotation gate is a per-app decision.
374
+
375
+ ```ruby
376
+ StandardHealth.configure do |c|
377
+ c.register_check :solid_cable, StandardHealth::Checks::SolidCable
378
+ c.register_check :env_spec, StandardHealth::Checks::EnvSpecAudit
379
+ end
380
+ ```
381
+
382
+ ### `Checks::SolidCable`
383
+
384
+ Bounded read against `solid_cable_messages`, confirming both that the cable
385
+ schema is migrated and that its connection is up. Uses
386
+ `SolidCable::Record.connection` when SolidCable is pointed at a separate
387
+ database, falling back to the primary connection when it isn't.
388
+
389
+ Cable is a **degradable feature dependency** — prefer this on the aggregate
390
+ tier. A broken cable store should mark the app `degraded`, never de-rotate it.
391
+
392
+ ### `Checks::EnvSpecAudit`
393
+
394
+ Runs the configured `env_spec` and fails on the violation statuses
395
+ (`:missing`, `:forbidden`, `:mismatch`), reporting the offending variable
396
+ **names** grouped by status. Values never appear.
397
+
398
+ It reads the spec directly, so there is one declaration and no second list to
399
+ keep in sync — which is the whole reason it exists. It skips `consumed_by`
400
+ resolution (that does file IO per entry, fine for an on-demand doctor endpoint
401
+ and not fine on a tier polled every few seconds), and it is `:ok` when no
402
+ `env_spec` is configured.
403
+
404
+ Non-critical by default, and that default is load-bearing: config drift is
405
+ *visibility*, not a rotation signal. An instance with a stale toggle set is
406
+ still serving traffic correctly, and de-rotating it converts a warning into an
407
+ outage. Registering it `critical: true` asserts "this app must not serve at all
408
+ with a bad env" — a real but rare posture. Know which you want.
409
+
410
+ The aggregator instantiates checks with `name:`/`critical:` only, so to narrow
411
+ what counts as a failure, subclass:
412
+
413
+ ```ruby
414
+ class ForbiddenTogglesOnly < StandardHealth::Checks::EnvSpecAudit
415
+ def initialize(name: :forbidden_toggles, critical: false)
416
+ super(name: name, critical: critical, fail_on: %i[forbidden])
417
+ end
418
+ end
419
+ ```
420
+
421
+ Note the failure message is subject to [redaction](#failure-detail-is-redacted)
422
+ on `/ready` like any other check; the detail reaches you through logs and
423
+ Sentry. The row carries a stable `error_class` of
424
+ `StandardHealth::EnvSpecViolation`, so the redacted body still groups on
425
+ `error_code: "standard_health_env_spec_violation"` rather than a useless
426
+ `standard_error`.
427
+
209
428
  ## Auth
210
429
 
211
430
  `/alive` and `/ready` are typically left open for orchestrator probes. `/diagnostics/env` enumerates which env vars are missing — that's potentially sensitive, so the host app is responsible for protecting it.
@@ -283,6 +502,130 @@ When `diagnostics_parent_controller` is unset, `DiagnosticsController` falls bac
283
502
 
284
503
  The orchestrator should pull the instance out of rotation only on 503; degraded means "still serving, page someone."
285
504
 
505
+ `skipped` also exists, for a check the total budget never reached (see
506
+ Timeouts). A skip floors the roll-up at `degraded` and **never** produces
507
+ `unavailable`, even for a critical check.
508
+
509
+ ## Failure detail is redacted
510
+
511
+ Since 0.4.1, a failing check reports the exception **class** rather than its
512
+ message:
513
+
514
+ ```json
515
+ { "name": "database", "critical": true, "status": "fail",
516
+ "error_class": "PG::ConnectionBad", "error_code": "pg_connection_bad" }
517
+ ```
518
+
519
+ `/ready` is unauthenticated by design — probes carry no credentials — and a
520
+ raw driver message will happily tell an anonymous caller the database host,
521
+ port and username. The full message still reaches your logs and Sentry through
522
+ the instrumentation below; it just isn't in the public body.
523
+
524
+ ```ruby
525
+ c.expose_check_errors = true # restore pre-0.4.1 verbose bodies
526
+ c.detail_token = ENV["HEALTH_DETAIL_TOKEN"] # X-Health-Token unlocks detail
527
+ ```
528
+
529
+ `status`, `name`, `critical`, `latency_ms` and `generated_at` are unchanged.
530
+
531
+ ## Instrumentation
532
+
533
+ The gem emits events on whichever bus is live — `Rails.event` on Rails 8.1+,
534
+ `ActiveSupport::Notifications` otherwise:
535
+
536
+ | Event | Payload |
537
+ |---|---|
538
+ | `standard_health.check.completed` | `name`, `critical`, `status`, `latency_ms`, `error_class`, `error_message` |
539
+ | `standard_health.check.timed_out` | `name`, `critical`, `timeout_s` |
540
+ | `standard_health.ready.evaluated` | `status`, `duration_ms`, `failed[]` |
541
+
542
+ Three subscribers are registered automatically. Their noise profiles differ on
543
+ purpose, because health events are **polls**, not state transitions — a 10s
544
+ probe period means ~6 evaluations/minute/instance:
545
+
546
+ - **Logger** — silent while healthy. `warn` on degraded, `error` on
547
+ unavailable. A line per evaluation would be ~8,640/day/instance of
548
+ "everything is fine".
549
+ - **Sentry** — only on status **change**, plus a 60s repeat floor for a
550
+ sustained bad state, plus one `info` on recovery. Capturing every non-ok
551
+ poll would turn a five-minute outage into ~30 duplicate issues. Sentry is a
552
+ soft dependency; no gemspec entry.
553
+ - **Metrics** — every poll, deliberately. Counters plus latency
554
+ distributions are what make `latency_ms` chartable.
555
+
556
+ ```ruby
557
+ StandardHealth.configure do |c|
558
+ c.instrumentation_enabled = true # default
559
+ c.logger = Rails.logger # default: Rails.logger
560
+ c.sentry_enabled = true # default
561
+ c.metric_prefix = "health" # default
562
+
563
+ c.add_notifier(MyNotifier.new) # must respond to call(event_name, payload)
564
+ end
565
+ ```
566
+
567
+ ## Timeouts
568
+
569
+ **Off by default.** Both settings are `nil`, which is byte-identical to 0.4.0.
570
+
571
+ ```ruby
572
+ c.default_check_timeout = 2.0 # seconds, per check
573
+ c.total_check_budget = 5.0 # checked BEFORE each check starts
574
+ c.register_check :db, MyCheck, critical: true, timeout: 1.0 # per-check override
575
+ ```
576
+
577
+ **`total_check_budget` bounds how many checks RUN, not how long the probe
578
+ takes.** It is evaluated before each check starts, not during one. With a 1s
579
+ budget and a first check that blocks for 30s, `/ready` still takes 30s — only
580
+ the checks *after* it are skipped.
581
+
582
+ Clamping each check to the remaining budget would close that gap, and is
583
+ deliberately not done: it would apply `Timeout.timeout` to checks whose author
584
+ never asked for one, and that mechanism raises into the thread at an arbitrary
585
+ point (see below). Setting a budget must not silently opt you into that. To
586
+ bound wall-clock, put a `timeout:` on the checks that can safely take one.
587
+
588
+ Enabling them is a semantic change — a check that has always been
589
+ slow-but-fine starts reporting `:fail`, and for a critical check that pulls
590
+ the instance out of rotation. Pick values from observed `latency_ms` rather
591
+ than intuition; that is what the `check.completed` events are for.
592
+
593
+ Checks the total budget never reaches report `:skipped`, never silently `:ok`.
594
+ A skip alone floors the roll-up at `degraded` — otherwise a slow *non-critical*
595
+ check could exhaust the budget, leave the database check unrun, and pull a
596
+ healthy instance out of rotation.
597
+
598
+ Checks run **sequentially**. They are not parallelised on purpose: running
599
+ them in threads would mean connection checkouts from the health path, and a
600
+ health endpoint that can exhaust the connection pool is a worse problem than
601
+ the one it was added to solve.
602
+
603
+ ### ⚠ Know what `Timeout.timeout` actually does before enabling this
604
+
605
+ Ruby's `Timeout.timeout` interrupts by raising **into the running thread at an
606
+ arbitrary point**. It cannot wait for a safe boundary. If a timeout fires
607
+ while a check is mid-way through a non-atomic operation — say a connection
608
+ checkout in the `ActiveRecord` check — the connection can be returned to the
609
+ pool in a broken state. You would have traded a slow check for a corrupted
610
+ pool, on the health path, during an incident.
611
+
612
+ The dedicated `CheckTimeout` class solves the *mislabelling* problem (a host's
613
+ own `Timeout::Error` is never mistaken for ours). **It does not solve this
614
+ one.** Nothing can, at this layer.
615
+
616
+ So, before turning timeouts on:
617
+
618
+ - Prefer a driver-level timeout where one exists — `connect_timeout` /
619
+ `statement_timeout` on the database, a client timeout on an HTTP dependency.
620
+ Those abort at a safe point because the driver owns the operation.
621
+ - Reserve `timeout:` here for checks whose `run` is genuinely interruptible —
622
+ typically your own custom checks — rather than the built-in datastore ones.
623
+ - Set the value comfortably above observed p99 (see the `latency_ms` in
624
+ `check.completed`), so it fires on a hang rather than on a slow day.
625
+
626
+ This is why the defaults ship as `nil` and why choosing them is deferred: the
627
+ right answer is usually a driver timeout, not this.
628
+
286
629
  ## License
287
630
 
288
631
  MIT.
@@ -25,9 +25,36 @@ module StandardHealth
25
25
 
26
26
  render json: {
27
27
  mode: mode,
28
+ status: audit_status(audit),
28
29
  audit: audit,
29
30
  generated_at: Time.now.utc.iso8601
30
31
  }
31
32
  end
33
+
34
+ private
35
+
36
+ # Top-level verdict so a caller can gate on ONE field instead of
37
+ # re-implementing the roll-up over `audit`.
38
+ #
39
+ # :ok — no violations
40
+ # :incomplete — at least one row is `:missing`, `:forbidden`, or
41
+ # `:mismatch` (`EnvSpec::VIOLATION_STATUSES`)
42
+ #
43
+ # The `forbidden`/`mismatch` statuses join the roll-up rather than
44
+ # getting a verdict of their own, so callers that already gate on
45
+ # `status == "incomplete"` pick up the new assertions for free. Note the
46
+ # level is deliberately NOT consulted: a `recommended` var declared with
47
+ # an `expected_value:` that does not hold is a failed assertion, not
48
+ # advice. The advisory status stays `:should_set`, which is not a
49
+ # violation.
50
+ #
51
+ # The endpoint still returns 200 either way, so nothing that asserts on
52
+ # the status code breaks.
53
+ def audit_status(audit)
54
+ violated = Array(audit).any? do |row|
55
+ EnvSpec::VIOLATION_STATUSES.include?(row[:status])
56
+ end
57
+ violated ? :incomplete : :ok
58
+ end
32
59
  end
33
60
  end