standard_health 0.4.1 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +74 -0
- data/README.md +213 -11
- data/app/controllers/standard_health/diagnostics_controller.rb +16 -10
- data/lib/generators/standard_health/install/install_generator.rb +150 -0
- data/lib/generators/standard_health/install/templates/initializer.rb.erb +99 -0
- data/lib/standard_health/checks/env_spec_audit.rb +105 -0
- data/lib/standard_health/checks/solid_cable.rb +54 -0
- data/lib/standard_health/configuration.rb +3 -2
- data/lib/standard_health/env_spec.rb +108 -6
- data/lib/standard_health/version.rb +1 -1
- data/lib/standard_health.rb +4 -0
- metadata +7 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 3d6a6ea3266864aaa15232529241e2040a836d810c4898eb512746e78a582901
|
|
4
|
+
data.tar.gz: fc9575f2c3e425a6065e3e5285094d03ba5cd694848c386937251b0fea5d75af
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 83a04f1025e9e8b6e62344ff53828a716c262d7779d30f64afeb69b704fb34289e9138f751a93fb46215ba24c71612e17e9cfd7a37c859ac67feca64bbb2e25a
|
|
7
|
+
data.tar.gz: 6cf0b8d3a4100e8d3737eb1bbaddfb7c1a40d6dd77ad4e9c097ad547308ff196fb012b46ebb7c1d267b30d784abc863f734df4bd8520971d99af9d705e2af51b
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,80 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.5.0] - 2026-07-30
|
|
11
|
+
|
|
12
|
+
Closes the gap that had four host apps writing their own checks. The env-spec
|
|
13
|
+
DSL could only audit **presence**, so "this toggle must not be set on
|
|
14
|
+
production" and "this value must be exactly `false`" were inexpressible — and
|
|
15
|
+
the workaround was a hand-written check carrying a duplicate list of variable
|
|
16
|
+
names, kept in sync with the spec by comment.
|
|
17
|
+
|
|
18
|
+
Everything here is **additive and opt-in**. No check is auto-registered and no
|
|
19
|
+
existing status changes meaning, so this is a pure `bundle update` for
|
|
20
|
+
consumers on `~> 0.4`.
|
|
21
|
+
|
|
22
|
+
### Added
|
|
23
|
+
|
|
24
|
+
- **`forbidden` EnvSpec level.** Inverts the presence rule: absent is `:ok`,
|
|
25
|
+
present reports the new `:forbidden` status. For dangerous ops toggles —
|
|
26
|
+
demo modes, auth bypasses, bootstrap flags — that are legitimate on staging
|
|
27
|
+
and must never survive promotion. Composes with `in:`, `mode_alias`, groups
|
|
28
|
+
and the `if:`/`unless:` predicates like any other level.
|
|
29
|
+
- **`expected_value:` assertion** on `required` / `recommended`. Asserts the
|
|
30
|
+
value, not just presence; a present-but-wrong value reports the new
|
|
31
|
+
`:mismatch` status. Accepts a String, an Array (any-of), a Regexp (matched,
|
|
32
|
+
not compared), or any object comparable by its string form. The **actual
|
|
33
|
+
value is never surfaced** — env values are routinely secrets — only the
|
|
34
|
+
declared expectation.
|
|
35
|
+
- **`Checks::EnvSpecAudit`** (opt-in). Runs the configured `env_spec` and
|
|
36
|
+
fails on `:missing` / `:forbidden` / `:mismatch`, reporting offending
|
|
37
|
+
variable *names* grouped by status. Reads the spec directly, so there is no
|
|
38
|
+
second list to drift. Non-critical by default: config drift is visibility,
|
|
39
|
+
not a rotation signal. Skips `consumed_by` resolution — that is file IO per
|
|
40
|
+
entry, fine for an on-demand doctor endpoint and not on a polled tier.
|
|
41
|
+
- **`Checks::SolidCable`** (opt-in). Bounded read against
|
|
42
|
+
`solid_cable_messages`. Promoted from sidekick-web. Falls back to the
|
|
43
|
+
primary connection when SolidCable isn't pointed at its own database.
|
|
44
|
+
- **`standard_health:install` generator.** Writes the initializer and mounts
|
|
45
|
+
the engine, with the aggregate-route ordering requirement noted inline.
|
|
46
|
+
Idempotent; `--skip-initializer`, `--skip-routes`, `--force`.
|
|
47
|
+
|
|
48
|
+
### Changed
|
|
49
|
+
|
|
50
|
+
- **`/diagnostics/env` top-level `status`** now reports `incomplete` for
|
|
51
|
+
`:forbidden` and `:mismatch` rows as well as `:missing`. Callers already
|
|
52
|
+
gating on that one field pick up the new assertions for free, which is why
|
|
53
|
+
they joined the existing roll-up instead of getting a verdict of their own.
|
|
54
|
+
The endpoint still returns 200 either way. `:should_set` remains advisory
|
|
55
|
+
and still never affects the roll-up.
|
|
56
|
+
- **Rails dependency relaxed** from `~> 8.0` to `>= 8.0`, matching the rest of
|
|
57
|
+
the `standard_*` family. `~> 8.0` was the only floor in the family that
|
|
58
|
+
would have blocked a Rails 9 host.
|
|
59
|
+
|
|
60
|
+
### Not in this release
|
|
61
|
+
|
|
62
|
+
- **Timeout defaults remain unset.** 0.4.1 said sensible `default_check_timeout`
|
|
63
|
+
/ budget values would be chosen "in 0.5.0". They are not: choosing them still
|
|
64
|
+
needs enough real p99 latency data from the instrumentation 0.4.1 added, and
|
|
65
|
+
guessing a timeout on a health check is how you cause the outage you were
|
|
66
|
+
trying to prevent. The machinery stays opt-in per check. The stale promise in
|
|
67
|
+
`Configuration` has been repointed (#48).
|
|
68
|
+
- **`/diagnostics/env` still returns 200** when `status` is `incomplete`.
|
|
69
|
+
Turning that into a 503 is a breaking contract change for any caller gating on
|
|
70
|
+
the response code, so it wants its own decision rather than riding along with
|
|
71
|
+
an otherwise additive release.
|
|
72
|
+
|
|
73
|
+
### Documentation
|
|
74
|
+
|
|
75
|
+
- **The aggregate `GET /health` tier and its ordering requirement are now
|
|
76
|
+
documented.** The engine draws sub-paths only; the aggregate tier is the
|
|
77
|
+
host's job and must be drawn **before** `mount`. An app that mounts first
|
|
78
|
+
and relies on the engine to serve it silently has no aggregate tier — no
|
|
79
|
+
boot error, no failing route spec. Also documents what each of the four
|
|
80
|
+
tiers is for, and that readiness gates only on hard infra the app owns.
|
|
81
|
+
- New README section on opt-in checks, including why nothing here
|
|
82
|
+
auto-registers.
|
|
83
|
+
|
|
10
84
|
## [0.4.1] - 2026-07-29
|
|
11
85
|
|
|
12
86
|
Observability release. Until now the gem emitted **nothing** — no events, no
|
data/README.md
CHANGED
|
@@ -18,7 +18,16 @@ Add to your Gemfile:
|
|
|
18
18
|
gem "standard_health"
|
|
19
19
|
```
|
|
20
20
|
|
|
21
|
-
Then `bundle install
|
|
21
|
+
Then `bundle install`, and run the install generator:
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
bin/rails generate standard_health:install
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
It writes `config/initializers/standard_health.rb` and mounts the engine in
|
|
28
|
+
`config/routes.rb` with the ordering requirement below noted inline. It is
|
|
29
|
+
idempotent — re-running skips what is already installed. Flags:
|
|
30
|
+
`--skip-initializer`, `--skip-routes`, `--force` (overwrite the initializer).
|
|
22
31
|
|
|
23
32
|
## Mounting
|
|
24
33
|
|
|
@@ -34,9 +43,45 @@ This wires up:
|
|
|
34
43
|
- `GET /health/ready`
|
|
35
44
|
- `GET /health/diagnostics/env`
|
|
36
45
|
|
|
46
|
+
### The aggregate `GET /health` is yours to draw — before the mount
|
|
47
|
+
|
|
48
|
+
**The engine draws sub-paths only.** There is no `GET /health` in
|
|
49
|
+
`config/routes.rb` here; the aggregate tier is the host's responsibility, and
|
|
50
|
+
the ordering is not optional:
|
|
51
|
+
|
|
52
|
+
```ruby
|
|
53
|
+
get "/health", to: "health_aggregate#show" # aggregate — FIRST
|
|
54
|
+
mount StandardHealth::Engine => "/health", as: :standard_health
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
An app that mounts the engine **first** and relies on it to serve the aggregate
|
|
58
|
+
tier **silently has no aggregate tier at all** — no boot error, no 404 at boot,
|
|
59
|
+
and no failing route spec unless one exists. The failure surfaces as a
|
|
60
|
+
dashboard that has been reading nothing for months. Add a route spec asserting
|
|
61
|
+
`GET /health` resolves.
|
|
62
|
+
|
|
63
|
+
Why this way round: `mount` claims the `/health` prefix, so a bare `GET /health`
|
|
64
|
+
drawn *after* it still resolves (the engine has no route for the bare path) —
|
|
65
|
+
but nothing about that is obvious from reading the file, and getting it
|
|
66
|
+
backwards fails silently either way. Draw it first and the ordering question
|
|
67
|
+
never arises.
|
|
68
|
+
|
|
69
|
+
The four tiers, and what each is for:
|
|
70
|
+
|
|
71
|
+
| Route | Tier | Consumer |
|
|
72
|
+
|---|---|---|
|
|
73
|
+
| `GET /health/alive` | liveness — process is up | restart probes, CI reachability gates |
|
|
74
|
+
| `GET /health/ready` | readiness — **rotation gate** | load balancer / platform health check |
|
|
75
|
+
| `GET /health` | aggregate — full picture | dashboards, humans |
|
|
76
|
+
| `GET /health/diagnostics/env` | doctor (authed) | on-call, config audits |
|
|
77
|
+
|
|
78
|
+
Readiness gates **only** on hard infra the app owns. A soft upstream that
|
|
79
|
+
degrades must not pull an instance out of rotation — put those on the aggregate
|
|
80
|
+
tier instead.
|
|
81
|
+
|
|
37
82
|
## Configuration
|
|
38
83
|
|
|
39
|
-
Create `config/initializers/standard_health.rb
|
|
84
|
+
Create `config/initializers/standard_health.rb` (or let the generator write it):
|
|
40
85
|
|
|
41
86
|
```ruby
|
|
42
87
|
StandardHealth.configure do |c|
|
|
@@ -62,18 +107,21 @@ end
|
|
|
62
107
|
|
|
63
108
|
## EnvSpec
|
|
64
109
|
|
|
65
|
-
The DSL has
|
|
110
|
+
The DSL has three declarations:
|
|
66
111
|
|
|
67
112
|
- `required :NAME` — missing value reports `status: :missing`
|
|
68
113
|
- `recommended :NAME` — missing value reports `status: :should_set`
|
|
114
|
+
- `forbidden :NAME` — **present** value reports `status: :forbidden`; absent is `:ok`
|
|
69
115
|
|
|
70
|
-
|
|
116
|
+
All three accept:
|
|
71
117
|
|
|
72
118
|
- `in: %w[staging production]` — restricts the entry to those `APP_ENVIRONMENT` values; ignored otherwise. May also be a Symbol resolved via `mode_alias` (see below).
|
|
73
119
|
- `description: "..."` — surfaced verbatim in the audit JSON
|
|
74
120
|
- `consumed_by: "config/initializers/sentry.rb"` — pointer (or `Array<String>`) to where the value is read; surfaced verbatim
|
|
75
121
|
- `if: -> { ... }` / `unless: -> { ... }` — Proc predicates evaluated at audit time. When `unless:` returns truthy or `if:` returns falsy, the entry is reported with `status: :not_applicable`
|
|
76
122
|
|
|
123
|
+
`required` and `recommended` additionally accept `expected_value:` (see below). `forbidden` does not — its assertion *is* "absent" — and combining them raises an `ArgumentError` at `define` time rather than silently ignoring the option.
|
|
124
|
+
|
|
77
125
|
Audit output (one row per applicable entry):
|
|
78
126
|
|
|
79
127
|
```json
|
|
@@ -85,7 +133,68 @@ Audit output (one row per applicable entry):
|
|
|
85
133
|
}
|
|
86
134
|
```
|
|
87
135
|
|
|
88
|
-
Possible `status` values
|
|
136
|
+
Possible `status` values:
|
|
137
|
+
|
|
138
|
+
| status | meaning |
|
|
139
|
+
|---|---|
|
|
140
|
+
| `ok` | nothing to report |
|
|
141
|
+
| `missing` | `required` + absent |
|
|
142
|
+
| `should_set` | `recommended` + absent (advisory) |
|
|
143
|
+
| `forbidden` | `forbidden` + **present** |
|
|
144
|
+
| `mismatch` | present, but `expected_value:` says otherwise |
|
|
145
|
+
| `not_applicable` | suppressed by an `if:`/`unless:` predicate |
|
|
146
|
+
|
|
147
|
+
`missing`, `forbidden` and `mismatch` are the **violation** statuses — they drive the top-level `status: "incomplete"` and the optional `EnvSpecAudit` check. `should_set` is advisory and never counts as a violation.
|
|
148
|
+
|
|
149
|
+
### `forbidden`: vars that must NOT be set
|
|
150
|
+
|
|
151
|
+
For dangerous ops toggles — demo modes, auth bypasses, bootstrap flags — that are legitimate on staging and must never survive promotion to production:
|
|
152
|
+
|
|
153
|
+
```ruby
|
|
154
|
+
mode_alias :live, %w[production]
|
|
155
|
+
|
|
156
|
+
group "Production-forbidden toggles" do
|
|
157
|
+
forbidden :DEMO_MODE_ENABLED, in: :live,
|
|
158
|
+
description: "Demo/ops dashboard surfaces; unset before promoting"
|
|
159
|
+
forbidden :STANDARD_ID_BYPASS_CODE, in: :live,
|
|
160
|
+
description: "Fixed E2E OTP bypass code; staging only"
|
|
161
|
+
end
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
```json
|
|
165
|
+
{ "name": "DEMO_MODE_ENABLED", "level": "forbidden", "status": "forbidden", "mode": "production" }
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Before this level existed, hosts expressed "forbidden" by declaring the var `recommended` with an `if: -> { ENV[...].present? }` predicate so that a set toggle at least produced a row — but the row was a green `:ok`, so the signal had to be carried by a hand-written custom check with its own duplicate list of toggle names. `forbidden` replaces both halves.
|
|
169
|
+
|
|
170
|
+
### `expected_value:`: assert the value, not just presence
|
|
171
|
+
|
|
172
|
+
Presence auditing misses the case where a var is set to the *wrong* thing — which for a security toggle is the failure mode that matters:
|
|
173
|
+
|
|
174
|
+
```ruby
|
|
175
|
+
# CSP is only enforced when this is exactly the string "false";
|
|
176
|
+
# "true" passes a presence audit while leaving the CSP report-only.
|
|
177
|
+
required :CONTENT_SECURITY_POLICY_REPORT_ONLY, in: :live,
|
|
178
|
+
expected_value: "false",
|
|
179
|
+
description: "Any other value leaves the CSP report-only"
|
|
180
|
+
|
|
181
|
+
required :LOG_LEVEL, expected_value: %w[info warn] # any of
|
|
182
|
+
required :DATABASE_URL, expected_value: /\Apostgres:/ # matched, not compared
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
A present-but-wrong value reports `status: :mismatch`. An absent value still reports `:missing` / `:should_set` — there is nothing to compare. Comparison is on the string form, so `expected_value: 3000` matches `"3000"`.
|
|
186
|
+
|
|
187
|
+
**The actual value is never surfaced.** Env values are routinely secrets, and the endpoint exists to report *that* something is wrong, not to echo it back. The row carries the declared `expected_value` (host config, safe) and nothing else:
|
|
188
|
+
|
|
189
|
+
```json
|
|
190
|
+
{
|
|
191
|
+
"name": "CONTENT_SECURITY_POLICY_REPORT_ONLY",
|
|
192
|
+
"level": "required",
|
|
193
|
+
"status": "mismatch",
|
|
194
|
+
"expected_value": "false",
|
|
195
|
+
"mode": "production"
|
|
196
|
+
}
|
|
197
|
+
```
|
|
89
198
|
|
|
90
199
|
### Predicates: `if:` and `unless:`
|
|
91
200
|
|
|
@@ -193,13 +302,31 @@ can gate on one field instead of re-implementing the roll-up:
|
|
|
193
302
|
{ "mode": "production", "status": "incomplete", "audit": [ ... ] }
|
|
194
303
|
```
|
|
195
304
|
|
|
196
|
-
`incomplete` means at least one
|
|
197
|
-
advisory and never affects it.
|
|
305
|
+
`incomplete` means at least one row is a **violation** — `missing`,
|
|
306
|
+
`forbidden`, or `mismatch`. `should_set` is advisory and never affects it.
|
|
198
307
|
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
308
|
+
The level is deliberately not consulted for `mismatch`: a `recommended` var
|
|
309
|
+
declared with an `expected_value:` that does not hold is a failed assertion,
|
|
310
|
+
not advice.
|
|
311
|
+
|
|
312
|
+
**The endpoint still returns 200 either way.** Callers that already gate on
|
|
313
|
+
`status == "incomplete"` pick up the `forbidden` and `mismatch` assertions for
|
|
314
|
+
free — which is why they joined this roll-up rather than getting a verdict of
|
|
315
|
+
their own.
|
|
316
|
+
|
|
317
|
+
### Surfacing the audit on a health tier
|
|
318
|
+
|
|
319
|
+
`/diagnostics/env` is authed and polled by nobody, so config drift declared in
|
|
320
|
+
the spec is invisible until someone looks. Register the opt-in `EnvSpecAudit`
|
|
321
|
+
check to put the same verdict on a health tier:
|
|
322
|
+
|
|
323
|
+
```ruby
|
|
324
|
+
c.register_check :env_spec, StandardHealth::Checks::EnvSpecAudit
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
It fails on `:missing` / `:forbidden` / `:mismatch` rows and reports the
|
|
328
|
+
offending **names** (never values). Non-critical by default — see
|
|
329
|
+
[Opt-in checks](#opt-in-checks).
|
|
203
330
|
|
|
204
331
|
### Backward compatibility
|
|
205
332
|
|
|
@@ -223,6 +350,81 @@ end
|
|
|
223
350
|
|
|
224
351
|
`with_timing` captures `latency_ms` on success and converts any `StandardError` into `{ status: :fail, error: <message> }`.
|
|
225
352
|
|
|
353
|
+
**A check must never raise.** `Aggregator` rescues `StandardError` per check, so a buggy check degrades to `:fail` rather than 500ing the endpoint — but don't rely on that as the only line of defence. Route fallible work through `with_timing`.
|
|
354
|
+
|
|
355
|
+
## Opt-in checks
|
|
356
|
+
|
|
357
|
+
Every check in this gem must be registered explicitly; **none are registered
|
|
358
|
+
automatically**, including the ones below. That is a deliberate constraint on
|
|
359
|
+
the gem: auto-registering a check turns it on for every host on a `bundle
|
|
360
|
+
update`, and a host that has never had SolidCable installed — or whose env spec
|
|
361
|
+
has a pre-existing violation — would go from green to yellow across its estate
|
|
362
|
+
for a check nobody asked for. New checks are additive only when they are opt-in.
|
|
363
|
+
|
|
364
|
+
| Check | Probes | `critical:` default |
|
|
365
|
+
|---|---|---|
|
|
366
|
+
| `Checks::ActiveRecord` | `SELECT 1` on the primary connection | `true` |
|
|
367
|
+
| `Checks::SolidQueue` | SolidQueue's tables | `true` |
|
|
368
|
+
| `Checks::SolidCache` | read-only `Rails.cache` probe | `false` |
|
|
369
|
+
| `Checks::SolidCable` | `solid_cable_messages` store | `false` |
|
|
370
|
+
| `Checks::EnvSpecAudit` | the configured `env_spec` | `false` |
|
|
371
|
+
|
|
372
|
+
The `critical:` default is only a default — `register_check` overrides it, and
|
|
373
|
+
what belongs on the rotation gate is a per-app decision.
|
|
374
|
+
|
|
375
|
+
```ruby
|
|
376
|
+
StandardHealth.configure do |c|
|
|
377
|
+
c.register_check :solid_cable, StandardHealth::Checks::SolidCable
|
|
378
|
+
c.register_check :env_spec, StandardHealth::Checks::EnvSpecAudit
|
|
379
|
+
end
|
|
380
|
+
```
|
|
381
|
+
|
|
382
|
+
### `Checks::SolidCable`
|
|
383
|
+
|
|
384
|
+
Bounded read against `solid_cable_messages`, confirming both that the cable
|
|
385
|
+
schema is migrated and that its connection is up. Uses
|
|
386
|
+
`SolidCable::Record.connection` when SolidCable is pointed at a separate
|
|
387
|
+
database, falling back to the primary connection when it isn't.
|
|
388
|
+
|
|
389
|
+
Cable is a **degradable feature dependency** — prefer this on the aggregate
|
|
390
|
+
tier. A broken cable store should mark the app `degraded`, never de-rotate it.
|
|
391
|
+
|
|
392
|
+
### `Checks::EnvSpecAudit`
|
|
393
|
+
|
|
394
|
+
Runs the configured `env_spec` and fails on the violation statuses
|
|
395
|
+
(`:missing`, `:forbidden`, `:mismatch`), reporting the offending variable
|
|
396
|
+
**names** grouped by status. Values never appear.
|
|
397
|
+
|
|
398
|
+
It reads the spec directly, so there is one declaration and no second list to
|
|
399
|
+
keep in sync — which is the whole reason it exists. It skips `consumed_by`
|
|
400
|
+
resolution (that does file IO per entry, fine for an on-demand doctor endpoint
|
|
401
|
+
and not fine on a tier polled every few seconds), and it is `:ok` when no
|
|
402
|
+
`env_spec` is configured.
|
|
403
|
+
|
|
404
|
+
Non-critical by default, and that default is load-bearing: config drift is
|
|
405
|
+
*visibility*, not a rotation signal. An instance with a stale toggle set is
|
|
406
|
+
still serving traffic correctly, and de-rotating it converts a warning into an
|
|
407
|
+
outage. Registering it `critical: true` asserts "this app must not serve at all
|
|
408
|
+
with a bad env" — a real but rare posture. Know which you want.
|
|
409
|
+
|
|
410
|
+
The aggregator instantiates checks with `name:`/`critical:` only, so to narrow
|
|
411
|
+
what counts as a failure, subclass:
|
|
412
|
+
|
|
413
|
+
```ruby
|
|
414
|
+
class ForbiddenTogglesOnly < StandardHealth::Checks::EnvSpecAudit
|
|
415
|
+
def initialize(name: :forbidden_toggles, critical: false)
|
|
416
|
+
super(name: name, critical: critical, fail_on: %i[forbidden])
|
|
417
|
+
end
|
|
418
|
+
end
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
Note the failure message is subject to [redaction](#failure-detail-is-redacted)
|
|
422
|
+
on `/ready` like any other check; the detail reaches you through logs and
|
|
423
|
+
Sentry. The row carries a stable `error_class` of
|
|
424
|
+
`StandardHealth::EnvSpecViolation`, so the redacted body still groups on
|
|
425
|
+
`error_code: "standard_health_env_spec_violation"` rather than a useless
|
|
426
|
+
`standard_error`.
|
|
427
|
+
|
|
226
428
|
## Auth
|
|
227
429
|
|
|
228
430
|
`/alive` and `/ready` are typically left open for orchestrator probes. `/diagnostics/env` enumerates which env vars are missing — that's potentially sensitive, so the host app is responsible for protecting it.
|
|
@@ -36,19 +36,25 @@ module StandardHealth
|
|
|
36
36
|
# Top-level verdict so a caller can gate on ONE field instead of
|
|
37
37
|
# re-implementing the roll-up over `audit`.
|
|
38
38
|
#
|
|
39
|
-
# :ok —
|
|
40
|
-
# :incomplete — at least one
|
|
39
|
+
# :ok — no violations
|
|
40
|
+
# :incomplete — at least one row is `:missing`, `:forbidden`, or
|
|
41
|
+
# `:mismatch` (`EnvSpec::VIOLATION_STATUSES`)
|
|
41
42
|
#
|
|
42
|
-
#
|
|
43
|
-
#
|
|
44
|
-
#
|
|
45
|
-
#
|
|
46
|
-
#
|
|
43
|
+
# The `forbidden`/`mismatch` statuses join the roll-up rather than
|
|
44
|
+
# getting a verdict of their own, so callers that already gate on
|
|
45
|
+
# `status == "incomplete"` pick up the new assertions for free. Note the
|
|
46
|
+
# level is deliberately NOT consulted: a `recommended` var declared with
|
|
47
|
+
# an `expected_value:` that does not hold is a failed assertion, not
|
|
48
|
+
# advice. The advisory status stays `:should_set`, which is not a
|
|
49
|
+
# violation.
|
|
50
|
+
#
|
|
51
|
+
# The endpoint still returns 200 either way, so nothing that asserts on
|
|
52
|
+
# the status code breaks.
|
|
47
53
|
def audit_status(audit)
|
|
48
|
-
|
|
49
|
-
|
|
54
|
+
violated = Array(audit).any? do |row|
|
|
55
|
+
EnvSpec::VIOLATION_STATUSES.include?(row[:status])
|
|
50
56
|
end
|
|
51
|
-
|
|
57
|
+
violated ? :incomplete : :ok
|
|
52
58
|
end
|
|
53
59
|
end
|
|
54
60
|
end
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "rails/generators"
|
|
4
|
+
|
|
5
|
+
module StandardHealth
|
|
6
|
+
module Generators
|
|
7
|
+
# Installs StandardHealth in a host Rails application.
|
|
8
|
+
#
|
|
9
|
+
# Writes `config/initializers/standard_health.rb` and mounts the engine in
|
|
10
|
+
# `config/routes.rb`.
|
|
11
|
+
#
|
|
12
|
+
# The routes step exists because of a failure mode that is otherwise
|
|
13
|
+
# invisible: the engine only draws SUB-paths (`/alive`, `/ready`,
|
|
14
|
+
# `/diagnostics/env`), so an app that mounts it and expects an aggregate
|
|
15
|
+
# `GET /health` silently has no aggregate tier — no boot error, no failing
|
|
16
|
+
# route spec. The generated block carries the ordering requirement as a
|
|
17
|
+
# comment right where someone will read it.
|
|
18
|
+
#
|
|
19
|
+
# Idempotent: re-running skips pieces already installed. `--skip-*` opts
|
|
20
|
+
# out of individual steps; `--force` overwrites an existing initializer.
|
|
21
|
+
class InstallGenerator < Rails::Generators::Base
|
|
22
|
+
source_root File.expand_path("templates", __dir__)
|
|
23
|
+
|
|
24
|
+
INITIALIZER_PATH = "config/initializers/standard_health.rb"
|
|
25
|
+
ROUTES_PATH = "config/routes.rb"
|
|
26
|
+
|
|
27
|
+
desc <<~DESC
|
|
28
|
+
Installs StandardHealth. By default this:
|
|
29
|
+
* writes config/initializers/standard_health.rb
|
|
30
|
+
* mounts StandardHealth::Engine in config/routes.rb, with the
|
|
31
|
+
aggregate-route ordering requirement noted inline
|
|
32
|
+
|
|
33
|
+
Use --skip-* flags to opt out of individual steps when re-running on an
|
|
34
|
+
existing install. The generator is idempotent — already-installed
|
|
35
|
+
pieces are skipped with a clear message. Pass --force to overwrite an
|
|
36
|
+
existing initializer.
|
|
37
|
+
DESC
|
|
38
|
+
|
|
39
|
+
class_option :skip_initializer, type: :boolean, default: false,
|
|
40
|
+
desc: "Do not write #{INITIALIZER_PATH}"
|
|
41
|
+
class_option :skip_routes, type: :boolean, default: false,
|
|
42
|
+
desc: "Do not mount the engine in #{ROUTES_PATH}"
|
|
43
|
+
class_option :force, type: :boolean, default: false,
|
|
44
|
+
desc: "Overwrite #{INITIALIZER_PATH} if it already exists"
|
|
45
|
+
|
|
46
|
+
def copy_initializer
|
|
47
|
+
if options[:skip_initializer]
|
|
48
|
+
say_status("skip", "#{INITIALIZER_PATH} (--skip-initializer)", :yellow)
|
|
49
|
+
return
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
if File.exist?(File.join(destination_root, INITIALIZER_PATH)) && !options[:force]
|
|
53
|
+
say_status("identical", "#{INITIALIZER_PATH} (already exists; pass --force to overwrite)", :blue)
|
|
54
|
+
return
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
template "initializer.rb.erb", INITIALIZER_PATH, force: options[:force]
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def mount_engine
|
|
61
|
+
if options[:skip_routes]
|
|
62
|
+
say_status("skip", "#{ROUTES_PATH} (--skip-routes)", :yellow)
|
|
63
|
+
return
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
unless routes_file_exists?
|
|
67
|
+
say_status("skip", "#{ROUTES_PATH} not found; mount the engine manually", :yellow)
|
|
68
|
+
return
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
if already_mounted?
|
|
72
|
+
say_status("identical", "#{ROUTES_PATH} (engine already mounted), skipping", :blue)
|
|
73
|
+
return
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
route(routes_snippet)
|
|
77
|
+
|
|
78
|
+
return unless aggregate_route_drawn?
|
|
79
|
+
|
|
80
|
+
# `route` prepends, so a host that already draws its own `GET /health`
|
|
81
|
+
# ends up with the mount ABOVE it. That still WORKS — the engine draws
|
|
82
|
+
# no bare `/health`, so its router returns X-Cascade: pass and the
|
|
83
|
+
# outer route matches (there is a spec pinning this). But it reads
|
|
84
|
+
# backwards against the documented ordering, so say so rather than
|
|
85
|
+
# leaving them to wonder.
|
|
86
|
+
say_status(
|
|
87
|
+
"review",
|
|
88
|
+
"#{ROUTES_PATH}: the mount was prepended ABOVE your existing `GET /health`. " \
|
|
89
|
+
"It still resolves via route cascading, but move the mount below it to match " \
|
|
90
|
+
"the documented ordering.",
|
|
91
|
+
:yellow
|
|
92
|
+
)
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
no_commands do
|
|
96
|
+
def routes_file_exists?
|
|
97
|
+
File.exist?(File.join(destination_root, ROUTES_PATH))
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
def routes_source
|
|
101
|
+
File.read(File.join(destination_root, ROUTES_PATH))
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
# Match the mount however it was written — `=>` or a trailing `at:` —
|
|
105
|
+
# so a hand-mounted app isn't given a duplicate.
|
|
106
|
+
#
|
|
107
|
+
# ANCHORED PER LINE, and that matters: an unanchored match would treat
|
|
108
|
+
# a commented example (`# mount StandardHealth::Engine => "/health"`)
|
|
109
|
+
# as a live mount and skip the install, leaving the host with no health
|
|
110
|
+
# routes at all and a "already mounted, skipping" message explaining
|
|
111
|
+
# why it was fine.
|
|
112
|
+
def already_mounted?
|
|
113
|
+
routes_source.each_line.any? do |line|
|
|
114
|
+
line.match?(/\A\s*mount\s+StandardHealth::Engine/)
|
|
115
|
+
end
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
# Does the host already draw its own aggregate `GET /health`? Same
|
|
119
|
+
# per-line anchoring, for the same reason.
|
|
120
|
+
def aggregate_route_drawn?
|
|
121
|
+
routes_source.each_line.any? do |line|
|
|
122
|
+
line.match?(%r{\A\s*(?:get|match)\s+["']/health["']})
|
|
123
|
+
end
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
# `route` prepends, so this block lands at the top of the draw block.
|
|
127
|
+
# The aggregate line is commented rather than generated live: the
|
|
128
|
+
# engine ships no aggregate controller (the tier is the host's job,
|
|
129
|
+
# and its target is host-specific — often standard_circuit's), so
|
|
130
|
+
# generating a route to a controller that does not exist would break
|
|
131
|
+
# boot. The comment is the deliverable here.
|
|
132
|
+
def routes_snippet
|
|
133
|
+
<<~ROUTES
|
|
134
|
+
# StandardHealth — liveness, readiness and the env doctor.
|
|
135
|
+
#
|
|
136
|
+
# ORDERING MATTERS. The aggregate `GET /health` must be drawn
|
|
137
|
+
# BEFORE the mount below. The engine draws only sub-paths
|
|
138
|
+
# (/alive, /ready, /diagnostics/env), so an app that mounts first
|
|
139
|
+
# and relies on the engine to serve the aggregate tier silently
|
|
140
|
+
# has NO aggregate tier — no boot error, no failing route spec.
|
|
141
|
+
#
|
|
142
|
+
# Uncomment and point at your aggregate controller:
|
|
143
|
+
# get "/health", to: "health_aggregate#show"
|
|
144
|
+
mount StandardHealth::Engine => "/health", as: :standard_health
|
|
145
|
+
ROUTES
|
|
146
|
+
end
|
|
147
|
+
end
|
|
148
|
+
end
|
|
149
|
+
end
|
|
150
|
+
end
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# StandardHealth configuration.
|
|
4
|
+
#
|
|
5
|
+
# ── ROUTE ORDERING (read this before touching config/routes.rb) ─────────────
|
|
6
|
+
#
|
|
7
|
+
# The engine draws SUB-paths only: /alive, /ready, /diagnostics/env. The
|
|
8
|
+
# aggregate `GET /health` is the HOST's responsibility, and it must be drawn
|
|
9
|
+
# BEFORE the mount:
|
|
10
|
+
#
|
|
11
|
+
# get "/health", to: "health_aggregate#show" # aggregate — FIRST
|
|
12
|
+
# mount StandardHealth::Engine => "/health", as: :standard_health
|
|
13
|
+
#
|
|
14
|
+
# An app that mounts the engine first and expects it to serve the aggregate
|
|
15
|
+
# tier silently has no aggregate tier at all — no boot error, and no failing
|
|
16
|
+
# route spec unless one exists. Add a route spec for `GET /health`.
|
|
17
|
+
#
|
|
18
|
+
# Tiers: /alive = liveness (process up) · /ready = readiness (rotation gate)
|
|
19
|
+
# /health = aggregate (dashboards) · /diagnostics/env = authed doctor.
|
|
20
|
+
# ────────────────────────────────────────────────────────────────────────────
|
|
21
|
+
|
|
22
|
+
StandardHealth.configure do |config|
|
|
23
|
+
# Controller the engine's controllers inherit from. Defaults to
|
|
24
|
+
# ActionController::API so the engine works unconfigured in API-only apps.
|
|
25
|
+
# config.parent_controller = "ApplicationController"
|
|
26
|
+
|
|
27
|
+
# /diagnostics/env enumerates which env vars are missing or wrong — protect
|
|
28
|
+
# it. Setting a diagnostics-only parent lets you put auth on the doctor tier
|
|
29
|
+
# without the callback leaking onto /alive and /ready.
|
|
30
|
+
# config.diagnostics_parent_controller = "DiagnosticsBaseController"
|
|
31
|
+
|
|
32
|
+
# ── Checks ───────────────────────────────────────────────────────────────
|
|
33
|
+
#
|
|
34
|
+
# READINESS GATES ONLY ON HARD INFRA THIS APP OWNS. A failing critical check
|
|
35
|
+
# pulls the instance out of rotation, so `critical: true` belongs on the
|
|
36
|
+
# database and little else. Soft upstreams degrade; they never gate rotation.
|
|
37
|
+
config.register_check :database, StandardHealth::Checks::ActiveRecord, critical: true
|
|
38
|
+
# config.register_check :cache, StandardHealth::Checks::SolidCache
|
|
39
|
+
# config.register_check :queue, StandardHealth::Checks::SolidQueue
|
|
40
|
+
|
|
41
|
+
# Opt-in checks — not registered by default. Uncomment what applies.
|
|
42
|
+
#
|
|
43
|
+
# SolidCable: probes the `solid_cable_messages` store. Prefer this on the
|
|
44
|
+
# AGGREGATE tier; a broken cable store should degrade, not de-rotate.
|
|
45
|
+
# config.register_check :solid_cable, StandardHealth::Checks::SolidCable
|
|
46
|
+
#
|
|
47
|
+
# EnvSpecAudit: surfaces the env_spec below as a health check, failing on
|
|
48
|
+
# :missing / :forbidden / :mismatch rows. This is how config drift becomes
|
|
49
|
+
# visible without anyone polling the authed doctor endpoint. Non-critical by
|
|
50
|
+
# default on purpose — drift is a warning, not a reason to stop serving.
|
|
51
|
+
# config.register_check :env_spec, StandardHealth::Checks::EnvSpecAudit
|
|
52
|
+
|
|
53
|
+
# ── Timeouts ─────────────────────────────────────────────────────────────
|
|
54
|
+
#
|
|
55
|
+
# Both OFF by default. Read the README's Timeout.timeout caveat before
|
|
56
|
+
# enabling: it raises into the thread at an arbitrary point and can return a
|
|
57
|
+
# broken connection to the pool. A driver-level timeout is almost always the
|
|
58
|
+
# better tool.
|
|
59
|
+
# config.default_check_timeout = 2
|
|
60
|
+
# config.total_check_budget = 5
|
|
61
|
+
|
|
62
|
+
# ── Env spec ─────────────────────────────────────────────────────────────
|
|
63
|
+
#
|
|
64
|
+
# Declare what this app needs from its environment. Audited by
|
|
65
|
+
# /diagnostics/env, and by the EnvSpecAudit check above if registered.
|
|
66
|
+
config.env_spec = StandardHealth::EnvSpec.define do
|
|
67
|
+
# Reusable groupings of APP_ENVIRONMENT values, referenced as Symbols in `in:`.
|
|
68
|
+
mode_alias :deployed, %w[staging preview production]
|
|
69
|
+
mode_alias :live, %w[production]
|
|
70
|
+
|
|
71
|
+
group "Core" do
|
|
72
|
+
required :SECRET_KEY_BASE
|
|
73
|
+
required :DATABASE_URL, in: :deployed
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
group "Observability" do
|
|
77
|
+
recommended :SENTRY_DSN,
|
|
78
|
+
description: "Error tracking DSN (unique per environment)",
|
|
79
|
+
consumed_by: "config/initializers/sentry.rb"
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
# `forbidden` inverts the presence rule: absent is :ok, present is
|
|
83
|
+
# :forbidden. This is the level for dangerous toggles that are fine on
|
|
84
|
+
# staging and must never survive promotion — no custom check needed.
|
|
85
|
+
# group "Production-forbidden toggles" do
|
|
86
|
+
# forbidden :DEMO_MODE_ENABLED, in: :live,
|
|
87
|
+
# description: "Demo surfaces; unset before promoting"
|
|
88
|
+
# forbidden :BOOTSTRAP_NEEDED, in: :live,
|
|
89
|
+
# description: "First-boot admin bootstrap; unset once an admin exists"
|
|
90
|
+
# end
|
|
91
|
+
|
|
92
|
+
# `expected_value:` asserts the VALUE, not just presence. A present value
|
|
93
|
+
# that does not match reports :mismatch. Accepts a String, an Array
|
|
94
|
+
# ("any of"), or a Regexp.
|
|
95
|
+
# required :CONTENT_SECURITY_POLICY_REPORT_ONLY, in: :live,
|
|
96
|
+
# expected_value: "false",
|
|
97
|
+
# description: "Any other value leaves the CSP report-only"
|
|
98
|
+
end
|
|
99
|
+
end
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "standard_health/check"
|
|
4
|
+
require "standard_health/env_spec"
|
|
5
|
+
|
|
6
|
+
module StandardHealth
|
|
7
|
+
module Checks
|
|
8
|
+
# Surfaces EnvSpec violations as a health check.
|
|
9
|
+
#
|
|
10
|
+
# WHY THIS EXISTS
|
|
11
|
+
#
|
|
12
|
+
# The env-spec audit is otherwise only visible on `/diagnostics/env`,
|
|
13
|
+
# which is authed and polled by nobody. So hosts that wanted config drift
|
|
14
|
+
# to show up on a health tier wrote their own check to re-derive it —
|
|
15
|
+
# fundbright-web carried two (`ForbiddenToggles`, `CspEnforcement`) built
|
|
16
|
+
# from hand-maintained constant lists that had to be kept in sync with the
|
|
17
|
+
# env spec by comment. This check reads the spec directly, so there is one
|
|
18
|
+
# declaration and no list to drift.
|
|
19
|
+
#
|
|
20
|
+
# NOT REGISTERED AUTOMATICALLY. See `Checks::SolidCable` for the reasoning
|
|
21
|
+
# — briefly: auto-registering would fail existing hosts on a `bundle
|
|
22
|
+
# update` the moment their spec had any pre-existing violation.
|
|
23
|
+
#
|
|
24
|
+
# Register it yourself:
|
|
25
|
+
#
|
|
26
|
+
# c.register_check :env_spec, StandardHealth::Checks::EnvSpecAudit
|
|
27
|
+
#
|
|
28
|
+
# NON-CRITICAL BY DEFAULT, and that default is load-bearing. Config drift
|
|
29
|
+
# is *visibility*, not a rotation signal: an instance with a stale toggle
|
|
30
|
+
# set is still serving traffic correctly, and pulling it out of rotation
|
|
31
|
+
# would convert a warning into an outage. It rolls the aggregate up to
|
|
32
|
+
# `:degraded` (HTTP 200). Registering it `critical: true` means "this app
|
|
33
|
+
# must not serve at all with a bad env", which is a real but rare posture
|
|
34
|
+
# — and on the readiness tier it will pull instances. Know which you want.
|
|
35
|
+
#
|
|
36
|
+
# To narrow what counts as a failure, or to make it critical, subclass —
|
|
37
|
+
# the aggregator instantiates checks with `name:`/`critical:` only, so
|
|
38
|
+
# per-registration options are not available:
|
|
39
|
+
#
|
|
40
|
+
# class StrictEnvSpec < StandardHealth::Checks::EnvSpecAudit
|
|
41
|
+
# def initialize(name: :env_spec, critical: false)
|
|
42
|
+
# super(name: name, critical: critical, fail_on: %i[forbidden])
|
|
43
|
+
# end
|
|
44
|
+
# end
|
|
45
|
+
class EnvSpecAudit < Check
|
|
46
|
+
# Reported as `error_class` so the redacted body carries a groupable
|
|
47
|
+
# `error_code` (`standard_health_env_spec_violation`) instead of a
|
|
48
|
+
# meaningless "StandardError". Not an exception class — nothing here
|
|
49
|
+
# raises — just a stable label.
|
|
50
|
+
VIOLATION_LABEL = "StandardHealth::EnvSpecViolation"
|
|
51
|
+
|
|
52
|
+
def initialize(name: :env_spec, critical: false, fail_on: EnvSpec::VIOLATION_STATUSES)
|
|
53
|
+
super(name: name, critical: critical)
|
|
54
|
+
@fail_on = Array(fail_on).map(&:to_sym)
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
# @return [Array<Symbol>] audit statuses this check treats as failures
|
|
58
|
+
attr_reader :fail_on
|
|
59
|
+
|
|
60
|
+
def run
|
|
61
|
+
violations = nil
|
|
62
|
+
|
|
63
|
+
# `with_timing` is what keeps the never-raise invariant: `audit` runs
|
|
64
|
+
# host-supplied `if:`/`unless:` procs and can raise `UnknownModeAlias`,
|
|
65
|
+
# and neither may reach the aggregator as an exception.
|
|
66
|
+
timing = with_timing { violations = violating_rows }
|
|
67
|
+
|
|
68
|
+
return timing unless timing[:status] == :ok
|
|
69
|
+
return timing if violations.empty?
|
|
70
|
+
|
|
71
|
+
timing.merge(
|
|
72
|
+
status: :fail,
|
|
73
|
+
error: describe(violations),
|
|
74
|
+
error_class: VIOLATION_LABEL
|
|
75
|
+
)
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
private
|
|
79
|
+
|
|
80
|
+
def violating_rows
|
|
81
|
+
spec = ::StandardHealth.config.env_spec
|
|
82
|
+
return [] unless spec
|
|
83
|
+
|
|
84
|
+
# No `root:`. Resolving `consumed_by` paths does file IO per entry,
|
|
85
|
+
# which is fine for an on-demand doctor endpoint and not fine on a
|
|
86
|
+
# health tier that gets polled every few seconds. Consumer presence is
|
|
87
|
+
# a code-review question anyway, not a runtime health signal.
|
|
88
|
+
spec.audit(ENV.to_h, mode: ENV["APP_ENVIRONMENT"].to_s)
|
|
89
|
+
.select { |row| @fail_on.include?(row[:status]) }
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
# Names only, grouped by status. Env var NAMES are not secrets and are
|
|
93
|
+
# the whole point of the message; VALUES never appear. This message is
|
|
94
|
+
# redacted out of an unauthenticated /ready body anyway and reaches
|
|
95
|
+
# operators via the instrumentation events.
|
|
96
|
+
def describe(violations)
|
|
97
|
+
violations
|
|
98
|
+
.group_by { |row| row[:status] }
|
|
99
|
+
.sort_by { |status, _| status.to_s }
|
|
100
|
+
.map { |status, rows| "#{status}: #{rows.map { |r| r[:name] }.sort.join(', ')}" }
|
|
101
|
+
.join("; ")
|
|
102
|
+
end
|
|
103
|
+
end
|
|
104
|
+
end
|
|
105
|
+
end
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "standard_health/check"
|
|
4
|
+
|
|
5
|
+
module StandardHealth
|
|
6
|
+
module Checks
|
|
7
|
+
# Verifies the SolidCable / ActionCable backing store is reachable.
|
|
8
|
+
#
|
|
9
|
+
# SolidCable persists messages in `solid_cable_messages`. The probe is a
|
|
10
|
+
# bounded read against that table, which confirms both that the cable
|
|
11
|
+
# schema is migrated and that the connection it lives on is up.
|
|
12
|
+
#
|
|
13
|
+
# Promoted from sidekick-web's `SolidCableCheck`, which was one of four
|
|
14
|
+
# host-local copies of a check the gem should have shipped.
|
|
15
|
+
#
|
|
16
|
+
# NOT REGISTERED AUTOMATICALLY. Registering it would be a behaviour change
|
|
17
|
+
# on `bundle update`: every host without SolidCable installed — or with the
|
|
18
|
+
# table not yet migrated — would start reporting a failing check, turning
|
|
19
|
+
# the aggregate tier yellow across the estate for a check nobody asked for.
|
|
20
|
+
# New checks in this gem are opt-in for that reason.
|
|
21
|
+
#
|
|
22
|
+
# c.register_check :solid_cable, StandardHealth::Checks::SolidCable
|
|
23
|
+
#
|
|
24
|
+
# NON-CRITICAL BY DEFAULT, deliberately: cable is a degradable feature
|
|
25
|
+
# dependency. A broken cable store should mark the app `degraded`, never
|
|
26
|
+
# pull an instance out of rotation — so prefer this on the AGGREGATE tier
|
|
27
|
+
# rather than in readiness.
|
|
28
|
+
class SolidCable < Check
|
|
29
|
+
def initialize(name: :solid_cable, critical: false)
|
|
30
|
+
super
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def run
|
|
34
|
+
with_timing do
|
|
35
|
+
connection.select_value("SELECT 1 FROM solid_cable_messages LIMIT 1")
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
private
|
|
40
|
+
|
|
41
|
+
# SolidCable may or may not be pointed at a separate database
|
|
42
|
+
# (`SolidCable::Record.connects_to`). When it isn't configured, the table
|
|
43
|
+
# lives on the primary connection, so fall back to it rather than failing
|
|
44
|
+
# a host that simply didn't split the database out.
|
|
45
|
+
def connection
|
|
46
|
+
if defined?(::SolidCable::Record) && ::SolidCable::Record.respond_to?(:connection)
|
|
47
|
+
::SolidCable::Record.connection
|
|
48
|
+
else
|
|
49
|
+
::ActiveRecord::Base.connection
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
end
|
|
54
|
+
end
|
|
@@ -93,8 +93,9 @@ module StandardHealth
|
|
|
93
93
|
#
|
|
94
94
|
# The machinery ships now so apps can opt in per check and so the events
|
|
95
95
|
# emitted in this release can tell us what the real p99 latencies are.
|
|
96
|
-
# Sensible defaults get chosen from that data in
|
|
97
|
-
#
|
|
96
|
+
# Sensible defaults get chosen from that data in a later release, once
|
|
97
|
+
# enough p99 data has accumulated to pick them from evidence rather than
|
|
98
|
+
# guesswork — deliberately NOT 0.5.0, which shipped without it.
|
|
98
99
|
attr_accessor :default_check_timeout
|
|
99
100
|
|
|
100
101
|
# Budget across all checks, evaluated BEFORE each check starts. Checks not
|
|
@@ -12,6 +12,8 @@ module StandardHealth
|
|
|
12
12
|
# required :SECRET_KEY_BASE
|
|
13
13
|
# required :APP_ENVIRONMENT, in: %w[staging production]
|
|
14
14
|
# recommended :SENTRY_DSN, description: "Error tracking DSN"
|
|
15
|
+
# forbidden :DEMO_MODE_ENABLED, in: :live
|
|
16
|
+
# required :CSP_REPORT_ONLY, in: :live, expected_value: "false"
|
|
15
17
|
#
|
|
16
18
|
# group "Singpass / MyInfo" do
|
|
17
19
|
# required :MYINFO_CLIENT_ID
|
|
@@ -23,7 +25,7 @@ module StandardHealth
|
|
|
23
25
|
#
|
|
24
26
|
# Each entry has:
|
|
25
27
|
# - `name` (Symbol)
|
|
26
|
-
# - `level` (:required | :recommended)
|
|
28
|
+
# - `level` (:required | :recommended | :forbidden)
|
|
27
29
|
# - `in:` (Array<String>, Symbol, or nil) — when an Array, the entry only
|
|
28
30
|
# applies while `APP_ENVIRONMENT` matches one of these modes. A Symbol
|
|
29
31
|
# is resolved against `mode_alias` declarations at audit time. When
|
|
@@ -47,11 +49,28 @@ module StandardHealth
|
|
|
47
49
|
# Surfaced verbatim in audit rows.
|
|
48
50
|
# - `replacement:` (String, optional) — what to use instead. Surfaced
|
|
49
51
|
# in audit rows.
|
|
52
|
+
# - `expected_value:` (String/Symbol/Numeric/Regexp/Array, optional) —
|
|
53
|
+
# asserts the *value*, not just presence. A present value that does
|
|
54
|
+
# not match reports `status: :mismatch`. An Array means "any of".
|
|
55
|
+
# Surfaced in audit rows as `expected_value:` so an operator can see
|
|
56
|
+
# what was demanded; the actual value is deliberately NEVER surfaced
|
|
57
|
+
# (env values are frequently secrets).
|
|
50
58
|
class EnvSpec
|
|
51
59
|
# Raised when `in:` references a Symbol that hasn't been declared via
|
|
52
60
|
# `mode_alias`.
|
|
53
61
|
class UnknownModeAlias < ArgumentError; end
|
|
54
62
|
|
|
63
|
+
# Levels whose absence is fine and whose *presence* is the failure.
|
|
64
|
+
FORBIDDEN_LEVEL = :forbidden
|
|
65
|
+
|
|
66
|
+
# Statuses that mean the environment is wrong, as opposed to merely
|
|
67
|
+
# advisory (`:should_set`) or inapplicable (`:not_applicable`).
|
|
68
|
+
#
|
|
69
|
+
# Shared by `DiagnosticsController#audit_status` and the opt-in
|
|
70
|
+
# `Checks::EnvSpecAudit`, so the doctor endpoint and the health check
|
|
71
|
+
# cannot drift apart on what counts as a violation.
|
|
72
|
+
VIOLATION_STATUSES = %i[missing forbidden mismatch].freeze
|
|
73
|
+
|
|
55
74
|
# Internal entry record. `modes` may be an Array<String> or a Symbol
|
|
56
75
|
# alias that gets resolved at audit time.
|
|
57
76
|
Entry = Struct.new(
|
|
@@ -66,6 +85,7 @@ module StandardHealth
|
|
|
66
85
|
:deprecated,
|
|
67
86
|
:sunset_on,
|
|
68
87
|
:replacement,
|
|
88
|
+
:expected_value,
|
|
69
89
|
keyword_init: true
|
|
70
90
|
)
|
|
71
91
|
|
|
@@ -128,6 +148,31 @@ module StandardHealth
|
|
|
128
148
|
add(:recommended, name, **opts)
|
|
129
149
|
end
|
|
130
150
|
|
|
151
|
+
# Declare an env var that must NOT be set (in the applicable modes).
|
|
152
|
+
# Inverts the presence rule: absent is `:ok`, present is `:forbidden`.
|
|
153
|
+
#
|
|
154
|
+
# This is the level for dangerous ops toggles — demo modes, auth
|
|
155
|
+
# bypasses, bootstrap flags — that are legitimate on staging and must
|
|
156
|
+
# never survive to production:
|
|
157
|
+
#
|
|
158
|
+
# forbidden :DEMO_MODE_ENABLED, in: :live,
|
|
159
|
+
# description: "Demo surfaces; unset before promoting"
|
|
160
|
+
#
|
|
161
|
+
# `expected_value:` is meaningless here (the assertion is "absent"), so
|
|
162
|
+
# combining them is a declaration error rather than a silently ignored
|
|
163
|
+
# option. Raised at `define` time — i.e. host boot, well off the health
|
|
164
|
+
# path — so it can never surface as a 500 from /ready.
|
|
165
|
+
def forbidden(name, **opts)
|
|
166
|
+
if opts.key?(:expected_value)
|
|
167
|
+
raise ArgumentError,
|
|
168
|
+
"`forbidden` asserts absence, so expected_value: is meaningless " \
|
|
169
|
+
"(got #{opts[:expected_value].inspect} for #{name}). Use " \
|
|
170
|
+
"`required ..., expected_value:` to assert a value instead."
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
add(FORBIDDEN_LEVEL, name, **opts)
|
|
174
|
+
end
|
|
175
|
+
|
|
131
176
|
# Run the audit against an env-like hash.
|
|
132
177
|
#
|
|
133
178
|
# @param env_hash [Hash{String, Symbol => String}] e.g. ENV.to_h
|
|
@@ -143,8 +188,8 @@ module StandardHealth
|
|
|
143
188
|
# least `name`, `level`, `status`, `mode`. When an entry is suppressed
|
|
144
189
|
# by an `if:`/`unless:` predicate, `status` is `:not_applicable` and a
|
|
145
190
|
# `reason` field explains why. `description`, `consumed_by`, `group`,
|
|
146
|
-
# `deprecated`, `sunset_on`, `replacement`,
|
|
147
|
-
# included when set on the entry / computed during audit.
|
|
191
|
+
# `deprecated`, `sunset_on`, `replacement`, `expected_value`, and
|
|
192
|
+
# `consumer` are included when set on the entry / computed during audit.
|
|
148
193
|
def audit(env_hash, mode:, root: nil)
|
|
149
194
|
mode_str = mode.to_s
|
|
150
195
|
env = stringify(env_hash)
|
|
@@ -185,7 +230,8 @@ module StandardHealth
|
|
|
185
230
|
group: @group_stack.last,
|
|
186
231
|
deprecated: opts[:deprecated] ? true : nil,
|
|
187
232
|
sunset_on: opts[:sunset_on],
|
|
188
|
-
replacement: opts[:replacement]
|
|
233
|
+
replacement: opts[:replacement],
|
|
234
|
+
expected_value: opts[:expected_value]
|
|
189
235
|
)
|
|
190
236
|
end
|
|
191
237
|
|
|
@@ -241,9 +287,28 @@ module StandardHealth
|
|
|
241
287
|
row[:deprecated] = true if entry.deprecated
|
|
242
288
|
row[:sunset_on] = entry.sunset_on.to_s if entry.sunset_on
|
|
243
289
|
row[:replacement] = entry.replacement if entry.replacement
|
|
290
|
+
# The DECLARED expectation is safe to surface — it is config the host
|
|
291
|
+
# wrote, not process state. The ACTUAL value is never surfaced: env
|
|
292
|
+
# values are routinely secrets, and /diagnostics/env exists to report
|
|
293
|
+
# that something is wrong, not to echo it back.
|
|
294
|
+
row[:expected_value] = serialize_expected(entry.expected_value) unless entry.expected_value.nil?
|
|
244
295
|
row
|
|
245
296
|
end
|
|
246
297
|
|
|
298
|
+
# Regexps have to become strings to survive JSON rendering; an Array of
|
|
299
|
+
# candidates stays an Array so a caller can see every accepted value.
|
|
300
|
+
def serialize_expected(expected)
|
|
301
|
+
if expected.is_a?(Array)
|
|
302
|
+
expected.map { |candidate| stringify_expected(candidate) }
|
|
303
|
+
else
|
|
304
|
+
stringify_expected(expected)
|
|
305
|
+
end
|
|
306
|
+
end
|
|
307
|
+
|
|
308
|
+
def stringify_expected(candidate)
|
|
309
|
+
candidate.is_a?(Regexp) ? candidate.inspect : candidate.to_s
|
|
310
|
+
end
|
|
311
|
+
|
|
247
312
|
# Resolve the consumer-presence state for an entry. Returns nil when
|
|
248
313
|
# we have nothing to check (no root, no consumed_by). Otherwise:
|
|
249
314
|
#
|
|
@@ -283,11 +348,48 @@ module StandardHealth
|
|
|
283
348
|
value.length == 1 ? value.first : value
|
|
284
349
|
end
|
|
285
350
|
|
|
351
|
+
# Resolve one entry against the value found in the env.
|
|
352
|
+
#
|
|
353
|
+
# :forbidden — a `forbidden` entry is set
|
|
354
|
+
# :missing — a `required` entry is absent
|
|
355
|
+
# :should_set — a `recommended` entry is absent
|
|
356
|
+
# :mismatch — present, but `expected_value:` says otherwise
|
|
357
|
+
# :ok — everything else
|
|
358
|
+
#
|
|
359
|
+
# Presence is checked first for every level, because "absent" is
|
|
360
|
+
# unambiguous: it satisfies `forbidden`, violates `required`, and there
|
|
361
|
+
# is nothing for `expected_value:` to compare against.
|
|
286
362
|
def classify(entry, value)
|
|
287
363
|
present = !value.nil? && !value.to_s.empty?
|
|
288
|
-
return :ok if present
|
|
289
364
|
|
|
290
|
-
entry.level ==
|
|
365
|
+
if entry.level == FORBIDDEN_LEVEL
|
|
366
|
+
return present ? :forbidden : :ok
|
|
367
|
+
end
|
|
368
|
+
|
|
369
|
+
unless present
|
|
370
|
+
return entry.level == :required ? :missing : :should_set
|
|
371
|
+
end
|
|
372
|
+
|
|
373
|
+
return :mismatch unless expected_value_satisfied?(entry, value)
|
|
374
|
+
|
|
375
|
+
:ok
|
|
376
|
+
end
|
|
377
|
+
|
|
378
|
+
# Compare a present value against `expected_value:`. Comparison is on
|
|
379
|
+
# the string form, since everything arriving from ENV is a String —
|
|
380
|
+
# `expected_value: 5` matching `"5"` is the intuitive reading, and the
|
|
381
|
+
# alternative (never matching) would be a silent trap.
|
|
382
|
+
#
|
|
383
|
+
# An Array means "any of". A Regexp is matched, not compared, which is
|
|
384
|
+
# what makes value ranges and prefixes expressible.
|
|
385
|
+
def expected_value_satisfied?(entry, value)
|
|
386
|
+
expected = entry.expected_value
|
|
387
|
+
return true if expected.nil?
|
|
388
|
+
|
|
389
|
+
actual = value.to_s
|
|
390
|
+
Array(expected).any? do |candidate|
|
|
391
|
+
candidate.is_a?(Regexp) ? candidate.match?(actual) : candidate.to_s == actual
|
|
392
|
+
end
|
|
291
393
|
end
|
|
292
394
|
|
|
293
395
|
def stringify(env_hash)
|
data/lib/standard_health.rb
CHANGED
|
@@ -8,6 +8,10 @@ require "standard_health/check"
|
|
|
8
8
|
require "standard_health/checks/active_record"
|
|
9
9
|
require "standard_health/checks/solid_queue"
|
|
10
10
|
require "standard_health/checks/solid_cache"
|
|
11
|
+
# Opt-in checks — required so the constants resolve, NOT registered. See each
|
|
12
|
+
# class for why auto-registration would break existing hosts on bundle update.
|
|
13
|
+
require "standard_health/checks/solid_cable"
|
|
14
|
+
require "standard_health/checks/env_spec_audit"
|
|
11
15
|
require "standard_health/event_emitter"
|
|
12
16
|
require "standard_health/notifiers/logger"
|
|
13
17
|
require "standard_health/notifiers/sentry"
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: standard_health
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.5.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Jaryl Sim
|
|
@@ -13,14 +13,14 @@ dependencies:
|
|
|
13
13
|
name: rails
|
|
14
14
|
requirement: !ruby/object:Gem::Requirement
|
|
15
15
|
requirements:
|
|
16
|
-
- - "
|
|
16
|
+
- - ">="
|
|
17
17
|
- !ruby/object:Gem::Version
|
|
18
18
|
version: '8.0'
|
|
19
19
|
type: :runtime
|
|
20
20
|
prerelease: false
|
|
21
21
|
version_requirements: !ruby/object:Gem::Requirement
|
|
22
22
|
requirements:
|
|
23
|
-
- - "
|
|
23
|
+
- - ">="
|
|
24
24
|
- !ruby/object:Gem::Version
|
|
25
25
|
version: '8.0'
|
|
26
26
|
description: StandardHealth is a mountable Rails engine providing /alive, /ready,
|
|
@@ -41,10 +41,14 @@ files:
|
|
|
41
41
|
- app/controllers/standard_health/diagnostics_controller.rb
|
|
42
42
|
- app/controllers/standard_health/health_controller.rb
|
|
43
43
|
- config/routes.rb
|
|
44
|
+
- lib/generators/standard_health/install/install_generator.rb
|
|
45
|
+
- lib/generators/standard_health/install/templates/initializer.rb.erb
|
|
44
46
|
- lib/standard_health.rb
|
|
45
47
|
- lib/standard_health/aggregator.rb
|
|
46
48
|
- lib/standard_health/check.rb
|
|
47
49
|
- lib/standard_health/checks/active_record.rb
|
|
50
|
+
- lib/standard_health/checks/env_spec_audit.rb
|
|
51
|
+
- lib/standard_health/checks/solid_cable.rb
|
|
48
52
|
- lib/standard_health/checks/solid_cache.rb
|
|
49
53
|
- lib/standard_health/checks/solid_queue.rb
|
|
50
54
|
- lib/standard_health/configuration.rb
|