foam-otel 1.2.1 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/GOTCHAS.md +44 -28
- data/README.md +114 -22
- data/RESEARCH.md +62 -36
- data/lib/foam/otel/api.rb +21 -19
- data/lib/foam/otel/config.rb +4 -2
- data/lib/foam/otel/constants.rb +103 -10
- data/lib/foam/otel/pipelines.rb +89 -12
- data/lib/foam/otel/redacting_exporter.rb +19 -16
- data/lib/foam/otel/redaction.rb +176 -55
- data/lib/foam/otel/version.rb +14 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 82f1791c9e7d9929a655760c4e2f67295bb70f55dee75a2738bfcae33f5658f6
|
|
4
|
+
data.tar.gz: 16917b6e7a4e1dc1f0c6160f7026d4fe90c6213a16fa0664807e459359c5ea92
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 38df8e89604a42d3db4698f5f12934f2bf3e25e288124caf10694a0e698e50f3a732fdd9f1dd2455fb52af90777cba00a3e825f23b967817d6fbda25b567fbdf
|
|
7
|
+
data.tar.gz: 5034a35f4d3a7c9d1f18fb0e7adc6691e75aacf0130eb74312ce14ad0ab902e787a25b75af45d6f618cd70a13df12b8323fc172d1cb0abee2950991bb839db6d
|
data/GOTCHAS.md
CHANGED
|
@@ -240,34 +240,48 @@ anthropic 1.59.0, gemini-ai 4.3.0, ruby_llm 1.16.0).
|
|
|
240
240
|
`@attributes = validated_attributes(@attributes).freeze` then sets `@ended`
|
|
241
241
|
(opentelemetry-sdk `trace/span.rb:277,280`); `to_span_data` returns the frozen
|
|
242
242
|
`@attributes` by reference (`trace/span.rb:296-307`).
|
|
243
|
-
- **
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
`
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
243
|
+
- **Redaction posture (holistic redesign 2026-07-26, amended same day by the
|
|
244
|
+
credential-floor ruling)**: foam does NO customer redaction by default — with
|
|
245
|
+
no `redact_keys`/`redact_pii_keys` every value is captured RAW (no
|
|
246
|
+
value-pattern masking) EXCEPT the ALWAYS-ON CREDENTIAL FLOOR: the frozen
|
|
247
|
+
header/key name list of `contract/credential-denylist.json` masks to
|
|
248
|
+
`[REDACTED]` unconditionally, with no off switch (README "The default
|
|
249
|
+
credential denylist"; design doc `docs/decisions/credential-denylist-design.md`).
|
|
250
|
+
The floor + any configured keys run at the EXPORTER boundary — a
|
|
251
|
+
`RedactingSpanExporter`/`RedactingLogRecordExporter` that rebuilds the
|
|
252
|
+
mutable `SpanData`/`LogRecordData` Structs with masked
|
|
253
|
+
attributes/events/body before serialization. The pass runs unconditionally
|
|
254
|
+
(the floor has no off state); values under unmatched names ride through raw.
|
|
255
|
+
- **Divergence (CLOSED in 1.3.0)**: foam's HELPERS apply the floor + key pass
|
|
256
|
+
AT CAPTURE (`api.rb` `stringify`/`set_attribute`/`log`; `metrics.rb` always
|
|
257
|
+
did), so a tenant `additional_*` instance receives foam-helper data
|
|
253
258
|
ALREADY-MASKED (rule 18 C.3) — the pass is idempotent, so the wire value is
|
|
254
|
-
unchanged.
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
`
|
|
259
|
-
`
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
259
|
+
unchanged. THIRD-PARTY INSTRUMENTATION attributes freeze at finish before
|
|
260
|
+
any processor runs (no safe pre-freeze hook — `on_finishing` runs inside
|
|
261
|
+
the span mutex, so a processor calling `set_attribute` there would
|
|
262
|
+
deadlock), so they CANNOT be masked in place — instead the guarded tenant
|
|
263
|
+
wrappers (`pipelines.rb` `GuardedSpanProcessor#on_finish` /
|
|
264
|
+
`GuardedLogRecordProcessor#on_emit`) hand tenants a READ-ONLY MASKED VIEW
|
|
265
|
+
(`MaskedSpanView`/`MaskedLogRecordView`, served through the same
|
|
266
|
+
`Redaction.mask_span_data`/`mask_attributes`/`mask_body` pass foam's export
|
|
267
|
+
applies) — a tenant never sees a raw floor (or configured-key) value, and a
|
|
268
|
+
span whose masked view cannot be built is WITHHELD loudly, never handed
|
|
269
|
+
over raw. Foam's own export applies the same pass, and the exporters FAIL
|
|
270
|
+
CLOSED once the pass has begun: a struct whose rebuild raises is DROPPED
|
|
271
|
+
with a loud warning (`Redaction.mask_*` return nil on hard failure), never
|
|
272
|
+
exported half-masked.
|
|
273
|
+
- **Mitigation**: capture-time pass in `lib/foam/otel/api.rb` + the masked
|
|
274
|
+
tenant view in `lib/foam/otel/pipelines.rb` +
|
|
264
275
|
`lib/foam/otel/redacting_exporter.rb` + `lib/foam/otel/redaction.rb`.
|
|
265
|
-
- **Test**: `spec/
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
`spec/
|
|
269
|
-
|
|
270
|
-
|
|
276
|
+
- **Test**: `spec/credential_floor_spec.rb` + `spec/credential_floor_wire_spec.rb`
|
|
277
|
+
(the floor per signal/door; the tenant masked-view regression gate — a
|
|
278
|
+
third-party-set `authorization` attribute never reaches a tenant raw),
|
|
279
|
+
`spec/tenant_seam_spec.rb` (tenant processors receive helper-set LISTED keys
|
|
280
|
+
ALREADY-MASKED, free-text bodies RAW, the byte-identical with/without-tenant
|
|
281
|
+
proofs for spans, logs, AND metrics), `spec/redacting_exporter_spec.rb`
|
|
282
|
+
(masking on the wire; "a POISONED SpanData (dup raises) is DROPPED — never
|
|
283
|
+
exported raw"), `spec/redaction_spec.rb` (raw-above-the-floor defaults + the
|
|
284
|
+
two opt-in key lists; capture+exporter idempotence).
|
|
271
285
|
|
|
272
286
|
## F2: OTEL_SDK_DISABLED and OTEL_PROPAGATORS are only honored by SDK.configure
|
|
273
287
|
|
|
@@ -623,8 +637,10 @@ anthropic 1.59.0, gemini-ai 4.3.0, ruby_llm 1.16.0).
|
|
|
623
637
|
mitigation (foam never rewrites or drops metric dimensions — rule 4/26);
|
|
624
638
|
the mitigation is the README metrics warning with the classic bad example —
|
|
625
639
|
*Documented* (README "Metrics" section). *Test*: `spec/metrics_spec.rb`
|
|
626
|
-
("captures a metric attribute RAW by default; masks it only when
|
|
627
|
-
redact_keys" — pins that unlisted attributes pass
|
|
640
|
+
("captures a NON-floor metric attribute RAW by default; masks it only when
|
|
641
|
+
the key is in redact_keys" — pins that unlisted, non-floor attributes pass
|
|
642
|
+
through VERBATIM; the credential floor's frozen names are the one always-on
|
|
643
|
+
exception, `spec/credential_floor_wire_spec.rb`).
|
|
628
644
|
- **G8 — never-ended spans leak memory**: a span started and never ended holds
|
|
629
645
|
memory and context forever (and never exports — the batch processor only
|
|
630
646
|
sees FINISHED spans, `on_finish`). `Foam::Otel.span {}` always ends the span
|
data/README.md
CHANGED
|
@@ -4,9 +4,12 @@ Foam's OpenTelemetry core for Ruby services. A thin, safe wrapper over the
|
|
|
4
4
|
official OpenTelemetry libraries: foam owns the pipeline (providers, batch
|
|
5
5
|
processors, OTLP export to the foam fleet endpoint), turns on automatic
|
|
6
6
|
tier-1/2 instrumentation, and hands you a small set of never-throw helpers —
|
|
7
|
-
and nothing else. **Redaction is fully opt-in
|
|
8
|
-
value RAW** (no
|
|
9
|
-
|
|
7
|
+
and nothing else. **Redaction is fully opt-in above one always-on exception:
|
|
8
|
+
by default foam captures every value RAW** (no value-pattern masking, no PII
|
|
9
|
+
preset) **except the credential floor** — a fixed list of credential/secret
|
|
10
|
+
header and field NAMES that is always masked to `[REDACTED]` (see "The
|
|
11
|
+
default credential denylist" below); you enumerate any further fields to mask
|
|
12
|
+
via `redact_keys`/`redact_pii_keys`.
|
|
10
13
|
|
|
11
14
|
Built to `docs/BASE_PACKAGE_SPEC.md`. This README is the manual you integrate
|
|
12
15
|
from.
|
|
@@ -75,10 +78,12 @@ drop-in upstream metrics gem, so foam ships its own collector),
|
|
|
75
78
|
**session stitching** (inbound `baggage: session.id` from the foam browser
|
|
76
79
|
package is stamped onto every span and log record), and the **LLM shims**
|
|
77
80
|
(below). Every inbound request, DB call, HTTP call, job, log line, and model
|
|
78
|
-
call now flows to foam. **Redaction is opt-in
|
|
79
|
-
attributes, URLs/query strings, DB statement text, log
|
|
80
|
-
is captured RAW
|
|
81
|
-
|
|
81
|
+
call now flows to foam. **Redaction is opt-in above the credential floor: by
|
|
82
|
+
default every value — attributes, URLs/query strings, DB statement text, log
|
|
83
|
+
bodies, LLM content — is captured RAW, except values whose NAME is on the
|
|
84
|
+
always-on credential denylist (masked `[REDACTED]`; see "The default
|
|
85
|
+
credential denylist" below).** Pass `redact_keys`/`redact_pii_keys` to mask
|
|
86
|
+
or erase further specific fields (see below).
|
|
82
87
|
|
|
83
88
|
> **Sidekiq trace shape** (GOTCHAS G16): the bundled instrumentation keeps
|
|
84
89
|
> its upstream default `propagation_style: :link` — a performed job runs as
|
|
@@ -134,8 +139,8 @@ API):** the allowlist must include BOTH headers —
|
|
|
134
139
|
| `enabled:` | Boolean | yes | — | The config switch. `false` = fully inert (no SDK, no providers, no network, helpers no-op). `true` = export, in EVERY environment. No default — you write the logic. |
|
|
135
140
|
| `token:` | String | when enabled | — | `Authorization: Bearer` for export. Required (and validated) only when `enabled: true`; wired explicitly from your secret source — never an env fallback read by the gem. **Fleet convention: read it from the `FOAM_OTEL_TOKEN` env var.** Blank while enabled → raises at boot. |
|
|
136
141
|
| `version:` | String | no | nil | `service.version`, verbatim (git SHA recommended). Missing → warns and continues. Never detected at runtime. |
|
|
137
|
-
| `redact_keys:` | Array<String> | no | `[]` | The ONLY field names tail-masked (shape-preserving, e.g. `********f456`). Opt-in — empty (the default) means
|
|
138
|
-
| `redact_pii_keys:` | Array<String> | no | `[]` | Field names fully erased to `[REDACTED]` (no tail). Opt-in — empty (the default) means
|
|
142
|
+
| `redact_keys:` | Array<String> | no | `[]` | The ONLY field names tail-masked (shape-preserving, e.g. `********f456`). Opt-in and ADDITIVE on top of the always-on credential floor — empty (the default) means no masking beyond the floor. Matches by case-insensitive substring, including keys nested in structured values and query-string keys. A key that is also on the floor stays fully `[REDACTED]` (the floor wins; it is never downgraded to a tail). |
|
|
143
|
+
| `redact_pii_keys:` | Array<String> | no | `[]` | Field names fully erased to `[REDACTED]` (no tail). Opt-in and additive above the floor — empty (the default) means no erasure beyond the floor. Foam ships no PII preset; this is your enumeration. |
|
|
139
144
|
| `additional_span_processors:` | Array | no | `[]` | Tenant seam: constructed SpanProcessor instances added to foam's pipeline (additive; never replace foam's export). See coexistence. |
|
|
140
145
|
| `additional_log_record_processors:` | Array | no | `[]` | Tenant seam, logs. |
|
|
141
146
|
| `additional_metric_readers:` | Array | no | `[]` | Tenant seam, metrics. |
|
|
@@ -193,6 +198,86 @@ override is the operator-level `OTEL_EXPORTER_OTLP_ENDPOINT` env var (below).
|
|
|
193
198
|
|
|
194
199
|
---
|
|
195
200
|
|
|
201
|
+
## The default credential denylist (the always-on floor)
|
|
202
|
+
|
|
203
|
+
**Fleet ruling 2026-07-26** (binding design:
|
|
204
|
+
`docs/decisions/credential-denylist-design.md`; frozen fixture:
|
|
205
|
+
`contract/credential-denylist.json`). This is the ONE exception to foam's
|
|
206
|
+
raw-by-default capture, and it has **no off switch** — no init option, no env
|
|
207
|
+
var, and no door-2 parameter can disable, shrink, or re-spell it.
|
|
208
|
+
|
|
209
|
+
**What is masked.** The VALUE of every header/attribute/field whose NAME is on
|
|
210
|
+
the frozen list, in every signal (spans — events and links included — logs,
|
|
211
|
+
metrics), on door 1, on all three door-2 ingest taps, and in the view handed
|
|
212
|
+
to tenant `additional_*` processors:
|
|
213
|
+
|
|
214
|
+
- **(a) the seven credential headers**, matched as header names wherever
|
|
215
|
+
headers are captured — including the semconv
|
|
216
|
+
`http.request.header.<name>` / `http.response.header.<name>` span-attribute
|
|
217
|
+
forms: `authorization`, `proxy-authorization`, `cookie`, `set-cookie`,
|
|
218
|
+
`x-api-key`, `x-auth-token`, `www-authenticate`;
|
|
219
|
+
- **(b) the frozen 52-entry key list** of
|
|
220
|
+
`contract/credential-denylist.json` (the deduplicated union of those seven,
|
|
221
|
+
sentry-python's default denylists, and foam's documented reference roots —
|
|
222
|
+
`password`, `token`, `secret`, `api_key`, `ssn`, `jwt`, `private_key`,
|
|
223
|
+
`connect.sid`, `phpsessid`, …), matched as attribute/field names at every
|
|
224
|
+
depth the engine walks, and per-key in URL query strings
|
|
225
|
+
(`?token=x&user=bob` → `token=[REDACTED]&user=bob`).
|
|
226
|
+
|
|
227
|
+
**The matching rule.** Case-insensitive, dash/underscore-normalized **EXACT
|
|
228
|
+
name-equality — never substring**: `authorization` masks the header/field
|
|
229
|
+
`authorization` (any casing, `-`≡`_`) and nothing else — `authorization_url`,
|
|
230
|
+
`x_api_key_id`, `secretary`, and `session_count` all ride RAW. Dotted entries
|
|
231
|
+
match verbatim and whole (`connect.sid` masks; `user.session` does not). This
|
|
232
|
+
is deliberately STRICTER matching than your own `redact_keys` lists, which
|
|
233
|
+
keep their documented substring behavior.
|
|
234
|
+
|
|
235
|
+
**The mask.** The full literal `[REDACTED]` — no tail, no length
|
|
236
|
+
preservation. These are credentials, not debug aids. Semconv header
|
|
237
|
+
attributes (string arrays) mask per element, preserving arity.
|
|
238
|
+
|
|
239
|
+
**Precedence.** The floor runs unconditionally BEFORE your
|
|
240
|
+
`redact_keys`/`redact_pii_keys`, which stay purely additive on top. Listing a
|
|
241
|
+
floor name in `redact_keys` does NOT downgrade it to a tail mask — the floor
|
|
242
|
+
is terminal. `enabled: false` stays fully inert exactly as before (nothing
|
|
243
|
+
exports at all — the floor masks data that leaves; inert mode has none).
|
|
244
|
+
|
|
245
|
+
**Everything else stays RAW.** The floor matches NAMES only — it never scans
|
|
246
|
+
values: bodies, query values, SQL text, LLM content, and PII under unlisted
|
|
247
|
+
names export byte-identical raw, exactly as documented everywhere else in
|
|
248
|
+
this README.
|
|
249
|
+
|
|
250
|
+
**Why.** Coverage-over-masking remains the mission, but raw `Authorization`
|
|
251
|
+
headers, cookies, and API keys on the wire are a breach in waiting for every
|
|
252
|
+
customer at once — the fleet ruling carves out exactly this list (and only
|
|
253
|
+
this list) as non-negotiable. The list is byte-identical in every foam core
|
|
254
|
+
and gate-checked in CI against the fleet fixture
|
|
255
|
+
(`spec/credential_floor_spec.rb`).
|
|
256
|
+
|
|
257
|
+
**Fleet param-name canon** (the customer options above the floor, identical
|
|
258
|
+
concept in every core — the floor itself has NO init surface anywhere):
|
|
259
|
+
|
|
260
|
+
| Concept | js/otel | js/browser | python | ruby | java |
|
|
261
|
+
|---|---|---|---|---|---|
|
|
262
|
+
| secret keys (tail-mask) | `redactKeys` | `redactKeys` | `redact_keys` | `redact_keys:` | `.redactKeys(String…)` |
|
|
263
|
+
| PII keys (full `[REDACTED]`) | `redactPiiKeys` | `redactPiiKeys` | `redact_pii_keys` | `redact_pii_keys:` | `.redactPiiKeys(String…)` |
|
|
264
|
+
|
|
265
|
+
> **Migration note — 1.3.0 (minor): always-on credential masking.** As of
|
|
266
|
+
> this version foam masks, by default and in every signal, the VALUES of a
|
|
267
|
+
> fixed list of credential/secret header and field NAMES (`authorization`,
|
|
268
|
+
> `cookie`, `set-cookie`, `proxy-authorization`, `x-api-key`, `x-auth-token`,
|
|
269
|
+
> `www-authenticate`, and the 52-name key list in
|
|
270
|
+
> `contract/credential-denylist.json` — Sentry-parity plus foam's documented
|
|
271
|
+
> roots) to the literal `[REDACTED]`. Matching is exact name-equality,
|
|
272
|
+
> case-insensitive, dash/underscore-insensitive — never substring:
|
|
273
|
+
> `authorization_url` is untouched. Everything else still exports RAW exactly
|
|
274
|
+
> as before; `redact_keys`/`redact_pii_keys` are unchanged and additive.
|
|
275
|
+
> There is no off switch — if a dashboard keyed off a raw credential value
|
|
276
|
+
> (it should not have), it will now see `[REDACTED]`. Customers who
|
|
277
|
+
> previously received these header/field values raw stop receiving them.
|
|
278
|
+
|
|
279
|
+
---
|
|
280
|
+
|
|
196
281
|
## The helpers
|
|
197
282
|
|
|
198
283
|
All helpers never raise, and no-op silently before `init` and when disabled.
|
|
@@ -288,10 +373,13 @@ Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FO
|
|
|
288
373
|
> Ruby note: when you configure `redact_keys`/`redact_pii_keys`, foam's HELPERS
|
|
289
374
|
> apply the key pass at capture, so a tenant processor receives foam-helper data
|
|
290
375
|
> (span/log helper attributes, log bodies, metric labels) ALREADY masked.
|
|
291
|
-
>
|
|
292
|
-
> processor
|
|
293
|
-
>
|
|
294
|
-
>
|
|
376
|
+
> The Ruby SDK freezes span attributes at finish, so historically a tenant
|
|
377
|
+
> processor saw third-party instrumentation attributes UNMASKED — as of 1.3.0
|
|
378
|
+
> that gap is closed: tenant span/log processors are handed a READ-ONLY MASKED
|
|
379
|
+
> VIEW (the credential floor plus your configured keys, the same pass foam's
|
|
380
|
+
> own export applies — GOTCHAS F1). A tenant never sees a raw floor value.
|
|
381
|
+
> With no keys configured (the default) only the credential floor is masked
|
|
382
|
+
> on either path.
|
|
295
383
|
>
|
|
296
384
|
> Loop-guard note: `ignored_outbound_hosts` suppresses spans for tenant/agent
|
|
297
385
|
> egress made through **Net::HTTP or Excon**. A tenant exporter using
|
|
@@ -329,9 +417,10 @@ end
|
|
|
329
417
|
|
|
330
418
|
**Outcome**: their pipeline keeps working exactly as before — their data,
|
|
331
419
|
their resource, their export, byte-identical — and foam ALSO receives the
|
|
332
|
-
signal, carrying THEIR resource identity plus foam's export-time stamp
|
|
333
|
-
|
|
334
|
-
|
|
420
|
+
signal, carrying THEIR resource identity plus foam's export-time stamp, the
|
|
421
|
+
always-on credential floor applied to foam's copy, and any per-tap
|
|
422
|
+
`redact_keys`/`redact_pii_keys` you set (opt-in — none by default; details in
|
|
423
|
+
the Door 2 section).
|
|
335
424
|
|
|
336
425
|
**Unhappy paths**:
|
|
337
426
|
|
|
@@ -431,8 +520,10 @@ readers only: they never mutate their data, their resource, or their export.
|
|
|
431
520
|
### The three entries
|
|
432
521
|
|
|
433
522
|
All three share one signature (`token:` and `environment:` required; the
|
|
434
|
-
redaction kwargs are the
|
|
435
|
-
scoped to that tap
|
|
523
|
+
redaction kwargs are the only CUSTOMER keys that tap redacts — opt-in, none by
|
|
524
|
+
default, scoped to that tap, additive above the always-on credential floor,
|
|
525
|
+
which every tap applies with zero configuration; `diagnostics:` is tap-scoped
|
|
526
|
+
narration). Construction
|
|
436
527
|
validates loudly at boot and NEVER
|
|
437
528
|
throws afterwards — a failure inside a tap can never break their pipeline.
|
|
438
529
|
Fleet canon mapping (pinned by `spec/export_surface_spec.rb`):
|
|
@@ -453,9 +544,10 @@ their_tracer_provider.add_span_processor(processor)
|
|
|
453
544
|
# raises ArgumentError at construction for a blank token/environment
|
|
454
545
|
```
|
|
455
546
|
|
|
456
|
-
**Logs** — a LogRecordProcessor for THEIR LoggerProvider. The
|
|
457
|
-
`redact_keys`/`redact_pii_keys` (if any) match over
|
|
458
|
-
nested in structured bodies; free-text bodies ride
|
|
547
|
+
**Logs** — a LogRecordProcessor for THEIR LoggerProvider. The credential
|
|
548
|
+
floor and the tap's `redact_keys`/`redact_pii_keys` (if any) match over
|
|
549
|
+
record attributes and keys nested in structured bodies; free-text bodies ride
|
|
550
|
+
RAW (no value-pattern pass):
|
|
459
551
|
|
|
460
552
|
```ruby
|
|
461
553
|
provider = OpenTelemetry::SDK::Logs::LoggerProvider.new(resource: their_resource)
|
|
@@ -499,7 +591,7 @@ in THEIR pipeline. Three layers keep it closed:
|
|
|
499
591
|
| | their pipeline | foam's copy |
|
|
500
592
|
| --- | --- | --- |
|
|
501
593
|
| `service.name` / `service.version` / `service.instance.id` | theirs | theirs, untouched (the data carries THEIR identity) |
|
|
502
|
-
| all attributes / events / bodies | raw, byte-identical | your per-tap `redact_keys`/`redact_pii_keys` applied (default:
|
|
594
|
+
| all attributes / events / bodies | raw, byte-identical | the credential floor + your per-tap `redact_keys`/`redact_pii_keys` applied (default: floor only — everything off the frozen list identical to theirs) |
|
|
503
595
|
| `deployment.environment.name` | theirs (if any) | the tap's `environment:` argument (stamped at export time) |
|
|
504
596
|
| `foam.ingest.tier` | absent | `"external"` (the wire-contract marker) |
|
|
505
597
|
| `telemetry.distro.name` / `telemetry.distro.version` | absent | `"foam"` / the gem version (support reads the tap's presence and version from telemetry alone) |
|
data/RESEARCH.md
CHANGED
|
@@ -144,8 +144,11 @@ statement posture is **captured RAW, never suppressed** (2026-07-26 ruling —
|
|
|
144
144
|
coverage is the mission), and the upstream `-pg`/`-mysql2`/`-redis`/`-mongo`
|
|
145
145
|
gems all DEFAULT to `db_statement: :obfuscate` — a sanitizer — so foam
|
|
146
146
|
explicitly activates them with `db_statement: :include` (GOTCHAS F9; pinned
|
|
147
|
-
by `spec/floor_spec.rb`). Redaction stays fully opt-in
|
|
148
|
-
|
|
147
|
+
by `spec/floor_spec.rb`). Redaction stays fully opt-in above the always-on
|
|
148
|
+
credential floor (the frozen NAME list of
|
|
149
|
+
`contract/credential-denylist.json` — statement TEXT is never scanned),
|
|
150
|
+
applied only for keys the customer lists in
|
|
151
|
+
`redact_keys`/`redact_pii_keys`. An FDE can still clamp
|
|
149
152
|
`db_statement: :omit`/`:obfuscate` per customer via the standard
|
|
150
153
|
`OTEL_RUBY_INSTRUMENTATION_<NAME>_CONFIG_OPTS` env var when a compliance
|
|
151
154
|
context calls for it — per-customer tightening (rule 18 / Area 4), not the
|
|
@@ -210,7 +213,8 @@ foam ships the logs pipeline + `log()` helper (trace-correlated) — the mechani
|
|
|
210
213
|
rule 20 requires. The stdlib **Logger bridge is LANDED as foam's own code**
|
|
211
214
|
(`lib/foam/otel/logger_bridge.rb`, console-logs ruling 2026-07-26): one
|
|
212
215
|
prepend on `Logger#add` turns every message a host logger accepts into a
|
|
213
|
-
trace-correlated OTel log record — body RAW (
|
|
216
|
+
trace-correlated OTel log record — body RAW (name-keyed masking only: the
|
|
217
|
+
always-on credential floor plus any customer keys; free text never scanned),
|
|
214
218
|
severity mapped, loop-guarded (the OTel SDK's internal logger is skipped by
|
|
215
219
|
identity + a thread-local re-entrancy flag), gated on foam owning the logs
|
|
216
220
|
slot, and never able to break the host's log call (GOTCHAS F10; tests
|
|
@@ -226,7 +230,8 @@ The official contrib `opentelemetry-instrumentation-logger 0.4.0` bridges the
|
|
|
226
230
|
same `Logger#add` seam and is deliberately NOT bundled — census disposition
|
|
227
231
|
EXCLUDED/superseded (§3): foam's bridge additionally holds the lazy-block
|
|
228
232
|
contract, the OTel-internal-logger identity guard, the re-entrancy flag, the
|
|
229
|
-
logs-slot gate, and the opt-in
|
|
233
|
+
logs-slot gate, and the redaction path (credential floor + opt-in keys), and
|
|
234
|
+
shipping/sweeping both
|
|
230
235
|
would double-export every host log line (rule 12). init's sweep pins the
|
|
231
236
|
contrib gem disabled even when an FDE bundles it (`spec/floor_spec.rb`).
|
|
232
237
|
|
|
@@ -289,38 +294,59 @@ https://opentelemetry.io/docs/specs/semconv/gen-ai/aws-bedrock/ ; traceloop Ruby
|
|
|
289
294
|
|
|
290
295
|
## 6. Redaction design record (rule 14, as amended 2026-07-26)
|
|
291
296
|
|
|
292
|
-
**Redaction is FULLY OPT-IN
|
|
293
|
-
2026-07-26, holistic-redesign decision log
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
`
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
297
|
+
**Redaction is FULLY OPT-IN above the CREDENTIAL FLOOR** (owner ruling
|
|
298
|
+
2026-07-26, holistic-redesign decision log, AMENDED the same day by the
|
|
299
|
+
credential-floor fleet ruling — `docs/decisions/credential-denylist-design.md`;
|
|
300
|
+
`lib/foam/otel/redaction.rb`). With neither `redact_keys` nor
|
|
301
|
+
`redact_pii_keys` set (or both empty) every value is captured RAW —
|
|
302
|
+
attributes, bodies, URL query params, `db.statement`, log bodies — including
|
|
303
|
+
PII, EXCEPT the one always-on exception: the CREDENTIAL FLOOR masks the
|
|
304
|
+
values of the frozen credential/secret header + key NAME list
|
|
305
|
+
(`contract/credential-denylist.json`: the seven headers + 52 keys, constants
|
|
306
|
+
`CREDENTIAL_HEADER_DENYLIST`/`CREDENTIAL_KEY_DENYLIST` in `constants.rb`) to
|
|
307
|
+
the literal `[REDACTED]`, in every signal, on every door, with NO off switch.
|
|
308
|
+
There is still NO value-pattern auto-masking and NO PII preset; above the
|
|
309
|
+
floor the customer owns the privacy/legal posture; the FDE reactive clamp is
|
|
310
|
+
the per-customer valve.
|
|
311
|
+
|
|
312
|
+
**The retired floor vs the credential floor**: earlier revisions shipped an
|
|
313
|
+
always-on researched secrets floor in `constants.rb` (roots + value-pattern
|
|
314
|
+
pass, sourced from the OSS-scrubber consensus — raven-ruby, sentry, Elastic
|
|
315
|
+
APM, Rails filter_parameters, gitleaks et al. — modeled on the Node core). It
|
|
316
|
+
was REMOVED by the 2026-07-26 zero-default ruling; the provenance record
|
|
317
|
+
lives in this file's git history and the decision log. The SAME-DAY
|
|
318
|
+
credential-floor ruling is the new fleet ruling that reintroduces exactly ONE
|
|
319
|
+
always-on list — the frozen credential denylist (name-match only, never
|
|
320
|
+
value patterns; rule-14 reference roots absorbed as entries). Its content
|
|
321
|
+
changes only by a new fleet ruling and is CI-gated bit-for-bit against the
|
|
322
|
+
fleet fixture (`spec/credential_floor_spec.rb`).
|
|
323
|
+
|
|
324
|
+
**The current engine** (`redaction.rb`): the floor predicate
|
|
325
|
+
(`floor_kind`/`floor_match?`) runs FIRST and is TERMINAL — canon-exact
|
|
326
|
+
matching (trim, ASCII-lowercase, `-`→`_` fold; NEVER substring:
|
|
327
|
+
`authorization` never masks `authorization_url`), including the semconv
|
|
328
|
+
`http.{request,response}.header.<name>` prefix peel. Customer redaction then
|
|
329
|
+
runs ONLY over the keys the customer enumerates. `redact_keys` matches → tail
|
|
330
|
+
mask, shape-preserving (`********` + last 4 when ≥ 12 chars; the fixed 8-char
|
|
331
|
+
body never reveals length; non-scalars redact in full); `redact_pii_keys`
|
|
332
|
+
matches → full `[REDACTED]`, no tail. CUSTOMER matching is SUBSTRING,
|
|
333
|
+
case-insensitive, over the lowercased key — the customer's lists, nothing
|
|
334
|
+
inferred (the FLOOR deliberately does NOT ride the substring matcher — it
|
|
335
|
+
would over-match). The pass runs at the exporter boundary
|
|
336
|
+
(`redacting_exporter.rb` — the Ruby SDK freezes span attributes at finish
|
|
337
|
+
before any `on_finish` processor runs, so the exporter boundary is the seam;
|
|
338
|
+
see GOTCHAS) and runs UNCONDITIONALLY (the floor has no off state); values
|
|
339
|
+
under unmatched names ride through raw, and tenant `additional_*` processors
|
|
340
|
+
receive the masked VIEW (`pipelines.rb`, GOTCHAS F1). The explicit
|
|
341
|
+
`redact(value)` helper always masks (calling it is the opt-in). Fail-closed
|
|
342
|
+
for floor + listed keys (mask on any error, cycle + depth guarded).
|
|
343
|
+
|
|
344
|
+
Proven by `spec/credential_floor_spec.rb` + `spec/credential_floor_wire_spec.rb`
|
|
345
|
+
(the floor: list identity gate, matcher, every signal, doors 1+2, tenant
|
|
346
|
+
view, negative controls), `spec/redaction_spec.rb` (raw-above-the-floor
|
|
347
|
+
defaults — nothing else masked with no keys; both mask shapes; fail-closed)
|
|
348
|
+
and `spec/redacting_exporter_spec.rb` + the conformance app (default-path RAW
|
|
349
|
+
capture on the wire, floor names excepted).
|
|
324
350
|
|
|
325
351
|
---
|
|
326
352
|
|
data/lib/foam/otel/api.rb
CHANGED
|
@@ -105,13 +105,14 @@ module Foam
|
|
|
105
105
|
span = OpenTelemetry::Trace.current_span
|
|
106
106
|
return false unless span.respond_to?(:recording?) && span.recording?
|
|
107
107
|
|
|
108
|
-
# Capture-time
|
|
109
|
-
# redact_keys/redact_pii_keys the value lands
|
|
110
|
-
# processors and inert-mode providers only ever
|
|
111
|
-
# helper-set data. The exporter pass
|
|
112
|
-
#
|
|
113
|
-
# scrub_utf8 is encoding hygiene, not masking: an invalid-UTF-8
|
|
114
|
-
# would make the upstream OTLP encoder drop the WHOLE batch
|
|
108
|
+
# Capture-time pass (rule 18 C.3): the always-on credential floor plus
|
|
109
|
+
# any configured redact_keys/redact_pii_keys, so the value lands
|
|
110
|
+
# ALREADY-MASKED — tenant processors and inert-mode providers only ever
|
|
111
|
+
# see the redacted view of helper-set data. The exporter pass
|
|
112
|
+
# re-applies (idempotent). Values under unmatched names are captured
|
|
113
|
+
# RAW. scrub_utf8 is encoding hygiene, not masking: an invalid-UTF-8
|
|
114
|
+
# value would make the upstream OTLP encoder drop the WHOLE batch
|
|
115
|
+
# (rule 15).
|
|
115
116
|
span.set_attribute(key.to_s, Redaction.mask_one(key.to_s, Redaction.scrub_utf8(value), active_config))
|
|
116
117
|
true
|
|
117
118
|
rescue StandardError, SystemStackError
|
|
@@ -260,14 +261,14 @@ module Foam
|
|
|
260
261
|
LEVEL_TEXT[severity.to_s.downcase.to_sym]
|
|
261
262
|
end
|
|
262
263
|
|
|
263
|
-
# Stringify keys, then apply the customer's key
|
|
264
|
-
# C.3):
|
|
265
|
-
#
|
|
266
|
-
#
|
|
267
|
-
# redacted view of foam-helper data. The exporter-boundary pass
|
|
268
|
-
# — masking is idempotent.
|
|
269
|
-
#
|
|
270
|
-
#
|
|
264
|
+
# Stringify keys, then apply the credential floor + the customer's key
|
|
265
|
+
# lists AT CAPTURE (rule 18 C.3): helper-set attributes are
|
|
266
|
+
# ALREADY-MASKED when they reach the span/log record, so a tenant
|
|
267
|
+
# processor (or a foreign provider in inert mode) only ever sees the
|
|
268
|
+
# redacted view of foam-helper data. The exporter-boundary pass
|
|
269
|
+
# re-applies — masking is idempotent. Values under unmatched names ride
|
|
270
|
+
# RAW. NEVER throws (rule 9): non-enumerable garbage degrades to nil
|
|
271
|
+
# (attributes dropped, telemetry still flows).
|
|
271
272
|
def stringify(attributes)
|
|
272
273
|
return nil if attributes.nil?
|
|
273
274
|
|
|
@@ -279,10 +280,11 @@ module Foam
|
|
|
279
280
|
nil
|
|
280
281
|
end
|
|
281
282
|
|
|
282
|
-
# The customer key pass over a log BODY at capture (same
|
|
283
|
-
# stringify).
|
|
284
|
-
#
|
|
285
|
-
#
|
|
283
|
+
# The floor + customer key pass over a log BODY at capture (same
|
|
284
|
+
# rationale as stringify). mask_body is fail-closed and depth-guarded,
|
|
285
|
+
# so a cyclic structure is bounded before it can reach any encoder; a
|
|
286
|
+
# free-text body (and every structured field under an unmatched name)
|
|
287
|
+
# is captured RAW.
|
|
286
288
|
def mask_log_body(body)
|
|
287
289
|
return body if body.nil?
|
|
288
290
|
|
data/lib/foam/otel/config.rb
CHANGED
|
@@ -42,8 +42,10 @@ module Foam
|
|
|
42
42
|
environment: environment,
|
|
43
43
|
version: version,
|
|
44
44
|
enabled: enabled,
|
|
45
|
-
# Opt-in redaction (holistic redesign)
|
|
46
|
-
#
|
|
45
|
+
# Opt-in CUSTOMER redaction (holistic redesign), additive above the
|
|
46
|
+
# always-on credential floor (which is a module constant, NOT config
|
|
47
|
+
# state — no config shape can alter it). Empty (the default) → no
|
|
48
|
+
# customer redaction, raw capture above the floor. Stored lowercased
|
|
47
49
|
# for the engine's case-insensitive substring match.
|
|
48
50
|
redact_keys: downcase_list(redact_keys),
|
|
49
51
|
redact_pii_keys: downcase_list(redact_pii_keys),
|
data/lib/foam/otel/constants.rb
CHANGED
|
@@ -13,24 +13,117 @@ module Foam
|
|
|
13
13
|
DISTRO_NAME = "foam"
|
|
14
14
|
|
|
15
15
|
# Deep-redaction depth guard (rule 14). Beyond this, a value redacts in
|
|
16
|
-
# full rather than recursing further.
|
|
17
|
-
|
|
16
|
+
# full rather than recursing further. Pinned at 10 fleet-wide (contract
|
|
17
|
+
# SPEC §4; credential-denylist-design §4 cross-core table) — every core
|
|
18
|
+
# walks to the same depth so the floor's coverage is byte-identical.
|
|
19
|
+
MAX_REDACT_DEPTH = 10
|
|
18
20
|
|
|
19
21
|
# deployment.environment.name values that do NOT warn (section 0: the
|
|
20
22
|
# value is exported verbatim regardless; anything outside this set warns
|
|
21
23
|
# but is never rewritten).
|
|
22
24
|
KNOWN_ENVIRONMENTS = %w[production staging development test].freeze
|
|
23
25
|
|
|
24
|
-
# ---- Redaction is FULLY OPT-IN
|
|
25
|
-
#
|
|
26
|
-
#
|
|
27
|
-
#
|
|
28
|
-
#
|
|
29
|
-
#
|
|
26
|
+
# ---- Redaction is FULLY OPT-IN above the CREDENTIAL FLOOR ---------------
|
|
27
|
+
# (holistic redesign 2026-07-26, amended the same day by the
|
|
28
|
+
# credential-floor fleet ruling.) The package does NO redaction by default
|
|
29
|
+
# — with neither redact_keys nor redact_pii_keys set every value is
|
|
30
|
+
# captured RAW, including PII / PHI — EXCEPT the one always-on exception:
|
|
31
|
+
# the CREDENTIAL FLOOR below, which masks the values of the frozen
|
|
32
|
+
# credential/secret NAMES to the literal [REDACTED] in every signal, on
|
|
33
|
+
# every export path, with NO off switch. Everything not on the frozen list
|
|
34
|
+
# still exports RAW (no value-pattern auto-masking, no PII preset).
|
|
35
|
+
# Customer redaction happens ONLY for the exact keys the customer
|
|
36
|
+
# enumerates, ADDITIVE on top of the floor:
|
|
30
37
|
# * redact_keys → tail-mask the value (shape-preserving; see below)
|
|
31
38
|
# * redact_pii_keys → erase the value ([REDACTED])
|
|
32
|
-
# (See docs/decisions/holistic-redesign-decision-log.md
|
|
33
|
-
#
|
|
39
|
+
# (See docs/decisions/holistic-redesign-decision-log.md and
|
|
40
|
+
# docs/decisions/credential-denylist-design.md.) Above the floor, the
|
|
41
|
+
# customer owns the privacy/legal posture.
|
|
42
|
+
|
|
43
|
+
# ---- THE CREDENTIAL FLOOR (fleet ruling 2026-07-26) ---------------------
|
|
44
|
+
# Frozen fleet constants, byte-identical in every core and gate-checked
|
|
45
|
+
# bit-for-bit against contract/credential-denylist.json (order included) by
|
|
46
|
+
# spec/credential_floor_spec.rb. They change ONLY by a new fleet ruling —
|
|
47
|
+
# never ship a subset, a superset, or a re-spelling. The floor is matched
|
|
48
|
+
# by Redaction.floor_kind: case-insensitive, dash/underscore-normalized
|
|
49
|
+
# EXACT name-equality — never substring ("authorization" never masks
|
|
50
|
+
# "authorization_url"; "secret" never masks "secretary"). It is deliberately
|
|
51
|
+
# NOT part of Config: no init option, env var, or door-2 parameter can
|
|
52
|
+
# build a floor-less engine (design §4 — safety is not configurable away).
|
|
53
|
+
|
|
54
|
+
# (a) The seven credential HEADERS, stored in wire-canonical dash form —
|
|
55
|
+
# matched as header names wherever headers are captured, including the
|
|
56
|
+
# semconv http.{request,response}.header.<name> span-attribute forms.
|
|
57
|
+
CREDENTIAL_HEADER_DENYLIST = %w[
|
|
58
|
+
authorization
|
|
59
|
+
proxy-authorization
|
|
60
|
+
cookie
|
|
61
|
+
set-cookie
|
|
62
|
+
x-api-key
|
|
63
|
+
x-auth-token
|
|
64
|
+
www-authenticate
|
|
65
|
+
].freeze
|
|
66
|
+
|
|
67
|
+
# (b) The 52-entry KEY denylist, stored in canonical (lowercase,
|
|
68
|
+
# underscore) form, ASCII byte order — the deduplicated union of the seven
|
|
69
|
+
# headers, sentry-python's DEFAULT_DENYLIST + DEFAULT_PII_DENYLIST, and
|
|
70
|
+
# BASE_PACKAGE_SPEC rule 14's reference roots (design doc §1.2). Matched as
|
|
71
|
+
# attribute/field names at every depth the central engine walks. Every
|
|
72
|
+
# header above is also an entry here (subset invariant, design §1.1).
|
|
73
|
+
CREDENTIAL_KEY_DENYLIST = %w[
|
|
74
|
+
_csrf
|
|
75
|
+
_csrf_token
|
|
76
|
+
_session
|
|
77
|
+
_xsrf
|
|
78
|
+
aiohttp_session
|
|
79
|
+
api_key
|
|
80
|
+
apikey
|
|
81
|
+
auth
|
|
82
|
+
authorization
|
|
83
|
+
bearer
|
|
84
|
+
card
|
|
85
|
+
connect.sid
|
|
86
|
+
cookie
|
|
87
|
+
credential
|
|
88
|
+
credentials
|
|
89
|
+
csrf
|
|
90
|
+
csrf_token
|
|
91
|
+
csrftoken
|
|
92
|
+
cvc
|
|
93
|
+
cvv
|
|
94
|
+
ip_address
|
|
95
|
+
jwt
|
|
96
|
+
mfa
|
|
97
|
+
mysql_pwd
|
|
98
|
+
otp
|
|
99
|
+
passphrase
|
|
100
|
+
passwd
|
|
101
|
+
password
|
|
102
|
+
phpsessid
|
|
103
|
+
private_key
|
|
104
|
+
privatekey
|
|
105
|
+
proxy_authorization
|
|
106
|
+
pwd
|
|
107
|
+
remote_addr
|
|
108
|
+
secret
|
|
109
|
+
session
|
|
110
|
+
sessionid
|
|
111
|
+
set_cookie
|
|
112
|
+
sid
|
|
113
|
+
signature
|
|
114
|
+
ssn
|
|
115
|
+
symfony
|
|
116
|
+
token
|
|
117
|
+
user_session
|
|
118
|
+
www_authenticate
|
|
119
|
+
x_api_key
|
|
120
|
+
x_auth_token
|
|
121
|
+
x_csrftoken
|
|
122
|
+
x_forwarded_for
|
|
123
|
+
x_real_ip
|
|
124
|
+
xsrf
|
|
125
|
+
xsrf_token
|
|
126
|
+
].freeze
|
|
34
127
|
|
|
35
128
|
# The two mask shapes. A redact_keys match on a scalar keeps a debug tail;
|
|
36
129
|
# a redact_pii_keys match (and any non-scalar value under a redact_keys
|
data/lib/foam/otel/pipelines.rb
CHANGED
|
@@ -4,6 +4,8 @@
|
|
|
4
4
|
# export path, after it has lazily required the OTel SDK + exporters. A
|
|
5
5
|
# provider is built and registered ONLY for a signal the classifier proved
|
|
6
6
|
# FREE (BASE_PACKAGE_SPEC rule 18): foam never registers into a claimed slot.
|
|
7
|
+
require "delegate"
|
|
8
|
+
|
|
7
9
|
require_relative "redacting_exporter"
|
|
8
10
|
require_relative "session_stitching"
|
|
9
11
|
|
|
@@ -58,7 +60,7 @@ module Foam
|
|
|
58
60
|
# on_finish). Foam's batch export stays FIRST (pinned invariant).
|
|
59
61
|
provider.add_span_processor(SessionStitching::SpanProcessor.new)
|
|
60
62
|
Array(extra_processors).each do |processor|
|
|
61
|
-
provider.add_span_processor(GuardedSpanProcessor.new(processor))
|
|
63
|
+
provider.add_span_processor(GuardedSpanProcessor.new(processor, config))
|
|
62
64
|
rescue StandardError => e
|
|
63
65
|
Diagnostics.warn("additional span processor skipped: #{e.class}: #{e.message}")
|
|
64
66
|
end
|
|
@@ -96,7 +98,7 @@ module Foam
|
|
|
96
98
|
# batch export stays FIRST (pinned invariant).
|
|
97
99
|
provider.add_log_record_processor(SessionStitching::LogRecordProcessor.new)
|
|
98
100
|
Array(extra_processors).each do |processor|
|
|
99
|
-
provider.add_log_record_processor(GuardedLogRecordProcessor.new(processor))
|
|
101
|
+
provider.add_log_record_processor(GuardedLogRecordProcessor.new(processor, config))
|
|
100
102
|
rescue StandardError => e
|
|
101
103
|
Diagnostics.warn("additional log processor skipped: #{e.class}: #{e.message}")
|
|
102
104
|
end
|
|
@@ -130,14 +132,14 @@ module Foam
|
|
|
130
132
|
inner = OpenTelemetry::Exporter::OTLP::Metrics::MetricsExporter.new(
|
|
131
133
|
endpoint: "#{endpoint}/v1/metrics", headers: headers
|
|
132
134
|
)
|
|
133
|
-
# The exporter-boundary
|
|
134
|
-
#
|
|
135
|
+
# The exporter-boundary pass (credential floor + customer keys), same
|
|
136
|
+
# as traces and logs above — always on, raw for unmatched names.
|
|
135
137
|
# get_meter is public passthrough surface: instruments created through it
|
|
136
138
|
# (including observable-gauge callbacks) and contrib instrumentation
|
|
137
139
|
# emitting metrics record attributes that pass NEITHER Metrics.stringify
|
|
138
|
-
# (four foam helpers only) NOR the capture-time pass — so
|
|
139
|
-
#
|
|
140
|
-
# metrics pipeline. The wrapper forwards `reset` for after_fork, so
|
|
140
|
+
# (four foam helpers only) NOR the capture-time pass — so this wrapper
|
|
141
|
+
# is what applies the floor (and any configured keys) on foam's own
|
|
142
|
+
# door-1 metrics pipeline. The wrapper forwards `reset` for after_fork, so
|
|
141
143
|
# PeriodicMetricReader's fork recovery is unaffected (GOTCHAS R1/R7).
|
|
142
144
|
exporter = RedactingMetricsExporter.new(inner, config)
|
|
143
145
|
reader = OpenTelemetry::SDK::Metrics::Export::PeriodicMetricReader.new(exporter: exporter)
|
|
@@ -166,14 +168,85 @@ module Foam
|
|
|
166
168
|
end
|
|
167
169
|
end
|
|
168
170
|
|
|
171
|
+
# The tenant-seam MASKED VIEW (credential-floor design §5.4 / rule 18 C):
|
|
172
|
+
# the Ruby SDK freezes span attributes at finish BEFORE any on_finish
|
|
173
|
+
# processor runs (GOTCHAS F1), so foam cannot rewrite the span a tenant
|
|
174
|
+
# sees — instead the guarded wrappers below hand tenants a READ-ONLY
|
|
175
|
+
# delegator whose attributes/events/links/to_span_data are served from the
|
|
176
|
+
# SAME Redaction.mask_span_data pass foam's exporter applies (the
|
|
177
|
+
# credential floor + the customer's init-time key lists). Tenants only
|
|
178
|
+
# ever see the masked view — a raw floor value never crosses the seam,
|
|
179
|
+
# third-party instrumentation attributes included. Everything else
|
|
180
|
+
# (name, kind, context, status, resource, scope…) delegates to the real
|
|
181
|
+
# span untouched.
|
|
182
|
+
class MaskedSpanView < SimpleDelegator
|
|
183
|
+
def initialize(span, masked_span_data)
|
|
184
|
+
super(span)
|
|
185
|
+
@masked = masked_span_data
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
def attributes = @masked.attributes
|
|
189
|
+
def events = @masked.events
|
|
190
|
+
def links = @masked.links
|
|
191
|
+
def to_span_data = @masked
|
|
192
|
+
end
|
|
193
|
+
|
|
194
|
+
# The logs twin: on_emit receives the LIVE record, so the view masks
|
|
195
|
+
# attributes and body lazily through the same engine (floor + customer
|
|
196
|
+
# keys), fail-closed — a read that cannot be masked degrades to the mask,
|
|
197
|
+
# never the raw value. Reads happen AFTER foam's earlier processors
|
|
198
|
+
# (export buffer, session stitching) ran, so the tenant misses nothing.
|
|
199
|
+
class MaskedLogRecordView < SimpleDelegator
|
|
200
|
+
def initialize(record, config)
|
|
201
|
+
super(record)
|
|
202
|
+
@config = config
|
|
203
|
+
end
|
|
204
|
+
|
|
205
|
+
def attributes
|
|
206
|
+
@attributes ||= begin
|
|
207
|
+
Foam::Otel::Redaction.mask_attributes(__getobj__.attributes, @config)
|
|
208
|
+
rescue StandardError, SystemStackError
|
|
209
|
+
{} # fail closed: never the raw attributes
|
|
210
|
+
end
|
|
211
|
+
end
|
|
212
|
+
|
|
213
|
+
def body
|
|
214
|
+
@body ||= begin
|
|
215
|
+
Foam::Otel::Redaction.mask_body(__getobj__.body, @config)
|
|
216
|
+
rescue StandardError, SystemStackError
|
|
217
|
+
Foam::Otel::REDACTED # fail closed: never the raw body
|
|
218
|
+
end
|
|
219
|
+
end
|
|
220
|
+
end
|
|
221
|
+
|
|
169
222
|
# A tenant span/log processor supplied via additional_* — wrapped so a
|
|
170
223
|
# throwing tenant instance is skipped with a loud warning and breaks
|
|
171
224
|
# neither the app nor foam's export (rule 18 C / rule 9). Never-throw is
|
|
172
|
-
# foam's guarantee for foam-invoked calls; the wrapper enforces it.
|
|
225
|
+
# foam's guarantee for foam-invoked calls; the wrapper enforces it. The
|
|
226
|
+
# wrapper is ALSO the tenant seam's masking point: on_finish/on_emit hand
|
|
227
|
+
# the tenant the masked view above, never the raw item (credential floor,
|
|
228
|
+
# capture-time clause — tenants only ever see the masked view).
|
|
173
229
|
class GuardedSpanProcessor
|
|
174
|
-
def initialize(inner
|
|
230
|
+
def initialize(inner, config = nil)
|
|
231
|
+
@inner = inner
|
|
232
|
+
@config = config
|
|
233
|
+
end
|
|
234
|
+
|
|
175
235
|
def on_start(span, context) = guarded { @inner.on_start(span, context) }
|
|
176
|
-
|
|
236
|
+
|
|
237
|
+
def on_finish(span)
|
|
238
|
+
guarded do
|
|
239
|
+
masked = Foam::Otel::Redaction.mask_span_data(span.to_span_data, @config)
|
|
240
|
+
if masked.nil?
|
|
241
|
+
# Fail closed, loudly (rule 15): a span whose masked view cannot be
|
|
242
|
+
# built is WITHHELD from the tenant, never handed over raw.
|
|
243
|
+
Foam::Otel::Diagnostics.warn("tenant span view masking failed — span withheld from " \
|
|
244
|
+
"additional span processor (fail-closed, rule 14)")
|
|
245
|
+
else
|
|
246
|
+
@inner.on_finish(MaskedSpanView.new(span, masked))
|
|
247
|
+
end
|
|
248
|
+
end
|
|
249
|
+
end
|
|
177
250
|
def force_flush(timeout: nil) = guarded { @inner.force_flush(timeout: timeout) } || OpenTelemetry::SDK::Trace::Export::SUCCESS
|
|
178
251
|
def shutdown(timeout: nil) = guarded { @inner.shutdown(timeout: timeout) } || OpenTelemetry::SDK::Trace::Export::SUCCESS
|
|
179
252
|
|
|
@@ -188,8 +261,12 @@ module Foam
|
|
|
188
261
|
end
|
|
189
262
|
|
|
190
263
|
class GuardedLogRecordProcessor
|
|
191
|
-
def initialize(inner
|
|
192
|
-
|
|
264
|
+
def initialize(inner, config = nil)
|
|
265
|
+
@inner = inner
|
|
266
|
+
@config = config
|
|
267
|
+
end
|
|
268
|
+
|
|
269
|
+
def on_emit(log_record, context) = guarded { @inner.on_emit(MaskedLogRecordView.new(log_record, @config), context) }
|
|
193
270
|
def force_flush(timeout: nil) = guarded { @inner.force_flush(timeout: timeout) } || OpenTelemetry::SDK::Logs::Export::SUCCESS
|
|
194
271
|
def shutdown(timeout: nil) = guarded { @inner.shutdown(timeout: timeout) } || OpenTelemetry::SDK::Logs::Export::SUCCESS
|
|
195
272
|
|
|
@@ -5,16 +5,19 @@ require_relative "diagnostics"
|
|
|
5
5
|
|
|
6
6
|
module Foam
|
|
7
7
|
module Otel
|
|
8
|
-
# The
|
|
9
|
-
#
|
|
10
|
-
#
|
|
11
|
-
#
|
|
12
|
-
#
|
|
13
|
-
#
|
|
14
|
-
#
|
|
15
|
-
#
|
|
16
|
-
#
|
|
17
|
-
#
|
|
8
|
+
# The central enforcement point: the ALWAYS-ON CREDENTIAL FLOOR
|
|
9
|
+
# (constants.rb / redaction.rb — fleet ruling 2026-07-26) plus the
|
|
10
|
+
# customer's opt-in key pass (redact_keys / redact_pii_keys) run HERE, at
|
|
11
|
+
# the exporter boundary, on door 1 AND door 2. Why not a SpanProcessor:
|
|
12
|
+
# the Ruby SDK freezes a span's attributes at finish, BEFORE any on_finish
|
|
13
|
+
# processor runs (opentelemetry-sdk span.rb — GOTCHAS entry), so no
|
|
14
|
+
# processor can mask them. SpanData / LogRecordData / Event are mutable
|
|
15
|
+
# Structs the exporter can rebuild, so masking happens on the copy foam
|
|
16
|
+
# serializes. The pass runs UNCONDITIONALLY — the floor has no off switch;
|
|
17
|
+
# values under unmatched names ship RAW. (Foam's own HELPERS additionally
|
|
18
|
+
# apply the same pass at capture — api.rb stringify / set_attribute — and
|
|
19
|
+
# the tenant seam hands additional_* processors a masked VIEW —
|
|
20
|
+
# pipelines.rb — so tenants only ever see masked data.)
|
|
18
21
|
#
|
|
19
22
|
# These wrap (never subclass) the real OTLP exporter by composition, so
|
|
20
23
|
# they are robust across exporter versions — they only rely on the
|
|
@@ -27,12 +30,12 @@ module Foam
|
|
|
27
30
|
# rescued explicitly (it is not a StandardError): a poisoned payload must
|
|
28
31
|
# kill neither the batch thread nor the caller (rule 9 / rule 15).
|
|
29
32
|
#
|
|
30
|
-
#
|
|
31
|
-
# freeze at finish,
|
|
32
|
-
#
|
|
33
|
-
#
|
|
34
|
-
#
|
|
35
|
-
# attributes
|
|
33
|
+
# The F1 tenant divergence is CLOSED (1.3.0, credential-floor design §5.4):
|
|
34
|
+
# attributes still freeze at finish, so they cannot be masked IN PLACE
|
|
35
|
+
# before a tenant additional_* processor — instead the guarded tenant
|
|
36
|
+
# wrappers (pipelines.rb) hand tenants a read-only MASKED VIEW built
|
|
37
|
+
# through the same Redaction pass these exporters apply, third-party
|
|
38
|
+
# instrumentation attributes included.
|
|
36
39
|
class RedactingSpanExporter
|
|
37
40
|
def initialize(inner, config)
|
|
38
41
|
@inner = inner
|
data/lib/foam/otel/redaction.rb
CHANGED
|
@@ -1,23 +1,44 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "json"
|
|
4
|
+
require "set"
|
|
4
5
|
require "uri"
|
|
5
6
|
|
|
6
7
|
require_relative "constants"
|
|
8
|
+
require_relative "diagnostics"
|
|
7
9
|
|
|
8
10
|
module Foam
|
|
9
11
|
module Otel
|
|
10
|
-
#
|
|
11
|
-
#
|
|
12
|
-
#
|
|
13
|
-
#
|
|
14
|
-
#
|
|
12
|
+
# The central redaction engine: the always-on CREDENTIAL FLOOR plus the
|
|
13
|
+
# customer's opt-in key lists (holistic redesign 2026-07-26, amended the
|
|
14
|
+
# same day by the credential-floor fleet ruling —
|
|
15
|
+
# docs/decisions/credential-denylist-design.md).
|
|
16
|
+
#
|
|
17
|
+
# THE FLOOR (unconditional, no off switch): every attribute/field whose
|
|
18
|
+
# NAME is on the frozen CREDENTIAL_KEY_DENYLIST (constants.rb) — including
|
|
19
|
+
# the semconv http.{request,response}.header.<name> forms of the
|
|
20
|
+
# CREDENTIAL_HEADER_DENYLIST — masks in FULL to the literal [REDACTED], in
|
|
21
|
+
# every signal, at every depth this engine walks, before the customer's
|
|
22
|
+
# lists run. Matching is canon-exact (lowercase + dash→underscore fold,
|
|
23
|
+
# design §1.0/§2.1): NEVER substring — "authorization" does not mask
|
|
24
|
+
# "authorization_url"; "secret" does not mask "secretary". The floor
|
|
25
|
+
# consults the module-level frozen constant directly (never Config), so no
|
|
26
|
+
# configuration path can disable, shrink, or re-spell it.
|
|
27
|
+
#
|
|
28
|
+
# ABOVE the floor, redaction stays fully opt-in: with neither redact_keys
|
|
29
|
+
# nor redact_pii_keys set (or both empty) every OTHER value is captured RAW
|
|
30
|
+
# — no value-pattern auto-masking. Customer redaction runs ONLY over the
|
|
31
|
+
# keys the customer enumerates, ADDITIVE on top of the floor:
|
|
15
32
|
#
|
|
16
33
|
# * redact_keys — tail-mask matching scalar values (shape-preserving,
|
|
17
34
|
# e.g. ********f456); non-scalars redact in full.
|
|
18
35
|
# * redact_pii_keys — erase matching values in full ([REDACTED]).
|
|
19
36
|
#
|
|
20
|
-
#
|
|
37
|
+
# CUSTOMER matching is SUBSTRING, case-insensitive, over the lowercased key
|
|
38
|
+
# (documented Ruby behavior, unchanged). The FLOOR deliberately does NOT
|
|
39
|
+
# ride that matcher (design §2.4 — substring would over-match): it uses its
|
|
40
|
+
# own canon-exact predicate, floor_kind, below. A floor match is TERMINAL:
|
|
41
|
+
# listing a floor name in redact_keys never downgrades it to a tail mask.
|
|
21
42
|
#
|
|
22
43
|
# Two entry points:
|
|
23
44
|
# * redact(value) — the public helper: mask a value the caller
|
|
@@ -27,9 +48,12 @@ module Foam
|
|
|
27
48
|
# calling it.
|
|
28
49
|
# * mask_span_data / mask_log_record_data / mask_metric_data — the
|
|
29
50
|
# exporter-boundary pass that applies the
|
|
30
|
-
# customer's key lists, run by
|
|
31
|
-
#
|
|
32
|
-
#
|
|
51
|
+
# floor plus the customer's key lists, run by
|
|
52
|
+
# the redacting exporters
|
|
53
|
+
# (redacting_exporter.rb) on door 1 AND all
|
|
54
|
+
# three door-2 taps. Runs UNCONDITIONALLY —
|
|
55
|
+
# the floor has no off switch; values under
|
|
56
|
+
# unlisted names pass through raw.
|
|
33
57
|
#
|
|
34
58
|
# Ruby SDK constraint (see GOTCHAS.md): a Span freezes its attributes at
|
|
35
59
|
# finish (opentelemetry-sdk span.rb) BEFORE any SpanProcessor#on_finish
|
|
@@ -94,8 +118,10 @@ module Foam
|
|
|
94
118
|
"#{MASK_BODY}#{s[-4..]}"
|
|
95
119
|
end
|
|
96
120
|
|
|
97
|
-
# True when the customer enumerated at least one key to redact
|
|
98
|
-
#
|
|
121
|
+
# True when the customer enumerated at least one key to redact — the
|
|
122
|
+
# CUSTOMER lists only. The credential floor runs regardless: this
|
|
123
|
+
# predicate gates only customer-list behavior (e.g. the exemplar-drop
|
|
124
|
+
# posture below), never the floor.
|
|
99
125
|
def redaction_configured?(config)
|
|
100
126
|
return false if config.nil?
|
|
101
127
|
|
|
@@ -106,25 +132,89 @@ module Foam
|
|
|
106
132
|
false
|
|
107
133
|
end
|
|
108
134
|
|
|
109
|
-
# ---- the
|
|
110
|
-
#
|
|
111
|
-
#
|
|
112
|
-
#
|
|
135
|
+
# ---- the credential-floor predicate (design §2, self-contained) -------
|
|
136
|
+
# canon(name): trim ASCII whitespace → ASCII-lowercase → fold '-' to '_'
|
|
137
|
+
# (design §1.0). Applied to BOTH sides of every floor comparison; the
|
|
138
|
+
# stored CREDENTIAL_KEY_DENYLIST entries are already canonical.
|
|
139
|
+
def canon(name)
|
|
140
|
+
name.to_s.strip.downcase(:ascii).tr("-", "_")
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
# The two semconv header-attribute prefixes — the ONLY prefixes the floor
|
|
144
|
+
# peels (design §2.2). Canonical (lowercase, no dashes), so matching the
|
|
145
|
+
# canon'd name covers any case variant of the prefix too.
|
|
146
|
+
FLOOR_HEADER_PREFIXES = ["http.request.header.", "http.response.header."].freeze
|
|
147
|
+
|
|
148
|
+
# O(1) lookup set for the frozen key list (the array stays the shipped,
|
|
149
|
+
# fixture-gated constant; this is a derived view, same object identity of
|
|
150
|
+
# entries).
|
|
151
|
+
FLOOR_KEY_SET = CREDENTIAL_KEY_DENYLIST.to_set.freeze
|
|
152
|
+
|
|
153
|
+
# Classify a name against the FLOOR ONLY (never the customer lists):
|
|
154
|
+
# :header — a semconv header attribute (http.{request,response}
|
|
155
|
+
# .header.<name>) whose peeled suffix canon-matches the key
|
|
156
|
+
# list (which contains all seven headers — subset invariant,
|
|
157
|
+
# design §1.1; the peel checks the FULL 52-entry list, §2.2,
|
|
158
|
+
# so e.g. http.request.header.x_forwarded_for masks too);
|
|
159
|
+
# :key — the full name itself canon-matches a key-list entry
|
|
160
|
+
# (full-name equality: "user.session" does NOT match
|
|
161
|
+
# "session"; "connect.sid" matches its own verbatim entry);
|
|
162
|
+
# nil — no floor match (customer lists may still apply).
|
|
163
|
+
def floor_kind(name)
|
|
164
|
+
c = canon(name)
|
|
165
|
+
FLOOR_HEADER_PREFIXES.each do |prefix|
|
|
166
|
+
next unless c.start_with?(prefix)
|
|
167
|
+
|
|
168
|
+
return FLOOR_KEY_SET.include?(c[prefix.length..]) ? :header : nil
|
|
169
|
+
end
|
|
170
|
+
FLOOR_KEY_SET.include?(c) ? :key : nil
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
def floor_match?(name)
|
|
174
|
+
!floor_kind(name).nil?
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
# The floor mask (design §3): a semconv header attribute is a string
|
|
178
|
+
# array (one element per header instance) — mask PER ELEMENT, preserving
|
|
179
|
+
# arity and the string-array type (OTel's own sanitize_header_values
|
|
180
|
+
# shape). Everything else replaces the ENTIRE value with the single
|
|
181
|
+
# literal [REDACTED] — scalar, object, array, number, binary. Never a
|
|
182
|
+
# tail, never length-preserving: these are credentials, not debug aids.
|
|
183
|
+
def floor_mask(kind, value)
|
|
184
|
+
return value.map { REDACTED } if kind == :header && value.is_a?(Array)
|
|
185
|
+
|
|
186
|
+
REDACTED
|
|
187
|
+
end
|
|
188
|
+
|
|
189
|
+
# ---- the floor + customer key pass over exported attributes -----------
|
|
190
|
+
# Returns a NEW attributes hash with the credential floor and the
|
|
191
|
+
# customer's key lists applied. Runs UNCONDITIONALLY (the floor has no
|
|
192
|
+
# off switch); values under unmatched names pass through raw (same
|
|
193
|
+
# objects). Never raises; a key that fails to mask redacts in full
|
|
194
|
+
# rather than leaking.
|
|
113
195
|
def mask_attributes(attributes, config)
|
|
114
196
|
return attributes unless attributes.is_a?(Hash)
|
|
115
|
-
return attributes unless redaction_configured?(config)
|
|
116
197
|
|
|
117
198
|
attributes.each_with_object({}) do |(key, value), out|
|
|
118
199
|
out[key] = mask_one(key.to_s, value, config)
|
|
119
200
|
rescue StandardError
|
|
201
|
+
# Fail closed at the VALUE level, loudly (rule 15): the key name only
|
|
202
|
+
# — never the value — reaches the diagnostics channel. The warning is
|
|
203
|
+
# itself guarded: a poisoned key (#to_s raises) must still redact.
|
|
204
|
+
begin
|
|
205
|
+
Diagnostics.warn("masking failed for attribute #{key.to_s.inspect} — value redacted in full " \
|
|
206
|
+
"(fail-closed, rule 14)")
|
|
207
|
+
rescue StandardError, SystemStackError
|
|
208
|
+
Diagnostics.warn("masking failed for an attribute (unprintable key) — value redacted in full " \
|
|
209
|
+
"(fail-closed, rule 14)")
|
|
210
|
+
end
|
|
120
211
|
out[key] = REDACTED
|
|
121
212
|
end
|
|
122
213
|
end
|
|
123
214
|
|
|
124
215
|
def mask_one(key, value, config)
|
|
125
|
-
return value unless redaction_configured?(config)
|
|
126
|
-
|
|
127
216
|
case classify_key(key, config)
|
|
217
|
+
when :floor then floor_mask(floor_kind(key), value)
|
|
128
218
|
when :erase then REDACTED
|
|
129
219
|
when :mask then value.is_a?(Array) ? value.map { REDACTED } : scalar_or_full(value)
|
|
130
220
|
else
|
|
@@ -146,10 +236,10 @@ module Foam
|
|
|
146
236
|
end
|
|
147
237
|
|
|
148
238
|
# Unmatched key: the value itself is captured RAW (no value-pattern
|
|
149
|
-
# auto-masking). Structured values still DESCEND so a
|
|
150
|
-
# redact_pii_keys key NESTED under an innocent key is honored
|
|
151
|
-
# MAX_REDACT_DEPTH so an adversarial/cyclic structure can
|
|
152
|
-
# downstream encoder into SystemStackError (rule 9).
|
|
239
|
+
# auto-masking). Structured values still DESCEND so a floor-listed or
|
|
240
|
+
# redact_keys/redact_pii_keys key NESTED under an innocent key is honored
|
|
241
|
+
# — bounded to MAX_REDACT_DEPTH so an adversarial/cyclic structure can
|
|
242
|
+
# never drive a downstream encoder into SystemStackError (rule 9).
|
|
153
243
|
def descend(value, config = nil, depth = 0)
|
|
154
244
|
case value
|
|
155
245
|
when Hash, Array then deep_mask(value, config || Foam::Otel.active_config, depth)
|
|
@@ -158,9 +248,17 @@ module Foam
|
|
|
158
248
|
end
|
|
159
249
|
|
|
160
250
|
# ---- key classification ----------------------------------------------
|
|
161
|
-
#
|
|
162
|
-
#
|
|
251
|
+
# The credential floor is checked FIRST and is TERMINAL (design §4): a
|
|
252
|
+
# floor-matched name is [REDACTED] in full even when the customer also
|
|
253
|
+
# lists it in redact_keys — the tail pass must never see a floor value (a
|
|
254
|
+
# tail mask over the [REDACTED] literal would emit ******** and destroy
|
|
255
|
+
# the contract literal). Below the floor: redact_keys (:mask) wins over
|
|
256
|
+
# redact_pii_keys (:erase) when both claim a key. Customer matching is
|
|
257
|
+
# substring, case-insensitive (unchanged).
|
|
163
258
|
def classify_key(key, config)
|
|
259
|
+
return :floor if floor_match?(key)
|
|
260
|
+
return nil if config.nil?
|
|
261
|
+
|
|
164
262
|
lower = key.downcase
|
|
165
263
|
return :mask if match_any_substring?(lower, config.redact_keys)
|
|
166
264
|
return :erase if match_any_substring?(lower, config.redact_pii_keys)
|
|
@@ -176,9 +274,11 @@ module Foam
|
|
|
176
274
|
|
|
177
275
|
# ---- URL / query redaction -------------------------------------------
|
|
178
276
|
# The instrumentation-set URL attributes (rack, http clients). Redact the
|
|
179
|
-
# query PER KEY so non-secret pairs survive:
|
|
180
|
-
#
|
|
181
|
-
#
|
|
277
|
+
# query PER KEY so non-secret pairs survive: `?token=x&user=bob` exports
|
|
278
|
+
# as `token=[REDACTED]&user=bob`. A pair is touched only when its key is
|
|
279
|
+
# on the credential floor (canon-exact — the floor's `signature` entry
|
|
280
|
+
# covers OTel's own PARAMS_TO_REDACT `Signature` case-insensitively) or
|
|
281
|
+
# on one of the customer's lists.
|
|
182
282
|
URL_ATTRIBUTES = %w[http.target url.query url.full http.url uri.query].freeze
|
|
183
283
|
|
|
184
284
|
def url_attribute?(key)
|
|
@@ -214,8 +314,9 @@ module Foam
|
|
|
214
314
|
rescue StandardError
|
|
215
315
|
raw_key.downcase
|
|
216
316
|
end
|
|
217
|
-
if
|
|
218
|
-
match_any_substring?(decoded, config
|
|
317
|
+
if floor_match?(decoded) ||
|
|
318
|
+
match_any_substring?(decoded, config&.redact_keys) ||
|
|
319
|
+
match_any_substring?(decoded, config&.redact_pii_keys)
|
|
219
320
|
"#{raw_key}=#{REDACTED}"
|
|
220
321
|
else
|
|
221
322
|
pair
|
|
@@ -225,18 +326,16 @@ module Foam
|
|
|
225
326
|
""
|
|
226
327
|
end
|
|
227
328
|
|
|
228
|
-
# ---- exporter-boundary
|
|
229
|
-
# Rebuild the mutable Struct with the
|
|
230
|
-
# keeps foam's export copy independent of the
|
|
231
|
-
#
|
|
232
|
-
#
|
|
329
|
+
# ---- exporter-boundary pass (SpanData / LogRecordData) ----------------
|
|
330
|
+
# Rebuild the mutable Struct with the credential floor and the customer's
|
|
331
|
+
# key lists applied. A dup keeps foam's export copy independent of the
|
|
332
|
+
# span the tenant seam saw. Runs UNCONDITIONALLY (the floor has no off
|
|
333
|
+
# switch); values under unmatched names are carried through raw.
|
|
233
334
|
#
|
|
234
335
|
# FAIL CLOSED: on a hard failure these return NIL — never the raw payload
|
|
235
|
-
# once
|
|
336
|
+
# once the pass has begun. The redacting exporters drop a nil entry
|
|
236
337
|
# (with a loud [foam] warning) rather than export a half-masked record.
|
|
237
338
|
def mask_span_data(span_data, config)
|
|
238
|
-
return span_data unless redaction_configured?(config)
|
|
239
|
-
|
|
240
339
|
masked = span_data.dup
|
|
241
340
|
masked.attributes = mask_attributes(span_data.attributes, config) if span_data.attributes
|
|
242
341
|
# Link attributes ride the key pass too: links are reachable on BOTH
|
|
@@ -263,8 +362,6 @@ module Foam
|
|
|
263
362
|
end
|
|
264
363
|
|
|
265
364
|
def mask_log_record_data(record, config)
|
|
266
|
-
return record unless redaction_configured?(config)
|
|
267
|
-
|
|
268
365
|
masked = record.dup
|
|
269
366
|
masked.attributes = mask_attributes(record.attributes, config) if record.attributes
|
|
270
367
|
masked.body = mask_body(record.body, config) if record.body
|
|
@@ -275,27 +372,21 @@ module Foam
|
|
|
275
372
|
|
|
276
373
|
# Door 1 masks metric attributes at capture; foreign (door-2) instrument
|
|
277
374
|
# data never passes those helpers, so the ingest metric path applies the
|
|
278
|
-
# key lists here. Rebuilds every data point as a COPY (the
|
|
279
|
-
# points' attributes hash is the LIVE aggregation key upstream —
|
|
280
|
-
# in place would corrupt the customer's aggregation state).
|
|
281
|
-
#
|
|
375
|
+
# floor + key lists here. Rebuilds every data point as a COPY (the
|
|
376
|
+
# collected points' attributes hash is the LIVE aggregation key upstream —
|
|
377
|
+
# masking in place would corrupt the customer's aggregation state). Runs
|
|
378
|
+
# UNCONDITIONALLY (the floor has no off switch).
|
|
282
379
|
#
|
|
283
380
|
# FAIL CLOSED: returns NIL on a hard failure — the caller drops the metric
|
|
284
381
|
# loudly rather than exporting a half-masked point.
|
|
285
382
|
def mask_metric_data(metric_data, config)
|
|
286
|
-
return metric_data unless redaction_configured?(config)
|
|
287
|
-
|
|
288
383
|
masked = metric_data.dup
|
|
289
384
|
points = metric_data.data_points
|
|
290
385
|
if points
|
|
291
386
|
masked.data_points = points.map do |point|
|
|
292
387
|
copy = point.dup
|
|
293
388
|
copy.attributes = mask_attributes(point.attributes || {}, config)
|
|
294
|
-
|
|
295
|
-
# one field over from the point attributes the key pass just masked.
|
|
296
|
-
# Once redaction is configured, drop them: fail-closed, no
|
|
297
|
-
# observability cost for foam's own use.
|
|
298
|
-
copy.exemplars = nil if copy.respond_to?(:exemplars=)
|
|
389
|
+
mask_exemplars!(copy, config)
|
|
299
390
|
copy
|
|
300
391
|
end
|
|
301
392
|
end
|
|
@@ -304,13 +395,42 @@ module Foam
|
|
|
304
395
|
nil
|
|
305
396
|
end
|
|
306
397
|
|
|
398
|
+
# Exemplars carry filtered_attributes the OTLP exporter encodes RAW, one
|
|
399
|
+
# field over from the point attributes just masked — the floor must reach
|
|
400
|
+
# them too (design §2.3). Two postures, both fail-closed:
|
|
401
|
+
# * customer keys configured → DROP exemplars entirely (the documented
|
|
402
|
+
# pre-floor behavior for configured redaction, unchanged);
|
|
403
|
+
# * floor only (the default path) → keep the exemplars but run their
|
|
404
|
+
# filtered_attributes through the same engine pass, so floor-listed
|
|
405
|
+
# names are [REDACTED] and everything else stays raw (§9: the only
|
|
406
|
+
# default-path wire delta is floor values becoming [REDACTED]). An
|
|
407
|
+
# exemplar whose rebuild fails is dropped, never exported raw.
|
|
408
|
+
def mask_exemplars!(point, config)
|
|
409
|
+
return unless point.respond_to?(:exemplars=)
|
|
410
|
+
|
|
411
|
+
if redaction_configured?(config)
|
|
412
|
+
point.exemplars = nil
|
|
413
|
+
return
|
|
414
|
+
end
|
|
415
|
+
exemplars = point.respond_to?(:exemplars) ? point.exemplars : nil
|
|
416
|
+
return if exemplars.nil?
|
|
417
|
+
|
|
418
|
+
point.exemplars = exemplars.filter_map do |exemplar|
|
|
419
|
+
copy = exemplar.dup
|
|
420
|
+
copy.filtered_attributes = mask_attributes(exemplar.filtered_attributes || {}, config)
|
|
421
|
+
copy
|
|
422
|
+
rescue StandardError
|
|
423
|
+
nil # fail closed: a poisoned exemplar drops, never ships raw
|
|
424
|
+
end
|
|
425
|
+
rescue StandardError
|
|
426
|
+
point.exemplars = nil # fail closed at the list level too
|
|
427
|
+
end
|
|
428
|
+
|
|
307
429
|
# A log body is AnyValue: a free-text string is captured RAW (no
|
|
308
|
-
# value-pattern masking
|
|
309
|
-
#
|
|
310
|
-
#
|
|
430
|
+
# value-pattern masking — the floor matches NAMES, never scans content), a
|
|
431
|
+
# structured map/array DESCENDS so a floor-listed or customer-listed key
|
|
432
|
+
# nested inside is honored (depth-guarded). Runs unconditionally.
|
|
311
433
|
def mask_body(body, config)
|
|
312
|
-
return body unless redaction_configured?(config)
|
|
313
|
-
|
|
314
434
|
deep_mask(body, config, 0)
|
|
315
435
|
rescue StandardError
|
|
316
436
|
REDACTED
|
|
@@ -323,6 +443,7 @@ module Foam
|
|
|
323
443
|
when Hash
|
|
324
444
|
value.each_with_object({}) do |(key, val), out|
|
|
325
445
|
case classify_key(key.to_s, config)
|
|
446
|
+
when :floor then out[key] = floor_mask(floor_kind(key.to_s), val)
|
|
326
447
|
when :erase then out[key] = REDACTED
|
|
327
448
|
when :mask then out[key] = scalar_or_full(val)
|
|
328
449
|
else out[key] = deep_mask(val, config, depth + 1)
|
data/lib/foam/otel/version.rb
CHANGED
|
@@ -17,6 +17,19 @@ module Foam
|
|
|
17
17
|
# -logger pinned disabled in the sweep (foam's LoggerBridge supersedes
|
|
18
18
|
# it — rule 12 double-bridge guard). Sidekiq + Faraday floor entries
|
|
19
19
|
# wire-proven (rule 8b); Sidekiq :link posture documented (GOTCHAS G16).
|
|
20
|
-
|
|
20
|
+
# 1.3.0: ALWAYS-ON CREDENTIAL FLOOR (fleet ruling 2026-07-26; MINOR per the
|
|
21
|
+
# ruling — security-motivated wire change, no API/config change,
|
|
22
|
+
# docs/decisions/credential-denylist-design.md §9). Foam now masks, by
|
|
23
|
+
# default and in every signal, the VALUES of the fixed credential/secret
|
|
24
|
+
# header + field NAME list in contract/credential-denylist.json (seven
|
|
25
|
+
# headers + 52 keys) to the literal [REDACTED]. Exact name-equality,
|
|
26
|
+
# case-insensitive, dash/underscore-insensitive — never substring:
|
|
27
|
+
# authorization_url is untouched. Everything else still exports RAW;
|
|
28
|
+
# redact_keys/redact_pii_keys unchanged and additive; NO off switch.
|
|
29
|
+
# Tenant additional_* processors now receive a MASKED read-only view
|
|
30
|
+
# (closes the GOTCHAS F1 third-party-attribute divergence). If a
|
|
31
|
+
# dashboard keyed off a raw credential value (it should not have), it
|
|
32
|
+
# will now see [REDACTED].
|
|
33
|
+
VERSION = "1.3.0"
|
|
21
34
|
end
|
|
22
35
|
end
|