foam-otel 1.2.0 → 1.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bb41e8c47c944ac8f252f47bededb07f8c73e152411f3a7069a8fcab3d1185a2
-  data.tar.gz: ab93229a7820e4b60439d8c30f903fd3d1be93a8d2dd723b032862813ac4278f
+  metadata.gz: 82f1791c9e7d9929a655760c4e2f67295bb70f55dee75a2738bfcae33f5658f6
+  data.tar.gz: 16917b6e7a4e1dc1f0c6160f7026d4fe90c6213a16fa0664807e459359c5ea92
 SHA512:
-  metadata.gz: 291f81494a360ee3ce94a09b6b98dafde0574e4078d61ec01bb3e3e3b7803a4f7034bc36ef17a33db79cbb886ad5c0d337c2ffa75762c5a9d04f52e89cdf6669
-  data.tar.gz: e81c1c075606402562ee74a8dc6752dd4d96f4b73235303a43cd0d20569ca7238a031b3c6c6d6726f5cf3f3c32858c2452cc5cbfac8ffd61f834f549738c6c1b
+  metadata.gz: 38df8e89604a42d3db4698f5f12934f2bf3e25e288124caf10694a0e698e50f3a732fdd9f1dd2455fb52af90777cba00a3e825f23b967817d6fbda25b567fbdf
+  data.tar.gz: 5034a35f4d3a7c9d1f18fb0e7adc6691e75aacf0130eb74312ce14ad0ab902e787a25b75af45d6f618cd70a13df12b8323fc172d1cb0abee2950991bb839db6d

data/GOTCHAS.md

@@ -240,34 +240,48 @@ anthropic 1.59.0, gemini-ai 4.3.0, ruby_llm 1.16.0).
   `@attributes = validated_attributes(@attributes).freeze` then sets `@ended`
   (opentelemetry-sdk `trace/span.rb:277,280`); `to_span_data` returns the frozen
   `@attributes` by reference (`trace/span.rb:296-307`).
-- **Opt-in redaction (holistic redesign, 2026-07-26)**: foam does NO redaction by
-  default — with no `redact_keys`/`redact_pii_keys` every value is captured RAW
-  (no secret floor, no value-pattern masking). When keys ARE configured, foam
-  applies the key pass at the EXPORTER boundary — a `RedactingSpanExporter`/
-  `RedactingLogRecordExporter` that rebuilds the mutable `SpanData`/
-  `LogRecordData` Structs with masked attributes/events/body before
-  serialization. With no keys configured the exporter is a true pass-through.
-- **Divergence (NARROWED)**: when keys are configured foam's HELPERS apply the
-  key pass AT CAPTURE (`api.rb` `stringify`/`set_attribute`/`log`; `metrics.rb`
-  always did), so a tenant `additional_*` instance receives foam-helper data
+- **Redaction posture (holistic redesign 2026-07-26, amended same day by the
+  credential-floor ruling)**: foam does NO customer redaction by default — with
+  no `redact_keys`/`redact_pii_keys` every value is captured RAW (no
+  value-pattern masking) EXCEPT the ALWAYS-ON CREDENTIAL FLOOR: the frozen
+  header/key name list of `contract/credential-denylist.json` masks to
+  `[REDACTED]` unconditionally, with no off switch (README "The default
+  credential denylist"; design doc `docs/decisions/credential-denylist-design.md`).
+  The floor + any configured keys run at the EXPORTER boundary — a
+  `RedactingSpanExporter`/`RedactingLogRecordExporter` that rebuilds the
+  mutable `SpanData`/`LogRecordData` Structs with masked
+  attributes/events/body before serialization. The pass runs unconditionally
+  (the floor has no off state); values under unmatched names ride through raw.
+- **Divergence (CLOSED in 1.3.0)**: foam's HELPERS apply the floor + key pass
+  AT CAPTURE (`api.rb` `stringify`/`set_attribute`/`log`; `metrics.rb` always
+  did), so a tenant `additional_*` instance receives foam-helper data
   ALREADY-MASKED (rule 18 C.3) — the pass is idempotent, so the wire value is
-  unchanged. The residual divergence is THIRD-PARTY INSTRUMENTATION attributes
-  only: those freeze at finish before any processor runs, so a tenant span
-  processor sees them as foam captured them and foam can only apply the key pass
-  at the exporter boundary. Ruby offers no safe pre-freeze hook — the
-  `on_finishing` hook runs inside the span mutex, so a processor calling
-  `set_attribute` there would deadlock. Foam's own export applies the configured
-  key pass, and the exporters FAIL CLOSED once a pass has begun: a struct whose
-  rebuild raises is DROPPED with a loud warning (`Redaction.mask_*` return nil on
-  hard failure), never exported half-masked.
-- **Mitigation**: capture-time key pass in `lib/foam/otel/api.rb` +
+  unchanged. THIRD-PARTY INSTRUMENTATION attributes freeze at finish before
+  any processor runs (no safe pre-freeze hook — `on_finishing` runs inside
+  the span mutex, so a processor calling `set_attribute` there would
+  deadlock), so they CANNOT be masked in place — instead the guarded tenant
+  wrappers (`pipelines.rb` `GuardedSpanProcessor#on_finish` /
+  `GuardedLogRecordProcessor#on_emit`) hand tenants a READ-ONLY MASKED VIEW
+  (`MaskedSpanView`/`MaskedLogRecordView`, served through the same
+  `Redaction.mask_span_data`/`mask_attributes`/`mask_body` pass foam's export
+  applies) — a tenant never sees a raw floor (or configured-key) value, and a
+  span whose masked view cannot be built is WITHHELD loudly, never handed
+  over raw. Foam's own export applies the same pass, and the exporters FAIL
+  CLOSED once the pass has begun: a struct whose rebuild raises is DROPPED
+  with a loud warning (`Redaction.mask_*` return nil on hard failure), never
+  exported half-masked.
+- **Mitigation**: capture-time pass in `lib/foam/otel/api.rb` + the masked
+  tenant view in `lib/foam/otel/pipelines.rb` +
   `lib/foam/otel/redacting_exporter.rb` + `lib/foam/otel/redaction.rb`.
-- **Test**: `spec/tenant_seam_spec.rb` (tenant processors receive helper-set
-  LISTED keys ALREADY-MASKED, free-text bodies RAW, the byte-identical
-  with/without-tenant proofs for spans, logs, AND metrics),
-  `spec/redacting_exporter_spec.rb` (opt-in masking on the wire; "a POISONED
-  SpanData (dup raises) is DROPPED — never exported raw"), `spec/redaction_spec.rb`
-  (default zero-redaction + the two opt-in key lists; capture+exporter idempotence).
+- **Test**: `spec/credential_floor_spec.rb` + `spec/credential_floor_wire_spec.rb`
+  (the floor per signal/door; the tenant masked-view regression gate — a
+  third-party-set `authorization` attribute never reaches a tenant raw),
+  `spec/tenant_seam_spec.rb` (tenant processors receive helper-set LISTED keys
+  ALREADY-MASKED, free-text bodies RAW, the byte-identical with/without-tenant
+  proofs for spans, logs, AND metrics), `spec/redacting_exporter_spec.rb`
+  (masking on the wire; "a POISONED SpanData (dup raises) is DROPPED — never
+  exported raw"), `spec/redaction_spec.rb` (raw-above-the-floor defaults + the
+  two opt-in key lists; capture+exporter idempotence).
 
 ## F2: OTEL_SDK_DISABLED and OTEL_PROPAGATORS are only honored by SDK.configure
 
@@ -623,8 +637,10 @@ anthropic 1.59.0, gemini-ai 4.3.0, ruby_llm 1.16.0).
   mitigation (foam never rewrites or drops metric dimensions — rule 4/26);
   the mitigation is the README metrics warning with the classic bad example —
   *Documented* (README "Metrics" section). *Test*: `spec/metrics_spec.rb`
-  ("captures a metric attribute RAW by default; masks it only when the key is in
-  redact_keys" — pins that unlisted attributes pass through VERBATIM).
+  ("captures a NON-floor metric attribute RAW by default; masks it only when
+  the key is in redact_keys" — pins that unlisted, non-floor attributes pass
+  through VERBATIM; the credential floor's frozen names are the one always-on
+  exception, `spec/credential_floor_wire_spec.rb`).
 - **G8 — never-ended spans leak memory**: a span started and never ended holds
   memory and context forever (and never exports — the batch processor only
   sees FINISHED spans, `on_finish`). `Foam::Otel.span {}` always ends the span
@@ -681,6 +697,22 @@ anthropic 1.59.0, gemini-ai 4.3.0, ruby_llm 1.16.0).
   beside it (`lib/foam/otel/runtime_metrics.rb` `guarded_callback`).
   *Test*: `spec/runtime_metrics_spec.rb` ("a poisoned reading is
   fail-to-dark", the foreign-slot and idempotence cases).
+- **G16 — Sidekiq's queue-hop trace shape is `propagation_style: :link`**:
+  the bundled contrib instrumentation's default (installed source:
+  opentelemetry-instrumentation-sidekiq-0.29.0 `instrumentation.rb:104`)
+  makes a performed job a SEPARATE trace whose first (consumer) span carries
+  a span LINK back to the enqueuing producer span — NOT a child inside the
+  web request's trace. Foam ships that upstream default untouched (rule 26:
+  the contrib pin sets the semconv; `FLOOR_INSTRUMENTATION_CONFIG` overrides
+  only the datastore raw-capture posture), so dashboards that follow ONLY
+  parent/child edges see the web trace end at the enqueue — follow the link,
+  or clamp the posture per deployment with the standard contrib lever
+  `OTEL_RUBY_INSTRUMENTATION_SIDEKIQ_CONFIG_OPTS='propagation_style=child'`
+  for one continuous web→job trace. Context is propagated across the hop
+  either way — a hop that DROPS it is a correctness bug, never accepted.
+  *Test*: `spec/floor_sidekiq_wire_spec.rb` (real Redis enqueue→perform:
+  producer span joins the request trace; consumer span LINKS the producer
+  span on the wire).
 
 ---
 

data/README.md

@@ -4,9 +4,12 @@ Foam's OpenTelemetry core for Ruby services. A thin, safe wrapper over the
 official OpenTelemetry libraries: foam owns the pipeline (providers, batch
 processors, OTLP export to the foam fleet endpoint), turns on automatic
 tier-1/2 instrumentation, and hands you a small set of never-throw helpers —
-and nothing else. **Redaction is fully opt-in: by default foam captures every
-value RAW** (no secret floor, no value-pattern masking); you enumerate the
-fields to mask via `redact_keys`/`redact_pii_keys`.
+and nothing else. **Redaction is fully opt-in above one always-on exception:
+by default foam captures every value RAW** (no value-pattern masking, no PII
+preset) **except the credential floor** — a fixed list of credential/secret
+header and field NAMES that is always masked to `[REDACTED]` (see "The
+default credential denylist" below); you enumerate any further fields to mask
+via `redact_keys`/`redact_pii_keys`.
 
 Built to `docs/BASE_PACKAGE_SPEC.md`. This README is the manual you integrate
 from.
@@ -75,10 +78,20 @@ drop-in upstream metrics gem, so foam ships its own collector),
 **session stitching** (inbound `baggage: session.id` from the foam browser
 package is stamped onto every span and log record), and the **LLM shims**
 (below). Every inbound request, DB call, HTTP call, job, log line, and model
-call now flows to foam. **Redaction is opt-in: by default every value —
-attributes, URLs/query strings, DB statement text, log bodies, LLM content —
-is captured RAW.** Pass `redact_keys`/`redact_pii_keys` to mask or erase
-specific fields (see below).
+call now flows to foam. **Redaction is opt-in above the credential floor: by
+default every value — attributes, URLs/query strings, DB statement text, log
+bodies, LLM content — is captured RAW, except values whose NAME is on the
+always-on credential denylist (masked `[REDACTED]`; see "The default
+credential denylist" below).** Pass `redact_keys`/`redact_pii_keys` to mask
+or erase further specific fields (see below).
+
+> **Sidekiq trace shape** (GOTCHAS G16): the bundled instrumentation keeps
+> its upstream default `propagation_style: :link` — a performed job runs as
+> its OWN trace whose first span LINKS back to the enqueuing span (context is
+> always propagated across the hop; proven against a real Redis in
+> `spec/floor_sidekiq_wire_spec.rb`). Prefer one continuous web→job trace?
+> Set the standard contrib lever:
+> `OTEL_RUBY_INSTRUMENTATION_SIDEKIQ_CONFIG_OPTS='propagation_style=child'`.
 
 ### The LLM surface — OpenAI, Anthropic, Gemini (and ruby_llm)
 
@@ -87,7 +100,14 @@ presence-checked shims over the SDKs' public call sites, emitting the standard
 `gen_ai.*` semantic conventions — activity (model, response id, finish
 reasons, token usage, latency) AND prompt/response **content, RAW**
 (`gen_ai.input.messages` / `gen_ai.output.messages` /
-`gen_ai.system_instructions`). Covered: the official `openai` SDK
+`gen_ai.system_instructions`). Agent tool loops ride in full: tool-call
+turns keep their tool names/arguments and tool-result turns keep their
+`tool_call_id` linkage inside the message content. `gen_ai.operation.name`
+(and the `{operation} {model}` span name) follows the semconv operation for
+each API shape — `chat` for the OpenAI/Anthropic/ruby_llm chat seams,
+`generate_content` for Gemini's generateContent, fleet-identical with the
+js/python Gemini paths (**wire change in 1.2.1**: Gemini spans previously
+said `chat`). Covered: the official `openai` SDK
 (chat.completions.create + responses.create), the official `anthropic` SDK
 (messages.create), Gemini via the `gemini-ai` gem (Google ships no official
 Ruby SDK), and `ruby_llm` (one seam covering all its providers). Absent SDK =
@@ -119,8 +139,8 @@ API):** the allowlist must include BOTH headers —
 | `enabled:` | Boolean | yes | — | The config switch. `false` = fully inert (no SDK, no providers, no network, helpers no-op). `true` = export, in EVERY environment. No default — you write the logic. |
 | `token:` | String | when enabled | — | `Authorization: Bearer` for export. Required (and validated) only when `enabled: true`; wired explicitly from your secret source — never an env fallback read by the gem. **Fleet convention: read it from the `FOAM_OTEL_TOKEN` env var.** Blank while enabled → raises at boot. |
 | `version:` | String | no | nil | `service.version`, verbatim (git SHA recommended). Missing → warns and continues. Never detected at runtime. |
-| `redact_keys:` | Array<String> | no | `[]` | The ONLY field names tail-masked (shape-preserving, e.g. `********f456`). Opt-in — empty (the default) means NO masking. Matches by case-insensitive substring, including keys nested in structured values and query-string keys. |
-| `redact_pii_keys:` | Array<String> | no | `[]` | Field names fully erased to `[REDACTED]` (no tail). Opt-in — empty (the default) means NO erasure. Foam ships no preset; this is your enumeration. |
+| `redact_keys:` | Array<String> | no | `[]` | The ONLY field names tail-masked (shape-preserving, e.g. `********f456`). Opt-in and ADDITIVE on top of the always-on credential floor — empty (the default) means no masking beyond the floor. Matches by case-insensitive substring, including keys nested in structured values and query-string keys. A key that is also on the floor stays fully `[REDACTED]` (the floor wins; it is never downgraded to a tail). |
+| `redact_pii_keys:` | Array<String> | no | `[]` | Field names fully erased to `[REDACTED]` (no tail). Opt-in and additive above the floor — empty (the default) means no erasure beyond the floor. Foam ships no PII preset; this is your enumeration. |
 | `additional_span_processors:` | Array | no | `[]` | Tenant seam: constructed SpanProcessor instances added to foam's pipeline (additive; never replace foam's export). See coexistence. |
 | `additional_log_record_processors:` | Array | no | `[]` | Tenant seam, logs. |
 | `additional_metric_readers:` | Array | no | `[]` | Tenant seam, metrics. |
@@ -178,6 +198,86 @@ override is the operator-level `OTEL_EXPORTER_OTLP_ENDPOINT` env var (below).
 
 ---
 
+## The default credential denylist (the always-on floor)
+
+**Fleet ruling 2026-07-26** (binding design:
+`docs/decisions/credential-denylist-design.md`; frozen fixture:
+`contract/credential-denylist.json`). This is the ONE exception to foam's
+raw-by-default capture, and it has **no off switch** — no init option, no env
+var, and no door-2 parameter can disable, shrink, or re-spell it.
+
+**What is masked.** The VALUE of every header/attribute/field whose NAME is on
+the frozen list, in every signal (spans — events and links included — logs,
+metrics), on door 1, on all three door-2 ingest taps, and in the view handed
+to tenant `additional_*` processors:
+
+- **(a) the seven credential headers**, matched as header names wherever
+  headers are captured — including the semconv
+  `http.request.header.<name>` / `http.response.header.<name>` span-attribute
+  forms: `authorization`, `proxy-authorization`, `cookie`, `set-cookie`,
+  `x-api-key`, `x-auth-token`, `www-authenticate`;
+- **(b) the frozen 52-entry key list** of
+  `contract/credential-denylist.json` (the deduplicated union of those seven,
+  sentry-python's default denylists, and foam's documented reference roots —
+  `password`, `token`, `secret`, `api_key`, `ssn`, `jwt`, `private_key`,
+  `connect.sid`, `phpsessid`, …), matched as attribute/field names at every
+  depth the engine walks, and per-key in URL query strings
+  (`?token=x&user=bob` → `token=[REDACTED]&user=bob`).
+
+**The matching rule.** Case-insensitive, dash/underscore-normalized **EXACT
+name-equality — never substring**: `authorization` masks the header/field
+`authorization` (any casing, `-`≡`_`) and nothing else — `authorization_url`,
+`x_api_key_id`, `secretary`, and `session_count` all ride RAW. Dotted entries
+match verbatim and whole (`connect.sid` masks; `user.session` does not). This
+is deliberately STRICTER matching than your own `redact_keys` lists, which
+keep their documented substring behavior.
+
+**The mask.** The full literal `[REDACTED]` — no tail, no length
+preservation. These are credentials, not debug aids. Semconv header
+attributes (string arrays) mask per element, preserving arity.
+
+**Precedence.** The floor runs unconditionally BEFORE your
+`redact_keys`/`redact_pii_keys`, which stay purely additive on top. Listing a
+floor name in `redact_keys` does NOT downgrade it to a tail mask — the floor
+is terminal. `enabled: false` stays fully inert exactly as before (nothing
+exports at all — the floor masks data that leaves; inert mode has none).
+
+**Everything else stays RAW.** The floor matches NAMES only — it never scans
+values: bodies, query values, SQL text, LLM content, and PII under unlisted
+names export byte-identical raw, exactly as documented everywhere else in
+this README.
+
+**Why.** Coverage-over-masking remains the mission, but raw `Authorization`
+headers, cookies, and API keys on the wire are a breach in waiting for every
+customer at once — the fleet ruling carves out exactly this list (and only
+this list) as non-negotiable. The list is byte-identical in every foam core
+and gate-checked in CI against the fleet fixture
+(`spec/credential_floor_spec.rb`).
+
+**Fleet param-name canon** (the customer options above the floor, identical
+concept in every core — the floor itself has NO init surface anywhere):
+
+| Concept | js/otel | js/browser | python | ruby | java |
+|---|---|---|---|---|---|
+| secret keys (tail-mask) | `redactKeys` | `redactKeys` | `redact_keys` | `redact_keys:` | `.redactKeys(String…)` |
+| PII keys (full `[REDACTED]`) | `redactPiiKeys` | `redactPiiKeys` | `redact_pii_keys` | `redact_pii_keys:` | `.redactPiiKeys(String…)` |
+
+> **Migration note — 1.3.0 (minor): always-on credential masking.** As of
+> this version foam masks, by default and in every signal, the VALUES of a
+> fixed list of credential/secret header and field NAMES (`authorization`,
+> `cookie`, `set-cookie`, `proxy-authorization`, `x-api-key`, `x-auth-token`,
+> `www-authenticate`, and the 52-name key list in
+> `contract/credential-denylist.json` — Sentry-parity plus foam's documented
+> roots) to the literal `[REDACTED]`. Matching is exact name-equality,
+> case-insensitive, dash/underscore-insensitive — never substring:
+> `authorization_url` is untouched. Everything else still exports RAW exactly
+> as before; `redact_keys`/`redact_pii_keys` are unchanged and additive.
+> There is no off switch — if a dashboard keyed off a raw credential value
+> (it should not have), it will now see `[REDACTED]`. Customers who
+> previously received these header/field values raw stop receiving them.
+
+---
+
 ## The helpers
 
 All helpers never raise, and no-op silently before `init` and when disabled.
@@ -273,10 +373,13 @@ Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FO
 > Ruby note: when you configure `redact_keys`/`redact_pii_keys`, foam's HELPERS
 > apply the key pass at capture, so a tenant processor receives foam-helper data
 > (span/log helper attributes, log bodies, metric labels) ALREADY masked.
-> Because the Ruby SDK freezes span attributes at finish, a tenant
-> processor sees third-party instrumentation attributes UNMASKED (for those,
-> foam's key pass runs at the exporter boundary — GOTCHAS F1). With no keys
-> configured (the default) nothing is masked on either path.
+> The Ruby SDK freezes span attributes at finish, so historically a tenant
+> processor saw third-party instrumentation attributes UNMASKED — as of 1.3.0
+> that gap is closed: tenant span/log processors are handed a READ-ONLY MASKED
+> VIEW (the credential floor plus your configured keys, the same pass foam's
+> own export applies — GOTCHAS F1). A tenant never sees a raw floor value.
+> With no keys configured (the default) only the credential floor is masked
+> on either path.
 >
 > Loop-guard note: `ignored_outbound_hosts` suppresses spans for tenant/agent
 > egress made through **Net::HTTP or Excon**. A tenant exporter using
@@ -314,9 +417,10 @@ end
 
 **Outcome**: their pipeline keeps working exactly as before — their data,
 their resource, their export, byte-identical — and foam ALSO receives the
-signal, carrying THEIR resource identity plus foam's export-time stamp and
-any per-tap `redact_keys`/`redact_pii_keys` you set (opt-in — none by default;
-details in the Door 2 section).
+signal, carrying THEIR resource identity plus foam's export-time stamp, the
+always-on credential floor applied to foam's copy, and any per-tap
+`redact_keys`/`redact_pii_keys` you set (opt-in — none by default; details in
+the Door 2 section).
 
 **Unhappy paths**:
 
@@ -416,8 +520,10 @@ readers only: they never mutate their data, their resource, or their export.
 ### The three entries
 
 All three share one signature (`token:` and `environment:` required; the
-redaction kwargs are the ONLY keys that tap redacts — opt-in, none by default,
-scoped to that tap; `diagnostics:` is tap-scoped narration). Construction
+redaction kwargs are the only CUSTOMER keys that tap redacts — opt-in, none by
+default, scoped to that tap, additive above the always-on credential floor,
+which every tap applies with zero configuration; `diagnostics:` is tap-scoped
+narration). Construction
 validates loudly at boot and NEVER
 throws afterwards — a failure inside a tap can never break their pipeline.
 Fleet canon mapping (pinned by `spec/export_surface_spec.rb`):
@@ -438,9 +544,10 @@ their_tracer_provider.add_span_processor(processor)
 # raises ArgumentError at construction for a blank token/environment
 ```
 
-**Logs** — a LogRecordProcessor for THEIR LoggerProvider. The tap's
-`redact_keys`/`redact_pii_keys` (if any) match over record attributes and keys
-nested in structured bodies; free-text bodies ride RAW (no value-pattern pass):
+**Logs** — a LogRecordProcessor for THEIR LoggerProvider. The credential
+floor and the tap's `redact_keys`/`redact_pii_keys` (if any) match over
+record attributes and keys nested in structured bodies; free-text bodies ride
+RAW (no value-pattern pass):
 
 ```ruby
 provider = OpenTelemetry::SDK::Logs::LoggerProvider.new(resource: their_resource)
@@ -484,7 +591,7 @@ in THEIR pipeline. Three layers keep it closed:
 | | their pipeline | foam's copy |
 | --- | --- | --- |
 | `service.name` / `service.version` / `service.instance.id` | theirs | theirs, untouched (the data carries THEIR identity) |
-| all attributes / events / bodies | raw, byte-identical | your per-tap `redact_keys`/`redact_pii_keys` applied (default: none → identical to theirs) |
+| all attributes / events / bodies | raw, byte-identical | the credential floor + your per-tap `redact_keys`/`redact_pii_keys` applied (default: floor only — everything off the frozen list identical to theirs) |
 | `deployment.environment.name` | theirs (if any) | the tap's `environment:` argument (stamped at export time) |
 | `foam.ingest.tier` | absent | `"external"` (the wire-contract marker) |
 | `telemetry.distro.name` / `telemetry.distro.version` | absent | `"foam"` / the gem version (support reads the tap's presence and version from telemetry alone) |

data/RESEARCH.md

@@ -144,13 +144,30 @@ statement posture is **captured RAW, never suppressed** (2026-07-26 ruling —
 coverage is the mission), and the upstream `-pg`/`-mysql2`/`-redis`/`-mongo`
 gems all DEFAULT to `db_statement: :obfuscate` — a sanitizer — so foam
 explicitly activates them with `db_statement: :include` (GOTCHAS F9; pinned
-by `spec/floor_spec.rb`). Redaction stays fully opt-in, applied only for keys
-the customer lists in `redact_keys`/`redact_pii_keys`. An FDE can still clamp
+by `spec/floor_spec.rb`). Redaction stays fully opt-in above the always-on
+credential floor (the frozen NAME list of
+`contract/credential-denylist.json` — statement TEXT is never scanned),
+applied only for keys the customer lists in
+`redact_keys`/`redact_pii_keys`. An FDE can still clamp
 `db_statement: :omit`/`:obfuscate` per customer via the standard
 `OTEL_RUBY_INSTRUMENTATION_<NAME>_CONFIG_OPTS` env var when a compliance
 context calls for it — per-customer tightening (rule 18 / Area 4), not the
 default.
 
+**Sidekiq queue-hop posture (documented, GOTCHAS G16):** `-sidekiq 0.29.0`
+defaults `propagation_style: :link` (installed source `instrumentation.rb:104`)
+— a performed job is a SEPARATE trace whose consumer span LINKS the enqueuing
+producer span. Foam ships the upstream default untouched
+(`FLOOR_INSTRUMENTATION_CONFIG` overrides only the datastore raw-capture
+posture); the continuous web→job shape is one env lever away
+(`OTEL_RUBY_INSTRUMENTATION_SIDEKIQ_CONFIG_OPTS='propagation_style=child'`).
+Context always survives the hop. Wire-proven per rule 8b against a REAL
+redis-server with the REAL sidekiq gem (enqueue → Redis → embedded-server
+perform; producer/consumer spans + the link asserted from the exporter):
+`spec/floor_sidekiq_wire_spec.rb`. The Faraday floor entry carries the same
+rule-8b emission proof against a local listener:
+`spec/floor_wire_spec.rb`.
+
 **Runtime + GC metrics (hand-written — landed,
 `lib/foam/otel/runtime_metrics.rb`).** Ruby has no drop-in upstream metrics
 instrumentation gem (unlike Python's `-system-metrics` or the Node
@@ -165,8 +182,19 @@ construction (rule 26). Tests: `spec/runtime_metrics_spec.rb`.
 
 **Excluded (with reason):** `-factory_bot`, `-rspec` (test-only, no production
 value); `-net_ldap` (niche, PII/credential-adjacent — explicit FDE opt-in);
+`-logger 0.4.0` (SUPERSEDED by foam's own stdlib Logger bridge, §4: both
+patch the SAME `Logger#add` seam, and foam's bridge keeps the lazy-block
+contract, both loop guards, the logs-slot gating, and the opt-in redaction
+path the contrib bridge lacks. Two bridges on one seam would export every
+host log line TWICE — rule 12 never-double-attach — so init's bundle-driven
+sweep pins the contrib gem disabled (`SUPERSEDED_INSTRUMENTATION_CONFIG`,
+`lib/foam/otel/init.rb`; proven by `spec/floor_spec.rb`); an FDE who
+deliberately wants the contrib bridge instead passes its instance via
+`additional_instrumentations`, which bypasses the sweep map);
 `all`/`base` (not instrumentations — foam deliberately does NOT depend on the
-`all` meta-gem, which would drag test gems).
+`all` meta-gem, which would drag test gems). With the 17-gem floor, the 27
+when-present gems, and these 4 exclusions, all 48 registry gems have a
+written disposition (rule 6: no silent omissions).
 
 **Known-bad / crash history (rule 6 / R3):** action_pack overrides rack config
 (contrib #88); active_record 0.2.2 failed on Rails 7 (#38); `-que` NoMethodError
@@ -185,7 +213,8 @@ foam ships the logs pipeline + `log()` helper (trace-correlated) — the mechani
 rule 20 requires. The stdlib **Logger bridge is LANDED as foam's own code**
 (`lib/foam/otel/logger_bridge.rb`, console-logs ruling 2026-07-26): one
 prepend on `Logger#add` turns every message a host logger accepts into a
-trace-correlated OTel log record — body RAW (opt-in key-masking only),
+trace-correlated OTel log record — body RAW (name-keyed masking only: the
+always-on credential floor plus any customer keys; free text never scanned),
 severity mapped, loop-guarded (the OTel SDK's internal logger is skipped by
 identity + a thread-local re-entrancy flag), gated on foam owning the logs
 slot, and never able to break the host's log call (GOTCHAS F10; tests
@@ -197,6 +226,15 @@ one prepend covers most framework logging in practice;
 https://github.com/open-telemetry/opentelemetry-ruby/discussions/1789 is
 open/unresolved) — a documented gap, narrowed to the non-Logger-backed exotics.
 
+The official contrib `opentelemetry-instrumentation-logger 0.4.0` bridges the
+same `Logger#add` seam and is deliberately NOT bundled — census disposition
+EXCLUDED/superseded (§3): foam's bridge additionally holds the lazy-block
+contract, the OTel-internal-logger identity guard, the re-entrancy flag, the
+logs-slot gate, and the redaction path (credential floor + opt-in keys), and
+shipping/sweeping both
+would double-export every host log line (rule 12). init's sweep pins the
+contrib gem disabled even when an FDE bundles it (`spec/floor_spec.rb`).
+
 ---
 
 ## 5. LLM ecosystem (rule 31)
@@ -210,7 +248,16 @@ shims patch the SDKs' public call sites — official `openai`
 (messages.create), `gemini-ai` (generate_content / stream_generate_content;
 Google ships no official Ruby SDK), and `ruby_llm` (Provider#complete, one
 seam covering all its providers) — emitting the same GenAI semconv attribute
-set the fleet's js/python gap-fillers emit. **LLM traces are captured in
+set the fleet's js/python gap-fillers emit. `gen_ai.operation.name` matches
+the fleet per API shape: `chat` on the chat seams, `generate_content` on the
+Gemini shim (aligned in 1.2.1 — Ruby previously emitted `chat` for Gemini, a
+cross-language vocabulary split against the js shim and python's official
+google-genai instrumentor). Agent-loop tool activity is captured on the
+ruby_llm seam too: assistant tool-call turns serialize their `tool_calls`
+(id/name/arguments) and tool-result turns their `tool_call_id` into
+`gen_ai.input.messages`/`gen_ai.output.messages`
+(`spec/llm_ruby_llm_spec.rb` drives a real `chat.with_tool` loop on the
+wire); the Anthropic shim carries tool_use blocks verbatim inside content. **LLM traces are captured in
 full — activity (model, response id, finish reasons, tokens, latency) AND
 prompt/completion content, raw by default** (2026-07-26 ruling: LLM content
 is never gated or dropped; the FDE reactive clamp is the only per-customer
@@ -247,38 +294,59 @@ https://opentelemetry.io/docs/specs/semconv/gen-ai/aws-bedrock/ ; traceloop Ruby
 
 ## 6. Redaction design record (rule 14, as amended 2026-07-26)
 
-**Redaction is FULLY OPT-IN — the default is ZERO redaction** (owner ruling
-2026-07-26, holistic-redesign decision log; `lib/foam/otel/redaction.rb`).
-With neither `redact_keys` nor `redact_pii_keys` set (or both empty) every
-value is captured RAW — attributes, headers, bodies, URL query params,
-`db.statement`, log bodies — including secrets and PII. There is NO always-on
-secrets floor and NO value-pattern auto-masking; the package auto-masks
-nothing and ships no preset. The customer owns the privacy/legal posture; the
-FDE reactive clamp is the per-customer valve.
-
-**The retired floor**: earlier revisions shipped an always-on researched
-secrets floor in `constants.rb` (roots + value-pattern pass, sourced from the
-OSS-scrubber consensus — raven-ruby, sentry, Elastic APM, Rails
-filter_parameters, gitleaks et al. — modeled on the Node core). It was
-REMOVED by the 2026-07-26 ruling; the provenance record lives in this file's
-git history and the decision log. Do not reintroduce it without a new ruling.
-
-**The current engine** (`redaction.rb`): redaction runs ONLY over the keys
-the customer enumerates. `redact_keys` matches → tail mask, shape-preserving
-(`********` + last 4 when ≥ 12 chars; the fixed 8-char body never reveals
-length; non-scalars redact in full); `redact_pii_keys` matches → full
-`[REDACTED]`, no tail. Matching is SUBSTRING, case-insensitive, over the
-lowercased key — the customer's lists, nothing inferred. The pass runs at the
-exporter boundary (`redacting_exporter.rb` — the Ruby SDK freezes span
-attributes at finish before any `on_finish` processor runs, so the exporter
-boundary is the seam; see GOTCHAS) and is a true pass-through when no keys
-are configured. The explicit `redact(value)` helper always masks (calling it
-is the opt-in). Fail-closed for listed keys (mask on any error, cycle + depth
-guarded).
-
-Proven by `spec/redaction_spec.rb` (opt-in defaults — nothing masked with no
-keys; both mask shapes; fail-closed) and `spec/redacting_exporter_spec.rb` +
-the conformance app (default-path RAW capture on the wire).
+**Redaction is FULLY OPT-IN above the CREDENTIAL FLOOR** (owner ruling
+2026-07-26, holistic-redesign decision log, AMENDED the same day by the
+credential-floor fleet ruling — `docs/decisions/credential-denylist-design.md`;
+`lib/foam/otel/redaction.rb`). With neither `redact_keys` nor
+`redact_pii_keys` set (or both empty) every value is captured RAW —
+attributes, bodies, URL query params, `db.statement`, log bodies — including
+PII, EXCEPT the one always-on exception: the CREDENTIAL FLOOR masks the
+values of the frozen credential/secret header + key NAME list
+(`contract/credential-denylist.json`: the seven headers + 52 keys, constants
+`CREDENTIAL_HEADER_DENYLIST`/`CREDENTIAL_KEY_DENYLIST` in `constants.rb`) to
+the literal `[REDACTED]`, in every signal, on every door, with NO off switch.
+There is still NO value-pattern auto-masking and NO PII preset; above the
+floor the customer owns the privacy/legal posture; the FDE reactive clamp is
+the per-customer valve.
+
+**The retired floor vs the credential floor**: earlier revisions shipped an
+always-on researched secrets floor in `constants.rb` (roots + value-pattern
+pass, sourced from the OSS-scrubber consensus — raven-ruby, sentry, Elastic
+APM, Rails filter_parameters, gitleaks et al. — modeled on the Node core). It
+was REMOVED by the 2026-07-26 zero-default ruling; the provenance record
+lives in this file's git history and the decision log. The SAME-DAY
+credential-floor ruling is the new fleet ruling that reintroduces exactly ONE
+always-on list — the frozen credential denylist (name-match only, never
+value patterns; rule-14 reference roots absorbed as entries). Its content
+changes only by a new fleet ruling and is CI-gated bit-for-bit against the
+fleet fixture (`spec/credential_floor_spec.rb`).
+
+**The current engine** (`redaction.rb`): the floor predicate
+(`floor_kind`/`floor_match?`) runs FIRST and is TERMINAL — canon-exact
+matching (trim, ASCII-lowercase, `-`→`_` fold; NEVER substring:
+`authorization` never masks `authorization_url`), including the semconv
+`http.{request,response}.header.<name>` prefix peel. Customer redaction then
+runs ONLY over the keys the customer enumerates. `redact_keys` matches → tail
+mask, shape-preserving (`********` + last 4 when ≥ 12 chars; the fixed 8-char
+body never reveals length; non-scalars redact in full); `redact_pii_keys`
+matches → full `[REDACTED]`, no tail. CUSTOMER matching is SUBSTRING,
+case-insensitive, over the lowercased key — the customer's lists, nothing
+inferred (the FLOOR deliberately does NOT ride the substring matcher — it
+would over-match). The pass runs at the exporter boundary
+(`redacting_exporter.rb` — the Ruby SDK freezes span attributes at finish
+before any `on_finish` processor runs, so the exporter boundary is the seam;
+see GOTCHAS) and runs UNCONDITIONALLY (the floor has no off state); values
+under unmatched names ride through raw, and tenant `additional_*` processors
+receive the masked VIEW (`pipelines.rb`, GOTCHAS F1). The explicit
+`redact(value)` helper always masks (calling it is the opt-in). Fail-closed
+for floor + listed keys (mask on any error, cycle + depth guarded).
+
+Proven by `spec/credential_floor_spec.rb` + `spec/credential_floor_wire_spec.rb`
+(the floor: list identity gate, matcher, every signal, doors 1+2, tenant
+view, negative controls), `spec/redaction_spec.rb` (raw-above-the-floor
+defaults — nothing else masked with no keys; both mask shapes; fail-closed)
+and `spec/redacting_exporter_spec.rb` + the conformance app (default-path RAW
+capture on the wire, floor names excepted).
 
 ---
 

data/lib/foam/otel/api.rb

@@ -105,13 +105,14 @@ module Foam
         span = OpenTelemetry::Trace.current_span
         return false unless span.respond_to?(:recording?) && span.recording?
 
-        # Capture-time key pass (rule 18 C.3): when the customer configured
-        # redact_keys/redact_pii_keys the value lands ALREADY-MASKED, so tenant
-        # processors and inert-mode providers only ever see the redacted view of
-        # helper-set data. The exporter pass re-applies (idempotent). With no
-        # keys configured this is a pass-through — the value is captured RAW.
-        # scrub_utf8 is encoding hygiene, not masking: an invalid-UTF-8 value
-        # would make the upstream OTLP encoder drop the WHOLE batch (rule 15).
+        # Capture-time pass (rule 18 C.3): the always-on credential floor plus
+        # any configured redact_keys/redact_pii_keys, so the value lands
+        # ALREADY-MASKED — tenant processors and inert-mode providers only ever
+        # see the redacted view of helper-set data. The exporter pass
+        # re-applies (idempotent). Values under unmatched names are captured
+        # RAW. scrub_utf8 is encoding hygiene, not masking: an invalid-UTF-8
+        # value would make the upstream OTLP encoder drop the WHOLE batch
+        # (rule 15).
         span.set_attribute(key.to_s, Redaction.mask_one(key.to_s, Redaction.scrub_utf8(value), active_config))
         true
       rescue StandardError, SystemStackError
@@ -260,14 +261,14 @@ module Foam
         LEVEL_TEXT[severity.to_s.downcase.to_sym]
       end
 
-      # Stringify keys, then apply the customer's key lists AT CAPTURE (rule 18
-      # C.3): when redact_keys/redact_pii_keys are configured, helper-set
-      # attributes are ALREADY-MASKED when they reach the span/log record, so a
-      # tenant processor (or a foreign provider in inert mode) only ever sees the
-      # redacted view of foam-helper data. The exporter-boundary pass re-applies
-      # — masking is idempotent. With no keys configured this is a pass-through
-      # (raw capture). NEVER throws (rule 9): non-enumerable garbage degrades to
-      # nil (attributes dropped, telemetry still flows).
+      # Stringify keys, then apply the credential floor + the customer's key
+      # lists AT CAPTURE (rule 18 C.3): helper-set attributes are
+      # ALREADY-MASKED when they reach the span/log record, so a tenant
+      # processor (or a foreign provider in inert mode) only ever sees the
+      # redacted view of foam-helper data. The exporter-boundary pass
+      # re-applies — masking is idempotent. Values under unmatched names ride
+      # RAW. NEVER throws (rule 9): non-enumerable garbage degrades to nil
+      # (attributes dropped, telemetry still flows).
       def stringify(attributes)
         return nil if attributes.nil?
 
@@ -279,10 +280,11 @@ module Foam
         nil
       end
 
-      # The customer key pass over a log BODY at capture (same rationale as
-      # stringify). When keys are configured mask_body is fail-closed and
-      # depth-guarded, so a cyclic structure is bounded before it can reach any
-      # encoder; with no keys configured the body is captured RAW.
+      # The floor + customer key pass over a log BODY at capture (same
+      # rationale as stringify). mask_body is fail-closed and depth-guarded,
+      # so a cyclic structure is bounded before it can reach any encoder; a
+      # free-text body (and every structured field under an unmatched name)
+      # is captured RAW.
       def mask_log_body(body)
         return body if body.nil?
 

data/lib/foam/otel/config.rb

@@ -42,8 +42,10 @@ module Foam
           environment: environment,
           version: version,
           enabled: enabled,
-          # Opt-in redaction (holistic redesign): the ONLY keys ever redacted.
-          # Empty (the default) → zero redaction, raw capture. Stored lowercased
+          # Opt-in CUSTOMER redaction (holistic redesign), additive above the
+          # always-on credential floor (which is a module constant, NOT config
+          # state — no config shape can alter it). Empty (the default) → no
+          # customer redaction, raw capture above the floor. Stored lowercased
           # for the engine's case-insensitive substring match.
           redact_keys: downcase_list(redact_keys),
           redact_pii_keys: downcase_list(redact_pii_keys),