foam-otel 1.4.0 → 1.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 180e8af99c1f5c9bfc35de221c4985fb3ebac7b52ddf6fc99d6552a52aaea836
-  data.tar.gz: 14a5d81c673af1a20a75e483e076d2c6b1820fa14e5a587e2a23451da3cccf77
+  metadata.gz: 1e23dd37962a70ea36799a92d914fd2a38aa967a6cd1aface19bbaa322ab8966
+  data.tar.gz: f0018c27fb2dfff9316118438688a4a8ab2baea18810dbda522880361dfe6681
 SHA512:
-  metadata.gz: a307d175e903abbbbd1c382e65694664f47aba18408fc7153f9a43e9fb72f8131fedcfc71a59e16ee4f7446dee0f3460233883f96a7048e1284fff75b54e16c7
-  data.tar.gz: 7322e0a91e04092d494150ad9b003e4a6fd61b23ebc7468a8c715eec2d5ab8aac64d1edc4c9fb3cb1a844c3d3a56c167b9bbf05bed2bb1d4cb491eba20695f5d
+  metadata.gz: 9731cb1fae1b54f236dd583102b2348ab4b45bff38745f446c584151e594ea685f2689d6a7fb4807136372006b7b10c0284b2e7c140c0c9625edb5a820294c8a
+  data.tar.gz: 91744d9fb0fb671bce62de973eec96152b147742ee5948fedf607626dfd89b8c3c6ed055522f21c4bbfd7d65ce5b558da9e67605578e3573ec15f8b0d4efeecc

data/README.md

@@ -142,8 +142,9 @@ API):** the allowlist must include BOTH headers —
 | `enabled:` | Boolean | yes | — | The config switch. `false` = fully inert (no SDK, no providers, no network, helpers no-op). `true` = export, in EVERY environment. No default — you write the logic. |
 | `token:` | String | when enabled | — | `Authorization: Bearer` for export. Required (and validated) only when `enabled: true`; wired explicitly from your secret source — never an env fallback read by the gem. **Fleet convention: read it from the `FOAM_OTEL_TOKEN` env var.** Blank while enabled → raises at boot. |
 | `version:` | String | no | nil | `service.version`, verbatim (git SHA recommended). Missing → warns and continues. Never detected at runtime. |
-| `redact_keys:` | Array<String> | no | `[]` | The ONLY field names tail-masked (shape-preserving, e.g. `********f456`). Opt-in and ADDITIVE on top of the always-on credential floor — empty (the default) means no masking beyond the floor. Matches by case-insensitive substring, including keys nested in structured values and query-string keys. A key that is also on the floor stays fully `[REDACTED]` (the floor wins; it is never downgraded to a tail). |
-| `redact_pii_keys:` | Array<String> | no | `[]` | Field names fully erased to `[REDACTED]` (no tail). Opt-in and additive above the floor — empty (the default) means no erasure beyond the floor. Foam ships no PII preset; this is your enumeration. |
+| `redact:` | Hash | no | nil | The grouped redaction object (1.5.0, the preferred surface): `{ secrets: [...], pii: [...], detect: [...] }` — symbol or string keys. `secrets:` has `redact_keys` semantics (tail mask) and `pii:` has `redact_pii_keys` semantics (full `[REDACTED]`); when both a legacy option and its `redact` field are set, the lists UNION. `detect:` is the opt-in PII detection tier's entity list — value-shape detection with typed placeholders (see "PII detection (opt-in)" below). An unknown field, a non-Array value, or an unknown `detect` entity name raises at boot. Absent (the default) → behavior byte-identical to before. |
+| `redact_keys:` | Array<String> | no | `[]` | Alias — the preferred spelling is `redact: { secrets: [...] }` (same semantics; the lists union when both are set). The ONLY field names tail-masked (shape-preserving, e.g. `********f456`). Opt-in and ADDITIVE on top of the always-on credential floor — empty (the default) means no masking beyond the floor. Matches by case-insensitive substring, including keys nested in structured values and query-string keys. A key that is also on the floor stays fully `[REDACTED]` (the floor wins; it is never downgraded to a tail). |
+| `redact_pii_keys:` | Array<String> | no | `[]` | Alias — the preferred spelling is `redact: { pii: [...] }` (same semantics; the lists union when both are set). Field names fully erased to `[REDACTED]` (no tail). Opt-in and additive above the floor — empty (the default) means no erasure beyond the floor. Foam ships no PII preset; this is your enumeration. |
 | `additional_span_processors:` | Array | no | `[]` | Tenant seam: constructed SpanProcessor instances added to foam's pipeline (additive; never replace foam's export). See coexistence. |
 | `additional_log_record_processors:` | Array | no | `[]` | Tenant seam, logs. |
 | `additional_metric_readers:` | Array | no | `[]` | Tenant seam, metrics. |
@@ -161,6 +162,22 @@ Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: EN
                 redact_pii_keys: %w[customer_email full_name])               # fully [REDACTED]
 ```
 
+```ruby
+# The grouped redact object (1.5.0, preferred — the flat options above are
+# aliases and UNION with it when both are set), plus the opt-in PII
+# detection tier (typed placeholders; see "PII detection (opt-in)"):
+Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
+                redact: {
+                  secrets: %w[internal_ref voucher_code],   # == redact_keys (tail mask)
+                  pii: %w[customer_email full_name],        # == redact_pii_keys (full [REDACTED])
+                  detect: %w[email phone ssn credit_card ip], # value-shape detection → [EMAIL], [PHONE], …
+                })
+# An unknown field, a non-Array value, or an unknown detect entity raises at boot:
+# Foam::Otel.init(..., redact: { detect: ["name"] })
+# => raises ArgumentError: Foam::Otel.init redact: unknown detect entity "name" — the valid
+#    entity names are exactly {email, phone, ssn, credit_card, ip} (contract/pii-detect.json)
+```
+
 ```ruby
 # Only if your legitimate telemetry collides with the generic secret
 # heuristics (e.g. base64 content-addressed ids masked as high-entropy
@@ -365,6 +382,58 @@ changed only by fleet ruling.
 
 ---
 
+## PII detection (opt-in)
+
+The floor and the secret layer protect credentials. PII in VALUES — an email
+address inside a log line, a card number inside an exception message — still
+exports RAW by default, because coverage-over-masking is the mission and only
+you know your privacy posture. As of 1.5.0 you can opt into value-shape PII
+detection per entity, via `redact: { detect: [...] }` (fleet ruling
+2026-07-28; frozen fixture: `contract/pii-detect.json` — the entity names and
+placeholders are byte-identical in every foam core and gate-checked in CI by
+`spec/pii_detect_spec.rb`). Nothing detects unless you list the entity —
+an empty/absent `detect` list is exactly today's behavior.
+
+**The exact entity set** (these five, nothing else — an unknown name raises
+at boot):
+
+| Entity | Placeholder | Catches |
+| --- | --- | --- |
+| `email` | `[EMAIL]` | `john.smith+test@example.co.uk` — requires a real TLD (`user@localhost` rides raw) |
+| `phone` | `[PHONE]` | `+1 (415) 555-0142`, `415-555-0199` — separated groups; bare digit runs, dotted versions (`2024.10.05`) and clock times ride raw |
+| `ssn` | `[SSN]` | `536-90-4399`, `536 90 4399` — delimited 3-2-4 only; structurally invalid SSNs (`000-…`, group `00`, serial `0000`) and undelimited runs ride raw |
+| `credit_card` | `[CREDIT_CARD]` | 13–19 digit PANs (spaced/dashed/contiguous) that pass **Luhn** — a card-shaped tracking id failing the checksum is never masked |
+| `ip` | `[IP]` | IPv4 (octet-validated — `999.1.1.1` and `10.4.1.2000` ride raw) and IPv6, `::`-compressed included |
+
+**How it masks.** Only the matched character span is replaced with the
+entity's typed placeholder — surrounding text keeps its diagnostic value
+(`login from [IP] flagged`). It runs everywhere the value-pattern secret
+layer runs (span/event/link attributes, `status.message`, log bodies, URL
+query and fragment leaves, nested structures, both doors), immediately AFTER
+the secret layer (a shaped credential still becomes `[REDACTED]` — detect
+never weakens it) and BEFORE your key lists; metric datapoint attributes stay
+exempt exactly like the secret layer. Detection is idempotent (placeholders
+never re-match), rides the same execution caps and fail-closed discipline
+(any detector fault masks the whole value to `[REDACTED]`, loudly), and the
+patterns follow the same bounded-quantifier authoring rules.
+
+**What it can NOT do — read this before relying on it.** This is value-SHAPE
+detection, nothing more: **person names and free-text prose are NOT
+detectable** — recognizing "the patient, John Smith, reported…" takes
+server-side NER, which is platform scope, not an SDK regex. Field-NAME-keyed
+PII (`patient_name`) is what `redact: { pii: [...] }` is for; enumerate those
+keys yourself. Foam still ships no PII preset — the detect list is your
+explicit, audited enumeration.
+
+```ruby
+# Door-2 taps take the same redact object, scoped to that tap:
+processor = Foam::Otel.create_ingest_span_processor(
+  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV"),
+  redact: { pii: %w[customer_email], detect: %w[email credit_card] })
+```
+
+---
+
 ## The helpers
 
 All helpers never raise, and no-op silently before `init` and when disabled.
@@ -607,7 +676,9 @@ readers only: they never mutate their data, their resource, or their export.
 ### The three entries
 
 All three share one signature (`token:` and `environment:` required; the
-redaction kwargs are the only CUSTOMER keys that tap redacts — opt-in, none by
+redaction kwargs — the grouped `redact:` object and its legacy aliases
+`redact_keys:`/`redact_pii_keys:`, union semantics exactly as on `init` —
+are the only CUSTOMER redaction that tap applies — opt-in, none by
 default, scoped to that tap, additive above the always-on credential floor,
 which every tap applies with zero configuration; `diagnostics:` is tap-scoped
 narration). Construction

data/lib/foam/otel/config.rb

@@ -11,11 +11,17 @@ module Foam
     # loop guard and the redacting exporters can read one authoritative value.
     Config = Struct.new(
       :name, :environment, :version, :enabled,
-      :redact_keys, :redact_pii_keys, :ignored_outbound_hosts,
+      :redact_keys, :redact_pii_keys, :redact_detect, :ignored_outbound_hosts,
       :diagnostics, :endpoint, :secret_heuristics,
       keyword_init: true
     )
 
+    # The exact field names the grouped `redact:` option accepts (fleet ruling
+    # 2026-07-28, docs/decisions/redact-object-detect-design.md §2): `secrets`
+    # unions with the legacy redact_keys, `pii` with redact_pii_keys, `detect`
+    # opts into the PII detection tier. Anything else raises at boot.
+    REDACT_OPTION_FIELDS = %w[secrets pii detect].freeze
+
     class << self
       # The inert config the helpers read before init() runs (everything a
       # no-op needs: empty redaction lists, export disabled).
@@ -23,6 +29,7 @@ module Foam
         Config.new(
           name: "", environment: "", version: nil, enabled: false,
           redact_keys: [].freeze, redact_pii_keys: [].freeze,
+          redact_detect: [].freeze,
           ignored_outbound_hosts: [].freeze,
           diagnostics: false, endpoint: FOAM_OTEL_ENDPOINT,
           secret_heuristics: true
@@ -37,7 +44,8 @@ module Foam
 
       def resolve_config(name:, environment:, version:, enabled:,
                          redact_keys:, redact_pii_keys:, ignored_outbound_hosts:,
-                         diagnostics:, endpoint:, secret_heuristics: true)
+                         diagnostics:, endpoint:, secret_heuristics: true, redact: nil)
+        keys, pii, detect = merged_redact_lists(redact_keys, redact_pii_keys, redact)
         Config.new(
           name: name,
           environment: environment,
@@ -47,9 +55,13 @@ module Foam
           # always-on credential floor (which is a module constant, NOT config
           # state — no config shape can alter it). Empty (the default) → no
           # customer redaction, raw capture above the floor. Stored lowercased
-          # for the engine's case-insensitive substring match.
-          redact_keys: downcase_list(redact_keys),
-          redact_pii_keys: downcase_list(redact_pii_keys),
+          # for the engine's case-insensitive substring match. The grouped
+          # `redact:` object (2026-07-28 ruling) UNIONS with the legacy flat
+          # lists; redact.detect is the opt-in PII detection tier's entity
+          # list — empty (the default) means zero PII detection.
+          redact_keys: keys,
+          redact_pii_keys: pii,
+          redact_detect: detect,
           ignored_outbound_hosts: Array(ignored_outbound_hosts).map { |h| h.to_s.downcase }.freeze,
           diagnostics: diagnostics ? true : false,
           endpoint: endpoint,
@@ -69,21 +81,84 @@ module Foam
         Array(list).map { |k| k.to_s.downcase }.reject(&:empty?).uniq.freeze
       end
 
+      # ---- the grouped `redact:` object (fleet ruling 2026-07-28) -----------
+      # Parse + validate the object into its three raw lists. LOUD AT BOOT
+      # (rule 10, exactly like the name/environment validation): a non-Hash
+      # redact, an unknown field, a non-Array field value, or an unknown
+      # detect entity name raises ArgumentError here — never a silent skip,
+      # never a dark boot. Symbol AND string keys/entities are accepted
+      # (design §2 Ruby row); entities are coerced with the same downcase
+      # normalization the legacy lists use, then validated against the frozen
+      # fixture entity names (contract/pii-detect.json — the ONLY valid
+      # spellings). Called by both doors: init() via resolve_config /
+      # refresh_post_init_config, and the ingest factory's step-2 validation
+      # (with the entry name as context).
+      def parse_redact!(redact, context: "Foam::Otel.init")
+        return { secrets: nil, pii: nil, detect: nil } if redact.nil?
+
+        unless redact.is_a?(Hash)
+          raise ArgumentError, "#{context} redact: must be a Hash with fields drawn from " \
+                               "{#{REDACT_OPTION_FIELDS.join(', ')}} (e.g. redact: { detect: [\"email\"] }), " \
+                               "got #{redact.class}"
+        end
+        out = { secrets: nil, pii: nil, detect: nil }
+        redact.each do |key, value|
+          field = key.to_s
+          unless REDACT_OPTION_FIELDS.include?(field)
+            raise ArgumentError, "#{context} redact: unknown field #{key.inspect} — the valid fields are " \
+                                 "exactly {#{REDACT_OPTION_FIELDS.join(', ')}}"
+          end
+          unless value.is_a?(Array)
+            raise ArgumentError, "#{context} redact: #{field} must be an Array of strings, got #{value.class}"
+          end
+          out[field.to_sym] = Array(out[field.to_sym]) + value
+        end
+        validate_detect_entities!(out[:detect], context)
+        out
+      end
+
+      # Fixture rule "unknown_entity": the valid names are exactly the keys of
+      # contract/pii-detect.json's `entities` — anything else raises at init.
+      def validate_detect_entities!(detect, context)
+        return if detect.nil?
+
+        valid = PII_DETECT_ENTITIES.keys
+        detect.each do |entity|
+          next if valid.include?(entity.to_s.downcase)
+
+          raise ArgumentError, "#{context} redact: unknown detect entity #{entity.inspect} — the valid entity " \
+                               "names are exactly {#{valid.join(', ')}} (contract/pii-detect.json)"
+        end
+      end
+
+      # Legacy flat lists ∪ the grouped object's fields (design §2: the legacy
+      # aliases live on unchanged; when both are set, the lists UNION), all
+      # three normalized exactly as the legacy options always were.
+      def merged_redact_lists(redact_keys, redact_pii_keys, redact, context: "Foam::Otel.init")
+        parsed = parse_redact!(redact, context: context)
+        [downcase_list(Array(redact_keys) + Array(parsed[:secrets])),
+         downcase_list(Array(redact_pii_keys) + Array(parsed[:pii])),
+         downcase_list(parsed[:detect])]
+      end
+
       # A second init() refreshes ONLY the redaction lists (and diagnostics,
       # and the heuristic-tier flag — redaction state) onto the EXISTING
       # config: enabled/endpoint/identity are process-global and immutable
       # after the first init (see init.rb — a second init must never flip
-      # enabled and leave foam half-dark). Private: not public API surface
-      # (pinned by export_surface_spec).
+      # enabled and leave foam half-dark). The `redact:` object refreshes
+      # exactly like the legacy lists (design §2 second-init row). Private:
+      # not public API surface (pinned by export_surface_spec).
       def refresh_post_init_config(existing, redact_keys:, redact_pii_keys:, diagnostics:,
-                                   secret_heuristics: true)
+                                   secret_heuristics: true, redact: nil)
+        keys, pii, detect = merged_redact_lists(redact_keys, redact_pii_keys, redact)
         Config.new(
           name: existing.name,
           environment: existing.environment,
           version: existing.version,
           enabled: existing.enabled,
-          redact_keys: downcase_list(redact_keys),
-          redact_pii_keys: downcase_list(redact_pii_keys),
+          redact_keys: keys,
+          redact_pii_keys: pii,
+          redact_detect: detect,
           ignored_outbound_hosts: existing.ignored_outbound_hosts,
           diagnostics: diagnostics ? true : false,
           endpoint: existing.endpoint,

data/lib/foam/otel/constants.rb

@@ -342,5 +342,74 @@ module Foam
       yourkey your_api_key your-api-key xxxxxxxx todo
       notreal loremipsum deadbeef 0000000000 1111111111
     ].freeze
+
+    # ---- THE PII DETECT TIER (fleet ruling 2026-07-28, FULLY OPT-IN) --------
+    # docs/decisions/redact-object-detect-design.md §3; frozen fixture:
+    # contract/pii-detect.json (entity names + placeholders byte-identical,
+    # gate-checked by spec/pii_detect_spec.rb; behavior proven against the
+    # fixture's positive/negative vectors). Nothing here runs unless the
+    # customer lists the entity in init's redact.detect (or a door-2 tap's
+    # redact:) — the empty/absent list is today's raw-by-default behavior,
+    # byte-identical. The engine (redaction.rb detect_pii) runs this tier
+    # immediately AFTER the value-pattern secret layer and BEFORE customer
+    # key masking, masking ONLY the matched character span with the entity's
+    # typed :placeholder (surrounding text preserved verbatim; emitted
+    # placeholders never re-match — fixture "placeholder_idempotence").
+    #
+    # Authoring rules mirror the secret layer's (design §V.5.2 / rule 49):
+    # compile-once at require (a bad pattern fails the require, loudly),
+    # bounded quantifiers only, no lookbehind, no atomic/possessive groups;
+    # :group names the span to mask (0 = the whole match; phone masks group 1
+    # behind a consumed non-digit boundary — never a lookbehind). Patterns
+    # are byte-consistent with the python core's PII_DETECT_ENTITIES; the one
+    # recorded portability translation is phone's `^` → `\A` (Ruby's ^ is a
+    # LINE anchor — same translation note as SECRET_HEURISTIC_H2). :filter
+    # (:luhn) names a mandatory post-match validator — a card-shaped digit
+    # string failing Luhn is NEVER masked (fixture rule "checksum"). NOT
+    # detectable by design: person names and free-text prose (server-side
+    # NER scope — the README states this plainly).
+    PII_DETECT_ENTITIES = {
+      "email" => {
+        placeholder: "[EMAIL]", group: 0,
+        regex: '\b[A-Za-z0-9._%+\-]{1,64}@[A-Za-z0-9\-]{1,63}' \
+               '(?:\.[A-Za-z0-9\-]{1,63}){0,10}\.[A-Za-z]{2,24}\b',
+      }.freeze,
+      # Group 1 is the number; the leading alternation is a consumed boundary
+      # guard (no lookbehind — rule-49 discipline). Separators are space/dash
+      # only, so dotted versions (2024.10.05) and clock times (12:30:45)
+      # never trip.
+      "phone" => {
+        placeholder: "[PHONE]", group: 1,
+        regex: '(?:\A|[^0-9A-Za-z])((?:\+[0-9]{1,3}[ \-]?)?' \
+               '(?:\([0-9]{1,4}\)[ \-]?|[0-9]{1,4}[ \-])' \
+               '[0-9]{3,4}[ \-][0-9]{3,4})(?![0-9])',
+      }.freeze,
+      # Delimited 3-2-4 with a CONSISTENT delimiter (the \1 backreference —
+      # a one-char group, linear by construction) — an undelimited 9-digit
+      # run never matches. Area 000/666/9xx, group 00, and serial 0000 are
+      # structurally invalid SSNs and ride raw.
+      "ssn" => {
+        placeholder: "[SSN]", group: 0,
+        regex: '\b(?!000|666|9[0-9]{2})[0-9]{3}([ \-])' \
+               '(?!00)[0-9]{2}\1(?!0000)[0-9]{4}\b',
+      }.freeze,
+      # 13-19 digits in 4-4-4-rest grouping (space/dash/contiguous). The
+      # regex is deliberately loose on grouping; the LUHN filter is the
+      # normative gate (fixture "checksum" rule).
+      "credit_card" => {
+        placeholder: "[CREDIT_CARD]", group: 0, filter: :luhn,
+        regex: '\b[0-9]{4}(?:[ \-]?[0-9]{4}){2}[ \-]?[0-9]{1,7}\b',
+      }.freeze,
+      # IPv4 with per-octet range validation (999.1.1.1 and 4-digit octets
+      # ride raw), full 8-group IPv6, and ::-compressed IPv6.
+      "ip" => {
+        placeholder: "[IP]", group: 0,
+        regex: '\b(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}' \
+               '(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\b' \
+               '|\b(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}\b' \
+               '|\b(?:[0-9A-Fa-f]{1,4}:){1,6}:' \
+               '(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,5})?(?![0-9A-Fa-f:])',
+      }.freeze,
+    }.freeze
   end
 end

data/lib/foam/otel/ingest.rb

@@ -38,31 +38,38 @@ module Foam
       # Door-2 traces entry: a SpanProcessor-duck instance for THEIR
       # TracerProvider (their setup block / add_span_processor). Foam's copy
       # is stamped + masked at export; their spans are never touched.
+      # `redact:` is the grouped redaction object (fleet ruling 2026-07-28,
+      # same shape and union semantics as init's — secrets ∪ redact_keys,
+      # pii ∪ redact_pii_keys, detect = the opt-in PII detection tier),
+      # scoped to this tap like the legacy kwargs.
       def create_ingest_span_processor(token:, environment:, redact_keys: nil,
-                                       redact_pii_keys: nil, diagnostics: false,
+                                       redact_pii_keys: nil, redact: nil, diagnostics: false,
                                        secret_heuristics: true)
         Ingest.build(:traces, token: token, environment: environment,
                              redact_keys: redact_keys, redact_pii_keys: redact_pii_keys,
-                             diagnostics: diagnostics, secret_heuristics: secret_heuristics)
+                             redact: redact, diagnostics: diagnostics,
+                             secret_heuristics: secret_heuristics)
       end
 
       # Door-2 logs entry: a LogRecordProcessor duck for THEIR LoggerProvider.
       def create_ingest_log_record_processor(token:, environment:, redact_keys: nil,
-                                             redact_pii_keys: nil, diagnostics: false,
+                                             redact_pii_keys: nil, redact: nil, diagnostics: false,
                                              secret_heuristics: true)
         Ingest.build(:logs, token: token, environment: environment,
                             redact_keys: redact_keys, redact_pii_keys: redact_pii_keys,
-                            diagnostics: diagnostics, secret_heuristics: secret_heuristics)
+                            redact: redact, diagnostics: diagnostics,
+                            secret_heuristics: secret_heuristics)
       end
 
       # Door-2 metrics entry: a PeriodicMetricReader subclass for THEIR
       # MeterProvider#add_metric_reader (the back-filling attach point).
       def create_ingest_metric_reader(token:, environment:, redact_keys: nil,
-                                      redact_pii_keys: nil, diagnostics: false,
+                                      redact_pii_keys: nil, redact: nil, diagnostics: false,
                                       secret_heuristics: true)
         Ingest.build(:metrics, token: token, environment: environment,
                                redact_keys: redact_keys, redact_pii_keys: redact_pii_keys,
-                               diagnostics: diagnostics, secret_heuristics: secret_heuristics)
+                               redact: redact, diagnostics: diagnostics,
+                               secret_heuristics: secret_heuristics)
       end
     end
 
@@ -112,7 +119,7 @@ module Foam
 
       class << self
         def build(signal, token:, environment:, redact_keys:, redact_pii_keys:, diagnostics:,
-                  secret_heuristics: true)
+                  secret_heuristics: true, redact: nil)
           # (1) the operator kill switch — read once at construction, exactly
           # like init (rule 19 / fleet audit R13). Trimmed + case-insensitive
           # so the fleet's one emergency switch behaves the same across
@@ -130,6 +137,12 @@ module Foam
           entry = ENTRY_NAMES.fetch(signal)
           validate_present!(entry, :token, token)
           validate_present!(entry, :environment, environment)
+          # The grouped `redact:` object validates HERE, in the loud step-2
+          # block (fleet ruling 2026-07-28, same rule-10 posture as init): an
+          # unknown field, a non-Array value, or an unknown detect entity
+          # raises ArgumentError at construction — never a degraded-to-inert
+          # boot the rescue below would swallow.
+          Foam::Otel.send(:parse_redact!, redact, context: "[foam] #{entry}")
           Foam::Otel.send(:warn_environment, environment)
 
           # (3) build. After validation NOTHING ever throws (rule 9): a broken
@@ -146,7 +159,7 @@ module Foam
             Foam::Otel.send(:warn_inert_per_signal_endpoints, endpoint)
             config = Foam::Otel.resolve_config(
               name: "", environment: environment.to_s, version: nil, enabled: true,
-              redact_keys: redact_keys, redact_pii_keys: redact_pii_keys,
+              redact_keys: redact_keys, redact_pii_keys: redact_pii_keys, redact: redact,
               ignored_outbound_hosts: [], diagnostics: diagnostics ? true : false,
               endpoint: endpoint, secret_heuristics: secret_heuristics
             )

data/lib/foam/otel/init.rb

@@ -49,7 +49,7 @@ module Foam
     class << self
       # rubocop:disable Metrics/MethodLength, Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
       def init(name:, environment:, enabled:, token: nil, version: nil,
-               redact_keys: nil, redact_pii_keys: nil,
+               redact_keys: nil, redact_pii_keys: nil, redact: nil,
                additional_instrumentations: nil,
                additional_span_processors: nil,
                additional_log_record_processors: nil,
@@ -78,10 +78,13 @@ module Foam
 
         endpoint = resolve_endpoint
         warn_inert_per_signal_endpoints(endpoint)
+        # The grouped `redact:` object (fleet ruling 2026-07-28) resolves here
+        # too — an unknown field, a non-Array value, or an unknown detect
+        # entity raises INSIDE resolve_config, loudly, at boot (rule 10).
         config = resolve_config(
           name: name, environment: environment, version: version,
           enabled: effective_enabled,
-          redact_keys: redact_keys, redact_pii_keys: redact_pii_keys,
+          redact_keys: redact_keys, redact_pii_keys: redact_pii_keys, redact: redact,
           ignored_outbound_hosts: ignored_outbound_hosts,
           diagnostics: diagnostics, endpoint: endpoint,
           secret_heuristics: secret_heuristics
@@ -105,7 +108,7 @@ module Foam
         if @initialized
           self.active_config = refresh_post_init_config(
             active_config, redact_keys: redact_keys, redact_pii_keys: redact_pii_keys,
-            diagnostics: diagnostics, secret_heuristics: secret_heuristics
+            redact: redact, diagnostics: diagnostics, secret_heuristics: secret_heuristics
           )
           Diagnostics.warn("init already called — returning the existing instance (providers are process-global; " \
                            "enabled and endpoint cannot change after the first init — only the redaction lists refresh)")

data/lib/foam/otel/redaction.rb

@@ -458,6 +458,127 @@ module Foam
         true
       end
 
+      # ---- THE OPT-IN PII DETECT TIER (fleet ruling 2026-07-28) --------------
+      # docs/decisions/redact-object-detect-design.md §3; fixture:
+      # contract/pii-detect.json. The THIRD value control at this seam, and the
+      # first that is FULLY OPT-IN: an entity from the frozen table
+      # (constants.rb PII_DETECT_ENTITIES) detects ONLY when its name is in the
+      # config's redact_detect list. It composes immediately AFTER the secret
+      # value layer and BEFORE customer key masking, masks ONLY the matched
+      # character span with the entity's TYPED placeholder ([EMAIL], [PHONE],
+      # [SSN], [CREDIT_CARD], [IP] — surrounding text preserved verbatim), and
+      # runs everywhere the secret layer runs — same exemption (metric
+      # datapoint attributes ride the value_scan: false path) and the same
+      # execution discipline: compile-once (per-Regexp timeout on Ruby >= 3.2),
+      # over-cap values are NEVER regex-scanned (the secret layer's anchor scan
+      # already made the oversize mask-whole/pass decision — detect adds
+      # nothing), a match flood masks the whole value, and ANY throw inside the
+      # detector masks the WHOLE value to [REDACTED] with a loud [foam]
+      # warning — never the raw value, never a crash (fixture "fail_closed").
+
+      # True only when the customer opted into at least one detect entity —
+      # the zero-cost gate for the byte-identical default path.
+      def detect_enabled?(config)
+        return false if config.nil?
+        return false unless config.respond_to?(:redact_detect)
+
+        list = config.redact_detect
+        !list.nil? && !list.empty?
+      rescue StandardError
+        false
+      end
+
+      # THE detect-tier entrypoint. Same guard shape as scan_value_secrets:
+      # scan a scrubbed COPY of an invalid-encoding value; when nothing
+      # matches, the ORIGINAL object rides through (raw capture stands,
+      # identity-preserving).
+      def detect_pii(text, config)
+        return text unless text.is_a?(String) && !text.empty?
+
+        entities = config.redact_detect
+        work = text.valid_encoding? ? text : scrub_utf8(text)
+        work = text unless work.is_a?(String)
+        result = detect_pii_unguarded(work, entities)
+        result.equal?(work) ? text : result
+      rescue StandardError, SystemStackError
+        # Fail closed (fixture "fail_closed"): a detector fault ships the
+        # mask, NEVER the raw value.
+        warn_secret_scan_once(:pii_fault, "PII detection failed for a value — masked whole (fail-closed)")
+        REDACTED
+      end
+
+      def detect_pii_unguarded(text, entities)
+        # Over-cap follows the secret layer's over-cap rule (fixture "caps"):
+        # NEVER regex-scanned. The secret layer's anchor scan already ran and
+        # decided mask-whole vs pass for oversized values; detect adds nothing.
+        return text if text.length > SECRET_VALUE_SCAN_CAP
+
+        spans = []
+        PII_DETECT_COMPILED.each do |name, entity|
+          next unless entities.include?(name)
+
+          group = entity[:group]
+          text.scan(entity[:re]) do
+            m = Regexp.last_match
+            cand = m[group]
+            next if cand.nil? || !detect_filter_ok?(entity[:filter], cand)
+
+            spans << [m.begin(group), m.end(group), entity[:placeholder]]
+            if spans.length > SECRET_MAX_MATCHES
+              # match flood ⇒ whole value masked (fail-closed, secret-layer parity)
+              warn_secret_scan_once(:pii_flood,
+                                    "PII detection exceeded the match-flood cap — masked whole (fail-closed)")
+              return REDACTED
+            end
+          end
+        end
+        return text if spans.empty?
+
+        apply_detect_spans(text, spans)
+      end
+
+      def detect_filter_ok?(filter, cand)
+        return true unless filter == :luhn
+
+        digits = cand.delete("^0-9")
+        digits.length.between?(13, 19) && luhn_valid?(digits)
+      end
+
+      # The Luhn checksum over a plain digit string. credit_card is masked
+      # ONLY when this passes (fixture "checksum" rule) — a card-shaped number
+      # failing Luhn (a tracking id, a serial) is never masked.
+      def luhn_valid?(digits)
+        sum = 0
+        digits.reverse.each_char.with_index do |ch, i|
+          d = ch.ord - 48
+          if i.odd?
+            d *= 2
+            d -= 9 if d > 9
+          end
+          sum += d
+        end
+        (sum % 10).zero?
+      end
+
+      # Replace each detected span with ITS entity's typed placeholder, left
+      # to right; overlaps resolved by earliest-start-longest-match (the
+      # secret layer's span-merge rule, fixture "span_masking"). Emitted
+      # placeholders are never rescanned — running detection twice is
+      # byte-identical (fixture "placeholder_idempotence").
+      def apply_detect_spans(text, spans)
+        spans.sort_by! { |s, e, _| [s, -e] }
+        out = +""
+        prev = 0
+        spans.each do |s, e, placeholder|
+          next if s < prev # overlapped by an earlier-start/longer match — dropped
+
+          out << text[prev...s] << placeholder
+          prev = e
+        end
+        out << text[prev..]
+        out
+      end
+
       # Rate-limited (once per process per class) scanner warnings — loud
       # (rule 15) but never a log flood an adversary controls.
       def warn_secret_scan_once(key, message)
@@ -525,14 +646,16 @@ module Foam
       end
 
       # A redact_keys match: the value-pattern scan runs FIRST (design §V.0 —
-      # a shaped secret must never survive as a 4-char tail reveal); a V hit
-      # is terminal for its span. Otherwise: tail-mask a scalar leaf, redact
-      # anything else in full.
+      # a shaped secret must never survive as a 4-char tail reveal), then the
+      # opt-in detect tier (2026-07-28 ruling: detect sits BETWEEN the secret
+      # layer and customer key masking); a V or detect hit is terminal for its
+      # span. Otherwise: tail-mask a scalar leaf, redact anything else in full.
       def masked_value_for(value, config, value_scan)
         return value.map { REDACTED } if value.is_a?(Array)
 
         if value_scan && value.is_a?(String)
           scanned = scan_value_secrets(value, heuristics_enabled?(config))
+          scanned = detect_pii(scanned, config) if detect_enabled?(config)
           return scanned unless scanned == value
         end
         scalar_or_full(value)
@@ -566,12 +689,16 @@ module Foam
       # query AND fragment, per-pair value scan), (2) the whole-string
       # value-pattern scan for shaped secrets outside k=v pairs (a bearer
       # token in an exception message, a PEM block, a bare JWT). [REDACTED]
-      # emitted by (1) never re-matches in (2).
+      # emitted by (1) never re-matches in (2). The OPT-IN detect tier (fleet
+      # ruling 2026-07-28) composes THIRD, after the secret layer — its typed
+      # placeholders never weaken (1)/(2), and with no opted-in entities it is
+      # a zero-cost no-op (the byte-identical default path).
       def redact_leaf_string(text, config)
         out = scan_value_secrets(redact_url_like(text, config), heuristics_enabled?(config))
-        # Raw capture stands byte-identical: when neither pass changed
-        # anything, the ORIGINAL object rides through (content- and
-        # identity-preserving, pinned by the floor negative controls).
+        out = detect_pii(out, config) if detect_enabled?(config)
+        # Raw capture stands byte-identical: when no pass changed anything,
+        # the ORIGINAL object rides through (content- and identity-preserving,
+        # pinned by the floor negative controls).
         out == text ? text : out
       end
 
@@ -928,6 +1055,14 @@ module Foam
       SECRET_H2_HEX_RE = build_secret_regexp(SECRET_HEURISTIC_H2[:hex_regex], false)
       SECRET_PLACEHOLDER_RES = SECRET_VALUE_PLACEHOLDERS.map { |p| build_secret_regexp(p, true) }.freeze
 
+      # The detect tier's compiled view of the frozen fixture table (fleet
+      # ruling 2026-07-28) — same compile-once discipline (and the same
+      # per-Regexp timeout belt) as the secret rules above: a pattern that
+      # fails to compile fails the require LOUDLY, never a silent skip.
+      PII_DETECT_COMPILED = PII_DETECT_ENTITIES.transform_values do |entity|
+        entity.merge(re: build_secret_regexp(entity[:regex], false)).freeze
+      end.freeze
+
       # Every lowercase literal anchor across Tier 1 + the linear scans + H1 —
       # the single pre-filter deciding whether ANY regex runs (and the whole
       # decision for oversized values).

data/lib/foam/otel/version.rb

@@ -46,6 +46,21 @@ module Foam
     #   exporter — OTEL_RUBY_EXPORTER_OTLP_SSL_VERIFY_NONE is now INERT and
     #   loudly warned about (F-RB1, CWE-295). If a dashboard keyed off a raw
     #   token value (it should not have), it will now see [REDACTED].
-    VERSION = "1.4.0"
+    # 1.5.0: `redact:` OBJECT + OPT-IN PII DETECT TIER (fleet ruling
+    #   2026-07-28, docs/decisions/redact-object-detect-design.md; MINOR —
+    #   additive, opt-in, zero default-path behavior change). init() and all
+    #   three door-2 ingest entries gain ONE additive option, redact: — a Hash
+    #   grouping the two key lists under intent-named fields (secrets: ==
+    #   redact_keys, pii: == redact_pii_keys; the legacy kwargs live on
+    #   unchanged and UNION when both are set) plus a new opt-in detect: list
+    #   of PII entity names (contract/pii-detect.json: email, phone, ssn,
+    #   credit_card, ip) whose value shapes are masked in every exported
+    #   string with typed placeholders ([EMAIL], [PHONE], [SSN],
+    #   [CREDIT_CARD], [IP]) — span-only, Luhn-gated credit_card, idempotent
+    #   over placeholders, same caps/fail-closed discipline and the same
+    #   metric-label exemption as the secret layer. Unknown redact fields,
+    #   non-Array values, and unknown detect entities raise at init (loud at
+    #   boot). With redact: absent, behavior is byte-identical to 1.4.0.
+    VERSION = "1.5.0"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: foam-otel
 version: !ruby/object:Gem::Version
-  version: 1.4.0
+  version: 1.5.0
 platform: ruby
 authors:
 - Foam