foam-otel 1.5.0 → 1.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1e23dd37962a70ea36799a92d914fd2a38aa967a6cd1aface19bbaa322ab8966
-  data.tar.gz: f0018c27fb2dfff9316118438688a4a8ab2baea18810dbda522880361dfe6681
+  metadata.gz: 48166d05b120dd8bdca15a9b45db5b4c16a6574fcb6890f4caff2c17c2ae9add
+  data.tar.gz: 2dcd4b3b146a0221c231ba5efc057085418b74e21b24d8441396d47cfc0a8288
 SHA512:
-  metadata.gz: 9731cb1fae1b54f236dd583102b2348ab4b45bff38745f446c584151e594ea685f2689d6a7fb4807136372006b7b10c0284b2e7c140c0c9625edb5a820294c8a
-  data.tar.gz: 91744d9fb0fb671bce62de973eec96152b147742ee5948fedf607626dfd89b8c3c6ed055522f21c4bbfd7d65ce5b558da9e67605578e3573ec15f8b0d4efeecc
+  metadata.gz: 4e92c7af8cc5cf7cc177b792f87a8107b918035efdce080e2a974750180ebbb9aac4a5460cb08372f3ded2724d136804bcb8f0574ee4685ebbdd94fde7afb3e8
+  data.tar.gz: ddd7b6e431034663ec94dacd8b7b051ee63bfecdbeb2a38d383c58531854ba0580f34c9a04ed56473b654b89155fc9a3b7ce140b073ba346d90a5a801b9c6d4b

data/GOTCHAS.md

@@ -633,6 +633,179 @@ exfiltratable — the value-pattern secret layer is the required second control
   `spec/vuln_suite_spec.rb` (the adversarial exfil/ReDoS/bypass suite +
   mutation spot-checks).
 
+## F14: Default-on header capture — the official rack options are the seam (a NAMED list, not capture-all), operator header config is never overridden, and Faraday middleware ORDER decides which span gets enriched
+
+- **Trap**: four ways foam's default-on header capture
+  (`header_capture.rb`) could go wrong. (1) The official rack gem's
+  `allowed_request_headers:`/`allowed_response_headers:` options are a
+  STATIC ALLOWLIST compiled at install time with no capture-all form — so
+  Ruby inbound cannot be capture-all like js/python; pretending otherwise
+  (or shipping an empty default) silently costs the whole signal. (2) The
+  gem applies operator env config
+  (`OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS`,
+  instrumentation-base `config_overrides_from_env`) per option at install,
+  where an env value BEATS any passed config — except the narrow-to-empty
+  spelling (`allowed_request_headers=`), which the gem's env parser DROPS
+  (`parts[1]` nil): foam blindly passing its defaults would then override
+  the operator's explicit opt-out. And `Instrumentation::Base#install` is
+  first-wins (`return true if installed?`) — foam pre-installing BEFORE the
+  operator's explicit instance would freeze the operator's header config
+  out entirely. (3) A middleware writing to
+  `OpenTelemetry::Trace.current_span` writes to WHATEVER span is ambient:
+  foam's Faraday middleware registered without `use :open_telemetry` FIRST
+  (the official faraday instrumentation auto-appends its tracer middleware
+  AFTER the customer's block, i.e. INSIDE any middleware the block added)
+  would stamp `http.request.header.*` onto the enclosing app span,
+  silently corrupting another producer's data. (4) Outbound Net::HTTP has
+  NO hook surface in its official instrumentation (a closed
+  `connect`/`request` patch, no request/response hook, no header option) —
+  the only route would be monkey-patching stdlib or gem internals.
+- **Sources**: installed source —
+  opentelemetry-instrumentation-rack-0.31.1 `instrumentation.rb:24-25`
+  (allowlist options), `instrumentation.rb:108-127` (compiled at install:
+  enumerate-up-front, no capture-all; `build_attribute_name` folds `-` to
+  `_`, so the emitted suffixes are underscored);
+  opentelemetry-instrumentation-base-0.26.1 `base.rb:218-227` (install is
+  first-wins/idempotent), `base.rb:274-313` (env override beats passed
+  config per option), `base.rb:343-363` (the `;`-separated `name=value`
+  env grammar; a valueless `name=` is dropped);
+  opentelemetry-instrumentation-faraday-0.33.0 `instrumentation.rb` (no
+  header option; `patches/stable/connection.rb:18-25` — the auto-insert
+  appends `use(:open_telemetry)` AFTER the customer's builder block) and
+  `middlewares/stable/tracer_middleware.rb:27-45` (the span is active only
+  around the handlers BELOW the tracer middleware);
+  opentelemetry-instrumentation-net_http-0.29.0 `patches/instrument.rb`
+  (no extension point).
+- **Decision & why** (user ruling 2026-07-28, superseding the first-pass
+  capture-all Rack middleware): INBOUND capture is the official rack gem's
+  OWN header options, which foam PRE-INSTALLS with documented default
+  lists (`DEFAULT_REQUEST_HEADERS`/`DEFAULT_RESPONSE_HEADERS` — the
+  standard APM set plus the seven floor headers, which arrive `[REDACTED]`
+  so presence stays visible) — the same pre-install shape as the loop
+  guard, ordered AFTER `install_additional` so an operator's explicit rack
+  instance wins entirely, and computed per option AGAINST the operator's
+  env var so an operator-named option is never filled by foam (the F-PY3
+  posture: a narrower operator list is never widened, and `name=` narrows
+  to the gem default `[]`, never to foam's list). The honest consequence
+  is documented, not papered over: a header outside the list is NOT
+  captured (README shows the env-var extension recipe). OUTBOUND stays
+  foam's Faraday middleware (Faraday's first-class public API — the
+  official faraday instrumentation has no header option), capture-all,
+  triple-gated per write: foam owns the traces slot (rule 18 B), the span
+  is recording, AND the span's `instrumentation_scope.name` is
+  "OpenTelemetry::Instrumentation::Faraday" — a mis-ordered stack captures
+  NOTHING instead of enriching the wrong span. Faraday placement stays a
+  per-connection line (`f.use :open_telemetry` then
+  `f.use :foam_otel_headers`): the only default-on mechanism is the
+  Connection prepend the official gem uses — exactly the
+  internals-patching foam never does. Net::HTTP (and Excon / httprb /
+  HTTPX) outbound headers are deliberately OUT OF SCOPE — skipped and
+  documented rather than hacked. Masking stays central: the credential
+  floor covers the emitted `http.{request,response}.header.<name>` forms
+  at the exporter boundary (dash/underscore-normalized, so the rack gem's
+  underscored suffixes match), never a key list in the capture path.
+- **Mitigation**: `lib/foam/otel/header_capture.rb` (the default lists,
+  `preinstall_rack_defaults!` + the operator-precedence parsing, the
+  Faraday middleware + public-registry registration, the shared gates);
+  `lib/foam/otel/init.rb` (`activate_instrumentations` pre-install
+  ordering — after `install_additional`, before `install_all` — gated on
+  foam owning traces; `activate_floor_extensions` Faraday registration).
+- **Test**: `spec/header_capture_spec.rb` — default-on capture both
+  directions on the official rack span, the honest negative (an unlisted
+  header is NOT captured), floor headers arrive `[REDACTED]` on the wire
+  with zero config, multi-value arity, operator env respect (named option
+  wins, narrow-to-empty honored, explicit instance wins entirely),
+  install idempotence (a second sweep never clobbers the config),
+  foreign-traces stand-down, hostile values (invalid UTF-8 / huge /
+  raising readers), Faraday scope/ownership gates + mis-ordered-stack
+  no-op + registration idempotence; `spec/conventions_spec.rb` (rule 21:
+  header capture never grows a `rack.input` body tee).
+
+## F15: Opt-in payload (body) capture must TEE, never consume — and the rack span's lifetime decides when body attributes can land
+
+- **Trap**: five ways foam's opt-in body capture (`payload_capture.rb`,
+  behind `capture_payloads:` — default `:off`) could break the customer or
+  silently lose data. (1) Reading `env['rack.input']` ahead of the app —
+  or rewinding it — breaks Rack 3 apps: inputs are no longer required to
+  be rewindable, and a consumed stream starves the framework's own parser.
+  (2) Buffering the response to capture it breaks streaming (SSE, large
+  downloads), and Rack 3 `#call`-only streaming bodies must not be wrapped
+  at all — defining `#each` on the wrapper would change how the server
+  drives the response. (3) The official rack instrumentation ends its span
+  at DIFFERENT times per mode: the default `Rack::Events` handler finishes
+  it when the response body is CLOSED (`EventedBodyProxy` fires
+  `on_finish` after the inner body's close — so a tee flushing at close
+  writes in time), but the non-events `TracerMiddleware` ends it when
+  `@app.call` RETURNS — attributes flushed from a streamed body's close
+  there would hit an ENDED span (upstream logs a warning per write). (4)
+  An OUTER middleware (`Rack::MethodOverride`) reads the form body,
+  rewinds — or repositions via `seek(0)`/`pos=`, equivalent idioms on the
+  rewindable inputs real servers hand out — and the framework re-reads it:
+  a naive tee (or one that dedupes `rewind` alone) captures and counts it
+  twice. (5) The `:errors` mode needs "an exception was recorded on the
+  span" — re-implementing that check would duplicate
+  `Errors.record_once`'s dedupe registry and drift from it.
+- **Sources**: installed source — rack `body_proxy.rb:28-35` (`close`
+  closes the inner body BEFORE the proxy's block) and `:47-53` (`to_ary`
+  auto-closes), rack `events.rb:140-142` (the body proxy that fires
+  `on_finish` on close); opentelemetry-instrumentation-rack-0.31.1 stable
+  `event_handler.rb:109-117,199-205` (`on_finish`/`detach_context` finish
+  the span at body close) vs stable `tracer_middleware.rb:78-90`
+  (`in_span` ends the span when the block — `@app.call(env).tap` —
+  returns); Rack SPEC (input `gets`/`read`/`each`; `to_ary` bodies;
+  streaming `#call` bodies); `errors.rb:17-43` (the span-carried
+  `@__foam_otel_recorded` ivar registry).
+- **Decision & why**: delegating tees only, on BOTH directions, behind the
+  ONE mode switch. `:off` (the default) ships ZERO middleware (the Railtie
+  hook runs after `:load_config_initializers` — when init has resolved the
+  mode — and stands down; a manual `use` line is a one-branch
+  passthrough). The input tee observes exactly what the APP reads
+  (read/gets/each), delegates everything else with truthful `respond_to?`
+  (a non-rewindable input stays non-rewindable), and dedupes across EVERY
+  repositioning method — `rewind`, `seek`, `pos=` — with a
+  position/high-water scheme — bytes are captured and counted once no
+  matter how often an outer middleware repositions the stream. The response side
+  captures `to_ary`-able (buffered) bodies SYNCHRONOUSLY at middleware
+  return — the span is provably alive in both official modes — returning
+  the array as the new body (the Rack spec's sanctioned move;
+  `BodyProxy#to_ary` closes the original). Genuinely streaming `#each`
+  bodies are teed chunk-by-chunk, unbuffered, flushing attributes on
+  iteration-complete/close: that lands in the default `Rack::Events` mode
+  and is silently skipped (a `recording?` gate, no upstream warn-spam) in
+  `TracerMiddleware` mode — a recorded, documented loss shape, never a
+  broken stream. `#call`-only bodies are never wrapped (sizes ride the
+  declared Content-Length when present). `:errors` reads the SAME ivar
+  registry `Errors.record_once` stamps, through the thin
+  `Errors.recorded?` seam — one registry, no duplicated dedupe — plus the
+  status >= 500 triplet check; an exception raised through the middleware
+  attaches the request side before the IDENTICAL re-raise. Capture is
+  gated on foam OWNING the traces slot (rule 18 B — a foreign SDK's rack
+  span is never enriched) and on a recording span; every path is
+  individually rescued (rule 9). Redaction stays central: JSON-shaped body
+  strings are deep-masked BY FIELD NAME at the exporter boundary
+  (`redaction.rb` `BODY_ATTRIBUTE_NAMES` — floor + customer lists reach
+  inside the body JSON; a cap-truncated body no longer parses, so only the
+  value-shape scans cover it — documented limit).
+- **Mitigation**: `lib/foam/otel/payload_capture.rb` (the whole module:
+  the mode gate + FOAM_CAPTURE_PAYLOADS clamp plumbing, ownership/
+  recording gates, tees, `to_ary`/streaming discrimination,
+  textual/identity classification, the 8192-char cap + fleet truncation
+  marker); `lib/foam/otel/init.rb` (option validation + env clamp,
+  mode-gated Railtie-fallback wiring); `lib/foam/otel/errors.rb`
+  (`recorded?`); `lib/foam/otel/redaction.rb` (`mask_body_attribute`).
+- **Test**: `spec/payload_capture_spec.rb` — all three modes on the real
+  redacting wire path (`:off` ships zero middleware and zero attrs;
+  `:errors` attaches on exception AND 5xx and on the record_exception ivar
+  seam, NOT on 200/404; `:always` attaches on 200), env-clamp precedence
+  both directions + invalid-env fallback, the streaming-body proof (chunks
+  intact, close preserved, attributes landed), the rewind/seek/pos=
+  re-read dedupe, the
+  never-reads case, cap/true-size/truncation, binary/gzip sizes-only,
+  concurrent-request isolation, the foreign-provider and non-recording
+  no-ops, the hostile-input battery, the error-path capture, and the
+  JSON-body deep-redaction wire proof; `spec/conventions_spec.rb` (rule 21
+  as amended: `rack.input` touched ONLY by payload_capture.rb).
+
 ---
 
 ## General gotchas (applicable to Ruby)

data/README.md

@@ -33,8 +33,11 @@ gem "foam-otel"
 # Faraday), the primary datastores (pg, mysql2, redis, mongo) with RAW
 # db.statement capture, Sidekiq, the stdlib Logger bridge, hand-written Ruby
 # runtime + GC metrics, session stitching (browser session.id via baggage),
-# and the LLM shims (OpenAI, Anthropic, Gemini, ruby_llm — activity + raw
-# content). Add only the niche long tail your app needs beyond the floor:
+# default-on HTTP header capture (a documented default header list on the
+# official rack server span; every Faraday client header — see "Header
+# capture" below), and the LLM shims
+# (OpenAI, Anthropic, Gemini, ruby_llm — activity + raw content). Add only
+# the niche long tail your app needs beyond the floor:
 gem "opentelemetry-instrumentation-graphql"  # e.g. GraphQL
 gem "opentelemetry-instrumentation-resque"   # e.g. Resque
 ```
@@ -131,6 +134,184 @@ baggage is absent. **CORS note (the FDE wires this on every frontend-called
 API):** the allowlist must include BOTH headers —
 `Access-Control-Allow-Headers: traceparent, baggage`.
 
+### Header capture — request + response headers on the official spans
+
+**Inbound (default-on, no init option):** foam configures the OFFICIAL
+`opentelemetry-instrumentation-rack` gem's own header options
+(`allowed_request_headers:` / `allowed_response_headers:`) with a
+documented **default list** at init, so the rack SERVER span carries the
+listed request and response headers as
+`http.request.header.<name>` / `http.response.header.<name>` attributes
+(the gem's form: lowercase with `-` folded to `_`, e.g.
+`http.request.header.x_request_id`), on error requests too.
+
+> **Honest limit (deliberate divergence from foam's js/python cores):**
+> Ruby inbound captures the **named default list below, NOT all headers** —
+> the official gem's options are an enumerated allowlist with no
+> capture-all form, and that official option is the chosen seam (ruling
+> 2026-07-28). A custom header outside the list is **not captured** unless
+> you extend the list (recipe below).
+
+The default lists (`Foam::Otel::HeaderCapture::DEFAULT_REQUEST_HEADERS` /
+`DEFAULT_RESPONSE_HEADERS` — frozen, documented constants):
+
+- **request** — `content-type`, `content-length`, `content-encoding`,
+  `accept`, `accept-charset`, `accept-encoding`, `accept-language`,
+  `user-agent`, `referer`, `origin`, `host`, `cache-control`, `pragma`,
+  `if-none-match`, `if-modified-since`, `range`, `via`, `forwarded`,
+  `x-forwarded-for`, `x-forwarded-proto`, `x-forwarded-host`,
+  `x-forwarded-port`, `x-real-ip`, `x-request-id`, `x-correlation-id`,
+  plus the request half of the credential floor — `authorization`,
+  `proxy-authorization`, `cookie`, `x-api-key`, `x-auth-token`;
+- **response** — `content-type`, `content-length`, `content-encoding`,
+  `content-language`, `content-range`, `cache-control`, `pragma`,
+  `expires`, `age`, `etag`, `last-modified`, `vary`, `location`,
+  `retry-after`, `x-request-id`, `x-correlation-id`, `x-runtime`,
+  `server-timing`, plus the response half of the floor — `set-cookie`,
+  `www-authenticate`.
+
+Propagation headers (`traceparent`, `tracestate`, `baggage`) are
+deliberately not listed — they are extracted as span context, not captured
+as attributes; hop-by-hop plumbing (`connection`, `keep-alive`, …) is
+skipped too.
+
+**Extending (or narrowing) the list — the gem's own standard env var,
+always respected, never overridden by foam:**
+
+```bash
+# The value REPLACES the list for that option — name every header you want:
+export OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS='allowed_request_headers=content-type,accept,x-request-id,x-tenant-id'
+```
+
+Precedence: (1) an explicit rack instrumentation instance you pass via
+`additional_instrumentations:` installs first and its header config wins
+entirely; (2) any header option you set in
+`OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS` governs that option (foam's
+default fills only the option you did not touch — a narrower operator list
+is never widened, even the narrow-to-empty `allowed_request_headers=`
+spelling); (3) otherwise foam's defaults apply. Foam never installs the
+rack instrumentation (or its header config) when a foreign SDK owns the
+traces slot.
+
+**Floor masking, zero config:** the always-on credential floor masks the
+seven credential headers (`authorization`, `proxy-authorization`, `cookie`,
+`set-cookie`, `x-api-key`, `x-auth-token`, `www-authenticate`) to
+`[REDACTED]` in the emitted attribute forms (dash/underscore-normalized
+match) — they ride the default lists deliberately so their PRESENCE is
+visible, value never (see "The default credential denylist";
+`x-forwarded-for` / `x-real-ip` are on the 52-entry key floor and arrive
+`[REDACTED]` too). **Masking further captured headers is redaction's job,
+not a capture switch:** list the emitted name form in `redact_keys` /
+`redact: { secrets: [...] }` (tail mask) or `redact_pii_keys` /
+`redact: { pii: [...] }` (full `[REDACTED]`) — the substring key match
+reaches the attribute names (inbound names are underscored by the rack gem,
+so list e.g. `x_runtime`; Faraday's outbound names keep dashes).
+
+**Outbound (Faraday):** the official faraday instrumentation has no header
+option at all, so outbound stays foam's own Faraday middleware (its
+first-class public extension API), enriching the official CLIENT span with
+**every** request header actually sent (the injected `traceparent`
+included) and every response header received — string-array attributes,
+dash-form names. It never creates a span and stands down when foam is
+disabled/killed, when a foreign SDK owns traces, or when the current span
+is not the official faraday instrumentation's own (a mis-ordered stack
+captures nothing rather than writing onto the wrong span). One line per
+connection (Faraday has no public add-to-every-connection hook — the
+middleware name is registered by init, placement is yours; ORDER MATTERS,
+the official middleware first):
+
+```ruby
+conn = Faraday.new(url: "https://partner-api.example") do |f|
+  f.use :open_telemetry     # the official client span (explicit, so it sits OUTSIDE)
+  f.use :foam_otel_headers  # foam's header capture, directly inside it
+end
+```
+
+**Deliberately out of scope** (no standard extension point exists —
+skipped, not monkey-patched): outbound **Net::HTTP / Excon / HTTP (httprb)
+/ HTTPX** headers (their official instrumentations expose no
+request/response hook or header option) and gRPC metadata. Outbound header
+coverage today is Faraday. **Bodies/payloads are not captured by header
+capture** — bodies are the separate, opt-in `capture_payloads:` capability
+below.
+
+### Payload (body) capture — opt-in, `capture_payloads:`
+
+**Off by default.** One init option turns on inbound HTTP **body** capture
+on the official rack SERVER span (headers stay the header-capture seam
+above — this never duplicates them):
+
+```ruby
+Foam::Otel.init(
+  name: "checkout-api", environment: ENV.fetch("APP_ENV"),
+  enabled: ENV.fetch("APP_ENV") == "production", token: ENV.fetch("FOAM_OTEL_TOKEN"),
+  capture_payloads: :errors # :off (default) | :errors | :always — equivalent strings accepted
+)
+```
+
+- **`:off` (default)** — zero body teeing, zero per-request allocation, and
+  **zero middleware shipped**: with the default, foam's Rails wiring
+  installs nothing at all.
+- **`:errors`** — bodies are teed per request but attributes attach to the
+  span **only when the request errored**: `Foam::Otel.record_exception`
+  called while the rack server span is current, an exception raised through
+  the middleware, or a response status **>= 500**. A clean 2xx/4xx
+  attaches **nothing**.
+- **`:always`** — attach on every request.
+
+An invalid option value raises `ArgumentError` at boot. The operator env
+clamp **`FOAM_CAPTURE_PAYLOADS=off|errors|always` overrides the option in
+both directions** (force payloads off on a misbehaving deploy, or force
+them on without a code change — one loud `[foam]` line when it changes the
+mode); an invalid env value warns and falls back to the option, never a
+crashed boot.
+
+**What lands on the span** (in `:always`, or in `:errors` on an errored
+request): `http.request.body` / `http.response.body` for **textual**
+payloads (`text/*`, JSON/`+json`, urlencoded forms, XML/`+xml`, GraphQL;
+skipped when `Content-Encoding` is not identity), capped at **8192 chars**
+with an `…[truncated]` marker, plus `http.request.body.size` /
+`http.response.body.size` — **true byte sizes** when known (declared
+`Content-Length`, else the bytes actually observed). Non-textual payloads
+contribute sizes only.
+
+**Stream safety (GOTCHAS F15):** bodies are TEED, never consumed — foam
+records exactly what your app reads from `rack.input` (an app that never
+reads its body captures nothing; a re-read form body — rewound, or
+repositioned via `seek`/`pos=` — is captured and counted once), and
+streaming/SSE responses pass through
+unbuffered chunk-by-chunk with `close` preserved. Rack 3 `#call`-only
+streaming bodies are never wrapped. The request and response are never
+altered, and an app exception re-raises identically.
+
+**Redaction stays central, on export:** a JSON-shaped body is
+**deep-redacted by field name** at the exporter boundary — the credential
+floor (`password`, `authorization`, …) and your `redact_keys` /
+`redact: {secrets:/pii:}` lists reach *inside* the body JSON; urlencoded
+bodies get their `k=v` pair names masked by the same tokenizer that covers
+query strings; and the value-pattern secret layer scans all captured body
+text. (A body truncated at the cap no longer parses as JSON, so field-name
+deep-redaction cannot apply to it — the value-shape scans still run.)
+
+**Wiring:** in **Rails**, automatic when the mode is not `:off` (foam's
+railtie inserts `Foam::Otel::PayloadCapture` directly inside the official
+rack middleware after your initializers run; when foam-otel loads before
+rails, `init` wires it instead, as long as init runs during boot — the
+standard `config/initializers/foam.rb` recipe). **Plain Rack / Sinatra**
+apps add one line under the official middleware:
+
+```ruby
+# config.ru
+use(*OpenTelemetry::Instrumentation::Rack::Instrumentation.instance.middleware_args)
+use Foam::Otel::PayloadCapture   # opt-in body capture (inert while capture_payloads is :off)
+run MyApp
+```
+
+The middleware enriches only: it never creates a span, and it stands down
+per request when foam is disabled/killed, when the span is not recording,
+or when a foreign SDK owns the traces slot. Outbound (client) bodies are
+not captured — inbound only.
+
 ---
 
 ## The `init` options
@@ -150,6 +331,7 @@ API):** the allowlist must include BOTH headers —
 | `additional_metric_readers:` | Array | no | `[]` | Tenant seam, metrics. |
 | `additional_instrumentations:` | Array | no | `[]` | Constructed tier-2 instrumentation instances to register (fault-isolated: one that throws is skipped with a `[foam]` warning). |
 | `ignored_outbound_hosts:` | Array<String> | no | `[]` | Hosts whose outbound calls produce no spans — EXTENDS the built-in export-loop guard. For a co-resident agent's intake or a tenant exporter's endpoint. Applies to the clients whose official instrumentation supports host suppression: **Net::HTTP and Excon**. Faraday/HTTP (httprb)/HTTPX have no upstream `untraced_hosts` option — foam warns loudly at init when one of those is bundled (GOTCHAS G13); foam's own export loop is guarded for every client regardless. |
+| `capture_payloads:` | Symbol/String | no | `:off` | Inbound HTTP **body** capture on the rack server span (see "Payload (body) capture"). `:off` = zero teeing, zero middleware (the default); `:errors` = tee per request, attach only on exception/5xx; `:always` = attach on every request. Equivalent strings accepted; anything else raises at boot. Overridden in both directions by the `FOAM_CAPTURE_PAYLOADS` env clamp (env table below). Textual payloads only, 8192-char cap + `…[truncated]`, true `.size` attributes; sizes-only for binary/compressed. |
 | `diagnostics:` | Boolean | no | false | Verbose `[foam]` self-reporting of init/health. Warnings and errors are always loud regardless. |
 | `secret_heuristics:` | Boolean | no | true | The value-pattern secret layer's HEURISTIC tier (generic keyword+entropy detection, see "The value-pattern secret layer"). `false` disables ONLY the heuristics — the named provider patterns, the credential floor and the redaction-coverage contract have no off switch. Disabling logs one loud `[foam]` line at init (an explicit, audited decision for telemetry whose legitimate values collide with the heuristics). |
 
@@ -823,6 +1005,8 @@ end
 | `OTEL_EXPORTER_OTLP_ENDPOINT` | HONORED — the one operator-level override of the pinned fleet endpoint (for foam's own conformance rig / enterprise egress). Active → loud `[foam]` warning naming the destination. Applies to door-2 taps identically (and moves the host the required loop step must name). |
 | `OTEL_PROPAGATORS=none` | HONORED — turns trace propagation OFF (links lost) while telemetry keeps flowing; warns. Any other value warns and is ignored (foam's propagator set is fixed: W3C tracecontext + baggage). |
 | `OTEL_BSP_SCHEDULE_DELAY` / `OTEL_BLRP_SCHEDULE_DELAY` / `OTEL_METRIC_EXPORT_INTERVAL` | HONORED — batch cadence, read natively by the upstream SDK. |
+| `FOAM_CAPTURE_PAYLOADS` | HONORED — the operator clamp over the `capture_payloads:` init option (`off`/`errors`/`always`, case-insensitive), overriding it in BOTH directions with one loud `[foam]` line when it changes the mode. An invalid value warns and falls back to the init option (never crashes a boot). This is the ONE foam-named env var the gem reads — an override valve over an init-declared option, never an on/off switch, token, or config fallback (those still arrive only through `init`'s explicit arguments). Read once at init; changing it implies a restart. |
+| `OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS` | HONORED (by the contrib rack gem itself) and RESPECTED by foam — a header option you set here (`allowed_request_headers=…` / `allowed_response_headers=…`) governs that option; foam's default header list fills only the option you did not touch (never widened, never overridden — see "Header capture"). The other `OTEL_RUBY_INSTRUMENTATION_<NAME>_CONFIG_OPTS` vars are likewise the contrib gems' own standard levers (e.g. the Sidekiq `propagation_style` note above). |
 | `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | INERT — foam wires ONE resolved endpoint into all three exporters explicitly, so per-signal endpoint vars never redirect (or split) foam's export. Set → loud `[foam]` warning that it is inert. |
 | `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | INERT — as above (warns when set). |
 | `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | INERT — as above (warns when set). |

data/lib/foam/otel/config.rb

@@ -12,7 +12,7 @@ module Foam
     Config = Struct.new(
       :name, :environment, :version, :enabled,
       :redact_keys, :redact_pii_keys, :redact_detect, :ignored_outbound_hosts,
-      :diagnostics, :endpoint, :secret_heuristics,
+      :diagnostics, :endpoint, :secret_heuristics, :capture_payloads,
       keyword_init: true
     )
 
@@ -22,6 +22,14 @@ module Foam
     # opts into the PII detection tier. Anything else raises at boot.
     REDACT_OPTION_FIELDS = %w[secrets pii detect].freeze
 
+    # The exact modes the `capture_payloads:` init option accepts
+    # (payload-capture mandate 2026-07-28; equivalent strings — trimmed,
+    # case-insensitive — are accepted and canonicalized to these symbols).
+    # :off is the default: zero body teeing, zero per-request allocation, no
+    # middleware shipped. The FOAM_CAPTURE_PAYLOADS operator env clamp
+    # (init.rb) accepts the same spellings and overrides the option.
+    CAPTURE_PAYLOAD_MODES = %i[off errors always].freeze
+
     class << self
       # The inert config the helpers read before init() runs (everything a
       # no-op needs: empty redaction lists, export disabled).
@@ -32,7 +40,7 @@ module Foam
           redact_detect: [].freeze,
           ignored_outbound_hosts: [].freeze,
           diagnostics: false, endpoint: FOAM_OTEL_ENDPOINT,
-          secret_heuristics: true
+          secret_heuristics: true, capture_payloads: :off
         ).freeze
       end
 
@@ -44,8 +52,13 @@ module Foam
 
       def resolve_config(name:, environment:, version:, enabled:,
                          redact_keys:, redact_pii_keys:, ignored_outbound_hosts:,
-                         diagnostics:, endpoint:, secret_heuristics: true, redact: nil)
+                         diagnostics:, endpoint:, secret_heuristics: true, redact: nil,
+                         capture_payloads: :off)
         keys, pii, detect = merged_redact_lists(redact_keys, redact_pii_keys, redact)
+        # capture_payloads is validated LOUDLY at boot (rule 10, exactly like
+        # the redact: object): a value outside CAPTURE_PAYLOAD_MODES (or its
+        # string spellings) raises here — never a silent :off.
+        capture_mode = validate_capture_payloads!(capture_payloads)
         Config.new(
           name: name,
           environment: environment,
@@ -71,12 +84,44 @@ module Foam
           # module-constant machinery no config shape can narrow. Anything
           # but literal false means ON (default-on preserves the fleet's
           # no-leakage bar; disabling is an explicit, audited decision).
-          secret_heuristics: secret_heuristics == false ? false : true
+          secret_heuristics: secret_heuristics == false ? false : true,
+          # The payload-capture mode (2026-07-28 mandate): :off (the default
+          # — zero teeing, zero middleware) | :errors | :always, canonical
+          # symbol form. init.rb applies the FOAM_CAPTURE_PAYLOADS operator
+          # env clamp BEFORE this resolves.
+          capture_payloads: capture_mode
         ).freeze
       end
 
       private
 
+      # ---- the capture_payloads option (payload-capture mandate 2026-07-28)
+      # Canonicalize a mode value: the three symbols, or their string
+      # spellings trimmed + case-insensitive, map to the canonical symbol;
+      # anything else is nil (the callers decide raise-vs-fallback).
+      def normalize_capture_payloads(value)
+        return value if CAPTURE_PAYLOAD_MODES.include?(value)
+        return nil unless value.is_a?(String) || value.is_a?(Symbol)
+
+        mode = value.to_s.strip.downcase.to_sym
+        CAPTURE_PAYLOAD_MODES.include?(mode) ? mode : nil
+      rescue StandardError
+        nil
+      end
+
+      # The loud-at-boot door (rule 10): an invalid INIT value is a
+      # programming error the engineer must catch on their machine.
+      # (The env clamp's invalid values warn + fall back instead — init.rb.)
+      def validate_capture_payloads!(value, context: "Foam::Otel.init")
+        mode = normalize_capture_payloads(value)
+        if mode.nil?
+          raise ArgumentError, "#{context} capture_payloads: must be one of " \
+                               "#{CAPTURE_PAYLOAD_MODES.map(&:inspect).join(', ')} " \
+                               "(equivalent strings accepted), got #{value.inspect}"
+        end
+        mode
+      end
+
       def downcase_list(list)
         Array(list).map { |k| k.to_s.downcase }.reject(&:empty?).uniq.freeze
       end
@@ -162,7 +207,12 @@ module Foam
           ignored_outbound_hosts: existing.ignored_outbound_hosts,
           diagnostics: diagnostics ? true : false,
           endpoint: existing.endpoint,
-          secret_heuristics: secret_heuristics == false ? false : true
+          secret_heuristics: secret_heuristics == false ? false : true,
+          # Like enabled/endpoint, capture_payloads is process-global after
+          # the first init: middleware insertion already happened (or
+          # deliberately did not — an :off boot shipped none), so a second
+          # init cannot meaningfully flip it. Carried forward unchanged.
+          capture_payloads: existing.capture_payloads
         ).freeze
       end
     end

data/lib/foam/otel/errors.rb

@@ -41,6 +41,18 @@ module Foam
         # exception" into a host-thread crash (rule 9).
         false
       end
+
+      # Read-only view of the registry above (the payload-capture :errors
+      # mode's thin seam, mandate 2026-07-28): true when at least one
+      # exception was recorded on this span via record_once. Reads the SAME
+      # span-carried ivar — never a second registry, never duplicated dedupe
+      # logic. Never raises (rule 9).
+      def recorded?(span)
+        seen = span.instance_variable_get(IVAR)
+        seen.is_a?(Array) && !seen.empty?
+      rescue StandardError
+        false
+      end
     end
   end
 end