foam-otel 1.5.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1e23dd37962a70ea36799a92d914fd2a38aa967a6cd1aface19bbaa322ab8966
-  data.tar.gz: f0018c27fb2dfff9316118438688a4a8ab2baea18810dbda522880361dfe6681
+  metadata.gz: cb5e4f91fea3170bbb206582adade6ca516824edad064a4be911ebe912ce7abf
+  data.tar.gz: 67ee9186826b4bc31c7f51deeaf8fd91bec6aaec393099f23b000384821230b9
 SHA512:
-  metadata.gz: 9731cb1fae1b54f236dd583102b2348ab4b45bff38745f446c584151e594ea685f2689d6a7fb4807136372006b7b10c0284b2e7c140c0c9625edb5a820294c8a
-  data.tar.gz: 91744d9fb0fb671bce62de973eec96152b147742ee5948fedf607626dfd89b8c3c6ed055522f21c4bbfd7d65ce5b558da9e67605578e3573ec15f8b0d4efeecc
+  metadata.gz: 4c4c7288e2b049a9f686d0eae3637a74261820bc779d6e8ec4daf14347b6ed3b423648c4c0d226982663465907465fc350a44cd52b998ad97b4766b69ecb727b
+  data.tar.gz: 7eb2f27f17fc9fcd9649b72c68b59493b366ff66950c86ff42f4baa414b7ea222b9e251f30264adbfe32400fe7851b2cf64505bf666c5385546b44f4a5714768

data/GOTCHAS.md

@@ -633,6 +633,94 @@ exfiltratable — the value-pattern secret layer is the required second control
   `spec/vuln_suite_spec.rb` (the adversarial exfil/ReDoS/bypass suite +
   mutation spot-checks).
 
+## F14: Default-on header capture — the official rack options are the seam (a NAMED list, not capture-all), operator header config is never overridden, and Faraday middleware ORDER decides which span gets enriched
+
+- **Trap**: four ways foam's default-on header capture
+  (`header_capture.rb`) could go wrong. (1) The official rack gem's
+  `allowed_request_headers:`/`allowed_response_headers:` options are a
+  STATIC ALLOWLIST compiled at install time with no capture-all form — so
+  Ruby inbound cannot be capture-all like js/python; pretending otherwise
+  (or shipping an empty default) silently costs the whole signal. (2) The
+  gem applies operator env config
+  (`OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS`,
+  instrumentation-base `config_overrides_from_env`) per option at install,
+  where an env value BEATS any passed config — except the narrow-to-empty
+  spelling (`allowed_request_headers=`), which the gem's env parser DROPS
+  (`parts[1]` nil): foam blindly passing its defaults would then override
+  the operator's explicit opt-out. And `Instrumentation::Base#install` is
+  first-wins (`return true if installed?`) — foam pre-installing BEFORE the
+  operator's explicit instance would freeze the operator's header config
+  out entirely. (3) A middleware writing to
+  `OpenTelemetry::Trace.current_span` writes to WHATEVER span is ambient:
+  foam's Faraday middleware registered without `use :open_telemetry` FIRST
+  (the official faraday instrumentation auto-appends its tracer middleware
+  AFTER the customer's block, i.e. INSIDE any middleware the block added)
+  would stamp `http.request.header.*` onto the enclosing app span,
+  silently corrupting another producer's data. (4) Outbound Net::HTTP has
+  NO hook surface in its official instrumentation (a closed
+  `connect`/`request` patch, no request/response hook, no header option) —
+  the only route would be monkey-patching stdlib or gem internals.
+- **Sources**: installed source —
+  opentelemetry-instrumentation-rack-0.31.1 `instrumentation.rb:24-25`
+  (allowlist options), `instrumentation.rb:108-127` (compiled at install:
+  enumerate-up-front, no capture-all; `build_attribute_name` folds `-` to
+  `_`, so the emitted suffixes are underscored);
+  opentelemetry-instrumentation-base-0.26.1 `base.rb:218-227` (install is
+  first-wins/idempotent), `base.rb:274-313` (env override beats passed
+  config per option), `base.rb:343-363` (the `;`-separated `name=value`
+  env grammar; a valueless `name=` is dropped);
+  opentelemetry-instrumentation-faraday-0.33.0 `instrumentation.rb` (no
+  header option; `patches/stable/connection.rb:18-25` — the auto-insert
+  appends `use(:open_telemetry)` AFTER the customer's builder block) and
+  `middlewares/stable/tracer_middleware.rb:27-45` (the span is active only
+  around the handlers BELOW the tracer middleware);
+  opentelemetry-instrumentation-net_http-0.29.0 `patches/instrument.rb`
+  (no extension point).
+- **Decision & why** (user ruling 2026-07-28, superseding the first-pass
+  capture-all Rack middleware): INBOUND capture is the official rack gem's
+  OWN header options, which foam PRE-INSTALLS with documented default
+  lists (`DEFAULT_REQUEST_HEADERS`/`DEFAULT_RESPONSE_HEADERS` — the
+  standard APM set plus the seven floor headers, which arrive `[REDACTED]`
+  so presence stays visible) — the same pre-install shape as the loop
+  guard, ordered AFTER `install_additional` so an operator's explicit rack
+  instance wins entirely, and computed per option AGAINST the operator's
+  env var so an operator-named option is never filled by foam (the F-PY3
+  posture: a narrower operator list is never widened, and `name=` narrows
+  to the gem default `[]`, never to foam's list). The honest consequence
+  is documented, not papered over: a header outside the list is NOT
+  captured (README shows the env-var extension recipe). OUTBOUND stays
+  foam's Faraday middleware (Faraday's first-class public API — the
+  official faraday instrumentation has no header option), capture-all,
+  triple-gated per write: foam owns the traces slot (rule 18 B), the span
+  is recording, AND the span's `instrumentation_scope.name` is
+  "OpenTelemetry::Instrumentation::Faraday" — a mis-ordered stack captures
+  NOTHING instead of enriching the wrong span. Faraday placement stays a
+  per-connection line (`f.use :open_telemetry` then
+  `f.use :foam_otel_headers`): the only default-on mechanism is the
+  Connection prepend the official gem uses — exactly the
+  internals-patching foam never does. Net::HTTP (and Excon / httprb /
+  HTTPX) outbound headers are deliberately OUT OF SCOPE — skipped and
+  documented rather than hacked. Masking stays central: the credential
+  floor covers the emitted `http.{request,response}.header.<name>` forms
+  at the exporter boundary (dash/underscore-normalized, so the rack gem's
+  underscored suffixes match), never a key list in the capture path.
+- **Mitigation**: `lib/foam/otel/header_capture.rb` (the default lists,
+  `preinstall_rack_defaults!` + the operator-precedence parsing, the
+  Faraday middleware + public-registry registration, the shared gates);
+  `lib/foam/otel/init.rb` (`activate_instrumentations` pre-install
+  ordering — after `install_additional`, before `install_all` — gated on
+  foam owning traces; `activate_floor_extensions` Faraday registration).
+- **Test**: `spec/header_capture_spec.rb` — default-on capture both
+  directions on the official rack span, the honest negative (an unlisted
+  header is NOT captured), floor headers arrive `[REDACTED]` on the wire
+  with zero config, multi-value arity, operator env respect (named option
+  wins, narrow-to-empty honored, explicit instance wins entirely),
+  install idempotence (a second sweep never clobbers the config),
+  foreign-traces stand-down, hostile values (invalid UTF-8 / huge /
+  raising readers), Faraday scope/ownership gates + mis-ordered-stack
+  no-op + registration idempotence; `spec/conventions_spec.rb` (rule 21:
+  header capture never grows a `rack.input` body tee).
+
 ---
 
 ## General gotchas (applicable to Ruby)

data/README.md

@@ -33,8 +33,11 @@ gem "foam-otel"
 # Faraday), the primary datastores (pg, mysql2, redis, mongo) with RAW
 # db.statement capture, Sidekiq, the stdlib Logger bridge, hand-written Ruby
 # runtime + GC metrics, session stitching (browser session.id via baggage),
-# and the LLM shims (OpenAI, Anthropic, Gemini, ruby_llm — activity + raw
-# content). Add only the niche long tail your app needs beyond the floor:
+# default-on HTTP header capture (a documented default header list on the
+# official rack server span; every Faraday client header — see "Header
+# capture" below), and the LLM shims
+# (OpenAI, Anthropic, Gemini, ruby_llm — activity + raw content). Add only
+# the niche long tail your app needs beyond the floor:
 gem "opentelemetry-instrumentation-graphql"  # e.g. GraphQL
 gem "opentelemetry-instrumentation-resque"   # e.g. Resque
 ```
@@ -131,6 +134,106 @@ baggage is absent. **CORS note (the FDE wires this on every frontend-called
 API):** the allowlist must include BOTH headers —
 `Access-Control-Allow-Headers: traceparent, baggage`.
 
+### Header capture — request + response headers on the official spans
+
+**Inbound (default-on, no init option):** foam configures the OFFICIAL
+`opentelemetry-instrumentation-rack` gem's own header options
+(`allowed_request_headers:` / `allowed_response_headers:`) with a
+documented **default list** at init, so the rack SERVER span carries the
+listed request and response headers as
+`http.request.header.<name>` / `http.response.header.<name>` attributes
+(the gem's form: lowercase with `-` folded to `_`, e.g.
+`http.request.header.x_request_id`), on error requests too.
+
+> **Honest limit (deliberate divergence from foam's js/python cores):**
+> Ruby inbound captures the **named default list below, NOT all headers** —
+> the official gem's options are an enumerated allowlist with no
+> capture-all form, and that official option is the chosen seam (ruling
+> 2026-07-28). A custom header outside the list is **not captured** unless
+> you extend the list (recipe below).
+
+The default lists (`Foam::Otel::HeaderCapture::DEFAULT_REQUEST_HEADERS` /
+`DEFAULT_RESPONSE_HEADERS` — frozen, documented constants):
+
+- **request** — `content-type`, `content-length`, `content-encoding`,
+  `accept`, `accept-charset`, `accept-encoding`, `accept-language`,
+  `user-agent`, `referer`, `origin`, `host`, `cache-control`, `pragma`,
+  `if-none-match`, `if-modified-since`, `range`, `via`, `forwarded`,
+  `x-forwarded-for`, `x-forwarded-proto`, `x-forwarded-host`,
+  `x-forwarded-port`, `x-real-ip`, `x-request-id`, `x-correlation-id`,
+  plus the request half of the credential floor — `authorization`,
+  `proxy-authorization`, `cookie`, `x-api-key`, `x-auth-token`;
+- **response** — `content-type`, `content-length`, `content-encoding`,
+  `content-language`, `content-range`, `cache-control`, `pragma`,
+  `expires`, `age`, `etag`, `last-modified`, `vary`, `location`,
+  `retry-after`, `x-request-id`, `x-correlation-id`, `x-runtime`,
+  `server-timing`, plus the response half of the floor — `set-cookie`,
+  `www-authenticate`.
+
+Propagation headers (`traceparent`, `tracestate`, `baggage`) are
+deliberately not listed — they are extracted as span context, not captured
+as attributes; hop-by-hop plumbing (`connection`, `keep-alive`, …) is
+skipped too.
+
+**Extending (or narrowing) the list — the gem's own standard env var,
+always respected, never overridden by foam:**
+
+```bash
+# The value REPLACES the list for that option — name every header you want:
+export OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS='allowed_request_headers=content-type,accept,x-request-id,x-tenant-id'
+```
+
+Precedence: (1) an explicit rack instrumentation instance you pass via
+`additional_instrumentations:` installs first and its header config wins
+entirely; (2) any header option you set in
+`OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS` governs that option (foam's
+default fills only the option you did not touch — a narrower operator list
+is never widened, even the narrow-to-empty `allowed_request_headers=`
+spelling); (3) otherwise foam's defaults apply. Foam never installs the
+rack instrumentation (or its header config) when a foreign SDK owns the
+traces slot.
+
+**Floor masking, zero config:** the always-on credential floor masks the
+seven credential headers (`authorization`, `proxy-authorization`, `cookie`,
+`set-cookie`, `x-api-key`, `x-auth-token`, `www-authenticate`) to
+`[REDACTED]` in the emitted attribute forms (dash/underscore-normalized
+match) — they ride the default lists deliberately so their PRESENCE is
+visible, value never (see "The default credential denylist";
+`x-forwarded-for` / `x-real-ip` are on the 52-entry key floor and arrive
+`[REDACTED]` too). **Masking further captured headers is redaction's job,
+not a capture switch:** list the emitted name form in `redact_keys` /
+`redact: { secrets: [...] }` (tail mask) or `redact_pii_keys` /
+`redact: { pii: [...] }` (full `[REDACTED]`) — the substring key match
+reaches the attribute names (inbound names are underscored by the rack gem,
+so list e.g. `x_runtime`; Faraday's outbound names keep dashes).
+
+**Outbound (Faraday):** the official faraday instrumentation has no header
+option at all, so outbound stays foam's own Faraday middleware (its
+first-class public extension API), enriching the official CLIENT span with
+**every** request header actually sent (the injected `traceparent`
+included) and every response header received — string-array attributes,
+dash-form names. It never creates a span and stands down when foam is
+disabled/killed, when a foreign SDK owns traces, or when the current span
+is not the official faraday instrumentation's own (a mis-ordered stack
+captures nothing rather than writing onto the wrong span). One line per
+connection (Faraday has no public add-to-every-connection hook — the
+middleware name is registered by init, placement is yours; ORDER MATTERS,
+the official middleware first):
+
+```ruby
+conn = Faraday.new(url: "https://partner-api.example") do |f|
+  f.use :open_telemetry     # the official client span (explicit, so it sits OUTSIDE)
+  f.use :foam_otel_headers  # foam's header capture, directly inside it
+end
+```
+
+**Deliberately out of scope** (no standard extension point exists —
+skipped, not monkey-patched): outbound **Net::HTTP / Excon / HTTP (httprb)
+/ HTTPX** headers (their official instrumentations expose no
+request/response hook or header option) and gRPC metadata. Outbound header
+coverage today is Faraday. **Bodies/payloads are not captured** — a
+separate, future capability, not part of header capture.
+
 ---
 
 ## The `init` options
@@ -823,6 +926,7 @@ end
 | `OTEL_EXPORTER_OTLP_ENDPOINT` | HONORED — the one operator-level override of the pinned fleet endpoint (for foam's own conformance rig / enterprise egress). Active → loud `[foam]` warning naming the destination. Applies to door-2 taps identically (and moves the host the required loop step must name). |
 | `OTEL_PROPAGATORS=none` | HONORED — turns trace propagation OFF (links lost) while telemetry keeps flowing; warns. Any other value warns and is ignored (foam's propagator set is fixed: W3C tracecontext + baggage). |
 | `OTEL_BSP_SCHEDULE_DELAY` / `OTEL_BLRP_SCHEDULE_DELAY` / `OTEL_METRIC_EXPORT_INTERVAL` | HONORED — batch cadence, read natively by the upstream SDK. |
+| `OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS` | HONORED (by the contrib rack gem itself) and RESPECTED by foam — a header option you set here (`allowed_request_headers=…` / `allowed_response_headers=…`) governs that option; foam's default header list fills only the option you did not touch (never widened, never overridden — see "Header capture"). The other `OTEL_RUBY_INSTRUMENTATION_<NAME>_CONFIG_OPTS` vars are likewise the contrib gems' own standard levers (e.g. the Sidekiq `propagation_style` note above). |
 | `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | INERT — foam wires ONE resolved endpoint into all three exporters explicitly, so per-signal endpoint vars never redirect (or split) foam's export. Set → loud `[foam]` warning that it is inert. |
 | `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | INERT — as above (warns when set). |
 | `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | INERT — as above (warns when set). |

data/lib/foam/otel/header_capture.rb

@@ -0,0 +1,416 @@
+# frozen_string_literal: true
+
+require "opentelemetry"
+
+require_relative "diagnostics"
+require_relative "redaction"
+
+module Foam
+  module Otel
+    # Default-on HTTP HEADER capture with NO init knob (the init signature is
+    # FROZEN; a capture option is forbidden surface), on both wire directions:
+    #
+    #   * INBOUND — the OFFICIAL opentelemetry-instrumentation-rack gem's own
+    #     header options, `allowed_request_headers:` /
+    #     `allowed_response_headers:`
+    #     (opentelemetry-instrumentation-rack-0.31.1 instrumentation.rb:24-25),
+    #     which foam PRE-INSTALLS with the default lists below
+    #     (preinstall_rack_defaults!, wired from init.rb's
+    #     activate_instrumentations — the same pre-install shape as
+    #     configure_loop_guard's untraced_hosts). The official middleware then
+    #     records each listed header onto its own SERVER span as
+    #     `http.request.header.<name>` / `http.response.header.<name>`
+    #     (the gem's build_attribute_name lowercases and folds `-` to `_`,
+    #     so `x-request-id` lands as http.request.header.x_request_id;
+    #     values are the raw header strings).
+    #   * OUTBOUND (Faraday) — foam's own FaradayMiddleware below (Faraday's
+    #     first-class public middleware API), enriching the official faraday
+    #     CLIENT span with EVERY request/response header. Faraday's official
+    #     instrumentation has no header option at all, so the middleware seam
+    #     stays (explicitly accepted); placement is one line per connection
+    #     (README recipe: `f.use :foam_otel_headers`).
+    #
+    # THE SEAM DECISION (superseded + re-ruled): the first pass shipped a foam
+    # Rack middleware capturing ALL headers, because the official gem's
+    # options are a static allowlist with no capture-all form. USER RULING
+    # 2026-07-28: inbound headers come from the official rack gem's own
+    # options, configured by foam with defaults — NOT from a custom Rack
+    # middleware. That capture-all middleware (and its Railtie/init wiring) is
+    # REMOVED; the official option is the chosen seam, and the honest
+    # consequence is documented everywhere it matters: Ruby inbound captures
+    # the NAMED DEFAULT LIST below, not all headers (the js/python cores
+    # capture all — the official Ruby gem cannot express capture-all). A
+    # header outside the list is NOT captured unless the operator extends the
+    # list via the gem's own standard env var (below).
+    #
+    # OPERATOR PRECEDENCE (never override an operator's own header config):
+    #   1. An explicit rack instrumentation instance passed through init's
+    #      `additional_instrumentations:` installs FIRST (install_additional
+    #      runs before this pre-install; Instrumentation::Base#install is
+    #      first-wins) — the operator's config, header options included, wins
+    #      entirely and foam's pre-install no-ops.
+    #   2. The gem's standard env var
+    #      OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS
+    #      (instrumentation-base config_overrides_from_env) is parsed by foam
+    #      BEFORE pre-installing: any header option the operator names there
+    #      is left OUT of foam's config — the operator's value (applied by
+    #      the gem itself at install, where an env override also beats any
+    #      passed config) governs that option, including the narrow-to-empty
+    #      spelling `allowed_request_headers=`. Foam's default fills ONLY the
+    #      header option(s) the operator did not touch. This mirrors the
+    #      python core's F-PY3 posture: a narrower operator allowlist is
+    #      respected, never widened.
+    #   3. Operator untouched → foam pre-installs the rack instrumentation
+    #      with BOTH default lists; the later install_all sweep skips the
+    #      already-installed gem (install is idempotent).
+    #      Extending the list is the same standard env var, e.g.:
+    #      OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS='allowed_request_headers=<foam defaults>,x-tenant-id'
+    #      (the env value REPLACES the list for that option — include the
+    #      defaults you still want).
+    #
+    # Masking stays central (never a key list here): the credential floor
+    # masks the seven credential headers in the emitted
+    # http.{request,response}.header.<name> forms at the exporter boundary
+    # (redaction.rb floor_kind — dash/underscore-normalized, so the gem's
+    # underscore forms match), and the customer masks further names via
+    # redact_keys / redact: {secrets:/pii:}. The floor-listed headers are
+    # deliberately IN the default lists so their PRESENCE is visible as
+    # [REDACTED] — matching the js/python wire shape.
+    #
+    # DELIBERATELY OUT OF SCOPE (no standard seam — skipped, not hacked):
+    # outbound Net::HTTP / Excon / HTTP (httprb) / HTTPX headers (their
+    # official instrumentations expose no request/response hook or header
+    # option) and gRPC metadata. Bodies are NOT captured (a separate future
+    # pass).
+    module HeaderCapture
+      # ---- THE DEFAULT HEADER LISTS (foam's Ruby fleet default) --------------
+      # Frozen, documented, grouped by purpose. Survey basis (recorded in the
+      # PR design record): New Relic's captured request/response header
+      # attributes, AppSignal's default request_headers config, Sentry's
+      # header capture w/ denylist, the OTel semconv opt-in capture model —
+      # erring toward MORE coverage. Extendable per customer via
+      # OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS (recipe above/README).
+      # Deliberately EXCLUDED: propagation headers (traceparent, tracestate,
+      # baggage, b3) — they are extracted as span context, capturing them
+      # would duplicate the trace ids; hop-by-hop plumbing (connection,
+      # keep-alive, transfer-encoding, upgrade, te, trailer) — no diagnostic
+      # value.
+      DEFAULT_REQUEST_HEADERS = %w[
+        content-type content-length content-encoding
+        accept accept-charset accept-encoding accept-language
+        user-agent referer origin host
+        cache-control pragma if-none-match if-modified-since range
+        via forwarded x-forwarded-for x-forwarded-proto x-forwarded-host
+        x-forwarded-port x-real-ip
+        x-request-id x-correlation-id
+        authorization proxy-authorization cookie x-api-key x-auth-token
+      ].freeze
+      # Group purposes (in list order):
+      #   content-type/-length/-encoding — body metadata (mismatch debugging);
+      #   accept*                        — content negotiation;
+      #   user-agent/referer/origin/host — client + navigation context
+      #                                    (CORS/vhost triage);
+      #   cache-control..range           — caching / conditional requests
+      #                                    (304-vs-200 and partial-GET triage);
+      #   via..x-real-ip                 — proxy/LB chain (x-forwarded-for and
+      #                                    x-real-ip are on the 52-entry
+      #                                    credential/PII key floor, so they
+      #                                    arrive [REDACTED] — presence still
+      #                                    visible);
+      #   x-request-id/x-correlation-id  — cross-system correlation ids;
+      #   authorization..x-auth-token    — the request half of the credential
+      #                                    floor: ALWAYS [REDACTED] on the
+      #                                    wire, captured so auth PRESENCE is
+      #                                    visible (matches js/python).
+
+      DEFAULT_RESPONSE_HEADERS = %w[
+        content-type content-length content-encoding content-language
+        content-range
+        cache-control pragma expires age etag last-modified vary
+        location retry-after
+        x-request-id x-correlation-id x-runtime server-timing
+        set-cookie www-authenticate
+      ].freeze
+      # Group purposes (in list order):
+      #   content-*                      — body metadata;
+      #   cache-control..vary            — caching + validators (CDN/cache
+      #                                    debugging);
+      #   location/retry-after           — redirect + backpressure flow control;
+      #   x-request-id..server-timing    — correlation ids + server-side
+      #                                    timing breakdowns (x-runtime is the
+      #                                    Rack/Rails standard);
+      #   set-cookie/www-authenticate    — the response half of the credential
+      #                                    floor: ALWAYS [REDACTED], presence
+      #                                    visible.
+
+      # The rack instrumentation class foam pre-installs (resolved lazily —
+      # same shape as init.rb's HTTP_CLIENT_INSTRUMENTATIONS: only the
+      # …::Instrumentation class responds to `.instance.install`).
+      RACK_INSTRUMENTATION = "OpenTelemetry::Instrumentation::Rack::Instrumentation"
+
+      # The gem's own standard per-instrumentation config env var
+      # (instrumentation-base config_overrides_from_env). Foam PARSES it only
+      # to stand down per option — foam never writes it, and the gem itself
+      # applies it at install (where it also overrides any passed config).
+      RACK_CONFIG_OPTS_VAR = "OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS"
+
+      # The two rack header options foam defaults (everything else is the
+      # gem's own business).
+      RACK_HEADER_OPTIONS = {
+        allowed_request_headers: DEFAULT_REQUEST_HEADERS,
+        allowed_response_headers: DEFAULT_RESPONSE_HEADERS,
+      }.freeze
+
+      # The official faraday instrumentation's tracer name — the scope gate
+      # FaradayMiddleware enriches through (Instrumentation::Base#install
+      # binds the tracer under the instrumentation's own name).
+      FARADAY_SCOPE_NAME = "OpenTelemetry::Instrumentation::Faraday"
+
+      # The public Faraday middleware-registry key foam registers under
+      # (README recipe: `f.use :foam_otel_headers`).
+      FARADAY_MIDDLEWARE_NAME = :foam_otel_headers
+
+      class << self
+        # ---- inbound: pre-install the OFFICIAL rack gem with foam defaults --
+        # Called from activate_instrumentations AFTER install_additional (an
+        # operator's explicit rack instance wins — precedence rule 1 above)
+        # and BEFORE install_all (which skips the already-installed gem).
+        # Never raises; a failure warns loudly (rule 15 — header capture
+        # silently off was the 1.0.0 loop-guard bug's hiding shape).
+        def preinstall_rack_defaults!
+          klass = rack_instrumentation_class
+          if klass.nil?
+            # The rack instrumentation gem is not bundled (it is a hard
+            # gemspec dependency, so this is belt-and-braces like
+            # configure_loop_guard's NameError path).
+            Diagnostics.info("header capture: #{RACK_INSTRUMENTATION} not present — inbound header capture skipped")
+            return false
+          end
+
+          instance = klass.instance
+          if instance.installed?
+            # Precedence rule 1: an explicit operator install (usually via
+            # additional_instrumentations) already configured the gem — foam
+            # touches nothing, including its header options.
+            Diagnostics.info("header capture: rack instrumentation already installed by the operator — " \
+                             "foam's default header lists NOT applied (operator config wins)")
+            return false
+          end
+
+          config = rack_header_defaults
+          if config.empty?
+            # Precedence rule 2, both options operator-set: nothing left to
+            # default; install_all installs the gem and the gem applies the
+            # operator's env values itself.
+            Diagnostics.info("header capture: both rack header options set via #{RACK_CONFIG_OPTS_VAR} — " \
+                             "operator lists win; foam defaults not applied")
+            return false
+          end
+
+          installed = instance.install(config)
+          unless installed
+            # Present-check false (a non-Rack process: Sidekiq-only worker) or
+            # OTEL_RUBY_INSTRUMENTATION_RACK_ENABLED=false — expected states,
+            # narrated under diagnostics only (rule 10). There are no server
+            # spans to enrich in either case.
+            Diagnostics.info("header capture: rack instrumentation not installable here (Rack absent or " \
+                             "disabled by env) — no inbound spans, no inbound header capture")
+          end
+          installed
+        rescue StandardError => e
+          Diagnostics.warn("header capture: rack pre-install failed (#{e.class}: #{e.message}) — inbound " \
+                           "header capture is OFF (the official rack spans still flow, without header attributes)")
+          false
+        end
+
+        # Foam's defaults MINUS every header option the operator named in the
+        # gem's standard env var (precedence rule 2: an operator-named option
+        # is theirs — even the narrow-to-empty `allowed_request_headers=`
+        # spelling, which the gem's own env parser drops, must fall back to
+        # the GEM default [], never to foam's list).
+        def rack_header_defaults
+          operator = operator_rack_options
+          RACK_HEADER_OPTIONS.reject { |name, _| operator.include?(name) }
+        end
+
+        # The option names the operator set in
+        # OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS, parsed with the exact
+        # upstream grammar (`;`-separated `name=value` pairs —
+        # instrumentation-base config_overrides_from_env). Never raises.
+        def operator_rack_options
+          raw = ENV[RACK_CONFIG_OPTS_VAR]
+          return [] if raw.nil? || raw.strip.empty?
+
+          raw.split(";").filter_map do |pair|
+            name = pair.split("=", 2).first.to_s.strip
+            name.to_sym unless name.empty?
+          end
+        rescue StandardError
+          []
+        end
+
+        # ---- outbound: Faraday wiring, PUBLIC API ONLY ----------------------
+        # Register the middleware under its registry name so the README
+        # one-liner (`f.use :foam_otel_headers`) works. Faraday has no public
+        # "add to every connection" hook — the official instrumentation gets
+        # its default-on placement by PREPENDING Faraday::Connection, which
+        # is exactly the third-party-internals patching foam never does — so
+        # per-connection insertion is the documented recipe, not automatic
+        # (README "Header capture"). Idempotent; inert without Faraday.
+        def register_faraday_middleware!
+          return false if @faraday_registered
+          return false unless defined?(::Faraday::Middleware) &&
+                              ::Faraday::Middleware.respond_to?(:register_middleware)
+
+          ::Faraday::Middleware.register_middleware(FARADAY_MIDDLEWARE_NAME => FaradayMiddleware)
+          @faraday_registered = true
+        rescue StandardError => e
+          Diagnostics.warn("header capture: Faraday middleware registration failed " \
+                           "(#{e.class}: #{e.message}) — wire it manually with " \
+                           "`use Foam::Otel::HeaderCapture::FaradayMiddleware` (README)")
+          false
+        end
+
+        # Test hook (mirrors Foam::Otel.reset_for_tests!).
+        def reset_for_tests!
+          @faraday_registered = false
+        end
+
+        # ---- gates shared with the Faraday middleware ------------------------
+        # Foam's OWN pipeline only: the traces slot must still hold the
+        # provider foam registered (rule 18 B — pre-init, disabled, killed,
+        # foreign-owned and displaced-after-init all land here as false).
+        def foam_owned_traces?
+          registered = Foam::Otel.instance_variable_get(:@foam_registered)
+          provider = registered && registered[:traces]
+          !provider.nil? && OpenTelemetry.tracer_provider.equal?(provider)
+        rescue StandardError
+          false
+        end
+
+        # The current span, but ONLY when it is recording AND was minted by
+        # the named official instrumentation (the scope gate): a mis-ordered
+        # middleware stack must silently capture nothing, never enrich some
+        # OTHER producer's span with header attributes.
+        def official_recording_span(scope_name)
+          span = OpenTelemetry::Trace.current_span
+          return nil unless span.respond_to?(:recording?) && span.recording?
+
+          scope = span.respond_to?(:instrumentation_scope) ? span.instrumentation_scope : nil
+          return nil unless scope.respond_to?(:name) && scope.name == scope_name
+
+          span
+        rescue StandardError
+          nil
+        end
+
+        # Never-throw attribute write with a re-checked recording gate (the
+        # span may have ended between capture start and this write).
+        def set_attr(span, key, value)
+          return if value.nil?
+
+          span.set_attribute(key, value) if span.respond_to?(:recording?) && span.recording?
+          nil
+        rescue StandardError, SystemStackError
+          nil
+        end
+
+        private
+
+        def rack_instrumentation_class
+          RACK_INSTRUMENTATION.split("::").reduce(Object) { |mod, name| mod.const_get(name) }
+        rescue NameError
+          nil
+        end
+      end
+
+      # Outbound header capture on the official Faraday CLIENT span, via
+      # Faraday's first-class public middleware API (a plain #call(env)
+      # middleware — no inheritance from Faraday internals, so this file
+      # loads without Faraday present). Unlike inbound (the official rack
+      # gem's named allowlist), this captures EVERY outbound header — the
+      # faraday instrumentation has no header option, so foam's middleware is
+      # the seam and it has no list to be limited by (explicitly accepted
+      # asymmetry, user ruling 2026-07-28). It must sit directly INSIDE the
+      # official :open_telemetry middleware (README recipe):
+      #
+      #   Faraday.new(url) do |f|
+      #     f.use :open_telemetry     # the official client span
+      #     f.use :foam_otel_headers  # foam's header capture, inside it
+      #   end
+      #
+      # because the official tracer middleware activates its span only
+      # around the handlers BELOW it. The scope gate makes any other
+      # placement a silent no-op — never a wrong-span write (GOTCHAS F14).
+      class FaradayMiddleware
+        def initialize(app, *_args)
+          @app = app
+        end
+
+        # Request headers are captured on the way in (the traceparent the
+        # official middleware injected included — it IS an outbound header);
+        # response headers via the response's public on_complete hook, which
+        # runs while the official span is still open (its own status hook
+        # rides the same mechanism). The app's/adapter's exception
+        # propagates untouched.
+        def call(env)
+          span = capturable_span
+          set_request_headers(span, env) unless span.nil?
+          response = @app.call(env)
+          unless span.nil?
+            begin
+              response.on_complete { |renv| set_response_headers(span, renv) }
+            rescue StandardError, SystemStackError
+              nil
+            end
+          end
+          response
+        end
+
+        private
+
+        def capturable_span
+          return nil unless HeaderCapture.foam_owned_traces?
+
+          HeaderCapture.official_recording_span(FARADAY_SCOPE_NAME)
+        rescue StandardError
+          nil
+        end
+
+        def set_request_headers(span, env)
+          each_header(env, :request_headers) do |name, value|
+            HeaderCapture.set_attr(span, "http.request.header.#{name}",
+                                   [Redaction.scrub_utf8(value.to_s)])
+          end
+        end
+
+        def set_response_headers(span, env)
+          each_header(env, :response_headers) do |name, value|
+            HeaderCapture.set_attr(span, "http.response.header.#{name}",
+                                   [Redaction.scrub_utf8(value.to_s)])
+          end
+        end
+
+        # Faraday header maps (Faraday::Utils::Headers) carry one string per
+        # name (repeats are comma-joined per RFC 9110 by the adapter), so
+        # each name emits a one-element semconv array.
+        def each_header(env, reader)
+          headers = env.respond_to?(reader) ? env.public_send(reader) : nil
+          return if headers.nil? || !headers.respond_to?(:each)
+
+          headers.each do |key, value|
+            name = key.to_s.downcase
+            next if name.empty?
+
+            yield name, value
+          rescue StandardError
+            next # one poisoned header never costs the rest
+          end
+          nil
+        rescue StandardError
+          nil
+        end
+      end
+    end
+  end
+end

data/lib/foam/otel/init.rb

@@ -18,6 +18,7 @@ require_relative "ingest" # door-2 entries (SDK still lazy — rule 41)
 require_relative "logger_bridge"
 require_relative "runtime_metrics"
 require_relative "llm"
+require_relative "header_capture" # default-on header capture (rule 32a module; wired below)
 
 module Foam
   module Otel
@@ -210,6 +211,7 @@ module Foam
         Metrics.reset_for_tests!
         RuntimeMetrics.reset_for_tests!
         Redaction.reset_secret_scan_warnings!
+        HeaderCapture.reset_for_tests!
       end
 
       private
@@ -554,6 +556,16 @@ module Foam
 
         configure_loop_guard(config)
         install_additional(additional)
+        # Inbound header capture (user ruling 2026-07-28): pre-install the
+        # OFFICIAL rack instrumentation with foam's default header lists
+        # (HeaderCapture::DEFAULT_{REQUEST,RESPONSE}_HEADERS) — the same
+        # pre-install shape as configure_loop_guard; install_all below skips
+        # the already-installed gem. Ordered AFTER install_additional so an
+        # operator's explicit rack instance (their own header config) wins,
+        # and per-option operator env config
+        # (OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS) is never overridden —
+        # precedence documented in header_capture.rb.
+        HeaderCapture.preinstall_rack_defaults!
         OpenTelemetry::Instrumentation.registry.install_all(
           FLOOR_INSTRUMENTATION_CONFIG.merge(SUPERSEDED_INSTRUMENTATION_CONFIG)
         )
@@ -586,6 +598,16 @@ module Foam
 
         return unless foam_exports_signal?(:traces)
 
+        begin
+          # Outbound: register foam's Faraday header middleware under its
+          # public registry name (`f.use :foam_otel_headers` — README
+          # recipe). Registration only — Faraday has no public add-to-every-
+          # connection hook, so placement stays a per-connection line.
+          HeaderCapture.register_faraday_middleware!
+        rescue StandardError => e
+          Diagnostics.warn("header capture: Faraday registration failed: #{e.class}: #{e.message}")
+        end
+
         begin
           LLM.activate!
         rescue StandardError => e

data/lib/foam/otel/version.rb

@@ -61,6 +61,28 @@ module Foam
     #   metric-label exemption as the secret layer. Unknown redact fields,
     #   non-Array values, and unknown detect entities raise at init (loud at
     #   boot). With redact: absent, behavior is byte-identical to 1.4.0.
-    VERSION = "1.5.0"
+    # 1.6.0: DEFAULT-ON HTTP HEADER CAPTURE (2026-07-28; MINOR — additive wire
+    #   change, no API/config change). New rule-32a module HeaderCapture.
+    #   INBOUND: foam pre-installs the OFFICIAL rack instrumentation with the
+    #   documented default header lists (DEFAULT_REQUEST_HEADERS /
+    #   DEFAULT_RESPONSE_HEADERS) via its own allowed_request_headers /
+    #   allowed_response_headers options (user ruling 2026-07-28 — the
+    #   official gem's options are the seam, not a custom Rack middleware),
+    #   so the rack SERVER span carries the LISTED headers as
+    #   http.{request,response}.header.<name> (underscored suffix — the gem's
+    #   form). HONEST LIMIT: a named enumeration, NOT capture-all like
+    #   js/python (the official option cannot express capture-all); extend
+    #   per service via OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS — operator
+    #   header config there (or an explicit instance via
+    #   additional_instrumentations) is always respected, never overridden.
+    #   OUTBOUND: a Faraday middleware (`f.use :foam_otel_headers`) enriching
+    #   the official faraday client span with every outbound header (no
+    #   upstream header option exists — accepted asymmetry). Credential floor
+    #   masks the seven auth headers, zero config, no off switch.
+    #   Net::HTTP/Excon/httprb/HTTPX outbound headers deliberately out of
+    #   scope (their official instrumentations expose no hook —
+    #   OTel-fundamentals ruling). Bodies are NOT captured (that is the
+    #   separate capture_payloads pass).
+    VERSION = "1.6.0"
   end
 end

data/lib/foam/otel.rb

@@ -10,9 +10,14 @@
 # the PIPELINE (providers, batch processors, OTLP export to the pinned fleet
 # endpoint, the always-on redaction floor) and turns on automatic tier-1/2
 # instrumentation; the ecosystem owns the INSTRUMENTATION. The package ships
-# a small set of never-throw helpers and nothing else — no stamping, no
-# request-input capture, no request-id enrichment (all removed per the spec;
-# stamps are the collector's job server-side).
+# a small set of never-throw helpers, plus default-on HTTP header capture —
+# inbound via the OFFICIAL rack gem's own allowed_request_headers /
+# allowed_response_headers options, pre-installed by foam with a documented
+# DEFAULT LIST (a named enumeration, NOT all headers — the honest divergence
+# from js/python; header_capture.rb), outbound via foam's Faraday middleware
+# on the official client span — and nothing else: no stamping, no
+# request-INPUT (body) capture, no request-id enrichment (removed per the
+# spec; stamps are the collector's job server-side).
 
 require_relative "otel/version"
 require_relative "otel/constants"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: foam-otel
 version: !ruby/object:Gem::Version
-  version: 1.5.0
+  version: 1.6.0
 platform: ruby
 authors:
 - Foam
@@ -384,6 +384,7 @@ files:
 - lib/foam/otel/diagnostics.rb
 - lib/foam/otel/errors.rb
 - lib/foam/otel/fork_hooks.rb
+- lib/foam/otel/header_capture.rb
 - lib/foam/otel/ingest.rb
 - lib/foam/otel/ingest_metric_reader.rb
 - lib/foam/otel/init.rb