foam-otel 3.0.0.alpha.7 → 3.0.0.alpha.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 929a47fe32e662ab012e7995078f1b0bab1f0ef3b58e3b7a9ca8b3b109c6ca29
-  data.tar.gz: 1608559fa17bd0f683492267a2db2f7daa4f3989d4a79f06d67f38e27632174d
+  metadata.gz: 240906547c665dd40513921db5ab9f00d65febef81d40848faa91a1f5ae02e15
+  data.tar.gz: 9a2c4c5045ba080c21b165e1a87836cd9288408343b049c293352162e9be7083
 SHA512:
-  metadata.gz: b8484befaf21fec24e8ce725931c368d9aaac7518916a8f0aebc69f02a5345fc10a0a73c6b25167d555554133eb84b7f6479d3ad676697beb2c03758352dd592
-  data.tar.gz: 56032f62347e799791dce8d37db046dd6fe6e4c8e372dd4aff7fa7c54813992b89e61b854b9b6565aeaf9fd7dd08c53747d5c9b8eae897b96ae869622594987e
+  metadata.gz: f891d7d8a749ce8a9042f08183581ec43924445d21137b863619a912cd0be0bb516e340d01e2ed6e238356a3b61f3cedb5739b769b04e38213d785712616cb9f
+  data.tar.gz: dd19f649f254cff7a00da71b14f158cc3d3d370a3affcc587ec86156d5439aaecd956564e106c54f0c0b2d229195065f69c03ab3eae9058459265ea479b71aaf

data/README.md

@@ -92,7 +92,7 @@ TODO(jhartquist): Gen AI providers beyond `anthropic`. There is no OpenTelemetry
 
 ## Public API
 
-Every method lives on the `Foam` module. Other than the ingest factories and `init`'s keyword errors, none of them raise exceptions. `set_baggage`, the metric helpers, and `log` need a Foam export path to function properly, so they are no-ops before `init`; the trace context helpers and `record_exception` work on the current OpenTelemetry context whether or not `init` ran.
+Every method lives on the `Foam` module. A missing or invalid value is reported in Foam's log output instead of raising; only an unknown keyword raises `ArgumentError`. `set_baggage`, the metric helpers, and `log` need a Foam export path to function properly, so they are no-ops before `init`; the trace context helpers and `record_exception` work on the current OpenTelemetry context whether or not `init` ran.
 
 ### `init`
 
@@ -221,7 +221,7 @@ All three export to the endpoint set by `set_endpoint` and take the same `redact
 
 Foam identifies the service by the `name` and `environment` passed in: on the copy sent to Foam they overwrite the provider's `service.name` and `deployment.environment.name`, and `foam.ingest.host` names the host of the endpoint the copy goes to. The provider's own telemetry is left unchanged. Pass the same values to all three.
 
-Unlike every other public method, these are not guarded: they raise `ArgumentError` when `name`, `environment`, or `token` is blank or `redact` or `before_send` is invalid, and let any other failure through.
+A missing or blank `name`, `environment`, or `token`, or an invalid `redact` or `before_send` value, is reported in Foam's log output and turns off just that helper: it returns a component that exports nothing, which the app adds to its provider as usual, and the signal stays `none` in `state`, so an unset environment variable never stops the app from booting.
 
 Do not combine `init` and an ingest component for the same signal. The ingest components do not install instrumentations; register the ones the app needs with the OpenTelemetry SDK directly.
 
@@ -280,7 +280,7 @@ Foam.set_metric("cache.size", 1024, unit: "By", description: "Entries in the cac
 - `add_up_down_counter(name, value, ...)` adds to a counter that can go down; pass a negative value to decrement.
 - `set_metric(name, value, ...)` sets a gauge to its latest value.
 
-Every helper takes `attributes: {}`, `unit: nil`, and `description: nil`. The instrument is created on the first call for a name, and `unit` and `description` apply then; later calls reuse it. Calls before `init`, when Foam is disabled, or in a setup that only attaches an ingest metric reader are dropped.
+Every helper takes `attributes: {}`, `unit: nil`, and `description: nil`. The instrument is created on the first call for a name, and `unit` and `description` apply then; later calls reuse it. Without `init`, when `create_foam_ingest_metric_reader` is attached to the app's own meter provider, the helpers record on the global meter provider, so their values reach Foam through that reader. Calls before `init` with no ingest metric reader, or when Foam is disabled, are dropped.
 
 TODO(jhartquist): observable (pull) instruments. The metrics SDK has observable counters, gauges, and up-down counters; the helpers cover the four synchronous kinds only. v1 used observables for runtime metrics (GC, heap, threads, RSS), which this version does not send.
 
@@ -314,7 +314,7 @@ The span gets an `exception` event with the type, message, and stack trace, plus
 
 ### `flush(timeout: 10)`
 
-Exports everything Foam's providers still hold. Call it before a point where the process may be killed without a normal exit, for example at the end of a job on a runner that stops the process.
+Exports everything Foam's providers and ingest components still hold. Call it before a point where the process may be killed without a normal exit, for example at the end of a job on a runner that stops the process.
 
 ```ruby
 Foam.flush(timeout: 5)
@@ -324,7 +324,7 @@ Foam.flush(timeout: 5)
 
 ### `shutdown(timeout: 10)`
 
-Flushes and shuts down Foam's providers, including the one behind `log` when an ingest component created it, and releases the global slots Foam claimed. Both `init` and the ingest log factory register it with `at_exit`, so most apps never call it. An app calls it before a process is killed without a normal exit.
+Flushes and shuts down Foam's providers, including the one behind `log` when an ingest component created it, and the ingest components, and releases the global slots Foam claimed. `init` and each ingest factory register it with `at_exit`, so most apps never call it, and an app that never shuts its own providers down still delivers the ingest components' last batch. An app calls it before a process is killed without a normal exit.
 
 `timeout` is in seconds and bounds the whole call. A provider that runs out of time is reported on stderr.
 
@@ -367,7 +367,7 @@ Each `signals` value names the source of the Foam export path for that signal:
 - `:local` - logs only: another SDK owns the global provider and Foam keeps its own so `log` still delivers.
 - `:none` - no Foam export path.
 
-A slot another SDK held before `init` stays with it and reports `:none` (`:local` for logs). Foam builds no provider for that signal, so the metric helpers and `set_baggage` do nothing while `log` still delivers. A slot another SDK takes after `init` reports the same way, but Foam's helpers keep exporting through Foam's own providers. Both cases are warned about once on stderr. Body capture is off while another SDK holds the logs slot. `baggage` is `:global` or `:none`. `profile` is not supported and always reports `:none`.
+A slot another SDK held before `init` stays with it and reports `:none` (`:local` for logs), or `:ingest` when a `create_foam_ingest_*` object was already built for that signal. Foam builds no provider for that signal, so the metric helpers and `set_baggage` do nothing while `log` still delivers. A slot another SDK takes after `init` reports the same way, but Foam's helpers keep exporting through Foam's own providers. Both cases are warned about once on stderr. Body capture is off while another SDK holds the logs slot. `baggage` is `:global` or `:none`. `profile` is not supported and always reports `:none`.
 
 ## Custom spans and other OpenTelemetry APIs
 
@@ -464,6 +464,27 @@ Foam.init(
 )
 ```
 
+If the app loads Datadog's OpenTelemetry support (`require "datadog/opentelemetry"`) and configures the OpenTelemetry SDK itself, `OpenTelemetry::SDK.configure` installs Datadog's span processor on the app's tracer provider, next to the ingest helpers. Two Datadog bugs then affect what Foam receives:
+
+- Once a child span ends, spans started afterwards under the same parent begin new traces. Foam receives the trace split at that point, and logs written afterwards are not correlated to the parent. Datadog's own copy of the trace is complete.
+- `force_flush` and `shutdown` on the app's tracer provider raise inside Datadog's span processor, and the processors after it, the ingest span processor included, are skipped. The app does not need either call for Foam: Foam flushes the ingest components at exit.
+
+Where the app does not need to own the OpenTelemetry SDK, call `init` instead and do not call `OpenTelemetry::SDK.configure`. Datadog's span processor is then not installed, so neither bug applies. Spans made through the OpenTelemetry API go to Foam only, and Datadog keeps what its own integrations record.
+
+### Rollbar
+
+Rollbar sends each report to its API over HTTP, which Foam would otherwise record as spans. Pass Rollbar's endpoint and request header to `init`. This works whether Rollbar reports inline, from a background thread, or from a Sidekiq job:
+
+```ruby
+rollbar = URI(Rollbar.configuration.endpoint)
+
+Foam.init(
+  name: "checkout-api", environment: "production", token: ENV["FOAM_OTEL_TOKEN"],
+  ignored_outbound_hosts: ["#{rollbar.host}:#{rollbar.port}"],
+  ignored_outbound_headers: ["X-Rollbar-Access-Token"]
+)
+```
+
 ## Redaction
 
 Foam masks the values of sensitive keys in everything it exports: span, span event, and span link attributes; log bodies and attributes; metric data point attributes; and resource attributes. A built-in list of credential and personal-data keys always applies, matched by exact name or by a sensitive segment of the key (`legacy_api_key_2`, `stripeToken`). Keys match after normalization: case, `-`, `.`, and camelCase boundaries are folded, so `apiKey`, `api-key`, and `API_KEY` are one key. Header attributes match on the header name. SQL statements arrive from the database instrumentations with every literal replaced with `?` (see Instrumentation) and are redacted like any other string. Captured HTTP bodies are masked by content type with the same keys. When masking a body fails, the whole body is replaced with `********`.
@@ -576,7 +597,7 @@ OpenTelemetry packages outside the instrumentation bundle that Foam does not wir
 
 ## TODO(jhartquist): Safely
 
-Every public method except the ingest factories, and every capture hook, runs under `Internal::Safely`: a failure inside the gem is written to stderr with the `[foam-otel]` prefix and the call returns a fallback, so nothing reaches the app.  One broad boundary per public call and per hook, instead of a wrap at each site, would give one place to rate-limit and report, at the cost of skipping the rest of a hook after its first failure. Additionally, a failure could also be reported to Foam over the path `init` uses for its own outcome.
+Every public method, and every capture hook, runs under `Internal::Safely`: a failure inside the gem is written to stderr with the `[foam-otel]` prefix and the call returns a fallback, so nothing reaches the app.  One broad boundary per public call and per hook, instead of a wrap at each site, would give one place to rate-limit and report, at the cost of skipping the rest of a hook after its first failure. Additionally, a failure could also be reported to Foam over the path `init` uses for its own outcome.
 
 ## Development
 

data/lib/foam/otel/internal/before_send.rb

@@ -12,34 +12,38 @@ module Foam
           "before_send must respond to call, got #{hook.inspect}"
         end
 
-        # Takes the before_send option as init or an ingest factory received it; nil means no hook.
+        # Takes the before_send option as init or an ingest factory received it, after checking it
+        # with invalid_option; nil means no hook.
         def initialize(hook)
-          problem = BeforeSend.invalid_option(hook)
-          raise ArgumentError, problem if problem
-
           @hook = hook
           freeze
         end
 
         # Called by the exporters with each hook event they built; returns what to send, or nil to drop it.
         # A hook that raises, returns something else, or gets an event that cannot be copied changes
-        # nothing, so a broken hook never loses telemetry.
+        # nothing, so a broken hook never loses telemetry. The block receives what the hook raised, so
+        # the exporter can say so once for its batch.
         def event_to_export(hook_event)
           return hook_event unless @hook
 
-          result = @hook.call(deep_copy(hook_event))
+          copy = deep_copy(hook_event) or return hook_event
+          result = @hook.call(copy)
           return if result.nil?
 
           hook_event?(result, hook_event[:type]) ? result : hook_event
-        rescue
+        rescue => error
+          yield error
           hook_event
         end
 
         private
 
-        # Marshal thaws the SDK's frozen attributes, so the hook can edit in place.
+        # Marshal thaws the SDK's frozen attributes, so the hook can edit in place. nil for an event
+        # holding something Marshal cannot dump, such as an IO.
         def deep_copy(hook_event)
           Marshal.load(Marshal.dump(hook_event))
+        rescue
+          nil
         end
 
         def hook_event?(result, type)

data/lib/foam/otel/internal/body_capture.rb

@@ -20,9 +20,15 @@ module Foam
         CHUNK_INDEX = "foam.http.body.chunk.index"
         CHUNK_BYTES = "foam.http.body.chunk.bytes"
 
-        # The context key a capturing middleware marks its span under, so the transport patches
-        # beneath it leave that span alone.
-        OWNED_SPAN = OpenTelemetry::Context.create_key("foam-otel.owned-span")
+        # Where a middleware that captures above its transport, such as Faraday, puts its Owner in
+        # the context. The transport patches beneath it then skip its span and report to it what
+        # they inflated.
+        OWNER = OpenTelemetry::Context.create_key("foam-otel.owner")
+
+        # The span a middleware captures on, above its transport. When the transport inflates the
+        # response, it sets encoding and wire_bytes to the response's encoding and size on the wire;
+        # otherwise both stay nil.
+        Owner = Struct.new(:span, :encoding, :wire_bytes)
 
         # The span attributes of each direction. Size and the header prefix are the semantic
         # conventions' names; the rest are Foam's.
@@ -96,10 +102,19 @@ module Foam
             Process.clock_gettime(Process::CLOCK_MONOTONIC)
           end
 
-          # Runs the block with span owned by the caller, so the transport patches inside it
-          # capture nothing on that span and a library capturing above its transport captures once.
-          def owning(span, &block)
-            OpenTelemetry::Context.with_value(OWNED_SPAN, span, &block)
+          # Runs the block under the owner, so the transport patches inside it capture nothing on
+          # its span and each request is captured once.
+          def owning(owner, &block)
+            OpenTelemetry::Context.with_value(OWNER, owner, &block)
+          end
+
+          # Called by a transport patch that inflated a response: records the response's encoding
+          # and size on the wire on the middleware that owns the request. Does nothing when no
+          # middleware owns it.
+          def inflated_below(encoding, wire_bytes)
+            owner = OpenTelemetry::Context.current.value(OWNER)
+            owner&.encoding = encoding
+            owner&.wire_bytes = wire_bytes
           end
 
           # Returns the recording client span an outbound request belongs to, or nil when nothing is
@@ -114,7 +129,7 @@ module Foam
             span = OpenTelemetry::Trace.current_span
             return unless span.recording? && span.kind == :client
 
-            span unless OpenTelemetry::Context.current.value(OWNED_SPAN).equal?(span)
+            span unless OpenTelemetry::Context.current.value(OWNER)&.span.equal?(span)
           end
 
           # Sets the safe headers of one direction on the span, in the semantic conventions' dash
@@ -140,6 +155,7 @@ module Foam
           @chunks = []
           @observed_bytes = 0
           @kept_bytes = 0
+          @wire_bytes = nil
           @truncated = false
           @open = true
           @lock = Mutex.new
@@ -183,6 +199,13 @@ module Foam
           end
         end
 
+        # Tells the capture that the client library already inflated the observed bytes, and that
+        # wire_bytes is the body's size on the wire. The bytes are then read as text without
+        # decoding, and the size attributes report wire_bytes.
+        def inflated(wire_bytes)
+          @wire_bytes = wire_bytes
+        end
+
         # Ends the capture and emits its span attributes and records. Runs once; later calls do
         # nothing. The records are emitted even when the span has already ended.
         def finalize(complete:)
@@ -217,11 +240,12 @@ module Foam
           text = @body_redactor.redact(text, headers["content-type"]) if text
           truncated = @truncated || cut
           outcome = outcome_of(complete)
+          size = @wire_bytes || @observed_bytes
 
           if @span.recording?
             @span.set_attribute(@attributes[:complete], complete)
             @span.set_attribute(@attributes[:truncated], truncated)
-            @span.set_attribute(@attributes[:size], @observed_bytes)
+            @span.set_attribute(@attributes[:size], size)
           end
           return if outcome == "empty"
 
@@ -233,8 +257,8 @@ module Foam
             COMPLETE => complete,
             TRUNCATED => truncated,
             OUTCOME => outcome,
-            @attributes[:size] => @observed_bytes,
-            CAPTURED_BYTES => text ? text.bytesize : @kept_bytes,
+            @attributes[:size] => size,
+            CAPTURED_BYTES => text ? text.bytesize : 0,
             CHUNK_COUNT => chunks.size
           }.merge(header_attributes(headers))
 
@@ -257,8 +281,8 @@ module Foam
         # body cannot be read as text: an unknown or corrupt encoding, one inflating past the
         # bound, or a binary content type. Only the metadata is exported then, never the bytes.
         def decoded_text(headers)
-          wire = (@chunks.size == 1) ? @chunks.first : @chunks.join
-          decoded = BodyDecoder.decode(wire, headers["content-encoding"])
+          kept = (@chunks.size == 1) ? @chunks.first : @chunks.join
+          decoded = @wire_bytes ? kept : BodyDecoder.decode(kept, headers["content-encoding"])
           return [nil, false] unless decoded
 
           BodyDecoder.text(decoded, ContentType.parse(headers["content-type"])) || [nil, false]

data/lib/foam/otel/internal/body_decoder.rb

@@ -11,6 +11,14 @@ module Foam
       module BodyDecoder
         INFLATED_ENCODINGS = %w[gzip x-gzip deflate].freeze
 
+        # How much of a body of unknown type is looked at to tell whether it is text.
+        SNIFF_BYTES = 1024
+        # Control characters other than tab, line feed, form feed, and carriage return.
+        CONTROL = /[\x00-\x08\x0B\x0E-\x1F\x7F]/
+        # The start of an archive, image, executable, or PDF, long enough that no text starts that
+        # way. A Windows executable is left to the NUL check, since its two-letter MZ is ordinary text.
+        FILE_SIGNATURE = /\A(?:PK\x03\x04|\x1F\x8B|%PDF-|\x89PNG|\xFF\xD8\xFF|GIF8[79]a|Rar!\x1A\x07|7z\xBC\xAF|BZh[1-9]1AY&SY|\x7FELF)/n
+
         module_function
 
         # Undoes the content encoding and returns the decoded bytes. The header lists encodings in
@@ -26,9 +34,12 @@ module Foam
 
         # Returns the body as UTF-8 text and whether it was cut at the capture limit. Invalid
         # bytes are replaced and the cut lands on a character boundary. Returns nil when the
-        # content type is not text.
+        # content type is binary, or unknown or missing and the bytes do not read as text, or the
+        # bytes are a file whatever the type says. A multipart body's parts are judged on their own.
         def text(bytes, content_type)
-          return unless content_type.text?
+          readable = content_type.text? || (!content_type.binary? && textual?(bytes))
+          readable &&= content_type.multipart? || !file?(bytes)
+          return unless readable
 
           cut = bytes.bytesize > Constants::BODY_CAPTURE_BYTES
           text = cut ? bytes.byteslice(0, Constants::BODY_CAPTURE_BYTES) : bytes.dup
@@ -42,6 +53,20 @@ module Foam
           [text, cut]
         end
 
+        # Whether the first KiB holds a NUL or starts with a file signature.
+        def file?(bytes)
+          start = bytes.byteslice(0, SNIFF_BYTES).b
+          start.include?("\x00") || FILE_SIGNATURE.match?(start)
+        end
+
+        # Whether the start of the bytes is UTF-8 without control characters, allowing a
+        # character cut off where the look ends before the bytes do.
+        def textual?(bytes)
+          start = bytes.byteslice(0, SNIFF_BYTES).force_encoding(Encoding::UTF_8)
+          start = whole_utf8(start) if bytes.bytesize > SNIFF_BYTES
+          start.valid_encoding? && !CONTROL.match?(start)
+        end
+
         # Splits the text into pieces of at most the chunk size that end on character boundaries.
         def chunks(text)
           pieces = []

data/lib/foam/otel/internal/body_redactor.rb

@@ -30,8 +30,12 @@ module Foam
         HTML_INPUT = /<input\b[^>]*>/i
         HTML_INPUT_NAME = /\bname\s*=\s*(["'])(.*?)\1/i
         HTML_INPUT_VALUE = /(\bvalue\s*=\s*)(["'])(.*?)\2/i
+        HTML_PASSWORD_TYPE = /\btype\s*=\s*(["']?)password\1(?=[\s>\/])/i
         YAML_PAIR = /\A(\s*(?:-\s*)?)(["']?)([A-Za-z_][\w.-]*)\2(\s*:\s*)(.*)\z/
         YAML_BLOCK_SCALAR = /\A[>|]/
+        YAML_ANCHOR = /\A&[^\s,\[\]{}]+\s*/
+        # An alias token: an asterisk after the line start, whitespace, or a flow indicator.
+        YAML_ALIAS = /(?<![^\s\[{,])\*([^\s,\[\]{}]+)/
         MULTIPART_HEADER_SEPARATOR = /\r\n\r\n|\n\n|\r\r/
         CONTENT_DISPOSITION = /\Acontent-disposition\s*:/i
         CONTENT_DISPOSITION_NAME = /\bname\s*=\s*(?:"([^"]*)"|([^;\s]*))/i
@@ -44,8 +48,9 @@ module Foam
           freeze
         end
 
-        # Masks the values under sensitive keys as the content type is read; a type it cannot read
-        # is left alone, and a failure masks the whole body rather than leak it.
+        # Masks the values under sensitive keys as the content type is read; a body of unknown or
+        # missing type is read as plain text, a binary type is left alone, and a failure masks the
+        # whole body rather than leak it.
         def redact(text, content_type)
           type = ContentType.parse(content_type)
           redacted =
@@ -65,7 +70,7 @@ module Foam
               redact_xml(text)
             elsif type.yaml?
               redact_yaml(text)
-            elsif type.text?
+            elsif !type.binary?
               redact_plain_text(text)
             end
           redacted || text
@@ -242,8 +247,7 @@ module Foam
           end
 
           output = output.gsub(HTML_INPUT) do |tag|
-            name = HTML_INPUT_NAME.match(tag)&.[](2)
-            next tag unless name && sensitive_name?(name)
+            next tag unless sensitive_input?(tag)
 
             replaced = tag.sub(HTML_INPUT_VALUE, "\\1\\2#{Constants::REDACTED_VALUE}\\2")
             changed = true unless replaced == tag
@@ -253,6 +257,14 @@ module Foam
           output if changed
         end
 
+        # An input of type password, or one named for a sensitive key or a query key.
+        def sensitive_input?(tag)
+          return true if HTML_PASSWORD_TYPE.match?(tag)
+
+          name = HTML_INPUT_NAME.match(tag)&.[](2)
+          name ? sensitive_name?(name) : false
+        end
+
         # The text of each sensitive element is masked; one holding markup or never closed gets the
         # plain mask, since its value cannot be read whole. Positions are bytes throughout, so the
         # body's length costs nothing.
@@ -304,26 +316,39 @@ module Foam
           colon ? name[(colon + 1)..] : name
         end
 
-        # A sensitive key's value is masked on its line; a block scalar under one spans lines that
-        # cannot be told apart from the rest, so the whole document is masked instead.
+        # A sensitive key's value is masked on its line, keeping an anchor on it, and every later
+        # alias of that anchor gets the same mask. A block scalar under a sensitive key spans lines
+        # that cannot be told apart from the rest, so the whole document is masked instead.
         def redact_yaml(text)
           block_scalar = false
+          masked_anchors = {}
           redacted = redact_by_line(text) do |line|
             match = YAML_PAIR.match(line)
-            next unless match
+            kind = match && @redactor.mask_kind(match[3])
+            next mask_yaml_aliases(line, masked_anchors) unless kind
 
             prefix, quote, key, separator, value = match.captures
-            kind = @redactor.mask_kind(key)
-            next unless kind
-
             value = value.strip
             block_scalar = true if YAML_BLOCK_SCALAR.match?(value)
+            anchor = value[YAML_ANCHOR].to_s
+            value = value.delete_prefix(anchor)
             value = value[1..-2] if value.match?(/\A(["']).*\1\z/)
-            "#{prefix}#{quote}#{key}#{quote}#{separator}#{@redactor.mask(kind, value)}"
+            masked = @redactor.mask(kind, value)
+            masked_anchors[anchor.strip.delete_prefix("&")] = masked unless anchor.empty?
+            "#{prefix}#{quote}#{key}#{quote}#{separator}#{anchor}#{masked}"
           end
           block_scalar ? Constants::REDACTED_VALUE : redacted
         end
 
+        # The line with each alias of a masked anchor replaced by that anchor's mask; nil when it
+        # has none.
+        def mask_yaml_aliases(line, masked_anchors)
+          return if masked_anchors.empty?
+
+          replaced = line.gsub(YAML_ALIAS) { |token| masked_anchors.fetch(Regexp.last_match(1), token) }
+          replaced unless replaced == line
+        end
+
         # Text with no declared structure is read as JSON when it looks like it, then as YAML, then
         # scanned for key-value pairs.
         def redact_plain_text(text)
@@ -336,7 +361,7 @@ module Foam
         end
 
         # Each part is redacted on its own: a part named for a sensitive key is masked whole, a file
-        # part is left alone, and any other part is read by its own content type.
+        # part is omitted, and any other part is read by its own content type.
         def redact_multipart(text, boundary)
           return if boundary.empty? || boundary.length > MAX_BOUNDARY_LENGTH
 
@@ -369,12 +394,23 @@ module Foam
         def redact_part(headers, body)
           name = content_disposition_name(headers)
           return Constants::REDACTED_VALUE if name && sensitive_name?(name)
-          return if FILENAME.match?(headers)
 
-          redacted = redact(body, part_content_type(headers) || "text/plain")
+          type = part_content_type(headers)
+          return omit(body) if FILENAME.match?(headers) || ContentType.parse(type).binary? || file_part?(body)
+
+          redacted = redact(body, type || "text/plain")
           redacted unless redacted == body
         end
 
+        def omit(body)
+          Constants::OMITTED_VALUE unless body == Constants::OMITTED_VALUE
+        end
+
+        # Multipart text holds each byte as one character, so the part is turned back into its bytes.
+        def file_part?(body)
+          BodyDecoder.file?(body.encode(Encoding::ISO_8859_1, invalid: :replace, undef: :replace))
+        end
+
         # A form field or part name is matched against the key lists and the query key list alike.
         def sensitive_name?(name)
           @redactor.mask_kind(name) || @redactor.query_key?(name)

data/lib/foam/otel/internal/clients/excon.rb

@@ -13,6 +13,9 @@ module Foam
           # response or the error.
           REQUEST_CAPTURE = :foam_request_capture
 
+          # The encodings Excon's Decompress middleware inflates.
+          DECOMPRESSED = %w[gzip x-gzip deflate].freeze
+
           def self.install
             return unless Instrumentations.instance(INSTRUMENTATION).installed?
 
@@ -75,11 +78,13 @@ module Foam
           end
 
           # Captures the parsed response body. A response handed to a response block never reaches
-          # the datum, so it is skipped.
+          # the datum, so it is skipped. When a middleware above owns the request, it is told
+          # instead what Decompress is about to inflate.
           def capture_response(datum)
             span = BodyCapture.client_span
             body = datum.dig(:response, :body)
-            return unless span && body.is_a?(String) && !datum.key?(:response_block)
+            return unless body.is_a?(String) && !datum.key?(:response_block)
+            return report_inflating(datum, body) unless span
 
             capture = BodyCapture.start(
               span, State.body_redactor, side: :client, direction: :response
@@ -90,6 +95,18 @@ module Foam
             capture.finalize(complete: true)
           end
 
+          # Tells the owning middleware the encoding and size on the wire of a body Decompress is
+          # about to inflate. Decompress inflates only the last encoding listed, and only one it
+          # knows, then removes it from content-encoding.
+          def report_inflating(datum, body)
+            return unless datum[:middlewares].include?(::Excon::Middleware::Decompress)
+
+            encoding = headers(datum[:response][:headers])["content-encoding"]&.split(",")&.last&.strip
+            return unless DECOMPRESSED.include?(encoding&.downcase)
+
+            BodyCapture.inflated_below(encoding, body.bytesize)
+          end
+
           # Matched without regard to case, since the request headers keep whatever case the app
           # wrote.
           def headers(hash)

data/lib/foam/otel/internal/clients/faraday.rb

@@ -31,6 +31,23 @@ module Foam
             return unless Instrumentations.instance(INSTRUMENTATION).installed?
 
             ::Faraday::Connection.prepend(Connection)
+            instrument_default_connection
+          end
+
+          # Faraday.get, Faraday.post and the other class-level calls reuse a cached default
+          # connection. If it was built before init (Datadog's integration does this),
+          # OpenTelemetry's and our initialize patches never ran on it, so add the tracer and this
+          # middleware now, matching what a new connection gets. Once that connection has sent a
+          # request, Faraday freezes its middleware, so it is left as is, traced by the HTTP client
+          # beneath it.
+          def self.instrument_default_connection
+            connection = ::Faraday.instance_variable_get(:@default_connection)
+            return if connection.nil? || connection.builder.locked?
+
+            tracer = ::Faraday::Middleware.lookup_middleware(:open_telemetry)
+            [tracer, self].each do |middleware|
+              connection.use(middleware) unless connection.builder.handlers.any? { |handler| handler.klass == middleware }
+            end
           end
 
           def initialize(app)
@@ -39,7 +56,8 @@ module Foam
 
           # Captures the request before the adapter runs and the response once it has it. The span
           # found here is kept for the response side, since inside the owned context client_span
-          # returns nil by design.
+          # returns nil by design. The transport patches beneath report to the owner what they
+          # inflated, so the response is described as it was on the wire.
           def call(env)
             span = Safely.call("body capture") { BodyCapture.client_span }
             return @app.call(env) unless span
@@ -51,19 +69,18 @@ module Foam
               BodyCapture.set_header_attributes(span, :request, request_headers(env, body))
             end
             request = Safely.call("request body capture") { capture_request(env, span, body) }
+            owner = BodyCapture::Owner.new(span)
             response = nil
             begin
-              response = BodyCapture.owning(span) { @app.call(env) }
+              response = BodyCapture.owning(owner) { @app.call(env) }
             ensure
               request&.finalize(complete: !response.nil?)
             end
             response.on_complete do |completed|
               Safely.call("response header capture") do
-                BodyCapture.set_header_attributes(
-                  span, :response, headers(completed.response_headers)
-                )
+                BodyCapture.set_header_attributes(span, :response, sent_headers(completed, owner))
               end
-              Safely.call("response body capture") { capture_response(completed, span) }
+              Safely.call("response body capture") { capture_response(completed, span, owner) }
             end
           end
 
@@ -91,26 +108,38 @@ module Foam
           end
 
           # Only a String body is the bytes the adapter received: one a middleware below this one has
-          # already parsed, or one streamed to the app through on_data, is skipped.
-          def capture_response(env, span)
+          # already parsed, or one streamed to the app through on_data, is skipped. A body the
+          # transport inflated is reported at its size on the wire.
+          def capture_response(env, span, owner)
             body = env.body
             return if env.request.stream_response? || !body.is_a?(String)
 
             capture = BodyCapture.start(
               span, State.body_redactor, side: :client, direction: :response
-            ) { body_headers(env, body) }
+            ) { body_headers(env, body, owner) }
             return unless capture
 
             capture.observe(body)
+            capture.inflated(owner.wire_bytes) if owner.wire_bytes
             capture.finalize(complete: true)
           end
 
-          # The safe response headers for the body capture. Every adapter inflates a gzip or deflate
-          # response for the caller but leaves content-encoding as the server sent it, and that
-          # header would make the capture try to inflate plain text. It is kept only when the body
-          # still starts like compressed data.
-          def body_headers(env, body)
+          # The safe response headers as the server sent them. Net::HTTP and Excon remove
+          # content-encoding once they inflate the body, so the encoding their patch reported is restored.
+          def sent_headers(env, owner)
             found = headers(env.response_headers)
+            found["content-encoding"] = owner.encoding if owner.encoding
+            found
+          end
+
+          # The safe response headers for the body capture. When an adapter Foam does not patch
+          # inflates the body, content-encoding still names the compression and the capture would
+          # try to inflate plain text, so it is dropped unless the body still starts like
+          # compressed data. A body a patched transport inflated keeps it.
+          def body_headers(env, body, owner)
+            found = sent_headers(env, owner)
+            return found if owner.encoding
+
             lead = COMPRESSED_LEAD[found["content-encoding"]&.downcase]
             found.delete("content-encoding") if lead && !body.start_with?(lead)
             found