foam-otel 3.0.0.alpha.5 → 3.0.0.alpha.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 34ec502fdc38ac38099bba65f1fc37c91f1edb79476bff598c6a0dda7245fcf0
-  data.tar.gz: 9596d8c1d746b66103b87587f5c52006e024d0498d93816912ed916014e563e4
+  metadata.gz: 929a47fe32e662ab012e7995078f1b0bab1f0ef3b58e3b7a9ca8b3b109c6ca29
+  data.tar.gz: 1608559fa17bd0f683492267a2db2f7daa4f3989d4a79f06d67f38e27632174d
 SHA512:
-  metadata.gz: 85207e2233f8a986bb874f68215960f20216e010978e5d1355808ec2ed6bfb44f19c9b8f8947cc3164cca65864326dab8b5ef95fb8ef6c37a1150fe06923ae2a
-  data.tar.gz: 3a3bc463b06b32e1d1bd4b6c439e5f8a92b031859301f8681e78a32e4e1db655ef73d4439a0755e336abc9fc1b068bca48049172046732da4adb514eb12c00cb
+  metadata.gz: b8484befaf21fec24e8ce725931c368d9aaac7518916a8f0aebc69f02a5345fc10a0a73c6b25167d555554133eb84b7f6479d3ad676697beb2c03758352dd592
+  data.tar.gz: 56032f62347e799791dce8d37db046dd6fe6e4c8e372dd4aff7fa7c54813992b89e61b854b9b6565aeaf9fd7dd08c53747d5c9b8eae897b96ae869622594987e

data/README.md

@@ -40,9 +40,9 @@ Foam.init(
 )
 ```
 
-During boot, `init` adds Foam's Rack middleware to the app directly after the OpenTelemetry Rack middleware, so server spans get the same header and body capture as under plain Rack. `init` must run before the app initializes: an initializer file or `config/application.rb`. An `init` from `after_initialize` or later warns on stderr and adds no middleware.
+`init` must run before the app initializes, either with an initializer file or in `config/application.rb`. An `init` from `after_initialize` or later warns on stderr and adds no middleware. During boot it inserts `Foam::Rack::Middleware` directly after the OpenTelemetry Rack middleware. `config.ru` stays as generated.
 
-Nothing else changes; `config.ru` stays as generated. A `RUBYOPT` preload runs before Rails loads, so it installs no Rails instrumentation and adds no middleware.
+Do not load Foam with a `RUBYOPT` preload. It runs before Rails loads, so it installs no Rails instrumentation and adds no middleware.
 
 ### Plain Ruby and Rack
 
@@ -64,7 +64,7 @@ use Foam::Rack::Middleware
 run App
 ```
 
-Foam's middleware sets the `content-type`, `content-length`, and `content-encoding` headers on server spans and, in `:advanced` mode, captures bodies. See Network capture.
+Foam's middleware records the `content-type`, `content-length`, and `content-encoding` headers as server span attributes. In `:advanced` mode it also sends the bodies as log records correlated to the span. See Network capture.
 
 ## Instrumentation
 
@@ -78,17 +78,13 @@ TODO(jhartquist): Revisit and review this compatibility guidance.
 
 ### Instrumentation versions
 
-Bundler keeps one version of each gem in the application. By the time Foam starts, those versions are already fixed. `additional_instrumentations` only configures an instrumentation that Bundler installed and Ruby loaded; it cannot choose or install another version.
+Bundler keeps one version of each gem, fixed before Foam starts. Each instrumentation checks its library's version when `init` runs and is left out of `state[:instrumentations]` when the version is unsupported. `additional_instrumentations` cannot bypass the check, choose another version, or install a gem.
 
-If a bundled instrumentation does not support the library version your app uses, pin a compatible `opentelemetry-instrumentation-*` gem in the `Gemfile` and run `bundle install` again.
-
-If Bundler accepts the pin, restart the app and the whole bundle will use that instrumentation version. If it conflicts with `foam-otel` or `opentelemetry-instrumentation-all`, those versions cannot live in the same bundle. At that point, change the application library, use a compatible Foam release, or accept the telemetry gap: the app can keep running, but Foam will not receive the spans that instrumentation would have produced.
-
-Even a successful `bundle install` only means the declared gem dependencies agree. The instrumentation checks the target library when `init` runs. If the target is unsupported, Foam leaves that instrumentation out of `state[:instrumentations]`. `additional_instrumentations` cannot bypass the check or load a second version of the gem.
+For an unsupported library version, change the app's library, use a compatible Foam release, or accept the gap. The app keeps running, but Foam receives no spans from that library.
 
 The `pg`, `mysql2`, and `trilogy` instrumentations obfuscate SQL statements (`db_statement: :obfuscate`, their default), replacing every literal with `?`, so `WHERE email = 'ada@example.com'` is exported as `WHERE email = ?`. A statement over 2000 characters is exported as a placeholder instead. The other instrumentations keep their upstream defaults.
 
-To disable one instrumentation or change its options, pass it in `additional_instrumentations` or use the upstream environment variables, which take precedence: `OTEL_RUBY_INSTRUMENTATION_<NAME>_ENABLED=false` and `OTEL_RUBY_INSTRUMENTATION_<NAME>_CONFIG_OPTS`, where `<NAME>` is the instrumentation's class name after `OpenTelemetry::Instrumentation::`, upper-cased, with `::` replaced by `_`: `PG`, `NET_HTTP`, `ACTIVERECORD`.
+To disable one instrumentation or change its options, pass it in `additional_instrumentations` or set the upstream environment variables `OTEL_RUBY_INSTRUMENTATION_<NAME>_ENABLED=false` and `OTEL_RUBY_INSTRUMENTATION_<NAME>_CONFIG_OPTS`, which take precedence. `<NAME>` is the instrumentation's registry name after `OpenTelemetry::Instrumentation::`, upper-cased, with `::` replaced by `_`, such as `PG`, `NET_HTTP`, or `ACTIVERECORD`.
 
 TODO(jhartquist): `db_capture` init option. The `pg`, `mysql2`, and `trilogy` instrumentations can include the statement (`db_statement: :include`) and `pg` can report row counts (`db_response_returned_rows`, `db_response_affected_rows`), but none hands over results. Result capture needs Foam patches on the drivers, opt-in, through the same redaction and size caps as bodies. See Expanded data capture at the end of this document.
 
@@ -96,13 +92,13 @@ TODO(jhartquist): Gen AI providers beyond `anthropic`. There is no OpenTelemetry
 
 ## Public API
 
-Every method lives on the `Foam` module. Other than the ingest factories, none of them raise exceptions. `set_baggage`, the metric helpers, and `log` need a Foam export path to function properly, so they are no-ops before `init`; the trace context helpers and `record_exception` work on the current OpenTelemetry context whether or not `init` ran.
+Every method lives on the `Foam` module. Other than the ingest factories and `init`'s keyword errors, none of them raise exceptions. `set_baggage`, the metric helpers, and `log` need a Foam export path to function properly, so they are no-ops before `init`; the trace context helpers and `record_exception` work on the current OpenTelemetry context whether or not `init` ran.
 
 ### `init`
 
 Sets up tracing, metrics, logs, automatic instrumentation, W3C `tracecontext` and `baggage` propagation, and OTLP export to Foam.
 
-`init` claims the four OpenTelemetry global slots only when they are free: the tracer provider, meter provider, logger provider, and propagator. A slot another SDK set before `init` stays with that SDK. Foam warns once on stderr, reports the signal in `state`, and builds no pipeline for it, so any `additional_*` components for that signal are unused. Foam always builds its own logger provider, so `log` delivers either way. `shutdown` releases the slots Foam claimed.
+`init` claims the four OpenTelemetry global slots only when they are free: the tracer provider, meter provider, logger provider, and propagator. A slot another SDK sets before `init` stays with that SDK. Foam warns once on stderr, reports the signal in `state`, and builds no pipeline for it, so any `additional_*` components for that signal are unused. Foam always builds its own logger provider, so `log` delivers either way. `shutdown` releases the slots that Foam claimed.
 
 Call this once. Use it when Foam should run OpenTelemetry for the process.
 
@@ -134,6 +130,7 @@ Foam.init(
   disable_log_sending: false,
   network_capture: :basic,
   ignored_outbound_hosts: [],
+  ignored_outbound_headers: [],
   redact: nil,
   before_send: nil
 )
@@ -147,10 +144,11 @@ Foam.init(
 - `token` - the Foam ingest token, sent as a bearer token on every export. Required when Foam is enabled; a missing token is reported on stderr and leaves Foam uninitialized.
 - `version` - the service version, sent as `service.version` when given.
 - `sample_rate` - the fraction of root traces kept, from `0.0` to `1.0`; child spans follow their parent. Leave it at `1.0` unless volume forces sampling. `OTEL_TRACES_SAMPLER` is not consulted.
-- `additional_span_processors`, `additional_log_record_processors`, `additional_metric_readers` - the app's own OpenTelemetry components, added to Foam's providers after Foam's own exporters.
+- `additional_span_processors`, `additional_log_record_processors`, `additional_metric_readers` - span processors, log record processors, and metric readers the app builds itself, such as an exporter to another backend. Foam adds them to its providers after its own exporters.
 - `additional_resource_attributes` - attributes merged into the resource of every signal. `service.name`, `service.version`, and `deployment.environment.name` come from the options above and `service.instance.id` from Foam; a value for one of them here is reported on stderr and ignored.
 - `additional_instrumentations` - instrumentations to install next to the bundle. An entry is an instrumentation's registry name, which is its class name without the trailing `::Instrumentation` (`"OpenTelemetry::Instrumentation::Logger"`), or a `[name, config]` pair to install it with options (`["OpenTelemetry::Instrumentation::PG", {db_statement: :include}]`). Require the instrumentation's gem before `init`; an entry whose instrumentation is not loaded fails `init`. An entry naming a bundled instrumentation replaces Foam's configuration for it.
-- `ignored_outbound_hosts` - skips tracing of outbound HTTP requests to these hostnames. The endpoint host is always skipped. Use it for hosts whose traffic would drown the traces, such as a sidecar health check on `localhost` or another vendor's OTLP ingest host. Hosts are compared as written; `localhost` and `127.0.0.1` are different hosts. A request to a skipped host still carries a `traceparent` marked not sampled, so an OpenTelemetry service behind it records nothing for that request.
+- `ignored_outbound_hosts` - skips tracing of outbound HTTP requests to these hosts, each written `host` or `host:port`. The endpoint host is always skipped. Use it for hosts whose traffic would drown the traces, such as a sidecar health check on `localhost`, another SDK's agent on `127.0.0.1:8126`, or another vendor's OTLP ingest host. Hosts are compared as written; `localhost` and `127.0.0.1` are different hosts, and an entry with a port matches that port only. A request to a skipped host still carries a `traceparent` marked not sampled, so an OpenTelemetry service behind it records nothing for that request.
+- `ignored_outbound_headers` - skips tracing of outbound Net::HTTP requests carrying any of these request headers, compared case-insensitively. Use it for another SDK's traffic to its backend when there is no host to ignore, as over a Unix socket. A connection opened before the request, as `Net::HTTP.start` does, has its connect span before any header is seen, so over TCP ignore the host as well.
 - `disable_log_sending` - `true` stops shipping the app's `Logger` lines to Foam. `Foam.log` is unaffected. See Loggers.
 - `network_capture` - `:off`, `:basic`, or `:advanced`. Use `:advanced` unless security and compliance rule it out. See Network capture.
 - `redact` - keys to mask beyond the built-in list, as `{secrets: [...], pii: [...]}`. A value under a `secrets` key keeps its last four characters when it is 12 characters or longer; a value under a `pii` key is fully masked. See Redaction.
@@ -168,15 +166,15 @@ Every signal carries a resource with these attributes:
 
 A process forked after `init` carries its own `service.instance.id` and `process.pid`. `OTEL_RESOURCE_ATTRIBUTES` can add further attributes. It cannot override any of the attributes above, and `OTEL_SERVICE_NAME` cannot override `name`.
 
-`init` never raises. A failure or an option holding an invalid value is reported on stderr with a `[foam-otel]` prefix and leaves `state` uninitialized.
+`init` raises only `ArgumentError`, for a missing `name` or `environment` or an unknown keyword. Any other failure, or an option holding an invalid value, is reported on stderr with a `[foam-otel]` prefix and leaves `state` uninitialized.
 
 `init` also reports its outcome to Foam: one log record with the message and the `state`, sent straight to the endpoint with the token. The report is sent in the background and never blocks the app. When the process exits before the report has been delivered, exit waits up to two seconds for the send to finish; a report that cannot be delivered is dropped. Nothing is sent without a token.
 
 ### `set_endpoint(url)`
 
-For tests and local development only. Sets the OTLP/HTTP base URL, without a signal path, that every export and report goes to, such as a local collector. Production apps do not call it: the default is Foam's ingest endpoint, `https://otel.api.foam.ai`.
+For tests and local development only. Call it before `init` to send every export to another OTLP/HTTP base URL, without a signal path, such as a local collector. The default is Foam's ingest endpoint, `https://otel.api.foam.ai`.
 
-Call it before `init` or before creating an ingest component. Exporters keep the endpoint they were built with, so a later call changes only later reports and ingest components; after `init` it is also reported on stderr. `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, and their per-signal forms are not consulted. The upstream exporters still read the timeout, compression, and certificate variables.
+`OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, and their per-signal forms are not consulted. The upstream exporters still read the timeout, compression, and certificate variables.
 
 ### `create_foam_ingest_span_processor(name, environment, token, redact: nil, before_send: nil)`
 
@@ -223,7 +221,7 @@ All three export to the endpoint set by `set_endpoint` and take the same `redact
 
 Foam identifies the service by the `name` and `environment` passed in: on the copy sent to Foam they overwrite the provider's `service.name` and `deployment.environment.name`, and `foam.ingest.host` names the host of the endpoint the copy goes to. The provider's own telemetry is left unchanged. Pass the same values to all three.
 
-Unlike every other public method, these are not guarded: they raise `ArgumentError` when `name`, `environment`, or `token` is blank or `redact` or `before_send` is invalid, and let any other failure through. Returning nothing instead would fail later, inside the app's provider, with less to go on.
+Unlike every other public method, these are not guarded: they raise `ArgumentError` when `name`, `environment`, or `token` is blank or `redact` or `before_send` is invalid, and let any other failure through.
 
 Do not combine `init` and an ingest component for the same signal. The ingest components do not install instrumentations; register the ones the app needs with the OpenTelemetry SDK directly.
 
@@ -254,6 +252,8 @@ end
 
 `inject_trace_context` and `extract_trace_context` always use the W3C `traceparent`, `tracestate`, and `baggage` headers, whatever propagator holds the global slot. Foam does not read `OTEL_PROPAGATORS`. An app that needs a different propagator registers it before `init`, and Foam leaves it in place.
 
+On outbound requests, Foam writes its propagation headers after any other SDK's. Another SDK's own propagation headers are left as written.
+
 ### `set_baggage(key, value)`
 
 Adds a key-value pair to the current context's baggage. Baggage travels with the trace context to every downstream service, where `baggage(key)` reads it; it is not written onto spans. The value lasts for the rest of the current scope, for example the rest of the request when called from a Rails controller, and is dropped when that scope ends. Outside any span or context scope the call is ignored and warned about once, so a value can never outlive the work it belongs to. No-op before `init`.
@@ -280,13 +280,13 @@ Foam.set_metric("cache.size", 1024, unit: "By", description: "Entries in the cac
 - `add_up_down_counter(name, value, ...)` adds to a counter that can go down; pass a negative value to decrement.
 - `set_metric(name, value, ...)` sets a gauge to its latest value.
 
-Every helper takes `attributes: {}`, `unit: nil`, and `description: nil`. The instrument is created on the first call for a name, and `unit` and `description` apply then; later calls reuse it. Values are exported on Foam's metrics pipeline under the `foam-otel` scope. Calls before `init`, when Foam is disabled, or in a setup that only attaches an ingest metric reader are dropped.
+Every helper takes `attributes: {}`, `unit: nil`, and `description: nil`. The instrument is created on the first call for a name, and `unit` and `description` apply then; later calls reuse it. Calls before `init`, when Foam is disabled, or in a setup that only attaches an ingest metric reader are dropped.
 
 TODO(jhartquist): observable (pull) instruments. The metrics SDK has observable counters, gauges, and up-down counters; the helpers cover the four synchronous kinds only. v1 used observables for runtime metrics (GC, heap, threads, RSS), which this version does not send.
 
 ### `log(body, severity: :info, attributes: {})`
 
-Sends a log record to Foam on Foam's own export path. The record goes only to Foam: nothing is written to the app's log output, and nothing is emitted through a logger provider Foam does not own. A record emitted inside a request carries the trace and span ids of the current span.
+Sends a log record to Foam on Foam's own export path. The record goes only to Foam: nothing is written to the app's log output, and nothing is emitted through a logger provider Foam does not own. A record emitted inside a request carries the trace and span IDs of the current span.
 
 Use it to send a log directly, for example when the app's logger is not captured automatically (see Loggers).
 
@@ -328,7 +328,7 @@ Flushes and shuts down Foam's providers, including the one behind `log` when an
 
 `timeout` is in seconds and bounds the whole call. A provider that runs out of time is reported on stderr.
 
-`at_exit` covers a normal exit, an unhandled exception, and a signal the process handles by exiting. `exit!`, `KILL`, and a crash lose the last batch, and so does a batch that outlasts the timeout. An exception outside any span is invisible unless the app logs it.
+`at_exit` covers a normal exit, an unhandled exception, and a signal the process handles by exiting. The last batch is lost when the process ends without running `at_exit`, as with `exit!`, a `KILL` signal, or a crash of the Ruby process itself, and when a batch outlasts the timeout. An exception outside any span is invisible unless the app logs it.
 
 Foam's span processor holds 4096 finished spans and drops the oldest when full; `OTEL_BSP_MAX_QUEUE_SIZE` overrides the size. The ingest span processor and the log record processors keep the OpenTelemetry default of 2048.
 
@@ -367,7 +367,7 @@ Each `signals` value names the source of the Foam export path for that signal:
 - `:local` - logs only: another SDK owns the global provider and Foam keeps its own so `log` still delivers.
 - `:none` - no Foam export path.
 
-A slot another SDK held before `init` stays with it and reports `:none` (`:local` for logs) from the start. Foam builds no provider for that signal, so the metric helpers and `set_baggage` do nothing while `log` still delivers. `:global` is checked against the live OpenTelemetry global on each call, so a slot another SDK takes after `init` reports the same way; in that case only the slot changes hands, and the helpers keep using Foam's own providers. Both cases are warned about once on stderr. Body capture is off while another SDK holds the logs slot. `baggage` is `:global` or `:none`. `profile` is not supported and always reports `:none`.
+A slot another SDK held before `init` stays with it and reports `:none` (`:local` for logs). Foam builds no provider for that signal, so the metric helpers and `set_baggage` do nothing while `log` still delivers. A slot another SDK takes after `init` reports the same way, but Foam's helpers keep exporting through Foam's own providers. Both cases are warned about once on stderr. Body capture is off while another SDK holds the logs slot. `baggage` is `:global` or `:none`. `profile` is not supported and always reports `:none`.
 
 ## Custom spans and other OpenTelemetry APIs
 
@@ -383,59 +383,43 @@ TODO(jhartquist): custom spans and the OpenTelemetry API: getting a tracer, `in_
 
 There is no option to record other headers.
 
-### Header capture
+Server spans get the capture from `Foam::Rack::Middleware`, placed after the OpenTelemetry Rack middleware (added by `init` in Rails apps). Client spans get it from `Net::HTTP`, Faraday with any adapter, Excon, and HTTPClient when their gem is loaded before `init`. RestClient and HTTParty are captured on the `Net::HTTP` span beneath them. `http`, `httpx`, and `ethon` get no capture, and neither does a Faraday connection built before `init`.
 
-Server spans get the headers from `Foam::Rack::Middleware`, placed after the OpenTelemetry Rack middleware (added by `init` in Rails apps).
-
-Client spans get the headers from Foam's hooks on `Net::HTTP`, Excon, HTTPClient, and Faraday, applied unless `network_capture` is `:off`. The `Net::HTTP` hook is always installed; the others install only when their gem is loaded before `init`, and Faraday's covers connections built after `init`. Faraday is covered on the Faraday span whatever the adapter. `http`, `httpx`, `ethon`, and `restclient` are not covered.
-
-Faraday over the `httpclient` adapter sends each request with the trace marked not sampled, so a Foam server on the other side records no span for it. The issue is in the upstream HTTPClient instrumentation, which ignores the untraced context Faraday runs its adapter in; Net::HTTP and Excon honor it.
+Streamed bodies are captured for a Rack response the server iterates, a Rack request body as the app reads it, a `Net::HTTP` request sent through `body_stream`, and an HTTPClient response read in a block. Other streamed bodies are not, such as a Rack `call` body, a file sent by path, a hijacked connection, a client request body sent from an IO, or a client response read in a block or through a streaming callback.
 
 TODO(jhartquist): cover `http`, `httpx`, `ethon`, and `restclient`.
 
-TODO(jhartquist): the HTTPClient trace break above. The fix is `return super if OpenTelemetry::Common::Utilities.untraced?` at the top of the instrumentation's `do_get_block`, upstream or redefined at init until a release carries it.
-
-### Body capture
-
-In `:advanced` mode each captured body arrives as `foam.http.body.chunk` log records correlated to the span, and the span carries the body's size and content and whether the capture was complete.
+### Bodies
 
+- Each captured body arrives as `foam.http.body.chunk` log records correlated to the span, each carrying a chunk of the body's text as its log body. The span carries the body's size and whether the capture was complete.
 - Bodies with a textual content type, or one declaring a charset, are captured. gzip and deflate are decoded. Other encodings and binary types yield one record with the size only; an empty body yields none.
 - At most 1 MiB of a body is kept, sent as records of at most 64 KiB each. The rest counts toward the size.
-- Bodies are parsed by content type and redacted with the same keys as span attributes.
-
-### Rack bodies
+- Bodies are parsed by content type and redacted with the same keys as span attributes. See Redaction.
+- A server request body is captured whether or not the app reads it when the input is rewindable, as it is under Puma. Otherwise only what the app reads is captured.
+- A streamed server response is observed as it is sent, without buffering or delaying it. Its records are emitted when the body closes; a stream idle for over 60 seconds is closed as incomplete and the rest is dropped.
 
-Server bodies need `Foam::Rack::Middleware` after the OpenTelemetry Rack middleware, as shown under "Plain Ruby and Rack" (added by `init` in Rails apps).
+### Known issue
 
-- The request body is captured whether or not the app reads it when the input is rewindable, as it is under Puma, which buffers request bodies. Otherwise what the app reads is captured.
-- A streamed response is observed as the server sends it, without buffering or delaying it. The records are emitted when the body closes; a stream idle for over 60 seconds is closed as incomplete and the rest is dropped.
-- A streaming body (`call`), a file sent by path (`to_path`), and a hijacked connection are not captured.
+Faraday over the `httpclient` adapter sends each request with the trace marked not sampled, so a Foam service on the other side records no span for it. Faraday over `Net::HTTP` or Excon is unaffected.
 
-### Client bodies
-
-Client bodies are captured in `:advanced` by the same hooks that set the headers.
-
-- `Net::HTTP`: a request is captured whether sent as a string or through `body_stream`. A response read in a block or into an IO is not captured.
-- Faraday: bodies are captured on the Faraday span through Foam's Faraday middleware, whatever the adapter. A request whose body is an IO and a response streamed with `on_data` are not captured.
-- Excon: a request sent with `request_block` and a response received with `response_block` are not captured.
-- HTTPClient: a request whose body is an IO or multipart parts is not captured. A response read in a block is captured.
-- RestClient and HTTParty: captured on the `Net::HTTP` span beneath them.
-- `http`, `httpx`, and `ethon`: no bodies, since they get no header capture either.
+TODO(jhartquist): the HTTPClient trace break above. The fix is `return super if OpenTelemetry::Common::Utilities.untraced?` at the top of the instrumentation's `do_get_block`, upstream or redefined at init until a release carries it.
 
 TODO(jhartquist): capture the streamed bodies listed above. For `Net::HTTP`, prepend on `Net::HTTPResponse#read_body` and tee the block or IO the app passes.
 
 ## Loggers
 
-Foam installs `opentelemetry-instrumentation-logger`, which hooks `Logger#format_message`, so every logger built on `Logger` is covered, including one created before `init`. Each line becomes one log record: the body is the formatted line and the severity follows the `Logger` level, except that `unknown` arrives with no severity. The app's own log output is unchanged; the trace and span IDs are on the record sent to Foam, not written into the app's lines. The record carries the resource attributes but none of its own: a JSON-formatted line arrives as one string body, with its fields inside the text. Use `Foam.log` to send a record with attributes.
+Foam installs `opentelemetry-instrumentation-logger`, so every logger built on `Logger` is covered, including one created before `init`. Each line becomes one log record: the body is the formatted line and the severity follows the `Logger` level, except that `unknown` arrives with no severity. The app's own log output is unchanged; the trace and span IDs are on the record sent to Foam, not written into the app's lines. The record carries the resource attributes but none of its own: a JSON-formatted line arrives as one string body, with its fields inside the text. Use `Foam.log` to send a record with attributes.
 
 Covered:
 
 - `Logger`: every line except `Logger#<<`, which writes to the device directly.
 - Rails 7.1 and later: `Rails.logger`, `ActiveSupport::Logger`, `TaggedLogging`, and `BroadcastLogger`. A broadcast sends one record per line, tags included in the body, provided ActiveSupport is loaded before `init`; otherwise every sink sends its own record.
-- `Sidekiq::Logger`, MonoLogger, LogStashLogger, Ougai, and fluent-logger build on `Logger` and should be covered; Lograge formats `Rails.logger` lines and is covered through it. LogStashLogger's `MultiLogger` should send one record per logger it wraps. None of these is tested here.
+- `Sidekiq::Logger`, MonoLogger, LogStashLogger, Ougai, and fluent-logger build on `Logger` and are covered; Lograge formats `Rails.logger` lines and is covered through it. LogStashLogger's `MultiLogger` sends one record per logger it wraps.
 
 Not covered, since they do not build on `Logger`: Semantic Logger, `logging`, log4r, yell, and Puma's own server lines. Use `Foam.log` for a record from these. Semantic Logger also ships an `:open_telemetry` appender. `puts` and direct writes to `$stdout` or `$stderr` are never captured, nor is a `Logger` whose device is `nil` or `File::NULL`.
 
+Lines another SDK writes through its own `Logger` are shipped like the app's; `before_send` can drop them.
+
 If another SDK owns logs, set `disable_log_sending: true` so records do not arrive twice, and attach `create_foam_ingest_log_record_processor` to that SDK's provider. `Foam.log` delivers either way.
 
 Secrets in a log line are masked before export. See Redaction.
@@ -459,19 +443,40 @@ Thread.new do
 end
 ```
 
+## Other backends installed
+
+An app may already run another vendor's SDK, such as Datadog, which records its own telemetry and sends it to that vendor. Do not change its configuration.
+
+Call `init` after configuring the existing SDK. If the app already configures OpenTelemetry itself with `OpenTelemetry::SDK.configure`, instead attach the ingest helpers to the app's providers and do not call `init`; see `create_foam_ingest_span_processor` under Public API.
+
+### Datadog
+
+Datadog sends its data to its agent over HTTP, which Foam would otherwise record as spans. Pass the agent's address and Datadog's request header to `init`. This works whether the agent is on TCP or a Unix socket:
+
+```ruby
+agent = Datadog::Core::Configuration::AgentSettingsResolver.call(Datadog.configuration)
+agent_hosts = agent.uds_path ? [] : ["#{agent.hostname}:#{agent.port}"]
+
+Foam.init(
+  name: "checkout-api", environment: "production", token: ENV["FOAM_OTEL_TOKEN"],
+  ignored_outbound_hosts: agent_hosts,
+  ignored_outbound_headers: ["Datadog-Meta-Tracer-Version"]
+)
+```
+
 ## Redaction
 
-Foam masks the values of sensitive keys in everything it exports: span, span event, and span link attributes; log bodies and attributes; metric data point attributes; and resource attributes. A built-in list of credential and personal-data keys always applies, matched by exact name or by a sensitive segment of the key (`legacy_api_key_2`, `stripeToken`). Keys match after normalization: case, `-`, `.`, and camelCase boundaries are folded, so `apiKey`, `api-key`, and `API_KEY` are one key. Header attributes match on the header name. SQL statements arrive from the database instrumentations with every literal replaced with `?` (see Instrumentation) and are redacted like any other string. Captured HTTP bodies are masked by content type with the same keys, with their framing kept. When masking a body fails, the whole body is replaced with `********`.
+Foam masks the values of sensitive keys in everything it exports: span, span event, and span link attributes; log bodies and attributes; metric data point attributes; and resource attributes. A built-in list of credential and personal-data keys always applies, matched by exact name or by a sensitive segment of the key (`legacy_api_key_2`, `stripeToken`). Keys match after normalization: case, `-`, `.`, and camelCase boundaries are folded, so `apiKey`, `api-key`, and `API_KEY` are one key. Header attributes match on the header name. SQL statements arrive from the database instrumentations with every literal replaced with `?` (see Instrumentation) and are redacted like any other string. Captured HTTP bodies are masked by content type with the same keys. When masking a body fails, the whole body is replaced with `********`.
 
 Every other string is scanned as text, log bodies included.
 
 - A string that is a whole JSON document is masked by key at every depth. A list or object under a sensitive key becomes one mask.
-- Any other string has the value after each sensitive key masked. The key may be quoted as JSON writes it or bare before `=`, `:`, or `=>`, in a query string, a header line, or prose. The value ends at the next delimiter, and a quoted URL keeps its closing quote.
+- Any other string has the value after each sensitive key masked. The key may be quoted as JSON writes it or bare before `=`, `:`, or `=>`, in a query string, a header line, or prose. The value ends at the next delimiter.
 - `url.query` is masked pair by pair. Names such as `code` and `sig` are sensitive only inside a query string.
 
 Scanning stops after 1 MiB, and everything past that point is replaced with a single mask. Masking is value-only: the key and the surrounding text remain intact.
 
-The `redact` option adds the app's own keys, matched exactly after the same normalization. A value under a built-in or `pii` key becomes `[REDACTED]`. A value under a `secrets` key becomes `********`, followed by its last four characters when it is 12 characters or longer. That tail is kept only where the value is read whole under its key: an attribute, a hash or array, or a whole JSON document. A value found by scanning text or a `url.query` pair becomes `[REDACTED]` whatever its key.
+The `redact` option adds the app's own keys, matched exactly after the same normalization. A value under a built-in or `pii` key becomes `[REDACTED]`. A value under a `secrets` key becomes `********`, followed by its last four characters when it is 12 characters or longer.
 
 Masking changes values only: keys stay present, and span names, event names, and severity are untouched. It is applied after `before_send`, to the copy sent to Foam, through `init` and through the ingest components alike. The app's own span and log objects, and the copies handed to other processors and exporters, are unchanged.
 
@@ -578,11 +583,8 @@ Every public method except the ingest factories, and every capture hook, runs un
 ```sh
 bundle install
 bundle exec rspec             # unit tests
-bundle exec rake e2e          # end-to-end tests
 bundle exec standardrb --fix  # lint and autoformat
-bundle exec rake              # unit, e2e, then lint (what CI checks)
+bundle exec rake              # unit, then lint (what CI checks)
 ```
 
-The end-to-end tests run a real OTLP collector and the fixture apps under `e2e/apps/` with puma, then assert on what the collector received. `rake e2e` downloads the pinned `otelcol-contrib` release to `~/.cache/foam-otel` the first time (about 350 MB; set `FOAM_OTELCOL_DIR` to cache it elsewhere), and the suite installs a fixture app's bundle when it starts the app. Run one file with `bundle exec rspec e2e/spec/rack_init_spec.rb`; that does not download the collector, so run `rake e2e:install` once first.
 
-Each run writes the collector config, the received telemetry, and the output of both processes to `tmp/e2e/`. A failure prints the tail of both stderr files.

data/lib/foam/otel/internal/body_capture.rb

@@ -19,18 +19,16 @@ module Foam
         CHUNK_COUNT = "foam.http.body.chunk.count"
         CHUNK_INDEX = "foam.http.body.chunk.index"
         CHUNK_BYTES = "foam.http.body.chunk.bytes"
-        CHUNK_CONTENT = "foam.http.body.chunk.content"
 
         # The context key a capturing middleware marks its span under, so the transport patches
         # beneath it leave that span alone.
         OWNED_SPAN = OpenTelemetry::Context.create_key("foam-otel.owned-span")
 
-        # The span attributes of each direction. Size, content, and the header prefix are the
-        # semantic conventions' names; the rest are Foam's.
+        # The span attributes of each direction. Size and the header prefix are the semantic
+        # conventions' names; the rest are Foam's.
         SPAN_ATTRIBUTES = %i[request response].to_h do |direction|
           [direction, {
             size: "http.#{direction}.body.size",
-            content: "http.#{direction}.body.content",
             capture_id: "foam.http.#{direction}.body.capture_id",
             complete: "foam.http.#{direction}.body.complete",
             truncated: "foam.http.#{direction}.body.truncated",
@@ -159,12 +157,16 @@ module Foam
             @last_activity = BodyCapture.now
             @observed_bytes += bytes.bytesize
             room = Constants::BODY_CAPTURE_BYTES - @kept_bytes
+            # The caller's string is copied, since the app may still write to it; a slice is a
+            # fresh string already, so it is relabeled as bytes rather than copied again.
             if bytes.bytesize > room
               @truncated = true
-              bytes = bytes.byteslice(0, room)
+              kept = bytes.byteslice(0, room).force_encoding(Encoding::BINARY)
+            else
+              kept = bytes.b
             end
-            @chunks << bytes.b unless bytes.empty?
-            @kept_bytes += bytes.bytesize
+            @chunks << kept unless kept.empty?
+            @kept_bytes += kept.bytesize
             nil
           end
         end
@@ -220,7 +222,6 @@ module Foam
             @span.set_attribute(@attributes[:complete], complete)
             @span.set_attribute(@attributes[:truncated], truncated)
             @span.set_attribute(@attributes[:size], @observed_bytes)
-            @span.set_attribute(@attributes[:content], text) if text
           end
           return if outcome == "empty"
 
@@ -243,7 +244,7 @@ module Foam
           records.each_with_index do |chunk, index|
             logger.on_emit(
               event_name: Constants::BODY_CHUNK_EVENT,
-              body: Constants::BODY_CHUNK_EVENT_BODY,
+              body: chunk,
               severity_number: Logs::SEVERITIES[:info],
               severity_text: "INFO",
               attributes: chunk ? common.merge(chunk_attributes(chunk, index)) : common,
@@ -256,7 +257,8 @@ module Foam
         # body cannot be read as text: an unknown or corrupt encoding, one inflating past the
         # bound, or a binary content type. Only the metadata is exported then, never the bytes.
         def decoded_text(headers)
-          decoded = BodyDecoder.decode(@chunks.join, headers["content-encoding"])
+          wire = (@chunks.size == 1) ? @chunks.first : @chunks.join
+          decoded = BodyDecoder.decode(wire, headers["content-encoding"])
           return [nil, false] unless decoded
 
           BodyDecoder.text(decoded, ContentType.parse(headers["content-type"])) || [nil, false]
@@ -283,7 +285,7 @@ module Foam
         end
 
         def chunk_attributes(chunk, index)
-          {CHUNK_INDEX => index, CHUNK_BYTES => chunk.bytesize, CHUNK_CONTENT => chunk}
+          {CHUNK_INDEX => index, CHUNK_BYTES => chunk.bytesize}
         end
       end
     end

data/lib/foam/otel/internal/body_decoder.rb

@@ -31,10 +31,11 @@ module Foam
           return unless content_type.text?
 
           cut = bytes.bytesize > Constants::BODY_CAPTURE_BYTES
-          text = bytes.byteslice(0, Constants::BODY_CAPTURE_BYTES)
+          text = cut ? bytes.byteslice(0, Constants::BODY_CAPTURE_BYTES) : bytes.dup
           text.force_encoding(encoding_for(content_type))
           text = if text.encoding == Encoding::UTF_8
-            whole_utf8(text).scrub
+            whole = whole_utf8(text)
+            whole.valid_encoding? ? whole : whole.scrub
           else
             text.encode(Encoding::UTF_8, invalid: :replace, undef: :replace)
           end

data/lib/foam/otel/internal/body_redactor.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require "json"
+require "strscan"
 
 module Foam
   module Otel
@@ -13,10 +14,16 @@ module Foam
         BLANK_LINE = /(\r\n\r\n|\n\n|\r\r)/
         RECORD_SEPARATOR = "\u001e"
 
-        # A JSON pair as written, with the closing quote optional so a string cut off by truncation
-        # still matches.
-        JSON_PAIR =
-          /"((?:[^"\\]|\\.)*)"(\s*:\s*)("(?:[^"\\]|\\.)*"?|-?\d[\d.eE+-]*|true|false|null)/
+        # A JSON key and its separator. The key is bounded so the pattern never walks a long string;
+        # the scalar after it is read by position, which costs nothing however long it is.
+        JSON_KEY = /"((?:[^"\\]|\\.){,#{Constants::KEY_SCAN_CHARS}})"(\s*:\s*)/
+        JSON_NUMBER_END = /[^\d.eE+-]/
+        JSON_LITERALS = %w[true false null].freeze
+        QUOTE = '"'.ord
+        MINUS = "-".ord
+        CLOSE_BRACKET = ">".ord
+        WHITESPACE = [" ", "\t", "\n", "\r", "\f", "\v"].map(&:ord).freeze
+        DIGITS = ("0".ord.."9".ord)
         SSE_DATA_LINE = /\A(data: ?)(.*)\z/
         XML_OPENING_TAG = /<([A-Za-z_][\w:.-]*)(?:\s[^>]*)?>/
         XML_ATTRIBUTE = /(\s)([A-Za-z_:][\w:.-]*)(\s*=\s*)(["'])(.*?)\4/
@@ -104,18 +111,38 @@ module Foam
         end
 
         # Sensitive pairs masked as written, for JSON that does not parse; a truncated body is the
-        # usual case. The mask is the plain one, since the value may be cut.
+        # usual case. The mask is the plain one, since the value may be cut. Every scalar is stepped
+        # over by position, so no pattern is tried inside a value.
         def redact_json_fallback(text)
-          changed = false
-          redacted = text.gsub(JSON_PAIR) do |pair|
-            key, separator = Regexp.last_match.captures
-            kind = @redactor.mask_kind(key)
-            next pair unless kind
+          scanner = StringScanner.new(text)
+          out = nil
+          last = 0
+          while scanner.scan_until(JSON_KEY)
+            key, separator = scanner[1], scanner[2]
+            pair_start = scanner.pos - scanner.matched_size
+            value_end = json_scalar_end(text, scanner.pos) or next
+            scanner.pos = value_end
+            kind = @redactor.mask_kind(key) or next
+
+            out ||= +""
+            out << text.byteslice(last...pair_start) << "\"#{key}\"#{separator}\"#{fallback_mask(kind)}\""
+            last = value_end
+          end
+          out << text.byteslice(last..) if out
+        end
 
-            changed = true
-            "\"#{key}\"#{separator}\"#{fallback_mask(kind)}\""
+        # The position after the JSON scalar at position, or nil when no scalar starts there. A
+        # string cut off by truncation ends with the text.
+        def json_scalar_end(text, position)
+          byte = text.getbyte(position)
+          if byte == QUOTE
+            Redactor.closing_quote(text, position + 1, QUOTE, text.bytesize) || text.bytesize
+          elsif byte == MINUS || DIGITS.cover?(byte)
+            text.byteindex(JSON_NUMBER_END, position) || text.bytesize
+          else
+            literal = JSON_LITERALS.find { |word| text.byteslice(position, word.bytesize) == word }
+            literal && position + literal.bytesize
           end
-          redacted if changed
         end
 
         # The mask for a value that cannot be read whole: no tail is kept for a secret.
@@ -227,40 +254,54 @@ module Foam
         end
 
         # The text of each sensitive element is masked; one holding markup or never closed gets the
-        # plain mask, since its value cannot be read whole.
+        # plain mask, since its value cannot be read whole. Positions are bytes throughout, so the
+        # body's length costs nothing.
         def redact_xml_elements(text)
-          changed = false
+          scanner = StringScanner.new(text, fixed_anchor: true)
+          output = nil
           cursor = 0
-          scan_from = 0
-          output = +""
 
-          while (opening = XML_OPENING_TAG.match(text, scan_from))
-            scan_from = opening.end(0)
-            next if opening[0].end_with?("/>")
+          while scanner.scan_until(XML_OPENING_TAG)
+            next if scanner.matched.end_with?("/>")
 
-            kind = @redactor.mask_kind(local_name(opening[1]))
+            name = scanner[1]
+            kind = @redactor.mask_kind(local_name(name))
             next unless kind
 
-            changed = true
-            output << text[cursor...scan_from]
-            closing = /<\/#{Regexp.escape(opening[1])}\s*>/.match(text, scan_from)
+            output ||= +""
+            output << text.byteslice(cursor...scanner.pos)
+            closing = xml_closing_tag(text, name, scanner.pos)
             unless closing
               output << fallback_mask(kind)
-              cursor = text.length
+              cursor = text.bytesize
               break
             end
 
-            value = text[scan_from...closing.begin(0)]
+            value = text.byteslice(scanner.pos...closing.begin)
             masked = value.include?("<") ? fallback_mask(kind) : @redactor.mask(kind, value)
-            output << masked << closing[0]
-            cursor = scan_from = closing.end(0)
+            output << masked << text.byteslice(closing)
+            cursor = scanner.pos = closing.end
           end
 
-          output << text[cursor..] if changed
+          output << text.byteslice(cursor..) if output
+        end
+
+        # The byte range of the first </name> at or after position, whitespace before the bracket
+        # allowed; nil when the element is never closed.
+        def xml_closing_tag(text, name, position)
+          needle = "</#{name}"
+          while (start = text.byteindex(needle, position))
+            stop = start + needle.bytesize
+            stop += 1 while WHITESPACE.include?(text.getbyte(stop))
+            return start...(stop + 1) if text.getbyte(stop) == CLOSE_BRACKET
+
+            position = start + 1
+          end
         end
 
         def local_name(name)
-          name.split(":").last
+          colon = name.rindex(":")
+          colon ? name[(colon + 1)..] : name
         end
 
         # A sensitive key's value is masked on its line; a block scalar under one spans lines that

data/lib/foam/otel/internal/constants.rb

@@ -22,11 +22,6 @@ module Foam
         # The prefixes of the header attributes, http.request.header.<name> and http.response.header.<name>.
         HTTP_HEADER_PREFIXES = ["http.request.header.", "http.response.header."].freeze
 
-        # Captured body content on a span. Foam names these; the semantic conventions define only the
-        # body sizes.
-        HTTP_REQUEST_BODY_CONTENT = "http.request.body.content"
-        HTTP_RESPONSE_BODY_CONTENT = "http.response.body.content"
-
         # The only HTTP headers captured as span attributes: they describe the body, never the caller.
         SAFE_NETWORK_HEADERS = %w[content-type content-length content-encoding].freeze
 
@@ -49,8 +44,8 @@ module Foam
         # The event name on every body chunk record.
         BODY_CHUNK_EVENT = "foam.http.body.chunk"
 
-        # The log body of every body chunk record; the chunk's text goes in its attributes.
-        BODY_CHUNK_EVENT_BODY = "HTTP body chunk"
+        # The longest key the redactors match in text.
+        KEY_SCAN_CHARS = 256
 
         # Replaces a redacted value.
         REDACTED_VALUE = "[REDACTED]"

data/lib/foam/otel/internal/exporters.rb

@@ -138,7 +138,7 @@ module Foam
             event = @before_send.event_to_export(hook_event(record)) or return
 
             record.dup.tap do |copy|
-              copy.body = @redactor.redact_value(event[:body])
+              copy.body = redacted_body(record, event[:body])
               copy.severity_text = event[:severity_text]
               copy.severity_number = event[:severity_number]
               copy.attributes = redacted_attributes(event[:attributes])
@@ -148,6 +148,13 @@ module Foam
             end
           end
 
+          # A body chunk's text was redacted by the body redactor when it was captured.
+          def redacted_body(record, body)
+            return body if record.event_name == Constants::BODY_CHUNK_EVENT
+
+            @redactor.redact_value(body)
+          end
+
           # What the hook sees: the fields it may edit, then the correlation fields, which are read from
           # the record on export whatever the hook returns.
           def hook_event(record)