foam-otel 1.0.2 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/GOTCHAS.md +522 -53
- data/README.md +341 -31
- data/RESEARCH.md +170 -70
- data/THIRD-PARTY-NOTICES +21 -10
- data/lib/foam/otel/api.rb +124 -16
- data/lib/foam/otel/classifier.rb +44 -6
- data/lib/foam/otel/config.rb +22 -2
- data/lib/foam/otel/constants.rb +15 -89
- data/lib/foam/otel/errors.rb +4 -1
- data/lib/foam/otel/fork_hooks.rb +7 -0
- data/lib/foam/otel/ingest.rb +617 -0
- data/lib/foam/otel/ingest_metric_reader.rb +87 -0
- data/lib/foam/otel/init.rb +351 -34
- data/lib/foam/otel/instance.rb +4 -1
- data/lib/foam/otel/llm/anthropic_shim.rb +84 -0
- data/lib/foam/otel/llm/gemini_shim.rb +113 -0
- data/lib/foam/otel/llm/openai_shim.rb +123 -0
- data/lib/foam/otel/llm/ruby_llm_shim.rb +131 -0
- data/lib/foam/otel/llm.rb +272 -0
- data/lib/foam/otel/logger_bridge.rb +187 -0
- data/lib/foam/otel/metrics.rb +32 -7
- data/lib/foam/otel/pipelines.rb +72 -3
- data/lib/foam/otel/redacting_exporter.rb +88 -17
- data/lib/foam/otel/redaction.rb +164 -87
- data/lib/foam/otel/runtime_metrics.rb +120 -0
- data/lib/foam/otel/session_stitching.rb +112 -0
- data/lib/foam/otel/version.rb +1 -1
- metadata +237 -19
data/README.md
CHANGED
|
@@ -2,9 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
Foam's OpenTelemetry core for Ruby services. A thin, safe wrapper over the
|
|
4
4
|
official OpenTelemetry libraries: foam owns the pipeline (providers, batch
|
|
5
|
-
processors, OTLP export to the foam fleet endpoint,
|
|
6
|
-
|
|
7
|
-
|
|
5
|
+
processors, OTLP export to the foam fleet endpoint), turns on automatic
|
|
6
|
+
tier-1/2 instrumentation, and hands you a small set of never-throw helpers —
|
|
7
|
+
and nothing else. **Redaction is fully opt-in: by default foam captures every
|
|
8
|
+
value RAW** (no secret floor, no value-pattern masking); you enumerate the
|
|
9
|
+
fields to mask via `redact_keys`/`redact_pii_keys`.
|
|
8
10
|
|
|
9
11
|
Built to `docs/BASE_PACKAGE_SPEC.md`. This README is the manual you integrate
|
|
10
12
|
from.
|
|
@@ -19,12 +21,16 @@ Foam builds on the OpenTelemetry authors' work (https://opentelemetry.io) — se
|
|
|
19
21
|
```ruby
|
|
20
22
|
# Gemfile
|
|
21
23
|
gem "foam-otel"
|
|
22
|
-
#
|
|
23
|
-
#
|
|
24
|
-
#
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
24
|
+
# foam-otel bundles the UNIVERSAL FLOOR and auto-activates each piece when its
|
|
25
|
+
# target library is present (presence-checked, and gated on foam owning the
|
|
26
|
+
# slot it produces into): Rack + the Rails family, the HTTP clients (Net::HTTP,
|
|
27
|
+
# Faraday), the primary datastores (pg, mysql2, redis, mongo) with RAW
|
|
28
|
+
# db.statement capture, Sidekiq, the stdlib Logger bridge, hand-written Ruby
|
|
29
|
+
# runtime + GC metrics, session stitching (browser session.id via baggage),
|
|
30
|
+
# and the LLM shims (OpenAI, Anthropic, Gemini, ruby_llm — activity + raw
|
|
31
|
+
# content). Add only the niche long tail your app needs beyond the floor:
|
|
32
|
+
gem "opentelemetry-instrumentation-graphql" # e.g. GraphQL
|
|
33
|
+
gem "opentelemetry-instrumentation-resque" # e.g. Resque
|
|
28
34
|
```
|
|
29
35
|
|
|
30
36
|
**Supported versions** (tested in CI, `spec/version_spec.rb`): Ruby **>= 3.1**;
|
|
@@ -54,10 +60,53 @@ Foam::Otel.init(
|
|
|
54
60
|
```
|
|
55
61
|
|
|
56
62
|
**Outcome**: foam registers the tracer/meter/logger providers, exports OTLP to
|
|
57
|
-
the foam fleet endpoint, and auto-activates the
|
|
58
|
-
instrumentation
|
|
59
|
-
|
|
60
|
-
|
|
63
|
+
the foam fleet endpoint, and auto-activates the **universal floor** — the
|
|
64
|
+
official rack/rails instrumentation plus the bundled HTTP (Net::HTTP/Faraday),
|
|
65
|
+
datastore (pg/mysql2/redis/mongo), and Sidekiq instrumentations — each
|
|
66
|
+
presence-gated, and gated on foam owning the traces slot (a foreign-owned
|
|
67
|
+
tracer stands the activation down). The datastore set is activated with
|
|
68
|
+
**`db_statement: :include`** — foam explicitly overrides the upstream
|
|
69
|
+
`:obfuscate` default so RAW SQL / redis args / mongo commands land on the wire.
|
|
70
|
+
On top of the contrib set the floor adds foam's own pieces: the **stdlib
|
|
71
|
+
Logger bridge** (every host `Logger` line also lands as a trace-correlated
|
|
72
|
+
OTel log record, body raw), hand-written Ruby **runtime + GC metrics**
|
|
73
|
+
(GC counts + timing, heap slots, allocation, thread count, RSS — Ruby has no
|
|
74
|
+
drop-in upstream metrics gem, so foam ships its own collector),
|
|
75
|
+
**session stitching** (inbound `baggage: session.id` from the foam browser
|
|
76
|
+
package is stamped onto every span and log record), and the **LLM shims**
|
|
77
|
+
(below). Every inbound request, DB call, HTTP call, job, log line, and model
|
|
78
|
+
call now flows to foam. **Redaction is opt-in: by default every value —
|
|
79
|
+
attributes, URLs/query strings, DB statement text, log bodies, LLM content —
|
|
80
|
+
is captured RAW.** Pass `redact_keys`/`redact_pii_keys` to mask or erase
|
|
81
|
+
specific fields (see below).
|
|
82
|
+
|
|
83
|
+
### The LLM surface — OpenAI, Anthropic, Gemini (and ruby_llm)
|
|
84
|
+
|
|
85
|
+
The Ruby contrib registry ships no LLM instrumentation, so foam does: thin,
|
|
86
|
+
presence-checked shims over the SDKs' public call sites, emitting the standard
|
|
87
|
+
`gen_ai.*` semantic conventions — activity (model, response id, finish
|
|
88
|
+
reasons, token usage, latency) AND prompt/response **content, RAW**
|
|
89
|
+
(`gen_ai.input.messages` / `gen_ai.output.messages` /
|
|
90
|
+
`gen_ai.system_instructions`). Covered: the official `openai` SDK
|
|
91
|
+
(chat.completions.create + responses.create), the official `anthropic` SDK
|
|
92
|
+
(messages.create), Gemini via the `gemini-ai` gem (Google ships no official
|
|
93
|
+
Ruby SDK), and `ruby_llm` (one seam covering all its providers). Absent SDK =
|
|
94
|
+
inert; version-guarded; fail-to-dark — a shim failure can NEVER break the
|
|
95
|
+
model call, and the SDK's own error re-raises identically. One model call =
|
|
96
|
+
one `gen_ai` client span (the SDK's HTTP send rides the same suppression
|
|
97
|
+
context foam's exporters use). Streaming call sites (`#stream`/`#stream_raw`
|
|
98
|
+
on the official SDKs) are a recorded follow-up.
|
|
99
|
+
|
|
100
|
+
### Session stitching — browser `session.id` on backend telemetry
|
|
101
|
+
|
|
102
|
+
The foam browser package stamps `session.id` on every browser signal and
|
|
103
|
+
sends it to your API as W3C Baggage beside `traceparent`. This gem's floor
|
|
104
|
+
copies `session.id` (and `session.previous_id` across a rotation) from the
|
|
105
|
+
inbound baggage onto every span and log record foam emits — browser session →
|
|
106
|
+
backend traces/logs, stitched with no ingest-side join. No-op when the
|
|
107
|
+
baggage is absent. **CORS note (the FDE wires this on every frontend-called
|
|
108
|
+
API):** the allowlist must include BOTH headers —
|
|
109
|
+
`Access-Control-Allow-Headers: traceparent, baggage`.
|
|
61
110
|
|
|
62
111
|
---
|
|
63
112
|
|
|
@@ -70,15 +119,56 @@ now flows to foam, with the redaction floor applied on the wire.
|
|
|
70
119
|
| `enabled:` | Boolean | yes | — | The config switch. `false` = fully inert (no SDK, no providers, no network, helpers no-op). `true` = export, in EVERY environment. No default — you write the logic. |
|
|
71
120
|
| `token:` | String | when enabled | — | `Authorization: Bearer` for export. Required (and validated) only when `enabled: true`; wired explicitly from your secret source — never an env fallback read by the gem. **Fleet convention: read it from the `FOAM_OTEL_TOKEN` env var.** Blank while enabled → raises at boot. |
|
|
72
121
|
| `version:` | String | no | nil | `service.version`, verbatim (git SHA recommended). Missing → warns and continues. Never detected at runtime. |
|
|
73
|
-
| `redact_keys:` | Array<String> | no | `[]` |
|
|
74
|
-
| `redact_pii_keys:` | Array<String> | no | `[]` | Field names
|
|
122
|
+
| `redact_keys:` | Array<String> | no | `[]` | The ONLY field names tail-masked (shape-preserving, e.g. `********f456`). Opt-in — empty (the default) means NO masking. Matches by case-insensitive substring, including keys nested in structured values and query-string keys. |
|
|
123
|
+
| `redact_pii_keys:` | Array<String> | no | `[]` | Field names fully erased to `[REDACTED]` (no tail). Opt-in — empty (the default) means NO erasure. Foam ships no preset; this is your enumeration. |
|
|
75
124
|
| `additional_span_processors:` | Array | no | `[]` | Tenant seam: constructed SpanProcessor instances added to foam's pipeline (additive; never replace foam's export). See coexistence. |
|
|
76
125
|
| `additional_log_record_processors:` | Array | no | `[]` | Tenant seam, logs. |
|
|
77
126
|
| `additional_metric_readers:` | Array | no | `[]` | Tenant seam, metrics. |
|
|
78
127
|
| `additional_instrumentations:` | Array | no | `[]` | Constructed tier-2 instrumentation instances to register (fault-isolated: one that throws is skipped with a `[foam]` warning). |
|
|
79
|
-
| `ignored_outbound_hosts:` | Array<String> | no | `[]` | Hosts whose outbound calls produce no spans — EXTENDS the built-in export-loop guard. For a co-resident agent's intake or a tenant exporter's endpoint. |
|
|
128
|
+
| `ignored_outbound_hosts:` | Array<String> | no | `[]` | Hosts whose outbound calls produce no spans — EXTENDS the built-in export-loop guard. For a co-resident agent's intake or a tenant exporter's endpoint. Applies to the clients whose official instrumentation supports host suppression: **Net::HTTP and Excon**. Faraday/HTTP (httprb)/HTTPX have no upstream `untraced_hosts` option — foam warns loudly at init when one of those is bundled (GOTCHAS G13); foam's own export loop is guarded for every client regardless. |
|
|
80
129
|
| `diagnostics:` | Boolean | no | false | Verbose `[foam]` self-reporting of init/health. Warnings and errors are always loud regardless. |
|
|
81
130
|
|
|
131
|
+
Examples for the remaining options (each is runnable as-is):
|
|
132
|
+
|
|
133
|
+
```ruby
|
|
134
|
+
# Redaction is OPT-IN (default: nothing is masked). List the fields to redact:
|
|
135
|
+
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
|
|
136
|
+
redact_keys: %w[password api_key internal_ref voucher_code], # tail-masked
|
|
137
|
+
redact_pii_keys: %w[customer_email full_name]) # fully [REDACTED]
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
```ruby
|
|
141
|
+
# Constructed tier-2 instrumentation instances (fault-isolated; one that
|
|
142
|
+
# throws is skipped with a [foam] warning, never a crashed boot):
|
|
143
|
+
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
|
|
144
|
+
additional_instrumentations: [
|
|
145
|
+
OpenTelemetry::Instrumentation::Sidekiq::Instrumentation.instance,
|
|
146
|
+
[OpenTelemetry::Instrumentation::PG::Instrumentation.instance, { db_statement: :omit }],
|
|
147
|
+
])
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
```ruby
|
|
151
|
+
# Tenant seam, logs + metrics (spans are shown in scenario 3 below):
|
|
152
|
+
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
|
|
153
|
+
additional_log_record_processors: [EvalTool::LogRecordProcessor.new],
|
|
154
|
+
additional_metric_readers: [EvalTool::MetricReader.new],
|
|
155
|
+
ignored_outbound_hosts: ["ingest.eval-tool.example"]) # the tenant loop guard — required
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
```ruby
|
|
159
|
+
# Verbose self-reporting while wiring foam up (warnings are loud regardless):
|
|
160
|
+
Foam::Otel.init(name: "checkout-api", environment: "development", enabled: true,
|
|
161
|
+
token: ENV.fetch("FOAM_OTEL_TOKEN"), diagnostics: true)
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
And the unhappy path — a blank required argument fails AT BOOT, on your
|
|
165
|
+
machine, never silently:
|
|
166
|
+
|
|
167
|
+
```ruby
|
|
168
|
+
Foam::Otel.init(name: "", environment: "production", enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"))
|
|
169
|
+
# => raises ArgumentError: Foam::Otel.init requires a non-empty name
|
|
170
|
+
```
|
|
171
|
+
|
|
82
172
|
**Deliberately absent** (and why): `endpoint`, `cadence`/`sampling`,
|
|
83
173
|
`is_production`, `force_export`, `disabled_environments`, and any
|
|
84
174
|
processor/exporter injection beyond the tenant seam. Customers identify their
|
|
@@ -159,14 +249,14 @@ Foam::Otel.get_logger("my.lib")
|
|
|
159
249
|
non-destructive thing. The classifier's output is information for you, never a
|
|
160
250
|
decision it makes for you.
|
|
161
251
|
|
|
162
|
-
### Scenario 2 — beside a proprietary APM agent
|
|
252
|
+
### Scenario 2 — beside a proprietary APM agent
|
|
163
253
|
|
|
164
254
|
Disjoint pipelines: foam runs its own beside the agent, touching nothing. Add the
|
|
165
255
|
agent's intake host so foam's outbound instrumentation doesn't shadow its egress.
|
|
166
256
|
|
|
167
257
|
```ruby
|
|
168
258
|
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FOAM_OTEL_TOKEN,
|
|
169
|
-
ignored_outbound_hosts: ["
|
|
259
|
+
ignored_outbound_hosts: ["agent-intake.vendor.example"])
|
|
170
260
|
```
|
|
171
261
|
|
|
172
262
|
### Scenario 3 — a scoped tenant rides foam's pipeline (LLM-eval, AI-observability)
|
|
@@ -180,17 +270,79 @@ Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FO
|
|
|
180
270
|
ignored_outbound_hosts: ["ingest.eval-tool.example"])
|
|
181
271
|
```
|
|
182
272
|
|
|
183
|
-
> Ruby note:
|
|
184
|
-
>
|
|
185
|
-
>
|
|
186
|
-
|
|
187
|
-
|
|
273
|
+
> Ruby note: when you configure `redact_keys`/`redact_pii_keys`, foam's HELPERS
|
|
274
|
+
> apply the key pass at capture, so a tenant processor receives foam-helper data
|
|
275
|
+
> (span/log helper attributes, log bodies, metric labels) ALREADY masked.
|
|
276
|
+
> Because the Ruby SDK freezes span attributes at finish, a tenant
|
|
277
|
+
> processor sees third-party instrumentation attributes UNMASKED (for those,
|
|
278
|
+
> foam's key pass runs at the exporter boundary — GOTCHAS F1). With no keys
|
|
279
|
+
> configured (the default) nothing is masked on either path.
|
|
280
|
+
>
|
|
281
|
+
> Loop-guard note: `ignored_outbound_hosts` suppresses spans for tenant/agent
|
|
282
|
+
> egress made through **Net::HTTP or Excon**. A tenant exporter using
|
|
283
|
+
> Faraday/HTTP (httprb)/HTTPX cannot be host-suppressed today (no upstream
|
|
284
|
+
> `untraced_hosts` option — foam warns at init; GOTCHAS G13).
|
|
285
|
+
|
|
286
|
+
### Scenario 4 — a foreign OTel SDK already owns a signal (door 2)
|
|
188
287
|
|
|
189
288
|
If another OTel SDK registered (say) the tracer before foam, foam does NOT
|
|
190
|
-
displace it.
|
|
191
|
-
tier-3 telemetry for it
|
|
192
|
-
|
|
193
|
-
|
|
289
|
+
displace it. `init` warns once, naming the owner, and foam is dark for that
|
|
290
|
+
signal — your tier-3 telemetry for it rides the foreign SDK to ITS backend.
|
|
291
|
+
To ALSO send that pipeline to foam, add ONE LINE per claimed signal in THEIR
|
|
292
|
+
setup — the ingest entries (see "Door 2 — the ingest entries" below). YOU
|
|
293
|
+
place the tap: foam never attaches itself to a pipeline it does not own
|
|
294
|
+
(their data moves only when a human says so), and the placement line doubles
|
|
295
|
+
as the off-switch — gate it on the same condition you would give `enabled:`.
|
|
296
|
+
|
|
297
|
+
```ruby
|
|
298
|
+
# config/initializers/opentelemetry.rb — THEIR setup, plus the foam lines
|
|
299
|
+
require "opentelemetry/sdk"
|
|
300
|
+
require "foam/otel"
|
|
301
|
+
|
|
302
|
+
OpenTelemetry::SDK.configure do |c|
|
|
303
|
+
c.service_name = "their-service"
|
|
304
|
+
c.use_all
|
|
305
|
+
# REQUIRED loop step (see "Door 2 — the required loop step" below):
|
|
306
|
+
c.use "OpenTelemetry::Instrumentation::Net::HTTP", { untraced_hosts: ["otel.api.foam.ai"] }
|
|
307
|
+
c.add_span_processor(their_vendor_processor) # their export, untouched
|
|
308
|
+
if ENV["APP_ENV"] == "production" # your consent AND your off-switch
|
|
309
|
+
c.add_span_processor(Foam::Otel.create_ingest_span_processor(
|
|
310
|
+
token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
|
|
311
|
+
end
|
|
312
|
+
end
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
**Outcome**: their pipeline keeps working exactly as before — their data,
|
|
316
|
+
their resource, their export, byte-identical — and foam ALSO receives the
|
|
317
|
+
signal, carrying THEIR resource identity plus foam's export-time stamp and
|
|
318
|
+
any per-tap `redact_keys`/`redact_pii_keys` you set (opt-in — none by default;
|
|
319
|
+
details in the Door 2 section).
|
|
320
|
+
|
|
321
|
+
**Unhappy paths**:
|
|
322
|
+
|
|
323
|
+
```ruby
|
|
324
|
+
Foam::Otel.create_ingest_span_processor(token: "", environment: "production")
|
|
325
|
+
# => raises ArgumentError: [foam] Foam::Otel.create_ingest_span_processor requires a non-empty token
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
- `OTEL_SDK_DISABLED=true` at construction → the factory returns an inert
|
|
329
|
+
no-op instance (their setup line keeps working, nothing is sent) and warns:
|
|
330
|
+
`[foam] OTEL_SDK_DISABLED=true — this foam ingest entry is disabled by the
|
|
331
|
+
operator kill switch and will send nothing.`
|
|
332
|
+
- Tap wired onto FOAM'S OWN pipeline (self-ingest wiring bug) → one loud
|
|
333
|
+
warning and the tap STANDS DOWN — door 1 already exports that data, so
|
|
334
|
+
nothing is lost and nothing is doubled.
|
|
335
|
+
- `init()` already exports a signal AND a tap sits on a separate foreign
|
|
336
|
+
pipeline for the same signal → one loud warning; BOTH keep flowing
|
|
337
|
+
(different pipelines; `foam.ingest.tier` distinguishes the copies).
|
|
338
|
+
|
|
339
|
+
When the TRACES slot is foreign-owned, foam also skips its automatic
|
|
340
|
+
tier-1/2 instrumentation activation (it never adds span producers or Rack
|
|
341
|
+
middleware to a pipeline it does not own) and warns once;
|
|
342
|
+
`additional_instrumentations` you pass explicitly are still installed. The
|
|
343
|
+
same holds for the process-wide propagator: a propagator the incumbent
|
|
344
|
+
installed first is KEPT (foam installs its W3C set only over the API default),
|
|
345
|
+
with one warning.
|
|
194
346
|
|
|
195
347
|
### Scenario 5 — per-signal composition
|
|
196
348
|
|
|
@@ -200,6 +352,13 @@ metrics and logs on the full contract:
|
|
|
200
352
|
```ruby
|
|
201
353
|
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FOAM_OTEL_TOKEN)
|
|
202
354
|
# → warns: traces foreign-owned (foam dark for traces); metrics + logs → foam, full contract.
|
|
355
|
+
|
|
356
|
+
# The doors COMPOSE: add the traces tap in THEIR setup and foam receives
|
|
357
|
+
# traces too — the tap rides THEIR tracer, and trace ids correlate with
|
|
358
|
+
# foam-exported logs/metrics through the shared context:
|
|
359
|
+
their_provider.add_span_processor(Foam::Otel.create_ingest_span_processor(
|
|
360
|
+
token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
|
|
361
|
+
# (no double-claim warning: init() serves metrics+logs, the tap serves traces)
|
|
203
362
|
```
|
|
204
363
|
|
|
205
364
|
### Scenario 6 — tests / CI (fully inert, no token needed)
|
|
@@ -236,6 +395,143 @@ end
|
|
|
236
395
|
|
|
237
396
|
---
|
|
238
397
|
|
|
398
|
+
## Door 2 — the ingest entries
|
|
399
|
+
|
|
400
|
+
When a foreign OTel SDK owns a signal (scenario 4), the ingest entries are
|
|
401
|
+
how that pipeline's data ALSO reaches foam: constructed instances you add —
|
|
402
|
+
one line per claimed signal — to the CUSTOMER'S OWN OTel setup. Additive
|
|
403
|
+
readers only: they never mutate their data, their resource, or their export.
|
|
404
|
+
|
|
405
|
+
> **All three entries are SHIPPED.** The traces entry rides the stable 1.x SDK;
|
|
406
|
+
> the log and metric entries ride upstream's pre-1.0 SDKs (logs "development",
|
|
407
|
+
> metrics alpha), pinned to this gem's capped ranges (GOTCHAS R2/R5) so a 0.x
|
|
408
|
+
> drift is a deliberate re-prove, never a silent change. The pre-1.0 SDK
|
|
409
|
+
> maturity is a version-pin caveat, not a deferral — the entries are available
|
|
410
|
+
> and supported today.
|
|
411
|
+
>
|
|
412
|
+
> **Version-pin posture:** because the log and metric taps ride those pre-1.0
|
|
413
|
+
> upstream SDKs, they are version-pinned and re-proven on every 0.x bump
|
|
414
|
+
> rather than assumed stable — shipped and supported either way.
|
|
415
|
+
|
|
416
|
+
### The three entries
|
|
417
|
+
|
|
418
|
+
All three share one signature (`token:` and `environment:` required; the
|
|
419
|
+
redaction kwargs are the ONLY keys that tap redacts — opt-in, none by default,
|
|
420
|
+
scoped to that tap; `diagnostics:` is tap-scoped narration). Construction
|
|
421
|
+
validates loudly at boot and NEVER
|
|
422
|
+
throws afterwards — a failure inside a tap can never break their pipeline.
|
|
423
|
+
Fleet canon mapping (pinned by `spec/export_surface_spec.rb`):
|
|
424
|
+
`create_ingest_span_processor` = `createFoamIngestSpanProcessor`,
|
|
425
|
+
`create_ingest_log_record_processor` = `createFoamIngestLogRecordProcessor`,
|
|
426
|
+
`create_ingest_metric_reader` = `createFoamIngestMetricReader`.
|
|
427
|
+
|
|
428
|
+
**Traces** — a SpanProcessor for THEIR TracerProvider (scenario 4 snippet):
|
|
429
|
+
|
|
430
|
+
```ruby
|
|
431
|
+
processor = Foam::Otel.create_ingest_span_processor(
|
|
432
|
+
token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV"),
|
|
433
|
+
redact_keys: %w[voucher_code], # optional: extra secret keys, this tap only
|
|
434
|
+
redact_pii_keys: %w[customer_email], # optional: your PII enumeration, this tap only
|
|
435
|
+
diagnostics: false) # optional: per-tap [foam] narration
|
|
436
|
+
their_tracer_provider.add_span_processor(processor)
|
|
437
|
+
# returns a new instance per call; no-op inert instance under OTEL_SDK_DISABLED=true;
|
|
438
|
+
# raises ArgumentError at construction for a blank token/environment
|
|
439
|
+
```
|
|
440
|
+
|
|
441
|
+
**Logs** — a LogRecordProcessor for THEIR LoggerProvider. The tap's
|
|
442
|
+
`redact_keys`/`redact_pii_keys` (if any) match over record attributes and keys
|
|
443
|
+
nested in structured bodies; free-text bodies ride RAW (no value-pattern pass):
|
|
444
|
+
|
|
445
|
+
```ruby
|
|
446
|
+
provider = OpenTelemetry::SDK::Logs::LoggerProvider.new(resource: their_resource)
|
|
447
|
+
provider.add_log_record_processor(their_log_processor)
|
|
448
|
+
provider.add_log_record_processor(Foam::Otel.create_ingest_log_record_processor(
|
|
449
|
+
token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
|
|
450
|
+
OpenTelemetry.logger_provider = provider
|
|
451
|
+
```
|
|
452
|
+
|
|
453
|
+
**Metrics** — a PeriodicMetricReader for THEIR MeterProvider. Attach order is
|
|
454
|
+
forgiving: `add_metric_reader` back-fills instruments created before the tap
|
|
455
|
+
attached. Foam's reader collects on its own schedule beside theirs:
|
|
456
|
+
|
|
457
|
+
```ruby
|
|
458
|
+
OpenTelemetry.meter_provider.add_metric_reader(Foam::Otel.create_ingest_metric_reader(
|
|
459
|
+
token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
|
|
460
|
+
```
|
|
461
|
+
|
|
462
|
+
### Door 2 — the required loop step
|
|
463
|
+
|
|
464
|
+
The tap's exports to foam are outbound HTTP calls THEIR instrumentation will
|
|
465
|
+
span, and their pipeline hands those spans back to the tap — a feedback loop
|
|
466
|
+
in THEIR pipeline. Three layers keep it closed:
|
|
467
|
+
|
|
468
|
+
1. **The SDK suppression context** (automatic): foam's exporters send inside
|
|
469
|
+
`untraced`, which the official http instrumentations respect.
|
|
470
|
+
2. **REQUIRED — suppress foam's endpoint host in THEIR http instrumentation**
|
|
471
|
+
(belt for non-contrib patches and older contrib versions): add
|
|
472
|
+
`untraced_hosts: ["otel.api.foam.ai"]` to their Net::HTTP/Excon config
|
|
473
|
+
(scenario 4 snippet). When the `OTEL_EXPORTER_OTLP_ENDPOINT` override is
|
|
474
|
+
active, list THAT host instead. Faraday/HTTP (httprb)/HTTPX cannot take
|
|
475
|
+
the option, but they also never see the tap's exports — foam's exporters
|
|
476
|
+
use Net::HTTP only.
|
|
477
|
+
3. **The echo filter** (structural backstop): a client-kind span naming the
|
|
478
|
+
ACTIVE foam export host is telemetry about foam's own export call — the
|
|
479
|
+
tap refuses to re-export its own echo, with one loud warning naming the
|
|
480
|
+
missing step. Their pipeline keeps that span untouched.
|
|
481
|
+
|
|
482
|
+
### Identity on the wire
|
|
483
|
+
|
|
484
|
+
| | their pipeline | foam's copy |
|
|
485
|
+
| --- | --- | --- |
|
|
486
|
+
| `service.name` / `service.version` / `service.instance.id` | theirs | theirs, untouched (the data carries THEIR identity) |
|
|
487
|
+
| all attributes / events / bodies | raw, byte-identical | your per-tap `redact_keys`/`redact_pii_keys` applied (default: none → identical to theirs) |
|
|
488
|
+
| `deployment.environment.name` | theirs (if any) | the tap's `environment:` argument (stamped at export time) |
|
|
489
|
+
| `foam.ingest.tier` | absent | `"external"` (the wire-contract marker) |
|
|
490
|
+
| `telemetry.distro.name` / `telemetry.distro.version` | absent | `"foam"` / the gem version (support reads the tap's presence and version from telemetry alone) |
|
|
491
|
+
|
|
492
|
+
The stamp is merged onto FOAM'S COPY at export serialization — never onto
|
|
493
|
+
the shared objects; it changes exactly those four keys and nothing else.
|
|
494
|
+
|
|
495
|
+
### External-tier caveats (recorded, never fought)
|
|
496
|
+
|
|
497
|
+
- THEIR sampler gates what the traces tap sees (unsampled spans never reach
|
|
498
|
+
processors).
|
|
499
|
+
- THEIR Views shape what the metrics tap receives; per-reader temporality is
|
|
500
|
+
not a Ruby knob — aggregations default cumulative (or the operator's
|
|
501
|
+
`OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE`), and cumulative totals
|
|
502
|
+
start at attach time.
|
|
503
|
+
- Lifecycle rides THEIR pipeline: their provider's `shutdown`/`force_flush`
|
|
504
|
+
drives the tap's. Serverless mirror: flush THEIR provider at handler end
|
|
505
|
+
(the platform freezes the process before batch timers fire).
|
|
506
|
+
- Kill switch and endpoint override apply to taps exactly as to `init`:
|
|
507
|
+
`OTEL_SDK_DISABLED=true` at construction → inert instance + one warning;
|
|
508
|
+
`OTEL_EXPORTER_OTLP_ENDPOINT` honored with the loud warning; per-signal
|
|
509
|
+
OTLP endpoint variants stay INERT + warned.
|
|
510
|
+
|
|
511
|
+
### Puma / Unicorn note (door 2)
|
|
512
|
+
|
|
513
|
+
Construct taps where THEIR setup runs — with `SDK.configure` in an
|
|
514
|
+
initializer under cluster preload, taps are inherited by workers. The span
|
|
515
|
+
and log taps self-heal in the child (the stock batch processors reset on
|
|
516
|
+
first use after a fork); the metrics tap is restarted by foam's own
|
|
517
|
+
`Process._fork` hook, which the factory installs (a separate registry from
|
|
518
|
+
`init`'s own readers, so a per-worker re-init never orphans a tap — GOTCHAS
|
|
519
|
+
R7). `Foam::Otel.after_fork!` covers both if you wire it manually.
|
|
520
|
+
|
|
521
|
+
### Tests / CI
|
|
522
|
+
|
|
523
|
+
Gate the add-line on the same condition you would give `enabled:` — the
|
|
524
|
+
placement line IS the switch (no foam-invented env var):
|
|
525
|
+
|
|
526
|
+
```ruby
|
|
527
|
+
if ENV["APP_ENV"] == "production"
|
|
528
|
+
c.add_span_processor(Foam::Otel.create_ingest_span_processor(
|
|
529
|
+
token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
|
|
530
|
+
end
|
|
531
|
+
```
|
|
532
|
+
|
|
533
|
+
---
|
|
534
|
+
|
|
239
535
|
## Failure modes — what happens and where the warning appears
|
|
240
536
|
|
|
241
537
|
| Situation | Behavior |
|
|
@@ -244,8 +540,13 @@ end
|
|
|
244
540
|
| `enabled: true` but blank `token` | raises `ArgumentError` at boot |
|
|
245
541
|
| `enabled: false` | fully inert; helpers no-op silently |
|
|
246
542
|
| missing `version` | `[foam]` warning, continues |
|
|
247
|
-
| a foreign OTel SDK owns a signal | `[foam]` warning once, naming the owner; foam dark for that signal |
|
|
543
|
+
| a foreign OTel SDK owns a signal | `[foam]` warning once, naming the owner AND the matching door-2 ingest entry; foam dark for that signal |
|
|
248
544
|
| `init` called twice | `[foam]` warning; returns the first instance (providers are process-global) |
|
|
545
|
+
| ingest entry with blank `token:`/`environment:` | raises `ArgumentError` at construction (on your machine) |
|
|
546
|
+
| ingest entry under `OTEL_SDK_DISABLED=true` | inert no-op instance + one `[foam]` warning; their setup line never crashes |
|
|
547
|
+
| ingest tap wired onto foam's OWN pipeline | `[foam]` wiring-bug warning once; the tap STANDS DOWN (zero duplicates) |
|
|
548
|
+
| `init()` exports a signal AND a tap on a separate foreign pipeline | `[foam]` wiring-bug warning once; both keep flowing (`foam.ingest.tier` distinguishes them) |
|
|
549
|
+
| their instrumentation traces foam's export calls (missing loop step) | echo filter refuses foam's own echo + one `[foam]` warning naming the required `untraced_hosts` step |
|
|
249
550
|
|
|
250
551
|
---
|
|
251
552
|
|
|
@@ -253,11 +554,14 @@ end
|
|
|
253
554
|
|
|
254
555
|
| Var | Posture |
|
|
255
556
|
| --- | --- |
|
|
256
|
-
| `OTEL_SDK_DISABLED=true` | HONORED — full kill switch, supersedes `enabled: true` (
|
|
257
|
-
| `OTEL_EXPORTER_OTLP_ENDPOINT` | HONORED — the one operator-level override of the pinned fleet endpoint (for foam's own conformance rig / enterprise egress). Active → loud `[foam]` warning naming the destination. |
|
|
557
|
+
| `OTEL_SDK_DISABLED=true` | HONORED — full kill switch, supersedes `enabled: true` (matched trimmed + case-insensitive, so `TRUE` / ` true ` also kill — the fleet's one emergency switch behaves the same across languages). Door-2 ingest entries honor it too: construction returns an inert instance (read once at construction; changing it implies a restart). |
|
|
558
|
+
| `OTEL_EXPORTER_OTLP_ENDPOINT` | HONORED — the one operator-level override of the pinned fleet endpoint (for foam's own conformance rig / enterprise egress). Active → loud `[foam]` warning naming the destination. Applies to door-2 taps identically (and moves the host the required loop step must name). |
|
|
258
559
|
| `OTEL_PROPAGATORS=none` | HONORED — turns trace propagation OFF (links lost) while telemetry keeps flowing; warns. Any other value warns and is ignored (foam's propagator set is fixed: W3C tracecontext + baggage). |
|
|
259
560
|
| `OTEL_BSP_SCHEDULE_DELAY` / `OTEL_BLRP_SCHEDULE_DELAY` / `OTEL_METRIC_EXPORT_INTERVAL` | HONORED — batch cadence, read natively by the upstream SDK. |
|
|
260
|
-
|
|
|
561
|
+
| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | INERT — foam wires ONE resolved endpoint into all three exporters explicitly, so per-signal endpoint vars never redirect (or split) foam's export. Set → loud `[foam]` warning that it is inert. |
|
|
562
|
+
| `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | INERT — as above (warns when set). |
|
|
563
|
+
| `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | INERT — as above (warns when set). |
|
|
564
|
+
| every other `OTEL_*` | INERT — `OTEL_SERVICE_NAME` / `OTEL_RESOURCE_ATTRIBUTES` never override the init-declared identity; sampler/exporter/span-limit vars are ignored. |
|
|
261
565
|
|
|
262
566
|
---
|
|
263
567
|
|
|
@@ -269,6 +573,12 @@ end
|
|
|
269
573
|
- **No data at all** → set `diagnostics: true` and read the `[foam]` lines; check
|
|
270
574
|
`init` runs before your app code, that `enabled` is true, and that a token is
|
|
271
575
|
wired.
|
|
576
|
+
- **No door-2 data** → check, in order: `OTEL_SDK_DISABLED` unset (a killed tap
|
|
577
|
+
is inert and warned once at boot); the tap is on the RIGHT provider (the one
|
|
578
|
+
their setup actually registers — a wiring-bug warning at first traffic means
|
|
579
|
+
self-ingest stand-down); the `[foam]` wiring-bug/echo warnings in the boot
|
|
580
|
+
log; construct with `diagnostics: true` for per-export narration; for
|
|
581
|
+
metrics under Puma, see the Door 2 Puma note.
|
|
272
582
|
- **A downstream rejects `traceparent`** → set `OTEL_PROPAGATORS=none` to stop
|
|
273
583
|
injection while keeping telemetry flowing.
|
|
274
584
|
|