foam-otel 1.8.0 → 1.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/GOTCHAS.md +61 -0
- data/README.md +35 -10
- data/lib/foam/otel/before_send.rb +128 -30
- data/lib/foam/otel/config.rb +48 -5
- data/lib/foam/otel/version.rb +22 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 6ef766d3509a7ee5daa25c7f53dce352b9c67c8ed22283099c72f3d6f0b8e957
|
|
4
|
+
data.tar.gz: caca5adca96f4e3dab3d0610ce54000858cb231cbf167e5bb6dd0c6f1dcd6a9b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 55da8fd55a07d287d81fddf5a65eeb8b7e0f743d1ec8b034362972f5c02a15a84bab18119aacf9e881b4deefc61d6958e062990fe3b205f66f2a88fd11ba514b
|
|
7
|
+
data.tar.gz: 70e46934c6250308cb8d15c90a160eeab5ce916b294cfc7395e46fc07e0d8d2027ded17f3f21cdcf3125c35773fb899324eb0374c6326c04d73a2908e0e1024d
|
data/GOTCHAS.md
CHANGED
|
@@ -808,6 +808,67 @@ exfiltratable — the value-pattern secret layer is the required second control
|
|
|
808
808
|
|
|
809
809
|
---
|
|
810
810
|
|
|
811
|
+
## F16: before_send hooks run inside the export lock, and record bookkeeping must be re-normalized behind them
|
|
812
|
+
|
|
813
|
+
- **Trap**: Three traps share the seam. (1) The batch processors call
|
|
814
|
+
`exporter.export` inside `@export_mutex.synchronize` with no lock timeout —
|
|
815
|
+
a customer `before_send` hook that blocks (DB lookup, un-timed HTTP, sleep)
|
|
816
|
+
wedges the worker thread AND any concurrent `force_flush`/`shutdown`
|
|
817
|
+
acquiring the same mutex: the process can hang at exit until SIGKILL.
|
|
818
|
+
(2) The OTLP exporters encode `dropped_attributes_count` as
|
|
819
|
+
`total_recorded_attributes - attributes.size` into a protobuf **uint32** —
|
|
820
|
+
a hook that ADDS an attribute without the total being fixed up makes the
|
|
821
|
+
subtraction negative, protobuf raises `RangeError`, `encode` rescues to
|
|
822
|
+
nil, and the WHOLE export request — every healthy record batched beside
|
|
823
|
+
the touched one — is dropped, silently, forever (the same trap
|
|
824
|
+
session_stitching already documents for foam's own stamped attributes; a
|
|
825
|
+
hand-built replacement record with nil totals is the NoMethodError variant
|
|
826
|
+
of the same encode failure). (3) A hook fault marks the whole batch
|
|
827
|
+
FAILURE (redacting-exporter parity), and `force_flush` aborts its drain
|
|
828
|
+
loop on the first non-SUCCESS batch — so a hook that faults
|
|
829
|
+
DETERMINISTICALLY on some record shape makes the at-exit flush abandon
|
|
830
|
+
everything queued behind the first faulted batch.
|
|
831
|
+
- **Sources**:
|
|
832
|
+
- Installed source: `OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor`
|
|
833
|
+
`#export_batch` inside `@export_mutex.synchronize`
|
|
834
|
+
(opentelemetry-sdk `trace/export/batch_span_processor.rb:189`), same shape
|
|
835
|
+
in `logs/export/batch_log_record_processor.rb:186`; `force_flush`'s
|
|
836
|
+
`return result_code unless result_code == SUCCESS` drain-abort
|
|
837
|
+
(`batch_span_processor.rb:113-115`).
|
|
838
|
+
- Installed source: `dropped_attributes_count: span_data.total_recorded_attributes - span_data.attributes&.size.to_i`
|
|
839
|
+
(opentelemetry-exporter-otlp `exporter.rb:343`; logs twin
|
|
840
|
+
`logs_exporter.rb:311`); `encode` rescues `StandardError` → nil →
|
|
841
|
+
`send_bytes(nil)` returns FAILURE for the whole request.
|
|
842
|
+
- In-repo precedent: `lib/foam/otel/session_stitching.rb` bumps
|
|
843
|
+
`@total_recorded_attributes` when foam adds its own attribute, guarding
|
|
844
|
+
against exactly trap (2).
|
|
845
|
+
- **Decision & why**: Hooks stay ON the export thread (any queue/thread
|
|
846
|
+
indirection would break the drop-before-serialize guarantee and add an
|
|
847
|
+
unbounded buffer); foam re-normalizes `total_recorded_*` on every
|
|
848
|
+
surviving record after the pipeline (`BeforeSend.restore_counters!` —
|
|
849
|
+
size + SDK-dropped delta captured pre-hook, nil-safe), hands hooks COPIED
|
|
850
|
+
attribute values and log bodies so in-place mutation cannot bleed into
|
|
851
|
+
the live record or a tenant's buffered view, and keeps FAILURE accounting
|
|
852
|
+
for faulted records (parity with the redacting exporters) while warning
|
|
853
|
+
ONCE per batch with error classes only. The blocking-hook risk is
|
|
854
|
+
accepted and documented (README "The `before_send` hook") — it is the
|
|
855
|
+
customer's own code on their own pipeline, the same trust grade as a
|
|
856
|
+
tenant processor, and rule 49's DoS bounds govern foam's code, not the
|
|
857
|
+
customer's.
|
|
858
|
+
- **Mitigation**: README pins the contract: non-blocking hooks only; stamp
|
|
859
|
+
request/job-thread context onto attributes at capture; replace (never
|
|
860
|
+
mutate) events/links/status; deterministic-fault hooks are a customer bug
|
|
861
|
+
surfaced by the aggregated per-batch `[foam]` warning + batch FAILURE.
|
|
862
|
+
Boot-time arity validation rejects hooks that cannot take one positional
|
|
863
|
+
argument (the Sentry two-arg port) so the fault-every-record shape is
|
|
864
|
+
caught on the engineer's machine (rule 10).
|
|
865
|
+
- **Test**: `spec/before_send_spec.rb` ("re-normalizes total_recorded",
|
|
866
|
+
"REAL OTLP encode", "aggregates hook faults", "arity", "in-place value
|
|
867
|
+
mutation never bleeds"); `spec/before_send_champ_scenario_spec.rb` (the
|
|
868
|
+
export-thread / stamped-context proof).
|
|
869
|
+
|
|
870
|
+
---
|
|
871
|
+
|
|
811
872
|
## General gotchas (applicable to Ruby)
|
|
812
873
|
|
|
813
874
|
- **G1 — init after target import / pre-init no-op**: the API's proxy providers
|
data/README.md
CHANGED
|
@@ -331,7 +331,7 @@ not captured — inbound only.
|
|
|
331
331
|
| `additional_metric_readers:` | Array | no | `[]` | Tenant seam, metrics. |
|
|
332
332
|
| `additional_instrumentations:` | Array | no | `[]` | Constructed tier-2 instrumentation instances to register (fault-isolated: one that throws is skipped with a `[foam]` warning). |
|
|
333
333
|
| `ignored_outbound_hosts:` | Array<String> | no | `[]` | Hosts whose outbound calls produce no spans — EXTENDS the built-in export-loop guard. For a co-resident agent's intake or a tenant exporter's endpoint. Applies to the clients whose official instrumentation supports host suppression: **Net::HTTP and Excon**. Faraday/HTTP (httprb)/HTTPX have no upstream `untraced_hosts` option — foam warns loudly at init when one of those is bundled (GOTCHAS G13); foam's own export loop is guarded for every client regardless. |
|
|
334
|
-
| `before_send:` | Proc/Array | no | `[]` | Export hook (1.8.0): one callable — or an Array run as a pipeline — called with every **span and log record** at the export boundary, after batching and BEFORE foam's redaction pass and serialization, so nothing a hook drops ever leaves the process. Return the record to export it
|
|
334
|
+
| `before_send:` | Proc/Array | no | `[]` | Export hook (1.8.0): one callable — or an Array run as a pipeline — called with every **span and log record** at the export boundary, after batching and BEFORE foam's redaction pass and serialization, so nothing a hook drops ever leaves the process. Return the record (or a same-type replacement) to export it — in-place mutation supported; return `nil` to drop it; a raise or foreign return drops **that record** loudly, fail-closed, never a raise into your threads. Runs BEFORE redaction — a hook can never widen what ships. Spans and logs only; door 1 only. A non-callable (or a one-arg-incompatible lambda) raises at boot. Process-global. Hooks run on the export thread — keep them non-blocking. Full contract: "The `before_send` hook" below. |
|
|
335
335
|
| `capture_payloads:` | Symbol/String | no | `:off` | Inbound HTTP **body** capture on the rack server span (see "Payload (body) capture"). `:off` = zero teeing, zero middleware (the default); `:errors` = tee per request, attach only on exception/5xx; `:always` = attach on every request. Equivalent strings accepted; anything else raises at boot. Overridden in both directions by the `FOAM_CAPTURE_PAYLOADS` env clamp (env table below). Textual payloads only, 8192-char cap + `…[truncated]`, true `.size` attributes; sizes-only for binary/compressed. |
|
|
336
336
|
| `diagnostics:` | Boolean | no | false | Verbose `[foam]` self-reporting of init/health. Warnings and errors are always loud regardless. |
|
|
337
337
|
| `secret_heuristics:` | Boolean | no | true | The value-pattern secret layer's HEURISTIC tier (generic keyword+entropy detection, see "The value-pattern secret layer"). `false` disables ONLY the heuristics — the named provider patterns, the credential floor and the redaction-coverage contract have no off switch. Disabling logs one loud `[foam]` line at init (an explicit, audited decision for telemetry whose legitimate values collide with the heuristics). |
|
|
@@ -661,17 +661,42 @@ set.
|
|
|
661
661
|
|
|
662
662
|
The per-record contract (Sentry `beforeSend` semantics):
|
|
663
663
|
|
|
664
|
-
* **return the record
|
|
665
|
-
Mutating it in place is supported —
|
|
666
|
-
|
|
667
|
-
`record.attributes.delete("k")
|
|
664
|
+
* **return the record — or a replacement of the same type** → it continues
|
|
665
|
+
to the next hook / to export. Mutating it in place is supported —
|
|
666
|
+
`record.attributes` arrives as your own unfrozen copy (values included),
|
|
667
|
+
so `record.attributes["k"] = v`, `record.attributes.delete("k")`,
|
|
668
|
+
`record.attributes["k"].gsub!(...)` and (for logs) editing `record.body`
|
|
669
|
+
all just work without bleeding into the span your app still holds or the
|
|
670
|
+
view a tenant processor saw. Events, links and status are the exception:
|
|
671
|
+
**replace them wholesale** (`record.events = [...]`) — never mutate them
|
|
672
|
+
in place, they are shared structures. Foam re-normalizes the record's
|
|
673
|
+
`total_recorded_*` bookkeeping after your hooks run, so adding/deleting
|
|
674
|
+
attributes never corrupts the wire's dropped-count accounting.
|
|
668
675
|
* **return `nil`** → the record is dropped, silently (that is the filter
|
|
669
676
|
mechanism, not an error).
|
|
670
|
-
* **raise, or return
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
677
|
+
* **raise, or return a foreign object** → that record is dropped
|
|
678
|
+
**loudly** — one aggregated `[foam]` warning per batch, error classes
|
|
679
|
+
only (messages can carry your data) — and the batch reports failure:
|
|
680
|
+
fail-closed, a faulted hook never ships a record you may have meant to
|
|
681
|
+
scrub, and never raises into your application or export threads. Healthy
|
|
682
|
+
records in the same batch still export. Because a fault marks the whole
|
|
683
|
+
batch failed, a hook that faults **deterministically** also makes the
|
|
684
|
+
at-exit flush report failure — fix faulting hooks promptly (GOTCHAS F16).
|
|
685
|
+
|
|
686
|
+
**Hooks run on foam's export thread, inside the batch processor's export
|
|
687
|
+
lock.** Keep them fast and non-blocking: no network calls, no DB lookups,
|
|
688
|
+
no sleeping — a blocked hook stalls the export pipeline and can wedge the
|
|
689
|
+
at-exit flush. Anything a hook needs from the request/job thread
|
|
690
|
+
(tenant context, user identity) must be stamped onto the record as an
|
|
691
|
+
attribute at capture time (see the `additional_span_processors` stamper
|
|
692
|
+
pattern); thread-locals are gone by the time the hook runs.
|
|
693
|
+
|
|
694
|
+
Boot validation is strict (rule 10): a non-callable — or a **lambda/Method
|
|
695
|
+
whose signature cannot accept one positional argument** (e.g. a Sentry-port
|
|
696
|
+
`->(event, hint) { ... }` with two required params, or a required keyword)
|
|
697
|
+
— raises `ArgumentError` at `init`, because it would otherwise fault on
|
|
698
|
+
every record and ship zero telemetry from a green boot. Make the extra
|
|
699
|
+
Sentry `hint` param optional (`->(event, hint = nil)`) and it is accepted.
|
|
675
700
|
|
|
676
701
|
Scope: **spans and log records** (the two record-shaped signals — the hook
|
|
677
702
|
receives `SpanData` or `LogRecordData`; distinguish them with `is_a?` or
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "constants"
|
|
3
4
|
require_relative "diagnostics"
|
|
4
5
|
|
|
5
6
|
module Foam
|
|
@@ -53,37 +54,134 @@ module Foam
|
|
|
53
54
|
|
|
54
55
|
# Run the hook pipeline over one record: each hook receives the
|
|
55
56
|
# previous hook's return. Returns the surviving record, nil for an
|
|
56
|
-
# intentional drop, or ERROR for a faulted one.
|
|
57
|
-
|
|
58
|
-
|
|
57
|
+
# intentional drop, or ERROR for a faulted one. Fault DETAIL is
|
|
58
|
+
# appended to `faults` (error class / return class only — never a
|
|
59
|
+
# message, which can embed record values) so the exporter can warn
|
|
60
|
+
# ONCE per batch instead of once per record (the redacting exporters'
|
|
61
|
+
# aggregation convention; an unthrottled per-record warn is a stderr
|
|
62
|
+
# flood an adversary-shaped record stream controls — rule 15's
|
|
63
|
+
# "loud but never a flood" posture).
|
|
64
|
+
def run(hooks, record, signal, faults)
|
|
65
|
+
current, counters = thaw(record)
|
|
59
66
|
hooks.each do |hook|
|
|
60
67
|
result = hook.call(current)
|
|
61
68
|
return nil if result.nil?
|
|
62
69
|
|
|
63
70
|
unless result.is_a?(record.class)
|
|
64
|
-
|
|
65
|
-
"dropped (return the record, a replacement of the same type, or nil to drop; " \
|
|
66
|
-
"fail-closed, rule 14)")
|
|
71
|
+
faults << "hook returned #{result.class}"
|
|
67
72
|
return ERROR
|
|
68
73
|
end
|
|
69
74
|
current = result
|
|
70
75
|
end
|
|
76
|
+
restore_counters!(current, counters)
|
|
71
77
|
current
|
|
72
78
|
rescue StandardError, SystemStackError => e
|
|
73
|
-
|
|
74
|
-
|
|
79
|
+
faults << e.class.name
|
|
80
|
+
# Full detail (message included) only under diagnostics: true —
|
|
81
|
+
# e.message can carry attribute values.
|
|
82
|
+
Diagnostics.info("before_send fault detail (#{signal}): #{e.class}: #{e.message}")
|
|
75
83
|
ERROR
|
|
76
84
|
end
|
|
77
85
|
|
|
78
|
-
# A
|
|
79
|
-
#
|
|
86
|
+
# A dup the hook can safely mutate IN PLACE: the struct and the
|
|
87
|
+
# attributes hash are copied, and so are the mutable attribute VALUES
|
|
88
|
+
# (strings, and strings inside string arrays — the OTel attribute
|
|
89
|
+
# types) plus a structured log body — so `record.attributes["k"] = v`,
|
|
90
|
+
# `attributes["k"].gsub!(...)` and `record.body["k"] = v` never bleed
|
|
91
|
+
# into the live span/record the customer still holds or the masked
|
|
92
|
+
# view a tenant processor buffered (pipelines.rb), and never hit a
|
|
93
|
+
# frozen value. Events/links/status are NOT deep-copied: replace them
|
|
94
|
+
# wholesale (`record.events = [...]`), never mutate them in place —
|
|
95
|
+
# documented in the README.
|
|
96
|
+
#
|
|
97
|
+
# Also returns the record's SDK-drop counters, captured BEFORE the
|
|
98
|
+
# hooks run: the OTLP exporters encode dropped_attributes_count as
|
|
99
|
+
# `total_recorded_attributes - attributes.size` (a protobuf uint32),
|
|
100
|
+
# so a hook that ADDS an attribute without the total being fixed up
|
|
101
|
+
# afterwards makes the count NEGATIVE and the encoder drops the WHOLE
|
|
102
|
+
# export request — every healthy record in the batch, silently (the
|
|
103
|
+
# same trap session_stitching.rb documents and guards for foam's own
|
|
104
|
+
# added attributes). restore_counters! below re-normalizes the
|
|
105
|
+
# survivor from these.
|
|
80
106
|
def thaw(record)
|
|
81
107
|
copy = record.dup
|
|
82
108
|
if copy.respond_to?(:attributes) && copy.respond_to?(:attributes=)
|
|
83
109
|
attributes = copy.attributes
|
|
84
|
-
copy.attributes = attributes.nil? ? {} : attributes.
|
|
110
|
+
copy.attributes = attributes.nil? ? {} : attributes.transform_values { |v| thaw_value(v) }
|
|
85
111
|
end
|
|
86
|
-
copy
|
|
112
|
+
copy.body = thaw_body(copy.body) if copy.respond_to?(:body) && copy.respond_to?(:body=)
|
|
113
|
+
[copy, sdk_drop_counters(record)]
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
def thaw_value(value)
|
|
117
|
+
case value
|
|
118
|
+
when String then value.dup
|
|
119
|
+
when Array then value.map { |e| e.is_a?(String) ? e.dup : e }
|
|
120
|
+
else value
|
|
121
|
+
end
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
# Bounded deep copy of a log body (String / Hash / Array / scalar) so
|
|
125
|
+
# in-place body mutation is safe too; the depth guard mirrors the
|
|
126
|
+
# redaction engine's (rule 9 — an adversarial structure never drives
|
|
127
|
+
# a stack overflow).
|
|
128
|
+
def thaw_body(body, depth = 0)
|
|
129
|
+
return body if depth > MAX_REDACT_DEPTH
|
|
130
|
+
|
|
131
|
+
case body
|
|
132
|
+
when String then body.dup
|
|
133
|
+
when Hash then body.each_with_object({}) { |(k, v), out| out[k] = thaw_body(v, depth + 1) }
|
|
134
|
+
when Array then body.map { |v| thaw_body(v, depth + 1) }
|
|
135
|
+
else body
|
|
136
|
+
end
|
|
137
|
+
end
|
|
138
|
+
|
|
139
|
+
# How many attributes/events/links the SDK itself dropped (limits)
|
|
140
|
+
# BEFORE the hooks ran — nil-safe, clamped at zero.
|
|
141
|
+
def sdk_drop_counters(record)
|
|
142
|
+
{
|
|
143
|
+
attributes: sdk_dropped(record, :total_recorded_attributes, :attributes),
|
|
144
|
+
events: sdk_dropped(record, :total_recorded_events, :events),
|
|
145
|
+
links: sdk_dropped(record, :total_recorded_links, :links),
|
|
146
|
+
}
|
|
147
|
+
end
|
|
148
|
+
|
|
149
|
+
def sdk_dropped(record, total_getter, list_getter)
|
|
150
|
+
return 0 unless record.respond_to?(total_getter)
|
|
151
|
+
|
|
152
|
+
total = record.public_send(total_getter).to_i
|
|
153
|
+
size = collection_size(record, list_getter)
|
|
154
|
+
total > size ? total - size : 0
|
|
155
|
+
end
|
|
156
|
+
|
|
157
|
+
def collection_size(record, list_getter)
|
|
158
|
+
return 0 unless record.respond_to?(list_getter)
|
|
159
|
+
|
|
160
|
+
list = record.public_send(list_getter)
|
|
161
|
+
list.respond_to?(:size) ? list.size.to_i : 0
|
|
162
|
+
rescue StandardError
|
|
163
|
+
0
|
|
164
|
+
end
|
|
165
|
+
|
|
166
|
+
# Re-normalize the survivor's total_recorded_* bookkeeping after the
|
|
167
|
+
# hooks ran: total = what the record NOW carries + what the SDK had
|
|
168
|
+
# already dropped. Without this, a hook-ADDED attribute makes the
|
|
169
|
+
# OTLP encoder's `total - size` subtraction negative (protobuf uint32
|
|
170
|
+
# → RangeError → the encoder returns nil → the ENTIRE batch is
|
|
171
|
+
# dropped, silently, forever), and a hand-built replacement record
|
|
172
|
+
# with nil totals raises NoMethodError at the same line. Also keeps a
|
|
173
|
+
# hook DELETE honest (the wire no longer claims an SDK-limit drop).
|
|
174
|
+
def restore_counters!(record, counters)
|
|
175
|
+
if record.respond_to?(:total_recorded_attributes=)
|
|
176
|
+
record.total_recorded_attributes = collection_size(record, :attributes) + counters[:attributes]
|
|
177
|
+
end
|
|
178
|
+
if record.respond_to?(:total_recorded_events=)
|
|
179
|
+
record.total_recorded_events = collection_size(record, :events) + counters[:events]
|
|
180
|
+
end
|
|
181
|
+
if record.respond_to?(:total_recorded_links=)
|
|
182
|
+
record.total_recorded_links = collection_size(record, :links) + counters[:links]
|
|
183
|
+
end
|
|
184
|
+
record
|
|
87
185
|
end
|
|
88
186
|
|
|
89
187
|
# The span-side wrapper (composition over the redacting exporter, same
|
|
@@ -97,18 +195,19 @@ module Foam
|
|
|
97
195
|
|
|
98
196
|
def export(span_data, timeout: nil)
|
|
99
197
|
batch = span_data.to_a
|
|
100
|
-
|
|
198
|
+
faults = []
|
|
101
199
|
kept = batch.filter_map do |sd|
|
|
102
|
-
result = BeforeSend.run(@hooks, sd, :span)
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
200
|
+
result = BeforeSend.run(@hooks, sd, :span, faults)
|
|
201
|
+
result.equal?(ERROR) ? nil : result
|
|
202
|
+
end
|
|
203
|
+
# ONE aggregated warn per batch (the redacting exporters'
|
|
204
|
+
# convention) — class names only, never messages/values.
|
|
205
|
+
unless faults.empty?
|
|
206
|
+
Diagnostics.warn("before_send faulted for #{faults.length} span record(s) — dropped, never " \
|
|
207
|
+
"exported half-transformed (fail-closed, rule 14): #{faults.uniq.take(3).join(', ')}")
|
|
109
208
|
end
|
|
110
209
|
status = kept.empty? ? OpenTelemetry::SDK::Trace::Export::SUCCESS : @inner.export(kept, timeout: timeout)
|
|
111
|
-
|
|
210
|
+
faults.empty? ? status : OpenTelemetry::SDK::Trace::Export::FAILURE
|
|
112
211
|
rescue StandardError, SystemStackError
|
|
113
212
|
# Fail closed past the per-record guards, loudly (rule 15) —
|
|
114
213
|
# never raise into the batch thread.
|
|
@@ -128,18 +227,17 @@ module Foam
|
|
|
128
227
|
|
|
129
228
|
def export(log_record_data, timeout: nil)
|
|
130
229
|
batch = log_record_data.to_a
|
|
131
|
-
|
|
230
|
+
faults = []
|
|
132
231
|
kept = batch.filter_map do |lrd|
|
|
133
|
-
result = BeforeSend.run(@hooks, lrd, :log)
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
end
|
|
232
|
+
result = BeforeSend.run(@hooks, lrd, :log, faults)
|
|
233
|
+
result.equal?(ERROR) ? nil : result
|
|
234
|
+
end
|
|
235
|
+
unless faults.empty?
|
|
236
|
+
Diagnostics.warn("before_send faulted for #{faults.length} log record(s) — dropped, never " \
|
|
237
|
+
"exported half-transformed (fail-closed, rule 14): #{faults.uniq.take(3).join(', ')}")
|
|
140
238
|
end
|
|
141
239
|
status = kept.empty? ? OpenTelemetry::SDK::Logs::Export::SUCCESS : @inner.export(kept, timeout: timeout)
|
|
142
|
-
|
|
240
|
+
faults.empty? ? status : OpenTelemetry::SDK::Logs::Export::FAILURE
|
|
143
241
|
rescue StandardError, SystemStackError
|
|
144
242
|
Diagnostics.warn("before_send pass raised past its guards — log batch dropped (fail-closed, rule 14/15)")
|
|
145
243
|
OpenTelemetry::SDK::Logs::Export::FAILURE
|
data/lib/foam/otel/config.rb
CHANGED
|
@@ -108,22 +108,65 @@ module Foam
|
|
|
108
108
|
# #call — a lambda, a Proc, a Method, or any hook object) or an Array
|
|
109
109
|
# of callables run as a pipeline. Anything else — including a nil or
|
|
110
110
|
# non-callable INSIDE the array — raises here, never a silent no-op
|
|
111
|
-
# that leaves the customer believing their filter runs.
|
|
111
|
+
# that leaves the customer believing their filter runs. The array is
|
|
112
|
+
# COPIED before freezing (never freeze the caller's own object).
|
|
113
|
+
#
|
|
114
|
+
# Arity is validated too, where Ruby lets us introspect it: a lambda
|
|
115
|
+
# or Method that cannot accept ONE positional argument (a Sentry-port
|
|
116
|
+
# `->(event, hint) { }`, a forgotten-param `-> { }`, a required
|
|
117
|
+
# keyword) would pass a bare respond_to?(:call) check and then fault
|
|
118
|
+
# on EVERY record at export — a green boot that ships zero telemetry
|
|
119
|
+
# forever. That is exactly the boot-detectable programming error rule
|
|
120
|
+
# 10 exists for. Non-lambda Procs are arity-forgiving by language
|
|
121
|
+
# semantics and uninspectable callables get the benefit of the doubt.
|
|
112
122
|
def validate_before_send!(value, context: "Foam::Otel.init")
|
|
113
123
|
hooks = if value.nil?
|
|
114
124
|
[]
|
|
115
125
|
else
|
|
116
|
-
value.is_a?(Array) ? value : [value]
|
|
126
|
+
value.is_a?(Array) ? value.dup : [value]
|
|
117
127
|
end
|
|
118
128
|
hooks.each_with_index do |hook, index|
|
|
119
|
-
|
|
129
|
+
unless hook.respond_to?(:call)
|
|
130
|
+
raise ArgumentError, "#{context} before_send: must be a callable (responds to #call) or an " \
|
|
131
|
+
"Array of callables, got #{hook.class} at position #{index}"
|
|
132
|
+
end
|
|
133
|
+
next if before_send_arity_ok?(hook)
|
|
120
134
|
|
|
121
|
-
raise ArgumentError, "#{context} before_send:
|
|
122
|
-
"
|
|
135
|
+
raise ArgumentError, "#{context} before_send: the callable at position #{index} cannot be " \
|
|
136
|
+
"called with one record argument (it takes a different number of " \
|
|
137
|
+
"required positional parameters, or requires keywords) — before_send " \
|
|
138
|
+
"hooks are called as hook.call(record)"
|
|
123
139
|
end
|
|
124
140
|
hooks.freeze
|
|
125
141
|
end
|
|
126
142
|
|
|
143
|
+
# True when the hook can be invoked as hook.call(record). Lambdas and
|
|
144
|
+
# Methods enforce their signatures, so they are checked; non-lambda
|
|
145
|
+
# Procs coerce arguments (always fine); anything whose #call cannot
|
|
146
|
+
# be introspected passes (never a false boot failure).
|
|
147
|
+
def before_send_arity_ok?(hook)
|
|
148
|
+
params = if hook.is_a?(Proc)
|
|
149
|
+
return true unless hook.lambda?
|
|
150
|
+
|
|
151
|
+
hook.parameters
|
|
152
|
+
elsif hook.is_a?(Method) || hook.is_a?(UnboundMethod)
|
|
153
|
+
hook.parameters
|
|
154
|
+
else
|
|
155
|
+
begin
|
|
156
|
+
hook.method(:call).parameters
|
|
157
|
+
rescue StandardError
|
|
158
|
+
return true
|
|
159
|
+
end
|
|
160
|
+
end
|
|
161
|
+
return false if params.any? { |kind, _| kind == :keyreq }
|
|
162
|
+
|
|
163
|
+
required = params.count { |kind, _| kind == :req }
|
|
164
|
+
flexible = params.any? { |kind, _| kind == :opt || kind == :rest }
|
|
165
|
+
required == 1 || (required.zero? && flexible)
|
|
166
|
+
rescue StandardError
|
|
167
|
+
true
|
|
168
|
+
end
|
|
169
|
+
|
|
127
170
|
# ---- the capture_payloads option (payload-capture mandate 2026-07-28)
|
|
128
171
|
# Canonicalize a mode value: the three symbols, or their string
|
|
129
172
|
# spellings trimmed + case-insensitive, map to the canonical symbol;
|
data/lib/foam/otel/version.rb
CHANGED
|
@@ -124,6 +124,27 @@ module Foam
|
|
|
124
124
|
# non-callable raises at boot; process-global like enabled/endpoint
|
|
125
125
|
# (a second init never changes it). Door-1 only — the door-2 ingest
|
|
126
126
|
# taps are unchanged.
|
|
127
|
-
|
|
127
|
+
# 1.8.1: before_send REVIEW HARDENING (adversarial review, 21 confirmed
|
|
128
|
+
# findings; PATCH — bugfix-only, no API change). CRITICAL: the OTLP
|
|
129
|
+
# encoders derive dropped_attributes_count as total_recorded - size
|
|
130
|
+
# (protobuf uint32) — a hook-ADDED attribute made the count negative,
|
|
131
|
+
# the encoder raised, and the WHOLE batch (healthy siblings included)
|
|
132
|
+
# was silently dropped; foam now re-normalizes total_recorded_* on
|
|
133
|
+
# every surviving record (nil-safe for hand-built replacement records;
|
|
134
|
+
# hook DELETES no longer fake an SDK-limit drop) — GOTCHAS F16. thaw
|
|
135
|
+
# now copies attribute VALUES and deep-copies structured log bodies
|
|
136
|
+
# (in-place mutation never bleeds into the live record or the tenant
|
|
137
|
+
# masked view; frozen values never fault). Boot validation gains an
|
|
138
|
+
# arity check: a lambda/Method that cannot take one positional arg
|
|
139
|
+
# (the Sentry ->(event, hint) port) raises at init instead of
|
|
140
|
+
# faulting on every record. Hook faults warn ONCE per batch (error
|
|
141
|
+
# classes only — messages can carry values; detail under
|
|
142
|
+
# diagnostics). validate_before_send! copies the caller's array
|
|
143
|
+
# (never freezes customer-owned input). Same-type replacement records
|
|
144
|
+
# documented (always accepted). Champ scenario spec: CI
|
|
145
|
+
# service-container fallback, pg_isready gate, leak cleanup, numeric
|
|
146
|
+
# version pick. Hooks run on the export thread — non-blocking only
|
|
147
|
+
# (documented, GOTCHAS F16).
|
|
148
|
+
VERSION = "1.8.1"
|
|
128
149
|
end
|
|
129
150
|
end
|