foam-otel 1.2.0 → 1.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bb41e8c47c944ac8f252f47bededb07f8c73e152411f3a7069a8fcab3d1185a2
-  data.tar.gz: ab93229a7820e4b60439d8c30f903fd3d1be93a8d2dd723b032862813ac4278f
+  metadata.gz: 4e10efb9aaacdda1498a19efaa4af8338afbb98d89880cc6f662aa99205aab59
+  data.tar.gz: 436060cbcfc7ed33c171ab230a60d4f8b62a882ce46be22bcd81e9c497edbc84
 SHA512:
-  metadata.gz: 291f81494a360ee3ce94a09b6b98dafde0574e4078d61ec01bb3e3e3b7803a4f7034bc36ef17a33db79cbb886ad5c0d337c2ffa75762c5a9d04f52e89cdf6669
-  data.tar.gz: e81c1c075606402562ee74a8dc6752dd4d96f4b73235303a43cd0d20569ca7238a031b3c6c6d6726f5cf3f3c32858c2452cc5cbfac8ffd61f834f549738c6c1b
+  metadata.gz: 528f57356698e39e691b08bcc0b6f00bc409ad133afca0ac0bdeeb2d0caffbc052a7a19d7fe63b0285230eb16cd26117ede4bc0824c7c0f31377a4a983ad7458
+  data.tar.gz: 9e445e3ce3924ff924afd66e98c35b58fe536ae2d546a0f151d3d870f9359974b3bbfd781871fd39fef67d3d509573cab7c2ce573531067eedf3623943967f32

data/GOTCHAS.md

@@ -681,6 +681,22 @@ anthropic 1.59.0, gemini-ai 4.3.0, ruby_llm 1.16.0).
   beside it (`lib/foam/otel/runtime_metrics.rb` `guarded_callback`).
   *Test*: `spec/runtime_metrics_spec.rb` ("a poisoned reading is
   fail-to-dark", the foreign-slot and idempotence cases).
+- **G16 — Sidekiq's queue-hop trace shape is `propagation_style: :link`**:
+  the bundled contrib instrumentation's default (installed source:
+  opentelemetry-instrumentation-sidekiq-0.29.0 `instrumentation.rb:104`)
+  makes a performed job a SEPARATE trace whose first (consumer) span carries
+  a span LINK back to the enqueuing producer span — NOT a child inside the
+  web request's trace. Foam ships that upstream default untouched (rule 26:
+  the contrib pin sets the semconv; `FLOOR_INSTRUMENTATION_CONFIG` overrides
+  only the datastore raw-capture posture), so dashboards that follow ONLY
+  parent/child edges see the web trace end at the enqueue — follow the link,
+  or clamp the posture per deployment with the standard contrib lever
+  `OTEL_RUBY_INSTRUMENTATION_SIDEKIQ_CONFIG_OPTS='propagation_style=child'`
+  for one continuous web→job trace. Context is propagated across the hop
+  either way — a hop that DROPS it is a correctness bug, never accepted.
+  *Test*: `spec/floor_sidekiq_wire_spec.rb` (real Redis enqueue→perform:
+  producer span joins the request trace; consumer span LINKS the producer
+  span on the wire).
 
 ---
 

data/README.md

@@ -80,6 +80,14 @@ attributes, URLs/query strings, DB statement text, log bodies, LLM content —
 is captured RAW.** Pass `redact_keys`/`redact_pii_keys` to mask or erase
 specific fields (see below).
 
+> **Sidekiq trace shape** (GOTCHAS G16): the bundled instrumentation keeps
+> its upstream default `propagation_style: :link` — a performed job runs as
+> its OWN trace whose first span LINKS back to the enqueuing span (context is
+> always propagated across the hop; proven against a real Redis in
+> `spec/floor_sidekiq_wire_spec.rb`). Prefer one continuous web→job trace?
+> Set the standard contrib lever:
+> `OTEL_RUBY_INSTRUMENTATION_SIDEKIQ_CONFIG_OPTS='propagation_style=child'`.
+
 ### The LLM surface — OpenAI, Anthropic, Gemini (and ruby_llm)
 
 The Ruby contrib registry ships no LLM instrumentation, so foam does: thin,
@@ -87,7 +95,14 @@ presence-checked shims over the SDKs' public call sites, emitting the standard
 `gen_ai.*` semantic conventions — activity (model, response id, finish
 reasons, token usage, latency) AND prompt/response **content, RAW**
 (`gen_ai.input.messages` / `gen_ai.output.messages` /
-`gen_ai.system_instructions`). Covered: the official `openai` SDK
+`gen_ai.system_instructions`). Agent tool loops ride in full: tool-call
+turns keep their tool names/arguments and tool-result turns keep their
+`tool_call_id` linkage inside the message content. `gen_ai.operation.name`
+(and the `{operation} {model}` span name) follows the semconv operation for
+each API shape — `chat` for the OpenAI/Anthropic/ruby_llm chat seams,
+`generate_content` for Gemini's generateContent, fleet-identical with the
+js/python Gemini paths (**wire change in 1.2.1**: Gemini spans previously
+said `chat`). Covered: the official `openai` SDK
 (chat.completions.create + responses.create), the official `anthropic` SDK
 (messages.create), Gemini via the `gemini-ai` gem (Google ships no official
 Ruby SDK), and `ruby_llm` (one seam covering all its providers). Absent SDK =

data/RESEARCH.md

@@ -151,6 +151,20 @@ the customer lists in `redact_keys`/`redact_pii_keys`. An FDE can still clamp
 context calls for it — per-customer tightening (rule 18 / Area 4), not the
 default.
 
+**Sidekiq queue-hop posture (documented, GOTCHAS G16):** `-sidekiq 0.29.0`
+defaults `propagation_style: :link` (installed source `instrumentation.rb:104`)
+— a performed job is a SEPARATE trace whose consumer span LINKS the enqueuing
+producer span. Foam ships the upstream default untouched
+(`FLOOR_INSTRUMENTATION_CONFIG` overrides only the datastore raw-capture
+posture); the continuous web→job shape is one env lever away
+(`OTEL_RUBY_INSTRUMENTATION_SIDEKIQ_CONFIG_OPTS='propagation_style=child'`).
+Context always survives the hop. Wire-proven per rule 8b against a REAL
+redis-server with the REAL sidekiq gem (enqueue → Redis → embedded-server
+perform; producer/consumer spans + the link asserted from the exporter):
+`spec/floor_sidekiq_wire_spec.rb`. The Faraday floor entry carries the same
+rule-8b emission proof against a local listener:
+`spec/floor_wire_spec.rb`.
+
 **Runtime + GC metrics (hand-written — landed,
 `lib/foam/otel/runtime_metrics.rb`).** Ruby has no drop-in upstream metrics
 instrumentation gem (unlike Python's `-system-metrics` or the Node
@@ -165,8 +179,19 @@ construction (rule 26). Tests: `spec/runtime_metrics_spec.rb`.
 
 **Excluded (with reason):** `-factory_bot`, `-rspec` (test-only, no production
 value); `-net_ldap` (niche, PII/credential-adjacent — explicit FDE opt-in);
+`-logger 0.4.0` (SUPERSEDED by foam's own stdlib Logger bridge, §4: both
+patch the SAME `Logger#add` seam, and foam's bridge keeps the lazy-block
+contract, both loop guards, the logs-slot gating, and the opt-in redaction
+path the contrib bridge lacks. Two bridges on one seam would export every
+host log line TWICE — rule 12 never-double-attach — so init's bundle-driven
+sweep pins the contrib gem disabled (`SUPERSEDED_INSTRUMENTATION_CONFIG`,
+`lib/foam/otel/init.rb`; proven by `spec/floor_spec.rb`); an FDE who
+deliberately wants the contrib bridge instead passes its instance via
+`additional_instrumentations`, which bypasses the sweep map);
 `all`/`base` (not instrumentations — foam deliberately does NOT depend on the
-`all` meta-gem, which would drag test gems).
+`all` meta-gem, which would drag test gems). With the 17-gem floor, the 27
+when-present gems, and these 4 exclusions, all 48 registry gems have a
+written disposition (rule 6: no silent omissions).
 
 **Known-bad / crash history (rule 6 / R3):** action_pack overrides rack config
 (contrib #88); active_record 0.2.2 failed on Rails 7 (#38); `-que` NoMethodError
@@ -197,6 +222,14 @@ one prepend covers most framework logging in practice;
 https://github.com/open-telemetry/opentelemetry-ruby/discussions/1789 is
 open/unresolved) — a documented gap, narrowed to the non-Logger-backed exotics.
 
+The official contrib `opentelemetry-instrumentation-logger 0.4.0` bridges the
+same `Logger#add` seam and is deliberately NOT bundled — census disposition
+EXCLUDED/superseded (§3): foam's bridge additionally holds the lazy-block
+contract, the OTel-internal-logger identity guard, the re-entrancy flag, the
+logs-slot gate, and the opt-in redaction path, and shipping/sweeping both
+would double-export every host log line (rule 12). init's sweep pins the
+contrib gem disabled even when an FDE bundles it (`spec/floor_spec.rb`).
+
 ---
 
 ## 5. LLM ecosystem (rule 31)
@@ -210,7 +243,16 @@ shims patch the SDKs' public call sites — official `openai`
 (messages.create), `gemini-ai` (generate_content / stream_generate_content;
 Google ships no official Ruby SDK), and `ruby_llm` (Provider#complete, one
 seam covering all its providers) — emitting the same GenAI semconv attribute
-set the fleet's js/python gap-fillers emit. **LLM traces are captured in
+set the fleet's js/python gap-fillers emit. `gen_ai.operation.name` matches
+the fleet per API shape: `chat` on the chat seams, `generate_content` on the
+Gemini shim (aligned in 1.2.1 — Ruby previously emitted `chat` for Gemini, a
+cross-language vocabulary split against the js shim and python's official
+google-genai instrumentor). Agent-loop tool activity is captured on the
+ruby_llm seam too: assistant tool-call turns serialize their `tool_calls`
+(id/name/arguments) and tool-result turns their `tool_call_id` into
+`gen_ai.input.messages`/`gen_ai.output.messages`
+(`spec/llm_ruby_llm_spec.rb` drives a real `chat.with_tool` loop on the
+wire); the Anthropic shim carries tool_use blocks verbatim inside content. **LLM traces are captured in
 full — activity (model, response id, finish reasons, tokens, latency) AND
 prompt/completion content, raw by default** (2026-07-26 ruling: LLM content
 is never gated or dropped; the FDE reactive clamp is the only per-customer

data/lib/foam/otel/init.rb

@@ -470,6 +470,21 @@ module Foam
         "OpenTelemetry::Instrumentation::Mongo" => { db_statement: :include }.freeze,
       }.freeze
 
+      # Contrib gems foam's OWN floor pieces supersede (rule-6 census:
+      # EXCLUDED with reason, RESEARCH.md §3): the contrib
+      # opentelemetry-instrumentation-logger bridges the SAME `Logger#add`
+      # seam foam's LoggerBridge owns (lazy-block contract, both loop
+      # guards, logs-slot gating — logger_bridge.rb). If an FDE bundles it,
+      # the bundle-driven sweep below would install it BESIDE foam's prepend
+      # and every host log line would export twice (rule 12: never
+      # double-attach) — so the sweep pins it disabled. An FDE who
+      # deliberately wants the contrib bridge instead passes its instance
+      # via `additional_instrumentations` (install_additional installs
+      # directly and never reads this map — the explicit choice wins).
+      SUPERSEDED_INSTRUMENTATION_CONFIG = {
+        "OpenTelemetry::Instrumentation::Logger" => { enabled: false }.freeze,
+      }.freeze
+
       # Tier-1/2 activation (rules 5-7): install every bundled official
       # opentelemetry-instrumentation-* gem, presence-checked by the gem
       # itself. The outbound loop guard (rule 24) is wired by pre-installing
@@ -505,7 +520,9 @@ module Foam
 
         configure_loop_guard(config)
         install_additional(additional)
-        OpenTelemetry::Instrumentation.registry.install_all(FLOOR_INSTRUMENTATION_CONFIG)
+        OpenTelemetry::Instrumentation.registry.install_all(
+          FLOOR_INSTRUMENTATION_CONFIG.merge(SUPERSEDED_INSTRUMENTATION_CONFIG)
+        )
         Diagnostics.info("instrumentations installed: #{gems.length} bundled gem(s)")
       rescue StandardError => e
         Diagnostics.warn("instrumentation activation failed: #{e.class}: #{e.message}")

data/lib/foam/otel/llm/gemini_shim.rb

@@ -10,9 +10,14 @@ module Foam
       # stream_generate_content (the gem consumes the SSE stream INSIDE the
       # call and returns the collected events, so both seams yield the full
       # response synchronously). gen_ai.provider.name uses the semconv
-      # well-known value for the Gemini API: "gcp.gemini".
+      # well-known value for the Gemini API: "gcp.gemini", and
+      # gen_ai.operation.name / the span name use "generate_content" — the
+      # semconv well-known operation for this API shape, exactly what the
+      # fleet's js shim and python's official google-genai instrumentor emit
+      # (GenAI-uniformity ruling 2026-07-26; aligned from "chat" in 1.2.1).
       module GeminiShim
         PROVIDER = "gcp.gemini"
+        OPERATION = "generate_content"
         SUPPORTED = Gem::Requirement.new(">= 4.0", "< 5")
 
         class << self
@@ -58,7 +63,7 @@ module Foam
               nil
             end
             LLM.request_attributes(
-              provider: PROVIDER, operation: "chat",
+              provider: PROVIDER, operation: OPERATION,
               model: model,
               input_messages: payload[:contents] || payload["contents"],
               system_instructions: payload[:system_instruction] || payload["system_instruction"]
@@ -90,7 +95,9 @@ module Foam
           def generate_content(payload, server_sent_events: nil, &callback)
             Foam::Otel::LLM.observe(
               provider: GeminiShim::PROVIDER,
-              request: Foam::Otel::LLM.safe_request(provider: GeminiShim::PROVIDER) do
+              operation: GeminiShim::OPERATION,
+              request: Foam::Otel::LLM.safe_request(provider: GeminiShim::PROVIDER,
+                                                    operation: GeminiShim::OPERATION) do
                 GeminiShim.request(self, payload)
               end,
               response_extractor: GeminiShim::RESPONSE
@@ -100,7 +107,9 @@ module Foam
           def stream_generate_content(payload, server_sent_events: nil, &callback)
             Foam::Otel::LLM.observe(
               provider: GeminiShim::PROVIDER,
-              request: Foam::Otel::LLM.safe_request(provider: GeminiShim::PROVIDER) do
+              operation: GeminiShim::OPERATION,
+              request: Foam::Otel::LLM.safe_request(provider: GeminiShim::PROVIDER,
+                                                    operation: GeminiShim::OPERATION) do
                 GeminiShim.request(self, payload)
               end,
               response_extractor: GeminiShim::RESPONSE

data/lib/foam/otel/llm/ruby_llm_shim.rb

@@ -73,15 +73,44 @@ module Foam
             )
           end
 
-          # RubyLLM::Message list → plain [{role:, content:}] for the RAW
+          # RubyLLM::Message list → plain message hashes for the RAW
           # input.messages JSON (Content objects degrade to_s, never dropped).
+          # Agent-loop turns keep their tool activity (rule 8a/8c floor —
+          # tool names/arguments are activity signal): an assistant tool-call
+          # turn carries "tool_calls" (id/name/arguments verbatim) and a
+          # tool-result turn carries its "tool_call_id" linkage.
           def serialize_messages(messages)
             Array(messages).map do |message|
-              {
+              entry = {
                 "role" => (message.respond_to?(:role) ? message.role.to_s : "user"),
                 "content" => content_of(message),
               }
+              tool_calls = serialize_tool_calls(message)
+              entry["tool_calls"] = tool_calls if tool_calls
+              tool_call_id = message.respond_to?(:tool_call_id) ? message.tool_call_id : nil
+              entry["tool_call_id"] = tool_call_id.to_s if tool_call_id && !tool_call_id.to_s.empty?
+              entry
+            end
+          rescue StandardError, SystemStackError
+            nil
+          end
+
+          # RubyLLM::Message#tool_calls is a Hash{id => ToolCall} — each call
+          # rides as {id:, name:, arguments:} verbatim. nil when absent/empty
+          # (the key is omitted, matching the non-agentic message shape).
+          def serialize_tool_calls(message)
+            return nil unless message.respond_to?(:tool_calls)
+
+            calls = message.tool_calls
+            calls = calls.values if calls.is_a?(Hash)
+            serialized = Array(calls).map do |call|
+              {
+                "id" => (call.respond_to?(:id) ? call.id : nil),
+                "name" => (call.respond_to?(:name) ? call.name : nil),
+                "arguments" => (call.respond_to?(:arguments) ? call.arguments : nil),
+              }
             end
+            serialized.empty? ? nil : serialized
           rescue StandardError, SystemStackError
             nil
           end
@@ -96,20 +125,26 @@ module Foam
           end
         end
 
-        # The returned RubyLLM::Message → response attributes.
+        # The returned RubyLLM::Message → response attributes. A tool-call
+        # turn (the assistant asking for a tool run) keeps its tool names +
+        # arguments on the output side too — without them the agent loop's
+        # model turns are invisible activity.
         RESPONSE = lambda do |message|
           next nil unless message.respond_to?(:content)
 
           input_tokens = message.respond_to?(:input_tokens) ? message.input_tokens : nil
           output_tokens = message.respond_to?(:output_tokens) ? message.output_tokens : nil
+          output_message = {
+            "role" => (message.respond_to?(:role) ? message.role.to_s : "assistant"),
+            "content" => RubyLLMShim.content_of(message),
+          }
+          tool_calls = RubyLLMShim.serialize_tool_calls(message)
+          output_message["tool_calls"] = tool_calls if tool_calls
           LLM.response_attributes(
             model: message.respond_to?(:model_id) ? message.model_id : nil,
             input_tokens: input_tokens.is_a?(Numeric) ? input_tokens.to_i : nil,
             output_tokens: output_tokens.is_a?(Numeric) ? output_tokens.to_i : nil,
-            output_messages: [{
-              "role" => (message.respond_to?(:role) ? message.role.to_s : "assistant"),
-              "content" => RubyLLMShim.content_of(message),
-            }]
+            output_messages: [output_message]
           )
         end
 

data/lib/foam/otel/version.rb

@@ -9,6 +9,14 @@ module Foam
     # request-context architecture is gone; the init surface changed.
     # 1.0.1 (#80): HTTP-client instrumentation class resolved for the loop guard.
     # 1.0.2: FOAM_OTEL_TOKEN doc convention (token comment source change); drop-in.
-    VERSION = "1.2.0"
+    # 1.2.1: review-fleet fixes. DELIBERATE WIRE CHANGE (GenAI-uniformity
+    #   ruling): Gemini shim gen_ai.operation.name + span name "chat" →
+    #   "generate_content", matching the js/python Gemini emissions. ruby_llm
+    #   shim now captures agent-loop tool activity (tool_calls id/name/
+    #   arguments + tool_call_id) in gen_ai input/output messages. Contrib
+    #   -logger pinned disabled in the sweep (foam's LoggerBridge supersedes
+    #   it — rule 12 double-bridge guard). Sidekiq + Faraday floor entries
+    #   wire-proven (rule 8b); Sidekiq :link posture documented (GOTCHAS G16).
+    VERSION = "1.2.1"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: foam-otel
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.2.1
 platform: ruby
 authors:
 - Foam