ask-opentelemetry 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 725e08edb09468eeddf352695a9683197d245f385783aa6b527f52bd29243ec3
+  data.tar.gz: 875068867c33e9d45a1e442875dcc3d7e487e345b74fe9f15104ab564813475a
+SHA512:
+  metadata.gz: 2b0d3a3c347fd6301e1f5e8b6c61da18fe78cd925eadd32bd3a060aea02099d84c91a8ccb4bdc64aee8aae7570b63c712e5ffdbc5c83b69acb6ca7970f3c6e06
+  data.tar.gz: 2ff2cceaa569cfc8e70f7d5f03589ffdb3f97c8593d77e11ccd3616d3481cbf493fa3db03ef802ed5bb16675090c536d6fb229adc69669c2b6d1f9658ee43dd7

data/CHANGELOG.md

@@ -0,0 +1,13 @@
+# Changelog
+
+## [0.1.0] - Unreleased
+
+### Added
+- Initial release
+- `Ask::OpenTelemetry.install` — subscribes to all `.ask` events
+- `Subscriber` — maps events to OpenTelemetry spans (llm.chat, llm.tool, llm.embedding, llm.image)
+- Span attributes: llm.provider, llm.model, llm.input_tokens, llm.output_tokens, llm.duration_ms, llm.tool, llm.image.size
+- Metadata forwarding: `Ask::Instrumentation.with_metadata` values set as `llm.metadata.*` attributes
+- Railtie for automatic installation in Rails apps
+- Full test suite with OpenTelemetry span verification
+- README with backend examples (Langfuse, Datadog, Honeycomb)

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Kaka Ruto
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,189 @@
+# ask-opentelemetry
+
+[![Gem Version](https://badge.fury.io/rb/ask-opentelemetry.svg)](https://badge.fury.io/rb/ask-opentelemetry)
+[![CI](https://github.com/ask-rb/ask-opentelemetry/actions/workflows/ci.yml/badge.svg)](https://github.com/ask-rb/ask-opentelemetry/actions/workflows/ci.yml)
+
+OpenTelemetry tracing for the ask-rb ecosystem. Subscribes to
+`ask-instrumentation` events and creates OpenTelemetry spans for chat
+completions, tool calls, embeddings, and image generation.
+
+Works with any OpenTelemetry-compatible backend: Langfuse, Datadog, Honeycomb,
+Jaeger, Arize Phoenix, and more.
+
+## Installation
+
+```ruby
+gem "ask-opentelemetry"
+gem "ask-instrumentation"
+```
+
+## Quick Start
+
+```ruby
+require "ask/open_telemetry"
+
+# Subscribe to all ask events and create OpenTelemetry spans
+Ask::OpenTelemetry.install
+```
+
+### In Rails
+
+The Railtie auto-installs — no manual setup required:
+
+```ruby
+# Gemfile
+gem "ask-opentelemetry"
+gem "ask-instrumentation"
+```
+
+That's it. Spans are created automatically for every LLM operation.
+
+## Spans Created
+
+| Event | Span Name | Key Attributes |
+|---|---|---|
+| `chat.ask` | `llm.chat` | provider, model, input_tokens, output_tokens, duration_ms |
+| `chat.stream.ask` | `llm.chat` | provider, model, input_tokens, output_tokens, duration_ms |
+| `tool.ask` | `llm.tool` | provider, tool, tool_args |
+| `embedding.ask` | `llm.embedding` | provider, model, input_tokens |
+| `image.ask` | `llm.image` | provider, model, image.size, duration_ms |
+
+### Standard Attributes
+
+All spans include:
+
+- `llm.provider` — The LLM provider (e.g., `openai`, `anthropic`)
+- `llm.model` — The model identifier (e.g., `gpt-4`, `claude-3`)
+- `llm.duration_ms` — The duration of the LLM operation in milliseconds
+- `llm.input_tokens` — Input token count (when available)
+- `llm.output_tokens` — Output token count (when available)
+
+### Metadata Attributes
+
+Any context set via `Ask::Instrumentation.with_metadata` is forwarded as
+`llm.metadata.*` attributes:
+
+```ruby
+Ask::Instrumentation.with_metadata(user_id: 42, session_id: "abc") do
+  # The resulting span has:
+  #   llm.metadata.user_id = 42
+  #   llm.metadata.session_id = "abc"
+end
+```
+
+## Backend Examples
+
+### Langfuse
+
+[Langfuse](https://langfuse.com) provides LLM observability (cost tracking,
+evaluations, prompt management) via OpenTelemetry.
+
+**Using Langfuse Cloud:**
+
+```ruby
+require "ask/open_telemetry"
+require "opentelemetry-sdk"
+require "opentelemetry-exporter-otlp"
+
+OpenTelemetry::SDK.configure do |c|
+  c.service_name = "my-app"
+  c.add_span_processor(
+    OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(
+      OpenTelemetry::Exporter::OTLP::Exporter.new(
+        endpoint: "https://cloud.langfuse.com/api/public/otel/v1/traces",
+        headers: {
+          "Authorization" => "Basic #{Base64.strict_encode64("#{LANGFUSE_PUBLIC_KEY}:#{LANGFUSE_SECRET_KEY}")}"
+        }
+      )
+    )
+  )
+end
+
+Ask::OpenTelemetry.install
+```
+
+**Using Langfuse Self-Hosted:**
+
+```ruby
+OpenTelemetry::SDK.configure do |c|
+  c.service_name = "my-app"
+  c.add_span_processor(
+    OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(
+      OpenTelemetry::Exporter::OTLP::Exporter.new(
+        endpoint: "https://your-instance.langfuse.com/api/public/otel/v1/traces",
+        headers: {
+          "Authorization" => "Basic #{Base64.strict_encode64("#{LANGFUSE_PUBLIC_KEY}:#{LANGFUSE_SECRET_KEY}")}"
+        }
+      )
+    )
+  )
+end
+```
+
+### Datadog
+
+```ruby
+require "ask/open_telemetry"
+require "opentelemetry-sdk"
+require "opentelemetry-exporter-otlp"
+
+OpenTelemetry::SDK.configure do |c|
+  c.service_name = "my-app"
+  c.add_span_processor(
+    OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(
+      OpenTelemetry::Exporter::OTLP::Exporter.new(
+        endpoint: "https://otlp.datadoghq.com",
+        headers: { "DD-API-KEY" => ENV["DD_API_KEY"] }
+      )
+    )
+  )
+end
+
+Ask::OpenTelemetry.install
+```
+
+### Honeycomb
+
+```ruby
+require "ask/open_telemetry"
+require "opentelemetry-sdk"
+require "opentelemetry-exporter-otlp"
+
+OpenTelemetry::SDK.configure do |c|
+  c.service_name = "my-app"
+  c.add_span_processor(
+    OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(
+      OpenTelemetry::Exporter::OTLP::Exporter.new(
+        endpoint: "https://api.honeycomb.io:443",
+        headers: { "x-honeycomb-team" => ENV["HONEYCOMB_API_KEY"] }
+      )
+    )
+  )
+end
+
+Ask::OpenTelemetry.install
+```
+
+## API
+
+### `.install`
+
+Subscribe to all `.ask` events and start creating spans. Idempotent — calling
+multiple times only subscribes once.
+
+```ruby
+Ask::OpenTelemetry.install
+```
+
+## Development
+
+```bash
+git clone https://github.com/ask-rb/ask-opentelemetry.git
+cd ask-opentelemetry
+bundle install
+bundle exec rake test
+```
+
+## License
+
+MIT

data/lib/ask/open_telemetry/railtie.rb

@@ -0,0 +1,17 @@
+module Ask
+  module OpenTelemetry
+    # Rails Railtie that auto-installs ask-opentelemetry in any Rails app.
+    #
+    # Automatically calls +Ask::OpenTelemetry.install+ during Rails
+    # initialization so LLM operations are traced without manual setup.
+    #
+    # @example
+    #   # No configuration needed — the railtie auto-installs.
+    #   # Bundler loads the gem, the railtie hooks into Rails init.
+    class Railtie < ::Rails::Railtie
+      initializer "ask.opentelemetry" do
+        Ask::OpenTelemetry.install
+      end
+    end
+  end
+end

data/lib/ask/open_telemetry/subscriber.rb

@@ -0,0 +1,72 @@
+module Ask
+  module OpenTelemetry
+    # Subscribes to Ask::Instrumentation events and creates OpenTelemetry spans.
+    #
+    # Maps each event name to a span name:
+    #   "chat.ask"        → "llm.chat"
+    #   "chat.stream.ask" → "llm.chat"
+    #   "tool.ask"        → "llm.tool"
+    #   "embedding.ask"   → "llm.embedding"
+    #   "image.ask"       → "llm.image"
+    #
+    # Forwards the event payload as span attributes under the +llm.*+ namespace
+    # and merges in any metadata from +Ask::Instrumentation.current_metadata+.
+    class Subscriber
+      # Event name pattern → OpenTelemetry span name.
+      SPAN_NAMES = {
+        "chat.ask"        => "llm.chat",
+        "chat.stream.ask" => "llm.chat",
+        "tool.ask"        => "llm.tool",
+        "embedding.ask"   => "llm.embedding",
+        "image.ask"       => "llm.image"
+      }.freeze
+
+      # Invoked by +ActiveSupport::Notifications+ for each matching event.
+      #
+      # @param event [ActiveSupport::Notifications::Event] The instrumentation event
+      def call(event)
+        span_name = SPAN_NAMES[event.name]
+        return unless span_name
+
+        tracer.in_span(span_name, attributes: attributes_for(event)) do |span|
+          span.set_attribute("llm.duration_ms", (event.duration * 1000).round(2))
+        end
+      end
+
+      private
+
+      def attributes_for(event)
+        payload = event.payload
+
+        attrs = {}
+
+        # Core llm attributes
+        attrs["llm.provider"]     = payload[:provider] if payload[:provider]
+        attrs["llm.model"]        = payload[:model]    if payload[:model]
+
+        # Token tracking (when available)
+        attrs["llm.input_tokens"]  = payload[:input_tokens]  if payload[:input_tokens]
+        attrs["llm.output_tokens"] = payload[:output_tokens] if payload[:output_tokens]
+
+        # Tool-specific
+        attrs["llm.tool"]    = payload[:tool_name] if payload[:tool_name]
+        attrs["llm.tool_args"] = payload[:tool_args] if payload[:tool_args] && payload[:tool_args].is_a?(String)
+
+        # Image-specific
+        attrs["llm.image.size"] = payload[:size] if payload[:size]
+
+        # Forward all metadata from the instrumentation context
+        Ask::Instrumentation.current_metadata.each do |key, value|
+          next if key.to_s.start_with?("llm.") # don't clobber explicit llm attributes
+          attrs["llm.metadata.#{key}"] = value
+        end
+
+        attrs
+      end
+
+      def tracer
+        @tracer ||= ::OpenTelemetry.tracer_provider.tracer("ask-opentelemetry", Ask::OpenTelemetry::VERSION)
+      end
+    end
+  end
+end

data/lib/ask/open_telemetry/version.rb

@@ -0,0 +1,5 @@
+module Ask
+  module OpenTelemetry
+    VERSION = "0.1.0"
+  end
+end

data/lib/ask/open_telemetry.rb

@@ -0,0 +1,34 @@
+require "ask/instrumentation"
+require "opentelemetry-api"
+require_relative "open_telemetry/version"
+
+module Ask
+  # OpenTelemetry tracing for the ask-rb ecosystem.
+  #
+  # Subscribes to +Ask::Instrumentation+ events and creates OpenTelemetry spans
+  # for chat completions, tool calls, embeddings, and image generation.
+  #
+  # == Usage
+  #
+  #   Ask::OpenTelemetry.install
+  #
+  # In a Rails app the railtie installs automatically — no manual call needed.
+  module OpenTelemetry
+    autoload :Subscriber, "ask/open_telemetry/subscriber"
+    autoload :Railtie,    "ask/open_telemetry/railtie"
+
+    class << self
+      # Subscribe to all ask events and start creating spans.
+      #
+      # Once called, every +Ask::Instrumentation+ event will be wrapped in an
+      # OpenTelemetry span. Safe to call multiple times — subsequent calls are
+      # no-ops.
+      def install
+        return if @installed
+
+        Ask::Instrumentation.subscribe(/\.ask$/, Subscriber.new)
+        @installed = true
+      end
+    end
+  end
+end

data/lib/ask-opentelemetry.rb

@@ -0,0 +1 @@
+require_relative "ask/open_telemetry"

metadata

@@ -0,0 +1,122 @@
+--- !ruby/object:Gem::Specification
+name: ask-opentelemetry
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Kaka Ruto
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: ask-instrumentation
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+- !ruby/object:Gem::Dependency
+  name: opentelemetry-api
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: opentelemetry-sdk
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.7'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.7'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.25'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.25'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+description: Adds OpenTelemetry tracing to ask-rb LLM operations. Subscribes to ask-instrumentation
+  events and creates spans for chat completions, embeddings, tool calls, and more.
+email:
+- kaka@myrrlabs.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE
+- README.md
+- lib/ask-opentelemetry.rb
+- lib/ask/open_telemetry.rb
+- lib/ask/open_telemetry/railtie.rb
+- lib/ask/open_telemetry/subscriber.rb
+- lib/ask/open_telemetry/version.rb
+homepage: https://github.com/ask-rb/ask-opentelemetry
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/ask-rb/ask-opentelemetry
+  source_code_uri: https://github.com/ask-rb/ask-opentelemetry
+  changelog_uri: https://github.com/ask-rb/ask-opentelemetry/blob/master/CHANGELOG.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.2'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.3
+specification_version: 4
+summary: OpenTelemetry tracing for the ask-rb ecosystem
+test_files: []