dspy-o11y-langfuse 1.0.1 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b8332abbd0abfcb0cb30aea98cc91d45b827cd5a77d674b89bdb2fbb6fb6da32
-  data.tar.gz: cd0006faf6a82befe4c67ad782fa9fca78088bfcd5d59e089772b6f7f05ef3d2
+  metadata.gz: 681cd7bc2d72f935e3e86ba63e8f7846ff385cf98c485e49ed8c67065eb5e254
+  data.tar.gz: '03792ca1ab58794ab07be57da2ff73c1f522212114b0fec18db8cea674d74e22'
 SHA512:
-  metadata.gz: 76a3716dd937b61d5bc29fe65bc146de17e2b16acf9ad8467f0de2322be26d14bbc30eede060bf34503fa6644ec1f2832dafa0e1e41149fc10b490cc1ab2be3c
-  data.tar.gz: 96c3283338d8ce2678efd17146ff3fafa5f1aba6ac0c65b7662069f580dbef44360cbac6b0f8788d68d910cb1aa9635a782db65dfb2a58d353b403b5df093296
+  metadata.gz: 5ae475d8c8d502806ea6c7edb647dc8378b9a68d35ebe9972717a0aac821a5d769e30147b0796efa7d94ead8e836e6f2169bfa0bb767384a14ad1190fc9bf12f
+  data.tar.gz: 3d9d35c62ead40d58d87ccf4eca2e0a23c811a864ed3e44f0c8ba23bab3c3e50e3406a7888012925c7f74af014ab9299ce33bfa3f6b66549424ca63c6f817413

data/README.md

@@ -3,7 +3,7 @@
 [![Gem Version](https://img.shields.io/gem/v/dspy)](https://rubygems.org/gems/dspy)
 [![Total Downloads](https://img.shields.io/gem/dt/dspy)](https://rubygems.org/gems/dspy)
 [![Build Status](https://img.shields.io/github/actions/workflow/status/vicentereig/dspy.rb/ruby.yml?branch=main&label=build)](https://github.com/vicentereig/dspy.rb/actions/workflows/ruby.yml)
-[![Documentation](https://img.shields.io/badge/docs-vicentereig.github.io%2Fdspy.rb-blue)](https://vicentereig.github.io/dspy.rb/)
+[![Documentation](https://img.shields.io/badge/docs-oss.vicente.services%2Fdspy.rb-blue)](https://oss.vicente.services/dspy.rb/)
 [![Discord](https://img.shields.io/discord/1161519468141355160?label=discord&logo=discord&logoColor=white)](https://discord.gg/zWBhrMqn)
 
 > [!NOTE]
@@ -16,7 +16,7 @@
 
 **Build reliable LLM applications in idiomatic Ruby using composable, type-safe modules.**
 
-DSPy.rb is the Ruby-first surgical port of Stanford's [DSPy framework](https://github.com/stanfordnlp/dspy). It delivers structured LLM programming, prompt engineering, and context engineering in the language we love. Instead of wrestling with brittle prompt strings, you define typed signatures in idiomatic Ruby and compose workflows and agents that actually behave.
+DSPy.rb is the Ruby-first surgical port of Stanford's [DSPy paradigm](https://github.com/stanfordnlp/dspy). It delivers structured LLM programming, prompt engineering, and context engineering in the language we love. Instead of wrestling with brittle prompt strings, you define typed signatures in idiomatic Ruby and compose workflows and agents that actually behave.
 
 **Prompts are just functions.** Traditional prompting is like writing code with string concatenation: it works until it doesn't. DSPy.rb brings you the programming approach pioneered by [dspy.ai](https://dspy.ai/): define modular signatures and let the framework deal with the messy bits.
 
@@ -102,6 +102,7 @@ DSPy.rb ships multiple gems from this monorepo so you can opt into features with
 | `dspy-openai` | Packages the OpenAI/OpenRouter/Ollama adapters plus the official SDK guardrails. Install whenever you call `openai/*`, `openrouter/*`, or `ollama/*`. [Adapter README](https://github.com/vicentereig/dspy.rb/blob/main/lib/dspy/openai/README.md) | **Stable** (v1.0.0) |
 | `dspy-anthropic` | Claude adapters, streaming, and structured-output helpers behind the official `anthropic` SDK. [Adapter README](https://github.com/vicentereig/dspy.rb/blob/main/lib/dspy/anthropic/README.md) | **Stable** (v1.0.0) |
 | `dspy-gemini` | Gemini adapters with multimodal + tool-call support via `gemini-ai`. [Adapter README](https://github.com/vicentereig/dspy.rb/blob/main/lib/dspy/gemini/README.md) | **Stable** (v1.0.0) |
+| `dspy-ruby_llm` | Unified access to 12+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Ollama, DeepSeek, etc.) via [RubyLLM](https://rubyllm.com). [Adapter README](https://github.com/vicentereig/dspy.rb/blob/main/lib/dspy/ruby_llm/README.md) | **Stable** (v0.1.0) |
 | `dspy-code_act` | Think-Code-Observe agents that synthesize and execute Ruby safely. (Add the gem or set `DSPY_WITH_CODE_ACT=1` before requiring `dspy/code_act`.) | **Stable** (v1.0.0) |
 | `dspy-datasets` | Dataset helpers plus Parquet/Polars tooling for richer evaluation corpora. (Toggle via `DSPY_WITH_DATASETS`.) | **Stable** (v1.0.0) |
 | `dspy-evals` | High-throughput evaluation harness with metrics, callbacks, and regression fixtures. (Toggle via `DSPY_WITH_EVALS`.) | **Stable** (v1.0.0) |
@@ -247,13 +248,24 @@ DSPy.rb has gone from experimental to production-ready in three fast releases.
 
 ## Documentation
 
-📖 **[Complete Documentation Website](https://vicentereig.github.io/dspy.rb/)**
+📖 **[Complete Documentation Website](https://oss.vicente.services/dspy.rb/)**
 
 ### LLM-Friendly Documentation
 
 For LLMs and AI assistants working with DSPy.rb:
-- **[llms.txt](https://vicentereig.github.io/dspy.rb/llms.txt)** - Concise reference optimized for LLMs
-- **[llms-full.txt](https://vicentereig.github.io/dspy.rb/llms-full.txt)** - Comprehensive API documentation
+- **[llms.txt](https://oss.vicente.services/dspy.rb/llms.txt)** - Concise reference optimized for LLMs
+- **[llms-full.txt](https://oss.vicente.services/dspy.rb/llms-full.txt)** - Comprehensive API documentation
+
+### Claude Skill
+
+A [Claude Skill](https://github.com/vicentereig/dspy-rb-skill) is available to help you build DSPy.rb applications with Claude Code or claude.ai.
+
+**Claude Code:**
+```bash
+git clone https://github.com/vicentereig/dspy-rb-skill ~/.claude/skills/dspy-rb
+```
+
+**Claude.ai (Pro/Max):** Download the [skill as a ZIP](https://github.com/vicentereig/dspy-rb-skill/archive/refs/heads/main.zip) and upload via Settings > Skills.
 
 ### Getting Started
 - **[Installation & Setup](docs/src/getting-started/installation.md)** - Detailed installation and configuration

data/lib/dspy/o11y/langfuse/scores_exporter.rb

@@ -0,0 +1,222 @@
+# frozen_string_literal: true
+
+require 'sorbet-runtime'
+require 'net/http'
+require 'json'
+require 'base64'
+
+module DSPy
+  class Observability
+    module Adapters
+      module Langfuse
+        # Async exporter for sending scores to Langfuse REST API
+        # Uses a background thread to avoid blocking the main application
+        class ScoresExporter
+          extend T::Sig
+
+          DEFAULT_HOST = 'https://cloud.langfuse.com'
+          SCORES_ENDPOINT = '/api/public/scores'
+          DEFAULT_MAX_RETRIES = 3
+          DEFAULT_TIMEOUT = 10
+
+          attr_reader :host
+
+          sig do
+            params(
+              public_key: String,
+              secret_key: String,
+              host: String,
+              max_retries: Integer,
+              timeout: Integer
+            ).void
+          end
+          def initialize(
+            public_key:,
+            secret_key:,
+            host: DEFAULT_HOST,
+            max_retries: DEFAULT_MAX_RETRIES,
+            timeout: DEFAULT_TIMEOUT
+          )
+            @public_key = public_key
+            @secret_key = secret_key
+            @host = host.chomp('/')
+            @max_retries = max_retries
+            @timeout = timeout
+            @queue = Thread::Queue.new
+            @running = false
+            @worker_thread = nil
+            @subscription_id = nil
+            @mutex = Mutex.new
+          end
+
+          # Factory method that creates, starts, and subscribes to events
+          sig do
+            params(
+              public_key: String,
+              secret_key: String,
+              host: String,
+              max_retries: Integer,
+              timeout: Integer
+            ).returns(ScoresExporter)
+          end
+          def self.configure(
+            public_key:,
+            secret_key:,
+            host: DEFAULT_HOST,
+            max_retries: DEFAULT_MAX_RETRIES,
+            timeout: DEFAULT_TIMEOUT
+          )
+            exporter = new(
+              public_key: public_key,
+              secret_key: secret_key,
+              host: host,
+              max_retries: max_retries,
+              timeout: timeout
+            )
+            exporter.start
+            exporter.subscribe_to_events
+            exporter
+          end
+
+          sig { void }
+          def start
+            @mutex.synchronize do
+              return if @running
+
+              @running = true
+              @worker_thread = Thread.new { process_queue }
+            end
+          end
+
+          sig { returns(T::Boolean) }
+          def running?
+            @mutex.synchronize { @running }
+          end
+
+          sig { params(score_event: DSPy::Scores::ScoreEvent).void }
+          def export(score_event)
+            return unless running?
+
+            @queue.push(score_event)
+          end
+
+          sig { returns(Integer) }
+          def queue_size
+            @queue.size
+          end
+
+          sig { void }
+          def subscribe_to_events
+            @subscription_id = DSPy.events.subscribe('score.create') do |_name, attrs|
+              # Reconstruct ScoreEvent from event attributes
+              score_event = DSPy::Scores::ScoreEvent.new(
+                id: attrs[:score_id],
+                name: attrs[:score_name],
+                value: attrs[:score_value],
+                data_type: DSPy::Scores::DataType.deserialize(attrs[:score_data_type]),
+                comment: attrs[:score_comment],
+                trace_id: attrs[:trace_id],
+                observation_id: attrs[:observation_id]
+              )
+              export(score_event)
+            end
+          end
+
+          sig { params(timeout: Integer).void }
+          def shutdown(timeout: 5)
+            @mutex.synchronize do
+              return unless @running
+
+              @running = false
+
+              # Unsubscribe from events
+              DSPy.events.unsubscribe(@subscription_id) if @subscription_id
+              @subscription_id = nil
+
+              # Signal worker to stop
+              @queue.push(:stop)
+            end
+
+            # Wait for worker thread to finish
+            @worker_thread&.join(timeout)
+          end
+
+          private
+
+          sig { void }
+          def process_queue
+            while running? || !@queue.empty?
+              item = @queue.pop
+
+              break if item == :stop
+
+              begin
+                send_with_retry(item)
+              rescue StandardError => e
+                DSPy.log('scores.export_error', error: e.message, score_name: item.name)
+              end
+            end
+          end
+
+          sig { params(score_event: DSPy::Scores::ScoreEvent).void }
+          def send_with_retry(score_event)
+            retries = 0
+
+            begin
+              send_to_langfuse(score_event)
+            rescue StandardError => e
+              retries += 1
+              if retries <= @max_retries
+                sleep(exponential_backoff(retries))
+                retry
+              else
+                raise e
+              end
+            end
+          end
+
+          sig { params(attempt: Integer).returns(Float) }
+          def exponential_backoff(attempt)
+            # 0.1s, 0.2s, 0.4s, 0.8s... with jitter
+            base_delay = 0.1 * (2 ** (attempt - 1))
+            base_delay + rand * 0.1
+          end
+
+          sig { params(score_event: DSPy::Scores::ScoreEvent).void }
+          def send_to_langfuse(score_event)
+            uri = URI("#{@host}#{SCORES_ENDPOINT}")
+            payload = build_payload(score_event)
+
+            http = Net::HTTP.new(uri.host, uri.port)
+            http.use_ssl = uri.scheme == 'https'
+            http.open_timeout = @timeout
+            http.read_timeout = @timeout
+
+            request = Net::HTTP::Post.new(uri.path)
+            request['Content-Type'] = 'application/json'
+            request['Authorization'] = "Basic #{auth_token}"
+            request.body = JSON.generate(payload)
+
+            response = http.request(request)
+
+            unless response.is_a?(Net::HTTPSuccess)
+              raise "Langfuse API error: #{response.code} - #{response.body}"
+            end
+
+            DSPy.log('scores.exported', score_name: score_event.name, score_id: score_event.id)
+          end
+
+          sig { params(score_event: DSPy::Scores::ScoreEvent).returns(T::Hash[Symbol, T.untyped]) }
+          def build_payload(score_event)
+            score_event.to_langfuse_payload
+          end
+
+          sig { returns(String) }
+          def auth_token
+            Base64.strict_encode64("#{@public_key}:#{@secret_key}")
+          end
+        end
+      end
+    end
+  end
+end

data/lib/dspy/o11y/langfuse/version.rb

@@ -3,7 +3,7 @@
 module DSPy
   module O11y
     module Langfuse
-      VERSION = '1.0.1'
+      VERSION = '1.1.0'
     end
   end
 end

data/lib/dspy/o11y/langfuse.rb

@@ -132,3 +132,6 @@ module DSPy
 end
 
 DSPy::Observability::Adapters::Langfuse.register!
+
+# Load scores exporter for Langfuse
+require_relative 'langfuse/scores_exporter'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dspy-o11y-langfuse
 version: !ruby/object:Gem::Version
-  version: 1.0.1
+  version: 1.1.0
 platform: ruby
 authors:
 - Vicente Reig Rincón de Arellano
@@ -62,6 +62,7 @@ files:
 - LICENSE
 - README.md
 - lib/dspy/o11y/langfuse.rb
+- lib/dspy/o11y/langfuse/scores_exporter.rb
 - lib/dspy/o11y/langfuse/version.rb
 homepage: https://github.com/vicentereig/dspy.rb
 licenses: