langfuse-rb 0.6.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e320f34169f77bea79aa259867b62aa7164d77a693e864cb1d03cb86d0d188a2
-  data.tar.gz: 59a40715dbb46604f6287b46b80c34e1524f7279fed261b7795d781854df987f
+  metadata.gz: ba92036fbe7b63b8355a113e44ff3f62cfcb11817ae3b3c1c444756af55ebf84
+  data.tar.gz: 44097d4d441ad6d2d8780a95cb836d0368ed0b772f16036e301ece7e018df0a1
 SHA512:
-  metadata.gz: d18c3cb807c759c20f72a4259ccfb0502f1800b671aadcf14ea24f88e0be227ed200677f1cbabef22e467c06f2f1ce5f1cf0a0d1a379e14fe29327fce5597ec0
-  data.tar.gz: ff132aea8911b044a43de1a9abe1e54f3a680069b3b480f2e8109953600600b80abe4b923a13d3672f22ff1fe5c4ab3aef93246f2a9946d1c4f593495228ac0f
+  metadata.gz: 6e8880c58683dc7ff719f0c966515841e9a3e3df024f88175297090949a6befaddf23528b24a65d3825fa40e33118348087c0dbe5312543c16bfef65856eaf12
+  data.tar.gz: a82f8be469125e99355c5b6a1bd747a1f2e395dd6e0eb098f8319a0976419dfd60087c6252e26c2570db15994ed084d3156069019541c3b32acc7f1827c92792

data/CHANGELOG.md

@@ -7,6 +7,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.8.0] - 2026-04-24
+
+### Added
+- Probabilistic trace sampling with score parity (#60)
+- Dataset run lifecycle methods: `get_dataset_run`, `list_dataset_runs`, `delete_dataset_run` (#62)
+
+### Fixed
+- Tracing is now isolated-by-default with lazy setup and smart export filtering (#77)
+
+### Documentation
+- Align docs with implementation (#78)
+
+## [0.7.0] - 2026-04-14
+
+### Added
+- Custom/deterministic trace ID support (#74)
+
+### Fixed
+- Bump faraday, json, and addressable to patch CVEs (#75)
+
+### Documentation
+- Align docs with implementation (#70, #76)
+
 ## [0.6.0] - 2026-03-06
 
 ### Added
@@ -69,7 +92,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - OpenTelemetry-based tracing with OTLP export
 - Distributed caching with Rails.cache backend and stampede protection
 - Prompt management (text and chat) with Mustache templating
-- In-memory caching with TTL and LRU eviction
+- In-memory caching with TTL and bounded expiration-ordered eviction
 - Fallback prompt support
 - Global configuration pattern with `Langfuse.configure`
 
@@ -77,7 +100,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Migrated from legacy ingestion API to OTLP endpoint
 - Removed `tracing_enabled` configuration flag (#2)
 
-[Unreleased]: https://github.com/simplepractice/langfuse-rb/compare/v0.6.0...HEAD
+[Unreleased]: https://github.com/simplepractice/langfuse-rb/compare/v0.8.0...HEAD
+[0.8.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.7.0...v0.8.0
+[0.7.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.6.0...v0.7.0
 [0.6.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.5.0...v0.6.0
 [0.5.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.4.0...v0.5.0
 [0.4.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.3.0...v0.4.0

data/README.md

@@ -8,69 +8,49 @@
 
 > Ruby SDK for [Langfuse](https://langfuse.com) - Open-source LLM observability and prompt management.
 
-<br>
-
-### Features
-
-- 🎯 **Prompt Management** - Centralized prompt versioning with Mustache templating
-- 📊 **LLM Tracing** - Zero-boilerplate observability built on OpenTelemetry
-- ⚡ **Performance** - In-memory or Redis-backed caching with stampede protection, both supporting stale-while-revalidate cache strategy
-- 💬 **Chat & Text Prompts** - First-class support for both formats
-- 🔄 **Automatic Retries** - Built-in exponential backoff for resilient API calls
-- 🛡️ **Fallback Support** - Graceful degradation when API unavailable
-- 🚀 **Rails-Friendly** - Global configuration pattern, works with any Ruby project
-
-<br>
-
-### Installation
+## Installation
 
 ```ruby
-# Add to Gemfile & bundle install
-gem 'langfuse-rb'
+gem "langfuse-rb"
 ```
 
-<br>
-
-### Quick Start
-
-> Configure once at startup
+## Quick Start
 
 ```ruby
-# config/initializers/langfuse.rb (Rails)
-# Or at the top of your script
 Langfuse.configure do |config|
-  config.public_key = ENV['LANGFUSE_PUBLIC_KEY']
-  config.secret_key = ENV['LANGFUSE_SECRET_KEY']
-  # Optional: for self-hosted instances
-  config.base_url = ENV.fetch('LANGFUSE_BASE_URL', 'https://cloud.langfuse.com')
-
-  # Optional: Enable stale-while-revalidate for best performance
-  config.cache_backend = :rails  # or :memory
-  config.cache_stale_while_revalidate = true
+  config.public_key = ENV["LANGFUSE_PUBLIC_KEY"]
+  config.secret_key = ENV["LANGFUSE_SECRET_KEY"]
+  config.base_url = ENV.fetch("LANGFUSE_BASE_URL", "https://cloud.langfuse.com")
+
+  # Optional: sample traces and trace-linked scores deterministically
+  config.sample_rate = 1.0
 end
+
+message = Langfuse.client.compile_prompt(
+  "greeting",
+  variables: { name: "Alice" }
+)
 ```
 
-> Fetch and use a prompt
+Langfuse tracing is isolated by default. `Langfuse.configure` stores configuration only; it does not replace `OpenTelemetry.tracer_provider`.
 
-```ruby
-prompt = Langfuse.client.get_prompt("greeting")
-message = prompt.compile(name: "Alice")
-# => "Hello Alice!"
-```
+`sample_rate` is applied to traces and trace-linked scores. Rebuild the client with `Langfuse.reset!` before expecting runtime sampling changes to take effect.
 
-> Trace an LLM call
+## Trace an LLM Call
 
 ```ruby
 Langfuse.observe("chat-completion", as_type: :generation) do |gen|
+  gen.model = "gpt-4.1-mini"
+  gen.input = [{ role: "user", content: "Hello!" }]
+
   response = openai_client.chat(
     parameters: {
-      model: "gpt-4",
+      model: "gpt-4.1-mini",
       messages: [{ role: "user", content: "Hello!" }]
     }
   )
 
   gen.update(
-    model: "gpt-4",
     output: response.dig("choices", 0, "message", "content"),
     usage_details: {
       prompt_tokens: response.dig("usage", "prompt_tokens"),
@@ -80,39 +60,14 @@ Langfuse.observe("chat-completion", as_type: :generation) do |gen|
 end
 ```
 
-> [!IMPORTANT]
-> For complete reference see [docs](./docs/) section.
-
-<br>
-
-### Requirements
-
-- Ruby >= 3.2.0
-- No Rails dependency (works with any Ruby project)
-
-<br>
-
-### Contributing
-
-We welcome contributions! Please:
-
-1. Check existing [issues](https://github.com/simplepractice/langfuse-rb/issues)
-2. Open an issue to discuss your idea
-3. Fork the repo and create a feature branch
-4. Write tests (maintain >95% coverage)
-5. Ensure `bundle exec rspec` and `bundle exec rubocop` pass
-6. Submit a pull request
-
-> [!TIP]
-> See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.
-
-<br>
-
-### Support
+## Start Here
 
-- **[GitHub Issues](https://github.com/simplepractice/langfuse-rb/issues)** - Bug reports and feature requests
-- **[Langfuse Documentation](https://langfuse.com/docs)** - Platform documentation
-- **[API Reference](https://api.reference.langfuse.com)** - REST API reference
+- [Documentation Hub](docs/README.md)
+- [Getting Started](docs/GETTING_STARTED.md)
+- [Prompts](docs/PROMPTS.md)
+- [Tracing](docs/TRACING.md)
+- [Scoring](docs/SCORING.md)
+- [Rails Patterns](docs/RAILS.md)
 
 ## License
 

data/lib/langfuse/api_client.rb

@@ -260,6 +260,61 @@ module Langfuse
       end
     end
 
+    # Fetch a dataset run by dataset and run name
+    #
+    # @param dataset_name [String] Dataset name (required)
+    # @param run_name [String] Run name (required)
+    # @return [Hash] The dataset run data
+    # @raise [NotFoundError] if the dataset run is not found
+    # @raise [UnauthorizedError] if authentication fails
+    # @raise [ApiError] for other API errors
+    def get_dataset_run(dataset_name:, run_name:)
+      with_faraday_error_handling do
+        response = connection.get(dataset_run_path(dataset_name: dataset_name, run_name: run_name))
+        handle_response(response)
+      end
+    end
+
+    # List dataset runs in a dataset
+    #
+    # @param dataset_name [String] Dataset name (required)
+    # @param page [Integer, nil] Optional page number for pagination
+    # @param limit [Integer, nil] Optional limit per page
+    # @return [Array<Hash>] Array of dataset run hashes
+    # @raise [UnauthorizedError] if authentication fails
+    # @raise [ApiError] for other API errors
+    def list_dataset_runs(dataset_name:, page: nil, limit: nil)
+      result = list_dataset_runs_paginated(dataset_name: dataset_name, page: page, limit: limit)
+      result["data"] || []
+    end
+
+    # Full paginated response including "meta" for internal pagination use
+    #
+    # @api private
+    # @return [Hash] Full response hash with "data" array and "meta" pagination info
+    def list_dataset_runs_paginated(dataset_name:, page: nil, limit: nil)
+      with_faraday_error_handling do
+        response = connection.get(dataset_runs_path(dataset_name), build_dataset_runs_params(page: page, limit: limit))
+        handle_response(response)
+      end
+    end
+
+    # Delete a dataset run by name
+    #
+    # @param dataset_name [String] Dataset name (required)
+    # @param run_name [String] Run name (required)
+    # @return [Hash, nil] Response body, or nil for 204 responses
+    # @raise [NotFoundError] if the dataset run is not found
+    # @raise [UnauthorizedError] if authentication fails
+    # @raise [ApiError] for other API errors
+    # @note 404 responses raise NotFoundError to preserve strict delete semantics
+    def delete_dataset_run(dataset_name:, run_name:)
+      with_faraday_error_handling do
+        response = connection.delete(dataset_run_path(dataset_name: dataset_name, run_name: run_name))
+        response.status == 204 ? nil : handle_response(response)
+      end
+    end
+
     # Fetch projects accessible with the current API keys
     #
     # @return [Hash] The parsed response body containing project data
@@ -604,6 +659,23 @@ module Langfuse
       }.compact
     end
 
+    # Build params for list_dataset_runs
+    def build_dataset_runs_params(page:, limit:)
+      { page: page, limit: limit }.compact
+    end
+
+    # Build endpoint path for dataset runs
+    def dataset_runs_path(dataset_name)
+      encoded_name = URI.encode_uri_component(dataset_name)
+      "/api/public/datasets/#{encoded_name}/runs"
+    end
+
+    # Build endpoint path for a specific dataset run
+    def dataset_run_path(dataset_name:, run_name:)
+      encoded_run_name = URI.encode_uri_component(run_name)
+      "#{dataset_runs_path(dataset_name)}/#{encoded_run_name}"
+    end
+
     # Build query params for list_traces, mapping snake_case to camelCase
     # rubocop:disable Metrics/ParameterLists
     def build_traces_params(page:, limit:, user_id:, name:, session_id:,

data/lib/langfuse/cache_warmer.rb

@@ -85,7 +85,7 @@ module Langfuse
     # @example Warm with a different default label
     #   results = warmer.warm_all(default_label: "staging")
     #
-    # @example Warm without any label (latest versions)
+    # @example Warm without any label (API-determined selection)
     #   results = warmer.warm_all(default_label: nil)
     #
     # @example With specific versions for some prompts

data/lib/langfuse/client.rb

@@ -588,6 +588,51 @@ module Langfuse
       )
     end
 
+    # Fetch a dataset run by dataset and run name
+    #
+    # @param dataset_name [String] Dataset name (required)
+    # @param run_name [String] Run name (required)
+    # @return [Hash] The dataset run data, including linked run items
+    # @raise [NotFoundError] if the dataset run is not found
+    # @raise [UnauthorizedError] if authentication fails
+    # @raise [ApiError] for other API errors
+    def get_dataset_run(dataset_name:, run_name:)
+      api_client.get_dataset_run(dataset_name: dataset_name, run_name: run_name)
+    end
+
+    # List dataset runs for a dataset
+    #
+    # When page is nil (default), auto-paginates to fetch all runs.
+    # When page is provided, returns only that single page.
+    #
+    # @param dataset_name [String] Dataset name (required)
+    # @param page [Integer, nil] Optional page number for pagination
+    # @param limit [Integer, nil] Optional limit per page
+    # @return [Array<Hash>] Array of dataset run hashes
+    # @raise [UnauthorizedError] if authentication fails
+    # @raise [ApiError] for other API errors
+    def list_dataset_runs(dataset_name:, page: nil, limit: nil)
+      if page
+        fetch_dataset_runs_page(dataset_name: dataset_name, page: page, limit: limit)
+      else
+        fetch_all_dataset_runs(dataset_name: dataset_name, limit: limit)
+      end
+    end
+
+    # Delete a dataset run by name
+    #
+    # @param dataset_name [String] Dataset name (required)
+    # @param run_name [String] Run name (required)
+    # @return [nil]
+    # @raise [NotFoundError] if the dataset run is not found
+    # @raise [UnauthorizedError] if authentication fails
+    # @raise [ApiError] for other API errors
+    # @note 404 responses raise NotFoundError to preserve strict delete semantics
+    def delete_dataset_run(dataset_name:, run_name:)
+      api_client.delete_dataset_run(dataset_name: dataset_name, run_name: run_name)
+      nil
+    end
+
     # Run an experiment against local data or a named dataset
     #
     # @param name [String] Experiment/run name (required)
@@ -656,17 +701,35 @@ module Langfuse
     end
 
     def fetch_all_dataset_items(limit:, **filters)
-      per_page = limit || DATASET_ITEMS_PAGE_SIZE
-      first_result = api_client.list_dataset_items_paginated(page: 1, limit: per_page, **filters)
-      items = first_result["data"] || []
+      fetch_all_paginated_data(limit: limit) do |page:, per_page:|
+        api_client.list_dataset_items_paginated(page: page, limit: per_page, **filters)
+      end
+    end
+
+    def fetch_dataset_runs_page(dataset_name:, page:, limit:)
+      api_client.list_dataset_runs(dataset_name: dataset_name, page: page, limit: limit)
+    end
+
+    def fetch_all_dataset_runs(dataset_name:, limit:)
+      fetch_all_paginated_data(limit: limit) do |page:, per_page:|
+        api_client.list_dataset_runs_paginated(dataset_name: dataset_name, page: page, limit: per_page)
+      end
+    end
+
+    # Keep dataset collection pagination semantics in one place so edge cases
+    # like missing or zero totalPages stay consistent across APIs.
+    def fetch_all_paginated_data(limit:)
+      page_size = limit || DATASET_ITEMS_PAGE_SIZE
+      first_result = yield(page: 1, per_page: page_size)
+      records = first_result["data"] || []
       total_pages = first_result.dig("meta", "totalPages") || 1
 
-      (2..total_pages).each do |pg|
-        result = api_client.list_dataset_items_paginated(page: pg, limit: per_page, **filters)
-        items.concat(result["data"] || [])
+      (2..total_pages).each do |page|
+        result = yield(page: page, per_page: page_size)
+        records.concat(result["data"] || [])
       end
 
-      items
+      records
     end
 
     def resolve_experiment_items(data, dataset_name)

data/lib/langfuse/config.rb

@@ -18,6 +18,7 @@ module Langfuse
   #     c.secret_key = "sk_..."
   #   end
   #
+  # rubocop:disable Metrics/ClassLength
   class Config
     # @return [String, nil] Langfuse public API key
     attr_accessor :public_key
@@ -74,6 +75,12 @@ module Langfuse
     # @return [String, nil] Default release identifier applied to new traces/observations
     attr_accessor :release
 
+    # @return [Float] Trace sampling rate from 0.0 to 1.0
+    attr_reader :sample_rate
+
+    # @return [#call, nil] Callback that decides whether an ended span should export to Langfuse.
+    attr_accessor :should_export_span
+
     # @return [#call, nil] Mask callable applied to input, output, and metadata before serialization.
     #   Receives `data:` keyword argument. nil disables masking.
     attr_accessor :mask
@@ -114,6 +121,9 @@ module Langfuse
     # @return [Symbol] Default ActiveJob queue name
     DEFAULT_JOB_QUEUE = :default
 
+    # @return [Float] Default trace sampling rate (sample all traces)
+    DEFAULT_SAMPLE_RATE = 1.0
+
     # @return [Integer] Number of seconds representing indefinite cache duration (~1000 years)
     INDEFINITE_SECONDS = 1000 * 365 * 24 * 60 * 60
 
@@ -136,7 +146,6 @@ module Langfuse
     # @yield [config] Optional block for configuration
     # @yieldparam config [Config] The config instance
     # @return [Config] a new Config instance
-    # rubocop:disable Metrics/AbcSize
     def initialize
       @public_key = ENV.fetch("LANGFUSE_PUBLIC_KEY", nil)
       @secret_key = ENV.fetch("LANGFUSE_SECRET_KEY", nil)
@@ -153,14 +162,11 @@ module Langfuse
       @batch_size = DEFAULT_BATCH_SIZE
       @flush_interval = DEFAULT_FLUSH_INTERVAL
       @job_queue = DEFAULT_JOB_QUEUE
-      @environment = env_value("LANGFUSE_TRACING_ENVIRONMENT")
-      @release = env_value("LANGFUSE_RELEASE") || detect_release_from_ci_env
-      @mask = nil
+      initialize_tracing_defaults
       @logger = default_logger
 
       yield(self) if block_given?
     end
-    # rubocop:enable Metrics/AbcSize
 
     # Validate the configuration
     #
@@ -183,7 +189,8 @@ module Langfuse
       validate_swr_config!
 
       validate_cache_backend!
-
+      validate_sample_rate!
+      validate_should_export_span!
       validate_mask!
     end
     # rubocop:enable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
@@ -205,6 +212,15 @@ module Langfuse
       cache_stale_ttl == :indefinite ? INDEFINITE_SECONDS : cache_stale_ttl
     end
 
+    # Set trace sampling rate.
+    #
+    # @param value [Numeric, String] Sampling rate from 0.0 to 1.0
+    # @raise [ConfigurationError] if value is non-numeric or outside 0.0..1.0
+    # @return [Float]
+    def sample_rate=(value)
+      @sample_rate = coerce_sample_rate(value)
+    end
+
     private
 
     def default_logger
@@ -215,6 +231,14 @@ module Langfuse
       end
     end
 
+    def initialize_tracing_defaults
+      @environment = env_value("LANGFUSE_TRACING_ENVIRONMENT")
+      @release = env_value("LANGFUSE_RELEASE") || detect_release_from_ci_env
+      self.sample_rate = env_value("LANGFUSE_SAMPLE_RATE") || DEFAULT_SAMPLE_RATE
+      @should_export_span = nil
+      @mask = nil
+    end
+
     def validate_cache_backend!
       valid_backends = %i[memory rails]
       return if valid_backends.include?(cache_backend)
@@ -255,12 +279,24 @@ module Langfuse
       raise ConfigurationError, "cache_refresh_threads must be positive"
     end
 
+    def validate_sample_rate!
+      return if sample_rate.is_a?(Numeric) && sample_rate.between?(0.0, 1.0)
+
+      raise ConfigurationError, "sample_rate must be between 0.0 and 1.0"
+    end
+
     def validate_mask!
       return if mask.nil? || mask.respond_to?(:call)
 
       raise ConfigurationError, "mask must respond to #call"
     end
 
+    def validate_should_export_span!
+      return if should_export_span.nil? || should_export_span.respond_to?(:call)
+
+      raise ConfigurationError, "should_export_span must respond to #call"
+    end
+
     def detect_release_from_ci_env
       COMMON_RELEASE_ENV_KEYS.each do |key|
         value = env_value(key)
@@ -276,5 +312,22 @@ module Langfuse
 
       value
     end
+
+    def coerce_sample_rate(value)
+      numeric_value = if value.is_a?(Numeric)
+                        value.to_f
+                      elsif value.is_a?(String)
+                        Float(value)
+                      else
+                        raise ConfigurationError, "sample_rate must be numeric"
+                      end
+
+      return numeric_value if numeric_value.between?(0.0, 1.0)
+
+      raise ConfigurationError, "sample_rate must be between 0.0 and 1.0"
+    rescue ArgumentError, TypeError
+      raise ConfigurationError, "sample_rate must be numeric"
+    end
   end
+  # rubocop:enable Metrics/ClassLength
 end

data/lib/langfuse/observations.rb

@@ -133,8 +133,7 @@ module Langfuse
     # @yield [observation] Optional block that receives the observation object
     # @return [BaseObservation, Object] The child observation (or block return value if block given)
     def start_observation(name, attrs = {}, as_type: :span, &block)
-      # Call module-level factory with parent context
-      # Skip validation to allow unknown types to fall back to Span
+      # Skip validation so unknown types fall back to Span in the factory.
       child = Langfuse.start_observation(
         name,
         attrs,
@@ -142,24 +141,9 @@ module Langfuse
         parent_span_context: @otel_span.context,
         skip_validation: true
       )
+      return child unless block
 
-      if block
-        # Block-based API: auto-ends when block completes
-        # Set context and execute block
-        current_context = OpenTelemetry::Context.current
-        result = OpenTelemetry::Context.with_current(
-          OpenTelemetry::Trace.context_with_span(child.otel_span, parent_context: current_context)
-        ) do
-          block.call(child)
-        end
-        # Only end if not already ended (events auto-end in start_observation)
-        child.end unless as_type.to_s == OBSERVATION_TYPES[:event]
-        result
-      else
-        # Stateful API - return observation
-        # Events already auto-ended in start_observation
-        child
-      end
+      child.send(:run_in_context, &block)
     end
 
     # Sets observation-level input attributes.
@@ -261,6 +245,27 @@ module Langfuse
         prompt
       end
     end
+
+    private
+
+    # Runs the block with this observation as the active OTel span,
+    # then ends the span in ensure (events excluded — they auto-end).
+    # @api private
+    def run_in_context
+      parent_ctx = OpenTelemetry::Context.current
+      span_ctx = OpenTelemetry::Trace.context_with_span(@otel_span, parent_context: parent_ctx)
+      OpenTelemetry::Context.with_current(span_ctx) { yield self }
+    ensure
+      safe_end
+    end
+
+    # Ends the span, swallowing errors so ensure never masks a block exception.
+    # @api private
+    def safe_end
+      self.end unless @type == OBSERVATION_TYPES[:event]
+    rescue StandardError
+      nil
+    end
   end
 
   # General-purpose observation for tracking operations, functions, or logical units of work.