langfuse-rb 0.7.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 2bf0fdf8f1b31f237397c90d8db3fc61c40bc8a69e20111a949aa0bdffc8dd3e
4
- data.tar.gz: b6b83329218d23b3ebd53562a4eb5bb1cd01179f07fdbf5d90b2b1c97fc192ef
3
+ metadata.gz: 1eedf02269727792fdb13a343429d2108556cc4c7df52dea9e2c638a011409a0
4
+ data.tar.gz: 0130f17065fa3e4233090e858a2699105d9be48dce85d6765b0c3ee394c88bff
5
5
  SHA512:
6
- metadata.gz: a0bda13a371bcc93d37d4ae17548f04c65f1cc8fdf4982756eebac468d280f5b3a44d51ecb5ed5406e2ffee204a958988b3423860916471d8eb1a9e728fcf51c
7
- data.tar.gz: a2bebff2e84e94c5fa30b6774c89c35f63ed4d2af214a76348a370ddbd9326ff0c8346fac9b3601e9bf9427bdb56f73a3f309f7eb71d439bd31b70426461ae3d
6
+ metadata.gz: 036b34ce7908114bdad51bf895c5f50159f005bbf53f3256a347dc7a6c813d9c49d18c9f7491dc42c8771d66fb6b509ae1441af6812d575ff45b80fe4e73ce6e
7
+ data.tar.gz: c4cf129fd0260c35f91e75797903d29e16d74b0bf543a89d9f566201a69e08eaf6245fc9d117e5f2f0e3bad5673cfee5cd54e0802d46a53809ab9c5ea825f172
data/CHANGELOG.md CHANGED
@@ -7,6 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.9.0] - 2026-04-28
11
+
12
+ ### Added
13
+ - Expose `type`, `commit_message`, and `resolution_graph` metadata on text and chat prompt clients (#87)
14
+
15
+ ### Fixed
16
+ - Preserve and compile chat prompt message placeholders in parity with Langfuse Python and JS SDKs (#86)
17
+ - Preserve raw prompt compile variables instead of HTML-escaping JSON, XML, and HTML-like values (#85)
18
+ - Suppress prompt name/version attribution on fallback prompt clients so fallback output is not reported as prompt version 0 (#84)
19
+
20
+ ### Documentation
21
+ - Link to upstream Langfuse agent skills and refresh README header image (#81, #83)
22
+
23
+ ## [0.8.0] - 2026-04-24
24
+
25
+ ### Added
26
+ - Probabilistic trace sampling with score parity (#60)
27
+ - Dataset run lifecycle methods: `get_dataset_run`, `list_dataset_runs`, `delete_dataset_run` (#62)
28
+
29
+ ### Fixed
30
+ - Tracing is now isolated-by-default with lazy setup and smart export filtering (#77)
31
+
32
+ ### Documentation
33
+ - Align docs with implementation (#78)
34
+
10
35
  ## [0.7.0] - 2026-04-14
11
36
 
12
37
  ### Added
@@ -88,7 +113,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
113
  - Migrated from legacy ingestion API to OTLP endpoint
89
114
  - Removed `tracing_enabled` configuration flag (#2)
90
115
 
91
- [Unreleased]: https://github.com/simplepractice/langfuse-rb/compare/v0.7.0...HEAD
116
+ [Unreleased]: https://github.com/simplepractice/langfuse-rb/compare/v0.9.0...HEAD
117
+ [0.9.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.8.0...v0.9.0
118
+ [0.8.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.7.0...v0.8.0
92
119
  [0.7.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.6.0...v0.7.0
93
120
  [0.6.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.5.0...v0.6.0
94
121
  [0.5.0]: https://github.com/simplepractice/langfuse-rb/compare/v0.4.0...v0.5.0
data/README.md CHANGED
@@ -1,4 +1,5 @@
1
- ![header](https://camo.githubusercontent.com/26d19b945bc752101b4aca468e07b118a44af07340db79af29f7df95505f2cea/68747470733a2f2f6c616e67667573652e636f6d2f6c616e67667573655f6c6f676f5f77686974652e706e67)
1
+ <img width="2255" height="527" alt="langfuse-wordart" src="https://github.com/user-attachments/assets/59422d0a-6ecb-4e5f-a21c-cae955b5ce75" />
2
+
2
3
 
3
4
  # Langfuse Ruby SDK
4
5
 
@@ -8,69 +9,49 @@
8
9
 
9
10
  > Ruby SDK for [Langfuse](https://langfuse.com) - Open-source LLM observability and prompt management.
10
11
 
11
- <br>
12
-
13
- ### Features
14
-
15
- - 🎯 **Prompt Management** - Centralized prompt versioning with Mustache templating
16
- - 📊 **LLM Tracing** - Zero-boilerplate observability built on OpenTelemetry
17
- - ⚡ **Performance** - In-memory or Redis-backed caching with stampede protection, both supporting stale-while-revalidate cache strategy
18
- - 💬 **Chat & Text Prompts** - First-class support for both formats
19
- - 🔄 **Automatic Retries** - Built-in exponential backoff for resilient API calls
20
- - 🛡️ **Fallback Support** - Graceful degradation when API unavailable
21
- - 🚀 **Rails-Friendly** - Global configuration pattern, works with any Ruby project
22
-
23
- <br>
24
-
25
- ### Installation
12
+ ## Installation
26
13
 
27
14
  ```ruby
28
- # Add to Gemfile & bundle install
29
- gem 'langfuse-rb'
15
+ gem "langfuse-rb"
30
16
  ```
31
17
 
32
- <br>
33
-
34
- ### Quick Start
35
-
36
- > Configure once at startup
18
+ ## Quick Start
37
19
 
38
20
  ```ruby
39
- # config/initializers/langfuse.rb (Rails)
40
- # Or at the top of your script
41
21
  Langfuse.configure do |config|
42
- config.public_key = ENV['LANGFUSE_PUBLIC_KEY']
43
- config.secret_key = ENV['LANGFUSE_SECRET_KEY']
44
- # Optional: for self-hosted instances
45
- config.base_url = ENV.fetch('LANGFUSE_BASE_URL', 'https://cloud.langfuse.com')
46
-
47
- # Optional: Enable stale-while-revalidate for best performance
48
- config.cache_backend = :rails # or :memory
49
- config.cache_stale_while_revalidate = true
22
+ config.public_key = ENV["LANGFUSE_PUBLIC_KEY"]
23
+ config.secret_key = ENV["LANGFUSE_SECRET_KEY"]
24
+ config.base_url = ENV.fetch("LANGFUSE_BASE_URL", "https://cloud.langfuse.com")
25
+
26
+ # Optional: sample traces and trace-linked scores deterministically
27
+ config.sample_rate = 1.0
50
28
  end
29
+
30
+ message = Langfuse.client.compile_prompt(
31
+ "greeting",
32
+ variables: { name: "Alice" }
33
+ )
51
34
  ```
52
35
 
53
- > Fetch and use a prompt
36
+ Langfuse tracing is isolated by default. `Langfuse.configure` stores configuration only; it does not replace `OpenTelemetry.tracer_provider`.
54
37
 
55
- ```ruby
56
- prompt = Langfuse.client.get_prompt("greeting")
57
- message = prompt.compile(name: "Alice")
58
- # => "Hello Alice!"
59
- ```
38
+ `sample_rate` is applied to traces and trace-linked scores. Rebuild the client with `Langfuse.reset!` before expecting runtime sampling changes to take effect.
60
39
 
61
- > Trace an LLM call
40
+ ## Trace an LLM Call
62
41
 
63
42
  ```ruby
64
43
  Langfuse.observe("chat-completion", as_type: :generation) do |gen|
44
+ gen.model = "gpt-4.1-mini"
45
+ gen.input = [{ role: "user", content: "Hello!" }]
46
+
65
47
  response = openai_client.chat(
66
48
  parameters: {
67
- model: "gpt-4",
49
+ model: "gpt-4.1-mini",
68
50
  messages: [{ role: "user", content: "Hello!" }]
69
51
  }
70
52
  )
71
53
 
72
54
  gen.update(
73
- model: "gpt-4",
74
55
  output: response.dig("choices", 0, "message", "content"),
75
56
  usage_details: {
76
57
  prompt_tokens: response.dig("usage", "prompt_tokens"),
@@ -80,39 +61,16 @@ Langfuse.observe("chat-completion", as_type: :generation) do |gen|
80
61
  end
81
62
  ```
82
63
 
83
- > [!IMPORTANT]
84
- > For complete reference see [docs](./docs/) section.
85
-
86
- <br>
87
-
88
- ### Requirements
89
-
90
- - Ruby >= 3.2.0
91
- - No Rails dependency (works with any Ruby project)
92
-
93
- <br>
94
-
95
- ### Contributing
96
-
97
- We welcome contributions! Please:
98
-
99
- 1. Check existing [issues](https://github.com/simplepractice/langfuse-rb/issues)
100
- 2. Open an issue to discuss your idea
101
- 3. Fork the repo and create a feature branch
102
- 4. Write tests (maintain >95% coverage)
103
- 5. Ensure `bundle exec rspec` and `bundle exec rubocop` pass
104
- 6. Submit a pull request
105
-
106
- > [!TIP]
107
- > See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.
108
-
109
- <br>
110
-
111
- ### Support
64
+ ## Start Here
112
65
 
113
- - **[GitHub Issues](https://github.com/simplepractice/langfuse-rb/issues)** - Bug reports and feature requests
114
- - **[Langfuse Documentation](https://langfuse.com/docs)** - Platform documentation
115
- - **[API Reference](https://api.reference.langfuse.com)** - REST API reference
66
+ - [Documentation Hub](docs/README.md)
67
+ - [Getting Started](docs/GETTING_STARTED.md)
68
+ - [Prompts](docs/PROMPTS.md)
69
+ - [Tracing](docs/TRACING.md)
70
+ - [Scoring](docs/SCORING.md)
71
+ - [Rails Patterns](docs/RAILS.md)
72
+ - [Agent Skills](https://github.com/langfuse/skills)
73
+ - [Agent Skill Docs](https://langfuse.com/docs/api-and-data-platform/features/agent-skill)
116
74
 
117
75
  ## License
118
76
 
@@ -260,6 +260,61 @@ module Langfuse
260
260
  end
261
261
  end
262
262
 
263
+ # Fetch a dataset run by dataset and run name
264
+ #
265
+ # @param dataset_name [String] Dataset name (required)
266
+ # @param run_name [String] Run name (required)
267
+ # @return [Hash] The dataset run data
268
+ # @raise [NotFoundError] if the dataset run is not found
269
+ # @raise [UnauthorizedError] if authentication fails
270
+ # @raise [ApiError] for other API errors
271
+ def get_dataset_run(dataset_name:, run_name:)
272
+ with_faraday_error_handling do
273
+ response = connection.get(dataset_run_path(dataset_name: dataset_name, run_name: run_name))
274
+ handle_response(response)
275
+ end
276
+ end
277
+
278
+ # List dataset runs in a dataset
279
+ #
280
+ # @param dataset_name [String] Dataset name (required)
281
+ # @param page [Integer, nil] Optional page number for pagination
282
+ # @param limit [Integer, nil] Optional limit per page
283
+ # @return [Array<Hash>] Array of dataset run hashes
284
+ # @raise [UnauthorizedError] if authentication fails
285
+ # @raise [ApiError] for other API errors
286
+ def list_dataset_runs(dataset_name:, page: nil, limit: nil)
287
+ result = list_dataset_runs_paginated(dataset_name: dataset_name, page: page, limit: limit)
288
+ result["data"] || []
289
+ end
290
+
291
+ # Full paginated response including "meta" for internal pagination use
292
+ #
293
+ # @api private
294
+ # @return [Hash] Full response hash with "data" array and "meta" pagination info
295
+ def list_dataset_runs_paginated(dataset_name:, page: nil, limit: nil)
296
+ with_faraday_error_handling do
297
+ response = connection.get(dataset_runs_path(dataset_name), build_dataset_runs_params(page: page, limit: limit))
298
+ handle_response(response)
299
+ end
300
+ end
301
+
302
+ # Delete a dataset run by name
303
+ #
304
+ # @param dataset_name [String] Dataset name (required)
305
+ # @param run_name [String] Run name (required)
306
+ # @return [Hash, nil] Response body, or nil for 204 responses
307
+ # @raise [NotFoundError] if the dataset run is not found
308
+ # @raise [UnauthorizedError] if authentication fails
309
+ # @raise [ApiError] for other API errors
310
+ # @note 404 responses raise NotFoundError to preserve strict delete semantics
311
+ def delete_dataset_run(dataset_name:, run_name:)
312
+ with_faraday_error_handling do
313
+ response = connection.delete(dataset_run_path(dataset_name: dataset_name, run_name: run_name))
314
+ response.status == 204 ? nil : handle_response(response)
315
+ end
316
+ end
317
+
263
318
  # Fetch projects accessible with the current API keys
264
319
  #
265
320
  # @return [Hash] The parsed response body containing project data
@@ -604,6 +659,23 @@ module Langfuse
604
659
  }.compact
605
660
  end
606
661
 
662
+ # Build params for list_dataset_runs
663
+ def build_dataset_runs_params(page:, limit:)
664
+ { page: page, limit: limit }.compact
665
+ end
666
+
667
+ # Build endpoint path for dataset runs
668
+ def dataset_runs_path(dataset_name)
669
+ encoded_name = URI.encode_uri_component(dataset_name)
670
+ "/api/public/datasets/#{encoded_name}/runs"
671
+ end
672
+
673
+ # Build endpoint path for a specific dataset run
674
+ def dataset_run_path(dataset_name:, run_name:)
675
+ encoded_run_name = URI.encode_uri_component(run_name)
676
+ "#{dataset_runs_path(dataset_name)}/#{encoded_run_name}"
677
+ end
678
+
607
679
  # Build query params for list_traces, mapping snake_case to camelCase
608
680
  # rubocop:disable Metrics/ParameterLists
609
681
  def build_traces_params(page:, limit:, user_id:, name:, session_id:,
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "mustache"
3
+ require_relative "prompt_renderer"
4
4
 
5
5
  module Langfuse
6
6
  # Chat prompt client for compiling chat prompts with variable substitution
@@ -20,6 +20,8 @@ module Langfuse
20
20
  # chat_prompt.labels # => ["production"]
21
21
  #
22
22
  class ChatPromptClient
23
+ PLACEHOLDER_TYPE = "placeholder"
24
+
23
25
  # @return [String] Prompt name
24
26
  attr_reader :name
25
27
 
@@ -35,14 +37,24 @@ module Langfuse
35
37
  # @return [Hash] Prompt configuration
36
38
  attr_reader :config
37
39
 
38
- # @return [Array<Hash>] Array of message hashes with role and content
40
+ # @return [Array<Hash>] Array of message hashes and placeholder entries
39
41
  attr_reader :prompt
40
42
 
43
+ # @return [String, nil] Optional commit message for this prompt version
44
+ attr_reader :commit_message
45
+
46
+ # @return [Hash, nil] Optional dependency resolution graph for composed prompts
47
+ attr_reader :resolution_graph
48
+
49
+ # @return [Boolean] Whether this client uses caller-provided fallback content
50
+ attr_reader :is_fallback
51
+
41
52
  # Initialize a new chat prompt client
42
53
  #
43
54
  # @param prompt_data [Hash] The prompt data from the API
55
+ # @param is_fallback [Boolean] Whether this client wraps caller-provided fallback content
44
56
  # @raise [ArgumentError] if prompt data is invalid
45
- def initialize(prompt_data)
57
+ def initialize(prompt_data, is_fallback: false)
46
58
  validate_prompt_data!(prompt_data)
47
59
 
48
60
  @name = prompt_data["name"]
@@ -51,16 +63,27 @@ module Langfuse
51
63
  @labels = prompt_data["labels"] || []
52
64
  @tags = prompt_data["tags"] || []
53
65
  @config = prompt_data["config"] || {}
66
+ @commit_message = prompt_data["commitMessage"]
67
+ @resolution_graph = prompt_data["resolutionGraph"]
68
+ @is_fallback = is_fallback
54
69
  end
55
70
 
56
- # Compile the chat prompt with variable substitution
71
+ # @return [String] Prompt type ("chat")
72
+ def type
73
+ "chat"
74
+ end
75
+
76
+ # Compile the chat prompt with variable substitution and message placeholders
57
77
  #
58
78
  # Returns an array of message hashes with roles and compiled content.
59
- # Each message in the prompt will have its content compiled with the
60
- # provided variables using Mustache templating.
79
+ # Placeholder entries are resolved from keyword arguments: arrays are
80
+ # expanded, empty arrays are skipped, unresolved placeholders stay in the
81
+ # output, and malformed values raise before invalid messages are sent to
82
+ # an LLM provider.
61
83
  #
62
- # @param kwargs [Hash] Variables to substitute in message templates (as keyword arguments)
63
- # @return [Array<Hash>] Array of compiled messages with :role and :content keys
84
+ # @param kwargs [Hash] Variables and placeholder values to compile
85
+ # @return [Array<Hash>] Array of compiled messages and unresolved placeholders
86
+ # @raise [ArgumentError] if a placeholder value is malformed
64
87
  #
65
88
  # @example
66
89
  # chat_prompt.compile(name: "Alice", topic: "Ruby")
@@ -69,9 +92,18 @@ module Langfuse
69
92
  # # { role: :user, content: "Hello Alice, let's discuss Ruby!" }
70
93
  # # ]
71
94
  def compile(**kwargs)
72
- prompt.map do |message|
73
- compile_message(message, kwargs)
95
+ unresolved = []
96
+ compiled = []
97
+ prompt.each do |message|
98
+ normalized = symbolize_keys(message)
99
+ if normalized[:type].to_s == PLACEHOLDER_TYPE
100
+ append_placeholder(normalized, kwargs, compiled, unresolved)
101
+ else
102
+ compiled << compile_message(normalized, kwargs)
103
+ end
74
104
  end
105
+ warn_unresolved(unresolved)
106
+ compiled
75
107
  end
76
108
 
77
109
  private
@@ -88,19 +120,102 @@ module Langfuse
88
120
  raise ArgumentError, "prompt must be an Array" unless prompt_data["prompt"].is_a?(Array)
89
121
  end
90
122
 
91
- # Compile a single message with variable substitution
123
+ # Compile a single role/content message with variable substitution
92
124
  #
93
- # @param message [Hash] The message with role and content
125
+ # @param normalized [Hash] Symbolized message hash
94
126
  # @param variables [Hash] Variables to substitute
95
127
  # @return [Hash] Compiled message with :role and :content as symbols
96
- def compile_message(message, variables)
97
- content = message["content"] || ""
98
- compiled_content = variables.empty? ? content : Mustache.render(content, variables)
99
-
100
- {
101
- role: normalize_role(message["role"]),
102
- content: compiled_content
103
- }
128
+ def compile_message(normalized, variables)
129
+ normalized.except(:type).merge(
130
+ role: normalize_role(normalized[:role]),
131
+ content: render(normalized[:content] || "", variables)
132
+ )
133
+ end
134
+
135
+ # @api private
136
+ def append_placeholder(message, variables, compiled, unresolved)
137
+ name = message[:name].to_s
138
+ found, value = lookup_placeholder(variables, name)
139
+ return append_unresolved(name, compiled, unresolved) unless found
140
+
141
+ expand_placeholder(name, value, variables, compiled)
142
+ end
143
+
144
+ # @api private
145
+ def append_unresolved(name, compiled, unresolved)
146
+ unresolved << name
147
+ compiled << { type: PLACEHOLDER_TYPE, name: name }
148
+ end
149
+
150
+ # @api private
151
+ def expand_placeholder(name, value, variables, compiled)
152
+ return if value.is_a?(Array) && value.empty?
153
+
154
+ unless value.is_a?(Array)
155
+ raise ArgumentError, "Placeholder '#{name}' must contain an array of chat message hashes, got #{value.class}."
156
+ end
157
+
158
+ value.each { |entry| compiled << placeholder_message(entry, variables, name) }
159
+ end
160
+
161
+ # @api private
162
+ def lookup_placeholder(variables, name)
163
+ return [true, variables[name.to_sym]] if variables.key?(name.to_sym)
164
+ return [true, variables[name]] if variables.key?(name)
165
+
166
+ [false, nil]
167
+ end
168
+
169
+ # @api private
170
+ def placeholder_message(message, variables, name)
171
+ unless message.is_a?(Hash)
172
+ raise ArgumentError,
173
+ "Placeholder '#{name}' must contain an array of chat message hashes with role and content fields."
174
+ end
175
+
176
+ normalized = symbolize_keys(message)
177
+ unless valid_placeholder_message?(normalized)
178
+ raise ArgumentError,
179
+ "Placeholder '#{name}' must contain an array of chat message hashes with role and content fields."
180
+ end
181
+
182
+ normalized.merge(
183
+ role: normalize_role(normalized[:role]),
184
+ content: render(normalized[:content] || "", variables)
185
+ )
186
+ end
187
+
188
+ # @api private
189
+ def render(content, variables)
190
+ variables.empty? ? content : PromptRenderer.render(content, variables)
191
+ end
192
+
193
+ # @api private
194
+ def valid_placeholder_message?(message)
195
+ message.is_a?(Hash) &&
196
+ message.key?(:role) &&
197
+ !message[:role].to_s.empty? &&
198
+ message.key?(:content)
199
+ end
200
+
201
+ # @api private
202
+ def warn_unresolved(names)
203
+ return if names.empty?
204
+
205
+ unresolved_names = names.uniq.sort
206
+ message = "Placeholders #{unresolved_names.inspect} have not been resolved. " \
207
+ "Pass them as keyword arguments to compile()."
208
+ warn_msg(message)
209
+ end
210
+
211
+ # @api private
212
+ def warn_msg(message)
213
+ Langfuse.configuration.logger.warn("Langfuse: #{message}")
214
+ end
215
+
216
+ # @api private
217
+ def symbolize_keys(hash)
218
+ hash.transform_keys(&:to_sym)
104
219
  end
105
220
 
106
221
  # Normalize role to symbol
@@ -588,6 +588,51 @@ module Langfuse
588
588
  )
589
589
  end
590
590
 
591
+ # Fetch a dataset run by dataset and run name
592
+ #
593
+ # @param dataset_name [String] Dataset name (required)
594
+ # @param run_name [String] Run name (required)
595
+ # @return [Hash] The dataset run data, including linked run items
596
+ # @raise [NotFoundError] if the dataset run is not found
597
+ # @raise [UnauthorizedError] if authentication fails
598
+ # @raise [ApiError] for other API errors
599
+ def get_dataset_run(dataset_name:, run_name:)
600
+ api_client.get_dataset_run(dataset_name: dataset_name, run_name: run_name)
601
+ end
602
+
603
+ # List dataset runs for a dataset
604
+ #
605
+ # When page is nil (default), auto-paginates to fetch all runs.
606
+ # When page is provided, returns only that single page.
607
+ #
608
+ # @param dataset_name [String] Dataset name (required)
609
+ # @param page [Integer, nil] Optional page number for pagination
610
+ # @param limit [Integer, nil] Optional limit per page
611
+ # @return [Array<Hash>] Array of dataset run hashes
612
+ # @raise [UnauthorizedError] if authentication fails
613
+ # @raise [ApiError] for other API errors
614
+ def list_dataset_runs(dataset_name:, page: nil, limit: nil)
615
+ if page
616
+ fetch_dataset_runs_page(dataset_name: dataset_name, page: page, limit: limit)
617
+ else
618
+ fetch_all_dataset_runs(dataset_name: dataset_name, limit: limit)
619
+ end
620
+ end
621
+
622
+ # Delete a dataset run by name
623
+ #
624
+ # @param dataset_name [String] Dataset name (required)
625
+ # @param run_name [String] Run name (required)
626
+ # @return [nil]
627
+ # @raise [NotFoundError] if the dataset run is not found
628
+ # @raise [UnauthorizedError] if authentication fails
629
+ # @raise [ApiError] for other API errors
630
+ # @note 404 responses raise NotFoundError to preserve strict delete semantics
631
+ def delete_dataset_run(dataset_name:, run_name:)
632
+ api_client.delete_dataset_run(dataset_name: dataset_name, run_name: run_name)
633
+ nil
634
+ end
635
+
591
636
  # Run an experiment against local data or a named dataset
592
637
  #
593
638
  # @param name [String] Experiment/run name (required)
@@ -656,17 +701,35 @@ module Langfuse
656
701
  end
657
702
 
658
703
  def fetch_all_dataset_items(limit:, **filters)
659
- per_page = limit || DATASET_ITEMS_PAGE_SIZE
660
- first_result = api_client.list_dataset_items_paginated(page: 1, limit: per_page, **filters)
661
- items = first_result["data"] || []
704
+ fetch_all_paginated_data(limit: limit) do |page:, per_page:|
705
+ api_client.list_dataset_items_paginated(page: page, limit: per_page, **filters)
706
+ end
707
+ end
708
+
709
+ def fetch_dataset_runs_page(dataset_name:, page:, limit:)
710
+ api_client.list_dataset_runs(dataset_name: dataset_name, page: page, limit: limit)
711
+ end
712
+
713
+ def fetch_all_dataset_runs(dataset_name:, limit:)
714
+ fetch_all_paginated_data(limit: limit) do |page:, per_page:|
715
+ api_client.list_dataset_runs_paginated(dataset_name: dataset_name, page: page, limit: per_page)
716
+ end
717
+ end
718
+
719
+ # Keep dataset collection pagination semantics in one place so edge cases
720
+ # like missing or zero totalPages stay consistent across APIs.
721
+ def fetch_all_paginated_data(limit:)
722
+ page_size = limit || DATASET_ITEMS_PAGE_SIZE
723
+ first_result = yield(page: 1, per_page: page_size)
724
+ records = first_result["data"] || []
662
725
  total_pages = first_result.dig("meta", "totalPages") || 1
663
726
 
664
- (2..total_pages).each do |pg|
665
- result = api_client.list_dataset_items_paginated(page: pg, limit: per_page, **filters)
666
- items.concat(result["data"] || [])
727
+ (2..total_pages).each do |page|
728
+ result = yield(page: page, per_page: page_size)
729
+ records.concat(result["data"] || [])
667
730
  end
668
731
 
669
- items
732
+ records
670
733
  end
671
734
 
672
735
  def resolve_experiment_items(data, dataset_name)
@@ -760,9 +823,9 @@ module Langfuse
760
823
 
761
824
  case type
762
825
  when :text
763
- TextPromptClient.new(prompt_data)
826
+ TextPromptClient.new(prompt_data, is_fallback: true)
764
827
  when :chat
765
- ChatPromptClient.new(prompt_data)
828
+ ChatPromptClient.new(prompt_data, is_fallback: true)
766
829
  end
767
830
  end
768
831
 
@@ -793,7 +856,8 @@ module Langfuse
793
856
 
794
857
  # Normalize prompt content for API request
795
858
  #
796
- # Converts Ruby symbol keys to string keys for chat messages
859
+ # Converts Ruby symbol keys to string keys for chat messages and preserves
860
+ # Langfuse message placeholder entries.
797
861
  #
798
862
  # @param prompt [String, Array] The prompt content
799
863
  # @param type [Symbol] The prompt type
@@ -803,18 +867,28 @@ module Langfuse
803
867
 
804
868
  # Normalize chat messages to use string keys
805
869
  prompt.map do |message|
806
- # Convert all keys to symbols first, then extract
807
- normalized = message.transform_keys do |k|
808
- k.to_sym
809
- rescue StandardError
810
- k
811
- end
812
- {
813
- "role" => normalized[:role]&.to_s,
814
- "content" => normalized[:content]
815
- }
870
+ normalized = message.transform_keys(&:to_s)
871
+ next placeholder_prompt_content(normalized) if normalized["type"] == ChatPromptClient::PLACEHOLDER_TYPE
872
+
873
+ normalize_chat_message_content(normalized)
816
874
  end
817
875
  end
876
+
877
+ # @api private
878
+ def placeholder_prompt_content(message)
879
+ {
880
+ "type" => ChatPromptClient::PLACEHOLDER_TYPE,
881
+ "name" => message["name"].to_s
882
+ }
883
+ end
884
+
885
+ # @api private
886
+ def normalize_chat_message_content(message)
887
+ message.merge(
888
+ "role" => message["role"]&.to_s,
889
+ "content" => message["content"]
890
+ )
891
+ end
818
892
  end
819
893
  # rubocop:enable Metrics/ClassLength
820
894
  end