langfuse-ruby 0.1.7 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +21 -10
- data/.github/workflows/release.yml +2 -2
- data/.mise.toml +8 -0
- data/.rubocop.yml +2 -2
- data/CHANGELOG.md +78 -10
- data/CLAUDE.md +54 -16
- data/Gemfile +1 -0
- data/Gemfile.lock +15 -5
- data/Makefile +6 -5
- data/README.md +338 -89
- data/Rakefile +0 -6
- data/docs/FINAL_SUMMARY.md +12 -185
- data/docs/PUBLISH_GUIDE.md +36 -272
- data/docs/README.md +12 -22
- data/docs/RELEASE_CHECKLIST.md +20 -145
- data/docs/V4.md +159 -0
- data/examples/basic_tracing.rb +11 -10
- data/examples/simplified_usage.rb +7 -6
- data/examples/v4_otel_tracing.rb +70 -0
- data/langfuse-ruby.gemspec +2 -0
- data/lib/langfuse/client.rb +757 -276
- data/lib/langfuse/event.rb +1 -17
- data/lib/langfuse/generation.rb +71 -124
- data/lib/langfuse/null_objects.rb +4 -0
- data/lib/langfuse/otel_exporter.rb +145 -63
- data/lib/langfuse/partial_updates.rb +30 -0
- data/lib/langfuse/prompt.rb +9 -83
- data/lib/langfuse/prompt_cache.rb +65 -0
- data/lib/langfuse/span.rb +29 -153
- data/lib/langfuse/span_wrappers.rb +32 -0
- data/lib/langfuse/template_compiler.rb +56 -0
- data/lib/langfuse/trace.rb +38 -164
- data/lib/langfuse/utils.rb +54 -20
- data/lib/langfuse/version.rb +1 -1
- data/lib/langfuse.rb +61 -56
- data/scripts/release.sh +12 -12
- metadata +37 -2
data/README.md
CHANGED
|
@@ -2,20 +2,27 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://badge.fury.io/rb/langfuse-ruby) [](https://github.com/ai-firstly/langfuse-ruby/actions/workflows/ci.yml) [](https://www.ruby-lang.org/) [](LICENSE)
|
|
4
4
|
|
|
5
|
-
Ruby SDK for [Langfuse](https://langfuse.com)
|
|
5
|
+
Ruby SDK for [Langfuse](https://langfuse.com) — the open-source LLM engineering platform. Tracing, prompt management, and evaluation for Ruby LLM apps, with first-class support for the **Langfuse v4** observations-first data model.
|
|
6
|
+
|
|
7
|
+
**New projects should use `ingestion_mode: :otel` (Langfuse v4).** That path sends traces over OTLP/HTTP (`/api/public/otel/v1/traces`) with `x-langfuse-ingestion-version: 4`, so data shows up in real time and observation-level evaluators, cost, and the Observations API v2 work as designed. The tracing API (`Langfuse.trace`, `#generation`, `#span`, `#agent`, …) is unchanged; only the transport and ID format differ.
|
|
8
|
+
|
|
9
|
+
On Langfuse Cloud, `POST /api/public/ingestion` stops accepting everything except scores on **16 November 2026**. Self-hosted v4 should use OTEL as well. See the [Langfuse v4 guide](docs/V4.md) and the official [custom-ingestion migration](https://langfuse.com/integrations/native/opentelemetry/migration-to-v4).
|
|
6
10
|
|
|
7
11
|
## Features
|
|
8
12
|
|
|
9
|
-
-
|
|
10
|
-
-
|
|
11
|
-
-
|
|
12
|
-
-
|
|
13
|
-
- 🚀 **Async Processing**: Background
|
|
14
|
-
- 🔒 **
|
|
15
|
-
- 🎯 **Framework Integration**: Easy integration with popular Ruby frameworks
|
|
13
|
+
- ⚡ **Langfuse v4 / OpenTelemetry**: OTLP ingestion, W3C hex IDs, `usage_details` / `cost_details`, observation types (`agent`, `tool`, `retriever`, …)
|
|
14
|
+
- 🔍 **Tracing**: Traces, spans, generations, events, and typed observations
|
|
15
|
+
- 📝 **Prompt Management**: Versioned prompts with a bounded cache and stale-on-outage reads
|
|
16
|
+
- 📊 **Evaluation**: Built-in evaluators and scores (trace, observation, session, dataset run)
|
|
17
|
+
- 🚀 **Async Processing**: Background batching, queue bounds, fork-safe flush, `at_exit` shutdown
|
|
18
|
+
- 🔒 **Resilience**: Typed errors, Retry-After + jittered backoff, null-object degradation for `Langfuse.trace`
|
|
16
19
|
|
|
17
20
|
## Installation
|
|
18
21
|
|
|
22
|
+
This gem requires Ruby >= 3.1 and is tested against Ruby 3.1–4.0. For
|
|
23
|
+
development, Ruby version is managed with [mise](https://mise.jdx.dev) (defaults
|
|
24
|
+
to the latest stable Ruby via `.mise.toml`).
|
|
25
|
+
|
|
19
26
|
Add this line to your application's Gemfile:
|
|
20
27
|
|
|
21
28
|
```ruby
|
|
@@ -34,87 +41,133 @@ Or install it yourself as:
|
|
|
34
41
|
$ gem install langfuse-ruby
|
|
35
42
|
```
|
|
36
43
|
|
|
37
|
-
|
|
44
|
+
### Development setup with mise
|
|
38
45
|
|
|
39
|
-
|
|
46
|
+
```bash
|
|
47
|
+
# Install mise (if not already installed), then trust the project config and
|
|
48
|
+
# install the pinned Ruby version.
|
|
49
|
+
brew install mise # macOS; see mise docs for other OSes
|
|
50
|
+
mise install # installs Ruby from .mise.toml
|
|
51
|
+
bundle install # install gem dependencies
|
|
52
|
+
bundle exec rake spec # run the test suite
|
|
53
|
+
```
|
|
40
54
|
|
|
41
|
-
|
|
42
|
-
require 'langfuse'
|
|
55
|
+
## Quick Start (Langfuse v4)
|
|
43
56
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
57
|
+
### 1. Configure the client for v4
|
|
58
|
+
|
|
59
|
+
The SDK default host is **US Cloud** (`https://us.cloud.langfuse.com`). Override
|
|
60
|
+
`host` / `LANGFUSE_HOST` / `LANGFUSE_BASE_URL` for EU, Japan, HIPAA, or
|
|
61
|
+
self-hosted. `ingestion_mode` still defaults to `:legacy` for compatibility;
|
|
62
|
+
set it to `:otel` for v4.
|
|
63
|
+
|
|
64
|
+
```ruby
|
|
65
|
+
require "langfuse"
|
|
50
66
|
|
|
51
|
-
# Or configure globally
|
|
52
67
|
Langfuse.configure do |config|
|
|
53
|
-
config.public_key = "
|
|
54
|
-
config.secret_key = "
|
|
55
|
-
config.host = "https://cloud.langfuse.com"
|
|
68
|
+
config.public_key = ENV.fetch("LANGFUSE_PUBLIC_KEY")
|
|
69
|
+
config.secret_key = ENV.fetch("LANGFUSE_SECRET_KEY")
|
|
70
|
+
config.host = ENV["LANGFUSE_HOST"] || ENV["LANGFUSE_BASE_URL"] || "https://us.cloud.langfuse.com"
|
|
71
|
+
config.ingestion_mode = :otel # Langfuse v4 — required for real-time OTEL ingestion
|
|
56
72
|
end
|
|
57
73
|
|
|
58
|
-
|
|
74
|
+
# Equivalent:
|
|
75
|
+
# Langfuse.new(..., ingestion_mode: :otel)
|
|
76
|
+
# LANGFUSE_INGESTION_MODE=otel
|
|
59
77
|
```
|
|
60
78
|
|
|
61
|
-
|
|
79
|
+
| Region | `host` |
|
|
80
|
+
| --- | --- |
|
|
81
|
+
| US (SDK default) | `https://us.cloud.langfuse.com` |
|
|
82
|
+
| EU | `https://cloud.langfuse.com` |
|
|
83
|
+
| Japan | `https://jp.cloud.langfuse.com` |
|
|
84
|
+
| HIPAA | `https://hipaa.cloud.langfuse.com` |
|
|
85
|
+
| Self-hosted | your Langfuse origin (no trailing path) |
|
|
62
86
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
87
|
+
No extra gems are required. The SDK maps traces/spans/generations to OTLP JSON
|
|
88
|
+
and sets `x-langfuse-ingestion-version: 4` on the OTEL connection.
|
|
89
|
+
|
|
90
|
+
### 2. Trace an LLM call (recommended API)
|
|
91
|
+
|
|
92
|
+
v4 is **observations-first**: a trace is the set of observations that share a
|
|
93
|
+
`trace_id`. Put the overall request/response on the trace (root span) **and**
|
|
94
|
+
on the generation/span that actually produced them. Prefer `usage_details` /
|
|
95
|
+
`cost_details` over the legacy `usage` hash — v4 uses `usage_details` for cost.
|
|
67
96
|
|
|
68
97
|
```ruby
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
98
|
+
Langfuse.trace("chat-completion", user_id: "user-123", session_id: "sess-456",
|
|
99
|
+
input: { message: "Hello, world!" }) do |trace|
|
|
100
|
+
generation = trace.generation(
|
|
101
|
+
name: "openai-completion",
|
|
102
|
+
model: "gpt-4o",
|
|
103
|
+
input: [{ role: "user", content: "Hello, world!" }],
|
|
104
|
+
model_parameters: { temperature: 0.7, max_tokens: 100 }
|
|
105
|
+
)
|
|
75
106
|
|
|
76
|
-
#
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
107
|
+
response = call_llm(...) # your code
|
|
108
|
+
|
|
109
|
+
generation.end(
|
|
110
|
+
output: response.content,
|
|
111
|
+
usage_details: { input: 12, output: 18, total: 30 }, # v4 cost model
|
|
112
|
+
cost_details: { input: 0.0001, output: 0.0006, total: 0.0007 }
|
|
113
|
+
)
|
|
114
|
+
generation.score(name: "faithfulness", value: 0.9)
|
|
82
115
|
|
|
83
|
-
|
|
84
|
-
#
|
|
116
|
+
trace.update(output: response.content)
|
|
117
|
+
end # flush is automatic in the block form
|
|
85
118
|
```
|
|
86
119
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
120
|
+
Scores always go through `/api/public/ingestion` (`score-create`), even in
|
|
121
|
+
`:otel` mode. Trace/observation IDs are normalized to W3C hex so they attach to
|
|
122
|
+
the OTEL-ingested spans. If an OTEL export fails mid-batch, both the OTEL
|
|
123
|
+
events and that batch's scores are re-queued.
|
|
90
124
|
|
|
91
|
-
###
|
|
125
|
+
### 3. Nested work with v4 observation types
|
|
92
126
|
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
name: "chat-completion",
|
|
97
|
-
user_id: "user123",
|
|
98
|
-
session_id: "session456",
|
|
99
|
-
environment: "production"
|
|
100
|
-
)
|
|
127
|
+
Typed observations (`agent`, `tool`, `chain`, `retriever`, `embedding`,
|
|
128
|
+
`evaluator`, `guardrail`) are spans with `langfuse.observation.type` set. They
|
|
129
|
+
filter and evaluate correctly in the v4 UI.
|
|
101
130
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
name: "
|
|
105
|
-
model: "gpt-3.5-turbo",
|
|
106
|
-
input: [{ role: "user", content: "Hello, world!" }],
|
|
107
|
-
model_parameters: { temperature: 0.7, max_tokens: 100 }
|
|
108
|
-
)
|
|
131
|
+
```ruby
|
|
132
|
+
Langfuse.trace("document-qa", user_id: "user-456", input: { query: "What is Ruby?" }) do |trace|
|
|
133
|
+
agent = trace.agent(name: "qa-agent", input: { query: "What is Ruby?" })
|
|
109
134
|
|
|
110
|
-
|
|
135
|
+
retrieval = agent.retriever(name: "vector-search", input: { query: "What is Ruby?", top_k: 5 })
|
|
136
|
+
retrieval.end(output: { documents: ["Ruby is a programming language..."] })
|
|
111
137
|
|
|
112
|
-
|
|
138
|
+
gen = agent.generation(
|
|
139
|
+
name: "openai-completion",
|
|
140
|
+
model: "gpt-4o",
|
|
141
|
+
input: [{ role: "user", content: "What is Ruby?" }],
|
|
142
|
+
prompt: Langfuse.get_prompt("qa-system") # optional prompt link
|
|
143
|
+
)
|
|
144
|
+
gen.end(
|
|
145
|
+
output: "Ruby is a dynamic programming language.",
|
|
146
|
+
usage_details: { input: 50, output: 20, total: 70 }
|
|
147
|
+
)
|
|
113
148
|
|
|
114
|
-
|
|
115
|
-
|
|
149
|
+
agent.end(output: { answer: "Ruby is a dynamic programming language." })
|
|
150
|
+
trace.update(output: { answer: "Ruby is a dynamic programming language." })
|
|
151
|
+
end
|
|
116
152
|
```
|
|
117
153
|
|
|
154
|
+
### What `:otel` changes (and what it does not)
|
|
155
|
+
|
|
156
|
+
| | `:otel` (v4, recommended) | `:legacy` |
|
|
157
|
+
| --- | --- | --- |
|
|
158
|
+
| Transport | OTLP/HTTP JSON `/api/public/otel/v1/traces` | batched POST `/api/public/ingestion` |
|
|
159
|
+
| Header | `x-langfuse-ingestion-version: 4` | none |
|
|
160
|
+
| IDs | W3C 32-char trace / 16-char span hex | UUIDs |
|
|
161
|
+
| Create + update | collapsed into **one** span (v4 is append-only) | sent as separate events |
|
|
162
|
+
| Usage | `usage` normalized into `langfuse.observation.usage_details` + `gen_ai.usage.*` | legacy `usage` object |
|
|
163
|
+
| Scores | still the ingestion API, IDs coerced to hex | ingestion API |
|
|
164
|
+
| Tracing API | identical | identical |
|
|
165
|
+
|
|
166
|
+
Do not dual-send the same IDs through both modes into one project. Switch with
|
|
167
|
+
`ingestion_mode:` / `LANGFUSE_INGESTION_MODE` (values are downcased; a typo
|
|
168
|
+
falls back to `:legacy` with a warning). Full attribute mapping, evaluator
|
|
169
|
+
notes, and a cutover checklist: [docs/V4.md](docs/V4.md).
|
|
170
|
+
|
|
118
171
|
## Simplified Usage (Recommended)
|
|
119
172
|
|
|
120
173
|
For most use cases, you can use the simplified class-level API with automatic flush:
|
|
@@ -124,10 +177,11 @@ For most use cases, you can use the simplified class-level API with automatic fl
|
|
|
124
177
|
```ruby
|
|
125
178
|
require 'langfuse'
|
|
126
179
|
|
|
127
|
-
# Configure once
|
|
180
|
+
# Configure once — v4 / OTEL is the recommended ingestion path
|
|
128
181
|
Langfuse.configure do |config|
|
|
129
|
-
config.public_key = ENV[
|
|
130
|
-
config.secret_key = ENV[
|
|
182
|
+
config.public_key = ENV["LANGFUSE_PUBLIC_KEY"]
|
|
183
|
+
config.secret_key = ENV["LANGFUSE_SECRET_KEY"]
|
|
184
|
+
config.ingestion_mode = :otel
|
|
131
185
|
end
|
|
132
186
|
|
|
133
187
|
# Use block-based tracing - flush happens automatically!
|
|
@@ -143,7 +197,10 @@ Langfuse.trace("my-trace", user_id: "user-1", input: { message: "Hello" }) do |t
|
|
|
143
197
|
response = call_openai(...)
|
|
144
198
|
|
|
145
199
|
# Record the response
|
|
146
|
-
generation.end(
|
|
200
|
+
generation.end(
|
|
201
|
+
output: response.content,
|
|
202
|
+
usage_details: { input: 10, output: 15, total: 25 } # or usage: response.usage
|
|
203
|
+
)
|
|
147
204
|
trace.update(output: response.content)
|
|
148
205
|
end # Automatic flush here!
|
|
149
206
|
```
|
|
@@ -176,20 +233,20 @@ end
|
|
|
176
233
|
### Other Class Methods
|
|
177
234
|
|
|
178
235
|
```ruby
|
|
179
|
-
# Get the thread-safe singleton client
|
|
236
|
+
# Get the process-wide, thread-safe singleton client
|
|
180
237
|
client = Langfuse.client
|
|
181
238
|
|
|
182
239
|
# Manual flush (when not using block-based tracing)
|
|
183
240
|
Langfuse.flush
|
|
184
241
|
|
|
185
|
-
# Shutdown the client
|
|
242
|
+
# Shutdown the client (idempotent; also runs via at_exit when shutdown_on_exit is true)
|
|
186
243
|
Langfuse.shutdown
|
|
187
244
|
|
|
188
|
-
# Reset the singleton (useful for testing)
|
|
245
|
+
# Reset the singleton (useful for testing; prefer Langfuse.new for isolated clients)
|
|
189
246
|
Langfuse.reset!
|
|
190
247
|
```
|
|
191
248
|
|
|
192
|
-
|
|
249
|
+
## Nested Spans
|
|
193
250
|
|
|
194
251
|
```ruby
|
|
195
252
|
trace = client.trace(name: "document-qa")
|
|
@@ -233,7 +290,46 @@ llm_gen = answer_span.generation(
|
|
|
233
290
|
]
|
|
234
291
|
)
|
|
235
292
|
|
|
236
|
-
|
|
293
|
+
llm_gen.end(
|
|
294
|
+
output: { answer: "Machine learning is a subset of AI..." },
|
|
295
|
+
usage_details: { input: 50, output: 30, total: 80 }
|
|
296
|
+
)
|
|
297
|
+
answer_span.end(output: { answer: "Machine learning is a subset of AI..." })
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
## Observation Types (v4)
|
|
301
|
+
|
|
302
|
+
Langfuse v4 queries **observations** directly. Use a specific type so traces
|
|
303
|
+
filter and evaluate correctly. Helpers exist on `Client`, `Trace`, `Span`, and
|
|
304
|
+
`Generation`. `as_type:` on `#span` does the same thing; an explicit `as_type:`
|
|
305
|
+
passed to a typed helper cannot override that helper's type.
|
|
306
|
+
|
|
307
|
+
| Helper | `langfuse.observation.type` | Use for |
|
|
308
|
+
| --- | --- | --- |
|
|
309
|
+
| `#span` | `span` | generic timed work |
|
|
310
|
+
| `#generation` | `generation` | LLM calls (model, tokens, cost, prompt link) |
|
|
311
|
+
| `#event` | `event` | point-in-time logs |
|
|
312
|
+
| `#agent` | `agent` | orchestration / tool-calling loops |
|
|
313
|
+
| `#tool` | `tool` | a single function or API call |
|
|
314
|
+
| `#chain` | `chain` | stitching retrieval → generation, etc. |
|
|
315
|
+
| `#retriever` | `retriever` | vector store / DB lookups |
|
|
316
|
+
| `#embedding` | `embedding` | embedding model calls (`model` / `usage` go into metadata) |
|
|
317
|
+
| `#evaluator` / `#evaluator_obs` | `evaluator` | scoring functions (`Client#evaluator` is an alias of `#evaluator_obs`) |
|
|
318
|
+
| `#guardrail` | `guardrail` | safety / moderation |
|
|
319
|
+
|
|
320
|
+
```ruby
|
|
321
|
+
trace = client.trace(name: "support-agent", user_id: "u1")
|
|
322
|
+
|
|
323
|
+
agent = trace.agent(name: "planner", input: { question: "Reset my password" })
|
|
324
|
+
tool = agent.tool(name: "lookup-user", input: { email: "a@example.com" })
|
|
325
|
+
tool.end(output: { user_id: "u1" })
|
|
326
|
+
|
|
327
|
+
guard = agent.guardrail(name: "content-filter", input: { text: "Reset my password" })
|
|
328
|
+
guard.end(output: { blocked: false })
|
|
329
|
+
|
|
330
|
+
gen = agent.generation(name: "reply", model: "gpt-4o", input: [...])
|
|
331
|
+
gen.end(output: "I can help with that.", usage_details: { input: 40, output: 12, total: 52 })
|
|
332
|
+
agent.end(output: { reply: "I can help with that." })
|
|
237
333
|
```
|
|
238
334
|
|
|
239
335
|
## Events
|
|
@@ -289,6 +385,11 @@ puts compiled
|
|
|
289
385
|
|
|
290
386
|
> **Note**: Prompt names containing special characters (like `/`, spaces, `?`, etc.) are automatically URL-encoded. You don't need to manually encode them.
|
|
291
387
|
|
|
388
|
+
Fetched prompts are cached per client for `cache_ttl_seconds` (default 60). The
|
|
389
|
+
cache is bounded (200 entries), TTLs use a monotonic clock, and if a refetch
|
|
390
|
+
fails while an expired entry exists, the stale copy is served with a warning —
|
|
391
|
+
a Langfuse outage does not break prompt resolution for prompts seen before.
|
|
392
|
+
|
|
292
393
|
### Create Prompts
|
|
293
394
|
|
|
294
395
|
```ruby
|
|
@@ -406,6 +507,108 @@ generation.score(
|
|
|
406
507
|
|
|
407
508
|
## Advanced Usage
|
|
408
509
|
|
|
510
|
+
### Tracing Environment, Sampling and Masking
|
|
511
|
+
|
|
512
|
+
```ruby
|
|
513
|
+
# Tag all events with a tracing environment (also via LANGFUSE_TRACING_ENVIRONMENT)
|
|
514
|
+
client = Langfuse.new(
|
|
515
|
+
public_key: "pk-lf-...",
|
|
516
|
+
secret_key: "sk-lf-...",
|
|
517
|
+
environment: "production"
|
|
518
|
+
)
|
|
519
|
+
|
|
520
|
+
# Sample a fraction of traces deterministically (also via LANGFUSE_SAMPLE_RATE)
|
|
521
|
+
# All events of a trace share the same keep/drop decision.
|
|
522
|
+
sampled_client = Langfuse.new(
|
|
523
|
+
public_key: "pk-lf-...",
|
|
524
|
+
secret_key: "sk-lf-...",
|
|
525
|
+
sample_rate: 0.1
|
|
526
|
+
)
|
|
527
|
+
|
|
528
|
+
# Mask sensitive fields before sending (applied to input/output/metadata)
|
|
529
|
+
masked_client = Langfuse.new(
|
|
530
|
+
public_key: "pk-lf-...",
|
|
531
|
+
secret_key: "sk-lf-...",
|
|
532
|
+
mask: ->(value) { value.to_s.gsub(/\b\d{16}\b/, "***CARD***") }
|
|
533
|
+
)
|
|
534
|
+
```
|
|
535
|
+
|
|
536
|
+
### Batch flushing
|
|
537
|
+
|
|
538
|
+
Events are flushed in the background every `flush_interval` seconds, or as soon
|
|
539
|
+
as `flush_at` events are queued (default 15, env `LANGFUSE_FLUSH_AT`). Batches
|
|
540
|
+
are automatically split to respect the 3.5 MB ingestion API limit, and a
|
|
541
|
+
process-wide `at_exit` hook flushes pending events on shutdown.
|
|
542
|
+
|
|
543
|
+
The queue is bounded by `max_queue_size` (default 10,000, env
|
|
544
|
+
`LANGFUSE_MAX_QUEUE_SIZE`): while Langfuse is unreachable the oldest events are
|
|
545
|
+
dropped with a warning instead of growing memory without bound, and batches
|
|
546
|
+
that fail with a permanent error (4xx) are dropped rather than retried forever.
|
|
547
|
+
`shutdown` lets the flush thread finish its current send before returning, and
|
|
548
|
+
after a `fork` (e.g. Puma workers) each process recreates its own flush thread.
|
|
549
|
+
|
|
550
|
+
`update` and `end` send only the fields you changed, so ending a generation does
|
|
551
|
+
not re-upload its prompt, input or model parameters.
|
|
552
|
+
|
|
553
|
+
```ruby
|
|
554
|
+
client = Langfuse.new(
|
|
555
|
+
public_key: "pk-lf-...",
|
|
556
|
+
secret_key: "sk-lf-...",
|
|
557
|
+
flush_at: 50, # flush after 50 events
|
|
558
|
+
flush_interval: 10, # or every 10 seconds
|
|
559
|
+
max_queue_size: 20_000, # drop the oldest events beyond this many queued
|
|
560
|
+
shutdown_on_exit: true # flush on process exit (default)
|
|
561
|
+
)
|
|
562
|
+
```
|
|
563
|
+
|
|
564
|
+
### Scores with full fields
|
|
565
|
+
|
|
566
|
+
Scores can target a trace, an observation, a session, or a dataset run, and
|
|
567
|
+
carry metadata, a config reference, and an annotation queue link:
|
|
568
|
+
|
|
569
|
+
```ruby
|
|
570
|
+
# Trace-level score
|
|
571
|
+
trace.score(name: "accuracy", value: 0.9, comment: "good")
|
|
572
|
+
|
|
573
|
+
# Observation-level score (trace_id is set automatically on Span/Generation)
|
|
574
|
+
generation.score(name: "faithfulness", value: 0.8, data_type: "NUMERIC")
|
|
575
|
+
|
|
576
|
+
# Session-level score
|
|
577
|
+
client.score(name: "csat", value: 5, session_id: "sess-1", data_type: "NUMERIC")
|
|
578
|
+
|
|
579
|
+
# Dataset-run score with metadata and config link
|
|
580
|
+
client.score(
|
|
581
|
+
name: "hallucination",
|
|
582
|
+
value: 0.2,
|
|
583
|
+
dataset_run_id: "run-1",
|
|
584
|
+
trace_id: "trace-1",
|
|
585
|
+
metadata: { evaluator: "llm-judge" },
|
|
586
|
+
config_id: "cfg-abc",
|
|
587
|
+
data_type: "NUMERIC"
|
|
588
|
+
)
|
|
589
|
+
|
|
590
|
+
# Categorical string value
|
|
591
|
+
client.score(name: "label", value: "good", trace_id: "t1", data_type: "CATEGORICAL")
|
|
592
|
+
```
|
|
593
|
+
|
|
594
|
+
### Generation usage details, cost details and prompt linking (v4)
|
|
595
|
+
|
|
596
|
+
```ruby
|
|
597
|
+
# New v4 usage model (arbitrary keys, e.g. cache tokens)
|
|
598
|
+
gen = trace.generation(
|
|
599
|
+
name: "chat",
|
|
600
|
+
model: "gpt-4o",
|
|
601
|
+
usage_details: { input: 100, output: 50, cache_read: 30 },
|
|
602
|
+
cost_details: { input: 0.001, output: 0.003, total: 0.004 }
|
|
603
|
+
)
|
|
604
|
+
gen.end(output: "response")
|
|
605
|
+
|
|
606
|
+
# Link a generation to a prompt version (accepts a Langfuse::Prompt or a hash)
|
|
607
|
+
prompt = Langfuse.get_prompt("chat-prompt")
|
|
608
|
+
gen = trace.generation(name: "chat", model: "gpt-4o", prompt: prompt)
|
|
609
|
+
# or: prompt: { name: "chat-prompt", version: 3 }
|
|
610
|
+
```
|
|
611
|
+
|
|
409
612
|
### Error Handling
|
|
410
613
|
|
|
411
614
|
```ruby
|
|
@@ -434,15 +637,49 @@ end
|
|
|
434
637
|
client = Langfuse.new(
|
|
435
638
|
public_key: "pk-lf-...",
|
|
436
639
|
secret_key: "sk-lf-...",
|
|
437
|
-
host: "https://
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
640
|
+
host: "https://us.cloud.langfuse.com", # US default; EU: cloud.langfuse.com
|
|
641
|
+
ingestion_mode: :otel, # Langfuse v4 (default remains :legacy)
|
|
642
|
+
debug: true, # Enable debug logging (or LANGFUSE_DEBUG=true)
|
|
643
|
+
timeout: 30, # Request timeout in seconds
|
|
644
|
+
retries: 3, # Number of retry attempts
|
|
645
|
+
flush_interval: 30, # Event flush interval in seconds (default: 5)
|
|
646
|
+
flush_at: 50, # Flush once this many events are queued (default: 15)
|
|
647
|
+
max_queue_size: 10_000, # Drop the oldest events beyond this many queued (default: 10_000)
|
|
648
|
+
auto_flush: true, # Enable automatic flushing (default: true)
|
|
649
|
+
environment: "prod", # Tracing environment (or LANGFUSE_TRACING_ENVIRONMENT)
|
|
650
|
+
sample_rate: 0.5, # Keep 50% of traces deterministically (or LANGFUSE_SAMPLE_RATE)
|
|
651
|
+
mask: ->(v) { v }, # Callable applied to input/output/metadata
|
|
652
|
+
shutdown_on_exit: true, # Flush pending events on process exit (default: true)
|
|
653
|
+
http_adapter: :net_http_persistent # Faraday adapter (default: Faraday's default)
|
|
443
654
|
)
|
|
444
655
|
```
|
|
445
656
|
|
|
657
|
+
### Retries
|
|
658
|
+
|
|
659
|
+
Timeouts, network errors, `429` and `5xx` responses are retried up to `retries`
|
|
660
|
+
times (default 3). A `Retry-After` header is honored (seconds or HTTP date,
|
|
661
|
+
capped at 10 s); otherwise the delay grows exponentially from 0.5 s with ±50%
|
|
662
|
+
jitter, so many processes do not retry in lockstep. Client errors such as `401`
|
|
663
|
+
and `422` are not retried, since repeating them cannot help.
|
|
664
|
+
|
|
665
|
+
### Connection reuse
|
|
666
|
+
|
|
667
|
+
By default every request opens a new TLS connection. To keep connections alive
|
|
668
|
+
between flushes, add a pooling adapter to your `Gemfile` and select it with
|
|
669
|
+
`http_adapter`:
|
|
670
|
+
|
|
671
|
+
```ruby
|
|
672
|
+
# Gemfile
|
|
673
|
+
gem "faraday-net_http_persistent"
|
|
674
|
+
|
|
675
|
+
Langfuse.configure do |config|
|
|
676
|
+
config.http_adapter = :net_http_persistent
|
|
677
|
+
end
|
|
678
|
+
```
|
|
679
|
+
|
|
680
|
+
If the adapter is not available, the client logs a warning and falls back to
|
|
681
|
+
Faraday's default adapter instead of raising.
|
|
682
|
+
|
|
446
683
|
### Environment Variables
|
|
447
684
|
|
|
448
685
|
You can also configure the client using environment variables:
|
|
@@ -450,9 +687,15 @@ You can also configure the client using environment variables:
|
|
|
450
687
|
```bash
|
|
451
688
|
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
|
|
452
689
|
export LANGFUSE_SECRET_KEY="sk-lf-..."
|
|
453
|
-
export LANGFUSE_HOST="https://cloud.langfuse.com"
|
|
690
|
+
export LANGFUSE_HOST="https://us.cloud.langfuse.com" # or LANGFUSE_BASE_URL; EU: https://cloud.langfuse.com
|
|
454
691
|
export LANGFUSE_FLUSH_INTERVAL=5
|
|
692
|
+
export LANGFUSE_FLUSH_AT=15
|
|
693
|
+
export LANGFUSE_MAX_QUEUE_SIZE=10000
|
|
455
694
|
export LANGFUSE_AUTO_FLUSH=true
|
|
695
|
+
export LANGFUSE_TRACING_ENVIRONMENT="production"
|
|
696
|
+
export LANGFUSE_SAMPLE_RATE=0.5
|
|
697
|
+
export LANGFUSE_DEBUG=false
|
|
698
|
+
export LANGFUSE_INGESTION_MODE=otel # v4; use `legacy` only for pre-v4 self-hosted
|
|
456
699
|
```
|
|
457
700
|
|
|
458
701
|
### Automatic Flush Control
|
|
@@ -539,6 +782,7 @@ client.shutdown
|
|
|
539
782
|
Langfuse.configure do |config|
|
|
540
783
|
config.public_key = Rails.application.credentials.langfuse_public_key
|
|
541
784
|
config.secret_key = Rails.application.credentials.langfuse_secret_key
|
|
785
|
+
config.ingestion_mode = :otel
|
|
542
786
|
config.debug = Rails.env.development?
|
|
543
787
|
end
|
|
544
788
|
|
|
@@ -594,18 +838,22 @@ end
|
|
|
594
838
|
|
|
595
839
|
Check out the `examples/` directory for more comprehensive examples:
|
|
596
840
|
|
|
597
|
-
- [
|
|
598
|
-
- [
|
|
599
|
-
- [
|
|
600
|
-
- [
|
|
841
|
+
- [Langfuse v4 / OTEL tracing](examples/v4_otel_tracing.rb) (recommended)
|
|
842
|
+
- [Simplified usage](examples/simplified_usage.rb)
|
|
843
|
+
- [Basic tracing](examples/basic_tracing.rb)
|
|
844
|
+
- [Prompt management](examples/prompt_management.rb)
|
|
845
|
+
- [Events](examples/event_usage.rb)
|
|
846
|
+
- [Auto-flush control](examples/auto_flush_control.rb)
|
|
847
|
+
- [Connection config](examples/connection_config_demo.rb)
|
|
601
848
|
|
|
602
849
|
## Documentation
|
|
603
850
|
|
|
604
|
-
|
|
605
|
-
|
|
851
|
+
- [Langfuse v4 usage](docs/V4.md) — OTEL ingestion, observations-first model, cutover checklist
|
|
852
|
+
- [Documentation index](docs/README.md)
|
|
606
853
|
- [Publishing Guide](docs/PUBLISH_GUIDE.md)
|
|
607
854
|
- [Release Checklist](docs/RELEASE_CHECKLIST.md)
|
|
608
|
-
- [
|
|
855
|
+
- [Official Langfuse docs](https://langfuse.com/docs)
|
|
856
|
+
- [Migrate custom ingestion to v4](https://langfuse.com/integrations/native/opentelemetry/migration-to-v4)
|
|
609
857
|
|
|
610
858
|
## Development
|
|
611
859
|
|
|
@@ -641,4 +889,5 @@ The gem is available as open source under the terms of the [MIT License](https:/
|
|
|
641
889
|
|
|
642
890
|
- [Langfuse Ruby SDK Documentation](https://rubydoc.info/gems/langfuse-ruby)
|
|
643
891
|
- [RubyGems](https://rubygems.org/gems/langfuse-ruby)
|
|
644
|
-
- [Langfuse Documentation](https://langfuse.com/docs)
|
|
892
|
+
- [Langfuse Documentation](https://langfuse.com/docs)
|
|
893
|
+
- [Langfuse v4 / OpenTelemetry](https://langfuse.com/integrations/native/opentelemetry)
|
data/Rakefile
CHANGED
|
@@ -8,12 +8,6 @@ RSpec::Core::RakeTask.new(:spec)
|
|
|
8
8
|
|
|
9
9
|
task default: :spec
|
|
10
10
|
|
|
11
|
-
# Custom release task
|
|
12
|
-
desc 'Release gem to RubyGems'
|
|
13
|
-
task release_gem: [:build] do
|
|
14
|
-
sh "gem push langfuse-ruby-#{Langfuse::VERSION}.gem"
|
|
15
|
-
end
|
|
16
|
-
|
|
17
11
|
# Offline test task
|
|
18
12
|
desc 'Run offline tests'
|
|
19
13
|
task :test_offline do
|