riffer 0.33.0 → 0.35.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.release-please-manifest.json +1 -1
- data/CHANGELOG.md +19 -0
- data/docs/03_AGENTS.md +44 -0
- data/docs/10_CONFIGURATION.md +15 -15
- data/docs/16_TRACING.md +32 -4
- data/docs/17_METRICS.md +36 -4
- data/lib/riffer/agent/run.rb +82 -49
- data/lib/riffer/agent.rb +20 -13
- data/lib/riffer/config.rb +23 -24
- data/lib/riffer/guardrails/runner.rb +40 -11
- data/lib/riffer/metrics/instruments.rb +6 -0
- data/lib/riffer/metrics/{null.rb → no_op.rb} +1 -1
- data/lib/riffer/metrics/otel.rb +4 -3
- data/lib/riffer/metrics.rb +7 -6
- data/lib/riffer/providers/amazon_bedrock.rb +7 -1
- data/lib/riffer/providers/anthropic.rb +8 -1
- data/lib/riffer/providers/base.rb +38 -26
- data/lib/riffer/providers/gemini.rb +5 -1
- data/lib/riffer/providers/open_ai.rb +10 -1
- data/lib/riffer/providers/open_router.rb +10 -1
- data/lib/riffer/tools/runtime.rb +30 -20
- data/lib/riffer/tracing/{null.rb → no_op.rb} +3 -3
- data/lib/riffer/tracing/otel.rb +4 -3
- data/lib/riffer/tracing.rb +9 -9
- data/lib/riffer/version.rb +1 -1
- data/sig/generated/riffer/agent/run.rbs +47 -30
- data/sig/generated/riffer/agent.rbs +16 -9
- data/sig/generated/riffer/config.rbs +14 -14
- data/sig/generated/riffer/guardrails/runner.rbs +15 -4
- data/sig/generated/riffer/metrics/instruments.rbs +2 -0
- data/sig/generated/riffer/metrics/{null.rbs → no_op.rbs} +2 -2
- data/sig/generated/riffer/metrics/otel.rbs +4 -3
- data/sig/generated/riffer/metrics.rbs +7 -6
- data/sig/generated/riffer/providers/base.rbs +26 -20
- data/sig/generated/riffer/tools/runtime.rbs +24 -18
- data/sig/generated/riffer/tracing/{null.rbs → no_op.rbs} +5 -5
- data/sig/generated/riffer/tracing/otel.rbs +4 -3
- data/sig/generated/riffer/tracing.rbs +9 -9
- data/sig/manual/riffer/metrics/no_op.rbs +5 -0
- data/sig/manual/riffer/tracing/no_op.rbs +5 -0
- metadata +11 -11
- data/sig/manual/riffer/metrics/null.rbs +0 -5
- data/sig/manual/riffer/tracing/null.rbs +0 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: acfc9a0c002c97b79d515172d71aa419359e27fac0a3809934c30057931e66ef
|
|
4
|
+
data.tar.gz: 322aaebffb66c41824fdd3a3c43d08ae6bb806f8e2e06ca5ee1834648704e1d5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: ba3554a4a135ae43f5d5a2d65928275e4acfcc27c03609f2578d5304e8825c3f42f05b7229fe5881fdc7c663372ef6ca3591af02b2719167bc7711fc7754e7e6
|
|
7
|
+
data.tar.gz: 49c2b41f0cb606ec9e86b72777d55b3b3222220d879b5e05ed211a07094d9561c7e1c054ce2689d3b81b6ff872d981f6e1f24306b916cf12a697f7df23f91a69
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,25 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.35.0](https://github.com/janeapp/riffer/compare/riffer/v0.34.0...riffer/v0.35.0) (2026-06-25)
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
### Features
|
|
12
|
+
|
|
13
|
+
* per-call tags: for Agent#generate and #stream ([#339](https://github.com/janeapp/riffer/issues/339)) ([ddfda89](https://github.com/janeapp/riffer/commit/ddfda8929ca3b594415f827957d8c91104387317))
|
|
14
|
+
|
|
15
|
+
## [0.34.0](https://github.com/janeapp/riffer/compare/riffer/v0.33.0...riffer/v0.34.0) (2026-06-22)
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
### ⚠ BREAKING CHANGES
|
|
19
|
+
|
|
20
|
+
* **telemetry:** `config.tracing.tracer_provider` and `config.metrics.meter_provider` are removed. Assign the OTEL backend instead: `config.tracing.backend = Riffer::Tracing::Otel.build` (pass `provider:` to override the global provider).
|
|
21
|
+
|
|
22
|
+
### Features
|
|
23
|
+
|
|
24
|
+
* emit a riffer.guardrail.duration metric for guardrail execution ([#338](https://github.com/janeapp/riffer/issues/338)) ([7b0dfb0](https://github.com/janeapp/riffer/commit/7b0dfb014f710d839e53d25ee345df5b404c8c60))
|
|
25
|
+
* **telemetry:** pluggable tracing/metrics backend (OTEL opt-in) ([#329](https://github.com/janeapp/riffer/issues/329)) ([d647f54](https://github.com/janeapp/riffer/commit/d647f54b814bb8e331b89811c2ed661d6534b20c))
|
|
26
|
+
|
|
8
27
|
## [0.33.0](https://github.com/janeapp/riffer/compare/riffer/v0.32.1...riffer/v0.33.0) (2026-06-18)
|
|
9
28
|
|
|
10
29
|
|
data/docs/03_AGENTS.md
CHANGED
|
@@ -317,6 +317,50 @@ agent.generate('Hello')
|
|
|
317
317
|
|
|
318
318
|
When `config:` is supplied, the class-level configuration is ignored for that instance.
|
|
319
319
|
|
|
320
|
+
## Per-Call Tags
|
|
321
|
+
|
|
322
|
+
`#generate` and `#stream` accept an optional `tags:` hash — a flat map of attribution labels scoped to that single call (for cost/usage attribution, filtering audit logs, slicing telemetry). It is **per-call only**.
|
|
323
|
+
|
|
324
|
+
```ruby
|
|
325
|
+
agent.generate("Summarize this ticket.",
|
|
326
|
+
tags: {team: "growth", feature: "summarizer", user_id: "u_abc123"})
|
|
327
|
+
|
|
328
|
+
agent.stream("...", tags: {team: "growth", environment: "production"})
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
Keys and values may be `String` or `Symbol`; both are stringified, and entries with a `nil` value are dropped. Passing a non-`Hash` raises `Riffer::ArgumentError`. An omitted or empty `tags:` is a complete no-op.
|
|
332
|
+
|
|
333
|
+
Tags propagate to **two** places:
|
|
334
|
+
|
|
335
|
+
1. The provider's native per-request metadata field (see the mapping below).
|
|
336
|
+
2. Observability — stamped as `riffer.tag.<key>` on **every** span and metric the call emits (`invoke_agent`, `chat`, `execute_tool`, `execute_guardrail` spans, and the duration/token-usage/cost metrics). See [Tracing](16_TRACING.md) and [Metrics](17_METRICS.md).
|
|
337
|
+
|
|
338
|
+
### Reserved key: `user_id`
|
|
339
|
+
|
|
340
|
+
`user_id` is a reserved tag. Beyond appearing like any other tag, it maps to the provider's native end-user identifier where one exists (see the table).
|
|
341
|
+
|
|
342
|
+
### Provider mapping
|
|
343
|
+
|
|
344
|
+
| Provider | Native request field | `user_id` handling |
|
|
345
|
+
| --------------------- | ----------------------------- | --------------------------------------- |
|
|
346
|
+
| Amazon Bedrock | `request_metadata` (all tags) | Ordinary entry; no dedicated user field |
|
|
347
|
+
| OpenAI / Azure OpenAI | `metadata` (all tags) | Also mapped to `safety_identifier` |
|
|
348
|
+
| OpenRouter | `metadata` (all tags) | Also mapped to `user` |
|
|
349
|
+
| Anthropic | `metadata.user_id` **only** | The only tag forwarded |
|
|
350
|
+
| Gemini | _(none — observability only)_ | Tag only; no request field |
|
|
351
|
+
|
|
352
|
+
**Anthropic silently drops non-`user_id` tags.** The Messages API has no free-form request-metadata field — only `metadata.user_id`. So for Anthropic, `user_id` is forwarded as `metadata: {user_id: …}` and **every other tag is dropped from the request** (it still appears on spans and metrics). This is intentional.
|
|
353
|
+
|
|
354
|
+
**Gemini is observability-only.** Riffer's Gemini adapter targets the Gemini Developer API (`generativelanguage.googleapis.com`), whose `generateContent` request has **no** `labels` field — sending unknown fields is rejected. So tags are **not** added to the Gemini request; they propagate to spans and metrics only. Native request labels (`labels`, lowercase `[a-z0-9_-]`, ≤63 chars each) are a Vertex AI feature and would arrive with a future Vertex adapter.
|
|
355
|
+
|
|
356
|
+
### Provider limits (not enforced)
|
|
357
|
+
|
|
358
|
+
Riffer does not validate tag count, key/value length, or charset — it forwards what you give it and lets the provider reject anything out of bounds. Known limits: Bedrock `requestMetadata` ≤16 entries, key/value ≤256 chars; OpenAI / OpenRouter `metadata` ≤16 pairs (OpenRouter: 64-char keys, 512-char values).
|
|
359
|
+
|
|
360
|
+
### Metric cardinality
|
|
361
|
+
|
|
362
|
+
Tags are promoted onto metric dimensions verbatim — there is no allow-list. High-cardinality values (e.g. a unique `user_id` per request) will create a correspondingly large number of metric series in your backend. Keep metric-bound tags low-cardinality.
|
|
363
|
+
|
|
320
364
|
## Expand Your Agent
|
|
321
365
|
|
|
322
366
|
| Goal | Feature | Guide |
|
data/docs/10_CONFIGURATION.md
CHANGED
|
@@ -103,38 +103,38 @@ Accepts a `Riffer::Skills::Backend` instance or a `Proc` that receives `context`
|
|
|
103
103
|
|
|
104
104
|
### Tracing
|
|
105
105
|
|
|
106
|
-
Tracing-related global configuration lives under `config.tracing`. Riffer
|
|
106
|
+
Tracing-related global configuration lives under `config.tracing`. Riffer emits spans only through the backend you assign to `config.tracing.backend` — there is **no auto-detection**. OpenTelemetry is a built-in backend you opt into explicitly with `Riffer::Tracing::Otel.build`; with no backend set every span is a silent no-op, and riffer carries no OTEL gem dependency either way.
|
|
107
107
|
|
|
108
108
|
```ruby
|
|
109
109
|
Riffer.configure do |config|
|
|
110
110
|
config.tracing.enabled = ENV.fetch("RIFFER_TRACING_ENABLED", "true")
|
|
111
|
+
# Opt into OpenTelemetry (uses the global tracer provider):
|
|
112
|
+
config.tracing.backend = Riffer::Tracing::Otel.build
|
|
111
113
|
end
|
|
112
114
|
```
|
|
113
115
|
|
|
114
|
-
| Option | Description
|
|
115
|
-
| ------------------ |
|
|
116
|
-
| `enabled` | The kill switch, consulted on every span — flipping it at runtime takes effect immediately. Accepts booleans or `'true'`/`'false'`/`'1'`/`'0'`. Defaults to `true`.
|
|
117
|
-
| `capture_messages` | Opt-in capture of full message content on LLM-call spans (`gen_ai.input.messages`, `gen_ai.output.messages`, `gen_ai.system_instructions`) as GenAI-semconv JSON. Defaults to `false` — message content routinely carries sensitive data. File attachments serialize as metadata-only stubs (media type and name, never bytes), and riffer applies no size limit of its own — cap oversized attributes with the OTEL SDK attribute length limits.
|
|
118
|
-
| `
|
|
119
|
-
|
|
120
|
-
Hosts own SDK and exporter wiring — riffer only emits spans through whatever provider the host configures. See [Tracing](16_TRACING.md) for the emitted span contract — names, attributes, hierarchy, and host wiring.
|
|
116
|
+
| Option | Description |
|
|
117
|
+
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
118
|
+
| `enabled` | The kill switch, consulted on every span — flipping it at runtime takes effect immediately, short-circuiting to a no-op ahead of the backend. Accepts booleans or `'true'`/`'false'`/`'1'`/`'0'`. Defaults to `true`. |
|
|
119
|
+
| `capture_messages` | Opt-in capture of full message content on LLM-call spans (`gen_ai.input.messages`, `gen_ai.output.messages`, `gen_ai.system_instructions`) as GenAI-semconv JSON. Defaults to `false` — message content routinely carries sensitive data. File attachments serialize as metadata-only stubs (media type and name, never bytes), and riffer applies no size limit of its own — cap oversized attributes with the OTEL SDK attribute length limits. |
|
|
120
|
+
| `backend` | The backend riffer routes spans through. Assign `Riffer::Tracing::Otel.build` (pass `provider:` to override the global tracer provider — e.g. an in-memory provider in tests), or any object satisfying the duck-typed contract (`in_span` / `current_context` / `with_context`) to route into a non-OTEL system (e.g. Datadog APM). Defaults to `nil` — a no-op. Raises `Riffer::ArgumentError` unless the value is `nil` or responds to `in_span`. See [Tracing → Routing to a non-OpenTelemetry backend](16_TRACING.md#routing-to-a-non-opentelemetry-backend). |
|
|
121
121
|
|
|
122
122
|
### Metrics
|
|
123
123
|
|
|
124
|
-
Metrics-related global configuration lives under `config.metrics`, **independent** of `config.tracing` — each has its own kill switch, so you can run one signal without the other. Riffer
|
|
124
|
+
Metrics-related global configuration lives under `config.metrics`, **independent** of `config.tracing` — each has its own kill switch and its own backend, so you can run one signal without the other. Riffer records measurements only through the backend you assign to `config.metrics.backend` — there is **no auto-detection**. OpenTelemetry is a built-in backend you opt into explicitly with `Riffer::Metrics::Otel.build`; with no backend set every measurement is a silent no-op, and riffer carries no OTEL gem dependency either way. The metrics API and SDK are separate, still-experimental (pre-1.0) gems from the traces API.
|
|
125
125
|
|
|
126
126
|
```ruby
|
|
127
127
|
Riffer.configure do |config|
|
|
128
128
|
config.metrics.enabled = ENV.fetch("RIFFER_METRICS_ENABLED", "true")
|
|
129
|
+
# Opt into OpenTelemetry (uses the global meter provider):
|
|
130
|
+
config.metrics.backend = Riffer::Metrics::Otel.build
|
|
129
131
|
end
|
|
130
132
|
```
|
|
131
133
|
|
|
132
|
-
| Option
|
|
133
|
-
|
|
|
134
|
-
| `enabled`
|
|
135
|
-
| `
|
|
136
|
-
|
|
137
|
-
Hosts own SDK, reader, exporter, and aggregation wiring — riffer only records instruments through whatever provider the host configures, and histogram bucket boundaries are set host-side via Views. See [Metrics](17_METRICS.md) for the instrument contract — names, units, attributes, and host wiring.
|
|
134
|
+
| Option | Description |
|
|
135
|
+
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
136
|
+
| `enabled` | The kill switch, consulted on every measurement — flipping it at runtime takes effect immediately, short-circuiting to a no-op ahead of the backend. Accepts booleans or `'true'`/`'false'`/`'1'`/`'0'`. Defaults to `true`. |
|
|
137
|
+
| `backend` | The backend riffer routes measurements through. Assign `Riffer::Metrics::Otel.build` (pass `provider:` to override the global meter provider — e.g. an in-memory provider in tests), or any object responding to `record_histogram` to route into a non-OTEL system (e.g. DogStatsD). Defaults to `nil` — a no-op. Raises `Riffer::ArgumentError` unless the value is `nil` or responds to `record_histogram`. See [Metrics → Routing to a non-OpenTelemetry backend](17_METRICS.md#routing-to-a-non-opentelemetry-backend). |
|
|
138
138
|
|
|
139
139
|
### Pricing
|
|
140
140
|
|
data/docs/16_TRACING.md
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
Riffer instruments its agent loop with [OpenTelemetry](https://opentelemetry.io/) spans, following the [GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). The emitted span shape — names, attributes, and hierarchy — is a public, versioned contract you can build dashboards, alerts, and cost reporting against. This page is the reference for that contract.
|
|
4
4
|
|
|
5
|
-
Riffer only _emits_ spans.
|
|
5
|
+
Riffer only _emits_ spans, and only through a backend you assign to `config.tracing.backend` — OpenTelemetry is the built-in option you opt into (the host application owns the SDK, exporter, sampling, and service naming — the standard OTEL split), but never a default. With no backend assigned, every span is a silent no-op and Riffer carries no OpenTelemetry gem dependency.
|
|
6
6
|
|
|
7
7
|
## Enabling tracing
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Riffer emits spans only through a backend you assign to `config.tracing.backend` — it does **not** auto-detect OpenTelemetry. To use OTEL, add the SDK, configure an exporter, and assign Riffer's built-in OTEL backend with `Riffer::Tracing::Otel.build`.
|
|
10
10
|
|
|
11
11
|
```ruby
|
|
12
12
|
# Gemfile
|
|
@@ -19,8 +19,14 @@ require "opentelemetry/sdk"
|
|
|
19
19
|
OpenTelemetry::SDK.configure do |c|
|
|
20
20
|
c.service_name = "my-agent-host"
|
|
21
21
|
end
|
|
22
|
+
|
|
23
|
+
Riffer.configure do |config|
|
|
24
|
+
config.tracing.backend = Riffer::Tracing::Otel.build
|
|
25
|
+
end
|
|
22
26
|
```
|
|
23
27
|
|
|
28
|
+
`Riffer::Tracing::Otel.build` wraps the global `OpenTelemetry.tracer_provider` by default; pass `provider:` to wrap a specific one (an in-memory provider in tests, say). It returns `nil` — leaving tracing a no-op rather than raising — when the `opentelemetry-api` gem is absent or outside the supported range (>= 1.1, < 2), so the same configuration is safe on a host that doesn't bundle OTEL.
|
|
29
|
+
|
|
24
30
|
To see Riffer's spans on stdout while developing locally, wire in the console exporter:
|
|
25
31
|
|
|
26
32
|
```ruby
|
|
@@ -36,12 +42,30 @@ OpenTelemetry::SDK.configure do |c|
|
|
|
36
42
|
end
|
|
37
43
|
```
|
|
38
44
|
|
|
39
|
-
Any backend that implements the OpenTelemetry Traces API ingests Riffer's spans with no second pipeline
|
|
45
|
+
Any backend that implements the OpenTelemetry Traces API then ingests Riffer's spans with no second pipeline. For real exporter and collector setup (OTLP, sampling, resource attributes), see the [OpenTelemetry Ruby docs](https://opentelemetry.io/docs/languages/ruby/). A host on a non-OTEL stack (e.g. Datadog APM) assigns its own backend instead — see [Routing to a non-OpenTelemetry backend](#routing-to-a-non-opentelemetry-backend).
|
|
40
46
|
|
|
41
|
-
The
|
|
47
|
+
The tracing knobs — the `enabled` kill switch, opt-in message-content capture, and the `backend` itself — live in [Configuration — Tracing](10_CONFIGURATION.md#tracing).
|
|
42
48
|
|
|
43
49
|
Spans are emitted under the instrumentation scope named `riffer`, versioned with the Riffer gem version. That scope version is the runtime signal for which release produced a span; see [Stability](#stability).
|
|
44
50
|
|
|
51
|
+
## Routing to a non-OpenTelemetry backend
|
|
52
|
+
|
|
53
|
+
OpenTelemetry is one backend, not the only one. A host already invested in another stack — Datadog APM, say — can route Riffer's spans into it with **no `opentelemetry-*` gem installed** by assigning its own backend to `config.tracing.backend` in place of `Riffer::Tracing::Otel.build`. Whatever you assign is the backend; there is no fallback and no auto-detection — an unset backend is a no-op.
|
|
54
|
+
|
|
55
|
+
```ruby
|
|
56
|
+
Riffer.configure do |config|
|
|
57
|
+
config.tracing.backend = MyDatadogTracingBackend.new
|
|
58
|
+
end
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The backend is duck-typed — any object satisfying the contract works, and the setter validates only that it responds to `in_span` (otherwise it raises `Riffer::ArgumentError`). It must respond to:
|
|
62
|
+
|
|
63
|
+
- `in_span(name, attributes:, kind:) { |span| … }` — open a span around the block, yield a span object, and return the block's value.
|
|
64
|
+
- `current_context` — return the active trace context (for re-attaching across fiber/thread boundaries), or `nil` when there is none.
|
|
65
|
+
- `with_context(context) { … }` — run the block with the given context active; a `nil` context passes straight through, so a span re-attached while tracing was dark stays harmless.
|
|
66
|
+
|
|
67
|
+
The yielded span must respond to `set_attribute(key, value)`, `add_event(name, attributes:)`, `record_exception(exception)`, `error!(description)`, and `recording?` — the same surface the OTEL span exposes. `Riffer::Tracing::NoOp` is the reference shape for both the backend and the span contract. The `enabled` kill switch is still honoured ahead of the backend: with `config.tracing.enabled = false`, spans short-circuit to the no-op without ever reaching a custom backend.
|
|
68
|
+
|
|
45
69
|
## Spans
|
|
46
70
|
|
|
47
71
|
Riffer emits four span types. A single agent run produces one `invoke_agent` span wrapping one `chat` span per model call, one `execute_tool` span per tool call, and one `execute_guardrail` span per guardrail execution, interleaved in execution order:
|
|
@@ -69,6 +93,10 @@ Every attribute a span can carry is listed below, including the conditional ones
|
|
|
69
93
|
|
|
70
94
|
The contract promise is: **when present**, a key carries the documented meaning and type. It is _not_ a promise that every key appears on every span.
|
|
71
95
|
|
|
96
|
+
### Per-call tags (`riffer.tag.*`)
|
|
97
|
+
|
|
98
|
+
Any tags passed to `#generate` / `#stream` via `tags:` are stamped on **all four** span types as `riffer.tag.<key>` (string), so the per-span tables below omit them. They appear on every span the tagged call emits and are absent otherwise. Example: `tags: {team: "growth"}` adds `riffer.tag.team` → `"growth"` to the `invoke_agent`, `chat`, `execute_tool`, and `execute_guardrail` spans. See [Per-Call Tags](03_AGENTS.md#per-call-tags) for the full surface and the per-provider request-metadata mapping.
|
|
99
|
+
|
|
72
100
|
## `invoke_agent {agent}` — the run span
|
|
73
101
|
|
|
74
102
|
`INTERNAL`. One per call to `Agent#generate` or `Agent#stream`. The span name suffix is the agent's identifier (e.g. `invoke_agent weather-agent`).
|
data/docs/17_METRICS.md
CHANGED
|
@@ -2,13 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
Riffer can record [OpenTelemetry](https://opentelemetry.io/) metric instruments alongside its [spans](16_TRACING.md), following the [GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). Metric names, instrument types, units, and attributes are a public, versioned contract you can build dashboards and alerts against. This page is the reference for that contract.
|
|
4
4
|
|
|
5
|
-
As with tracing, Riffer only _records_ instruments.
|
|
5
|
+
As with tracing, Riffer only _records_ instruments, and only through a backend you assign to `config.metrics.backend` — OpenTelemetry is the built-in option you opt into (the host application owns the SDK, metric reader, exporter, and aggregation — the standard OTEL split), but never a default. With no backend assigned, every measurement is a silent no-op and Riffer carries no OpenTelemetry gem dependency.
|
|
6
6
|
|
|
7
7
|
> **OpenTelemetry metrics for Ruby is still pre-1.0.** The metrics API and SDK ship as separate, experimental gems (`opentelemetry-metrics-api`, `opentelemetry-metrics-sdk`) from the stable 1.x traces API. Riffer guards against an incompatible API and falls back to a no-op outside the supported range, but expect the host-side wiring below to evolve with those gems.
|
|
8
8
|
|
|
9
9
|
## Enabling metrics
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
Riffer records measurements only through a backend you assign to `config.metrics.backend` — it does **not** auto-detect OpenTelemetry. To use OTEL, add the metrics SDK, register a metric reader with an exporter, and assign Riffer's built-in OTEL backend with `Riffer::Metrics::Otel.build`.
|
|
12
12
|
|
|
13
13
|
```ruby
|
|
14
14
|
# Gemfile
|
|
@@ -21,14 +21,32 @@ require "opentelemetry-metrics-sdk"
|
|
|
21
21
|
OpenTelemetry::SDK.configure do |c|
|
|
22
22
|
c.service_name = "my-agent-host"
|
|
23
23
|
end
|
|
24
|
+
|
|
25
|
+
Riffer.configure do |config|
|
|
26
|
+
config.metrics.backend = Riffer::Metrics::Otel.build
|
|
27
|
+
end
|
|
24
28
|
```
|
|
25
29
|
|
|
26
|
-
The metrics SDK is **separate** from the traces SDK (`opentelemetry-sdk`); add it explicitly even if you already trace. Any backend implementing the OpenTelemetry Metrics API ingests Riffer's instruments. For real reader and exporter setup (OTLP, periodic export, Views), see the [OpenTelemetry Ruby docs](https://opentelemetry.io/docs/languages/ruby/).
|
|
30
|
+
The metrics SDK is **separate** from the traces SDK (`opentelemetry-sdk`); add it explicitly even if you already trace. `Riffer::Metrics::Otel.build` wraps the global `OpenTelemetry.meter_provider` by default; pass `provider:` to wrap a specific one. It returns `nil` — leaving metrics a no-op rather than raising — when the `opentelemetry-metrics-api` gem is absent or outside the supported range (>= 0.2, < 1.0). Any backend implementing the OpenTelemetry Metrics API then ingests Riffer's instruments. For real reader and exporter setup (OTLP, periodic export, Views), see the [OpenTelemetry Ruby docs](https://opentelemetry.io/docs/languages/ruby/).
|
|
27
31
|
|
|
28
|
-
The
|
|
32
|
+
The metrics knobs — the `enabled` kill switch and the `backend` itself — live in [Configuration — Metrics](10_CONFIGURATION.md#metrics). They are **independent** of the tracing knobs (each signal has its own backend): you can run tracing while metrics are off, or the reverse.
|
|
29
33
|
|
|
30
34
|
Instruments are recorded under the instrumentation scope named `riffer`, versioned with the Riffer gem version — the runtime signal for which release produced a measurement; see [Stability](#stability).
|
|
31
35
|
|
|
36
|
+
## Routing to a non-OpenTelemetry backend
|
|
37
|
+
|
|
38
|
+
OpenTelemetry is one backend, not the only one. A host that already runs another metrics stack — DogStatsD, say — can route Riffer's measurements into it with **no `opentelemetry-*` gem installed** by assigning its own backend to `config.metrics.backend` in place of `Riffer::Metrics::Otel.build`. Whatever you assign is the backend; there is no fallback and no auto-detection — an unset backend is a no-op.
|
|
39
|
+
|
|
40
|
+
```ruby
|
|
41
|
+
Riffer.configure do |config|
|
|
42
|
+
config.metrics.backend = MyDogStatsdMetricsBackend.new
|
|
43
|
+
end
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
The backend is duck-typed: any object that responds to `record_histogram(name, value, unit:, description:, attributes:)` works, and the setter validates only that (otherwise it raises `Riffer::ArgumentError`). `Riffer::Metrics::NoOp` is the reference shape. All four instruments are histograms, so the single `record_histogram` method covers the full contract; tell them apart by `name`. Two carry unit `s` — `gen_ai.client.operation.duration` and `riffer.guardrail.duration` — so `name` is the reliable discriminator; `{token}` is `gen_ai.client.token.usage` and `USD` is `riffer.gen_ai.cost`.
|
|
47
|
+
|
|
48
|
+
A custom backend counts as a **live** sink, so the providers still compute the token counts and cost that feed it — the same data that, under OTEL, populates `gen_ai.client.token.usage` and `riffer.gen_ai.cost`. The `enabled` kill switch is honoured ahead of the backend: with `config.metrics.enabled = false`, measurements short-circuit to the no-op without ever reaching a custom backend.
|
|
49
|
+
|
|
32
50
|
### Bucket boundaries
|
|
33
51
|
|
|
34
52
|
Histogram bucket boundaries are a **host-side** concern. The OpenTelemetry metrics API does not let an instrumenting library attach bucket boundaries at instrument creation, so Riffer does not set them — the SDK's default buckets apply unless you override them. To match the GenAI semantic conventions' recommended boundaries (or your own), register a [View](https://opentelemetry.io/docs/specs/otel/metrics/sdk/#view) on the meter provider that targets the instrument by name and sets explicit bucket boundaries.
|
|
@@ -56,6 +74,8 @@ OpenTelemetry.meter_provider.add_view(
|
|
|
56
74
|
|
|
57
75
|
Each instrument is documented here as a row carrying its name, instrument type, unit, and attribute set.
|
|
58
76
|
|
|
77
|
+
> **Per-call tags become dimensions.** Any tags passed to `#generate` / `#stream` via `tags:` are added as `riffer.tag.<key>` attributes to **every** instrument below — `operation.duration` (each operation), `token.usage` (each point), `cost`, and `guardrail.duration` — so the attribute lists omit them. Tags are promoted verbatim with no allow-list — a high-cardinality value like a unique `user_id` per request multiplies time series accordingly; drop it with a [View](https://opentelemetry.io/docs/specs/otel/metrics/sdk/#view) or keep metric-bound tags low-cardinality. See [Per-Call Tags](03_AGENTS.md#per-call-tags).
|
|
78
|
+
|
|
59
79
|
### `gen_ai.client.operation.duration`
|
|
60
80
|
|
|
61
81
|
Histogram, unit `s`. The latency of a single GenAI operation, recorded around the same wrap as the matching [span](16_TRACING.md) on both the success and error paths and timed with a monotonic clock. Recording is independent of tracing — the metric fires even with `config.tracing.enabled = false`. Tell the three operations apart by `gen_ai.operation.name`.
|
|
@@ -99,6 +119,18 @@ Pricing is **consumer-configured** — no price table ships with the gem (see [C
|
|
|
99
119
|
|
|
100
120
|
> **Per-call only.** Cost is never recorded at the run (`invoke_agent`) level, for the same reason as token usage: metrics pre-aggregate, so emitting both per-call points and a run total would double-count. Sum the per-call points in your backend for a run total.
|
|
101
121
|
|
|
122
|
+
### `riffer.guardrail.duration`
|
|
123
|
+
|
|
124
|
+
Histogram, unit `s`. The latency of a single guardrail execution, recorded around the same wrap as the [`execute_guardrail` span](16_TRACING.md) on both the success and raise paths and timed with a monotonic clock. Guardrails run on the request hot path — before every model turn and after every response — so this is the guardrail counterpart to `gen_ai.client.operation.duration`. Recording is independent of tracing — the metric fires even with `config.tracing.enabled = false`.
|
|
125
|
+
|
|
126
|
+
This instrument is Riffer-owned (`riffer.*`, not `gen_ai.*`) by the same reasoning as its span: a guardrail is not a GenAI semantic-convention operation, so the `execute_guardrail` span deliberately carries no `gen_ai.operation.name` and lives in riffer's own `riffer.guardrail.*` namespace (see [Tracing](16_TRACING.md)). Folding the metric into `gen_ai.client.operation.duration` would contradict that, so the duration is its own riffer-owned histogram instead; see [Stability](#stability).
|
|
127
|
+
|
|
128
|
+
| Value | Attributes |
|
|
129
|
+
| ----------------------------------- | -------------------------------------------------------------------------- |
|
|
130
|
+
| duration of the guardrail execution | `riffer.guardrail.name`, `riffer.guardrail.phase`, `error.type` (on raise) |
|
|
131
|
+
|
|
132
|
+
`riffer.guardrail.phase` is `before` or `after`. A pass, transform, or block is a **handled** outcome and records no `error.type` — that attribute carries the exception class only when a guardrail raises, mirroring the span. One time series exists per `riffer.guardrail.name` × `riffer.guardrail.phase`; guardrail names are bounded by the guardrails you configure, so cardinality is safe — unlike the dynamic tool names on `gen_ai.client.operation.duration`.
|
|
133
|
+
|
|
102
134
|
## Stability
|
|
103
135
|
|
|
104
136
|
The instrument shape is a public, versioned contract, in two tiers — mirroring the [tracing contract](16_TRACING.md#stability):
|
data/lib/riffer/agent/run.rb
CHANGED
|
@@ -10,37 +10,45 @@ module Riffer::Agent::Run
|
|
|
10
10
|
# for prompt/files semantics.
|
|
11
11
|
#
|
|
12
12
|
#--
|
|
13
|
-
#: (agent: Riffer::Agent, ?prompt: String?, ?files: Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart]?) -> Riffer::Agent::Response
|
|
14
|
-
def generate(agent:, prompt: nil, files: nil)
|
|
13
|
+
#: (agent: Riffer::Agent, ?prompt: String?, ?files: Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart]?, ?tags: Hash[(String | Symbol), untyped]) -> Riffer::Agent::Response
|
|
14
|
+
def generate(agent:, prompt: nil, files: nil, tags: {})
|
|
15
15
|
append_user_message(agent, prompt, files: files)
|
|
16
|
-
run_loop(agent)
|
|
16
|
+
run_loop(agent, tags: tags)
|
|
17
17
|
end
|
|
18
18
|
|
|
19
19
|
# Runs the streaming loop for the given agent. See Riffer::Agent#stream
|
|
20
20
|
# for prompt/files semantics.
|
|
21
21
|
#
|
|
22
22
|
#--
|
|
23
|
-
#: (agent: Riffer::Agent, ?prompt: String?, ?files: Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart]?) -> Enumerator[Riffer::StreamEvents::Base, void]
|
|
24
|
-
def stream(agent:, prompt: nil, files: nil)
|
|
23
|
+
#: (agent: Riffer::Agent, ?prompt: String?, ?files: Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart]?, ?tags: Hash[(String | Symbol), untyped]) -> Enumerator[Riffer::StreamEvents::Base, void]
|
|
24
|
+
def stream(agent:, prompt: nil, files: nil, tags: {})
|
|
25
25
|
append_user_message(agent, prompt, files: files)
|
|
26
26
|
# The enumerator body runs in its own fiber, where the fiber-local OTEL
|
|
27
|
-
# context is empty — capture here so the run span parents to the caller's
|
|
27
|
+
# context is empty — capture here so the run span parents to the caller's
|
|
28
|
+
# trace. tags ride as an ordinary argument captured in the closure, so they
|
|
29
|
+
# cross the fiber boundary without any re-propagation.
|
|
28
30
|
trace_context = Riffer::Tracing.current_context
|
|
29
31
|
Enumerator.new do |stream_yielder|
|
|
30
|
-
Riffer::Tracing.with_context(trace_context) { run_loop(agent, stream_yielder: stream_yielder) }
|
|
32
|
+
Riffer::Tracing.with_context(trace_context) { run_loop(agent, tags: tags, stream_yielder: stream_yielder) }
|
|
31
33
|
end
|
|
32
34
|
end
|
|
33
35
|
|
|
34
36
|
private
|
|
35
37
|
|
|
38
|
+
# Both +generate+ and +stream+ funnel here, so this is the single place raw
|
|
39
|
+
# +tags+ are normalized. The clean <tt>String => String</tt> map is then
|
|
40
|
+
# threaded to every span/metric builder in the run as +riffer.tag.*+ and to
|
|
41
|
+
# each provider call (via +merged_model_options+) for native request-metadata
|
|
42
|
+
# mapping.
|
|
36
43
|
#--
|
|
37
|
-
#: (Riffer::Agent, ?stream_yielder: Enumerator::Yielder?) -> Riffer::Agent::Response
|
|
38
|
-
def run_loop(agent, stream_yielder: nil)
|
|
44
|
+
#: (Riffer::Agent, ?tags: Hash[(String | Symbol), untyped]?, ?stream_yielder: Enumerator::Yielder?) -> Riffer::Agent::Response
|
|
45
|
+
def run_loop(agent, tags: {}, stream_yielder: nil)
|
|
46
|
+
tags = normalize_tags(tags)
|
|
39
47
|
start = Riffer::Metrics.monotonic_now
|
|
40
48
|
error_type = nil #: String?
|
|
41
49
|
begin
|
|
42
|
-
Riffer::Tracing.in_span("invoke_agent #{agent.class.identifier}", attributes: run_span_attributes(agent), kind: :internal) do |span|
|
|
43
|
-
response = execute_run(agent, stream_yielder)
|
|
50
|
+
Riffer::Tracing.in_span("invoke_agent #{agent.class.identifier}", attributes: run_span_attributes(agent, tags), kind: :internal) do |span|
|
|
51
|
+
response = execute_run(agent, stream_yielder, tags)
|
|
44
52
|
record_run_outcome(span, response)
|
|
45
53
|
response
|
|
46
54
|
rescue => error
|
|
@@ -55,18 +63,18 @@ module Riffer::Agent::Run
|
|
|
55
63
|
error_type = error.class.name #: String?
|
|
56
64
|
raise
|
|
57
65
|
ensure
|
|
58
|
-
Riffer::Metrics::Instruments::OPERATION_DURATION.record(Riffer::Metrics.monotonic_now - start, attributes: run_metric_attributes(agent, error_type))
|
|
66
|
+
Riffer::Metrics::Instruments::OPERATION_DURATION.record(Riffer::Metrics.monotonic_now - start, attributes: run_metric_attributes(agent, error_type, tags))
|
|
59
67
|
end
|
|
60
68
|
end
|
|
61
69
|
|
|
62
70
|
#--
|
|
63
|
-
#: (Riffer::Agent, Enumerator::Yielder?) -> Riffer::Agent::Response
|
|
64
|
-
def execute_run(agent, stream_yielder)
|
|
71
|
+
#: (Riffer::Agent, Enumerator::Yielder?, ?Hash[String, String]) -> Riffer::Agent::Response
|
|
72
|
+
def execute_run(agent, stream_yielder, tags = {})
|
|
65
73
|
all_modifications = [] #: Array[Riffer::Guardrails::Modification]
|
|
66
74
|
run_usage = nil #: Riffer::Providers::TokenUsage?
|
|
67
75
|
run_steps = 0
|
|
68
76
|
|
|
69
|
-
run_before_guardrails(agent, stream_yielder, all_modifications) do |tripwire|
|
|
77
|
+
run_before_guardrails(agent, stream_yielder, all_modifications, tags) do |tripwire|
|
|
70
78
|
return tripwire_response(agent, stream_yielder, tripwire, all_modifications, steps: run_steps)
|
|
71
79
|
end
|
|
72
80
|
|
|
@@ -84,16 +92,16 @@ module Riffer::Agent::Run
|
|
|
84
92
|
step = agent.session.steps
|
|
85
93
|
|
|
86
94
|
reason = catch(:riffer_interrupt) do
|
|
87
|
-
execute_pending_tool_calls(agent)
|
|
95
|
+
execute_pending_tool_calls(agent, tags)
|
|
88
96
|
|
|
89
97
|
loop do
|
|
90
|
-
response = stream_yielder ? accumulate_streamed_response(agent, stream_yielder) : call_llm(agent)
|
|
98
|
+
response = stream_yielder ? accumulate_streamed_response(agent, stream_yielder, tags) : call_llm(agent, tags)
|
|
91
99
|
step += 1
|
|
92
100
|
run_steps += 1
|
|
93
101
|
track_token_usage(agent, response.token_usage)
|
|
94
102
|
run_usage = sum_usage(run_usage, response.token_usage)
|
|
95
103
|
|
|
96
|
-
processed_response = run_after_guardrails(agent, response, stream_yielder, all_modifications) do |tripwire|
|
|
104
|
+
processed_response = run_after_guardrails(agent, response, stream_yielder, all_modifications, tags) do |tripwire|
|
|
97
105
|
return tripwire_response(agent, stream_yielder, tripwire, all_modifications, token_usage: run_usage, steps: run_steps)
|
|
98
106
|
end
|
|
99
107
|
|
|
@@ -104,7 +112,7 @@ module Riffer::Agent::Run
|
|
|
104
112
|
max_steps = agent.config.max_steps
|
|
105
113
|
throw :riffer_interrupt, Riffer::Agent::INTERRUPT_MAX_STEPS if max_steps && step >= max_steps
|
|
106
114
|
|
|
107
|
-
execute_tool_calls(agent, processed_response)
|
|
115
|
+
execute_tool_calls(agent, processed_response, tags: tags)
|
|
108
116
|
end
|
|
109
117
|
|
|
110
118
|
return final_response(agent, all_modifications, token_usage: run_usage, steps: run_steps)
|
|
@@ -122,14 +130,14 @@ module Riffer::Agent::Run
|
|
|
122
130
|
end
|
|
123
131
|
|
|
124
132
|
#--
|
|
125
|
-
#: (Riffer::Agent, Enumerator::Yielder) -> Riffer::Messages::Assistant
|
|
126
|
-
def accumulate_streamed_response(agent, stream_yielder)
|
|
133
|
+
#: (Riffer::Agent, Enumerator::Yielder, ?Hash[String, String]) -> Riffer::Messages::Assistant
|
|
134
|
+
def accumulate_streamed_response(agent, stream_yielder, tags = {})
|
|
127
135
|
accumulated_content = ""
|
|
128
136
|
accumulated_tool_calls = [] #: Array[Riffer::Messages::Assistant::ToolCall]
|
|
129
137
|
accumulated_token_usage = nil #: Riffer::Providers::TokenUsage?
|
|
130
138
|
accumulated_finish_reason = nil #: Symbol?
|
|
131
139
|
|
|
132
|
-
call_llm_stream(agent).each do |event|
|
|
140
|
+
call_llm_stream(agent, tags).each do |event|
|
|
133
141
|
stream_yielder << event
|
|
134
142
|
|
|
135
143
|
case event
|
|
@@ -180,33 +188,33 @@ module Riffer::Agent::Run
|
|
|
180
188
|
end
|
|
181
189
|
|
|
182
190
|
#--
|
|
183
|
-
#: (Riffer::Agent) -> Riffer::Messages::Assistant
|
|
184
|
-
def call_llm(agent)
|
|
191
|
+
#: (Riffer::Agent, ?Hash[String, String]) -> Riffer::Messages::Assistant
|
|
192
|
+
def call_llm(agent, tags = {})
|
|
185
193
|
agent.provider.generate_text(
|
|
186
194
|
messages: agent.session.messages,
|
|
187
195
|
model: agent.model_name,
|
|
188
196
|
tools: effective_tools(agent),
|
|
189
|
-
**merged_model_options(agent)
|
|
197
|
+
**merged_model_options(agent, tags)
|
|
190
198
|
)
|
|
191
199
|
end
|
|
192
200
|
|
|
193
201
|
#--
|
|
194
|
-
#: (Riffer::Agent) -> Enumerator[Riffer::StreamEvents::Base, void]
|
|
195
|
-
def call_llm_stream(agent)
|
|
202
|
+
#: (Riffer::Agent, ?Hash[String, String]) -> Enumerator[Riffer::StreamEvents::Base, void]
|
|
203
|
+
def call_llm_stream(agent, tags = {})
|
|
196
204
|
agent.provider.stream_text(
|
|
197
205
|
messages: agent.session.messages,
|
|
198
206
|
model: agent.model_name,
|
|
199
207
|
tools: effective_tools(agent),
|
|
200
|
-
**merged_model_options(agent)
|
|
208
|
+
**merged_model_options(agent, tags)
|
|
201
209
|
)
|
|
202
210
|
end
|
|
203
211
|
|
|
204
212
|
#--
|
|
205
|
-
#: (Riffer::Agent, Riffer::Messages::Assistant, ?tool_calls: Array[Riffer::Messages::Assistant::ToolCall]) -> void
|
|
206
|
-
def execute_tool_calls(agent, assistant_message, tool_calls: assistant_message.tool_calls)
|
|
213
|
+
#: (Riffer::Agent, Riffer::Messages::Assistant, ?tool_calls: Array[Riffer::Messages::Assistant::ToolCall], ?tags: Hash[String, String]) -> void
|
|
214
|
+
def execute_tool_calls(agent, assistant_message, tool_calls: assistant_message.tool_calls, tags: {})
|
|
207
215
|
return if tool_calls.empty?
|
|
208
216
|
|
|
209
|
-
results = agent.tool_runtime.execute(tool_calls, tools: effective_tools(agent), context: agent.context, assistant_message: assistant_message)
|
|
217
|
+
results = agent.tool_runtime.execute(tool_calls, tools: effective_tools(agent), context: agent.context, assistant_message: assistant_message, tags: tags)
|
|
210
218
|
|
|
211
219
|
inject_discovered_tools(agent, results)
|
|
212
220
|
|
|
@@ -233,19 +241,19 @@ module Riffer::Agent::Run
|
|
|
233
241
|
end
|
|
234
242
|
|
|
235
243
|
#--
|
|
236
|
-
#: (Riffer::Agent) -> void
|
|
237
|
-
def execute_pending_tool_calls(agent)
|
|
244
|
+
#: (Riffer::Agent, ?Hash[String, String]) -> void
|
|
245
|
+
def execute_pending_tool_calls(agent, tags = {})
|
|
238
246
|
assistant_message, pending = agent.session.pending_tool_calls
|
|
239
|
-
execute_tool_calls(agent, assistant_message, tool_calls: pending) if assistant_message
|
|
247
|
+
execute_tool_calls(agent, assistant_message, tool_calls: pending, tags: tags) if assistant_message
|
|
240
248
|
end
|
|
241
249
|
|
|
242
250
|
#--
|
|
243
|
-
#: (Riffer::Agent, Enumerator::Yielder?, Array[Riffer::Guardrails::Modification]) { (Riffer::Guardrails::Tripwire) -> void } -> void
|
|
244
|
-
def run_before_guardrails(agent, stream_yielder, all_modifications)
|
|
251
|
+
#: (Riffer::Agent, Enumerator::Yielder?, Array[Riffer::Guardrails::Modification], ?Hash[String, String]) { (Riffer::Guardrails::Tripwire) -> void } -> void
|
|
252
|
+
def run_before_guardrails(agent, stream_yielder, all_modifications, tags = {})
|
|
245
253
|
guardrails = agent.config.guardrails_for(:before)
|
|
246
254
|
return if guardrails.empty?
|
|
247
255
|
|
|
248
|
-
runner = Riffer::Guardrails::Runner.new(guardrails, phase: :before, context: agent.context)
|
|
256
|
+
runner = Riffer::Guardrails::Runner.new(guardrails, phase: :before, context: agent.context, tags: tags)
|
|
249
257
|
processed_messages, tripwire, modifications = runner.run(agent.session.messages)
|
|
250
258
|
agent.session.set(processed_messages) unless tripwire
|
|
251
259
|
record_modifications!(stream_yielder, all_modifications, modifications)
|
|
@@ -253,12 +261,12 @@ module Riffer::Agent::Run
|
|
|
253
261
|
end
|
|
254
262
|
|
|
255
263
|
#--
|
|
256
|
-
#: (Riffer::Agent, Riffer::Messages::Assistant, Enumerator::Yielder?, Array[Riffer::Guardrails::Modification]) { (Riffer::Guardrails::Tripwire) -> void } -> untyped
|
|
257
|
-
def run_after_guardrails(agent, response, stream_yielder, all_modifications)
|
|
264
|
+
#: (Riffer::Agent, Riffer::Messages::Assistant, Enumerator::Yielder?, Array[Riffer::Guardrails::Modification], ?Hash[String, String]) { (Riffer::Guardrails::Tripwire) -> void } -> untyped
|
|
265
|
+
def run_after_guardrails(agent, response, stream_yielder, all_modifications, tags = {})
|
|
258
266
|
guardrails = agent.config.guardrails_for(:after)
|
|
259
267
|
return response if guardrails.empty?
|
|
260
268
|
|
|
261
|
-
runner = Riffer::Guardrails::Runner.new(guardrails, phase: :after, context: agent.context)
|
|
269
|
+
runner = Riffer::Guardrails::Runner.new(guardrails, phase: :after, context: agent.context, tags: tags)
|
|
262
270
|
processed_response, tripwire, modifications = runner.run(response, messages: agent.session.messages)
|
|
263
271
|
|
|
264
272
|
response_index = agent.session.messages.length
|
|
@@ -285,11 +293,16 @@ module Riffer::Agent::Run
|
|
|
285
293
|
discovered.empty? ? agent.tools : agent.tools + discovered
|
|
286
294
|
end
|
|
287
295
|
|
|
296
|
+
# +tags+ rides in the options hash as a curated key the providers extract for
|
|
297
|
+
# native request-metadata mapping (alongside +:structured_output+); it never
|
|
298
|
+
# reaches an SDK call verbatim. Span/metric tagging is threaded separately to
|
|
299
|
+
# each builder.
|
|
288
300
|
#--
|
|
289
|
-
#: (Riffer::Agent) -> Hash[Symbol, untyped]
|
|
290
|
-
def merged_model_options(agent)
|
|
301
|
+
#: (Riffer::Agent, ?Hash[String, String]) -> Hash[Symbol, untyped]
|
|
302
|
+
def merged_model_options(agent, tags = {})
|
|
291
303
|
opts = agent.config.model_options.dup
|
|
292
304
|
opts[:structured_output] = agent.structured_output if agent.structured_output
|
|
305
|
+
opts[:tags] = tags unless tags.empty?
|
|
293
306
|
opts
|
|
294
307
|
end
|
|
295
308
|
|
|
@@ -329,19 +342,19 @@ module Riffer::Agent::Run
|
|
|
329
342
|
end
|
|
330
343
|
|
|
331
344
|
#--
|
|
332
|
-
#: (Riffer::Agent) -> Hash[String, untyped]
|
|
333
|
-
def run_span_attributes(agent)
|
|
345
|
+
#: (Riffer::Agent, ?Hash[String, String]) -> Hash[String, untyped]
|
|
346
|
+
def run_span_attributes(agent, tags = {})
|
|
334
347
|
{
|
|
335
348
|
"gen_ai.operation.name" => "invoke_agent",
|
|
336
349
|
"gen_ai.agent.name" => agent.class.identifier,
|
|
337
350
|
"gen_ai.provider.name" => agent.provider.class.semconv_provider_name,
|
|
338
351
|
"gen_ai.request.model" => agent.model_name
|
|
339
|
-
}
|
|
352
|
+
}.merge(tag_attributes(tags))
|
|
340
353
|
end
|
|
341
354
|
|
|
342
355
|
#--
|
|
343
|
-
#: (Riffer::Agent, String?) -> Hash[String, untyped]
|
|
344
|
-
def run_metric_attributes(agent, error_type)
|
|
356
|
+
#: (Riffer::Agent, String?, ?Hash[String, String]) -> Hash[String, untyped]
|
|
357
|
+
def run_metric_attributes(agent, error_type, tags = {})
|
|
345
358
|
attributes = {
|
|
346
359
|
"gen_ai.operation.name" => "invoke_agent",
|
|
347
360
|
"gen_ai.provider.name" => agent.provider.class.semconv_provider_name,
|
|
@@ -349,11 +362,31 @@ module Riffer::Agent::Run
|
|
|
349
362
|
"gen_ai.agent.name" => agent.class.identifier
|
|
350
363
|
} #: Hash[String, untyped]
|
|
351
364
|
attributes["error.type"] = error_type if error_type
|
|
352
|
-
attributes
|
|
365
|
+
attributes.merge(tag_attributes(tags))
|
|
366
|
+
end
|
|
367
|
+
|
|
368
|
+
#--
|
|
369
|
+
#: (Hash[(String | Symbol), untyped]?) -> Hash[String, String]
|
|
370
|
+
def normalize_tags(tags)
|
|
371
|
+
return {} if tags.nil?
|
|
372
|
+
raise Riffer::ArgumentError, "tags: must be a Hash, got #{tags.class}" unless tags.is_a?(Hash)
|
|
373
|
+
|
|
374
|
+
result = {} #: Hash[String, String]
|
|
375
|
+
tags.each do |key, value|
|
|
376
|
+
next if value.nil?
|
|
377
|
+
result[key.to_s] = value.to_s
|
|
378
|
+
end
|
|
379
|
+
result
|
|
380
|
+
end
|
|
381
|
+
|
|
382
|
+
#--
|
|
383
|
+
#: (Hash[String, String]) -> Hash[String, String]
|
|
384
|
+
def tag_attributes(tags)
|
|
385
|
+
tags.transform_keys { |key| "riffer.tag.#{key}" }
|
|
353
386
|
end
|
|
354
387
|
|
|
355
388
|
#--
|
|
356
|
-
#: (Riffer::Tracing::Otel::Span | Riffer::Tracing::
|
|
389
|
+
#: (Riffer::Tracing::Otel::Span | Riffer::Tracing::NoOp::Span, Riffer::Agent::Response) -> void
|
|
357
390
|
def record_run_outcome(span, response)
|
|
358
391
|
span.set_attribute("riffer.steps", response.steps)
|
|
359
392
|
Riffer::Tracing.record_usage(span, response.token_usage)
|