riffer 0.52.1 → 0.53.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.release-please-manifest.json +1 -1
- data/CHANGELOG.md +7 -0
- data/docs/AGENTS.md +15 -0
- data/docs/CATALOG.md +205 -0
- data/docs/CONFIGURATION.md +35 -23
- data/docs/MESSAGES.md +1 -1
- data/docs/SERIALIZATION.md +1 -0
- data/docs/TRACING.md +2 -1
- data/docs/providers/AMAZON_BEDROCK.md +2 -0
- data/docs/providers/ANTHROPIC.md +2 -0
- data/docs/providers/AZURE_OPENAI.md +2 -0
- data/docs/providers/CUSTOM_PROVIDERS.md +15 -1
- data/docs/providers/GEMINI.md +10 -0
- data/docs/providers/MOCK_PROVIDER.md +2 -0
- data/docs/providers/OPENAI.md +2 -0
- data/docs/providers/OPENROUTER.md +2 -0
- data/docs-site/manifest.yml +3 -0
- data/lib/riffer/agent/config.rb +17 -0
- data/lib/riffer/agent/run.rb +2 -0
- data/lib/riffer/agent/serializer.rb +2 -0
- data/lib/riffer/agent.rb +7 -0
- data/lib/riffer/{config/pricing → catalog}/rates.rb +21 -6
- data/lib/riffer/catalog.rb +225 -0
- data/lib/riffer/config/pricing.rb +17 -27
- data/lib/riffer/config.rb +29 -0
- data/lib/riffer/providers/amazon_bedrock.rb +14 -0
- data/lib/riffer/providers/anthropic.rb +8 -0
- data/lib/riffer/providers/base.rb +58 -6
- data/lib/riffer/providers/gemini.rb +6 -0
- data/lib/riffer/providers/mock.rb +7 -0
- data/lib/riffer/providers/open_ai.rb +6 -0
- data/lib/riffer/providers/open_router.rb +6 -0
- data/lib/riffer/version.rb +1 -1
- data/lib/riffer.rb +2 -0
- data/sig/generated/riffer/agent/config.rbs +10 -1
- data/sig/generated/riffer/agent.rbs +4 -0
- data/sig/generated/riffer/{config/pricing → catalog}/rates.rbs +10 -4
- data/sig/generated/riffer/catalog.rbs +77 -0
- data/sig/generated/riffer/config/pricing.rbs +12 -11
- data/sig/generated/riffer/config.rbs +17 -0
- data/sig/generated/riffer/providers/amazon_bedrock.rbs +6 -0
- data/sig/generated/riffer/providers/anthropic.rbs +4 -0
- data/sig/generated/riffer/providers/base.rbs +22 -2
- data/sig/generated/riffer/providers/gemini.rbs +4 -0
- data/sig/generated/riffer/providers/mock.rbs +5 -0
- data/sig/generated/riffer/providers/open_ai.rbs +4 -0
- data/sig/generated/riffer/providers/open_router.rbs +4 -0
- data/sig/generated/riffer.rbs +1 -0
- metadata +6 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 70784c56c20102b8da3e1e147bc843db2287e2be910f1e10d26b8ee9a2f8e0f9
|
|
4
|
+
data.tar.gz: 6df766acde899e39867a06de11fca0d013671e4bda34d3759b4ae310a43d04de
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 39af9afa9f695fe6dcc661915e1173ddf812db2ee425b83f77ee42d2080cea623b0f03f82c80ae606e7b4caf2f42bb3799a63ed0955ab7baa13b2c30861a2781
|
|
7
|
+
data.tar.gz: 1f79ef71cf1a838092444368f6986cd772803aeffe0de9acadc5cf8c23834e0e56c04e5049001eaeab3360d13a0eb3767a8118f429cacdbad7372879706ce332
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,13 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.53.0](https://github.com/janeapp/riffer/compare/riffer/v0.52.1...riffer/v0.53.0) (2026-10-13)
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
### Features
|
|
12
|
+
|
|
13
|
+
* **catalog:** add a model catalog and a cross-provider reasoning setting ([#480](https://github.com/janeapp/riffer/issues/480)) ([7144f54](https://github.com/janeapp/riffer/commit/7144f54022257217f4e9ebee3263b5627cb96544))
|
|
14
|
+
|
|
8
15
|
## [0.52.1](https://github.com/janeapp/riffer/compare/riffer/v0.52.0...riffer/v0.52.1) (2026-10-09)
|
|
9
16
|
|
|
10
17
|
|
data/docs/AGENTS.md
CHANGED
|
@@ -139,6 +139,21 @@ class MyAgent < Riffer::Agent
|
|
|
139
139
|
end
|
|
140
140
|
```
|
|
141
141
|
|
|
142
|
+
### reasoning
|
|
143
|
+
|
|
144
|
+
Sets how much the model thinks before it answers: `:off`, `:low`, `:medium`, `:high`, `:xhigh`, or `:max` (a String works too). Anything else raises `Riffer::ArgumentError`. Leave it unset to send nothing and keep the model's default.
|
|
145
|
+
|
|
146
|
+
```ruby
|
|
147
|
+
class MyAgent < Riffer::Agent
|
|
148
|
+
model 'anthropic/claude-sonnet-4-6'
|
|
149
|
+
reasoning :low
|
|
150
|
+
end
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Riffer turns the level into the provider's own params. It uses the model's [catalog](CATALOG.md#reasoning) entry when one defines the level, and otherwise the provider's default, which sends the level name as the provider's effort value (for example `output_config: { effort: "low" }` with adaptive thinking on Anthropic). Riffer doesn't check whether the model supports the level; a model that rejects it returns the provider's error.
|
|
154
|
+
|
|
155
|
+
`model_options` are deep-merged over those params, so they override single fields rather than whole blocks. For example, `model_options max_tokens: 8000` keeps the catalog's thinking budget and replaces only its `max_tokens`.
|
|
156
|
+
|
|
142
157
|
### max_steps
|
|
143
158
|
|
|
144
159
|
Sets the maximum number of LLM call steps in the tool-use loop. When the limit is reached, the loop interrupts with reason `:max_steps`. Defaults to `16`. Set to `nil` (`max_steps nil`) for unlimited steps:
|
data/docs/CATALOG.md
ADDED
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
# Model Catalog
|
|
2
|
+
|
|
3
|
+
The model catalog holds per-model data that riffer can't derive from the model name: how a [`reasoning`](AGENTS.md#reasoning) level maps to request params, and what the model costs. It lives in JSON files you list in your configuration, so a new model or a price change is a data edit, not a riffer release.
|
|
4
|
+
|
|
5
|
+
Riffer ships no catalog. Without one:
|
|
6
|
+
|
|
7
|
+
- `reasoning` still works through each provider's default mapping (see [Provider defaults](#provider-defaults)).
|
|
8
|
+
- Calls carry no cost (`token_usage.cost` is `nil`), unless the deprecated [`config.pricing`](CONFIGURATION.md#pricing-deprecated) prices the model.
|
|
9
|
+
|
|
10
|
+
## Loading catalog files
|
|
11
|
+
|
|
12
|
+
```ruby
|
|
13
|
+
Riffer.configure do |config|
|
|
14
|
+
config.catalog_files = ["config/riffer/models.json"]
|
|
15
|
+
end
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
- `catalog_files` takes an Array of paths (Strings or Pathnames). Files load in the order listed.
|
|
19
|
+
- `Riffer.configure` builds the catalog at the end of the block, so a missing file or invalid data raises `Riffer::ArgumentError` at boot.
|
|
20
|
+
- Calling `configure` again rebuilds the catalog from scratch.
|
|
21
|
+
- Files load at boot only. There is no API to add entries at runtime. The built catalog is frozen.
|
|
22
|
+
|
|
23
|
+
## File format
|
|
24
|
+
|
|
25
|
+
```json
|
|
26
|
+
{
|
|
27
|
+
"version": 1,
|
|
28
|
+
"models": {
|
|
29
|
+
"anthropic/claude-haiku-4-5-20251001": {
|
|
30
|
+
"aliases": ["anthropic/claude-haiku-4-5"],
|
|
31
|
+
"pricing": {
|
|
32
|
+
"input": 1.0,
|
|
33
|
+
"output": 5.0,
|
|
34
|
+
"cache_read": 0.1,
|
|
35
|
+
"cache_write": 1.25
|
|
36
|
+
},
|
|
37
|
+
"reasoning": {
|
|
38
|
+
"off": { "thinking": { "type": "disabled" } },
|
|
39
|
+
"low": {
|
|
40
|
+
"thinking": { "type": "enabled", "budget_tokens": 1024 },
|
|
41
|
+
"max_tokens": 5120
|
|
42
|
+
},
|
|
43
|
+
"high": {
|
|
44
|
+
"thinking": { "type": "enabled", "budget_tokens": 24576 },
|
|
45
|
+
"max_tokens": 28672
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
},
|
|
49
|
+
"gemini/gemini-2.5-flash": {
|
|
50
|
+
"reasoning": {
|
|
51
|
+
"off": { "thinkingConfig": { "thinkingBudget": 0 } },
|
|
52
|
+
"low": { "thinkingConfig": { "thinkingBudget": 1024 } }
|
|
53
|
+
}
|
|
54
|
+
},
|
|
55
|
+
"amazon_bedrock/us.amazon.nova-2-lite-v1:0": {
|
|
56
|
+
"reasoning": {
|
|
57
|
+
"low": {
|
|
58
|
+
"additional_model_request_fields": {
|
|
59
|
+
"reasoningConfig": {
|
|
60
|
+
"type": "enabled",
|
|
61
|
+
"maxReasoningEffort": "low"
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
| Key | Description |
|
|
72
|
+
| --------- | ------------------------------------------------------------------------------------------ |
|
|
73
|
+
| `version` | Required. The file format version. Only `1` is supported. |
|
|
74
|
+
| `models` | An object keyed by exact `provider/model` id: the same string you give an agent's `model`. |
|
|
75
|
+
|
|
76
|
+
Each model entry can have any of these keys:
|
|
77
|
+
|
|
78
|
+
| Key | Description |
|
|
79
|
+
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
80
|
+
| `aliases` | Other `provider/model` ids that share this entry, pricing included. An alias may name a different provider, e.g. `"azure_openai/my-gpt-prod"` on an OpenAI entry. |
|
|
81
|
+
| `pricing` | Per-million-token rates. See [Pricing](#pricing). |
|
|
82
|
+
| `reasoning` | An object keyed by level (`off`, `low`, `medium`, `high`, `xhigh`, `max`). Each value is a `model_options` fragment. See [Reasoning](#reasoning). |
|
|
83
|
+
|
|
84
|
+
### Lookup
|
|
85
|
+
|
|
86
|
+
A model id matches an entry by its exact key first, then by an alias. There are no patterns, families, or fallback entries.
|
|
87
|
+
|
|
88
|
+
The provider part of the key is the provider's [repository](providers/CUSTOM_PROVIDERS.md#registering-your-provider) identifier, so a custom provider registered as `:my_provider` reads entries under `my_provider/...`.
|
|
89
|
+
|
|
90
|
+
## Reasoning
|
|
91
|
+
|
|
92
|
+
A reasoning fragment is written exactly as you would write it in `model_options`, using the provider's own param names. Bedrock fragments spell out `additional_model_request_fields` and `inference_config` themselves.
|
|
93
|
+
|
|
94
|
+
When an agent sets `reasoning`, riffer picks the params for that level in this order:
|
|
95
|
+
|
|
96
|
+
1. The catalog entry's fragment for the level (by exact key, then alias).
|
|
97
|
+
2. The provider's default mapping, when the entry is missing or doesn't define the level.
|
|
98
|
+
3. If the provider has no default either, riffer raises `Riffer::ArgumentError` before the request:
|
|
99
|
+
`No reasoning mapping for my_provider/my-model at level :low. Add one in a catalog file (see docs/CATALOG.md).`
|
|
100
|
+
|
|
101
|
+
Riffer doesn't check whether a model supports a level. If the model rejects the params, you get the provider's error unchanged.
|
|
102
|
+
|
|
103
|
+
### Provider defaults
|
|
104
|
+
|
|
105
|
+
The defaults send the level name as the provider's effort value:
|
|
106
|
+
|
|
107
|
+
| Provider | `:off` | Other levels |
|
|
108
|
+
| -------------------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
|
109
|
+
| OpenAI, Azure OpenAI | `reasoning: "none"` | `reasoning: "<level>"` |
|
|
110
|
+
| OpenRouter | `reasoning: "none"` | `reasoning: "<level>"` |
|
|
111
|
+
| Anthropic | `thinking: { type: "disabled" }` | `thinking: { type: "adaptive" }, output_config: { effort: "<level>" }` |
|
|
112
|
+
| Amazon Bedrock | `additional_model_request_fields: { thinking: { type: "disabled" } }` | `additional_model_request_fields: { thinking: { type: "adaptive" }, output_config: { effort: "<level>" } }` |
|
|
113
|
+
| Gemini | `thinkingConfig: { thinkingLevel: "off" }` | `thinkingConfig: { thinkingLevel: "<level>" }` |
|
|
114
|
+
| Mock | nothing | nothing |
|
|
115
|
+
|
|
116
|
+
Add a catalog entry for models that need different params, for example:
|
|
117
|
+
|
|
118
|
+
- Claude 4.5 and earlier, which take a `budget_tokens` thinking budget instead of adaptive thinking.
|
|
119
|
+
- Gemini 2.5, which takes `thinkingBudget`.
|
|
120
|
+
- Non-Claude models on Amazon Bedrock (Nova, OpenAI), which use their own fields.
|
|
121
|
+
|
|
122
|
+
A [custom provider](providers/CUSTOM_PROVIDERS.md#reasoning-level) can supply its own default.
|
|
123
|
+
|
|
124
|
+
### Merging with model_options
|
|
125
|
+
|
|
126
|
+
The agent's `model_options` always win. Riffer deep-merges them over the fragment: nested objects merge key by key, and any other value (including arrays) replaces the fragment's. So you can override one field without repeating the rest:
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
class SummaryAgent < Riffer::Agent
|
|
130
|
+
model "anthropic/claude-haiku-4-5-20251001"
|
|
131
|
+
reasoning :low
|
|
132
|
+
model_options max_tokens: 8000 # keeps the catalog's thinking budget, replaces its max_tokens
|
|
133
|
+
end
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
Riffer doesn't check the merged result. If a partial override makes the request invalid, the provider rejects it.
|
|
137
|
+
|
|
138
|
+
## Pricing
|
|
139
|
+
|
|
140
|
+
```json
|
|
141
|
+
{
|
|
142
|
+
"pricing": {
|
|
143
|
+
"input": 3.0,
|
|
144
|
+
"output": 15.0,
|
|
145
|
+
"cache_read": 0.3,
|
|
146
|
+
"cache_write": 3.75
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
| Key | Description |
|
|
152
|
+
| ------------- | ------------------------------------------------------------------------------------------------ |
|
|
153
|
+
| `input` | Required. Price per **million** input tokens. Applies to the uncached portion of `input_tokens`. |
|
|
154
|
+
| `output` | Required. Price per **million** output tokens. |
|
|
155
|
+
| `cache_read` | Price per million cache-read tokens. When omitted, cache reads bill at the `input` rate. |
|
|
156
|
+
| `cache_write` | Price per million cache-write tokens. When omitted, cache writes bill at the `input` rate. |
|
|
157
|
+
|
|
158
|
+
Because the cache buckets are subsets of `input_tokens`, the cost formula subtracts them before applying the input rate:
|
|
159
|
+
|
|
160
|
+
```text
|
|
161
|
+
cost = (input − cache_read − cache_write) × input_rate
|
|
162
|
+
+ cache_read × cache_read_rate
|
|
163
|
+
+ cache_write × cache_write_rate
|
|
164
|
+
+ output × output_rate
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
(all rates ÷ 1,000,000; an unset cache rate falls back to `input_rate`.) Cost is for observability, not billing: it's a `Float`, and sub-cent rounding can accumulate over a long run. See [Messages → Token Usage Semantics](MESSAGES.md#token-usage-semantics) for how cost surfaces and aggregates.
|
|
168
|
+
|
|
169
|
+
A model with no catalog pricing carries no cost. The deprecated [`config.pricing`](CONFIGURATION.md#pricing-deprecated) is still read as a fallback; catalog pricing wins when both price a model.
|
|
170
|
+
|
|
171
|
+
## Layering files
|
|
172
|
+
|
|
173
|
+
Later files override earlier ones, so you can keep shared data in one file and local overrides in another:
|
|
174
|
+
|
|
175
|
+
- For an entry with the same key, a later file replaces whole sections (`pricing`, `reasoning`) and leaves the others alone.
|
|
176
|
+
- `aliases` arrays are combined, so a later file can add aliases without repeating earlier ones.
|
|
177
|
+
- If a later file lists an alias an earlier file gave to a different entry, the later file wins.
|
|
178
|
+
|
|
179
|
+
```json
|
|
180
|
+
{
|
|
181
|
+
"version": 1,
|
|
182
|
+
"models": {
|
|
183
|
+
"anthropic/claude-haiku-4-5-20251001": {
|
|
184
|
+
"pricing": { "input": 0.8, "output": 4.0 }
|
|
185
|
+
},
|
|
186
|
+
"openai/gpt-5.1": {
|
|
187
|
+
"aliases": ["azure_openai/my-gpt-prod"]
|
|
188
|
+
}
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Listed after the first example, this file replaces Haiku's pricing but keeps its reasoning, and routes an Azure deployment to the `openai/gpt-5.1` entry.
|
|
194
|
+
|
|
195
|
+
## Validation
|
|
196
|
+
|
|
197
|
+
Loading raises `Riffer::ArgumentError`, naming the file and the model key, when:
|
|
198
|
+
|
|
199
|
+
- The file is missing or isn't valid JSON, or `version` is missing or unsupported.
|
|
200
|
+
- The file has keys other than `version` and `models`, or an entry has keys other than `aliases`, `pricing`, and `reasoning`.
|
|
201
|
+
- A model key or alias isn't in `provider/model` form, or `aliases` isn't an array of strings.
|
|
202
|
+
- `pricing` is missing `input` or `output`, has unknown keys, or has a rate that isn't a non-negative number.
|
|
203
|
+
- A reasoning level isn't one of `off`, `low`, `medium`, `high`, `xhigh`, `max`, or its value isn't an object.
|
|
204
|
+
- Two entries in the same file list the same alias.
|
|
205
|
+
- After all files are layered, an alias is also a model key.
|
data/docs/CONFIGURATION.md
CHANGED
|
@@ -224,39 +224,38 @@ Every download in step 3 or 4 is gated by `allow_downloads`; with it `false` (th
|
|
|
224
224
|
|
|
225
225
|
A file that fails resolution raises a `Riffer::FileError` subclass — `Riffer::FileUnsupportedError`, `Riffer::FileDownloadsDisabledError`, `Riffer::TooManyFilesError`, `Riffer::FileChecksumMismatchError`, `Riffer::FileTooLargeError`, `Riffer::FileDownloadError`, or `Riffer::FileEncodingError` — so callers can `rescue Riffer::FileError` for any attachment problem, or a specific subclass to handle one case.
|
|
226
226
|
|
|
227
|
-
###
|
|
227
|
+
### Model Catalog
|
|
228
228
|
|
|
229
|
-
|
|
229
|
+
List JSON catalog files that hold per-model reasoning mappings and pricing. Riffer ships no catalog, so without one, `reasoning` uses each provider's default mapping, and calls carry no cost unless the deprecated [`config.pricing`](#pricing-deprecated) prices the model.
|
|
230
230
|
|
|
231
231
|
```ruby
|
|
232
232
|
Riffer.configure do |config|
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
233
|
+
config.catalog_files = ["config/riffer/models.json"]
|
|
234
|
+
end
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
| Option | Description |
|
|
238
|
+
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
|
|
239
|
+
| `catalog_files` | Array of paths (Strings or Pathnames), loaded in order; later files override earlier ones. Built at the end of `Riffer.configure`. |
|
|
240
|
+
|
|
241
|
+
See [Model Catalog](CATALOG.md) for the file format, lookup, layering, and validation.
|
|
236
242
|
|
|
237
|
-
|
|
238
|
-
|
|
243
|
+
### Pricing (deprecated)
|
|
244
|
+
|
|
245
|
+
`config.pricing` is deprecated and will be removed in a future release. Move rates into a [catalog file](CATALOG.md#pricing); the first `set` call prints a deprecation warning. Riffer still reads these rates as a fallback, and catalog pricing wins when both price a model.
|
|
246
|
+
|
|
247
|
+
```ruby
|
|
248
|
+
# before
|
|
249
|
+
Riffer.configure do |config|
|
|
250
|
+
config.pricing.set("anthropic/claude-sonnet-4-6", input: 3.0, output: 15.0)
|
|
239
251
|
end
|
|
240
252
|
```
|
|
241
253
|
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
| `models` | A `provider/model` id (e.g. `"openai/gpt-4"`) — the same string you pass to `model` — or an array of ids that share one set of rates. No alias matching; raises on a malformed id. |
|
|
245
|
-
| `input:` | Price per **million** input tokens. Required. Applies to the uncached portion of `input_tokens`. |
|
|
246
|
-
| `output:` | Price per **million** output tokens. Required. |
|
|
247
|
-
| `cache_read:` | Price per million cache-read tokens. Optional — when omitted, cache reads bill at the `input:` rate. |
|
|
248
|
-
| `cache_write:` | Price per million cache-write tokens. Optional — when omitted, cache writes bill at the `input:` rate. |
|
|
249
|
-
|
|
250
|
-
Because the cache buckets are subsets of `input_tokens`, the cost formula subtracts them before applying the input rate:
|
|
251
|
-
|
|
252
|
-
```text
|
|
253
|
-
cost = (input − cache_read − cache_write) × input_rate
|
|
254
|
-
+ cache_read × cache_read_rate
|
|
255
|
-
+ cache_write × cache_write_rate
|
|
256
|
-
+ output × output_rate
|
|
254
|
+
```json
|
|
255
|
+
{ "version": 1, "models": { "anthropic/claude-sonnet-4-6": { "pricing": { "input": 3.0, "output": 15.0 } } } }
|
|
257
256
|
```
|
|
258
257
|
|
|
259
|
-
|
|
258
|
+
`set` takes a `provider/model` id or an array of ids, plus `input:`, `output:`, and optional `cache_read:` and `cache_write:` rates per million tokens. The rates and cost formula are the same as [catalog pricing](CATALOG.md#pricing).
|
|
260
259
|
|
|
261
260
|
### Message ID Strategy
|
|
262
261
|
|
|
@@ -297,6 +296,19 @@ class MyAgent < Riffer::Agent
|
|
|
297
296
|
end
|
|
298
297
|
```
|
|
299
298
|
|
|
299
|
+
### reasoning
|
|
300
|
+
|
|
301
|
+
Set how much the model thinks before it answers, with one setting that works on every provider:
|
|
302
|
+
|
|
303
|
+
```ruby
|
|
304
|
+
class MyAgent < Riffer::Agent
|
|
305
|
+
model 'anthropic/claude-sonnet-4-6'
|
|
306
|
+
reasoning :low # :off, :low, :medium, :high, :xhigh, or :max
|
|
307
|
+
end
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
Riffer maps the level to the provider's own params, from a [catalog](CATALOG.md#reasoning) entry or the provider's default. `model_options` are deep-merged over those params and win. See [Agents — reasoning](AGENTS.md#reasoning).
|
|
311
|
+
|
|
300
312
|
## Common Model Options
|
|
301
313
|
|
|
302
314
|
### OpenAI
|
data/docs/MESSAGES.md
CHANGED
|
@@ -91,7 +91,7 @@ tool_call.to_h # => {call_id: "call_123", name: "weather_tool", argu
|
|
|
91
91
|
|
|
92
92
|
The cache buckets are subsets of `input_tokens`, never additions to it — summing `input_tokens + cache_read_tokens` double-counts. `total_tokens` (input + output) matches the totals providers report on their dashboards.
|
|
93
93
|
|
|
94
|
-
- `cost` — the computed cost of the call, set when pricing
|
|
94
|
+
- `cost` — the computed cost of the call, set when a [catalog](CATALOG.md#pricing) or the deprecated [`config.pricing`](CONFIGURATION.md#pricing-deprecated) prices the model in use; `nil` when the model is unpriced. It's for observability, not billing. Run-level usage sums per-call costs through `TokenUsage#+`, so `response.token_usage.cost` is the total spend across the run — but the sum is `nil` if any call in the run used an unpriced model, rather than silently under-reporting.
|
|
95
95
|
|
|
96
96
|
#### Finish Reasons
|
|
97
97
|
|
data/docs/SERIALIZATION.md
CHANGED
|
@@ -46,6 +46,7 @@ The session is used **as-is**: the rebuilt agent does not prepend anything to it
|
|
|
46
46
|
max_steps: 8, # integer; -1 = unlimited (see below)
|
|
47
47
|
structured_output: { type: "object", … }, # JSON Schema, or null
|
|
48
48
|
structured_output_strategy: :native, # :native or :prompted ("native"/"prompted" over JSON)
|
|
49
|
+
reasoning: :low, # reasoning level, or null
|
|
49
50
|
tools: [ { name:, description:, parameters_schema:, timeout: }, … ]
|
|
50
51
|
}
|
|
51
52
|
```
|
data/docs/TRACING.md
CHANGED
|
@@ -142,6 +142,7 @@ Usage on this span is the run total, aggregated across every step. See [Token us
|
|
|
142
142
|
| `gen_ai.request.presence_penalty` | float | When the caller set it |
|
|
143
143
|
| `gen_ai.request.seed` | int | When the caller set it |
|
|
144
144
|
| `gen_ai.request.stop_sequences` | string[] | When the caller set it |
|
|
145
|
+
| `riffer.request.reasoning_level` | string | When the agent sets [`reasoning`](AGENTS.md#reasoning) |
|
|
145
146
|
| `gen_ai.usage.input_tokens` | int | When the provider reported usage |
|
|
146
147
|
| `gen_ai.usage.output_tokens` | int | When the provider reported usage |
|
|
147
148
|
| `gen_ai.usage.cache_read.input_tokens` | int | When the provider reported cache reads |
|
|
@@ -252,7 +253,7 @@ Riffer normalizes this across providers, so the number may differ from a provide
|
|
|
252
253
|
|
|
253
254
|
### Cost
|
|
254
255
|
|
|
255
|
-
`riffer.cost` is the modeled cost of one call (on a `chat` span) or a whole run (on the `invoke_agent` span). It lives in Riffer's own namespace because the GenAI semantic conventions define no cost attribute by design — Riffer never squats `gen_ai.*` for it. The attribute appears only when
|
|
256
|
+
`riffer.cost` is the modeled cost of one call (on a `chat` span) or a whole run (on the `invoke_agent` span). It lives in Riffer's own namespace because the GenAI semantic conventions define no cost attribute by design — Riffer never squats `gen_ai.*` for it. The attribute appears only when a [catalog](CATALOG.md#pricing) or the deprecated [`config.pricing`](CONFIGURATION.md#pricing-deprecated) prices the model in use: Riffer ships no price table and never guesses, so an unpriced model simply carries no `riffer.cost`.
|
|
256
257
|
|
|
257
258
|
The value is **unitless on the wire** — Riffer attaches no currency. It is the sum of the per-token rates you configured, in whatever currency you expressed them, so a `riffer.cost` of `0.0123` means 0.0123 of that unit. The raw float is emitted unrounded; round for display in your backend, not before.
|
|
258
259
|
|
|
@@ -189,6 +189,8 @@ end
|
|
|
189
189
|
|
|
190
190
|
The reasoning is kept on the assistant message as [reasoning parts](../MESSAGES.md#reasoning), whether you call `generate` or `stream`. Read it with `response.reasoning`, or as plain text with `reasoning_text` on the message. Bedrock sometimes redacts part of Claude's reasoning. Those parts are stored and sent back like any other, but they have no readable text, so `reasoning_text` leaves them out.
|
|
191
191
|
|
|
192
|
+
The agent-level [`reasoning`](../AGENTS.md#reasoning) setting defaults to Claude's fields inside `additional_model_request_fields`: adaptive thinking at the level's effort, and `thinking: {type: "disabled"}` for `:off`. Other model families (Nova, OpenAI) and Claude 4.5 and earlier need a [catalog](../CATALOG.md#reasoning) entry with their own fields.
|
|
193
|
+
|
|
192
194
|
### Reasoning Replay
|
|
193
195
|
|
|
194
196
|
Riffer sends the reasoning back to Bedrock on every later turn. How much of it Claude uses depends on the model; some only use the reasoning from the current tool-calling turn. For tool calls, sending it back is required: when thinking is enabled, Claude needs its earlier reasoning back to carry on after a tool result. You don't have to do anything; it happens as long as the assistant messages stay in the history.
|
data/docs/providers/ANTHROPIC.md
CHANGED
|
@@ -170,6 +170,8 @@ class ReasoningAgent < Riffer::Agent
|
|
|
170
170
|
end
|
|
171
171
|
```
|
|
172
172
|
|
|
173
|
+
The agent-level [`reasoning`](../AGENTS.md#reasoning) setting defaults to adaptive thinking at the level's effort (`thinking: {type: "adaptive"}, output_config: {effort: "low"}`), and `:off` sends `thinking: {type: "disabled"}`. Claude 4.5 and earlier need a `budget_tokens` mapping in a [catalog](../CATALOG.md#reasoning) entry instead.
|
|
174
|
+
|
|
173
175
|
When streaming with extended thinking enabled, you'll receive `ReasoningDelta` events containing the model's thought process, followed by a `ReasoningDone` event when thinking completes:
|
|
174
176
|
|
|
175
177
|
```ruby
|
|
@@ -87,6 +87,8 @@ Enables extended thinking (for supported models):
|
|
|
87
87
|
model_options reasoning: 'medium' # 'low', 'medium', or 'high'
|
|
88
88
|
```
|
|
89
89
|
|
|
90
|
+
The agent-level [`reasoning`](../AGENTS.md#reasoning) setting maps to this option the same way as on the [OpenAI provider](OPENAI.md#reasoning).
|
|
91
|
+
|
|
90
92
|
Reasoning is captured and replayed on later turns exactly as with the [OpenAI provider](OPENAI.md#reasoning-replay), except that parts are tagged `format: "azure-openai-v1"`. Azure only accepts reasoning produced by the same resource, so parts from the OpenAI provider are never sent to Azure, and Azure's are never sent to OpenAI.
|
|
91
93
|
|
|
92
94
|
### structured_output
|
|
@@ -157,7 +157,7 @@ Register your provider under an identifier.
|
|
|
157
157
|
Riffer::Providers::Repository.register(:my_provider) { Riffer::Providers::MyProvider }
|
|
158
158
|
```
|
|
159
159
|
|
|
160
|
-
The block resolves the provider class lazily, so it need not be loaded at registration time. Registration is idempotent — re-registering the same identifier replaces the previous factory — and a custom registration takes precedence over a built-in sharing the identifier, which lets you route an existing prefix (e.g. `openai`) through your own backend. Both `find` and `key_for` (used for
|
|
160
|
+
The block resolves the provider class lazily, so it need not be loaded at registration time. Registration is idempotent — re-registering the same identifier replaces the previous factory — and a custom registration takes precedence over a built-in sharing the identifier, which lets you route an existing prefix (e.g. `openai`) through your own backend. Both `find` and `key_for` (used for [catalog](../CATALOG.md) keys) honor the registration. Remove one with `Riffer::Providers::Repository.unregister(:my_provider)`.
|
|
161
161
|
|
|
162
162
|
Register during application boot — from a Rails initializer or equivalent — before you start handling requests. The registry is not synchronized for concurrent mutation, so treat registration as a one-time setup step rather than something you do from a live request path.
|
|
163
163
|
|
|
@@ -308,6 +308,20 @@ yielder << Riffer::StreamEvents::ReasoningDone.new(part)
|
|
|
308
308
|
|
|
309
309
|
If your adapter surfaces reasoning text but cannot yet replay it, call `yield_reasoning_done(yielder, text)` instead. It wraps the text in a `:text` part with no `format`, which persists for display and is skipped on replay.
|
|
310
310
|
|
|
311
|
+
## Reasoning Level
|
|
312
|
+
|
|
313
|
+
When an agent sets [`reasoning`](../AGENTS.md#reasoning), riffer resolves the level to `model_options` params before calling `build_request_params`, so your provider receives ordinary options and never sees the level itself. It uses the model's [catalog](../CATALOG.md#reasoning) entry under your registration key first, then your provider's default. Override the private `default_reasoning_options` to supply one:
|
|
314
|
+
|
|
315
|
+
```ruby
|
|
316
|
+
private
|
|
317
|
+
|
|
318
|
+
def default_reasoning_options(level)
|
|
319
|
+
{ reasoning_effort: level.to_s }
|
|
320
|
+
end
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
The base class returns `nil`, which makes riffer raise `Riffer::ArgumentError` for any level the catalog doesn't map.
|
|
324
|
+
|
|
311
325
|
## Tags
|
|
312
326
|
|
|
313
327
|
Every agent and judge call passes a `:tags` option: a flat `String => String` hash of the caller's [per-call tags](../AGENTS.md#per-call-tags) plus the [default tags](../AGENTS.md#default-tags). `kind` (`"agent"` or `"judge"`) and `agent` (the agent or evaluator identifier) tell you who the call is on behalf of:
|
data/docs/providers/GEMINI.md
CHANGED
|
@@ -80,6 +80,16 @@ Nucleus sampling:
|
|
|
80
80
|
model_options topP: 0.9
|
|
81
81
|
```
|
|
82
82
|
|
|
83
|
+
### thinkingConfig
|
|
84
|
+
|
|
85
|
+
Controls thinking, passed through as Gemini expects:
|
|
86
|
+
|
|
87
|
+
```ruby
|
|
88
|
+
model_options thinkingConfig: {thinkingLevel: "low"}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
The agent-level [`reasoning`](../AGENTS.md#reasoning) setting writes `thinkingConfig: {thinkingLevel: "<level>"}` by default, including `"off"` for `:off`, which Gemini rejects. Gemini 2.5 models take `thinkingBudget` instead; map them in a [catalog](../CATALOG.md#reasoning) entry.
|
|
92
|
+
|
|
83
93
|
## Usage
|
|
84
94
|
|
|
85
95
|
### Basic Generation
|
|
@@ -64,6 +64,8 @@ agent.session.messages.last.reasoning_text # => "The user wants the answer."
|
|
|
64
64
|
|
|
65
65
|
When streaming, each part is emitted as a `ReasoningDelta` (only when it carries `text`) followed by a `ReasoningDone` carrying the part, ahead of the text events.
|
|
66
66
|
|
|
67
|
+
An agent's [`reasoning`](../AGENTS.md#reasoning) level needs no catalog entry on the mock provider: it maps every level to no params, so `calls` show only your `model_options` unless a [catalog](../CATALOG.md) entry under `mock/...` maps the level.
|
|
68
|
+
|
|
67
69
|
## Queueing Multiple Responses
|
|
68
70
|
|
|
69
71
|
Responses are consumed in order:
|
data/docs/providers/OPENAI.md
CHANGED
|
@@ -82,6 +82,8 @@ model_options reasoning: 'medium' # 'low', 'medium', or 'high'
|
|
|
82
82
|
|
|
83
83
|
When reasoning is enabled, you'll receive `ReasoningDelta` and `ReasoningDone` events during streaming.
|
|
84
84
|
|
|
85
|
+
The agent-level [`reasoning`](../AGENTS.md#reasoning) setting writes this same option by default: `reasoning :low` sends `reasoning: 'low'`, and `:off` sends `'none'`. A [catalog](../CATALOG.md#reasoning) entry can map a level differently.
|
|
86
|
+
|
|
85
87
|
#### Reasoning Replay
|
|
86
88
|
|
|
87
89
|
Each `reasoning` item in a Responses API output becomes [reasoning parts](../MESSAGES.md#reasoning) on the assistant message, on both `generate_text` and `stream_text`, all tagged `format: "openai-v1"` and carrying the item's `id`:
|
|
@@ -80,6 +80,8 @@ model_options reasoning: {effort: 'medium', max_tokens: 5000}
|
|
|
80
80
|
|
|
81
81
|
Streaming yields `Riffer::StreamEvents::ReasoningDelta` and `ReasoningDone` events when the model returns reasoning content.
|
|
82
82
|
|
|
83
|
+
The agent-level [`reasoning`](../AGENTS.md#reasoning) setting writes this same option by default: `reasoning :low` sends `reasoning: 'low'`, and `:off` sends `'none'`. A [catalog](../CATALOG.md#reasoning) entry can map a level differently.
|
|
84
|
+
|
|
83
85
|
### provider (routing preferences)
|
|
84
86
|
|
|
85
87
|
Pin which upstream provider OpenRouter should use, set allow/deny lists, or prefer a sort order:
|
data/docs-site/manifest.yml
CHANGED
|
@@ -45,6 +45,9 @@ groups:
|
|
|
45
45
|
- source: CONFIGURATION.md
|
|
46
46
|
slug: configuration
|
|
47
47
|
description: Framework configuration
|
|
48
|
+
- source: CATALOG.md
|
|
49
|
+
slug: catalog
|
|
50
|
+
description: Reasoning mappings and pricing per model
|
|
48
51
|
- source: EVALS.md
|
|
49
52
|
slug: evals
|
|
50
53
|
description: Evaluating agent quality
|
data/lib/riffer/agent/config.rb
CHANGED
|
@@ -5,6 +5,8 @@ class Riffer::Agent::Config
|
|
|
5
5
|
DEFAULT_MAX_STEPS = 16 #: Integer
|
|
6
6
|
STRUCTURED_OUTPUT_STRATEGIES = %i[native prompted].freeze #: Array[Symbol]
|
|
7
7
|
|
|
8
|
+
REASONING_LEVELS = %i[off low medium high xhigh max].freeze #: Array[Symbol]
|
|
9
|
+
|
|
8
10
|
# @rbs @tool_runtime: (singleton(Riffer::Tools::Runtime) | Riffer::Tools::Runtime | Proc)?
|
|
9
11
|
|
|
10
12
|
attr_reader :identifier #: String? # @dynamic identifier
|
|
@@ -14,6 +16,7 @@ class Riffer::Agent::Config
|
|
|
14
16
|
attr_reader :structured_output #: Riffer::Params? # @dynamic structured_output
|
|
15
17
|
attr_reader :structured_output_strategy #: Symbol # @dynamic structured_output_strategy
|
|
16
18
|
attr_accessor :max_steps #: Numeric? # @dynamic max_steps, max_steps=
|
|
19
|
+
attr_reader :reasoning #: Symbol? # @dynamic reasoning
|
|
17
20
|
attr_accessor :tools_config #: (Array[singleton(Riffer::Tool)] | Proc)? # @dynamic tools_config, tools_config=
|
|
18
21
|
attr_reader :mcp_configs #: Array[Hash[Symbol, untyped]] # @dynamic mcp_configs
|
|
19
22
|
attr_accessor :skills_config #: Riffer::Skills::Config? # @dynamic skills_config, skills_config=
|
|
@@ -28,6 +31,7 @@ class Riffer::Agent::Config
|
|
|
28
31
|
# ?structured_output: Riffer::Params?,
|
|
29
32
|
# ?structured_output_strategy: Symbol,
|
|
30
33
|
# ?max_steps: Numeric?,
|
|
34
|
+
# ?reasoning: (Symbol | String)?,
|
|
31
35
|
# ?tools_config: (Array[singleton(Riffer::Tool)] | Proc)?,
|
|
32
36
|
# ?mcp_configs: Array[Hash[Symbol, untyped]],
|
|
33
37
|
# ?tool_runtime: (singleton(Riffer::Tools::Runtime) | Riffer::Tools::Runtime | Proc)?,
|
|
@@ -42,6 +46,7 @@ class Riffer::Agent::Config
|
|
|
42
46
|
structured_output: nil,
|
|
43
47
|
structured_output_strategy: :native,
|
|
44
48
|
max_steps: DEFAULT_MAX_STEPS,
|
|
49
|
+
reasoning: nil,
|
|
45
50
|
tools_config: nil,
|
|
46
51
|
mcp_configs: [],
|
|
47
52
|
tool_runtime: nil,
|
|
@@ -59,6 +64,7 @@ class Riffer::Agent::Config
|
|
|
59
64
|
self.instructions = instructions
|
|
60
65
|
self.structured_output = structured_output
|
|
61
66
|
self.structured_output_strategy = structured_output_strategy
|
|
67
|
+
self.reasoning = reasoning
|
|
62
68
|
self.tool_runtime = tool_runtime if tool_runtime
|
|
63
69
|
end
|
|
64
70
|
|
|
@@ -89,6 +95,17 @@ class Riffer::Agent::Config
|
|
|
89
95
|
@structured_output_strategy = value
|
|
90
96
|
end
|
|
91
97
|
|
|
98
|
+
#--
|
|
99
|
+
#: ((Symbol | String)?) -> Symbol?
|
|
100
|
+
def reasoning=(value)
|
|
101
|
+
level = value&.to_s&.to_sym
|
|
102
|
+
unless level.nil? || REASONING_LEVELS.include?(level)
|
|
103
|
+
raise Riffer::ArgumentError, "reasoning must be one of #{REASONING_LEVELS.join(', ')}, got #{value.inspect}"
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
@reasoning = level
|
|
107
|
+
end
|
|
108
|
+
|
|
92
109
|
#--
|
|
93
110
|
#: () -> (singleton(Riffer::Tools::Runtime) | Riffer::Tools::Runtime | Proc)
|
|
94
111
|
def tool_runtime
|
data/lib/riffer/agent/run.rb
CHANGED
|
@@ -354,6 +354,8 @@ module Riffer::Agent::Run
|
|
|
354
354
|
def merged_model_options(agent, tags = {})
|
|
355
355
|
opts = agent.config.model_options.dup
|
|
356
356
|
opts[:structured_output] = agent.structured_output if agent.structured_output
|
|
357
|
+
reasoning = agent.config.reasoning
|
|
358
|
+
opts[:riffer_reasoning_level] = reasoning if reasoning
|
|
357
359
|
# Providers extract :tags for native request metadata; it never reaches an
|
|
358
360
|
# SDK call verbatim.
|
|
359
361
|
opts[:tags] = tags unless tags.empty?
|
|
@@ -28,6 +28,7 @@ module Riffer::Agent::Serializer
|
|
|
28
28
|
max_steps: encode_max_steps(config.max_steps),
|
|
29
29
|
structured_output: config.structured_output&.to_json_schema(strict: false),
|
|
30
30
|
structured_output_strategy: config.structured_output_strategy,
|
|
31
|
+
reasoning: config.reasoning,
|
|
31
32
|
tools: agent.tools.map { |tool_class| tool_descriptor(tool_class) },
|
|
32
33
|
}
|
|
33
34
|
end
|
|
@@ -79,6 +80,7 @@ module Riffer::Agent::Serializer
|
|
|
79
80
|
structured_output: decode_structured_output(hash[:structured_output]),
|
|
80
81
|
structured_output_strategy: (hash[:structured_output_strategy] || :native).to_sym,
|
|
81
82
|
max_steps: decode_max_steps(hash),
|
|
83
|
+
reasoning: hash[:reasoning],
|
|
82
84
|
tools_config: tools,
|
|
83
85
|
} #: Hash[Symbol, untyped]
|
|
84
86
|
# Config#tool_runtime= rejects nil.
|
data/lib/riffer/agent.rb
CHANGED
|
@@ -73,6 +73,13 @@ class Riffer::Agent
|
|
|
73
73
|
config.max_steps = value.first
|
|
74
74
|
end
|
|
75
75
|
|
|
76
|
+
#--
|
|
77
|
+
#: (?(Symbol | String)?) -> Symbol?
|
|
78
|
+
def self.reasoning(value = nil)
|
|
79
|
+
config.reasoning = value if value
|
|
80
|
+
config.reasoning
|
|
81
|
+
end
|
|
82
|
+
|
|
76
83
|
#--
|
|
77
84
|
#: (?(Array[singleton(Riffer::Tool)] | Proc)?) -> (Array[singleton(Riffer::Tool)] | Proc)?
|
|
78
85
|
def self.uses_tools(value = nil)
|