axn-ruby_llm 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +179 -0
- data/README.md +67 -35
- data/lib/axn/ruby_llm/ask.rb +130 -112
- data/lib/axn/ruby_llm/rspec.rb +23 -43
- data/lib/axn/ruby_llm/tool_adapter.rb +100 -75
- data/lib/axn/ruby_llm/version.rb +1 -1
- data/lib/axn/ruby_llm.rb +8 -40
- metadata +22 -8
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1c013cfd72ea5be2662e93875a9f0f01b8d8761cfe008aabed920bb3ee4ab8d1
|
|
4
|
+
data.tar.gz: 2977d6ab65433f53e59f019c796a9a10adc016efa290990ee3596d4b0680df4f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 532fa712897c472636dff0085d9df2b9696f0a355e88db3f18978da9128b6d4c9d0a7251170b8f84b9354e55c7fd83406b0d0ab7863b4291b1f255dfc9d7c5b1
|
|
7
|
+
data.tar.gz: 8f77714a318a5bbf674d467bd2a0d5bb4b82c6f4d199d0faf84dc82a5d298fba9825aa083d2a71ca79c89e67d147cf150d02f2d044d6a573f88a936602782173
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,184 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.3.0] - 2026-09-22
|
|
4
|
+
|
|
5
|
+
RubyLLM 2.0 is a breaking rewrite (renamed Tool DSL, restructured error hierarchy, a usage ledger
|
|
6
|
+
replacing per-message token/cost readers, `RubyLLM::Schema` moved to the separate `schematist` gem)
|
|
7
|
+
with no compatibility shim for 1.x, so this is a hard cut: `ruby_llm ">= 2.0", "< 3.0"` replaces the
|
|
8
|
+
previous `">= 1.15", "< 2.0"` floor, and 1.x is no longer supported by this gem.
|
|
9
|
+
|
|
10
|
+
This release was developed against ruby_llm's `2.0.0.rc4` release candidate (as `0.3.0.rc1`, never
|
|
11
|
+
published as a final release) and raised to the `2.0.0` GA floor once RubyLLM released it
|
|
12
|
+
(2026-09-18) — re-diffed first and confirmed every path this gem touches (`Tool`, the
|
|
13
|
+
Gemini/Anthropic/Chat-Completions protocols) is byte-identical between rc4 and GA.
|
|
14
|
+
|
|
15
|
+
### Breaking
|
|
16
|
+
|
|
17
|
+
- **Tool DSL renamed to match `RubyLLM::Tool` 2.0.** Internal to `Axn::RubyLLM.wrap`'s generated
|
|
18
|
+
tool class (`params`/`with_params` → `parameters`/`provider_options`); not observable unless you
|
|
19
|
+
subclassed or introspected a wrapped tool directly. `Tool#params_schema` is now
|
|
20
|
+
`#parameters_schema`, and `Tool#parameters` (the empty `Parameter` DSL reader) is now
|
|
21
|
+
`#declared_parameters`.
|
|
22
|
+
- **`provider_params:` renamed to `provider_options:`** (the `wrap` kwarg and the
|
|
23
|
+
`configure(:ruby_llm)` setting), matching RubyLLM 2.0's own `Tool.provider_options`. A leftover
|
|
24
|
+
`provider_params:` raises `ArgumentError` with a pointer, the same hard-error pattern as the
|
|
25
|
+
pre-existing `render_as:` → `present_as:` rename.
|
|
26
|
+
- **`halt_after:` removed outright** — the `wrap` kwarg, the `configure(:ruby_llm)` setting, and the
|
|
27
|
+
`RubyLLM::Tool::Halt`-wrapping behavior. RubyLLM 2.0 deleted `Tool::Halt`; the conversation loop is
|
|
28
|
+
now caller-controlled (`chat.step` / `chat.complete?`). See RubyLLM's Agentic Workflows guide for
|
|
29
|
+
the replacement pattern.
|
|
30
|
+
- **`json:` input removed from `Axn::RubyLLM::Ask`.** There is no protocol-agnostic JSON-mode
|
|
31
|
+
request field in RubyLLM 2.0 (OpenAI's `response_format: {type: "json_object"}` is Chat
|
|
32
|
+
Completions-only, and OpenAI now defaults to the Responses API). `schema:` is the replacement —
|
|
33
|
+
it now additionally accepts a plain Hash (already supported, previously undocumented) or an Axn
|
|
34
|
+
class (new: forwards `axn_class.output_schema`), alongside a `Schematist::Schema` class/instance.
|
|
35
|
+
The Axn-class form makes adjustments axn's own `output_schema` has no reason to make on its own,
|
|
36
|
+
each confirmed against a real provider (Anthropic, live) rather than assumed from docs alone
|
|
37
|
+
(which turned out unreliable on this point — a docs summary claimed `minLength` was also
|
|
38
|
+
unsupported by Anthropic; a flat schema with `minLength: 1` on a String field succeeded live
|
|
39
|
+
regardless, so only what's actually confirmed failing is adjusted):
|
|
40
|
+
- Injects `additionalProperties: false` on every fixed-shape object node — required
|
|
41
|
+
*unconditionally* by Anthropic's structured output (confirmed live: without it, the request
|
|
42
|
+
fails with `"For 'object' type, 'additionalProperties' must be explicitly set to false"`) and
|
|
43
|
+
by OpenAI's strict mode — while leaving a map's own `additionalProperties` (its value schema)
|
|
44
|
+
untouched.
|
|
45
|
+
- Strips `minProperties`/`maxProperties` from every object *schema node* — axn emits
|
|
46
|
+
`minProperties: 1` by default on a nested fixed-shape `Hash` field, and Anthropic's schema
|
|
47
|
+
validator rejects it outright (confirmed live: `"For 'object' type, property 'minProperties' is
|
|
48
|
+
not supported"`). Gated on the node actually being an object schema (declaring `type`), not on
|
|
49
|
+
the key name alone — a code-review catch: the recursive pass walks every Hash in the schema,
|
|
50
|
+
but `properties` is a name-to-schema *map*, so an Axn exposing a field literally named
|
|
51
|
+
`minProperties` would otherwise have that field silently dropped while `required` still named
|
|
52
|
+
it, producing an invalid schema (confirmed live before the fix).
|
|
53
|
+
- Pins `strict: false`, since RubyLLM's own strict-inference would otherwise turn on for the
|
|
54
|
+
common case of every property being required, and OpenAI's *full* strict mode additionally
|
|
55
|
+
requires every property to appear in `required` even when conceptually optional, which axn's
|
|
56
|
+
reflection doesn't promise.
|
|
57
|
+
|
|
58
|
+
**Known limitation, confirmed live, not worked around:** a `Hash` map field (`type: Hash, of:
|
|
59
|
+
{...}`) doesn't survive this path against every provider. Anthropic's structured output rejects
|
|
60
|
+
`additionalProperties` set to anything but the literal `false` (confirmed live:
|
|
61
|
+
`"'additionalProperties: object' is not supported. Please set 'additionalProperties' to false"`),
|
|
62
|
+
and OpenAI's strict mode has the same restriction by design — neither provider's structured-output
|
|
63
|
+
feature represents dynamic/arbitrary keys, only a fixed shape, so there's no schema-legal way to
|
|
64
|
+
route around it. Use a fixed-shape `Hash` (`shape:`) instead, or pass your own schema for that
|
|
65
|
+
provider if you specifically need a map. Pass a `Schematist::Schema` instead of an Axn class if you
|
|
66
|
+
need OpenAI's *full* strict-mode guarantee.
|
|
67
|
+
- **`RubyLLM::Schema` is `Schematist::Schema` in RubyLLM 2.0** — this gem does not shim the old
|
|
68
|
+
name. Any `schema:` class you declare must subclass `Schematist::Schema`.
|
|
69
|
+
- **`Axn::RubyLLM.configuration` / `.reset_configuration!` removed** — these were deprecated in
|
|
70
|
+
0.2.0 for `.config` / `.reset_config!` and scheduled for removal in 0.3.0 regardless of the
|
|
71
|
+
RubyLLM 2.0 port; landing in the same release since both are breaking-change cuts.
|
|
72
|
+
|
|
73
|
+
### Changed
|
|
74
|
+
|
|
75
|
+
- **Token and cost exposures now read RubyLLM 2.0's own usage ledger** (`Chat#tokens` /
|
|
76
|
+
`Chat#cost`) instead of summing `Message#input_tokens`/`#cache_read_tokens`/etc. across
|
|
77
|
+
`chat.messages` by hand. Exposure names (`input_tokens`, `output_tokens`, `cache_read_tokens`,
|
|
78
|
+
`cache_write_tokens`, `prompt_tokens`, `cost`, `cost_breakdown`) are unchanged, but the totals now
|
|
79
|
+
additionally include failed retries and no-message provider attempts within a tool loop — strictly
|
|
80
|
+
more accurate than the previous per-message sum, which only ever saw messages that made it onto
|
|
81
|
+
the chat.
|
|
82
|
+
- **`Message#model_id` reads are now `Message#model`**; `response_model` in OTel attributes follows.
|
|
83
|
+
- **`with_params(temperature:)` is now `with_temperature(temperature)`.** RubyLLM 2.0 sends the
|
|
84
|
+
temperature you set verbatim; 1.x sometimes rewrote it to `1.0` or dropped it for models that
|
|
85
|
+
didn't support it. A model that now rejects your value raises `RubyLLM::BadRequestError` instead
|
|
86
|
+
of silently ignoring it.
|
|
87
|
+
- **`Ask`'s `KNOWN_ERROR_CLASSES` grew three entries** — `RubyLLM::ModelRegistryError`,
|
|
88
|
+
`RubyLLM::PendingToolCallsError`, `RubyLLM::CancelledError` — new in RubyLLM 2.0, so their
|
|
89
|
+
messages surface instead of falling into the generic `"LLM request failed"` bucket.
|
|
90
|
+
- **Removed ~170 lines of Gemini schema workarounds** (`normalize_nullable_types`,
|
|
91
|
+
`annotate_object_constraints`, and six helpers) from the tool adapter. RubyLLM 2.0's Gemini
|
|
92
|
+
protocol reads a tool's schema via `parametersJsonSchema` — the wire form verbatim — rather than
|
|
93
|
+
rebuilding each property from the fixed whitelist that dropped array-valued `type`,
|
|
94
|
+
`additionalProperties`, and `min`/`maxProperties`. A wrapped tool's schema is now passed to every
|
|
95
|
+
provider unmodified.
|
|
96
|
+
- **The gemspec now declares `faraday` directly** (previously arrived only transitively through
|
|
97
|
+
`ruby_llm`), since `ask.rb` rescues `::Faraday::Error` directly.
|
|
98
|
+
- **`axn` floor bumped to `0.1.0-alpha.6.1`** (from `0.1.0-alpha.6`). No new API is required — this
|
|
99
|
+
just picks up alpha.6.1's logger-raise `best_effort` hardening and `model:`/`Result#declared_fields`
|
|
100
|
+
fixes.
|
|
101
|
+
|
|
102
|
+
### Fixed
|
|
103
|
+
|
|
104
|
+
- **`stub_axn_ruby_llm`'s message double no longer needs `RubyLLM.models.find` stubbed.** The test
|
|
105
|
+
helper's `chat.cost`/`chat.tokens` stubs mirror the real 2.0 ledger reads directly, removing a
|
|
106
|
+
layer of indirection through a stubbed model registry lookup that the production code no longer
|
|
107
|
+
performs either.
|
|
108
|
+
|
|
109
|
+
## [0.2.1] - 2026-09-03
|
|
110
|
+
|
|
111
|
+
### Added
|
|
112
|
+
|
|
113
|
+
- **[FEAT] Every tool call is now stamped `invoked_via: :ruby_llm`** (PRO-3332), via
|
|
114
|
+
`Axn::Tools::Invoker.new(adapter: :ruby_llm)`. No adapter-side work required — the stamp applies to
|
|
115
|
+
the wrapped Axn and any nested sub-axn or enqueued Sidekiq job for the life of the call tree, so a
|
|
116
|
+
Datadog dashboard (or any `Axn.config.on_exception`/tracing consumer reading the resolved
|
|
117
|
+
`invoked_via` dimension) can separate tool-driven traffic from an ordinary direct `.call`.
|
|
118
|
+
|
|
119
|
+
### Changed
|
|
120
|
+
|
|
121
|
+
- **`record_otel_attributes!` now uses `Axn::Extensions::Tracing.annotate_span`** instead of the
|
|
122
|
+
unreliable ambient `OpenTelemetry::Trace.current_span` lookup, which could disagree with the span
|
|
123
|
+
axn's own tracer actually opened (PRO-3278) and silently drop every `gen_ai.*` attribute. Requires
|
|
124
|
+
axn `>= 0.1.0-alpha.6`.
|
|
125
|
+
|
|
126
|
+
### Fixed
|
|
127
|
+
|
|
128
|
+
- **[BUGFIX] `reject_opaque_exposed_values` is now resolved per tool call instead of once at wrap
|
|
129
|
+
time.** The setting is `overridable:`, so a per-tool `configure(:ruby_llm) { |c| ... }` /
|
|
130
|
+
`tool ruby_llm: { ... }` bag or a gem-wide `Axn::RubyLLM.configure` assignment is meant to be what
|
|
131
|
+
a tool honors — but `wrap` resolved the value eagerly and the built tool class closed over that
|
|
132
|
+
Boolean forever. Since the normal way to build tools is once at boot (`chat.with_tools(*Axn::RubyLLM.tools)`),
|
|
133
|
+
any change to the setting *after* that point silently did nothing to a live tool: the wrapped class
|
|
134
|
+
kept whatever the flag happened to resolve to at wrap time, with no warning and no way to tell from
|
|
135
|
+
the outside. Resolution now happens inside `#execute`, at the moment the result is rendered, so a
|
|
136
|
+
tool always reflects the currently-configured value — the same per-call semantics `axn-mcp` already
|
|
137
|
+
had. **Old vs new:** a tool wrapped while the flag was `false` and later switched to `true` used to
|
|
138
|
+
keep shipping opaque renderings; it now fails those calls with the generic tool error, as configured.
|
|
139
|
+
Only the *timing* changed — the resolution order (per-class override, then gem-wide config, then the
|
|
140
|
+
`false` default) is unchanged, so a setup that configures before wrapping (the overwhelmingly common
|
|
141
|
+
case, and every documented example) behaves exactly as before.
|
|
142
|
+
|
|
143
|
+
- **A map's `additionalProperties` (and a Hash's `minProperties`/`maxProperties`) are no longer silently
|
|
144
|
+
lost at Gemini.** RubyLLM's Gemini converter rebuilds each property from a fixed whitelist that omits all
|
|
145
|
+
three, so `expects :scores, type: Hash, of: { keys: String, values: Integer }` reached the model as an empty
|
|
146
|
+
`{type: OBJECT}` — with no error raised, leaving the model to guess the value type and the call to be rejected
|
|
147
|
+
at runtime instead (PRO-3172). The adapter now restates these constraints as prose in the same node's
|
|
148
|
+
`description`, which Gemini does forward, appending after any `description:` you supplied; a structured value
|
|
149
|
+
type carries its compact JSON Schema too. The enforceable keys are still advertised unchanged, so OpenAI and
|
|
150
|
+
Anthropic are unaffected apart from the redundant sentence.
|
|
151
|
+
|
|
152
|
+
- **The transport-failure guard now logs an operator hint when `reject_opaque_exposed_values` may be the
|
|
153
|
+
cause.** The tool-facing error stays generic (`"The tool could not produce a valid response"`), but the
|
|
154
|
+
logged line now names the offending tool and both places the setting could be set
|
|
155
|
+
(`configure(:ruby_llm)` / `Axn::RubyLLM.config.reject_opaque_exposed_values`) whenever the resolved
|
|
156
|
+
value is `true` — matching `axn-openapi`'s dispatcher hint and `axn-mcp`'s guard. Previously an operator
|
|
157
|
+
had to guess which knob caused a rejection since the setting is per-tool overridable.
|
|
158
|
+
|
|
159
|
+
### Internal
|
|
160
|
+
|
|
161
|
+
- **[INTERNAL] Adopted axn's `Axn::Tools::AdapterSerialization` mixin (PRO-2996)** in place of this
|
|
162
|
+
gem's hand-rolled copies of the same three things. `Axn::RubyLLM` now `extend`s the mixin alongside
|
|
163
|
+
`Axn::Tools::AdapterRoots` and uses `declare_reject_opaque_exposed_values! default: false` for the
|
|
164
|
+
setting, `Axn::RubyLLM.serialize_exposed(result)` for the render (which resolves the per-tool flag
|
|
165
|
+
itself — see the Fixed entry above), and `Axn::RubyLLM.guard_tool_response(axn_class, on_error:)`
|
|
166
|
+
for the transport-mapping guard. The default `tool_roots` moved to `tool_roots_default %w[agent_tools]`,
|
|
167
|
+
which drops this gem's hand-copied `AdapterRoots.validate!` lambda and validates the default eagerly
|
|
168
|
+
at gem load rather than at the registry's first read. No public API, config name, default, or
|
|
169
|
+
user-facing string changed. Two second-order effects worth knowing: the mixin's guard runs its
|
|
170
|
+
dev-mode re-raise *before* reporting (so under `best_effort_raises_in_dev` a mapping failure now
|
|
171
|
+
raises without first emitting the `reject_opaque_exposed_values` log hint — production behavior is
|
|
172
|
+
unchanged), and it rescues `SystemStackError`/`ScriptError` in addition to `StandardError`, so a
|
|
173
|
+
runaway `as_json`/`to_h` on an exposed value now becomes a tool error rather than escaping into the
|
|
174
|
+
chat.
|
|
175
|
+
|
|
176
|
+
### Requires
|
|
177
|
+
|
|
178
|
+
- **axn `>= 0.1.0-alpha.6`** (released), for `Axn::Tools::AdapterSerialization` (PRO-2996) and
|
|
179
|
+
`Axn::Extensions::Tracing.annotate_span` (PRO-3278). No Gemfile override needed — resolves from
|
|
180
|
+
RubyGems.
|
|
181
|
+
|
|
3
182
|
## [0.2.0] - 2026-08-05
|
|
4
183
|
|
|
5
184
|
> **Upgrade notes (behavior changes):**
|
data/README.md
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# axn-ruby_llm
|
|
2
2
|
|
|
3
|
-
Call LLMs from [Axn](https://github.com/teamshares/axn) actions using [RubyLLM](https://github.com/crmne/ruby_llm), with declarative error handling,
|
|
3
|
+
Call LLMs from [Axn](https://github.com/teamshares/axn) actions using [RubyLLM](https://github.com/crmne/ruby_llm), with declarative error handling, schema-based structured output, configurable defaults, and cost/token tracking — and wrap any Axn as a `RubyLLM::Tool` a chat can call.
|
|
4
|
+
|
|
5
|
+
> **RubyLLM 2.0 required.** As of `0.3.0`, this gem requires `ruby_llm >= 2.0, < 3.0` and no longer supports RubyLLM 1.x — see [CHANGELOG.md](CHANGELOG.md) for the full breaking-change rundown if you're upgrading from an earlier `axn-ruby_llm` release.
|
|
4
6
|
|
|
5
7
|
Part of the `axn-*` extension ecosystem — see also [axn-mcp](https://github.com/teamshares/axn-mcp).
|
|
6
8
|
|
|
@@ -12,7 +14,7 @@ Four things you'd otherwise hand-build:
|
|
|
12
14
|
|
|
13
15
|
2. **Production gating.** A single `c.enabled = -> { Rails.env.production? }` in an initializer stubs every LLM call in non-prod environments — no per-callsite guards needed. The stub is typed (`stubbed: true`, `input_tokens: 0`, etc.) so downstream code doesn't need to branch on it either.
|
|
14
16
|
|
|
15
|
-
3. **Cost/token tracking, exposed automatically.** Every call exposes `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens`, `prompt_tokens` (the total), `cost`, and `cost_breakdown
|
|
17
|
+
3. **Cost/token tracking, exposed automatically.** Every call exposes `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens`, `prompt_tokens` (the total), `cost`, and `cost_breakdown`, read straight off RubyLLM's own usage ledger (`Chat#tokens` / `Chat#cost`) — no manual model lookup, and a tool-call loop's retries and multiple round-trips are already aggregated for you. If your app uses OpenTelemetry, these values are also set as attributes on the existing `axn.call` span — no configuration required.
|
|
16
18
|
|
|
17
19
|
4. **Author-once tools.** `Axn::RubyLLM.wrap` turns any Axn into a `RubyLLM::Tool` your chat can call — reuse the same Axn classes you already expose through [axn-mcp](https://github.com/teamshares/axn-mcp), or plain Axns, with no rewrite. The tool's name, JSON Schema, and argument validation all come from the Axn's own contract.
|
|
18
20
|
|
|
@@ -52,13 +54,6 @@ result = Axn::RubyLLM.ask(
|
|
|
52
54
|
)
|
|
53
55
|
result.response # => "The team decided to..."
|
|
54
56
|
|
|
55
|
-
# JSON mode
|
|
56
|
-
result = Axn::RubyLLM.ask(
|
|
57
|
-
prompt: build_extraction_prompt(doc),
|
|
58
|
-
json: true
|
|
59
|
-
)
|
|
60
|
-
result.response # => { "company" => "Acme", "founded" => 1999 }
|
|
61
|
-
|
|
62
57
|
# With system prompt and model override
|
|
63
58
|
result = Axn::RubyLLM.ask(
|
|
64
59
|
prompt: user_message,
|
|
@@ -71,10 +66,14 @@ result = Axn::RubyLLM.ask(
|
|
|
71
66
|
|
|
72
67
|
### Structured output via schema
|
|
73
68
|
|
|
74
|
-
Pass `schema:` to enable provider-enforced structured output (e.g. OpenAI strict mode) via `RubyLLM::Chat#with_schema`. The result's `response` is the parsed Hash.
|
|
69
|
+
Pass `schema:` to enable provider-enforced structured output (e.g. OpenAI strict mode) via `RubyLLM::Chat#with_schema`. The result's `response` is the parsed Hash — read via `RubyLLM::Message#parsed` under the hood, not `#content` (which is now always the raw JSON text).
|
|
70
|
+
|
|
71
|
+
`schema:` accepts any of three forms:
|
|
72
|
+
|
|
73
|
+
**A [`Schematist::Schema`](https://github.com/crmne/schematist) class or instance** — anything `RubyLLM::Chat#with_schema` itself accepts:
|
|
75
74
|
|
|
76
75
|
```ruby
|
|
77
|
-
class CompanyMatch <
|
|
76
|
+
class CompanyMatch < Schematist::Schema
|
|
78
77
|
integer :company_id, description: "ID of the matched company, or null"
|
|
79
78
|
number :confidence, description: "0.0–1.0"
|
|
80
79
|
string :reasoning
|
|
@@ -87,11 +86,41 @@ result = Axn::RubyLLM.ask(
|
|
|
87
86
|
result.response # => { "company_id" => 42, "confidence" => 0.92, "reasoning" => "..." }
|
|
88
87
|
```
|
|
89
88
|
|
|
90
|
-
|
|
89
|
+
The `schematist` gem (installed automatically by RubyLLM 2.0) is recommended but not required; declare it in your own Gemfile if you want the DSL.
|
|
90
|
+
|
|
91
|
+
**A raw JSON Schema Hash**, passed straight through unchanged:
|
|
92
|
+
|
|
93
|
+
```ruby
|
|
94
|
+
Axn::RubyLLM.ask(prompt: "...", schema: { type: "object", properties: { answer: { type: "string" } } })
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
**An Axn class**, via its own `output_schema` reflection — the same contract you'd already write with `exposes`, no separate schema to maintain:
|
|
98
|
+
|
|
99
|
+
```ruby
|
|
100
|
+
class CompanyMatch
|
|
101
|
+
include Axn
|
|
102
|
+
exposes :company_id, type: Integer, allow_nil: true
|
|
103
|
+
exposes :confidence, type: Float
|
|
104
|
+
exposes :reasoning, type: String
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
result = Axn::RubyLLM.ask(prompt: "...", schema: CompanyMatch)
|
|
108
|
+
result.response # => { "company_id" => 42, "confidence" => 0.92, "reasoning" => "..." }
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
A few adjustments happen automatically here, confirmed against real provider calls (Anthropic live; OpenAI by its documented contract):
|
|
112
|
+
|
|
113
|
+
- `additionalProperties: false` is injected on every fixed-shape object node — required unconditionally by Anthropic's structured output and by OpenAI's strict mode, and axn's `exposes` contract has no reason to emit it on its own.
|
|
114
|
+
- `minProperties`/`maxProperties` are stripped — axn emits `minProperties: 1` by default on a nested fixed-shape `Hash` field, and Anthropic's schema validator rejects it outright (confirmed live).
|
|
115
|
+
- `strict: false` is always sent, rather than left to RubyLLM's own strict-inference (which would otherwise turn on for the common case of every property being required).
|
|
116
|
+
|
|
117
|
+
You get the declared shape, required keys, and "no extra keys" enforcement; you don't get OpenAI's *full* strict-mode guarantee, which additionally requires every property to appear in `required` — even conceptually optional ones, via a nullable type — which axn's reflection doesn't promise. Pass a `Schematist::Schema` instead if you need that.
|
|
118
|
+
|
|
119
|
+
> **A `Hash` map field (`type: Hash, of: {...}`) doesn't survive this path against every provider.** Confirmed live: Anthropic's structured output rejects `additionalProperties` set to anything but the literal `false` (`"additionalProperties: object' is not supported. Please set 'additionalProperties' to false"`), and OpenAI's strict mode has the same restriction by design — neither provider's structured-output feature represents dynamic/arbitrary keys, only a fixed shape. This isn't something the adapter can paper over (there's no schema-legal way to say "arbitrary keys, but still typed" in either provider's strict mode), so a map field is left as-is and the provider's own rejection surfaces as the request's error. Use a fixed-shape `Hash` (`shape:`) instead, or pass your own schema Hash / `Schematist::Schema` if you specifically need a map with that provider.
|
|
91
120
|
|
|
92
121
|
### Token counts and cost
|
|
93
122
|
|
|
94
|
-
Every successful result exposes token usage and cost:
|
|
123
|
+
Every successful result exposes token usage and cost, read off RubyLLM's own usage ledger (`Chat#tokens` / `Chat#cost`) — which already sums every provider attempt for the call, including a tool loop's multiple round-trips and any retries:
|
|
95
124
|
|
|
96
125
|
```ruby
|
|
97
126
|
result = Axn::RubyLLM.ask(prompt: "...")
|
|
@@ -103,14 +132,14 @@ result.prompt_tokens # => 512 (input_tokens + cache_read_tokens + cache_wr
|
|
|
103
132
|
result.output_tokens # => 78
|
|
104
133
|
result.cost # => 0.00056 (Float USD total; nil if RubyLLM has no pricing for the model)
|
|
105
134
|
|
|
106
|
-
# Full breakdown — RubyLLM::Cost
|
|
135
|
+
# Full breakdown — RubyLLM::Cost, RubyLLM's own aggregated-cost object
|
|
107
136
|
result.cost_breakdown # => #<Cost input: 0.0004, output: 0.00016, cache_read: 0.0, ..., total: 0.00056>
|
|
108
137
|
|
|
109
138
|
# Raw RubyLLM::Message for thinking tokens, raw provider data, etc.
|
|
110
139
|
result.raw_message # => #<RubyLLM::Message ...>
|
|
111
140
|
```
|
|
112
141
|
|
|
113
|
-
`cost`
|
|
142
|
+
`cost` is `nil` when RubyLLM lacks pricing for the model (e.g. unknown/custom endpoints); `cost_breakdown` itself is still a `Cost` object in that case (only its component readers are `nil`). Token counts are nil only if the provider did not return them. `prompt_tokens` is nil only if all three input token fields are nil.
|
|
114
143
|
|
|
115
144
|
### Errors
|
|
116
145
|
|
|
@@ -119,8 +148,8 @@ Errors are handled via Axn's declarative `error` DSL. Every failure shares a con
|
|
|
119
148
|
- `RubyLLM::RateLimitError` (HTTP 429, provider-agnostic) → `"LLM request failed: Rate limit reached: <message>"`
|
|
120
149
|
- `RubyLLM::OverloadedError` / `ServiceUnavailableError` / `ServerError` (5xx, transient) → `"LLM request failed: Provider temporarily unavailable, try again later: <message>"`
|
|
121
150
|
- `RubyLLM::ContextLengthExceededError` → `"LLM request failed: Prompt exceeds the model's context window: <message>"` (message retains the provider's token counts)
|
|
122
|
-
- `schema:` set but LLM returned non-JSON → `"LLM request failed: Schema response was not valid JSON"`
|
|
123
|
-
- Any other known RubyLLM error — `RubyLLM::Error` (auth, bad request, payment, etc.), `RubyLLM::ConfigurationError`, `ModelNotFoundError`, `PromptNotFoundError`, `InvalidRoleError`, `InvalidToolChoiceError`, `UnsupportedAttachmentError` — or `Faraday::Error` (network/transport failure) → `"LLM request failed: <message>"`
|
|
151
|
+
- `schema:` set but LLM returned non-JSON, or valid JSON that isn't an object → `"LLM request failed: Response was not valid JSON"` (malformed JSON text) or `"LLM request failed: Schema response was not valid JSON"` (valid JSON, wrong shape)
|
|
152
|
+
- Any other known RubyLLM error — `RubyLLM::Error` (auth, bad request, payment, etc.), `RubyLLM::ConfigurationError`, `ModelNotFoundError`, `ModelRegistryError`, `PromptNotFoundError`, `InvalidRoleError`, `InvalidToolChoiceError`, `PendingToolCallsError`, `CancelledError`, `UnsupportedAttachmentError` — or `Faraday::Error` (network/transport failure) → `"LLM request failed: <message>"`
|
|
124
153
|
- Any other `StandardError` (i.e. not a recognized RubyLLM/network failure — most likely a bug) → `"LLM request failed"`, with no exception detail leaked into the message
|
|
125
154
|
|
|
126
155
|
## Tool adapter — wrap any Axn as a RubyLLM::Tool
|
|
@@ -153,20 +182,19 @@ result.response # => "Created widget Sprocket (id: 42)."
|
|
|
153
182
|
|
|
154
183
|
`tools:` accepts a mix of **bare Axn classes** (wrapped automatically) and **already-wrapped tools** from `Axn::RubyLLM.wrap` (a class, or an instance that closed over `ambient_context:` — see below). Pass `tools: Axn::RubyLLM.tools` to expose everything registered under the `:ruby_llm` adapter (see [Enumerating tools](#enumerating-tools-from-the-registry)). The same Axn classes you expose through [axn-mcp](https://github.com/teamshares/axn-mcp) work here unchanged.
|
|
155
184
|
|
|
156
|
-
> **Token/cost in a tool loop:** a tool call makes multiple model round-trips inside one `ask`. The token counts, `cost`, and `cost_breakdown`
|
|
185
|
+
> **Token/cost in a tool loop:** a tool call makes multiple model round-trips inside one `ask`. The token counts, `cost`, and `cost_breakdown` come from RubyLLM's own usage ledger (`Chat#tokens` / `Chat#cost`), which already aggregates **every turn** — so they reflect the whole call, not just the final response. (`raw_message` is still the final response.)
|
|
157
186
|
|
|
158
187
|
`Axn::RubyLLM.wrap` is also available directly if you're driving `RubyLLM.chat` yourself rather than going through `ask` — see [Using wrapped tools with RubyLLM directly](#using-wrapped-tools-with-rubyllm-directly).
|
|
159
188
|
|
|
160
|
-
The tool's name, description, and JSON Schema parameters come straight from the Axn's own contract — the same `description`/`expects`/`exposes` you'd write for any Axn — so a minimal class just works (the tool name defaults from the class name: `CreateWidget` → `create_widget`). Arguments the model supplies are run through axn core's tool `Invoker`: wire types are coerced, and any contract violation — a missing required field, an out-of-schema argument, a wrong type, or a value outside an `inclusion` set (validated at **full depth**, not just the top-level type) — comes back to the model as a clean, correctable `{ error: "Invalid tool arguments: <reason>" }`, and does **not** page `on_exception` as though it were a bug. A model-supplied `ambient_context` is stripped before the Axn runs, so a prompt-injected context can never override the caller's — the wrap's own `ambient_context:` (below) is injected instead.
|
|
189
|
+
The tool's name, description, and JSON Schema parameters come straight from the Axn's own contract — the same `description`/`expects`/`exposes` you'd write for any Axn — so a minimal class just works (the tool name defaults from the class name: `CreateWidget` → `create_widget`). Arguments the model supplies are run through axn core's tool `Invoker`: wire types are coerced, and any contract violation — a missing required field, an out-of-schema argument, a wrong type, or a value outside an `inclusion` set (validated at **full depth**, not just the top-level type) — comes back to the model as a clean, correctable `{ error: "Invalid tool arguments: <reason>" }`, and does **not** page `on_exception` as though it were a bug. A model-supplied `ambient_context` is stripped before the Axn runs, so a prompt-injected context can never override the caller's — the wrap's own `ambient_context:` (below) is injected instead. The `Invoker` also stamps every call (including any nested sub-Axn) with the `invoked_via: :ruby_llm` dimension, so a Datadog dashboard can query tool-driven traffic separately from ordinary direct `.call`s — see [OpenTelemetry](#opentelemetry) below.
|
|
161
190
|
|
|
162
|
-
On success, `execute` returns the exposed values (via `Axn::
|
|
191
|
+
On success, `execute` returns the exposed values (via `Axn::RubyLLM.serialize_exposed`, honoring `reject_opaque_exposed_values` — see below) as a JSON **string**, not a Hash — `RubyLLM::Tool.split_result` sends a returned String through as-is, but a returned Hash/Array is `#to_json`'d via Ruby's own dispatch, which may not match axn's own serialization contract (Symbol keys/values, `BigDecimal`, `Time`, opaque-value rejection); serializing ourselves keeps the wire form aligned with what `output_schema` advertises. On failure, `{ error: result.error }`. The same `CreateWidget` class can be wrapped for other transports (e.g. `Axn::MCP.wrap`) with no changes — the contract is declared once.
|
|
163
192
|
|
|
164
193
|
Options, settable either per-call via `wrap` keywords or once on the Axn via axn's namespaced per-class `configure(:ruby_llm) { |c| ... }` (a `wrap` keyword wins when both are present, then the class-level `configure(:ruby_llm)` value, then this gem's own `Axn::RubyLLM.configure { |c| ... }` global, then the default below):
|
|
165
194
|
|
|
166
195
|
| Option | Effect |
|
|
167
196
|
|---|---|
|
|
168
|
-
| `
|
|
169
|
-
| `provider_params:` | Hash deep-merged into the tool definition sent to the provider (via RubyLLM's `with_params`) — an escape hatch for provider-specific tool fields RubyLLM doesn't model first-class (e.g. OpenAI `strict` function calling, Anthropic tool `cache_control`). Keys mirror that provider's tool shape. Default `{}`. |
|
|
197
|
+
| `provider_options:` | Hash merged into the tool definition sent to the provider (via RubyLLM's `Tool.provider_options`) — an escape hatch for provider-specific tool fields RubyLLM doesn't model first-class (e.g. OpenAI `strict` function calling, Anthropic tool `cache_control`). Keys mirror that provider's tool shape. Default `{}`. |
|
|
170
198
|
| `present_as:` | `:structured` (default) returns the exposed values as a JSON string; `:message` returns `result.message` instead. Same knob as axn-mcp's `present_as:`. |
|
|
171
199
|
|
|
172
200
|
Set a default once on the Axn with `configure(:ruby_llm)`, and still override per call:
|
|
@@ -174,14 +202,16 @@ Set a default once on the Axn with `configure(:ruby_llm)`, and still override pe
|
|
|
174
202
|
```ruby
|
|
175
203
|
class CreateWidget
|
|
176
204
|
include Axn
|
|
177
|
-
configure(:ruby_llm) { |c| c.
|
|
205
|
+
configure(:ruby_llm) { |c| c.present_as = :message } # default for this tool
|
|
178
206
|
# ...
|
|
179
207
|
end
|
|
180
208
|
|
|
181
|
-
Axn::RubyLLM.wrap(CreateWidget)
|
|
182
|
-
Axn::RubyLLM.wrap(CreateWidget,
|
|
209
|
+
Axn::RubyLLM.wrap(CreateWidget) # returns result.message
|
|
210
|
+
Axn::RubyLLM.wrap(CreateWidget, present_as: :structured) # per-call override
|
|
183
211
|
```
|
|
184
212
|
|
|
213
|
+
> **No more `halt_after:`.** RubyLLM 2.0 removed `Tool::Halt` along with the rest of the auto-halting machinery — the conversation loop is now caller-controlled. If you need a tool call to stop the loop, drive it yourself: `loop { chat.step; break if chat.complete? || done_condition }`. See RubyLLM's [Agentic Workflows](https://rubyllm.com/agentic-workflows/) guide.
|
|
214
|
+
|
|
185
215
|
`configure(:ruby_llm)` needs no `include` beyond `Axn` — every Axn gets it for free (core's namespaced per-class config). It's usually written in the class body as above, but since it's a plain class method you can also call it from outside — e.g. `SomeThirdPartyAxn.configure(:ruby_llm) { |c| ... }` in an initializer, to configure an Axn you don't own. Namespacing is what lets **one base Axn be configured for multiple adapters at once**, each in its own namespace, without collision even when two adapters share a setting name (both this gem and axn-mcp expose `present_as`):
|
|
186
216
|
|
|
187
217
|
```ruby
|
|
@@ -200,14 +230,14 @@ Pass `ambient_context:` to close over explicit caller context (e.g. `current_use
|
|
|
200
230
|
Axn::RubyLLM.wrap(CreateWidget, ambient_context: { company_id: current_company.id })
|
|
201
231
|
```
|
|
202
232
|
|
|
203
|
-
Passing `ambient_context:` returns a tool **instance** (closing over that context) rather than the tool class, since `chat.
|
|
233
|
+
Passing `ambient_context:` returns a tool **instance** (closing over that context) rather than the tool class, since `chat.with_tools` accepts either.
|
|
204
234
|
|
|
205
235
|
### Using wrapped tools with RubyLLM directly
|
|
206
236
|
|
|
207
|
-
`Axn::RubyLLM.ask(tools:)` covers the common single-call case. When you're driving `RubyLLM.chat` yourself — multi-turn conversations, streaming, or anything else beyond `ask` — register wrapped tools with RubyLLM's own `
|
|
237
|
+
`Axn::RubyLLM.ask(tools:)` covers the common single-call case. When you're driving `RubyLLM.chat` yourself — multi-turn conversations, streaming, or anything else beyond `ask` — register wrapped tools with RubyLLM's own `with_tools`, which accepts one or many `RubyLLM::Tool` classes/instances:
|
|
208
238
|
|
|
209
239
|
```ruby
|
|
210
|
-
chat = RubyLLM.chat.
|
|
240
|
+
chat = RubyLLM.chat.with_tools(Axn::RubyLLM.wrap(CreateWidget))
|
|
211
241
|
chat.ask("Create a widget called Sprocket")
|
|
212
242
|
|
|
213
243
|
# or register everything under the :ruby_llm adapter at once:
|
|
@@ -283,12 +313,13 @@ So the adapter guards that mapping step (only — the Axn call already reports i
|
|
|
283
313
|
|
|
284
314
|
### Schema reflection — provider notes
|
|
285
315
|
|
|
286
|
-
The advertised tool schema is axn's reflected `input_schema
|
|
316
|
+
The advertised tool schema is axn's reflected `input_schema`, passed through to RubyLLM **unmodified** — the adapter does no per-provider schema rewriting. RubyLLM 2.0's Gemini protocol reads a tool's schema via `parametersJsonSchema`, the wire form verbatim, rather than rebuilding each property from a fixed whitelist (as its 1.x converter did), so nullable fields, `additionalProperties`, and entry-count bounds all reach Gemini the same way they reach OpenAI and Anthropic.
|
|
317
|
+
|
|
318
|
+
A few things worth knowing about what axn itself reflects, independent of RubyLLM version:
|
|
287
319
|
|
|
288
|
-
- **
|
|
289
|
-
- **Array fields — declare `of:`.** `expects :ids, type: Array` reflects to `{type: array}` with no `items`, so the element type isn't advertised (Gemini then assumes `string`; OpenAI *strict* mode requires `items`). Declare the element type — `expects :ids, type: Array, of: Integer` — to advertise `items` correctly.
|
|
320
|
+
- **Array fields — declare `of:`.** `expects :ids, type: Array` reflects to `{type: array}` with no `items`, so the element type isn't advertised (some providers then assume `string`; OpenAI *strict* mode requires `items`). Declare the element type — `expects :ids, type: Array, of: Integer` — to advertise `items` correctly.
|
|
290
321
|
- **Enums are advertised — use the top-level `inclusion:` key.** `expects :color, type: String, inclusion: %w[red green blue]` (a bare Array, or the long form `inclusion: { in: %w[red green blue] }`) reflects to `{ type: "string", enum: ["red", "green", "blue"] }`, so the model is told the allowed set (an `optional:` field keeps `null` in the enum; a dynamic `in: -> { ... }` is correctly skipped rather than guessed). Note this is the **top-level `inclusion:` option**, not `validate: { inclusion: ... }` — `validate:` is axn's custom-callable hook (it needs `with:`), so that spelling neither enforces nor reflects the set.
|
|
291
|
-
- **Conditional expectations** (`expects :token, if: :use_token`) reflect to a JSON Schema `allOf`/`if`/`then` clause.
|
|
322
|
+
- **Conditional expectations** (`expects :token, if: :use_token`) reflect to a JSON Schema `allOf`/`if`/`then` clause. Whether a given provider's tool-schema handling honors that clause is between RubyLLM and the provider, not something this adapter controls — check the provider's own function-calling docs if it matters for your case. Encoding the rule in `description:` remains the portable option regardless.
|
|
292
323
|
|
|
293
324
|
## Testing
|
|
294
325
|
|
|
@@ -304,7 +335,7 @@ it "summarizes the thread" do
|
|
|
304
335
|
end
|
|
305
336
|
```
|
|
306
337
|
|
|
307
|
-
The response is the only required argument — pass it positionally (as above) or as `response:`.
|
|
338
|
+
The response is the only required argument — pass it positionally (as above) or as `response:`. Pass `schema:` to route a Hash response through the schema path (stubbing `raw_message.parsed` so `result.response` gets the Hash back unparsed, matching what a real `schema:` call returns). Token counts and cost default to zero and can be set explicitly to exercise cost/usage logic:
|
|
308
339
|
|
|
309
340
|
```ruby
|
|
310
341
|
stub_axn_ruby_llm({ "company_id" => 42 }, schema: CompanyMatch)
|
|
@@ -326,6 +357,7 @@ If your app uses OpenTelemetry, `axn` already wraps every action in an `axn.call
|
|
|
326
357
|
| `gen_ai.usage.output_tokens` | Completion token count |
|
|
327
358
|
| `gen_ai.usage.cost` | USD total (non-standard; useful for spend filtering) |
|
|
328
359
|
| `axn.ruby_llm.stubbed` | `true` when production gating returned a stub |
|
|
360
|
+
| `axn.dimension.invoked_via` | `"ruby_llm"` — set by axn core on every tool call (including nested sub-Axns), not by this gem; lets you separate tool-driven traffic from ordinary direct `.call`s in the same span schema |
|
|
329
361
|
|
|
330
362
|
For LLM-level tracing (individual `RubyLLM.chat` calls, tool calls, embeddings, prompt content), add [`opentelemetry-instrumentation-ruby_llm`](https://github.com/thoughtbot/opentelemetry-instrumentation-ruby_llm) to your own Gemfile and configure it per its README. It is not a dependency of this gem.
|
|
331
363
|
|
|
@@ -347,8 +379,8 @@ When disabled, `Axn::RubyLLM.ask` returns a **success** result with obvious stub
|
|
|
347
379
|
|
|
348
380
|
| Field | Stubbed value |
|
|
349
381
|
|---|---|
|
|
350
|
-
| `response` | `"stubbed response value"` (plain) / `{ "stubbed" => true }` (`
|
|
351
|
-
| `raw_message` | Stub struct with `.content`, `.
|
|
382
|
+
| `response` | `"stubbed response value"` (plain) / `{ "stubbed" => true }` (`schema:`) |
|
|
383
|
+
| `raw_message` | Stub struct with `.content`, `.tokens` (a real `RubyLLM::Tokens`, all zero), `.model`, `.parsed` |
|
|
352
384
|
| `input_tokens` / `output_tokens` / `cache_read_tokens` / `cache_write_tokens` / `prompt_tokens` | `0` |
|
|
353
385
|
| `cost` | `0.0` |
|
|
354
386
|
| `cost_breakdown` | `nil` |
|