llm.rb 16.0.0.pre.5 → 16.0.0.pre.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/CHANGELOG.md CHANGED
@@ -13,6 +13,9 @@
13
13
  > Changelog <br>
14
14
  > [r.uby.dev](https://r.uby.dev) project
15
15
 
16
+ This file covers the v15 series and the unreleased changes that follow it.
17
+ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
18
+
16
19
  ## What's next
17
20
 
18
21
  ### Breaking
@@ -23,212 +26,195 @@
23
26
  |-----|-----|
24
27
  | Ruby 3.3 or later | Ruby 3.4 or later |
25
28
  | `record.messages` returns the messages the runtime holds | a `:jsonb` record returns a relation of `LLM::ActiveRecord::Message` rows |
29
+ | a `set_tracer` method on an ORM model | `set tracer:` in the block, or `tracer:` on the wrapper |
30
+ | a `:thread` or `:fiber` cancel that arrives before the tool starts is dropped | the cancel is held and raised once the tool starts |
31
+ | a cancel that arrives between two requests does nothing | the turn ends there, and the caller sees `LLM::Interrupt` |
26
32
 
27
33
  * **drop Ruby 3.3 support** <br>
28
- The gem now requires Ruby 3.4 or later: `required_ruby_version` is
29
- `>= 3.4.0`, the CI matrix runs 3.4 and 4.0, and RuboCop targets 3.4.
30
- Ruby 3.3 cannot hold an interrupt for a ractor-backed tool call, so a
31
- cancel that arrives before the tool starts is not delivered there, while
32
- Ruby 3.4 and 4.0 both deliver it.
34
+ The gem now requires Ruby 3.4 or later. Ruby 3.3 cannot hold an
35
+ interrupt for a ractor-backed tool call, so a cancel can arrive
36
+ before the tool starts and never be delivered.
33
37
 
34
38
  * **activerecord: read a `:jsonb` record's messages from the column** <br>
35
- [`LLM::ActiveRecord`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord.html)
36
- now answers `#messages` for a record whose `format` is `:jsonb` with an
37
- [`ActiveRecord::Relation`](https://api.rubyonrails.org/classes/ActiveRecord/Relation.html)
38
- over the column, through
39
- [`LLM::ActiveRecord::Message`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord/Message.html),
40
- so a conversation can be filtered, counted, and ordered in SQL, and reading it
41
- needs no provider and no credentials. Before this, every record loaded the
42
- runtime and returned the messages it held, whatever the format. The relation
43
- is not a drop-in replacement for that buffer: it reads what is persisted
44
- rather than what a runtime holds in memory, it is unordered, so order by
45
- `position`, and its rows answer `unwrap!` rather than being
46
- [`LLM::Message`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html) objects.
47
- Every other format, and Sequel, still loads the runtime.
39
+ The `#messages` of a `:jsonb` record is now a relation over the stored
40
+ messages, so a conversation can be filtered and counted in SQL without a
41
+ provider or credentials. Before, every record loaded the runtime; reach
42
+ that again with `#messages!`, and order the relation by `position`
43
+ because it is unordered.
44
+
45
+ * **function: hold a cancel on `:thread` and `:fiber` instead of dropping it** <br>
46
+ A cancel that arrived before the tool started used to be dropped, and the call
47
+ ran to completion. It is now held and raised once the tool starts, so a caller
48
+ that cancelled a task it had not yet spawned sees
49
+ [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html) where
50
+ it used to see a return.
51
+
52
+ * **interrupt: reach a turn that is between its requests** <br>
53
+ A cancel that arrived between two requests used to do nothing, because
54
+ there was nothing in flight to close and nothing running to raise into.
55
+ It now ends the turn where it is, raising `LLM::Interrupt` into the
56
+ caller the turn runs under and announcing `scope: :agent` to the tracer
57
+ first. `LLM::Agent#run_loop` records that caller for as long as the turn
58
+ lasts, and it answers `interrupt!` through
59
+ [`LLM::Agent::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent/Interrupt.html).
60
+ `LLM::Interrupt` is a `StandardError`, so a broad `rescue` catches it
61
+ wherever the turn had reached.
48
62
 
49
63
  ### ActiveRecord
50
64
 
51
65
  * **activerecord: add `messages!` for the runtime's own messages** <br>
52
66
  [`LLM::ActiveRecord#messages!`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord.html#messages!-instance_method)
53
- returns the messages the runtime holds whatever the storage format is, so a
54
- jsonb record can still reach them now that its `#messages` answers with the
55
- view over the column. It reads what the context holds, including state that
56
- has not been saved, where `#messages` reads the column. The Sequel plugin
57
- answers to `#messages!` too, where it is the same call as `#messages`.
67
+ returns the messages the runtime holds, including state that has not been
68
+ saved, where `#messages` reads the column. Use it to reach the live
69
+ conversation of a `:jsonb` record. The Sequel plugin answers to
70
+ `#messages!` too, where it is the same call as `#messages`.
58
71
 
59
72
  ### Agent
60
73
 
61
74
  * **agent: keep the instructions it injected in step** <br>
62
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now brings
63
- the instructions it injected up to date before a request goes out, so a
75
+ An agent now refreshes its instructions before each request, so a
64
76
  conversation restored from saved state runs on the agent's current
65
- instructions rather than on the ones it was saved with. The message is
66
- found by a mark the agent puts on it, so a message the caller composed is
67
- never replaced, and the mark is written by
68
- [`LLM::Message#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html#to_h-instance_method)
69
- and read back on a restore, which is the conversation the refresh exists
70
- for. Before this, the instructions were injected once and the message
71
- stayed as it was for the life of the conversation.
77
+ instructions instead of the ones it was saved with. A message the caller
78
+ composed is never touched.
72
79
 
73
80
  * **agent: identify its own instructions by the mark, not by the role** <br>
74
- Fix a bug where the agent decided whether it had already injected its
75
- instructions by looking for a message with the `system` role. Google gives
76
- instructions the `user` role, so the agent did not recognise the message it
77
- had already injected and added another copy of the instructions on every
78
- turn. It now asks the provider which role instructions are given, and
79
- identifies the message it wrote by its mark, because on Google the message
80
- a caller seeds a conversation with is a user message too.
81
+ Fix a bug where an agent on Google added a fresh copy of its instructions
82
+ every turn, because it looked for a `system` message and Google gives
83
+ instructions the `user` role. It now marks the message it wrote and finds
84
+ it by that mark.
81
85
 
82
86
  ### Console
83
87
 
84
88
  * **console: erase with the backspace key on OpenBSD** <br>
85
- The console now erases a character for every code a backspace key sends. A
86
- terminal that follows the Linux convention sends DEL (127), the OpenBSD
87
- console sends ^H (8), and curses sends its own `KEY_BACKSPACE` when the
88
- terminal's terminfo describes the key. All three erase. Before this only 127
89
- did, so the backspace key did nothing on the OpenBSD console.
89
+ The console now erases for every code a backspace key sends: DEL (127),
90
+ ^H (8), and curses's `KEY_BACKSPACE`. Before, only 127 worked, so
91
+ backspace did nothing on the OpenBSD console.
90
92
 
91
93
  ### Fix
92
94
 
93
95
  * **fork: require xchan.rb `~> 0.24`** <br>
94
- The `:fork` concurrency strategy now requires the `xchan.rb` gem at `~> 0.24`
95
- instead of `~> 0.23`, and the gemspec's development dependency follows. An
96
- application that uses `:fork` needs `xchan.rb` 0.24 or later.
96
+ The `:fork` strategy now needs `xchan.rb` 0.24 or later, up from 0.23.
97
97
 
98
98
  ### Function
99
99
 
100
100
  * **function: deliver an interrupt to the tool, not to whatever is running** <br>
101
- [`LLM::Function::Window`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Window.html)
102
- is the stretch of a call that an interrupt belongs to. The `:fork` and `:ractor`
103
- strategies now raise
101
+ A cancel now raises
104
102
  [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html) on the
105
- thread that runs the tool, and only while the tool runs: a cancel that arrives
106
- before the tool starts is held until it does, so the tool's own `rescue` can
107
- handle it, and one that arrives after the tool has returned is a no-op, which
108
- no longer throws the result away. Before this, the interrupt was raised on the
109
- child process's or the ractor's main thread wherever that thread had got to.
103
+ thread running the tool, and only while it runs: arriving early, it is held
104
+ until the tool starts, so the tool's own `rescue` sees it; arriving late, it
105
+ is a no-op instead of throwing the result away. Applies to `:fork` and
106
+ `:ractor`.
110
107
 
111
108
  * **function: answer a ractor-backed task once its ractor has gone** <br>
112
- A `:ractor` tool's result is handed to a ractor the task holds it in,
113
- rather than asked of the ractor that ran the tool, which answers `alive?`
114
- while the tool runs and goes once the result is in. So a wait that arrives
115
- after the tool has run is answered, a second wait is answered from memory,
116
- the way
117
- [`LLM::Function::Thread::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Thread/Task.html#wait-instance_method)
118
- answers from the thread's value, and `alive?` answers `false` once the task
119
- has been waited on. A cancel returns `nil` rather than raising, which is what
120
- lets
121
- [`LLM::Function::Ractor::Group#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Ractor/Group.html#interrupt!-instance_method)
122
- reach every task it holds. Before this, a wait that arrived once the tool had
123
- run raised `Ractor::ClosedError` or never came back, a cancel raised the same
124
- error, and a group's cancel stopped at the first task that had already
125
- returned, leaving the calls after it uncancelled.
109
+ A `:ractor` tool's result is now held by the task instead of asked of the
110
+ ractor that ran it, so a late wait is answered, a second wait comes from
111
+ memory, and a cancel returns `nil` instead of raising. A group's cancel now
112
+ reaches every task, not just the ones still running.
126
113
 
127
114
  * **function: tell an `:async` tool it was cancelled** <br>
128
- A cancel now raises `LLM::Interrupt` on the fiber the tool runs in, through the
129
- reactor's scheduler, so the tool is told and can answer with its own value. A
130
- cancel that arrives before the tool started stops it from running, and the
131
- reactor is stopped by whoever waits. Before this, the cancel pushed an
132
- interrupt sentinel to the task's result queue and left the tool running, its
133
- side effects happening and its result written to a queue nobody read, and the
134
- reactor thread stayed alive until a group's `wait` stopped it.
115
+ A cancel now raises `LLM::Interrupt` on the fiber the tool runs in, so the
116
+ tool is told and can clean up. Before, the cancel was queued and the tool
117
+ kept running, its result written to a queue nobody read.
135
118
 
136
119
  * **function: run the interrupt hook on every strategy** <br>
137
120
  A tool that implements
138
121
  [`LLM::Tool#on_interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#on_interrupt-instance_method)
139
- or
140
- [`LLM::Tool#on_cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#on_cancel-instance_method)
141
- is now told on every concurrency strategy. `:thread`, `:fiber`, and `:async`
142
- run the hook on the thread or fiber that ran the call, `:fork` and `:ractor`
143
- run it inside the child or the ractor before the interrupt is delivered, and
144
- a sequential group's cancel tells every task it holds. Before this, `:async`,
145
- `:fork`, and `:ractor` never ran the hook, the others ran it on the thread
146
- that cancelled, and a class-backed tool ran it on no strategy at all, because
147
- `interrupt!` read the hook off the tool's definition and a class answers
148
- `respond_to?` with false for an instance method. A hook that raises becomes
149
- what the caller sees in place of the call's result.
122
+ is now told on every concurrency strategy. Before, `:async`, `:fork`, and
123
+ `:ractor` never ran it, and a class-backed tool ran it on no strategy at all.
124
+
125
+ * **function: hold a cancel that arrives before a thread or fiber starts** <br>
126
+ Fix a bug where a cancel that arrived before a `:thread` or `:fiber` tool
127
+ started was dropped, and the call ran to completion as if nobody had asked it
128
+ to stop. The cancel is now held on the task and raised once the body is live,
129
+ so the tool's `on_interrupt` hook runs and the caller sees `LLM::Interrupt`.
130
+
131
+ * **function: answer a task that has already answered** <br>
132
+ A second `#wait` on an `:async` or `:fork` task is answered from what the
133
+ first one took, which is what `Thread#value` does, so a task that has answered
134
+ answers again. Before, a second wait blocked on a queue nothing would fill, or
135
+ read a channel the task had already closed. An interrupt re-raises the same
136
+ exception rather than blocking.
150
137
 
151
138
  ### ORM
152
139
 
153
140
  * **orm: save a conversation after each request, not once per turn** <br>
154
- [`LLM::Step`](https://r.uby.dev/api-docs/llm.rb/LLM/Step.html) is prepended
155
- onto every stream by
156
- [`LLM::Stream.try`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#try-class_method),
157
- so a conversation bound to a record is written down as it goes and a turn
158
- that is interrupted can be continued rather than started over. It saves
159
- through
160
- [`LLM::Context#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#record-instance_method),
161
- an ActiveRecord model through `LLM::ActiveRecord::Utils.save!` and a Sequel
162
- model through `LLM::Sequel::Plugin::Utils.save!`, and the write is best
163
- effort, so a failure is swallowed rather than raised inside the request that
164
- has just completed. Before this, `talk` and `ask` on both wrappers saved
165
- once, when the whole turn had finished, so an interrupted turn persisted
166
- nothing.
141
+ A record-backed conversation is now written down as it goes, so an
142
+ interrupted turn can be continued instead of started over. Before, the
143
+ wrappers saved once when the whole turn finished, so an interrupted turn
144
+ persisted nothing.
145
+
146
+ * **orm: drop the `set_tracer` callback** <br>
147
+ The wrappers no longer resolve a tracer from a `set_tracer` method on the
148
+ model. Declare it where an agent's other defaults are declared, with
149
+ `set tracer:` in the block, or pass it to the wrapper as `tracer:`, which
150
+ is what a context has. A model that implements `set_tracer` today loses
151
+ its tracer silently.
167
152
 
168
153
  ### Prompt
169
154
 
170
155
  * **prompt: let a message carry fields a caller attaches to it** <br>
171
156
  [`LLM::Prompt#system`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#system-instance_method),
172
- [`#user`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#user-instance_method),
173
- [`#developer`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#developer-instance_method),
174
- and [`#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#talk-instance_method),
175
- along with its `#chat` alias, now take an `extra:` keyword, so a message a
176
- caller composes can carry fields of its own. The runtime reads those fields
177
- back rather than inferring provenance from a role, which is how an agent
178
- tells the instructions it injected from a message the caller wrote.
157
+ `#user`, `#developer`, and `#talk` (and its `#chat` alias) now take an
158
+ `extra:` keyword, so a caller can tag a message with fields of its own. The
159
+ runtime reads those fields back instead of guessing provenance from the role.
179
160
 
180
161
  ### Provider
181
162
 
182
163
  * **deepseek: support image attachments in chat completions** <br>
183
- DeepSeek's vision models now take an image through the chat completions
184
- adapter. A tagged prompt object of kind `:image_url` is sent as an `image_url`
185
- content item, and a local image file is sent as a base64 encoded data URI.
186
- Anything else is rejected with a reason of its own: a remote file or an
187
- `LLM::Response` raises
164
+ DeepSeek's vision models now accept an image. A `:image_url` object is sent
165
+ as an `image_url` item, and a local image file as a base64 data URI.
166
+ Anything else is rejected with a reason: a remote file or an `LLM::Response`
167
+ raises
188
168
  [`LLM::PromptError`](https://r.uby.dev/api-docs/llm.rb/LLM/PromptError.html)
189
- because DeepSeek has no Files API, so a file id cannot be resolved, and a
190
- local file that is not an image raises because DeepSeek's models read images
191
- only.
169
+ because DeepSeek has no Files API, and a non-image local file raises because
170
+ its models read images only.
192
171
 
193
172
  ### Registry
194
173
 
195
174
  * **refresh model metadata** <br>
196
- Update `data/` with current model listings, limits, and pricing. Alibaba adds
197
- `qwen3.5-flash`, `qwen3.7-flash`, and `qwen3.8-omni-flash`, and gives
198
- `qwen3.7-plus` attachments, structured output, and a lower price; OpenAI adds
199
- `gpt-daybreak-blue-latest` and `gpt-daybreak-red-latest`; Bedrock adds
200
- `openai.gpt-6-sol`, `openai.gpt-6-luna`, and
201
- `global.anthropic.claude-sonnet-5-5`; and DeepInfra adds `tencent/Hy4-preview`
202
- and the Xiaomi MiMo V2.6 Pro and V2.6 Flash models.
203
- Bedrock also corrects the context and output limits of fifteen models, Google
204
- lowers the limits of `gemini-2.5-computer-use-preview-10-2025` and
205
- `gemini-3-pro-image`, and OpenRouter adds five models, drops six, and reprices
206
- many of the DeepSeek, Z.ai, and Qwen entries.
175
+ Update `data/` with current listings, limits, and pricing. Alibaba, OpenAI,
176
+ Bedrock, DeepInfra, Google, and OpenRouter gain models and correct limits or
177
+ prices, and OpenRouter drops six entries.
207
178
 
208
179
  ### Schema
209
180
 
210
181
  * **schema: apply every `parameter` option to the leaf** <br>
211
- [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) now applies each
212
- option given to `parameter` to the leaf its type resolves to, so settings such
213
- as `min:`, `max:`, `multiple_of:`, and `const:` reach the schema instead of
214
- being dropped. Before this only `required`, `default`, and `enum` were read, so
215
- `parameter :age, Integer, "Age", min: 1, max: 9` sent a plain integer and lost
216
- the range. A `default:` of `false` survives now as well, where the old code
217
- discarded a false default, and `required: false` marks the parameter optional
218
- rather than being ignored.
182
+ Every option given to `parameter` now reaches the schema, so `min:`, `max:`,
183
+ `multiple_of:`, and `const:` are no longer dropped, a `default: false`
184
+ survives, and `required: false` marks a parameter optional. Before, only
185
+ `required`, `default`, and `enum` were read, so a tool that declared a range
186
+ sent a plain number and lost it.
219
187
 
220
188
  ### Stream
221
189
 
222
190
  * **stream: add `on_step`, called when a request completes** <br>
223
191
  [`LLM::Stream#on_step`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_step-instance_method)
224
- is a new callback, emitted from
225
- [`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk-instance_method)
226
- once the prompt and the response are in the conversation. It marks the
227
- boundary between one request and the next, which is the point a stream can
228
- checkpoint the conversation at, because a provider will accept it again
229
- there. It fires once per successful request, so a request retried after a
230
- rate limit or a timeout calls it when it lands and not once per attempt, and
231
- it fires even when the stream is disabled.
192
+ is a new callback, fired once the prompt and response are in the
193
+ conversation. It marks where one request ends and the next begins, which is
194
+ where a stream can checkpoint, and it fires once per successful request even
195
+ when streaming is off.
196
+
197
+ ### Tracers
198
+
199
+ * **tracer: report a request that failed** <br>
200
+ [`LLM::Tracer#on_request_error`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#on_request_error-instance_method)
201
+ is now called whenever a request ends without a response, so a span is always
202
+ closed. Before, only provider errors that became exceptions were reported, so
203
+ a dropped connection left a span open, which reads like a process that died.
204
+
205
+ * **tracer: add `on_interrupt`, called when a request is interrupted** <br>
206
+ [`LLM::Tracer#on_interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#on_interrupt-instance_method)
207
+ is a new hook, called with `scope: :request`, `:tool`, or `:agent` before
208
+ `LLM::Interrupt` reaches the caller, so a tracer can record an interrupted
209
+ turn. An `:agent` scope means the interrupt landed between a turn's
210
+ requests, where there was nothing more precise to interrupt. It does
211
+ nothing by default, since a raise here would replace the interrupt
212
+ callers are written against.
213
+
214
+ * **tracer: announce an interrupted tool phase once** <br>
215
+ [`LLM::Context#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#wait-instance_method)
216
+ now calls `on_interrupt` once when an interrupt unwinds through it, rather
217
+ than once per tool, matching the single exception the caller receives.
232
218
 
233
219
  ## v15.5.0
234
220
 
@@ -1443,1752 +1429,3 @@ rate-limited requests, and a range of REPL and skills improvements.
1443
1429
  Update `data/deepinfra.json` with current pricing for the DeepSeek
1444
1430
  V4, DeepSeek-V3, DeepSeek-R1-0528, and Kimi-K3 models, and mark
1445
1431
  `structured_output` support for one model.
1446
-
1447
- ## v14.0.0
1448
-
1449
- Changes since `v13.1.0`.
1450
-
1451
- This release replaces the `transformer=` setter with the new
1452
- `LLM::Transformer` class hierarchy, refactors guards into the
1453
- `LLM::Guard` superclass with per-tool-call interception, and replaces
1454
- the agent `tool_attempts` parameter with the `tool_budget` class DSL.
1455
- It also adds the Moonshot (Kimi) provider, the `LLM::Tool.set`
1456
- bulk-assignment DSL, a `LLM::Function#return` shorthand, and a wide
1457
- range of REPL improvements.
1458
-
1459
- ### Breaking
1460
-
1461
- #### Migration
1462
-
1463
- | Old | New |
1464
- |-----|-----|
1465
- | `ctx.transformer = MyTransformer` | `LLM::Context.new(transformer: MyTransformer)` |
1466
- | `transformer.call(ctx, prompt, params)` | `transformer.call(message:, **opts)` |
1467
- | `~/.llm.rb/session.json` (shared across providers) | `~/.llm.rb/<provider>/<uuid>.json` (scoped per provider and directory) |
1468
- | `agent.talk(tool_attempts: 25)` | `set :tool_budget => 50` (disabled by default) |
1469
- | `LLM::LoopGuard` | `LLM::Guard::Loop` |
1470
- | `guard: true` / `ctx.guard = MyGuard` | `guard: MyGuard, guard_options: {}` |
1471
- | `guard.call(ctx)` (warning string) | `guard.call(function:)` (`LLM::Function::Return` or nil) |
1472
- | `LLM::GuardError` | `"guard_error"` |
1473
-
1474
- * **replace the transformer setter with `LLM::Transformer`** <br>
1475
- The previous `transformer=` setter and 3-argument
1476
- `call(ctx, prompt, params)` interface on `LLM::Context` have been
1477
- replaced by the new
1478
- [`LLM::Transformer`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer.html)
1479
- class interface. Configure a transformer class through `transformer:`
1480
- and options through `transformer_options:` instead.
1481
-
1482
- * **cli: scope session persistence per provider and directory** <br>
1483
- `bin/llm.rb` no longer shares a single session file between providers.
1484
- Each provider now has a `~/.llm.rb/<provider>.json` file that maps the
1485
- current working directory to a UUID-scoped session file under
1486
- `~/.llm.rb/<provider>/<uuid>.json`, so sessions are scoped to both the
1487
- provider and the directory they were started in.
1488
-
1489
- * **cli: harden the executable against bad inputs** <br>
1490
- `bin/llm.rb` now prints an error message followed by the help menu and
1491
- exits with status 1 when the `-p` switch is given without an argument or
1492
- when an unknown option is passed. Previously unknown options produced a
1493
- warning but the run continued. The session-file lookup also no longer
1494
- rewrites `~/.llm.rb/<provider>.json` when it already exists.
1495
-
1496
- * **agent: replace `tool_attempts` with the `tool_budget` class DSL** <br>
1497
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
1498
- replaces the `tool_attempts` parameter with a `tool_budget` class DSL
1499
- (`tool_budget { 50 }`) that caps the number of tool calls allowed in a
1500
- single turn. Once the budget is spent, the agent sends an in-band
1501
- advisory message back through the model telling it to solve the problem
1502
- with fewer tool calls.
1503
- <br><br>
1504
- The feature is now disabled by default; the old `tool_attempts`
1505
- parameter defaulted to 25, which long-horizon agents could easily
1506
- exhaust in a single turn.
1507
-
1508
- * **guard: replace `LLM::LoopGuard` with the `LLM::Guard` class hierarchy** <br>
1509
- [`LLM::Guard`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html)
1510
- is a new superclass for context-level supervisors, with
1511
- [`LLM::Guard::Loop`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard/Loop.html)
1512
- (replacing `LLM::LoopGuard`) and
1513
- [`LLM::Guard::Null`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard/Null.html)
1514
- as the built-in implementations.
1515
- [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
1516
- now accepts `guard:` (a guard class defaulting to `LLM::Guard::Null`)
1517
- and `guard_options:` (a hash forwarded to the guard's `call` method),
1518
- matching the transformer and compactor interfaces. The old boolean and
1519
- hash forms of `guard` and the `guard=` setter are removed. `LLM::Agent`
1520
- enables `LLM::Guard::Loop` by default.
1521
-
1522
- * **guard: block individual tool calls instead of the whole batch** <br>
1523
- [`LLM::Guard#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html#call-instance_method)
1524
- now receives the pending `function:` and returns an
1525
- [`LLM::Function::Return`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Return.html)
1526
- (or nil) instead of a warning string for the entire batch, so a guard
1527
- can block a single tool call while the rest of the batch still
1528
- executes. Custom guards that implemented the old `call(ctx)`
1529
- warning-string interface must be updated to return a
1530
- `LLM::Function::Return` instead.
1531
-
1532
- * **errors: drop `LLM::GuardError`** <br>
1533
- Remove `LLM::GuardError`. The constant was never raised as an
1534
- exception; it only named the in-band error type for guarded tool
1535
- returns. Guarded tool returns now use the string `"guard_error"` as
1536
- their error type.
1537
-
1538
- ### Core
1539
-
1540
- * **add `LLM::Provider#build_messages` for assembling outgoing messages** <br>
1541
- [`LLM::Provider#build_messages`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#build_messages-instance_method)
1542
- normalizes a prompt into `LLM::Message` objects and prepends the existing
1543
- history, replacing the per-provider `build_complete_messages`
1544
- implementation. The method is idempotent: prompts that are already
1545
- [`LLM::Message`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html)
1546
- instances or arrays of messages are returned as-is.
1547
-
1548
- * **copy the `params` hash in `LLM::Context` and `LLM::Agent`** <br>
1549
- [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
1550
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now copy
1551
- the `params` hash in their constructors before mutating it, leaving the
1552
- caller's hash untouched. Previously the constructors deleted keys from
1553
- the caller's hash in place.
1554
-
1555
- * **gemspec: ship the deepdive sub-files in the gem** <br>
1556
- The gemspec now includes `resources/deepdive/*/*.md` in the gem
1557
- package, so the full deepdive guide (fundamentals, advanced,
1558
- protocols, and everything-else chapters) is available after
1559
- installation.
1560
-
1561
- * **add short aliases to `LLM::Cost`** <br>
1562
- [`LLM::Cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html) now
1563
- offers short aliases for its cost accessors: `input`, `output`,
1564
- `input_audio`, `output_audio`, `input_image`, `cache_read`,
1565
- `cache_write`, and `reasoning`. Each alias matches the key used by
1566
- `#to_h`, so `cost.input` reads the same value as `cost.input_costs`.
1567
-
1568
- ### Provider
1569
-
1570
- * **add `LLM::Moonshot` for the Moonshot AI provider** <br>
1571
- [`LLM::Moonshot`](https://r.uby.dev/api-docs/llm.rb/LLM/Moonshot.html)
1572
- is a new provider that talks to
1573
- [Moonshot AI](https://platform.moonshot.ai) through its
1574
- OpenAI-compatible Kimi API. Create an instance with
1575
- [`LLM.moonshot`](https://r.uby.dev/api-docs/llm.rb/LLM.html#moonshot-class_method),
1576
- which accepts the same `key:`, `host:`, and `base_path:` options as the
1577
- OpenAI provider. The provider defaults to the `kimi-k3` model and
1578
- supports chat completions, streaming, tool calls, and structured output
1579
- through the shared OpenAI-compatible path; image, audio, moderation,
1580
- responses, and vector store endpoints raise `NotImplementedError`.
1581
- Model metadata ships in `data/moonshot.json` for the registry.
1582
-
1583
- ### Transformer
1584
-
1585
- * **add `LLM::Transformer` for rewriting messages before they reach the provider** <br>
1586
- [`LLM::Transformer`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer.html)
1587
- is a new superclass for message transformers. A transformer is bound to a
1588
- context and rewrites a single message before it is sent to the provider,
1589
- which makes it possible to redact personal information or rewrite any
1590
- message before it goes out over the wire. Each subclass implements
1591
- `call(message:, **opts)` and returns the message to send, either by
1592
- mutating it in place or returning a new one.
1593
- [`LLM::Transformer::Null`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer/Null.html)
1594
- is a no-op transformer used as the default.
1595
-
1596
- * **hook the transformer API into `LLM::Context`** <br>
1597
- [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) now
1598
- accepts `transformer:` (a transformer class defaulting to
1599
- `LLM::Transformer::Null`) and `transformer_options:` (a hash forwarded to
1600
- the transformer's `call` method). The transformer runs on the most recent
1601
- message in both chat and responses turns.
1602
- [`LLM::Stream#on_transform`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_transform-instance_method)
1603
- and
1604
- [`LLM::Stream#on_transform_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_transform_finish-instance_method)
1605
- now receive the transformer instance as their single argument.
1606
-
1607
- ### Tool
1608
-
1609
- * **add `LLM::Tool.set` for bulk-assigning tool properties** <br>
1610
- [`LLM::Tool.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#set-class_method)
1611
- accepts a hash of `name`, `description`, `parameters`, `required`, and
1612
- `defaults` to configure a tool subclass in a single call. Parameters are
1613
- defined as tuples of `[name, type, description, options]`, matching the
1614
- same interface as the existing `parameter` DSL. Unknown keys raise
1615
- `KeyError`.
1616
-
1617
- ### Function
1618
-
1619
- * **add `LLM::Function#return` for building tool returns** <br>
1620
- [`LLM::Function#return`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#return-instance_method)
1621
- returns an
1622
- [`LLM::Function::Return`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Return.html)
1623
- built from the function's own id and name, using the given hash as its
1624
- value. It is a shorthand mainly useful inside a
1625
- [`LLM::Guard`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html)
1626
- subclass and is defined via `define_method` because `return` is a Ruby
1627
- keyword.
1628
-
1629
- ### Guard
1630
-
1631
- * **run the guard for streamed tool calls** <br>
1632
- Fix a gap where the guard was not consulted when a tool call was queued
1633
- while a response was still streaming. The guard is now stamped onto the
1634
- functions a context binds, so it runs wherever a task is spawned,
1635
- including tool calls queued from a stream. A blocked call yields its
1636
- `guard_error` return without executing.
1637
-
1638
- ### Agent
1639
-
1640
- * **add `LLM::Agent#compacted?`** <br>
1641
- [`LLM::Agent#compacted?`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#compacted%3F-instance_method)
1642
- delegates to the wrapped
1643
- [`LLM::Context#compacted?`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#compacted%3F-instance_method)
1644
- and reports whether the conversation has been compacted, so callers
1645
- can detect when history was trimmed.
1646
-
1647
- ### Change
1648
-
1649
- * **openai: default to `gpt-5.6-luna`** <br>
1650
- The default OpenAI chat model has changed from `gpt-5.4-mini` to
1651
- `gpt-5.6-luna`. The new model is OpenAI's fastest and most affordable
1652
- option, matching the kind of default llm.rb aims for.
1653
-
1654
- ### Repl
1655
-
1656
- * **show an unknown context state after `/compact`** <br>
1657
- After running `/compact`, the REPL status line now renders `Context
1658
- compacted` and the context-usage bar shows `???` instead of a percentage,
1659
- because the used context is unknown until the next response.
1660
-
1661
- * **land on a blank line after Ctrl+N at the end of history** <br>
1662
- When recalling history with Ctrl+P and Ctrl+N, Ctrl+N at the last item
1663
- now advances to a blank line so you can start typing new input, instead
1664
- of staying stuck on the last item in history (the previous behavior).
1665
- Recalling with Ctrl+P or Ctrl+N also no longer overwrites the input when
1666
- there is no history to show.
1667
-
1668
- * **restore history wrap for Ctrl+P and Ctrl+N** <br>
1669
- Fix a regression where Ctrl+P and Ctrl+N recalled history text without
1670
- reflowing it into rows, so recalled lines wider than the terminal were
1671
- clipped. Recalled text now flows through the same word-wrap path as
1672
- typed input and wraps at the terminal width.
1673
-
1674
- * **restore Ctrl+D deletion across rows** <br>
1675
- Fix a bug where Ctrl+D at the end of an input row was a no-op, so
1676
- multiline input could not be joined by deleting a row break. Deleting
1677
- at the end of a row now consumes the break and pulls the next row up,
1678
- restoring the split space so merged words do not run together.
1679
-
1680
- * **center the buffer with 20% gutters** <br>
1681
- The curses-based REPL now centers
1682
- [`LLM::Repl::Buffer`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Buffer.html)
1683
- in a content area that is 60% of the terminal width, with an unused 20%
1684
- gutter on each side. The drawing area is based on the available rows and
1685
- columns instead of a fixed 80-column width, and `Buffer#wrap` now
1686
- hard-breaks words that overflow the width onto the next row, fixing a
1687
- bug where a word could be cut off between rows.
1688
-
1689
- * **apply markdown to previous messages** <br>
1690
- The curses-based REPL now renders every message in the buffer with
1691
- markdown styling, including messages that were already present when
1692
- the session started or restored from disk. Previously only newly
1693
- streamed responses were styled; older messages fell back to plain
1694
- text.
1695
-
1696
- * **add `LLM::Repl#sender` for the user label** <br>
1697
- [`LLM::Repl#sender`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl.html#sender-instance_method)
1698
- returns the label used for user messages in the curses-based REPL. It
1699
- defaults to `"You"` (previously `"user"`), and the buffer layout now
1700
- places each label on its own line followed by the message content and a
1701
- blank line.
1702
-
1703
- * **add `LLM::Repl::Color` for coloring the curses UI** <br>
1704
- Add
1705
- [`LLM::Repl::Color`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Color.html)
1706
- as a new module that returns Curses color bitmasks. `Color.enable`
1707
- initializes 8 color pairs, and methods like `Color.blue` return the
1708
- corresponding `Curses.color_pair(X)` bitmask, which can be bitwise
1709
- OR'ed with other attributes such as `Curses::A_BOLD`. User labels in
1710
- the REPL are now rendered in blue instead of plain bold text.
1711
-
1712
- * **split on words rather than characters** <br>
1713
- `LLM::Repl::Buffer#wrap` now breaks text on word boundaries instead of
1714
- wrapping one character at a time. A word that does not fit on the
1715
- current row moves to the next, and only a single word longer than the
1716
- whole width is hard-broken, so text is never clipped by the window.
1717
-
1718
- * **render kramdown typographic symbols and smart quotes** <br>
1719
- Fix a bug where certain character sequences such as `...` were not
1720
- rendered at all in the curses-based REPL. Kramdown parses them into
1721
- `:typographic_sym` and `:smart_quote` nodes, which previously fell
1722
- through to the children clause and were dropped. The markdown renderer
1723
- now maps them to their unicode equivalents: ellipsis, en and em
1724
- dashes, guillemets, and single and double quotation marks.
1725
-
1726
- * **apply colors to the markdown renderer** <br>
1727
- The curses-based REPL now renders markdown with the `LLM::Repl::Color`
1728
- palette: headers and strong text in white, code spans and code blocks
1729
- in green, and links in underlined green, on the black background.
1730
- Previously markdown styling used bold, underline, and reverse video
1731
- attributes only.
1732
-
1733
- * **wrap the input line at word boundaries** <br>
1734
- The curses-based REPL input line now wraps words whole onto the next
1735
- row at the terminal width instead of cutting them in half. A word that
1736
- does not fit on the current row moves to the next row, and only a
1737
- single word longer than the whole width is hard-broken, so typed text
1738
- is never clipped by the window.
1739
-
1740
- * **distinguish the connecting and thinking status bar phases** <br>
1741
- The curses-based REPL status bar now shows `Connecting • Esc to
1742
- cancel` while the model is establishing a connection, then switches
1743
- to `Thinking • Esc to cancel` once a tool call or text fragment
1744
- arrives on the stream. Active tool calls appear in the status bar
1745
- with a lambda indicator.
1746
-
1747
- * **add emoji to the status bar phases** <br>
1748
- The curses-based REPL status bar now uses emoji to identify each
1749
- phase at a glance: a globe (`🌐`) while the model is connecting, and
1750
- a brain (`🧠`) while it is thinking. The text after the emoji still
1751
- reads `Connecting • Esc to cancel` and `Thinking • Esc to cancel`
1752
- respectively.
1753
-
1754
- * **render the status bar with color and attributes** <br>
1755
- The curses-based REPL status bar now supports colored and attributed
1756
- status text, so the lambda indicator for active tool calls is drawn
1757
- in bold red.
1758
-
1759
- ### Registry
1760
-
1761
- * **refresh model metadata across providers** <br>
1762
- Update `data/*.json` files with current provider model listings and
1763
- pricing. Remove the deprecated Claude Opus 4.1 entries from the
1764
- Anthropic registry, add `Qwen/Qwen3.8-Max` to DeepInfra, and add a
1765
- `low` reasoning-effort option to DeepSeek.
1766
-
1767
- ## v13.1.0
1768
-
1769
- Changes since `v13.0.0`.
1770
-
1771
- This release adds `LLM::Agent` class DSL attributes (`path`, `description`),
1772
- extends skills with file-path loading and the `tools: all` directive, adds new
1773
- built-in tools (`LLM::Tool::Ruby`, `LLM::Tool::EditFile`), introduces the
1774
- `LLM::Tracer::PrettyLogger` for human-readable tracing, renames `Transcript`
1775
- to `Buffer` across the REPL, ships a `bin/llm.rb` CLI entry point, and fixes
1776
- several agent and tool bugs around persistence, interruption, and naming.
1777
-
1778
- ### Core
1779
-
1780
- * **add post install message with deepdive link** <br>
1781
- The gemspec now includes a `post_install_message` that points users to
1782
- the deepdive guide at `https://r.uby.dev/llm/deepdive` after installation,
1783
- making it easier for new users to discover the project documentation.
1784
-
1785
- ### Agent
1786
-
1787
- * **add `description` class DSL and instance method** <br>
1788
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now
1789
- has a `description` class DSL (`description "release engineer"`) and a
1790
- corresponding `#description` instance method. The description is an
1791
- optional self-documenting string that serves as a brief summary of the
1792
- agent's purpose. It can be set via the class DSL,
1793
- `LLM::Agent.set(description: ...)`, or `LLM::Agent.new(description: ...)`.
1794
-
1795
- * **add `path` class DSL and instance method** <br>
1796
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now
1797
- has a `path` class DSL (`path "contexts/admin.json"`) and a
1798
- corresponding `#path` instance method. When a path is set, the agent
1799
- automatically restores its conversation history from that file on
1800
- initialization and saves it back after each `talk` or `ask` turn,
1801
- making session persistence across process restarts transparent.
1802
-
1803
- ### Skills
1804
-
1805
- * **accept a path to a markdown file** <br>
1806
- [`LLM::Skill.load`](https://r.uby.dev/api-docs/llm.rb/LLM/Skill.html#load-class_method)
1807
- now accepts a path to a markdown file in addition to a directory path.
1808
- When given a file path, the file is read directly instead of looking for
1809
- a `SKILL.md` inside a directory. This makes it possible to load a single
1810
- markdown file as a skill without placing it in a dedicated directory.
1811
-
1812
- * **extend with `all` keyword for loading the full tool registry** <br>
1813
- `LLM::Skill` now supports `tools: all` (or `tools: "*"`) in the frontmatter
1814
- to load all tools from the global
1815
- [`LLM::Tool.registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#registry-class_method).
1816
- Previously, the `tools:` frontmatter only accepted `inherit`, an array of tool
1817
- names, or nothing. The new `all` keyword makes it possible to give a skill
1818
- access to every registered tool without listing them individually.
1819
-
1820
- ### Tools
1821
-
1822
- * **add `LLM::Tool::Ruby` for executing Ruby code in a subprocess** <br>
1823
- [`LLM::Tool::Ruby`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Ruby.html)
1824
- is a new built-in tool that runs a string of Ruby code in a separate
1825
- Ruby process with a configurable timeout (default 15s). The code runs
1826
- in an isolated address space unaware of its parent, making it useful
1827
- for safe(ish) dynamic code execution. It must be required explicitly
1828
- with `require "llm/tools/ruby"` and requires the `test-cmd.rb` gem.
1829
-
1830
- * **rename `LLM::Tool::SwapText` to `LLM::Tool::EditFile`** <br>
1831
- The `SwapText` tool has been renamed to
1832
- [`LLM::Tool::EditFile`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/EditFile.html)
1833
- to better match the naming of sibling tools (`ReadFile`, `WriteFile`).
1834
- The old `require "llm/tools/swap_text"` path no longer exists; use
1835
- `require "llm/tools/edit-file"` instead.
1836
-
1837
- ### Tracer
1838
-
1839
- * **add `LLM::Tracer::PrettyLogger` for human-readable tracing** <br>
1840
- [`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html)
1841
- is a new tracer that writes human-readable request and tool-call logs to a
1842
- console or file. Unlike the structured JSON output of
1843
- `LLM::Tracer::Logger`, the pretty logger emits single-line entries with
1844
- inline context, making it easier to follow agent activity at a glance.
1845
- It writes to `$stderr` by default and accepts an `io:` option for file
1846
- output.
1847
-
1848
- ### Repl
1849
-
1850
- * **rename `LLM::Repl::Transcript` to `LLM::Repl::Buffer`** <br>
1851
- `LLM::Repl::Transcript` has been renamed to
1852
- [`LLM::Repl::Buffer`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Buffer.html)
1853
- to better reflect its role as a conversation state manager. The old
1854
- `start` and `finish` methods have been renamed to `open` and `close`
1855
- respectively. The public accessor on
1856
- [`LLM::Repl`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl.html) has been
1857
- renamed from `transcript` to `buffer`.
1858
-
1859
- * **add `write_message` for formatted message writing** <br>
1860
- [`LLM::Repl::Buffer#write_message`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Buffer.html#write_message-instance_method)
1861
- and
1862
- [`LLM::Repl#write_message`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl.html#write_message-instance_method)
1863
- provide a convenience method that takes a username and content string,
1864
- formatting the output with a bold `user:` label and a trailing newline.
1865
- This is simpler than the equivalent sequence of `write` calls.
1866
-
1867
- * **add `Command#write_message` and refactor `Command#write`** <br>
1868
- [`LLM::Command#write_message`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html#write_message-instance_method)
1869
- provides a convenience method that takes a username and content string,
1870
- matching the same interface on `LLM::Repl` and `LLM::Buffer`. The
1871
- `Command#write` method is now implemented on top of `write_message`,
1872
- always prefixing output with `command(<name>): `. The `who:` keyword
1873
- argument previously accepted by `write` has been removed; use
1874
- `write_message` instead.
1875
-
1876
- * **display pre-existing agent messages when the repl starts** <br>
1877
- When
1878
- [`LLM::Agent#repl`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
1879
- starts, any messages already in the agent's buffer are now rendered
1880
- in the REPL window. Previously the REPL started with an empty
1881
- transcript even when the agent carried prior conversation history,
1882
- making it harder to resume a session. Tool-call and tool-return
1883
- messages are skipped to avoid visual noise.
1884
-
1885
- ### CLI
1886
-
1887
- * **add `bin/llm.rb` for launching the REPL from the command line** <br>
1888
- A new executable script (`bin/llm.rb`) provides a convenient way to start
1889
- an interactive REPL session directly from the terminal. It auto-detects
1890
- the provider from environment variables like `OPENAI_API_KEY`, supports
1891
- a `-p PROVIDER` flag for explicit provider selection, a `-t` flag for
1892
- temporary (non-persistent) sessions, and `-h` for help. Sessions are
1893
- automatically saved to `~/.llm.rb/` by default.
1894
-
1895
- ### Fix
1896
-
1897
- * **agent: fix `path` restore on first run** <br>
1898
- Fix a bug where [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
1899
- called `@ctx.restore(path:)` even when the path's file did not exist.
1900
- The fix checks `File.readable?(@path)` before attempting to restore,
1901
- so the agent starts with a blank conversation on first use instead of
1902
- failing with a file-not-found error.
1903
-
1904
- * **tools: re-raise `LLM::Interrupt` to abort the turn** <br>
1905
- [`LLM::Tool::Git`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Git.html),
1906
- [`LLM::Tool::Mkdir`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Mkdir.html),
1907
- [`LLM::Tool::Rg`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html),
1908
- [`LLM::Tool::Ruby`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Ruby.html),
1909
- and
1910
- [`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html)
1911
- now re-raise `LLM::Interrupt` after killing their running command. The
1912
- previous behavior rescued the interrupt and killed the child process but
1913
- let the turn continue, which meant a cancelled tool call did not abort
1914
- the conversation turn. Re-raising ensures the entire turn is interrupted.
1915
-
1916
- * **tools: rescue `LLM::Interrupt` in shell-based tools** <br>
1917
- [`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html),
1918
- [`LLM::Tool::Git`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Git.html),
1919
- [`LLM::Tool::Mkdir`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Mkdir.html),
1920
- and
1921
- [`LLM::Tool::Rg`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html)
1922
- now rescue `LLM::Interrupt` and kill their running command, preventing
1923
- orphaned child processes when a tool is interrupted during execution.
1924
-
1925
- * **agent: fix default name derivation** <br>
1926
- Fix a bug where `LLM::Agent` used without a subclass derived its default
1927
- name as `"l-lm-agent"` instead of `"agent"`. The fix replaces the
1928
- regex-based parameterization with a pattern that correctly handles
1929
- single-word class names and multi-word namespaced names.
1930
-
1931
- * **function: `#params` always returns an `LLM::Object`** <br>
1932
- [`LLM::Function#params`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#params-instance_method)
1933
- now always returns an `LLM::Object` representing the function's parameter
1934
- schema. Previously it returned `nil` when a function defined no parameters,
1935
- forcing every caller to guard against `nil`. All provider adapters now use
1936
- `fn.params.to_h` instead of `fn.params || {type: "object", properties: {}}`.
1937
-
1938
- ## v13.0.0
1939
-
1940
- v13.0.0 is released under the MIT license. Commercial, personal,
1941
- educational, and all other uses are permitted under the standard
1942
- MIT terms.
1943
-
1944
- Seven breaking changes. Concurrency strategies have been renamed
1945
- (`:call` → `:sequential`, `:task` → `:async`), `spawn` is now
1946
- `task`, and the `:async` strategy has been rebuilt from the ground
1947
- up. It no longer blocks and now supports interruption. The compactor
1948
- has been refactored into pluggable strategies. Interruption is now
1949
- reliable across all six concurrency backends. The `functions` and
1950
- `functions?` methods have been renamed to `pending_functions` and
1951
- `pending_functions?`.
1952
-
1953
- ### Migration from v12.6.0
1954
-
1955
- | Old | New |
1956
- |-----|-----|
1957
- | `fn.spawn(:call)` | `fn.task(:sequential)` |
1958
- | `fn.spawn(:task)` | `fn.task(:async)` |
1959
- | `ctx.wait(:call)` | `ctx.wait(:sequential)` |
1960
- | `agent.concurrency :task` | `agent.concurrency :async` |
1961
- | `LLM::Function::FiberGroup` | `LLM::Function::Fiber::Group` |
1962
- | `LLM::Function::CallGroup` | `LLM::Function::Sequential::Group` |
1963
- | `LLM::Function::TaskGroup` | `LLM::Function::Async::Group` |
1964
- | `Compactor.new(model:, token_threshold:)` | `Compactor::Truncate.new(ctx)` |
1965
- | `on_compaction(ctx, compactor)` | `on_compaction(compactor)` |
1966
- | `ctx.functions` / `ctx.functions?` | `ctx.pending_functions` / `ctx.pending_functions?` |
1967
- | `agent.functions` / `agent.functions?` | `agent.pending_functions` |
1968
-
1969
- ### Breaking
1970
-
1971
- * **rename `LLM::Function#spawn` as `LLM::Function#task`** <br>
1972
- [`LLM::Function#task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#task-instance_method)
1973
- (previously `spawn`) now consistently returns a
1974
- [`LLM::Function::Task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Task.html)
1975
- object that can be spawned, waited on, and passed to
1976
- [`LLM::Function::Group`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Group.html).
1977
- The old implementation alternated between spawning immediately or
1978
- returning a raw thread or fiber.
1979
-
1980
- * **rename concurrency strategies (`:call` → `:sequential`,**
1981
- **`:task` → `:async`)** <br>
1982
- The `:call` concurrency strategy is now `:sequential`, and the `:task`
1983
- strategy is now `:async`.
1984
- [`LLM::Agent.concurrency`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#concurrency-class_method),
1985
- [`LLM::Context#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#wait-instance_method),
1986
- `LLM::Function::Array#task`, and
1987
- [`LLM::Function#task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#task-instance_method)
1988
- all accept the new names. The old names raise `ArgumentError`.
1989
-
1990
- * **rename group classes** <br>
1991
- Group classes have been moved into their strategy's namespace:
1992
- `FiberGroup` → `Fiber::Group`, `ThreadGroup` → `Thread::Group`,
1993
- `CallGroup` → `Sequential::Group`, `TaskGroup` → `Async::Group`,
1994
- `Fork::Group` → `Fork::Group`, `Ractor::Group` → `Ractor::Group`.
1995
-
1996
- * **repurpose `LLM::Function::Task` as a task interface superclass** <br>
1997
- `LLM::Function::Task` has been repurposed from a general-purpose class
1998
- that tried to support multiple concurrency strategies into an abstract
1999
- base class that defines the task interface. Individual strategies
2000
- (`Sequential::Task`, `Thread::Task`, `Fiber::Task`, `Async::Task`,
2001
- `Fork::Task`, `Ractor::Task`) now subclass it and implement
2002
- `spawn`, `alive?`, `interrupt!`, and `wait`.
2003
-
2004
- * **fix `:async` concurrency (now backed by a managed**
2005
- **`LLM::Function::Async::Reactor` on a background thread)** <br>
2006
- The `:async` strategy previously used `Async {}` which blocked the
2007
- caller until all tasks completed and did not support interruption.
2008
- The fix replaces it with a per-turn
2009
- `LLM::Function::Async::Reactor` on a background thread. Work is
2010
- submitted via `submit(&block)` and consumed by the reactor's event
2011
- loop through a thread-safe `Queue`. `Async::Group` manages the
2012
- reactor lifecycle and spawns tasks lazily on `wait`.
2013
- <br><br>
2014
- Interruption pushes [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
2015
- to the task's result queue instead of using `Fiber#raise`, and
2016
- results are bridged back to the caller through a second `Queue`.
2017
- This work drove the broader refactor of strategy naming, the
2018
- spawn/wait split, and the `Task` superclass. The `:async` strategy
2019
- needed the same interface the other strategies already had.
2020
-
2021
- * **compactor: refactor to strategy-based interface** <br>
2022
- [`LLM::Compactor`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor.html)
2023
- has been refactored from a single class that performed LLM-based
2024
- summarization into a strategy-based superclass. Each subclass
2025
- implements a different compaction strategy via `call(**opts)`. The old
2026
- summarization approach (using `model:`, `token_threshold:`,
2027
- `message_threshold:`, and `retention_window:` options) has been removed.
2028
- The built-in
2029
- [`LLM::Compactor::Truncate`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor/Truncate.html)
2030
- strategy drops the oldest messages when the conversation exceeds a
2031
- configured size.
2032
-
2033
- * **rename `LLM::Context#{functions,functions?}` and `LLM::Agent#{functions,functions?}`** <br>
2034
- [`LLM::Context#functions`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#functions-instance_method)
2035
- and
2036
- [`LLM::Context#functions?`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#functions%3F-instance_method)
2037
- have been renamed to
2038
- [`LLM::Context#pending_functions`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#pending_functions-instance_method)
2039
- and
2040
- [`LLM::Context#pending_functions?`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#pending_functions%3F-instance_method)
2041
- respectively. The same rename applies to
2042
- [`LLM::Agent#functions`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#functions-instance_method)
2043
- (now
2044
- [`LLM::Agent#pending_functions`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#pending_functions-instance_method)).
2045
- The `pending_functions` name was already available as an alias in
2046
- v12.5.0; this change removes the old `functions` name entirely.
2047
-
2048
- ### Core
2049
-
2050
- * **extend `LLM.require` with an optional version argument** <br>
2051
- `LLM.require` now accepts a second `version` parameter that is passed
2052
- to `Kernel#gem` before loading, enabling version constraints for
2053
- optional runtime dependencies. For example,
2054
- `LLM.require "test-cmd.rb", "~> 2.2"` ensures a minimum gem version
2055
- is available. This is used internally by the `Git`, `Rg`, `Mkdir`,
2056
- and `Shell` tools to enforce compatibility with the `test-cmd.rb` gem.
2057
-
2058
- ### Compactor
2059
-
2060
- * **add `Truncate` strategy for dropping oldest messages** <br>
2061
- [`LLM::Compactor::Truncate`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor/Truncate.html)
2062
- is a new built-in compaction strategy that drops the oldest messages
2063
- when the conversation exceeds a configured size. It preserves tool
2064
- call/return pairs so the algorithm never breaks in the middle of a
2065
- sequence. Configured with `keep:` (default 64), it emits the standard
2066
- `on_compaction` and `on_compaction_finish`
2067
- [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
2068
- lifecycle callbacks. No LLM call is made; the strategy is purely
2069
- lossy but fast and requires no network.
2070
-
2071
- * **raise when given an unparseable `keep:` value** <br>
2072
- `LLM::Compactor::Truncate` now raises `ArgumentError` when the `keep:`
2073
- parameter cannot be parsed as an integer or percentage string, instead
2074
- of failing with an obscure error later during execution.
2075
-
2076
- * **accept percentage string for the `keep:` parameter** <br>
2077
- `LLM::Compactor::Truncate#call` now accepts a percentage string such
2078
- as `"80%"` for the `keep:` parameter, which keeps approximately 80%
2079
- of the most recent messages. Integer values continue to work as
2080
- before. This makes it easy to trim proportionally rather than to an
2081
- absolute number of messages.
2082
-
2083
- * **add `Null` strategy for no-op compaction** <br>
2084
- [`LLM::Compactor::Null`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor/Null.html)
2085
- is a new built-in compaction strategy that does nothing. It is used as
2086
- the default compactor when no strategy is configured on a context,
2087
- ensuring the compactor interface is always present without requiring a
2088
- separate nil check.
2089
-
2090
- * **accept both `LLM::Agent` and `LLM::Context`** <br>
2091
- [`LLM::Compactor#initialize`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor.html#initialize-instance_method)
2092
- now accepts both `LLM::Agent` and `LLM::Context` instances. When given
2093
- an agent, the internal context is unwrapped automatically, making the
2094
- compactor API more flexible when working with agents.
2095
-
2096
- #### Context integration
2097
-
2098
- * **accept `compactor` and `compactor_options` parameters** <br>
2099
- [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
2100
- now accepts `compactor:` (a compactor class defaulting to
2101
- [`LLM::Compactor::Null`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor/Null.html))
2102
- and `compactor_options:` (a hash of options forwarded to the
2103
- compactor's `call` method) parameters. The compactor is automatically
2104
- invoked at the beginning of each `talk` turn. The previous `compactor=`
2105
- setter has been removed in favour of constructor-driven configuration.
2106
-
2107
- * **`on_compaction` and `on_compaction_finish` receive a single argument** <br>
2108
- [`LLM::Stream#on_compaction`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction-instance_method)
2109
- and
2110
- [`LLM::Stream#on_compaction_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction_finish-instance_method)
2111
- now accept a single argument (the compactor instance) instead of two
2112
- arguments (context and compactor). The context is still available via
2113
- `LLM::Compactor#ctx`, so access to the context is not lost. This
2114
- simplifies the callback interface for compaction lifecycle observers.
2115
-
2116
- ### Tools
2117
-
2118
- * **add `LLM::Tool::Utils` module for shared command execution logic** <br>
2119
- A new
2120
- [`LLM::Tool::Utils`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Utils.html)
2121
- module provides shared `wait(command:, timeout:)` and `now` helper
2122
- methods for tools that execute commands. Tools that include `Utils` can
2123
- wait on a running command and automatically kill it when it exceeds the
2124
- configured timeout, using `Process.clock_gettime` with `CLOCK_MONOTONIC`
2125
- for precise timing. The module is used by both the `Shell` and `Rg`
2126
- tools internally.
2127
-
2128
- * **shell: add `timeout` parameter for command execution deadlines** <br>
2129
- The
2130
- [`LLM::Tool::Shell`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Shell.html)
2131
- tool now accepts a `timeout` parameter (default 60s) that automatically
2132
- kills commands exceeding the specified time limit, preventing hung
2133
- processes from blocking the agent indefinitely.
2134
-
2135
- * **rg: add `timeout` parameter for search execution deadlines** <br>
2136
- The
2137
- [`LLM::Tool::Rg`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Rg.html)
2138
- tool now accepts a `timeout` parameter (default 5s) that automatically
2139
- kills search commands exceeding the specified time limit, preventing
2140
- long-running searches from blocking the agent indefinitely.
2141
-
2142
- * **git: add `timeout` parameter for command execution deadlines** <br>
2143
- The
2144
- [`LLM::Tool::Git`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Git.html)
2145
- tool now accepts a `timeout` parameter (default 5s) that automatically
2146
- kills git commands exceeding the specified time limit, preventing hung
2147
- processes from blocking the agent indefinitely.
2148
-
2149
- ### Schema
2150
-
2151
- * **properties are now ordered and support indifferent access** <br>
2152
- [`LLM::Schema::Leaf`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema/Leaf.html)
2153
- tracks property definition order in a new `index` attribute, matching
2154
- the convention already used by
2155
- [`LLM::Command::Parameter`](https://r.uby.dev/api-docs/llm.rb/LLM/Command/Parameter.html).
2156
- Internally, `@properties` is stored as an
2157
- [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
2158
- instead of a plain `Hash`, so lookups with both string and symbol keys
2159
- work.
2160
-
2161
- ### Buffer
2162
-
2163
- * **more array-like message management** <br>
2164
- [`LLM::Buffer`](https://r.uby.dev/api-docs/llm.rb/LLM/Buffer.html)
2165
- now exposes `first`, `reject!`, `select!`, `shift`, `clear`, `drop`,
2166
- `take`, and `reverse`, making it easier to query and mutate
2167
- `LLM::Context#messages` like an ordinary Array. `reject!` is aliased
2168
- as `delete_if` for familiarity.
2169
-
2170
- * **`last(nil)` no longer returns the last message** <br>
2171
- `LLM::Buffer#last` now uses an internal `UNDEFINED` sentinel to
2172
- distinguish between no argument (`last` returns the last message)
2173
- and `nil` (`last(nil)` is treated as an argument). Previously `nil`
2174
- was indistinguishable from no argument.
2175
-
2176
- ### Function
2177
-
2178
- * **consolidate `call` and `call!` into one method** <br>
2179
- The private `call!` method has been merged into the public
2180
- [`LLM::Function#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#call-instance_method).
2181
- The separate `call!` method existed for tracer-scoping logic now
2182
- handled directly inside `call`. All internal call sites now use
2183
- `function.call` instead of `function.call!`.
2184
-
2185
- * **add `LLM::Function::Group` as an abstract base class** <br>
2186
- A new abstract base class
2187
- ([`LLM::Function::Group`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Group.html))
2188
- defines the interface that all concurrency strategy groups must
2189
- implement: `alive?`, `interrupt!`, and `wait`. Each strategy group
2190
- (`Sequential::Group`, `Thread::Group`, `Fiber::Group`,
2191
- `Async::Group`, `Fork::Group`, `Ractor::Group`) now subclasses
2192
- this base.
2193
-
2194
- * **split `spawn` and `wait` across all strategies** <br>
2195
- `spawn` now starts execution without blocking, and `wait` collects
2196
- the result. `on_tool_start` moved into each task's `spawn` so the
2197
- tracer span covers execution rather than construction. Each task and
2198
- group now exposes a public `spawn` method alongside the existing
2199
- `wait`/`value` methods.
2200
-
2201
- * **spawn tasks lazily in `Group#wait`** <br>
2202
- All concurrency strategy groups (Fiber::Group, Fork::Group,
2203
- Ractor::Group, Thread::Group) now automatically spawn their tasks
2204
- when `wait` is called if they haven't been spawned yet, matching
2205
- the existing `Async::Group` behavior. This makes the spawn/wait
2206
- contract consistent across all six concurrency backends.
2207
-
2208
- ### Agent
2209
-
2210
- * **add `name` class DSL and instance method** <br>
2211
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now
2212
- has a `name` class DSL (`name "admin"`) and a corresponding `#name`
2213
- instance method. The name is resolved through the same lazy-resolution
2214
- path as other agent attributes. It can be set via
2215
- `LLM::Agent.set(name: ...)`, `LLM::Agent.new(name: ...)`, or the class
2216
- DSL. When no name is given, a default is derived from the class name
2217
- (e.g., `SystemAdmin` becomes `system-admin`). The REPL uses the name
2218
- as the prompt label and transcript prefix, making it easier to
2219
- distinguish multiple sessions.
2220
-
2221
- ### REPL
2222
-
2223
- * **agent identity in the prompt** <br>
2224
- [`LLM::Agent#repl`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
2225
- and
2226
- [`LLM::Repl.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl.html#initialize-instance_method)
2227
- accept a `name:` parameter (defaulting to `LLM::Agent#name`) that sets
2228
- the input prompt to `provider(name)> ` and labels transcript messages
2229
- with the agent's name instead of a hardcoded `agent:`. Useful when
2230
- running multiple sessions.
2231
-
2232
- * **`/compact` command** <br>
2233
- New built-in `/compact` command frees context window space by dropping
2234
- the oldest messages via
2235
- [`LLM::Compactor::Truncate`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor/Truncate.html).
2236
- Supports both integer (`/compact 32`) and percentage (`/compact 75%`)
2237
- arguments. Defaults to keeping the last 128 messages.
2238
-
2239
- * **tab-completion for `/` commands** <br>
2240
- Pressing Tab on an input line starting with `/` autocompletes the
2241
- command name. Repeated Tab presses cycle through matching commands.
2242
- Powered by
2243
- [`LLM::Command.complete(str)`](https://r.uby.dev/api-docs/llm.rb/LLM/Command.html#complete-class_method)
2244
- which is available outside the REPL too.
2245
-
2246
- * **command system enhancements** <br>
2247
- Commands can set parameter defaults in their `call` method signature
2248
- (e.g., `def call(n: 128)`). Aliases like `/quit` now inherit their
2249
- parent's description and parameters. Commands also have access to
2250
- the active `agent` and `repl` via public readers.
2251
-
2252
- * **tool argument sorting** <br>
2253
- Tool parameters in the status bar are now displayed in definition
2254
- order (using the new `index` attribute), regardless of the order
2255
- the model returns them.
2256
-
2257
- * **expanded markdown rendering** <br>
2258
- The curses-based markdown renderer now handles lists (`<ul>`, `<ol>`),
2259
- blockquotes, horizontal rules, hyperlinks (underline), images
2260
- (`[image: alt text]`), and tables (aligned columns).
2261
-
2262
- * **input improvements** <br>
2263
- Ctrl+P and Ctrl+N walk through conversation history (user messages
2264
- only, managed by
2265
- [`LLM::Repl::Walker`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Walker.html)).
2266
- Page Up/Down scroll the transcript by a page. ENTER and BACKSPACE are
2267
- now mapped to raw character codes from `Curses.getch` instead of
2268
- `Curses::Key` constants.
2269
-
2270
- ### Object
2271
-
2272
- * **preserve the original key name in `KeyError` messages** <br>
2273
- `LLM::Object#fetch` now preserves the original key name when a key
2274
- is not found, instead of raising `KeyError` with `key not found: nil`.
2275
- The previous behavior occurred when the given key was not found in
2276
- the stored hash, causing internal lookup to return `nil` and lose
2277
- the original key reference.
2278
-
2279
- ### Registry
2280
-
2281
- * **refresh model metadata across providers** <br>
2282
- Update `data/*.json` files with current provider model listings and
2283
- pricing. Mark several DeepInfra models as deprecated (`meta-llama/
2284
- Meta-Llama-3.1-8B-Instruct`, `Qwen/Qwen1.5-110B-Chat`, and
2285
- `mistralai/Mixtral-8x7B-Instruct-v0.1`). Correct xAI cache-read
2286
- pricing from $0.50 to $0.30 per million input tokens.
2287
-
2288
- ### Fix
2289
-
2290
- * **agent: fix default name resolution when name is not explicitly set** <br>
2291
- Fix a bug where `LLM::Agent` derived its default name from
2292
- `self.class` instead of `self`, causing the name to be `"class"`
2293
- instead of a parameterized version of the actual class name
2294
- (e.g., `"system-admin"` for `SystemAdmin`). The fix uses `self`
2295
- directly, which correctly resolves the class name at the instance
2296
- level.
2297
-
2298
- * **function: make Fork::Task and Ractor::Task inherit LLM::Function::Task** <br>
2299
- `LLM::Function::Fork::Task` and `LLM::Function::Ractor::Task` now
2300
- explicitly subclass `LLM::Function::Task` and accept an options hash
2301
- as their second argument, matching the constructor signature used
2302
- by the other four task classes. The interface was already compatible
2303
- but the inheritance was missing by mistake. It is now consistent across
2304
- all six concurrency backends.
2305
-
2306
- * **google: fix `stream` parameter leakage that broke the provider** <br>
2307
- Fix a bug in the Google provider where `stream: stream.enabled?` was
2308
- being merged into request parameters, causing API-level errors. The
2309
- Google provider does not use a `stream` parameter. Streaming is
2310
- controlled via the URL path (`streamGenerateContent` vs
2311
- `generateContent`). The fix removes the leaked parameter and correctly
2312
- routes streaming requests through the appropriate path.
2313
-
2314
- * **fork: fix deadlock on xchan.rb channel** <br>
2315
- Fix a deadlock in the `:fork` concurrency strategy where both the
2316
- writer and reader could get stuck on the xchan channel, preventing the
2317
- reader from draining the channel. The deadlock surfaced as an errno
2318
- failure, especially with large tool returns. The fix requires xchan.rb
2319
- v0.22.0 and uses the `SOCK_STREAM` socket type for communicating a
2320
- tool's return value.
2321
-
2322
- ## v12.6.0
2323
-
2324
- Changes since `v12.5.1`.
2325
-
2326
- This release adds bulk defaults for tools and agents: `LLM::Tool.defaults`
2327
- for setting parameter defaults and `LLM::Agent.set` for mass-assigning
2328
- class-level defaults, both mirrored on ActiveRecord and Sequel agent models.
2329
-
2330
- It also makes `LLM::Interrupt` reliable across every concurrency strategy
2331
- (:thread, :call, :fiber, :task, :fork, and :ractor) so tool cancellation
2332
- works consistently regardless of execution backend, and fixes a stale fiber
2333
- reference in `LLM::Context#talk` that could prevent interruption after a
2334
- prior call.
2335
-
2336
- ### Add
2337
-
2338
- * **tool: add `defaults` method for setting parameter defaults** <br>
2339
- Add `LLM::Tool.defaults(properties)` for bulk-setting default values
2340
- on tool parameters, matching the same interface as `LLM::Schema.defaults`.
2341
- Each key maps to a parameter name; unknown keys raise `KeyError`.
2342
-
2343
- * **agent: add `set` method for bulk-assigning class-level defaults** <br>
2344
- Add `LLM::Agent.set(properties)` for mass-assigning agent defaults
2345
- from a Hash. Each key maps to a class-level accessor; unknown keys
2346
- raise `KeyError`.
2347
-
2348
- * **active_record: expose `set` on `acts_as_agent` models** <br>
2349
- ActiveRecord models using `acts_as_agent` can call `set` to
2350
- bulk-assign agent class-level defaults.
2351
-
2352
- * **sequel: expose `set` on `plugin :agent` models** <br>
2353
- Sequel models using `plugin :agent` can call `set` to bulk-assign
2354
- agent class-level defaults.
2355
-
2356
- ### Fix
2357
-
2358
- * **function: raise `LLM::Interrupt` on thread where tool is running** <br>
2359
- On cancel, `LLM::Interrupt` is now raised on the thread that is
2360
- running a tool. The tool can rescue `LLM::Interrupt` and gracefully
2361
- terminate (e.g., clean up resources). The previous approach used
2362
- `Thread#interrupt` which was less reliable. It did not interrupt a
2363
- sleeping thread.
2364
-
2365
- * **function: suppress thread exception reporting in `:thread` concurrency** <br>
2366
- Threads spawned by the `:thread` concurrency strategy now have
2367
- `report_on_exception` set to `false`, preventing noisy exception
2368
- messages from appearing on stderr when a thread is interrupted
2369
- during tool execution.
2370
-
2371
- * **context: clear `@owner` after `talk` completes** <br>
2372
- `LLM::Context#talk` now clears the `@owner` reference in an
2373
- `ensure` block after the method completes, so `interrupt!` does
2374
- not attempt to interrupt a stale fiber reference from a prior
2375
- call.
2376
-
2377
- * **function: raise `LLM::Interrupt` on thread waiting in `CallGroup#wait`** <br>
2378
- When using `ctx.wait(:call)`, `LLM::Interrupt` is now raised on the
2379
- thread executing the sequential tool wait. `CallGroup#wait` tracks the
2380
- active thread and `interrupt!` raises `LLM::Interrupt` on it, enabling
2381
- interruption of the `:call` concurrency strategy just like the existing
2382
- `:thread` strategy.
2383
-
2384
- * **function: raise `LLM::Interrupt` on fiber-backed tool tasks** <br>
2385
- `LLM::Interrupt` is now raised on the active fiber via `Fiber#raise`
2386
- when interrupting `:fiber`-concurrency tools.
2387
- <br><br>
2388
- `Task#interrupt!` now dispatches by task type: `Thread#raise` for
2389
- threads, `Fiber#raise` for fibers. Making interruption reliable
2390
- across all concurrency strategies.
2391
-
2392
- * **function: raise `LLM::Interrupt` on fork-backed tool tasks** <br>
2393
- `LLM::Interrupt` is now raised on the main thread of a fork child
2394
- process via `Thread.main.raise(LLM::Interrupt)` when interrupting
2395
- `:fork`-concurrency tools, and the fork `Task#wait` re-raises the
2396
- interrupt on the parent side. Making interruption reliable across
2397
- all concurrency strategies including `:fork`.
2398
-
2399
- * **function: raise `LLM::Interrupt` on `Async::Task`-backed tool tasks** <br>
2400
- `LLM::Interrupt` is now raised on the underlying fiber of an
2401
- `Async::Task` via `Fiber#raise` when interrupting `:task`-concurrency
2402
- tools. `Task#interrupt!` now detects `Async::Task` instances and
2403
- dispatches to `Fiber#raise`, extending reliable interruption to the
2404
- `:task` concurrency strategy under the Async runtime.
2405
-
2406
- * **function: raise `LLM::Interrupt` on ractor-backed tool tasks** <br>
2407
- `LLM::Interrupt` is now raised on the main thread inside a ractor
2408
- via `Thread.main.raise(LLM::Interrupt)` when interrupting
2409
- `:ractor`-concurrency tools. A listener thread inside the tool
2410
- ractor waits for an interrupt message via `Ractor.receive` and
2411
- raises `LLM::Interrupt` on the ractor's main thread.
2412
- <br><br>
2413
- `Task#interrupt!` delegates to the mailbox to send the interrupt
2414
- message. Extending reliable interruption to the `:ractor`
2415
- concurrency strategy.
2416
-
2417
- ## v12.5.1
2418
-
2419
- Changes since `v12.5.0`.
2420
-
2421
- This release reverts the global `LLM::Function` registry fallback for tool
2422
- resolution that was added in v12.5.0.
2423
-
2424
- ### Change
2425
-
2426
- * **function: remove the global registry fallback for tool resolution** <br>
2427
- Remove the `LLM::Function.find_by_name` fallback that was added in
2428
- v12.5.0 as an intermediate step between available-tools lookup and
2429
- raising `LLM::NoSuchToolError`. Tool calls not found in the available
2430
- tools list now go directly to `function_missing` (which raises
2431
- `LLM::NoSuchToolError`) without first checking the global
2432
- `LLM::Function` registry.
2433
-
2434
- ## v12.5.0
2435
-
2436
- Changes since `v12.4.0`.
2437
-
2438
- This release extends the REPL command system with typed parameters, a
2439
- built-in `/help` command, command aliases (`/quit`), and cancellation
2440
- via the 'Esc' key.
2441
-
2442
- The default HTTP timeout is increased to 15 minutes (900s) to better
2443
- accommodate reasoning models and large structured outputs.
2444
- `LLM::Agent#deserialize` and `LLM::Agent#restore` now return `self` for
2445
- method chaining, and `LLM::Buffer#pop` is added for tail-end message
2446
- removal.
2447
-
2448
- Tool resolution gains a fallback to the global `LLM::Function` registry
2449
- before raising `LLM::NoSuchToolError`, and `pending_functions` aliases
2450
- are added on both contexts and agents for a consistent interface.
2451
-
2452
- Several REPL bugs are fixed including parameter state leakage across
2453
- turns and invalid tool-call error routing.
2454
-
2455
- Model metadata is refreshed across all providers with new Anthropic,
2456
- OpenAI, Google, DeepInfra, DeepSeek, and xAI model entries.
2457
-
2458
- ### Add
2459
-
2460
- #### Buffer & function internals
2461
-
2462
- * **buffer: add `LLM::Buffer#pop`** <br>
2463
- Add `LLM::Buffer#pop` for removing the last message from the tail
2464
- of the buffer, complementing the existing `#<<` and array-style
2465
- message management.
2466
-
2467
- * **function: add registry fallback for tool resolution** <br>
2468
- When resolving tool calls from a message, if the tool is not found
2469
- in the available tools list, it now also looks up the global
2470
- `LLM::Function` registry via `LLM::Function.find_by_name` before
2471
- creating a placeholder function. This improves tool resolution for
2472
- tools that are registered globally but not passed directly through
2473
- the request tool set.
2474
-
2475
- #### Consistent `pending_functions` aliases
2476
-
2477
- * **context: alias `LLM::Context#functions` as `LLM::Context#pending_functions`** <br>
2478
- Add `LLM::Context#pending_functions` as an alias for `LLM::Context#functions`,
2479
- so callers that prefer the more descriptive `pending_functions` name can use
2480
- it instead of `functions` when checking for unresolved tool work.
2481
-
2482
- * **agent: alias `LLM::Agent#functions` as `LLM::Agent#pending_functions`** <br>
2483
- Add `LLM::Agent#pending_functions` as an alias for `LLM::Agent#functions`,
2484
- matching the same alias on `LLM::Context`, so callers have a consistent
2485
- `pending_functions` interface across both contexts and agents.
2486
-
2487
- #### REPL command system
2488
-
2489
- * **repl: extend command system with parameter support** <br>
2490
- Commands can now declare typed parameters using the `parameter`
2491
- DSL, modelled after `LLM::Tool` and `LLM::Schema` conventions.
2492
- Parameters can be marked as required with `required %i[...]`,
2493
- and values are type-checked before being passed to `call`.
2494
- Argument parsing is handled by the repl: arguments are split
2495
- from the input string and assigned to parameters by position.
2496
-
2497
- ```ruby
2498
- class Greeter < LLM::Command
2499
- name "greet"
2500
- description "Greets the given name"
2501
- parameter :name, String, "The person's name"
2502
- required %i[name]
2503
-
2504
- def call(name:)
2505
- write("Welcome #{name}!\n")
2506
- end
2507
- end
2508
- ```
2509
-
2510
- * **repl: add `help` command** <br>
2511
- Add `LLM::Repl::Help` as a new built-in command, registered
2512
- automatically via the command registry. Typing `/help` shows
2513
- the `help` command's own name, description, and parameters,
2514
- while `/help <name>` shows details for a specific command,
2515
- including its parameters and whether each is required or
2516
- optional. Unknown command names produce an error message.
2517
-
2518
- ```ruby
2519
- class Help < Command
2520
- name "help"
2521
- description "show help for a given command"
2522
- parameter :name, String, "The name of a command"
2523
-
2524
- def call(name: nil)
2525
- if name.nil?
2526
- write("\n#{self.class.help}\n\n")
2527
- elsif command = LLM::Command.find_by(name:)
2528
- write("\n#{command.help}\n\n")
2529
- else
2530
- write "\nNo help for #{name} was found" \
2531
- "\nThat command doesn't exist.\n\n"
2532
- end
2533
- end
2534
- end
2535
- ```
2536
-
2537
- * **repl: add support for command aliases** <br>
2538
- Commands can now be aliased by creating a subclass of another
2539
- command (with `LLM::Command` as an indirect ancestor). The
2540
- first alias introduced is `/quit` as an alias of `/exit`.
2541
-
2542
- ```ruby
2543
- class Quit < Command::Exit
2544
- name "quit"
2545
- end
2546
- ```
2547
-
2548
- * **repl: add `Command::Parameter#optional?`** <br>
2549
- Parameters now expose an `#optional?` method that returns `true`
2550
- when a parameter has not been marked as required, making it
2551
- possible to query parameter optionality programmatically.
2552
-
2553
- * **repl: add `LLM::Repl::Command#write`** <br>
2554
- Commands can now write output to the transcript via the `write`
2555
- method. Commands also receive a reference to the active repl
2556
- through their `#initialize` method, making it possible to
2557
- interact with the repl window from within a command.
2558
-
2559
- * **repl: display command errors in the curses UI** <br>
2560
- Commands invoked with too few arguments now display an error
2561
- message: `command(<name>): too few arguments`. Displayed directly in
2562
- the curses transcript area, giving immediate feedback instead
2563
- of silently failing.
2564
-
2565
- * **repl: add `LLM::Command` convenience constant** <br>
2566
- Add `LLM::Command = LLM::Repl::Command` as a shorter alias,
2567
- available once `"llm/repl"` is required.
2568
-
2569
- #### Misc
2570
-
2571
- * **repl: implement cancellation with the 'Esc' key** <br>
2572
- The curses-based REPL now supports cancelling an active model
2573
- request by pressing the 'Esc' key. When a request is in progress,
2574
- the status line shows `thinking • Esc to cancel`, and pressing
2575
- Esc calls `LLM::Agent#cancel!` to interrupt the request. The
2576
- transcript displays `request cancelled!` to confirm the
2577
- cancellation.
2578
-
2579
- ### Change
2580
-
2581
- #### Misc
2582
-
2583
- * **provider: increase default timeout to 900s** <br>
2584
- The default HTTP timeout for all providers has been increased from
2585
- 180 to 900 seconds (15 minutes) to better accommodate long-running
2586
- requests such as reasoning models and large structured outputs.
2587
-
2588
- * **agent: `deserialize` and `restore` return `self`** <br>
2589
- `LLM::Agent#deserialize` and `LLM::Agent#restore` now return `self`
2590
- (the agent instance) instead of forwarding the context's return
2591
- value, enabling method chaining after restoring agent state.
2592
-
2593
- * **context: discard all messages from a cancelled turn** <br>
2594
- When `LLM::Context#cancel!` is called, all messages added during
2595
- that turn are now discarded via `Buffer#slice!`, preventing edge
2596
- cases where dangling tool calls between turns caused repeated
2597
- cancellation loops. The `#repair!` method now handles tool call
2598
- cancellations on the next turn instead of mutating the conversation
2599
- buffer directly at cancellation time.
2600
-
2601
- * **stream: drop the `error` argument from `on_tool_call`** <br>
2602
- The `on_tool_call` callback no longer accepts an `error` argument.
2603
- Previously, stream parsers passed both a tool and an optional error,
2604
- requiring boilerplate like `if error; queue << error; end` in every
2605
- callback. Error handling is now pushed directly onto the stream queue
2606
- inside each provider's stream parser, so `on_tool_call(tool)` is the
2607
- only signature. The REPL stream and base `LLM::Stream` class have
2608
- been updated accordingly.
2609
-
2610
- #### REPL internals
2611
-
2612
- * **repl: pass the repl instance to command constructors** <br>
2613
- `LLM::Repl::Command` subclasses now receive the active repl
2614
- instance via `initialize(repl)`, enabling commands to write
2615
- to the transcript and interact with the repl window.
2616
-
2617
- * **repl: `Command#write` prefixes messages with the command name** <br>
2618
- The `#write` method now prefixes output with `command(<name>): `
2619
- so command messages are consistent with the `user:` and `agent:`
2620
- labels in the transcript. The prefix can be customised with the
2621
- `who:` keyword argument, or set to `who: nil` to disable it
2622
- entirely.
2623
-
2624
- ### Fix
2625
-
2626
- #### Misc
2627
-
2628
- * **function: avoid silent skip of tools not found in available tools** <br>
2629
- When a model calls a tool that is not present in the available tools
2630
- list, instead of silently skipping the tool call (via `next`), a
2631
- `LLM::NoSuchToolError` is now raised so the model receives feedback
2632
- about the invalid tool call and can correct course.
2633
- <br><br>
2634
- An additional fallback to the global `LLM::Function` registry is
2635
- tried before raising, so globally registered tools are still
2636
- resolved even when not in the per-request tool set.
2637
-
2638
- #### REPL bugs
2639
-
2640
- * **repl: don't persist parameter state between turns** <br>
2641
- Parameter state (such as `Parameter#value`) was leaking across
2642
- turns because the same parameter objects were being mutated
2643
- in place. A duplicate set of parameters is now created for each
2644
- turn, keeping the original parameter definitions intact and
2645
- preventing stale state from carrying over.
2646
-
2647
- * **repl: reply with error when given an invalid tool** <br>
2648
- When the model tries to call a tool that does not exist, the
2649
- error is now pushed onto the stream queue so the model can
2650
- see the error and correct course, instead of silently dropping
2651
- the invalid tool call and leaving it to `Context#repair` to
2652
- remove it from history.
2653
-
2654
- * **repl: fix save of initial runtime state** <br>
2655
- Fix a bug in `LLM::Repl#configure` where a non-existent path
2656
- argument was treated as no path at all, preventing the initial
2657
- runtime state from being saved after the first turn. The correct
2658
- behavior is to create the file so it can be written to after
2659
- the first turn completes.
2660
-
2661
- ### Refresh
2662
-
2663
- * **Refresh model metadata across all providers** <br>
2664
- Update model listings, pricing, capabilities, reasoning options,
2665
- modality support, context limits, and release dates across all
2666
- provider registries (Anthropic, AWS Bedrock, DeepInfra, DeepSeek,
2667
- Google, Mistral, OpenAI, xAI, and ZAI). Notable changes include
2668
- Anthropic claude-opus-4-8 and claude-sonnet-4-6 additions with
2669
- effort-based reasoning, OpenAI gpt-5.6-sol/terra/luna and
2670
- gpt-5-codex additions, Google gemini-3-pro-preview and
2671
- gemini-3-flash-preview additions, DeepInfra Qwen3.5 and DeepSeek
2672
- V4 model additions, and updated xAI Grok model entries.
2673
-
2674
- ## v12.4.0
2675
-
2676
- Changes since `v12.3.1`.
2677
-
2678
- This release brings major improvements to the curses-based REPL
2679
- (`LLM::Agent#repl`). The REPL now supports saving and restoring runtime
2680
- state across sessions, automatic paste-mode detection for fast bulk input,
2681
- a command system foundation with the `/exit` command, and several new
2682
- keybindings (Ctrl+F, Ctrl+K, Ctrl+Y). Tool calls are rendered with a
2683
- compact function-call syntax in the status bar.
2684
-
2685
- Two new built-in tools: `LLM::Tool::Ls` and `LLM::Tool::Which` are
2686
- available as opt-in additions for file listing and executable lookup.
2687
-
2688
- Model metadata has been refreshed across providers, the REPL loop
2689
- internals have been refactored to use `catch`/`throw` for cleaner command
2690
- routing, and several bugs have been fixed including a tracer restoration
2691
- issue in the agent ensure clause and a missing cursor in the REPL input
2692
- area.
2693
-
2694
- ### Add
2695
-
2696
- * **repl: allow runtime state to be saved and restored** <br>
2697
- `LLM::Agent#repl` now accepts a `path:` option that serializes
2698
- runtime state to the filesystem. When the path already exists,
2699
- runtime state is restored when the read-eval-print loop starts.
2700
- Otherwise the path is written after the first turn, making it
2701
- possible to resume a session across process restarts.
2702
-
2703
- * **repl: scroll to the bottom on submit** <br>
2704
- The curses-based REPL now scrolls the transcript to the bottom when
2705
- the user submits their input, so the latest response is visible
2706
- without needing to scroll down manually.
2707
-
2708
- * **repl: add Ctrl+F to move the cursor forward** <br>
2709
- The curses-based REPL input now supports Ctrl+F to move the cursor
2710
- forward by one column, matching common terminal editing conventions
2711
- found in shells like `/bin/sh`.
2712
-
2713
- * **repl: add Ctrl+K to erase from cursor to end of line** <br>
2714
- The curses-based REPL input now supports Ctrl+K to erase all text
2715
- from the cursor position to the end of the input buffer, matching
2716
- common terminal editing conventions found in shells like `/bin/sh`.
2717
-
2718
- * **repl: add Ctrl+Y to paste previously killed text** <br>
2719
- The curses-based REPL input now supports Ctrl+Y to insert the most
2720
- recently killed text (via Ctrl+K) at the current cursor position,
2721
- matching the yank/paste convention found in shells like `/bin/sh`.
2722
- The killed text is stored in an internal copy buffer so it can be
2723
- pasted multiple times or at different cursor positions.
2724
-
2725
- * **repl: add command system foundation** <br>
2726
- Add `LLM::Repl::Command` as a new base class for REPL commands,
2727
- along with the first built-in command `LLM::Repl::Command::Exit`
2728
- which exits the read-eval-print loop via `throw(:exit)`.
2729
- Commands are identified by a name and can be looked up through
2730
- `Command.find_by`. This is the foundation for the `/` command
2731
- syntax used in the REPL input line.
2732
-
2733
- * **repl: connect the command system to user input** <br>
2734
- The curses-based REPL now routes user input through the command
2735
- system. Any input string beginning with `"/"` is matched against
2736
- the command registry via `Command.find_by`, and the corresponding
2737
- command is executed instead of being forwarded to the model.
2738
- This makes built-in commands like `/exit` functional from the
2739
- input line. Command arguments are not yet supported.
2740
-
2741
- * **repl: add `LLM::Repl::Command.registry`** <br>
2742
- Add `LLM::Repl::Command.registry` for auto-registering command
2743
- subclasses. The `inherited` hook captures each new subclass and
2744
- stores it in the registry, making it possible to enumerate all
2745
- available commands at runtime. Built-in commands like Exit are
2746
- automatically registered when the command file is loaded.
2747
-
2748
- * **repl: detect and handle paste mode in the input line** <br>
2749
- The curses-based REPL input now detects paste operations by tracking
2750
- the rate at which characters arrive. A paste rate of ≤50ms is
2751
- assumed to be a burst of characters that could only be explained by
2752
- a paste. No human types that fast. Multiline pastes are supported
2753
- through internal refactoring of the input handling logic.
2754
-
2755
- * **repl: optimize paste mode rendering** <br>
2756
- Track the paste state with an internal `@paste` variable and switch
2757
- to a faster input path during paste operations. While in paste mode,
2758
- the input buffer is drained via `Curses.getch`, bypassing the more
2759
- expensive char-by-char render path used for ordinary interactive
2760
- input. This makes pasting large amounts of text noticeably faster.
2761
-
2762
- * **Add `LLM::Tool::Ls`** <br>
2763
- Add a built-in tool for listing files and directories, with optional
2764
- glob pattern filtering to narrow results. <br>
2765
- It must be required explicitly with `require "llm/tools/ls"`.
2766
-
2767
- * **Add `LLM::Tool::Which`** <br>
2768
- Add a built-in tool for locating an executable on the system PATH.
2769
- This lets an agent check whether a command is available before
2770
- attempting to run it, avoiding failed subprocess calls. <br>
2771
- It must be required explicitly with `require "llm/tools/which"`.
2772
-
2773
- * **repl: render tool calls in a function-call syntax** <br>
2774
- The curses-based REPL status bar now renders tool calls with a
2775
- compact function-call syntax: `tool(key: value)` instead of
2776
- `tool: name`. Strings are quoted and truncated, arrays show their
2777
- first two elements, and hashes collapse to `{…}`, making it easier
2778
- to see what arguments the model is passing. The `tool done` status
2779
- message has been removed since the tool call itself conveys
2780
- completion information.
2781
-
2782
- ### Change
2783
-
2784
- * **Refresh model metadata** <br>
2785
- Update model listings, pricing, and capabilities across providers.
2786
- Fix GPT-5.6 model family names in the OpenAI registry (`gpt` to
2787
- `gpt-sol`, `gpt-nano` to `gpt-luna`, `gpt-mini` to `gpt-terra`).
2788
- Add OpenAI models (`gpt-5.6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`)
2789
- to the AWS Bedrock registry. Update DeepInfra pricing for
2790
- `DeepSeek-V3` and `Sky-T1-32B-Preview`. Fix Google model knowledge
2791
- cutoff dates.
2792
-
2793
- * **repl: control the loop with catch & throw** <br>
2794
- The curses-based REPL input loop now uses `catch(:exit)` and
2795
- `throw(:exit)` instead of returning the `:exit` symbol and
2796
- breaking out of the loop. This enables the `/command` syntax
2797
- without requiring an `:exit` return value to be propagated
2798
- through a potentially deeply nested call path.
2799
-
2800
- * **repl: replace Ctrl+D with shell-like delete-at-cursor** <br>
2801
- The curses-based REPL input now treats Ctrl+D as a delete action
2802
- that removes the character at the current cursor position, matching
2803
- the shell/Emacs convention where Ctrl+D deletes the character under
2804
- the cursor instead of signalling end-of-file. The previous Ctrl+D
2805
- behaviour (exiting the REPL) is superseded by the `/exit` command.
2806
-
2807
- * **repl: switch to 'Thinking' mode after tool return** <br>
2808
- The curses-based REPL status line now switches to "Thinking" mode
2809
- after a tool returns, so the user can see the agent is processing
2810
- the tool result rather than showing a stale tool-call status.
2811
-
2812
- ### Fix
2813
-
2814
- * **agent: fix a subtle typo in the ensure clause** <br>
2815
- Fix a subtle typo in `LLM::Agent` where the deprecated `trace` local
2816
- variable was given preference over `tracer` (the preferred local name)
2817
- in an `ensure` clause. The `trace` local was supported for backward
2818
- compatibility but the ensure clause still referenced `trace` instead of
2819
- `tracer`, which meant the previous tracer was never restored when the
2820
- REPL session ended.
2821
-
2822
- * **repl: restore the cursor in the input area** <br>
2823
- Remove the `Curses.curs_set(0)` call from the REPL redraw method,
2824
- which was inadvertently hiding the cursor and making it impossible
2825
- to see the current position in the input area. The input field is
2826
- now always drawn at its full height so the cursor position is
2827
- correctly maintained after each redraw.
2828
-
2829
- ## v12.3.1
2830
-
2831
- Changes since `v12.3.0`.
2832
-
2833
- This release fixes a flickering issue in the curses-based REPL redraw.
2834
- The full-screen clear that caused visible flickering has been replaced
2835
- with a targeted cursor-hide approach, and stale rows from a larger
2836
- transcript are now explicitly cleared to prevent ghost text from
2837
- lingering when the transcript shrinks.
2838
-
2839
- ### Fix
2840
-
2841
- * **repl: fix redraw flicker** <br>
2842
- Replace `Curses.clear` with `Curses.curs_set(0)` in the REPL redraw
2843
- method to avoid a full screen clear that caused visible flickering
2844
- during redraws. The drawing order is also adjusted so the status line
2845
- is drawn before the divider, and stale rows left over from a larger
2846
- transcript are now explicitly cleared to prevent ghost text from
2847
- lingering when the transcript shrinks.
2848
-
2849
- ## v12.3.0
2850
-
2851
- Changes since `v12.2.0`.
2852
-
2853
- This release brings major improvements to the curses-based REPL
2854
- (`LLM::Agent#repl`). The status line now shows a context-usage bar and
2855
- running cost counter, the input field expands to three rows with
2856
- full cursor navigation, model responses are rendered as styled markdown,
2857
- and the UI stays responsive while the agent is working by running
2858
- requests in a separate thread. A new `LLM::Stream::IO` and
2859
- `LLM::Stream::Disabled` provide a uniform stream representation across
2860
- all stream types.
2861
-
2862
- Mistral OCR support is added for extracting text from images and
2863
- documents via the `/v1/ocr` endpoint. The `skills:` and `tools:` options
2864
- on `LLM::Agent#repl` let you attach additional tools or skill directories
2865
- for the duration of a session. `LLM::Object#merge!` rounds out the
2866
- in-place merge API, and a new `LLM.logger` convenience method creates
2867
- tracer logger instances with less verbosity.
2868
-
2869
- ### Add
2870
-
2871
- * **Add `LLM.logger` convenience method** <br>
2872
- Add `LLM.logger(llm, ...)` as a shorter, less verbose way to create an
2873
- `LLM::Tracer::Logger` instance. Takes a provider and optional keyword
2874
- arguments forwarded to the logger constructor.
2875
-
2876
- * **Add `skills:` option to `LLM::Agent#repl`** <br>
2877
- `LLM::Agent#repl` now accepts a `skills:` keyword argument that attaches
2878
- one or more skill directories (containing `SKILL.md`) for the duration of
2879
- the repl session. Skills are loaded and converted to tools, combining with
2880
- any tools already configured on the agent, and are discarded when the
2881
- session ends.
2882
-
2883
- * **Add `LLM::Provider#ocr` base method** <br>
2884
- Add a base `ocr(...)` method to `LLM::Provider` that raises `NotImplementedError`
2885
- by default, establishing a common interface for providers that support OCR
2886
- (Optical Character Recognition) on images and documents.
2887
-
2888
- * **Add Mistral OCR endpoint support** <br>
2889
- The Mistral provider now supports OCR via its `/v1/ocr` endpoint. Call
2890
- `mistral.ocr(image_url: ...)` for images or `mistral.ocr(document_url: ...)`
2891
- for documents (e.g., PDFs). Returns an `LLM::Response` with extracted pages,
2892
- markdown content, and structured block data.
2893
-
2894
- * **Add `LLM::Object#merge!`** <br>
2895
- Add `LLM::Object#merge!` for in-place merging of hash data into an
2896
- `LLM::Object` instance, complementing the existing `#merge` method.
2897
-
2898
- * **Add `LLM::Stream::IO` and `LLM::Stream::Disabled`** <br>
2899
- `LLM::Stream::IO` wraps IO-like objects as stream targets, forwarding
2900
- streamed content via `#<<`. `LLM::Stream::Disabled` represents an explicitly
2901
- disabled stream with no-op callbacks.
2902
-
2903
- This is part of an internal refactoring that lets all stream values: IO
2904
- objects, `true`, `false`, `nil`, and `LLM::Stream` instances themselves
2905
- be represented by the same `LLM::Stream` interface via the new
2906
- `LLM::Stream.try` factory method.
2907
-
2908
- Before this change the codebase had to perform ad-hoc type checks
2909
- (e.g. `if LLM::Stream === stream`) scattered throughout. After this
2910
- change all stream handling goes through a single uniform path, and
2911
- providers check `#enabled?` to decide whether to request streaming
2912
- from the API.
2913
-
2914
- ### Change
2915
-
2916
- * **repl: rename `trace:` to `tracer:`** <br>
2917
- The `trace:` keyword argument in `LLM::Agent#repl` has been renamed to
2918
- `tracer:` for consistency with the rest of the codebase. The old `trace:`
2919
- name still works with a deprecation warning.
2920
-
2921
- * **repl: add context-usage bar and cost counter to the status line** <br>
2922
- The curses-based REPL status line now shows a small progress bar that
2923
- indicates how much of the model's context window remains as a percentage,
2924
- alongside a running cost estimate rendered on the right side of the status
2925
- line. The input line has been updated to show the provider name as a prefix.
2926
- Estimates are best-effort and depend on registry pricing data (see `data/`).
2927
-
2928
- * **repl: keep the UI responsive while a request is in progress** <br>
2929
- The curses-based REPL now spawns the agent request in a separate thread
2930
- and communicates streamed output through a queue, so the curses UI stays
2931
- responsive during model processing. Users can continue to scroll through
2932
- the transcript while the agent is working.
2933
-
2934
- * **repl: style transcript rows as structured data with bold labels** <br>
2935
- The curses-based REPL transcript now stores rows as structured data with
2936
- style metadata instead of plain strings, enabling bold rendering of the
2937
- `user:` and `agent:` labels for improved readability during interactive
2938
- sessions.
2939
-
2940
- * **repl: render a small subset of markdown** <br>
2941
- The curses-based REPL now renders model responses as styled markdown.
2942
- Headers and strong text render in bold, emphasis renders in underline,
2943
- and code spans and blocks are highlighted with inverted colors. Streaming
2944
- content is buffered and re-rendered on each tick so the transcript reads
2945
- cleanly as the agent responds. Requires the optional `kramdown` gem.
2946
-
2947
- * **repl: add cursor LEFT/RIGHT movement to the input line** <br>
2948
- The curses-based REPL input now supports cursor movement with the left
2949
- and right arrow keys, enabling in-place text editing before submitting
2950
- a prompt. The cursor position is tracked visually and moves backwards
2951
- on left-arrow and forwards on right-arrow.
2952
-
2953
- * **repl: add Ctrl+A and Ctrl+E keybindings to the input line** <br>
2954
- The curses-based REPL input now supports Ctrl+A to jump the cursor to
2955
- the start of the input line and Ctrl+E to jump it to the end, matching
2956
- common terminal editing conventions.
2957
-
2958
- * **repl: add `tools:` option to `LLM::Agent#repl`** <br>
2959
- `LLM::Agent#repl` now accepts a `tools:` keyword argument that attaches
2960
- additional tool classes or instances for the duration of the repl session.
2961
- These tools are combined with any tools already configured on the agent,
2962
- and are discarded when the session ends.
2963
-
2964
- * **repl: add repl support to ActiveRecord and Sequel agent models** <br>
2965
- `acts_as_agent` (ActiveRecord) and `plugin :agent` (Sequel) models now
2966
- expose a `repl` method that delegates to the underlying agent's
2967
- read-eval-print loop. This allows interactive debugging and inspection
2968
- of persisted agent state at runtime. Note that changes made during a
2969
- repl session do not persist back to the database.
2970
-
2971
- * **repl: add extra padding between markdown nodes** <br>
2972
- The curses-based REPL markdown renderer now adds extra vertical spacing
2973
- between certain markdown elements: paragraphs, headers, and codeblocks
2974
- for improved readability of model responses.
2975
-
2976
- * **repl: add a visual divider between transcript and the rows below it** <br>
2977
- The curses-based REPL now draws a horizontal divider line (using a unicode
2978
- `─` character) to separate the transcript area from the status and input
2979
- rows below it. A single empty buffer row is also added between the
2980
- transcript and the divider, preventing transcript text from running too
2981
- close to the status and input rows.
2982
-
2983
- * **repl: expand input field to 3 rows** <br>
2984
- The curses-based REPL input field now spans three rows instead of one,
2985
- wrapping text that exceeds the terminal width onto subsequent lines. A
2986
- scrollable viewport follows the cursor so the active line stays visible,
2987
- and common navigation commands (Ctrl+A, Ctrl+E, cursor keys) work across
2988
- all three rows of the expanded input area.
2989
-
2990
- * **Refresh OpenAI model metadata** <br>
2991
- Add new OpenAI models to the registry, including `gpt-5.6`,
2992
- `gpt-5.6-luna`, `gpt-5.6-terra`, `gpt-5.6-sol`, and
2993
- `gpt-realtime-2.1`, with associated pricing, capabilities, and
2994
- limits.
2995
-
2996
- ### Fix
2997
-
2998
- * **Fix Ollama non-streaming response handling** <br>
2999
- Fix the Ollama provider to properly handle the non-streaming path. When
3000
- the provider returns a raw NDJSON response body (instead of streaming),
3001
- the response is now parsed and merged into a single `LLM::Object` before
3002
- being returned to the caller. Previously the non-streaming path was
3003
- effectively broken and would fail to produce a valid completion response.
3004
-
3005
- * **repl: handle a negative context window allowance in the usage bar** <br>
3006
- Fix a crash in the curses-based REPL context-usage bar when the context
3007
- window allowance is exceeded (used > total). The negative width value that
3008
- resulted from this edge case could cause curses errors; it now gracefully
3009
- defaults to `0%` and zero bar width.
3010
-
3011
- * **Fix YARD documentation across provider and tool files** <br>
3012
- Fix unnamed, misnamed, and missing `@param` tags in `LLM::Repl::Status`,
3013
- `LLM::Tool::Git`, `LLM::Tool::Pwd`, `LLM::Tool::Rg`, and
3014
- `LLM::Tool::SwapText`.
3015
-
3016
- ## v12.2.0
3017
-
3018
- Changes since `v12.1.0`.
3019
-
3020
- This release adds Mistral as a new provider with chat completions, streaming, tool calls,
3021
- structured outputs, file/image attachments, and embeddings support. It introduces
3022
- the `trace:` option to `LLM::Agent#repl` for keeping the tracer active during
3023
- interactive sessions.
3024
-
3025
- Several fixes land for the Google provider (generationConfig parameter leakage),
3026
- `LLM::Context#tracer=` (always assigning nil), `LLM::Provider#with_tracer(nil)`
3027
- (nil fallback), and `LLM::Context#repair!` (dropping Struct returns).
3028
-
3029
- The default HTTP timeout has been increased from 60s to 180s to better accommodate
3030
- reasoning models and large structured outputs, and the Anthropic default model has
3031
- been updated to `claude-opus-4-8`. Model metadata has been refreshed across
3032
- Anthropic, AWS Bedrock, DeepInfra, Google, and xAI, with Mistral model data added
3033
- to the registry.
3034
-
3035
- ### Add
3036
-
3037
- * **Add `trace:` option to `LLM::Agent#repl`** <br>
3038
- `LLM::Agent#repl` now accepts a `trace:` keyword argument. By default
3039
- the tracer is disabled for the duration of the repl session to prevent
3040
- curses UI interference from output written to `$stdout` or `$stderr`.
3041
- Set `trace: true` to keep the tracer active during the session, which
3042
- is useful when the tracer writes to a file rather than the terminal.
3043
-
3044
- * **Add a new provider: LLM::Mistral** <br>
3045
- [Mistral](https://mistral.ai) is now supported through its
3046
- OpenAI-compatible API. The provider supports chat completions,
3047
- streaming, tool calls, structured output (schema), file/image
3048
- attachments, and embeddings. Use `LLM.mistral(...)` to create a
3049
- provider instance.
3050
-
3051
- * **Add `LLM.mistral(...)` convenience method** <br>
3052
- A new top-level accessor (`LLM.mistral`) returns an `LLM::Mistral`
3053
- provider instance, matching the pattern used by other providers.
3054
-
3055
- ### Fix
3056
-
3057
- * **Fix Google `generationConfig` parameter leakage** <br>
3058
- Fix a bug in the Google provider where non-generation parameters
3059
- (`role`, `model`, `messages`, `stream`) were leaking into the
3060
- `generationConfig` object alongside legitimate generation config
3061
- parameters such as `temperature`. Non-config parameters are now
3062
- filtered out before constructing `generationConfig`.
3063
-
3064
- * **Fix `LLM::Context#tracer=` always assigning `nil`** <br>
3065
- The `LLM::Context#tracer=` setter had a bug where it always assigned
3066
- `nil` regardless of the tracer value passed. It now correctly assigns
3067
- the given tracer or falls back to `LLM::Tracer::Null`.
3068
-
3069
- * **Fix `LLM::Provider#with_tracer(nil)` fallback** <br>
3070
- `LLM::Provider#with_tracer(nil)` now falls back to
3071
- `LLM::Tracer::Null` instead of setting a `nil` tracer directly.
3072
-
3073
- * **Fix `LLM::Context#repair!` dropping `Struct` returns** <br>
3074
- `LLM::Context#repair!` used `[*prompt]` to wrap the prompt before
3075
- grepping for return objects. Since `LLM::Function::Return` is a
3076
- `Struct`, the splat operator expanded it into its member values
3077
- instead of wrapping it, causing the grep to silently drop returns.
3078
- The fix wraps both sources in an array before flattening.
3079
-
3080
- ### Change
3081
-
3082
- * **Increase default provider timeout from 60s to 180s** <br>
3083
- The default HTTP timeout for all providers has been increased from
3084
- 60 to 180 seconds to better accommodate long-running requests such
3085
- as reasoning models and large structured outputs.
3086
-
3087
- * **Change Anthropic default model to `claude-opus-4-8`** <br>
3088
- The default Anthropic chat model has been updated from
3089
- `claude-sonnet-4-20250514` to `claude-opus-4-8`, reflecting the
3090
- latest model release from Anthropic.
3091
-
3092
- * **Refresh model metadata** <br>
3093
- Update model listings, pricing, and capabilities for Anthropic,
3094
- AWS Bedrock, DeepInfra, Google, and xAI. Add Mistral model data
3095
- to the registry.
3096
-
3097
- ## v12.1.0
3098
-
3099
- Changes since `v12.0.0`.
3100
-
3101
- This release adds `LLM::Agent#repl` with a curses-based
3102
- interactive read-eval-print loop. It requires the optional
3103
- dependency `curses` and it is probably the most notable
3104
- feature in this release.
3105
-
3106
- Multiple _opt-in_ tools have been added to the `llm/tools/*.rb`
3107
- directory. They serve as examples and as general-purpose tools
3108
- that happen to power the repository's agents.
3109
-
3110
- Other changes include small-ish bug fixes. <br>
3111
- As always, see the changelog details for a thorough overview.
3112
-
3113
- ### Add
3114
-
3115
- * **Add `LLM::Agent#repl`** <br>
3116
- Add a curses-based read-eval-print loop for `LLM::Agent` that lets
3117
- developers interact with an agent after it has been set up or has
3118
- performed a task. It is similar to `binding.irb`: once you exit,
3119
- you can continue with the rest of your program. It requires the
3120
- `curses` gem.
3121
-
3122
- * **Add `#tracer=` setter on Provider, Context and Agent** <br>
3123
- `LLM::Provider`, `LLM::Context` and `LLM::Agent` can now configure
3124
- the tracer after initialization via the `#tracer=` setter. It accepts
3125
- a subclass of `LLM::Tracer` or `nil` to disable the tracer.
3126
-
3127
- * **Add `LLM::Tool::Shell`** <br>
3128
- Add a built-in shell tool that can run a command with arguments. <br>
3129
- It must be required explicitly with `require "llm/tools/shell"` and
3130
- requires the `test-cmd.rb` gem.
3131
-
3132
- * **Add `LLM::Tool::ReadFile`** <br>
3133
- Add a built-in tool for reading the contents of a file, with optional
3134
- `start` and `stop` line offsets. <br>
3135
- It must be required explicitly with `require "llm/tools/read_file"`.
3136
-
3137
- * **Add `LLM::Tool::Chdir`** <br>
3138
- Add a built-in tool for changing the current working directory. <br>
3139
- It must be required explicitly with `require "llm/tools/chdir"`.
3140
-
3141
- * **Add `LLM::Tool::Git`** <br>
3142
- Add a built-in tool that can perform git actions (`log`, `diff`,
3143
- `show`, `commit`, `checkout`, `branch`). <br>
3144
- It must be required explicitly with `require "llm/tools/git"` and requires the `test-cmd.rb` gem.
3145
-
3146
- * **Add `LLM::Tool::Rg`** <br>
3147
- Add a built-in tool that wraps the `rg` (ripgrep) command for
3148
- recursively searching the current directory for patterns. <br>
3149
- It must be required explicitly with `require "llm/tools/rg"` and requires the `test-cmd.rb` gem.
3150
-
3151
- * **Add `LLM::Tool::SwapText`** <br>
3152
- Add a built-in tool that can replace an exact snippet of text in a
3153
- file with a new piece of text. <br>
3154
- It must be required explicitly with `require "llm/tools/swap_text"`.
3155
-
3156
- * **Add `LLM::Tool::Pwd`** <br>
3157
- Add a built-in tool that returns the current working directory. <br>
3158
- It must be required explicitly with `require "llm/tools/pwd"`.
3159
-
3160
- * **Add `LLM::Tool::WriteFile`** <br>
3161
- Add a built-in tool that can write a given string to a given file
3162
- path. <br>
3163
- It must be required explicitly with `require "llm/tools/write_file"`.
3164
-
3165
- * **Add `LLM::Tool::Mkdir`** <br>
3166
- Add a built-in tool that can create a tree of new directories. <br>
3167
- It must be required explicitly with `require "llm/tools/mkdir"` and
3168
- requires the `test-cmd.rb` gem.
3169
-
3170
- ### Change
3171
-
3172
- * **Change LlamaCpp default port (8080 => 8013)** <br>
3173
- The default port for the LlamaCpp provider has changed from `8080` to
3174
- `8013` since llamacpp itself defaults to that port.
3175
-
3176
- * **Change LlamaCpp default model to `nil`** <br>
3177
- The default model for LlamaCpp is now `nil`, letting whatever model
3178
- is served by the llamacpp server act as the default. Previously it
3179
- defaulted to `qwen3`.
3180
-
3181
- ### Fix
3182
-
3183
- * **Fix `LLM::Agent.tools` Symbol resolution** <br>
3184
- When an agent defined tools via `tools :method_name`, the resolved
3185
- symbol was incorrectly forwarded as `[:method_name]` (an array) to
3186
- `LLM::Context`. This fix copies the same pattern used by other
3187
- attribute resolvers (e.g., `skills`) so a single Symbol is resolved
3188
- through the agent instance correctly.
3189
-
3190
- * **Encode strings as UTF-8 in the JSON adapter** <br>
3191
- The `json` gem will reject BINARY-encoded strings from version 3 and
3192
- beyond. The `LLM::JSONAdapter.dump` method now walks serialized data
3193
- and encodes every string into UTF-8, using `String#scrub` to replace
3194
- bytes that are not valid UTF-8.