llm.rb 16.0.0.pre.9 → 16.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/CHANGELOG.md CHANGED
@@ -1,23 +1,25 @@
1
- <p align="center">
2
- <a href="https://r.uby.dev">
3
- <img
4
- src="https://github.com/r-uby-dev/llm.rb/raw/main/rubydev.svg"
5
- width="400"
6
- height="200"
7
- border="0"
8
- alt="a r.uby.dev project"
9
- >
10
- </a>
11
- </p>
12
-
13
1
  > Changelog <br>
14
2
  > [r.uby.dev](https://r.uby.dev) project
15
3
 
16
- This file covers the v15 series and the unreleased changes that follow it.
17
- Releases before v15 are kept in [changelog/old.md](changelog/old.md).
4
+ This file covers the v16 series. Releases up to and including v15 are kept in
5
+ [changelog/old.md](changelog/old.md).
18
6
 
19
7
  ## What's next
20
8
 
9
+ *No unreleased changes yet. Check back after the next release.*
10
+
11
+ ## v16.0.0
12
+
13
+ Changes since `v15.5.0`.
14
+
15
+ This release makes an interrupt precise: `LLM::Interrupt` moves outside
16
+ `StandardError`, a turn between its requests ends where it is, and a cancel is
17
+ delivered inside the tool that has to handle it. It also requires Ruby 3.4 or
18
+ later, reads a `:jsonb` record's messages from the column, saves a
19
+ record-backed conversation after each request, and adds
20
+ `LLM::ActiveRecord#messages!`, `LLM::Stream#on_step`, and a tracer's
21
+ `on_interrupt` hook.
22
+
21
23
  ### Breaking
22
24
 
23
25
  #### Migration
@@ -232,6 +234,17 @@ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
232
234
  where a stream can checkpoint, and it fires once per successful request even
233
235
  when streaming is off.
234
236
 
237
+ ### Tools
238
+
239
+ * **tools: wait for a command's status, not only for it to stop running** <br>
240
+ Fix a bug where an exec-backed tool could answer `ok: nil` for a command
241
+ that ran and printed its output.
242
+ [`LLM::Tool::Utils#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Utils.html#wait-instance_method)
243
+ left as soon as `running?` said the process was gone, and the status is not
244
+ decided at that moment, so `command.success?` was read before it had an
245
+ answer. It waits for the status now, and a command that was never found is
246
+ the exception, because there was nothing to reap.
247
+
235
248
  ### Tracers
236
249
 
237
250
  * **tracer: report a request that failed** <br>
@@ -253,1217 +266,3 @@ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
253
266
  [`LLM::Context#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#wait-instance_method)
254
267
  now calls `on_interrupt` once when an interrupt unwinds through it, rather
255
268
  than once per tool, matching the single exception the caller receives.
256
-
257
- ## v15.5.0
258
-
259
- Changes since `v15.4.1`.
260
-
261
- This release makes the exec-backed tools report how long a command ran, calls a
262
- tracer's `on_exit` hook once when the last scope ends, and resolves a `Proc`
263
- parameter type when a schema is serialized. It also fixes `LLM::Schema.to_s`
264
- for a schema with a deferred type.
265
-
266
- ### Tools
267
-
268
- * **tools: report how long a command ran** <br>
269
- [`LLM::Tool::Exec#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Exec.html#call-instance_method)
270
- now includes a `duration` field in its result, a string such as `"0.4 seconds"`
271
- that reports how long the command took. The tools that route through `exec`
272
- (`git`, `rg`, `mkdir`, `ruby`, and `bundle`) return it as well. A command that
273
- was not found still returns the error hash, which has no `duration`.
274
-
275
- ### Tracers
276
-
277
- * **tracer: call `on_exit` once, when the last scope ends** <br>
278
- [`LLM::Tracer#on_exit`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#on_exit-instance_method)
279
- is a new hook, called once when the last scope that is open for a tracer
280
- ends. A tracer that holds a connection, a file, or a span releases it here.
281
- The count of open scopes belongs to the tracer rather than to the thread that
282
- opened one, because a tool runs on a thread of its own and scopes the turn's
283
- tracer while it does. Before this, a tool's scope looked like the outermost
284
- one: a tracer that released its resource in `on_exit` lost it in the middle
285
- of the turn, and the rest of the trace was written without it.
286
-
287
- ### Schema
288
-
289
- * **schema: resolve a parameter type when the schema is serialized** <br>
290
- [`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) and
291
- [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) accept a `Proc`
292
- as the type of a parameter or a property, and the proc is called when the
293
- schema is serialized rather than when the class that declares it is defined.
294
- A type that is only known at runtime no longer has to be known at load time:
295
- `parameter :kind, proc { Enum[*Article.kinds] }, "The article kind"` asks the
296
- model for the values a store holds right now, and the proc runs again on the
297
- next request, so it sees the values that are current then. Whatever the proc
298
- returns is resolved in turn, so it may be a leaf, a class, or an array of
299
- either, and the description, `required`, `default`, and `enum` given with the
300
- parameter are applied to it.
301
-
302
- * **schema: keep `to_s` working for a deferred type** <br>
303
- Fix a bug where a schema with a proc-typed property raised from
304
- [`LLM::Schema.to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html#to_s-class_method)
305
- (and so from `inspect` and `p`) because the placeholder resolved its leaf
306
- through a private method that shadowed the one `LLM::Schema::Leaf#default`
307
- uses. Serialization, and therefore the request sent to a provider, was
308
- unaffected.
309
-
310
- ## v15.4.1
311
-
312
- Changes since `v15.4.0`.
313
-
314
- This release removes the gemspec's post-install message, so installing the gem
315
- no longer prints the r.uby.dev notice. It also refreshes the model registry
316
- with current model listings, limits, and pricing.
317
-
318
- ### Core
319
-
320
- * **remove the gemspec post install message** <br>
321
- The gemspec no longer sets `post_install_message`, so installing the
322
- gem no longer prints the r.uby.dev website notice.
323
-
324
- ### Registry
325
-
326
- * **refresh model metadata** <br>
327
- Update `data/` with current model listings, limits, and pricing for the
328
- Alibaba, Bedrock, DeepSeek, Google, Moonshot, and OpenRouter registries.
329
- Bedrock adds the Kimi K3 and GPT-6 Sol and GPT-6 Luna families in both
330
- the global and US regions and raises the context limit to 1M tokens for
331
- two models, while Moonshot raises the Kimi K3 output limit to 1M tokens.
332
- OpenRouter adds `qwen/qwen3.8-max-prime`, `z-ai/glm-5.3-prime`,
333
- `upstage/solar-mini4`, the Aion 3.5 models, and `stealth/space-bunny-alpha`,
334
- drops `mistralai/devstral-2512` and a free Ling 3.0 Flash VL entry, and
335
- reprices several DeepSeek and Mistral models.
336
-
337
- ## v15.4.0
338
-
339
- Changes since `v15.3.0`.
340
-
341
- This release saves token usage with the context state, moves the retry budget
342
- onto the provider, groups a turn's spans under one trace, and lets an agent
343
- declare `name`, `description`, `path`, and `tool_budget` with a block. It also
344
- gives every tracer request an id, adds `LLM::ActiveRecord::Message`, reads
345
- `AGENTS.md` as the console's system prompt, removes `LLM::Guard::Loop`, and
346
- refreshes the model registry.
347
-
348
- ### Core
349
-
350
- * **context: save token usage with the state** <br>
351
- [`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h-instance_method)
352
- now writes `context_used` and `context_window`, so a saved state can be
353
- inspected without loading the runtime. They are never read back, and a payload
354
- without them still loads.
355
-
356
- ### Provider
357
-
358
- * **provider: decide the retry budget on the provider** <br>
359
- [`LLM::Provider#retry_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#retry_budget-instance_method)
360
- returns how many times a rate-limited request is retried before the error is
361
- raised, and defaults to 5. An agent that sets no `retry_budget` of its own
362
- now takes the budget from its provider instead of special-casing Alibaba, so
363
- [`LLM::Alibaba`](https://r.uby.dev/api-docs/llm.rb/LLM/Alibaba.html)
364
- still retries 8 times and a provider that recovers from rate limits slowly can
365
- return a higher budget. An explicit `retry_budget:` still takes precedence.
366
-
367
- ### Agent
368
-
369
- * **agent: group a turn's spans into one trace** <br>
370
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now opens a
371
- `llm.turn` trace group around every turn, so all spans a turn produces share
372
- one trace id. It uses the agent's tracer when it has one, the provider's
373
- otherwise; previously a provider-wide tracer split one turn across traces.
374
-
375
- * **agent: declare `name`, `description`, `path`, and `tool_budget` with a block** <br>
376
- A block passed to any of these four class-level setters was previously
377
- ignored, because the call was read as a getter. Now
378
- [`LLM::Agent.name`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#name-class_method)
379
- and [`LLM::Agent.description`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#description-class_method)
380
- store the block, and an instance resolves it against itself, or against its
381
- ORM record when it is bound to one. `name` and `description` also accept a
382
- `Symbol` or a `Proc`, and the class-level reader still returns what was
383
- configured, so read them on an instance for the resolved string.
384
-
385
- ### Console
386
-
387
- * **console: use `AGENTS.md` as the system prompt** <br>
388
- `bin/llm.rb` now looks for `AGENTS.md` in the current working directory when
389
- it boots, and when the file exists its contents become the agent's
390
- instructions for the session. The instructions are injected once, so a
391
- resumed session that already has a system message keeps the one it has.
392
-
393
- * **console: stop a cancelled turn from crashing the console** <br>
394
- A turn cancelled with Esc can still have chunks queued, and they arrive after
395
- the buffer that renders them is closed. The streaming path then tried to
396
- replace a row that no longer existed and raised.
397
- [`LLM::Console::Buffer`](https://r.uby.dev/api-docs/llm.rb/LLM/Console/Buffer.html)
398
- now reports whether it has a row to replace through `open?`, the stream drops
399
- chunks that arrive for a closed buffer, and `replace` keeps the current text
400
- instead of raising.
401
-
402
- ### Guard
403
-
404
- * **guard: remove `LLM::Guard::Loop`** <br>
405
- `LLM::Guard::Loop` is removed. It stopped repeated tool-call patterns, but
406
- could also interrupt a loop that was making progress, so it did not hold up as
407
- a default. Agents no longer enable a guard of their own; bound the loop with
408
- `tool_budget` instead.
409
-
410
- ### ActiveRecord
411
-
412
- * **activerecord: add `LLM::ActiveRecord::Message`** <br>
413
- [`LLM::ActiveRecord::Message`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord/Message.html)
414
- is a virtual model over the JSONB column that holds an agent's state, so
415
- messages can be filtered, ordered, and counted in SQL instead of in memory.
416
- `for(agent:)` returns an
417
- [`ActiveRecord::Relation`](https://api.rubyonrails.org/classes/ActiveRecord/Relation.html)
418
- with `id`, `role`, `content`, `tools`, and `position` columns; a row's
419
- `unwrap!` returns the message. The class never materializes as a table.
420
-
421
- ### Tracer
422
-
423
- * **tracer: give every request an id** <br>
424
- [`LLM::Tracer#on_request_start`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#on_request_start-instance_method)
425
- now takes a `request_id:`, a UUIDv7 that the runtime mints when a request
426
- begins and passes to `on_request_finish` and `on_request_error` for that same
427
- request. A turn can make many requests, so `trace_group_id` groups a turn but
428
- not the events inside one request; the id lets a tracer correlate a request's
429
- start, finish, and error events. The keyword is required, so a subclass that
430
- overrides these hooks must accept it.
431
-
432
- * **tracer: record the model the provider answers with** <br>
433
- [`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html)
434
- now sets `gen_ai.response.model` from the model on the response instead of the
435
- model requested, so a span reports the model a provider actually served. A
436
- router model such as `openrouter/auto` records the model it resolved to,
437
- while `gen_ai.request.model` keeps the name that was asked for.
438
-
439
- ### Registry
440
-
441
- * **refresh model metadata** <br>
442
- Update `data/` with current pricing, limits, and capabilities for the
443
- Alibaba, Anthropic, Bedrock, DeepInfra, DeepSeek, Mistral, OpenAI,
444
- OpenRouter, xAI, and Z.ai registries. Claude Opus 5.5 reaches Anthropic,
445
- Bedrock, and OpenRouter, OpenAI adds GPT-6 Sol and GPT-6 Luna, Bedrock adds
446
- Gemma 4 and more regional Claude Sonnet 4 and GPT-5.6 entries, and
447
- OpenRouter adds the Xiaomi MiMo V2.6, Nex N2.5, Cohere Command A+, and
448
- Qwen3.8 Omni Flash models. Z.ai adds `glm-4.6v-flash` and `glm-5.3-flashx`,
449
- xAI adds Grok 4.7, DeepSeek adds a `low` reasoning effort to
450
- `deepseek-v4-pro` and deprecates `deepseek-v4-flash` and
451
- `deepseek-v4-flash-vision-exp`, DeepInfra reprices `tencent/Hy3`, and
452
- OpenRouter drops `anthropic/claude-opus-4` and `kwaipilot/kat-coder-pro-v2`.
453
-
454
- ## v15.3.0
455
-
456
- Changes since `v15.2.2`.
457
-
458
- This release gives contexts, agents, and messages UUIDv7 ids that survive
459
- persistence and back their `created_at`, scopes per-request headers through a
460
- block on `LLM::Provider#with`, and pins an OpenRouter session per context. It
461
- also fixes the OpenAI tool schema sent through OpenRouter and Azure, which
462
- mixed the Responses and Chat Completions shapes and left nested objects open.
463
-
464
- ### Core
465
-
466
- * **context: give every context an id you can sort and trace** <br>
467
- [`LLM::Context#id`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#id-instance_method)
468
- returns a UUIDv7 string that encodes its creation time, so contexts and their
469
- agents sort by creation order and stay identifiable within and across
470
- sessions. The id is generated on creation and restored with the runtime state
471
- on load. [`LLM::Agent#id`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#id-instance_method)
472
- delegates to the context it wraps; a model and its agent keep separate ids.
473
-
474
- * **context: read the creation time without storing it** <br>
475
- [`LLM::Context#created_at`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#created_at-instance_method)
476
- returns when the context was created, read from the timestamp its UUIDv7 id
477
- already carries, so no separate timestamp has to be persisted or kept in sync.
478
- `LLM::Agent#created_at` delegates to it. The time is recomputed rather than
479
- stored, so it returns `nil` for a context whose id is not a UUIDv7 string.
480
-
481
- * **message: give every message an id that survives persistence** <br>
482
- [`LLM::Message#id`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html#id-instance_method)
483
- returns a UUIDv7 string generated once when the message is created and restored
484
- with the runtime state on load, so a message keeps the same identity across
485
- saves and loads. `LLM::Message#created_at` is derived from the id rather than
486
- stored. `LLM::Message#==` ignores the id, so two messages with the same role
487
- and content are equal even when they were created at different times;
488
- previously the timestamp took part in the comparison.
489
-
490
- * **utils: decode a UUIDv7 timestamp once, in one place** <br>
491
- [`LLM::Utils.timestamp`](https://r.uby.dev/api-docs/llm.rb/LLM/Utils.html#timestamp-instance_method)
492
- returns the UTC time encoded in a UUIDv7 string, or `nil` when the value is not
493
- a UUIDv7. It backs `created_at` on messages, contexts, and agents, so the
494
- decoding logic lives in a single method instead of three.
495
-
496
- ### Provider
497
-
498
- * **provider: set a per-request header without leaking it** <br>
499
- [`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method)
500
- now takes a block: headers set inside apply to the current fiber only and are
501
- restored when the block returns, so a header that varies per request can be set
502
- for one call without leaking into the next. Without a block the headers are
503
- merged into the provider's defaults as before. The method returns the provider
504
- without a block and the block's value with one.
505
-
506
- * **openrouter: keep one session so requests reuse a cached model** <br>
507
- A context now sends its id as the `x-session-id` header on OpenRouter requests,
508
- so consecutive requests share a session and OpenRouter can route them to the
509
- same cached model instead of starting fresh each time. Other providers are
510
- unaffected.
511
-
512
- ### Fix
513
-
514
- * **openai: make tools callable through OpenRouter and Azure again** <br>
515
- Fix a bug where [`LLM::OpenAI`](https://r.uby.dev/api-docs/llm.rb/LLM/OpenAI.html)
516
- sent a function schema that mixed the Responses and Chat Completions shapes,
517
- carrying a top-level `name` next to a nested `function` object. OpenRouter and
518
- OpenAI through Azure read it as a Responses function and could not describe the
519
- tool correctly. The name, description, parameters, and `strict: false` now sit
520
- under `function:`, the shape chat completions documents.
521
-
522
- * **openai: stop Azure rejecting nested schema objects** <br>
523
- Tool parameters and structured-output schemas now carry
524
- `additionalProperties: false` on every object they contain, not just the root,
525
- in the chat completions and Responses paths alike. OpenAI and Azure reject a
526
- schema that leaves any object open, so nested objects were failing validation.
527
- The chat completions structured-output schema, which was sent open, is now
528
- closed too.
529
-
530
- ## v15.2.2
531
-
532
- Changes since `v15.2.1`.
533
-
534
- This release fixes the `set_provider`, `set_context`, and `set_tracer`
535
- callbacks installed by `acts_as_llm` and `plugin :llm` so a subclass
536
- inherits them from its superclass instead of raising.
537
-
538
- ### Fix
539
-
540
- * **activerecord, sequel: inherit `set_provider` from a superclass** <br>
541
- The `set_provider`, `set_context`, and `set_tracer` callbacks installed by
542
- `acts_as_llm` / `plugin :llm` (and their agent counterparts) are now resolved
543
- through the normal method lookup chain. Defining a callback on a superclass,
544
- the usual single-table inheritance setup, no longer raises
545
- `NotImplementedError`, and a subclass can still override it.
546
-
547
- ## v15.2.1
548
-
549
- Changes since `v15.2.0`.
550
-
551
- This release makes the agent count its tool budget across the whole turn
552
- and emit tool returns to the stream once that budget is spent, and lets
553
- `LLM::Function#cancel` carry extra return fields.
554
-
555
- ### Agent
556
-
557
- * **agent: count the tool budget across the whole turn** <br>
558
- [`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
559
- now caps the tool calls a turn runs in total. A batch of calls is
560
- spent as a batch, and a batch that would take the turn past its
561
- budget is not run at all; once the budget is spent no tool call runs,
562
- and the agent keeps sending its in-band advisory until the model
563
- answers without requesting more tools. Previously the agent ran up to
564
- the budget, then ran further batches after each advisory, so a turn
565
- could run more tool calls than its budget allowed.
566
-
567
- * **agent: emit tool returns when the budget is spent** <br>
568
- Fix a bug where, once a turn's tool budget was spent, the agent passed
569
- its in-band returns to the model without emitting them to the stream.
570
- The stream went silent, so a stream that tracked tool-call state left
571
- the calls in the `call` state. The returns are now emitted through
572
- [`LLM::Stream#on_tool_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_return-instance_method)
573
- like any other tool return.
574
-
575
- ### Function
576
-
577
- * **function: `LLM::Function#cancel` takes extra return fields** <br>
578
- [`LLM::Function#cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#cancel-instance_method)
579
- now accepts keywords beyond `reason:` and merges them into the
580
- [`LLM::Function::Return`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Return.html)
581
- it builds. `LLM::Function#budget_spent` now builds its return through
582
- `cancel` instead of hand-rolling an error, so a budget-spent return is
583
- marked `cancelled: true` and carries `action:` and `advice:` hints. A
584
- callback that receives returns can now tell a cancellation apart from
585
- an ordinary error.
586
-
587
- ## v15.2.0
588
-
589
- Changes since `v15.1.0`.
590
-
591
- This release renames the REPL to `LLM::Console` (with `/keep` replacing
592
- `/compact`) and routes every shell-out tool through a shared, bounded
593
- `exec` runner. It also adds `LLM::Message#created_at`, the `LLM::Tracer`
594
- factory methods, a `bundle` tool, per-tool `max_bytes` output limits,
595
- and a `-v` switch to the CLI, and refreshes the model registry.
596
-
597
- ### Core
598
-
599
- * **message: add `LLM::Message#created_at`** <br>
600
- [`LLM::Message#created_at`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html#created_at-instance_method)
601
- returns the time the message was created, defaulting to the moment the
602
- message is initialized. The timestamp is serialized into
603
- [`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json-instance_method)
604
- as an ISO-8601 string and restored on deserialization, so it can be
605
- stored alongside the rest of the conversation.
606
-
607
- ### Agent
608
-
609
- * **agent: inherit the ORM model's name** <br>
610
- An `acts_as_agent` (ActiveRecord) or `plugin :agent` (Sequel) model now
611
- names its generated
612
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) after
613
- the model class. Previously the agent was an anonymous subclass, so
614
- without an explicit name it defaulted to a gibberish `#<Class:0x...>`
615
- string. The wrapper now initializes the agent's name before `.agent`
616
- returns, and
617
- [`LLM::Agent.name`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#name-class_method)
618
- kebab-cases a `Class` argument, so an `AdminUser` model yields an agent
619
- named `admin-user`.
620
-
621
- ### Cli
622
-
623
- * **cli: add a `-v` switch** <br>
624
- `bin/llm.rb` gains a `-v` switch that prints the version (`llm.rb v#{LLM::VERSION}`) and exits.
625
-
626
- ### Console
627
-
628
- * **console: raise `LLM::Interrupt` on the agent's thread** <br>
629
- Pressing Esc to cancel now also raises `LLM::Interrupt` on the agent's
630
- thread. `LLM::Agent#cancel!` alone can be a no-op at some stages of the
631
- request lifecycle, so the console backs it up by interrupting the thread
632
- that runs the agent.
633
-
634
- * **console: add a `/keep` command and retire `/compact`** <br>
635
- The console now offers `/keep` for freeing space in the context window;
636
- the `/compact` command is removed. `/keep` takes the same argument, so
637
- `/keep 20%` keeps 20% of the context window. Closes
638
- [issue #161](https://github.com/r-uby-dev/llm.rb/issues/161).
639
-
640
- * **console: keep the UI responsive during long streams** <br>
641
- A model can emit many chunks in a single turn. The console now draws
642
- at most four streamed chunks at a time, then checks for input, so the
643
- UI stays responsive even when a turn produces a large amount of
644
- output.
645
-
646
- * **console: persist the conversation when a turn is done** <br>
647
- The console now saves the agent's state after the turn finishes,
648
- rather than while the response is still streaming. State is still
649
- saved every turn, but not until the turn has completed.
650
-
651
- * **console: render markdown text as typed** <br>
652
- Fix a bug where [`LLM::Console::Markdown`](https://r.uby.dev/api-docs/llm.rb/LLM/Console/Markdown.html)
653
- mangled the model's output: HTML could render invisible, and
654
- sequences like `...` were converted to unicode glyphs. The renderer
655
- now uses a custom kramdown parser that disables the HTML, smart-quote,
656
- and typographic-symbol parsers, so tags and punctuation come through
657
- exactly as written.
658
-
659
- * **console: fix a crash in the markdown parser** <br>
660
- Fix a bug where the markdown renderer raised an error on an unclosed
661
- HTML tag or a partial tag taken out of context, such as `4 < 5`. The
662
- parser now emits the `<...` run literally when there is no closing
663
- `>`, so the text renders instead of crashing.
664
-
665
- * **console: stop rendering bare pipes as tables** <br>
666
- Fix a bug where the markdown renderer treated a lone `|foo|` in prose
667
- as a table and mangled its output. A pipe line now parses as a table
668
- only when a header row is followed by a delimiter row, so bare pipes
669
- come through literally while real tables still render.
670
-
671
- * **console: find the worker thread when cancelling** <br>
672
- Fix a bug where pressing Esc to cancel raised `LLM::Interrupt` on an
673
- instance variable that does not exist, so the interrupt was a no-op and
674
- a cancel could leave the turn running. The console now resolves the
675
- worker thread through its `#thread` reader and interrupts it.
676
-
677
- * **console: protect the state write from cancellation** <br>
678
- The console now defers `LLM::Interrupt` while it saves the agent's
679
- state after a turn, so a cancel that arrives during the write cannot
680
- interrupt `agent.save` mid-flight and risk a lost or corrupted session
681
- file.
682
-
683
- ### Tools
684
-
685
- * **tools: the command runner is now `exec`** <br>
686
- The command tool that spawns a process without a shell is now
687
- [`LLM::Tool::Exec`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Exec.html),
688
- with the tool name `exec` instead of the previous `shell`. This is an
689
- internal refactor of the shell-out tools: `git`, `rg`, `mkdir`,
690
- `ruby`, and `bundle` all route through it and inherit
691
- its bounded output.
692
-
693
- * **tools: report when a command cannot be found** <br>
694
- `LLM::Tool::Exec` now returns `{ok: false, error: "command 'NAME' was
695
- not found on this system"}` when the requested command is missing,
696
- instead of a bare `{ok: false}` result that did not tell the model why
697
- the tool failed.
698
-
699
- * **tools: drop the `name:` parameter from `LLM::Tool::Exec#call`** <br>
700
- `LLM::Tool::Exec#call` now takes a single `arguments:` array instead of
701
- separate `name:` and `arguments:` parameters, with the command name as
702
- the first element (for example `arguments: ["rg", "-m", "10", "lib"]`).
703
- The `Git`, `Mkdir`, `Rg`, `Ruby`, and `Bundle` tools build their calls
704
- the same way. The change was made after models were observed confusing
705
- the two parameters, so a single list is simpler and more reliable.
706
-
707
- * **tools: rename `repl` as `console`** <br>
708
- The interactive loop is renamed to
709
- [`LLM::Console`](https://r.uby.dev/api-docs/llm.rb/LLM/Console.html),
710
- which better reflects what it does. `agent.console` is the primary
711
- entry point, and the require path moves from `llm/repl` to
712
- `llm/console`. Backwards-compatible aliases remain: `LLM::Repl`,
713
- `LLM::Agent#repl`, the ORM wrappers' `#repl`, and `LLM::Command =`
714
- `LLM::Console::Command`.
715
-
716
- * **tools: `LLM::Tool::Git#call` takes an `arguments:` array** <br>
717
- [`LLM::Tool::Git#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Git.html)
718
- now takes a single `arguments:` array in place of the previous
719
- `subcommand:` parameter. The first element must be one of `log`,
720
- `diff`, `commit`, `checkout`, `branch`, or `show`, validated before the
721
- command is spawned; the remaining elements are forwarded to git.
722
-
723
- * **tools: `LLM::Tool::Utils` now owns command spawning** <br>
724
- The shared [`LLM::Tool::Utils`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Utils.html)
725
- module now requires the `test-cmd.rb` gem (at `~> 2.7.1`) itself and
726
- exposes the `spawn` and `wait` helpers, so any tool that includes
727
- `Utils` gets command spawning without requiring `exec` directly.
728
- `Exec` and `ReadFile` include `Utils`, and the tools that shell out
729
- (`Git`, `Rg`, `Mkdir`, `Ruby`, and `Bundle`) route through `Exec`,
730
- so they all get the same bounded output.
731
-
732
- * **tools: route `git`, `rg`, `mkdir`, and `ruby` through `exec`** <br>
733
- `LLM::Tool::Git`, `LLM::Tool::Rg`, `LLM::Tool::Mkdir`, and
734
- `LLM::Tool::Ruby` now implement their calls through the `exec` tool,
735
- completing the refactor so every tool that shells out flows through
736
- the shared command runner with its bounded output.
737
-
738
- * **tools: read-file returns structured lines** <br>
739
- `LLM::Tool::ReadFile#call` now returns its content as structured
740
- `{lineno:, content:}` lines under a `lines:` key instead of a single
741
- `content:` string, and adds a `truncated:` flag. A reversed range
742
- (`start: 20, stop: 2`) is swapped to read lines 2 through 20. The
743
- truncation marker is kept out of the returned lines, so the model
744
- does not mistake it for a real file line.
745
-
746
- * **tools: write-file appends a trailing newline by default** <br>
747
- `LLM::Tool::WriteFile` now ensures written content ends with a newline,
748
- adding one when the content does not already end with `\n`. It previously
749
- wrote the content exactly as given. A new `newline:` parameter (default
750
- `true`) controls this, so `newline: false` writes the content exactly as
751
- given.
752
-
753
- * **tools: fix `edit-file` treating `before` as a regex** <br>
754
- `LLM::Tool::EditFile` now escapes the `before` snippet with
755
- `Regexp.escape`, so regex metacharacters are matched literally, and
756
- switches to the block form of `sub` so the `after` replacement keeps
757
- backslash sequences like `\1` and `\&` literal.
758
-
759
- * **tools: bound tool output with a per-tool `max_bytes`** <br>
760
- Each of the `Exec`, `ReadFile`, `Rg`, `Mkdir`, `Ruby`, and
761
- `Bundle` tools gains a `max_bytes` limit (default 75,000) for the
762
- maximum number of bytes a tool returns to the model. `Exec` and
763
- `ReadFile` add the class-level `max_bytes` accessor, which the other
764
- tools inherit through `Exec`, so each tool's cap can be configured
765
- independently, for example `LLM::Tool::ReadFile.max_bytes(175_000)`.
766
- It does not enforce the limit by itself;
767
- [`LLM::Tool::Utils#truncate`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Utils.html#truncate-instance_method)
768
- trims a string within the limit and marks the trailing content as
769
- truncated, and `truncate!` returns a `[content, truncated]` tuple for
770
- callers that structure truncated output themselves. `rg` also gains a
771
- `max_count:` parameter that caps the number of results per file.
772
-
773
- * **tools: add a `bundle` tool** <br>
774
- A new [`LLM::Tool::Bundle`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool/Bundle.html)
775
- tool runs a command through `bundle`. It uses the `BUNDLE_GEMFILE`
776
- environment variable when set, or a `Gemfile` in the current working
777
- directory otherwise. The tool takes an `arguments:` array, so the
778
- model passes the bundle command and its arguments as a single list,
779
- for example `arguments: ["exec", "rspec"]`.
780
-
781
- * **tools: resolve defaults through `LLM::Utils.resolve_option`** <br>
782
- A tool parameter default can now be an immediate value, a Symbol resolved
783
- as a method on the tool, or a Proc evaluated lazily at runtime, matching
784
- how `LLM::Agent` resolves its attributes. This lets a default track a
785
- value that can change between boot and runtime, such as a tool's
786
- `max_bytes`.
787
-
788
- ### Registry
789
-
790
- * **refresh model metadata** <br>
791
- Update `data/` with current pricing, limits, and capabilities for the
792
- OpenRouter, OpenAI, Bedrock, DeepInfra, DeepSeek, Google, Mistral,
793
- Moonshot, Z.ai, and Alibaba registries.
794
-
795
- ### Provider
796
-
797
- * **provider: retry `Net::WriteTimeout`, too** <br>
798
- Requests that raise `Net::WriteTimeout` are now retried alongside the
799
- other timed-out and rate-limited requests, up to the `retry_budget`,
800
- matching how `Net::OpenTimeout` and `Net::ReadTimeout` are handled. The
801
- console status bar also reports a write timeout as `Timed out`.
802
-
803
- * **alibaba: default to a retry budget of 8** <br>
804
- An agent that runs on the Alibaba provider now defaults to a retry
805
- budget of 8 instead of 5, because Alibaba (token plan) frequently rate
806
- limits and times out requests that it later recovers from. An explicit
807
- `retry_budget:` still overrides the default.
808
-
809
- * **deepseek: default to the `deepseek-flash` model** <br>
810
- The default DeepSeek chat model is now `deepseek-flash` instead of
811
- `deepseek-v4-flash`. DeepSeek resolves `deepseek-flash` to
812
- `deepseek-v4.1-flash` and recommends the name in its documentation and
813
- API error messages, so the default follows the current model alias
814
- instead of a pinned version.
815
-
816
- ### Tracer
817
-
818
- * **tracer: add `LLM::Tracer` factory methods** <br>
819
- Add
820
- [`LLM::Tracer.logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#logger-class_method),
821
- [`LLM::Tracer.pretty_logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#pretty_logger-class_method),
822
- and
823
- [`LLM::Tracer.telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#telemetry-class_method)
824
- as the preferred way to build a tracer for a provider, so switching
825
- between tracers means changing a factory method instead of a class name.
826
- The old `LLM.logger(llm, ...)` convenience method is removed in favor
827
- of `LLM::Tracer.logger(llm, ...)`.
828
-
829
- * **tracer: add `path:` support to `LLM::Tracer::PrettyLogger`** <br>
830
- [`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html)
831
- now accepts a `path:` option to write its human-readable entries to a
832
- file, matching `LLM::Tracer::Logger`. It previously only accepted `io:`.
833
-
834
- ### Fix
835
-
836
- * **json: scrub invalid UTF-8 on dump** <br>
837
- Fix a bug where [`LLM::JSONAdapter`](https://r.uby.dev/api-docs/llm.rb/LLM/JSONAdapter.html)
838
- raised a JSON generator error when dumping a string tagged as UTF-8 that
839
- carried invalid bytes. The normalize step now transcodes every string to
840
- valid UTF-8, replacing invalid sequences with the replacement character,
841
- so dumping works on `json ~> 3.0`. The `oj` and `yajl` adapters now run
842
- the same normalization, so every backend scrubs invalid bytes before
843
- serializing.
844
-
845
- * **fork: require xchan.rb `~> 0.23`** <br>
846
- The `:fork` concurrency strategy now requires the `xchan.rb` gem at
847
- `~> 0.23` instead of `~> 0.22`. xchan.rb 0.23.0 replaces the external
848
- `lockf.rb` gem with a built-in, Fiddle-based `Chan::Lockf`, so fork
849
- channels no longer carry that extra dependency. (The socket
850
- length-header deadlock fix shipped earlier, in xchan.rb 0.22.0.)
851
-
852
- * **async: fix a shutdown exception on the reactor thread** <br>
853
- Fix a bug where the `:async` strategy's
854
- [`LLM::Function::Async::Reactor`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Async/Reactor.html)
855
- raised a `TypeError` on shutdown with recent `async` and `io-event`
856
- versions, because their internals tried to raise an integer as an
857
- exception. The scheduler is now detached from the reactor thread
858
- before it exits, which avoids that code path entirely, and teardown
859
- is managed by `reactor.stop`, so the thread exits promptly instead of
860
- abruptly or hanging.
861
-
862
- * **openai: prevent the loss of user messages in the completions path** <br>
863
- Fix a bug where the request body was built from `params[:messages]`
864
- alone when that key was present, discarding the messages built from the
865
- prompt. The DeepSeek and Alibaba schema support injects the schema
866
- system message into `params[:messages]`, so a request with a `schema:`
867
- could be sent with the schema message only, dropping the user's
868
- messages. The built messages now lead with `params[:messages]`, and the
869
- key is removed before the body is assembled.
870
-
871
- ## v15.1.0
872
-
873
- Changes since `v15.0.3`.
874
-
875
- This release adds the OpenRouter provider, splits provider timeouts
876
- into `connect_timeout` and `read_timeout`, retries timed-out requests,
877
- and renames `on_rate_limit` to `on_retry`. Skills now gain a
878
- frontmatter `model:` parameter and inherit the parent agent's model,
879
- while the CLI gains `-m` and `-x` switches, and the console shows retry
880
- progress and measures text by display width.
881
-
882
- ### Provider
883
-
884
- * **add `LLM::OpenRouter` for the OpenRouter provider** <br>
885
- [`LLM::OpenRouter`](https://r.uby.dev/api-docs/llm.rb/LLM/OpenRouter.html)
886
- is a new provider that talks to [OpenRouter](https://openrouter.ai)
887
- through its OpenAI-compatible API, contributed via
888
- [PR #165](https://github.com/r-uby-dev/llm.rb/pull/165). Create an
889
- instance with
890
- [`LLM.openrouter`](https://r.uby.dev/api-docs/llm.rb/LLM.html#openrouter-class_method),
891
- which accepts the same `key:`, `host:`, and `base_path:` options as the
892
- OpenAI provider. It defaults to the `openrouter/auto` router model and
893
- supports chat completions, streaming, tool calls, structured output,
894
- and embeddings; image, audio, moderation, files, and vector store
895
- endpoints raise `NotImplementedError`. Model metadata ships in
896
- `data/openrouter.json` for the registry.
897
-
898
- * **provider: `LLM::Provider#with` accepts headers without the `headers:` keyword** <br>
899
- [`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method)
900
- now accepts headers directly as a Hash (`llm.with("User-Agent" => "llmrb/1.0")`)
901
- in addition to the legacy `headers:` keyword form. Both are merged, and
902
- the keyword form remains for backwards compatibility.
903
-
904
- * **providers: `model: nil` falls back to `default_model`** <br>
905
- [`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
906
- now treats a `model: nil` payload as "use the default model" instead of
907
- sending a model value that providers reject. Previously a `{model: nil}`
908
- param overwrote the default and then was removed by `.compact`, leaving
909
- undefined behavior where providers could reject the request.
910
-
911
- * **openai: handle `model: nil` in the Responses API** <br>
912
- The OpenAI Responses API path now also falls back to the default model
913
- when `model: nil`, matching the completions path.
914
-
915
- * **provider: separate `connect_timeout` and `read_timeout`** <br>
916
- [`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
917
- now accepts a `connect_timeout:` for opening the connection (default 5
918
- seconds) and a `read_timeout:` for waiting on an idle connection (default
919
- 600 seconds), instead of a single `timeout`. The legacy `timeout:` option
920
- remains as a shorthand for `read_timeout`. The provider exposes the new
921
- `#read_timeout` and `#connect_timeout` accessors.
922
-
923
- * **provider: retry timed-out requests** <br>
924
- Requests that time out (`Net::OpenTimeout` or `Net::ReadTimeout`) are now
925
- retried along with rate-limited requests, up to the `retry_budget`. When
926
- a heavily loaded provider (for example DeepSeek) drops the connection
927
- during the connect or read phase, the request is retried instead of
928
- failing, and the stream is notified through `on_retry`.
929
-
930
- * **context: retry on `LLM::InsufficientQuotaError`, too** <br>
931
- A request that raises
932
- [`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html)
933
- is now retried like other rate-limited requests. The error is a subclass
934
- of `LLM::RateLimitError` and can be raised at regular intervals, in
935
- particular by the Alibaba provider.
936
-
937
- * **cost: return `LLM::Cost.zero` when the registry has no pricing** <br>
938
- Add
939
- [`LLM::Cost.zero`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#zero-class_method),
940
- a factory for a zero-valued cost breakdown.
941
- [`LLM::Context#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#cost-instance_method)
942
- now returns it when the active model has no pricing in the registry (for
943
- example OpenRouter's `openrouter/auto` router model), instead of crashing
944
- on a nil pricing entry.
945
-
946
- ### Skills
947
-
948
- * **skills: add a `model` frontmatter parameter** <br>
949
- A skill's `SKILL.md` frontmatter can now declare a `model:` value, so a
950
- skill's sub-agent runs on a specific model instead of the default. This
951
- lets a parent agent run on one model (for example `deepseek-v4-flash`)
952
- while the skill runs on another (`deepseek-v4-pro`). The `model` value is
953
- not strictly portable between providers.
954
-
955
- * **skills: inherit the model of the parent agent** <br>
956
- A skill's sub-agent now inherits the active model of the agent that
957
- spawns it instead of falling back to the provider default. The frontmatter
958
- `model:` parameter overrides it explicitly when set.
959
- [`LLM::Context#model`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#model-instance_method)
960
- now falls back to the provider's default model when no model is set.
961
-
962
- ### Stream
963
-
964
- * **stream: rename `on_rate_limit` to `on_retry`** <br>
965
- [`LLM::Stream#on_rate_limit`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_rate_limit-instance_method)
966
- is renamed to
967
- [`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry-instance_method),
968
- which fires when a request is retried after a rate limit or a timeout,
969
- and now receives the one-based retry attempt number as a second argument.
970
- The old `on_rate_limit` name remains as an alias.
971
-
972
- ### CLI
973
-
974
- * **cli: add a `-m` switch for choosing the model** <br>
975
- `bin/llm.rb` now accepts `-m MODEL` to run the session on a model other
976
- than the provider default, for example
977
- `llm.rb -p deepseek -m deepseek-v4-pro`. A model unknown to the provider's
978
- registry prints an error and exits.
979
-
980
- * **cli: add a `-x` switch for the read timeout** <br>
981
- `bin/llm.rb` now accepts `-x SECONDS` to set the provider read timeout.
982
-
983
- * **cli: print a backtrace on fatal crashes** <br>
984
- When `bin/llm.rb` hits an unexpected error, the crash message now includes
985
- up to three stack lines from the backtrace, so the failure is easier to
986
- locate and report than a bare diagnostic.
987
-
988
- ### Console
989
-
990
- * **console: show retry progress in the status bar** <br>
991
- When a request is rate limited or times out, the curses-based console status
992
- bar shows a retry indicator with the error and the remaining attempts, for
993
- example `🔁 Rate limited • attempt 2 of 5`.
994
-
995
- * **console: measure text width with `unicode-display_width`** <br>
996
- The curses-based console now counts and slices text by display column width
997
- instead of character count, so wrapping, table columns, and clipping stay
998
- aligned for wide characters such as emoji. It requires the optional
999
- `unicode-display_width` gem.
1000
-
1001
- * **console: treat `LLM::InsufficientQuotaError` as a rate limit in the status bar** <br>
1002
- The curses-based console status bar now shows `Rate limited` when a request
1003
- raises
1004
- [`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html),
1005
- matching how ordinary `LLM::RateLimitError`s are shown, instead of falling
1006
- through to the raw class name.
1007
-
1008
- ### Registry
1009
-
1010
- * **refresh model metadata** <br>
1011
- Update `data/*.json` with current model listings and pricing, adding
1012
- GPT-5.6 Sol, Terra, and Luna models to Bedrock, Grok 4.6 and Grok Imagine
1013
- Image 2.0 to xAI, DeepSeek V4 Flash Vision to DeepSeek, and DeepSeek V4
1014
- Pro 0813, Qwen3 VL, and Qwen3.8 models to DeepInfra.
1015
-
1016
- ## v15.0.3
1017
-
1018
- Changes since `v15.0.2`.
1019
-
1020
- This release fixes ActiveRecord `:json`/`:jsonb` serialization so tool
1021
- call arguments round-trip as JSON objects instead of arrays of pairs.
1022
-
1023
- ### Fix
1024
-
1025
- * **activerecord: serialize tool call arguments properly** <br>
1026
- Fix a bug where the ActiveRecord `:json`/`:jsonb` layer serialized tool
1027
- call arguments as an array of pairs instead of a Hash. The context is
1028
- now serialized through its JSON form, so tool call arguments round-trip
1029
- as JSON objects that providers accept.
1030
-
1031
- ## v15.0.2
1032
-
1033
- Changes since `v15.0.1`.
1034
-
1035
- This release fixes a concurrency race in MCP tool calls by
1036
- reference-counting the transport session, so overlapping tool calls
1037
- reuse the running transport instead of racing `start`/`stop`.
1038
-
1039
- ### Fix
1040
-
1041
- * **mcp: fix concurrent MCP tool call race** <br>
1042
- `LLM::MCP` now reference-counts its transport session: the first
1043
- caller starts the transport and the last caller stops it. Concurrent
1044
- or overlapping tool calls reuse the running transport instead of
1045
- racing `start`/`stop`, avoiding "MCP transport is not running" errors
1046
- that could occur with the `async` strategy. An externally started
1047
- transport is never stopped by a borrower.
1048
-
1049
- ## v15.0.1
1050
-
1051
- Changes since `v15.0.0`.
1052
-
1053
- This release fixes agent `set` handling of single `Symbol`/`Proc`
1054
- values and resolves ORM options as `Symbol`s through the bound record
1055
- instead of the `LLM::Agent` instance.
1056
-
1057
- ### Agent
1058
-
1059
- * **agent: fix `set (skills|tools): Symbol|Proc`** <br>
1060
- The `skills`, `tools`, `confirm`, and `schema` class accessors now
1061
- consistently resolve a single `Symbol` or `Proc` lazily at agent
1062
- initialization, so `LLM::Agent.set(skills: proc { [...] })` and
1063
- `LLM::Agent.set(tools: :tools)` no longer raise. The `set` family
1064
- shares one `single_callable?` helper rather than each accessor
1065
- duplicating its own logic.
1066
-
1067
- ### Fix
1068
-
1069
- * **orm: resolve a `Symbol` through the bound record** <br>
1070
- When an ORM model using `acts_as_agent` (ActiveRecord), `plugin :agent`
1071
- (Sequel), or `acts_as_llm` / `plugin :llm` configures an option as a
1072
- `Symbol`, that symbol is now resolved on the model instance (or its
1073
- bound record) instead of the `LLM::Agent` instance. The contexts and
1074
- agents built by the wrappers are now bound to the record, so a model like
1075
- `agent.set :tools` with a `tools` method on the record works as
1076
- expected.
1077
-
1078
- ## v15.0.0
1079
-
1080
- Changes since `v14.0.0`.
1081
-
1082
- This release renames `usage` to `token_usage` across contexts, agents,
1083
- and messages, makes `context_window` return `nil` when unknown, and
1084
- makes `LLM::Cost` accessors always return `Float`. It also adds the
1085
- Alibaba provider, automatic API key discovery from the environment,
1086
- new context-usage and context-used methods, a retry budget for
1087
- rate-limited requests, and a range of REPL and skills improvements.
1088
-
1089
- ### Breaking
1090
-
1091
- #### Migration
1092
-
1093
- | Old | New |
1094
- |-----|-----|
1095
- | `ctx.usage` / `agent.usage` / `msg.usage` | `ctx.token_usage` / `agent.token_usage` / `msg.token_usage` (`usage` remains an alias) |
1096
- | `LLM::Message#usage` returns `LLM::Object` | `LLM::Message#token_usage` returns a copy of `LLM::Usage`, for assistant messages only |
1097
- | `ctx.context_window` returns `0` when the model isn't in the registry | returns `nil` when unknown |
1098
- | `LLM::Cost#input` (and other accessors) return `nil` when unused | return `0.0` |
1099
- | `ctx.usage` returns the most recent assistant message usage | sums token usage across all assistant messages |
1100
- | a skill exposes its tool as `weather` (the skill name) | the generated tool is now named `weather-skill` |
1101
-
1102
- * **rename `#usage` to `#token_usage` across contexts, agents, and messages** <br>
1103
- [`LLM::Context#usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#usage-instance_method),
1104
- [`LLM::Agent#usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#usage-instance_method),
1105
- and
1106
- [`LLM::Message#usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html#usage-instance_method)
1107
- are now aliases of `token_usage`.
1108
- [`LLM::Message#token_usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html#token_usage-instance_method)
1109
- now returns a copy of `LLM::Usage` instead of `LLM::Object`, and only
1110
- returns a value for assistant messages.
1111
-
1112
- * **`LLM::Context#context_window` now returns `nil` when unknown** <br>
1113
- [`LLM::Context#context_window`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#context_window-instance_method)
1114
- now returns `nil` when the model's context window size is not known to the
1115
- runtime, instead of `0`. This makes the code check for a window instead
1116
- of a number, so an unknown window no longer reads as a real (zero) size.
1117
-
1118
- * **`LLM::Cost` accessors always return `Float` objects** <br>
1119
- The cost accessors on
1120
- [`LLM::Cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
1121
- (`input`, `output`, `input_audio`, `output_audio`, `input_image`,
1122
- `cache_read`, `cache_write`, and `reasoning`) now always return a
1123
- `Float`, returning `0.0` when no tokens of that kind were used, instead
1124
- of `nil`. Callers can sum and compare cost values without guarding
1125
- against `nil`.
1126
-
1127
- * **expose new context methods on the ActiveRecord and Sequel wrappers** <br>
1128
- The `acts_as_llm` (ActiveRecord) and `plugin :llm` (Sequel) wrappers
1129
- now expose `context_used` and `context_usage`, delegating to the
1130
- wrapped `LLM::Context`. `token_usage` replaces `usage` (which remains
1131
- as an alias), and `context_window` now returns `nil` when the model's
1132
- context window is unknown instead of `0`.
1133
-
1134
- ### Core
1135
-
1136
- * **discover API keys from the environment** <br>
1137
- [Cloud provider factories](https://r.uby.dev/api-docs/llm.rb/LLM.html)
1138
- (`LLM.anthropic`, `LLM.google`, `LLM.deepseek`, `LLM.openai`,
1139
- `LLM.xai`, `LLM.mistral`, `LLM.zai`, `LLM.moonshot`,
1140
- `LLM.alibaba`, and `LLM.aliyun`) now resolve the provider's API key
1141
- automatically when no `key:` is given, by walking the environment
1142
- variable names listed in the models.dev registry. So `LLM.openai`
1143
- works without an explicit key as long as `OPENAI_API_KEY` (or one of
1144
- the registry's alternative names) is set in the environment. A
1145
- missing key raises `ArgumentError`.
1146
-
1147
- * **cli: auto-discover credentials and support Bedrock** <br>
1148
- `bin/llm.rb` now resolves the provider through the `LLM` factory
1149
- methods instead of mapping environment variable names directly, so it
1150
- picks up Bedrock (all three AWS credentials) and relies on the same
1151
- automatic key discovery as the library. The CLI also always starts
1152
- now: without arguments it falls back to `ollama` or `llamacpp`. A
1153
- provider whose credentials are not set exits with status 1.
1154
-
1155
- * **cli: add `-c` and `-n` switches** <br>
1156
- `bin/llm.rb` now accepts a `-c STRATEGY` switch to choose the
1157
- concurrency strategy used for tool calls (`thread`, `async`, `fork`,
1158
- or any of the other strategies) and a `-n TRANSPORT` switch to choose
1159
- the HTTP transport (`net-http`, `net-http-persistent`, or `curb`),
1160
- both forwarded to the session's agent and provider.
1161
-
1162
- * **context: keep runtime parameters from reaching the provider** <br>
1163
- [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
1164
- now strips its runtime-only parameters (`guard`, `retry_budget`,
1165
- `concurrency`, `transformer`, and `compactor`) before merging params
1166
- into a provider request, so they can never cross the context-provider
1167
- boundary and risk an API-level error.
1168
-
1169
- * **add `retry_budget` support for rate-limited requests** <br>
1170
- [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
1171
- now accepts a `retry_budget:` that automatically sleeps and retries a
1172
- rate-limited request up to the given number of times before raising
1173
- `LLM::RateLimitError`. Each retry sleeps a growing interval (2s, 4s,
1174
- 6s, ...) and notifies the stream through
1175
- [`LLM::Stream#on_rate_limit`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_rate_limit-instance_method).
1176
- [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
1177
- enables a budget of 5 by default, while a raw context disables it (0)
1178
- unless configured.
1179
-
1180
- * **add `LLM::Usage.zero`** <br>
1181
- Add
1182
- [`LLM::Usage.zero`](https://r.uby.dev/api-docs/llm.rb/LLM/Usage.html#zero-class_method)
1183
- as a zero-valued usage object. `LLM::Context#usage`, `LLM::Agent#usage`,
1184
- and the ActiveRecord and Sequel wrappers now return `LLM::Usage` objects
1185
- instead of `LLM::Object` when no provider usage has been recorded yet.
1186
-
1187
- * **add `LLM::Context#context_usage` and `LLM::Agent#context_usage`** <br>
1188
- Add
1189
- [`LLM::Context#context_usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#context_usage-instance_method)
1190
- and
1191
- [`LLM::Agent#context_usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#context_usage-instance_method),
1192
- which return the fraction of the model's context window currently used as
1193
- a `Rational` (for example `Rational(100, 10_000)`), or `nil` when the used
1194
- amount or the window size is unknown. The REPL status bar now renders this
1195
- fraction instead of computing the remainder from raw token counts.
1196
-
1197
- * **add `LLM::Context#context_used` and `LLM::Agent#context_used`** <br>
1198
- Add
1199
- [`LLM::Context#context_used`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#context_used-instance_method)
1200
- and
1201
- [`LLM::Agent#context_used`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#context_used-instance_method),
1202
- which return the live context size (in tokens) of the most recent
1203
- assistant message, or `nil` when no assistant message has a recorded token
1204
- usage. This fills the gap left after `token_usage` became accumulative and
1205
- no longer represented a single turn, so callers can read how much of the
1206
- context window has been used without walking the messages themselves.
1207
-
1208
- ### Provider
1209
-
1210
- * **add `LLM::Provider#registry`** <br>
1211
- Add [`LLM::Provider#registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#registry-instance_method),
1212
- which returns the provider's model registry. `LLM::Context#registry`
1213
- and `LLM::Agent#registry` now delegate to their underlying provider
1214
- instead of looking it up on their own.
1215
-
1216
- * **add `LLM::Alibaba` for Alibaba Cloud Model Studio** <br>
1217
- [`LLM::Alibaba`](https://r.uby.dev/api-docs/llm.rb/LLM/Alibaba.html)
1218
- is a new provider that talks to
1219
- [Alibaba Cloud Model Studio](https://www.alibabacloud.com/help/en/model-studio/models)
1220
- through its OpenAI-compatible API, including the Qwen3 family of
1221
- models. Create an instance with
1222
- [`LLM.alibaba`](https://r.uby.dev/api-docs/llm.rb/LLM.html#alibaba-class_method),
1223
- also aliased as `LLM.aliyun`, which accepts the same `key:`, `host:`,
1224
- and `base_path:` options as the OpenAI provider. The provider defaults
1225
- to the `deepseek-v4-flash-0731` model and supports chat completions,
1226
- streaming, tool calls, and structured output through the shared
1227
- OpenAI-compatible path; image, audio, moderation, responses, and
1228
- vector store endpoints raise `NotImplementedError`. Model metadata
1229
- ships in `data/alibaba.json` for the registry.
1230
-
1231
- * **alibaba: support structured outputs via `json_object`** <br>
1232
- [`LLM::Alibaba`](https://r.uby.dev/api-docs/llm.rb/LLM/Alibaba.html)
1233
- now supports structured output through a shared `json_object` fallback,
1234
- since Alibaba models do not support `json_schema` natively. The schema is
1235
- described in an injected system message that also satisfies the
1236
- "messages must contain the word json" requirement. The same shared
1237
- fallback now also backs DeepSeek.
1238
-
1239
- * **alibaba: default to the pay-as-you-go host** <br>
1240
- The default
1241
- [`LLM::Alibaba`](https://r.uby.dev/api-docs/llm.rb/LLM/Alibaba.html)
1242
- host is now `dashscope-intl.aliyuncs.com`. Override it globally with
1243
- the `DASHSCOPE_API_HOST` environment variable, or per instance with
1244
- `LLM.alibaba(host: ...)`, for example to point at a Token Plan
1245
- endpoint.
1246
-
1247
- * **alibaba: use `DASHSCOPE_API_KEY` as the default key env var** <br>
1248
- [`LLM::Alibaba`](https://r.uby.dev/api-docs/llm.rb/LLM/Alibaba.html)
1249
- now discovers its API key from `DASHSCOPE_API_KEY` instead of
1250
- `ALIBABA_API_KEY`, following the models.dev registry convention.
1251
-
1252
- * **alibaba: raise `LLM::InsufficientQuotaError` for exhausted quota** <br>
1253
- Add
1254
- [`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html),
1255
- a subclass of `LLM::RateLimitError`, for when a provider reports a
1256
- tokens-per-minute (TPM) quota limit.
1257
- [`LLM::Alibaba`](https://r.uby.dev/api-docs/llm.rb/LLM/Alibaba.html)
1258
- now raises it when Alibaba responds with an `insufficient_quota`
1259
- error. Since it subclasses `RateLimitError`, quota errors are retried
1260
- like other rate limits.
1261
-
1262
- * **bedrock: auto-discover AWS credentials from the environment** <br>
1263
- [`LLM.bedrock`](https://r.uby.dev/api-docs/llm.rb/LLM.html#bedrock-class_method)
1264
- now infers its credentials from the `AWS_ACCESS_KEY_ID`,
1265
- `AWS_SECRET_ACCESS_KEY`, and `AWS_REGION` environment variables when
1266
- they are not passed explicitly, matching the other cloud providers. A
1267
- missing key raises `ArgumentError`.
1268
-
1269
- * **add `LLM::Bedrock#key?`** <br>
1270
- Add
1271
- [`LLM::Bedrock#key?`](https://r.uby.dev/api-docs/llm.rb/LLM/Bedrock.html#key%3F-instance_method),
1272
- which overrides the superclass method to check all three Bedrock
1273
- credentials (`access_key_id`, `secret_access_key`, and `region`)
1274
- instead of a single API key.
1275
-
1276
- ### Function
1277
-
1278
- * **make `Sequential::Group` abide by the `LLM::Function::Group` contract** <br>
1279
- [`LLM::Function::Sequential::Group`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Sequential/Group.html)
1280
- now receives an array of
1281
- [`LLM::Function::Sequential::Task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Sequential/Task.html)
1282
- objects instead of raw `LLM::Function` objects, matching the interface
1283
- shared by every other concurrency strategy.
1284
- [`LLM::Function::Array#task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Array.html#task-instance_method)
1285
- wraps each function as a `Sequential::Task` before constructing the
1286
- group, and the group delegates `spawn`, `alive?`, and `wait` to those
1287
- tasks. This fixes `Sequential::Group#alive?`, which always returned
1288
- `false`, and restores guard handling for sequential execution by
1289
- honoring the shared `guarded:` option on `Sequential::Task`.
1290
-
1291
- * **function: redirect output streams in `:fork` tool processes** <br>
1292
- The `:fork` concurrency strategy (via
1293
- [`LLM::Function::Fork::Task`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Fork/Task.html))
1294
- now redirects the child process's `$stdout` and `$stderr` to
1295
- `File::NULL`, so a forked tool can no longer clobber the parent
1296
- terminal, for example by blanking the curses REPL display. A tool that
1297
- genuinely needs the terminal can still reopen `/dev/tty`; the file
1298
- descriptor stays available to the child.
1299
-
1300
- ### Fix
1301
-
1302
- * **a2a: fix a typo in the HTTP transport** <br>
1303
- Fix a bug in
1304
- [`LLM::A2A::Transport::HTTP`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Transport/HTTP.html)
1305
- where the constructor read `uri.port` instead of `@uri.port`, which
1306
- crashed the program whenever
1307
- [`LLM::A2A.rest`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A.html#rest-class_method)
1308
- or
1309
- [`LLM::A2A.jsonrpc`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A.html#jsonrpc-class_method)
1310
- was used. The transport now reads the port from the parsed `@uri`.
1311
-
1312
- * **openai: report usage for streamed completions requests** <br>
1313
- Fix a bug in the OpenAI completions path where `params[:stream]` was
1314
- checked after it had been deleted from the params hash, so the check
1315
- always evaluated to `false`. The fix checks the resolved stream's
1316
- `enabled?` instead, so `stream_options: {include_usage: true}` is
1317
- added to streamed requests and API usage is reported back to the
1318
- caller.
1319
-
1320
- * **curb: read the stream body and resolve streaming requests** <br>
1321
- Fix two bugs in [`LLM::Transport::Curb`](https://r.uby.dev/api-docs/llm.rb/LLM/Transport/Curb.html)
1322
- that left the `curb` transport unusable. The request body setter now
1323
- reads a streaming request's body stream into a string (dropping the
1324
- chunked transfer header, which curb replaces with a content length),
1325
- and the result builder now accumulates the response body from the
1326
- `on_body` callback instead of leaving it empty.
1327
-
1328
- * **cli: handle errors in `main`** <br>
1329
- Wrap all of `bin/llm.rb`'s `main` method in error handling: an
1330
- interrupted session exits gracefully with `Bye!`, an explicit provider
1331
- is passed the resolved transport, and any unexpected error prints a
1332
- formatted diagnostic with a link to issue tracking before exiting.
1333
-
1334
- * **cli: persist the session mapping file** <br>
1335
- Fix a bug where `bin/llm.rb` saved the session file at
1336
- `~/.llm.rb/<provider>/<uuid>.json` but never wrote the updated
1337
- working-directory mapping back to `~/.llm.rb/<provider>.json`. The
1338
- mapping file is now written whenever a new session is registered.
1339
-
1340
- * **context: aggregate usage across all assistant messages** <br>
1341
- [`LLM::Context#usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#usage-instance_method)
1342
- now sums token usage across every assistant message in the conversation
1343
- instead of returning only the first message's usage.
1344
- [`LLM::Cost.from`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
1345
- now subtracts reasoning tokens from the output total and cache-read
1346
- tokens from the input total before pricing, and prices reasoning tokens
1347
- with the model's reasoning rate when one is available.
1348
-
1349
- ### Repl
1350
-
1351
- * **draw a top chrome row with the cwd and active model** <br>
1352
- The curses-based REPL now draws a white-on-blue row at the very top of
1353
- the screen showing the current working directory on the left and the
1354
- active model on the right. The row is drawn above the transcript and
1355
- uses a new blue status-bar color pair.
1356
-
1357
- * **redraw the window on resize** <br>
1358
- The curses-based REPL now handles the terminal resize signal
1359
- (`KEY_RESIZE`) while reading input, clearing and redrawing the entire
1360
- window so the layout stays aligned after the terminal is resized.
1361
-
1362
- * **hide the cursor until the window is ready** <br>
1363
- Fix a visual glitch where the curses-based REPL showed the cursor at
1364
- position 0,0 at startup and then jumped it to the input area once the
1365
- window was drawn. The cursor is now hidden until the input field has
1366
- been drawn and the cursor can be placed directly into it.
1367
-
1368
- * **collapse the cost to two decimal places** <br>
1369
- [`LLM::Cost#to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
1370
- now renders the total cost with two decimal places (for example
1371
- `$0.01`), so the REPL status bar shows a compact cost estimate instead
1372
- of a long run of digits.
1373
-
1374
- * **add auto-complete ability for commands** <br>
1375
- [`LLM::Repl::Command`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html)
1376
- subclasses can now override a `complete` method to autocomplete their
1377
- arguments. The method receives the command's parameters as keyword
1378
- arguments, with the non-nil keyword being the active fragment, and
1379
- returns candidate completions. Repeated TAB presses cycle through the
1380
- candidate list.
1381
-
1382
- * **add `LLM::Repl#model` and `LLM::Repl#model=`** <br>
1383
- [`LLM::Repl`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl.html#model-instance_method)
1384
- now tracks the active model in its own `model` attribute, seeded from
1385
- the wrapped agent's model. The status bar reads the model through the
1386
- repl instead of the agent, so the model can be switched within a
1387
- session.
1388
-
1389
- * **add `/model` command** <br>
1390
- A new `/model <name>` command switches the active model within a
1391
- single REPL session. Its argument auto-completes through the
1392
- text-to-text models in the registry.
1393
-
1394
- * **repl: restrict autocomplete to text-to-text models** <br>
1395
- The `/model` command's argument auto-complete now suggests only
1396
- text-to-text models, so embedding and other non-chat models are
1397
- left out of the completion list.
1398
-
1399
- * **repl: highlight GitHub-flavored codeblocks** <br>
1400
- The curses-based REPL now parses the GitHub-style ``` fences that
1401
- models commonly emit as real code blocks. Kramdown's native fenced-code
1402
- syntax uses `~~~`, so the ``` fences were previously parsed as inline
1403
- code spans. The language name is now shown in bold white above the code,
1404
- which renders in green.
1405
-
1406
- * **repl: fix a scroll render artifact** <br>
1407
- Fix a bug where scrolling upward could leave a piece of text just
1408
- above the status row as a render artifact. The row above the status
1409
- row is now cleared on every buffer render.
1410
-
1411
- * **repl: add a buffer row below the blue status bar** <br>
1412
- The curses-based REPL buffer now starts with an empty row below the
1413
- blue status bar, improving the visual spacing of the first exchange
1414
- in the chat.
1415
-
1416
- * **repl: pin `curses` and `kramdown` to tested versions** <br>
1417
- The REPL now pins `curses` to `~> 1.6` and `kramdown` to `~> 2.5`
1418
- through `LLM.require`, so it loads gem versions known to have been
1419
- tested instead of whatever happens to be installed.
1420
-
1421
- ### Skills
1422
-
1423
- * **skills: append `-skill` to the generated tool name** <br>
1424
- A skill is now exposed as a tool named `"<skill>-skill"` instead of
1425
- just the skill's name, so a skill like `weather` that also uses a tool
1426
- named `weather` no longer collides with it (or a same-named global
1427
- tool) in the tool registry.
1428
-
1429
- * **skills: add `LLM::Stream` skill lifecycle callbacks** <br>
1430
- Add
1431
- [`LLM::Stream#on_skill_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_call-instance_method)
1432
- and
1433
- [`LLM::Stream#on_skill_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_skill_return-instance_method),
1434
- which are called before a skill's sub-agent runs and after it finishes.
1435
- `on_skill_return` receives the `LLM::Agent` sub-agent that ran the skill
1436
- along with the resulting `LLM::Response`, so a stream can inspect the
1437
- sub-agent's conversation, tally its usage, or add a verification step. A
1438
- stream can use the two callbacks to know when a skill sub-agent is
1439
- running.
1440
-
1441
- ### Registry
1442
-
1443
- * **add `LLM::Registry::Model` as a comparable model wrapper** <br>
1444
- Add [`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html),
1445
- a wrapper around a model's registry metadata (pricing, limits,
1446
- capabilities, and modalities). Models are comparable by price, so
1447
- `models.sort` orders them from cheapest to most expensive. The class
1448
- exposes predicate helpers such as `tool_call?`, `reasoning?`,
1449
- `structured_output?`, `open_weights?`, `text?`, `image?`, `audio?`,
1450
- `pdf?`, and `video?`, plus `input_cost`, `output_cost`, and
1451
- `context_window` accessors.
1452
-
1453
- * **gemspec: bundle the deepdive guide from `docs/`** <br>
1454
- The gemspec now packages the deepdive guide from `docs/deepdive.md`
1455
- and `docs/deepdive/*/*.md` after the deepdive sources moved from
1456
- `resources/` to `docs/`, so the full guide ships with the gem.
1457
-
1458
- * **make `LLM::Registry#models` return model objects** <br>
1459
- [`LLM::Registry#models`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#models-instance_method)
1460
- now returns a list of
1461
- [`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html)
1462
- objects instead of model name strings. Use the new
1463
- [`LLM::Registry#keys`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#keys-instance_method)
1464
- method to get the model names.
1465
-
1466
- * **refresh DeepInfra model metadata** <br>
1467
- Update `data/deepinfra.json` with current pricing for the DeepSeek
1468
- V4, DeepSeek-V3, DeepSeek-R1-0528, and Kimi-K3 models, and mark
1469
- `structured_output` support for one model.