llm.rb 15.2.0 → 15.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5c55a3fa1151afad5720fc33bbe60a71feed5577242e53b4e8d4e2f9962a18ed
-  data.tar.gz: 3099ef73435ed6d91b003a40e2e5c195a82a8f631418b21e7c8cbce9088a5c76
+  metadata.gz: 8437f48c9a995054696648dfbe3386a5f0d8c8a62379765d2eca8694cf3adb3b
+  data.tar.gz: d2c36a69342e2fef94472abc25e813a0ee8bbef9aba15e68b275d89d307a6405
 SHA512:
-  metadata.gz: 8e0d5dd6c98355412a31dcb638081ea9f0553725cbf30aa6598eed5d16cd6d4693b5edfae8c063625ee2f5198b342e9b11850897370404267d291ddae4d747fe
-  data.tar.gz: 25102554a840fee5046804d17ff91673793603b96ab9ae246a30b7d3192a792e9bd7e384d76ce20abf2d504910330202cb2d4aef0057c7875c1ce3866ccfe177
+  metadata.gz: d1d120fe448ee693ba627cfdfe213cd42030fe8448f72c8cf6ecaddb62d2ac5f7b07ded6191142ec28571b41acbdf8c96b458420889e7fb75990f53f51b3ab43
+  data.tar.gz: 816b6feffe0339f52798062de3f36188b3d6574203289f450fd07de2e1e600b29aa956a8ba50947d3ae27540a4132222f3b31bb865784f88e24074388a852058

data/CHANGELOG.md

@@ -17,6 +17,46 @@
 
 *No unreleased changes yet. Check back after the next release.*
 
+## v15.2.1
+
+Changes since `v15.2.0`.
+
+This release makes the agent count its tool budget across the whole turn
+and emit tool returns to the stream once that budget is spent, and lets
+`LLM::Function#cancel` carry extra return fields.
+
+### Agent
+
+* **agent: count the tool budget across the whole turn** <br>
+  [`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
+  now caps the tool calls a turn runs in total. A batch of calls is
+  spent as a batch, and a batch that would take the turn past its
+  budget is not run at all; once the budget is spent no tool call runs,
+  and the agent keeps sending its in-band advisory until the model
+  answers without requesting more tools. Previously the agent ran up to
+  the budget, then ran further batches after each advisory, so a turn
+  could run more tool calls than its budget allowed.
+
+* **agent: emit tool returns when the budget is spent** <br>
+  Fix a bug where, once a turn's tool budget was spent, the agent passed
+  its in-band returns to the model without emitting them to the stream.
+  The stream went silent, so a stream that tracked tool-call state left
+  the calls in the `call` state. The returns are now emitted through
+  [`LLM::Stream#on_tool_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_return-instance_method)
+  like any other tool return.
+
+### Function
+
+* **function: `LLM::Function#cancel` takes extra return fields** <br>
+  [`LLM::Function#cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#cancel-instance_method)
+  now accepts keywords beyond `reason:` and merges them into the
+  [`LLM::Function::Return`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Return.html)
+  it builds. `LLM::Function#budget_spent` now builds its return through
+  `cancel` instead of hand-rolling an error, so a budget-spent return is
+  marked `cancelled: true` and carries `action:` and `advice:` hints. A
+  callback that receives returns can now tell a cancellation apart from
+  an ordinary error.
+
 ## v15.2.0
 
 Changes since `v15.1.0`.

data/README.md

@@ -19,10 +19,10 @@ on CRuby. It has zero runtime dependencies by default, supports
 concurrent and parallel tool execution and has a single coherent API
 that spans 14+ providers.
 
-The easiest way to learn about llm.rb is to ask [the r.uby.dev chatbot](https://r.uby.dev)
+The most effective way to learn about llm.rb is to ask [the r.uby.dev chatbot](https://r.uby.dev)
 a question. It is connected to the llm.rb GitHub repository, backed by
-ActiveRecord and uses the builtin MCP feature to connect to GitHub. All
-answers are grounded in the llm.rb source code.
+ActiveRecord and uses the builtin MCP feature to connect to GitHub. The chatbot
+is an llm.rb agent that is deployed with [roda-llm](https://github.com/r-uby-dev/roda-llm#readme).
 
 ## Install
 
@@ -37,9 +37,22 @@ gem install llm.rb
 The
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
 class is the default high-level interface,
-and it is recommended for most use-cases. It manages tool execution
-automatically and guards against infinite loops,
-manages conversation state, and much more.
+and it is recommended for most use-cases. It manages the tool loop
+and provides configurable features on top of it. For example you can
+manage the tool loop with a retry budget alongside a tool call budget,
+among other features.
+
+The runtime is designed to keep the tool loop alive and it will
+avoid exceptions. When an error is encountered in a tool or during
+the lifecycle of an agent it is almost always reported back to the
+model as an in-band error that allows the model to correct course.
+
+A lot of care also goes into keeping the tool loop from entering
+an invalid state that would lead to API-level errors. For example,
+when a tool call is interrupted it could leave an unanswered tool
+call that a model will reject on the next turn. The runtime takes
+care of this by pruning orphaned tool calls and ensuring that the
+tool loop always remains valid. 
 
 ```ruby
 require "llm"
@@ -249,12 +262,12 @@ end
 <br>
 
 The [LLM::Agent#console](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#console-instance_method)
-method drops you into a highly capable interactive console
-that is built on top of curses. It can help you debug agents,
-test your tools, connect to MCP servers, and even A2A agents.
-The console stands out because it connects to the surrounding
-runtime and it can be extended by your code. Think of it as
-`binding.irb` but for agents.
+method drops you into an interactive console that is built on
+top of curses. It can help you debug agents, test your tools,
+connect to MCP servers, and other A2A agents. The console stands
+out because it connects to the surrounding runtime and it can
+be extended by your code. Think of it as `binding.irb` but
+for agents.
 
 ##### Demo
 

data/lib/llm/active_record/acts_as_agent.rb

@@ -37,7 +37,7 @@ module LLM::ActiveRecord
       #     acts_as_agent
       #     set instructions: "You are a system administrator",
       #         model: "gpt-4.1-nano",
-      #         tools: [Shell]
+      #         tools: [LLM::Tool::Exec]
       #   end
       #
       # @param [Hash] properties

data/lib/llm/agent.rb

@@ -22,9 +22,12 @@ module LLM
   #   tool-call patterns and blocks stuck execution before more tool work is
   #   queued.
   # * The tool loop can be bounded with `tool_budget`. Once the budget is
-  #   spent, the agent sends an in-band advisory message back through the
-  #   model and keeps the loop in-band. By default no budget is set
-  #   (`nil`), so the feature is disabled.
+  #   spent, no further tool calls are run for that turn: the agent sends an
+  #   in-band advisory message back through the model instead, and keeps
+  #   sending it while the model keeps asking for tools. The budget counts
+  #   tool calls, so a batch of calls is spent as a batch, and a batch that
+  #   would take the turn past its budget is not run at all. By default no
+  #   budget is set (`nil`), so the feature is disabled.
   # * Tool loop execution can be configured with `concurrency :sequential`,
   #   `:thread`, `:async`, `:fiber`, `:fork`, or `:ractor`.
   #
@@ -32,7 +35,7 @@ module LLM
   #   class SystemAdmin < LLM::Agent
   #     set model: "gpt-4.1-nano",
   #         instructions: "You are a Linux system admin",
-  #         tools: [Shell],
+  #         tools: [LLM::Tool::Exec],
   #         schema: Result
   #   end
   #
@@ -99,7 +102,7 @@ module LLM
     #     set name: "admin",
     #         instructions: "You are a system administrator",
     #         model: "gpt-4.1-nano",
-    #         tools: [Shell, ReadFile]
+    #         tools: [LLM::Tool::Exec, LLM::Tool::ReadFile]
     #   end
     #
     # @param [Hash] properties
@@ -345,10 +348,15 @@ module LLM
     ##
     # Set or get the maximum number of tool calls
     # that are allowed in a single turn. Once the
-    # budget is spent, we will return an in-band
-    # message that informs the model it has spent
-    # its tool call budget - and usually a model
-    # will change course afterwards.
+    # budget is spent, no further tool calls are run:
+    # we return an in-band message that informs the
+    # model it has spent its tool call budget, and
+    # keep returning it while the model keeps asking
+    # for tools - a model will usually change course
+    # afterwards. The budget counts tool calls, so a
+    # batch of calls is spent as a batch, and a batch
+    # that would take the turn past its budget is not
+    # run at all.
     # @note
     #  By default this feature is disabled
     #  (set to `nil`).
@@ -823,14 +831,21 @@ module LLM
         stream = params[:stream] || @ctx.params[:stream]
         params[:stream] = LLM::Stream.try(stream, extra: {concurrency:})
         res = talk.call(apply_instructions(prompt), params)
+        spent = 0
         while @ctx.pending_functions?
-          if max
-            max.times do
-              break unless @ctx.pending_functions?
-              res = talk.call(call_functions, params)
-            end
-            res = talk.call(@ctx.pending_functions.map(&:budget_spent), params) if @ctx.pending_functions?
+          batch = @ctx.pending_functions.size
+          if max and spent + batch > max
+            ##
+            # The budget is spent, so the calls are not run.
+            # They still have to be answered though: each one
+            # gets an in-band return and, just as important,
+            # that return is emitted to the stream.
+            tools = @ctx.pending_functions
+            returns = tools.map(&:budget_spent)
+            @ctx.method(:emit_tool_returns).call(tools, returns)
+            res = talk.call(returns, params)
           else
+            spent += batch if max
             res = talk.call(call_functions, params)
           end
         end

data/lib/llm/function.rb

@@ -294,8 +294,8 @@ class LLM::Function
   #   ctx.talk "I want to run the functions"
   #   ctx.talk ctx.pending_functions.map(&:cancel)
   # @return [LLM::Function::Return]
-  def cancel(reason: "function call cancelled")
-    Return.new(id, name, {cancelled: true, reason:})
+  def cancel(reason: "function call cancelled", **extra)
+    Return.new(id, name, extra.merge(cancelled: true, reason:))
   ensure
     @cancelled = true
   end
@@ -356,12 +356,11 @@ class LLM::Function
   # call budget has been spent.
   # @return [LLM::Function::Return]
   def budget_spent
-    LLM::Function::Return.new(id, name, {
-      error: true,
-      type: "LLM::BudgetSpentError",
-      message: "the tool call budget for this turn has been spent. " \
-               "try to solve the problem with less tool calls."
-    })
+    cancel(
+      reason: "you are making too many tool calls",
+      action: ["stop requesting tool calls"],
+      advice: ["produce a response from tool calls already made", "temporary error that resets on the next turn"]
+    )
   end
 
   ##

data/lib/llm/sequel/agent.rb

@@ -52,7 +52,7 @@ module LLM::Sequel
       #     plugin :llm_agent
       #     set instructions: "You are a system administrator",
       #         model: "gpt-4.1-nano",
-      #         tools: [Shell]
+      #         tools: [LLM::Tool::Exec]
       #   end
       #
       # @param [Hash] properties

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "15.2.0"
+  VERSION = "15.2.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 15.2.0
+  version: 15.2.1
 platform: ruby
 authors:
 - Robert Gleeson