llm.rb 16.0.0 → 17.0.1

data/docs/deepdive/reference/cost.md

@@ -100,7 +100,7 @@ at the end of a session to keep a spend trail.
 [`LLM::Cost#to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_s-instance_method)
 returns the total in a compact, human-friendly format, rounded to
 two decimals (`"0.01"`). Components that were not used are `0`, so
-you never need to guard against `nil` when aggregating.
+you never need to check for `nil` when aggregating.
 
 The console renders context usage as a proportion, not a cost.
 [`LLM::Context#context_usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#context_usage-instance_method)

data/docs/deepdive/reference/response.md

@@ -16,7 +16,7 @@ their own.
 
 `LLM::Response` carries the transport response and a little common
 state: `#res` (the underlying HTTP response), `#body` (the parsed body,
-or a raw string), `#ok?`, `#id`, and `#file?`. A name the base class
+or a raw string), `#ok?`, `#id`, and `#file?`. A name the superclass
 does not define falls through to the body, so a provider field such as
 `content` is reachable directly.
 

data/docs/deepdive/reference/tracer.md

@@ -134,6 +134,60 @@ OpenTelemetry.
 The tracer can also write to a file with the `path:` option to
 [`LLM::Tracer::Logger.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html#initialize-instance_method).
 
+### Multi
+
+#### Overview
+
+[`LLM::Tracer::Multi`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Multi.html)
+fans every tracing callback out to a set of tracers, so one request can
+be logged to a file and exported to OpenTelemetry at the same time
+rather than choosing between them.
+
+#### How it works
+
+[`LLM::Tracer.multi`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#multi-class_method)
+builds a `Multi` from a provider and the tracers to fan out to. The
+provider comes first and is required, so a tracer that is given as a class
+is built with it, while one that is given as an instance is used as it is:
+
+```ruby
+require "llm"
+
+llm = LLM.deepseek(key: ENV["KEY"])
+llm.tracer = LLM::Tracer.multi(
+  llm,
+  LLM::Tracer::Logger,
+  LLM::Tracer.telemetry(llm)
+)
+agent = LLM::Agent.new(llm)
+agent.talk "Hello"
+```
+
+`LLM::Tracer::Multi.new` takes the tracers themselves, so one can also be
+built by hand when a tracer has to be constructed some other way.
+
+`on_request_start` and `on_tool_start` return an array with one span per
+tracer, in the same order as
+[`LLM::Tracer::Multi#tracers`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Multi.html#tracers-instance_method).
+Pass that array back as `span:` to the matching finish, error, or
+interrupt callback and each tracer is handed the span it returned. A
+tracer that raises is reported the way
+[`LLM::Tracer::Rescue`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Rescue.html)
+reports any other, and the remaining tracers still run.
+
+#### Why would I use it?
+
+It is how you keep a human-readable log and a machine-readable trace from
+diverging: one tracer sees the events in development, another sees them
+in production, and neither has to know about the other.
+
+#### Notes
+
+A `Multi` holds no provider, so `spans` and `flush!` are its own rather
+than any child's: the callbacks are fanned out, but a span list or a
+flush is asked of the tracer that owns it. `on_exit` is fanned out like
+the rest, so every tracer it holds is told when the last scope ends.
+
 ### Hooks
 
 #### Overview
@@ -153,8 +207,13 @@ for that request, so a tracer can correlate its events even when a
 turn makes several requests.
 
 Three more hooks cover a local tool call. `on_tool_start` fires before
-the tool runs and returns the span that `on_tool_finish` and
-`on_tool_error` receive.
+the tool runs and returns the span that `on_tool_finish`,
+`on_tool_error`, and `on_tool_interrupt` receive. Each ending is also
+handed a `started_at:` and a `finished_at:`, the monotonic clock times
+the call opened and ended at. The endings are delivered in the order
+the group holds its calls, so a call that finished first can still be
+announced after one that finished later. Those two times, and not the
+moment the callback runs, are what say how long a call took.
 
 An interrupt is neither a finish nor an error, and it is reported to a
 hook of its own. `on_interrupt` is called with a `scope:` of `:request`,
@@ -167,6 +226,12 @@ between a turn's requests, where there was nothing more precise to
 interrupt. A cancel reaches every tool that is running, so the tool phase
 is announced once rather than once per tool.
 
+A tool call that was interrupted is closed through `on_tool_interrupt`
+as well, which is handed the exception the caller is given and the span
+`on_tool_start` returned. That span is the only thing that can name a
+call which never returned a result, so a tracer can pair a start with an
+end rather than leaving the span open.
+
 A tracer's own lifetime is bracketed as well. `on_exit` fires once,
 when the last scope that is open for that tracer ends. That scope can
 belong to a different thread than the one that opened the first: a tool
@@ -184,7 +249,7 @@ and a tracer that supports it (such as
 uses that id to give every span of the turn the same trace id:
 
 ```ruby
-class MyTracer < LLM::Tracer
+class MyTracer < LLM::Tracer::Null
   def on_request_start(operation:, model: nil, **)
     warn "start #{operation} #{model}"
   end
@@ -193,20 +258,12 @@ class MyTracer < LLM::Tracer
     warn "finish #{operation}"
   end
 
-  def on_request_error(ex:, **)
-    warn "error #{ex.class}"
-  end
-
   def on_tool_start(id:, name:, arguments:, model:, **)
     warn "tool #{name}"
   end
 
-  def on_tool_finish(result:, **)
-    warn "tool #{result.name} done"
-  end
-
-  def on_tool_error(ex:, **)
-    warn "tool error #{ex.class}"
+  def on_tool_interrupt(ex:, span:, **)
+    warn "tool interrupted: #{ex.class}"
   end
 end
 
@@ -225,15 +282,26 @@ as hooks gain parameters.
 
 #### Notes
 
-The base class raises `NotImplementedError` for the six request and tool
-hooks above, so a tracer has to answer for each of them. The remaining
-hooks do nothing by default. `on_exit` does nothing because most tracers
-hold nothing that needs releasing, and `start_trace` and `stop_trace`
-are no-ops too. `on_interrupt` does nothing because an interrupt is
-delivered to whatever tracer happens to be bound, and a hook that raised
-would replace the interrupt every caller is written against. Accept `**`
-to absorb keywords you do not read, as the built-in tracers do, so a
-hook that gains a parameter does not break your subclass.
+The superclass raises `NotImplementedError` for every hook the runtime
+calls, so subclassing
+[`LLM::Tracer`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html)
+means answering all of them: the six request and tool hooks above,
+`on_tool_interrupt`, `on_interrupt`, and `on_exit`.
+[`LLM::Tracer::Null`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Null.html)
+answers them all with a no-op, which is the shorter route when you want
+only a few, and what the example above takes. `start_trace` and
+`stop_trace` are the exception: their base implementations are no-ops
+rather than raises, because a tracer that keeps no trace has nothing to
+open or close.
+
+A hook that raises does not take the turn with it.
+[`LLM::Tracer::Rescue`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Rescue.html)
+is prepended onto every subclass of `LLM::Tracer`, so the raise is
+reported on standard error and the turn carries on. `LLM::Interrupt` is
+re-raised, since a cancel is not a tracer's to swallow.
+
+Accept `**` to absorb keywords you do not read, as the built-in tracers
+do, so a hook that gains a parameter does not break your subclass.
 
 ### PrettyLogger
 

data/docs/deepdive.md

@@ -81,7 +81,7 @@ is the best place to start if you are new to llm.rb.
 - [Cancellation](deepdive/advanced/cancellation.md)
 - [Transports](deepdive/advanced/transports.md)
 - [Transformer](deepdive/advanced/transformer.md)
-- [Guard](deepdive/advanced/guard.md)
+- [Intercept](deepdive/advanced/intercept.md)
 
 ## Media
 

data/lib/llm/agent/interrupt.rb

@@ -2,63 +2,53 @@
 
 class LLM::Agent
   ##
-  # What a turn's caller answers to.
+  # Stop an agent in its tracks.
   #
-  # A turn is a loop, and between two of its requests there is nothing in
-  # flight to close and nothing running to raise into. {LLM::Agent#run_loop}
-  # records where it is running - the thread, the fiber, the scheduler that
-  # fiber belongs to, and the tracer the turn is traced with - and extends
-  # this onto that record, so an interrupt with nothing more precise to do
-  # has somewhere to land: {LLM::Context#interrupt!} calls `interrupt!` on
-  # the caller it holds, and what the caller does with the names is its own
-  # business.
+  # The {LLM::Agent::Interrupt} module extends
+  # the instance variable `@caller` (an instance
+  # of {LLM::Object}) that belongs to an agent
+  # and its context.
+  #
+  # The `@caller` instance variable captures the
+  # agent's thread, fiber, and scheduler. The
+  # variable is in scope for as long as an agent
+  # is running and it provides other threads and
+  # fibers with access to the agent's thread,
+  # fiber and scheduler.
+  #
+  # A different thread or fiber can use this module
+  # and the object it extends (`@caller`) to stop
+  # an agent in its tracks via the raise of
+  # {LLM::Interrupt LLM::Interrupt} on the
+  # agent's thread, or fiber.
+  #
+  # @note
+  #  The method provided by this module is
+  #  not called directly, and the `@caller`
+  #  instance variable is private. The
+  #  {LLM::Agent#interrupt!} and {LLM::Context#interrupt!}
+  #  methods call it on your behalf.
   #
-  # It is public rather than private because the runtime is not the only
-  # thing that runs a loop: an application with a loop of its own can record
-  # a caller the same way, and a spec can build one without running a turn.
   # @see LLM::Agent#run_loop
   # @see LLM::Context#interrupt!
   module Interrupt
     ##
-    # Ends the turn where it is, and tells the tracer first.
-    #
-    # Which name receives the raise is decided by who is cancelling:
-    #
-    #   another thread  the thread, because a fiber belongs to the thread
-    #                   that made it and cannot be entered from another
-    #                   one. The raise lands in whichever fiber that
-    #                   thread is running, which is the turn's, because
-    #                   the turn is what it is doing.
-    #
-    #   the same thread the fiber, because raising on the thread would
-    #                   raise in the canceller that asked for it.
+    # Stops an agent in its tracks.
     #
-    # A fiber scheduler is the case the second rule is for: a turn under
-    # Falcon or Async runs on a fiber of the reactor's thread, and a cancel
-    # that arrives on that thread is another fiber asking. Such a fiber is
-    # asked for through the scheduler, the way
-    # {LLM::Function::Fiber::Task#interrupt!} asks, because a direct raise
-    # into a scheduled fiber does not transfer: it suspends the thread that
-    # raises, and that thread is the canceller's. A fiber with no scheduler
-    # behind it - a turn an application ran in a fiber of its own - is raised
-    # into directly, which is what such a fiber is for.
+    # When this method is called from
+    # the agent's thread but a different
+    # fiber an interrupt is raised on the
+    # caller's fiber.
     #
-    # The tracer is told `scope: :agent`, before the raise, the way
-    # {LLM::Context#wait} tells it `scope: :tool` before the caller is given
-    # that interrupt. A tracer that has to account for every interrupt a turn
-    # receives needs this one too, and the alternative - announcing it from
-    # the turn as it unwinds - is not available: between two requests there
-    # is no frame of the turn's own to announce it from.
+    # And when it is called from a thread
+    # other than the agent's thread, the
+    # interrupt is raised on the caller's
+    # thread.
     #
-    # Nothing is raised, and nothing is announced, when there is no fiber to
-    # raise into - and nothing is raised when the turn ended between the read
-    # and the raise, which is the ordinary race, because a cancel that
-    # arrives after a turn is not a failure.
-    # @return [nil]
+    # @return [void]
     def interrupt!
       if thread.equal?(Thread.current)
         if fiber
-          tracer&.on_interrupt(scope: :agent)
           if scheduler
             scheduler.fiber_interrupt(fiber, LLM::Interrupt.new("agent interrupted"))
           else
@@ -66,7 +56,6 @@ class LLM::Agent
           end
         end
       else
-        tracer&.on_interrupt(scope: :agent)
         thread.raise(LLM::Interrupt, "agent interrupted")
       end
       nil

data/lib/llm/agent/registry.rb

@@ -0,0 +1,106 @@
+# frozen_string_literal: true
+
+class LLM::Agent
+  ##
+  # {LLM::Agent::Registry LLM::Agent::Registry} holds the agents that are
+  # running a turn in this process, so that a cancel can find one.
+  #
+  # Delivery is not the problem it solves. `LLM::Agent#interrupt!` can be
+  # called from any thread, and {LLM::Agent::Interrupt LLM::Agent::Interrupt}
+  # decides what receives the raise - but a canceller has to hold the agent,
+  # and in an application the agent is built inside the job that runs the
+  # turn and stays a local variable for as long as the turn lasts. A cancel
+  # button has nothing to reach.
+  #
+  # So a turn registers itself for as long as it runs, and a canceller names
+  # it by the agent, or by the id it already has: the record's id when the
+  # agent was built around one, the agent's own otherwise.
+  #
+  # ## What it does not promise
+  #
+  # It is **one process**, and it holds the agents *that process* is running.
+  # A cancel that lands elsewhere - a second worker, a deploy that leaves two
+  # up for a minute - finds nothing, and `false` must not be read as "the turn
+  # is over". An application that needs a guarantee keeps one of its own; this
+  # is the fast path.
+  #
+  # @api private
+  class Registry
+    ##
+    # @return [LLM::Agent::Registry]
+    def initialize
+      @agents = {}
+      @mutex  = Mutex.new
+    end
+
+    ##
+    # Records that an agent is running a turn.
+    #
+    # Strongly, and that is the point: an entry has to reach an agent that
+    # is blocked inside `talk`, so a weak reference would let a turn that is
+    # running be collected. The lifetime is the caller's `ensure`.
+    # @param [LLM::Agent] agent
+    # @return [LLM::Agent]
+    #  Returns the agent
+    def enter(agent)
+      @mutex.synchronize { @agents[key(agent)] = agent }
+      agent
+    end
+
+    ##
+    # Records that an agent's turn is over.
+    #
+    # Removed only if it is still the agent that registered. Two turns under
+    # one identity - a second attempt alongside one that is finishing - are
+    # not the same turn, and the one that finishes first must not take the
+    # other one's registration with it.
+    # @param [LLM::Agent] agent
+    # @return [void]
+    def exit(agent)
+      @mutex.synchronize do
+        id = key(agent)
+        @agents.delete(id) if @agents[id].equal?(agent)
+      end
+    end
+
+    ##
+    # The agent running a turn under this identity, or nil.
+    #
+    # Named by the agent or by the id, and not by both: the two are the same
+    # question asked with different things in hand. An id is compared with
+    # `==`, so a host may pass a string, a number, or whatever it holds that
+    # equals the id a turn registered under.
+    #
+    # Looked up under the lock and answered outside it, because the interrupt
+    # that follows is not this class's to make: a caller holding the lock
+    # while raising into another thread's turn would be holding it against
+    # that turn's own `exit`.
+    # @param [LLM::Agent, nil] agent
+    # @param [String, Integer, nil] id
+    # @raise [ArgumentError]
+    #  When neither is given, or when both are
+    # @return [LLM::Agent, nil]
+    def find(agent: nil, id: nil)
+      if (agent.nil? and id.nil?) or (!agent.nil? and !id.nil?)
+        raise ArgumentError, "provide an agent or an id, and not both"
+      else
+        @mutex.synchronize { @agents[agent ? key(agent) : id] }
+      end
+    end
+
+    private
+
+    ##
+    # The identity an agent registers under.
+    #
+    # The record's id when the agent was built around a record, because that
+    # is what a host already has - a row it can name from a route - and the
+    # agent's own otherwise. One value, not two: an agent with a record
+    # answers to the record's id and not also to its own.
+    # @param [LLM::Agent] agent
+    # @return [String, Integer]
+    def key(agent)
+      agent.record&.id || agent.id
+    end
+  end
+end