llm.rb 15.3.0 → 15.4.0

data/docs/deepdive/reference/tracer.md

@@ -134,6 +134,82 @@ OpenTelemetry.
 The tracer can also write to a file with the `path:` option to
 [`LLM::Tracer::Logger.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html#initialize-instance_method).
 
+### Hooks
+
+#### Overview
+
+[`LLM::Tracer`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html)
+exposes one method per event in a request's lifecycle. A subclass
+implements the events it cares about and routes them anywhere: a
+logger, a metrics counter, or a database table.
+
+#### How it works
+
+Three hooks cover a provider request. `on_request_start` fires before
+the request is sent and returns the span that `on_request_finish` and
+`on_request_error` receive. Every request carries a `request_id`, a
+UUIDv7 minted when the request begins and passed to all three hooks
+for that request, so a tracer can correlate its events even when a
+turn makes several requests.
+
+Three more hooks cover a local tool call. `on_tool_start` fires before
+the tool runs and returns the span that `on_tool_finish` and
+`on_tool_error` receive.
+
+A turn is additionally bracketed with `start_trace` and `stop_trace`.
+The runtime calls them around every agent turn with a `trace_group_id`,
+and a tracer that supports it (such as
+[`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html))
+uses that id to give every span of the turn the same trace id:
+
+```ruby
+class MyTracer < LLM::Tracer
+  def on_request_start(operation:, model: nil, **)
+    warn "start #{operation} #{model}"
+  end
+
+  def on_request_finish(operation:, res:, **)
+    warn "finish #{operation}"
+  end
+
+  def on_request_error(ex:, **)
+    warn "error #{ex.class}"
+  end
+
+  def on_tool_start(id:, name:, arguments:, model:, **)
+    warn "tool #{name}"
+  end
+
+  def on_tool_finish(result:, **)
+    warn "tool #{result.name} done"
+  end
+
+  def on_tool_error(ex:, **)
+    warn "tool error #{ex.class}"
+  end
+end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+llm.tracer = MyTracer.new(llm)
+agent = LLM::Agent.new(llm)
+agent.talk "Hello"
+```
+
+#### Why would I use it?
+
+The built-in tracers cover logging and OpenTelemetry. A hook lets you
+send the same events somewhere else, and because the built-in tracers
+accept the keywords they do not use, existing tracer code keeps working
+as hooks gain parameters.
+
+#### Notes
+
+The base class raises `NotImplementedError` for any hook it does not
+implement, so a tracer must cover every hook the runtime calls: the six
+request and tool hooks above. Accept `**` to absorb keywords you do not
+read, as the built-in tracers do, so a hook that gains a parameter does
+not break your subclass.
+
 ### PrettyLogger
 
 #### Overview

data/docs/deepdive.md

@@ -44,7 +44,7 @@ useful when you need to go beyond the basics.
 #### Notes
 
 The deepdive is a living document. Sections are added as new
-features land. The [README.md](https://github.com/r-uby-dev/llm#readme)
+features land. The [README.md](https://github.com/r-uby-dev/llm.rb#readme)
 is the best place to start if you are new to llm.rb.
 
 ---

data/lib/llm/active_record/message.rb

@@ -0,0 +1,113 @@
+# frozen_string_literal: true
+
+module LLM::ActiveRecord
+  ##
+  # Represents a message in an agent's memory.
+  #
+  # This class is virtual and never materializes
+  # in the database as a real table. It provides
+  # a SQL view into the messages array stored in
+  # the JSONB column that carries an agent's
+  # runtime state, and it returns relations, so
+  # messages can be filtered and ordered in the
+  # database instead of in memory.
+  #
+  # It expects the agent and this class to share a
+  # connection, which they do when both live on the
+  # same database - the default in Rails.
+  #
+  # @example
+  #   LLM::ActiveRecord::Message.for(agent:)
+  #                             .where(role: "assistant")
+  #                             .count
+  class Message < ActiveRecord::Base
+    ##
+    # The name the derived table is given. It is "created"
+    # on-demand and it is here so that ActiveRecord has
+    # something to select from.
+    self.table_name = "llm_agent_messages"
+
+    ##
+    # The derived table does not exist, so there is no schema
+    # to load. Without this, ActiveRecord asks the database
+    # for the columns of a table that was never created.
+    # @return [void]
+    def self.load_schema!
+      @columns_hash = {}.freeze
+    end
+
+    ##
+    # Build a relation over one agent's messages.
+    #
+    # The table, the column and its type come from the
+    # agent's own class, so this works for any model that
+    # keeps its conversation in a column.
+    #
+    # The base fields the query guarantees are columns -
+    # id, role, content, tools - so they are what a
+    # caller's `where` and `order` are written against.
+    # The whole message is carried along as `data`, for
+    # fields the query does not name yet.
+    #
+    # The derived table is aliased as `llm_agent_messages`
+    # because ActiveRecord qualifies its SELECT with the
+    # class' table name.
+    #
+    # @param [ActiveRecord::Base] agent
+    #  An instance of an ActiveRecord model.
+    # @return [ActiveRecord::Relation]
+    def self.for(agent:)
+      klass      = agent.class
+      connection = klass.connection
+      table      = connection.quote_table_name(klass.table_name)
+      options    = klass.llm_plugin_options
+      column     = connection.quote_column_name(options.fetch(:data_column))
+      from(<<~SQL).where(agent_id: agent&.id)
+        (SELECT #{table}.id                AS agent_id,
+                (message ->> 'id')         AS id,
+                (message ->> 'role')       AS role,
+                (message ->> 'content')    AS content,
+                (message -> 'tools')       AS tools,
+                ordinality                 AS position,
+                message                    AS data
+         FROM #{table},
+              jsonb_array_elements(#{table}.#{column} -> 'messages')
+                WITH ORDINALITY AS each(message, ordinality))
+          AS llm_agent_messages
+      SQL
+    end
+
+    ##
+    # @return (see LLM::Message#tool_call?)
+    def tool_call?
+      unwrap!.tool_call?
+    end
+
+    ##
+    # @return (see LLM::Message#tool_return?)
+    def tool_return?
+      unwrap!.tool_return?
+    end
+
+    ##
+    # The message, as the runtime would hand it back.
+    #
+    # The whole serialized message is used, so fields the
+    # query does not name: usage, reasoning content,
+    # and compaction survive the round trip through the
+    # database.
+    #
+    # @return [LLM::Message]
+    def unwrap!
+      @message ||= begin
+        stored = data.is_a?(Hash) ? data : {}
+        extra = stored.each_with_object({}) { |(key, value), acc| acc[key.to_sym] = value }
+        extra[:id] = stored["id"] || id
+        extra[:role] ||= role
+        extra[:content] ||= content
+        extra[:tool_calls] = stored["tools"] || tools
+        LLM::Message.new(extra[:role], extra[:content], extra)
+      end
+    end
+  end
+end

data/lib/llm/active_record.rb

@@ -74,4 +74,5 @@ module LLM::ActiveRecord
 
   require "llm/active_record/acts_as_llm"
   require "llm/active_record/acts_as_agent"
+  require "llm/active_record/message"
 end

data/lib/llm/agent.rb

@@ -17,10 +17,6 @@ module LLM
   # **Notes:**
   # * Instructions are injected once unless a system message is already present.
   # * An agent automatically executes tool loops (unlike {LLM::Context LLM::Context}).
-  # * The automatic tool loop enables the wrapped context's `guard` by default.
-  #   The built-in {LLM::Guard::Loop LLM::Guard::Loop} detects repeated
-  #   tool-call patterns and blocks stuck execution before more tool work is
-  #   queued.
   # * The tool loop can be bounded with `tool_budget`. Once the budget is
   #   spent, no further tool calls are run for that turn: the agent sends an
   #   in-band advisory message back through the model instead, and keeps
@@ -106,6 +102,8 @@ module LLM
     #   end
     #
     # @param [Hash] properties
+    # @option properties [String, Symbol, Proc] :name
+    # @option properties [String, Symbol, Proc] :description
     # @option properties [String] :instructions
     # @option properties [String] :model
     # @option properties [Array<LLM::Function>] :tools
@@ -138,7 +136,7 @@ module LLM
     # @return [String]
     #  Return's the agents name
     def self.name(name = UNDEFINED, &block)
-      if name.equal?(UNDEFINED)
+      if name.equal?(UNDEFINED) and block.nil?
         if @name.nil?
           name  = to_s.split("::").last
           @name = name.gsub(CASE_PATTERN, "-").downcase
@@ -158,6 +156,11 @@ module LLM
     ##
     # Set or get an agent's description
     # @note
+    #  Reading this on the class returns what was configured - a
+    #  Symbol or Proc included - because an instance is what
+    #  resolves those. Read it on an instance for the description
+    #  itself.
+    # @note
     #  This method serves as a self-documenting string.
     #  It is optional but recommended.
     # @param [String] desc
@@ -165,7 +168,7 @@ module LLM
     # @return [String, nil]
     #  Returns the agent's description
     def self.description(desc = UNDEFINED, &block)
-      if desc.equal?(UNDEFINED)
+      if desc.equal?(UNDEFINED) and block.nil?
         @desc
       else
         @desc = block || desc
@@ -338,10 +341,10 @@ module LLM
     #  The path to a file
     # @return [String, nil]
     def self.path(path = UNDEFINED, &block)
-      if path.equal?(UNDEFINED)
+      if path.equal?(UNDEFINED) and block.nil?
         @path
       else
-        @path = path || block
+        @path = block || path
       end
     end
 
@@ -365,10 +368,10 @@ module LLM
     #  a single turn.
     # @return [Integer, nil]
     def self.tool_budget(budget = UNDEFINED, &block)
-      if budget.equal?(UNDEFINED)
+      if budget.equal?(UNDEFINED) and block.nil?
         @tool_budget
       else
-        @tool_budget = budget || block
+        @tool_budget = block || budget
       end
     end
 
@@ -434,13 +437,11 @@ module LLM
         end
       end
       ##
-      # Alibaba (token plan) will frequently issue rate
-      # limits or time outs that it recovers from. The
-      # higher retry count is to account for scenarios
-      # where it takes longer than expected to recover.
-      retry_budget = llm.name == :alibaba ? 8 : 5
-      params[:retry_budget] = retry_budget if params[:retry_budget].equal?(UNDEFINED)
-      @ctx = LLM::Context.new(llm, {guard: LLM::Guard::Loop}.merge(params))
+      # The provider decides how patient a turn should be: it
+      # knows its own API, and how often it recovers from a
+      # rate limit rather than failing outright.
+      params[:retry_budget] = llm.retry_budget if params[:retry_budget].equal?(UNDEFINED)
+      @ctx = LLM::Context.new(llm, params)
       @path and File.readable?(@path) ? @ctx.restore(path:) : nil
     end
 
@@ -867,8 +868,16 @@ module LLM
         end
         res
       end
-      return run.call unless @tracer
-      @llm.with_tracer(@tracer, &run)
+      ##
+      # One turn, one trace group. A tracer
+      # is told where the turn begins and
+      # where it ends. The trace group ID
+      # identifies the turn.
+      tracer = @tracer || @llm.tracer
+      tracer.start_trace(name: "llm.turn", trace_group_id: SecureRandom.uuid_v7)
+      @llm.with_tracer(tracer, &run)
+    ensure
+      tracer&.stop_trace
     end
 
     ##

data/lib/llm/console/buffer.rb

@@ -72,6 +72,14 @@ class LLM::Console
       @snapshot = nil
     end
 
+    ##
+    # Returns true while the buffer has an active row
+    # for the streaming path to replace.
+    # @return [Boolean]
+    def open?
+      !@snapshot.nil?
+    end
+
     ##
     # @return [void]
     def scroll_up(height)
@@ -131,7 +139,7 @@ class LLM::Console
     #  One or more chunks.
     # @return [void]
     def replace(chunks)
-      @rows = @snapshot.map(&:dup)
+      @rows = @snapshot ? @snapshot.map(&:dup) : @rows
       chunks.each { wrap(_1, @rows) }
     end
 

data/lib/llm/console.rb

@@ -284,6 +284,11 @@ module LLM
           # UI can repaint. Cap the stream chunks drained per call so
           # read! gives control back to the key loop: leftover chunks
           # stay queued and are drained by the next read!.
+          #
+          # A turn that was cancelled can still have chunks queued, and
+          # they arrive after the buffer is closed. There is no active
+          # row to replace at that point, so they are dropped.
+          next unless buffer.open?
           status.text = think_text if stream.tools.empty?
           write_message name, markdown(value), method: :replace
           burst += 1
@@ -292,7 +297,7 @@ module LLM
           self.status = value
         when :done
           status.text = "idle"
-          write_message name, markdown(value), method: :replace
+          write_message(name, markdown(value), method: :replace) if buffer.open?
           buffer.close
           @thread = nil
         when :cancel

data/lib/llm/context/deserializer.rb

@@ -5,7 +5,12 @@ class LLM::Context
   # @api private
   module Deserializer
     ##
-    # Restore a saved context state
+    # Restore a saved context state.
+    #
+    # The `context_used` and `context_window` keys carried by a payload
+    # are projections for queryability. They are deliberately ignored
+    # here: the runtime derives both from the messages and the
+    # registry, so a payload can never seed them.
     # @param [String, nil] path
     #  The path to a JSON file
     # @param [String, nil] string

data/lib/llm/context.rb

@@ -183,10 +183,6 @@ module LLM
     # via {LLM::Stream#on_tool_call}. A blocked call yields its in-band
     # `guard_error` return without executing.
     #
-    # The built-in implementation is {LLM::Guard::Loop LLM::Guard::Loop}, which
-    # detects repeated tool-call patterns and turns them into in-band
-    # `guard_error` tool returns.
-    #
     # @return [Class<LLM::Guard>]
     def guard
       @guard[:klass]
@@ -506,6 +502,12 @@ module LLM
     end
 
     ##
+    # Returns the runtime state as a Hash.
+    #
+    # `context_used` and `context_window` are projections written for
+    # queryability, so that saved state can be inspected at rest
+    # without loading it. They are never read back; the runtime
+    # always derives them from the messages and the registry.
     # @return [Hash]
     def to_h
       {
@@ -513,6 +515,8 @@ module LLM
         id: @id,
         model:,
         compacted:,
+        context_used:,
+        context_window:,
         messages: @messages.map { serialize_message(_1) }
       }
     end

data/lib/llm/guard.rb

@@ -10,16 +10,10 @@ module LLM
   # an {LLM::Function::Return LLM::Function::Return} that closes the
   # pending tool call, or `nil` when execution should continue. The guard
   # is stamped onto the functions the context binds, so it runs whenever a
-  # task is spawned — including tool calls queued from a stream. The
-  # built-in implementation is {LLM::Guard::Loop LLM::Guard::Loop}, which
-  # detects repeated tool-call patterns. {LLM::Guard::Null LLM::Guard::Null}
-  # is a no-op and the default.
-  #
-  # {LLM::Agent LLM::Agent} enables {LLM::Guard::Loop LLM::Guard::Loop}
-  # by default through its wrapped context.
+  # task is spawned — including tool calls queued from a stream.
+  # {LLM::Guard::Null LLM::Guard::Null} is a no-op and the default.
   class Guard
     require_relative "guard/null"
-    require_relative "guard/loop"
 
     ##
     # @return [LLM::Context]

data/lib/llm/provider.rb

@@ -255,6 +255,22 @@ class LLM::Provider
     raise NotImplementedError
   end
 
+  ##
+  # Returns the number of times a rate-limited
+  # request is retried before the error is raised.
+  #
+  # Each retry sleeps a growing interval, so a budget
+  # that is exhausted surfaces the rate-limit error
+  # instead of blocking indefinitely. A provider can
+  # return a different budget when its API recovers
+  # from rate limits more slowly.
+  #
+  # @see LLM::Agent#retry_budget
+  # @return [Integer]
+  def retry_budget
+    5
+  end
+
   ##
   # Returns an object that can generate a JSON schema
   # @return [LLM::Schema]

data/lib/llm/providers/alibaba.rb

@@ -89,5 +89,20 @@ module LLM
     def default_model
       "deepseek-v4-flash-0731"
     end
+
+    ##
+    # Returns the number of times a rate-limited
+    # request is retried.
+    #
+    # Alibaba (token plan) will frequently issue
+    # rate limits or time outs that it recovers from.
+    # The higher retry count is to account for scenarios
+    # where it takes longer than expected to recover.
+    #
+    # @see LLM::Provider#retry_budget
+    # @return [Integer]
+    def retry_budget
+      8
+    end
   end
 end

data/lib/llm/providers/anthropic/error_handler.rb

@@ -21,10 +21,13 @@ class LLM::Anthropic
     #  The span
     # @param [LLM::Transport::Response, Net::HTTPResponse] res
     #  The response from the server
+    # @param [String] request_id
+    #  The id of the request that failed
     # @return [LLM::Anthropic::ErrorHandler]
-    def initialize(tracer, span, res)
+    def initialize(tracer, span, res, request_id)
       @tracer = tracer
       @span = span
+      @request_id = request_id
       @res = LLM::Transport::Response.from(res)
     end
 
@@ -33,7 +36,7 @@ class LLM::Anthropic
     #  Raises a subclass of {LLM::Error LLM::Error}
     def raise_error!
       ex = error
-      @tracer.on_request_error(ex:, span:)
+      @tracer.on_request_error(ex:, span:, request_id: @request_id)
     ensure
       raise(ex) if ex
     end

data/lib/llm/providers/anthropic/files.rb

@@ -38,9 +38,9 @@ class LLM::Anthropic
     def all(**params)
       query = URI.encode_www_form(params)
       req = LLM::Transport::Request.get("/v1/files?#{query}", headers)
-      res, span, tracer = execute(request: req, operation: "request")
+      res, span, tracer, request_id = execute(request: req, operation: "request")
       res = ResponseAdapter.adapt(res, type: :enumerable)
-      tracer.on_request_finish(operation: "request", res:, span:)
+      tracer.on_request_finish(operation: "request", res:, span:, request_id:)
       res
     end
 
@@ -59,9 +59,9 @@ class LLM::Anthropic
       req = LLM::Transport::Request.post("/v1/files", headers)
       req["content-type"] = multi.content_type
       transport.set_body_stream(req, multi.body)
-      res, span, tracer = execute(request: req, operation: "request")
+      res, span, tracer, request_id = execute(request: req, operation: "request")
       res = ResponseAdapter.adapt(res, type: :file)
-      tracer.on_request_finish(operation: "request", res:, span:)
+      tracer.on_request_finish(operation: "request", res:, span:, request_id:)
       res
     end
 
@@ -80,9 +80,9 @@ class LLM::Anthropic
       file_id = file.respond_to?(:id) ? file.id : file
       query = URI.encode_www_form(params)
       req = LLM::Transport::Request.get("/v1/files/#{file_id}?#{query}", headers)
-      res, span, tracer = execute(request: req, operation: "request")
+      res, span, tracer, request_id = execute(request: req, operation: "request")
       res = ResponseAdapter.adapt(res, type: :file)
-      tracer.on_request_finish(operation: "request", res:, span:)
+      tracer.on_request_finish(operation: "request", res:, span:, request_id:)
       res
     end
 
@@ -101,9 +101,9 @@ class LLM::Anthropic
       query = URI.encode_www_form(params)
       file_id = file.respond_to?(:id) ? file.id : file
       req = LLM::Transport::Request.get("/v1/files/#{file_id}?#{query}", headers)
-      res, span, tracer = execute(request: req, operation: "request")
+      res, span, tracer, request_id = execute(request: req, operation: "request")
       res = ResponseAdapter.adapt(res, type: :file)
-      tracer.on_request_finish(operation: "request", res:, span:)
+      tracer.on_request_finish(operation: "request", res:, span:, request_id:)
       res
     end
     alias_method :retrieve_metadata, :get_metadata
@@ -121,9 +121,9 @@ class LLM::Anthropic
     def delete(file:)
       file_id = file.respond_to?(:id) ? file.id : file
       req = LLM::Transport::Request.delete("/v1/files/#{file_id}", headers)
-      res, span, tracer = execute(request: req, operation: "request")
+      res, span, tracer, request_id = execute(request: req, operation: "request")
       res = LLM::Response.new(res)
-      tracer.on_request_finish(operation: "request", res:, span:)
+      tracer.on_request_finish(operation: "request", res:, span:, request_id:)
       res
     end
 
@@ -147,9 +147,9 @@ class LLM::Anthropic
       file_id = file.respond_to?(:id) ? file.id : file
       req = LLM::Transport::Request.get("/v1/files/#{file_id}/content?#{query}", headers)
       io = StringIO.new("".b)
-      res, span, tracer = execute(request: req, operation: "request") { |res| res.read_body { |chunk| io << chunk } }
+      res, span, tracer, request_id = execute(request: req, operation: "request") { |res| res.read_body { |chunk| io << chunk } }
       res = LLM::Response.new(res).tap { _1.define_singleton_method(:file) { io } }
-      tracer.on_request_finish(operation: "request", res:, span:)
+      tracer.on_request_finish(operation: "request", res:, span:, request_id:)
       res
     end
 

data/lib/llm/providers/anthropic/models.rb

@@ -40,9 +40,9 @@ class LLM::Anthropic
     def all(**params)
       query = URI.encode_www_form(params)
       req = LLM::Transport::Request.get("/v1/models?#{query}", headers)
-      res, span, tracer = execute(request: req, operation: "request")
+      res, span, tracer, request_id = execute(request: req, operation: "request")
       res = ResponseAdapter.adapt(res, type: :models)
-      tracer.on_request_finish(operation: "request", res:, span:)
+      tracer.on_request_finish(operation: "request", res:, span:, request_id:)
       res
     end
 

data/lib/llm/providers/anthropic.rb

@@ -52,10 +52,10 @@ module LLM
     def complete(prompt, params = {})
       params, stream, tools, role = normalize_complete_params(params)
       req = build_complete_request(prompt, params, role)
-      res, span, tracer = execute(request: req, stream: stream, operation: "chat", model: params[:model])
+      res, span, tracer, request_id = execute(request: req, stream: stream, operation: "chat", model: params[:model])
       res = ResponseAdapter.adapt(res, type: :completion)
         .extend(Module.new { define_method(:__tools__) { tools } })
-      tracer.on_request_finish(operation: "chat", model: params[:model], res:, span:)
+      tracer.on_request_finish(operation: "chat", model: params[:model], res:, span:, request_id:)
       res
     end
 

data/lib/llm/providers/bedrock/error_handler.rb

@@ -23,9 +23,10 @@ class LLM::Bedrock
     # @param [Object, nil] span
     # @param [LLM::Transport::Response, Net::HTTPResponse] res
     # @return [LLM::Bedrock::ErrorHandler]
-    def initialize(tracer, span, res)
+    def initialize(tracer, span, res, request_id)
       @tracer = tracer
       @span = span
+      @request_id = request_id
       @res = LLM::Transport::Response.from(res)
     end
 
@@ -33,7 +34,7 @@ class LLM::Bedrock
     # @raise [LLM::Error]
     def raise_error!
       ex = error
-      @tracer.on_request_error(ex:, span:)
+      @tracer.on_request_error(ex:, span:, request_id: @request_id)
     ensure
       raise(ex) if ex
     end

data/lib/llm/providers/bedrock/models.rb

@@ -40,9 +40,10 @@ class LLM::Bedrock
     # @return [LLM::Response]
     def all(**params)
       host = credentials.host
+      request_id = SecureRandom.uuid_v7
       req = build_request(host, params)
       res = build_transport(host).request(req, owner: self)
-      handle_response(res)
+      handle_response(res, request_id)
     end
 
     private
@@ -69,9 +70,10 @@ class LLM::Bedrock
 
     ##
     # @param [LLM::Transport::Response, Net::HTTPResponse] res
+    # @param [String] request_id
     # @return [LLM::Response]
     # @raise [LLM::Error]
-    def handle_response(res)
+    def handle_response(res, request_id)
       res = LLM::Transport::Response.from(res)
       if res.success?
         res.body = LLM::Object.from(LLM.json.load(res.body || "{}"))
@@ -79,7 +81,7 @@ class LLM::Bedrock
       else
         body = +""
         res.read_body { body << _1 } if res.body.nil?
-        LLM::Bedrock::ErrorHandler.new(tracer, nil, res).raise_error!
+        LLM::Bedrock::ErrorHandler.new(tracer, nil, res, request_id).raise_error!
       end
     end
 

data/lib/llm/providers/bedrock.rb

@@ -89,10 +89,10 @@ module LLM
       tracer.set_request_metadata(user_input: extract_user_input(messages, fallback: prompt))
       sign!(req, body)
       model_id = model_id_for(req.path)
-      res, span, tracer = execute(request: req, stream:, operation: "chat", stream_parser:, model: model_id)
+      res, span, tracer, request_id = execute(request: req, stream:, operation: "chat", stream_parser:, model: model_id)
       res = ResponseAdapter.adapt(res, type: :completion)
         .extend(Module.new { define_method(:__tools__) { tools } })
-      tracer.on_request_finish(operation: "chat", model: model_id, res:, span:)
+      tracer.on_request_finish(operation: "chat", model: model_id, res:, span:, request_id:)
       res
     end