llm.rb 5.3.0 → 6.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 39e1632fb63f83a65c5a146ea2a2f4178d0d99d26d2a347f36d360c09ea9845d
-  data.tar.gz: 04b2236d5cac243cc496b686d8d8a5097676e7bcd6973bacfa8d1f7e8d48e270
+  metadata.gz: 8cc16b548be77e0a2bb78c0e130b14f0ac8fdabb2c2f082dd2824282e5a5733b
+  data.tar.gz: 668655ba6a7d65d44b53cc1b8b33afddf3563dc216df83181fb2b7394a2847ff
 SHA512:
-  metadata.gz: 1b4a68bd3b3e109a00f996f520296405ad6066b4f17c1e59da4077c2023c5fe0c95e770b9bd563a531748a16d79355f8db5fb2dcd66ac42673698ddcbea07704
-  data.tar.gz: 12713c07834164f3d13d01613126488cc19e937b8f127fcdbf839d8c75f2f6b77c6207e7e3e6f515d996e35aedb6d6e7333ba563a3f4578f4c33f454d41c6088
+  metadata.gz: 2257aeec49a43c56bfc974e3fc190a850ea27c98c437c7c242efe6c645c000eebdbe2e731fcc0b287303690c1055768f2de32be6ac900391c5156260da9c2ce5
+  data.tar.gz: 8f4f8f3475ac1bd2d9ffbd8c0b413b46224936459eb0ed6bb395876b994c1ab67ec02cf670176f90b8d9b892b59ab4b67271f4b69c80fa7c094aa89310ab5c58

data/CHANGELOG.md

@@ -1,9 +1,70 @@
 # Changelog
 
-## Unreleased
+## v6.0.0
+
+Changes since `v5.4.0`.
+
+This release simplifies the ORM persistence contract around serialized
+`data` state, removing the assumption of reserved `provider`, `model`, and
+usage columns. Provider selection must now come from `provider:` hooks,
+model defaults come from `context:` or agent DSL, and usage is read from the
+serialized runtime state. Alongside this breaking change, Sequel JSON and
+JSONB persistence is fixed, ractor-backed tools now fire tracer callbacks,
+and `LLM::RactorError` is raised for unsupported ractor tool work.
+
+### Change
+
+* **Simplify ORM persistence to serialized `data` state** <br>
+  Change the built-in ActiveRecord and Sequel wrappers to treat serialized
+  `data` as the persistence contract, instead of assuming reserved
+  `provider`, `model`, and usage columns. Provider selection must now come
+  from `provider:` hooks that resolve a real `LLM::Provider` instance, model
+  defaults come from `context:` or agent DSL, and `usage` is read from the
+  serialized runtime state.
+
+### Fix
+
+* **Fix Sequel JSON and JSONB persistence** <br>
+  Load Sequel PostgreSQL JSON support when `plugin :llm` is configured with
+  `format: :json` or `:jsonb`, and wrap structured payloads correctly so
+  persisted context state can be stored in PostgreSQL JSON columns.
+
+* **Trace ractor-backed tool callbacks** <br>
+  Make tool tracers fire `on_tool_start` and `on_tool_finish` for
+  class-based `:ractor` execution too, so ractor-backed tool calls show up
+  in tracer callbacks like the other concurrent tool paths.
+
+* **Raise `LLM::RactorError` for unsupported ractor tool work** <br>
+  Add `LLM::RactorError` and fail fast when `:ractor` execution is requested
+  for unsupported tool types such as skill-backed tools, instead of letting
+  deeper Ruby isolation errors leak out later in execution.
+
+## v5.4.0
 
 Changes since `v5.3.0`.
 
+This release expands tracer support around agentic execution. It lets
+`LLM::Agent` define scoped tracers through the agent DSL and fixes concurrent
+tool execution so those scoped tracers stay attached when work crosses
+thread, task, fiber, and skill boundaries.
+
+### Change
+
+* **Add agent-scoped tracers** <br>
+  Let `LLM::Agent` classes define `tracer ...` or `tracer { ... }` so an
+  agent can carry its own tracer without replacing the provider's default
+  tracer. The resolved tracer is scoped to that agent's turns, tool loops,
+  and pending tool access. Available through the `acts_as_agent` and Sequel
+  agent plugin `tracer` DSL too.
+
+### Fix
+
+* **Preserve scoped tracers across concurrent tool work** <br>
+  Keep agent- and request-scoped tracers attached when tool execution
+  crosses `:thread`, `:task`, or `:fiber` boundaries, including skill
+  execution, so spawned work does not fall back to the provider default
+  tracer.
+
 ## v5.3.0
 
 Changes since `v5.2.1`.

data/README.md

@@ -4,7 +4,7 @@
 <p align="center">
   <a href="https://0x1eef.github.io/x/llm.rb?rebuild=1"><img src="https://img.shields.io/badge/docs-0x1eef.github.io-blue.svg" alt="RubyDoc"></a>
   <a href="https://opensource.org/license/0bsd"><img src="https://img.shields.io/badge/License-0BSD-orange.svg?" alt="License"></a>
-  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-5.3.0-green.svg?" alt="Version"></a>
+  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-6.0.0-green.svg?" alt="Version"></a>
 </p>
 
 ## About
@@ -25,7 +25,6 @@ schemas, files, and persisted state, so real systems can be built out of one coh
 execution model instead of a pile of adapters.
 
 Want to see some code? Jump to [the examples](#examples) section. <br>
-Want to see an agentic framework built on top of llm.rb? Check out [general-intelligence-systems/brute](https://github.com/general-intelligence-systems/brute). <br>
 Want to see a self-hosted LLM environment built on llm.rb? Check out [Relay](https://github.com/llmrb/relay).
 
 ## Architecture
@@ -87,6 +86,7 @@ Review the release state, summarize what changed, and prepare the release.
 class Agent < LLM::Agent
   model "gpt-5.4-mini"
   skills "./skills/release"
+  tracer { LLM::Tracer::Logger.new(llm, path: "logs/release-agent.log") }
 end
 
 llm = LLM.openai(key: ENV["KEY"])
@@ -101,20 +101,26 @@ separate agent table or a second persistence layer.
 
 `acts_as_agent` extends a model with agent capabilities: the same runtime
 surface as [`LLM::Agent`](https://0x1eef.github.io/x/llm.rb/LLM/Agent.html),
-because it actually wraps an `LLM::Agent`, plus persistence through a text,
-JSON, or JSONB-backed column on the same table.
+because it actually wraps an `LLM::Agent`, plus persistence through one text,
+JSON, or JSONB-backed `data` column on the same table. If your app also has
+provider or model columns, provide them to llm.rb through `set_provider` and
+`set_context`.
 
 
 ```ruby
 class Ticket < ApplicationRecord
-  acts_as_agent provider: :set_provider
+  acts_as_agent provider: :set_provider, context: :set_context
   model "gpt-5.4-mini"
   instructions "You are a support assistant."
 
   private
 
   def set_provider
-    { key: ENV["#{provider.upcase}_SECRET"], persistent: true }
+    LLM.openai(key: ENV["OPENAI_SECRET"])
+  end
+
+  def set_context
+    { mode: :responses, store: false }
   end
 end
 ```
@@ -302,7 +308,7 @@ finer sequential control across several steps before shutting the client down.
 ```ruby
 mcp = LLM::MCP.http(
   url: "https://api.githubcopilot.com/mcp/",
-  headers: {"Authorization" => "Bearer #{ENV.fetch("GITHUB_PAT")}"}
+  headers: {"Authorization" => "Bearer #{ENV["GITHUB_PAT"]}"}
 ).persistent
 mcp.run do
   ctx = LLM::Context.new(llm, tools: mcp.tools)
@@ -375,7 +381,8 @@ worker.join
   Use threads, fibers, async tasks, or experimental ractors without
   rewriting your tool layer. The current `:ractor` mode is for class-based
   tools and does not support MCP tools, but mixed workloads can branch on
-  `tool.mcp?` and choose a supported strategy per tool. `:ractor` is
+  `tool.mcp?` and choose a supported strategy per tool. Class-based
+  `:ractor` tools still emit normal tool tracer callbacks. `:ractor` is
   especially useful for CPU-bound tools, while `:task`, `:fiber`, or
   `:thread` may be a better fit for I/O-bound work.
 - **Advanced workloads are built in, not bolted on** <br>
@@ -566,6 +573,7 @@ class Agent < LLM::Agent
   model "gpt-5.4-mini"
   instructions "You are a concise release assistant."
   skills "./skills/release", "./skills/review"
+  tracer { LLM::Tracer::Logger.new(llm, path: "logs/release-agent.log") }
 end
 
 llm = LLM.openai(key: ENV["KEY"])
@@ -694,7 +702,7 @@ worker.join
 
 #### Sequel (ORM)
 
-The `plugin :llm` integration wraps [`LLM::Context`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html) on a `Sequel::Model` and keeps tool execution explicit. <br> See the [deepdive (web)](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) or [deepdive (markdown)](resources/deepdive.md) for more examples.
+The `plugin :llm` integration wraps [`LLM::Context`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html) on a `Sequel::Model` and keeps tool execution explicit. Like the ActiveRecord wrappers, its built-in persistence contract is the serialized `data` column, while `provider:` resolves a real `LLM::Provider` instance and `context:` injects defaults such as `model:`. <br> See the [deepdive (web)](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) or [deepdive (markdown)](resources/deepdive.md) for more examples.
 
 ```ruby
 require "llm"
@@ -703,10 +711,20 @@ require "sequel"
 require "sequel/plugins/llm"
 
 class Context < Sequel::Model
-  plugin :llm, provider: -> { { key: ENV["#{provider.upcase}_SECRET"], persistent: true } }
+  plugin :llm, provider: :set_provider, context: :set_context
+
+  private
+
+  def set_provider
+    LLM.openai(key: ENV["OPENAI_SECRET"])
+  end
+
+  def set_context
+    {model: "gpt-5.4-mini", mode: :responses, store: false}
+  end
 end
 
-ctx = Context.create(provider: "openai", model: "gpt-5.4-mini")
+ctx = Context.create
 ctx.talk("Remember that my favorite language is Ruby")
 puts ctx.talk("What is my favorite language?").content
 ```
@@ -714,36 +732,76 @@ puts ctx.talk("What is my favorite language?").content
 #### ActiveRecord (ORM): acts_as_llm
 
 The `acts_as_llm` method wraps [`LLM::Context`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html) and
-provides full control over tool execution. <br> See the [deepdive (web)](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) or [deepdive (markdown)](resources/deepdive.md) for more examples.
+provides full control over tool execution. Its built-in persistence contract is
+one serialized `data` column. If your app has provider, model, or usage
+columns, provide them to llm.rb through `provider:` and `context:` instead of
+relying on reserved wrapper columns.
+
+See the [deepdive (web)](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) or [deepdive (markdown)](resources/deepdive.md) for more examples.
 
 ```ruby
 require "llm"
-require "net/http/persistent"
 require "active_record"
 require "llm/active_record"
 
 class Context < ApplicationRecord
-  acts_as_llm provider: -> { { key: ENV["#{provider.upcase}_SECRET"], persistent: true } }
+  acts_as_llm provider: :set_provider, context: :set_context
+
+  private
+
+  def set_provider
+    LLM.openai(key: ENV["OPENAI_SECRET"])
+  end
+
+  def set_context
+    {model: "gpt-5.4-mini", mode: :responses, store: false}
+  end
 end
 
-ctx = Context.create!(provider: "openai", model: "gpt-5.4-mini")
+ctx = Context.create!
 ctx.talk("Remember that my favorite language is Ruby")
 puts ctx.talk("What is my favorite language?").content
 ```
 
+```ruby
+require "llm"
+require "active_record"
+require "llm/active_record"
+
+class Context < ApplicationRecord
+  acts_as_llm provider: :set_provider, context: :set_context
+
+  # Optional application columns can still provide the provider and context.
+  # For example, `provider_name` and `model_name` can be normal columns.
+
+  private
+
+  def set_provider
+    LLM.public_send(provider_name, key: provider_key)
+  end
+
+  def set_context
+    {model: model_name, mode: :responses, store: false}
+  end
+end
+```
+
 #### ActiveRecord (ORM): acts_as_agent
 
 The `acts_as_agent` method wraps [`LLM::Agent`](https://0x1eef.github.io/x/llm.rb/LLM/Agent.html) and
-manages tool execution for you. <br> See the [deepdive (web)](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) or [deepdive (markdown)](resources/deepdive.md) for more examples.
+manages tool execution for you. Like `acts_as_llm`, its built-in persistence
+contract is one serialized `data` column. If your app has provider or model
+columns, provide them to llm.rb through your hooks and agent DSL.
+
+See the [deepdive (web)](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) or [deepdive (markdown)](resources/deepdive.md) for more examples.
 
 ```ruby
 require "llm"
-require "net/http/persistent"
 require "active_record"
 require "llm/active_record"
 
 class Ticket < ApplicationRecord
-  acts_as_agent provider: :set_provider
+  acts_as_agent provider: :set_provider, context: :set_context
   model "gpt-5.4-mini"
   instructions "You are a concise support assistant."
   tools SearchDocs, Escalate
@@ -752,14 +810,40 @@ class Ticket < ApplicationRecord
   private
 
   def set_provider
-    { key: ENV["#{provider.upcase}_SECRET"], persistent: true }
+    LLM.openai(key: ENV["OPENAI_SECRET"])
+  end
+
+  def set_context
+    {mode: :responses, store: false}
   end
 end
 
-ticket = Ticket.create!(provider: "openai", model: "gpt-5.4-mini")
+ticket = Ticket.create!
 puts ticket.talk("How do I rotate my API key?").content
 ```
 
+```ruby
+require "llm"
+require "active_record"
+require "llm/active_record"
+
+class Ticket < ApplicationRecord
+  acts_as_agent provider: :set_provider, context: :set_context
+  model "gpt-5.4-mini"
+  instructions "You are a concise support assistant."
+
+  private
+
+  def set_provider
+    LLM.public_send(provider_name, key: provider_key)
+  end
+
+  def set_context
+    {mode: :responses, store: false}
+  end
+end
+```
+
 #### MCP
 
 This example uses [`LLM::MCP`](https://0x1eef.github.io/x/llm.rb/LLM/MCP.html) over HTTP so remote GitHub MCP tools run through the same `LLM::Context` tool path as local tools. It expects a GitHub token in `ENV["GITHUB_PAT"]`. See the [deepdive (web)](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) or [deepdive (markdown)](resources/deepdive.md) for more examples.
@@ -771,7 +855,7 @@ require "net/http/persistent"
 llm = LLM.openai(key: ENV["KEY"])
 mcp = LLM::MCP.http(
   url: "https://api.githubcopilot.com/mcp/",
-  headers: {"Authorization" => "Bearer #{ENV.fetch("GITHUB_PAT")}"}
+  headers: {"Authorization" => "Bearer #{ENV["GITHUB_PAT"]}"}
 ).persistent
 
 mcp.start
@@ -786,7 +870,7 @@ For scoped work, `mcp.run do ... end` is shorter and handles cleanup for you:
 ```ruby
 mcp = LLM::MCP.http(
   url: "https://api.githubcopilot.com/mcp/",
-  headers: {"Authorization" => "Bearer #{ENV.fetch("GITHUB_PAT")}"}
+  headers: {"Authorization" => "Bearer #{ENV["GITHUB_PAT"]}"}
 ).persistent
 mcp.run do
   ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)

data/lib/llm/active_record/acts_as_agent.rb

@@ -11,7 +11,6 @@ module LLM::ActiveRecord
   # class and forwarded to an internal agent subclass.
   module ActsAsAgent
     EMPTY_HASH = LLM::ActiveRecord::ActsAsLLM::EMPTY_HASH
-    DEFAULT_USAGE_COLUMNS = LLM::ActiveRecord::ActsAsLLM::DEFAULT_USAGE_COLUMNS
     DEFAULTS = LLM::ActiveRecord::ActsAsLLM::DEFAULTS
     Utils = LLM::ActiveRecord::ActsAsLLM::Utils
 
@@ -41,6 +40,11 @@ module LLM::ActiveRecord
         agent.concurrency(concurrency)
       end
 
+      def tracer(tracer = nil, &block)
+        return agent.tracer if tracer.nil? && !block
+        agent.tracer(tracer, &block)
+      end
+
       def agent
         @agent ||= Class.new(LLM::Agent)
       end
@@ -53,8 +57,6 @@ module LLM::ActiveRecord
       # @param [Class] model
       # @return [void]
       def self.extended(model)
-        options = model.llm_plugin_options
-        model.validates options[:provider_column], options[:model_column], presence: true
         model.include LLM::ActiveRecord::ActsAsLLM::InstanceMethods unless model.ancestors.include?(LLM::ActiveRecord::ActsAsLLM::InstanceMethods)
         model.include InstanceMethods unless model.ancestors.include?(InstanceMethods)
         model.extend ClassMethods unless model.singleton_class.ancestors.include?(ClassMethods)
@@ -72,6 +74,8 @@ module LLM::ActiveRecord
     # @option options [Proc, Symbol, LLM::Tracer, nil] :tracer
     #   Optional tracer, method name, or proc that resolves to one and is
     #   assigned through `llm.tracer = ...` on the resolved provider.
+    # @option options [Proc, Symbol, LLM::Provider] :provider
+    #   Must resolve to an `LLM::Provider` instance for the current record.
     # @yield
     #   Evaluated in the model class after the wrapper is installed, so agent
     #   DSL methods such as `model`, `tools`, `schema`, `instructions`, and
@@ -79,9 +83,8 @@ module LLM::ActiveRecord
     # @return [void]
     def acts_as_agent(options = EMPTY_HASH, &block)
       options = DEFAULTS.merge(options)
-      usage_columns = DEFAULT_USAGE_COLUMNS.merge(options[:usage_columns] || EMPTY_HASH)
       class_attribute :llm_plugin_options, instance_accessor: false, default: DEFAULTS unless respond_to?(:llm_plugin_options)
-      self.llm_plugin_options = options.merge(usage_columns: usage_columns.freeze).freeze
+      self.llm_plugin_options = options.freeze
       extend Hooks
       class_exec(&block) if block
     end
@@ -92,11 +95,8 @@ module LLM::ActiveRecord
       # @return [LLM::Provider]
       def llm
         options = self.class.llm_plugin_options
-        columns = Utils.columns(options)
-        provider = self[columns[:provider_column]]
-        kwargs = Utils.resolve_options(self, options[:provider], ActsAsAgent::EMPTY_HASH)
         return @llm if @llm
-        @llm = LLM.method(provider).call(**kwargs)
+        @llm = Utils.resolve_provider(self, options, ActsAsAgent::EMPTY_HASH)
         @llm.tracer = Utils.resolve_option(self, options[:tracer]) if options[:tracer]
         @llm
       end
@@ -108,10 +108,9 @@ module LLM::ActiveRecord
       def ctx
         @ctx ||= begin
           options = self.class.llm_plugin_options
-          columns = Utils.columns(options)
           params = Utils.resolve_options(self, options[:context], ActsAsAgent::EMPTY_HASH).dup
-          params[:model] ||= self[columns[:model_column]]
           ctx = self.class.agent.new(llm, params.compact)
+          columns = Utils.columns(options)
           data = self[columns[:data_column]]
           if data.nil? || data == ""
             ctx

data/lib/llm/active_record/acts_as_llm.rb

@@ -17,19 +17,11 @@ module LLM::ActiveRecord
   # `tracer:` can also be configured as symbols that are called on the model.
   module ActsAsLLM
     EMPTY_HASH = {}.freeze
-    DEFAULT_USAGE_COLUMNS = {
-      input_tokens: :input_tokens,
-      output_tokens: :output_tokens,
-      total_tokens: :total_tokens
-    }.freeze
     DEFAULTS = {
-      provider_column: :provider,
-      model_column: :model,
       data_column: :data,
       format: :string,
-      usage_columns: DEFAULT_USAGE_COLUMNS,
       tracer: nil,
-      provider: EMPTY_HASH,
+      provider: nil,
       context: EMPTY_HASH
     }.freeze
 
@@ -78,28 +70,26 @@ module LLM::ActiveRecord
       # Maps wrapper options onto the record's storage columns.
       # @return [Hash]
       def self.columns(options)
-        usage_columns = options[:usage_columns]
         {
-          provider_column: options[:provider_column],
-          model_column: options[:model_column],
-          data_column: options[:data_column],
-          input_tokens: usage_columns[:input_tokens],
-          output_tokens: usage_columns[:output_tokens],
-          total_tokens: usage_columns[:total_tokens]
+          data_column: options[:data_column]
         }.freeze
       end
 
+      ##
+      # Resolves the provider runtime for a record.
+      # @return [LLM::Provider]
+      def self.resolve_provider(obj, options, empty_hash)
+        provider = resolve_option(obj, options[:provider])
+        return provider if LLM::Provider === provider
+        raise ArgumentError, "provider: must resolve to an LLM::Provider instance"
+      end
+
       ##
       # Persists the runtime state and usage columns back onto the record.
       # @return [void]
       def self.save(obj, ctx, options)
         columns = self.columns(options)
-        obj.assign_attributes(
-          columns[:data_column] => serialize_context(ctx, options[:format]),
-          columns[:input_tokens] => ctx.usage.input_tokens,
-          columns[:output_tokens] => ctx.usage.output_tokens,
-          columns[:total_tokens] => ctx.usage.total_tokens
-        )
+        obj.assign_attributes(columns[:data_column] => serialize_context(ctx, options[:format]))
         obj.save!
       end
     end
@@ -111,8 +101,6 @@ module LLM::ActiveRecord
       # @param [Class] model
       # @return [void]
       def self.extended(model)
-        options = model.llm_plugin_options
-        model.validates options[:provider_column], options[:model_column], presence: true
         model.include InstanceMethods unless model.ancestors.include?(InstanceMethods)
       end
     end
@@ -128,12 +116,13 @@ module LLM::ActiveRecord
     # @option options [Proc, Symbol, LLM::Tracer, nil] :tracer
     #   Optional tracer, method name, or proc that resolves to one and is
     #   assigned through `llm.tracer = ...` on the resolved provider.
+    # @option options [Proc, Symbol, LLM::Provider] :provider
+    #   Must resolve to an `LLM::Provider` instance for the current record.
     # @return [void]
     def acts_as_llm(options = EMPTY_HASH)
       options = DEFAULTS.merge(options)
-      usage_columns = DEFAULT_USAGE_COLUMNS.merge(options[:usage_columns] || EMPTY_HASH)
       class_attribute :llm_plugin_options, instance_accessor: false, default: DEFAULTS unless respond_to?(:llm_plugin_options)
-      self.llm_plugin_options = options.merge(usage_columns: usage_columns.freeze).freeze
+      self.llm_plugin_options = options.freeze
       extend Hooks
     end
 
@@ -228,12 +217,7 @@ module LLM::ActiveRecord
       # Returns usage from the mapped usage columns.
       # @return [LLM::Object]
       def usage
-        columns = Utils.columns(self.class.llm_plugin_options)
-        LLM::Object.from(
-          input_tokens: self[columns[:input_tokens]] || 0,
-          output_tokens: self[columns[:output_tokens]] || 0,
-          total_tokens: self[columns[:total_tokens]] || 0
-        )
+        ctx.usage || LLM::Object.from(input_tokens: 0, output_tokens: 0, total_tokens: 0)
       end
 
       ##
@@ -285,11 +269,8 @@ module LLM::ActiveRecord
       # @return [LLM::Provider]
       def llm
         options = self.class.llm_plugin_options
-        columns = Utils.columns(options)
-        provider = self[columns[:provider_column]]
-        kwargs = Utils.resolve_options(self, options[:provider], ActsAsLLM::EMPTY_HASH)
         return @llm if @llm
-        @llm = LLM.method(provider).call(**kwargs)
+        @llm = Utils.resolve_provider(self, options, ActsAsLLM::EMPTY_HASH)
         @llm.tracer = Utils.resolve_option(self, options[:tracer]) if options[:tracer]
         @llm
       end
@@ -303,7 +284,6 @@ module LLM::ActiveRecord
           options = self.class.llm_plugin_options
           columns = Utils.columns(options)
           params = Utils.resolve_options(self, options[:context], ActsAsLLM::EMPTY_HASH).dup
-          params[:model] ||= self[columns[:model_column]]
           ctx = LLM::Context.new(llm, params.compact)
           data = self[columns[:data_column]]
           if data.nil? || data == ""

data/lib/llm/agent.rb

@@ -115,6 +115,26 @@ module LLM
       @concurrency = concurrency
     end
 
+    ##
+    # Set or get the default tracer.
+    #
+    # When a block is provided, it is stored and evaluated lazily against the
+    # agent instance during initialization so it can build a tracer from the
+    # resolved provider.
+    #
+    # @example
+    #   class Agent < LLM::Agent
+    #     tracer { LLM::Tracer::Logger.new(llm, io: $stdout) }
+    #   end
+    #
+    # @param [LLM::Tracer, Proc, nil] tracer
+    # @yieldreturn [LLM::Tracer, nil]
+    # @return [LLM::Tracer, Proc, nil]
+    def self.tracer(tracer = nil, &block)
+      return @tracer if tracer.nil? && !block
+      @tracer = block || tracer
+    end
+
     ##
     # @param [LLM::Provider] provider
     #  A provider
@@ -131,6 +151,7 @@ module LLM
       defaults = {model: self.class.model, tools: self.class.tools, skills: self.class.skills, schema: self.class.schema}.compact
       @concurrency = params.delete(:concurrency) || self.class.concurrency
       @llm = llm
+      @tracer = resolve_option(self.class.tracer) unless self.class.tracer.nil?
       @ctx = LLM::Context.new(llm, defaults.merge({guard: true}).merge(params))
     end
 
@@ -179,7 +200,7 @@ module LLM
     ##
     # @return [Array<LLM::Function>]
     def functions
-      @ctx.functions
+      @tracer ? @llm.with_tracer(@tracer) { @ctx.functions } : @ctx.functions
     end
 
     ##
@@ -193,14 +214,14 @@ module LLM
     # @see LLM::Context#call
     # @return [Object]
     def call(...)
-      @ctx.call(...)
+      @tracer ? @llm.with_tracer(@tracer) { @ctx.call(...) } : @ctx.call(...)
     end
 
     ##
     # @see LLM::Context#wait
     # @return [Array<LLM::Function::Return>]
     def wait(...)
-      @ctx.wait(...)
+      @tracer ? @llm.with_tracer(@tracer) { @ctx.wait(...) } : @ctx.wait(...)
     end
 
     ##
@@ -257,7 +278,7 @@ module LLM
     # @return [LLM::Tracer]
     #  Returns an LLM tracer
     def tracer
-      @ctx.tracer
+      @tracer || @ctx.tracer
     end
 
     ##
@@ -371,14 +392,21 @@ module LLM
     end
 
     def run_loop(method, prompt, params)
-      max = Integer(params.delete(:tool_attempts) || 25)
-      res = @ctx.public_send(method, apply_instructions(prompt), params)
-      max.times do
-        break if @ctx.functions.empty?
-        res = @ctx.public_send(method, call_functions, params)
+      loop = proc do
+        max = Integer(params.delete(:tool_attempts) || 25)
+        res = @ctx.public_send(method, apply_instructions(prompt), params)
+        max.times do
+          break if @ctx.functions.empty?
+          res = @ctx.public_send(method, call_functions, params)
+        end
+        raise LLM::ToolLoopError, "pending tool calls remain" unless @ctx.functions.empty?
+        res
       end
-      raise LLM::ToolLoopError, "pending tool calls remain" unless @ctx.functions.empty?
-      res
+      @tracer ? @llm.with_tracer(@tracer, &loop) : loop.call
+    end
+
+    def resolve_option(option)
+      Proc === option ? instance_exec(&option) : option
     end
   end
 end

data/lib/llm/error.rb

@@ -63,6 +63,10 @@ module LLM
   # When a request is interrupted
   Interrupt = Class.new(Error)
 
+  ##
+  # When a concurrency strategy cannot execute a given tool
+  RactorError = Class.new(Error)
+
   ##
   # When a tool call cannot be mapped to a local tool
   NoSuchToolError = Class.new(Error)

data/lib/llm/function/ractor/task.rb

@@ -15,8 +15,12 @@ class LLM::Function
     # @param [String, nil] id
     # @param [String] name
     # @param [Hash, Array, nil] arguments
+    # @param [LLM::Tracer, nil] tracer
+    # @param [Object, nil] span
     # @return [LLM::Function::Ractor::Task]
-    def initialize(runner_class, id, name, arguments)
+    def initialize(runner_class, id, name, arguments, tracer: nil, span: nil)
+      @tracer = tracer
+      @span = span
       @mailbox = Ractor::Mailbox.new(build_task(runner_class, id, name, arguments))
     end
 
@@ -37,7 +41,9 @@ class LLM::Function
     # @return [LLM::Function::Return]
     def wait
       id, name, value = mailbox.wait
-      Return.new(id, name, value)
+      result = Return.new(id, name, value)
+      @tracer&.on_tool_finish(result:, span: @span)
+      result
     end
     alias_method :value, :wait
 

data/lib/llm/function.rb

@@ -218,18 +218,22 @@ class LLM::Function
     task = case strategy
     when :task
       require "async" unless defined?(::Async)
-      Async { call }
+      Async { call! }
     when :thread
-      Thread.new { call }
+      Thread.new { call! }
     when :fiber
       Fiber.new do
-        call
+        call!
       ensure
         Fiber.yield
       end.tap(&:resume)
     when :ractor
-      raise ArgumentError, "Ractor concurrency only supports class-based tools" unless Class === @runner
-      Ractor::Task.new(@runner, id, name, arguments)
+      raise LLM::RactorError, "Ractor concurrency only supports class-based tools" unless Class === @runner
+      if @runner.respond_to?(:skill?) && @runner.skill?
+        raise LLM::RactorError, "Ractor concurrency does not support skill-backed tools"
+      end
+      span = @tracer&.on_tool_start(id:, name:, arguments:, model:)
+      Ractor::Task.new(@runner, id, name, arguments, tracer: @tracer, span:)
     else
       raise ArgumentError, "Unknown strategy: #{strategy.inspect}. Expected :thread, :task, :fiber, or :ractor"
     end
@@ -328,9 +332,16 @@ class LLM::Function
   #   Returns a Return object with either the function result or error information.
   def call_function
     runner = ((Class === @runner) ? @runner.new : @runner)
+    runner.tracer = @tracer if runner.respond_to?(:tracer=)
     kwargs = Hash === arguments ? arguments.transform_keys(&:to_sym) : arguments
     Return.new(id, name, runner.call(**kwargs))
   rescue => ex
     Return.new(id, name,  {error: true, type: ex.class.name, message: ex.message})
   end
+
+  def call!
+    llm = @tracer&.llm
+    return call unless llm.respond_to?(:with_tracer)
+    llm.with_tracer(@tracer) { call }
+  end
 end

data/lib/llm/sequel/agent.rb

@@ -12,7 +12,6 @@ module LLM::Sequel
   module Agent
     require_relative "plugin"
     EMPTY_HASH = LLM::Sequel::Plugin::EMPTY_HASH
-    DEFAULT_USAGE_COLUMNS = LLM::Sequel::Plugin::DEFAULT_USAGE_COLUMNS
     DEFAULTS = LLM::Sequel::Plugin::DEFAULTS
     Utils = LLM::Sequel::Plugin::Utils
 
@@ -24,11 +23,8 @@ module LLM::Sequel
 
     def self.configure(model, options = EMPTY_HASH, &block)
       options = DEFAULTS.merge(options)
-      usage_columns = DEFAULT_USAGE_COLUMNS.merge(options[:usage_columns] || EMPTY_HASH)
-      model.instance_variable_set(
-        :@llm_agent_options,
-        options.merge(usage_columns: usage_columns.freeze).freeze
-      )
+      model.db.extension :pg_json if %i[json jsonb].include?(options[:format])
+      model.instance_variable_set(:@llm_agent_options, options.freeze)
       model.instance_exec(&block) if block
     end
 
@@ -62,6 +58,11 @@ module LLM::Sequel
         agent.concurrency(concurrency)
       end
 
+      def tracer(tracer = nil, &block)
+        return agent.tracer if tracer.nil? && !block
+        agent.tracer(tracer, &block)
+      end
+
       def agent
         @agent ||= Class.new(LLM::Agent)
       end
@@ -75,7 +76,6 @@ module LLM::Sequel
           options = self.class.llm_plugin_options
           columns = Agent::Utils.columns(options)
           params = Agent::Utils.resolve_options(self, options[:context], Agent::EMPTY_HASH).dup
-          params[:model] ||= self[columns[:model_column]]
           ctx = self.class.agent.new(llm, params.compact)
           data = self[columns[:data_column]]
           if data.nil? || data == ""

data/lib/llm/sequel/plugin.rb

@@ -17,11 +17,6 @@ module LLM::Sequel
   # can also be configured as symbols that are called on the model.
   module Plugin
     EMPTY_HASH = {}.freeze
-    DEFAULT_USAGE_COLUMNS = {
-      input_tokens: :input_tokens,
-      output_tokens: :output_tokens,
-      total_tokens: :total_tokens
-    }.freeze
 
     ##
     # Shared helper methods for the ORM wrapper.
@@ -68,38 +63,46 @@ module LLM::Sequel
       # Maps wrapper options onto the record's storage columns.
       # @return [Hash]
       def self.columns(options)
-        usage_columns = options[:usage_columns]
         {
-          provider_column: options[:provider_column],
-          model_column: options[:model_column],
-          data_column: options[:data_column],
-          input_tokens: usage_columns[:input_tokens],
-          output_tokens: usage_columns[:output_tokens],
-          total_tokens: usage_columns[:total_tokens]
+          data_column: options[:data_column]
         }.freeze
       end
 
+      ##
+      # Resolves the provider runtime for a record.
+      # @return [LLM::Provider]
+      def self.resolve_provider(obj, options, empty_hash)
+        provider = resolve_option(obj, options[:provider])
+        return provider if LLM::Provider === provider
+        raise ArgumentError, "provider: must resolve to an LLM::Provider instance"
+      end
+
       ##
       # Persists the runtime state and usage columns back onto the record.
       # @return [void]
       def self.save(obj, ctx, options)
         columns = self.columns(options)
-        obj.update(
-          columns[:data_column] => serialize_context(ctx, options[:format]),
-          columns[:input_tokens] => ctx.usage.input_tokens,
-          columns[:output_tokens] => ctx.usage.output_tokens,
-          columns[:total_tokens] => ctx.usage.total_tokens
-        )
+        payload = serialize_context(ctx, options[:format])
+        payload = wrap_json_payload(payload, options[:format])
+        obj.update(columns[:data_column] => payload)
+      end
+
+      ##
+      # Wraps JSON payloads for Sequel PostgreSQL adapters when needed.
+      # @return [Object]
+      def self.wrap_json_payload(payload, format)
+        case format
+        when :json then Sequel.pg_json_wrap(payload)
+        when :jsonb then Sequel.pg_jsonb_wrap(payload)
+        else payload
+        end
       end
     end
     DEFAULTS = {
-      provider_column: :provider,
-      model_column: :model,
       data_column: :data,
       format: :string,
-      usage_columns: DEFAULT_USAGE_COLUMNS,
       tracer: nil,
-      provider: EMPTY_HASH,
+      provider: nil,
       context: EMPTY_HASH
     }.freeze
 
@@ -134,14 +137,13 @@ module LLM::Sequel
     # @option options [Proc, Symbol, LLM::Tracer, nil] :tracer
     #   Optional tracer, method name, or proc that resolves to one and is
     #   assigned through `llm.tracer = ...` on the resolved provider.
+    # @option options [Proc, Symbol, LLM::Provider] :provider
+    #   Must resolve to an `LLM::Provider` instance for the current record.
     # @return [void]
     def self.configure(model, options = EMPTY_HASH)
       options = DEFAULTS.merge(options)
-      usage_columns = DEFAULT_USAGE_COLUMNS.merge(options[:usage_columns] || EMPTY_HASH)
-      model.instance_variable_set(
-        :@llm_plugin_options,
-        options.merge(usage_columns: usage_columns.freeze).freeze
-      )
+      model.db.extension :pg_json if %i[json jsonb].include?(options[:format])
+      model.instance_variable_set(:@llm_plugin_options, options.freeze)
     end
   end
 
@@ -247,12 +249,7 @@ module LLM::Sequel
     # Returns usage from the mapped usage columns.
     # @return [LLM::Object]
     def usage
-      columns = Utils.columns(self.class.llm_plugin_options)
-      LLM::Object.from(
-        input_tokens: self[columns[:input_tokens]] || 0,
-        output_tokens: self[columns[:output_tokens]] || 0,
-        total_tokens: self[columns[:total_tokens]] || 0
-      )
+      ctx.usage || LLM::Object.from(input_tokens: 0, output_tokens: 0, total_tokens: 0)
     end
 
     ##
@@ -304,11 +301,8 @@ module LLM::Sequel
     # @return [LLM::Provider]
     def llm
       options = self.class.llm_plugin_options
-      columns = Utils.columns(options)
-      provider = self[columns[:provider_column]]
-      kwargs = Utils.resolve_options(self, options[:provider], Plugin::EMPTY_HASH)
       return @llm if @llm
-      @llm = LLM.method(provider).call(**kwargs)
+      @llm = Utils.resolve_provider(self, options, Plugin::EMPTY_HASH)
       @llm.tracer = Utils.resolve_option(self, options[:tracer]) if options[:tracer]
       @llm
     end
@@ -322,7 +316,6 @@ module LLM::Sequel
         options = self.class.llm_plugin_options
         columns = Utils.columns(options)
         params = Utils.resolve_options(self, options[:context], Plugin::EMPTY_HASH).dup
-        params[:model] ||= self[columns[:model_column]]
         ctx = LLM::Context.new(llm, params.compact)
         data = self[columns[:data_column]]
         if data.nil? || data == ""

data/lib/llm/skill.rb

@@ -74,11 +74,12 @@ module LLM
     # @param [LLM::Context] ctx
     # @return [Hash]
     def call(ctx)
-      instructions, tools = self.instructions, self.tools
+      instructions, tools, tracer = self.instructions, self.tools, ctx.llm.tracer
       params = ctx.params.merge(mode: ctx.mode).reject { [:tools, :schema].include?(_1) }
       agent = Class.new(LLM::Agent) do
         instructions(instructions)
         tools(*tools)
+        tracer(tracer)
       end.new(ctx.llm, params)
       agent.messages.concat(messages_for(ctx))
       res = agent.talk("Solve the user's query.")
@@ -95,6 +96,11 @@ module LLM
       Class.new(LLM::Tool) do
         name skill.name
         description skill.description
+        attr_accessor :tracer
+
+        define_singleton_method(:skill?) do
+          true
+        end
 
         define_method(:call) do
           skill.call(ctx)

data/lib/llm/tracer/logger.rb

@@ -114,7 +114,7 @@ module LLM
     # @param [LLM::Response] res
     # @api private
     def finish_attributes(operation, res)
-      case @provider.class.to_s
+      case @llm.class.to_s
       when "LLM::OpenAI" then openai_attributes(operation, res)
       else {}
       end

data/lib/llm/tracer/telemetry.rb

@@ -233,7 +233,7 @@ module LLM
     # @param [LLM::Response] res
     # @api private
     def finish_attributes(operation, res)
-      case @provider.class.to_s
+      case @llm.class.to_s
       when "LLM::OpenAI" then openai_attributes(operation, res)
       else {}
       end

data/lib/llm/tracer.rb

@@ -14,13 +14,17 @@ module LLM
     require_relative "tracer/langsmith"
     require_relative "tracer/null"
 
+    ##
+    # @return [LLM::Provider]
+    attr_reader :llm
+
     ##
     # @param [LLM::Provider] provider
     #  A provider
     # @param [Hash] options
     #  A hash of options
     def initialize(provider, options = {})
-      @provider = provider
+      @llm = provider
       @options = {}
     end
 
@@ -124,7 +128,7 @@ module LLM
     ##
     # @return [String]
     def inspect
-      "#<#{self.class.name}:0x#{object_id.to_s(16)} @provider=#{@provider.class} @tracer=#{@tracer.inspect}>"
+      "#<#{self.class.name}:0x#{object_id.to_s(16)} @provider=#{@llm.class} @tracer=#{@tracer.inspect}>"
     end
 
     ##
@@ -245,19 +249,19 @@ module LLM
     ##
     # @return [String]
     def provider_name
-      @provider.class.name.split("::").last.downcase
+      @llm.class.name.split("::").last.downcase
     end
 
     ##
     # @return [String]
     def provider_host
-      @provider.instance_variable_get(:@host)
+      @llm.instance_variable_get(:@host)
     end
 
     ##
     # @return [String]
     def provider_port
-      @provider.instance_variable_get(:@port)
+      @llm.instance_variable_get(:@port)
     end
   end
 end

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "5.3.0"
+  VERSION = "6.0.0"
 end

data/llm.gemspec

@@ -57,4 +57,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "activerecord", "~> 8.0"
   spec.add_development_dependency "sequel", "~> 5.0"
   spec.add_development_dependency "sqlite3", "~> 2.0"
+  spec.add_development_dependency "pg", "~> 1.5"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 5.3.0
+  version: 6.0.0
 platform: ruby
 authors:
 - Antar Azri
@@ -236,6 +236,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: pg
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
 description: |
   llm.rb is a lightweight runtime for building capable AI systems in Ruby.
   It is not just an API wrapper. llm.rb gives you one runtime for providers,