llm.rb 12.5.1 → 13.0.0

data/README.md

@@ -24,7 +24,8 @@ optional dependencies that are opt-in.
 The runtime supports OpenAI, OpenAI-compatible endpoints, Anthropic, Google
 Gemini, Mistral, DeepSeek, DeepInfra, xAI, Z.ai, AWS Bedrock, Ollama, and llama.cpp.
 It has first-class support for streaming, tool calls,  MCP
-and A2A, embeddings, vector stores and the RAG pattern.
+and A2A, embeddings, vector stores, OCR, context compaction,
+and the RAG pattern.
 
 There are multiple HTTP backends to choose from, tools can be run concurrently
 or in parallel via threads, async tasks, fibers, ractors, and fork, and it is
@@ -35,6 +36,9 @@ so once you learn the fundamentals, everything else falls into place naturally.
 you learn llm.rb, you will also be able to use <a href="https://r.uby.dev/mruby-llm">mruby-llm</a> and
 <a href="https://r.uby.dev/wasm-llm">wasm-llm</a> because the API is pretty much identical.
 
+For detailed explanations, configuration, and advanced patterns, see the
+[deepdive.md](https://r.uby.dev/llm/deepdive/).
+
 ## Install
 
 ```bash
@@ -116,153 +120,52 @@ agent = LLM::Agent.new(llm, stream: MyStream.new)
 agent.talk "Explain Ruby fibers."
 ```
 
-#### LLM::REPL
-
-The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
-method allows an agent to spawn a read-eval-print loop
-that can be useful while developing or operating agents.
-It can be used to debug tool calls, confirm an
-agent has done what was expected, or improve an agent by
-asking questions about what it has done up to that point.
-
-This feature requires that the [curses](https://github.com/ruby/curses)
-and [kramdown](https://github.com/gettalong/kramdown) libraries are
-installed and available to require.
-
-The TUI displays a status line with a context-usage bar and cost
-counter, a scrollable transcript with markdown rendering, and a
-multi-line input area. The UI stays responsive while the model
-is generating a response.
-
-##### REPL: Agent
-
-A REPL session is started by calling `repl` on any agent
-instance. The session inherits the agent's model, tools,
-skills, and instructions.
+#### LLM::Schema
 
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl
-```
-
-##### REPL: State
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) subclasses produce typed, structured
+output from any model call. Pass a schema to `LLM::Context#talk`,
+`LLM::Agent#talk`, or `LLM::Provider#complete` to receive validated
+JSON instead of free text. Schemas work alongside tools and streams.
 
-The `path:` option accepts a file path where runtime state
-is read from and written to. This lets you resume a
-conversation across REPL sessions.
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) can define objects, arrays, enums, nested schemas,
+and more. It is also used internally by [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
+definitions, so you already benefit from it when you declare tool
+parameters.
 
 ```ruby
-require "llm"
+class Weather < LLM::Schema
+  property :city, String, "The city name"
+  property :temperature, Float, "Current temperature"
+  property :conditions, String, "Weather conditions"
+  required %i[city temperature conditions]
+end
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(path: "session.json")
+llm = LLM.openai(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, schema: Weather)
+res = agent.talk "Weather in Paris?"
+res.content!  # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
 ```
 
-##### REPL: Tools
-
-The `tools` option lets you attach additional tools
-for the duration of the session. This is in addition to
-any tools that might already be associated with an agent.
-
-A number of optional tools are distributed as part of
-llm.rb. They power the agents that can be found in the
-[agents/](agents/) directory.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(tools: [Debugger])
-```
+#### LLM::REPL
 
-The following example starts a read-eval-print loop
-with all of the builtin tools available.
+The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
+method drops you into a curses-based TUI for talking to an
+agent interactively. The `path:` option saves and restores
+runtime state across sessions. The `tools:` option attaches
+extra tools for the duration of the session. It is like
+`binding.pry` but for agents. For the full reference see the
+[REPL section](https://r.uby.dev/llm/deepdive/#repl) in the
+deepdive.
 
 ```ruby
 require "llm"
 require "llm/tools"
 
 llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(tools: LLM::Tool.subclasses)
+agent = LLM::Agent.new(llm, name: "my-agent")
+agent.repl(path: "agent.json", tools: LLM::Tool.subclasses)
 ```
 
-##### REPL: Skills
-
-The `skills` option lets you load extra skill directories
-without attaching them to an agent permanently.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(skills: [__dir__])
-```
-
-##### REPL: Tracer
-
-By default the tracer is disabled for the duration of the
-session. Setting `tracer: true` configures the REPL to use
-the tracer associated with an instance of
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
-
-```ruby
-require "llm"
-
-llm    = LLM.deepseek(key: ENV["KEY"])
-tracer = LLM.logger(llm, path: "agent.log")
-agent  = LLM::Agent.new(llm, tracer:)
-agent.repl(tracer: true, tools: [Debugger])
-```
-
-##### REPL: Commands
-
-Commands are recognized by a `/` prefix and are backed by the
-[`LLM::Repl::Command`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html)
-class, which can be subclassed to add custom commands. Once you
-create a subclass, it is automatically added to the repl. A command
-can have zero or more parameters, and all parameters are presumed
-to be a String (at least for now).
-
-```ruby
-require "llm"
-require "llm/repl"
-
-class Greeter < LLM::Command
-  name "greet"
-  description "Greets the given name"
-  parameter :name, String, "The person's name"
-  required %i[name]
-
-  def call(name:)
-    write("Welcome #{name}!\n")
-  end
-end
-```
-
-##### REPL: Input
-
-The input area supports several keyboard shortcuts:
-
-| Key | Action |
-|---|---|
-| `Enter` | Submit the current prompt |
-| `Ctrl+A` | Jump to the start of the line |
-| `Ctrl+E` | Jump to the end of the line |
-| `Ctrl+F` | Move the cursor forward |
-| `Ctrl+K` | Erase from cursor to the end of the line |
-| `Ctrl+Y` | Paste previously killed text |
-| `Ctrl+D` | Delete the character at the cursor |
-| `Left / Right` | Move the cursor |
-| `Up / Down` | Scroll the transcript |
-| `/exit` | Leave the REPL |
-
 #### LLM::MCP
 
 The Model Context Protocol (MCP) has first-class support
@@ -326,16 +229,20 @@ Document.create!(
 
 #### Concurrency
 
-The runtime supports five different concurrency strategies that have
+The runtime supports six different concurrency strategies that have
 different attributes. The choice between all of them often depends
 on the requirements of your application.
 
-IO-bound tools are a good fit for the `:task`, `:thread`,
+IO-bound tools are a good fit for the `:async`, `:thread`,
 and `:fiber` strategies while true parallelism can be achieved
 with the `:fork` and `:ractor` strategies. The
-`:fork` strategy also provides a separate process that offers
+`:sequential` strategy runs tools one at a time and is the default.
+The `:fork` strategy also provides a separate process that offers
 isolation from its parent.
 
+You can learn more about the llm.rb concurrency model in the
+[deepdive.md](https://r.uby.dev/llm/deepdive/#concurrency).
+
 ```ruby
 require "llm"
 
@@ -362,11 +269,11 @@ require "llm"
 require "llm/active_record"
 
 class Agent < ApplicationRecord
-  acts_as_agent do |agent|
-    agent.model "deepseek-v4-pro"
-    agent.instructions "solve the user's query"
-    agent.tools [Research, FinalizeResearch, ActOnResearch]
-  end
+  acts_as_agent
+  set name: "my-agent",
+      instructions: "solve the user's query",
+      model: "deepseek-v4-pro",
+      tools: [Research, FinalizeResearch, ActOnResearch]
 
   private
 
@@ -482,22 +389,5 @@ and resources.
 
 ## License
 
-[Business Source License 1.1](./LICENSE)
-<br>
-Commercial production use requires a commercial license.
-<br>
-Each version converts to the [BSD Zero Clause](https://choosealicense.com/licenses/0bsd/)
-four years after its first public release.
-<br>
-Contact [robert@r.uby.dev](mailto:robert@r.uby.dev) for a commercial license.
-
-### Waivers
-
-Waivers are automatically granted for: <br>
-
-  * Personal use
-  * Students
-  * Teachers
-  * Evaluation, development, and testing
-  * Non-profits and charities
-  * Companies with less than or equal to 50 employees
+This software is released under the terms of the MIT license. <br>
+See [LICENSE](./LICENSE) for details.

data/data/deepinfra.json

@@ -503,6 +503,7 @@
         "context": 131072,
         "output": 131072
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.4,
         "output": 0.4
@@ -537,6 +538,7 @@
         "context": 262144,
         "output": 65536
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.2,
         "output": 0.8
@@ -832,6 +834,7 @@
         "context": 196608,
         "output": 131072
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.15,
         "output": 1.15,

data/data/xai.json

@@ -312,7 +312,7 @@
       "cost": {
         "input": 2,
         "output": 6,
-        "cache_read": 0.5,
+        "cache_read": 0.3,
         "tiers": [
           {
             "input": 4,

data/lib/llm/a2a.rb

@@ -27,7 +27,7 @@
 #   a2a = LLM::A2A.rest(url: "https://agent.example.com")
 #   ctx = LLM::Context.new(llm, tools: a2a.skills)
 #   ctx.talk("Analyze this data using the remote agent.")
-#   ctx.talk(ctx.wait(:call)) while ctx.functions?
+#   ctx.talk(ctx.wait(:sequential)) while ctx.pending_functions?
 class LLM::A2A
   require_relative "a2a/card"
   require_relative "a2a/error"

data/lib/llm/active_record/acts_as_agent.rb

@@ -17,9 +17,41 @@ module LLM::ActiveRecord
   # class and forwarded to an internal agent subclass.
   module ActsAsAgent
     module ClassMethods
+      ##
+      # @return [Class<LLM::Agent>]
       def agent
         @agent ||= Class.new(LLM::Agent)
       end
+
+      ##
+      # Bulk-assign class-level agent defaults.
+      #
+      # Each key is resolved by calling the corresponding class method on the
+      # internal agent subclass.
+      #
+      # @example
+      #   class Agent < ApplicationRecord
+      #     acts_as_agent
+      #     set instructions: "You are a system administrator",
+      #         model: "gpt-4.1-nano",
+      #         tools: [Shell]
+      #   end
+      #
+      # @param [Hash] properties
+      # @option properties [String] :instructions
+      # @option properties [String] :model
+      # @option properties [Array<LLM::Function>] :tools
+      # @option properties [Array<String>] :skills
+      # @option properties [#to_json] :schema
+      # @option properties [Symbol, Array<Symbol>] :concurrency
+      # @option properties [LLM::Tracer, Proc] :tracer
+      # @option properties [Object, Proc] :stream
+      # @option properties [String, Symbol, Array<String, Symbol>, Proc] :confirm
+      # @raise [KeyError] when a property key does not match a class-level accessor
+      # @return [void]
+      def set(properties)
+        agent.set(properties)
+      end
     end
 
     module Hooks

data/lib/llm/active_record/acts_as_llm.rb

@@ -104,17 +104,17 @@ module LLM::ActiveRecord
       end
 
       ##
-      # @see LLM::Context#functions
+      # @see LLM::Context#pending_functions
       # @return [Array<LLM::Function>]
-      def functions
-        ctx.functions
+      def pending_functions
+        ctx.pending_functions
       end
 
       ##
-      # @see LLM::Context#functions?
+      # @see LLM::Context#pending_functions?
       # @return [Boolean]
-      def functions?
-        ctx.functions?
+      def pending_functions?
+        ctx.pending_functions?
       end
 
       ##

data/lib/llm/agent.rb

@@ -2,9 +2,10 @@
 
 module LLM
   ##
-  # {LLM::Agent LLM::Agent} provides a class-level DSL for defining
-  # reusable, preconfigured assistants with defaults for model,
-  # tools, schema, and instructions.
+  # {LLM::Agent LLM::Agent} is the recommended entry point for most
+  # use-cases. It provides a class-level DSL for defining reusable,
+  # preconfigured assistants with defaults for model, tools, schema,
+  # and instructions.
   #
   # It wraps the same stateful runtime surface as
   # {LLM::Context LLM::Context}: message history, usage, persistence,
@@ -22,26 +23,91 @@ module LLM
   # * The default tool attempt budget is `25`. After that, the agent sends
   #   advisory tool errors back through the model and keeps the loop in-band.
   #   Set `tool_attempts: nil` to disable that advisory behavior.
-  # * Tool loop execution can be configured with `concurrency :call`,
-  #   `:thread`, `:task`, `:fiber`, or `:ractor`.
+  # * Tool loop execution can be configured with `concurrency :sequential`,
+  #   `:thread`, `:async`, `:fiber`, or `:ractor`.
   #
-  # @example
+  # @example Subclass with defaults
   #   class SystemAdmin < LLM::Agent
-  #     model "gpt-4.1-nano"
-  #     instructions "You are a Linux system admin"
-  #     tools Shell
-  #     schema Result
+  #     set model: "gpt-4.1-nano",
+  #         instructions: "You are a Linux system admin",
+  #         tools: [Shell],
+  #         schema: Result
   #   end
   #
   #   llm = LLM.openai(key: ENV["KEY"])
   #   agent = SystemAdmin.new(llm)
   #   agent.talk("Run 'date'")
+  #
+  # @example Direct instance
+  #   llm = LLM.deepseek(key: ENV["KEY"])
+  #   agent = LLM::Agent.new(llm, stream: $stdout)
+  #   agent.talk "Hello world"
+  #
+  # @see LLM::Context The low-level runtime that Agent wraps
+  # @see LLM::Tool Tools that Agent can call on your behalf
+  # @see LLM::Stream Stream callbacks for model output
   class Agent
+    ##
+    # @api private
+    UNDEFINED = Object.new
+    private_constant :UNDEFINED
+
     ##
     # Returns a provider
     # @return [LLM::Provider]
     attr_reader :llm
 
+    ##
+    # Bulk-assign class-level agent defaults from a Hash.
+    #
+    # Each key is resolved by calling the corresponding class method on the
+    # agent subclass. An error is raised for unknown keys so that typos are
+    # caught early.
+    #
+    # @example
+    #   class AdminAgent < LLM::Agent
+    #     set name: "admin",
+    #         instructions: "You are a system administrator",
+    #         model: "gpt-4.1-nano",
+    #         tools: [Shell, ReadFile]
+    #   end
+    #
+    # @param [Hash] properties
+    # @option properties [String] :instructions
+    # @option properties [String] :model
+    # @option properties [Array<LLM::Function>] :tools
+    # @option properties [Array<String>] :skills
+    # @option properties [#to_json] :schema
+    # @option properties [Symbol, Array<Symbol>] :concurrency
+    # @option properties [LLM::Tracer, Proc] :tracer
+    # @option properties [Object, Proc] :stream
+    # @option properties [String, Symbol, Array<String, Symbol>, Proc] :confirm
+    # @raise [KeyError] when a property key does not match a class-level accessor
+    # @return [void]
+    def self.set(properties)
+      properties.each do
+        if respond_to?(_1)
+          public_send(_1, _2)
+        else
+          raise KeyError, "key not found: #{_1}"
+        end
+      end
+    end
+
+    ##
+    # Set or get an agent's name
+    # @param [String] name
+    #  The agent name
+    # @return [String]
+    #  Return's the agents name
+    def self.name(name = UNDEFINED, &block)
+      if name.equal?(UNDEFINED)
+        @name || self.to_s.gsub(/(.)([A-Z])/, '\\1-\\2').downcase
+      else
+        @name = block || name
+      end
+    end
+
     ##
     # Set or get the default model
     # @param [String, nil] model
@@ -110,9 +176,9 @@ module LLM
     #
     # @param [Symbol, Array<Symbol>, nil] concurrency
     #  Controls how pending tool loops are executed:
-    #  - `:call`: sequential calls
+    #  - `:sequential`: sequential calls
     #  - `:thread`: concurrent threads
-    #  - `:task`: concurrent async tasks
+    #  - `:async`: concurrent async tasks
     #  - `:fiber`: concurrent scheduler-backed fibers
     #  - `:fork`: forked child processes
     #  - `:ractor`: concurrent Ruby ractors for class-based tools; MCP tools are not supported,
@@ -211,8 +277,8 @@ module LLM
     # @option params [Symbol, Array<Symbol>, nil] :concurrency Defaults to the agent class concurrency
     def initialize(llm, params = {})
       @llm = llm
-      fields = %i[model skills schema tracer stream tools concurrency instructions confirm]
-      fields_ivar = %i[tracer concurrency instructions confirm]
+      fields = %i[name model skills schema tracer stream tools concurrency instructions confirm]
+      fields_ivar = %i[name tracer concurrency instructions confirm]
       fields.each do |field|
         resolvable = params.key?(field) ? params.delete(field) : self.class.public_send(field)
         resolve_symbol = !%i[concurrency].include?(field)
@@ -229,6 +295,13 @@ module LLM
       @ctx = LLM::Context.new(llm, {guard: true}.merge(params))
     end
 
+    ##
+    # Returns the agent's name
+    # @return [String]
+    def name
+      @name
+    end
+
     ##
     # Maintain a conversation via the chat completions API.
     # This method immediately sends a request to the LLM and returns the response.
@@ -263,10 +336,9 @@ module LLM
 
     ##
     # @return [Array<LLM::Function>]
-    def functions
-      @tracer ? @llm.with_tracer(@tracer) { @ctx.functions } : @ctx.functions
+    def pending_functions
+      @tracer ? @llm.with_tracer(@tracer) { @ctx.pending_functions } : @ctx.pending_functions
     end
-    alias_method :pending_functions, :functions
 
     ##
     # @see LLM::Context#returns
@@ -399,6 +471,9 @@ module LLM
     #  By default this method disables the tracer for
     #  the duration of the repl session, and restores
     #  it afterwards.
+    # @param [String] name
+    #  The agent's name.
+    #  Defaults to {LLM::Agent#name}.
     # @param [String] path
     #  The path to a file where runtime state is read
     #  from, and written to
@@ -410,7 +485,7 @@ module LLM
     #  When true, the tracer is kept alive during the
     #  repl session. Default is false.
     # @return [void]
-    def repl(path: nil, tools: [], skills: [], tracer: false, trace: nil)
+    def repl(name: self.name, path: nil, tools: [], skills: [], tracer: false, trace: nil)
       if trace != nil
         warn "llm.rb: trace option is deprecated, use tracer instead"
         tracer = trace
@@ -420,7 +495,7 @@ module LLM
         self.tracer = nil
       end
       require_relative "repl" unless defined?(::LLM::Repl)
-      LLM::Repl.new(agent: self, path:, tools:, skills:).start
+      LLM::Repl.new(agent: self, name:, path:, tools:, skills:).start
     ensure
       if !tracer
         self.tracer = previous
@@ -480,7 +555,7 @@ module LLM
     # @param [Symbol, Array<Symbol>] strategy
     #  The execution strategy that would be used for the tool call.
     # @return [LLM::Function::Return]
-    #  Return either `fn.spawn(strategy).wait` to approve execution or
+    #  Return either `fn.task(strategy).wait` to approve execution or
     #  `fn.cancel(...)` to cancel the call.
     def on_tool_confirmation(fn, strategy)
       fn.cancel
@@ -518,12 +593,12 @@ module LLM
     ##
     # @return [Array<LLM::Function::Return>]
     def call_functions
-      strategy = concurrency || :call
+      strategy = concurrency || :sequential
       return wait(strategy) unless @confirm&.any?
-      confirmables = @ctx.functions.select { @confirm.include?(_1.name.to_s) }
+      confirmables = @ctx.pending_functions.select { @confirm.include?(_1.name.to_s) }
       results = confirmables.map { method(:on_tool_confirmation).call(_1, strategy) }
       @ctx.method(:emit_tool_returns).call(confirmables, results)
-      if (@ctx.functions - confirmables).any?
+      if (@ctx.pending_functions - confirmables).any?
         [*results, *wait(strategy, except: confirmables)]
       else
         results
@@ -541,13 +616,13 @@ module LLM
         stream = params[:stream] || @ctx.params[:stream]
         params[:stream] = LLM::Stream.try(stream, extra: {concurrency:})
         res = talk.call(apply_instructions(prompt), params)
-        while @ctx.functions?
+        while @ctx.pending_functions?
           if max
             max.times do
-              break unless @ctx.functions?
+              break unless @ctx.pending_functions?
               res = talk.call(call_functions, params)
             end
-            res = talk.call(@ctx.functions.map(&:rate_limit), params) if @ctx.functions?
+            res = talk.call(@ctx.pending_functions.map(&:rate_limit), params) if @ctx.pending_functions?
           else
             res = talk.call(call_functions, params)
           end