llm.rb 12.6.0 → 13.0.0

data/README.md

@@ -24,7 +24,8 @@ optional dependencies that are opt-in.
 The runtime supports OpenAI, OpenAI-compatible endpoints, Anthropic, Google
 Gemini, Mistral, DeepSeek, DeepInfra, xAI, Z.ai, AWS Bedrock, Ollama, and llama.cpp.
 It has first-class support for streaming, tool calls,  MCP
-and A2A, embeddings, vector stores and the RAG pattern.
+and A2A, embeddings, vector stores, OCR, context compaction,
+and the RAG pattern.
 
 There are multiple HTTP backends to choose from, tools can be run concurrently
 or in parallel via threads, async tasks, fibers, ractors, and fork, and it is
@@ -35,6 +36,9 @@ so once you learn the fundamentals, everything else falls into place naturally.
 you learn llm.rb, you will also be able to use <a href="https://r.uby.dev/mruby-llm">mruby-llm</a> and
 <a href="https://r.uby.dev/wasm-llm">wasm-llm</a> because the API is pretty much identical.
 
+For detailed explanations, configuration, and advanced patterns, see the
+[deepdive.md](https://r.uby.dev/llm/deepdive/).
+
 ## Install
 
 ```bash
@@ -116,153 +120,52 @@ agent = LLM::Agent.new(llm, stream: MyStream.new)
 agent.talk "Explain Ruby fibers."
 ```
 
-#### LLM::REPL
-
-The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
-method allows an agent to spawn a read-eval-print loop
-that can be useful while developing or operating agents.
-It can be used to debug tool calls, confirm an
-agent has done what was expected, or improve an agent by
-asking questions about what it has done up to that point.
-
-This feature requires that the [curses](https://github.com/ruby/curses)
-and [kramdown](https://github.com/gettalong/kramdown) libraries are
-installed and available to require.
-
-The TUI displays a status line with a context-usage bar and cost
-counter, a scrollable transcript with markdown rendering, and a
-multi-line input area. The UI stays responsive while the model
-is generating a response.
+#### LLM::Schema
 
-##### REPL: Agent
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) subclasses produce typed, structured
+output from any model call. Pass a schema to `LLM::Context#talk`,
+`LLM::Agent#talk`, or `LLM::Provider#complete` to receive validated
+JSON instead of free text. Schemas work alongside tools and streams.
 
-A REPL session is started by calling `repl` on any agent
-instance. The session inherits the agent's model, tools,
-skills, and instructions.
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) can define objects, arrays, enums, nested schemas,
+and more. It is also used internally by [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
+definitions, so you already benefit from it when you declare tool
+parameters.
 
 ```ruby
-require "llm"
+class Weather < LLM::Schema
+  property :city, String, "The city name"
+  property :temperature, Float, "Current temperature"
+  property :conditions, String, "Weather conditions"
+  required %i[city temperature conditions]
+end
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl
+llm = LLM.openai(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, schema: Weather)
+res = agent.talk "Weather in Paris?"
+res.content!  # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
 ```
 
-##### REPL: State
-
-The `path:` option accepts a file path where runtime state
-is read from and written to. This lets you resume a
-conversation across REPL sessions.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(path: "session.json")
-```
-
-##### REPL: Tools
-
-The `tools` option lets you attach additional tools
-for the duration of the session. This is in addition to
-any tools that might already be associated with an agent.
-
-A number of optional tools are distributed as part of
-llm.rb. They power the agents that can be found in the
-[agents/](agents/) directory.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(tools: [Debugger])
-```
+#### LLM::REPL
 
-The following example starts a read-eval-print loop
-with all of the builtin tools available.
+The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
+method drops you into a curses-based TUI for talking to an
+agent interactively. The `path:` option saves and restores
+runtime state across sessions. The `tools:` option attaches
+extra tools for the duration of the session. It is like
+`binding.pry` but for agents. For the full reference see the
+[REPL section](https://r.uby.dev/llm/deepdive/#repl) in the
+deepdive.
 
 ```ruby
 require "llm"
 require "llm/tools"
 
 llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(tools: LLM::Tool.subclasses)
-```
-
-##### REPL: Skills
-
-The `skills` option lets you load extra skill directories
-without attaching them to an agent permanently.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(skills: [__dir__])
-```
-
-##### REPL: Tracer
-
-By default the tracer is disabled for the duration of the
-session. Setting `tracer: true` configures the REPL to use
-the tracer associated with an instance of
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
-
-```ruby
-require "llm"
-
-llm    = LLM.deepseek(key: ENV["KEY"])
-tracer = LLM.logger(llm, path: "agent.log")
-agent  = LLM::Agent.new(llm, tracer:)
-agent.repl(tracer: true, tools: [Debugger])
+agent = LLM::Agent.new(llm, name: "my-agent")
+agent.repl(path: "agent.json", tools: LLM::Tool.subclasses)
 ```
 
-##### REPL: Commands
-
-Commands are recognized by a `/` prefix and are backed by the
-[`LLM::Repl::Command`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html)
-class, which can be subclassed to add custom commands. Once you
-create a subclass, it is automatically added to the repl. A command
-can have zero or more parameters, and all parameters are presumed
-to be a String (at least for now).
-
-```ruby
-require "llm"
-require "llm/repl"
-
-class Greeter < LLM::Command
-  name "greet"
-  description "Greets the given name"
-  parameter :name, String, "The person's name"
-  required %i[name]
-
-  def call(name:)
-    write("Welcome #{name}!\n")
-  end
-end
-```
-
-##### REPL: Input
-
-The input area supports several keyboard shortcuts:
-
-| Key | Action |
-|---|---|
-| `Enter` | Submit the current prompt |
-| `Ctrl+A` | Jump to the start of the line |
-| `Ctrl+E` | Jump to the end of the line |
-| `Ctrl+F` | Move the cursor forward |
-| `Ctrl+K` | Erase from cursor to the end of the line |
-| `Ctrl+Y` | Paste previously killed text |
-| `Ctrl+D` | Delete the character at the cursor |
-| `Left / Right` | Move the cursor |
-| `Up / Down` | Scroll the transcript |
-| `/exit` | Leave the REPL |
-
 #### LLM::MCP
 
 The Model Context Protocol (MCP) has first-class support
@@ -326,16 +229,20 @@ Document.create!(
 
 #### Concurrency
 
-The runtime supports five different concurrency strategies that have
+The runtime supports six different concurrency strategies that have
 different attributes. The choice between all of them often depends
 on the requirements of your application.
 
-IO-bound tools are a good fit for the `:task`, `:thread`,
+IO-bound tools are a good fit for the `:async`, `:thread`,
 and `:fiber` strategies while true parallelism can be achieved
 with the `:fork` and `:ractor` strategies. The
-`:fork` strategy also provides a separate process that offers
+`:sequential` strategy runs tools one at a time and is the default.
+The `:fork` strategy also provides a separate process that offers
 isolation from its parent.
 
+You can learn more about the llm.rb concurrency model in the
+[deepdive.md](https://r.uby.dev/llm/deepdive/#concurrency).
+
 ```ruby
 require "llm"
 
@@ -363,7 +270,8 @@ require "llm/active_record"
 
 class Agent < ApplicationRecord
   acts_as_agent
-  set instructions: "solve the user's query",
+  set name: "my-agent",
+      instructions: "solve the user's query",
       model: "deepseek-v4-pro",
       tools: [Research, FinalizeResearch, ActOnResearch]
 
@@ -481,22 +389,5 @@ and resources.
 
 ## License
 
-[Business Source License 1.1](./LICENSE)
-<br>
-Commercial production use requires a commercial license.
-<br>
-Each version converts to the [BSD Zero Clause](https://choosealicense.com/licenses/0bsd/)
-four years after its first public release.
-<br>
-Contact [robert@r.uby.dev](mailto:robert@r.uby.dev) for a commercial license.
-
-### Waivers
-
-Waivers are automatically granted for: <br>
-
-  * Personal use
-  * Students
-  * Teachers
-  * Evaluation, development, and testing
-  * Non-profits and charities
-  * Companies with less than or equal to 50 employees
+This software is released under the terms of the MIT license. <br>
+See [LICENSE](./LICENSE) for details.

data/data/deepinfra.json

@@ -503,6 +503,7 @@
         "context": 131072,
         "output": 131072
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.4,
         "output": 0.4
@@ -537,6 +538,7 @@
         "context": 262144,
         "output": 65536
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.2,
         "output": 0.8
@@ -832,6 +834,7 @@
         "context": 196608,
         "output": 131072
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.15,
         "output": 1.15,

data/data/xai.json

@@ -312,7 +312,7 @@
       "cost": {
         "input": 2,
         "output": 6,
-        "cache_read": 0.5,
+        "cache_read": 0.3,
         "tiers": [
           {
             "input": 4,

data/lib/llm/a2a.rb

@@ -27,7 +27,7 @@
 #   a2a = LLM::A2A.rest(url: "https://agent.example.com")
 #   ctx = LLM::Context.new(llm, tools: a2a.skills)
 #   ctx.talk("Analyze this data using the remote agent.")
-#   ctx.talk(ctx.wait(:call)) while ctx.functions?
+#   ctx.talk(ctx.wait(:sequential)) while ctx.pending_functions?
 class LLM::A2A
   require_relative "a2a/card"
   require_relative "a2a/error"

data/lib/llm/active_record/acts_as_llm.rb

@@ -104,17 +104,17 @@ module LLM::ActiveRecord
       end
 
       ##
-      # @see LLM::Context#functions
+      # @see LLM::Context#pending_functions
       # @return [Array<LLM::Function>]
-      def functions
-        ctx.functions
+      def pending_functions
+        ctx.pending_functions
       end
 
       ##
-      # @see LLM::Context#functions?
+      # @see LLM::Context#pending_functions?
       # @return [Boolean]
-      def functions?
-        ctx.functions?
+      def pending_functions?
+        ctx.pending_functions?
       end
 
       ##

data/lib/llm/agent.rb

@@ -2,9 +2,10 @@
 
 module LLM
   ##
-  # {LLM::Agent LLM::Agent} provides a class-level DSL for defining
-  # reusable, preconfigured assistants with defaults for model,
-  # tools, schema, and instructions.
+  # {LLM::Agent LLM::Agent} is the recommended entry point for most
+  # use-cases. It provides a class-level DSL for defining reusable,
+  # preconfigured assistants with defaults for model, tools, schema,
+  # and instructions.
   #
   # It wraps the same stateful runtime surface as
   # {LLM::Context LLM::Context}: message history, usage, persistence,
@@ -22,10 +23,10 @@ module LLM
   # * The default tool attempt budget is `25`. After that, the agent sends
   #   advisory tool errors back through the model and keeps the loop in-band.
   #   Set `tool_attempts: nil` to disable that advisory behavior.
-  # * Tool loop execution can be configured with `concurrency :call`,
-  #   `:thread`, `:task`, `:fiber`, or `:ractor`.
+  # * Tool loop execution can be configured with `concurrency :sequential`,
+  #   `:thread`, `:async`, `:fiber`, or `:ractor`.
   #
-  # @example
+  # @example Subclass with defaults
   #   class SystemAdmin < LLM::Agent
   #     set model: "gpt-4.1-nano",
   #         instructions: "You are a Linux system admin",
@@ -36,7 +37,21 @@ module LLM
   #   llm = LLM.openai(key: ENV["KEY"])
   #   agent = SystemAdmin.new(llm)
   #   agent.talk("Run 'date'")
+  #
+  # @example Direct instance
+  #   llm = LLM.deepseek(key: ENV["KEY"])
+  #   agent = LLM::Agent.new(llm, stream: $stdout)
+  #   agent.talk "Hello world"
+  #
+  # @see LLM::Context The low-level runtime that Agent wraps
+  # @see LLM::Tool Tools that Agent can call on your behalf
+  # @see LLM::Stream Stream callbacks for model output
   class Agent
+    ##
+    # @api private
+    UNDEFINED = Object.new
+    private_constant :UNDEFINED
+
     ##
     # Returns a provider
     # @return [LLM::Provider]
@@ -51,7 +66,8 @@ module LLM
     #
     # @example
     #   class AdminAgent < LLM::Agent
-    #     set instructions: "You are a system administrator",
+    #     set name: "admin",
+    #         instructions: "You are a system administrator",
     #         model: "gpt-4.1-nano",
     #         tools: [Shell, ReadFile]
     #   end
@@ -78,6 +94,20 @@ module LLM
       end
     end
 
+    ##
+    # Set or get an agent's name
+    # @param [String] name
+    #  The agent name
+    # @return [String]
+    #  Return's the agents name
+    def self.name(name = UNDEFINED, &block)
+      if name.equal?(UNDEFINED)
+        @name || self.to_s.gsub(/(.)([A-Z])/, '\\1-\\2').downcase
+      else
+        @name = block || name
+      end
+    end
+
     ##
     # Set or get the default model
     # @param [String, nil] model
@@ -146,9 +176,9 @@ module LLM
     #
     # @param [Symbol, Array<Symbol>, nil] concurrency
     #  Controls how pending tool loops are executed:
-    #  - `:call`: sequential calls
+    #  - `:sequential`: sequential calls
     #  - `:thread`: concurrent threads
-    #  - `:task`: concurrent async tasks
+    #  - `:async`: concurrent async tasks
     #  - `:fiber`: concurrent scheduler-backed fibers
     #  - `:fork`: forked child processes
     #  - `:ractor`: concurrent Ruby ractors for class-based tools; MCP tools are not supported,
@@ -247,8 +277,8 @@ module LLM
     # @option params [Symbol, Array<Symbol>, nil] :concurrency Defaults to the agent class concurrency
     def initialize(llm, params = {})
       @llm = llm
-      fields = %i[model skills schema tracer stream tools concurrency instructions confirm]
-      fields_ivar = %i[tracer concurrency instructions confirm]
+      fields = %i[name model skills schema tracer stream tools concurrency instructions confirm]
+      fields_ivar = %i[name tracer concurrency instructions confirm]
       fields.each do |field|
         resolvable = params.key?(field) ? params.delete(field) : self.class.public_send(field)
         resolve_symbol = !%i[concurrency].include?(field)
@@ -265,6 +295,13 @@ module LLM
       @ctx = LLM::Context.new(llm, {guard: true}.merge(params))
     end
 
+    ##
+    # Returns the agent's name
+    # @return [String]
+    def name
+      @name
+    end
+
     ##
     # Maintain a conversation via the chat completions API.
     # This method immediately sends a request to the LLM and returns the response.
@@ -299,10 +336,9 @@ module LLM
 
     ##
     # @return [Array<LLM::Function>]
-    def functions
-      @tracer ? @llm.with_tracer(@tracer) { @ctx.functions } : @ctx.functions
+    def pending_functions
+      @tracer ? @llm.with_tracer(@tracer) { @ctx.pending_functions } : @ctx.pending_functions
     end
-    alias_method :pending_functions, :functions
 
     ##
     # @see LLM::Context#returns
@@ -435,6 +471,9 @@ module LLM
     #  By default this method disables the tracer for
     #  the duration of the repl session, and restores
     #  it afterwards.
+    # @param [String] name
+    #  The agent's name.
+    #  Defaults to {LLM::Agent#name}.
     # @param [String] path
     #  The path to a file where runtime state is read
     #  from, and written to
@@ -446,7 +485,7 @@ module LLM
     #  When true, the tracer is kept alive during the
     #  repl session. Default is false.
     # @return [void]
-    def repl(path: nil, tools: [], skills: [], tracer: false, trace: nil)
+    def repl(name: self.name, path: nil, tools: [], skills: [], tracer: false, trace: nil)
       if trace != nil
         warn "llm.rb: trace option is deprecated, use tracer instead"
         tracer = trace
@@ -456,7 +495,7 @@ module LLM
         self.tracer = nil
       end
       require_relative "repl" unless defined?(::LLM::Repl)
-      LLM::Repl.new(agent: self, path:, tools:, skills:).start
+      LLM::Repl.new(agent: self, name:, path:, tools:, skills:).start
     ensure
       if !tracer
         self.tracer = previous
@@ -516,7 +555,7 @@ module LLM
     # @param [Symbol, Array<Symbol>] strategy
     #  The execution strategy that would be used for the tool call.
     # @return [LLM::Function::Return]
-    #  Return either `fn.spawn(strategy).wait` to approve execution or
+    #  Return either `fn.task(strategy).wait` to approve execution or
     #  `fn.cancel(...)` to cancel the call.
     def on_tool_confirmation(fn, strategy)
       fn.cancel
@@ -554,12 +593,12 @@ module LLM
     ##
     # @return [Array<LLM::Function::Return>]
     def call_functions
-      strategy = concurrency || :call
+      strategy = concurrency || :sequential
       return wait(strategy) unless @confirm&.any?
-      confirmables = @ctx.functions.select { @confirm.include?(_1.name.to_s) }
+      confirmables = @ctx.pending_functions.select { @confirm.include?(_1.name.to_s) }
       results = confirmables.map { method(:on_tool_confirmation).call(_1, strategy) }
       @ctx.method(:emit_tool_returns).call(confirmables, results)
-      if (@ctx.functions - confirmables).any?
+      if (@ctx.pending_functions - confirmables).any?
         [*results, *wait(strategy, except: confirmables)]
       else
         results
@@ -577,13 +616,13 @@ module LLM
         stream = params[:stream] || @ctx.params[:stream]
         params[:stream] = LLM::Stream.try(stream, extra: {concurrency:})
         res = talk.call(apply_instructions(prompt), params)
-        while @ctx.functions?
+        while @ctx.pending_functions?
           if max
             max.times do
-              break unless @ctx.functions?
+              break unless @ctx.pending_functions?
               res = talk.call(call_functions, params)
             end
-            res = talk.call(@ctx.functions.map(&:rate_limit), params) if @ctx.functions?
+            res = talk.call(@ctx.pending_functions.map(&:rate_limit), params) if @ctx.pending_functions?
           else
             res = talk.call(call_functions, params)
           end

data/lib/llm/buffer.rb

@@ -3,10 +3,24 @@
 module LLM
   ##
   # {LLM::Buffer LLM::Buffer} provides an Enumerable object that
-  # tracks messages in a conversation thread.
+  # tracks messages in a conversation thread. Access it through
+  # {LLM::Context#messages}.
+  #
+  # @example Working with message history
+  #   ctx.messages.last           # => most recent message
+  #   ctx.messages.first          # => oldest message
+  #   ctx.messages.select! { |m| m.assistant? }
+  #   ctx.messages.reverse        # => reversed copy
+  #   ctx.messages.reject! { |m| m.compaction? }
+  #
+  # @see LLM::Message Individual messages in the buffer
+  # @see LLM::Context Where the buffer lives (ctx.messages)
   class Buffer
     include Enumerable
 
+    UNDEFINED = Object.new
+    private_constant :UNDEFINED
+
     ##
     # @param [LLM::Provider] provider
     # @return [LLM::Buffer]
@@ -65,8 +79,69 @@ module LLM
     # @param [Integer, nil] n
     #  The number of messages to return
     # @return [LLM::Message, Array<LLM::Message>, nil]
-    def last(n = nil)
-      n.nil? ? @messages.last : @messages.last(n)
+    def last(n = UNDEFINED)
+      n.equal?(UNDEFINED) ? @messages.last : @messages.last(n)
+    end
+
+    ##
+    # Returns the first message(s) in the buffer
+    # @param [Integer, nil] n
+    #  The number of messages to return
+    # @return [LLM::Message, Array<LLM::Message>, nil]
+    def first(n = UNDEFINED)
+      n.equal?(UNDEFINED) ? @messages.first : @messages.first(n)
+    end
+
+    ##
+    # Removes messages matching the block in-place.
+    # @yield [LLM::Message]
+    # @return [LLM::Buffer]
+    def reject!(&)
+      @messages.reject!(&)
+      self
+    end
+    alias_method :delete_if, :reject!
+
+    ##
+    # Keeps messages matching the block in-place.
+    # @yield [LLM::Message]
+    # @return [LLM::Buffer]
+    def select!(&)
+      @messages.select!(&)
+      self
+    end
+
+    ##
+    # Removes and returns the first message.
+    # @return [LLM::Message, nil]
+    def shift
+      @messages.shift
+    end
+
+    ##
+    # Removes all messages.
+    # @return [LLM::Buffer]
+    def clear
+      @messages.clear
+      self
+    end
+
+    ##
+    # Returns all elements after the first n.
+    # @param [Integer] n
+    #  The number of messages to skip
+    # @return [Array<LLM::Message>]
+    def drop(n)
+      @messages.drop(n)
+    end
+
+    ##
+    # Returns the first n elements without removing them.
+    # @param [Integer] n
+    #  The number of messages to return
+    # @return [Array<LLM::Message>]
+    def take(n)
+      @messages.take(n)
     end
 
     ##
@@ -103,6 +178,13 @@ module LLM
       @messages[index]
     end
 
+    ##
+    # Returns a reversed copy of the internal array.
+    # @return [Array]
+    def reverse
+      @messages.reverse
+    end
+
     ##
     # @return [String]
     def to_json(...)

data/lib/llm/compactor/null.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+class LLM::Compactor
+  ##
+  # An {LLM::Compactor::Null LLM::Compactor::Null} is a compactor that
+  # does nothing. It is used as the default when no compactor strategy
+  # is configured.
+  #
+  # All methods return nil and produce no side effects.
+  class Null < self
+    ##
+    # @param [Hash] opts
+    #  Ignored
+    # @return [nil]
+    def call(**opts)
+      nil
+    end
+  end
+end