llm.rb 15.1.0 → 15.2.1

data/README.md

@@ -1,7 +1,7 @@
 <p align="center">
   <a href="https://r.uby.dev">
     <img
-      src="https://github.com/r-uby-dev/llm.rb/raw/main/rubydev.svg"
+      src="rubydev.svg"
       width="400"
       height="200"
       border="0"
@@ -10,21 +10,19 @@
   </a>
 </p>
 
-> A [r.uby.dev](https://r.uby.dev/llm) project.
+> [r.uby.dev](https://r.uby.dev/llm) project.
 
 Welcome to the canonical llm.rb repository.
 
 llm.rb is an advanced runtime for building agentic AI applications
 on CRuby. It has zero runtime dependencies by default, supports
 concurrent and parallel tool execution and has a single coherent API
-that spans 14+ providers. Streaming, tools, guards, compaction, the
-REPL, builtin MCP/A2A support and the database integrations all build
-on the same three concepts: providers, contexts, and agents.
+that spans 14+ providers.
 
-The easiest way to learn about llm.rb is to ask [the r.uby.dev chatbot](https://r.uby.dev)
+The most effective way to learn about llm.rb is to ask [the r.uby.dev chatbot](https://r.uby.dev)
 a question. It is connected to the llm.rb GitHub repository, backed by
-ActiveRecord and uses the builtin MCP feature to connect to GitHub. All
-answers are grounded in the llm.rb source code.
+ActiveRecord and uses the builtin MCP feature to connect to GitHub. The chatbot
+is an llm.rb agent that is deployed with [roda-llm](https://github.com/r-uby-dev/roda-llm#readme).
 
 ## Install
 
@@ -39,9 +37,22 @@ gem install llm.rb
 The
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
 class is the default high-level interface,
-and it is recommended for most use-cases. It manages tool execution
-automatically and guards against infinite loops,
-manages conversation state, and much more.
+and it is recommended for most use-cases. It manages the tool loop
+and provides configurable features on top of it. For example you can
+manage the tool loop with a retry budget alongside a tool call budget,
+among other features.
+
+The runtime is designed to keep the tool loop alive and it will
+avoid exceptions. When an error is encountered in a tool or during
+the lifecycle of an agent it is almost always reported back to the
+model as an in-band error that allows the model to correct course.
+
+A lot of care also goes into keeping the tool loop from entering
+an invalid state that would lead to API-level errors. For example,
+when a tool call is interrupted it could leave an unanswered tool
+call that a model will reject on the next turn. The runtime takes
+care of this by pruning orphaned tool calls and ensuring that the
+tool loop always remains valid. 
 
 ```ruby
 require "llm"
@@ -206,7 +217,7 @@ isolation from its parent.
 
 A couple of concurrency strategies require optional, opt-in dependencies.
 The `async` strategy requires the [async](https://github.com/socketry/async)
-gem and the `fork` strategy requires the [xchan.rb](https://github.com/0x1eef/xchan.rb)
+gem and the `fork` strategy requires the [xchan.rb](https://github.com/r-uby-dev/xchan.rb)
 gem. The `fiber` strategy requires a scheduler (`Fiber.scheduler`) but by
 default Ruby does not provide one.
 
@@ -247,27 +258,25 @@ end
 ```
 </details>
 <details>
-<summary>Console (<code>binding.pry</code> for agents)</summary>
+<summary>Console (<code>binding.irb</code> for agents)</summary>
 <br>
 
-The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
-method drops you into a highly capable read-eval-print loop (REPL)
-that is built on top of curses. It can help you debug agents,
-test your tools, connect to MCP servers, and even A2A agents.
-The REPL stands out because it connects to the surrounding
-runtime and it can be extended by your code. Think of it as
-`binding.pry` but for agents.
+The [LLM::Agent#console](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#console-instance_method)
+method drops you into an interactive console that is built on
+top of curses. It can help you debug agents, test your tools,
+connect to MCP servers, and other A2A agents. The console stands
+out because it connects to the surrounding runtime and it can
+be extended by your code. Think of it as `binding.irb` but
+for agents.
 
 ##### Demo
 
-[Watch in high quality on asciinema](https://asciinema.org/a/OsS8wwaasKasoDDz)
-
-![llm.rb REPL demo](demo.gif)
+![llm.rb console demo](demo.gif)
 
 
 ##### Installation
 
-The REPL is distributed with llm.rb so you don't have to install
+The console is distributed with llm.rb so you don't have to install
 a separate gem but it requires a number of optional dependencies
 to be installed separately. The following gems provide the full
 experience:
@@ -276,8 +285,8 @@ experience:
 
 ##### Persistence
 
-The `path:` option can be set on an agent for automatic persistence
-across REPL sessions. The `tools:` option attaches extra tools
+the `path:` option can be set on an agent for automatic persistence
+across console sessions. The `tools:` option attaches extra tools
 for the duration of the session. Recall previous turns with Ctrl+P and
 Ctrl+N.
 
@@ -287,21 +296,26 @@ require "llm/tools"
 
 llm = LLM.deepseek(key: ENV["KEY"])
 agent = LLM::Agent.new(llm, name: "my-agent", path: "agent.json")
-agent.repl(tools: LLM::Tool.subclasses)
+agent.console(tools: LLM::Tool.subclasses)
 ```
 
 ##### CLI
 
 The `llm.rb` executable is available on your PATH after installation.
-It starts a REPL session from any directory.The CLI auto-detects your
+It starts a console session from any directory. The CLI auto-detects your
 provider from standard environment variables (`DEEPSEEK_API_KEY`,
 `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.). Persistent sessions are
 stored under `~/.llm.rb/` and restored automatically on your next visit.
 
 ```bash
-llm.rb                     # auto-detect from $DEEPSEEK_API_KEY
+llm.rb                     # auto-detect from $PROVIDER_API_KEY
 llm.rb -p openai           # use OpenAI explicitly
+llm.rb -m gpt-5.6          # use a model other than the provider default
+llm.rb -c thread           # run tool calls on a separate thread
+llm.rb -n curb             # use libcurl as the HTTP transport
+llm.rb -x 900              # read timeout of 15 minutes
 llm.rb -t                  # temporary session, no persistence
+llm.rb -v                  # print the version
 ```
 </details>
 <details>
@@ -423,8 +437,8 @@ agent = Raven.find(agent.id).tap(&:research_codebase)
 ##
 # Start an agent console.
 # Query agent's state, debug, etc.
-# The REPL does not persist back to the database.
-agent.repl
+# The console does not persist back to the database.
+agent.console
 ```
 </details>
 
@@ -534,15 +548,15 @@ the call, or `nil` to let it run:
 ```ruby
 class PolicyGuard < LLM::Guard
   def call(function:)
-    if function.name == "shell"
+    if function.name == "exec"
       function.return(error: true, type: "policy_error",
-                      message: "shell is disabled")
+                      message: "exec is disabled")
     end
   end
 end
 
 llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, tools: [Shell, ReadFile], guard: PolicyGuard)
+agent = LLM::Agent.new(llm, tools: [LLM::Tool::Exec, ReadFile], guard: PolicyGuard)
 ```
 </details>
 
@@ -629,21 +643,28 @@ agent.talk "Hello"
 <summary>Observability</summary>
 <br>
 
-Trace what an agent is doing by attaching a tracer. Hook into
-requests, tool calls, and other runtime events to debug a
-misbehaving agent, monitor latency, or export spans to an
-observability backend. All built-in tracers share one interface,
-so switching between them means changing a class name:
+It is possible to trace what an agent is doing by attaching a
+tracer. A tracer can hook into requests, tool calls, and other
+runtime events to debug an agent, provide insights, monitor latency,
+or export spans to an observability backend. All built-in tracers
+share one interface, so switching between them means changing a
+factory method:
 
-* [`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html): human-readable single-line logs to stderr, ideal during development.
-* [`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html):
+* [`LLM::Tracer.pretty_logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#pretty_logger-class_method): human-readable single-line logs to stderr, ideal during development.
+* [`LLM::Tracer.telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#telemetry-class_method):
 exports spans via OTLP for OpenTelemetry in production.
-* [`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html):
+* [`LLM::Tracer.logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#logger-class_method):
 structured JSON to stdout or a file.
 
+It is also possible to create your own tracer by creating a subclass
+of [`LLM::Tracer`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html)
+that implements a number of callbacks that cover an agent's lifecycle.
+The tracer feature provides visibility into what the runtime is doing,
+and the tracer API lets other code hook into that feature.
+
 ```ruby
 llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, tracer: LLM::Tracer::PrettyLogger.new(llm))
+agent = LLM::Agent.new(llm, tracer: LLM::Tracer.pretty_logger(llm))
 agent.talk "Hello"
 ```
 </details>
@@ -668,7 +689,7 @@ class Agent < LLM::Agent
   set name: "sysadmin",
       description: "system administration agent",
       model: "deepseek-v4-pro",
-      tools: [LLM::Tool::Shell]
+      tools: [LLM::Tool::Exec]
 end
 
 llm = LLM.deepseek(key: ENV["KEY"])
@@ -756,7 +777,7 @@ require "llm"
 
 llm      = LLM.openai
 registry = llm.registry                # => LLM::Provider#registry
-cheapest = registry.models.sort.first  # => LLM::Model
+cheapest = registry.models.sort.first  # => LLM::Registry::Model
 cheapest.id                            # => "text-embedding-3-small"
 cheapest.context_window                # => 8191
 cheapest.structured_output?            # => false
@@ -895,6 +916,17 @@ IO.copy_stream res.images[0], "rocket-with-dog.svg"
 
 ## FAQ
 
+<details>
+<summary>Where can I see llm.rb in action?</summary>
+<br>
+<p>
+
+The [r.uby.dev](https://r.uby.dev) website is powered
+by llm.rb and its builtin MCP feature. It is connected
+to this very GitHub repository. It is designed to help
+you learn and troubleshoot llm.rb.
+</p>
+</details>
 <details>
 <summary>What about local LLM support?</summary>
 <br>
@@ -962,7 +994,7 @@ than three years and over that time multiple other
 contributors have contributed to llm.rb as well. New
 contributors are always welcome.
 
-I use the repl that is distributed with llm.rb to build
+I use the console that is distributed with llm.rb to build
 llm.rb itself so there is a healthy feedback loop and
 llm.rb has also been battle tested in production
 environments.
@@ -975,13 +1007,19 @@ I am constantly focused on improving llm.rb by using
 it as my primary driver for development.
 </details>
 
-## Resources
+## See also
+
+The [roda-llm](https://github.com/r-uby-dev/roda-llm#readme) project
+is how I deploy multiple ActiveRecord-backed llm.rb agents over HTTP.
+Each agent has an identical interface at a unique path that provide
+CRUD operations and stream support (via SSE - Server Side Events).
+It lets you focus on implementing agents rather than the glue that
+brings them together. It is implemented as a Roda plugin that could
+be hosted within a Rails application or other Rack-based applications.
 
-The [r.uby.dev chatbot](https://r.uby.dev) is connected
-to this very GitHub repository. It can read documentation,
-source code, issues, and pull requests. The [docs/](docs/)
-directory contains the full documentation and the chatbot
-can find the answers to your questions there.
+The [docs/](docs/) directory contains the full documentation and
+the chatbot can find the answers to your questions there. Or you
+can read them yourself.
 
 ## License
 

data/bin/llm.rb

@@ -42,6 +42,10 @@ def wrap(text, width)
   end
 end
 
+def version
+  warn "llm.rb v#{LLM::VERSION}"
+end
+
 def help
   prog = File.basename($PROGRAM_NAME)
   warn ""
@@ -54,6 +58,7 @@ def help
   warn "  -n TRANSPORT   HTTP transports - net-http (default), net-http-persistent, and curb"
   warn "  -x TIMEOUT     The default read timeout (in seconds)"
   warn "  -t             Temporary session that doesn't persist to disk"
+  warn "  -v             Print version information"
   warn "  -h             Show this help"
   warn ""
   warn "Examples:"
@@ -74,12 +79,12 @@ def loaderror(ex)
   warn ""
   warn_title "Missing dependency: #{gem}"
   warn ""
-  warn "     The repl needs this gem, but it's not installed."
+  warn "     The console needs this gem, but it's not installed."
   warn ""
   wrapped "Fix:  gem install #{gem}", "     "
   wrapped "Or:   bundle add #{gem}", "     "
   warn ""
-  warn "     Tip:  If you don't need the repl, you can use the"
+  warn "     Tip:  If you don't need the console, you can use the"
   wrapped "library directly with:  require \"llm\"", "           "
   warn ""
   warn "  ───────────────────────────────────────────────────────"
@@ -120,7 +125,7 @@ def main(argv)
   # Make sure the dependencies are satisified first
   begin
     require "llm/tools"
-    require "llm/repl"
+    require "llm/console"
   rescue LLM::LoadError => ex
     loaderror(ex)
     exit 1
@@ -135,6 +140,9 @@ def main(argv)
       when '-h'
         help
         exit 0
+      when '-v'
+        version
+        exit 0
       when '-t'
         temp = true
       when '-c'
@@ -249,7 +257,7 @@ def main(argv)
     concurrency ||= :sequential
     path  = temp ? nil : data[Dir.getwd]
     agent = LLM::Agent.new(llm, model:, path:, concurrency:, tools: LLM::Tool.subclasses)
-    agent.repl
+    agent.console
   rescue Interrupt
     warn "llm.rb: Bye!"
   rescue => ex