llm.rb 14.0.0 → 15.0.0

data/README.md

@@ -14,155 +14,17 @@
 
 Welcome to the canonical llm.rb repository.
 
-llm.rb is an advanced runtime for building capable AI applications
-on CRuby. It has zero runtime dependencies by default, and a single
-coherent API that spans 12+ providers. Streaming, tools, guards,
-compaction, the REPL, builtin MCP/A2A support and the database
-integrations all build on the same three concepts: providers,
-contexts, and agents.
+llm.rb is an advanced runtime for building agentic AI applications
+on CRuby. It has zero runtime dependencies by default, it supports
+concurrent and parallel tool execution and has a single coherent API
+that spans 13+ providers. Streaming, tools, guards, compaction, the
+REPL, builtin MCP/A2A support and the database integrations all build
+on the same three concepts: providers, contexts, and agents.
 
 Once you learn the fundamentals, everything else falls into place
 naturally. Some features, such as ActiveRecord support, require
 optional dependencies that are opt-in.
 
-## Features
-
-One runtime, 12+ providers. The same API drives OpenAI, Anthropic,
-Google Gemini, Moonshot (kimi), Mistral, DeepSeek, DeepInfra, xAI, Z.ai,
-AWS Bedrock, Ollama, and llama.cpp, so switching models or providers
-can be done with minimal code change.
-
-<details>
-<summary><b>Agents</b></summary>
-
-* **First-class support** <br>
-  llm.rb is designed to build agents. They can be attached to a
-  terminal-based read-eval-print loop (repl), persisted to disk
-  or a database column, run tools concurrently and be safely
-  interrupted.
-
-* **Builtin REPL** <br>
-  A curses-based TUI for talking to an agent interactively. It
-  renders markdown, shows a live status line with context usage
-  and running cost, and recalls previous turns, so a
-  conversation survives a restart.
-
-* **Persistence** <br>
-  Set `path:` and the agent saves its conversation to disk
-  automatically. ActiveRecord and Sequel support keep the same
-  state in a single database column, so you pick the storage
-  and the API stays identical.
-
-</details>
-
-<details>
-<summary><b>MCP &amp; A2A</b></summary>
-
-* **MCP** <br>
-  The Model Context Protocol is first-class. Point an MCP client
-  at any tool server over stdio or HTTP, and its tools translate
-  into local `LLM::Tool` subclasses, with the same tracing and
-  error handling.
-
-* **A2A** <br>
-  The Agent 2 Agent protocol is first-class. Point an A2A client
-  at another agent over HTTP or JSON-RPC, and call its skills
-  exactly like local tools.
-
-</details>
-
-<details>
-<summary><b>ORM</b></summary>
-
-* **ActiveRecord** <br>
-  Add `acts_as_agent` to a model and the agent state lives in a
-  single database column, saved after every turn and restored
-  on load. Works in Rack and Rails apps, with `jsonb` on
-  PostgreSQL.
-
-* **Sequel** <br>
-  Add `plugin :agent` to a Sequel model for the same single-
-  column persistence, with the `pg_json` extension loaded
-  automatically on PostgreSQL.
-
-</details>
-
-<details>
-<summary><b>RAG</b></summary>
-
-* **RAG, out of the box** <br>
-  Embeddings, OCR, and OpenAI's vector stores API come first-
-  class. Ground answers in your own documents, with vectors in
-  a managed store or in your own database such as sqlite-vec
-  or pgvector.
-
-</details>
-
-<details>
-<summary><b>Runtime</b></summary>
-
-* **Streaming** <br>
-  Streaming is first-class, with structured callbacks for
-  content, reasoning, and tool calls. Tools can start while
-  the model is still talking, so the first result lands
-  before the response finishes.
-
-* **Concurrency** <br>
-  Six ways to run tools: sequential, threads, async, fibers,
-  forks, and ractors. Plus three HTTP backends, so you pick
-  the concurrency model that fits the workload, not the other
-  way around.
-
-* **Interruption** <br>
-  Cancel an in-flight request or a running tool at any moment,
-  on any transport or concurrency strategy. A stuck call never
-  leaves a thread running that you can't stop.
-
-</details>
-
-<details>
-<summary><b>Provider extras</b></summary>
-
-* **DeepSeek-optimized** <br>
-  DeepSeek is the most cost-effective option for API users, and the
-  runtime closes its gaps: [`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
-  makes structured outputs work despite no official API, and
-  `images.create`/`edit` produce SVG vector graphics.
-</details>
-
-<details>
-<summary><b>Portable</b></summary>
-
-* **mruby-llm** <br>
-  The same runtime runs on mruby as
-  [mruby-llm](https://github.com/r-uby-dev/mruby-llm), with an
-  almost identical interface and the same set of capabilities.
-
-</details>
-
-<details>
-<summary><b>Everything else</b></summary>
-
-* **Skills** <br>
-  Write a SKILL.md, get a tool. The runtime spawns a
-  disposable subagent with the skill's instructions and tool
-  set for one turn, then discards it. Fresh and stateless
-  every call.
-
-* **A unified plugin family** <br>
-  Compactors, transformers, and guards all share one
-  interface. Context management, message rewriting, and tool
-  supervision (policy, quotas, loop detection) plug in the
-  same way and compose freely.
-
-* **Cost and usage tracking** <br>
-  Every context tracks its own cost and token usage, per turn.
-  Break the spend down by input, output, cache, and reasoning,
-  so the exact cost of any conversation is visible at a
-  glance.
-
-</details>
-
 ## Install
 
 ```bash
@@ -171,7 +33,7 @@ gem install llm.rb
 
 ## Quick start
 
-#### LLM::Agent
+### Agents
 
 The
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
@@ -186,82 +48,82 @@ require "llm"
 
 llm = LLM.deepseek(key: ENV["KEY"])
 agent = LLM::Agent.new(llm, stream: $stdout)
-agent.talk "Hello world"
+agent.talk "hello world"
 ```
+<details>
+<summary>Stream</summary>
+<br>
 
-##### set
+Streams can be simple IO objects or subclasses of
+[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
+with structured callbacks for content,
+reasoning, tool calls, tool returns, and compaction.
+Streams can also observe message transformers, which rewrite
+outgoing messages before they reach the provider.
 
-[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
-is a class-level DSL that accepts a Hash of properties. Each key resolves to a
-corresponding class accessor: `name`, `description`, `model`, `tools`,
-`instructions`, `schema`, `stream`, `tracer`, `concurrency`, `confirm`,
-`path`, `skills`, and `tool_budget`. All options are optional; zero or
-more can be set.
-An error is raised for unknown keys so that typos are caught early.
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/stream/)
+to learn more.
 
 ```ruby
-class SystemAdmin < LLM::Agent
-  set name: "sysadmin",
-      description: "system administration agent",
-      model: "deepseek-v4-pro",
-      tools: [Shell]
-end
+class MyStream < LLM::Stream
+  # Visible assistant output.
+  def on_content(content)
+    print content
+  end
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = SystemAdmin.new(llm)
-agent.talk "Run 'date'"
-```
+  # Reasoning output streamed separately from visible content.
+  def on_reasoning_content(content)
+    warn content
+  end
 
-##### Persistence
+  # A streamed tool call has been fully parsed.
+  def on_tool_call(tool)
+  end
 
-Set `path:` on an agent for automatic filesystem persistence;
-the agent restores conversation history from the file on startup
-and saves it back after every turn, with no manual serialization
-code. For database-backed persistence, ActiveRecord and Sequel
-integrations are also available (see the
-[database deepdive](https://r.uby.dev/llm/deepdive/advanced/database)
-for details). All persistence options use the same underlying
-serialization.
+  # Queued streamed tool work has returned.
+  def on_tool_return(tool, result)
+  end
 
-```ruby
-require "llm"
+  # Before a transformer rewrites an outgoing message.
+  def on_transform(transformer)
+  end
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, path: "session.json")
-agent.talk "remember my name is robert"
+  # Aftter a transformer rewrites an outgoing message.
+  def on_transform_finish(transformer)
+  end
 
-# Next time, the conversation is restored automatically:
-agent = LLM::Agent.new(llm, path: "session.json")
-agent.talk "what's my name?"
-```
+  # Before a compactor trims the conversation.
+  def on_compaction(compactor)
+  end
 
-#### LLM::Context
+  # After a compactor trims the conversation.
+  def on_compaction_finish(compactor)
+  end
 
-The
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
-class is at the heart of the runtime
-and it is what
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-uses under the hood.
-It requires that the tool call loop be managed manually -
-sometimes that can be useful, but usually for advanced use-cases.
-If you're new to llm.rb, try
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) first.
+  # Before a skill's subagent runs.
+  def on_skill_call(skill)
+  end
 
-Every context tracks its own token usage and estimated cost. After any
-turn, you can read the cost breakdown through
-[`LLM::Context#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#cost-instance_method),
-and the REPL shows the running total in the status line.
+  # After a skill's subagent runs.
+  # The subagent that ran it, the skill, and its response are passed
+  # through, so you can introspect the agent, tally skill usage, or
+  # track costs.
+  def on_skill_return(agent, skill, result)
+  end
 
-```ruby
-require "llm"
+  # A request was rate limited and will be retried.
+  def on_rate_limit(error)
+  end
+end
 
 llm = LLM.deepseek(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: $stdout)
-ctx.talk "Hello world"
+agent = LLM::Agent.new(llm, stream: MyStream.new)
+agent.talk "Explain Ruby fibers."
 ```
+</details>
 
-#### LLM::Tool
+<details><summary>Tools</summary>
+<br>
 
 Subclasses of
 [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
@@ -271,9 +133,7 @@ call them on your behalf, and they're one of the most powerful features
 for extending the feature set or abilities of a model.
 
 The runtime also ships with a catalog of built-in tools for
-filesystem, search, and shell operations. See the
-[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/builtin_tools)
-for details.
+filesystem, search, and shell operations. <br> See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/tools) to learn more.
 
 ```ruby
 class ReadFile < LLM::Tool
@@ -286,114 +146,145 @@ class ReadFile < LLM::Tool
     {contents: File.read(path)}
   end
 end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, tools: [ReadFile], stream: $stdout)
+agent.talk "summarize README.md"
 ```
+</details>
+<details>
+<summary>Skills</summary>
+<br>
 
-##### set
+A skill turns a markdown file into a callable tool. When the model
+calls it, the runtime spawns a subagent with the skill's instructions
+as its system prompt and the skill's own tool set. The subagent runs
+one turn and returns the result, then is discarded. Each call
+is fresh and stateless.
 
-[`LLM::Tool.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#set-class_method)
-is an alternative way to define tool properties using a Hash. It works
-the same way as
-[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
-and accepts the same keys that the individual methods do: `name`,
-`description`, `parameters`, `required`, and `defaults`:
+A [LLM::Stream](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
+can be notified as a skill starts and when it returns. The `on_skill_return`
+callback hands back the subagent that ran the skill, so you can inspect
+its conversation, measure its usage, track costs or add a verification
+step (eg `subagent.talk("verify your work")`).
 
-```ruby
-class MathTool < LLM::Tool
-  set name: "math",
-      description: "Performs arithmetic",
-      parameters: [
-        [:x, Integer, "first number" , {required: true}],
-        [:y, Integer, "second number", {default: 0}]
-      ]
-
-  def call(x:, y: 0)
-    {result: x + y}
-  end
-end
-```
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills) to learn more.
 
-#### LLM::Stream
+##### summary.md
 
-Streams can be simple IO objects or subclasses of
-[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
-with structured callbacks for content,
-reasoning, tool calls, tool returns, and compaction.
-Streams can also observe message transformers, which rewrite
-outgoing messages before they reach the provider (see the
-[deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transformer)).
+```markdown
+---
+name: summary
+description: Reads recent git history and writes a summary
+tools: all
+---
 
-```ruby
-class MyStream < LLM::Stream
-  def on_content(content)
-    print content
-  end
+Collect the recent git log, analyze each commit,
+and write a summary to summary.txt.
+```
 
-  def on_reasoning_content(content)
-    warn content
-  end
-end
+##### agent.rb
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, stream: MyStream.new)
-agent.talk "Explain Ruby fibers."
+```ruby
+require "llm"
+
+llm   = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, skills: ["summary.md"])
+agent.talk "Summarize the last week of work"
 ```
+</details>
 
-#### LLM::Schema
+<details>
+<summary>Concurrency</summary>
+<br>
 
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
-subclasses produce typed, structured
-output from any model call. Pass a schema to
-[`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk-instance_method),
-[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method),
-or
-[`LLM::Provider#complete`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#complete-instance_method)
-to receive validated JSON instead of free text. Schemas work alongside tools and streams.
+The runtime supports six different concurrency strategies that have
+different attributes. The choice between all of them often depends
+on the requirements of your application.
 
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
-can define objects, arrays, enums, nested schemas,
-and more. It is also used internally by
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
-definitions, so you already benefit from it when you declare tool
-parameters.
+IO-bound tools are a good fit for the `:async`, `:thread`,
+and `:fiber` strategies while true parallelism can be achieved
+with the `:fork` and `:ractor` strategies. The
+`:sequential` strategy runs tools one at a time and is the default.
+The `:fork` strategy also provides a separate process that offers
+isolation from its parent.
 
-The
-[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
-provider includes runtime-level optimisations such as structured
-output support (despite no official structured outputs API) and
-SVG image generation. This example uses
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) with
-DeepSeek:
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/concurrency) to learn more.
 
 ```ruby
-class Weather < LLM::Schema
-  property :city, String, "The city name"
-  property :temperature, Float, "Current temperature"
-  property :conditions, String, "Weather conditions"
-  required %i[city temperature conditions]
-end
+require "llm"
+require "llm/tools"
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, schema: Weather)
-res = agent.talk "Weather in Paris?"
-res.content!  # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
+llm   = LLM.deepseek(key: ENV["KEY"])
+tools = LLM::Tool.subclasses
+agent = LLM::Agent.new(llm, tools:, concurrency: :fork)
+agent.talk "Run the tools in parallel"
 ```
 
-#### LLM::REPL
+</details>
+<details>
+<summary>Cancellation</summary>
+<br>
+
+Abort a request mid-stream and interrupt any running tools with
+[`LLM::Agent#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#interrupt!)
+(or `cancel!`), from any thread. The runtime raises
+[`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
+on the caller and on every active tool. A forked tool gets interrupted over
+the control channel, a ractor via message passing, and pending tools
+are stopped before they run. The in-flight HTTP request is closed
+too, so a turn you no longer want stops without burning tokens.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/cancellation) to learn more.
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm)
+Thread.new { sleep(1); agent.cancel! }
+
+begin
+  agent.talk "write a very long poem", stream: $stdout
+rescue LLM::Interrupt
+  puts "cancelled"
+end
+```
+</details>
+<details>
+<summary>Console (<code>binding.pry</code> for agents)</summary>
+<br>
 
 The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
-method drops you into a curses-based TUI for talking to an
-agent interactively. It renders markdown directly in the
-terminal and shows a live status line with context usage,
-running cost, and the current tool call. A second thread keeps
-the UI responsive while the model works. Think of it as
+method drops you into a highly capable read-eval-print loop (REPL)
+that is built on top of curses. It can help you debug agents,
+test your tools, connect to MCP servers, and even A2A agents.
+The REPL stands out because it connects to the surrounding
+runtime and it can be extended by your code. Think of it as
 `binding.pry` but for agents.
 
-Set `path:` on the agent for automatic persistence across REPL
-sessions. The `tools:` option attaches extra tools for the
-duration of the session. Recall previous turns with Ctrl+P and
-Ctrl+N. For the full reference, see the
-[REPL section](https://r.uby.dev/llm/deepdive/fundamentals/repl) in the
-deepdive.
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/repl) to learn more.
+
+##### Demo
+
+[Watch in high quality on asciinema](https://asciinema.org/a/OsS8wwaasKasoDDz)
+
+![llm.rb REPL demo](demo.gif)
+
+
+##### Installation
+
+The REPL is distributed with llm.rb so you don't have to install
+a separate gem but it requires a number of optional dependencies
+to be installed separately. The following gems provide the full
+experience:
+
+    gem install curses kramdown xchan.rb test-cmd.rb
+
+##### Persistence
+
+The `path:` option can be set on an agent for automatic persistence
+across REPL sessions. The `tools:` option attaches extra tools
+for the duration of the session. Recall previous turns with Ctrl+P and
+Ctrl+N.
 
 ```ruby
 require "llm"
@@ -407,20 +298,110 @@ agent.repl(tools: LLM::Tool.subclasses)
 ##### CLI
 
 The `llm.rb` executable is available on your PATH after installation.
-It starts a REPL session from any directory:
+It starts a REPL session from any directory.The CLI auto-detects your
+provider from standard environment variables (`DEEPSEEK_API_KEY`,
+`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.). Persistent sessions are
+stored under `~/.llm.rb/` and restored automatically on your next visit.
 
 ```bash
 llm.rb                     # auto-detect from $DEEPSEEK_API_KEY
 llm.rb -p openai           # use OpenAI explicitly
 llm.rb -t                  # temporary session, no persistence
 ```
+</details>
+<details>
+<summary>Persistence</summary>
+<br>
+
+Set `path:` on an agent for automatic filesystem persistence:
+the agent restores conversation history from the file on startup
+and saves it back after every turn, with no manual serialization
+code. For database-backed persistence, ActiveRecord and Sequel
+integrations are also available. All persistence options use the same
+underlying serialization.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/database) to learn more.
+
+```ruby
+require "llm"
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, path: "session.json")
+agent.talk "remember my name is robert"
+
+# Next time, the conversation is restored automatically:
+agent = LLM::Agent.new(llm, path: "session.json")
+agent.talk "what's my name?"
+```
+</details>
+<details><summary>ActiveRecord | Sequel</summary>
+<br>
+
+Because both
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+can be serialized to JSON and stored in a simple string, both ActiveRecord
+and Sequel support can be implemented within a single column on a single row.
+
+The runtime includes first-class support for both ActiveRecord / Sequel, and
+for both Rack-based / Rails-based applications. On databases
+where it is supported, such as PostgreSQL, the column can be optimized by using
+the `jsonb` type.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/database) to learn more.
+
+```ruby
+require "active_record"
+require "llm"
+require "llm/active_record"
+
+class Email < ApplicationRecord
+  acts_as_agent do |agent|
+    agent.set name: "mail",
+              instructions: "Write concise, friendly replies to emails",
+              model: "deepseek-v4-pro"
+  end
+
+  def draft_reply!
+    talk("Draft a reply to:\n\n#{body}")
+  end
+
+  def summarize
+    talk("Summarize this email thread in a few sentences")
+  end
+
+  private
+
+  ##
+  # By convention, this method defines the provider for a model.
+  # If necessary, it can be renamed with: provider: :your_method.
+  def set_provider
+    LLM.deepseek(key: ENV["KEY"])
+  end
+
+  ##
+  # By convention, this method returns the context options given
+  # to LLM::Context or LLM::Agent. This method can be left undefined.
+  def set_context
+    {}
+  end
+end
 
-The CLI auto-detects your provider from standard environment variables
-(`DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.).
-Persistent sessions are stored under `~/.llm.rb/` and restored
-automatically on your next visit.
+email = Email.create!(subject: "Streaming support", body: "How do I stream responses?")
+email.draft_reply!
+
+##
+# The conversation (the email and the draft
+# reply) is persisted to the email's column. A
+# fresh instance restores it and continues the
+# thread, so the summary below knows what was
+# already drafted:
+Email.find(email.id).summarize
+```
+</details>
 
-#### LLM::MCP
+<details><summary>MCP</summary>
+<br>
 
 The Model Context Protocol (MCP) has first-class support
 in llm.rb. The stdio and http transports work out of the
@@ -430,6 +411,9 @@ used with
 [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
 
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/protocols/mcp/), and the
+[deepdive.md on persistent connections](https://r.uby.dev/llm/deepdive/features/transports) to learn more.
+
 ```ruby
 require "llm"
 
@@ -438,24 +422,9 @@ mcp   = LLM::MCP.stdio(argv: ["ruby", "server.rb"])
 agent = LLM::Agent.new(llm, stream: $stdout, tools: mcp.tools)
 agent.talk "Run the tool"
 ```
-
-##### Persistent connections
-
-Set `persistent: true` on HTTP transports to reuse connections
-across requests. This uses
-[`Net::HTTP::Persistent`](https://github.com/drbrain/net-http-persistent)
-under the hood and avoids opening a new TCP connection for every
-request:
-
-```ruby
-mcp = LLM::MCP.http(
-  url: "https://api.githubcopilot.com/mcp/",
-  headers: {"Authorization" => "Bearer #{ENV.fetch('GITHUB_PAT')}"},
-  persistent: true
-)
-```
-
-#### LLM::A2A
+</details>
+<details><summary>A2A</summary>
+<br>
 
 The Agent 2 Agent (A2A) protocol has first-class support
 in llm.rb. The http and jsonrpc transports work out of the
@@ -465,6 +434,9 @@ used with
 [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
 
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/protocols/a2a/), and the
+[deepdive.md on persistent connections](https://r.uby.dev/llm/deepdive/features/transports) to learn more.
+
 ```ruby
 require "llm"
 
@@ -473,21 +445,51 @@ a2a   = LLM::A2A.rest(url: "https://remote-agent.example.com")
 agent = LLM::Agent.new(llm, stream: $stdout, tools: a2a.skills)
 agent.talk "Run the skill"
 ```
+</details>
+
+<details><summary>Structured outputs</summary>
+<br>
+
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
+subclasses produce typed, structured
+output from any model call. Pass a schema to
+[`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk-instance_method),
+[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method),
+or
+[`LLM::Provider#complete`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#complete-instance_method)
+to receive validated JSON instead of free text. Schemas work alongside tools and streams.
 
-##### Persistent connections
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
+can define objects, arrays, enums, nested schemas,
+and more. It is also used internally by
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
+definitions, so you already benefit from it when you declare tool
+parameters.
 
-Set `persistent: true` on HTTP transports to reuse connections
-across requests. This uses
-[`Net::HTTP::Persistent`](https://github.com/drbrain/net-http-persistent)
-under the hood and avoids opening a new TCP connection for every
-request:
+The
+[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
+provider includes runtime-level optimisations such as structured
+output support (despite no official structured outputs API) and
+SVG image generation. This example uses
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) with
+DeepSeek:
 
 ```ruby
-a2a = LLM::A2A.rest(url: "https://agent.example.com", persistent: true)
-a2a = LLM::A2A.jsonrpc(url: "https://agent.example.com", persistent: true)
-```
+class Weather < LLM::Schema
+  property :city, String, "The city name"
+  property :temperature, Number, "Current temperature"
+  property :conditions, String, "Weather conditions"
+  required %i[city temperature conditions]
+end
 
-#### LLM::Guard
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, schema: Weather)
+res = agent.talk "Weather in Paris?"
+res.content!  # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
+```
+</details>
+<details><summary>Guards</summary>
+<br>
 
 [`LLM::Guard`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html)
 is the hook that sees every tool call before it runs. A guard
@@ -507,6 +509,8 @@ and implement
 The pending call arrives as `function:`. Return a value to close
 the call, or `nil` to let it run:
 
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/guard) to learn more.
+
 ```ruby
 class PolicyGuard < LLM::Guard
   def call(function:)
@@ -517,141 +521,285 @@ class PolicyGuard < LLM::Guard
   end
 end
 
-agent = LLM::Agent.new(llm, guard: PolicyGuard)
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, tools: [Shell, ReadFile], guard: PolicyGuard)
 ```
+</details>
 
-#### LLM::Skill
+<details>
+<summary>Transformers</summary>
+<br>
 
-A skill turns a markdown file into a callable tool. When the model
-calls it, the runtime spawns a subagent with the skill's instructions
-as its system prompt and the skill's own tool set. The subagent runs
-one turn and returns the result, then is discarded. Each call
-is fresh and stateless. For a deeper explanation see the
-[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills).
+It is possible to rewrite outgoing messages before they reach the provider with
+[`LLM::Transformer`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer.html).
+Create a subclass and implement `call(message:)` to scrub sensitive data,
+inject context, or normalize content. The transform runs automatically
+on every turn, so you never have to change your prompt code.
 
-##### SKILL.md
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transformer) to learn more.
 
-```markdown
----
-name: summary
-description: Reads recent git history and writes a summary
-tools: all
----
+```ruby
+class RedactEmails < LLM::Transformer
+  def call(message:)
+    content = message.content.to_s.gsub(/[\w.+-]+@[\w-]+\.[\w.]+/, "[EMAIL]")
+    LLM::Message.new(message.role, content, message.extra)
+  end
+end
 
-Collect the recent git log, analyze each commit,
-and write a summary to summary.txt.
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, transformer: RedactEmails)
+agent.talk "Contact support@example.com for help"
 ```
+</details>
 
-##### agent.rb
+<details>
+<summary>Compactors</summary>
+<br>
+
+Every model has a context window: the finite number of tokens it can
+consider in a single request. Generally a compactor will drop or
+summarize older messages to keep the conversation within that window,
+and it runs automatically before every turn. By default it is disabled
+so it is a feature you must opt into.
+
+[`LLM::Compactor::Truncate`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor/Truncate.html)
+keeps the most recent messages via an integer count or a percentage like
+`"80%"`. It preserves tool call and return pairs so the conversation
+never contains an orphaned result. It is also possible to subclass
+[`LLM::Compactor`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor.html)
+to implement your own compactor with its own logic. Streams can observe the
+process through the
+[`LLM::Stream#on_compaction`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction)
+and
+[`LLM::Stream#on_compaction_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction_finish)
+callbacks.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/compaction) to learn more.
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(
+  llm,
+  compactor: LLM::Compactor::Truncate,
+  compactor_options: {keep: 64}
+)
+agent.talk "Hello"
+```
+</details>
+
+<details>
+<summary>Automatic retries</summary>
+<br>
+
+Rate-limited requests are retried automatically by default. Agents
+retry a 429 up to five times with a growing backoff before giving
+up, so most request failures resolve on their own. Set `retry_budget`
+to change the number of retries, or `retry_budget: 0` to disable
+them.
 
 ```ruby
 require "llm"
 
-llm   = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, skills: ["./skills/summary"])
-agent.talk "Summarize the last week of work"
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, retry_budget: 0)
+agent.talk "Hello"
 ```
 
-#### RAG
+</details>
+
 
-Most providers offer an embedding model that can be
-used for semantic search, or similarity search. An
-embedding model can generate embeddings that can then
-be stored in a database that is optimized for storing
-and querying vectors, such as SQLite's [sqlite-vec](https://github.com/asg017/sqlite-vec)
-or PostgreSQL's [pg-vector](https://github.com/pgvector/pgvector).
+<details>
+<summary>Observability</summary>
+<br>
 
-llm.rb also includes support for OpenAI's vector store API. It
-provides a vector database as a HTTP service but we won't cover
-that here. For a deeper explanation see the
-[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/embeddings).
+Trace what an agent is doing by attaching a tracer. Hook into
+requests, tool calls, and other runtime events to debug a
+misbehaving agent, monitor latency, or export spans to an
+observability backend. All built-in tracers share one interface,
+so switching between them means changing a class name:
+
+* [`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html): human-readable single-line logs to stderr, ideal during development.
+* [`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html):
+exports spans via OTLP for OpenTelemetry in production.
+* [`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html):
+structured JSON to stdout or a file.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/reference/tracer) to learn more.
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, tracer: LLM::Tracer::PrettyLogger.new(llm))
+agent.talk "Hello"
+```
+</details>
+
+<details>
+<summary>As a subclass</summary>
+<br>
+
+[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
+is a class-level DSL that accepts a Hash of properties. Each key resolves to a
+corresponding class accessor: `name`, `description`, `model`, `tools`,
+`instructions`, `schema`, `stream`, `tracer`, `concurrency`, `confirm`,
+`path`, `skills`, `tool_budget`, and `retry_budget`. All options are
+optional; zero or more can be set.
+An error is raised for unknown keys so that typos are caught early.
 
 ```ruby
 require "llm"
+require "llm/tools"
 
-llm  = LLM.openai(key: ENV["KEY"])
-body = "llm.rb is Ruby's capable AI runtime."
-embedding = llm.embed([body]).embeddings.first
+class Agent < LLM::Agent
+  set name: "sysadmin",
+      description: "system administration agent",
+      model: "deepseek-v4-pro",
+      tools: [LLM::Tool::Shell]
+end
 
-# Document is your ActiveRecord or Sequel model
-# with a vector column (e.g. sqlite-vec or pgvector)
-Document.create!(
-  title: "llm.rb",
-  body:,
-  embedding:,
-)
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = Agent.new(llm)
+agent.talk "Run 'date'"
 ```
+</details>
 
-#### Concurrency
+### Providers
 
-The runtime supports six different concurrency strategies that have
-different attributes. The choice between all of them often depends
-on the requirements of your application.
+Each provider is constructed with a class-level factory method on
+`LLM`, and the resulting instance is passed to
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
+or
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html). The
+same API drives every one of them, so switching models is a one-line
+change. See the [deepdive](https://r.uby.dev/llm/deepdive/fundamentals/providers)
+for a full provider reference.
+
+#### What providers does llm.rb support?
+
+* **Anthropic** (`LLM.anthropic`)
+* **Google** (`LLM.google`)
+* **OpenAI** (`LLM.openai`)
+* **DeepSeek** (`LLM.deepseek`)
+* **DeepInfra** (`LLM.deepinfra`)
+* **xAI** (`LLM.xai`)
+* **Z.ai** (`LLM.zai`)
+* **Moonshot (Kimi)** (`LLM.moonshot`)
+* **Alibaba (Qwen3)** (`LLM.alibaba`, also `LLM.aliyun`)
+* **Mistral** (`LLM.mistral`)
+* **AWS Bedrock** (`LLM.bedrock`)
+* **Ollama** (`LLM.ollama`)
+* **llama.cpp** (`LLM.llamacpp`)
 
-IO-bound tools are a good fit for the `:async`, `:thread`,
-and `:fiber` strategies while true parallelism can be achieved
-with the `:fork` and `:ractor` strategies. The
-`:sequential` strategy runs tools one at a time and is the default.
-The `:fork` strategy also provides a separate process that offers
-isolation from its parent.
+<details>
+<summary>Implicit</summary>
+<br>
 
-You can learn more about the llm.rb concurrency model in the
-[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/concurrency).
+Cloud providers can infer their API key automatically
+from a set of common defaults that are defined by
+the [models.dev](https://models.dev) registry that
+is also distributed with llm.rb.
 
 ```ruby
-require "llm"
+llm = LLM.openai
+llm = LLM.anthropic
+llm = LLM.deepseek
+llm = LLM.alibaba  # also: LLM.aliyun
+llm = LLM.moonshot
+llm = LLM.mistral
+```
+</details>
+<details>
+<summary>Explicit</summary>
+<br>
 
-llm   = LLM.deepseek(key: ENV["KEY"])
-tools = [FetchNews, FetchStocks, FetchFeeds]
-agent = LLM::Agent.new(llm, tools:, concurrency: :fork)
-agent.talk "Run the tools in parallel"
+The `key` option can also be providied explicitly, and certain
+providers (eg ollama, llamacpp) usually do not require an API
+key at all.
+
+```ruby
+llm = LLM.openai(key: ENV["OPENAI_API_KEY"])
+llm = LLM.anthropic(key: ENV["ANTHROPIC_API_KEY"])
+llm = LLM.deepseek(key: ENV["DEEPSEEK_API_KEY"])
+llm = LLM.alibaba(key: ENV["DASHSCOPE_API_KEY"]) # also: LLM.aliyun
+llm = LLM.moonshot(key: ENV["MOONSHOT_API_KEY"])
+llm = LLM.mistral(key: ENV["MISTRAL_API_KEY"])
 ```
+</details>
 
-#### ORM
+<details>
+<summary>Model Registry</summary>
+<br>
 
-Because both
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-can be serialized to JSON and stored in a simple string, both ActiveRecord
-and Sequel support can be implemented within a single column on a single row.
+Each provider ships its model catalog, pricing, limits, and
+modalities with the gem, sourced from [models.dev](https://models.dev).
+Reach it from any provider, context, or agent, enumerate models, or
+sort them by price.
 
-The runtime includes first-class support for both ActiveRecord *and* Sequel, and
-for both Rack-based applications *and* Rails-based applications. On databases
-where it is supported, such as PostgreSQL, the column can be optimized by using
-the `jsonb` type.
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/reference/model_registry) to learn more.
 
 ```ruby
-require "active_record"
 require "llm"
-require "llm/active_record"
 
-class Agent < ApplicationRecord
-  acts_as_agent
-  set name: "my-agent",
-      instructions: "solve the user's query",
-      model: "deepseek-v4-pro",
-      tools: [Research, FinalizeResearch, ActOnResearch]
+llm      = LLM.openai
+registry = llm.registry                # => LLM::Provider#registry
+cheapest = registry.models.sort.first  # => LLM::Model
+cheapest.id                            # => "text-embedding-3-small"
+cheapest.context_window                # => 8191
+cheapest.structured_output?            # => false
+```
+</details>
 
-  private
+<details>
+<summary>Transports</summary>
+<br>
 
-  # By convention, this method defines the provider for a model.
-  # If necessary, it can be renamed with: provider: :your_method.
-  def set_provider
-    LLM.deepseek(key: ENV["KEY"])
-  end
+The `transport:` option selects which HTTP library a provider uses for
+network communication. Three backends ship out of the box: `net/http`
+is always available and the default, `net/http/persistent` pools
+connections for many requests to the same host, and `curb` wraps
+libcurl. They share one interface, so switching is a one-word change.
 
-  # By convention, this method returns the context options given
-  # to LLM::Context or LLM::Agent.
-  def set_context
-    {}
-  end
-end
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transports) to learn more.
 
-agent = Agent.create!
-agent.talk "perform research"
+```ruby
+llm = LLM.deepseek(
+  key: ENV["KEY"],
+  transport: :net_http_persistent
+)
+```
+</details>
+
+### RAG
+
+Most providers offer an embedding model that can be
+used for semantic search, or similarity search. An
+embedding model can generate embeddings that can then
+be stored in a database that is optimized for storing
+and querying vectors, such as SQLite's [sqlite-vec](https://github.com/asg017/sqlite-vec)
+or PostgreSQL's [pg-vector](https://github.com/pgvector/pgvector).
+
+llm.rb also includes support for OpenAI's vector store API. It
+provides a vector database as a HTTP service but we won't cover
+that here.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/embeddings) to learn more.
+
+```ruby
+require "llm"
+
+llm  = LLM.openai(key: ENV["KEY"])
+body = "llm.rb is Ruby's capable AI runtime."
+embedding = llm.embed([body]).embeddings.first
+
+# Document is your ActiveRecord or Sequel model
+# with a vector column (e.g. sqlite-vec or pgvector)
+Document.create!(
+  title: "llm.rb",
+  body:,
+  embedding:,
+)
 ```
 
-#### Images
+### Images
 
 A handful of providers can generate images from a text prompt.
 OpenAI, Google, xAI, and DeepInfra all support it. The API is
@@ -692,46 +840,16 @@ IO.copy_stream res.images[0], "rocket-with-dog.svg"
 ## FAQ
 
 <details>
-<summary>What providers does llm.rb support?</summary>
+<summary>What about local LLM support?</summary>
 <br>
 <p>
-
-**Cloud**
-
-The following cloud-based providers are available to choose from. <br>
-In no particular order:
-
-🇺🇸 OpenAI <br>
-🇺🇸 DeepInfra <br>
-🇺🇸 xAI <br>
-🇺🇸 Google (Gemini) <br>
-🇺🇸 AWS bedrock <br>
-🇺🇸 Anthropic <br>
-🇨🇳 DeepSeek <br>
-🇨🇳 zAI <br>
-🇨🇳 Moonshot AI (Kimi) <br>
-🇪🇺 Mistral <br>
-
-**Weights**
-
-The following providers provide access to open-weight models. <br>
-In no particular order:
-
-🇺🇸 DeepInfra <br>
-🇺🇸 AWS bedrock <br>
-🇨🇳 DeepSeek <br>
-🇨🇳 zAI <br>
-🇨🇳 Moonshot AI (Kimi) <br>
-🇪🇺 Mistral <br>
-
-**Local**
-
-The following providers can be run locally on your own hardware. <br>
-In no particular order:
+The following providers can be run used with models that
+are running on your own hardware. They're reasonably well
+tested but not my main driver:
+</p>
 
 * Ollama
 * Llamacpp
-</p>
 </details>
 
 <details>
@@ -763,11 +881,10 @@ type.
 If you're on a budget, DeepSeek is hard to beat.
 </details>
 <details>
-<summary>Can I download llm.rb via a decentralized network?</summary>
-<br>
-Yes.
+<summary>Sources other than GitHub?</summary>
 <br>
-We are on the <a href="https://radicle.network">radicle.network</a>
+<p>
+We are on the <a href="https://radicle.network">radicle.network</a> as well.
 <br>
 Every commit that lands on GitHub also lands on Radicle.
 <br>
@@ -776,6 +893,30 @@ Our repository ID is z2PtfQ6dYwyYaW2aGrztG1sMyDmCE.
 Browse on <a
 href="https://radicle.network/nodes/iris.radicle.network/z2PtfQ6dYwyYaW2aGrztG1sMyDmCE">the
 web</a>.
+</p>
+</details>
+
+<details>
+<summary>Who maintains llm.rb?</summary>
+<br>
+
+The llm.rb project is maintained primarily by one
+person. llm.rb has been in active development for more
+than three years and over that time multiple other
+contributors have contributed to llm.rb as well. New
+contributors are always welcome.
+
+I use the repl that is distributed with llm.rb to build
+llm.rb itself so there is a healthy feedback loop and
+llm.rb has also been battle tested in production
+environments.
+
+I have also also written llm.rb agents within the
+repository that help me maintain the documentation,
+and backport changes to the mruby-llm runtime as well.
+
+I am constantly focused on improving llm.rb by using
+it as my primary driver for development.
 </details>
 
 ## Resources
@@ -786,41 +927,6 @@ wasn't possible to cover every feature without the README becoming a small book.
 The [r.uby.dev](https://r.uby.dev) homepage also includes more learning material
 and resources.
 
-## Developers
-
-The llm.rb project is quite large and maintained primarily by one
-person. It would be near impossible for me to maintain both the codebase
-and its documentation, especially the [deepdive.md](https://r.uby.dev/llm/deepdive/)
-so I have written agents that maintain the documentation assets and that
-allows me to put more focus on the code.
-
-The following agents are available for those tasks, and all of them
-use the most cost effective option: DeepSeek. Feel free to use them
-in your own fork.
-
-```sh
-##
-# Maintains the deepdive and API docs
-rake agents:scribe:yardoc
-rake agents:scribe:coverage
-rake agents:scribe:regressions
-rake agents:scribe:style
-
-##
-# Maintains the release
-rake agents:dexter:changelog
-rake agents:dexter:release
-
-##
-# Maintains mruby-llm backports
-rake agents:mruby:research
-rake agents:mruby:implement
-
-##
-# Refresh the data/ registry
-rake models.dev:download
-```
-
 ## License
 
 This software is released under the terms of the MIT license. <br>