llm.rb 13.1.0 → 15.0.0

data/README.md

@@ -14,32 +14,17 @@
 
 Welcome to the canonical llm.rb repository.
 
-llm.rb is an advanced runtime for building capable AI applications
-on CRuby. By default it has zero runtime dependencies although certain
-functionality (such as ActiveRecord support) require
+llm.rb is an advanced runtime for building agentic AI applications
+on CRuby. It has zero runtime dependencies by default, it supports
+concurrent and parallel tool execution and has a single coherent API
+that spans 13+ providers. Streaming, tools, guards, compaction, the
+REPL, builtin MCP/A2A support and the database integrations all build
+on the same three concepts: providers, contexts, and agents.
+
+Once you learn the fundamentals, everything else falls into place
+naturally. Some features, such as ActiveRecord support, require
 optional dependencies that are opt-in.
 
-When you want to learn more than what the README covers, checkout
-the [deepdive.md](https://r.uby.dev/llm/deepdive/).
-
-## Features
-
-The runtime supports OpenAI, OpenAI-compatible endpoints, Anthropic, Google
-Gemini, Mistral, DeepSeek, DeepInfra, xAI, Z.ai, AWS Bedrock, Ollama, and llama.cpp.
-It has first-class support for streaming, tool calls,  MCP
-and A2A, embeddings, vector stores, OCR, context compaction,
-and the RAG pattern.
-
-There are multiple HTTP backends to choose from, tools can be run concurrently
-or in parallel via threads, async tasks, fibers, ractors, and fork, and it is
-also possible to make a tool call while the model is still streaming.
-
-The runtime builds on top of three core concepts: providers, contexts, and agents,
-so once you learn the fundamentals, everything else falls into place naturally. And once
-you learn llm.rb, you will also be able to use
-<a href="https://r.uby.dev/mruby-llm">mruby-llm</a> and
-<a href="https://r.uby.dev/wasm-llm">wasm-llm</a> because the API is pretty much identical.
-
 ## Install
 
 ```bash
@@ -48,90 +33,97 @@ gem install llm.rb
 
 ## Quick start
 
-#### LLM::Agent
+### Agents
 
 The
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
 class is the default high-level interface,
 and it is recommended for most use-cases. It manages tool execution
-automatically, guards against infinite loops, manages conversation
-state, and much more.
+automatically and
+[guards against infinite loops](https://r.uby.dev/llm/deepdive/advanced/guard),
+manages conversation state, and much more.
 
 ```ruby
 require "llm"
 
 llm = LLM.deepseek(key: ENV["KEY"])
 agent = LLM::Agent.new(llm, stream: $stdout)
-agent.talk "Hello world"
+agent.talk "hello world"
 ```
+<details>
+<summary>Stream</summary>
+<br>
 
-##### set
+Streams can be simple IO objects or subclasses of
+[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
+with structured callbacks for content,
+reasoning, tool calls, tool returns, and compaction.
+Streams can also observe message transformers, which rewrite
+outgoing messages before they reach the provider.
 
-[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
-is a class-level DSL that accepts a Hash of properties. Each key resolves to a
-corresponding class accessor: `name`, `description`, `model`, `tools`,
-`instructions`, `schema`, `stream`, `tracer`, `concurrency`, `confirm`,
-`path`, and `skills`. All options are optional; zero or more can be set.
-An error is raised for unknown keys so that typos are caught early.
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/stream/)
+to learn more.
 
 ```ruby
-class SystemAdmin < LLM::Agent
-  set name: "sysadmin",
-      description: "system administration agent",
-      model: "deepseek-v4-pro",
-      tools: [Shell]
-end
+class MyStream < LLM::Stream
+  # Visible assistant output.
+  def on_content(content)
+    print content
+  end
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = SystemAdmin.new(llm)
-agent.talk "Run 'date'"
-```
+  # Reasoning output streamed separately from visible content.
+  def on_reasoning_content(content)
+    warn content
+  end
 
-##### Persistence
+  # A streamed tool call has been fully parsed.
+  def on_tool_call(tool)
+  end
 
-Set `path:` on an agent for automatic filesystem persistence;
-the agent restores conversation history from the file on startup
-and saves it back after every turn, with no manual serialization
-code. For database-backed persistence, ActiveRecord and Sequel
-integrations are also available (see the
-[database deepdive](https://r.uby.dev/llm/deepdive/advanced/database)
-for details). All persistence options use the same underlying
-serialization.
+  # Queued streamed tool work has returned.
+  def on_tool_return(tool, result)
+  end
 
-```ruby
-require "llm"
+  # Before a transformer rewrites an outgoing message.
+  def on_transform(transformer)
+  end
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, path: "session.json")
-agent.talk "remember my name is robert"
+  # Aftter a transformer rewrites an outgoing message.
+  def on_transform_finish(transformer)
+  end
 
-# Next time, the conversation is restored automatically:
-agent = LLM::Agent.new(llm, path: "session.json")
-agent.talk "what's my name?"
-```
+  # Before a compactor trims the conversation.
+  def on_compaction(compactor)
+  end
 
-#### LLM::Context
+  # After a compactor trims the conversation.
+  def on_compaction_finish(compactor)
+  end
 
-The
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
-class is at the heart of the runtime
-and it is what
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-uses under the hood.
-It requires that the tool call loop be managed manually -
-sometimes that can be useful, but usually for advanced use-cases.
-If you're new to llm.rb, try
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) first.
+  # Before a skill's subagent runs.
+  def on_skill_call(skill)
+  end
 
-```ruby
-require "llm"
+  # After a skill's subagent runs.
+  # The subagent that ran it, the skill, and its response are passed
+  # through, so you can introspect the agent, tally skill usage, or
+  # track costs.
+  def on_skill_return(agent, skill, result)
+  end
+
+  # A request was rate limited and will be retried.
+  def on_rate_limit(error)
+  end
+end
 
 llm = LLM.deepseek(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: $stdout)
-ctx.talk "Hello world"
+agent = LLM::Agent.new(llm, stream: MyStream.new)
+agent.talk "Explain Ruby fibers."
 ```
+</details>
 
-#### LLM::Tool
+<details><summary>Tools</summary>
+<br>
 
 Subclasses of
 [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
@@ -140,6 +132,9 @@ an optional set of typed parameters. <br> The model can choose to
 call them on your behalf, and they're one of the most powerful features
 for extending the feature set or abilities of a model.
 
+The runtime also ships with a catalog of built-in tools for
+filesystem, search, and shell operations. <br> See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/tools) to learn more.
+
 ```ruby
 class ReadFile < LLM::Tool
   name "read-file"
@@ -151,78 +146,145 @@ class ReadFile < LLM::Tool
     {contents: File.read(path)}
   end
 end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, tools: [ReadFile], stream: $stdout)
+agent.talk "summarize README.md"
 ```
+</details>
+<details>
+<summary>Skills</summary>
+<br>
 
-#### LLM::Stream
+A skill turns a markdown file into a callable tool. When the model
+calls it, the runtime spawns a subagent with the skill's instructions
+as its system prompt and the skill's own tool set. The subagent runs
+one turn and returns the result, then is discarded. Each call
+is fresh and stateless.
 
-Streams can be simple IO objects or subclasses of
-[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
-with structured callbacks for content,
-reasoning, tool calls, tool returns, and compaction.
+A [LLM::Stream](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
+can be notified as a skill starts and when it returns. The `on_skill_return`
+callback hands back the subagent that ran the skill, so you can inspect
+its conversation, measure its usage, track costs or add a verification
+step (eg `subagent.talk("verify your work")`).
 
-```ruby
-class MyStream < LLM::Stream
-  def on_content(content)
-    print content
-  end
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills) to learn more.
 
-  def on_reasoning_content(content)
-    warn content
-  end
-end
+##### summary.md
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, stream: MyStream.new)
-agent.talk "Explain Ruby fibers."
+```markdown
+---
+name: summary
+description: Reads recent git history and writes a summary
+tools: all
+---
+
+Collect the recent git log, analyze each commit,
+and write a summary to summary.txt.
 ```
 
-#### LLM::Schema
+##### agent.rb
 
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
-subclasses produce typed, structured
-output from any model call. Pass a schema to `LLM::Context#talk`,
-`LLM::Agent#talk`, or `LLM::Provider#complete` to receive validated
-JSON instead of free text. Schemas work alongside tools and streams.
+```ruby
+require "llm"
 
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
-can define objects, arrays, enums, nested schemas,
-and more. It is also used internally by
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
-definitions, so you already benefit from it when you declare tool
-parameters.
+llm   = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, skills: ["summary.md"])
+agent.talk "Summarize the last week of work"
+```
+</details>
 
-The
-[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
-provider includes runtime-level optimisations such as structured
-output support (despite no official structured outputs API) and
-SVG image generation. This example uses
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) with
-DeepSeek:
+<details>
+<summary>Concurrency</summary>
+<br>
+
+The runtime supports six different concurrency strategies that have
+different attributes. The choice between all of them often depends
+on the requirements of your application.
+
+IO-bound tools are a good fit for the `:async`, `:thread`,
+and `:fiber` strategies while true parallelism can be achieved
+with the `:fork` and `:ractor` strategies. The
+`:sequential` strategy runs tools one at a time and is the default.
+The `:fork` strategy also provides a separate process that offers
+isolation from its parent.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/concurrency) to learn more.
 
 ```ruby
-class Weather < LLM::Schema
-  property :city, String, "The city name"
-  property :temperature, Float, "Current temperature"
-  property :conditions, String, "Weather conditions"
-  required %i[city temperature conditions]
-end
+require "llm"
+require "llm/tools"
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, schema: Weather)
-res = agent.talk "Weather in Paris?"
-res.content!  # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
+llm   = LLM.deepseek(key: ENV["KEY"])
+tools = LLM::Tool.subclasses
+agent = LLM::Agent.new(llm, tools:, concurrency: :fork)
+agent.talk "Run the tools in parallel"
 ```
 
-#### LLM::REPL
+</details>
+<details>
+<summary>Cancellation</summary>
+<br>
+
+Abort a request mid-stream and interrupt any running tools with
+[`LLM::Agent#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#interrupt!)
+(or `cancel!`), from any thread. The runtime raises
+[`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
+on the caller and on every active tool. A forked tool gets interrupted over
+the control channel, a ractor via message passing, and pending tools
+are stopped before they run. The in-flight HTTP request is closed
+too, so a turn you no longer want stops without burning tokens.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/cancellation) to learn more.
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm)
+Thread.new { sleep(1); agent.cancel! }
+
+begin
+  agent.talk "write a very long poem", stream: $stdout
+rescue LLM::Interrupt
+  puts "cancelled"
+end
+```
+</details>
+<details>
+<summary>Console (<code>binding.pry</code> for agents)</summary>
+<br>
 
 The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
-method drops you into a curses-based TUI for talking to an
-agent interactively. Set `path:` on the agent for automatic
-persistence across REPL sessions. The `tools:` option attaches
-extra tools for the duration of the session. It is like
-`binding.pry` but for agents. For the full reference see the
-[REPL section](https://r.uby.dev/llm/deepdive/fundamentals/repl) in the
-deepdive.
+method drops you into a highly capable read-eval-print loop (REPL)
+that is built on top of curses. It can help you debug agents,
+test your tools, connect to MCP servers, and even A2A agents.
+The REPL stands out because it connects to the surrounding
+runtime and it can be extended by your code. Think of it as
+`binding.pry` but for agents.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/repl) to learn more.
+
+##### Demo
+
+[Watch in high quality on asciinema](https://asciinema.org/a/OsS8wwaasKasoDDz)
+
+![llm.rb REPL demo](demo.gif)
+
+
+##### Installation
+
+The REPL is distributed with llm.rb so you don't have to install
+a separate gem but it requires a number of optional dependencies
+to be installed separately. The following gems provide the full
+experience:
+
+    gem install curses kramdown xchan.rb test-cmd.rb
+
+##### Persistence
+
+The `path:` option can be set on an agent for automatic persistence
+across REPL sessions. The `tools:` option attaches extra tools
+for the duration of the session. Recall previous turns with Ctrl+P and
+Ctrl+N.
 
 ```ruby
 require "llm"
@@ -236,20 +298,110 @@ agent.repl(tools: LLM::Tool.subclasses)
 ##### CLI
 
 The `llm.rb` executable is available on your PATH after installation.
-It starts a REPL session from any directory:
+It starts a REPL session from any directory.The CLI auto-detects your
+provider from standard environment variables (`DEEPSEEK_API_KEY`,
+`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.). Persistent sessions are
+stored under `~/.llm.rb/` and restored automatically on your next visit.
 
 ```bash
 llm.rb                     # auto-detect from $DEEPSEEK_API_KEY
 llm.rb -p openai           # use OpenAI explicitly
 llm.rb -t                  # temporary session, no persistence
 ```
+</details>
+<details>
+<summary>Persistence</summary>
+<br>
+
+Set `path:` on an agent for automatic filesystem persistence:
+the agent restores conversation history from the file on startup
+and saves it back after every turn, with no manual serialization
+code. For database-backed persistence, ActiveRecord and Sequel
+integrations are also available. All persistence options use the same
+underlying serialization.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/database) to learn more.
+
+```ruby
+require "llm"
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, path: "session.json")
+agent.talk "remember my name is robert"
+
+# Next time, the conversation is restored automatically:
+agent = LLM::Agent.new(llm, path: "session.json")
+agent.talk "what's my name?"
+```
+</details>
+<details><summary>ActiveRecord | Sequel</summary>
+<br>
+
+Because both
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+can be serialized to JSON and stored in a simple string, both ActiveRecord
+and Sequel support can be implemented within a single column on a single row.
+
+The runtime includes first-class support for both ActiveRecord / Sequel, and
+for both Rack-based / Rails-based applications. On databases
+where it is supported, such as PostgreSQL, the column can be optimized by using
+the `jsonb` type.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/database) to learn more.
+
+```ruby
+require "active_record"
+require "llm"
+require "llm/active_record"
+
+class Email < ApplicationRecord
+  acts_as_agent do |agent|
+    agent.set name: "mail",
+              instructions: "Write concise, friendly replies to emails",
+              model: "deepseek-v4-pro"
+  end
+
+  def draft_reply!
+    talk("Draft a reply to:\n\n#{body}")
+  end
+
+  def summarize
+    talk("Summarize this email thread in a few sentences")
+  end
+
+  private
+
+  ##
+  # By convention, this method defines the provider for a model.
+  # If necessary, it can be renamed with: provider: :your_method.
+  def set_provider
+    LLM.deepseek(key: ENV["KEY"])
+  end
+
+  ##
+  # By convention, this method returns the context options given
+  # to LLM::Context or LLM::Agent. This method can be left undefined.
+  def set_context
+    {}
+  end
+end
 
-The CLI auto-detects your provider from standard environment variables
-(`DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.).
-Persistent sessions are stored under `~/.llm.rb/` and restored
-automatically on your next visit.
+email = Email.create!(subject: "Streaming support", body: "How do I stream responses?")
+email.draft_reply!
 
-#### LLM::MCP
+##
+# The conversation (the email and the draft
+# reply) is persisted to the email's column. A
+# fresh instance restores it and continues the
+# thread, so the summary below knows what was
+# already drafted:
+Email.find(email.id).summarize
+```
+</details>
+
+<details><summary>MCP</summary>
+<br>
 
 The Model Context Protocol (MCP) has first-class support
 in llm.rb. The stdio and http transports work out of the
@@ -259,6 +411,9 @@ used with
 [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
 
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/protocols/mcp/), and the
+[deepdive.md on persistent connections](https://r.uby.dev/llm/deepdive/features/transports) to learn more.
+
 ```ruby
 require "llm"
 
@@ -267,8 +422,9 @@ mcp   = LLM::MCP.stdio(argv: ["ruby", "server.rb"])
 agent = LLM::Agent.new(llm, stream: $stdout, tools: mcp.tools)
 agent.talk "Run the tool"
 ```
-
-#### LLM::A2A
+</details>
+<details><summary>A2A</summary>
+<br>
 
 The Agent 2 Agent (A2A) protocol has first-class support
 in llm.rb. The http and jsonrpc transports work out of the
@@ -278,6 +434,9 @@ used with
 [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
 
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/protocols/a2a/), and the
+[deepdive.md on persistent connections](https://r.uby.dev/llm/deepdive/features/transports) to learn more.
+
 ```ruby
 require "llm"
 
@@ -286,40 +445,330 @@ a2a   = LLM::A2A.rest(url: "https://remote-agent.example.com")
 agent = LLM::Agent.new(llm, stream: $stdout, tools: a2a.skills)
 agent.talk "Run the skill"
 ```
+</details>
 
-#### LLM::Skill
+<details><summary>Structured outputs</summary>
+<br>
 
-A skill turns a markdown file into a callable tool. When the model
-calls it, the runtime spawns a subagent with the skill's instructions
-as its system prompt and the skill's own tool set. The subagent runs
-one turn and returns the result, then is discarded. Each call
-is fresh and stateless. For a deeper explanation see the
-[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills).
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
+subclasses produce typed, structured
+output from any model call. Pass a schema to
+[`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk-instance_method),
+[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method),
+or
+[`LLM::Provider#complete`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#complete-instance_method)
+to receive validated JSON instead of free text. Schemas work alongside tools and streams.
 
-**SKILL.md**
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
+can define objects, arrays, enums, nested schemas,
+and more. It is also used internally by
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
+definitions, so you already benefit from it when you declare tool
+parameters.
 
-```markdown
----
-name: summary
-description: Reads recent git history and writes a summary
-tools: all
----
+The
+[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
+provider includes runtime-level optimisations such as structured
+output support (despite no official structured outputs API) and
+SVG image generation. This example uses
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) with
+DeepSeek:
 
-Collect the recent git log, analyze each commit,
-and write a summary to summary.txt.
+```ruby
+class Weather < LLM::Schema
+  property :city, String, "The city name"
+  property :temperature, Number, "Current temperature"
+  property :conditions, String, "Weather conditions"
+  required %i[city temperature conditions]
+end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, schema: Weather)
+res = agent.talk "Weather in Paris?"
+res.content!  # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
 ```
+</details>
+<details><summary>Guards</summary>
+<br>
 
-**agent.rb**
+[`LLM::Guard`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html)
+is the hook that sees every tool call before it runs. A guard
+can let a call through, cancel it, block it with an error, or
+even answer for it. Because it runs before the tool, anything
+it intercepts never executes. Policy, validation, quotas, and
+cost ceilings all live here.
+
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+enables
+[`LLM::Guard::Loop`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard/Loop.html)
+by default, so agents get loop protection out of the box. To
+write your own guard, subclass
+[`LLM::Guard`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html)
+and implement
+[`LLM::Guard#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html#call-instance_method).
+The pending call arrives as `function:`. Return a value to close
+the call, or `nil` to let it run:
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/guard) to learn more.
+
+```ruby
+class PolicyGuard < LLM::Guard
+  def call(function:)
+    if function.name == "shell"
+      function.return(error: true, type: "policy_error",
+                      message: "shell is disabled")
+    end
+  end
+end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, tools: [Shell, ReadFile], guard: PolicyGuard)
+```
+</details>
+
+<details>
+<summary>Transformers</summary>
+<br>
+
+It is possible to rewrite outgoing messages before they reach the provider with
+[`LLM::Transformer`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer.html).
+Create a subclass and implement `call(message:)` to scrub sensitive data,
+inject context, or normalize content. The transform runs automatically
+on every turn, so you never have to change your prompt code.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transformer) to learn more.
+
+```ruby
+class RedactEmails < LLM::Transformer
+  def call(message:)
+    content = message.content.to_s.gsub(/[\w.+-]+@[\w-]+\.[\w.]+/, "[EMAIL]")
+    LLM::Message.new(message.role, content, message.extra)
+  end
+end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, transformer: RedactEmails)
+agent.talk "Contact support@example.com for help"
+```
+</details>
+
+<details>
+<summary>Compactors</summary>
+<br>
+
+Every model has a context window: the finite number of tokens it can
+consider in a single request. Generally a compactor will drop or
+summarize older messages to keep the conversation within that window,
+and it runs automatically before every turn. By default it is disabled
+so it is a feature you must opt into.
+
+[`LLM::Compactor::Truncate`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor/Truncate.html)
+keeps the most recent messages via an integer count or a percentage like
+`"80%"`. It preserves tool call and return pairs so the conversation
+never contains an orphaned result. It is also possible to subclass
+[`LLM::Compactor`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor.html)
+to implement your own compactor with its own logic. Streams can observe the
+process through the
+[`LLM::Stream#on_compaction`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction)
+and
+[`LLM::Stream#on_compaction_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction_finish)
+callbacks.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/compaction) to learn more.
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(
+  llm,
+  compactor: LLM::Compactor::Truncate,
+  compactor_options: {keep: 64}
+)
+agent.talk "Hello"
+```
+</details>
+
+<details>
+<summary>Automatic retries</summary>
+<br>
+
+Rate-limited requests are retried automatically by default. Agents
+retry a 429 up to five times with a growing backoff before giving
+up, so most request failures resolve on their own. Set `retry_budget`
+to change the number of retries, or `retry_budget: 0` to disable
+them.
 
 ```ruby
 require "llm"
 
-llm   = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, skills: ["./skills/summary"])
-agent.talk "Summarize the last week of work"
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, retry_budget: 0)
+agent.talk "Hello"
+```
+
+</details>
+
+
+<details>
+<summary>Observability</summary>
+<br>
+
+Trace what an agent is doing by attaching a tracer. Hook into
+requests, tool calls, and other runtime events to debug a
+misbehaving agent, monitor latency, or export spans to an
+observability backend. All built-in tracers share one interface,
+so switching between them means changing a class name:
+
+* [`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html): human-readable single-line logs to stderr, ideal during development.
+* [`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html):
+exports spans via OTLP for OpenTelemetry in production.
+* [`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html):
+structured JSON to stdout or a file.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/reference/tracer) to learn more.
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, tracer: LLM::Tracer::PrettyLogger.new(llm))
+agent.talk "Hello"
 ```
+</details>
+
+<details>
+<summary>As a subclass</summary>
+<br>
 
-#### RAG
+[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
+is a class-level DSL that accepts a Hash of properties. Each key resolves to a
+corresponding class accessor: `name`, `description`, `model`, `tools`,
+`instructions`, `schema`, `stream`, `tracer`, `concurrency`, `confirm`,
+`path`, `skills`, `tool_budget`, and `retry_budget`. All options are
+optional; zero or more can be set.
+An error is raised for unknown keys so that typos are caught early.
+
+```ruby
+require "llm"
+require "llm/tools"
+
+class Agent < LLM::Agent
+  set name: "sysadmin",
+      description: "system administration agent",
+      model: "deepseek-v4-pro",
+      tools: [LLM::Tool::Shell]
+end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = Agent.new(llm)
+agent.talk "Run 'date'"
+```
+</details>
+
+### Providers
+
+Each provider is constructed with a class-level factory method on
+`LLM`, and the resulting instance is passed to
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
+or
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html). The
+same API drives every one of them, so switching models is a one-line
+change. See the [deepdive](https://r.uby.dev/llm/deepdive/fundamentals/providers)
+for a full provider reference.
+
+#### What providers does llm.rb support?
+
+* **Anthropic** (`LLM.anthropic`)
+* **Google** (`LLM.google`)
+* **OpenAI** (`LLM.openai`)
+* **DeepSeek** (`LLM.deepseek`)
+* **DeepInfra** (`LLM.deepinfra`)
+* **xAI** (`LLM.xai`)
+* **Z.ai** (`LLM.zai`)
+* **Moonshot (Kimi)** (`LLM.moonshot`)
+* **Alibaba (Qwen3)** (`LLM.alibaba`, also `LLM.aliyun`)
+* **Mistral** (`LLM.mistral`)
+* **AWS Bedrock** (`LLM.bedrock`)
+* **Ollama** (`LLM.ollama`)
+* **llama.cpp** (`LLM.llamacpp`)
+
+<details>
+<summary>Implicit</summary>
+<br>
+
+Cloud providers can infer their API key automatically
+from a set of common defaults that are defined by
+the [models.dev](https://models.dev) registry that
+is also distributed with llm.rb.
+
+```ruby
+llm = LLM.openai
+llm = LLM.anthropic
+llm = LLM.deepseek
+llm = LLM.alibaba  # also: LLM.aliyun
+llm = LLM.moonshot
+llm = LLM.mistral
+```
+</details>
+<details>
+<summary>Explicit</summary>
+<br>
+
+The `key` option can also be providied explicitly, and certain
+providers (eg ollama, llamacpp) usually do not require an API
+key at all.
+
+```ruby
+llm = LLM.openai(key: ENV["OPENAI_API_KEY"])
+llm = LLM.anthropic(key: ENV["ANTHROPIC_API_KEY"])
+llm = LLM.deepseek(key: ENV["DEEPSEEK_API_KEY"])
+llm = LLM.alibaba(key: ENV["DASHSCOPE_API_KEY"]) # also: LLM.aliyun
+llm = LLM.moonshot(key: ENV["MOONSHOT_API_KEY"])
+llm = LLM.mistral(key: ENV["MISTRAL_API_KEY"])
+```
+</details>
+
+<details>
+<summary>Model Registry</summary>
+<br>
+
+Each provider ships its model catalog, pricing, limits, and
+modalities with the gem, sourced from [models.dev](https://models.dev).
+Reach it from any provider, context, or agent, enumerate models, or
+sort them by price.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/reference/model_registry) to learn more.
+
+```ruby
+require "llm"
+
+llm      = LLM.openai
+registry = llm.registry                # => LLM::Provider#registry
+cheapest = registry.models.sort.first  # => LLM::Model
+cheapest.id                            # => "text-embedding-3-small"
+cheapest.context_window                # => 8191
+cheapest.structured_output?            # => false
+```
+</details>
+
+<details>
+<summary>Transports</summary>
+<br>
+
+The `transport:` option selects which HTTP library a provider uses for
+network communication. Three backends ship out of the box: `net/http`
+is always available and the default, `net/http/persistent` pools
+connections for many requests to the same host, and `curb` wraps
+libcurl. They share one interface, so switching is a one-word change.
+
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transports) to learn more.
+
+```ruby
+llm = LLM.deepseek(
+  key: ENV["KEY"],
+  transport: :net_http_persistent
+)
+```
+</details>
+
+### RAG
 
 Most providers offer an embedding model that can be
 used for semantic search, or similarity search. An
@@ -332,6 +781,8 @@ llm.rb also includes support for OpenAI's vector store API. It
 provides a vector database as a HTTP service but we won't cover
 that here.
 
+See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/embeddings) to learn more.
+
 ```ruby
 require "llm"
 
@@ -348,116 +799,57 @@ Document.create!(
 )
 ```
 
-#### Concurrency
-
-The runtime supports six different concurrency strategies that have
-different attributes. The choice between all of them often depends
-on the requirements of your application.
-
-IO-bound tools are a good fit for the `:async`, `:thread`,
-and `:fiber` strategies while true parallelism can be achieved
-with the `:fork` and `:ractor` strategies. The
-`:sequential` strategy runs tools one at a time and is the default.
-The `:fork` strategy also provides a separate process that offers
-isolation from its parent.
+### Images
 
-You can learn more about the llm.rb concurrency model in the
-[deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/concurrency).
+A handful of providers can generate images from a text prompt.
+OpenAI, Google, xAI, and DeepInfra all support it. The API is
+the same across providers:
 
 ```ruby
 require "llm"
 
-llm   = LLM.deepseek(key: ENV["KEY"])
-tools = [FetchNews, FetchStocks, FetchFeeds]
-agent = LLM::Agent.new(llm, tools:, concurrency: :fork)
-agent.talk "Run the tools in parallel"
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.images.create(prompt: "a dog on a rocket to the moon")
+IO.copy_stream res.images[0], "rocket.png"
 ```
 
-#### ORM
+##### DeepSeek
 
-Because both
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-can be serialized to JSON and stored in a simple string, both ActiveRecord
-and Sequel support can be implemented within a single column on a single row.
-
-The runtime includes first-class support for both ActiveRecord *and* Sequel, and
-for both Rack-based applications *and* Rails-based applications. On databases
-where it is supported, such as PostgreSQL, the column can be optimized by using
-the `jsonb` type.
+DeepSeek does not have a dedicated image model, but the runtime
+generates SVG vector graphics through its text model. Each
+generation produces a valid SVG document that can be converted
+to PNG with tools like `rsvg-convert`. Pass an existing agent
+to maintain a session across generations:
 
 ```ruby
-require "active_record"
 require "llm"
-require "llm/active_record"
-
-class Agent < ApplicationRecord
-  acts_as_agent
-  set name: "my-agent",
-      instructions: "solve the user's query",
-      model: "deepseek-v4-pro",
-      tools: [Research, FinalizeResearch, ActOnResearch]
-
-  private
-
-  # By convention, this method defines the provider for a model.
-  # If necessary, it can be renamed with: provider: :your_method.
-  def set_provider
-    LLM.deepseek(key: ENV["KEY"])
-  end
+llm = LLM.deepseek(key: ENV["KEY"])
 
-  # By convention, this method returns the context options given
-  # to LLM::Context or LLM::Agent.
-  def set_context
-    {}
-  end
-end
+##
+# First generation
+res = llm.images.create(prompt: "a rocket on the moon")
+IO.copy_stream res.images[0], "rocket.svg"
 
-agent = Agent.create!
-agent.talk "perform research"
+##
+# Refine with follow-up prompts (shares context)
+res = llm.images.create(prompt: "add a dog next to the rocket",
+                        agent: res.agent)
+IO.copy_stream res.images[0], "rocket-with-dog.svg"
 ```
 
 ## FAQ
 
 <details>
-<summary>What providers does llm.rb support?</summary>
+<summary>What about local LLM support?</summary>
 <br>
 <p>
-
-**Cloud**
-
-The following cloud-based providers are available to choose from. <br>
-In no particular order:
-
-🇺🇸 OpenAI <br>
-🇺🇸 DeepInfra <br>
-🇺🇸 xAI <br>
-🇺🇸 Google (Gemini) <br>
-🇺🇸 AWS bedrock <br>
-🇺🇸 Anthropic <br>
-🇨🇳 DeepSeek <br>
-🇨🇳 zAI <br>
-🇪🇺 Mistral <br>
-
-**Weights**
-
-The following providers provide access to open-weight models. <br>
-In no particular order:
-
-🇺🇸 DeepInfra <br>
-🇺🇸 AWS bedrock <br>
-🇨🇳 DeepSeek <br>
-🇨🇳 zAI <br>
-🇪🇺 Mistral <br>
-
-**Local**
-
-The following providers can be run locally on your own hardware. <br>
-In no particular order:
+The following providers can be run used with models that
+are running on your own hardware. They're reasonably well
+tested but not my main driver:
+</p>
 
 * Ollama
 * Llamacpp
-</p>
 </details>
 
 <details>
@@ -489,11 +881,10 @@ type.
 If you're on a budget, DeepSeek is hard to beat.
 </details>
 <details>
-<summary>Can I download llm.rb via a decentralized network?</summary>
-<br>
-Yes.
+<summary>Sources other than GitHub?</summary>
 <br>
-We are on the <a href="https://radicle.network">radicle.network</a>
+<p>
+We are on the <a href="https://radicle.network">radicle.network</a> as well.
 <br>
 Every commit that lands on GitHub also lands on Radicle.
 <br>
@@ -502,6 +893,30 @@ Our repository ID is z2PtfQ6dYwyYaW2aGrztG1sMyDmCE.
 Browse on <a
 href="https://radicle.network/nodes/iris.radicle.network/z2PtfQ6dYwyYaW2aGrztG1sMyDmCE">the
 web</a>.
+</p>
+</details>
+
+<details>
+<summary>Who maintains llm.rb?</summary>
+<br>
+
+The llm.rb project is maintained primarily by one
+person. llm.rb has been in active development for more
+than three years and over that time multiple other
+contributors have contributed to llm.rb as well. New
+contributors are always welcome.
+
+I use the repl that is distributed with llm.rb to build
+llm.rb itself so there is a healthy feedback loop and
+llm.rb has also been battle tested in production
+environments.
+
+I have also also written llm.rb agents within the
+repository that help me maintain the documentation,
+and backport changes to the mruby-llm runtime as well.
+
+I am constantly focused on improving llm.rb by using
+it as my primary driver for development.
 </details>
 
 ## Resources