llm.rb 16.0.0 → 17.0.0

data/README.md

@@ -18,16 +18,9 @@ llm.rb is a runtime for building agentic AI applications
 on CRuby. It has zero runtime dependencies by default, supports
 concurrent and parallel tool execution and has a single coherent API
 that spans 14+ providers.
-The README covers the common cases. For everything else there is the
-[deepdive](docs/deepdive.md), a reference with a chapter for each
-topic, and for what changes between releases there is the
-[changelog](CHANGELOG.md).
-
-[The r.uby.dev website](https://r.uby.dev) hosts
-an agentic platform that provides users with
-personalized agents who can access GitHub, and
-other services. It is built with llm.rb. Check it
-out if curious. **Still in early development.**
+The README covers a lot of ground and the
+[changelog](CHANGELOG.md)
+tracks what has changed between releases.
 
 ## Install
 
@@ -46,101 +39,191 @@ The
 class is the default high-level interface,
 and it is recommended for most use-cases. It manages the tool loop
 and provides configurable features on top of it. For example you can
-manage the tool loop with a retry budget alongside a tool call budget,
-among other features.
+manage the tool loop with a retry budget and a tool call budget -
+alongside other features.
 
 The runtime is designed to keep the tool loop alive and it will
-avoid exceptions. When an error is encountered in a tool or during
-the lifecycle of an agent it is almost always reported back to the
-model as an in-band error that allows the model to correct course.
-
-A lot of care also goes into keeping the tool loop from entering
-an invalid state that would lead to API-level errors. For example,
-when a tool call is interrupted it could leave an unanswered tool
-call that a model will reject on the next turn. The runtime closes
-every tool call that has no return before the next request is sent,
-and each one is answered with an in-band return of its own. The
-conversation a provider sees is therefore always valid, and a
-cancelled call is something the model is told about rather than
-something that quietly disappears.
+rescue exceptions. When an exception is encountered in a tool it
+is reported back to the model as an in-band error that allows the
+model to change course or retry with different parameters.
+
+A tool call requires a tool return (or response), and the lack of
+one can corrupt the conversation and lead to API-level errors from
+a provider. But sometimes it is unavoidable (for example, via an
+interrupt or power loss) so the runtime automatically closes
+tool calls that fall into that category by telling the model
+the tool call(s) were cancelled.
+
+##### helloworld.rb
+
+A classic "hello world" example. <br> The environment variable
+`$DEEPSEEK_API_KEY` is assumed to have been set and reference
+a valid DeepSeek API key, otherwise you can provide a key explicitly
+(eg: `LLM.deepseek(key: ENV["MYKEY"])`) and you could swap
+deepseek for any of the other 13 providers as well
+(eg: `llm = LLM.openai`):
 
 ```ruby
 require "llm"
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, stream: $stdout)
+class GetTime < LLM::Tool
+  name "get-time"
+  description "returns the current time"
+
+  def call
+    {now: Time.now}
+  end
+end
+
+llm    = LLM.deepseek
+tracer = LLM::Tracer.pretty(llm)
+tools  = [GetTime]
+stream = StringIO.new
+
+agent  = LLM::Agent.new(llm, tools:, tracer:, stream:)
 agent.talk "hello world"
+agent.talk "what time is it?"
+puts stream.string
 ```
+
+#### Essentials
+
 <details>
 <summary>Stream</summary>
 <br>
 
-Streams can be simple IO objects or subclasses of
-[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
-with structured callbacks for content,
-reasoning, tool calls, tool returns, steps in a turn, and compaction.
-Streams can also observe message transformers, which rewrite
-outgoing messages before they reach the provider.
+A stream can be a simple IO object (eg `$stdout`)
+or it can be a subclass of [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html).
+An IO object can receive content but it cannot
+receive the other callbacks that are available
+to subclasses of [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html).
+It is generally a good idea to return as
+quickly as possible from all of these methods,
+but especially for the `on_content` and
+`on_reasoning_content` methods: they run
+inline with the request.
+
+A subclass of [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
+can implement callbacks that the runtime will call
+throughout an agent's lifetime. All callbacks are
+optional. The callbacks provide for content, reasoning,
+tool calls, tool returns, steps in a turn, retries,
+compaction and more:
 
 ```ruby
-class MyStream < LLM::Stream
-  # Visible assistant output.
+class Stream < LLM::Stream
+  ##
+  # @param [String] content
+  #  A chunk of text
   def on_content(content)
     print content
   end
 
-  # Reasoning output streamed separately from visible content.
+  ##
+  # @param [String] content
+  #  A chunk of text
   def on_reasoning_content(content)
     warn content
   end
 
-  # A streamed tool call has been fully parsed.
+  ##
+  # @param [LLM::Function] tool
+  #  The tool being called
   def on_tool_call(tool)
+    nil
   end
 
-  # Queued streamed tool work has returned.
+  ##
+  # @param [LLM::Function] tool
+  #  The tool that returned
+  # @param [LLM::Function::Return] result
+  #  The return from the tool call
   def on_tool_return(tool, result)
+    nil
   end
 
-  # A request has completed: the response is in the conversation, and
-  # any tools it asked for run after this.
-  def on_step(ctx, res)
-  end
-
-  # Before a transformer rewrites an outgoing message.
+  ##
+  # @note
+  #  This method is called _before_ a transformer runs
+  # @param [LLM::Transformer] transformer
+  #  A transformer
   def on_transform(transformer)
+    nil
   end
 
-  # Aftter a transformer rewrites an outgoing message.
+  ##
+  # @note
+  #  This method is called _after_ a transformer runs
+  # @param [LLM::Transformer] transformer
+  #  A transformer
   def on_transform_finish(transformer)
+    nil
   end
 
-  # Before a compactor trims the conversation.
+  ##
+  # @note
+  #  This method is called _before_ a compactor runs
+  # @param [LLM::Compactor] compactor
+  #  A compactor
   def on_compaction(compactor)
+    nil
   end
 
-  # After a compactor trims the conversation.
+  ##
+  # @note
+  #  This method is called _after_ a compactor runs
+  # @param [LLM::Compactor] compactor
+  #  A compactor
   def on_compaction_finish(compactor)
+    nil
   end
 
-  # Before a skill's subagent runs.
-  def on_skill_call(skill)
+  ##
+  # @note
+  #  This method is called once per request
+  #  in a turn (which can contain multiple
+  #  requests)
+  # @param [LLM::Context] ctx
+  #  The context
+  # @param [LLM::Response] res
+  #  The response
+  def on_step(ctx, res)
+    nil
   end
 
-  # After a skill's subagent runs.
-  # The subagent that ran it, the skill, and its response are passed
-  # through, so you can introspect the agent, tally skill usage, or
-  # track costs.
-  def on_skill_return(agent, skill, result)
+  ##
+  # @note
+  #  This method is called when a request is
+  #  rate limited and retried.
+  # @param [LLM::RateLimitError] error
+  # @param [Integer] attempt
+  def on_retry(error, attempt)
+    nil
   end
 
-  # A request was rate limited or timed out and will be retried.
-  def on_retry(error, attempt)
+  ##
+  # @note
+  #  This method is called _before_ a skill runs
+  # @param [LLM::Skill] skill
+  #  A skill
+  def on_skill_call(skill)
+    nil
+  end
+
+  ##
+  # @note
+  #  This method is called _after_ a skill runs
+  # @param [LLM::Agent] agent
+  #  The agent who ran the skill
+  # @param [LLM::Skill] skill
+  # @param [LLM::Response] res
+  def on_skill_return(agent, skill, res)
+    nil
   end
 end
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, stream: MyStream.new)
+llm = LLM.deepseek
+agent = LLM::Agent.new(llm, stream: Stream.new)
 agent.talk "Explain Ruby fibers."
 ```
 </details>
@@ -148,17 +231,23 @@ agent.talk "Explain Ruby fibers."
 <details><summary>Tools</summary>
 <br>
 
-Subclasses of
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
-are plain Ruby classes with
-an optional set of typed parameters. <br> The model can choose to
-call them on your behalf, and they're one of the most powerful features
-for extending the feature set or abilities of a model.
-
-The runtime also ships with a catalog of built-in tools for
-filesystem, search, and shell operations, and providers expose
-platform-native tools such as web search and code execution that run
-on the provider's side.
+An agent requires one or more tools to be
+able to interact with the "outside" world.
+At a high-level a tool is how a model can
+access your filesystem, search the internet,
+post a comment on your behalf and anything
+else that your own code could do.
+
+The runtime represents a tool as a subclass
+of [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
+that provides a name, a description, an optional
+set of parameters and a method that the runtime
+will call on the model's behalf. A tool is implemented
+on top of
+an [`LLM::Function`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html)
+object, and you might come across it in stream and
+tracer callbacks. The model decides when and how a
+tool is called:
 
 ```ruby
 class ReadFile < LLM::Tool
@@ -172,255 +261,441 @@ class ReadFile < LLM::Tool
   end
 end
 
-llm = LLM.deepseek(key: ENV["KEY"])
+llm = LLM.deepseek
 agent = LLM::Agent.new(llm, tools: [ReadFile], stream: $stdout)
 agent.talk "summarize README.md"
 ```
 </details>
-<details>
-<summary>Skills</summary>
+<details><summary>Schema</summary>
 <br>
 
-A skill turns a markdown file into a callable tool. When the model
-calls it, the runtime spawns a subagent with the skill's instructions
-as its system prompt and the skill's own tool set. The subagent runs
-one turn and returns the result, then is discarded. Each call
-is fresh and stateless.
-
-A [LLM::Stream](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
-can be notified as a skill starts and when it returns. The `on_skill_return`
-callback hands back the subagent that ran the skill, so you can inspect
-its conversation, measure its usage, track costs or add a verification
-step (eg `subagent.talk("verify your work")`).
+The
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
+class describes an object that has properties
+of different types, and it can also reference
+other objects, or schemas. The object describes
+the shape of a JSON object that the model will
+return in place of the usual free-form text that
+a model replies with.
+
+This feature is implemented with
+[JSONSchema](https://json-schema.org/)
+under the hood and a large portion
+of the spec is supported: objects, enums,
+arrays, nested objects, strings, integers,
+and so on.
+[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
+also provides the parameter definitions
+available to
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
+subclasses and the same range of types are
+supported.
 
-##### summary.md
+The
+[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
+provider does not support structured outputs
+officially but the runtime implements structured
+outputs on top of the `json_object` API and it
+works the same as all other providers who do
+support structured outputs:
 
-```markdown
----
-name: summary
-description: Reads recent git history and writes a summary
-tools: all
----
+```ruby
+class Forecast < LLM::Schema
+  property :city, String, "The city name"
+  property :temperature, Number, "Current temperature"
+  property :conditions, String, "Weather conditions"
+  required %i[city temperature conditions]
+end
 
-Collect the recent git log, analyze each commit,
-and write a summary to summary.txt.
+llm = LLM.deepseek
+agent = LLM::Agent.new(llm)
+res = agent.talk "Weather in Paris?", schema: Forecast
+res.content!  # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
 ```
+</details>
+<details>
+<summary>Serialization</summary>
+<br>
 
-##### agent.rb
+Both [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
+and
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+can be serialized to JSON and written to disk.
+This feature is what supports the ActiveRecord
+and Sequel integrations too but rather than
+store the agent directly on disk it is stored
+in a database column instead.
+
+An agent can be configured to read from and
+write to a file automatically with the `path`
+option. When the file already exists, the agent
+is restored from the file and continues where
+he left off. After each turn the agent flushes
+its state to the file. The text file can be shared
+like any other text file and it can be used to
+restore the agent in another process or machine:
 
 ```ruby
 require "llm"
 
-llm   = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, skills: ["summary.md"])
-agent.talk "Summarize the last week of work"
+path  = File.join(Dir.home, ".agents", "myagent.json")
+llm   = LLM.deepseek
+agent = LLM::Agent.new(llm, path:)
+agent.talk "remember my name is robert"
+
+##
+# Resume the conversation where the agent left off
+agent = LLM::Agent.new(llm, path:)
+agent.talk "what's my name?"
 ```
 </details>
-
 <details>
-<summary>Concurrency</summary>
+<summary>Tracer</summary>
 <br>
 
-The runtime supports six different concurrency strategies that have
-different attributes. The choice between all of them often depends
-on the requirements of your application.
+An agent can have its own tracer that observes runtime
+events throughout its lifetime. A tracer is local to an
+agent and the tracer runs on the same thread as the agent.
+There are three builtin tracers, and a fourth tracer that
+lets you combine multiple tracers into a single tracer -
+so you can log to a file, and export to OpenTelemetry
+at the same time.
+
+All tracers protect themselves and the agent being traced
+from a crash through a rescue that dumps a tracer error
+to standard error (`$stderr`). Beyond the default tracers,
+it is not unusual to write your own. A tracer can let you
+export runtime state to your own database, an external
+service, or any other medium you might have in your stack.
+A new tracer can be created through a subclass of
+[`LLM::Tracer`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html)
+that implements the expected callback methods. This feature
+also works for ActiveRecord and Sequel models.
+
+The builtin tracers:
+
+* [`LLM::Tracer.pretty`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#pretty_logger-class_method): human-readable single-line logs to stderr, ideal during development.
+* [`LLM::Tracer.telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#telemetry-class_method):
+exports spans via OTLP for OpenTelemetry in production.
+* [`LLM::Tracer.logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#logger-class_method):
+structured JSON to stdout or a file.
+* [`LLM::Tracer.multi`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#multi-class_method):
+fans out every callback to multiple tracers.
+
+A simple example:
 
-IO-bound tools are a good fit for the `:async`, `:thread`,
-and `:fiber` strategies while true parallelism can be achieved
-with the `:fork` and `:ractor` strategies. The
-`:sequential` strategy runs tools one at a time and is the default.
-The `:fork` strategy also provides a separate process that offers
-isolation from its parent.
+```ruby
+llm    = LLM.deepseek
+tracer = LLM::Tracer.pretty(llm)
+agent  = LLM::Agent.new(llm, tracer:)
+agent.talk "Hello"
+```
 
-A couple of concurrency strategies require optional, opt-in dependencies.
-The `async` strategy requires the [async](https://github.com/socketry/async)
-gem and the `fork` strategy requires the [xchan.rb](https://github.com/r-uby-dev/xchan.rb)
-gem (`~> 0.24`). The `fiber` strategy requires a scheduler (`Fiber.scheduler`) but by
-default Ruby does not provide one.
+A multi-tracer example:
+
+```ruby
+llm    = LLM.deepseek
+tracer = LLM::Tracer.multi(
+  llm,
+  LLM::Tracer::Logger,
+  LLM::Tracer::Telemetry,
+  LLM::Tracer::Foobar.new(llm, {option1: "foo", option2: "bar"})
+)
+agent = LLM::Agent.new(llm, tracer:)
+agent.talk "Hello"
+```
 
-The `:ractor` strategy is the least interchangeable of the six. It runs
-class-based tools only, and a tool's arguments have to be
-ractor-shareable.
+</details>
+
+<details><summary>MCP</summary>
+<br>
+
+The runtime can communicate with MCP servers via HTTP(S)
+and stdio. The stdio transport spawns a local process that
+uses its standard input and output streams for communication.
+The runtime provides a full-featured MCP client and the tools
+that an MCP server has announced are available as
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
+subclasses that can be used like any other subclass
+of
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html).
+
+
+The first example uses the HTTP transport to connect to
+GitHub's MCP server. The example uses a Personal Access
+Token (get yours [here](https://github.com/settings/apps))
+although it could also use a token that was obtained
+through OAuth and belongs to a user other than you:
 
 ```ruby
 require "llm"
-require "llm/tools"
 
-llm   = LLM.deepseek(key: ENV["KEY"])
-tools = LLM::Tool.subclasses
-agent = LLM::Agent.new(llm, tools:, concurrency: :fork)
-agent.talk "Run the tools in parallel"
+llm = LLM.deepseek
+mcp = LLM::MCP.http(
+  url: "https://api.githubcopilot.com/mcp/",
+  headers: {"Authorization" => "Bearer #{ENV['GITHUB_PAT']}"},
+  transport: :net_http_persistent
+)
+
+agent = LLM::Agent.new(llm, stream: $stdout, tools: mcp.tools)
+agent.talk "Find me an interesting Ruby project on GitHub"
 ```
 
-</details>
-<details>
-<summary>Cancellation</summary>
-<br>
+The second example focuses on the second transport.
+It is recommended (but not required) to launch a
+stdio MCP server with a session so that the process
+is kept alive for as long as you use it, otherwise
+the process can be launched multiple times and it
+can add latency to your requests.
 
-It is possible to abort a request mid-stream and interrupt
-running tool calls with
-[`LLM::Agent#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#interrupt!)
-(or `cancel!`) as long as it is done by another thread or fiber in
-the same process.
-
-The runtime raises
-[`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
-on the fiber making a request and on every active tool
-regardless of the active concurrency strategy. Before that
-happens the request socket is forcibly closed so the request
-stops burning tokens. It works the same across all strategies
-although there can be subtle differences that usually go unnoticed.
-
-A tool can implement the `#on_interrupt` callback
-to be notified when a tool call has been interrupted
-and it can also rescue the `LLM::Interrupt` exception
-to free resources or perform other actions before the
-tool dies.
+The example uses a local stdio MCP server from the
+[sem](https://github.com/Ataraxy-Labs/sem)
+project. <br> It works on top of git and parses your code
+with tree-sitter:
 
 ```ruby
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-Thread.new { sleep(1); agent.cancel! }
+require "llm"
 
-begin
-  agent.talk "write a very long poem", stream: $stdout
-rescue LLM::Interrupt
-  puts "cancelled"
+mcp = LLM::MCP.stdio(
+  argv: ["sem", "mcp"],
+  env: {'SEM_REPO' => Dir.getwd},
+  cwd: Dir.getwd,
+)
+
+##
+# 'mcp.session' is optional, but recommended.
+# It can be dropped but then the process may
+# be launched once per-request.
+mcp.session do
+  llm    = LLM.deepseek
+  tracer = LLM::Tracer.pretty(llm)
+  tools  = mcp.tools.select { _1.name == "sem_log" }
+  stream = $stdout
+  agent  = LLM::Agent.new(llm, tracer:, stream:, tools:)
+  agent.talk "summarize recent git activity"
 end
 ```
 </details>
+
 <details>
-<summary>Console (<code>binding.irb</code> for agents)</summary>
+<summary>Console</summary>
 <br>
 
 The [LLM::Agent#console](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#console-instance_method)
 method drops you into an interactive console that is built on
-top of curses. It can help you debug agents, test your tools,
-connect to MCP servers, and other A2A agents. The console stands
-out because it connects to the surrounding runtime and it can
-be extended by your code. Think of it as `binding.irb` but
-for agents.
+top of (n)curses. The `llm.rb` executable packaged with the gem
+is another way to access the console and ActiveRecord models who
+have called `acts_as_agent` can access the console as well
+(via `agent.console`).
+
+A console for an ActiveRecord model does not write back to the
+database. The `llm.rb` executable automatically associates a
+session with the current working directory and it can be resumed
+by calling `llm.rb` in the same directory at a later point.
+
+The console is not intended to compete with Claude, Codex and
+friends. It is much more limited, serves an entirely different
+purpose and is more like a debugger for your agents. The
+dependencies required by the console are not installed
+by default, and the easiest way to grab them is via
+`gem install llm-shell`.
 
 ##### Demo
 
 ![llm.rb console demo](demo.gif)
 
+</details>
+<details><summary>Budgets</summary>
+<br>
 
-##### Installation
-
-The console is distributed with llm.rb but it requires a number
-of optional dependencies to be installed separately. The following
-gems provide the full experience:
-
-    gem install unicode-display_width curses kramdown xchan.rb test-cmd.rb
+An agent has a retry budget, and a tool call
+budget. The retry budget is enabled by default,
+and it determines how many times we will retry
+when a request raises an error. The errors
+that are retried are returned by
+[`LLM::Provider#retry_errors`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#retry_errors-instance_method)
+and can vary by provider but typically cover
+rate limits, and time outs. Between retries
+we will sleep, and each retry sleeps a little
+longer (`sleep 2.0 * attempts`) than the last.
+
+The default retry budget is usually 5 but
+it can also vary by provider and it is determined by the
+[`LLM::Provider#retry_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#retry_budget-instance_method)
+method. When the budget is spent but the an
+exception continues to raise we will give up
+and let the exception bubble up to the caller.
+The budget can be disabled by setting
+`retry_budget` to zero:
 
-For convenience it is also possible to just use the following, it
-is a metagem that depends on llm.rb and all the dependencies it requires
-to run the console:
+```ruby
+require "llm"
 
-    gem install llm-shell
+llm = LLM.deepseek
+agent = LLM::Agent.new(llm, retry_budget: 0)
+```
 
-##### Persistence
+The second budget describes how many tool calls
+are allowed in a single turn, and it is disabled
+by default. When the limit is reached or breached
+the agent will reply with an in-band error that
+tells the model it has spent its tool call budget,
+and that it should stop requesting tool calls for
+this turn. This exchange repeats until the model
+stops requesting tool calls.
 
-the `path:` option can be set on an agent for automatic persistence
-across console sessions. The `tools:` option attaches extra tools
-for the duration of the session. Recall previous turns with Ctrl+P and
-Ctrl+N.
+The budget is restored after a turn completes and the
+model will normally provide a response from the tool
+calls made before the budget was spent:
 
 ```ruby
 require "llm"
 require "llm/tools"
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, name: "my-agent", path: "agent.json")
-agent.console(tools: LLM::Tool.subclasses)
+llm   = LLM.deepseek
+tools = LLM::Tool.subclasses
+agent = LLM::Agent.new(llm, tools:, tool_budget: 25)
+agent.talk "fix every FIXME in this repository"
 ```
+</details>
+<details>
+<summary>Usage|Cost</summary>
+<br>
 
-##### CLI
+An agent knows how many tokens it has spent,
+and how much those tokens cost. The runtime
+embeds the [models.dev](https://models.dev)
+registry (only the parts it uses, and on-demand)
+that gives it access to model information that
+includes costs and limits.
+
+The `token_usage` method sums the whole conversation as a
+[LLM::Usage](https://r.uby.dev/api-docs/llm.rb/LLM/Usage.html)
+object and [LLM::Cost](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
+combines it with the model registry to determine how much money
+has been spent. These numbers should not be taken as gospel though.
+They're a best-guess estimate, and your provider's dashboard
+might disagree:
 
-The `llm.rb` executable is available on your PATH after installation.
-It starts a console session from any directory. The CLI auto-detects your
-provider from standard environment variables (`DEEPSEEK_API_KEY`,
-`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.). Persistent sessions are
-stored under `~/.llm.rb/` and restored automatically on your next visit.
+```ruby
+require "llm"
 
-```bash
-llm.rb                     # auto-detect from $PROVIDER_API_KEY
-llm.rb -p openai           # use OpenAI explicitly
-llm.rb -m gpt-5.6          # use a model other than the provider default
-llm.rb -c thread           # run tool calls on a separate thread
-llm.rb -n curb             # use libcurl as the HTTP transport
-llm.rb -x 900              # read timeout of 15 minutes
-llm.rb -t                  # temporary session, no persistence
-llm.rb -v                  # print the version
-llm.rb -h                  # print usage
+llm = LLM.deepseek
+agent = LLM::Agent.new(llm)
+agent.talk "Hello"
+
+agent.token_usage  # => LLM::Usage for the whole conversation
+agent.cost         # => LLM::Cost, priced from the registry
 ```
+
+The `context_*` methods can tell you about the model's context
+window: its total capacity, and how much of it have we already
+spent:
+
+```ruby
+agent.context_used    # => tokens in the latest turn
+agent.context_window  # => the model's limit, or nil when unknown
+agent.context_usage   # => Rational, eg Rational(100, 10_000)
+```
+
 </details>
 <details>
-<summary>Persistence</summary>
+<summary><code>class Agent < LLM::Agent</code></summary>
 <br>
 
-Set `path:` on an agent for automatic filesystem persistence:
-the agent restores conversation history from the file on startup
-and saves it back after every turn, with no manual serialization
-code. For database-backed persistence, ActiveRecord and Sequel
-integrations are also available. All persistence options use the same
-underlying serialization.
+The examples so far have focused on
+[LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+as an object but you can create a subclass that inherits from
+[LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+too. Both support the same features and accept the same options
+but a subclass can be configured with the `set` method, implement
+its own methods, inherit from other modules and create multiple
+instances of itself.
+
+The
+[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
+method can be used to configure your agent, and
+it accepts all the same options that an instance
+of [LLM::Agent](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+accepts. An option can be resolved lazily via
+a Proc or a Symbol that represents a method
+on your class, and all options are optional -
+you can set many or none, and your subclass
+will still work. For example:
 
 ```ruby
 require "llm"
+require "llm/tools"
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, path: "session.json")
-agent.talk "remember my name is robert"
+class Agent < LLM::Agent
+  set name: "sysadmin",
+      description: "performs system administration tasks",
+      instructions: proc { File.read("prompt.md") },
+      concurrency: :fork,
+      tools: :mytools
 
-# Next time, the conversation is restored automatically:
-agent = LLM::Agent.new(llm, path: "session.json")
-agent.talk "what's my name?"
+  private
+
+  def mytools
+    [LLM::Tool::Exec]
+  end
+end
+
+llm = LLM.deepseek
+agent = Agent.new(llm)
+agent.talk "What's our uptime?", stream: $stdout
 ```
 </details>
-<details><summary>ActiveRecord | Sequel</summary>
+
+#### Database
+
+<details><summary>ActiveRecord </summary>
 <br>
 
-Because both
+Both
 [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) and
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-can be serialized to JSON and stored in a simple string, both ActiveRecord
-and Sequel support can be implemented within a single column on a single row.
-
-The runtime includes first-class support for both ActiveRecord / Sequel, and
-for both Rack-based / Rails-based applications. On databases
-where it is supported, such as PostgreSQL, the column can be optimized by using
-the `jsonb` type.
+can be serialized to JSON and stored in a database column.  The `jsonb`
+column type from PostgreSQL is recommended but it can also be stored as
+a string on other databases. ActiveRecord support is optimized
+for the `jsonb` column type and PostgreSQL.
+
+The column captures everything an agent has done up to that point,
+it includes tool calls and returns, exchanged messages, and other
+metadata that carries runtime state. Each agent is an ActiveRecord
+model that calls `acts_as_agent` and each row represents an instance
+of that agent. It can be used with new and existing models alike.
+
+The column should have the name `data` but this can be
+changed when the `acts_as_agent` method is called (eg
+`acts_as_agent(data_column: :my_column)`). The column
+is updated after every request that an agent makes rather
+than every turn, so an unexpected interrupt can be resumed
+from from the last request and no progress (or spent tokens)
+are lost:
 
 ```ruby
 require "active_record"
 require "llm"
 require "llm/active_record"
 
-##
-# The Robert agent.
-class Robert < ActiveRecord::Base
+class Octocat < ActiveRecord::Base
   acts_as_agent(format: :jsonb) do |agent|
-    agent.set name: "robert",
-              description: "robert is an agent that has access to the official " \
-                            "r.uby.dev GitHub repositories. He can access the repositories " \
-                            "to answer your question(s) about r.uby.dev projects.",
-              instructions: proc { File.read(File.join(__dir__, "robert", "prompt.md")) },
-              tools: :tools,
-              concurrency: :async,
-
-              ##
-              # The maximum number of tool calls per-turn.
-              tool_budget: 25,
-
-              ##
-              # The default tracer that all agents have associated
-              # with them. The tracer exports a trace to a couple of
-              # SQL tables.
-              tracer: proc { Raven::Tracer::SQL.new(llm, agent: self) }
+    agent.set name: "octocat",
+              description: "performs actions on GitHub",
+              instructions: proc { File.read("prompt.md") },
+              tools: :toolset,
+              tracer: proc { SQLTracer.new(llm, agent: self) }
   end
 
+  ##
+  # @return [Array<LLM::Tool>]
+  def toolset
+    github.tools
+  end
+
+  private
+
   ##
   # @return [LLM::MCP]
   def github
@@ -432,32 +707,17 @@ class Robert < ActiveRecord::Base
   end
 
   ##
-  # @return [Array<LLM::Tool>]
-  def tools
-    github.tools.select { allowlist.include?(_1.name.to_s) }
-  end
-
-  private
-
-  def allowlist
-    %w[
-        get_commit
-        get_file_contents
-        list_branches
-        list_commits
-        search_code
-        search_commits
-        search_repositories
-        search_issues
-        pull_request_read
-        list_pull_requests
-        list_issues
-        issue_read
-    ].freeze
+  # @note
+  #  All models who call 'acts_as_agent' must implement
+  #  this method. It returns the provider that the agent
+  #  will use. It can be determined dynamically.
+  # @return [LLM::Provider]
+  def set_provider
+    LLM.deepseek
   end
 end
 
-agent = Robert.create!
+agent = Octocat.create!
 
 ##
 # Every call to `talk` automatically persists
@@ -468,7 +728,7 @@ agent.talk "what's new on the llm.rb repository?"
 # The conversation was persisted to database. A
 # fresh instance restores it and continues where
 # we left off
-agent = Robert.find(agent.id).talk "and what about roda-llm?"
+agent = Octocat.find(agent.id).talk "and what about roda?"
 
 ##
 # Start an agent console.
@@ -477,8 +737,9 @@ agent = Robert.find(agent.id).talk "and what about roda-llm?"
 agent.console
 ```
 </details>
+
 <details>
-<summary> SQL optimizations </summary>
+<summary>SQL optimizations </summary>
 <br>
 
 In a database environment the runtime optimizes for
@@ -572,123 +833,238 @@ the whole column.
 
 </details>
 
-<details><summary>MCP</summary>
+#### Concurrency
+
+<details>
+<summary>Strategies</summary>
 <br>
 
-The Model Context Protocol (MCP) has first-class support
-in llm.rb. The stdio and http transports work out of the
-box. MCP tools are translated into subclasses of
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
-used with
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
+There are six different concurrency strategies
+to choose from. A tool can be spawned on its own
+process, on its own ractor, on its own thread,
+on a thread that manages an `Async::Reactor`, or
+on its own fiber.
+
+Each strategy is built around an abstraction that
+makes them interchangeable - you can switch between
+processes, threads, ractors, async fibers, or a ruby
+fiber by changing a single line of code. **All** subclasses
+of [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
+are supported, so you can launch MCP tools concurrently
+or in parallel as well.
+
+The strategy that you choose depends on the workload
+and the requirements of your application, as well as
+what the strategy offers. The `:fork` and `:ractor`
+strategies provide true parallelism, and the other
+strategies - especially `:thread`, and `:async` -
+can be great for IO-bound tools.
+
+The following example configures an agent to use
+all builtin tools (`LLM::Tool.subclasses`) and it
+spawns each tool in its own process
+(`concurrency: :fork`). It could be changed to use
+async instead with `concurrency: :async`, or with
+threads via `concurrency: :thread`, and so on. The
+default concurrency strategy is `:sequential`, and
+it runs tools sequentially on a single thread.
 
 ```ruby
 require "llm"
+require "llm/tools"
 
-llm   = LLM.deepseek(key: ENV["KEY"])
-mcp   = LLM::MCP.stdio(argv: ["ruby", "server.rb"])
-agent = LLM::Agent.new(llm, stream: $stdout, tools: mcp.tools)
-agent.talk "Run the tool"
+llm   = LLM.deepseek
+tools = LLM::Tool.subclasses
+agent = LLM::Agent.new(llm, tools:, concurrency: :fork)
+agent.talk "implement the feature described in todo.txt"
 ```
+
 </details>
-<details><summary>A2A</summary>
+<details>
+<summary>Cancellation</summary>
 <br>
 
-The Agent 2 Agent (A2A) protocol has first-class support
-in llm.rb. The http and jsonrpc transports work out of the
-box. A2A skills are translated into subclasses of
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
-used with
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
+It is possible to interrupt a running agent who
+is between requests or tool calls as long as the
+cancel request is sent from another thread or fiber
+running in the same process as the agent. A cancel
+request can be sent with the
+[`LLM::Agent#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#interrupt!-instance_method)
+method.
+
+An interrupted tool call is made aware of the interrupt
+and it can both rescue `LLM::Interrupt` and/or implement
+the `on_interrupt` callback on the tool class. The option to
+cancel on [r.uby.dev](https://r.uby.dev) is built on top of
+this feature, and it has a single background process
+with 24 threads. Each thread can run an agent request that
+can be interrupted via another thread in the same process.
+
+It is solid and reliable but r.uby.dev had to meet this
+feature halfway, so expect to build your own infrastructure
+around it. See
+[cancellation chapter](docs/deepdive/advanced/cancellation.md)
+to learn more.
 
 ```ruby
-require "llm"
+class Search < LLM::Tool
+  name "search"
+  description "Search many files"
+  parameter :pattern, String, "The pattern to search for"
+  required %i[pattern]
 
-llm   = LLM.deepseek(key: ENV["KEY"])
-a2a   = LLM::A2A.rest(url: "https://remote-agent.example.com")
-agent = LLM::Agent.new(llm, stream: $stdout, tools: a2a.skills)
-agent.talk "Run the skill"
+  ##
+  # A raise is delivered here; `on_interrupt` is a notification, and it
+  # runs on every strategy - `:sequential` included.
+  def call(pattern:)
+    search(pattern)
+  rescue LLM::Interrupt
+    ##
+    # A tool can return a value from here, and the turn carries on with
+    # it, or re-raise and the fiber that made the request is raised
+    # into as well.
+    cleanup
+    raise
+  end
+
+  ##
+  # Told on the thread or fiber the call runs on, before the raise on
+  # `:fork` and `:ractor` and after the rescue above on the other three.
+  def on_interrupt
+    cleanup
+  end
+
+  private
+
+  def cleanup
+    # Release a file, a socket, or a lock here.
+  end
+end
+
+llm = LLM.deepseek
+agent = LLM::Agent.new(llm, tools: [Search], concurrency: :async)
+queue = Queue.new
+Thread.new { [sleep(queue.pop), agent.interrupt!] }
+
+begin
+  queue.push(3)
+  agent.talk "find every TODO in the repository", stream: $stdout
+rescue LLM::Interrupt
+  puts
+  puts "!! cancelled !!"
+end
 ```
 </details>
-
-<details><summary>Structured outputs</summary>
+<details>
+<summary>Cancel by record ID</summary>
 <br>
 
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
-subclasses produce typed, structured
-output from any model call. Pass a schema to
-[`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk-instance_method),
-[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk-instance_method),
-or
-[`LLM::Provider#complete`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#complete-instance_method)
-to receive validated JSON instead of free text. Schemas work alongside tools and streams.
-
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
-can define objects, arrays, enums, nested schemas,
-and more. It is also used internally by
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) for parameter
-definitions, so you already benefit from it when you declare tool
-parameters.
+A common deployment setup is to run your agents in a
+background process that a web frontend can communicate
+with (usually via a database). The background process
+would have one thread per agent, and it could run as
+many agents as it has threads. This is how the
+[r.uby.dev](https://r.uby.dev) website is configured,
+and it is the configuration that the
+[`LLM.interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM.html#interrupt-class_method)
+method is optimized for: a single process with each agent
+running in its own thread.
 
 The
-[`LLM::DeepSeek`](https://r.uby.dev/api-docs/llm.rb/LLM/DeepSeek.html)
-provider includes runtime-level optimisations such as structured
-output support (despite no official structured outputs API) and
-SVG image generation. This example uses
-[`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html) with
-DeepSeek:
+[`LLM.interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM.html#interrupt-class_method)
+method has access to a process-wide
+registry that contains every active instance of
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html),
+and that includes Sequel and ActiveRecord agents, too. An
+agent enters the registry when it starts a turn, and it
+exits the registry afterwards. The method returns true
+when it sent an interrupt, and otherwise it returns false.
+
+There is often a window between when an agent is queued
+and when it runs, so a poll approach lets you eventually
+interrupt the agent, or give up trying:
 
 ```ruby
-class Weather < LLM::Schema
-  property :city, String, "The city name"
-  property :temperature, Number, "Current temperature"
-  property :conditions, String, "Weather conditions"
-  required %i[city temperature conditions]
+class InterruptJob
+  def call(agent_id:)
+    LLM.interrupt(
+      id: agent_id,
+      attempts: 10,
+      interval: 0.1
+    )
+  end
 end
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, schema: Weather)
-res = agent.talk "Weather in Paris?"
-res.content!  # => {city: "Paris", temperature: 15.0, conditions: "Cloudy"}
 ```
 </details>
-<details><summary>Guards</summary>
+
+#### Hooks
+
+<details><summary>Intercepts</summary>
 <br>
 
-[`LLM::Guard`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html)
-is the hook that sees every tool call before it runs. A guard
-can let a call through, cancel it, block it with an error, or
-even answer for it. Because it runs before the tool, anything
-it intercepts never executes. Policy, validation, quotas, and
-cost ceilings all live here.
-
-Agents and contexts use
-[`LLM::Guard::Null`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard/Null.html)
-by default, so a guard only runs when you configure one. To
-write your own guard, subclass
-[`LLM::Guard`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html)
-and implement
-[`LLM::Guard#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Guard.html#call-instance_method).
-The pending call arrives as `function:`. Return a value to close
-the call, or `nil` to let it run:
+When a model requests a tool call, the
+runtime represents it as a [`LLM::Function`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html)
+object that a subclass of [`LLM::Intercept`](https://r.uby.dev/api-docs/llm.rb/LLM/Intercept.html)
+can see _before_ it is called. An intercept
+can return a
+[`LLM::Function::Return`]((https://r.uby.dev/api-docs/llm.rb/LLM/Function.html))
+object on behalf of the tool call, and so the tool
+call is never made. Or it can return any other
+value to let the tool continue as normal.
+
+Maybe best of all, an intercept can modify
+the arguments given to a [`LLM::Function`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html)
+in-place, before the tool call is made. This
+allows an intercept to modify what the model
+wanted to send to a function, and then let
+the function be called as normal.
+
+This also works for MCP tools - so you might
+be able to imagine an intercept that wrapped
+a comment body at 80 columns, or parsed a
+markdown post into HTML, before the tool
+call is made.
+
+A simple example:
 
 ```ruby
-class PolicyGuard < LLM::Guard
+require "llm"
+require "llm/tools"
+
+class Sandbox < LLM::Intercept
   def call(function:)
     if function.name == "exec"
-      function.return(error: true, type: "policy_error",
-                      message: "exec is disabled")
+      function.return(error: true, type: "permission_error", message: "exec is disabled")
     end
   end
 end
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, tools: [LLM::Tool::Exec, ReadFile], guard: PolicyGuard)
+llm   = LLM.deepseek
+tools = [LLM::Tool::Exec, LLM::Tool::ReadFile]
+agent = LLM::Agent.new(llm, tools:, intercept: Sandbox)
+agent.talk "run 'rm -rf myapp/' right now", stream: $stdout
 ```
-</details>
 
+A multi-intercept example. <br>
+
+Each intercept is called in order they're given
+and the first one that returns an
+[`LLM::Function::Return`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Return.html)
+stops the tool call from being made. This feature
+lets you combine multiple intercepts into a single
+intercept that calls out to all of them:
+
+```ruby
+require "llm"
+
+llm       = LLM.deepseek
+mcp       = LLM::MCP.stdio(argv: ["bundle", "exec", "slack-mcp"])
+intercept = LLM::Intercept.multi(llm, WrapComment, BlockBadLanguage)
+agent     = LLM::Agent.new(llm, tools: [mcp.tools], intercept:)
+
+agent.talk "Tell slack channel I'm AFK", stream: $stdout
+```
+</details>
 <details>
 <summary>Transformers</summary>
 <br>
@@ -706,7 +1082,7 @@ class RedactEmails < LLM::Transformer
   end
 end
 
-llm = LLM.deepseek(key: ENV["KEY"])
+llm = LLM.deepseek
 agent = LLM::Agent.new(llm, transformer: RedactEmails)
 agent.talk "Contact support@example.com for help"
 ```
@@ -729,13 +1105,13 @@ never contains an orphaned result. It is also possible to subclass
 [`LLM::Compactor`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor.html)
 to implement your own compactor with its own logic. Streams can observe the
 process through the
-[`LLM::Stream#on_compaction`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction)
+[`LLM::Stream#on_compaction`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction-instance_method)
 and
-[`LLM::Stream#on_compaction_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction_finish)
+[`LLM::Stream#on_compaction_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction_finish-instance_method)
 callbacks.
 
 ```ruby
-llm = LLM.deepseek(key: ENV["KEY"])
+llm = LLM.deepseek
 agent = LLM::Agent.new(
   llm,
   compactor: LLM::Compactor::Truncate,
@@ -745,119 +1121,66 @@ agent.talk "Hello"
 ```
 </details>
 
-<details>
-<summary>Automatic retries</summary>
-<br>
+#### Everything else
 
-Rate-limited requests are retried automatically by default. Agents
-retry a 429 up to five times with a growing backoff before giving
-up, so most request failures resolve on their own. Connection and
-read timeouts are retried the same way. Set `retry_budget`
-to change the number of retries, or `retry_budget: 0` to disable
-them.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, retry_budget: 0)
-agent.talk "Hello"
-```
-
-</details>
-
-
-<details>
-<summary>Usage and cost</summary>
+<details><summary>A2A</summary>
 <br>
 
-Every context and agent reports what a conversation has spent and how
-much room is left, and the numbers answer different questions. A
-`token_usage` is the whole conversation, summed as an
-[LLM::Usage](https://r.uby.dev/api-docs/llm.rb/LLM/Usage.html), and it
-is what [LLM::Cost](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
-prices against the model registry:
+The Agent 2 Agent (A2A) protocol has first-class support
+in llm.rb. The http and jsonrpc transports work out of the
+box. A2A skills are translated into subclasses of
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be
+used with
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
 
 ```ruby
 require "llm"
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.talk "Hello"
-
-agent.token_usage  # => LLM::Usage for the whole conversation
-agent.cost         # => LLM::Cost, priced from the registry
-```
-
-A `context_used` is one turn's worth - the live size of the most recent
-assistant message - so it is what a context window is really being
-spent on, and `context_usage` is that as a fraction of the window:
-
-```ruby
-agent.context_used    # => tokens in the latest turn
-agent.context_window  # => the model's limit, or nil when unknown
-agent.context_usage   # => Rational, eg Rational(100, 10_000)
+llm   = LLM.deepseek
+a2a   = LLM::A2A.rest(url: "https://remote-agent.example.com")
+agent = LLM::Agent.new(llm, stream: $stdout, tools: a2a.skills)
+agent.talk "Run the skill"
 ```
-
 </details>
 
-
 <details>
-<summary>Observability</summary>
+<summary>Skills</summary>
 <br>
 
-It is possible to trace what an agent is doing by attaching a
-tracer. A tracer can hook into requests, tool calls, and other
-runtime events to debug an agent, provide insights, monitor latency,
-or export spans to an observability backend. All built-in tracers
-share one interface, so switching between them means changing a
-factory method:
+A skill turns a markdown file into a callable tool. When the model
+calls it, the runtime spawns a subagent with the skill's instructions
+as its system prompt and the skill's own tool set. The subagent runs
+one turn and returns the result, then is discarded. Each call
+is fresh and stateless.
 
-* [`LLM::Tracer.pretty_logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#pretty_logger-class_method): human-readable single-line logs to stderr, ideal during development.
-* [`LLM::Tracer.telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#telemetry-class_method):
-exports spans via OTLP for OpenTelemetry in production.
-* [`LLM::Tracer.logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html#logger-class_method):
-structured JSON to stdout or a file.
+A [LLM::Stream](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
+can be notified as a skill starts and when it returns. The `on_skill_return`
+callback hands back the subagent that ran the skill, so you can inspect
+its conversation, measure its usage, track costs or add a verification
+step (eg `subagent.talk("verify your work")`).
 
-It is also possible to create your own tracer by creating a subclass
-of [`LLM::Tracer`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html)
-that implements a number of callbacks that cover an agent's lifecycle.
-The tracer feature provides visibility into what the runtime is doing,
-and the tracer API lets other code hook into that feature.
+##### summary.md
 
-```ruby
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, tracer: LLM::Tracer.pretty_logger(llm))
-agent.talk "Hello"
-```
-</details>
+```markdown
+---
+name: summary
+description: Reads recent git history and writes a summary
+tools: all
+---
 
-<details>
-<summary>As a subclass</summary>
-<br>
+Collect the recent git log, analyze each commit,
+and write a summary to summary.txt.
+```
 
-[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
-is a class-level DSL that accepts a Hash of properties. Each key resolves to a
-corresponding class accessor: `name`, `description`, `model`, `tools`,
-`instructions`, `schema`, `stream`, `tracer`, `concurrency`, `confirm`,
-`path`, `skills`, `tool_budget`, and `retry_budget`. All options are
-optional; zero or more can be set.
-An error is raised for unknown keys so that typos are caught early.
+##### agent.rb
 
 ```ruby
 require "llm"
-require "llm/tools"
 
-class Agent < LLM::Agent
-  set name: "sysadmin",
-      description: "system administration agent",
-      model: "deepseek-v4-pro",
-      tools: [LLM::Tool::Exec]
-end
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = Agent.new(llm)
-agent.talk "Run 'date'"
+llm   = LLM.deepseek
+agent = LLM::Agent.new(llm, skills: ["summary.md"])
+agent.talk "Summarize the last week of work"
 ```
 </details>
 
@@ -869,7 +1192,9 @@ Each provider is constructed with a class-level factory method on
 or
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html). The
 same API drives every one of them, so switching providers is a one-line
-change.
+change. One default differs: an OpenAI context speaks the Responses API
+(`mode: :responses`) unless it is built with `mode: :completions`, while
+every other provider defaults to chat completions.
 
 #### What providers does llm.rb support?
 
@@ -941,13 +1266,19 @@ llm = LLM.bedrock(
 </details>
 
 <details>
-<summary>Model Registry</summary>
+<summary>Registry</summary>
 <br>
 
-Each provider ships its model catalog, pricing, limits, and
-modalities with the gem, sourced from [models.dev](https://models.dev).
-Reach it from any provider, context, or agent, enumerate models, or
-sort them by price.
+The runtime ships with the [models.dev](https://models.dev)
+registry. The registry is a moving target and each new release
+of llm.rb includes an updated copy of it. The registry is loaded
+per-provider, on-demand and it provides information about the limits,
+costs, and capabilities of a model. The runtime uses the registry
+to implement methods like
+[LLM::Agent#cost](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#cost-instance_method)
+and the registry can be accessed through the
+[LLM::Provider#registry](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#registry-instance_method)
+method as well:
 
 ```ruby
 require "llm"
@@ -973,7 +1304,6 @@ libcurl. They share one interface, so switching is a one-word change.
 
 ```ruby
 llm = LLM.deepseek(
-  key: ENV["KEY"],
   transport: :net_http_persistent
 )
 ```
@@ -987,13 +1317,16 @@ Providers accept two timeouts:
 
 * `connect_timeout` - opening the connection. Defaults to 5 seconds.
 * `read_timeout` - waiting for a response on an idle connection.
-  Defaults to 600 seconds (10 minutes).
+  Defaults to 240 seconds (4 minutes).
+
+A timeout is retriable, so it is often better to fail early and open a
+new session than to hold a connection open for a response that never
+comes. This is especially true for `read_timeout`. Both timeouts can
+be given as a custom value that is higher or lower than the default.
 
-The longer read timeout leaves room for slow reasoning models and
-local models. The legacy `timeout:` option remains as a shorthand for
-`read_timeout`. Timeouts are retriable:
+A request that times out is retried automatically. The
 [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-retries a timed out request up to its
+class retries a timed out request up to its
 [`retry_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#retry_budget-class_method)
 (five by default), so a dropped connection or a slow first token
 is often something we can recover from.
@@ -1001,7 +1334,7 @@ is often something we can recover from.
 ```ruby
 llm = LLM.deepseek(
   connect_timeout: 5,   # opening the connection
-  read_timeout: 600     # waiting for the next bytes
+  read_timeout: 600     # raise it when a model needs longer
 )
 ```
 </details>
@@ -1040,7 +1373,7 @@ that here.
 ```ruby
 require "llm"
 
-llm  = LLM.openai(key: ENV["KEY"])
+llm  = LLM.openai
 body = "llm.rb is Ruby's capable AI runtime."
 embedding = llm.embed([body]).embeddings.first
 
@@ -1062,7 +1395,7 @@ the same across providers:
 ```ruby
 require "llm"
 
-llm = LLM.openai(key: ENV["KEY"])
+llm = LLM.openai
 res = llm.images.create(prompt: "a dog on a rocket to the moon")
 IO.copy_stream res.images[0], "rocket.png"
 ```
@@ -1077,7 +1410,7 @@ to maintain a session across generations:
 
 ```ruby
 require "llm"
-llm = LLM.deepseek(key: ENV["KEY"])
+llm = LLM.deepseek
 
 ##
 # First generation
@@ -1091,8 +1424,74 @@ res = llm.images.create(prompt: "add a dog next to the rocket",
 IO.copy_stream res.images[0], "rocket-with-dog.svg"
 ```
 
+## Docs
+
+The [deepdive](docs/deepdive.md) is the reference for everything below
+the quick start. Each chapter is a topic of its own:
+
+- **Fundamentals**:
+  [Providers](docs/deepdive/fundamentals/providers.md),
+  [Agents](docs/deepdive/fundamentals/agents.md),
+  [Tools](docs/deepdive/fundamentals/tools.md),
+  [Stream](docs/deepdive/fundamentals/stream.md),
+  [Schema](docs/deepdive/fundamentals/schema.md),
+  [Skills](docs/deepdive/fundamentals/skills.md)
+- **Features**:
+  [Built-in tools](docs/deepdive/features/builtin_tools.md),
+  [Concurrency](docs/deepdive/features/concurrency.md),
+  [Embeddings](docs/deepdive/features/embeddings.md),
+  [Files](docs/deepdive/features/files.md),
+  [Moderations](docs/deepdive/features/moderations.md),
+  [Responses](docs/deepdive/features/responses.md),
+  [Database](docs/deepdive/features/database.md),
+  [Console](docs/deepdive/features/console.md)
+- **Advanced**:
+  [Context](docs/deepdive/advanced/context.md),
+  [Compaction](docs/deepdive/advanced/compaction.md),
+  [Cancellation](docs/deepdive/advanced/cancellation.md),
+  [Intercept](docs/deepdive/advanced/intercept.md),
+  [Transformer](docs/deepdive/advanced/transformer.md),
+  [Transports](docs/deepdive/advanced/transports.md)
+- **Media**:
+  [Images](docs/deepdive/media/images.md),
+  [Audio](docs/deepdive/media/audio.md),
+  [OCR](docs/deepdive/media/ocr.md)
+- **Protocols**:
+  [MCP](docs/deepdive/protocols/mcp.md),
+  [A2A](docs/deepdive/protocols/a2a.md)
+- **Reference**:
+  [LLM::Response](docs/deepdive/reference/response.md),
+  [LLM::Object](docs/deepdive/reference/object.md),
+  [LLM::Cost](docs/deepdive/reference/cost.md),
+  [Tracer](docs/deepdive/reference/tracer.md),
+  [Model registry](docs/deepdive/reference/model_registry.md)
+
 ## FAQ
 
+<details>
+<summary>Does it work on Windows?</summary>
+<br>
+I think so. Our Windows build is green on CI.
+That's the only signal I have to go by.
+
+However support is limited compared to
+other platforms. The `fork` strategy is not
+available (Windows does not provide fork),
+and the `ractor` strategy has one test out
+of many that hangs - so it is assumed ractor
+support is not perfect.
+
+The `curses` gem that the console requires has
+to be compiled from source, and this can be a
+challenge on Windows without a compiler toolchain.
+By default the console does not work but it might
+work if you are able to install the `curses` gem.
+
+So it mostly works as long as you avoid the
+parts that don't work. Pull requests that
+improve Windows support are very welcome.
+</details>
+
 <details>
 <summary>Where can I see llm.rb in action?</summary>
 <br>
@@ -1115,42 +1514,8 @@ are running on your own hardware.
 <summary>I have a limited budget. What should I do?</summary>
 <br>
 
-There are a few options. The first option is to host
-your own model, and use the ollama or llamacpp
-providers. This can be difficult though because
-a capable model requires hardware that can
-match it. If you have the ability to self-host,
-this would be my first option.
-
-The second option is DeepSeek. <br>
-The deepseek-v4-flash model costs pennies to use. <br>
-And llm.rb has been optimized for deepseek. For example,
-DeepSeek does not have image generation capabilities
-but on the llm.rb runtime it does (vector graphics only,
-though).
-
-The same is true for structured outputs. DeepSeek does
-not support structured outputs in the same way as OpenAI or
-Google, but the llm.rb runtime makes it appear as
-though it does, through the `json_object` response
-type.
-
-If you're on a budget, DeepSeek is hard to beat.
-</details>
-<details>
-<summary>Sources other than GitHub?</summary>
-<br>
-<p>
-We are on the <a href="https://radicle.network">radicle.network</a> as well.
-<br>
-Every commit that lands on GitHub also lands on Radicle.
-<br>
-Our repository ID is z2PtfQ6dYwyYaW2aGrztG1sMyDmCE.
-<br>
-Browse on <a
-href="https://radicle.network/nodes/iris.radicle.network/z2PtfQ6dYwyYaW2aGrztG1sMyDmCE">the
-web</a>.
-</p>
+If you're on a budget, [DeepSeek](https://platform.deepseek.com)
+is hard to beat and it works really well with llm.rb.
 </details>
 
 <details>
@@ -1162,9 +1527,9 @@ years ago by
 [@altruby](https://github.com/altruby) and
 [@antaz](https://github.com/altruby). The primary
 maintainer is [@altruby](https://github.com/altruby).
-Over those three years multiple other contributors have
-contributed to llm.rb as well, and new contributors are
-always welcome.
+There have been multiple other contributors over the
+last three years and new contributors are always
+welcome.
 </details>
 
 <details>
@@ -1173,18 +1538,9 @@ always welcome.
 
 It is battle tested daily.
 
-The console that is distributed with llm.rb is used
-to build llm.rb so there is a healthy, active feedback
-loop. It also powers the [r.uby.dev](https://r.uby.dev)
-website where multiple llm.rb agents are deployed. I'm
-aware of at least one production Rails deployment
-at a large-ish company.
-
-And this git repository includes llm.rb agents that help me
-maintain the documentation and perform other repository
-maintainence. The feedback loop is constant. Outside of that
-there is a large test suite that covers live requests (recorded
-by VCR) and database interactions.
+It powers the [r.uby.dev](https://r.uby.dev)
+website and I'm aware of at least one production
+Rails deployment at a large-ish company.
 </details>
 
 ## License