llm.rb 12.6.0 → 13.1.0

data/resources/deepdive.md

@@ -14,1220 +14,67 @@
 
 ## Welcome
 
-Welcome to the llm.rb deepdive. You are reading this document
-in the markdown format. An optimized version exists
-at [https://r.uby.dev/llm/deepdive](https://r.uby.dev/llm/deepdive)
-and it is both easier to read and navigate.
-
-This document is a continuation of the [homepage documentation](https://r.uby.dev/llm).
-It assumes you are familiar with the basics already, and focuses on
-features that didn't make it into the homepage documentation.
-
-## Table of contents
-
-**Overview**
-
-- [Welcome](#welcome)
-
-**Core**
-
-<details>
-<summary>Agents</summary>
-
-- [As a subclass](#as-a-subclass)
-- [As an object](#as-an-object)
-</details>
-
-<details>
-<summary>Tools</summary>
-
-- [LLM::Tool](#llmtool)
-- [Errors](#errors)
-</details>
-
-<details>
-<summary>Skills</summary>
-
-- [SKILL.md](#skillmd)
-- [Run it](#run-it)
-</details>
-
-<details>
-<summary>Schema</summary>
-
-- [Estimation](#estimation)
-</details>
-
-**Runtime**
-
-<details>
-<summary>Stream</summary>
-
-- [IO-like object](#io-like-object)
-- [LLM::Stream](#llmstream)
-</details>
-
-<details>
-<summary>Cancellation</summary>
-
-- [Cancel a request](#cancel-a-request)
-- [Tool interrupts](#tool-interrupts)
-</details>
-
-<details>
-<summary>Transports</summary>
-
-- [net/http](#nethttp)
-- [net/http/persistent](#nethttppersistent)
-- [curb](#curb)
-</details>
-
-<details>
-<summary>Tracer</summary>
-
-- [Provider-wide tracer](#provider-wide-tracer)
-- [Agent-local tracer](#agent-local-tracer)
-</details>
-
-<details>
-<summary>REPL</summary>
-
-- [LLM::Agent](#llmagent)
-- [State](#state)
-- [Tools](#tools)
-- [Skills](#skills-1)
-- [Tracer](#tracer-1)
-- [Input](#input)
-- [Commands](#commands)
-</details>
-
-**Persistence**
-
-<details>
-<summary>Serialization</summary>
-
-- [Save to disk](#save-to-disk)
-</details>
-
-<details>
-<summary>ORM</summary>
-
-- [ActiveRecord](#activerecord)
-- [Sequel](#sequel)
-</details>
-
-**Media**
-
-<details>
-<summary>Images</summary>
-
-- [Generation](#generation)
-- [Edits](#edits)
-- [DeepSeek](#deepseek)
-</details>
-
-<details>
-<summary>Audio</summary>
-
-- [text-to-speech](#text-to-speech)
-- [speech-to-text](#speech-to-text)
-- [translation](#translation)
-</details>
-
-**Protocols**
-
-<details>
-<summary>MCP</summary>
-
-- [stdio](#stdio)
-- [http](#http)
-</details>
-
-<details>
-<summary>A2A</summary>
-
-- [rest](#rest)
-- [jsonrpc](#jsonrpc)
-</details>
-
-## Agents
-
-An agent is represented by the
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-class, and it is built on top of
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) -
-the heart of the runtime. An agent manages the tool loop automatically,
-implements a tool loop guard for misbehaving models, and
-it can use five different concurrency strategies to execute
-tools.
-
-An agent can be a subclass of
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html),
-or a direct
-instance of it. The subclass approach is useful when you
-want reusable agents that can attach behavior (as methods)
-to their own class.
-
-#### As a subclass
-
-A subclass of
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-can define its model, tools,
-and other attributes at the class-level. All of these
-attributes are optional, and they act as defaults that
-can be overriden on the instance level.
-
-The example uses the `:fork` concurrency model. It has
-two primary benefits: tools are run in parallel, and in
-a separate process with a separate memory address space.
-
-The example purposefully demonstrates how the attributes
-can be lazily defined with a block, or a Symbol that is
-evaluated as an instance method on the subclass. It is
-not strictly neccessary, though, and the example would
-be simpler without it.
-
-```ruby
-class Agent < LLM::Agent
-  set model: "deepseek-v4-pro",
-      tools: [DoResearch, FinalizeResearch, ActOnResearch],
-      stream: -> { $stdout },
-      tracer: :set_tracer,
-      concurrency: :fork
-
-  def research!
-    talk "start the research"
-  end
-
-  private
-
-  def set_tracer
-    LLM::Tracer::Logger.new(llm, io: $stderr)
-  end
-end
-llm   = LLM.deepseek(key: ENV["KEY"])
-agent = Agent.new(llm).tap(&:research!)
-agent.talk "How did the research go?"
-```
-
-#### As an object
-
-The more direct, and sometimes more convienent approach, is to
-create an instance of
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-directly. The same attributes can be provided as the
-second argument given to
-[`LLM::Agent.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html),
-and the same lazy evaluation rules apply. This approach can be
-great for prototyping quickly, and you can always turn to a
-subclass later if that makes more sense.
-
-```ruby
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, stream: $stdout)
-agent.talk "Hello, fellow agent"
-```
-
-[Back to top](#table-of-contents)
-
-## Tools
-
-A tool extends the capabilities of a model. <br>
-A tool is a subclass of
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
-that has a name,
-a description, and an optional set of typed parameters.
-
-A tool also has a method associated with it, and when the
-model calls a tool it will do so through this method &ndash;
-alongside any parameters the tool might have defined.
-
-In other words, a tool provides a way for a model to
-call a method you have written, and it returns a value
-to the model that is considered the tool's response.
-The model then proceeds to process the tool's response,
-and then might generate its own response, or perhaps call
-another tool.
-
-#### LLM::Tool
-
-A tool can be defined by subclassing
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
-with
-a name, description, and optional set of parameters. The
-tool name, and description should be informative so the
-model can understand what the tool does and how it can
-serve a user's query.
-
-```ruby
-require "llm"
-require "shellwords"
-
-class Shell < LLM::Shell
-  name "shell"
-  description "execute a shell command"
-  parameter :name, String, "the command's name"
-  parameter :arguments, Array[String], "One or more arguments"
-  required %i[name]
-  defaults arguments: []
-
-  def call(name:, arguments:)
-    out = `#{name.shellscape} #{arguments.map(&:shellescape).join(" ")}`
-    {ok: $?.success?, out:}
-  end
-end
+### Introduction
 
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, tools: [Shell], stream: $stdout)
-agent.talk "What files are in the current working directory?"
-```
+#### Overview
 
-#### Errors
+Welcome to the llm.rb deepdive. This document is a continuation
+of the [homepage documentation](https://r.uby.dev/llm). It assumes
+you are familiar with the basics already, and focuses on features
+that didn't make it into the homepage documentation.
 
-Exceptions that might be raised by a tool are automatically
-rescued and returned to the model as a structured error.
-Otherwise &ndash; the conversation's history could be left
-in an invalid state.
-
-That's because a tool call must complete with a tool response,
-that's the only valid response a model expects, so even in the
-case of an error, something must be returned that communicates
-what happened.
-
-```ruby
-class Error < LLM::Tool
-  name "error"
-  description "demo how errors are handled"
+An optimized version exists
+at [https://r.uby.dev/llm/deepdive](https://r.uby.dev/llm/deepdive)
+that is both easier to read and navigate.
 
-  ##
-  # Returns
-  # {error: true, kind: "RuntimeError", message: "boom"}
-  def call
-    raise "boom"
-  end
-end
-```
+#### How it works
 
-## Skills
+Each topic file follows a consistent four-part pattern:
+`#### Overview` introduces the concept, `#### How it works` shows
+code, `#### Why would I use it?` explains the use case, and
+`#### Notes` covers caveats and edge cases.
 
-The skill concept is borrowed from tools like Claude and
-Codex, but llm.rb gives it a runtime of its own. A skill
-is a directory with a `SKILL.md` file. That file contains
-frontmatter where the skill's name, description, and tools
-can be declared.
+#### Why would I use it?
 
-#### SKILL.md
+The deepdive documents everything the homepage leaves out:
+advanced patterns, configuration options, ORM integrations,
+protocol support, and edge cases. Read it when you need to go
+beyond the basics.
 
-The `SKILL.md` file can look like this. When a skill runs,
-the runtime spawns a subagent with its own context window
-and message history. Some context is inherited from the
-parent agent, though.
+#### Notes
 
-By default the subagent can only access the tools declared
-by the skill. The `inherit` directive lets it inherit the
-parent agent's tools instead, including A2A and MCP tools.
+The deepdive is a living document. Sections are added as new
+features land. The [homepage](https://r.uby.dev/llm) is the best
+place to start if you are new to llm.rb.
 
-```markdown
----
-name: git-skill
-description: reads my git history and writes a summary
-tools: ['git-log', 'git-show', 'write-file']
 ---
 
-## Task
-
-Collect a log of recent history.
-Analyze each commit.
-Write a summary to summary.txt
-```
-
-#### Run it
-
-Given the skill above, llm.rb only needs the path to the
-directory that contains `SKILL.md`. Under the hood, a skill
-is represented as a tool the model can call. That means
-a skill can be called whenever it satisfies the user's
-request &ndash; in the same way that a regular tool can.
-
-This feature also works with both the ActiveRecord, and
-Sequel integrations.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, skills: [__dir__])
-agent.talk "run the git skill"
-```
-
-[Back to top](#table-of-contents)
-
-## MCP
-
-#### stdio
-
-The stdio transport connects to an MCP server that is launched as a
-separate process, and both its standard input and standard output
-streams are used for communication. It is recommended but not
-required to execute commands for a stdio transport over a
-persistent session via the
-[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
-method &ndash; otherwise
-you could end up launching the same process multiple times.
-
-```ruby
-require "llm"
-
-llm   = LLM.deepseek(key: ENV["KEY"])
-mcp   = LLM::MCP.stdio(argv: ["npx", "-y", "@forgejo/mcp-server"])
-agent = LLM::Agent.new(llm)
-
-mcp.session do
-  agent.talk "What's happening on forgejo?", tools: mcp.tools
-end
-```
-
-#### http
-
-The http transport connects to an MCP server over HTTP, and unlike
-the stdio transport, the MCP server does not have to be running
-locally. Popular services like GitHub provide their own MCP server
-over HTTP, and it is one of the most capable MCP servers I have
-used.
-
-Unlike the stdio transport,
-[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session-instance_method)
-carries little benefit for the http transport and it can be
-omitted.  It is recommended to consider the `net_http_persistent`
-transport for MCP interactions that run over HTTP, otherwise
-you could end up tearing down and setting up the same connection
-multiple times.
-
-```ruby
-require "llm"
-
-llm   = LLM.deepseek(key: ENV["KEY"])
-mcp   = LLM::MCP.http(
-  url: "https://api.githubcopilot.com/mcp/",
-  headers: {
-    "Authorization" => "Bearer #{ENV.fetch('GITHUB_PAT')}"
-  },
-  transport: :net_http_persistent
-)
-agent = LLM::Agent.new(llm)
-agent.talk "What's happening on GitHub?", tools: mcp.tools
-```
-
-[Back to top](#table-of-contents)
-
-## A2A
-
-#### rest
-
-The rest transport communicates with other agents via A2A
-endpoints that speak both HTTP and JSON. The skills advertised
-by an agent become subclasses of
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
-that can be used by both
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html),
-and [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-&ndash; similar to how MCP tools become subclasses of
-[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html).
-
-```ruby
-require "llm"
-
-llm   = LLM.deepseek(key: ENV["KEY"])
-a2a   = LLM::A2A.rest(url: "https://agent.example.com")
-agent = LLM::Agent.new(llm, tools: a2a.skills)
-agent.talk "What's happening, fellow agent?"
-```
-
-#### jsonrpc
-
-The jsonrpc transport communicates with other agents via HTTP
-and a protocol known as jsonrpc. Sometimes an agent will
-implement both, or just one of each. An agent's card, which
-is represented by an instance of
-[`LLM::A2A::Card`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html),
-can be
-used to discover available transports via the
-[`LLM::A2A::Card#interfaces`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A/Card.html#interfaces-instance_method)
-method.
-
-```ruby
-require "llm"
-llm   = LLM.deepseek(key: ENV["KEY"])
-a2a   = LLM::A2A.jsonrpc(url: "https://agent.example.com")
-agent = LLM::Agent.new(llm, tools: a2a.skills)
-agent.talk "What's happening, fellow agent?"
-```
-
-[Back to top](#table-of-contents)
-
-## Transports
-
-The [`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html),
-[`LLM::MCP`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html), and
-[`LLM::A2A`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A.html) classes
-all accept a `transport` option that decides which library
-will be used for HTTP communication. There are three options out
-of the box:
-[`net-http`](https://github.com/ruby/net-http),
-[`net-http-persistent`](https://github.com/drbrain/net-http-persistent),
-and [`curb`](https://github.com/taf2/curb).
-
-#### net/http
-
-The [`net/http`](https://github.com/ruby/net-http) transport is represented by the symbol `:net_http`. <br>
-It is the default transport.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: "...", transport: :net_http)
-mcp = LLM::MCP.http(url: "...", transport: :net_http)
-a2a = LLM::A2A.rest(url: "...", transport: :net_http)
-```
-
-#### net/http/persistent
-
-The [`net/http/persistent`](https://github.com/drbrain/net-http-persistent) transport is represented by the symbol `:net_http_persistent`. <br>
-It maintains a connection pool so the cost of tearing down and
-setting up a connection repeatedly is kept low, and it is built
-on top of [`net/http`](https://github.com/ruby/net-http).
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: "...", transport: :net_http_persistent)
-mcp = LLM::MCP.http(url: "...", transport: :net_http_persistent)
-a2a = LLM::A2A.rest(url: "...", transport: :net_http_persistent)
-```
-
-#### curb
-
-The [`curb`](https://github.com/taf2/curb) transport is represented by the symbol `:curb`. <br>
-It provides bindings for libcurl &ndash; a widely used, highly portable
-and feature-rich HTTP library written in C.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: "...", transport: :curb)
-mcp = LLM::MCP.http(url: "...", transport: :curb)
-a2a = LLM::A2A.rest(url: "...", transport: :curb)
-```
-
-[Back to top](#table-of-contents)
-
-## Stream
-
-#### IO-like object
-
-Any object that implements the `#<<` method can receive
-chunks from a stream. That includes objects like `$stdout`.
-This form of streaming is simple and limited. It is the
-equivalent of
-[`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content-instance_method),
-and doesn't include
-any of the other
-[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
-hooks.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, stream: $stdout)
-agent.talk "hello world"
-```
-
-#### LLM::Stream
-
-The [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
-class provides many hooks that a subclass
-can implement. They range from being notified when a tool call
-starts to when a tool call finishes, or when a conversation is
-due to be compacted because the context window exceeded a defined
-limit. All these callbacks support a responsive user interface
-where the user is always aware of what is happening behind the
-scenes.
-
-```ruby
-class Stream < LLM::Stream
-  def on_content(content)
-    puts content
-  end
-
-  def on_reasoning_content(content)
-    puts content
-  end
-
-  def on_tool_call(tool)
-    # this callback can be used to either log a tool call,
-    # or execute a tool call during a stream.
-  end
-
-  def on_tool_return(tool, result)
-  end
-
-  def on_compaction(ctx, compactor)
-    # this callback is called *before* a compact happens
-  end
-
-  def on_compaction_finish(ctx, compactor)
-    # this callback is called *after* a compact happens
-  end
-end
-```
-
-[Back to top](#table-of-contents)
-
-## Serialization
-
-The [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
-class can be serialized to JSON and stored in a string or on disk.
-That is powerful because a context contains runtime state that can
-be restored later, in a different process or even on a different
-machine. And because an agent is implemented on top of
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
-this feature works for [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html),
-too.
-
-#### Save to disk
-
-The runtime can serialize its state to a string, a text file, or
-a database column. The option that fits best depends on your application
-and environment. Web applications might be more interested in the [ORM](#orm)
-feature, which is built on top of the serialization feature.
-
-```ruby
-##
-# Create a provider
-llm = LLM.deepseek(key: ENV["KEY"])
-
-##
-# Save agent
-agent1 = LLM::Agent.new(llm)
-agent1.talk "remember my name is robert"
-agent1.save(path: "agent.json")
-
-##
-# Restore agent
-agent2 = LLM::Agent.new(llm, stream: $stdout)
-agent2.restore(path: "agent.json")
-agent2.talk "what's my name?"
-```
-
-## ORM
-
-Both ActiveRecord, and Sequel have first-class support on the
-llm.rb runtime. In both cases an ActiveRecord or Sequel model
-can be turned into a model that has the same capabilities as
-[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html),
-or [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
-
-The main difference is that the runtime persists directly into
-the database with no requirements beyond a single column on a
-single row. That means it is usually trivial to turn an existing
-model into an AI-aware model.
-
-#### ActiveRecord
-
-The ActiveRecord interface for
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
-is
-[`acts_as_agent`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord/ActsAsAgent.html).
-It yields an instance of
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html),
-and that can be used
-to configure the agent (eg which model, instructions, skills,
-tools, etc).
-
-An interesting option is the `format` option, by default it
-defaults to `:string` but it can also be changed to `:json`
-or `:jsonb` depending on the configuration and type of underlying
-column. The JSONB column type is recommended.
-
-```ruby
-require "active_record"
-require "llm"
-require "llm/active_record"
-
-class Agent < ApplicationRecord
-  acts_as_agent(format: :jsonb) do |agent|
-    agent.model "deepseek-v4-pro"
-    agent.instructions "solve the user's query"
-    agent.tools [Research, FinalizeResearch, ActOnResearch]
-  end
-
-  private
-
-  ##
-  # By convention, this method defines the provider
-  # for a model. If neccessary, it can be renamed and
-  # configured via `provider: :your_method` instead.
-  def set_provider
-    LLM.deepseek(key: ENV["KEY"])
-  end
-
-  ##
-  # By convention, this method should return what is
-  # given as the second argument to `LLM::Context` or
-  # `LLM::Agent`.
-  #
-  # Often, there is no need to set it, so it can be left
-  # undefined or it can be reassigned in the same way as
-  # `set_provider`. For example: `context: :your_method`
-  def set_context
-    {}
-  end
-end
-
-agent = Agent.create!
-agent.talk "perform research"
-```
-
-#### Sequel
-
-The following is a Sequel equivalent to the ActiveRecord example,
-but to keep it interesting and informative, this example also
-configures a per-model tracer that logs to `$stdout`. Works the
-same for ActiveRecord.
-
-```ruby
-require "sequel"
-require "llm"
-require "llm/sequel/plugin"
-
-class Agent < Sequel::Model
-  plugin(:agent, format: :jsonb) do |agent|
-    agent.model "deepseek-v4-pro"
-    agent.instructions "solve the user's query"
-    agent.tools [Research, FinalizeResearch, ActOnResearch]
-    agent.tracer { LLM::Tracer::Logger.new(llm, io: $stdout) }
-  end
-
-  private
-
-  def set_provider
-    LLM.deepseek(key: ENV["KEY"])
-  end
-end
-
-agent = Agent.create
-agent.talk "perform research"
-```
-
-[Back to top](#table-of-contents)
-
-## Schema
-
-The [`LLM::Schema`](https://r.uby.dev/api-docs/llm.rb/LLM/Schema.html)
-class can be subclassed to describe
-the shape of a JSON object or objects that you expect
-the model to respond with.
-
-It can be useful for a wide range of use cases but the
-most popular might be classification, data extraction,
-and transferring structured data between different software
-rather than blobs of text that a machine cannot easily parse
-in a structured way.
-
-#### Estimation
-
-The following example asks the model to estimate the age
-of a person in a photo. The model provides a structured response
-that's represented by an instance of
-[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html).
-
-The object returned by
-[`LLM::Response#content!`](https://r.uby.dev/api-docs/llm.rb/LLM/Contract/Completion.html#content!-instance_method)
-has methods that can access the age, confidence, and comments
-properties.
-This approach can also work for extracting data or an analysis
-from a PDF, and other file types.
-
-```ruby
-require "llm"
-require "pp"
-
-class Estimation < LLM::Schema
-  property :age, Integer, "The estimated age of the person"
-  property :confidence, Number, "Your confidence in the estimate"
-  property :applicable, Boolean, "True when the photo contains a person"
-  property :comments, String, "Any additional comments or input"
-  required %i[age confidence applicable comments]
-end
-
-llm = LLM.openai(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, schema: Estimation)
-res = agent.ask "Given this photo, provide an age estimate", with: "photo.jpg"
-
-##
-# Coerces the model's response from a JSON string
-# to an instance of LLM::Object.
-estimate = res.content!
-
-##
-# Let's print the estimate
-if estimate.applicable
-  print "The person is approx ", estimate.age.to_s, " years old", "\n"
-  print "I have a confidence rating of ", estimate.confidence.to_s, "\n"
-else
-  print "This photo is not applicable:", "\n"
-  print estimate.comments
-end
-```
-
-[Back to top](#table-of-contents)
-
-## Cancellation
-
-#### Cancel a request
-
-A common scenario when communicating with a model is to
-want to cancel the request mid-stream. This could be done
-for a number of different reasons, most often because the
-user made a mistake, or the model is making a mistake and
-the user wants to cancel the action.
-
-The runtime has built-in support for cancellation. So for
-example it is possible to cancel a request on the main
-thread from a secondary thread. A number of things happen
-when a request is cancelled. First the request is cancelled
-at the transport level, and each transport handles it a little
-differently. The net effect in every case is that the connection
-is closed.
-
-The runtime then notifies the rest of the system. so for example,
-if a tool was running, it will receive the `on_interrupt` / `on_cancel`
-callback that lets the tool do any necessary cleanup, or execute its own
-cancellation plan. Tools that were pending (not yet run but requetsed to
-run) are cancelled through
-[`LLM::Function#cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#cancel-instance_method).
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["DEEPSEEK_SECRET"])
-agent = LLM::Agent.new(llm)
-queue = Queue.new
-
-Thread.new do
-  queue.push(nil)
-  sleep(2)
-  agent.cancel!
-end
-
-begin
-  queue.pop
-  agent.talk "write me a very long poem", stream: $stdout
-rescue LLM::Interrupt
-  puts "request cancelled!"
-end
-```
-
-#### Tool interrupts
-
-When a running tool is interrupted &ndash; for example the user presses
-ESC in the [REPL](#repl) &ndash; the runtime raises
-[`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
-on the tool's execution context. This behavior is uniform across
-all six concurrency strategies (`:call`, `:thread`, `:fiber`,
-`:task`, `:fork`, and `:ractor`).
-
-A tool has two choices:
-
-**Re-raise** `LLM::Interrupt` to cancel the entire turn. The
-exception propagates out of the tool loop and the request is
-aborted. This is the default when you don't rescue the exception.
-
-```ruby
-def call
-  # ... do work ...
-rescue LLM::Interrupt
-  cleanup
-  raise   # cancel the turn
-end
-```
-
-**Return a value** to continue the tool loop. The model receives
-the result and decides what to do next, aware that the tool was
-interrupted.
-
-```ruby
-def call
-  # ... do work ...
-rescue LLM::Interrupt
-  cleanup
-  {ok: false, reason: "interrupted"}  # continue the loop
-end
-```
-
-The right choice depends on the situation. A hard cancel aborts
-the request outright &ndash; useful when continuing would produce
-garbage. Returning a value lets the model adapt, which can be
-helpful when the interrupt is temporary (e.g. a timeout).
-
-The `:ractor` strategy is cooperative by nature &ndash; the ractor
-must check for a cancel message at safe points in its execution.
-The `:fork` strategy delivers the interrupt via a signal, which
-interrupts blocking syscalls immediately. All other strategies
-raise the exception directly on the executing thread or fiber.
-
-[Back to top](#table-of-contents)
-
-## Tracer
-
-The runtime can be observed by subclasses of
-[`LLM::Tracer`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html). <br>
-The default tracers include a tracer that can write to standard
-output
-([`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html)),
-and a generic OpenTelemetry tracer that can export spans via OTLP
-([`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html)).
-
-llm.rb has numerous hooks implemented throughout the runtime that
-[`LLM::Tracer`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html)
-subclasses can hook into, and the tracer is
-purposefully designed to be extensible. The scope of a trace
-can vary from an individual agent (an instance of
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)),
-or for every request a provider makes (an indirect instance of
-[`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)).
-
-#### Provider-wide tracer
-
-The following two examples demonstrate provider-wide tracers that
-cover every request made for a single provider.
-
-```ruby
-##
-# Provider-wide tracer
-# Writes to $stdout
-llm = LLM.deepseek(key: ENV["KEY"])
-llm.tracer = LLM::Tracer::Logger.new(llm, io: $stdout)
-
-##
-# Provider-wide tracer
-# Writes to deepseek.log
-llm = LLM.deepseek(key: ENV["KEY"])
-llm.tracer = LLM::Tracer::Logger.new(llm, path: "deepseek.log")
-```
-
-#### Agent-local tracer
-
-The next two examples demonstrate a tracer that is local
-to an agent.
-
-```ruby
-##
-# Agent-local
-# Writes to $stdout
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, tracer: LLM::Tracer::Logger.new(llm, io: $stdout))
-
-##
-# Agent-local
-# Writes to deepseek-agent.log
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, tracer: LLM::Tracer::Logger.new(llm, path: "deepseek-agent.log"))
-```
-
-[Back to top](#table-of-contents)
-
-## REPL
-
-During the development and operation of agents it can often
-be helpful to drop into a read-eval-print loop. This gives
-you a way to confirm the work was successful, inspect
-anything that went wrong, and keep talking to the same
-agent while its state is still intact.
-
-The REPL is a curses-based TUI with a status line showing
-a context-usage bar and cost counter, a scrollable transcript
-that renders markdown, and a multi-line input area. The UI
-thread stays responsive while a second thread communicates
-with the model.
-
-#### LLM::Agent
-
-The [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)
-method allows an agent to spawn a read-eval-print loop
-that can be useful while developing or operating agents.
-It can be used to debug tool calls, confirm an
-agent has done what was expected, or improve an agent by
-asking questions about what it has done up to that point.
-
-This feature requires that the [curses](https://github.com/ruby/curses)
-and [kramdown](https://github.com/gettalong/kramdown) libraries are
-installed and available to require.
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl
-```
-
-#### State
-
-The `path:` option accepts a file path where runtime state
-is read from and written to. This lets you resume a
-conversation across REPL sessions. When the file does not
-exist the agent starts fresh; when it does, the agent
-restores its previous state.
-
-```ruby
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(path: "session.json")
-```
-
-#### Tools
-
-The read-eval-print loop accepts a `tools` option that lets
-you attach additional tools for the duration of the session.
-
-```ruby
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(tools: [Debugger])
-```
-
-#### Skills
-
-The read-eval-print loop also accepts a `skills` option.
-This can be useful when you want to load extra skills
-without attaching them to an agent permanently.
-
-```ruby
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm)
-agent.repl(skills: [__dir__])
-```
-
-#### Tracer
-
-By default the tracer is disabled for the duration of the
-session. This can be configured through the
-`tracer` option. Setting it to `true` will configure
-the REPL to use the tracer associated with an instance
-of [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
-
-```ruby
-llm = LLM.deepseek(key: ENV["KEY"])
-agent = LLM::Agent.new(llm, tracer: LLM.logger(llm, path: "agent.log"))
-agent.repl(tracer: true, tools: [Debugger])
-```
-
-#### Input
-
-The input area supports several keyboard shortcuts:
-
-| Key | Action |
-|---|---|
-| `Enter` | Submit the current prompt |
-| `Ctrl+A` | Jump to the start of the line |
-| `Ctrl+E` | Jump to the end of the line |
-| `Ctrl+F` | Move the cursor forward |
-| `Ctrl+K` | Erase from cursor to the end of the line |
-| `Ctrl+Y` | Paste previously killed text |
-| `Ctrl+D` | Delete the character at the cursor |
-| `Left / Right` | Move the cursor |
-| `Up / Down` | Scroll the transcript |
-
-When characters arrive faster than a threshold the REPL
-detects that text is being pasted rather than typed. In
-paste mode pressing `Enter` inserts a newline instead of
-submitting, allowing multi-line prompts.
-
-#### Commands
-
-Commands are recognized by a `/` prefix on the input line.
-The built-in command is `/exit`, which leaves the REPL.
-The [`LLM::Repl::Command`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html)
-class can be subclassed to add custom commands &ndash;
-any subclass that defines a `name`, `description`, and
-`call` method is automatically registered and available.
-
-```ruby
-class LLM::Repl::Command
-  class Clear < self
-    name "clear"
-    description "clears the transcript"
-
-    def call
-      system("clear") || system("cls")
-    end
-  end
-end
-```
-
-[Back to top](#table-of-contents)
-
-## Images
-
-The OpenAI, Google, xAI, DeepInfra, and DeepSeek providers have
-builtin image generation capabilities. OpenAI, xAI, and DeepInfra
-also support image edits. Google only supports image generation.
-DeepSeek supports generation and edits too, but only through SVG
-output rather than raster image models.
-
-#### Generation
-
-The [`LLM::Provider#images`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#images-instance_method)
-method returns an Image
-object that a subset of providers implement. At the
-moment Google, xAI, OpenAI, DeepInfra, and DeepSeek have image
-generation capabilities. DeepSeek is the odd one out: it generates
-SVG documents rather than raster images.
-
-```ruby
-require "llm"
-
-##
-# Store dogrocket.png
-llm = LLM.openai(key: ENV["KEY"])
-res = llm.images.create(prompt: "a dog on a rocket to the moon")
-IO.copy_stream res.images[0], "dogrocket.png"
-```
-
-The API is the same across providers. <br>
-For example &ndash; xAI:
-
-```ruby
-require "llm"
-
-##
-# Store dogrocket.png
-# Same API as OpenAI
-llm = LLM.xai(key: ENV["KEY"])
-res = llm.images.create(prompt: "a dog on a rocket to the moon")
-IO.copy_stream res.images[0], "dogrocket.png"
-```
-
-#### Edits
-
-OpenAI, xAI, and DeepInfra have the same interface for image edits. <br>
-DeepSeek also supports edits, but only for SVG files. <br>
-Google does not have edit image support. <br>
-
-```ruby
-require "llm"
-
-##
-# Edit self.jpg and add a mustache
-# Save to mustache.png
-llm = LLM.openai(key: ENV["KEY"])
-res = llm.images.edit(prompt: "add a mustache", image: "self.jpg")
-IO.copy_stream res.images[0], "mustache.png"
-```
-
-#### DeepSeek
-
-The DeepSeek provider does not provide an image generation model
-but it is possible to ask a text-to-text model to produce
-vector graphics (SVGs), and in that limited sense, it can become
-a capable text-to-image model.
-
-```ruby
-require "llm"
-
-##
-# Edit rocket.svg and change its color
-# Save to rocket-edited.svg
-llm = LLM.deepseek(key: ENV["KEY"])
-res = llm.images.edit(prompt: "make the rocket red", image: "rocket.svg")
-IO.copy_stream res.images[0], "rocket-edited.svg"
-```
-
-An interesting property of the DeepSeek implementation is that
-it can maintain a session that can perform multiple image generations
-or edits rather than just one-shot generations.
-
-It's possible because under the hood
-[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html),
-is attached to the
-[`LLM::Response`](https://r.uby.dev/api-docs/llm.rb/LLM/Response.html)
-object that is returned to the caller. So the response includes an
-`agent` method, and it can be carried across multiple generations.
-It is specific to this endpoint though. It works like this:
-
-```ruby
-require "llm"
-
-llm = LLM.deepseek(key: ENV["DEEPSEEK_SECRET"])
-agent = nil
-loop do
-  print "> "
-  prompt = $stdin.gets
-  res = llm.images.create(prompt:, agent:)
-  agent = res.agent
-  IO.copy_stream res.images[0], "image.svg"
-  print "ok: saved image.svg", "\n"
-end
-```
-
-[Back to top](#table-of-contents)
-
-## Audio
-
-The audio interface defined by llm.rb describes three methods,
-although not every provider implements all of them. Generally
-speaking the audio interface is for text-to-speech, and
-speech-to-text models.
-
-The following providers have audio support:
-
-* OpenAI - full support
-* Google - partial support
-* DeepInfra - partial support
-
-#### text-to-speech
-
-The `create_speech` method generates an audio clip based
-on the given input. This method returns a
-[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
-object. OpenAI, and DeepInfra support this method.
-
-```ruby
-require "llm"
-
-llm = LLM.openai(key: ENV["KEY"])
-res = llm.audio.create_speech(input: "Hello world")
-IO.copy_stream res.audio.decoded, "helloworld.mp3"
-```
-
-#### speech-to-text
-
-The `create_transcription` method transcribes a given
-audio clip as text. OpenAI, Google and DeepInfra support
-this method.
+## Fundamentals
 
-```ruby
-require "llm"
+- [Agents](deepdive/fundamentals/agents.md)
+- [Tools](deepdive/fundamentals/tools.md)
+- [Skills](deepdive/fundamentals/skills.md)
+- [Schema](deepdive/fundamentals/schema.md)
+- [Stream](deepdive/fundamentals/stream.md)
+- [Database](deepdive/fundamentals/database.md)
+- [Concurrency](deepdive/fundamentals/concurrency.md)
+- [REPL](deepdive/fundamentals/repl.md)
 
-llm = LLM.google(key: ENV["KEY"])
-res = llm.audio.create_transcription(file: "helloworld.mp3")
-res.text # => "Hello world"
-```
+## Advanced
 
-#### translation
+- [Context](deepdive/advanced/context.md)
+- [Compaction](deepdive/advanced/compaction.md)
+- [Cancellation](deepdive/advanced/cancellation.md)
+- [Transports](deepdive/advanced/transports.md)
+- [Tracer](deepdive/advanced/tracer.md)
 
-The `create_translation` method translates a given audio
-clip, then transcribes it as text. OpenAI, and Google
-support this method.
+## Protocols
 
-```ruby
-require "llm"
+- [MCP](deepdive/protocols/mcp.md)
+- [A2A](deepdive/protocols/a2a.md)
 
-llm = LLM.google(key: ENV["KEY"])
-res = llm.audio.create_translation(file: "bomdia.mp3")
-res.text # => "Good day"
-```
+## Everything else
 
-[Back to top](#table-of-contents)
+- [Images](deepdive/everything_else/images.md)
+- [Audio](deepdive/everything_else/audio.md)
+- [OCR](deepdive/everything_else/ocr.md)