llm.rb 13.1.0 → 14.0.0

data/resources/deepdive/everything_else/audio.md

@@ -0,0 +1,122 @@
+
+## Audio
+
+### Introduction
+
+#### Overview
+
+The audio interface covers three things: turning text into speech,
+transcribing audio into text, and translating spoken language.
+OpenAI supports all three. Google and DeepInfra support subsets.
+Each method follows the same pattern: pass input, get output,
+copy the result somewhere useful.
+
+#### How it works
+
+When you want to convert text to speech, call
+[`LLM::Audio#create_speech`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_speech).
+The provider returns an audio clip as a
+[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
+object. The generated audio can be copied
+to a file or streamed directly. Each provider supports different
+output formats and voice options:
+
+```ruby
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.audio.create_speech(input: "Hello world")
+IO.copy_stream res.audio.decoded, "helloworld.mp3"
+```
+
+#### Why would I use it?
+
+Audio support lets you build voice interfaces, add accessibility
+features, and work across languages without wiring up a separate
+speech service. Generate audio for notifications, narrate written
+content, or add speech output to an existing application with a
+single method call.
+
+#### Notes
+
+OpenAI has full audio support. Google and DeepInfra have partial
+support. The
+[`LLM::Audio#create_speech`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_speech)
+method returns a
+[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
+object.
+
+### Transcription
+
+#### Overview
+
+Transcription turns an audio file into text. Pass a file path
+to
+[`LLM::Audio#create_transcription`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_transcription)
+and get back the spoken content as
+a string. OpenAI, Google, and DeepInfra support it. Transcribe
+meeting notes, voice memos, or podcast episodes for search and
+processing. The response is plain text you can feed into any
+downstream pipeline.
+
+#### How it works
+
+When you want to transcribe audio into text, call
+[`LLM::Audio#create_transcription`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_transcription).
+The provider processes the audio and returns the transcribed text.
+The file can be a local path or a URL depending on provider support:
+
+```ruby
+llm = LLM.google(key: ENV["KEY"])
+res = llm.audio.create_transcription(file: "helloworld.mp3")
+res.text  # => "Hello world"
+```
+
+#### Why would I use it?
+
+Transcribe recorded meetings, voice memos, or podcast episodes for
+search and processing. The returned text feeds directly into search
+indexes, summarization pipelines, or downstream extraction. Spoken
+content becomes as queryable as written text.
+
+#### Notes
+
+OpenAI has full audio support. Google and DeepInfra have partial
+support.
+
+### Translation
+
+#### Overview
+
+Translation transcribes audio and translates it into English in
+one step. Pass a file to
+[`LLM::Audio#create_translation`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_translation)
+and get back the
+translated text. OpenAI and Google support it. Translate
+multilingual podcasts, interviews, or any audio where you need
+the content in English without running a separate translation
+pipeline.
+
+#### How it works
+
+When you want to translate spoken audio into English, call
+[`LLM::Audio#create_translation`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_translation).
+The provider transcribes the spoken language and translates the
+result into English in a single operation. The returned text is the
+English translation:
+
+```ruby
+llm = LLM.google(key: ENV["KEY"])
+res = llm.audio.create_translation(file: "bomdia.mp3")
+res.text  # => "Good day"
+```
+
+#### Why would I use it?
+
+Translate a podcast or interview into another language without a
+separate speech service. The provider handles both transcription and
+translation in a single call, so you get English text from any
+supported source language without chaining two operations together.
+
+#### Notes
+
+Each method works independently and each provider supports a
+different subset.

data/resources/deepdive/everything_else/cost.md

@@ -0,0 +1,99 @@
+
+## LLM::Cost
+
+### Introduction
+
+#### Overview
+
+[`LLM::Cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
+represents the approximate cost of a conversation. It breaks the
+total down by token type, so you can see how much was spent on input,
+output, cached tokens, reasoning, audio, and images. Cost is computed
+from token usage and the pricing data shipped in the model registry.
+
+#### How it works
+
+When you want to know what a conversation cost so far, call
+[`LLM::Context#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#cost-instance_method)
+(or
+[`LLM::Agent#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#cost-instance_method))
+and read the breakdown. The REPL shows this live in its status bar
+after every turn:
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+ctx = LLM::Context.new(llm)
+ctx.talk "Hello"
+
+cost = ctx.cost
+cost.input    # => 0.0000042
+cost.output   # => 0.0000084
+cost.total    # => 0.0000126
+cost.to_s     # => "0.0000126"
+```
+
+#### Why would I use it?
+
+Cost tracking matters in production. Monitoring spend per
+conversation, per agent, or per provider tells you which workflows
+are expensive and when to switch models. Log a structured breakdown
+with
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method),
+or read
+[`LLM::Cost#total`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#total-instance_method)
+for a single number.
+
+#### Notes
+
+Cost is an approximation based on the pricing in the model registry.
+[`LLM::Cost.from`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#from-class_method)
+returns an empty cost when the model or registry cannot be found, so
+a missing model never crashes your code.
+
+### Reading the breakdown
+
+#### Overview
+
+[`LLM::Cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
+exposes each cost component as a reader, plus
+[`LLM::Cost#total`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#total-instance_method),
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method),
+and
+[`LLM::Cost#to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_s-instance_method).
+
+#### How it works
+
+Each component is a Float, or `nil` when no tokens of that type were
+used. The
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method)
+method returns a Hash with only the non-nil components and the total:
+
+```ruby
+cost = ctx.cost
+
+cost.input
+cost.output
+cost.cache_read
+cost.cache_write
+cost.reasoning
+cost.input_audio
+cost.output_audio
+cost.input_image
+
+cost.to_h  # => {input: 4.2e-06, output: 8.4e-06, total: 1.26e-05}
+```
+
+#### Why would I use it?
+
+The per-component breakdown shows where the money goes. High cache
+read costs suggest a conversation benefits from prompt caching.
+High reasoning costs point at a model that thinks a lot. Log
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method)
+at the end of a session to keep a spend trail.
+
+#### Notes
+
+[`LLM::Cost#to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_s-instance_method)
+returns the total in a compact, human-friendly format
+(`"0.0000126"`). Components that were not used are `nil`, so sum
+them with `compact` if you aggregate across conversations.

data/resources/deepdive/everything_else/images.md

@@ -0,0 +1,89 @@
+
+## Images
+
+### Introduction
+
+#### Overview
+
+A handful of providers can generate images from a text prompt.
+OpenAI, Google, xAI, DeepInfra, and DeepSeek all support it.
+OpenAI, xAI, and DeepInfra also let you edit existing images.
+The API is the same across providers, so switching between them
+requires no code changes.
+
+#### How it works
+
+The
+[`LLM::Images#create`](https://r.uby.dev/api-docs/llm.rb/LLM/Images.html#create)
+method sends a prompt to the provider and
+returns the result as a
+[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
+object. The same API works
+across providers: swap
+[`LLM.openai`](https://r.uby.dev/api-docs/llm.rb/LLM.html#openai-class_method) for
+[`LLM.xai`](https://r.uby.dev/api-docs/llm.rb/LLM.html#xai-class_method) and the rest
+of the code is identical.
+
+```ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.images.create(prompt: "a dog on a rocket to the moon")
+IO.copy_stream res.images[0], "dogrocket.png"
+```
+
+#### Why would I use it?
+
+Image generation and editing let the model produce visual output
+directly from your prompts. The API is the same across providers,
+so switching between OpenAI and xAI requires changing one line.
+Prototype visual concepts, generate assets, or augment datasets
+without wiring up a separate image API.
+
+#### Notes
+
+Google only supports image generation, not edits. DeepSeek
+generates SVGs rather than raster images. DeepSeek can also
+maintain a session across multiple generations through the
+`agent` parameter on the response object.
+
+### Editing
+
+#### Overview
+
+Image editing takes an existing image and a text prompt, then
+produces a modified version. You can add objects, change colours,
+or alter the scene while keeping the original composition.
+OpenAI, xAI, and DeepInfra support raster image edits. DeepSeek
+generates SVG documents which can be refined with follow-up
+text-to-image prompts, giving you iterative vector editing.
+
+#### How it works
+
+The
+[`LLM::Images#edit`](https://r.uby.dev/api-docs/llm.rb/LLM/Images.html#edit)
+method takes a prompt and an image path. It
+returns a modified image as a
+[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
+object. Copy the result
+to a file the same way you would with generated images.
+
+```ruby
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.images.edit(prompt: "add a mustache", image: "self.jpg")
+IO.copy_stream res.images[0], "mustache.png"
+```
+
+#### Why would I use it?
+
+Editing lets the model modify existing images rather than starting
+from scratch. DeepSeek's SVG output is particularly useful here
+because vector graphics can be refined iteratively. Make targeted
+adjustments: add objects, change colours, or alter the scene,
+all while keeping the original composition intact.
+
+#### Notes
+
+Google does not support image edits. DeepSeek can maintain a
+session across multiple generations through the `agent` parameter
+on the response object.

data/resources/deepdive/everything_else/object.md

@@ -0,0 +1,108 @@
+
+## LLM::Object
+
+### Introduction
+
+#### Overview
+
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+is the hash-like object that llm.rb uses everywhere structured data
+flows through the runtime. Response bodies, tool arguments, schema
+results, usage and cost data, and function parameters all come back
+as [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+instances. It is similar in spirit to OpenStruct,
+and it was introduced after OpenStruct became a bundled gem rather
+than a default gem in Ruby 3.5.
+
+#### How it works
+
+When you want to read a value from an
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html),
+use either method-style or bracket access. Keys are indifferent, so
+strings and symbols work interchangeably:
+
+```ruby
+obj = LLM::Object.from(city: "Paris", temperature: 15.0)
+
+obj.city          # => "Paris"
+obj["city"]       # => "Paris"
+obj[:city]        # => "Paris"
+obj[:temperature] # => 15.0
+```
+
+Nested hashes and arrays are converted recursively, so deep chains
+read naturally:
+
+```ruby
+obj = LLM::Object.from(person: {name: "John"})
+obj.person.name   # => "John"
+obj.person.class  # => LLM::Object
+```
+
+#### Why would I use it?
+
+Most of the time you do not construct
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+instances yourself. They come back from `talk`, `ask`, `embed`, and
+every other call that returns structured data. Knowing how they
+behave lets you read response fields, pass tool arguments, and
+inspect usage without reaching for `to_h` on every line.
+
+#### Notes
+
+An
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+is enumerable and supports the usual Hash operations: `keys`,
+`values`, `key?`, `fetch`, `dig`, `slice`, `merge`, `merge!`,
+`delete`, `size`, and `empty?`. Use `to_h` for a plain Hash and
+`to_hash` for one with symbol keys. A missing key returns `nil`
+rather than raising. Because it subclasses `BasicObject`, `to_json`
+is defined explicitly and serializes through the configured JSON
+adapter via
+[`LLM.json.dump`](https://r.uby.dev/api-docs/llm.rb/LLM.html#json-class_method).
+
+### Reading and writing
+
+#### Overview
+
+Beyond simple reads,
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+supports assignment, mutation, and iteration, so you can treat it
+like a Hash in place.
+
+#### How it works
+
+Assign values with method or bracket syntax, mutate in place, and
+iterate like a Hash:
+
+```ruby
+obj = LLM::Object.from({})
+
+obj.city = "Paris"       # method-style write
+obj["country"] = "France"
+
+obj.key?(:city)          # => true
+obj.keys                 # => ["city", "country"]
+obj.merge!(population: 2_100_000)
+
+obj.each { |key, value| puts "#{key}: #{value}" }
+obj.transform_values!(&:upcase) if obj.any?
+```
+
+#### Why would I use it?
+
+Tool implementations receive their arguments as an
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+and often build a result Hash from them. Merging defaults, deleting
+optional keys, and transforming values in place keeps that code
+concise without converting back and forth between Hash and
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html).
+
+#### Notes
+
+`merge` returns a new
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html);
+`merge!` mutates in place.
+Assignment always stores the key as a string internally, which is
+why string and symbol lookups both work. Equality compares against
+anything that responds to `to_h`.

data/resources/deepdive/everything_else/ocr.md

@@ -0,0 +1,48 @@
+
+## OCR
+
+### Introduction
+
+#### Overview
+
+OCR pulls text out of images and PDFs so you can search, index,
+or process scanned content. Mistral is the only provider with a
+dedicated OCR endpoint, and it accepts both image URLs and
+document URLs.
+
+#### How it works
+
+Mistral is the only provider with a dedicated OCR endpoint. The
+[`LLM::Provider#ocr`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#ocr)
+method accepts an `image_url:` or `document_url:` parameter.
+Document URLs can point to PDFs.
+
+```ruby
+require "llm"
+
+llm = LLM.mistral(key: ENV["KEY"])
+
+# Extract text from an image
+res = llm.ocr(image_url: "https://example.com/photo.png")
+res.pages.each { |page| puts page.markdown }
+
+# Extract text from a PDF
+res = llm.ocr(document_url: "https://example.com/report.pdf")
+res.pages.each { |page| puts page.markdown }
+```
+
+#### Why would I use it?
+
+OCR extracts text from scanned documents and images. The result
+feeds into search indexes, extraction pipelines that pull out dates
+and amounts, or archives so scanned contracts become searchable
+records. PDFs and images both work, and the response is structured
+per page with markdown.
+
+#### Notes
+
+Only Mistral currently supports OCR through the llm.rb runtime.
+The response exposes pages through
+[`LLM::OCR::Response#pages`](https://r.uby.dev/api-docs/llm.rb/LLM/OCR/Response.html#pages),
+where each page
+has a `markdown` field containing the extracted text.

data/resources/deepdive/fundamentals/agents.md

@@ -0,0 +1,202 @@
+
+## Agents
+### Introduction
+
+#### Overview
+
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+is the recommended entry point for most use-cases. It provides a
+class-level DSL for defining reusable, preconfigured assistants
+with defaults for model, tools, schema, and instructions. Under
+the hood it delegates to
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html),
+so it has the same runtime surface: message history, streaming,
+serialization, compaction, and concurrency.
+
+#### How it works
+
+An agent holds a conversation with a model. You send input with
+[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk),
+the model responds, and if it requests tools the agent
+executes them automatically and feeds the results back. It enables
+[a loop guard by default](https://r.uby.dev/llm/deepdive/advanced/guard)
+that detects repeated tool-call patterns
+and blocks stuck execution. The tool loop can also be bounded with
+[`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
+(see the Tool budget section). Instructions are injected once
+unless a system message is already present.
+
+#### Why would I use it?
+
+Agents manage the tool loop for you. They guard against infinite
+loops, keep conversation state across turns, and let you define
+reusable configurations at the class level. If you need manual
+control over the tool loop, use
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
+directly instead.
+
+#### Notes
+
+Agents support the same concurrency strategies, compaction,
+cancellation, and serialization as contexts. The trade-off between
+a subclass and a direct instance is only in how the agent is
+organized, not in what it can do. Tool loop execution can be
+configured with `concurrency: :sequential`, `:thread`, `:async`,
+`:fiber`, `:fork`, or `:ractor`.
+
+### Class-based
+
+#### Overview
+
+A subclass of
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+gives you a reusable agent with its own behavior. You define the model, tools, and other
+attributes at the class level, and each instance picks them up
+as defaults. Attributes can be overridden per-instance, and they
+can be plain values, blocks, or Symbols that resolve to methods.
+The class becomes a self-contained worker that you can instantiate
+and talk to from anywhere.
+
+#### How it works
+
+A subclass declares its defaults with
+[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method).
+Each key is a
+class-level accessor: `name`, `description`, `model`, `tools`,
+`skills`, `instructions`, `stream`, `tracer`, `concurrency`,
+`schema`, `confirm`, `path`, `tool_budget`.
+Keyword arguments in the constructor override these defaults.
+
+```ruby
+class Agent < LLM::Agent
+  set model: "deepseek-v4-pro",
+      description: "system administration agent",
+      tools: [Shell]
+end
+
+llm = LLM.openai(key: ENV["KEY"])
+agent = Agent.new(llm)
+agent.talk "Run 'date'"
+```
+
+#### Why would I use it?
+
+A subclass is useful when multiple parts of an application need to
+call the same agent. The configuration and any helper methods live in one place. Define a
+`research!` method that kicks off the agent's work. The subclass
+becomes a self-contained worker.
+
+#### Notes
+
+Attributes passed to
+[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
+can be plain values, blocks, or
+Symbols. A Symbol is evaluated as an instance method on the
+subclass, so `tracer: :set_tracer` calls `set_tracer` on the
+instance. A block like `stream: -> { $stdout }` is evaluated
+when the attribute is first accessed.
+
+Set `path:` on a subclass or instance for automatic filesystem
+persistence; the agent restores conversation history from the
+file on startup and saves it back after every turn with no
+manual
+[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)/
+[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
+calls. See the
+[database deepdive](../fundamentals/database.md) for details.
+
+### Object-based
+
+#### Overview
+
+An
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+instance is the simplest way to get started. You pass a provider and any configuration as keyword
+arguments, and the agent runs the tool loop and manages state
+just like a subclass would. This is the right choice when you
+are prototyping, running a one-off task, or when the agent's
+configuration is determined at runtime.
+
+#### How it works
+
+A direct instance takes the same attributes as keyword arguments
+to
+[`LLM::Agent.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#initialize-instance_method).
+The first argument is always the provider.
+Everything else is optional. The agent runs the tool loop
+and manages state under the hood through a
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html).
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, stream: $stdout)
+agent.talk "Hello world"
+```
+
+#### Why would I use it?
+
+A direct instance is the right choice for quick experiments, one-shot
+tasks, or when defining a class would be overkill. It is also
+the right choice when the agent's configuration is determined at
+runtime and a class hierarchy adds unnecessary complexity.
+
+#### Notes
+
+Direct instances accept all the same options as subclasses. The
+difference is only in how the agent is organized, not in what it
+can do. Under the hood,
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+creates a
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
+that manages the message history.
+
+### Tool budget
+
+#### Overview
+
+[`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
+caps the number of tool calls allowed in a single turn. Once the
+budget is spent, the agent sends an in-band advisory message back
+through the model instead of running more tools. By default no
+budget is set, so the feature is disabled.
+
+#### How it works
+
+When you want to bound how many tools an agent can call in one
+turn, set the budget with
+[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method).
+The budget can be a plain number or a block evaluated against the
+agent instance. Once the agent has made the budgeted number of tool
+calls, it stops and returns an in-band advisory message describing
+the spent budget. A model will usually change course afterwards:
+
+```ruby
+class Researcher < LLM::Agent
+  set model: "deepseek-v4-pro",
+      tools: [FetchNews, FetchStocks],
+      tool_budget: 5
+end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = Researcher.new(llm)
+agent.talk "Research the market"
+```
+
+#### Why would I use it?
+
+A tool budget prevents runaway tool loops. A misbehaving model can
+otherwise keep calling tools, spending tokens on every round trip.
+Capping the budget turns that into a bounded conversation: after
+the cap, the model is told it has run out of tool calls and must
+respond from what it has.
+
+#### Notes
+
+The budget is disabled by default (`nil`). Set it on a subclass
+with the `tool_budget` DSL, through
+[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
+with `tool_budget:`, or per-instance with the `tool_budget:` keyword
+argument to
+[`LLM::Agent.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#initialize-instance_method).
+This replaces the previous `tool_attempts` parameter, which is no
+longer used.