llm.rb 13.1.0 → 15.0.0

data/docs/deepdive/protocols/mcp.md

@@ -0,0 +1,111 @@
+
+## MCP
+
+### Introduction
+
+#### Overview
+
+The Model Context Protocol (MCP) connects agents to external tools
+and data sources through a standardized interface. Instead of
+wiring each service directly into your agent, you run an MCP
+server that exposes its capabilities. The runtime translates the
+server's tool list into
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
+subclasses the model can call.
+
+#### How it works
+
+The stdio transport runs the server as a child process and
+communicates over stdin/stdout. Use
+[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
+to avoid launching the same process multiple times.
+
+```ruby
+require "llm"
+
+llm   = LLM.deepseek(key: ENV["KEY"])
+mcp   = LLM::MCP.stdio(argv: ["npx", "-y", "@forgejo/mcp-server"])
+agent = LLM::Agent.new(llm)
+
+mcp.session do
+  agent.talk "What's happening on forgejo?", tools: mcp.tools
+end
+```
+
+#### Why would I use it?
+
+MCP decouples tool implementation from the agent. Add tools by
+launching a new MCP server. Update them by restarting an existing
+one. Remove them without touching the agent's code.
+
+#### Notes
+
+For stdio, use
+[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
+to avoid launching the same process multiple times. For HTTP,
+[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
+carries little benefit.
+
+### HTTP
+
+#### Overview
+
+The HTTP transport connects to a remote MCP server over HTTP.
+It is the right choice for cloud-hosted servers like GitHub's
+MCP endpoint. Tools exposed by the server become
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html)
+subclasses the model can call. The server does not need to run
+locally or even on the same machine. Configure headers for
+authentication and pick a transport backend for connection
+management.
+
+#### How it works
+
+When you want to connect to a remote MCP server, provide a URL and
+optional headers. The server's tool list is fetched and translated
+into
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) subclasses the model can call.
+
+```ruby
+require "llm"
+
+llm   = LLM.deepseek(key: ENV["KEY"])
+mcp   = LLM::MCP.http(
+  url: "https://api.githubcopilot.com/mcp/",
+  headers: {
+    "Authorization" => "Bearer #{ENV.fetch('GITHUB_PAT')}"
+  },
+  transport: :net_http_persistent
+)
+agent = LLM::Agent.new(llm)
+agent.talk "What's happening on GitHub?", tools: mcp.tools
+```
+
+#### Why would I use it?
+
+The HTTP transport connects to remote MCP servers. This matters
+when the server is not running locally or when tools are maintained
+by a different team and exposed as a service.
+
+#### Notes
+
+For HTTP,
+[`LLM::MCP#session`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html#session)
+carries little benefit.
+
+##### Persistent connections
+
+Set `persistent: true` to reuse HTTP connections across requests
+to the same MCP server. This uses
+[`Net::HTTP::Persistent`](https://github.com/drbrain/net-http-persistent)
+under the hood and avoids the overhead of opening a new TCP
+connection for every request.
+
+```ruby
+mcp = LLM::MCP.http(
+  url: "https://api.githubcopilot.com/mcp/",
+  headers: {"Authorization" => "Bearer #{ENV.fetch('GITHUB_PAT')}"},
+  persistent: true
+)
+```
+

data/docs/deepdive/reference/cost.md

@@ -0,0 +1,109 @@
+
+## LLM::Cost
+
+### Introduction
+
+#### Overview
+
+[`LLM::Cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
+represents the approximate cost of a conversation. It breaks the
+total down by token type, so you can see how much was spent on input,
+output, cached tokens, reasoning, audio, and images. Cost is computed
+from token usage and the pricing data shipped in the
+[model registry](../reference/model_registry.md).
+
+#### How it works
+
+When you want to know what a conversation cost so far, call
+[`LLM::Context#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#cost-instance_method)
+(or
+[`LLM::Agent#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#cost-instance_method))
+and read the breakdown. The REPL shows this live in its status bar
+after every turn:
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+ctx = LLM::Context.new(llm)
+ctx.talk "Hello"
+
+cost = ctx.cost
+cost.input    # => 0.0000042
+cost.output   # => 0.0000084
+cost.total    # => 0.0000126
+cost.to_s     # => "0.00"
+```
+
+#### Why would I use it?
+
+Cost tracking matters in production. Monitoring spend per
+conversation, per agent, or per provider tells you which workflows
+are expensive and when to switch models. Log a structured breakdown
+with
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method),
+or read
+[`LLM::Cost#total`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#total-instance_method)
+for a single number.
+
+#### Notes
+
+Cost is an approximation based on the pricing in the
+[model registry](../reference/model_registry.md).
+[`LLM::Cost.from`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#from-class_method)
+returns an empty cost when the model or registry cannot be found, so
+a missing model never crashes your code.
+
+### Reading the breakdown
+
+#### Overview
+
+[`LLM::Cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
+exposes each cost component as a reader, plus
+[`LLM::Cost#total`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#total-instance_method),
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method),
+and
+[`LLM::Cost#to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_s-instance_method).
+
+#### How it works
+
+Each component is a Float, and defaults to `0` when no tokens of
+that type were used. The
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method)
+method returns a Hash with every component and the total:
+
+```ruby
+cost = ctx.cost
+
+cost.input
+cost.output
+cost.cache_read
+cost.cache_write
+cost.reasoning
+cost.input_audio
+cost.output_audio
+cost.input_image
+
+cost.to_h  # => {input: 4.2e-06, output: 8.4e-06, cache_read: 0.0,
+           #     cache_write: 0.0, input_audio: 0.0, output_audio: 0.0,
+           #     input_image: 0.0, reasoning: 0.0, total: 1.26e-05}
+```
+
+#### Why would I use it?
+
+The per-component breakdown shows where the money goes. High cache
+read costs suggest a conversation benefits from prompt caching.
+High reasoning costs point at a model that thinks a lot. Log
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method)
+at the end of a session to keep a spend trail.
+
+#### Notes
+
+[`LLM::Cost#to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_s-instance_method)
+returns the total in a compact, human-friendly format, rounded to
+two decimals (`"0.01"`). Components that were not used are `0`, so
+you never need to guard against `nil` when aggregating.
+
+The REPL renders context usage as a proportion, not a cost.
+[`LLM::Context#context_usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#context_usage-instance_method)
+returns a `Rational` of the tokens used over the context window
+(for example `Rational(100, 10_000)`), or `nil` when the window is
+unknown or the conversation is too short.

data/docs/deepdive/reference/model_registry.md

@@ -0,0 +1,271 @@
+## Model registry
+
+### Introduction
+
+#### Overview
+
+The model registry is a catalog of every model each provider offers,
+shipped with llm.rb under `data/<provider>.json`. The data is sourced
+from [models.dev](https://models.dev) and powers cost estimation,
+context-window limits, and the `/model` auto-complete in the REPL.
+
+[`LLM::Registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html)
+exposes that catalog, and
+[`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html)
+represents a single model's record. A `Model` is comparable **by price**,
+so you can sort a provider's whole catalog from cheapest to most
+expensive. This document covers reading the registry directly; the
+runtime uses the same data when it estimates the cost or context window
+of a conversation.
+
+#### How it works
+
+Get a registry with
+[`LLM::Registry.for`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#for-class_method)
+and a provider name, or through
+[`LLM::Provider#registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#registry-instance_method).
+[`LLM::Context#registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#registry-instance_method)
+and
+[`LLM::Agent#registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#registry-instance_method)
+delegate to their provider's registry, so
+[`LLM::Registry#keys`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#keys-instance_method)
+returns the model names, and
+[`LLM::Registry#models`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#models-instance_method)
+returns the full
+[`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html)
+objects:
+
+```ruby
+require "llm"
+
+registry = LLM::Registry.for(:openai)
+registry.keys             # => ["gpt-image-2", "gpt-5.2-pro", ...]
+registry.models           # => [LLM::Registry::Model, ...]
+registry.models.map(&:id) # => same names as #keys
+```
+
+Use *`keys`* when you only need model names, and *`models`* when you want
+to inspect pricing, limits, or capabilities.
+
+#### Why would I use it?
+
+The registry lets you discover what a provider offers without
+hardcoding model names. Build a model picker, validate a model, check a
+context window, or price a request. Because a
+[`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html)
+is comparable by price, you can also sort a provider's catalog and offer
+only the cheapest models that still meet a requirement.
+
+#### Notes
+
+Each provider ships a `data/<provider>.json` registry file. A missing
+model raises `LLM::NoSuchModelError`, and a missing registry raises
+`LLM::NoSuchRegistryError`. The runtime rescues both to degrade
+gracefully, so an unknown model's context window reads as `nil` rather
+than crashing your code.
+
+### Reading models
+
+#### Overview
+
+[`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html)
+wraps one model record and exposes its identity, pricing, limits,
+capabilities, and modalities.
+
+#### How it works
+
+Pick a model from
+[`LLM::Registry#models`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#models-instance_method)
+and call its readers. Note that the raw `cost`, `limit`, and `modalities`
+readers return
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+metadata objects, while the convenience accessors return plain values:
+
+```ruby
+require "llm"
+
+registry = LLM::Registry.for(:openai)
+model    = registry.models.find { _1.id == "gpt-4.1" }
+model.id # => "gpt-4.1"
+model.name # => "GPT-4.1"
+model.context_window # => 1_047_576
+model.input_cost # => 2.0   (per million input tokens)
+model.output_cost # => 8.0   (per million output tokens)
+model.cost # => #<LLM::Object ...>  raw pricing
+model.limit # => #<LLM::Object ...>  raw limits
+model.modalities # => #<LLM::Object ...>  raw input/output lists
+```
+
+#### Why would I use it?
+
+Pricing and limits are what most applications need from a catalog.
+`input_cost` and `output_cost` let you compare models by spend without
+unwrapping the raw cost object, and `context_window` tells you how large
+a conversation a model can hold. That is exactly the data the runtime
+uses to estimate cost and track context usage.
+
+#### Notes
+
+`name` falls back to `id` when a model has no human-readable name.
+`input_cost` and `output_cost` return `nil` when a model is unpriced.
+`context_window` returns `nil` when the limit is unknown, so it is safe
+to compare against numbers.
+
+### Capabilities
+
+#### Overview
+
+[`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html)
+exposes a model's capabilities through predicate methods, so you can
+filter a provider's catalog by what you need before you even make a
+call.
+
+#### How it works
+
+Capability predicates return booleans for whether a model supports tool
+calls, structured output, reasoning, or open weights:
+
+```ruby
+require "llm"
+
+model = LLM::Registry.for(:openai).models.find { _1.id == "gpt-4.1" }
+model.tool_call?         # => true  can be called as a tool
+model.structured_output? # => true  supports typed output
+model.reasoning?         # => false not a reasoning model
+model.open_weights?      # => false weights are not public
+```
+
+#### Why would I use it?
+
+Not every model supports every feature. If you are building a model
+picker, filter the registry to models that support tool calls and
+structured output before offering them to a user, and only include
+reasoning models where you want the extra capability (and cost).
+
+#### Notes
+
+Capabilities vary per model and per provider, and are absent from some
+records. The predicates coerce a missing flag to `false`, so you never
+need a `nil` check.
+
+### Modalities
+
+#### Overview
+
+Modalities describe what a model can read as input and produce as
+output: text, image, audio, video, and PDF. A model is multimodal when
+it handles more than text.
+
+#### How it works
+
+A model offers direction-aware tests and a set of convenience
+predicates. `input?("pdf")` asks whether the model *reads* PDFs,
+`output?("image")` whether it *produces* images, and `image?`/`audio?`/
+`pdf?`/`video?` report support on either side:
+
+```ruby
+require "llm"
+
+model = LLM::Registry.for(:openai).models.find { _1.id == "gpt-4.1" }
+model.text?            # => true  reads and writes text
+model.image?           # => true  reads or produces images
+model.pdf?             # => true  reads or produces PDFs
+model.audio?           # => false
+model.input?("pdf")    # => true  reads PDFs as input
+model.output?("image") # => false does not produce images
+```
+
+#### Why would I use it?
+
+`text?` is a quick way to identify a general-purpose chat LLM as opposed
+to a single-purpose generation model (such as a speech or image model),
+and the per-modality tests let you find a model that can accept a PDF or
+produce audio.
+
+#### Notes
+
+The `supports?`-style convenience predicates (`image?`, `audio?`, `pdf?`,
+`video?`) return `true` when the model handles a modality in *either*
+direction, so a model that only reads PDFs and writes text still reports
+`pdf?` as `true`.
+
+### Sorting by price
+
+#### Overview
+
+[`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html)
+is `Comparable` by price, so sorting a provider's models orders them
+from cheapest to most expensive.
+
+#### How it works
+
+Call `sort` on
+[`LLM::Registry#models`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#models-instance_method).
+Ordering compares input cost first, then output cost. Unpriced models
+sort to the end:
+
+```ruby
+require "llm"
+
+reg = LLM::Registry.for(:openai)
+list = reg.models.sort
+list.first.id # => "text-embedding-3-small"
+list.last.id  # => "chatgpt-image-latest"
+```
+
+#### Why would I use it?
+
+Offering the cheapest capable model is a common product decision. Sort
+by price, then filter to models that satisfy your capability and
+modality requirements, and take the first.
+
+#### Notes
+
+Models without a price are treated as the most expensive so they sort
+last rather than first. If you only compare a handful of models, the
+`<` comparison works directly (`cheap < expensive` returns `true` when
+`cheap` is cheaper).
+
+### Raw lookups
+
+#### Overview
+
+[`LLM::Registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html)
+also offers direct `model:` keyed lookups that resolve a single model and
+return its raw metadata object.
+
+#### How it works
+
+[`LLM::Registry#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#cost-instance_method),
+[`LLM::Registry#limit`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#limit-instance_method),
+and
+[`LLM::Registry#modalities`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#modalities-instance_method)
+take a `model:` keyword and return the corresponding raw
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html).
+These lookups also apply fallback resolution, so a dated or numbered
+model id matches its base model:
+
+```ruby
+require "llm"
+
+registry = LLM::Registry.for(:openai)
+registry.limit(model: "gpt-4.1").context # => 1_047_576
+registry.cost(model: "gpt-4.1-2025-01-01") ==
+  registry.cost(model: "gpt-4.1") # => true, dated id falls back
+```
+
+#### Why would I use it?
+
+These methods are convenient when you already know a model's id and only
+want one piece of raw metadata, and they are what the runtime itself
+uses internally. For richer access, prefer the
+[`LLM::Registry::Model`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry/Model.html)
+objects from
+[`LLM::Registry#models`](https://r.uby.dev/api-docs/llm.rb/LLM/Registry.html#models-instance_method).
+
+#### Notes
+
+A missing model raises `LLM::NoSuchModelError`. The fallback matching
+strips date suffixes (`-YYYY-MM-DD`) and collapses numbered OpenAI ids
+(`gpt-N-XXXX` → `gpt-N`) so a slightly out-of-date model id still
+resolves.

data/docs/deepdive/reference/object.md

@@ -0,0 +1,108 @@
+
+## LLM::Object
+
+### Introduction
+
+#### Overview
+
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+is the hash-like object that llm.rb uses everywhere structured data
+flows through the runtime. Response bodies, tool arguments, schema
+results, usage and cost data, and function parameters all come back
+as [`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+instances. It is similar in spirit to OpenStruct,
+and it was introduced after OpenStruct became a bundled gem rather
+than a default gem in Ruby 3.5.
+
+#### How it works
+
+When you want to read a value from an
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html),
+use either method-style or bracket access. Keys are indifferent, so
+strings and symbols work interchangeably:
+
+```ruby
+obj = LLM::Object.from(city: "Paris", temperature: 15.0)
+
+obj.city          # => "Paris"
+obj["city"]       # => "Paris"
+obj[:city]        # => "Paris"
+obj[:temperature] # => 15.0
+```
+
+Nested hashes and arrays are converted recursively, so deep chains
+read naturally:
+
+```ruby
+obj = LLM::Object.from(person: {name: "John"})
+obj.person.name   # => "John"
+obj.person.class  # => LLM::Object
+```
+
+#### Why would I use it?
+
+Most of the time you do not construct
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+instances yourself. They come back from `talk`, `ask`, `embed`, and
+every other call that returns structured data. Knowing how they
+behave lets you read response fields, pass tool arguments, and
+inspect usage without reaching for `to_h` on every line.
+
+#### Notes
+
+An
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+is enumerable and supports the usual Hash operations: `keys`,
+`values`, `key?`, `fetch`, `dig`, `slice`, `merge`, `merge!`,
+`delete`, `size`, and `empty?`. Use `to_h` for a plain Hash and
+`to_hash` for one with symbol keys. A missing key returns `nil`
+rather than raising. Because it subclasses `BasicObject`, `to_json`
+is defined explicitly and serializes through the configured JSON
+adapter via
+[`LLM.json.dump`](https://r.uby.dev/api-docs/llm.rb/LLM.html#json-class_method).
+
+### Reading and writing
+
+#### Overview
+
+Beyond simple reads,
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+supports assignment, mutation, and iteration, so you can treat it
+like a Hash in place.
+
+#### How it works
+
+Assign values with method or bracket syntax, mutate in place, and
+iterate like a Hash:
+
+```ruby
+obj = LLM::Object.from({})
+
+obj.city = "Paris"       # method-style write
+obj["country"] = "France"
+
+obj.key?(:city)          # => true
+obj.keys                 # => ["city", "country"]
+obj.merge!(population: 2_100_000)
+
+obj.each { |key, value| puts "#{key}: #{value}" }
+obj.transform_values!(&:upcase) if obj.any?
+```
+
+#### Why would I use it?
+
+Tool implementations receive their arguments as an
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html)
+and often build a result Hash from them. Merging defaults, deleting
+optional keys, and transforming values in place keeps that code
+concise without converting back and forth between Hash and
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html).
+
+#### Notes
+
+`merge` returns a new
+[`LLM::Object`](https://r.uby.dev/api-docs/llm.rb/LLM/Object.html);
+`merge!` mutates in place.
+Assignment always stores the key as a string internally, which is
+why string and symbol lookups both work. Equality compares against
+anything that responds to `to_h`.