llm.rb 13.0.0 → 14.0.0

data/resources/deepdive/advanced/tracer.md

@@ -0,0 +1,180 @@
+
+## Tracer
+
+### Introduction
+
+#### Overview
+
+Tracers let you observe what the runtime is doing. They hook into
+requests, tool calls, compactions, and other events. Debug a
+misbehaving agent, monitor request latency, or export spans to
+an observability backend. A provider-wide tracer intercepts every
+request through that provider. An agent-local tracer only covers
+requests made by that agent.
+
+#### How it works
+
+When you want to observe runtime events, subclass
+[`LLM::Tracer`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer.html)
+and implement the hooks you need, or use one of the built-in
+tracers. The built-in tracers include
+[`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html)
+(writes structured JSON to stdout or a file),
+[`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html)
+(writes human-readable single-line logs to stderr), and
+[`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html)
+(exports spans via OTLP for OpenTelemetry). Attach a tracer to a
+provider or an agent:
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+llm.tracer = LLM::Tracer::PrettyLogger.new(llm)
+agent = LLM::Agent.new(llm)
+agent.talk "Hello"
+```
+
+#### Why would I use it?
+
+Tracers give you visibility into what the runtime is doing. Debug
+a misbehaving agent by tracing every request it makes. Monitor
+request latency and token usage across providers. Export spans to
+OpenTelemetry for integration with existing observability pipelines.
+[`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html)
+is the best choice during development for compact, human-readable
+output.
+[`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html)
+provides structured JSON, and
+[`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html)
+exports spans to OpenTelemetry for production observability.
+
+#### Notes
+
+The tracer is extensible. You can implement custom hooks for any
+runtime event. The scope can be an individual agent or every
+request a provider makes. Three built-in tracers are available:
+[`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html)
+(human-readable),
+[`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html)
+(structured JSON), and
+[`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html)
+(OpenTelemetry).
+
+### Provider
+
+#### Overview
+
+A provider-wide tracer intercepts every request made through that
+provider. All agents sharing the same provider share the same
+tracer. Use this to trace at the infrastructure level without
+configuring each agent individually.
+
+#### How it works
+
+When you want every request through a provider to be traced, set
+the tracer on the provider directly. Every request made through
+that provider, regardless of which agent initiates it, flows
+through the same tracer hooks. The provider holds a reference to the
+tracer and passes it to every new context it creates. This ensures
+consistent observability without configuring each agent individually.
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+llm.tracer = LLM::Tracer::Logger.new(llm, io: $stdout)
+```
+
+#### Why would I use it?
+
+A provider-wide tracer captures every request at the infrastructure level.
+All agents sharing the same provider share the same tracer.
+
+#### Notes
+
+The tracer can also write to a file with the `path:` option to
+[`LLM::Tracer::Logger.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html#initialize-instance_method).
+
+### Agent
+
+#### Overview
+
+An agent-local tracer only covers requests made by that agent.
+Attach it via the `tracer:` keyword argument to
+[`LLM::Agent.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#initialize-instance_method)
+and it follows that agent wherever it goes. Different agents can
+have different tracers.
+
+#### How it works
+
+When you want a tracer for a specific agent, pass it to the agent
+on creation. Only requests made by
+that agent flow through the tracer, leaving other agents on the
+same provider unaffected.
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, tracer: LLM::Tracer::Logger.new(llm, io: $stdout))
+```
+
+#### Why would I use it?
+
+Agent-local tracers let each agent log differently.
+One agent might log to stdout, another to a file, a third to
+OpenTelemetry.
+
+#### Notes
+
+The tracer can also write to a file with the `path:` option to
+[`LLM::Tracer::Logger.new`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html#initialize-instance_method).
+
+### PrettyLogger
+
+#### Overview
+
+[`LLM::Tracer::PrettyLogger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/PrettyLogger.html)
+writes human-readable single-line logs to stderr. Unlike
+[`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html)
+(which emits structured JSON), the pretty logger is designed for
+interactive development sessions where you want to see request and
+tool-call activity at a glance.
+
+#### How it works
+
+Each request and tool call produces a single line on stderr with
+the model, duration, and a summary of the activity. The logger
+accepts an `io:` option to redirect output.
+
+##### Provider-wide
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+llm.tracer = LLM::Tracer::PrettyLogger.new(llm)
+```
+
+##### Agent-local
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, tracer: LLM::Tracer::PrettyLogger.new(llm))
+```
+
+##### Custom output
+
+```ruby
+tracer = LLM::Tracer::PrettyLogger.new(llm, io: $stdout)
+tracer = LLM::Tracer::PrettyLogger.new(llm, io: File.open("trace.log", "a"))
+```
+
+#### Why would I use it?
+
+The pretty logger is the best choice for development. The output is
+compact enough to follow in real time while still showing the model
+name, duration, and tool calls. Switch to
+[`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html)
+when you need structured JSON for programmatic analysis, or to
+[`LLM::Tracer::Telemetry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Telemetry.html)
+when you need OpenTelemetry exports.
+
+#### Notes
+
+The pretty logger writes to `$stderr` by default. Set `io:` to
+redirect output. All three built-in tracers share the same interface,
+so switching between them requires changing only the class name.

data/resources/deepdive/advanced/transformer.md

@@ -0,0 +1,67 @@
+
+## Transformer
+
+### Introduction
+
+#### Overview
+
+[`LLM::Transformer`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer.html)
+is the superclass for message transformers. A transformer is bound
+to a context and rewrites a single message before it is sent to the
+provider. This lets you scrub sensitive data, inject context, or
+otherwise modify outgoing messages without changing your prompt
+code.
+
+#### How it works
+
+A transformer is a subclass of
+[`LLM::Transformer`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer.html)
+that implements
+[`LLM::Transformer#call`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer.html#call-instance_method).
+The method receives the message to transform and returns a
+message.
+You can mutate the message in place or return a new one; either way,
+the returned message is what gets sent.
+
+Configure the transformer on a context with the `transformer:` option,
+passing a class rather than an instance. The runtime instantiates it
+once per turn. Options passed through `transformer_options:` are
+forwarded to `call` as keyword arguments. The transformer runs on
+the most recent message in both chat completions and Responses API
+turns, before the request reaches the provider:
+
+```ruby
+class RedactEmails < LLM::Transformer
+  def call(message:)
+    content = message.content.to_s.gsub(/[\w.+-]+@[\w-]+\.[\w.]+/, "[EMAIL]")
+    LLM::Message.new(message.role, content, message.extra)
+  end
+end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+ctx = LLM::Context.new(
+  llm,
+  transformer: RedactEmails
+)
+ctx.talk "Contact support@example.com for help"
+```
+
+#### Why would I use it?
+
+Transformers give you a single hook point for all outgoing messages.
+Common uses include redacting PII before it leaves your process,
+injecting a timestamp or request ID, or normalizing content for a
+particular provider. Because the transformer runs automatically on
+every turn, you never need to remember to apply the transform in
+your prompt code.
+
+#### Notes
+
+[`LLM::Transformer::Null`](https://r.uby.dev/api-docs/llm.rb/LLM/Transformer/Null.html)
+is the default transformer; it returns the message unchanged. The
+`transformer_options:` hash is passed to `call` on every turn.
+Streams can observe transformation through the
+[`LLM::Stream#on_transform`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_transform-instance_method)
+and
+[`LLM::Stream#on_transform_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_transform_finish-instance_method)
+callbacks, which receive the transformer instance.

data/resources/deepdive/advanced/transports.md

@@ -0,0 +1,45 @@
+
+## Transports
+
+### Introduction
+
+#### Overview
+
+The transport option selects which HTTP library is used for network
+communication.
+[`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html),
+[`LLM::MCP`](https://r.uby.dev/api-docs/llm.rb/LLM/MCP.html), and
+[`LLM::A2A`](https://r.uby.dev/api-docs/llm.rb/LLM/A2A.html)
+all accept this option. Three backends are available out of the box:
+`net/http` is always there, `net/http/persistent` pools connections,
+and `curb` wraps libcurl. Each implements the same internal interface
+so switching between them is a one-word change.
+
+#### How it works
+
+**Net/HTTP** (`:net_http`) is the
+default and is always available. **Net/HTTP/Persistent**
+(`:net_http_persistent`) maintains a connection pool so the cost
+of tearing down and setting up connections is kept low. **Curb**
+(`:curb`) provides bindings for libcurl.
+
+```ruby
+llm = LLM.deepseek(key: "...", transport: :net_http)
+mcp = LLM::MCP.http(url: "...", transport: :net_http_persistent)
+a2a = LLM::A2A.rest(url: "...", transport: :curb)
+```
+
+#### Why would I use it?
+
+Different environments have different HTTP needs.
+
+The default `net/http` transport is always available and works
+everywhere. The persistent transport reduces connection overhead
+when your agent makes many requests to the same provider in quick
+succession. Curb gives you libcurl bindings for environments where
+that is already configured or preferred.
+
+#### Notes
+
+The persistent transport is built on top of `net/http`. Curb
+requires the `curb` gem.

data/resources/deepdive/everything_else/audio.md

@@ -0,0 +1,122 @@
+
+## Audio
+
+### Introduction
+
+#### Overview
+
+The audio interface covers three things: turning text into speech,
+transcribing audio into text, and translating spoken language.
+OpenAI supports all three. Google and DeepInfra support subsets.
+Each method follows the same pattern: pass input, get output,
+copy the result somewhere useful.
+
+#### How it works
+
+When you want to convert text to speech, call
+[`LLM::Audio#create_speech`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_speech).
+The provider returns an audio clip as a
+[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
+object. The generated audio can be copied
+to a file or streamed directly. Each provider supports different
+output formats and voice options:
+
+```ruby
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.audio.create_speech(input: "Hello world")
+IO.copy_stream res.audio.decoded, "helloworld.mp3"
+```
+
+#### Why would I use it?
+
+Audio support lets you build voice interfaces, add accessibility
+features, and work across languages without wiring up a separate
+speech service. Generate audio for notifications, narrate written
+content, or add speech output to an existing application with a
+single method call.
+
+#### Notes
+
+OpenAI has full audio support. Google and DeepInfra have partial
+support. The
+[`LLM::Audio#create_speech`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_speech)
+method returns a
+[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
+object.
+
+### Transcription
+
+#### Overview
+
+Transcription turns an audio file into text. Pass a file path
+to
+[`LLM::Audio#create_transcription`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_transcription)
+and get back the spoken content as
+a string. OpenAI, Google, and DeepInfra support it. Transcribe
+meeting notes, voice memos, or podcast episodes for search and
+processing. The response is plain text you can feed into any
+downstream pipeline.
+
+#### How it works
+
+When you want to transcribe audio into text, call
+[`LLM::Audio#create_transcription`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_transcription).
+The provider processes the audio and returns the transcribed text.
+The file can be a local path or a URL depending on provider support:
+
+```ruby
+llm = LLM.google(key: ENV["KEY"])
+res = llm.audio.create_transcription(file: "helloworld.mp3")
+res.text  # => "Hello world"
+```
+
+#### Why would I use it?
+
+Transcribe recorded meetings, voice memos, or podcast episodes for
+search and processing. The returned text feeds directly into search
+indexes, summarization pipelines, or downstream extraction. Spoken
+content becomes as queryable as written text.
+
+#### Notes
+
+OpenAI has full audio support. Google and DeepInfra have partial
+support.
+
+### Translation
+
+#### Overview
+
+Translation transcribes audio and translates it into English in
+one step. Pass a file to
+[`LLM::Audio#create_translation`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_translation)
+and get back the
+translated text. OpenAI and Google support it. Translate
+multilingual podcasts, interviews, or any audio where you need
+the content in English without running a separate translation
+pipeline.
+
+#### How it works
+
+When you want to translate spoken audio into English, call
+[`LLM::Audio#create_translation`](https://r.uby.dev/api-docs/llm.rb/LLM/Audio.html#create_translation).
+The provider transcribes the spoken language and translates the
+result into English in a single operation. The returned text is the
+English translation:
+
+```ruby
+llm = LLM.google(key: ENV["KEY"])
+res = llm.audio.create_translation(file: "bomdia.mp3")
+res.text  # => "Good day"
+```
+
+#### Why would I use it?
+
+Translate a podcast or interview into another language without a
+separate speech service. The provider handles both transcription and
+translation in a single call, so you get English text from any
+supported source language without chaining two operations together.
+
+#### Notes
+
+Each method works independently and each provider supports a
+different subset.

data/resources/deepdive/everything_else/cost.md

@@ -0,0 +1,99 @@
+
+## LLM::Cost
+
+### Introduction
+
+#### Overview
+
+[`LLM::Cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
+represents the approximate cost of a conversation. It breaks the
+total down by token type, so you can see how much was spent on input,
+output, cached tokens, reasoning, audio, and images. Cost is computed
+from token usage and the pricing data shipped in the model registry.
+
+#### How it works
+
+When you want to know what a conversation cost so far, call
+[`LLM::Context#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#cost-instance_method)
+(or
+[`LLM::Agent#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#cost-instance_method))
+and read the breakdown. The REPL shows this live in its status bar
+after every turn:
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+ctx = LLM::Context.new(llm)
+ctx.talk "Hello"
+
+cost = ctx.cost
+cost.input    # => 0.0000042
+cost.output   # => 0.0000084
+cost.total    # => 0.0000126
+cost.to_s     # => "0.0000126"
+```
+
+#### Why would I use it?
+
+Cost tracking matters in production. Monitoring spend per
+conversation, per agent, or per provider tells you which workflows
+are expensive and when to switch models. Log a structured breakdown
+with
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method),
+or read
+[`LLM::Cost#total`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#total-instance_method)
+for a single number.
+
+#### Notes
+
+Cost is an approximation based on the pricing in the model registry.
+[`LLM::Cost.from`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#from-class_method)
+returns an empty cost when the model or registry cannot be found, so
+a missing model never crashes your code.
+
+### Reading the breakdown
+
+#### Overview
+
+[`LLM::Cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
+exposes each cost component as a reader, plus
+[`LLM::Cost#total`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#total-instance_method),
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method),
+and
+[`LLM::Cost#to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_s-instance_method).
+
+#### How it works
+
+Each component is a Float, or `nil` when no tokens of that type were
+used. The
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method)
+method returns a Hash with only the non-nil components and the total:
+
+```ruby
+cost = ctx.cost
+
+cost.input
+cost.output
+cost.cache_read
+cost.cache_write
+cost.reasoning
+cost.input_audio
+cost.output_audio
+cost.input_image
+
+cost.to_h  # => {input: 4.2e-06, output: 8.4e-06, total: 1.26e-05}
+```
+
+#### Why would I use it?
+
+The per-component breakdown shows where the money goes. High cache
+read costs suggest a conversation benefits from prompt caching.
+High reasoning costs point at a model that thinks a lot. Log
+[`LLM::Cost#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_h-instance_method)
+at the end of a session to keep a spend trail.
+
+#### Notes
+
+[`LLM::Cost#to_s`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#to_s-instance_method)
+returns the total in a compact, human-friendly format
+(`"0.0000126"`). Components that were not used are `nil`, so sum
+them with `compact` if you aggregate across conversations.

data/resources/deepdive/everything_else/images.md

@@ -0,0 +1,89 @@
+
+## Images
+
+### Introduction
+
+#### Overview
+
+A handful of providers can generate images from a text prompt.
+OpenAI, Google, xAI, DeepInfra, and DeepSeek all support it.
+OpenAI, xAI, and DeepInfra also let you edit existing images.
+The API is the same across providers, so switching between them
+requires no code changes.
+
+#### How it works
+
+The
+[`LLM::Images#create`](https://r.uby.dev/api-docs/llm.rb/LLM/Images.html#create)
+method sends a prompt to the provider and
+returns the result as a
+[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
+object. The same API works
+across providers: swap
+[`LLM.openai`](https://r.uby.dev/api-docs/llm.rb/LLM.html#openai-class_method) for
+[`LLM.xai`](https://r.uby.dev/api-docs/llm.rb/LLM.html#xai-class_method) and the rest
+of the code is identical.
+
+```ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.images.create(prompt: "a dog on a rocket to the moon")
+IO.copy_stream res.images[0], "dogrocket.png"
+```
+
+#### Why would I use it?
+
+Image generation and editing let the model produce visual output
+directly from your prompts. The API is the same across providers,
+so switching between OpenAI and xAI requires changing one line.
+Prototype visual concepts, generate assets, or augment datasets
+without wiring up a separate image API.
+
+#### Notes
+
+Google only supports image generation, not edits. DeepSeek
+generates SVGs rather than raster images. DeepSeek can also
+maintain a session across multiple generations through the
+`agent` parameter on the response object.
+
+### Editing
+
+#### Overview
+
+Image editing takes an existing image and a text prompt, then
+produces a modified version. You can add objects, change colours,
+or alter the scene while keeping the original composition.
+OpenAI, xAI, and DeepInfra support raster image edits. DeepSeek
+generates SVG documents which can be refined with follow-up
+text-to-image prompts, giving you iterative vector editing.
+
+#### How it works
+
+The
+[`LLM::Images#edit`](https://r.uby.dev/api-docs/llm.rb/LLM/Images.html#edit)
+method takes a prompt and an image path. It
+returns a modified image as a
+[`LLM::URIData`](https://r.uby.dev/api-docs/llm.rb/LLM/URIData.html)
+object. Copy the result
+to a file the same way you would with generated images.
+
+```ruby
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.images.edit(prompt: "add a mustache", image: "self.jpg")
+IO.copy_stream res.images[0], "mustache.png"
+```
+
+#### Why would I use it?
+
+Editing lets the model modify existing images rather than starting
+from scratch. DeepSeek's SVG output is particularly useful here
+because vector graphics can be refined iteratively. Make targeted
+adjustments: add objects, change colours, or alter the scene,
+all while keeping the original composition intact.
+
+#### Notes
+
+Google does not support image edits. DeepSeek can maintain a
+session across multiple generations through the `agent` parameter
+on the response object.