llm.rb 4.10.0 → 4.11.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7fc9e03f0a1b44775f414d310b202dcc68c1a8ebbb4bd7e6e0517902551ffbdf
-  data.tar.gz: 40a11ffd6d8f91ec0babbfb442174e68dac38fc2183922ff5994ff33670c572c
+  metadata.gz: f4c449483ce7a3b53411760d6376157fed3e23b4f013f23ae397255398bef368
+  data.tar.gz: a9a9c82b107cde72edfe6fe5f68ea7b1ea5e493314883d101c453a94db81b601
 SHA512:
-  metadata.gz: ac72f357d340917b99462f12f3a18d30039b3c4f34e8cbb1686da84f3c75b0d67919fc2d90732268301fa45a25d9e9d721d5e1f33a46a387196349878e384dd0
-  data.tar.gz: 92d7f611de8229d7f5cbe065169f0688940753eb1bffcc0ef907fd1d66ec18be840848b692aca81d24b1dfc6068011b644ed9be890e863ed8fa5d51194f089ed
+  metadata.gz: 71a389b2fe654cfd053f45bd749c34b96c9d89ac60e984960f4a2720896588ba39056a3a92ab75a429572cd099961d9f3c02474f7dc43460b59866e41d8b5f28
+  data.tar.gz: 4532ec55176751b32ed21b281f2f71395dcd32cdf318973a751decf171af0a9e5f3f75b75871542c578fd9a2a134f8fc5cbf6a54b1df3b2dbe0c47745122b900

data/CHANGELOG.md

@@ -0,0 +1,152 @@
+# Changelog
+
+## Unreleased
+
+Changes since `v4.11.1`.
+
+## v4.11.1
+
+Changes since `v4.10.0`.
+
+### Add
+
+- Add `LLM::Stream` for richer streaming callbacks, including `on_content`,
+  `on_reasoning_content`, and `on_tool_call` for concurrent tool execution.
+- Add `LLM::Stream#wait` as a shortcut for `queue.wait`.
+- Add `LLM::Context#wait` as a shortcut for the configured stream's `wait`.
+- Add `LLM::Context#call(:functions)` as a shortcut for `functions.call`.
+- Add `LLM::Function.registry` and enhanced support for MCP tools in
+  `LLM::Tool.registry` for tool resolution during streaming.
+- Add normalized `LLM::Response` for OpenAI Responses, providing `content`,
+  `content!`, `messages` / `choices`, `usage`, and `reasoning_content`.
+- Add `mode: :responses` to `LLM::Context` for routing `talk` through the
+  Responses API.
+- Add `LLM::Context#returns` for collecting pending tool returns from the context.
+- Add persistent HTTP connection pooling for repeated MCP tool calls via
+  `LLM.mcp(http: ...).persist!`.
+- Add explicit MCP transport constructors via `LLM::MCP.stdio(...)` and
+  `LLM::MCP.http(...)`.
+
+### Fix
+
+- Fix Google tool-call handling by synthesizing stable ids when Gemini does
+  not provide a direct tool-call id.
+
+## v4.10.0
+
+Changes since `v4.9.0`.
+
+### Add
+
+- Add HTTP transport for MCP with `LLM::MCP::Transport::HTTP` for remote servers
+- Add JSON Schema union types (`any_of`, `all_of`, `one_of`) with parser integration
+- Add JSON Schema type array union support (e.g., `"type": ["object", "null"]`)
+- Add JSON Schema type inference from `const`, `enum`, or `default` fields
+
+### Change
+
+- Update `LLM::MCP` constructor for exclusive `http:` or `stdio:` transport
+- Update `LLM::MCP` documentation for HTTP transport support
+
+## v4.9.0
+
+Changes since `v4.8.0`.
+
+### Add
+
+- Add fiber-based concurrency with `LLM::Function::FiberGroup` and
+  `LLM::Function::TaskGroup` classes for lightweight async execution.
+- Add `:thread`, `:task`, and `:fiber` strategy parameter to
+  `LLM::Function#spawn` for explicit concurrency control.
+- Add stdio MCP client support, including remote tool discovery and
+  invocation through `LLM.mcp`, `LLM::Context`, and existing function/tool
+  APIs.
+- Add model registry support via `LLM::Registry`, including model
+  metadata lookup, pricing, modalities, limits, and cost estimation.
+- Add context access to a model context window via
+  `LLM::Context#context_window`.
+- Add tracking of defined tools in the tool registry.
+- Add `LLM::Schema::Enum`, enabling `Enum[...]` as a schema/tool
+  parameter type.
+- Add top-level Anthropic system instruction support using Anthropic's
+  provider-specific request format.
+- Add richer tracing hooks and extra metadata support for
+  LangSmith/OpenTelemetry-style traces.
+- Add rack/websocket and Relay-related example work, including MCP-focused
+  examples.
+- Add concurrent tool execution with `LLM::Function#spawn`,
+  `LLM::Function::Array` (`call`, `wait`, `spawn`), and
+  `LLM::Function::ThreadGroup`.
+- Add `LLM::Function::ThreadGroup#alive?` method for non-blocking
+  monitoring of concurrent tool execution.
+- Add `LLM::Function::ThreadGroup#value` alias for `ThreadGroup#wait` for
+  consistency with Ruby's `Thread#value`.
+
+### Change
+
+- Rename `LLM::Session` to `LLM::Context` throughout the codebase to better
+  reflect the concept of a stateful interaction environment.
+- Rename `LLM::Gemini` to `LLM::Google` to better reflect provider naming.
+- Standardize model objects across providers around a smaller common
+  interface.
+- Switch registry cost internals from `LLM::Estimate` to `LLM::Cost`.
+- Update image generation defaults so OpenAI and xAI consistently return
+  base64-encoded image data by default.
+- Update `LLM::Bot` deprecation warning from v5.0 to v6.0, giving users
+  more time to migrate to `LLM::Context`.
+- Rework the README and screencast documentation to better cover MCP,
+  registry, contexts, prompts, concurrency, providers, and example flow.
+- Expand the README with architecture, production, and provider guidance
+  while improving readability and example ordering.
+
+### Fix
+
+- Fix local schema `$ref` resolution in `LLM::Schema::Parser`.
+- Fix multiple MCP issues around stdio env handling, request IDs, registry
+  interaction, tool registration, and filtering of MCP tools from the
+  standard tool registry.
+- Fix stream parsing issues, including chunk-splitting bugs and safer
+  handling of streamed error responses.
+- Fix prompt handling across contexts, agents, and provider adapters so
+  prompt turns remain consistent in history and completions.
+- Fix several tool/context issues, including function return wrapping,
+  tool lookup after deserialization, unnamed subclass filtering, and
+  thread-safety around tool registry mutations.
+- Fix Google tool-call handling to preserve `thoughtSignature`.
+- Fix `LLM::Tracer::Logger` argument handling.
+- Fix packaging/docs issues such as registry files in the gemspec and
+  stale provider docs.
+- Fix Google provider handling of `nil` function IDs during context
+  deserialization.
+- Fix MCP stdio transport by increasing poll timeout for better
+  reliability.
+- Fix Google provider to properly cast non-Hash tool results into Hash
+  format for API compatibility.
+- Fix schema parser to support recursive normalization of `Array`,
+  `LLM::Object`, and nested structures.
+- Fix DeepSeek provider to tolerate malformed tool arguments.
+- Fix `LLM::Function::TaskGroup#alive?` to properly delegate to
+  `Async::Task#alive?`.
+- Fix various RuboCop errors across the codebase.
+- Fix DeepSeek provider to handle JSON that might be valid but unexpected.
+
+### Notes
+
+Notable merged work in this range includes:
+
+- `feat(function): add fiber-based concurrency for async environments (#64)`
+- `feat(mcp): add stdio MCP support (#134)`
+- `Add LLM::Registry + cost support (#133)`
+- `Consistent model objects across providers (#131)`
+- `Add rack + websocket example (#130)`
+- `feat(gemspec): add changelog URI (#136)`
+- `feat(function): alias ThreadGroup#wait as ThreadGroup#value (#62)`
+- README and screencast refresh across `#66`, `#67`, `#68`, `#71`, and
+  `#72`
+- `chore(bot): update deprecation warning from v5.0 to v6.0`
+- `fix(deepseek): tolerate malformed tool arguments`
+- `refactor(context): Rename Session as Context (#70)`
+
+Comparison base:
+- Latest tag: `v4.8.0` (`6468f2426ee125823b7ae43b4af507b125f96ffc`)
+- HEAD used for this changelog: `915c48da6fda9bef1554ff613947a6ce26d382e3`

data/README.md

@@ -4,7 +4,7 @@
 <p align="center">
   <a href="https://0x1eef.github.io/x/llm.rb?rebuild=1"><img src="https://img.shields.io/badge/docs-0x1eef.github.io-blue.svg" alt="RubyDoc"></a>
   <a href="https://opensource.org/license/0bsd"><img src="https://img.shields.io/badge/License-0BSD-orange.svg?" alt="License"></a>
-  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-4.10.0-green.svg?" alt="Version"></a>
+  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-4.11.1-green.svg?" alt="Version"></a>
 </p>
 
 ## About
@@ -30,8 +30,16 @@ llm.rb is built around the state and execution model around them:
 
 - **Contexts are central** <br>
   They hold history, tools, schema, usage, cost, persistence, and execution state.
+- **Contexts can be serialized** <br>
+  A context can be serialized to JSON and stored on disk, in a database, in a
+  job queue, or anywhere else your application needs to persist state.
 - **Tool execution is explicit** <br>
   Run local, provider-native, and MCP tools sequentially or concurrently with threads, fibers, or async tasks.
+- **Run tools while streaming** <br>
+  Start tool work while a response is still streaming instead of waiting for the turn to finish. <br>
+  This overlaps tool latency with model output and exposes streamed tool-call events for introspection, making it one of llm.rb's strongest execution features.
+- **HTTP MCP can reuse connections** <br>
+  Opt into persistent HTTP pooling for repeated remote MCP tool calls with `persist!`.
 - **One API across providers and capabilities** <br>
   The same model covers chat, files, images, audio, embeddings, vector stores, and more.
 - **Thread-safe where it matters** <br>
@@ -75,12 +83,14 @@ llm.rb is built in layers, each providing explicit control:
 llm.rb provides a complete set of primitives for building LLM-powered systems:
 
 - **Chat & Contexts** — stateless and stateful interactions with persistence
-- **Streaming** — real-time responses across providers
+- **Streaming** — real-time responses across providers, including structured stream callbacks
+- **Reasoning Support** — full stream, message, and response support when providers expose reasoning
 - **Tool Calling** — define and execute functions with automatic orchestration
+- **Run Tools While Streaming** — begin tool work before the model finishes its turn
 - **Concurrent Execution** — threads, async tasks, and fibers
 - **Agents** — reusable, preconfigured assistants with tool auto-execution
 - **Structured Outputs** — JSON schema-based responses
-- **MCP Support** — integrate external tool servers dynamically
+- **MCP Support** — integrate external tool servers dynamically over stdio or HTTP
 - **Multimodal Inputs** — text, images, audio, documents, URLs
 - **Audio** — text-to-speech, transcription, translation
 - **Images** — generation and editing
@@ -93,163 +103,234 @@ llm.rb provides a complete set of primitives for building LLM-powered systems:
 
 ## Quick Start
 
-#### Concurrent Tools
+#### Simple Streaming
 
-llm.rb provides explicit concurrency control for tool execution. The
-`wait(:thread)` method spawns each pending function in its own thread and waits
-for all to complete. You can also use `:fiber` for cooperative multitasking or
-`:task` for async/await patterns (requires the `async` gem). The context
-automatically collects all results and reports them back to the LLM in a
-single turn, maintaining conversation flow while parallelizing independent
-operations:
+At the simplest level, any object that implements `#<<` can receive visible
+output as it arrives. This works with `$stdout`, `StringIO`, files, sockets,
+and other Ruby IO-style objects.
+
+For more control, llm.rb also supports advanced streaming patterns through
+[`LLM::Stream`](lib/llm/stream.rb). See [Advanced Streaming](#advanced-streaming)
+for a structured callback-based example:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: $stdout, tools: [FetchWeather, FetchNews, FetchStock])
-
-# Execute multiple independent tools concurrently
-ctx.talk("Summarize the weather, headlines, and stock price.")
-ctx.talk(ctx.functions.wait(:thread)) while ctx.functions.any?
+ctx = LLM::Context.new(llm, stream: $stdout)
+loop do
+  print "> "
+  ctx.talk(STDIN.gets || break)
+  puts
+end
 ```
 
-#### MCP
+#### Structured Outputs
 
-llm.rb integrates with the Model Context Protocol (MCP) to dynamically discover
-and use tools from external servers. This example starts a filesystem MCP
-server over stdio and makes its tools available to a context, enabling the LLM
-to interact with the local file system through a standardized interface:
+The `LLM::Schema` system lets you define JSON schemas for structured outputs.
+Schemas can be defined as classes with `property` declarations or built
+programmatically using a fluent interface. When you pass a schema to a context,
+llm.rb adapts it into the provider's structured-output format when that
+provider supports one. The `content!` method then parses the assistant's JSON
+response into a Ruby object:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
+require "pp"
+
+class Report < LLM::Schema
+  property :category, Enum["performance", "security", "outage"], "Report category", required: true
+  property :summary, String, "Short summary", required: true
+  property :impact, OneOf[String, Integer], "Primary impact, as text or a count", required: true
+  property :services, Array[String], "Impacted services", required: true
+  property :timestamp, String, "When it happened", optional: true
+end
 
 llm = LLM.openai(key: ENV["KEY"])
-mcp = LLM.mcp(stdio: {argv: ["npx", "-y", "@modelcontextprotocol/server-filesystem", Dir.pwd]})
+ctx = LLM::Context.new(llm, schema: Report)
+res = ctx.talk("Structure this report: 'Database latency spiked at 10:42 UTC, causing 5% request timeouts for 12 minutes.'")
+pp res.content!
 
-begin
-  mcp.start
-  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
-  ctx.talk("List the directories in this project.")
-  ctx.talk(ctx.functions.call) while ctx.functions.any?
-ensure
-  mcp.stop
-end
+# {
+#   "category" => "performance",
+#   "summary" => "Database latency spiked, causing 5% request timeouts for 12 minutes.",
+#   "impact" => "5% request timeouts",
+#   "services" => ["Database"],
+#   "timestamp" => "2024-06-05T10:42:00Z"
+# }
 ```
 
-You can also connect to a hosted MCP server over HTTP. This is useful when the
-server already runs remotely and exposes MCP through a URL instead of a local
-process:
+#### Tool Calling
+
+Tools in llm.rb can be defined as classes inheriting from `LLM::Tool` or as
+closures using `LLM.function`. When the LLM requests a tool call, the context
+stores `Function` objects in `ctx.functions`. The `call()` method executes all
+pending functions and returns their results to the LLM. Tools describe
+structured parameters with JSON Schema and adapt those definitions to each
+provider's tool-calling format (OpenAI, Anthropic, Google, etc.):
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
-llm = LLM.openai(key: ENV["KEY"])
-mcp = LLM.mcp(http: {
-  url: "https://api.githubcopilot.com/mcp/",
-  headers: {"Authorization" => "Bearer #{ENV.fetch("GITHUB_PAT")}"}
-})
+class System < LLM::Tool
+  name "system"
+  description "Run a shell command"
+  param :command, String, "Command to execute", required: true
 
-begin
-  mcp.start
-  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
-  ctx.talk("List the available GitHub MCP toolsets.")
-  ctx.talk(ctx.functions.call) while ctx.functions.any?
-ensure
-  mcp.stop
+  def call(command:)
+    {success: system(command)}
+  end
 end
+
+llm = LLM.openai(key: ENV["KEY"])
+ctx = LLM::Context.new(llm, stream: $stdout, tools: [System])
+ctx.talk("Run `date`.")
+ctx.talk(ctx.call(:functions)) while ctx.functions.any?
 ```
 
-#### Streaming Chat
+#### Concurrent Tools
 
-This example demonstrates llm.rb's streaming support. The `stream: $stdout`
-parameter tells the context to write responses incrementally as they arrive
-from the LLM. The `Context` object manages the conversation history, and
-`talk()` sends your input while automatically appending both your message and
-the LLM's response to the context. Streams accept any object with `#<<`,
-giving you flexibility to pipe output to files, network sockets, or custom
-buffers:
+llm.rb provides explicit concurrency control for tool execution. The
+`wait(:thread)` method spawns each pending function in its own thread and waits
+for all to complete. You can also use `:fiber` for cooperative multitasking or
+`:task` for async/await patterns (requires the `async` gem). The context
+automatically collects all results and reports them back to the LLM in a
+single turn, maintaining conversation flow while parallelizing independent
+operations:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: $stdout)
-loop do
-  print "> "
-  ctx.talk(STDIN.gets || break)
-  puts
-end
+ctx = LLM::Context.new(llm, stream: $stdout, tools: [FetchWeather, FetchNews, FetchStock])
+
+# Execute multiple independent tools concurrently
+ctx.talk("Summarize the weather, headlines, and stock price.")
+ctx.talk(ctx.wait(:thread)) while ctx.functions.any?
 ```
 
-#### Tool Calling
+#### Advanced Streaming
 
-Tools in llm.rb can be defined as classes inheriting from `LLM::Tool` or as
-closures using `LLM.function`. When the LLM requests a tool call, the context
-stores `Function` objects in `ctx.functions`. The `call()` method executes all
-pending functions and returns their results to the LLM. Tools support
-structured parameters with JSON Schema validation and automatically adapt to
-each provider's API format (OpenAI, Anthropic, Google, etc.):
+llm.rb also supports the [`LLM::Stream`](lib/llm/stream.rb) interface for
+structured streaming events:
+
+- `on_content` for visible assistant output
+- `on_reasoning_content` for separate reasoning output
+- `on_tool_call` for streamed tool-call notifications
+
+Subclass [`LLM::Stream`](lib/llm/stream.rb) when you want features like
+`queue` and `wait`, or implement the same methods on your own object. Keep these
+callbacks fast: they run inline with the parser.
+
+`on_tool_call` lets tools start before the model finishes its turn, for
+example with `tool.spawn(:thread)`, `tool.spawn(:fiber)`, or
+`tool.spawn(:task)`. That can overlap tool latency with streaming output and
+gives you a first-class place to observe and instrument tool-call execution as
+it unfolds.
+
+If a stream cannot resolve a tool, `error` is an `LLM::Function::Return` that
+communicates the failure back to the LLM. That lets the tool-call path recover
+and keeps the session alive. It also leaves control in the callback: it can
+send `error`, spawn the tool when `error == nil`, or handle the situation
+however it sees fit.
+
+In normal use this should be rare, since `on_tool_call` is usually called with
+a resolved tool and `error == nil`. To resolve a tool call, the tool must be
+found in `LLM::Function.registry`. That covers `LLM::Tool` subclasses,
+including MCP tools, but not `LLM.function` closures, which are excluded
+because they may be bound to local state:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
+# Assume `System < LLM::Tool` is already defined.
 
-class System < LLM::Tool
-  name "system"
-  description "Run a shell command"
-  param :command, String, "Command to execute", required: true
+class Stream < LLM::Stream
+  attr_reader :content, :reasoning_content
 
-  def call(command:)
-    {success: system(command)}
+  def initialize
+    @content = +""
+    @reasoning_content = +""
+  end
+
+  def on_content(content)
+    @content << content
+    print content
+  end
+
+  def on_reasoning_content(content)
+    @reasoning_content << content
+  end
+
+  def on_tool_call(tool, error)
+    queue << (error || tool.spawn(:thread))
   end
 end
 
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: $stdout, tools: [System])
-ctx.talk("Run `date`.")
-ctx.talk(ctx.functions.call) while ctx.functions.any?
+ctx = LLM::Context.new(llm, stream: Stream.new, tools: [System])
+
+ctx.talk("Run `date` and `uname -a`.")
+while ctx.functions.any?
+  ctx.talk(ctx.wait(:thread))
+end
 ```
 
-#### Structured Outputs
+#### MCP
 
-The `LLM::Schema` system lets you define JSON schemas that LLMs must follow.
-Schemas can be defined as classes with `property` declarations or built
-programmatically using a fluent interface. When you pass a schema to a context,
-llm.rb automatically configures the provider's JSON mode and validates
-responses against your schema. The `content!` method returns the parsed JSON
-object, while errors are captured as structured data rather than raising
-exceptions:
+llm.rb integrates with the Model Context Protocol (MCP) to dynamically discover
+and use tools from external servers. This example starts a filesystem MCP
+server over stdio and makes its tools available to a context, enabling the LLM
+to interact with the local file system through a standardized interface.
+Use `LLM::MCP.stdio` or `LLM::MCP.http` when you want to make the transport
+explicit. Like `LLM::Context`, an MCP client is stateful and should remain
+isolated to a single thread:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
-require "pp"
 
-class Report < LLM::Schema
-  property :category, Enum["performance", "security", "outage"], "Report category", required: true
-  property :summary, String, "Short summary", required: true
-  property :impact, OneOf[String, Integer], "Primary impact, as text or a count", required: true
-  property :services, Array[String], "Impacted services", required: true
-  property :timestamp, String, "When it happened", optional: true
+llm = LLM.openai(key: ENV["KEY"])
+mcp = LLM::MCP.stdio(argv: ["npx", "-y", "@modelcontextprotocol/server-filesystem", Dir.pwd])
+
+begin
+  mcp.start
+  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
+  ctx.talk("List the directories in this project.")
+  ctx.talk(ctx.call(:functions)) while ctx.functions.any?
+ensure
+  mcp.stop
 end
+```
+
+You can also connect to an MCP server over HTTP. This is useful when the
+server already runs remotely and exposes MCP through a URL instead of a local
+process. If you expect repeated tool calls, use `persist!` to reuse a
+process-wide HTTP connection pool. This requires the optional
+`net-http-persistent` gem:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, schema: Report)
-res = ctx.talk("Structure this report: 'Database latency spiked at 10:42 UTC, causing 5% request timeouts for 12 minutes.'")
-pp res.content!
+mcp = LLM::MCP.http(
+  url: "https://api.githubcopilot.com/mcp/",
+  headers: {"Authorization" => "Bearer #{ENV.fetch("GITHUB_PAT")}"}
+).persist!
 
-# {
-#   "category" => "performance",
-#   "summary" => "Database latency spiked, causing 5% request timeouts for 12 minutes.",
-#   "impact" => "5% request timeouts",
-#   "services" => ["Database"],
-#   "timestamp" => "2024-06-05T10:42:00Z"
-# }
+begin
+  mcp.start
+  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
+  ctx.talk("List the available GitHub MCP toolsets.")
+  ctx.talk(ctx.call(:functions)) while ctx.functions.any?
+ensure
+  mcp.stop
+end
 ```
 
 ## Providers
@@ -405,27 +486,31 @@ puts "Cost: $#{model_info.cost.input}/1M input tokens"
 
 #### Responses API
 
-llm.rb also supports OpenAI's Responses API through `llm.responses` and
-`ctx.respond`. This API can maintain response state server-side and can reduce
-how much conversation state needs to be sent on each turn:
+llm.rb also supports OpenAI's Responses API through `LLM::Context` with
+`mode: :responses`. The important switch is `store:`. With `store: false`, the
+Responses API stays stateless while still using the Responses endpoint, which
+is useful for models or features that are only available through the Responses
+API. With `store: true`, OpenAI can keep
+response state server-side and reduce how much conversation state needs to be
+sent on each turn:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm)
+ctx = LLM::Context.new(llm, mode: :responses, store: false)
 
-ctx.respond("Your task is to answer the user's questions", role: :developer)
-res = ctx.respond("What is the capital of France?")
-puts res.output_text
+ctx.talk("Your task is to answer the user's questions", role: :developer)
+res = ctx.talk("What is the capital of France?")
+puts res.content
 ```
 
-#### Context Persistence
+#### Context Persistence: Vanilla
 
-Contexts can be serialized and restored across process boundaries. This makes
-it possible to persist conversation state in a file, database, or queue and
-resume work later:
+Contexts can be serialized and restored across process boundaries. A context
+can be serialized to JSON and stored on disk, in a database, in a job queue,
+or anywhere else your application needs to persist state:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -435,12 +520,79 @@ llm = LLM.openai(key: ENV["KEY"])
 ctx = LLM::Context.new(llm)
 ctx.talk("Hello")
 ctx.talk("Remember that my favorite language is Ruby")
-ctx.save(path: "context.json")
+
+# Serialize to a string when you want to store the context yourself,
+# for example in a database row or job payload.
+payload = ctx.to_json
 
 restored = LLM::Context.new(llm)
-restored.restore(path: "context.json")
+restored.restore(string: payload)
 res = restored.talk("What is my favorite language?")
 puts res.content
+
+# You can also persist the same state to a file:
+ctx.save(path: "context.json")
+restored = LLM::Context.new(llm)
+restored.restore(path: "context.json")
+```
+
+#### Context Persistence: ActiveRecord (Rails)
+
+In a Rails application, you can also wrap persisted context state in an
+ActiveRecord model. A minimal schema would include a `snapshot` column for the
+serialized context payload (`jsonb` is recommended) and a `provider` column
+for the provider name:
+
+```ruby
+create_table :contexts do |t|
+  t.jsonb :snapshot
+  t.string :provider, null: false
+  t.timestamps
+end
+```
+
+For example:
+
+```ruby
+class Context < ApplicationRecord
+  def talk(...)
+    ctx.talk(...).tap { flush }
+  end
+
+  def wait(...)
+    ctx.wait(...).tap { flush }
+  end
+
+  def messages
+    ctx.messages
+  end
+
+  def model
+    ctx.model
+  end
+
+  def flush
+    update_column(:snapshot, ctx.to_json)
+  end
+
+  private
+
+  def ctx
+    @ctx ||= begin
+      ctx = LLM::Context.new(llm)
+      ctx.restore(string: snapshot) if snapshot
+      ctx
+    end
+  end
+
+  def llm
+    LLM.method(provider).call(key: ENV.fetch(key))
+  end
+
+  def key
+    "#{provider.upcase}_KEY"
+  end
+end
 ```
 
 #### Agents