llm.rb 4.11.0 → 4.11.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a2af34506e099996b451951da8fb892ecdacebe9f29217bbf7a9e3ee3382d942
-  data.tar.gz: f49edb6d166ae113618139f0b118f37acbbd001b9b256d76d5c66b2828915a88
+  metadata.gz: f4c449483ce7a3b53411760d6376157fed3e23b4f013f23ae397255398bef368
+  data.tar.gz: a9a9c82b107cde72edfe6fe5f68ea7b1ea5e493314883d101c453a94db81b601
 SHA512:
-  metadata.gz: 8dbdbde04bf04fd714ce5ab3689f078f6a77243853bdb7ea287124295b2a5b5878493a36e4ec0c703a10466306f13ca503de9132b2a8a31c2c39b2f721b1bf78
-  data.tar.gz: 5bcb9be7c664bbee548cdc305878bc62fe1c8b5ab23d64630719084dab3581b8f4abf875a235a0e33ee05430cda8d69b0b6cc8fce538abafa4e8f85bbbbaead0
+  metadata.gz: 71a389b2fe654cfd053f45bd749c34b96c9d89ac60e984960f4a2720896588ba39056a3a92ab75a429572cd099961d9f3c02474f7dc43460b59866e41d8b5f28
+  data.tar.gz: 4532ec55176751b32ed21b281f2f71395dcd32cdf318973a751decf171af0a9e5f3f75b75871542c578fd9a2a134f8fc5cbf6a54b1df3b2dbe0c47745122b900

data/CHANGELOG.md

@@ -2,9 +2,9 @@
 
 ## Unreleased
 
-Changes since `v4.11.0`.
+Changes since `v4.11.1`.
 
-## v4.11.0
+## v4.11.1
 
 Changes since `v4.10.0`.
 

data/README.md

@@ -4,7 +4,7 @@
 <p align="center">
   <a href="https://0x1eef.github.io/x/llm.rb?rebuild=1"><img src="https://img.shields.io/badge/docs-0x1eef.github.io-blue.svg" alt="RubyDoc"></a>
   <a href="https://opensource.org/license/0bsd"><img src="https://img.shields.io/badge/License-0BSD-orange.svg?" alt="License"></a>
-  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-4.11.0-green.svg?" alt="Version"></a>
+  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-4.11.1-green.svg?" alt="Version"></a>
 </p>
 
 ## About
@@ -30,11 +30,14 @@ llm.rb is built around the state and execution model around them:
 
 - **Contexts are central** <br>
   They hold history, tools, schema, usage, cost, persistence, and execution state.
+- **Contexts can be serialized** <br>
+  A context can be serialized to JSON and stored on disk, in a database, in a
+  job queue, or anywhere else your application needs to persist state.
 - **Tool execution is explicit** <br>
   Run local, provider-native, and MCP tools sequentially or concurrently with threads, fibers, or async tasks.
 - **Run tools while streaming** <br>
   Start tool work while a response is still streaming instead of waiting for the turn to finish. <br>
-  This lets tool latency overlap with model output and is one of llm.rb's strongest execution features.
+  This overlaps tool latency with model output and exposes streamed tool-call events for introspection, making it one of llm.rb's strongest execution features.
 - **HTTP MCP can reuse connections** <br>
   Opt into persistent HTTP pooling for repeated remote MCP tool calls with `persist!`.
 - **One API across providers and capabilities** <br>
@@ -100,142 +103,114 @@ llm.rb provides a complete set of primitives for building LLM-powered systems:
 
 ## Quick Start
 
-#### Run Tools While Streaming
-
-llm.rb can start tool execution from streamed tool-call events before the
-assistant turn is fully finished. That means tool latency can overlap with
-streaming output instead of happening strictly after it. If your model emits
-tool calls early, this can noticeably reduce end-to-end latency for real
-systems.
+#### Simple Streaming
 
-This is different from plain concurrent tool execution. The tool starts while
-the response is still arriving, not after the turn has fully completed.
+At the simplest level, any object that implements `#<<` can receive visible
+output as it arrives. This works with `$stdout`, `StringIO`, files, sockets,
+and other Ruby IO-style objects.
 
-For example:
+For more control, llm.rb also supports advanced streaming patterns through
+[`LLM::Stream`](lib/llm/stream.rb). See [Advanced Streaming](#advanced-streaming)
+for a structured callback-based example:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
-class System < LLM::Tool
-  name "system"
-  description "Run a shell command"
-  params { _1.object(command: _1.string.required) }
-
-  def call(command:)
-    {success: Kernel.system(command)}
-  end
-end
-
-class Stream < LLM::Stream
-  def on_content(content)
-    print content
-  end
-
-  def on_tool_call(tool, error)
-    queue << (error || tool.spawn(:thread))
-  end
-end
-
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: Stream.new, tools: [System])
-
-ctx.talk("Run `date` and tell me what command you ran.")
-ctx.talk(ctx.wait(:thread)) while ctx.functions.any?
+ctx = LLM::Context.new(llm, stream: $stdout)
+loop do
+  print "> "
+  ctx.talk(STDIN.gets || break)
+  puts
+end
 ```
 
-#### Concurrent Tools
+#### Structured Outputs
 
-llm.rb provides explicit concurrency control for tool execution. The
-`wait(:thread)` method spawns each pending function in its own thread and waits
-for all to complete. You can also use `:fiber` for cooperative multitasking or
-`:task` for async/await patterns (requires the `async` gem). The context
-automatically collects all results and reports them back to the LLM in a
-single turn, maintaining conversation flow while parallelizing independent
-operations:
+The `LLM::Schema` system lets you define JSON schemas for structured outputs.
+Schemas can be defined as classes with `property` declarations or built
+programmatically using a fluent interface. When you pass a schema to a context,
+llm.rb adapts it into the provider's structured-output format when that
+provider supports one. The `content!` method then parses the assistant's JSON
+response into a Ruby object:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
+require "pp"
+
+class Report < LLM::Schema
+  property :category, Enum["performance", "security", "outage"], "Report category", required: true
+  property :summary, String, "Short summary", required: true
+  property :impact, OneOf[String, Integer], "Primary impact, as text or a count", required: true
+  property :services, Array[String], "Impacted services", required: true
+  property :timestamp, String, "When it happened", optional: true
+end
 
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: $stdout, tools: [FetchWeather, FetchNews, FetchStock])
+ctx = LLM::Context.new(llm, schema: Report)
+res = ctx.talk("Structure this report: 'Database latency spiked at 10:42 UTC, causing 5% request timeouts for 12 minutes.'")
+pp res.content!
 
-# Execute multiple independent tools concurrently
-ctx.talk("Summarize the weather, headlines, and stock price.")
-ctx.talk(ctx.wait(:thread)) while ctx.functions.any?
+# {
+#   "category" => "performance",
+#   "summary" => "Database latency spiked, causing 5% request timeouts for 12 minutes.",
+#   "impact" => "5% request timeouts",
+#   "services" => ["Database"],
+#   "timestamp" => "2024-06-05T10:42:00Z"
+# }
 ```
 
-#### MCP
+#### Tool Calling
 
-llm.rb integrates with the Model Context Protocol (MCP) to dynamically discover
-and use tools from external servers. This example starts a filesystem MCP
-server over stdio and makes its tools available to a context, enabling the LLM
-to interact with the local file system through a standardized interface.
-Use `LLM::MCP.stdio` or `LLM::MCP.http` when you want to make the transport
-explicit. Like `LLM::Context`, an MCP client is stateful and should remain
-isolated to a single thread:
+Tools in llm.rb can be defined as classes inheriting from `LLM::Tool` or as
+closures using `LLM.function`. When the LLM requests a tool call, the context
+stores `Function` objects in `ctx.functions`. The `call()` method executes all
+pending functions and returns their results to the LLM. Tools describe
+structured parameters with JSON Schema and adapt those definitions to each
+provider's tool-calling format (OpenAI, Anthropic, Google, etc.):
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
-llm = LLM.openai(key: ENV["KEY"])
-mcp = LLM::MCP.stdio(argv: ["npx", "-y", "@modelcontextprotocol/server-filesystem", Dir.pwd])
+class System < LLM::Tool
+  name "system"
+  description "Run a shell command"
+  param :command, String, "Command to execute", required: true
 
-begin
-  mcp.start
-  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
-  ctx.talk("List the directories in this project.")
-  ctx.talk(ctx.call(:functions)) while ctx.functions.any?
-ensure
-  mcp.stop
+  def call(command:)
+    {success: system(command)}
+  end
 end
-```
-
-You can also connect to an MCP server over HTTP. This is useful when the
-server already runs remotely and exposes MCP through a URL instead of a local
-process. If you expect repeated tool calls, use `persist!` to reuse a
-process-wide HTTP connection pool. This requires the optional
-`net-http-persistent` gem:
-
-```ruby
-#!/usr/bin/env ruby
-require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-mcp = LLM::MCP.http(
-  url: "https://api.githubcopilot.com/mcp/",
-  headers: {"Authorization" => "Bearer #{ENV.fetch("GITHUB_PAT")}"}
-).persist!
-
-begin
-  mcp.start
-  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
-  ctx.talk("List the available GitHub MCP toolsets.")
-  ctx.talk(ctx.call(:functions)) while ctx.functions.any?
-ensure
-  mcp.stop
-end
+ctx = LLM::Context.new(llm, stream: $stdout, tools: [System])
+ctx.talk("Run `date`.")
+ctx.talk(ctx.call(:functions)) while ctx.functions.any?
 ```
 
-#### Simple Streaming
+#### Concurrent Tools
 
-At the simplest level, any object that implements `#<<` can receive visible
-output as it arrives. This works with `$stdout`, `StringIO`, files, sockets,
-and other Ruby IO-style objects:
+llm.rb provides explicit concurrency control for tool execution. The
+`wait(:thread)` method spawns each pending function in its own thread and waits
+for all to complete. You can also use `:fiber` for cooperative multitasking or
+`:task` for async/await patterns (requires the `async` gem). The context
+automatically collects all results and reports them back to the LLM in a
+single turn, maintaining conversation flow while parallelizing independent
+operations:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: $stdout)
-loop do
-  print "> "
-  ctx.talk(STDIN.gets || break)
-  puts
-end
+ctx = LLM::Context.new(llm, stream: $stdout, tools: [FetchWeather, FetchNews, FetchStock])
+
+# Execute multiple independent tools concurrently
+ctx.talk("Summarize the weather, headlines, and stock price.")
+ctx.talk(ctx.wait(:thread)) while ctx.functions.any?
 ```
 
 #### Advanced Streaming
@@ -253,10 +228,11 @@ callbacks fast: they run inline with the parser.
 
 `on_tool_call` lets tools start before the model finishes its turn, for
 example with `tool.spawn(:thread)`, `tool.spawn(:fiber)`, or
-`tool.spawn(:task)`. This is the mechanism behind running tools while
-streaming.
+`tool.spawn(:task)`. That can overlap tool latency with streaming output and
+gives you a first-class place to observe and instrument tool-call execution as
+it unfolds.
 
-If a stream cannot execute a tool, `error` is an `LLM::Function::Return` that
+If a stream cannot resolve a tool, `error` is an `LLM::Function::Return` that
 communicates the failure back to the LLM. That lets the tool-call path recover
 and keeps the session alive. It also leaves control in the callback: it can
 send `error`, spawn the tool when `error == nil`, or handle the situation
@@ -304,69 +280,57 @@ while ctx.functions.any?
 end
 ```
 
-#### Tool Calling
+#### MCP
 
-Tools in llm.rb can be defined as classes inheriting from `LLM::Tool` or as
-closures using `LLM.function`. When the LLM requests a tool call, the context
-stores `Function` objects in `ctx.functions`. The `call()` method executes all
-pending functions and returns their results to the LLM. Tools describe
-structured parameters with JSON Schema and adapt those definitions to each
-provider's tool-calling format (OpenAI, Anthropic, Google, etc.):
+llm.rb integrates with the Model Context Protocol (MCP) to dynamically discover
+and use tools from external servers. This example starts a filesystem MCP
+server over stdio and makes its tools available to a context, enabling the LLM
+to interact with the local file system through a standardized interface.
+Use `LLM::MCP.stdio` or `LLM::MCP.http` when you want to make the transport
+explicit. Like `LLM::Context`, an MCP client is stateful and should remain
+isolated to a single thread:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
-class System < LLM::Tool
-  name "system"
-  description "Run a shell command"
-  param :command, String, "Command to execute", required: true
+llm = LLM.openai(key: ENV["KEY"])
+mcp = LLM::MCP.stdio(argv: ["npx", "-y", "@modelcontextprotocol/server-filesystem", Dir.pwd])
 
-  def call(command:)
-    {success: system(command)}
-  end
+begin
+  mcp.start
+  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
+  ctx.talk("List the directories in this project.")
+  ctx.talk(ctx.call(:functions)) while ctx.functions.any?
+ensure
+  mcp.stop
 end
-
-llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, stream: $stdout, tools: [System])
-ctx.talk("Run `date`.")
-ctx.talk(ctx.call(:functions)) while ctx.functions.any?
 ```
 
-#### Structured Outputs
-
-The `LLM::Schema` system lets you define JSON schemas for structured outputs.
-Schemas can be defined as classes with `property` declarations or built
-programmatically using a fluent interface. When you pass a schema to a context,
-llm.rb adapts it into the provider's structured-output format when that
-provider supports one. The `content!` method then parses the assistant's JSON
-response into a Ruby object:
+You can also connect to an MCP server over HTTP. This is useful when the
+server already runs remotely and exposes MCP through a URL instead of a local
+process. If you expect repeated tool calls, use `persist!` to reuse a
+process-wide HTTP connection pool. This requires the optional
+`net-http-persistent` gem:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
-require "pp"
-
-class Report < LLM::Schema
-  property :category, Enum["performance", "security", "outage"], "Report category", required: true
-  property :summary, String, "Short summary", required: true
-  property :impact, OneOf[String, Integer], "Primary impact, as text or a count", required: true
-  property :services, Array[String], "Impacted services", required: true
-  property :timestamp, String, "When it happened", optional: true
-end
 
 llm = LLM.openai(key: ENV["KEY"])
-ctx = LLM::Context.new(llm, schema: Report)
-res = ctx.talk("Structure this report: 'Database latency spiked at 10:42 UTC, causing 5% request timeouts for 12 minutes.'")
-pp res.content!
+mcp = LLM::MCP.http(
+  url: "https://api.githubcopilot.com/mcp/",
+  headers: {"Authorization" => "Bearer #{ENV.fetch("GITHUB_PAT")}"}
+).persist!
 
-# {
-#   "category" => "performance",
-#   "summary" => "Database latency spiked, causing 5% request timeouts for 12 minutes.",
-#   "impact" => "5% request timeouts",
-#   "services" => ["Database"],
-#   "timestamp" => "2024-06-05T10:42:00Z"
-# }
+begin
+  mcp.start
+  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
+  ctx.talk("List the available GitHub MCP toolsets.")
+  ctx.talk(ctx.call(:functions)) while ctx.functions.any?
+ensure
+  mcp.stop
+end
 ```
 
 ## Providers
@@ -542,11 +506,11 @@ res = ctx.talk("What is the capital of France?")
 puts res.content
 ```
 
-#### Context Persistence
+#### Context Persistence: Vanilla
 
-Contexts can be serialized and restored across process boundaries. This makes
-it possible to persist conversation state in a file, database, or queue and
-resume work later:
+Contexts can be serialized and restored across process boundaries. A context
+can be serialized to JSON and stored on disk, in a database, in a job queue,
+or anywhere else your application needs to persist state:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -556,12 +520,79 @@ llm = LLM.openai(key: ENV["KEY"])
 ctx = LLM::Context.new(llm)
 ctx.talk("Hello")
 ctx.talk("Remember that my favorite language is Ruby")
-ctx.save(path: "context.json")
+
+# Serialize to a string when you want to store the context yourself,
+# for example in a database row or job payload.
+payload = ctx.to_json
 
 restored = LLM::Context.new(llm)
-restored.restore(path: "context.json")
+restored.restore(string: payload)
 res = restored.talk("What is my favorite language?")
 puts res.content
+
+# You can also persist the same state to a file:
+ctx.save(path: "context.json")
+restored = LLM::Context.new(llm)
+restored.restore(path: "context.json")
+```
+
+#### Context Persistence: ActiveRecord (Rails)
+
+In a Rails application, you can also wrap persisted context state in an
+ActiveRecord model. A minimal schema would include a `snapshot` column for the
+serialized context payload (`jsonb` is recommended) and a `provider` column
+for the provider name:
+
+```ruby
+create_table :contexts do |t|
+  t.jsonb :snapshot
+  t.string :provider, null: false
+  t.timestamps
+end
+```
+
+For example:
+
+```ruby
+class Context < ApplicationRecord
+  def talk(...)
+    ctx.talk(...).tap { flush }
+  end
+
+  def wait(...)
+    ctx.wait(...).tap { flush }
+  end
+
+  def messages
+    ctx.messages
+  end
+
+  def model
+    ctx.model
+  end
+
+  def flush
+    update_column(:snapshot, ctx.to_json)
+  end
+
+  private
+
+  def ctx
+    @ctx ||= begin
+      ctx = LLM::Context.new(llm)
+      ctx.restore(string: snapshot) if snapshot
+      ctx
+    end
+  end
+
+  def llm
+    LLM.method(provider).call(key: ENV.fetch(key))
+  end
+
+  def key
+    "#{provider.upcase}_KEY"
+  end
+end
 ```
 
 #### Agents

data/lib/llm/tracer/telemetry.rb

@@ -126,7 +126,7 @@ module LLM
         "gen_ai.operation.name" => "execute_tool",
         "gen_ai.request.model" => model,
         "gen_ai.tool.call.id" => id,
-        "gen_ai.tool.name" => name,
+        "gen_ai.tool.name" => name&.to_s,
         "gen_ai.tool.call.arguments" => LLM.json.dump(arguments),
         "gen_ai.provider.name" => provider_name,
         "server.address" => provider_host,
@@ -145,7 +145,7 @@ module LLM
       return nil unless span
       attributes = {
         "gen_ai.tool.call.id" => result.id,
-        "gen_ai.tool.name" => result.name,
+        "gen_ai.tool.name" => result.name&.to_s,
         "gen_ai.tool.call.result" => LLM.json.dump(result.value)
       }.compact
       attributes.each { span.set_attribute(_1, _2) }

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "4.11.0"
+  VERSION = "4.11.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 4.11.0
+  version: 4.11.1
 platform: ruby
 authors:
 - Antar Azri