llm.rb 4.20.0 → 4.20.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d57e4d2af8568cdd6328c0a956fddb40aecbd2943a268b595dbd87ee811553a4
-  data.tar.gz: 8cf576171c3bfd7328b8316d42aecbb364a7e4d0a6bbff707cdc65cc9ddfbd01
+  metadata.gz: a182d595ad65c1cb2f1a796b83e48cba4f1038031ec140709e902734051a8b46
+  data.tar.gz: b8cdb2e051bc620f111a97236bd64fe7940ff9f3d5b44c9f07b115641d74abcd
 SHA512:
-  metadata.gz: 4d9087909b30c47e5ddb9c9407b53efdbbd2a3732629579dfd53415d60e1457a56b738b9942b578434888b97ee597d78955f21a1af5235847e6daa944810e8d7
-  data.tar.gz: a890e08d0129ccfa18188efb503e8ac32e4d5424a79851bf2bfa15424b713cae4431afeb78fe7645437d8b3d0c11cc2d975d7415279a2880ca5cd557de57cf5f
+  metadata.gz: a6fd61aaa9479ec34af93a1e732acf553a055e36a4f5e822a2c643ef2bf537923a7d0a968b40c6a8cfa9a09af8186ba31467fe627462da49389f1c6594d7ee41
+  data.tar.gz: df56a4624eca8f7007ea2054d79812df553df69d867297230c9b38368c87e67c06187dbf03195b5fcaae1b1701b82a79cd7be10ed86364a49802573367910d10

data/CHANGELOG.md

@@ -2,8 +2,55 @@
 
 ## Unreleased
 
+Changes since `v4.20.2`.
+
+## v4.20.2
+
+Changes since `v4.20.1`.
+
+This patch release improves runtime behavior around interruption and mixed
+concurrency waits. It also rounds out response API uniformity for Google
+completion responses.
+
+### Fix
+
+* **Expose Google completion response IDs through `.id`** <br>
+  Add `LLM::Response#id` support to Google completion responses so tracer
+  and caller code can rely on the same API used by other providers.
+
+* **Track interrupt ownership on the active request** <br>
+  Bind `LLM::Context` interruption to the fiber running `talk` or `respond`
+  so `interrupt!` works correctly when requests are started outside the
+  context's initialization fiber.
+
+### Change
+
+* **Allow mixed concurrency strategies in `wait(...)`** <br>
+  Let `LLM::Context#wait`, `LLM::Stream#wait`, and `LLM::Agent.concurrency`
+  accept arrays such as `[:thread, :ractor]` so mixed tool sets can wait on
+  more than one concurrency strategy.
+
+## v4.20.1
+
 Changes since `v4.20.0`.
 
+This patch release fixes ORM option resolution in the Sequel and
+ActiveRecord wrappers. Symbol-based `provider:` and `context:` hooks now
+resolve correctly, and internal default option constants are referenced
+explicitly instead of relying on nested constant lookup.
+
+### Fix
+
+* **Fix symbol-based ORM option hooks for provider and context hashes** <br>
+  Make `provider:` and `context:` resolve symbol hooks through the model in
+  the Sequel plugin and ActiveRecord wrappers instead of falling back to an
+  empty hash.
+
+* **Fix ORM wrapper constant lookup for option defaults** <br>
+  Qualify internal `EMPTY_HASH` / `DEFAULTS` references in the Sequel plugin
+  and ActiveRecord wrappers so option resolution does not depend on nested
+  constant lookup quirks.
+
 ## v4.20.0
 
 Changes since `v4.19.0`.

data/README.md

@@ -4,7 +4,7 @@
 <p align="center">
   <a href="https://0x1eef.github.io/x/llm.rb?rebuild=1"><img src="https://img.shields.io/badge/docs-0x1eef.github.io-blue.svg" alt="RubyDoc"></a>
   <a href="https://opensource.org/license/0bsd"><img src="https://img.shields.io/badge/License-0BSD-orange.svg?" alt="License"></a>
-  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-4.20.0-green.svg?" alt="Version"></a>
+  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-4.20.2-green.svg?" alt="Version"></a>
 </p>
 
 ## About
@@ -23,7 +23,8 @@ pieces only when needed, includes built-in ActiveRecord support through
 long-lived, tool-capable, stateful AI workflows instead of just
 request/response helpers.
 
-Want to see some code? Jump to [the examples](#examples) section.
+Want to see some code? Jump to [the examples](#examples) section. <br>
+Want a taste of what llm.rb can build? See [the screencast](#screencast).
 
 ## Architecture
 
@@ -186,7 +187,7 @@ gem install llm.rb
 
 ## Examples
 
-**REPL**
+#### REPL
 
 This example uses [`LLM::Context`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html) directly for an interactive REPL. <br> See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
 
@@ -203,7 +204,61 @@ loop do
 end
 ```
 
-**Sequel (ORM)**
+#### Streaming
+
+This example uses [`LLM::Stream`](https://0x1eef.github.io/x/llm.rb/LLM/Stream.html) directly so visible output and tool execution can happen together. <br> See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
+
+```ruby
+require "llm"
+
+class Stream < LLM::Stream
+  def on_content(content)
+    $stdout << content
+  end
+
+  def on_tool_call(tool, error)
+    return queue << error if error
+    $stdout << "\nRunning tool #{tool.name}...\n"
+    queue << tool.spawn(:thread)
+  end
+
+  def on_tool_return(tool, result)
+    if result.error?
+      $stdout << "Tool #{tool.name} failed\n"
+    else
+      $stdout << "Finished tool #{tool.name}\n"
+    end
+  end
+end
+
+llm = LLM.openai(key: ENV["KEY"])
+ctx = LLM::Context.new(llm, stream: Stream.new, tools: [System])
+
+ctx.talk("Run `date` and `uname -a`.")
+ctx.talk(ctx.wait(:thread)) while ctx.functions.any?
+```
+
+#### Request Cancellation
+
+Need to cancel a stream? llm.rb has you covered through [`LLM::Context#interrupt!`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html#interrupt-21-instance_method). <br> See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
+
+```ruby
+require "llm"
+require "io/console"
+
+llm = LLM.openai(key: ENV["KEY"])
+ctx = LLM::Context.new(llm, stream: $stdout)
+
+worker = Thread.new do
+  ctx.talk("Write a very long essay about network protocols.")
+end
+
+STDIN.getch
+ctx.interrupt!
+worker.join
+```
+
+#### Sequel (ORM)
 
 The `plugin :llm` integration wraps [`LLM::Context`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html) on a `Sequel::Model` and keeps tool execution explicit. <br> See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
 
@@ -222,7 +277,7 @@ ctx.talk("Remember that my favorite language is Ruby")
 puts ctx.talk("What is my favorite language?").content
 ```
 
-**ActiveRecord (ORM): acts_as_llm**
+#### ActiveRecord (ORM): acts_as_llm
 
 The `acts_as_llm` method wraps [`LLM::Context`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html) and
 provides full control over tool execution. <br> See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
@@ -242,7 +297,7 @@ ctx.talk("Remember that my favorite language is Ruby")
 puts ctx.talk("What is my favorite language?").content
 ```
 
-**ActiveRecord (ORM): acts_as_agent**
+#### ActiveRecord (ORM): acts_as_agent
 
 The `acts_as_agent` method wraps [`LLM::Agent`](https://0x1eef.github.io/x/llm.rb/LLM/Agent.html) and
 manages tool execution for you. <br> See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
@@ -272,7 +327,7 @@ ticket = Ticket.create!(provider: "openai", model: "gpt-5.4-mini")
 puts ticket.talk("How do I rotate my API key?").content
 ```
 
-**Agent**
+#### Agent
 
 This example uses [`LLM::Agent`](https://0x1eef.github.io/x/llm.rb/LLM/Agent.html) directly and lets the agent manage tool execution. <br> See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
 
@@ -291,6 +346,37 @@ agent = ShellAgent.new(llm)
 puts agent.talk("What time is it on this system?").content
 ```
 
+#### MCP
+
+This example uses [`LLM::MCP`](https://0x1eef.github.io/x/llm.rb/LLM/MCP.html) over HTTP so remote GitHub MCP tools run through the same `LLM::Context` tool path as local tools. <br> See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
+
+```ruby
+require "llm"
+require "net/http/persistent"
+
+llm = LLM.openai(key: ENV["KEY"])
+mcp = LLM::MCP.http(
+  url: "https://api.githubcopilot.com/mcp/",
+  headers: {"Authorization" => "Bearer #{ENV.fetch("GITHUB_PAT")}"}
+).persistent
+
+begin
+  mcp.start
+  ctx = LLM::Context.new(llm, stream: $stdout, tools: mcp.tools)
+  ctx.talk("Pull information about my GitHub account.")
+  ctx.talk(ctx.call(:functions)) while ctx.functions.any?
+ensure
+  mcp.stop
+end
+```
+
+## Screencast
+
+This screencast was built on an older version of llm.rb, but it still shows
+how capable the runtime can be in a real application:
+
+[![Watch the llm.rb screencast](https://img.youtube.com/vi/Jb7LNUYlCf4/maxresdefault.jpg)](https://www.youtube.com/watch?v=x1K4wMeO_QA)
+
 ## Resources
 
 - [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) is the

data/lib/llm/active_record/acts_as_agent.rb

@@ -150,8 +150,8 @@ module LLM::ActiveRecord
       # @return [Hash]
       def resolve_options(option)
         case option
-        when Proc, Hash then resolve_option(option)
-        else EMPTY_HASH.dup
+        when Proc, Symbol, Hash then resolve_option(option)
+        else ActsAsAgent::EMPTY_HASH.dup
         end
       end
 

data/lib/llm/active_record/acts_as_llm.rb

@@ -270,8 +270,8 @@ module LLM::ActiveRecord
       # @return [Hash]
       def resolve_options(option)
         case option
-        when Proc, Hash then resolve_option(option)
-        else EMPTY_HASH.dup
+        when Proc, Symbol, Hash then resolve_option(option)
+        else ActsAsLLM::EMPTY_HASH.dup
         end
       end
 

data/lib/llm/agent.rb

@@ -17,7 +17,8 @@ module LLM
   # * Instructions are injected only on the first request.
   # * An agent automatically executes tool loops (unlike {LLM::Context LLM::Context}).
   # * Tool loop execution can be configured with `concurrency :call`,
-  #   `:thread`, `:task`, `:fiber`, or `:ractor`.
+  #   `:thread`, `:task`, `:fiber`, `:ractor`, or a list of queued task
+  #   types such as `[:thread, :ractor]`.
   #
   # @example
   #   class SystemAdmin < LLM::Agent
@@ -83,7 +84,7 @@ module LLM
     ##
     # Set or get the tool execution concurrency.
     #
-    # @param [Symbol, nil] concurrency
+    # @param [Symbol, Array<Symbol>, nil] concurrency
     #  Controls how pending tool loops are executed:
     #  - `:call`: sequential calls
     #  - `:thread`: concurrent threads
@@ -91,7 +92,10 @@ module LLM
     #  - `:fiber`: concurrent raw fibers
     #  - `:ractor`: concurrent Ruby ractors for class-based tools; MCP tools are not supported,
     #    and this mode is especially useful for CPU-bound tool work
-    # @return [Symbol, nil]
+    #  - `[:thread, :ractor]`: the possible concurrency strategies to wait on, in the
+    #    given order. This is useful for mixed tool sets or when work may have been
+    #    spawned with more than one concurrency strategy.
+    # @return [Symbol, Array<Symbol>, nil]
     def self.concurrency(concurrency = nil)
       return @concurrency if concurrency.nil?
       @concurrency = concurrency
@@ -107,7 +111,7 @@ module LLM
     # @option params [String] :model Defaults to the provider's default model
     # @option params [Array<LLM::Function>, nil] :tools Defaults to nil
     # @option params [#to_json, nil] :schema Defaults to nil
-    # @option params [Symbol, nil] :concurrency Defaults to the agent class concurrency
+    # @option params [Symbol, Array<Symbol>, nil] :concurrency Defaults to the agent class concurrency
     def initialize(llm, params = {})
       defaults = {model: self.class.model, tools: self.class.tools, schema: self.class.schema}.compact
       @concurrency = params.delete(:concurrency) || self.class.concurrency
@@ -270,7 +274,7 @@ module LLM
 
     ##
     # Returns the configured tool execution concurrency.
-    # @return [Symbol, nil]
+    # @return [Symbol, Array<Symbol>, nil]
     def concurrency
       @concurrency
     end
@@ -348,8 +352,8 @@ module LLM
     def call_functions
       case concurrency || :call
       when :call then call(:functions)
-      when :thread, :task, :fiber, :ractor then wait(concurrency)
-      else raise ArgumentError, "Unknown concurrency: #{concurrency.inspect}. Expected :call, :thread, :task, :fiber, or :ractor"
+      when :thread, :task, :fiber, :ractor, Array then wait(concurrency)
+      else raise ArgumentError, "Unknown concurrency: #{concurrency.inspect}. Expected :call, :thread, :task, :fiber, :ractor, or an array of queued task types"
       end
     end
   end

data/lib/llm/context.rb

@@ -69,7 +69,6 @@ module LLM
       @mode = params.delete(:mode) || :completions
       @params = {model: llm.default_model, schema: nil}.compact.merge!(params)
       @messages = LLM::Buffer.new(llm)
-      @owner = Fiber.current
     end
 
     ##
@@ -86,6 +85,7 @@ module LLM
     #   puts res.messages[0].content
     def talk(prompt, params = {})
       return respond(prompt, params) if mode == :responses
+      @owner = Fiber.current
       params = params.merge(messages: @messages.to_a)
       params = @params.merge(params)
       bind!(params[:stream], params[:model])
@@ -112,6 +112,7 @@ module LLM
     #   res = ctx.respond("What is the capital of France?")
     #   puts res.output_text
     def respond(prompt, params = {})
+      @owner = Fiber.current
       params = @params.merge(params)
       bind!(params[:stream], params[:model])
       res_id = params[:store] == false ? nil : @messages.find(&:assistant?)&.response&.response_id
@@ -182,8 +183,10 @@ module LLM
     # exposes a non-empty queue. Otherwise it falls back to waiting on
     # the context's pending functions directly.
     #
-    # @param [Symbol] strategy
-    #  The concurrency strategy to use
+    # @param [Symbol, Array<Symbol>] strategy
+    #  The concurrency strategy to use, or the possible concurrency strategies to
+    #  wait on. For example, `[:thread, :ractor]` waits for any queued thread or
+    #  ractor work, in that order.
     # @return [Array<LLM::Function::Return>]
     def wait(strategy)
       stream = @params[:stream]

data/lib/llm/providers/google/response_adapter/completion.rb

@@ -9,6 +9,12 @@ module LLM::Google::ResponseAdapter
     end
     alias_method :choices, :messages
 
+    ##
+    # (see LLM::Contract::Completion#id)
+    def id
+      body["responseId"]
+    end
+
     ##
     # (see LLM::Contract::Completion#input_tokens)
     def input_tokens

data/lib/llm/sequel/plugin.rb

@@ -79,7 +79,7 @@ module LLM::Sequel
     ##
     # @return [Hash]
     def llm_plugin_options
-      @llm_plugin_options || DEFAULTS
+      @llm_plugin_options || Plugin::DEFAULTS
     end
   end
 
@@ -287,8 +287,8 @@ module LLM::Sequel
     # @return [Hash]
     def resolve_options(option)
       case option
-      when Proc, Hash then resolve_option(option)
-      else EMPTY_HASH.dup
+      when Proc, Symbol, Hash then resolve_option(option)
+      else Plugin::EMPTY_HASH.dup
       end
     end
 

data/lib/llm/stream/queue.rb

@@ -33,27 +33,57 @@ class LLM::Stream
 
     ##
     # Waits for queued work to finish and returns function results.
-    # @param [Symbol] strategy
-    #   Controls concurrency strategy:
+    # @param [Symbol, Array<Symbol>] strategy
+    #   Controls concurrency strategy, or lists the possible concurrency strategies
+    #   to wait on:
     #   - `:thread`: Use threads
     #   - `:task`: Use async tasks (requires async gem)
     #   - `:fiber`: Use raw fibers
     #   - `:ractor`: Use Ruby ractors (class-based tools only; MCP tools are not supported)
+    #   - `[:thread, :ractor]`: Wait for any queued thread or ractor work, in the
+    #     given order. This is useful when different tools were spawned with
+    #     different concurrency strategies.
     # @return [Array<LLM::Function::Return>]
     def wait(strategy)
       returns, tasks = @items.shift(@items.length).partition { LLM::Function::Return === _1 }
-      results = case strategy
+      results = wait_tasks(tasks, strategy)
+      returns.concat fire_hooks(tasks, results)
+    end
+    alias_method :value, :wait
+
+    private
+
+    def wait_tasks(tasks, strategy)
+      strategies = Array(strategy)
+      return wait_group(tasks, strategies.first) unless strategies.length > 1
+      grouped = strategies.to_h { [_1, []] }
+      tasks.each do |task|
+        grouped[task_strategy(task)] << task
+      end
+      strategies.flat_map do |name|
+        selected = grouped.fetch(name)
+        selected.empty? ? [] : wait_group(selected, name)
+      end
+    end
+
+    def wait_group(tasks, strategy)
+      case strategy
       when :thread then LLM::Function::ThreadGroup.new(tasks).wait
       when :task then LLM::Function::TaskGroup.new(tasks).wait
       when :fiber then LLM::Function::FiberGroup.new(tasks).wait
       when :ractor then LLM::Function::Ractor::Group.new(tasks).wait
       else raise ArgumentError, "Unknown strategy: #{strategy.inspect}. Expected :thread, :task, :fiber, or :ractor"
       end
-      returns.concat fire_hooks(tasks, results)
     end
-    alias_method :value, :wait
 
-    private
+    def task_strategy(task)
+      case task.task
+      when Thread then :thread
+      when Fiber then :fiber
+      when LLM::Function::Ractor::Task then :ractor
+      else :task
+      end
+    end
 
     def fire_hooks(tasks, results)
       results.each_with_index do |result, idx|

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "4.20.0"
+  VERSION = "4.20.2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 4.20.0
+  version: 4.20.2
 platform: ruby
 authors:
 - Antar Azri