llm.rb 4.13.0 → 4.15.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7847fee7ea1e63553ad5323750fc2e5ac1b4a9082c2f4c5aba71f4587440ea75
-  data.tar.gz: e63bdae085b2f0f606cbdb4633a7eff93fd6e2428fcb85ff5fe94fc78851bf5d
+  metadata.gz: 40217f9b44b00028739994a8f6f6a278b366d7fa7f4799b86afd2b793f367084
+  data.tar.gz: cf7e6d7935cf6ab8479ac864e09ea7a4403a97345f98e30ae03de9ae941c97a0
 SHA512:
-  metadata.gz: b1c8d8600b3214da5613d152677d13fde796b42e6a29cf8af035e4ad5f28b7cea0466a375b9b444a748e9e063d2e6ad6720b653609cb2b7038e8040cd2b44e39
-  data.tar.gz: c76882f9cd5416312e26f4e25493403df8f9f8c61ee14cba5096383b449bd7a4ce8b9d70834d12176648c3d9206f0f555a1eec4b22bdb6426d88c0c36c8ed592
+  metadata.gz: 3850bb93244032d3ee721c1a7bc85efd0e86f605a421960abfbab56b89b9b4c36d97efb68970759561f08f165069e4c2e213132c0bb485b3366c57bebb71e3ad
+  data.tar.gz: 2856c27c38e6d6d8d659d8a21c13b4ea514fdc0a0bd024935b2ea2f20d10eac96f324489f1c06a08c9bad6be5f97756eedc38b9f301e7a78533a8422f02cd329

data/CHANGELOG.md

@@ -2,8 +2,115 @@
 
 ## Unreleased
 
+Changes since `v4.15.0`.
+
+## v4.15.0
+
+Changes since `v4.14.0`.
+
+### Change
+
+* **Reduce OpenAI stream parser merge overhead** <br>
+  Special-case the most common single-field deltas, streamline
+  incremental tool-call merging, and avoid repeated JSON parse attempts
+  until streamed tool arguments look complete.
+
+* **Cache streaming callback capabilities in parsers** <br>
+  Cache callback support checks once at parser initialization time in
+  the OpenAI, OpenAI Responses, Anthropic, Google, and Ollama stream
+  parsers instead of repeating `respond_to?` checks on hot streaming
+  paths.
+
+* **Reduce OpenAI Responses parser lookup overhead** <br>
+  Special-case the hot Responses API event paths and cache the current
+  output item and content part so streamed output text deltas do less
+  repeated nested lookup work.
+
+* **Add a Sequel context persistence plugin** <br>
+  Add `plugin :llm` for Sequel models so apps can persist
+  `LLM::Context` state with default columns and pass provider setup
+  through `provider:` when needed. The plugin now also supports
+  `format: :string`, `:json`, or `:jsonb` for text and native JSON
+  storage when Sequel JSON typecasting is enabled.
+
+* **Improve streaming parser performance** <br>
+  In the local replay-based `stream_parser` benchmark versus
+  `v4.14.0` (median of 20 samples, 5000 iterations), plain Ruby is a
+  small overall win: the generic eventstream path is about 0.4%
+  faster, the OpenAI stream parser is about 0.5% faster, and the
+  OpenAI Responses parser is about 1.6% faster, with unchanged
+  allocations. Under YJIT on the same benchmark, the generic
+  eventstream path is about 0.9% faster and the OpenAI stream parser
+  is about 0.4% faster, while the OpenAI Responses parser is about
+  0.7% slower, also with unchanged allocations.
+
+  Compared to `v4.13.0`, the larger `v4.14.0` streaming gains still
+  hold. The generic eventstream path remains dramatically faster than
+  `v4.13.0`, the OpenAI stream parser remains modestly faster, and the
+  OpenAI Responses parser is roughly flat to slightly better depending
+  on runtime. In other words, current keeps the large eventstream win
+  from `v4.14.0`, adds only small incremental changes beyond that, and
+  does not turn the post-`v4.14.0` parser work into another large
+  benchmark jump.
+
+## v4.14.0
+
 Changes since `v4.13.0`.
 
+This release adds request interruption for contexts, reworks provider
+HTTP internals for lower-overhead streaming, and fixes MCP clients so
+parallel tool calls can safely share one connection.
+
+### Add
+
+* **Add request interruption support** <br>
+  Add `LLM::Context#interrupt!`, `LLM::Context#cancel!`, and
+  `LLM::Interrupt` for interrupting in-flight provider requests,
+  inspired by Go's context cancellation.
+
+### Change
+
+* **Rework provider HTTP transport internals** <br>
+  Rework provider HTTP around `LLM::Provider::Transport::HTTP` with
+  explicit transient and persistent transport handling.
+
+* **Reduce SSE parser overhead** <br>
+  Dispatch raw parsed values to registered visitors instead of building
+  an `Event` object for every streamed line.
+
+* **Reduce provider streaming allocations** <br>
+  Decode streamed provider payloads directly in
+  `LLM::Provider::Transport::HTTP` before handing them to provider
+  parsers, which cuts allocation churn and gives a smaller streaming
+  speed bump.
+
+* **Reduce generic SSE parser allocations** <br>
+  Keep unread event-stream buffer data in place until compaction is
+  worthwhile, which lowers allocation churn in the remaining generic
+  SSE path.
+
+* **Improve streaming parser performance** <br>
+  In the local replay-based `stream_parser` benchmark versus `v4.13.0`
+  (median of 20 samples, 5000 iterations):
+  Plain Ruby: the generic eventstream path is about 53% faster with
+  about 32% fewer allocations, the OpenAI stream parser is about 11%
+  faster with about 4% fewer allocations, and the OpenAI Responses
+  parser is about 3% faster with unchanged allocations.
+  YJIT on the current parser benchmark harness: the current tree is
+  about 26% faster than non-YJIT on the generic eventstream path,
+  about 18% faster on the OpenAI stream parser, and about 16% faster
+  on the OpenAI Responses parser, with allocations unchanged.
+
+### Fix
+
+* **Support parallel MCP tool calls on one client** <br>
+  Route MCP responses by JSON-RPC id so concurrent tool calls can
+  share one client and transport without mismatching replies.
+
+* **Use explicit MCP non-blocking read errors** <br>
+  Use `IO::EAGAINWaitReadable` while continuing to retry on
+  `IO::WaitReadable`.
+
 ## v4.13.0
 
 Changes since `v4.12.0`.

data/README.md

@@ -4,7 +4,7 @@
 <p align="center">
   <a href="https://0x1eef.github.io/x/llm.rb?rebuild=1"><img src="https://img.shields.io/badge/docs-0x1eef.github.io-blue.svg" alt="RubyDoc"></a>
   <a href="https://opensource.org/license/0bsd"><img src="https://img.shields.io/badge/License-0BSD-orange.svg?" alt="License"></a>
-  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-4.13.0-green.svg?" alt="Version"></a>
+  <a href="https://github.com/llmrb/llm.rb/tags"><img src="https://img.shields.io/badge/version-4.15.0-green.svg?" alt="Version"></a>
 </p>
 
 ## About
@@ -17,9 +17,9 @@ state.
 It is built for engineers who want control over how these systems run. llm.rb
 stays close to Ruby, runs on the standard library by default, loads optional
 pieces only when needed, and remains easy to extend. It also works well in
-Rails or ActiveRecord applications, where a small wrapper around context
-persistence is enough to save and restore long-lived conversation state across
-requests, jobs, or retries.
+Rails or ActiveRecord applications, and it includes built-in Sequel plugin
+support, where a small wrapper around context persistence is enough to save
+and restore long-lived conversation state across requests, jobs, or retries.
 
 Most LLM libraries stop at request/response APIs. Building real systems means
 stitching together streaming, tools, state, persistence, and external
@@ -28,22 +28,14 @@ so they compose naturally instead of becoming separate subsystems.
 
 ## Architecture
 
-```
-    External MCP      Internal MCP      OpenAPI / REST
-         │                 │                  │
-         └────────── Tools / MCP Layer ───────┘
-                            │
-                      llm.rb Contexts
-                            │
-                       LLM Providers
-                 (OpenAI, Anthropic, etc.)
-                            │
-                      Your Application
-```
+<p align="center">
+  <img src="https://github.com/llmrb/llm.rb/raw/main/resources/architecture.png" alt="llm.rb architecture" width="790">
+</p>
 
 ## Core Concept
 
-`LLM::Context` is the execution boundary in llm.rb.
+[`LLM::Context`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html)
+is the execution boundary in llm.rb.
 
 It holds:
 - message history
@@ -59,54 +51,89 @@ same context object.
 
 ### Execution Model
 
-- **A system layer, not just an API wrapper**  
+- **A system layer, not just an API wrapper** <br>
   Put providers, tools, MCP servers, and application APIs behind one runtime
   model instead of stitching them together by hand.
-- **Contexts are central**  
+- **Contexts are central** <br>
   Keep history, tools, schema, usage, persistence, and execution state in one
   place instead of spreading them across your app.
-- **Contexts can be serialized**  
+- **Contexts can be serialized** <br>
   Save and restore live state for jobs, databases, retries, or long-running
   workflows.
 
 ### Runtime Behavior
 
-- **Streaming and tool execution work together**  
+- **Streaming and tool execution work together** <br>
   Start tool work while output is still streaming so you can hide latency
   instead of waiting for turns to finish.
-- **Concurrency is a first-class feature**  
+- **Tool calls have an explicit lifecycle** <br>
+  A tool call can be executed, cancelled through
+  [`LLM::Function#cancel`](https://0x1eef.github.io/x/llm.rb/LLM/Function.html#cancel-instance_method),
+  or left unresolved for manual handling, but the normal runtime contract is
+  still that a model-issued tool request is answered with a tool return.
+- **Requests can be interrupted cleanly** <br>
+  Stop in-flight provider work through the same runtime instead of treating
+  cancellation as a separate concern.
+  [`LLM::Context#cancel!`](https://0x1eef.github.io/x/llm.rb/LLM/Context.html#cancel-21-instance_method)
+  is inspired by Go's context cancellation model.
+- **Concurrency is a first-class feature** <br>
   Use threads, fibers, or async tasks without rewriting your tool layer.
-- **Advanced workloads are built in, not bolted on**  
+- **Advanced workloads are built in, not bolted on** <br>
   Streaming, concurrent tool execution, persistence, tracing, and MCP support
   all fit the same runtime model.
 
 ### Integration
 
-- **MCP is built in**  
+- **MCP is built in** <br>
   Connect to MCP servers over stdio or HTTP without bolting on a separate
   integration stack.
-- **Tools are explicit**  
+- **Sequel persistence is built in** <br>
+  Use `plugin :llm` to persist `LLM::Context` state on a Sequel model with
+  sensible default columns, then pass provider setup through
+  `provider:` when you need it. Use `format: :string` for text columns or
+  `format: :jsonb` when you want native PostgreSQL JSON storage with Sequel's
+  JSON typecasting support enabled.
+- **Persistent HTTP pooling is shared process-wide** <br>
+  When enabled, separate
+  [`LLM::Provider`](https://0x1eef.github.io/x/llm.rb/LLM/Provider.html)
+  instances with the same endpoint settings can share one persistent
+  pool, and separate HTTP
+  [`LLM::MCP`](https://0x1eef.github.io/x/llm.rb/LLM/MCP.html)
+  instances can do the same, instead of each object creating its own
+  isolated per-instance transport.
+- **Provider support is broad** <br>
+  Work with OpenAI, OpenAI-compatible endpoints, Anthropic, Google, DeepSeek,
+  Z.ai, xAI, llama.cpp, and Ollama through the same runtime.
+- **Tools are explicit** <br>
   Run local tools, provider-native tools, and MCP tools through the same path
   with fewer special cases.
-- **Providers are normalized, not flattened**  
+- **Providers are normalized, not flattened** <br>
   Share one API surface across providers without losing access to provider-
   specific capabilities where they matter.
-- **Local model metadata is included**  
+- **Responses keep a uniform shape** <br>
+  Provider calls return
+  [`LLM::Response`](https://0x1eef.github.io/x/llm.rb/LLM/Response.html)
+  objects as a common base shape, then extend them with endpoint- or
+  provider-specific behavior when needed.
+- **Low-level access is still there** <br>
+  Normalized responses still keep the raw `Net::HTTPResponse` available when
+  you need headers, status, or other HTTP details.
+- **Local model metadata is included** <br>
   Model capabilities, pricing, and limits are available locally without extra
   API calls.
 
 ### Design Philosophy
 
-- **Runs on the stdlib**  
+- **Runs on the stdlib** <br>
   Start with Ruby's standard library and add extra dependencies only when you
   need them.
-- **It is highly pluggable**  
+- **It is highly pluggable** <br>
   Add tools, swap providers, change JSON backends, plug in tracing, or layer
   internal APIs and MCP servers into the same execution path.
-- **It scales from scripts to long-lived systems**  
+- **It scales from scripts to long-lived systems** <br>
   The same primitives work for one-off scripts, background jobs, and more
   demanding application workloads with streaming, persistence, and tracing.
-- **Thread boundaries are clear**  
+- **Thread boundaries are clear** <br>
   Providers are shareable. Contexts are stateful and should stay thread-local.
 
 ## Capabilities
@@ -114,6 +141,7 @@ same context object.
 - **Chat & Contexts** — stateless and stateful interactions with persistence
 - **Context Serialization** — save and restore state across processes or time
 - **Streaming** — visible output, reasoning output, tool-call events
+- **Request Interruption** — stop in-flight provider work cleanly
 - **Tool Calling** — class-based tools and closure-based functions
 - **Run Tools While Streaming** — overlap model output with tool latency
 - **Concurrent Execution** — threads, async tasks, and fibers
@@ -138,7 +166,11 @@ same context object.
 gem install llm.rb
 ```
 
-## Example
+## Examples
+
+**REPL**
+
+See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
 
 ```ruby
 require "llm"
@@ -153,6 +185,24 @@ loop do
 end
 ```
 
+**Sequel (ORM)**
+
+See the [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) for more examples.
+
+```ruby
+require "llm"
+require "sequel"
+require "sequel/plugins/llm"
+
+class Context < Sequel::Model
+  plugin :llm, provider: -> { { key: ENV["#{provider.upcase}_SECRET"], persistent: true } }
+end
+
+ctx = Context.create(provider: "openai", model: "gpt-5.4-mini")
+ctx.talk("Remember that my favorite language is Ruby")
+puts ctx.talk("What is my favorite language?").content
+```
+
 ## Resources
 
 - [deepdive](https://0x1eef.github.io/x/llm.rb/file.deepdive.html) is the

data/lib/llm/context.rb

@@ -2,16 +2,21 @@
 
 module LLM
   ##
-  # {LLM::Context LLM::Context} represents a stateful interaction with
-  # an LLM, including conversation history, tools, execution state,
-  # and cost tracking. It evolves over time as the system runs.
+  # {LLM::Context LLM::Context} is the stateful execution boundary in
+  # llm.rb.
   #
-  # Context is the stateful environment in which an LLM operates.
-  # This is not just prompt context; it is an active, evolving
-  # execution boundary for LLM workflows.
+  # It holds the evolving runtime state for an LLM workflow:
+  # conversation history, tool calls and returns, schema and streaming
+  # configuration, accumulated usage, and request ownership for
+  # interruption.
   #
-  # A context can use the chat completions API that all providers
-  # support or the responses API that currently only OpenAI supports.
+  # This is broader than prompt context alone. A context is the object
+  # that lets one-off prompts, streaming turns, tool execution,
+  # persistence, retries, and serialized long-lived workflows all run
+  # through the same model.
+  #
+  # A context can drive the chat completions API that all providers
+  # support or the Responses API on providers that expose it.
   #
   # @example
   #   #!/usr/bin/env ruby
@@ -62,6 +67,7 @@ module LLM
       @mode = params.delete(:mode) || :completions
       @params = {model: llm.default_model, schema: nil}.compact.merge!(params)
       @messages = LLM::Buffer.new(llm)
+      @owner = Fiber.current
     end
 
     ##
@@ -184,6 +190,15 @@ module LLM
       end
     end
 
+    ##
+    # Interrupt the active request, if any.
+    # This is inspired by Go's context cancellation model.
+    # @return [nil]
+    def interrupt!
+      llm.interrupt!(@owner)
+    end
+    alias_method :cancel!, :interrupt!
+
     ##
     # Returns token usage accumulated in this context
     # @note
@@ -262,13 +277,13 @@ module LLM
     ##
     # @return [Hash]
     def to_h
-      {model:, messages:}
+      {schema_version: 1, model:, messages:}
     end
 
     ##
     # @return [String]
     def to_json(...)
-      {schema_version: 1}.merge!(to_h).to_json(...)
+      to_h.to_json(...)
     end
 
     ##

data/lib/llm/error.rb

@@ -55,6 +55,10 @@ module LLM
   # When stuck in a tool call loop
   ToolLoopError = Class.new(Error)
 
+  ##
+  # When a request is interrupted
+  Interrupt = Class.new(Error)
+
   ##
   # When a tool call cannot be mapped to a local tool
   NoSuchToolError = Class.new(Error)

data/lib/llm/eventhandler.rb

@@ -13,13 +13,15 @@ module LLM
 
     ##
     # "data:" event callback
-    # @param [LLM::EventStream::Event] event
+    # @param [LLM::EventStream::Event, String, nil] event
+    # @param [String, nil] chunk
     # @return [void]
-    def on_data(event)
-      return if event.end?
-      chunk = LLM.json.load(event.value)
-      return unless chunk
-      @parser.parse!(chunk)
+    def on_data(event, chunk = nil)
+      value = chunk ? event : event.value
+      return if value == "[DONE]"
+      payload = LLM.json.load(value)
+      return unless payload
+      @parser.parse!(payload)
     rescue *LLM.json.parser_error
     end
 
@@ -28,13 +30,15 @@ module LLM
     # is received, regardless of whether it has
     # a field name or not. Primarily for ollama,
     # which does emit Server-Sent Events (SSE).
-    # @param [LLM::EventStream::Event] event
+    # @param [LLM::EventStream::Event, String, nil] event
+    # @param [String, nil] chunk
     # @return [void]
-    def on_chunk(event)
-      return if event.end?
-      chunk = LLM.json.load(event.chunk)
-      return unless chunk
-      @parser.parse!(chunk)
+    def on_chunk(event, chunk = nil)
+      raw_chunk = chunk || event&.chunk || event
+      return if raw_chunk == "[DONE]"
+      payload = LLM.json.load(raw_chunk)
+      return unless payload
+      @parser.parse!(payload)
     rescue *LLM.json.parser_error
     end
 

data/lib/llm/eventstream/event.rb

@@ -4,8 +4,17 @@ module LLM::EventStream
   ##
   # @private
   class Event
-    FIELD_REGEXP = /[^:]+/
-    VALUE_REGEXP = /(?<=: ).+/
+    UNSET = Object.new.freeze
+
+    def self.parse(chunk)
+      newline = chunk.end_with?("\n") ? chunk.bytesize - 1 : chunk.bytesize
+      separator = chunk.index(":")
+      return [nil, nil] unless separator
+      field = chunk.byteslice(0, separator)
+      value_start = separator + (chunk.getbyte(separator + 1) == 32 ? 2 : 1)
+      value = value_start < newline ? chunk.byteslice(value_start, newline - value_start) : nil
+      [field, value]
+    end
 
     ##
     # Returns the field name
@@ -25,9 +34,10 @@ module LLM::EventStream
     ##
     # @param [String] chunk
     # @return [LLM::EventStream::Event]
-    def initialize(chunk)
-      @field = chunk[FIELD_REGEXP]
-      @value = chunk[VALUE_REGEXP]
+    def initialize(chunk, field: UNSET, value: UNSET)
+      @field, @value = self.class.parse(chunk) if field.equal?(UNSET) || value.equal?(UNSET)
+      @field = field unless field.equal?(UNSET)
+      @value = value unless value.equal?(UNSET)
       @chunk = chunk
     end
 

data/lib/llm/eventstream/parser.rb

@@ -4,6 +4,9 @@ module LLM::EventStream
   ##
   # @private
   class Parser
+    COMPACT_THRESHOLD = 4096
+    Visitor = Struct.new(:target, :on_data, :on_event, :on_id, :on_retry, :on_chunk)
+
     ##
     # @return [LLM::EventStream::Parser]
     def initialize
@@ -18,7 +21,12 @@ module LLM::EventStream
     # @param [#on_data] visitor
     # @return [void]
     def register(visitor)
-      @visitors << visitor
+      @visitors << Visitor.new(
+        visitor,
+        visitor.respond_to?(:on_data), visitor.respond_to?(:on_event),
+        visitor.respond_to?(:on_id), visitor.respond_to?(:on_retry),
+        visitor.respond_to?(:on_chunk)
+      )
     end
 
     ##
@@ -42,7 +50,8 @@ module LLM::EventStream
     # Returns the internal buffer
     # @return [String]
     def body
-      @buffer.dup
+      return @buffer.dup if @cursor.zero?
+      @buffer.byteslice(@cursor, @buffer.bytesize - @cursor) || +""
     end
 
     ##
@@ -55,34 +64,72 @@ module LLM::EventStream
 
     private
 
-    def parse!(event)
-      event = Event.new(event)
-      dispatch(event)
+    def parse_event!(chunk, field, value)
+      dispatch_visitors(field, value, chunk)
+      dispatch_callbacks(field, value, chunk)
+    end
+
+    def parse!(chunk)
+      field, value = Event.parse(chunk)
+      parse_event!(chunk, field, value)
     end
 
-    def dispatch(event)
-      @visitors.each { dispatch_visitor(_1, event) }
-      @events[event.field].each { _1.call(event) }
+    def dispatch_visitors(field, value, chunk)
+      @visitors.each { dispatch_visitor(_1, field, value, chunk) }
     end
 
-    def dispatch_visitor(visitor, event)
-      method = "on_#{event.field}"
-      if visitor.respond_to?(method)
-        visitor.public_send(method, event)
-      elsif visitor.respond_to?("on_chunk")
-        visitor.on_chunk(event)
+    def dispatch_callbacks(field, value, chunk)
+      callbacks = @events[field]
+      return if callbacks.empty?
+      event = Event.new(chunk, field:, value:)
+      callbacks.each { _1.call(event) }
+    end
+
+    def dispatch_visitor(visitor, field, value, chunk)
+      target = visitor.target
+      if field == "data"
+        if visitor.on_data
+          target.on_data(value, chunk)
+        elsif visitor.on_chunk
+          target.on_chunk(nil, chunk)
+        end
+      elsif field == "event"
+        if visitor.on_event
+          target.on_event(value, chunk)
+        elsif visitor.on_chunk
+          target.on_chunk(nil, chunk)
+        end
+      elsif field == "id"
+        if visitor.on_id
+          target.on_id(value, chunk)
+        elsif visitor.on_chunk
+          target.on_chunk(nil, chunk)
+        end
+      elsif field == "retry"
+        if visitor.on_retry
+          target.on_retry(value, chunk)
+        elsif visitor.on_chunk
+          target.on_chunk(nil, chunk)
+        end
+      elsif visitor.on_chunk
+        target.on_chunk(nil, chunk)
       end
     end
 
     def each_line
       while (newline = @buffer.index("\n", @cursor))
-        line = @buffer[@cursor..newline]
+        line = @buffer.byteslice(@cursor, newline - @cursor + 1)
         @cursor = newline + 1
         yield(line)
       end
       return if @cursor.zero?
-      @buffer = @buffer[@cursor..] || +""
-      @cursor = 0
+      if @cursor >= @buffer.bytesize
+        @buffer.clear
+        @cursor = 0
+      elsif @cursor >= COMPACT_THRESHOLD
+        @buffer = @buffer.byteslice(@cursor, @buffer.bytesize - @cursor) || +""
+        @cursor = 0
+      end
     end
   end
 end

data/lib/llm/mcp/command.rb

@@ -74,7 +74,7 @@ class LLM::MCP
     #  The IO stream to read from (:stdout, :stderr)
     # @raise [LLM::Error]
     #  When the command is not running
-    # @raise [IO::WaitReadable]
+    # @raise [IO::EAGAINWaitReadable]
     #  When no complete message is available to read
     # @return [String]
     #  The next complete line from the specified IO stream

data/lib/llm/mcp/mailbox.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+class LLM::MCP
+  ##
+  # A per-request mailbox for routing a JSON-RPC response back to the
+  # caller waiting on that request id.
+  class Mailbox
+    def initialize
+      @queue = Queue.new
+    end
+
+    def <<(message)
+      @queue << message
+      self
+    end
+
+    def pop
+      @queue.pop(true)
+    rescue ThreadError
+      nil
+    end
+  end
+end

data/lib/llm/mcp/pipe.rb

@@ -27,7 +27,7 @@ class LLM::MCP
 
     ##
     # Reads from the reader end without blocking.
-    # @raise [IO::WaitReadable]
+    # @raise [IO::EAGAINWaitReadable]
     #  When no data is available to read
     # @return [String]
     def read_nonblock(...)