llm_gateway 0.8.1 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c70b5fdd6e77e60cbe437a5698e078b7aa88fb41dca96311dcf9a58585238476
-  data.tar.gz: cb9dd1f8aa71c33751ddd270acb849d964ed137b6b3fdd6c96e072082bc6e488
+  metadata.gz: 6110cca0d13016825c5b47cf76dae5a6d5ecfad786bc4f15b343e6ff03ef0710
+  data.tar.gz: 7f71d8817c123293e2ff3f0914a2209f8e12ab4f9522e14e020506424613b78b
 SHA512:
-  metadata.gz: d21ab094dd5f600731e12541c6ba0a672eb8d0862c732ce2f5f2a415c394024ff87565dee5820e2d1dcec5ee665715d7beacc5eb5fb5a35d518e98b4c3c6c6e1
-  data.tar.gz: 97f4cabb1416beddc261fc849d23823722f9c36afb70843fcb1ef9e748e7f7eeec7f8bb4cfa0f92b25dbc864bf2c27e8751eeb9367f5c6233a906ac79d483219
+  metadata.gz: e949f1d3f12fc6fc67202bdd3787caf8a1afaf27032ffb75e502e537e8c6d0a9fb55f4c393fe8e0306667a80f73c8b64a02e7ca688761d52a053ce519d42c1ae
+  data.tar.gz: 460024732ffc2e2668cfee803f82ea029825abfbadb7964d8e9847bbd612b20d08d8f8fa7661d7842868da33f8cc8258fb21d2be9dd2ffa12f7b304bededd2ff

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## [v0.9.0](https://github.com/Hyper-Unearthing/llm_gateway/tree/v0.9.0) (2026-07-13)
+
+[Full Changelog](https://github.com/Hyper-Unearthing/llm_gateway/compare/v0.8.1...v0.9.0)
+
+**Merged pull requests:**
+
+- Improve tool result [\#99](https://github.com/Hyper-Unearthing/llm_gateway/pull/99) ([billybonks](https://github.com/billybonks))
+- fix: harness queue draining [\#97](https://github.com/Hyper-Unearthing/llm_gateway/pull/97) ([billybonks](https://github.com/billybonks))
+- feat: pass session event to tool call [\#95](https://github.com/Hyper-Unearthing/llm_gateway/pull/95) ([billybonks](https://github.com/billybonks))
+
 ## [v0.8.1](https://github.com/Hyper-Unearthing/llm_gateway/tree/v0.8.1) (2026-06-27)
 
 [Full Changelog](https://github.com/Hyper-Unearthing/llm_gateway/compare/v0.8.0...v0.8.1)

data/README.md

@@ -30,6 +30,7 @@ Provide a unified translation interface for LLM Provider API's, While allowing d
   - [How reasoning values are mapped](#how-reasoning-values-are-mapped)
 - [Cross-Provider Handoffs](#cross-provider-handoffs)
 - [Context Serialization](#context-serialization)
+  - [Message metadata](#message-metadata)
 - [OAuth](#oauth)
   - [Get initial tokens (Codex / OpenAI OAuth)](#get-initial-tokens-codex--openai-oauth)
   - [Get initial tokens (Anthropic OAuth)](#get-initial-tokens-anthropic-oauth)
@@ -271,8 +272,8 @@ class AddTool < LlmGateway::Tool
   input_schema(type: "object")
   cache true # optional: mark the tool definition as cacheable where supported
 
-  def execute(input)
-    input[:left] + input[:right]
+  def execute(input, tool_use_id:)
+    tool_result(input[:left] + input[:right], tool_use_id: tool_use_id)
   end
 end
 
@@ -309,6 +310,8 @@ How `Prompt` works now:
 - `run(provider:, model:, reasoning:, **options)` calls `stream` and returns the final normalized `AssistantMessage` after any tool calls complete.
 - `stream(input = prompt, provider:, model:, reasoning:, **options, &block)` forwards to the provider and returns the normalized `AssistantMessage`.
 - Tools are declared as tool classes in a `TOOLS` constant. `run` automatically executes returned `tool_use` blocks, appends `tool_result` messages, and loops until no tool calls remain.
+- `LlmGateway::Tool#execute(input, tool_use_id:)` should return a `LlmGateway::Agents::Event::ToolCallResult` or a descendant; use the inherited `tool_result(content, tool_use_id: tool_use_id)` helper for the common case. The original tool call id is passed into `execute` so custom tools can construct their own result object and decide what is serialized into sessions.
+- Override the protected `execute_tool_requests(requests:, assistant_message:, session_event:)` hook to wrap tool execution with setup/teardown, result post-processing, or wrapper-message customization; call `yield requests` to let the harness/prompt execute tools normally, then return a `LlmGateway::Agents::Event::ToolResultMessage`. A `Harness` receives the persisted assistant event as `session_event:`; a `Prompt` receives `nil`. The default serializes as `{ role: "user", content: tool_results.map(&:to_h) }`.
 - `system_prompt`, `tools`, `model`, `reasoning`, `cache_key`, and `cache_retention` are forwarded as stream options.
 - `cache_retention` can also enable provider cache control for prompt-owned system/tool blocks where supported, and `Tool.cache true` marks a tool definition with `cache_control`.
 - `before_execute` callbacks receive the resolved input. `after_execute` callbacks receive the final `AssistantMessage`.
@@ -316,6 +319,7 @@ How `Prompt` works now:
 
 ## Migration guides
 
+- [0.9.0 migration guide](docs/migration_guide_0.9.0.md) — update custom tools to return `ToolCallResult`, migrate harness queue behavior, and update tool-result event consumers.
 - [0.7.0 migration guide](docs/migration_guide_0.7.0.md) — update `Prompt` subclasses for normalized `AssistantMessage` return values, automatic tool loops, `TOOLS`, and removed response hooks.
 - [0.6.0 migration guide](docs/migration_guide_0.6.0.md) — move `model_key` to per-request `model:`, update provider keys, update `Prompt` usage, and migrate stream event/usage changes.
 - [Migrating from `chat` to `stream`](docs/migration-guide.md) — use `stream` without a block when you only need the final response.
@@ -420,6 +424,8 @@ end
 Notes:
 - Tool calls are returned as `ToolCall` blocks with `type: "tool_use"`, `id`, `name`, and `input`.
 - Tool results are sent back in the transcript as `{ type: "tool_result", tool_use_id:, content: }` blocks.
+- When using `LlmGateway::Tool`, the tool call id is passed to `execute` as `tool_use_id:`. Return `ToolCallResult` directly, or return a descendant if you need custom fields/serialization behavior for session persistence.
+- `Prompt`/`Harness` subclasses can override `execute_tool_requests(requests:, assistant_message:, session_event:)` to wrap tool execution, then call `yield requests` for the default execution path and return a `ToolResultMessage` that wraps those tool result blocks.
 - For multimodal-capable models, `tool_result` content can include image blocks when supported by the provider/model.
 
 ### Server Tool Use
@@ -485,13 +491,16 @@ class WeatherTool < LlmGateway::Tool
     required: ["location"]
   )
 
-  def execute(input)
+  def execute(input, tool_use_id:)
     location = input[:location] || input["location"]
 
-    JSON.generate(
-      location: location,
-      temperature: 14,
-      condition: "Cloudy"
+    tool_result(
+      JSON.generate(
+        location: location,
+        temperature: 14,
+        condition: "Cloudy"
+      ),
+      tool_use_id: tool_use_id
     )
   end
 end
@@ -548,7 +557,7 @@ Harness behavior:
 - Harnesses pass `tools`, `system_prompt`, `model`, `reasoning`, `cache_key`, and `cache_retention` through the inherited `Prompt#stream` defaults.
 - Pass `model:` and optional `reasoning:` to `new`, or set them later with `harness.model = "..."` / `harness.reasoning = "..."`. Model and reasoning changes are recorded as session events.
 - `harness.transcript` (also aliased as `prompt`) returns the current model input: the latest compaction summary, if any, followed by active messages.
-- `harness.run` / `harness.continue` continues from the current session state without adding a new user message.
+- `harness.run` continues from the current session state without adding a new user message. `harness.continue` requires an idle agent; it marks the agent busy, drains queued `:steer` messages and then queued `:follow_up` messages, runs, and marks the agent idle when finished. `prompt_message`/`steer_message`/`follow_up_message` enqueue the new user message; if the agent is idle, they then call `continue`, so existing queued messages in that queue stay ahead of the new message.
 
 ### Agent events
 
@@ -560,8 +569,8 @@ When a block is passed to `prompt_message`, `run`, or `continue`, the harness em
 - `:message_update` with `event.stream_event` containing the normalized streaming event from the provider
 - `:message_end` with `event.message`
 - `:tool_execution_start` with `event.parameters` (`id`, `type`, `name`, `input`)
-- `:tool_execution_end` with `event.parameters` and `event.result`
-- `:turn_end` with `event.message` and `event.tool_results`
+- `:tool_execution_end` with `event.parameters` and `event.result` (`LlmGateway::Agents::Event::ToolCallResult`, or a descendant returned by the tool)
+- `:turn_end` with `event.message` and `event.tool_results` (`LlmGateway::Agents::Event::ToolCallResult` objects, or descendants returned by tools)
 - `:agent_end`
 
 ### Session managers and persistence
@@ -575,17 +584,16 @@ When a block is passed to `prompt_message`, `run`, or `continue`, the harness em
 
 Calls made while a harness is already processing are queued instead of recursively starting another run.
 
-- `prompt_message(message)` queues to the harness's default queue while busy. The default is `:next_turn`.
-- `steer_message(message)`, `follow_up_message(message)`, and `next_turn_message(message)` enqueue to their matching queue while busy. When idle, they behave like `prompt_message`.
+- `prompt_message(message)` queues to the harness's default queue while busy. The default is `:follow_up`.
+- `steer_message(message)` and `follow_up_message(message)` enqueue to their matching queue. When idle, they also start `continue` after enqueueing.
 - `:steer` messages are drained before the next model request in the current run.
-- `:follow_up` messages run after the current turn finishes and before `:next_turn` messages.
-- `:next_turn` messages run after the current agent run completes.
+- `:follow_up` messages run after the current turn finishes and after any tool-call loop has completed.
 - Queued messages drain as `:all` by default. Set `harness.queue_drain_mode = :one_at_a_time` to drain one FIFO message at a time.
-- Set `harness.default_queue_mode = :steer`, `:follow_up`, or `:next_turn` to change where busy `prompt_message` calls are queued.
+- Set `harness.default_queue_mode = :steer` or `:follow_up` to change where busy `prompt_message` calls are queued.
 
 ### Compaction
 
-Before starting a new user message and before draining queued follow-up/next-turn work, the harness checks whether compaction is needed. It compacts when either:
+Before starting a new user message and before draining queued follow-up work, the harness checks whether compaction is needed. It compacts when either:
 
 - the latest recorded message usage exceeds `LlmGateway::Agents::Harness::COMPACTION_TOKEN_THRESHOLD`, or
 - the latest assistant message is older than `LlmGateway::Agents::Harness::COMPACTION_IDLE_THRESHOLD_SECONDS`.
@@ -797,6 +805,24 @@ What to persist:
 
 Tip: if you serialize to JSON, keys become strings on parse; `llm_gateway` accepts standard hash input and normalizes internally.
 
+### Message metadata
+
+Input messages may include app-owned metadata, for example a `details` hash used for trace IDs, database IDs, UI state, or other per-message decorations:
+
+```ruby
+transcript = [
+  {
+    role: "user",
+    content: "Hello",
+    details: { trace_id: "msg-123", ui_thread_id: "thread-456" }
+  }
+]
+
+adapter.stream(transcript, model: "gpt-5.4")
+```
+
+`llm_gateway` preserves this as part of your local transcript shape, but strips `details` before sending user or assistant messages to provider APIs. This lets applications keep message metadata next to the message without leaking unsupported fields to OpenAI, Anthropic, Groq, or Codex request payloads.
+
 ## OAuth
 
 Use OAuth-capable providers (for example `openai_codex` and `anthropic_messages`) by supplying an `access_token` when building the adapter.

data/docs/migration_guide_0.9.0.md

@@ -0,0 +1,166 @@
+# Migration guide: v0.8.1 to v0.9.0
+
+This guide covers user-facing changes since the latest released gem, `v0.8.1`.
+
+Relevant PRs/changes:
+
+- #95: pass the persisted session event into tool execution
+- #97: fix harness queue draining semantics
+- current branch: tools return `LlmGateway::Agents::Event::ToolCallResult` objects instead of hashes/strings, with `is_error` support
+
+## 1. Update custom tools to accept `tool_use_id:` and return `ToolCallResult`
+
+Custom `LlmGateway::Tool` subclasses must now return a `LlmGateway::Agents::Event::ToolCallResult` object, or a subclass of it. Use the inherited `tool_result` helper for the common case.
+
+### Before
+
+```ruby
+class AddTool < LlmGateway::Tool
+  name "add"
+  description "Add two numbers"
+  input_schema(type: "object")
+
+  def execute(input)
+    input[:left] + input[:right]
+  end
+end
+```
+
+### After
+
+```ruby
+class AddTool < LlmGateway::Tool
+  name "add"
+  description "Add two numbers"
+  input_schema(type: "object")
+
+  def execute(input, tool_use_id:)
+    tool_result(input[:left] + input[:right], tool_use_id: tool_use_id)
+  end
+end
+```
+
+If a tool returns anything other than `ToolCallResult`, the prompt/harness catches the resulting `TypeError` and sends an error result to the model instead. The exception does not escape from the normal tool loop.
+
+## 2. Use `is_error:` for failed tool results when needed
+
+`ToolCallResult` now includes an `is_error` boolean and serializes it through `to_h`.
+
+```ruby
+LlmGateway::Agents::Event::ToolCallResult.new(
+  tool_use_id: tool_use_id,
+  content: "Something went wrong",
+  is_error: true
+)
+```
+
+The default is `false`.
+
+## 3. Update event consumers for tool result objects
+
+Harness events now expose tool results as `ToolCallResult` objects, not raw hashes or provider-specific `ToolResult` structs.
+
+Affected events:
+
+- `:tool_execution_end` via `event.result`
+- `:turn_end` via `event.tool_results`
+
+### Before
+
+```ruby
+case event.type
+when :tool_execution_end
+  puts event.result[:content]
+when :turn_end
+  event.tool_results.each { |result| persist(result) }
+end
+```
+
+### After
+
+```ruby
+case event.type
+when :tool_execution_end
+  puts event.result.content
+when :turn_end
+  event.tool_results.each { |result| persist(result.to_h) }
+end
+```
+
+## 4. Wrap tool execution if needed
+
+`Prompt` and `Harness` now call the protected `execute_tool_requests` hook to execute a batch of tool requests. If you already override tool execution, migrate that customization to this hook. Override it on a `Prompt` or `Harness` subclass when you need setup before tools run, post-processing after results return, or customization of the transcript message that carries tool results. Call `yield requests` to let llm_gateway execute tools normally, then return a `LlmGateway::Agents::Event::ToolResultMessage`.
+
+```ruby
+class MyHarness < LlmGateway::Agents::Harness
+  def execute_tool_requests(requests:, assistant_message:, session_event:)
+    context = build_context(requests, assistant_message)
+    results = yield requests
+    audit_results(context, results)
+
+    LlmGateway::Agents::Event::ToolResultMessage.new(
+      content: results,
+      details: { assistant_message_id: assistant_message.id }
+    )
+  end
+end
+```
+
+The default `ToolResultMessage` serializes as:
+
+```ruby
+{
+  role: "user",
+  content: tool_results.map(&:to_h)
+}
+```
+
+## 5. Review harness queue behavior
+
+Harness queue semantics changed to avoid stale queues and recursive runs.
+
+- `default_queue_mode` is now `:follow_up` instead of `:next_turn`.
+- `next_turn_message` was removed.
+- Valid `default_queue_mode` values are now `:steer` and `:follow_up`.
+- `prompt_message`, `steer_message`, and `follow_up_message` always enqueue first.
+- If the agent is idle, those methods enqueue and then call `continue`.
+- `continue` now raises `RuntimeError, "Cannot continue a busy agent"` if called while the session is busy.
+- `continue` marks the session busy, drains `:steer`, drains `:follow_up`, runs, and marks the session idle.
+
+### Before
+
+```ruby
+harness.default_queue_mode = :next_turn
+harness.next_turn_message("do this after the current run")
+```
+
+### After
+
+```ruby
+harness.default_queue_mode = :follow_up
+harness.follow_up_message("do this after the current turn")
+```
+
+If you previously relied on `next_turn` work running after the entire agent run, move that behavior into your application-level scheduler or enqueue a `follow_up` after the current operation completes.
+
+## 6. Tool execution can access the persisted session event
+
+For a `Harness`, `execute_tool_requests` receives the persisted assistant session event as `session_event:`. This is useful for subclasses/custom harnesses that need access to the stored message/event that produced the tool call. `Prompt` invokes the same hook with `session_event: nil`, because it has no session manager.
+
+Existing simple tools do not receive this event directly and do not need to use it; they only need the `execute(input, tool_use_id:)` signature and `ToolCallResult` return value described above.
+
+## 7. Message metadata is safe to keep in transcripts
+
+Input messages may now carry app-owned metadata, such as a `details` hash. The gateway preserves it locally but strips unsupported metadata before sending user/assistant messages to providers.
+
+```ruby
+adapter.stream([
+  {
+    role: "user",
+    content: "Hello",
+    details: { trace_id: "msg-123" }
+  }
+])
+```
+
+No migration is required unless you previously stripped this metadata yourself; that cleanup can now be simplified.

data/lib/llm_gateway/agents/event.rb

@@ -30,15 +30,17 @@ module LlmGateway
       )
 
       class ToolCallResult < ::BaseStruct
-        attribute :type, Types::Coercible::Symbol.enum(:tool_result)
+        attribute :type, Types::Coercible::Symbol.default(:tool_result).enum(:tool_result)
         attribute :tool_use_id, Types::String
         attribute :content, Types::Any
+        attribute :is_error, Types::Bool.default(false)
 
         def to_h
           {
             type: type.to_s,
             tool_use_id: tool_use_id,
-            content: content
+            content: content,
+            is_error: is_error
           }
         end
 
@@ -47,6 +49,32 @@ module LlmGateway
         end
       end
 
+      class ToolResultMessage < ::BaseStruct
+        attribute :role, Types::String.default("user")
+        attribute :content, Types::Array.of(ToolCallResult)
+        attribute? :details, Types::Hash.optional
+
+        def empty?
+          content.empty?
+        end
+
+        def any?
+          content.any?
+        end
+
+        def tool_results
+          content
+        end
+
+        def to_h
+          {
+            role: role,
+            content: content.map(&:to_h),
+            details: details
+          }.compact
+        end
+      end
+
       class Base < ::BaseStruct
         attribute :type, AgentEventType
 
@@ -93,7 +121,7 @@ module LlmGateway
       class TurnEnd < Base
         attribute :type, Types::Coercible::Symbol.default(:turn_end).enum(:turn_end)
         attribute :message, Types.Instance(AssistantMessage)
-        attribute :tool_results, Types::Array.of(Types.Instance(::ToolResult))
+        attribute :tool_results, Types::Array.of(ToolCallResult)
       end
 
       class AgentEnd < Base

data/lib/llm_gateway/agents/harness.rb

@@ -17,7 +17,7 @@ module LlmGateway
         super(provider: provider, model: model, reasoning: reasoning)
         @session_manager = session_manager
         sync_initial_configuration_events
-        self.default_queue_mode = :next_turn
+        self.default_queue_mode = :follow_up
         self.queue_drain_mode = :all
       end
 
@@ -27,19 +27,15 @@ module LlmGateway
       alias :prompt :transcript
 
       def prompt_message(message, &block)
-        enqueue_or_run_message(message, default_queue_mode, &block)
+        enqueue_and_continue_if_idle(message, default_queue_mode, &block)
       end
 
       def steer_message(message, &block)
-        enqueue_or_run_message(message, :steer, &block)
+        enqueue_and_continue_if_idle(message, :steer, &block)
       end
 
       def follow_up_message(message, &block)
-        enqueue_or_run_message(message, :follow_up, &block)
-      end
-
-      def next_turn_message(message, &block)
-        enqueue_or_run_message(message, :next_turn, &block)
+        enqueue_and_continue_if_idle(message, :follow_up, &block)
       end
 
       def default_queue_mode=(mode)
@@ -80,22 +76,20 @@ module LlmGateway
           emit(Event::MessageUpdate.new(stream_event: event), &block)
         end
 
-        session_manager.push_message(assistant_message.to_h)
+        persisted_message = session_manager.push_message(assistant_message.to_h)
         emit(Event::MessageEnd.new(message: assistant_message), &block)
 
-        tool_results = tool_requests(assistant_message).map do |message|
-          parameters = message.to_h
-          emit(Event::ToolExecutionStart.new(parameters: parameters), &block)
-          tool_result = find_and_execute_tool(message)
-          emit(Event::ToolExecutionEnd.new(parameters: parameters, result: tool_result), &block)
-          tool_result
+        tool_request_blocks = tool_requests(assistant_message)
+        tool_result_message = execute_tool_requests(
+          requests: tool_request_blocks,
+          assistant_message: assistant_message,
+          session_event: persisted_message
+        ) do |requests_to_execute|
+          run_tool_requests(requests_to_execute, session_event: persisted_message, &block)
         end
+        tool_results = tool_result_message.tool_results
 
-        tool_result_content = tool_results.map(&:to_h)
-        session_manager.push_message(
-          role: "user",
-          content: tool_result_content,
-        ) unless tool_result_content.empty?
+        session_manager.push_message(tool_result_message.to_h) if tool_result_message.any?
 
         turn_end_event = Event::TurnEnd.new(message: assistant_message, tool_results: tool_results)
         emit(turn_end_event, &block)
@@ -112,7 +106,19 @@ module LlmGateway
         emit(Event::AgentEnd.new(messages: []), &block)
         assistant_message
       end
-      alias :continue   :run
+
+      def continue(&block)
+        raise RuntimeError, "Cannot continue a busy agent" if session_manager.busy?
+
+        session_manager.busy!
+        begin
+          drain_queue(:steer)
+          drain_queue(:follow_up)
+          run(&block)
+        ensure
+          session_manager.idle!
+        end
+      end
 
       private
 
@@ -127,33 +133,16 @@ module LlmGateway
         end
       end
 
-      def enqueue_or_run_message(message, queue, &block)
-        if session_manager.idle?
-          drain_queue(:steer)
-          drain_queue(:next_turn)
-          drain_queue(:follow_up)
-        end
+      def enqueue_and_continue_if_idle(message, queue, &block)
         prepared_input = LlmGateway::Utils.deep_symbolize_keys(message)
-        result = session_manager.start_or_enqueue_user_message(prepared_input, queue: queue) do
-          compact_if_needed
+        if session_manager.busy?
+          session_manager.push_message_to_queue(prepared_input, queue)
+          return
         end
-        return if result == session_manager.class::MESSAGE_QUEUED
-
-        begin
 
-
-          continue(&block)
-
-          loop do
-            break unless session_manager.queued_messages?(:next_turn)
-
-            compact_if_needed
-            drain_queue(:next_turn)
-            continue(&block)
-          end
-        ensure
-          session_manager.idle!
-        end
+        compact_if_needed
+        session_manager.push_message_to_queue(prepared_input, queue)
+        continue(&block)
       end
 
       def compact_if_needed
@@ -173,6 +162,16 @@ module LlmGateway
         session_manager.drain_message_queue(queue, mode: queue_drain_mode)
       end
 
+      def run_tool_requests(requests, session_event:, &block)
+        requests.map do |tool_content_block|
+          parameters = tool_content_block.to_h
+          emit(Event::ToolExecutionStart.new(parameters: parameters), &block)
+          tool_result = find_and_execute_tool(tool_content_block, session_event: session_event)
+          emit(Event::ToolExecutionEnd.new(parameters: parameters, result: tool_result), &block)
+          tool_result
+        end
+      end
+
       def emit(event, &block)
         return unless block
 

data/lib/llm_gateway/agents/in_memory_session_manager.rb

@@ -8,7 +8,7 @@ module LlmGateway
     class InMemorySessionManager
       MESSAGE_QUEUED = :queued
       MESSAGE_STARTED = :started
-      QUEUES = [ :steer, :follow_up, :next_turn ].freeze
+      QUEUES = [ :steer, :follow_up ].freeze
       DRAIN_MODES = [ :one_at_a_time, :all ].freeze
 
       attr_reader :session_id, :session_start
@@ -27,17 +27,17 @@ module LlmGateway
         @state = :idle
       end
 
-      def drain_message_queue(queue = :next_turn, mode: :all)
+      def drain_message_queue(queue = :follow_up, mode: :all)
         messages = queued_messages(queue, mode)
         messages.each { |message| push_message(message) }
         messages
       end
 
-      def queued_messages?(queue = :next_turn)
+      def queued_messages?(queue = :follow_up)
         @message_queues[validate_queue!(queue)].any?
       end
 
-      def push_message_to_queue(message, queue = :next_turn)
+      def push_message_to_queue(message, queue = :follow_up)
         @message_queues[validate_queue!(queue)] << message
       end
 
@@ -63,7 +63,7 @@ module LlmGateway
         mode
       end
 
-      def start_or_enqueue_user_message(payload, queue: :next_turn)
+      def start_or_enqueue_user_message(payload, queue: :follow_up)
         if busy?
           push_message_to_queue(payload, queue)
           MESSAGE_QUEUED

data/lib/llm_gateway/agents/tools/bash_tool.rb

@@ -1,5 +1,6 @@
 require "securerandom"
 require "tmpdir"
+require_relative "../../tool"
 require_relative "tool_utils"
 
 class BashTool < LlmGateway::Tool
@@ -17,7 +18,7 @@ class BashTool < LlmGateway::Tool
     required: [ "command" ]
   })
 
-  def execute(input)
+  def execute(input, tool_use_id:)
     command = input[:command]
     timeout = input[:timeout]
 
@@ -25,16 +26,16 @@ class BashTool < LlmGateway::Tool
     out = format_output(result[:output], empty_text: result[:timed_out] ? "" : "(no output)")
 
     if result[:timed_out]
-      return append_status(out, "Command timed out after #{timeout} seconds")
+      return tool_result(append_status(out, "Command timed out after #{timeout} seconds"), tool_use_id: tool_use_id)
     end
 
     if result[:exit_status] && result[:exit_status] != 0
-      return append_status(out, "Command exited with code #{result[:exit_status]}")
+      return tool_result(append_status(out, "Command exited with code #{result[:exit_status]}"), tool_use_id: tool_use_id)
     end
 
-    out
+    tool_result(out, tool_use_id: tool_use_id)
   rescue StandardError => e
-    e.message
+    tool_result(e.message, tool_use_id: tool_use_id)
   end
 
   private

data/lib/llm_gateway/agents/tools/edit_tool.rb

@@ -1,4 +1,5 @@
 require "json"
+require_relative "../../tool"
 require_relative "tool_utils"
 
 class EditTool < LlmGateway::Tool
@@ -30,11 +31,11 @@ class EditTool < LlmGateway::Tool
     required: [ "path", "edits" ]
   })
 
-  def execute(input)
+  def execute(input, tool_use_id:)
     path = input[:path]
     edits = prepare_edits(input[:edits])
 
-    return "Edit tool input is invalid. edits must contain at least one replacement." if !edits.is_a?(Array) || edits.empty?
+    return tool_result("Edit tool input is invalid. edits must contain at least one replacement.", tool_use_id: tool_use_id) if !edits.is_a?(Array) || edits.empty?
 
     absolute_path = ToolUtils.resolve_to_cwd(path)
 
@@ -42,7 +43,7 @@ class EditTool < LlmGateway::Tool
       begin
         File.open(absolute_path, File::RDWR) { }
       rescue SystemCallError => e
-        return "Could not edit file: #{path}. Error code: #{e.class.name.split("::").last}."
+        return tool_result("Could not edit file: #{path}. Error code: #{e.class.name.split("::").last}.", tool_use_id: tool_use_id)
       end
 
       raw_content = File.binread(absolute_path)
@@ -59,12 +60,12 @@ class EditTool < LlmGateway::Tool
       final_bytes = bom + restored.encode("UTF-8").b
 
       File.binwrite(absolute_path, final_bytes)
-      "Successfully replaced #{edits.length} block(s) in #{path}."
+      tool_result("Successfully replaced #{edits.length} block(s) in #{path}.", tool_use_id: tool_use_id)
     end
   rescue EditError => e
-    e.message
+    tool_result(e.message, tool_use_id: tool_use_id)
   rescue StandardError => e
-    "Error editing file: #{e.message}"
+    tool_result("Error editing file: #{e.message}", tool_use_id: tool_use_id)
   end
 
   private

data/lib/llm_gateway/agents/tools/read_tool.rb

@@ -1,4 +1,5 @@
 require "base64"
+require_relative "../../tool"
 require_relative "tool_utils"
 
 class ReadTool < LlmGateway::Tool
@@ -21,24 +22,24 @@ class ReadTool < LlmGateway::Tool
   IMAGE_TYPE_SNIFF_BYTES = 4100
   PNG_SIGNATURE = [ 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a ].freeze
 
-  def execute(input)
+  def execute(input, tool_use_id:)
     path = input[:path] || input["path"]
     offset = input[:offset] || input["offset"]
     limit = input[:limit] || input["limit"]
 
     absolute_path = ToolUtils.resolve_read_path(path)
 
-    return "File not found: #{path}" unless File.exist?(absolute_path)
-    return "Cannot read directory: #{path}" if File.directory?(absolute_path)
-    return "File is not readable: #{path}" unless File.readable?(absolute_path)
+    return tool_result("File not found: #{path}", tool_use_id: tool_use_id) unless File.exist?(absolute_path)
+    return tool_result("Cannot read directory: #{path}", tool_use_id: tool_use_id) if File.directory?(absolute_path)
+    return tool_result("File is not readable: #{path}", tool_use_id: tool_use_id) unless File.readable?(absolute_path)
 
     mime_type = detect_supported_image_mime_type_from_file(absolute_path)
     if mime_type
       data = Base64.strict_encode64(File.binread(absolute_path))
-      return [
+      return tool_result([
         { type: "text", text: "Read image file [#{mime_type}]" },
         { type: "image", data: data, media_type: mime_type }
-      ]
+      ], tool_use_id: tool_use_id)
     end
 
     content = File.read(absolute_path, mode: "r:bom|utf-8")
@@ -46,7 +47,7 @@ class ReadTool < LlmGateway::Tool
     total_file_lines = all_lines.length
 
     start_line = [ 0, (offset || 1).to_i - 1 ].max
-    return "Offset #{offset} is beyond end of file (#{all_lines.length} lines total)" if start_line >= all_lines.length
+    return tool_result("Offset #{offset} is beyond end of file (#{all_lines.length} lines total)", tool_use_id: tool_use_id) if start_line >= all_lines.length
 
     selected_content = if limit
       end_line = [ start_line + limit.to_i, all_lines.length ].min
@@ -60,7 +61,7 @@ class ReadTool < LlmGateway::Tool
 
     if truncation[:first_line_exceeds_limit]
       first_line_size = ToolUtils.format_size(all_lines[start_line].to_s.bytesize)
-      return "[Line #{start_display} is #{first_line_size}, exceeds #{ToolUtils.format_size(ToolUtils::DEFAULT_MAX_BYTES)} limit. Use bash: sed -n '#{start_display}p' #{path} | head -c #{ToolUtils::DEFAULT_MAX_BYTES}]"
+      return tool_result("[Line #{start_display} is #{first_line_size}, exceeds #{ToolUtils.format_size(ToolUtils::DEFAULT_MAX_BYTES)} limit. Use bash: sed -n '#{start_display}p' #{path} | head -c #{ToolUtils::DEFAULT_MAX_BYTES}]", tool_use_id: tool_use_id)
     end
 
     output = truncation[:content]
@@ -80,9 +81,9 @@ class ReadTool < LlmGateway::Tool
       output = "#{output}\n\n[#{remaining} more lines in file. Use offset=#{next_offset} to continue.]"
     end
 
-    output
+    tool_result(output, tool_use_id: tool_use_id)
   rescue StandardError => e
-    "Error reading file: #{e.message}"
+    tool_result("Error reading file: #{e.message}", tool_use_id: tool_use_id)
   end
 
   private

data/lib/llm_gateway/agents/tools/write_tool.rb

@@ -1,4 +1,5 @@
 require "fileutils"
+require_relative "../../tool"
 require_relative "tool_utils"
 
 class WriteTool < LlmGateway::Tool
@@ -16,7 +17,7 @@ class WriteTool < LlmGateway::Tool
     required: [ "path", "content" ]
   })
 
-  def execute(input)
+  def execute(input, tool_use_id:)
     path = input[:path] || input["path"]
     content = input[:content] || input["content"]
 
@@ -27,8 +28,8 @@ class WriteTool < LlmGateway::Tool
       File.write(absolute_path, content)
     end
 
-    "Successfully wrote #{content.bytesize} bytes to #{path}"
+    tool_result("Successfully wrote #{content.bytesize} bytes to #{path}", tool_use_id: tool_use_id)
   rescue StandardError => e
-    e.message
+    tool_result(e.message, tool_use_id: tool_use_id)
   end
 end

data/lib/llm_gateway/prompt.rb

@@ -1,5 +1,8 @@
 # frozen_string_literal: true
 
+require_relative "tool"
+require_relative "agents/event"
+
 module LlmGateway
   class Prompt
     class_attribute :provider, :model, :reasoning
@@ -61,31 +64,47 @@ module LlmGateway
       nil
     end
 
+    protected
+
+    def execute_tool_requests(requests:, assistant_message: nil, session_event: nil)
+      Agents::Event::ToolResultMessage.new(content: yield(requests))
+    end
+
     private
 
-    def find_and_execute_tool(tool_request)
-      tool_name = tool_request.name
-      tool_input = tool_request.input
+    def find_and_execute_tool(tool_content_block, tool_call_id: nil, **kwargs)
+      tool_call_id ||= tool_content_block.id
+      tool_name = tool_content_block.name
+      tool_input = tool_content_block.input
       tool_class = self.class.find_tool(tool_name)
 
-      result = begin
+      begin
         if tool_class
-          execute_tool(tool_class, tool_input)
-        else
-          "Unknown tool: #{tool_name}"
+          result = execute_tool(tool_class, tool_input, tool_call_id: tool_call_id, **kwargs)
+          return result if result.is_a?(Agents::Event::ToolCallResult)
+
+          raise TypeError, "Tool #{tool_name} returned #{result.class}; expected LlmGateway::Agents::Event::ToolCallResult"
         end
+
+        build_tool_call_result(tool_call_id, "Unknown tool: #{tool_name}")
       rescue StandardError => e
-        "Error executing tool: #{e.message}"
+        build_tool_call_result(tool_call_id, "Error executing tool: #{e.message}")
       end
-      ToolResult.new(
-        type: "tool_result",
-        tool_use_id: tool_request.id,
-        content: result,
+    end
+
+    def execute_tool(tool_class, tool_input, tool_call_id:, **_kwargs)
+      tool_class.new.execute(tool_input, tool_use_id: tool_call_id)
+    end
+
+    def build_tool_call_result(tool_use_id, content)
+      Agents::Event::ToolCallResult.new(
+        tool_use_id: tool_use_id,
+        content: content
       )
     end
 
-    def execute_tool(tool_class, tool_input)
-      tool_class.new.execute(tool_input)
+    def run_tool_requests(requests, **kwargs)
+      requests.map { |request| find_and_execute_tool(request, **kwargs) }
     end
 
     def run_tool_loop(input, provider: nil, model: nil, reasoning: nil, **options, &block)
@@ -108,10 +127,15 @@ module LlmGateway
     def prompt_with_tool_results(input, response, requests)
       messages = input.is_a?(Array) ? input.dup : [ { role: "user", content: input } ]
       messages << response.to_h
-      messages << {
-        role: "user",
-        content: requests.map { |request| find_and_execute_tool(request).to_h }
-      }
+
+      tool_result_message = execute_tool_requests(
+        requests: requests,
+        assistant_message: response,
+        session_event: nil
+      ) do |requests_to_execute|
+        run_tool_requests(requests_to_execute)
+      end
+      messages << tool_result_message.to_h if tool_result_message.any?
       messages
     end
 

data/lib/llm_gateway/tool.rb

@@ -1,46 +1,55 @@
 # frozen_string_literal: true
 
+require_relative "agents/event"
+
 module LlmGateway
   class Tool
-      def initialize(*args)
-        # Empty constructor to allow subclasses to call super
-      end
-
-      def self.name(value = nil)
-        @name = value if value
-        @name
-      end
-
-      def self.description(value = nil)
-        @description = value if value
-        @description
-      end
-
-      def self.input_schema(value = nil)
-        @input_schema = value if value
-        @input_schema
-      end
-
-      def self.cache(value = nil)
-        @cache = value if value
-        @cache
-      end
-
-      def self.definition
-        {
-          name: @name,
-          description: @description,
-          input_schema: @input_schema,
-          cache_control: @cache ? { type: "ephemeral" } : nil
-        }.compact
-      end
-
-      def self.tool_name
-        definition[:name]
-      end
-
-      def execute(input)
-        raise NotImplementedError, "Subclasses must implement execute"
-      end
+    def initialize(*_args, **_kwargs)
+      # Empty constructor to allow subclasses to call super.
+    end
+
+    def self.name(value = nil)
+      @name = value if value
+      @name
+    end
+
+    def self.description(value = nil)
+      @description = value if value
+      @description
+    end
+
+    def self.input_schema(value = nil)
+      @input_schema = value if value
+      @input_schema
+    end
+
+    def self.cache(value = nil)
+      @cache = value if value
+      @cache
+    end
+
+    def self.definition
+      {
+        name: @name,
+        description: @description,
+        input_schema: @input_schema,
+        cache_control: @cache ? { type: "ephemeral" } : nil
+      }.compact
+    end
+
+    def self.tool_name
+      definition[:name]
+    end
+
+    def tool_result(content, tool_use_id:)
+      LlmGateway::Agents::Event::ToolCallResult.new(
+        tool_use_id: tool_use_id,
+        content: content
+      )
+    end
+
+    def execute(input, tool_use_id:)
+      raise NotImplementedError, "Subclasses must implement execute"
+    end
   end
 end

data/lib/llm_gateway/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LlmGateway
-  VERSION = "0.8.1"
+  VERSION = "0.9.0"
 end

data/lib/llm_gateway.rb

@@ -5,8 +5,8 @@ require_relative "llm_gateway/version"
 require_relative "llm_gateway/errors"
 require_relative "llm_gateway/base_client"
 require_relative "llm_gateway/client"
-require_relative "llm_gateway/prompt"
 require_relative "llm_gateway/tool"
+require_relative "llm_gateway/prompt"
 require_relative "llm_gateway/agents/event"
 require_relative "llm_gateway/agents/in_memory_session_manager"
 require_relative "llm_gateway/agents/file_session_manager"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: llm_gateway
 version: !ruby/object:Gem::Version
-  version: 0.8.1
+  version: 0.9.0
 platform: ruby
 authors:
 - billybonks
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-06-27 00:00:00.000000000 Z
+date: 2026-07-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: dry-struct
@@ -44,6 +44,7 @@ files:
 - docs/migration-guide.md
 - docs/migration_guide_0.6.0.md
 - docs/migration_guide_0.7.0.md
+- docs/migration_guide_0.9.0.md
 - lib/llm_gateway.rb
 - lib/llm_gateway/adapters/adapter.rb
 - lib/llm_gateway/adapters/anthropic/acts_like_messages.rb