patient_llm 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 7873a0b7daf57787415d4484495a17f130cbf3a14137613799651aa0007f2d63
+  data.tar.gz: 728755787c1a52c805ebd70eaf163fae702d55b9525325b101b9dae571ad1652
+SHA512:
+  metadata.gz: c101043564f214413f1a50b3536d56eff6f611e227013cc26507fd7b8465f1554bb6922de98f736f0712bda809afa2654bf1c366740c8b5ca3e673074e5738d7
+  data.tar.gz: 80d9a476d5fbe99a96946558a1d847c7faad08250584f9d0b5b452d6faeb906066d64b9c5e822139aed84f22d41c0059e035d59cefc9dc75979b8c4bffb7fd20

data/CHANGELOG.md

@@ -0,0 +1,14 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## 0.1.0
+
+### Added
+
+- Initial implementation: async OpenAI Chat Completions calls via `patient_http`.
+- Tool calling with automatic execution loop and `halt` short-circuit via `PromptBuilder.tool_registry`.
+- Provider registry, JSON-schema structured output, reasoning effort, custom headers/params.
+- Session state serialization/restoration via `PromptBuilder::Session`.

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2026 Brian Durand
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,323 @@
+# PatientLLM
+
+[![Continuous Integration](https://github.com/bdurand/patient_llm/actions/workflows/continuous_integration.yml/badge.svg)](https://github.com/bdurand/patient_llm/actions/workflows/continuous_integration.yml)
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
+[![Gem Version](https://badge.fury.io/rb/patient_llm.svg)](https://badge.fury.io/rb/patient_llm)
+
+Integrate LLM APIs with your Ruby backend applications without blocking threads. This gem uses asynchronous HTTP requests to call LLM providers and handles the response via callbacks. It supports multiple API formats natively via [PromptBuilder](https://github.com/bdurand/prompt_builder) serializers:
+
+- **OpenAI Chat Completions** (`:chat_completion`) -- for OpenAI and compatible providers
+- **OpenAI Responses** (`:open_responses`) -- for the newer OpenAI Responses API
+- **Anthropic Messages** (`:messages`) -- for the Anthropic Claude API
+- **Bedrock Converse** (`:converse`) -- for AWS Bedrock Converse API
+- **Gemini** (`:gemini`) -- for the Google Gemini API
+
+LLM API calls can take a long time to complete. With traditional synchronous HTTP clients, these requests tie up application threads while waiting for responses. This gem solves that problem by using async HTTP via [PatientHttp](https://github.com/bdurand/patient_http), freeing up your threads to do other work while waiting for the LLM provider to respond.
+
+## Prerequisites
+
+This gem delegates actual HTTP dispatch to `patient_http`, which requires a registered request handler before any `PatientLLM.ask` call will succeed. In a normal app you get this handler by adding one of the job-system integrations:
+
+- [patient_http-sidekiq](https://github.com/bdurand/patient_http-sidekiq)
+- [patient_http-solid_queue](https://github.com/bdurand/patient_http-solid_queue)
+
+Without a handler, `PatientLLM.ask` raises `RuntimeError: No request handler registered`.
+
+## Usage
+
+### Configuration
+
+Register your LLM providers with their API base URLs and authentication headers:
+
+```ruby
+PatientLLM.configure do |config|
+  config.provider :openai,
+    url: "https://api.openai.com",
+    headers: {"Authorization" => "Bearer #{ENV["OPENAI_API_KEY"]}"}
+
+  config.provider :anthropic,
+    url: "https://api.anthropic.com",
+    headers: {"x-api-key" => ENV["ANTHROPIC_API_KEY"]},
+    serializer: :messages
+end
+```
+
+> [!NOTE]
+> Authentication headers configured on the provider are re-attached to every request at dispatch time and are persisted in the asynchronous job payload.
+>
+> You should set up encryption for you job payloads to prevent leaking credentials. See the documentation for [patient_http-sidekiq](https://github.com/bdurand/patient_http-sidekiq#sensitive-data-handling) or [patient_http-solid_queue](https://github.com/bdurand/patient_http-solid_queue#sensitive-data-handling) for details.
+
+### Creating a Callback Class
+
+Create a callback class with `on_complete` and `on_error` methods. Callbacks receive
+**keyword arguments**, and you only declare the ones you need — the dispatcher inspects your
+method signature and passes just those values (or everything if you declare `**kwargs`):
+
+```ruby
+class LLMCallback
+  def on_complete(session:, provider:, llm_response:, callback_args:, http_response:, request_id:)
+    # session       - the PromptBuilder::Session with the response already added
+    # provider      - the provider name (String)
+    # llm_response  - a PromptBuilder::Response with the assistant's response
+    # callback_args - a PatientHttp::CallbackArgs containing data you passed in the `ask` call
+    # http_response - the raw PatientHttp::Response
+    # request_id    - the original request id (stable across tool-call iterations)
+
+    # Access the response content
+    puts llm_response.text
+    puts "Tokens: #{llm_response.usage.input_tokens} in / #{llm_response.usage.output_tokens} out"
+    puts "Duration: #{http_response.duration}s"
+
+    # Save the session state for future turns (response is already in the session)
+    save_session_state(callback_args[:user_id], session.to_h)
+  end
+
+  def on_error(session:, provider:, callback_args:, error:, http_response:, request_id:)
+    # error is a PatientHttp::RequestError, ClientError (HTTP 4xx),
+    # or ServerError (HTTP 5xx). All respond to:
+    #   error.error_type  - :timeout, :connection, :ssl, :http_error, etc.
+    #   error.message     - human-readable message
+    #   error.error_class - the original exception class (for RequestError)
+    #   error.request_id
+    # http_response is the raw PatientHttp::Response for HTTP errors, or nil for
+    # transport errors (timeouts, connection failures).
+
+    log_error(error.error_type, error.message)
+  end
+end
+```
+
+#### Callback keyword parameters
+
+Each callback may declare any subset of the keywords below, in any order. Declaring
+`**kwargs` receives them all. `PatientLLM.ask` validates your callback's signatures up
+front and raises an `ArgumentError` if a method uses an unsupported name, a positional
+parameter, or omits the required keyword.
+
+| Callback              | Supported keywords                                                          | Required       |
+|-----------------------|-----------------------------------------------------------------------------|----------------|
+| `on_complete`         | `session`, `provider`, `llm_response`, `callback_args`, `http_response`, `request_id` | `llm_response` |
+| `on_tool_use` (optional) | `session`, `provider`, `llm_response`, `callback_args`, `http_response`, `request_id` | `llm_response` |
+| `on_error`            | `session`, `provider`, `callback_args`, `error`, `http_response`, `request_id`        | `error`        |
+
+For example, a callback that only cares about the response text can be as small as:
+
+```ruby
+class LLMCallback
+  def on_complete(llm_response:)
+    puts llm_response.text
+  end
+
+  def on_error(error:)
+    log_error(error.error_type, error.message)
+  end
+end
+```
+
+### Making LLM Requests
+
+Create a `PromptBuilder::Session` and call `PatientLLM.ask` to make an async request:
+
+```ruby
+session = PromptBuilder::Session.new(model: "gpt-4o")
+session.instructions = "You are a helpful assistant."
+session.user("What is the capital of France?")
+
+PatientLLM.ask(session, provider: :openai, callback: LLMCallback)
+```
+
+You can pass custom data to your callback using `callback_args`:
+
+```ruby
+PatientLLM.ask(session, provider: :openai, callback: LLMCallback, callback_args: {
+  user_id: current_user.id,
+  conversation_id: conversation.id
+})
+```
+
+The request is sent asynchronously. When the LLM responds, your callback's `on_complete` method will be called with the result.
+
+### Session Configuration Options
+
+`PromptBuilder::Session` supports various configuration:
+
+```ruby
+session = PromptBuilder::Session.new(model: "gpt-5.4")
+
+# Set system instructions
+session.instructions = "You are a helpful assistant."
+
+# Set temperature
+session.temperature = 0.7
+
+# Enable reasoning for supported models (OpenAI o1/o3 family)
+session.reasoning = {effort: "high"}
+
+# Set a JSON schema for structured output
+session.text = {
+  format: {
+    type: "json_schema",
+    json_schema: {
+      name: "response",
+      schema: {
+        type: "object",
+        properties: {
+          answer: { type: "string" },
+          confidence: { type: "number" }
+        }
+      }
+    }
+  }
+}
+
+# Set the maximum output tokens
+session.max_output_tokens = 1000
+```
+
+`PatientLLM.ask` accepts additional options:
+
+```ruby
+PatientLLM.ask(session,
+  provider: :openai,
+  callback: LLMCallback,
+  url: "http://localhost:1234",           # Override the provider's base URL
+  serializer: :messages,                   # Override the API format
+  completion_path: "/chat/completions",    # Override the endpoint path
+  headers: {"X-Custom" => "value"},        # Additional HTTP headers
+  params: {max_completion_tokens: 1000}    # Additional request parameters
+)
+```
+
+### URL composition
+
+The full request URL is built by concatenating the base URL (from the provider registry or the `url:` option) with the `completion_path`. When you don't set `completion_path`, it defaults to the path for the active serializer (`/v1/chat/completions` for `:chat_completion`, `/v1/responses` for `:open_responses`, `/v1/messages` for `:messages`, `/converse` for `:converse`, `/v1beta/models/{model}:generateContent` for `:gemini`). A `{model}` placeholder in the path is replaced with the session's model at dispatch time, which is how the Gemini default targets Google's `/v1beta/models/{model}:generateContent` endpoint. Trailing slashes on the base and leading slashes on the path are normalized, so:
+
+```
+url = "https://api.openai.com"            completion_path = "/v1/chat/completions"
+-> https://api.openai.com/v1/chat/completions
+
+url = "http://localhost:1234"             completion_path = "/v1/chat/completions"
+-> http://localhost:1234/v1/chat/completions
+```
+
+If your base URL already includes a `/v1` prefix, override the completion path to avoid duplication:
+
+```ruby
+PatientLLM.ask(session,
+  provider: :openai,
+  callback: LLMCallback,
+  url: "https://my-gateway.internal/openai/v1",
+  completion_path: "/chat/completions"
+)
+```
+
+### Tool calling
+
+Register tools on the global `PromptBuilder.tool_registry`:
+
+```ruby
+PromptBuilder.tool_registry.register(
+  "weather",
+  description: "Get the current weather for a location",
+  parameters: {
+    type: "object",
+    properties: {
+      location: {type: "string", description: "City name"}
+    },
+    required: ["location"]
+  }
+) do |args|
+  WeatherService.lookup(args["location"])
+end
+```
+
+Then add tools to the session and ask normally:
+
+```ruby
+session = PromptBuilder::Session.new(model: "gpt-4o")
+session.register_tool("weather",
+  description: "Get the current weather for a location",
+  parameters: {type: "object", properties: {location: {type: "string"}}, required: ["location"]}
+)
+session.user("What's the weather in NYC?")
+
+PatientLLM.ask(session, provider: :openai, callback: LLMCallback)
+```
+
+When the model responds with tool calls, the gem automatically:
+
+1. Appends the assistant tool-call response to the session.
+2. Invokes the matching tool handler from the registry with the LLM-provided arguments.
+3. Appends a tool-response item to the session.
+4. Re-issues the request asynchronously.
+5. Repeats until the model returns a plain text response (or a tool raises `HaltError`). Your `on_complete` callback only fires for the final text response.
+
+If you define an optional `on_tool_use` method on your callback, it is invoked once per tool-execution round (after the tools run, before the next request is issued) so you can observe intermediate progress.
+
+The loop is capped at `PatientLLM::Callback::MAX_TOOL_ITERATIONS` (10) iterations per conversation to prevent runaway calls. When the cap is exceeded, your `on_error` callback is invoked with a `PatientHttp::RequestError` whose `error_type` is `:max_tool_iterations` and whose `error_class` is `PatientLLM::MaxToolIterationsError`, so you can handle it alongside transport and HTTP errors.
+
+> [!NOTE]
+> Tool handlers execute synchronously inside the callback worker (e.g. a Sidekiq job). Keep handlers fast to avoid blocking the worker pool. If a tool needs to do slow work (external API calls, heavy queries), consider offloading that work and using `HaltError` to stop the auto-loop.
+
+#### Halting the loop
+
+Raise `PatientLLM::HaltError` from a tool handler to stop the auto-loop and surface custom content as the final assistant message:
+
+```ruby
+PromptBuilder.tool_registry.register("auth", description: "Authenticate", parameters: {...}) do |args|
+  unless AuthService.valid?(args["token"])
+    raise PatientLLM::HaltError.new(content: "Authentication failed.")
+  end
+  AuthService.session_info(args["token"])
+end
+```
+
+### Serializing Conversations
+
+Sessions can be serialized to JSON for storage and later restored:
+
+```ruby
+# Initial request
+session = PromptBuilder::Session.new(model: "gpt-4o")
+session.instructions = "You are a helpful assistant."
+session.user("Hello!")
+
+PatientLLM.ask(session, provider: :openai, callback: LLMCallback,
+  callback_args: {conversation_id: conversation.id})
+
+# In your callback, save the state (response is already in the session):
+def on_complete(session:, callback_args:, **)
+  save_to_database(callback_args[:conversation_id], session.to_h)
+end
+
+# Later, restore and continue:
+session_data = load_from_database(conversation_id)
+session = PromptBuilder::Session.from_h(session_data)
+session.user("Tell me more about that.")
+
+PatientLLM.ask(session, provider: :openai, callback: LLMCallback,
+  callback_args: {conversation_id: conversation_id})
+```
+
+## Installation
+
+This gem is not yet published to RubyGems. Add it from GitHub:
+
+```ruby
+gem "patient_llm", github: "bdurand/patient_llm"
+```
+
+Then execute:
+```bash
+$ bundle
+```
+
+## Contributing
+
+Open a pull request on [GitHub](https://github.com/bdurand/patient_llm).
+
+Please use the [standardrb](https://github.com/testdouble/standard) syntax and lint your code with `standardrb --fix` before submitting.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/VERSION

@@ -0,0 +1 @@
+0.1.0

data/lib/patient_llm/callback.rb

@@ -0,0 +1,260 @@
+# frozen_string_literal: true
+
+require "json"
+
+module PatientLLM
+  # Callback class that receives async HTTP responses from PatientHttp and
+  # dispatches to the user's callback.
+  #
+  # When the response contains tool calls and the global PromptBuilder tool
+  # registry has handlers for those tools, this class executes them
+  # automatically and re-issues the request until the model returns a final
+  # text response or a tool raises {HaltError}. Iteration is capped at
+  # {MAX_TOOL_ITERATIONS} to prevent runaway loops.
+  #
+  # The user callback receives a `PromptBuilder::Response` object. Access
+  # the response text via `response.text`, token usage via `response.usage`,
+  # and model id via `response.model`.
+  class Callback
+    # Maximum number of tool-execution rounds before the loop raises.
+    MAX_TOOL_ITERATIONS = 10
+
+    # Supported keyword parameters for each user callback method, along with the
+    # one parameter that must always be declared.
+    CALLBACK_PARAMS = {
+      on_complete: {allowed: %i[session provider llm_response callback_args http_response request_id]},
+      on_tool_use: {allowed: %i[session provider llm_response callback_args http_response request_id]},
+      on_error: {allowed: %i[session provider callback_args error http_response request_id], required: :error}
+    }.freeze
+
+    # Validate that a user callback class declares supported keyword parameters.
+    #
+    # Each defined callback method must use keyword parameters drawn from the
+    # supported set for that method and must declare the required parameter
+    # (`error` is required for on_error). A
+    # `**kwargs` splat is permitted and receives every available value.
+    #
+    # @param callback_class [Class] The user callback class
+    # @raise [ArgumentError] If a method uses positional or unsupported parameters
+    # @return [void]
+    def self.validate_callback_class!(callback_class)
+      CALLBACK_PARAMS.each do |method_name, spec|
+        next unless callback_class.method_defined?(method_name)
+
+        params = callback_class.instance_method(method_name).parameters
+        splat = params.any? { |type, _| type == :keyrest }
+        declared = []
+        params.each do |type, name|
+          case type
+          when :key, :keyreq
+            declared << name
+          when :keyrest, :block
+            next
+          else
+            raise ArgumentError, "#{callback_class}##{method_name} must use keyword parameters; found positional parameter #{name.inspect}"
+          end
+        end
+
+        allowed = Array(spec[:allowed])
+        unknown = declared - allowed
+        unless unknown.empty?
+          raise ArgumentError, "#{callback_class}##{method_name} has unsupported parameter(s): #{unknown.map(&:inspect).join(", ")}. Allowed: #{allowed.map(&:inspect).join(", ")}"
+        end
+
+        required = Array(spec[:required])
+        unless splat || required.empty? || (required & declared).any?
+          raise ArgumentError, "#{callback_class}##{method_name} must declare the #{required.map(&:inspect).join(", ")} keyword parameter(s)"
+        end
+      end
+    end
+
+    # Handle a successful LLM completion response.
+    #
+    # @param response [PatientHttp::Response] The async HTTP response
+    # @return [void]
+    def on_complete(response)
+      callback_args = response.callback_args
+      session = restore_session(callback_args)
+      provider_name = callback_args[:provider]
+      request_options = callback_args[:request_options] || {}
+      user_callback = resolve_user_callback(callback_args)
+      original_request_id = callback_args.fetch(:original_request_id, nil) || response.request_id
+
+      serializer = resolve_serializer(provider_name, request_options)
+      llm_response = PromptBuilder::Response.parse(response.json, serializer)
+
+      if should_auto_execute_tools?(llm_response)
+        continue_tool_loop(session, provider_name, llm_response, callback_args, response, user_callback, request_options, original_request_id)
+      else
+        session.add_response(llm_response)
+        invoke_user_callback(user_callback, :on_complete, session: session, provider: provider_name, llm_response: llm_response, callback_args: user_callback_args(callback_args), http_response: response, request_id: original_request_id)
+      end
+    end
+
+    # Handle an error during an LLM request.
+    #
+    # @param error [PatientHttp::Error] The error
+    # @return [void]
+    def on_error(error)
+      callback_args = error.callback_args
+      session = restore_session(callback_args)
+      provider_name = callback_args[:provider]
+      http_response = error.respond_to?(:response) ? error.response : nil
+      original_request_id = callback_args.fetch(:original_request_id, http_response&.request_id)
+      user_callback = resolve_user_callback(callback_args)
+      invoke_user_callback(user_callback, :on_error, session: session, provider: provider_name, callback_args: user_callback_args(callback_args), error: error, http_response: http_response, request_id: original_request_id)
+    end
+
+    private
+
+    def invoke_user_callback(user_callback, method_name, **values)
+      params = user_callback.method(method_name).parameters
+      kwargs =
+        if params.any? { |type, _| type == :keyrest || type == :rest }
+          values
+        else
+          names = params.map { |_, name| name }
+          values.slice(*names)
+        end
+      user_callback.public_send(method_name, **kwargs)
+    end
+
+    def restore_session(callback_args)
+      session_hash = callback_args.fetch(:session, {})
+      PromptBuilder::Session.from_h(session_hash)
+    end
+
+    def resolve_user_callback(callback_args)
+      class_name = callback_args.fetch(:callback, nil)
+      if class_name.nil? || class_name == ""
+        raise ArgumentError, "No callback registered"
+      end
+
+      callback_class = PatientHttp::ClassHelper.resolve_class_name(class_name)
+      callback_class.new
+    end
+
+    def user_callback_args(callback_args)
+      PatientHttp::CallbackArgs.new(callback_args[:custom] || {})
+    end
+
+    def resolve_serializer(provider_name, request_options)
+      if request_options["serializer"] && !request_options["serializer"].empty?
+        return request_options["serializer"].to_sym
+      end
+
+      provider_config = PatientLLM.provider(provider_name)
+      provider_config&.dig(:serializer) || :chat_completion
+    end
+
+    def should_auto_execute_tools?(llm_response)
+      return false unless llm_response.has_tool_calls?
+
+      llm_response.tool_calls.any? do |call|
+        PromptBuilder.tool_registry.handler_for(call.name)
+      end
+    end
+
+    def continue_tool_loop(session, provider_name, llm_response, callback_args, http_response, user_callback, request_options, original_request_id)
+      iteration = callback_args.fetch(:tool_iteration, 0).to_i
+      if iteration >= MAX_TOOL_ITERATIONS
+        error = max_tool_iterations_error(http_response, original_request_id)
+        invoke_user_callback(user_callback, :on_error, session: session, provider: provider_name, callback_args: user_callback_args(callback_args), error: error, http_response: http_response, request_id: original_request_id)
+        return
+      end
+
+      session.add_response(llm_response)
+
+      halt = nil
+      llm_response.tool_calls.each do |function_call|
+        result, halted = execute_tool(function_call)
+        halt = halted if halted
+
+        session.add_item(
+          PromptBuilder::Items::FunctionCallOutput.new(
+            call_id: function_call.call_id,
+            output: result
+          )
+        )
+        break if halt
+      end
+
+      if halt
+        content = halt.content
+        halt_response = PromptBuilder::Response.new(
+          model: llm_response.model,
+          status: "completed",
+          output: [
+            PromptBuilder::Items::Message.new(
+              role: "assistant",
+              content: [PromptBuilder::Content::OutputText.new(text: content || "")]
+            )
+          ],
+          usage: llm_response.usage
+        )
+        session.add_response(halt_response)
+        invoke_user_callback(user_callback, :on_complete, session: session, provider: provider_name, llm_response: halt_response, callback_args: user_callback_args(callback_args), http_response: http_response, request_id: original_request_id)
+        return
+      end
+
+      if user_callback.respond_to?(:on_tool_use)
+        invoke_user_callback(user_callback, :on_tool_use, session: session, provider: provider_name, llm_response: llm_response, callback_args: user_callback_args(callback_args), http_response: http_response, request_id: original_request_id)
+      end
+
+      # Re-ask with the updated session
+      ask_kwargs = {
+        provider: provider_name.to_sym,
+        callback: callback_args[:callback],
+        callback_args: callback_args[:custom],
+        tool_iteration: iteration + 1,
+        original_request_id: original_request_id
+      }
+
+      # Restore per-request overrides
+      ask_kwargs[:url] = request_options["url"] if request_options["url"]
+      ask_kwargs[:serializer] = request_options["serializer"].to_sym if request_options["serializer"]
+      ask_kwargs[:completion_path] = request_options["completion_path"] if request_options["completion_path"]
+      ask_kwargs[:headers] = request_options["headers"] if request_options["headers"]
+      ask_kwargs[:params] = request_options["params"] if request_options["params"]
+
+      PatientLLM.ask(session, **ask_kwargs)
+    end
+
+    def max_tool_iterations_error(http_response, request_id)
+      exception = MaxToolIterationsError.new("Tool-call loop exceeded #{MAX_TOOL_ITERATIONS} iterations")
+      PatientHttp::RequestError.new(
+        class_name: exception.class.name,
+        message: exception.message,
+        backtrace: [],
+        error_type: :max_tool_iterations,
+        duration: http_response&.duration,
+        request_id: request_id,
+        url: http_response&.url,
+        http_method: http_response&.http_method
+      )
+    end
+
+    def execute_tool(function_call)
+      name = function_call.name
+      args = function_call.parsed_arguments
+
+      result = PromptBuilder.tool_registry.invoke(name, args)
+      [format_result(result), nil]
+    rescue HaltError => e
+      [format_result(e.content), e]
+    rescue PromptBuilder::ToolNotFoundError => e
+      [e.message, nil]
+    rescue => e
+      ["Error executing tool #{function_call.name}: #{e.class}: #{e.message}", nil]
+    end
+
+    def format_result(result)
+      case result
+      when String then result
+      when nil then ""
+      else
+        JSON.generate(result)
+      end
+    end
+  end
+end

data/lib/patient_llm/configuration.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module PatientLLM
+  # Configuration for provider registry.
+  #
+  # @example
+  #   PatientLLM.configure do |config|
+  #     config.provider :openai,
+  #       url: "https://api.openai.com",
+  #       headers: {"Authorization" => "Bearer #{ENV["OPENAI_API_KEY"]}"},
+  #       serializer: :chat_completion
+  #   end
+  class Configuration
+    def initialize
+      @providers = {}
+    end
+
+    # Register a provider with a base URL and default headers.
+    #
+    # @param name [Symbol, String] Provider name
+    # @param url [String] Base URL for the provider API
+    # @param headers [Hash] Default headers for requests
+    # @param serializer [Symbol] API format (:chat_completion, :open_responses, :messages, :converse, :gemini)
+    # @param completion_path [String, nil] Override the default endpoint path
+    # @param params [Hash] Additional parameters to merge into every request payload
+    # @return [void]
+    def provider(name, url:, headers: {}, serializer: :chat_completion, completion_path: nil, params: {})
+      sym = serializer.to_sym
+      unless PatientLLM::VALID_SERIALIZERS.include?(sym)
+        raise ArgumentError, "Unknown serializer: #{sym.inspect}. Valid options: #{PatientLLM::VALID_SERIALIZERS.map(&:inspect).join(", ")}"
+      end
+
+      @providers[name.to_s] = {
+        url: url,
+        headers: headers,
+        serializer: sym,
+        completion_path: completion_path,
+        params: params
+      }
+    end
+
+    # Look up a registered provider by name.
+    #
+    # @param name [Symbol, String] Provider name
+    # @return [Hash, nil] Provider config hash
+    def lookup(name)
+      @providers[name&.to_s]
+    end
+  end
+end

data/lib/patient_llm/halt_error.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module PatientLLM
+  # Raised by a tool handler to short-circuit the auto-execution loop.
+  # The +content+ is delivered to the user callback as the final assistant
+  # message.
+  #
+  # @example
+  #   PromptBuilder.register_tool("stop", description: "Stop") do |args|
+  #     raise PatientLLM::HaltError.new(content: "Done!")
+  #   end
+  class HaltError < StandardError
+    # @return [String, nil] Content to surface as the assistant response
+    attr_reader :content
+
+    # @param content [String, nil] The content for the final assistant message
+    # @param message [String] Optional error message (defaults to "Tool halted execution")
+    def initialize(content: nil, message: "Tool halted execution")
+      @content = content
+      super(message)
+    end
+  end
+end

data/lib/patient_llm/max_tool_iterations_error.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+module PatientLLM
+  # Raised internally when the automatic tool-execution loop exceeds
+  # {Callback::MAX_TOOL_ITERATIONS}. It is wrapped in a
+  # `PatientHttp::RequestError` (with error type +:max_tool_iterations+) and
+  # delivered to the user callback's +on_error+ method rather than propagating
+  # out of the callback worker.
+  class MaxToolIterationsError < StandardError
+  end
+end

data/lib/patient_llm.rb

@@ -0,0 +1,151 @@
+# frozen_string_literal: true
+
+require "patient_http"
+require "prompt_builder"
+
+module PatientLLM
+  VERSION = File.read(File.join(__dir__, "../VERSION")).strip
+
+  autoload :Configuration, File.expand_path("patient_llm/configuration", __dir__)
+  autoload :HaltError, File.expand_path("patient_llm/halt_error", __dir__)
+  autoload :MaxToolIterationsError, File.expand_path("patient_llm/max_tool_iterations_error", __dir__)
+  autoload :Callback, File.expand_path("patient_llm/callback", __dir__)
+
+  # Default API paths per serializer format. The Gemini path embeds a
+  # `{model}` placeholder that is replaced with the session's model at
+  # dispatch time, matching Google's `/v1beta/models/{model}:generateContent`
+  # endpoint.
+  SERIALIZER_PATHS = {
+    chat_completion: "/v1/chat/completions",
+    open_responses: "/v1/responses",
+    messages: "/v1/messages",
+    converse: "/converse",
+    gemini: "/v1beta/models/{model}:generateContent"
+  }.freeze
+
+  # Required version header for the Anthropic Messages API.
+  ANTHROPIC_VERSION = "2023-06-01"
+
+  # Valid serializer format names.
+  VALID_SERIALIZERS = SERIALIZER_PATHS.keys.freeze
+
+  class << self
+    # Configure providers for LLM requests.
+    #
+    # @yield [Configuration]
+    # @return [void]
+    def configure
+      @configuration ||= Configuration.new
+      yield @configuration
+    end
+
+    # Reset configuration. Primarily useful in tests.
+    #
+    # @return [void]
+    def reset!
+      @configuration = nil
+    end
+
+    # Look up a registered provider by name.
+    #
+    # @param name [Symbol, String] Provider name
+    # @return [Hash, nil] Provider config
+    def provider(name)
+      @configuration&.lookup(name)
+    end
+
+    # Send an LLM request asynchronously using the given session and provider.
+    #
+    # @param session [PromptBuilder::Session] The prompt session containing conversation state
+    # @param provider [Symbol, String] Registered provider name
+    # @param callback [Class, String] Callback class for handling completion/error
+    # @param callback_args [Hash] Custom arguments passed through to the callback
+    # @param url [String, nil] Override the provider's base URL for this request
+    # @param serializer [Symbol, nil] Override the provider's serializer for this request
+    # @param completion_path [String, nil] Override the endpoint path for this request
+    # @param headers [Hash, nil] Additional headers merged on top of provider headers
+    # @param params [Hash, nil] Additional params merged into the request payload
+    # @return [Object] Handler-specific identifier for the enqueued request
+    def ask(session, provider:, callback:, callback_args: {}, url: nil, serializer: nil, completion_path: nil, headers: nil, params: nil, tool_iteration: 0, original_request_id: nil) # :nodoc: tool_iteration and original_request_id are internal
+      provider_config = self.provider(provider) || {}
+      provider_name = provider.to_s
+
+      if tool_iteration.zero?
+        PatientLLM::Callback.validate_callback_class!(PatientHttp::ClassHelper.resolve_class_name(callback.to_s))
+      end
+
+      resolved_url = url || provider_config[:url]
+      raise ArgumentError, "No API base URL configured. Set url: or register a provider with a url." unless resolved_url
+
+      resolved_serializer = (serializer || provider_config[:serializer] || :chat_completion).to_sym
+      validate_serializer!(resolved_serializer)
+      resolved_completion_path = completion_path || provider_config[:completion_path] || SERIALIZER_PATHS[resolved_serializer] || "/v1/chat/completions"
+      if resolved_completion_path.include?("{model}")
+        resolved_completion_path = resolved_completion_path.gsub("{model}", session.model.to_s)
+      end
+      resolved_headers = (provider_config[:headers] || {}).merge(headers || {})
+      if resolved_serializer == :messages && !resolved_headers.key?("anthropic-version")
+        resolved_headers = {"anthropic-version" => ANTHROPIC_VERSION}.merge(resolved_headers)
+      end
+      resolved_params = (provider_config[:params] || {}).merge(params || {})
+
+      payload = session.request_payload(resolved_serializer)
+      payload = deep_merge(payload, deep_stringify_keys(resolved_params)) unless resolved_params.empty?
+
+      request_url = join_url(resolved_url, resolved_completion_path)
+
+      request_options = {}
+      request_options["url"] = url if url
+      request_options["serializer"] = serializer.to_s if serializer
+      request_options["completion_path"] = completion_path if completion_path
+      request_options["headers"] = headers if headers && !headers.empty?
+      request_options["params"] = params if params && !params.empty?
+
+      PatientHttp.post(
+        request_url,
+        json: payload,
+        headers: resolved_headers,
+        raise_error_responses: true,
+        callback: PatientLLM::Callback,
+        callback_args: {
+          session: session.to_h,
+          provider: provider_name,
+          callback: callback.to_s,
+          custom: callback_args.transform_keys(&:to_s),
+          request_options: request_options,
+          tool_iteration: tool_iteration,
+          original_request_id: original_request_id
+        }
+      )
+    end
+
+    private
+
+    def validate_serializer!(serializer)
+      unless VALID_SERIALIZERS.include?(serializer)
+        raise ArgumentError, "Unknown serializer: #{serializer.inspect}. Valid options: #{VALID_SERIALIZERS.map(&:inspect).join(", ")}"
+      end
+    end
+
+    def join_url(base, path)
+      "#{base.sub(%r{/\z}, "")}/#{path.to_s.sub(%r{\A/}, "")}"
+    end
+
+    def deep_merge(hash1, hash2)
+      hash1.merge(hash2) do |_key, old_val, new_val|
+        if old_val.is_a?(Hash) && new_val.is_a?(Hash)
+          deep_merge(old_val, new_val)
+        else
+          new_val
+        end
+      end
+    end
+
+    def deep_stringify_keys(hash)
+      return {} if hash.nil?
+      hash.each_with_object({}) do |(k, v), acc|
+        acc[k.to_s] = v.is_a?(Hash) ? deep_stringify_keys(v) : v
+      end
+    end
+  end
+end

data/patient_llm.gemspec

@@ -0,0 +1,45 @@
+Gem::Specification.new do |spec|
+  spec.name = "patient_llm"
+  spec.version = File.read(File.expand_path("../VERSION", __FILE__)).strip
+  spec.authors = ["Brian Durand"]
+  spec.email = ["bbdurand@gmail.com"]
+
+  spec.summary = "Asynchronous LLM API requests via patient_http using prompt_builder for multi-format LLM API support."
+
+  spec.homepage = "https://github.com/bdurand/patient_llm"
+  spec.license = "MIT"
+
+  spec.metadata = {
+    "homepage_uri" => spec.homepage,
+    "source_code_uri" => spec.homepage,
+    "changelog_uri" => "#{spec.homepage}/blob/main/CHANGELOG.md"
+  }
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  ignore_files = %w[
+    .
+    AGENTS.md
+    Appraisals
+    Gemfile
+    Gemfile.lock
+    Rakefile
+    bin/
+    gemfiles/
+    spec/
+    test_app/
+  ]
+  spec.files = Dir.chdir(File.expand_path("..", __FILE__)) do
+    `git ls-files -z`.split("\x0").reject { |f| ignore_files.any? { |path| f.start_with?(path) } }
+  end
+
+  spec.require_paths = ["lib"]
+
+  spec.required_ruby_version = ">= 3.0"
+
+  spec.add_dependency "patient_http"
+  spec.add_dependency "prompt_builder"
+
+  spec.add_development_dependency "bundler"
+  spec.add_development_dependency "rspec", "~> 3.12"
+end

metadata

@@ -0,0 +1,109 @@
+--- !ruby/object:Gem::Specification
+name: patient_llm
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Brian Durand
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: patient_http
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: prompt_builder
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+email:
+- bbdurand@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- MIT-LICENSE
+- README.md
+- VERSION
+- lib/patient_llm.rb
+- lib/patient_llm/callback.rb
+- lib/patient_llm/configuration.rb
+- lib/patient_llm/halt_error.rb
+- lib/patient_llm/max_tool_iterations_error.rb
+- patient_llm.gemspec
+homepage: https://github.com/bdurand/patient_llm
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/bdurand/patient_llm
+  source_code_uri: https://github.com/bdurand/patient_llm
+  changelog_uri: https://github.com/bdurand/patient_llm/blob/main/CHANGELOG.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.3
+specification_version: 4
+summary: Asynchronous LLM API requests via patient_http using prompt_builder for multi-format
+  LLM API support.
+test_files: []