agent-harness 0.40.0 → 0.42.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ee00d0e61838facc87483e6bdcac764ed26dc5bb227a15075b4e8f0daa8983d4
-  data.tar.gz: 6034f1733639711ad2b7df6bf9c26da839d45c26d1ddb6b0b4e5ba413854fc8a
+  metadata.gz: 9c2a3cf60957839163bbda4f4e202f663ee68915e49ba3853abce85b5f6f16e9
+  data.tar.gz: b5a9950414fc72fe2626abbb3ef50275e2e28267346414d04d330d3ea37c1297
 SHA512:
-  metadata.gz: 130a2897522e8b1aa73b2a9cf6094bd24b3b529696445e84356a78faa02540e0c4c2677602e306fca60e1f1a3040e48c04b3ff0a9ca28f144a24de6fc3fa5887
-  data.tar.gz: 647ab39ee27728d28a3441ccbbd84edd3521e7e434a120ac236e411849b22aa431ae7fc6dbe499ec8c525aee6159c9fdba42a5ee8ff7ec329281dd7ea3a4ad33
+  metadata.gz: b6147210553af7975d99dbf53cc52c0b37e7ba25c8424a9cfb24a290bae9172d25bcbc8a6e5b489d1b2ae363251facc756ea0eaae55a6efd3904cdb64fcda025
+  data.tar.gz: f869749041ff450642b8ba940a3f0f9f8ea2cd7a3d788b2414ae7f0115dcb5fde4dcb7f5aa725b4e1ba2216568a1412e2bf5439c47b61bf75b2d06390faebfae

data/.release-please-manifest.json

@@ -1,3 +1,3 @@
 {
-  ".": "0.40.0"
+  ".": "0.42.0"
 }

data/CHANGELOG.md

@@ -5,6 +5,20 @@
 * add runner model compatibility contract (`AgentHarness.model_compatibility`) with structured `ModelCompatibility::Result` outcomes. Codex exposes static facts for CLI-gated models (e.g. `gpt-5.5` requires Codex CLI `>= 0.116.0`), a baseline supported-model list, supported auth modes, and a `DEFAULT_COMPATIBLE_MODEL_ID` fallback so downstream orchestrators can validate tier/model assignments before scheduling agent runs ([#259](https://github.com/viamin/agent-harness/issues/259)).
 * **auth:** add provider-owned PKCE code-exchange API for Claude OAuth (`AgentHarness::Authentication.exchange_code`). Takes an authorization code plus PKCE verifier (and `redirect_uri`/`client_id`), posts an `authorization_code` grant to the Claude token endpoint, and persists the resulting access/refresh tokens in the native `claudeAiOauth` shape. Adds `exchange_code_supported?` and a `code_exchange` key to `auth_capabilities` ([#266](https://github.com/viamin/agent-harness/issues/266)).
 
+## [0.42.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.41.0...agent-harness/v0.42.0) (2026-09-25)
+
+
+### Features
+
+* Expose Attempt-Level Usage and Cost (RDR-072) ([#443](https://github.com/viamin/agent-harness/issues/443)) ([818f8ca](https://github.com/viamin/agent-harness/commit/818f8ca0f873f002fc4d26b2a864238ae41fa866))
+
+## [0.41.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.40.0...agent-harness/v0.41.0) (2026-09-25)
+
+
+### Features
+
+* Normalize API Chat Transport, Tools and Streaming (RDR-072) ([#441](https://github.com/viamin/agent-harness/issues/441)) ([761c118](https://github.com/viamin/agent-harness/commit/761c11842ccacd18d91e9412f71cb98d76afa3b2))
+
 ## [0.40.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.39.0...agent-harness/v0.40.0) (2026-09-25)
 
 

data/docs/provider-neutral-api-execution-contract.md

@@ -7,19 +7,83 @@ usage, and optional conversation-persistence work.
 
 ## Status and rollout boundary
 
-This is a design contract, not a claim that the described API is implemented.
-RDR-072's rollout guard is **docs-only now**, so this change adds no runtime
-behavior or dependency. Each capability below needs its own failing-first
-contract tests, implementation, release evidence, and downstream adoption
-evidence before a caller enables it.
+RDR-072's rollout guard was **docs-only** for the design phase. The normalized
+chat and attempt-accounting capabilities described below are now implemented
+behind `AgentHarness::Api::ChatTransport`. Other capabilities remain design contracts and each still needs
+its own failing-first contract tests, implementation, release evidence, and
+downstream adoption evidence before a caller enables it.
 
 Existing CLI and subscription behavior remains the default. Existing
 `TextTransport`, `OpenAICompatibleTransport`, `Conversation`, and `Response`
-interfaces remain available, but they do not yet satisfy this contract. A
-caller must not infer support from a gem version or issue closure.
+interfaces remain available and unchanged; they are not aliases for the
+normalized API. A caller must not infer support for another capability from a
+gem version or issue closure.
 
 Normative words such as MUST and MUST NOT describe the future public boundary.
 
+## Shipped normalized chat surface
+
+`AgentHarness::Api::ChatTransport#call` implements one normalized assistant
+response while leaving the conversation loop and all application tool
+execution with the caller:
+
+```ruby
+transport = AgentHarness::Api::ChatTransport.new
+result = transport.call(request.merge(
+  operation: :chat,
+  messages: [
+    {id: "system-1", role: :system,
+     content: [{type: :text, text: "Be concise"}]},
+    {id: "user-1", role: :user,
+     content: [{type: :text, text: "Summarize this"}]}
+  ],
+  tools: [{
+    name: "lookup",
+    description: "Looks up a record",
+    input_schema: {type: "object", properties: {id: {type: "string"}}}
+  }],
+  max_output_tokens: 1_000,
+  stream: true
+), observer: ->(event) { events << event })
+```
+
+The returned value is the normalized result hash in this document. The
+observer is either callable or responds to `on_chat_event`. It receives ordered
+events with request/attempt identity and sequence numbers. This transport never
+invokes a supplied tool; callers append completed tool results to a later
+request. A failed partial stream is terminal, so its content cannot be appended
+to a fallback response and its tool calls cannot be replayed automatically.
+
+Streamed tool-call arguments are emitted as raw, appendable JSON fragments
+correlated to one `tool_call_started` event per provider call, and cumulative
+provider token counts are emitted as deduplicated `usage_updated` events. An
+observer that raises aborts the in-flight request and surfaces as
+`AgentHarness::Api::ChatTransport::ObserverError` with the original failure as
+its `cause`; it is never classified as a provider error, never retried, and the
+failed observer is not invoked again.
+
+The verified scopes are Anthropic with `protocol: :messages`, OpenAI with
+`protocol: :responses` or `:chat_completions`, and OpenAI-compatible endpoints
+with `provider: :openai`, an explicit `endpoint`, and
+`protocol: :chat_completions`. Compatible endpoints that do not implement the
+Responses API must select `:chat_completions`; the transport never probes and
+silently switches protocols. Only `authentication_mode: :api_key` is currently
+supported.
+
+Credentials, endpoint, custom headers, timeout, and RubyLLM configuration are
+isolated with a request-local `RubyLLM::Context`. RubyLLM middleware retries
+are disabled; `retry.max_attempts` is the total physical-attempt limit owned by
+the harness. Authentication headers cannot be overridden by custom headers.
+`max_output_tokens` is forwarded without changing it.
+
+Unknown model IDs are allowed only because a complete provider and protocol
+are explicit in every candidate (`assume_model_exists: true` in the RubyLLM
+adapter). This skips registry validation; it does not assert that the endpoint
+supports the model. Provider rejection returns a classified failure. Custom
+endpoints retain the selected provider's wire protocol and authentication
+shape. Custom provider types, authentication modes, media content, and
+automatic protocol discovery are not supported by this capability.
+
 ## Ownership boundary
 
 AgentHarness owns protocol translation and one bounded provider request:
@@ -272,6 +336,23 @@ observer so durable accounting can persist an attempt even when no message is
 created. Callers deduplicate on `attempt_id`. Cost identifies its source as
 provider-reported or harness-estimated; unknown cost remains `nil`.
 
+The observer receives `attempt_completed` after every physical request, with
+the same report later returned in `result[:attempts]`. Delivery is at least
+once across process recovery: consumers MUST enforce a unique key on
+`attempt_id` and treat a repeated report as an idempotent upsert, not another
+charge. `AttemptReport.from_h` restores JSON-decoded reports. Stored cost
+includes USD component amounts, `source` (`provider_reported` or `estimated`),
+and `priced_at`; restoration never consults current prices. Missing counts and
+prices remain absent/`nil`, while a reported zero remains zero. Usage may also
+include `cache_read_tokens`, `cache_write_tokens`, and `thinking_tokens`.
+
+Attempt events contain only identifiers, provider/model, outcome, normalized
+usage/cost, timestamps, and sanitized classified errors. They do not include
+prompts, messages, request headers, endpoints, or credentials. RubyLLM API chat
+requests are in this ledger. Existing CLI providers, `TextTransport`,
+`OpenAICompatibleTransport`, token trackers, embeddings, and other
+non-`Api::ChatTransport` paths remain outside it.
+
 Only errors classified `transient` are eligible for bounded request retry:
 connection failure, timeout before a partial stream, rate limit, server error,
 service unavailable, and overload. Authentication, authorization, billing,
@@ -505,6 +586,27 @@ types stay behind the harness boundary.
 
 ## Compatibility and release evidence
 
+### Attempt-accounting capability evidence
+
+- Release: pending the first published version containing issue #435; a Git
+  branch or tag alone is not downstream adoption evidence.
+- Scope: normalized chat through `Api::ChatTransport` for Anthropic Messages
+  and OpenAI Responses/Chat Completions (including compatible endpoints), with
+  request-local API-key authentication. It is stacked on the normalized chat
+  capability from #433.
+- Verification: the API contract specs cover request-local credentials,
+  endpoint/header isolation, error classification, bounded non-nested retry,
+  fallback, cancellation, partial usage, cache usage, repeated identity,
+  observer redaction, and JSON reload with preserved pricing. The full upstream
+  suite and lint run on the repository's supported Ruby environment.
+- Migration: no Rails tables or migrations are loaded or required. Paid and
+  agent-image consumer versions remain unverified and MUST NOT adopt this
+  capability until their integration suites record the exact released gem and
+  image versions.
+- Retained paths: CLI/subscription providers, legacy HTTP transports,
+  embeddings, and token trackers remain outside this ledger and require
+  separate migration issues.
+
 AgentHarness currently supports Ruby 3.2 and later and must remain usable as a
 plain Ruby gem. RubyLLM 2.0.0 itself supports Ruby 3.1.3 and later, but adding it
 would introduce Faraday, event-stream parsing, Schematist, Marcel, and Zeitwerk

data/lib/agent_harness/api/attempt_report.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+require "time"
+
+module AgentHarness
+  module Api
+    # Serializable accounting facts for one physical provider request.
+    class AttemptReport
+      STATUSES = %i[succeeded failed cancelled partial].freeze
+      USAGE_KEYS = %i[input_tokens output_tokens cache_read_tokens cache_write_tokens thinking_tokens total_tokens].freeze
+      COST_KEYS = %i[input output cache_read cache_write thinking total].freeze
+
+      attr_reader :attributes
+
+      def self.from_h(attributes)
+        new(**symbolize(attributes))
+      end
+
+      def self.symbolize(value)
+        return value.to_h { |key, child| [key.to_sym, symbolize(child)] } if value.is_a?(Hash)
+        return value.map { |child| symbolize(child) } if value.is_a?(Array)
+
+        value
+      end
+      private_class_method :symbolize
+
+      def initialize(attempt_id:, request_id:, number:, provider:, model:, status:, started_at:, finished_at:,
+        usage: nil, cost: nil, provider_reported: false, error: nil)
+        @attributes = {
+          attempt_id: attempt_id.to_s, request_id: request_id.to_s, number: number,
+          provider: provider.to_sym, model: model&.to_s, status: status.to_sym,
+          started_at: timestamp(started_at), finished_at: timestamp(finished_at),
+          usage: normalize_usage(usage), cost: normalize_cost(cost),
+          provider_reported: provider_reported == true, error: error
+        }
+        validate!
+        deep_freeze(@attributes)
+      end
+
+      def to_h
+        deep_dup(attributes)
+      end
+
+      private
+
+      def timestamp(value)
+        value.respond_to?(:iso8601) ? value.iso8601(6) : Time.iso8601(value.to_s).utc.iso8601(6)
+      end
+
+      def normalize_usage(usage)
+        return unless usage
+
+        normalized = self.class.send(:symbolize, usage)
+        USAGE_KEYS.each_with_object({}) do |key, result|
+          result[key] = normalized[key] if normalized.key?(key)
+        end
+      end
+
+      def normalize_cost(cost)
+        return unless cost
+
+        normalized = self.class.send(:symbolize, cost)
+        amounts = COST_KEYS.to_h { |key| [key, normalized[key]] }
+        amounts.merge(currency: normalized.fetch(:currency, "USD"), source: normalized.fetch(:source).to_sym,
+          priced_at: timestamp(normalized.fetch(:priced_at)))
+      end
+
+      def validate!
+        raise ArgumentError, "attempt_id is required" if attributes[:attempt_id].empty?
+        raise ArgumentError, "request_id is required" if attributes[:request_id].empty?
+        raise ArgumentError, "number must be a positive integer" unless attributes[:number].is_a?(Integer) && attributes[:number].positive?
+        raise ArgumentError, "unknown attempt status" unless STATUSES.include?(attributes[:status])
+      end
+
+      def deep_freeze(value)
+        value.each_value { |child| deep_freeze(child) } if value.is_a?(Hash)
+        value.each { |child| deep_freeze(child) } if value.is_a?(Array)
+        value.freeze
+      end
+
+      def deep_dup(value)
+        return value.to_h { |key, child| [key, deep_dup(child)] } if value.is_a?(Hash)
+        return value.map { |child| deep_dup(child) } if value.is_a?(Array)
+
+        value
+      end
+    end
+  end
+end

data/lib/agent_harness/api/chat_transport.rb

@@ -0,0 +1,455 @@
+# frozen_string_literal: true
+
+require "securerandom"
+require "ruby_llm"
+require_relative "attempt_report"
+require_relative "ruby_llm_chat_adapter"
+
+module AgentHarness
+  module Api
+    # Executes one normalized chat response without running application tools.
+    class ChatTransport
+      RESERVED_HEADERS = {
+        anthropic: %w[x-api-key anthropic-version],
+        openai: %w[authorization]
+      }.freeze
+      PROTOCOLS = {
+        anthropic: %i[messages],
+        openai: %i[responses chat_completions]
+      }.freeze
+      DEFAULT_RETRY = {max_attempts: 1, base_delay_seconds: 0, max_delay_seconds: 0}.freeze
+
+      # Raised when the caller's observer fails. Observer bugs stay outside
+      # provider error classification, abort the in-flight request, and are
+      # never re-invoked with a synthetic terminal event.
+      class ObserverError < StandardError
+      end
+
+      def initialize(adapter: RubyLlmChatAdapter.new, id_generator: -> { SecureRandom.uuid }, sleeper: Kernel.method(:sleep))
+        @adapter = adapter
+        @id_generator = id_generator
+        @sleeper = sleeper
+      end
+
+      def call(request, observer: nil, &on_event)
+        execution = Execution.new(request, adapter: @adapter, id_generator: @id_generator,
+          sleeper: @sleeper, observer: observer || on_event)
+        execution.call
+      end
+
+      # Holds mutable state for exactly one request, keeping the transport reusable.
+      class Execution
+        def initialize(request, adapter:, id_generator:, sleeper:, observer:)
+          @request = symbolize(request)
+          @adapter = adapter
+          @id_generator = id_generator
+          @sleeper = sleeper
+          @observer = observer
+          @attempts = []
+          @sequence = 0
+          @partial_content = +""
+          @stream_tool_calls = {}
+          @tool_ids = {}
+          validate!
+        end
+
+        def call
+          return cancelled_result unless active?
+
+          candidates.each_with_index do |candidate, candidate_index|
+            result = attempt_candidate(candidate, candidate_index)
+            return result if terminal?(result)
+          end
+          failed_result(@last_error, @last_candidate)
+        end
+
+        private
+
+        attr_reader :request, :attempts
+
+        def attempt_candidate(candidate, candidate_index)
+          loop do
+            return cancelled_result(candidate) unless active?
+            return failed_result(@last_error, candidate) if attempts.length >= retry_config[:max_attempts]
+
+            result = perform_attempt(candidate)
+            return result if result[:status] == :succeeded || result[:status] == :partial
+            return cancelled_result(candidate) unless active?
+
+            @last_error = result[:error]
+            @last_candidate = candidate
+            return result if select_fallback(result, candidate_index)
+            return result unless retryable?(result)
+
+            backoff
+          end
+        end
+
+        def perform_attempt(candidate)
+          accounting = nil
+          provider_usage = -> { !streamed_usage.nil? }
+          prepared_chat = @adapter.prepare(
+            candidate: candidate,
+            messages: request[:messages],
+            tools: request[:tools] || [],
+            max_output_tokens: request[:max_output_tokens],
+            temperature: request[:temperature],
+            timeout: request[:timeout],
+            on_accounting: ->(facts) { accounting = facts },
+            provider_usage: provider_usage
+          )
+          attempt_id = @id_generator.call
+          started_at = Time.now.utc
+          emitted = false
+          streamed_usage = nil
+          emit(:response_started, attempt_id:, candidate: candidate_identity(candidate))
+
+          adapter_result = @adapter.call(
+            candidate: candidate,
+            messages: request[:messages],
+            tools: request[:tools] || [],
+            max_output_tokens: request[:max_output_tokens],
+            temperature: request[:temperature],
+            stream: request[:stream] == true,
+            timeout: request[:timeout],
+            cancellation: request[:cancellation],
+            on_accounting: ->(facts) { accounting = facts },
+            prepared_chat: prepared_chat
+          ) do |event|
+            event = normalize_stream_event(event)
+            emitted = true if output_event?(event)
+            streamed_usage = event.except(:type) if event[:type] == :usage_updated
+            accumulate(event)
+            emit(event.fetch(:type), attempt_id:, **event.except(:type))
+          end
+          raise RubyLLM::CancelledError unless active?
+
+          success(candidate, attempt_id, started_at, adapter_result, accounting)
+        rescue ObserverError
+          raise
+        rescue => error
+          return failed_result(classify(error), candidate) unless attempt_id
+
+          failure(candidate, attempt_id, started_at, classify(error), partial: emitted,
+            accounting: accounting || {
+              usage: streamed_usage, provider_reported: !streamed_usage.nil?
+            })
+        end
+
+        def success(candidate, attempt_id, started_at, adapter_result, accounting)
+          accounting ||= {usage: adapter_result[:usage], provider_reported: !adapter_result[:usage].nil?}
+          append_attempt(candidate, attempt_id, started_at, :succeeded, **accounting)
+          result = base_result(candidate).merge(
+            status: :succeeded,
+            content: adapter_result[:content] || "",
+            tool_calls: normalize_tool_calls(adapter_result[:tool_calls]),
+            finish_reason: adapter_result[:finish_reason],
+            usage: aggregate_usage,
+            error: nil
+          )
+          emit(:response_completed, attempt_id:, result: result)
+          result
+        end
+
+        def failure(candidate, attempt_id, started_at, error, partial:, accounting:)
+          status = failure_status(error, partial)
+          error = error.merge(retryable: false) if partial
+          append_attempt(candidate, attempt_id, started_at, status, error: error, **accounting)
+          result = base_result(candidate).merge(
+            status: status,
+            content: partial ? @partial_content.dup : "",
+            tool_calls: partial_tool_calls,
+            finish_reason: nil,
+            usage: aggregate_usage,
+            error: error
+          )
+          emit((status == :cancelled) ? :response_cancelled : :response_failed, attempt_id:, result: result)
+          result
+        end
+
+        def fallback_allowed?(result, candidate_index)
+          return false if result[:status] == :partial
+          return false unless fallback_categories.include?(result.dig(:error, :category))
+          return false unless candidates[candidate_index + 1]
+
+          true
+        end
+
+        def select_fallback(result, candidate_index)
+          return false unless fallback_allowed?(result, candidate_index)
+
+          next_candidate = candidates[candidate_index + 1]
+          emit(:fallback_selected, attempt_id: attempts.last[:attempt_id],
+            from: candidate_identity(candidates[candidate_index]),
+            to: candidate_identity(next_candidate), error: result[:error])
+          true
+        end
+
+        def retryable?(result)
+          result.dig(:error, :retryable) && attempts.length < retry_config[:max_attempts]
+        end
+
+        def terminal?(result)
+          return true if %i[succeeded partial cancelled].include?(result[:status])
+          return true unless fallback_categories.include?(result.dig(:error, :category))
+
+          attempts.length >= retry_config[:max_attempts]
+        end
+
+        def backoff
+          exponent = [attempts.length - 1, 0].max
+          delay = retry_config[:base_delay_seconds] * (2**exponent)
+          delay = [delay, @last_error[:retry_after_seconds].to_f].max
+          cap = retry_config[:max_delay_seconds]
+          remaining = cap.nil? ? delay : [delay, cap].min
+          while remaining.positive? && active?
+            interval = [remaining, 0.05].min
+            @sleeper.call(interval)
+            remaining -= interval
+          end
+        end
+
+        def emit(type, attempt_id:, **payload)
+          @sequence += 1
+          event = payload.merge(type: type, request_id: request[:request_id], attempt_id: attempt_id, sequence: @sequence)
+          deliver(event) if @observer
+        end
+
+        # Observer exceptions are caller bugs, not provider failures: they
+        # abort the in-flight request and surface directly instead of being
+        # classified or retried as provider errors.
+        def deliver(event)
+          @observer.respond_to?(:on_chat_event) ? @observer.on_chat_event(event) : @observer.call(event)
+        rescue => error
+          raise ObserverError, "chat observer failed: #{error.class} #{error.message}", error.backtrace
+        end
+
+        def accumulate(event)
+          @partial_content << event[:content].to_s if event[:type] == :text_delta
+          return unless event[:id]
+
+          call = (@stream_tool_calls[event[:id]] ||= {
+            id: event[:id], provider_id: event[:provider_id], name: event[:name],
+            arguments_json: +"", status: :incomplete
+          })
+          call[:arguments_json] << event[:arguments_json].to_s if event[:type] == :tool_call_delta
+          if event[:type] == :tool_call_completed
+            call[:arguments_json] = event[:arguments_json]
+            call[:status] = :completed
+          end
+        end
+
+        def output_event?(event)
+          %i[text_delta tool_call_started tool_call_delta tool_call_completed].include?(event[:type])
+        end
+
+        def partial_tool_calls
+          @stream_tool_calls.values.map(&:dup)
+        end
+
+        def normalize_tool_calls(tool_calls)
+          Array(tool_calls).map do |call|
+            id = call[:provider_id].nil? ? @id_generator.call : tool_id(call[:provider_id])
+            call.merge(id: id, status: :completed)
+          end
+        end
+
+        def normalize_stream_event(event)
+          return event unless event[:provider_id]
+
+          event.merge(id: tool_id(event[:provider_id]))
+        end
+
+        def tool_id(provider_id)
+          @tool_ids[provider_id] ||= @id_generator.call
+        end
+
+        def base_result(candidate)
+          {
+            request_id: request[:request_id],
+            provider: candidate&.dig(:provider),
+            model: candidate&.dig(:model),
+            protocol: candidate&.dig(:protocol),
+            authentication_mode: candidate&.dig(:authentication_mode),
+            parsed: nil,
+            attempts: attempts.dup,
+            provider_request_id: nil
+          }
+        end
+
+        def failed_result(error, candidate)
+          base_result(candidate).merge(status: :failed, content: "", tool_calls: [], finish_reason: nil,
+            usage: aggregate_usage, error: error)
+        end
+
+        def cancelled_result(candidate = candidates.first)
+          error = {category: :cancelled, code: :cancelled, retryable: false, message: "Request cancelled"}
+          base_result(candidate).merge(status: :cancelled, content: "", tool_calls: [], finish_reason: nil,
+            usage: aggregate_usage, error: error)
+        end
+
+        def append_attempt(candidate, attempt_id, started_at, status, error: nil, usage: nil, cost: nil,
+          provider_reported: false)
+          cost = cost.merge(priced_at: Time.now.utc) if cost
+          report = AttemptReport.new(
+            attempt_id: attempt_id,
+            request_id: request[:request_id],
+            number: attempts.length + 1,
+            provider: candidate[:provider],
+            model: candidate[:model],
+            status: status,
+            started_at: started_at.iso8601(6),
+            finished_at: Time.now.utc.iso8601(6),
+            usage: usage,
+            cost: cost,
+            provider_reported: provider_reported,
+            error: error
+          ).attributes
+          attempts << report
+          emit(:attempt_completed, attempt_id:, attempt: report)
+        end
+
+        def aggregate_usage
+          reports = attempts.filter_map { |attempt| attempt[:usage] }
+          return if reports.empty?
+
+          %i[input_tokens output_tokens cache_read_tokens cache_write_tokens thinking_tokens total_tokens].to_h do |key|
+            values = reports.filter_map { |usage| usage[key] }
+            [key, values.empty? ? nil : values.sum]
+          end.compact
+        end
+
+        def classify(error)
+          ErrorClassifier.call(error)
+        end
+
+        def failure_status(error, partial)
+          return :cancelled if error[:category] == :cancelled
+          return :partial if partial
+
+          :failed
+        end
+
+        def candidates = request[:candidates]
+        def retry_config = @retry_config ||= DEFAULT_RETRY.merge(request[:retry] || {})
+        def fallback_categories = Array(request.dig(:fallback, :on_error_categories)).map(&:to_sym)
+
+        def active?
+          token = request[:cancellation]
+          return true unless token
+
+          cancelled = token.respond_to?(:cancelled?) ? token.cancelled? : token.call
+          !cancelled
+        end
+
+        def validate!
+          raise ArgumentError, "operation must be :chat" unless request[:operation]&.to_sym == :chat
+          raise ArgumentError, "request_id is required" if request[:request_id].to_s.empty?
+          raise ArgumentError, "candidates must not be empty" if !request[:candidates].is_a?(Array) || request[:candidates].empty?
+          raise ArgumentError, "messages must be an array" unless request[:messages].is_a?(Array)
+          validate_attempt_limit!
+          candidates.each { |candidate| validate_candidate!(candidate) }
+        end
+
+        def validate_attempt_limit!
+          limit = retry_config[:max_attempts]
+          raise ArgumentError, "retry.max_attempts must be a positive integer" unless limit.is_a?(Integer) && limit.positive?
+        end
+
+        def validate_candidate!(candidate)
+          candidate.replace(symbolize(candidate))
+          %i[provider model protocol authentication_mode credentials].each do |key|
+            raise ArgumentError, "candidate.#{key} is required" if candidate[key].nil?
+          end
+          raise ArgumentError, "only api_key authentication is supported" unless candidate[:authentication_mode].to_sym == :api_key
+          supported_protocols = PROTOCOLS[candidate[:provider].to_sym]
+          unless supported_protocols&.include?(candidate[:protocol].to_sym)
+            raise ArgumentError, "unsupported provider/protocol combination"
+          end
+
+          headers = candidate[:headers] || {}
+          reserved_names = RESERVED_HEADERS.fetch(candidate[:provider].to_sym, RESERVED_HEADERS.values.flatten)
+          reserved = headers.keys.map { |key| key.to_s.downcase } & reserved_names
+          raise ArgumentError, "reserved header override: #{reserved.first}" if reserved.any?
+        end
+
+        def candidate_identity(candidate)
+          candidate.slice(:provider, :model, :protocol, :authentication_mode)
+        end
+
+        def symbolize(value)
+          if value.is_a?(Hash)
+            return value.each_with_object({}) do |(key, child), normalized|
+              symbol = key.to_sym
+              normalized[symbol] = (symbol == :headers) ? child.dup : symbolize(child)
+            end
+          end
+          return value.map { |child| symbolize(child) } if value.is_a?(Array)
+
+          value
+        end
+      end
+    end
+
+    module ErrorClassifier
+      TRANSIENT = {
+        Faraday::TimeoutError => :timeout,
+        Faraday::ConnectionFailed => :connection_failed,
+        Faraday::SSLError => :connection_failed,
+        RubyLLM::RateLimitError => :rate_limited,
+        RubyLLM::ServerError => :server_error,
+        RubyLLM::ServiceUnavailableError => :service_unavailable,
+        RubyLLM::OverloadedError => :overloaded
+      }.freeze
+
+      NON_RETRYABLE = {
+        RubyLlmChatAdapter::MissingCredentialError => [:authentication, :invalid_credential],
+        RubyLLM::UnauthorizedError => [:authentication, :invalid_credential],
+        RubyLLM::ForbiddenError => [:authorization, :permission_denied],
+        RubyLLM::PaymentRequiredError => [:billing, :billing_unavailable],
+        RubyLLM::ContextLengthExceededError => [:context_length, :context_length_exceeded],
+        RubyLLM::BadRequestError => [:invalid_request, :invalid_request],
+        RubyLlmChatAdapter::UnsupportedOptionError => [:unsupported, :unsupported_capability],
+        RubyLLM::UnsupportedServerToolError => [:unsupported, :unsupported_capability],
+        RubyLLM::ToolCallParseError => [:invalid_response, :invalid_tool_arguments],
+        JSON::ParserError => [:invalid_response, :invalid_tool_arguments],
+        RubyLLM::ModelNotFoundError => [:configuration, :invalid_configuration],
+        RubyLLM::ConfigurationError => [:configuration, :invalid_configuration],
+        RubyLLM::CancelledError => [:cancelled, :cancelled]
+      }.freeze
+
+      def self.call(error)
+        transient = TRANSIENT.find { |klass, _| error.is_a?(klass) }
+        return payload(error, :transient, transient.last, true) if transient
+
+        permanent = NON_RETRYABLE.find { |klass, _| error.is_a?(klass) }
+        return payload(error, *permanent.last, false) if permanent
+
+        payload(error, :unknown, :unclassified_provider_error, false)
+      end
+
+      def self.payload(error, category, code, retryable)
+        payload = {category: category, code: code, retryable: retryable, message: safe_message(category, code)}
+        retry_after = retry_after_seconds(error)
+        payload[:retry_after_seconds] = retry_after if retry_after
+        payload
+      end
+      private_class_method :payload
+
+      def self.retry_after_seconds(error)
+        return unless error.respond_to?(:response) && error.response
+
+        headers = error.response.respond_to?(:headers) ? error.response.headers : error.response[:response_headers]
+        value = headers&.find { |key, _| key.to_s.casecmp?("retry-after") }&.last
+        delay = Float(value, exception: false)
+        delay if delay&.finite? && delay >= 0
+      end
+      private_class_method :retry_after_seconds
+
+      def self.safe_message(category, code)
+        "Chat request failed (#{category}/#{code})"
+      end
+      private_class_method :safe_message
+    end
+  end
+end

data/lib/agent_harness/api/ruby_llm_chat_adapter.rb

@@ -0,0 +1,324 @@
+# frozen_string_literal: true
+
+require "json"
+require "ruby_llm"
+
+module AgentHarness
+  module Api
+    # Translates the normalized public chat values to RubyLLM public objects.
+    class RubyLlmChatAdapter
+      class UnsupportedOptionError < StandardError; end
+      class MissingCredentialError < StandardError; end
+
+      PROVIDER_CONFIG = {
+        anthropic: %i[anthropic_api_key anthropic_api_base],
+        openai: %i[openai_api_key openai_api_base]
+      }.freeze
+      DEFAULT_REQUEST_TIMEOUT = 300
+      OPENAI_UNSUPPLIED_CONFIG = %i[openai_organization_id openai_project_id openai_use_system_role].freeze
+
+      def call(candidate:, messages:, tools:, max_output_tokens:, temperature:, stream:, timeout:, cancellation:,
+        on_accounting: nil, prepared_chat: nil, &on_event)
+        provider_usage_reported = false
+        chat = prepared_chat || prepare(candidate:, messages:, tools:, max_output_tokens:, temperature:, timeout:,
+          on_accounting:, provider_usage: -> { provider_usage_reported })
+        response = generate(chat, stream, cancellation) do |event|
+          provider_usage_reported = true if event[:type] == :usage_updated
+          on_event&.call(event)
+        end
+        emit_completed_tool_calls(response, &on_event) if stream
+        normalize_response(response)
+      end
+
+      def prepare(candidate:, messages:, tools:, max_output_tokens:, temperature:, timeout:, on_accounting: nil,
+        provider_usage: -> { false })
+        context = build_context(candidate, timeout, on_accounting, provider_usage)
+        context.chat(model: candidate[:model], provider: candidate[:provider], protocol: ruby_llm_protocol(candidate),
+          assume_model_exists: true).tap do |chat|
+          configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature)
+        end
+      end
+
+      # RubyLLM keys streamed tool-call chunks by a stream index while
+      # continuation chunks carry no call id, so correlation state must live
+      # between chunks. Also remembers the latest cumulative token counts so
+      # duplicate cumulative reports are not re-emitted.
+      class StreamState
+        attr_accessor :input_tokens, :output_tokens
+
+        def initialize
+          @provider_id_by_key = {}
+          @started_ids = {}
+          @latest_provider_id = nil
+          @input_tokens = nil
+          @output_tokens = nil
+        end
+
+        # Links a stream chunk key to its provider call id, returning true
+        # only the first time +provider_id+ is seen.
+        def start(stream_key, provider_id)
+          @provider_id_by_key[stream_key] = provider_id unless stream_key.nil?
+          @latest_provider_id = provider_id
+          return false if @started_ids.key?(provider_id)
+
+          @started_ids[provider_id] = true
+        end
+
+        def provider_id(stream_key)
+          stream_key.nil? ? @latest_provider_id : @provider_id_by_key[stream_key]
+        end
+
+        def usage
+          total = (input_tokens + output_tokens) if input_tokens && output_tokens
+          {input_tokens: input_tokens, output_tokens: output_tokens, total_tokens: total}
+        end
+      end
+
+      private
+
+      def ruby_llm_protocol(candidate)
+        return :anthropic if candidate[:provider].to_sym == :anthropic && candidate[:protocol].to_sym == :messages
+
+        candidate[:protocol]
+      end
+
+      def build_context(candidate, timeout, on_accounting, provider_usage)
+        provider = candidate[:provider].to_sym
+        config_keys = PROVIDER_CONFIG[provider]
+        raise RubyLLM::ConfigurationError, "Unsupported chat provider: #{provider}" unless config_keys
+        raise MissingCredentialError, "API key is required" if candidate.dig(:credentials, :api_key).to_s.empty?
+
+        RubyLLM.context do |config|
+          config.public_send("#{config_keys[0]}=", candidate.dig(:credentials, :api_key))
+          config.public_send("#{config_keys[1]}=", candidate[:endpoint])
+          clear_unsupplied_openai_config(config) if provider == :openai
+          config.max_retries = 0
+          config.instrumenter = UsageInstrumenter.new(on_accounting, provider_usage) if on_accounting
+          apply_timeout(config, timeout)
+        end
+      end
+
+      # Retains only accounting instrumentation; other events may contain
+      # request content and are intentionally discarded.
+      class UsageInstrumenter
+        def initialize(callback, provider_usage)
+          @callback = callback
+          @provider_usage = provider_usage
+        end
+
+        def instrument(name, payload)
+          @callback.call(normalize(payload)) if name == "usage.ruby_llm"
+          yield(payload) if block_given?
+        end
+
+        private
+
+        def normalize(payload)
+          tokens = payload.fetch(:tokens)
+          cost = payload.fetch(:cost)
+          usage = tokens.to_h
+          usage[:total_tokens] = tokens.input + tokens.output if tokens.input && tokens.output
+          {
+            usage: usage,
+            cost: cost.total.nil? ? nil : cost.to_h.merge(source: cost_source(tokens), currency: "USD"),
+            provider_reported: provider_reported?(payload, usage)
+          }
+        end
+
+        def provider_reported?(payload, usage)
+          return false if usage.empty?
+          return true unless payload[:status]&.to_sym == :failed
+          return true if @provider_usage.call
+
+          usage.values.compact.any?(&:positive?)
+        end
+
+        def cost_source(tokens)
+          tokens.reported_cost.nil? ? :estimated : :provider_reported
+        end
+      end
+
+      def clear_unsupplied_openai_config(config)
+        OPENAI_UNSUPPLIED_CONFIG.each { |key| config.public_send("#{key}=", nil) }
+      end
+
+      def apply_timeout(config, timeout)
+        if timeout&.dig(:connect_seconds)
+          raise UnsupportedOptionError, "RubyLLM does not support request-local connect timeouts"
+        end
+
+        config.request_timeout = timeout&.dig(:read_seconds) || DEFAULT_REQUEST_TIMEOUT
+      end
+
+      def configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature)
+        chat.messages = normalize_messages(messages)
+        chat.with_tools(tools.map { |tool| normalized_tool(tool) }) unless tools.empty?
+        chat.with_headers(candidate[:headers] || {})
+        chat.with_max_output_tokens(max_output_tokens) if max_output_tokens
+        chat.with_temperature(temperature) unless temperature.nil?
+      end
+
+      def generate(chat, stream, cancellation)
+        return generate_without_events(chat, cancellation) unless stream
+
+        state = StreamState.new
+        chat.generate do |chunk|
+          if cancelled?(cancellation)
+            chat.cancel
+            raise RubyLLM::CancelledError
+          end
+
+          stream_events(chunk, state).each { |event| yield event }
+        end
+      end
+
+      def generate_without_events(chat, cancellation)
+        raise RubyLLM::CancelledError if cancelled?(cancellation)
+
+        chat.generate
+      end
+
+      def cancelled?(token)
+        return false unless token
+
+        token.respond_to?(:cancelled?) ? token.cancelled? : token.call
+      end
+
+      def normalize_messages(messages)
+        stable_to_provider = tool_id_map(messages)
+        messages.map do |message|
+          role = message.fetch(:role).to_sym
+          normalized = {role: role, content: text_content(message[:content])}
+          normalized[:tool_calls] = normalize_input_tool_calls(message[:tool_calls]) if message[:tool_calls]
+          normalized[:tool_call_id] = stable_to_provider.fetch(message[:tool_call_id], message[:tool_call_id]) if role == :tool
+          normalized
+        end
+      end
+
+      def tool_id_map(messages)
+        messages.each_with_object({}) do |message, ids|
+          Array(message[:tool_calls]).each do |call|
+            ids[call[:id]] = call[:provider_id] || call[:id]
+          end
+        end
+      end
+
+      def normalize_input_tool_calls(tool_calls)
+        Array(tool_calls).to_h do |call|
+          provider_id = call[:provider_id] || call[:id]
+          arguments = JSON.parse(call.fetch(:arguments_json, "{}"))
+          [provider_id, {id: provider_id, name: call[:name], arguments: arguments}]
+        end
+      end
+
+      def text_content(content)
+        return content if content.is_a?(String) || content.nil?
+
+        Array(content).map do |part|
+          unless part[:type]&.to_sym == :text
+            raise UnsupportedOptionError, "unsupported content type: #{part[:type]}"
+          end
+
+          part[:text].to_s
+        end.join
+      end
+
+      def normalized_tool(tool)
+        definition = tool.transform_keys(&:to_sym)
+        name = definition.fetch(:name).to_s
+        Class.new(RubyLLM::Tool).tap do |klass|
+          klass.define_singleton_method(:tool_name) { name }
+          klass.description(definition[:description].to_s)
+          klass.parameters(definition[:input_schema] || definition[:parameters] || {type: "object", properties: {}})
+          klass.define_method(:execute) { |**| raise "AgentHarness transports never execute tools" }
+        end.new
+      end
+
+      def stream_events(chunk, state)
+        [text_event(chunk), *tool_call_events(chunk, state), usage_event(chunk, state)].compact
+      end
+
+      def text_event(chunk)
+        {type: :text_delta, content: chunk.content} unless chunk.content.nil? || chunk.content.empty?
+      end
+
+      def tool_call_events(chunk, state)
+        return [] unless chunk.tool_calls
+
+        chunk.tool_calls.flat_map { |stream_key, call| call_events(stream_key, call, state) }
+      end
+
+      # Continuation chunks have a nil id and carry the provider's raw JSON
+      # fragment as arguments; they join the call that started their key.
+      def call_events(stream_key, call, state)
+        provider_id = call.id || state.provider_id(stream_key)
+        return [] unless provider_id
+
+        events = []
+        if state.start(stream_key, provider_id)
+          events << {type: :tool_call_started, provider_id: provider_id, name: call.name}
+        end
+        fragment = argument_fragment(call.arguments)
+        events << {type: :tool_call_delta, provider_id: provider_id, arguments_json: fragment} if fragment
+        events
+      end
+
+      # Deltas stay appendable JSON text; a start chunk may instead carry a
+      # complete parsed Hash, which becomes JSON once.
+      def argument_fragment(arguments)
+        case arguments
+        when String then arguments.empty? ? nil : arguments
+        when Hash then arguments.empty? ? nil : JSON.generate(arguments)
+        end
+      end
+
+      def usage_event(chunk, state)
+        return unless cumulative_usage_changed?(chunk, state)
+
+        {type: :usage_updated, **state.usage}
+      end
+
+      def cumulative_usage_changed?(chunk, state)
+        tokens = chunk.tokens
+        return false unless tokens
+
+        changed = false
+        if tokens.input && tokens.input != state.input_tokens
+          state.input_tokens = tokens.input
+          changed = true
+        end
+        if tokens.output && tokens.output != state.output_tokens
+          state.output_tokens = tokens.output
+          changed = true
+        end
+        changed
+      end
+
+      def emit_completed_tool_calls(response)
+        Array(response.tool_calls&.values).each do |call|
+          yield(type: :tool_call_completed, **normalize_tool_call(call))
+        end
+      end
+
+      def normalize_response(response)
+        {
+          content: response.content || "",
+          model: response.model,
+          finish_reason: response.finish_reason,
+          usage: normalize_usage(response.tokens),
+          tool_calls: Array(response.tool_calls&.values).map { |call| normalize_tool_call(call) }
+        }
+      end
+
+      def normalize_tool_call(call)
+        {provider_id: call.id, name: call.name, arguments_json: JSON.generate(call.arguments || {})}
+      end
+
+      def normalize_usage(tokens)
+        return unless tokens&.to_h&.any?
+
+        tokens.to_h.merge(total_tokens: (tokens.input && tokens.output) ? tokens.input + tokens.output : nil)
+      end
+    end
+  end
+end

data/lib/agent_harness/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module AgentHarness
-  VERSION = "0.40.0"
+  VERSION = "0.42.0"
 end

data/lib/agent_harness.rb

@@ -471,6 +471,8 @@ require_relative "agent_harness/error_taxonomy"
 require_relative "agent_harness/text_transport"
 require_relative "agent_harness/openai_compatible_transport"
 require_relative "agent_harness/conversation"
+require_relative "agent_harness/api/attempt_report"
+require_relative "agent_harness/api/chat_transport"
 require_relative "agent_harness/quota_status"
 require_relative "agent_harness/authentication"
 require_relative "agent_harness/provider_health_check"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: agent-harness
 version: !ruby/object:Gem::Version
-  version: 0.40.0
+  version: 0.42.0
 platform: ruby
 authors:
 - Bart Agapinan
@@ -135,6 +135,9 @@ files:
 - json-2.18.1.gem
 - lib/agent-harness.rb
 - lib/agent_harness.rb
+- lib/agent_harness/api/attempt_report.rb
+- lib/agent_harness/api/chat_transport.rb
+- lib/agent_harness/api/ruby_llm_chat_adapter.rb
 - lib/agent_harness/authentication.rb
 - lib/agent_harness/command_executor.rb
 - lib/agent_harness/configuration.rb