agent-harness 0.41.0 → 0.43.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c4406422d8b90ec6406bb4cdcc6e1d7496b708f43aad306d258c2570ed4f710e
-  data.tar.gz: cd23fe2f7d4b05e2ac18ac9aa2a631a0b292bc2703ae092f4f55a1ca6f3febbb
+  metadata.gz: 40d84099230c75237b23dca08330a46b739029768ce345b57cb232ba0e66a935
+  data.tar.gz: 47d36a8e160bcd29f9d50f6eabc1d0ded6795d6e05f2e45e93038d1987cd352c
 SHA512:
-  metadata.gz: efb5e61347d70b28e03bb0cd2843429928843147bf8f196d1292af97be97c8ec67ca4ef81e6f169562d1fac5e88e277b06d07a7cd44bf3d85a772effb2c46051
-  data.tar.gz: 7c5fb37c7e14edbc3da68f74dc3f1a6d10e04630e70509a79bd04b9ebbfde2397d6a4753e288dd33d962efae79003728d30cdb0ae113d1bd4864de5995daa112
+  metadata.gz: f059d918fed14691fe444dba2e5e93aaeb2cdb185785a580b8a7341db8198616a0a830fd43461b4dfceffc9258f71ce25a8cbb69351747af46980ba0bd783757
+  data.tar.gz: 2f09db993da48a0ca0ed852eee9653898f474e7c61a6e0603dde3588457d226c8f51de71a8b322e7eba46b7e64203cc61a7607aa31764ed63188ef8b38f7c1e5

data/.release-please-manifest.json

@@ -1,3 +1,3 @@
 {
-  ".": "0.41.0"
+  ".": "0.43.0"
 }

data/CHANGELOG.md

@@ -2,9 +2,24 @@
 
 ### Features
 
+* expose normalized API chat transport and schema-constrained responses with preserved JSON text, locally validated parsed values, and explicit refusal, truncation, invalid JSON, schema mismatch, and unsupported-mode outcomes; this combined delivery supersedes the separate chat-transport work item ([#433](https://github.com/viamin/agent-harness/issues/433), [#434](https://github.com/viamin/agent-harness/issues/434)).
 * add runner model compatibility contract (`AgentHarness.model_compatibility`) with structured `ModelCompatibility::Result` outcomes. Codex exposes static facts for CLI-gated models (e.g. `gpt-5.5` requires Codex CLI `>= 0.116.0`), a baseline supported-model list, supported auth modes, and a `DEFAULT_COMPATIBLE_MODEL_ID` fallback so downstream orchestrators can validate tier/model assignments before scheduling agent runs ([#259](https://github.com/viamin/agent-harness/issues/259)).
 * **auth:** add provider-owned PKCE code-exchange API for Claude OAuth (`AgentHarness::Authentication.exchange_code`). Takes an authorization code plus PKCE verifier (and `redirect_uri`/`client_id`), posts an `authorization_code` grant to the Claude token endpoint, and persists the resulting access/refresh tokens in the native `claudeAiOauth` shape. Adds `exchange_code_supported?` and a `code_exchange` key to `auth_capabilities` ([#266](https://github.com/viamin/agent-harness/issues/266)).
 
+## [0.43.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.42.0...agent-harness/v0.43.0) (2026-09-25)
+
+
+### Features
+
+* Expose Schema-Constrained Parsed Responses (RDR-072) ([#442](https://github.com/viamin/agent-harness/issues/442)) ([3d05cfa](https://github.com/viamin/agent-harness/commit/3d05cfac68624c218eda5f6f9871a3a4d1044ec0))
+
+## [0.42.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.41.0...agent-harness/v0.42.0) (2026-09-25)
+
+
+### Features
+
+* Expose Attempt-Level Usage and Cost (RDR-072) ([#443](https://github.com/viamin/agent-harness/issues/443)) ([818f8ca](https://github.com/viamin/agent-harness/commit/818f8ca0f873f002fc4d26b2a864238ae41fa866))
+
 ## [0.41.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.40.0...agent-harness/v0.41.0) (2026-09-25)
 
 

data/docs/provider-neutral-api-execution-contract.md

@@ -8,8 +8,8 @@ usage, and optional conversation-persistence work.
 ## Status and rollout boundary
 
 RDR-072's rollout guard was **docs-only** for the design phase. The normalized
-chat capability described in "Shipped normalized chat surface" below is now
-implemented. Other capabilities remain design contracts and each still needs
+chat, attempt-accounting, schema, and embedding capabilities described below
+are now implemented. Other capabilities remain design contracts and each still needs
 its own failing-first contract tests, implementation, release evidence, and
 downstream adoption evidence before a caller enables it.
 
@@ -70,11 +70,13 @@ Responses API must select `:chat_completions`; the transport never probes and
 silently switches protocols. Only `authentication_mode: :api_key` is currently
 supported.
 
-Credentials, endpoint, custom headers, timeout, and RubyLLM configuration are
-isolated with a request-local `RubyLLM::Context`. RubyLLM middleware retries
-are disabled; `retry.max_attempts` is the total physical-attempt limit owned by
-the harness. Authentication headers cannot be overridden by custom headers.
-`max_output_tokens` is forwarded without changing it.
+Credentials, endpoint, custom headers, read timeout, and RubyLLM configuration
+are isolated with a request-local `RubyLLM::Context`. Only
+`timeout.read_seconds` is supported; request-local connection timeouts are not.
+RubyLLM middleware retries are disabled; `retry.max_attempts` is the total
+physical-attempt limit owned by the harness. Authentication headers cannot be
+overridden by custom headers. `max_output_tokens` is forwarded without changing
+it.
 
 Unknown model IDs are allowed only because a complete provider and protocol
 are explicit in every candidate (`assume_model_exists: true` in the RubyLLM
@@ -84,6 +86,41 @@ endpoints retain the selected provider's wire protocol and authentication
 shape. Custom provider types, authentication modes, media content, and
 automatic protocol discovery are not supported by this capability.
 
+## Shipped schema-constrained response surface
+
+The same transport accepts `operation: :schema` with a JSON Schema and an
+optional name:
+
+```ruby
+result = transport.call(request.merge(
+  operation: :schema,
+  schema_name: "person",
+  schema: {
+    type: "object",
+    properties: {name: {type: "string"}, age: {type: "integer"}},
+    required: %w[name age],
+    additionalProperties: false
+  }
+))
+
+result[:content] # => '{"name":"Ada","age":37}'
+result[:parsed]  # => {"name" => "Ada", "age" => 37}
+```
+
+Schema operations use provider-native JSON Schema output and the same verified
+Anthropic Messages, OpenAI Responses, and OpenAI Chat Completions scopes as
+normalized chat. Provider adapters infer strictness from the schema; callers
+using a `{schema:, strict:}` envelope can explicitly select strictness.
+`schema_mode: :json_schema` is the only supported mode. JSON-only mode returns
+`unsupported/structured_output_not_supported`.
+
+The harness parses the exact provider text and validates it locally. It does
+not remove Markdown fences or repair malformed JSON. Invalid JSON, schema
+mismatch, refusal, and output-limit truncation return non-retryable
+`invalid_response` outcomes with codes `invalid_json`, `invalid_schema`,
+`refusal`, and `truncated_output`. They retain the original `content` and leave
+`parsed` as `nil`.
+
 ## Ownership boundary
 
 AgentHarness owns protocol translation and one bounded provider request:
@@ -139,7 +176,7 @@ request = {
     }
   ],
   fallback: {on_error_categories: [:transient]},
-  timeout: {connect_seconds: 5, read_seconds: 60},
+  timeout: {read_seconds: 60},
   retry: {max_attempts: 3, base_delay_seconds: 0.25, max_delay_seconds: 2},
   cancellation: cancellation_token,
   metadata: {tenant_id: "tenant-123", workflow_id: "workflow-456"}
@@ -160,9 +197,11 @@ part of a serializable request document.
 gets a distinct `attempt_id`. Redelivering an already reported attempt retains
 its `attempt_id`; initiating another outbound request does not.
 
-Credentials, endpoint, headers, timeouts, retry limits, and cancellation are
-request-local. Implementations MUST prevent concurrent requests from observing
-one another's credentials or headers. They MUST reject reserved header
+Credentials, endpoint, headers, the read timeout, retry limits, and cancellation
+are request-local. Only `timeout.read_seconds` is supported; supplying a
+connection timeout returns `unsupported/unsupported_capability` before any
+provider request. Implementations MUST prevent concurrent requests from
+observing one another's credentials or headers. They MUST reject reserved header
 overrides that would conflict with the selected protocol's authentication.
 Logs and errors MUST NOT contain credentials, authorization headers, message
 bodies, tool arguments, or full provider responses.
@@ -336,6 +375,23 @@ observer so durable accounting can persist an attempt even when no message is
 created. Callers deduplicate on `attempt_id`. Cost identifies its source as
 provider-reported or harness-estimated; unknown cost remains `nil`.
 
+The observer receives `attempt_completed` after every physical request, with
+the same report later returned in `result[:attempts]`. Delivery is at least
+once across process recovery: consumers MUST enforce a unique key on
+`attempt_id` and treat a repeated report as an idempotent upsert, not another
+charge. `AttemptReport.from_h` restores JSON-decoded reports. Stored cost
+includes USD component amounts, `source` (`provider_reported` or `estimated`),
+and `priced_at`; restoration never consults current prices. Missing counts and
+prices remain absent/`nil`, while a reported zero remains zero. Usage may also
+include `cache_read_tokens`, `cache_write_tokens`, and `thinking_tokens`.
+
+Attempt events contain only identifiers, provider/model, outcome, normalized
+usage/cost, timestamps, and sanitized classified errors. They do not include
+prompts, messages, request headers, endpoints, or credentials. RubyLLM API chat
+requests are in this ledger. Existing CLI providers, `TextTransport`,
+`OpenAICompatibleTransport`, token trackers, embeddings, and other
+non-`Api::ChatTransport` paths remain outside it.
+
 Only errors classified `transient` are eligible for bounded request retry:
 connection failure, timeout before a partial stream, rate limit, server error,
 service unavailable, and overload. Authentication, authorization, billing,
@@ -487,7 +543,7 @@ records.
 | Contract area | RubyLLM 2.0 mapping | Decision or gap |
 | --- | --- | --- |
 | Chat/protocols | `RubyLLM.chat`, messages, tools, stream callbacks | Candidate adapter; normalize all values and errors |
-| Schema | `with_schema`, `response.parsed`, model capability registry | Candidate; distinguish enforced schema from JSON mode |
+| Schema | `with_schema`; harness JSON parsing and validation | Implemented for the verified chat scopes; JSON-only mode stays distinct |
 | Embeddings | `RubyLLM.embed` and normalized vectors/usage | Candidate first capability; persistence remains in Paid |
 | Custom headers | `with_headers` | Candidate; contract-test merging and secret redaction |
 | Endpoint/credentials | provider configuration | Global mutable configuration is unsuitable; require request-local isolation or an upstream-supported client boundary |
@@ -569,11 +625,47 @@ types stay behind the harness boundary.
 
 ## Compatibility and release evidence
 
+### Attempt-accounting capability evidence
+
+- Release: pending the first published version containing issue #435; a Git
+  branch or tag alone is not downstream adoption evidence.
+- Scope: normalized chat through `Api::ChatTransport` for Anthropic Messages
+  and OpenAI Responses/Chat Completions (including compatible endpoints), with
+  request-local API-key authentication. It is stacked on the normalized chat
+  capability from #433.
+- Verification: the API contract specs cover request-local credentials,
+  endpoint/header isolation, error classification, bounded non-nested retry,
+  fallback, cancellation, partial usage, cache usage, repeated identity,
+  observer redaction, and JSON reload with preserved pricing. The full upstream
+  suite and lint run on the repository's supported Ruby environment.
+- Migration: no Rails tables or migrations are loaded or required. Paid and
+  agent-image consumer versions remain unverified and MUST NOT adopt this
+  capability until their integration suites record the exact released gem and
+  image versions.
+- Retained paths: CLI/subscription providers, legacy HTTP transports,
+  embeddings, and token trackers remain outside this ledger and require
+  separate migration issues.
+
+### Schema capability release evidence
+
+- Publication: unreleased; record the first installable version before
+  downstream adoption.
+- Verified scopes: `:schema` with Anthropic Messages, OpenAI Responses, and
+  OpenAI Chat Completions using API-key authentication, including compatible
+  endpoints that explicitly select Chat Completions.
+- Contract coverage: valid and required-field schemas, classified failures,
+  bounded retries, cancellation, refusal, truncation, malformed JSON, and
+  schema mismatch.
+- Retained paths: CLI and subscription execution remain on existing provider
+  interfaces. JSON-only mode and model-specific capability discovery are not
+  migrated.
+
 AgentHarness currently supports Ruby 3.2 and later and must remain usable as a
-plain Ruby gem. RubyLLM 2.0.0 itself supports Ruby 3.1.3 and later, but adding it
-would introduce Faraday, event-stream parsing, Schematist, Marcel, and Zeitwerk
-runtime dependencies. A capability issue must measure and publish the resolved
-dependency set and test the harness minimum Ruby version before adoption.
+plain Ruby gem. RubyLLM 2.0.0 itself supports Ruby 3.1.3 and later and adds
+Faraday, event-stream parsing, Schematist, Marcel, and Zeitwerk runtime
+dependencies. Local response validation adds `json_schemer` and its bounded
+dependency set. A capability issue must test the harness minimum Ruby version
+before adoption.
 
 Rails and Active Record remain optional. Requiring `agent_harness` in a process
 without Rails MUST NOT load Active Record, connect to a database, or require

data/lib/agent_harness/api/attempt_report.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+require "time"
+
+module AgentHarness
+  module Api
+    # Serializable accounting facts for one physical provider request.
+    class AttemptReport
+      STATUSES = %i[succeeded failed cancelled partial].freeze
+      USAGE_KEYS = %i[input_tokens output_tokens cache_read_tokens cache_write_tokens thinking_tokens total_tokens].freeze
+      COST_KEYS = %i[input output cache_read cache_write thinking total].freeze
+
+      attr_reader :attributes
+
+      def self.from_h(attributes)
+        new(**symbolize(attributes))
+      end
+
+      def self.symbolize(value)
+        return value.to_h { |key, child| [key.to_sym, symbolize(child)] } if value.is_a?(Hash)
+        return value.map { |child| symbolize(child) } if value.is_a?(Array)
+
+        value
+      end
+      private_class_method :symbolize
+
+      def initialize(attempt_id:, request_id:, number:, provider:, model:, status:, started_at:, finished_at:,
+        usage: nil, cost: nil, provider_reported: false, error: nil)
+        @attributes = {
+          attempt_id: attempt_id.to_s, request_id: request_id.to_s, number: number,
+          provider: provider.to_sym, model: model&.to_s, status: status.to_sym,
+          started_at: timestamp(started_at), finished_at: timestamp(finished_at),
+          usage: normalize_usage(usage), cost: normalize_cost(cost),
+          provider_reported: provider_reported == true, error: error
+        }
+        validate!
+        deep_freeze(@attributes)
+      end
+
+      def to_h
+        deep_dup(attributes)
+      end
+
+      private
+
+      def timestamp(value)
+        value.respond_to?(:iso8601) ? value.iso8601(6) : Time.iso8601(value.to_s).utc.iso8601(6)
+      end
+
+      def normalize_usage(usage)
+        return unless usage
+
+        normalized = self.class.send(:symbolize, usage)
+        USAGE_KEYS.each_with_object({}) do |key, result|
+          result[key] = normalized[key] if normalized.key?(key)
+        end
+      end
+
+      def normalize_cost(cost)
+        return unless cost
+
+        normalized = self.class.send(:symbolize, cost)
+        amounts = COST_KEYS.to_h { |key| [key, normalized[key]] }
+        amounts.merge(currency: normalized.fetch(:currency, "USD"), source: normalized.fetch(:source).to_sym,
+          priced_at: timestamp(normalized.fetch(:priced_at)))
+      end
+
+      def validate!
+        raise ArgumentError, "attempt_id is required" if attributes[:attempt_id].empty?
+        raise ArgumentError, "request_id is required" if attributes[:request_id].empty?
+        raise ArgumentError, "number must be a positive integer" unless attributes[:number].is_a?(Integer) && attributes[:number].positive?
+        raise ArgumentError, "unknown attempt status" unless STATUSES.include?(attributes[:status])
+      end
+
+      def deep_freeze(value)
+        value.each_value { |child| deep_freeze(child) } if value.is_a?(Hash)
+        value.each { |child| deep_freeze(child) } if value.is_a?(Array)
+        value.freeze
+      end
+
+      def deep_dup(value)
+        return value.to_h { |key, child| [key, deep_dup(child)] } if value.is_a?(Hash)
+        return value.map { |child| deep_dup(child) } if value.is_a?(Array)
+
+        value
+      end
+    end
+  end
+end

data/lib/agent_harness/api/chat_transport.rb

@@ -2,7 +2,9 @@
 
 require "securerandom"
 require "ruby_llm"
+require_relative "attempt_report"
 require_relative "ruby_llm_chat_adapter"
+require_relative "schema_response"
 
 module AgentHarness
   module Api
@@ -53,6 +55,7 @@ module AgentHarness
         end
 
         def call
+          return unsupported_schema_result unless schema_mode_supported?
           return cancelled_result unless active?
 
           candidates.each_with_index do |candidate, candidate_index|
@@ -85,6 +88,19 @@ module AgentHarness
         end
 
         def perform_attempt(candidate)
+          accounting = nil
+          provider_usage = -> { !streamed_usage.nil? }
+          prepared_chat = @adapter.prepare(
+            candidate: candidate,
+            messages: request[:messages],
+            tools: request[:tools] || [],
+            max_output_tokens: request[:max_output_tokens],
+            temperature: request[:temperature],
+            timeout: request[:timeout],
+            schema: schema_payload,
+            on_accounting: ->(facts) { accounting = facts },
+            provider_usage: provider_usage
+          )
           attempt_id = @id_generator.call
           started_at = Time.now.utc
           emitted = false
@@ -99,7 +115,10 @@ module AgentHarness
             temperature: request[:temperature],
             stream: request[:stream] == true,
             timeout: request[:timeout],
-            cancellation: request[:cancellation]
+            cancellation: request[:cancellation],
+            schema: schema_payload,
+            on_accounting: ->(facts) { accounting = facts },
+            prepared_chat: prepared_chat
           ) do |event|
             event = normalize_stream_event(event)
             emitted = true if output_event?(event)
@@ -109,31 +128,47 @@ module AgentHarness
           end
           raise RubyLLM::CancelledError unless active?
 
-          success(candidate, attempt_id, started_at, adapter_result)
+          success(candidate, attempt_id, started_at, adapter_result, accounting)
         rescue ObserverError
           raise
         rescue => error
-          failure(candidate, attempt_id, started_at, classify(error), partial: emitted, usage: streamed_usage)
+          return failed_result(classify(error), candidate) unless attempt_id
+
+          failure(candidate, attempt_id, started_at, classify(error), partial: emitted,
+            accounting: accounting || {
+              usage: streamed_usage, provider_reported: !streamed_usage.nil?
+            })
         end
 
-        def success(candidate, attempt_id, started_at, adapter_result)
-          attempts << attempt_report(candidate, attempt_id, started_at, :succeeded, usage: adapter_result[:usage])
+        def success(candidate, attempt_id, started_at, adapter_result, accounting)
+          accounting ||= {usage: adapter_result[:usage], provider_reported: !adapter_result[:usage].nil?}
+          schema_result = normalize_schema_response(adapter_result)
+          status = schema_result[:error] ? :failed : :succeeded
+          append_attempt(candidate, attempt_id, started_at, status, error: schema_result[:error], **accounting)
           result = base_result(candidate).merge(
-            status: :succeeded,
-            content: adapter_result[:content] || "",
+            status: status,
+            content: schema_result[:content],
+            parsed: schema_result[:parsed],
             tool_calls: normalize_tool_calls(adapter_result[:tool_calls]),
             finish_reason: adapter_result[:finish_reason],
             usage: aggregate_usage,
-            error: nil
+            error: schema_result[:error]
           )
-          emit(:response_completed, attempt_id:, result: result)
+          event = (status == :succeeded) ? :response_completed : :response_failed
+          emit(event, attempt_id:, result: result)
           result
         end
 
-        def failure(candidate, attempt_id, started_at, error, partial:, usage:)
+        def normalize_schema_response(adapter_result)
+          return {content: adapter_result[:content] || "", parsed: nil, error: nil} unless schema_operation?
+
+          SchemaResponse.new(schema_definition).call(adapter_result)
+        end
+
+        def failure(candidate, attempt_id, started_at, error, partial:, accounting:)
           status = failure_status(error, partial)
           error = error.merge(retryable: false) if partial
-          attempts << attempt_report(candidate, attempt_id, started_at, status, error: error, usage: usage)
+          append_attempt(candidate, attempt_id, started_at, status, error: error, **accounting)
           result = base_result(candidate).merge(
             status: status,
             content: partial ? @partial_content.dup : "",
@@ -178,8 +213,9 @@ module AgentHarness
         def backoff
           exponent = [attempts.length - 1, 0].max
           delay = retry_config[:base_delay_seconds] * (2**exponent)
+          delay = [delay, @last_error[:retry_after_seconds].to_f].max
           cap = retry_config[:max_delay_seconds]
-          remaining = cap&.positive? ? [delay, cap].min : delay
+          remaining = cap.nil? ? delay : [delay, cap].min
           while remaining.positive? && active?
             interval = [remaining, 0.05].min
             @sleeper.call(interval)
@@ -227,7 +263,8 @@ module AgentHarness
 
         def normalize_tool_calls(tool_calls)
           Array(tool_calls).map do |call|
-            call.merge(id: tool_id(call[:provider_id]), status: :completed)
+            id = call[:provider_id].nil? ? @id_generator.call : tool_id(call[:provider_id])
+            call.merge(id: id, status: :completed)
           end
         end
 
@@ -265,8 +302,10 @@ module AgentHarness
             usage: aggregate_usage, error: error)
         end
 
-        def attempt_report(candidate, attempt_id, started_at, status, error: nil, usage: nil)
-          {
+        def append_attempt(candidate, attempt_id, started_at, status, error: nil, usage: nil, cost: nil,
+          provider_reported: false)
+          cost = cost.merge(priced_at: Time.now.utc) if cost
+          report = AttemptReport.new(
             attempt_id: attempt_id,
             request_id: request[:request_id],
             number: attempts.length + 1,
@@ -276,20 +315,22 @@ module AgentHarness
             started_at: started_at.iso8601(6),
             finished_at: Time.now.utc.iso8601(6),
             usage: usage,
-            cost: nil,
-            provider_reported: false,
+            cost: cost,
+            provider_reported: provider_reported,
             error: error
-          }
+          ).attributes
+          attempts << report
+          emit(:attempt_completed, attempt_id:, attempt: report)
         end
 
         def aggregate_usage
           reports = attempts.filter_map { |attempt| attempt[:usage] }
           return if reports.empty?
 
-          %i[input_tokens output_tokens total_tokens].to_h do |key|
+          %i[input_tokens output_tokens cache_read_tokens cache_write_tokens thinking_tokens total_tokens].to_h do |key|
             values = reports.filter_map { |usage| usage[key] }
             [key, values.empty? ? nil : values.sum]
-          end
+          end.compact
         end
 
         def classify(error)
@@ -316,14 +357,48 @@ module AgentHarness
         end
 
         def validate!
-          raise ArgumentError, "operation must be :chat" unless request[:operation]&.to_sym == :chat
+          unless %i[chat schema].include?(request[:operation]&.to_sym)
+            raise ArgumentError, "operation must be :chat or :schema"
+          end
           raise ArgumentError, "request_id is required" if request[:request_id].to_s.empty?
           raise ArgumentError, "candidates must not be empty" if !request[:candidates].is_a?(Array) || request[:candidates].empty?
           raise ArgumentError, "messages must be an array" unless request[:messages].is_a?(Array)
+          raise ArgumentError, "schema is required for a schema operation" if schema_operation? && !request[:schema].is_a?(Hash)
           validate_attempt_limit!
           candidates.each { |candidate| validate_candidate!(candidate) }
         end
 
+        def schema_operation?
+          request[:operation].to_sym == :schema
+        end
+
+        def schema_mode_supported?
+          !schema_operation? || !request[:schema_mode] || request[:schema_mode].to_sym == :json_schema
+        end
+
+        def schema_payload
+          return unless schema_operation?
+
+          raw = request[:schema]
+          return raw.merge(name: request[:schema_name] || raw[:name]) if raw[:schema]
+
+          {name: request[:schema_name] || raw[:title] || "response", schema: raw}
+        end
+
+        def schema_definition
+          schema_payload.fetch(:schema)
+        end
+
+        def unsupported_schema_result
+          error = {
+            category: :unsupported,
+            code: :structured_output_not_supported,
+            retryable: false,
+            message: "Schema request failed (unsupported/structured_output_not_supported)"
+          }
+          failed_result(error, candidates.first)
+        end
+
         def validate_attempt_limit!
           limit = retry_config[:max_attempts]
           raise ArgumentError, "retry.max_attempts must be a positive integer" unless limit.is_a?(Integer) && limit.positive?
@@ -402,10 +477,23 @@ module AgentHarness
       end
 
       def self.payload(error, category, code, retryable)
-        {category: category, code: code, retryable: retryable, message: safe_message(category, code)}
+        payload = {category: category, code: code, retryable: retryable, message: safe_message(category, code)}
+        retry_after = retry_after_seconds(error)
+        payload[:retry_after_seconds] = retry_after if retry_after
+        payload
       end
       private_class_method :payload
 
+      def self.retry_after_seconds(error)
+        return unless error.respond_to?(:response) && error.response
+
+        headers = error.response.respond_to?(:headers) ? error.response.headers : error.response[:response_headers]
+        value = headers&.find { |key, _| key.to_s.casecmp?("retry-after") }&.last
+        delay = Float(value, exception: false)
+        delay if delay&.finite? && delay >= 0
+      end
+      private_class_method :retry_after_seconds
+
       def self.safe_message(category, code)
         "Chat request failed (#{category}/#{code})"
       end

data/lib/agent_harness/api/ruby_llm_chat_adapter.rb

@@ -5,6 +5,22 @@ require "ruby_llm"
 
 module AgentHarness
   module Api
+    # RubyLLM 2.0.0 flattens Responses API refusal deltas into ordinary text
+    # chunks and offers no public access to their event type. The gemspec pins
+    # that exact release so this compatibility shim cannot silently outlive the
+    # private parser shape it targets. Remove it when RubyLLM exposes refusals.
+    module RubyLlmResponsesStreamingRefusal
+      module RefusalChunk; end
+
+      def build_chunk(data)
+        super.tap do |chunk|
+          chunk.extend(RefusalChunk) if data["type"] == "response.refusal.delta"
+        end
+      end
+    end
+
+    RubyLLM::Protocols::Responses.prepend(RubyLlmResponsesStreamingRefusal)
+
     # Translates the normalized public chat values to RubyLLM public objects.
     class RubyLlmChatAdapter
       class UnsupportedOptionError < StandardError; end
@@ -14,16 +30,29 @@ module AgentHarness
         anthropic: %i[anthropic_api_key anthropic_api_base],
         openai: %i[openai_api_key openai_api_base]
       }.freeze
+      DEFAULT_REQUEST_TIMEOUT = 300
       OPENAI_UNSUPPLIED_CONFIG = %i[openai_organization_id openai_project_id openai_use_system_role].freeze
 
-      def call(candidate:, messages:, tools:, max_output_tokens:, temperature:, stream:, timeout:, cancellation:, &on_event)
-        context = build_context(candidate, timeout)
-        chat = context.chat(model: candidate[:model], provider: candidate[:provider], protocol: ruby_llm_protocol(candidate),
-          assume_model_exists: true)
-        configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature)
-        response = generate(chat, stream, cancellation, &on_event)
+      def call(candidate:, messages:, tools:, max_output_tokens:, temperature:, stream:, timeout:, cancellation:,
+        schema: nil, on_accounting: nil, prepared_chat: nil, &on_event)
+        provider_usage_reported = false
+        chat = prepared_chat || prepare(candidate:, messages:, tools:, max_output_tokens:, temperature:, timeout:,
+          schema:, on_accounting:, provider_usage: -> { provider_usage_reported })
+        response, streamed_refusal = generate(chat, stream, cancellation) do |event|
+          provider_usage_reported = true if event[:type] == :usage_updated
+          on_event&.call(event)
+        end
         emit_completed_tool_calls(response, &on_event) if stream
-        normalize_response(response)
+        normalize_response(response, streamed_refusal: streamed_refusal)
+      end
+
+      def prepare(candidate:, messages:, tools:, max_output_tokens:, temperature:, timeout:, on_accounting: nil,
+        schema: nil, provider_usage: -> { false })
+        context = build_context(candidate, timeout, on_accounting, provider_usage)
+        context.chat(model: candidate[:model], provider: candidate[:provider], protocol: ruby_llm_protocol(candidate),
+          assume_model_exists: true).tap do |chat|
+          configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature, schema)
+        end
       end
 
       # RubyLLM keys streamed tool-call chunks by a stream index while
@@ -32,6 +61,7 @@ module AgentHarness
       # duplicate cumulative reports are not re-emitted.
       class StreamState
         attr_accessor :input_tokens, :output_tokens
+        attr_reader :refusal
 
         def initialize
           @provider_id_by_key = {}
@@ -39,6 +69,7 @@ module AgentHarness
           @latest_provider_id = nil
           @input_tokens = nil
           @output_tokens = nil
+          @refusal = false
         end
 
         # Links a stream chunk key to its provider call id, returning true
@@ -59,6 +90,10 @@ module AgentHarness
           total = (input_tokens + output_tokens) if input_tokens && output_tokens
           {input_tokens: input_tokens, output_tokens: output_tokens, total_tokens: total}
         end
+
+        def observe(chunk)
+          @refusal ||= chunk.is_a?(RubyLlmResponsesStreamingRefusal::RefusalChunk)
+        end
       end
 
       private
@@ -69,7 +104,7 @@ module AgentHarness
         candidate[:protocol]
       end
 
-      def build_context(candidate, timeout)
+      def build_context(candidate, timeout, on_accounting, provider_usage)
         provider = candidate[:provider].to_sym
         config_keys = PROVIDER_CONFIG[provider]
         raise RubyLLM::ConfigurationError, "Unsupported chat provider: #{provider}" unless config_keys
@@ -80,10 +115,51 @@ module AgentHarness
           config.public_send("#{config_keys[1]}=", candidate[:endpoint])
           clear_unsupplied_openai_config(config) if provider == :openai
           config.max_retries = 0
+          config.instrumenter = UsageInstrumenter.new(on_accounting, provider_usage) if on_accounting
           apply_timeout(config, timeout)
         end
       end
 
+      # Retains only accounting instrumentation; other events may contain
+      # request content and are intentionally discarded.
+      class UsageInstrumenter
+        def initialize(callback, provider_usage)
+          @callback = callback
+          @provider_usage = provider_usage
+        end
+
+        def instrument(name, payload)
+          @callback.call(normalize(payload)) if name == "usage.ruby_llm"
+          yield(payload) if block_given?
+        end
+
+        private
+
+        def normalize(payload)
+          tokens = payload.fetch(:tokens)
+          cost = payload.fetch(:cost)
+          usage = tokens.to_h
+          usage[:total_tokens] = tokens.input + tokens.output if tokens.input && tokens.output
+          {
+            usage: usage,
+            cost: cost.total.nil? ? nil : cost.to_h.merge(source: cost_source(tokens), currency: "USD"),
+            provider_reported: provider_reported?(payload, usage)
+          }
+        end
+
+        def provider_reported?(payload, usage)
+          return false if usage.empty?
+          return true unless payload[:status]&.to_sym == :failed
+          return true if @provider_usage.call
+
+          usage.values.compact.any?(&:positive?)
+        end
+
+        def cost_source(tokens)
+          tokens.reported_cost.nil? ? :estimated : :provider_reported
+        end
+      end
+
       def clear_unsupplied_openai_config(config)
         OPENAI_UNSUPPLIED_CONFIG.each { |key| config.public_send("#{key}=", nil) }
       end
@@ -93,23 +169,23 @@ module AgentHarness
           raise UnsupportedOptionError, "RubyLLM does not support request-local connect timeouts"
         end
 
-        seconds = timeout&.dig(:read_seconds)
-        config.request_timeout = seconds if seconds
+        config.request_timeout = timeout&.dig(:read_seconds) || DEFAULT_REQUEST_TIMEOUT
       end
 
-      def configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature)
+      def configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature, schema)
         chat.messages = normalize_messages(messages)
         chat.with_tools(tools.map { |tool| normalized_tool(tool) }) unless tools.empty?
         chat.with_headers(candidate[:headers] || {})
         chat.with_max_output_tokens(max_output_tokens) if max_output_tokens
         chat.with_temperature(temperature) unless temperature.nil?
+        chat.with_schema(schema) if schema
       end
 
       def generate(chat, stream, cancellation)
-        return generate_without_events(chat, cancellation) unless stream
+        return [generate_without_events(chat, cancellation), false] unless stream
 
         state = StreamState.new
-        chat.generate do |chunk|
+        response = chat.generate do |chunk|
           if cancelled?(cancellation)
             chat.cancel
             raise RubyLLM::CancelledError
@@ -117,6 +193,7 @@ module AgentHarness
 
           stream_events(chunk, state).each { |event| yield event }
         end
+        [response, state.refusal]
       end
 
       def generate_without_events(chat, cancellation)
@@ -182,6 +259,7 @@ module AgentHarness
       end
 
       def stream_events(chunk, state)
+        state.observe(chunk)
         [text_event(chunk), *tool_call_events(chunk, state), usage_event(chunk, state)].compact
       end
 
@@ -247,26 +325,54 @@ module AgentHarness
         end
       end
 
-      def normalize_response(response)
+      def normalize_response(response, streamed_refusal: false)
         {
           content: response.content || "",
           model: response.model,
           finish_reason: response.finish_reason,
           usage: normalize_usage(response.tokens),
-          tool_calls: Array(response.tool_calls&.values).map { |call| normalize_tool_call(call) }
+          tool_calls: Array(response.tool_calls&.values).map { |call| normalize_tool_call(call) },
+          refusal: streamed_refusal || refusal?(response)
         }
       end
 
+      def refusal?(response)
+        return true if response.finish_reason == :content_filter
+        return false unless response.respond_to?(:raw)
+
+        body = response.raw&.body
+        return false unless body.is_a?(Hash)
+
+        responses_api_refusal?(body) || chat_completions_refusal?(body)
+      end
+
+      def responses_api_refusal?(body)
+        Array(body["output"] || body[:output]).any? do |item|
+          next false unless item.is_a?(Hash)
+
+          Array(item["content"] || item[:content]).any? do |part|
+            part.is_a?(Hash) && (part["type"] || part[:type]) == "refusal"
+          end
+        end
+      end
+
+      def chat_completions_refusal?(body)
+        Array(body["choices"] || body[:choices]).any? do |choice|
+          next false unless choice.is_a?(Hash)
+
+          message = choice["message"] || choice[:message]
+          message.is_a?(Hash) && !(message["refusal"] || message[:refusal]).nil?
+        end
+      end
+
       def normalize_tool_call(call)
         {provider_id: call.id, name: call.name, arguments_json: JSON.generate(call.arguments || {})}
       end
 
       def normalize_usage(tokens)
-        input = tokens&.input
-        output = tokens&.output
-        return unless input || output
+        return unless tokens&.to_h&.any?
 
-        {input_tokens: input, output_tokens: output, total_tokens: (input && output) ? input + output : nil}
+        tokens.to_h.merge(total_tokens: (tokens.input && tokens.output) ? tokens.input + tokens.output : nil)
       end
     end
   end

data/lib/agent_harness/api/schema_response.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+require "json"
+require "json_schemer"
+
+module AgentHarness
+  module Api
+    # Parses and validates one schema-constrained provider response.
+    class SchemaResponse
+      def initialize(schema)
+        @validator = JSONSchemer.schema(schema)
+      end
+
+      def call(response)
+        content = response[:content].to_s
+        return failure(content, :refusal) if response[:refusal]
+        return failure(content, :truncated_output) if response[:finish_reason]&.to_sym == :max_tokens
+
+        parsed = JSON.parse(content)
+        return failure(content, :invalid_schema) unless @validator.valid?(parsed)
+
+        {content: content, parsed: parsed, error: nil}
+      rescue JSON::ParserError
+        failure(content, :invalid_json)
+      end
+
+      private
+
+      def failure(content, code)
+        {
+          content: content,
+          parsed: nil,
+          error: {
+            category: :invalid_response,
+            code: code,
+            retryable: false,
+            message: "Schema response failed (invalid_response/#{code})"
+          }
+        }
+      end
+    end
+  end
+end

data/lib/agent_harness/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module AgentHarness
-  VERSION = "0.41.0"
+  VERSION = "0.43.0"
 end

data/lib/agent_harness.rb

@@ -471,7 +471,9 @@ require_relative "agent_harness/error_taxonomy"
 require_relative "agent_harness/text_transport"
 require_relative "agent_harness/openai_compatible_transport"
 require_relative "agent_harness/conversation"
+require_relative "agent_harness/api/attempt_report"
 require_relative "agent_harness/api/chat_transport"
+require_relative "agent_harness/api/schema_response"
 require_relative "agent_harness/quota_status"
 require_relative "agent_harness/authentication"
 require_relative "agent_harness/provider_health_check"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: agent-harness
 version: !ruby/object:Gem::Version
-  version: 0.41.0
+  version: 0.43.0
 platform: ruby
 authors:
 - Bart Agapinan
@@ -30,25 +30,33 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '2.0'
 - !ruby/object:Gem::Dependency
-  name: ruby_llm
+  name: json_schemer
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '2.0'
-    - - "<"
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '2.4'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.0'
-    - - "<"
+        version: '2.4'
+- !ruby/object:Gem::Dependency
+  name: ruby_llm
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: 2.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 2.0.0
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -135,8 +143,10 @@ files:
 - json-2.18.1.gem
 - lib/agent-harness.rb
 - lib/agent_harness.rb
+- lib/agent_harness/api/attempt_report.rb
 - lib/agent_harness/api/chat_transport.rb
 - lib/agent_harness/api/ruby_llm_chat_adapter.rb
+- lib/agent_harness/api/schema_response.rb
 - lib/agent_harness/authentication.rb
 - lib/agent_harness/command_executor.rb
 - lib/agent_harness/configuration.rb