agent-harness 0.42.0 → 0.44.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9c2a3cf60957839163bbda4f4e202f663ee68915e49ba3853abce85b5f6f16e9
-  data.tar.gz: b5a9950414fc72fe2626abbb3ef50275e2e28267346414d04d330d3ea37c1297
+  metadata.gz: 4908f5b12da6f2ae8bc0a93ebf3f682e99fbe017146a9bce9f0727ca397a3f72
+  data.tar.gz: 571c10e6f65f9a2a9c03e7b3ee83c8c13b461e66e27d3ff40ddd3dc7b2d883f7
 SHA512:
-  metadata.gz: b6147210553af7975d99dbf53cc52c0b37e7ba25c8424a9cfb24a290bae9172d25bcbc8a6e5b489d1b2ae363251facc756ea0eaae55a6efd3904cdb64fcda025
-  data.tar.gz: f869749041ff450642b8ba940a3f0f9f8ea2cd7a3d788b2414ae7f0115dcb5fde4dcb7f5aa725b4e1ba2216568a1412e2bf5439c47b61bf75b2d06390faebfae
+  metadata.gz: 77c952417ea0b5f2dbf2a9c646350d3b2154f864cab17639d56abbd2ad95d13757244a2c6680c8b071ce938087acb68fdf5c0ca41d5156c01c0791106efaf4f4
+  data.tar.gz: 2164b3afcbb40f241417950b9d126e0c82b5df261a544f9e5e11255b487a374bd401ef632fd9ed9b75fe1b42e778732b6092d5e215f61ab30e53ff84e14fca7e

data/.release-please-manifest.json

@@ -1,3 +1,3 @@
 {
-  ".": "0.42.0"
+  ".": "0.44.0"
 }

data/CHANGELOG.md

@@ -2,9 +2,24 @@
 
 ### Features
 
+* expose normalized API chat transport and schema-constrained responses with preserved JSON text, locally validated parsed values, and explicit refusal, truncation, invalid JSON, schema mismatch, and unsupported-mode outcomes; this combined delivery supersedes the separate chat-transport work item ([#433](https://github.com/viamin/agent-harness/issues/433), [#434](https://github.com/viamin/agent-harness/issues/434)).
 * add runner model compatibility contract (`AgentHarness.model_compatibility`) with structured `ModelCompatibility::Result` outcomes. Codex exposes static facts for CLI-gated models (e.g. `gpt-5.5` requires Codex CLI `>= 0.116.0`), a baseline supported-model list, supported auth modes, and a `DEFAULT_COMPATIBLE_MODEL_ID` fallback so downstream orchestrators can validate tier/model assignments before scheduling agent runs ([#259](https://github.com/viamin/agent-harness/issues/259)).
 * **auth:** add provider-owned PKCE code-exchange API for Claude OAuth (`AgentHarness::Authentication.exchange_code`). Takes an authorization code plus PKCE verifier (and `redirect_uri`/`client_id`), posts an `authorization_code` grant to the Claude token endpoint, and persists the resulting access/refresh tokens in the native `claudeAiOauth` shape. Adds `exchange_code_supported?` and a `code_exchange` key to `auth_capabilities` ([#266](https://github.com/viamin/agent-harness/issues/266)).
 
+## [0.44.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.43.0...agent-harness/v0.44.0) (2026-09-25)
+
+
+### Features
+
+* Evaluate resumable loop delegation and implement only if simpler (RDR-072) ([#448](https://github.com/viamin/agent-harness/issues/448)) ([02dedfb](https://github.com/viamin/agent-harness/commit/02dedfb08096631e59f326333df61feb57eba395))
+
+## [0.43.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.42.0...agent-harness/v0.43.0) (2026-09-25)
+
+
+### Features
+
+* Expose Schema-Constrained Parsed Responses (RDR-072) ([#442](https://github.com/viamin/agent-harness/issues/442)) ([3d05cfa](https://github.com/viamin/agent-harness/commit/3d05cfac68624c218eda5f6f9871a3a4d1044ec0))
+
 ## [0.42.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.41.0...agent-harness/v0.42.0) (2026-09-25)
 
 

data/docs/provider-neutral-api-execution-contract.md

@@ -8,10 +8,13 @@ usage, and optional conversation-persistence work.
 ## Status and rollout boundary
 
 RDR-072's rollout guard was **docs-only** for the design phase. The normalized
-chat and attempt-accounting capabilities described below are now implemented
-behind `AgentHarness::Api::ChatTransport`. Other capabilities remain design contracts and each still needs
-its own failing-first contract tests, implementation, release evidence, and
-downstream adoption evidence before a caller enables it.
+chat, attempt-accounting, schema, and embedding capabilities described below
+are now implemented. The resumable-loop delegation evaluation is also
+complete: it closed with a **retained-loop outcome** (see "Resumable-loop
+delegation evaluation"), so no loop runtime was added. Other capabilities
+remain design contracts and each still needs its own failing-first contract
+tests, implementation, release evidence, and downstream adoption evidence
+before a caller enables it.
 
 Existing CLI and subscription behavior remains the default. Existing
 `TextTransport`, `OpenAICompatibleTransport`, `Conversation`, and `Response`
@@ -70,11 +73,13 @@ Responses API must select `:chat_completions`; the transport never probes and
 silently switches protocols. Only `authentication_mode: :api_key` is currently
 supported.
 
-Credentials, endpoint, custom headers, timeout, and RubyLLM configuration are
-isolated with a request-local `RubyLLM::Context`. RubyLLM middleware retries
-are disabled; `retry.max_attempts` is the total physical-attempt limit owned by
-the harness. Authentication headers cannot be overridden by custom headers.
-`max_output_tokens` is forwarded without changing it.
+Credentials, endpoint, custom headers, read timeout, and RubyLLM configuration
+are isolated with a request-local `RubyLLM::Context`. Only
+`timeout.read_seconds` is supported; request-local connection timeouts are not.
+RubyLLM middleware retries are disabled; `retry.max_attempts` is the total
+physical-attempt limit owned by the harness. Authentication headers cannot be
+overridden by custom headers. `max_output_tokens` is forwarded without changing
+it.
 
 Unknown model IDs are allowed only because a complete provider and protocol
 are explicit in every candidate (`assume_model_exists: true` in the RubyLLM
@@ -84,6 +89,41 @@ endpoints retain the selected provider's wire protocol and authentication
 shape. Custom provider types, authentication modes, media content, and
 automatic protocol discovery are not supported by this capability.
 
+## Shipped schema-constrained response surface
+
+The same transport accepts `operation: :schema` with a JSON Schema and an
+optional name:
+
+```ruby
+result = transport.call(request.merge(
+  operation: :schema,
+  schema_name: "person",
+  schema: {
+    type: "object",
+    properties: {name: {type: "string"}, age: {type: "integer"}},
+    required: %w[name age],
+    additionalProperties: false
+  }
+))
+
+result[:content] # => '{"name":"Ada","age":37}'
+result[:parsed]  # => {"name" => "Ada", "age" => 37}
+```
+
+Schema operations use provider-native JSON Schema output and the same verified
+Anthropic Messages, OpenAI Responses, and OpenAI Chat Completions scopes as
+normalized chat. Provider adapters infer strictness from the schema; callers
+using a `{schema:, strict:}` envelope can explicitly select strictness.
+`schema_mode: :json_schema` is the only supported mode. JSON-only mode returns
+`unsupported/structured_output_not_supported`.
+
+The harness parses the exact provider text and validates it locally. It does
+not remove Markdown fences or repair malformed JSON. Invalid JSON, schema
+mismatch, refusal, and output-limit truncation return non-retryable
+`invalid_response` outcomes with codes `invalid_json`, `invalid_schema`,
+`refusal`, and `truncated_output`. They retain the original `content` and leave
+`parsed` as `nil`.
+
 ## Ownership boundary
 
 AgentHarness owns protocol translation and one bounded provider request:
@@ -139,7 +179,7 @@ request = {
     }
   ],
   fallback: {on_error_categories: [:transient]},
-  timeout: {connect_seconds: 5, read_seconds: 60},
+  timeout: {read_seconds: 60},
   retry: {max_attempts: 3, base_delay_seconds: 0.25, max_delay_seconds: 2},
   cancellation: cancellation_token,
   metadata: {tenant_id: "tenant-123", workflow_id: "workflow-456"}
@@ -160,9 +200,11 @@ part of a serializable request document.
 gets a distinct `attempt_id`. Redelivering an already reported attempt retains
 its `attempt_id`; initiating another outbound request does not.
 
-Credentials, endpoint, headers, timeouts, retry limits, and cancellation are
-request-local. Implementations MUST prevent concurrent requests from observing
-one another's credentials or headers. They MUST reject reserved header
+Credentials, endpoint, headers, the read timeout, retry limits, and cancellation
+are request-local. Only `timeout.read_seconds` is supported; supplying a
+connection timeout returns `unsupported/unsupported_capability` before any
+provider request. Implementations MUST prevent concurrent requests from
+observing one another's credentials or headers. They MUST reject reserved header
 overrides that would conflict with the selected protocol's authentication.
 Logs and errors MUST NOT contain credentials, authorization headers, message
 bodies, tool arguments, or full provider responses.
@@ -504,7 +546,7 @@ records.
 | Contract area | RubyLLM 2.0 mapping | Decision or gap |
 | --- | --- | --- |
 | Chat/protocols | `RubyLLM.chat`, messages, tools, stream callbacks | Candidate adapter; normalize all values and errors |
-| Schema | `with_schema`, `response.parsed`, model capability registry | Candidate; distinguish enforced schema from JSON mode |
+| Schema | `with_schema`; harness JSON parsing and validation | Implemented for the verified chat scopes; JSON-only mode stays distinct |
 | Embeddings | `RubyLLM.embed` and normalized vectors/usage | Candidate first capability; persistence remains in Paid |
 | Custom headers | `with_headers` | Candidate; contract-test merging and secret redaction |
 | Endpoint/credentials | provider configuration | Global mutable configuration is unsuitable; require request-local isolation or an upstream-supported client boundary |
@@ -514,6 +556,7 @@ records.
 | Attempt usage | `usage.ruby_llm` per physical attempt | Useful facts, but the public payload has no stable attempt ID; harness must add one |
 | Plain Ruby resume | transcript can be reconstructed manually | No documented state export/import API; implement normalized export/import outside RubyLLM |
 | Rails resume | `acts_as_chat` transcript plus supporting records | Technically restart-safe at checkpoints; at-least-once side effects remain |
+| Loop controls | `Chat#step`, `#complete`, `#run_tools`, `#approve`, `#deny`, `Tool.requires_approval`, `#cancel` | Evaluated for delegation; retained-loop outcome (see below) |
 
 ### Optional Rails supporting tables
 
@@ -552,14 +595,96 @@ historical/pending-conversation tests. Reverting the gem is not a data rollback.
 2. **Use RubyLLM tool-call and usage tables selectively.** Potentially removes
    bookkeeping, but only after stable-attempt mapping, tenant-scoped access,
    audit, and migration tests are complete.
-3. **Delegate the full loop and all supporting tables.** Not recommended now.
-   It does not yet demonstrate reduced maintenance across both repositories and
-   increases migration and recovery coupling.
+3. **Delegate the full loop and all supporting tables.** Evaluated and
+   rejected for now: the delegation review below found maintenance increases
+   across both repositories. Retaining Paid's loop over the normalized
+   transport is the completed outcome and creates no future delegation
+   obligation.
 
 The state investigation is therefore positive for checkpoint-based Rails
 restoration and normalized plain Ruby reconstruction, and negative for
 exactly-once recovery or an off-the-shelf plain Ruby export/import mechanism.
 
+## Resumable-loop delegation evaluation
+
+RDR-072 permits delegating the chat loop only when behavior is preserved and
+maintenance decreases across both repositories, counting adapters,
+persistence, and recovery code. Retaining Paid's loop over the normalized
+transport is an acceptable completed outcome. This section records the
+evaluation against RubyLLM 2.0.0's public loop controls
+(`Chat#step`, `#complete`, `#run_tools`, `#complete?`, `#awaiting_approval?`,
+`#pending_approvals`, `#approve`, `#deny`, `#cancel`, and
+`Tool.requires_approval`) and its conclusion.
+
+### Verified behaviors
+
+`spec/ruby_llm_loop_delegation_evaluation_spec.rb` drives a real
+`RubyLLM::Chat` through the public API with stubbed Anthropic Messages
+responses and pins each fact with a contract test:
+
+| Required behavior | Result | Evidence |
+| --- | --- | --- |
+| Single-step execution | Verified | `#step` advances one move; `#complete?` flips only on a final answer |
+| Mixed read/write batches | Verified | Reads without approval execute; `requires_approval` writes stay pending (`#awaiting_approval?`) |
+| Multiple pending decisions | Verified | `#pending_approvals` lists every undecided write; `#approve`/`#deny` resolve them independently |
+| Denial | Verified | A denied call receives a structured denial result and the model continues |
+| Completion | Verified | `#complete?` reports the terminal state |
+| Cancellation | Verified | `#cancel` raises `CancelledError` at the next loop checkpoint, then clears |
+| Completed tool results preserved | Verified | `#run_tools` skips calls that already carry results, so a resumed round executes only the remainder |
+
+### Gaps that fail the delegation criteria
+
+| Criterion | Finding | Evidence |
+| --- | --- | --- |
+| Stable tool IDs | `ToolCall#id` is the provider wire id verbatim; no caller-stable identity is generated, so the contract's "provider_id is not a durable application identifier" rule needs a new mapping adapter | provider-id example |
+| Iteration limits | The loop has no bound: `#complete` steps until `#complete?` or `#awaiting_approval?`, and `tool_options` exposes only choice/calls/concurrency; bounding stays caller-owned | unbounded-iteration example |
+| Restart restoration without Rails | Decisions recorded with `#approve` live in per-chat in-memory state; a plain Ruby chat rebuilt from the same messages is awaiting approval again. Durable decisions require `acts_as_chat` (Active Record), which plain Ruby consumers must not require | reconstruction example |
+| Crash recovery | Same as above: checkpoint restoration is Rails-only, so a plain Ruby consumer re-implements decision and transcript persistence | reconstruction example |
+| Runner-isolated tool execution | `Tool#execute` runs in the calling process; the loop has no dispatch boundary. Paid executes tools in runner processes under caller-owned authorization and atomic claims, so delegation needs new dispatch adapters rather than removing code | in-process execution throughout |
+| Bounded, non-nested retries | Each loop `#generate` retries beneath the caller through Faraday middleware by default (`max_retries` defaults to three), violating the single-retry-owner and shared `max_attempts` budget rules; even with retries disabled, per-step budget/fallback sequencing is absent | hidden-retry example |
+| Attempt accounting | `usage.ruby_llm` payloads carry only operation/provider/model/status/tokens/cost — no attempt or request identity — so the harness `AttemptReport` ledger with stable `attempt_id` would be bypassed | usage-payload example |
+
+### Maintenance comparison
+
+Delegating the loop would remove Paid's turn-taking mechanics (step dispatch
+and approve/deny plumbing) while keeping Paid's authorization, atomic claims,
+runner dispatch, iteration bounds, workflow recovery, durable accounting, and
+cross-process cancellation. It would add, across the two repositories:
+
+- a provider-id to stable-id mapping adapter;
+- in-process `Tool#execute` to runner-process dispatch adapters;
+- a per-step retry, shared-budget, and fallback sequencing wrapper;
+- a caller-side iteration-limit driver replacing `#complete`;
+- plain Ruby decision/transcript persistence and export/import (Rails
+  `acts_as_chat` cannot be a dependency); and
+- attempt-accounting extraction with harness-side attempt identity.
+
+The additions exceed the removals, so maintenance increases across both
+repositories and the criteria fail.
+
+### Outcome
+
+**Retained loop.** Paid keeps its conversation loop over
+`Api::ChatTransport`; the harness keeps one normalized response per call
+with bounded retries, fallback, cancellation, and the attempt ledger. Tool
+execution, approval decisions, authorization, atomic claims, iteration
+limits, and crash recovery remain caller responsibilities under the existing
+ownership boundary. No loop runtime, adapter, persistence, or release was
+added, so there is no migration cost and no consumer activation step. This
+closes the evaluation without creating a future delegation obligation.
+
+Re-evaluate only if RubyLLM later documents a public, plain-Ruby-resumable
+loop with caller-stable tool identity, an external tool-dispatch boundary, a
+bounded `#complete`, and per-attempt accounting identity; the evaluation
+spec's examples are the tripwire that detects such changes on upgrade.
+
+### Communication
+
+The retained-loop outcome, the behavior-test evidence above, and the zero
+migration cost are the message for the Paid adoption and closeout tracking
+(viamin/paid#4014 and the parent viamin/agent-harness#430). No scope beyond
+the accepted RDR-072 alternatives is requested or implied.
+
 ## Current harness gaps and incremental delivery
 
 The current transports already normalize basic text, tool calls, token totals,
@@ -576,7 +701,8 @@ Ship capabilities independently in this order:
 3. normalized non-streaming and streaming chat transport;
 4. structured output for verified model/protocol combinations;
 5. plain Ruby state round trips; and
-6. optional Rails persistence evaluation, then loop evaluation.
+6. optional Rails persistence evaluation, then loop evaluation (closed with
+   the retained-loop outcome recorded above; no runtime change).
 
 Each implementation issue starts with failing contract tests for request-local
 credential isolation, custom endpoints/headers, unsupported outcomes,
@@ -607,11 +733,43 @@ types stay behind the harness boundary.
   embeddings, and token trackers remain outside this ledger and require
   separate migration issues.
 
+### Schema capability release evidence
+
+- Publication: unreleased; record the first installable version before
+  downstream adoption.
+- Verified scopes: `:schema` with Anthropic Messages, OpenAI Responses, and
+  OpenAI Chat Completions using API-key authentication, including compatible
+  endpoints that explicitly select Chat Completions.
+- Contract coverage: valid and required-field schemas, classified failures,
+  bounded retries, cancellation, refusal, truncation, malformed JSON, and
+  schema mismatch.
+- Retained paths: CLI and subscription execution remain on existing provider
+  interfaces. JSON-only mode and model-specific capability discovery are not
+  migrated.
+
+### Loop-delegation evaluation evidence (retained loop)
+
+- Publication: none required. The evaluation changed documentation and tests
+  only; no runtime path, adapter, table, or public API was added or altered,
+  so every released agent-harness version already carries the outcome.
+- Verification: `spec/ruby_llm_loop_delegation_evaluation_spec.rb` exercises
+  only the public RubyLLM 2.0 API (pinned `ruby_llm = 2.0.0`) against stubbed
+  Anthropic Messages responses and runs with the upstream suite and lint on
+  the repository's supported Ruby environment. It records the verified loop
+  behaviors and the gaps that failed the delegation criteria.
+- Migration: none. Paid and other consumers keep their existing loops over
+  `Api::ChatTransport`; no consumer activation, feature flag, or rollout step
+  applies.
+- Retained responsibilities: documented in "Resumable-loop delegation
+  evaluation" above; loop ownership stays with Paid under RDR-072's accepted
+  alternatives.
+
 AgentHarness currently supports Ruby 3.2 and later and must remain usable as a
-plain Ruby gem. RubyLLM 2.0.0 itself supports Ruby 3.1.3 and later, but adding it
-would introduce Faraday, event-stream parsing, Schematist, Marcel, and Zeitwerk
-runtime dependencies. A capability issue must measure and publish the resolved
-dependency set and test the harness minimum Ruby version before adoption.
+plain Ruby gem. RubyLLM 2.0.0 itself supports Ruby 3.1.3 and later and adds
+Faraday, event-stream parsing, Schematist, Marcel, and Zeitwerk runtime
+dependencies. Local response validation adds `json_schemer` and its bounded
+dependency set. A capability issue must test the harness minimum Ruby version
+before adoption.
 
 Rails and Active Record remain optional. Requiring `agent_harness` in a process
 without Rails MUST NOT load Active Record, connect to a database, or require
@@ -629,6 +787,7 @@ For every capability, release evidence must name:
 - retained paths and follow-up issues for combinations not migrated.
 
 Paid issue `viamin/paid#4014` should receive this compatibility result, the
-state-restoration conclusion, the stable-ID gap, and the per-capability release
-evidence. Downstream adoption cannot proceed from this design issue closing or
+state-restoration conclusion, the stable-ID gap, the per-capability release
+evidence, and the retained-loop outcome of the delegation evaluation.
+Downstream adoption cannot proceed from this design issue closing or
 from a Git tag alone.

data/lib/agent_harness/api/chat_transport.rb

@@ -4,6 +4,7 @@ require "securerandom"
 require "ruby_llm"
 require_relative "attempt_report"
 require_relative "ruby_llm_chat_adapter"
+require_relative "schema_response"
 
 module AgentHarness
   module Api
@@ -54,6 +55,7 @@ module AgentHarness
         end
 
         def call
+          return unsupported_schema_result unless schema_mode_supported?
           return cancelled_result unless active?
 
           candidates.each_with_index do |candidate, candidate_index|
@@ -95,6 +97,7 @@ module AgentHarness
             max_output_tokens: request[:max_output_tokens],
             temperature: request[:temperature],
             timeout: request[:timeout],
+            schema: schema_payload,
             on_accounting: ->(facts) { accounting = facts },
             provider_usage: provider_usage
           )
@@ -113,6 +116,7 @@ module AgentHarness
             stream: request[:stream] == true,
             timeout: request[:timeout],
             cancellation: request[:cancellation],
+            schema: schema_payload,
             on_accounting: ->(facts) { accounting = facts },
             prepared_chat: prepared_chat
           ) do |event|
@@ -138,19 +142,29 @@ module AgentHarness
 
         def success(candidate, attempt_id, started_at, adapter_result, accounting)
           accounting ||= {usage: adapter_result[:usage], provider_reported: !adapter_result[:usage].nil?}
-          append_attempt(candidate, attempt_id, started_at, :succeeded, **accounting)
+          schema_result = normalize_schema_response(adapter_result)
+          status = schema_result[:error] ? :failed : :succeeded
+          append_attempt(candidate, attempt_id, started_at, status, error: schema_result[:error], **accounting)
           result = base_result(candidate).merge(
-            status: :succeeded,
-            content: adapter_result[:content] || "",
+            status: status,
+            content: schema_result[:content],
+            parsed: schema_result[:parsed],
             tool_calls: normalize_tool_calls(adapter_result[:tool_calls]),
             finish_reason: adapter_result[:finish_reason],
             usage: aggregate_usage,
-            error: nil
+            error: schema_result[:error]
           )
-          emit(:response_completed, attempt_id:, result: result)
+          event = (status == :succeeded) ? :response_completed : :response_failed
+          emit(event, attempt_id:, result: result)
           result
         end
 
+        def normalize_schema_response(adapter_result)
+          return {content: adapter_result[:content] || "", parsed: nil, error: nil} unless schema_operation?
+
+          SchemaResponse.new(schema_definition).call(adapter_result)
+        end
+
         def failure(candidate, attempt_id, started_at, error, partial:, accounting:)
           status = failure_status(error, partial)
           error = error.merge(retryable: false) if partial
@@ -343,14 +357,48 @@ module AgentHarness
         end
 
         def validate!
-          raise ArgumentError, "operation must be :chat" unless request[:operation]&.to_sym == :chat
+          unless %i[chat schema].include?(request[:operation]&.to_sym)
+            raise ArgumentError, "operation must be :chat or :schema"
+          end
           raise ArgumentError, "request_id is required" if request[:request_id].to_s.empty?
           raise ArgumentError, "candidates must not be empty" if !request[:candidates].is_a?(Array) || request[:candidates].empty?
           raise ArgumentError, "messages must be an array" unless request[:messages].is_a?(Array)
+          raise ArgumentError, "schema is required for a schema operation" if schema_operation? && !request[:schema].is_a?(Hash)
           validate_attempt_limit!
           candidates.each { |candidate| validate_candidate!(candidate) }
         end
 
+        def schema_operation?
+          request[:operation].to_sym == :schema
+        end
+
+        def schema_mode_supported?
+          !schema_operation? || !request[:schema_mode] || request[:schema_mode].to_sym == :json_schema
+        end
+
+        def schema_payload
+          return unless schema_operation?
+
+          raw = request[:schema]
+          return raw.merge(name: request[:schema_name] || raw[:name]) if raw[:schema]
+
+          {name: request[:schema_name] || raw[:title] || "response", schema: raw}
+        end
+
+        def schema_definition
+          schema_payload.fetch(:schema)
+        end
+
+        def unsupported_schema_result
+          error = {
+            category: :unsupported,
+            code: :structured_output_not_supported,
+            retryable: false,
+            message: "Schema request failed (unsupported/structured_output_not_supported)"
+          }
+          failed_result(error, candidates.first)
+        end
+
         def validate_attempt_limit!
           limit = retry_config[:max_attempts]
           raise ArgumentError, "retry.max_attempts must be a positive integer" unless limit.is_a?(Integer) && limit.positive?

data/lib/agent_harness/api/ruby_llm_chat_adapter.rb

@@ -5,6 +5,22 @@ require "ruby_llm"
 
 module AgentHarness
   module Api
+    # RubyLLM 2.0.0 flattens Responses API refusal deltas into ordinary text
+    # chunks and offers no public access to their event type. The gemspec pins
+    # that exact release so this compatibility shim cannot silently outlive the
+    # private parser shape it targets. Remove it when RubyLLM exposes refusals.
+    module RubyLlmResponsesStreamingRefusal
+      module RefusalChunk; end
+
+      def build_chunk(data)
+        super.tap do |chunk|
+          chunk.extend(RefusalChunk) if data["type"] == "response.refusal.delta"
+        end
+      end
+    end
+
+    RubyLLM::Protocols::Responses.prepend(RubyLlmResponsesStreamingRefusal)
+
     # Translates the normalized public chat values to RubyLLM public objects.
     class RubyLlmChatAdapter
       class UnsupportedOptionError < StandardError; end
@@ -18,24 +34,24 @@ module AgentHarness
       OPENAI_UNSUPPLIED_CONFIG = %i[openai_organization_id openai_project_id openai_use_system_role].freeze
 
       def call(candidate:, messages:, tools:, max_output_tokens:, temperature:, stream:, timeout:, cancellation:,
-        on_accounting: nil, prepared_chat: nil, &on_event)
+        schema: nil, on_accounting: nil, prepared_chat: nil, &on_event)
         provider_usage_reported = false
         chat = prepared_chat || prepare(candidate:, messages:, tools:, max_output_tokens:, temperature:, timeout:,
-          on_accounting:, provider_usage: -> { provider_usage_reported })
-        response = generate(chat, stream, cancellation) do |event|
+          schema:, on_accounting:, provider_usage: -> { provider_usage_reported })
+        response, streamed_refusal = generate(chat, stream, cancellation) do |event|
           provider_usage_reported = true if event[:type] == :usage_updated
           on_event&.call(event)
         end
         emit_completed_tool_calls(response, &on_event) if stream
-        normalize_response(response)
+        normalize_response(response, streamed_refusal: streamed_refusal)
       end
 
       def prepare(candidate:, messages:, tools:, max_output_tokens:, temperature:, timeout:, on_accounting: nil,
-        provider_usage: -> { false })
+        schema: nil, provider_usage: -> { false })
         context = build_context(candidate, timeout, on_accounting, provider_usage)
         context.chat(model: candidate[:model], provider: candidate[:provider], protocol: ruby_llm_protocol(candidate),
           assume_model_exists: true).tap do |chat|
-          configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature)
+          configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature, schema)
         end
       end
 
@@ -45,6 +61,7 @@ module AgentHarness
       # duplicate cumulative reports are not re-emitted.
       class StreamState
         attr_accessor :input_tokens, :output_tokens
+        attr_reader :refusal
 
         def initialize
           @provider_id_by_key = {}
@@ -52,6 +69,7 @@ module AgentHarness
           @latest_provider_id = nil
           @input_tokens = nil
           @output_tokens = nil
+          @refusal = false
         end
 
         # Links a stream chunk key to its provider call id, returning true
@@ -72,6 +90,10 @@ module AgentHarness
           total = (input_tokens + output_tokens) if input_tokens && output_tokens
           {input_tokens: input_tokens, output_tokens: output_tokens, total_tokens: total}
         end
+
+        def observe(chunk)
+          @refusal ||= chunk.is_a?(RubyLlmResponsesStreamingRefusal::RefusalChunk)
+        end
       end
 
       private
@@ -150,19 +172,20 @@ module AgentHarness
         config.request_timeout = timeout&.dig(:read_seconds) || DEFAULT_REQUEST_TIMEOUT
       end
 
-      def configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature)
+      def configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature, schema)
         chat.messages = normalize_messages(messages)
         chat.with_tools(tools.map { |tool| normalized_tool(tool) }) unless tools.empty?
         chat.with_headers(candidate[:headers] || {})
         chat.with_max_output_tokens(max_output_tokens) if max_output_tokens
         chat.with_temperature(temperature) unless temperature.nil?
+        chat.with_schema(schema) if schema
       end
 
       def generate(chat, stream, cancellation)
-        return generate_without_events(chat, cancellation) unless stream
+        return [generate_without_events(chat, cancellation), false] unless stream
 
         state = StreamState.new
-        chat.generate do |chunk|
+        response = chat.generate do |chunk|
           if cancelled?(cancellation)
             chat.cancel
             raise RubyLLM::CancelledError
@@ -170,6 +193,7 @@ module AgentHarness
 
           stream_events(chunk, state).each { |event| yield event }
         end
+        [response, state.refusal]
       end
 
       def generate_without_events(chat, cancellation)
@@ -235,6 +259,7 @@ module AgentHarness
       end
 
       def stream_events(chunk, state)
+        state.observe(chunk)
         [text_event(chunk), *tool_call_events(chunk, state), usage_event(chunk, state)].compact
       end
 
@@ -300,16 +325,46 @@ module AgentHarness
         end
       end
 
-      def normalize_response(response)
+      def normalize_response(response, streamed_refusal: false)
         {
           content: response.content || "",
           model: response.model,
           finish_reason: response.finish_reason,
           usage: normalize_usage(response.tokens),
-          tool_calls: Array(response.tool_calls&.values).map { |call| normalize_tool_call(call) }
+          tool_calls: Array(response.tool_calls&.values).map { |call| normalize_tool_call(call) },
+          refusal: streamed_refusal || refusal?(response)
         }
       end
 
+      def refusal?(response)
+        return true if response.finish_reason == :content_filter
+        return false unless response.respond_to?(:raw)
+
+        body = response.raw&.body
+        return false unless body.is_a?(Hash)
+
+        responses_api_refusal?(body) || chat_completions_refusal?(body)
+      end
+
+      def responses_api_refusal?(body)
+        Array(body["output"] || body[:output]).any? do |item|
+          next false unless item.is_a?(Hash)
+
+          Array(item["content"] || item[:content]).any? do |part|
+            part.is_a?(Hash) && (part["type"] || part[:type]) == "refusal"
+          end
+        end
+      end
+
+      def chat_completions_refusal?(body)
+        Array(body["choices"] || body[:choices]).any? do |choice|
+          next false unless choice.is_a?(Hash)
+
+          message = choice["message"] || choice[:message]
+          message.is_a?(Hash) && !(message["refusal"] || message[:refusal]).nil?
+        end
+      end
+
       def normalize_tool_call(call)
         {provider_id: call.id, name: call.name, arguments_json: JSON.generate(call.arguments || {})}
       end

data/lib/agent_harness/api/schema_response.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+require "json"
+require "json_schemer"
+
+module AgentHarness
+  module Api
+    # Parses and validates one schema-constrained provider response.
+    class SchemaResponse
+      def initialize(schema)
+        @validator = JSONSchemer.schema(schema)
+      end
+
+      def call(response)
+        content = response[:content].to_s
+        return failure(content, :refusal) if response[:refusal]
+        return failure(content, :truncated_output) if response[:finish_reason]&.to_sym == :max_tokens
+
+        parsed = JSON.parse(content)
+        return failure(content, :invalid_schema) unless @validator.valid?(parsed)
+
+        {content: content, parsed: parsed, error: nil}
+      rescue JSON::ParserError
+        failure(content, :invalid_json)
+      end
+
+      private
+
+      def failure(content, code)
+        {
+          content: content,
+          parsed: nil,
+          error: {
+            category: :invalid_response,
+            code: code,
+            retryable: false,
+            message: "Schema response failed (invalid_response/#{code})"
+          }
+        }
+      end
+    end
+  end
+end

data/lib/agent_harness/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module AgentHarness
-  VERSION = "0.42.0"
+  VERSION = "0.44.0"
 end

data/lib/agent_harness.rb

@@ -473,6 +473,7 @@ require_relative "agent_harness/openai_compatible_transport"
 require_relative "agent_harness/conversation"
 require_relative "agent_harness/api/attempt_report"
 require_relative "agent_harness/api/chat_transport"
+require_relative "agent_harness/api/schema_response"
 require_relative "agent_harness/quota_status"
 require_relative "agent_harness/authentication"
 require_relative "agent_harness/provider_health_check"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: agent-harness
 version: !ruby/object:Gem::Version
-  version: 0.42.0
+  version: 0.44.0
 platform: ruby
 authors:
 - Bart Agapinan
@@ -30,25 +30,33 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '2.0'
 - !ruby/object:Gem::Dependency
-  name: ruby_llm
+  name: json_schemer
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '2.0'
-    - - "<"
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '2.4'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.0'
-    - - "<"
+        version: '2.4'
+- !ruby/object:Gem::Dependency
+  name: ruby_llm
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: 2.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 2.0.0
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -138,6 +146,7 @@ files:
 - lib/agent_harness/api/attempt_report.rb
 - lib/agent_harness/api/chat_transport.rb
 - lib/agent_harness/api/ruby_llm_chat_adapter.rb
+- lib/agent_harness/api/schema_response.rb
 - lib/agent_harness/authentication.rb
 - lib/agent_harness/command_executor.rb
 - lib/agent_harness/configuration.rb