agent-harness 0.42.0 → 0.43.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9c2a3cf60957839163bbda4f4e202f663ee68915e49ba3853abce85b5f6f16e9
4
- data.tar.gz: b5a9950414fc72fe2626abbb3ef50275e2e28267346414d04d330d3ea37c1297
3
+ metadata.gz: 40d84099230c75237b23dca08330a46b739029768ce345b57cb232ba0e66a935
4
+ data.tar.gz: 47d36a8e160bcd29f9d50f6eabc1d0ded6795d6e05f2e45e93038d1987cd352c
5
5
  SHA512:
6
- metadata.gz: b6147210553af7975d99dbf53cc52c0b37e7ba25c8424a9cfb24a290bae9172d25bcbc8a6e5b489d1b2ae363251facc756ea0eaae55a6efd3904cdb64fcda025
7
- data.tar.gz: f869749041ff450642b8ba940a3f0f9f8ea2cd7a3d788b2414ae7f0115dcb5fde4dcb7f5aa725b4e1ba2216568a1412e2bf5439c47b61bf75b2d06390faebfae
6
+ metadata.gz: f059d918fed14691fe444dba2e5e93aaeb2cdb185785a580b8a7341db8198616a0a830fd43461b4dfceffc9258f71ce25a8cbb69351747af46980ba0bd783757
7
+ data.tar.gz: 2f09db993da48a0ca0ed852eee9653898f474e7c61a6e0603dde3588457d226c8f51de71a8b322e7eba46b7e64203cc61a7607aa31764ed63188ef8b38f7c1e5
@@ -1,3 +1,3 @@
1
1
  {
2
- ".": "0.42.0"
2
+ ".": "0.43.0"
3
3
  }
data/CHANGELOG.md CHANGED
@@ -2,9 +2,17 @@
2
2
 
3
3
  ### Features
4
4
 
5
+ * expose normalized API chat transport and schema-constrained responses with preserved JSON text, locally validated parsed values, and explicit refusal, truncation, invalid JSON, schema mismatch, and unsupported-mode outcomes; this combined delivery supersedes the separate chat-transport work item ([#433](https://github.com/viamin/agent-harness/issues/433), [#434](https://github.com/viamin/agent-harness/issues/434)).
5
6
  * add runner model compatibility contract (`AgentHarness.model_compatibility`) with structured `ModelCompatibility::Result` outcomes. Codex exposes static facts for CLI-gated models (e.g. `gpt-5.5` requires Codex CLI `>= 0.116.0`), a baseline supported-model list, supported auth modes, and a `DEFAULT_COMPATIBLE_MODEL_ID` fallback so downstream orchestrators can validate tier/model assignments before scheduling agent runs ([#259](https://github.com/viamin/agent-harness/issues/259)).
6
7
  * **auth:** add provider-owned PKCE code-exchange API for Claude OAuth (`AgentHarness::Authentication.exchange_code`). Takes an authorization code plus PKCE verifier (and `redirect_uri`/`client_id`), posts an `authorization_code` grant to the Claude token endpoint, and persists the resulting access/refresh tokens in the native `claudeAiOauth` shape. Adds `exchange_code_supported?` and a `code_exchange` key to `auth_capabilities` ([#266](https://github.com/viamin/agent-harness/issues/266)).
7
8
 
9
+ ## [0.43.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.42.0...agent-harness/v0.43.0) (2026-09-25)
10
+
11
+
12
+ ### Features
13
+
14
+ * Expose Schema-Constrained Parsed Responses (RDR-072) ([#442](https://github.com/viamin/agent-harness/issues/442)) ([3d05cfa](https://github.com/viamin/agent-harness/commit/3d05cfac68624c218eda5f6f9871a3a4d1044ec0))
15
+
8
16
  ## [0.42.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.41.0...agent-harness/v0.42.0) (2026-09-25)
9
17
 
10
18
 
@@ -8,8 +8,8 @@ usage, and optional conversation-persistence work.
8
8
  ## Status and rollout boundary
9
9
 
10
10
  RDR-072's rollout guard was **docs-only** for the design phase. The normalized
11
- chat and attempt-accounting capabilities described below are now implemented
12
- behind `AgentHarness::Api::ChatTransport`. Other capabilities remain design contracts and each still needs
11
+ chat, attempt-accounting, schema, and embedding capabilities described below
12
+ are now implemented. Other capabilities remain design contracts and each still needs
13
13
  its own failing-first contract tests, implementation, release evidence, and
14
14
  downstream adoption evidence before a caller enables it.
15
15
 
@@ -70,11 +70,13 @@ Responses API must select `:chat_completions`; the transport never probes and
70
70
  silently switches protocols. Only `authentication_mode: :api_key` is currently
71
71
  supported.
72
72
 
73
- Credentials, endpoint, custom headers, timeout, and RubyLLM configuration are
74
- isolated with a request-local `RubyLLM::Context`. RubyLLM middleware retries
75
- are disabled; `retry.max_attempts` is the total physical-attempt limit owned by
76
- the harness. Authentication headers cannot be overridden by custom headers.
77
- `max_output_tokens` is forwarded without changing it.
73
+ Credentials, endpoint, custom headers, read timeout, and RubyLLM configuration
74
+ are isolated with a request-local `RubyLLM::Context`. Only
75
+ `timeout.read_seconds` is supported; request-local connection timeouts are not.
76
+ RubyLLM middleware retries are disabled; `retry.max_attempts` is the total
77
+ physical-attempt limit owned by the harness. Authentication headers cannot be
78
+ overridden by custom headers. `max_output_tokens` is forwarded without changing
79
+ it.
78
80
 
79
81
  Unknown model IDs are allowed only because a complete provider and protocol
80
82
  are explicit in every candidate (`assume_model_exists: true` in the RubyLLM
@@ -84,6 +86,41 @@ endpoints retain the selected provider's wire protocol and authentication
84
86
  shape. Custom provider types, authentication modes, media content, and
85
87
  automatic protocol discovery are not supported by this capability.
86
88
 
89
+ ## Shipped schema-constrained response surface
90
+
91
+ The same transport accepts `operation: :schema` with a JSON Schema and an
92
+ optional name:
93
+
94
+ ```ruby
95
+ result = transport.call(request.merge(
96
+ operation: :schema,
97
+ schema_name: "person",
98
+ schema: {
99
+ type: "object",
100
+ properties: {name: {type: "string"}, age: {type: "integer"}},
101
+ required: %w[name age],
102
+ additionalProperties: false
103
+ }
104
+ ))
105
+
106
+ result[:content] # => '{"name":"Ada","age":37}'
107
+ result[:parsed] # => {"name" => "Ada", "age" => 37}
108
+ ```
109
+
110
+ Schema operations use provider-native JSON Schema output and the same verified
111
+ Anthropic Messages, OpenAI Responses, and OpenAI Chat Completions scopes as
112
+ normalized chat. Provider adapters infer strictness from the schema; callers
113
+ using a `{schema:, strict:}` envelope can explicitly select strictness.
114
+ `schema_mode: :json_schema` is the only supported mode. JSON-only mode returns
115
+ `unsupported/structured_output_not_supported`.
116
+
117
+ The harness parses the exact provider text and validates it locally. It does
118
+ not remove Markdown fences or repair malformed JSON. Invalid JSON, schema
119
+ mismatch, refusal, and output-limit truncation return non-retryable
120
+ `invalid_response` outcomes with codes `invalid_json`, `invalid_schema`,
121
+ `refusal`, and `truncated_output`. They retain the original `content` and leave
122
+ `parsed` as `nil`.
123
+
87
124
  ## Ownership boundary
88
125
 
89
126
  AgentHarness owns protocol translation and one bounded provider request:
@@ -139,7 +176,7 @@ request = {
139
176
  }
140
177
  ],
141
178
  fallback: {on_error_categories: [:transient]},
142
- timeout: {connect_seconds: 5, read_seconds: 60},
179
+ timeout: {read_seconds: 60},
143
180
  retry: {max_attempts: 3, base_delay_seconds: 0.25, max_delay_seconds: 2},
144
181
  cancellation: cancellation_token,
145
182
  metadata: {tenant_id: "tenant-123", workflow_id: "workflow-456"}
@@ -160,9 +197,11 @@ part of a serializable request document.
160
197
  gets a distinct `attempt_id`. Redelivering an already reported attempt retains
161
198
  its `attempt_id`; initiating another outbound request does not.
162
199
 
163
- Credentials, endpoint, headers, timeouts, retry limits, and cancellation are
164
- request-local. Implementations MUST prevent concurrent requests from observing
165
- one another's credentials or headers. They MUST reject reserved header
200
+ Credentials, endpoint, headers, the read timeout, retry limits, and cancellation
201
+ are request-local. Only `timeout.read_seconds` is supported; supplying a
202
+ connection timeout returns `unsupported/unsupported_capability` before any
203
+ provider request. Implementations MUST prevent concurrent requests from
204
+ observing one another's credentials or headers. They MUST reject reserved header
166
205
  overrides that would conflict with the selected protocol's authentication.
167
206
  Logs and errors MUST NOT contain credentials, authorization headers, message
168
207
  bodies, tool arguments, or full provider responses.
@@ -504,7 +543,7 @@ records.
504
543
  | Contract area | RubyLLM 2.0 mapping | Decision or gap |
505
544
  | --- | --- | --- |
506
545
  | Chat/protocols | `RubyLLM.chat`, messages, tools, stream callbacks | Candidate adapter; normalize all values and errors |
507
- | Schema | `with_schema`, `response.parsed`, model capability registry | Candidate; distinguish enforced schema from JSON mode |
546
+ | Schema | `with_schema`; harness JSON parsing and validation | Implemented for the verified chat scopes; JSON-only mode stays distinct |
508
547
  | Embeddings | `RubyLLM.embed` and normalized vectors/usage | Candidate first capability; persistence remains in Paid |
509
548
  | Custom headers | `with_headers` | Candidate; contract-test merging and secret redaction |
510
549
  | Endpoint/credentials | provider configuration | Global mutable configuration is unsuitable; require request-local isolation or an upstream-supported client boundary |
@@ -607,11 +646,26 @@ types stay behind the harness boundary.
607
646
  embeddings, and token trackers remain outside this ledger and require
608
647
  separate migration issues.
609
648
 
649
+ ### Schema capability release evidence
650
+
651
+ - Publication: unreleased; record the first installable version before
652
+ downstream adoption.
653
+ - Verified scopes: `:schema` with Anthropic Messages, OpenAI Responses, and
654
+ OpenAI Chat Completions using API-key authentication, including compatible
655
+ endpoints that explicitly select Chat Completions.
656
+ - Contract coverage: valid and required-field schemas, classified failures,
657
+ bounded retries, cancellation, refusal, truncation, malformed JSON, and
658
+ schema mismatch.
659
+ - Retained paths: CLI and subscription execution remain on existing provider
660
+ interfaces. JSON-only mode and model-specific capability discovery are not
661
+ migrated.
662
+
610
663
  AgentHarness currently supports Ruby 3.2 and later and must remain usable as a
611
- plain Ruby gem. RubyLLM 2.0.0 itself supports Ruby 3.1.3 and later, but adding it
612
- would introduce Faraday, event-stream parsing, Schematist, Marcel, and Zeitwerk
613
- runtime dependencies. A capability issue must measure and publish the resolved
614
- dependency set and test the harness minimum Ruby version before adoption.
664
+ plain Ruby gem. RubyLLM 2.0.0 itself supports Ruby 3.1.3 and later and adds
665
+ Faraday, event-stream parsing, Schematist, Marcel, and Zeitwerk runtime
666
+ dependencies. Local response validation adds `json_schemer` and its bounded
667
+ dependency set. A capability issue must test the harness minimum Ruby version
668
+ before adoption.
615
669
 
616
670
  Rails and Active Record remain optional. Requiring `agent_harness` in a process
617
671
  without Rails MUST NOT load Active Record, connect to a database, or require
@@ -4,6 +4,7 @@ require "securerandom"
4
4
  require "ruby_llm"
5
5
  require_relative "attempt_report"
6
6
  require_relative "ruby_llm_chat_adapter"
7
+ require_relative "schema_response"
7
8
 
8
9
  module AgentHarness
9
10
  module Api
@@ -54,6 +55,7 @@ module AgentHarness
54
55
  end
55
56
 
56
57
  def call
58
+ return unsupported_schema_result unless schema_mode_supported?
57
59
  return cancelled_result unless active?
58
60
 
59
61
  candidates.each_with_index do |candidate, candidate_index|
@@ -95,6 +97,7 @@ module AgentHarness
95
97
  max_output_tokens: request[:max_output_tokens],
96
98
  temperature: request[:temperature],
97
99
  timeout: request[:timeout],
100
+ schema: schema_payload,
98
101
  on_accounting: ->(facts) { accounting = facts },
99
102
  provider_usage: provider_usage
100
103
  )
@@ -113,6 +116,7 @@ module AgentHarness
113
116
  stream: request[:stream] == true,
114
117
  timeout: request[:timeout],
115
118
  cancellation: request[:cancellation],
119
+ schema: schema_payload,
116
120
  on_accounting: ->(facts) { accounting = facts },
117
121
  prepared_chat: prepared_chat
118
122
  ) do |event|
@@ -138,19 +142,29 @@ module AgentHarness
138
142
 
139
143
  def success(candidate, attempt_id, started_at, adapter_result, accounting)
140
144
  accounting ||= {usage: adapter_result[:usage], provider_reported: !adapter_result[:usage].nil?}
141
- append_attempt(candidate, attempt_id, started_at, :succeeded, **accounting)
145
+ schema_result = normalize_schema_response(adapter_result)
146
+ status = schema_result[:error] ? :failed : :succeeded
147
+ append_attempt(candidate, attempt_id, started_at, status, error: schema_result[:error], **accounting)
142
148
  result = base_result(candidate).merge(
143
- status: :succeeded,
144
- content: adapter_result[:content] || "",
149
+ status: status,
150
+ content: schema_result[:content],
151
+ parsed: schema_result[:parsed],
145
152
  tool_calls: normalize_tool_calls(adapter_result[:tool_calls]),
146
153
  finish_reason: adapter_result[:finish_reason],
147
154
  usage: aggregate_usage,
148
- error: nil
155
+ error: schema_result[:error]
149
156
  )
150
- emit(:response_completed, attempt_id:, result: result)
157
+ event = (status == :succeeded) ? :response_completed : :response_failed
158
+ emit(event, attempt_id:, result: result)
151
159
  result
152
160
  end
153
161
 
162
+ def normalize_schema_response(adapter_result)
163
+ return {content: adapter_result[:content] || "", parsed: nil, error: nil} unless schema_operation?
164
+
165
+ SchemaResponse.new(schema_definition).call(adapter_result)
166
+ end
167
+
154
168
  def failure(candidate, attempt_id, started_at, error, partial:, accounting:)
155
169
  status = failure_status(error, partial)
156
170
  error = error.merge(retryable: false) if partial
@@ -343,14 +357,48 @@ module AgentHarness
343
357
  end
344
358
 
345
359
  def validate!
346
- raise ArgumentError, "operation must be :chat" unless request[:operation]&.to_sym == :chat
360
+ unless %i[chat schema].include?(request[:operation]&.to_sym)
361
+ raise ArgumentError, "operation must be :chat or :schema"
362
+ end
347
363
  raise ArgumentError, "request_id is required" if request[:request_id].to_s.empty?
348
364
  raise ArgumentError, "candidates must not be empty" if !request[:candidates].is_a?(Array) || request[:candidates].empty?
349
365
  raise ArgumentError, "messages must be an array" unless request[:messages].is_a?(Array)
366
+ raise ArgumentError, "schema is required for a schema operation" if schema_operation? && !request[:schema].is_a?(Hash)
350
367
  validate_attempt_limit!
351
368
  candidates.each { |candidate| validate_candidate!(candidate) }
352
369
  end
353
370
 
371
+ def schema_operation?
372
+ request[:operation].to_sym == :schema
373
+ end
374
+
375
+ def schema_mode_supported?
376
+ !schema_operation? || !request[:schema_mode] || request[:schema_mode].to_sym == :json_schema
377
+ end
378
+
379
+ def schema_payload
380
+ return unless schema_operation?
381
+
382
+ raw = request[:schema]
383
+ return raw.merge(name: request[:schema_name] || raw[:name]) if raw[:schema]
384
+
385
+ {name: request[:schema_name] || raw[:title] || "response", schema: raw}
386
+ end
387
+
388
+ def schema_definition
389
+ schema_payload.fetch(:schema)
390
+ end
391
+
392
+ def unsupported_schema_result
393
+ error = {
394
+ category: :unsupported,
395
+ code: :structured_output_not_supported,
396
+ retryable: false,
397
+ message: "Schema request failed (unsupported/structured_output_not_supported)"
398
+ }
399
+ failed_result(error, candidates.first)
400
+ end
401
+
354
402
  def validate_attempt_limit!
355
403
  limit = retry_config[:max_attempts]
356
404
  raise ArgumentError, "retry.max_attempts must be a positive integer" unless limit.is_a?(Integer) && limit.positive?
@@ -5,6 +5,22 @@ require "ruby_llm"
5
5
 
6
6
  module AgentHarness
7
7
  module Api
8
+ # RubyLLM 2.0.0 flattens Responses API refusal deltas into ordinary text
9
+ # chunks and offers no public access to their event type. The gemspec pins
10
+ # that exact release so this compatibility shim cannot silently outlive the
11
+ # private parser shape it targets. Remove it when RubyLLM exposes refusals.
12
+ module RubyLlmResponsesStreamingRefusal
13
+ module RefusalChunk; end
14
+
15
+ def build_chunk(data)
16
+ super.tap do |chunk|
17
+ chunk.extend(RefusalChunk) if data["type"] == "response.refusal.delta"
18
+ end
19
+ end
20
+ end
21
+
22
+ RubyLLM::Protocols::Responses.prepend(RubyLlmResponsesStreamingRefusal)
23
+
8
24
  # Translates the normalized public chat values to RubyLLM public objects.
9
25
  class RubyLlmChatAdapter
10
26
  class UnsupportedOptionError < StandardError; end
@@ -18,24 +34,24 @@ module AgentHarness
18
34
  OPENAI_UNSUPPLIED_CONFIG = %i[openai_organization_id openai_project_id openai_use_system_role].freeze
19
35
 
20
36
  def call(candidate:, messages:, tools:, max_output_tokens:, temperature:, stream:, timeout:, cancellation:,
21
- on_accounting: nil, prepared_chat: nil, &on_event)
37
+ schema: nil, on_accounting: nil, prepared_chat: nil, &on_event)
22
38
  provider_usage_reported = false
23
39
  chat = prepared_chat || prepare(candidate:, messages:, tools:, max_output_tokens:, temperature:, timeout:,
24
- on_accounting:, provider_usage: -> { provider_usage_reported })
25
- response = generate(chat, stream, cancellation) do |event|
40
+ schema:, on_accounting:, provider_usage: -> { provider_usage_reported })
41
+ response, streamed_refusal = generate(chat, stream, cancellation) do |event|
26
42
  provider_usage_reported = true if event[:type] == :usage_updated
27
43
  on_event&.call(event)
28
44
  end
29
45
  emit_completed_tool_calls(response, &on_event) if stream
30
- normalize_response(response)
46
+ normalize_response(response, streamed_refusal: streamed_refusal)
31
47
  end
32
48
 
33
49
  def prepare(candidate:, messages:, tools:, max_output_tokens:, temperature:, timeout:, on_accounting: nil,
34
- provider_usage: -> { false })
50
+ schema: nil, provider_usage: -> { false })
35
51
  context = build_context(candidate, timeout, on_accounting, provider_usage)
36
52
  context.chat(model: candidate[:model], provider: candidate[:provider], protocol: ruby_llm_protocol(candidate),
37
53
  assume_model_exists: true).tap do |chat|
38
- configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature)
54
+ configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature, schema)
39
55
  end
40
56
  end
41
57
 
@@ -45,6 +61,7 @@ module AgentHarness
45
61
  # duplicate cumulative reports are not re-emitted.
46
62
  class StreamState
47
63
  attr_accessor :input_tokens, :output_tokens
64
+ attr_reader :refusal
48
65
 
49
66
  def initialize
50
67
  @provider_id_by_key = {}
@@ -52,6 +69,7 @@ module AgentHarness
52
69
  @latest_provider_id = nil
53
70
  @input_tokens = nil
54
71
  @output_tokens = nil
72
+ @refusal = false
55
73
  end
56
74
 
57
75
  # Links a stream chunk key to its provider call id, returning true
@@ -72,6 +90,10 @@ module AgentHarness
72
90
  total = (input_tokens + output_tokens) if input_tokens && output_tokens
73
91
  {input_tokens: input_tokens, output_tokens: output_tokens, total_tokens: total}
74
92
  end
93
+
94
+ def observe(chunk)
95
+ @refusal ||= chunk.is_a?(RubyLlmResponsesStreamingRefusal::RefusalChunk)
96
+ end
75
97
  end
76
98
 
77
99
  private
@@ -150,19 +172,20 @@ module AgentHarness
150
172
  config.request_timeout = timeout&.dig(:read_seconds) || DEFAULT_REQUEST_TIMEOUT
151
173
  end
152
174
 
153
- def configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature)
175
+ def configure_chat(chat, candidate, messages, tools, max_output_tokens, temperature, schema)
154
176
  chat.messages = normalize_messages(messages)
155
177
  chat.with_tools(tools.map { |tool| normalized_tool(tool) }) unless tools.empty?
156
178
  chat.with_headers(candidate[:headers] || {})
157
179
  chat.with_max_output_tokens(max_output_tokens) if max_output_tokens
158
180
  chat.with_temperature(temperature) unless temperature.nil?
181
+ chat.with_schema(schema) if schema
159
182
  end
160
183
 
161
184
  def generate(chat, stream, cancellation)
162
- return generate_without_events(chat, cancellation) unless stream
185
+ return [generate_without_events(chat, cancellation), false] unless stream
163
186
 
164
187
  state = StreamState.new
165
- chat.generate do |chunk|
188
+ response = chat.generate do |chunk|
166
189
  if cancelled?(cancellation)
167
190
  chat.cancel
168
191
  raise RubyLLM::CancelledError
@@ -170,6 +193,7 @@ module AgentHarness
170
193
 
171
194
  stream_events(chunk, state).each { |event| yield event }
172
195
  end
196
+ [response, state.refusal]
173
197
  end
174
198
 
175
199
  def generate_without_events(chat, cancellation)
@@ -235,6 +259,7 @@ module AgentHarness
235
259
  end
236
260
 
237
261
  def stream_events(chunk, state)
262
+ state.observe(chunk)
238
263
  [text_event(chunk), *tool_call_events(chunk, state), usage_event(chunk, state)].compact
239
264
  end
240
265
 
@@ -300,16 +325,46 @@ module AgentHarness
300
325
  end
301
326
  end
302
327
 
303
- def normalize_response(response)
328
+ def normalize_response(response, streamed_refusal: false)
304
329
  {
305
330
  content: response.content || "",
306
331
  model: response.model,
307
332
  finish_reason: response.finish_reason,
308
333
  usage: normalize_usage(response.tokens),
309
- tool_calls: Array(response.tool_calls&.values).map { |call| normalize_tool_call(call) }
334
+ tool_calls: Array(response.tool_calls&.values).map { |call| normalize_tool_call(call) },
335
+ refusal: streamed_refusal || refusal?(response)
310
336
  }
311
337
  end
312
338
 
339
+ def refusal?(response)
340
+ return true if response.finish_reason == :content_filter
341
+ return false unless response.respond_to?(:raw)
342
+
343
+ body = response.raw&.body
344
+ return false unless body.is_a?(Hash)
345
+
346
+ responses_api_refusal?(body) || chat_completions_refusal?(body)
347
+ end
348
+
349
+ def responses_api_refusal?(body)
350
+ Array(body["output"] || body[:output]).any? do |item|
351
+ next false unless item.is_a?(Hash)
352
+
353
+ Array(item["content"] || item[:content]).any? do |part|
354
+ part.is_a?(Hash) && (part["type"] || part[:type]) == "refusal"
355
+ end
356
+ end
357
+ end
358
+
359
+ def chat_completions_refusal?(body)
360
+ Array(body["choices"] || body[:choices]).any? do |choice|
361
+ next false unless choice.is_a?(Hash)
362
+
363
+ message = choice["message"] || choice[:message]
364
+ message.is_a?(Hash) && !(message["refusal"] || message[:refusal]).nil?
365
+ end
366
+ end
367
+
313
368
  def normalize_tool_call(call)
314
369
  {provider_id: call.id, name: call.name, arguments_json: JSON.generate(call.arguments || {})}
315
370
  end
@@ -0,0 +1,43 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "json"
4
+ require "json_schemer"
5
+
6
+ module AgentHarness
7
+ module Api
8
+ # Parses and validates one schema-constrained provider response.
9
+ class SchemaResponse
10
+ def initialize(schema)
11
+ @validator = JSONSchemer.schema(schema)
12
+ end
13
+
14
+ def call(response)
15
+ content = response[:content].to_s
16
+ return failure(content, :refusal) if response[:refusal]
17
+ return failure(content, :truncated_output) if response[:finish_reason]&.to_sym == :max_tokens
18
+
19
+ parsed = JSON.parse(content)
20
+ return failure(content, :invalid_schema) unless @validator.valid?(parsed)
21
+
22
+ {content: content, parsed: parsed, error: nil}
23
+ rescue JSON::ParserError
24
+ failure(content, :invalid_json)
25
+ end
26
+
27
+ private
28
+
29
+ def failure(content, code)
30
+ {
31
+ content: content,
32
+ parsed: nil,
33
+ error: {
34
+ category: :invalid_response,
35
+ code: code,
36
+ retryable: false,
37
+ message: "Schema response failed (invalid_response/#{code})"
38
+ }
39
+ }
40
+ end
41
+ end
42
+ end
43
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module AgentHarness
4
- VERSION = "0.42.0"
4
+ VERSION = "0.43.0"
5
5
  end
data/lib/agent_harness.rb CHANGED
@@ -473,6 +473,7 @@ require_relative "agent_harness/openai_compatible_transport"
473
473
  require_relative "agent_harness/conversation"
474
474
  require_relative "agent_harness/api/attempt_report"
475
475
  require_relative "agent_harness/api/chat_transport"
476
+ require_relative "agent_harness/api/schema_response"
476
477
  require_relative "agent_harness/quota_status"
477
478
  require_relative "agent_harness/authentication"
478
479
  require_relative "agent_harness/provider_health_check"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: agent-harness
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.42.0
4
+ version: 0.43.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Bart Agapinan
@@ -30,25 +30,33 @@ dependencies:
30
30
  - !ruby/object:Gem::Version
31
31
  version: '2.0'
32
32
  - !ruby/object:Gem::Dependency
33
- name: ruby_llm
33
+ name: json_schemer
34
34
  requirement: !ruby/object:Gem::Requirement
35
35
  requirements:
36
- - - ">="
37
- - !ruby/object:Gem::Version
38
- version: '2.0'
39
- - - "<"
36
+ - - "~>"
40
37
  - !ruby/object:Gem::Version
41
- version: '3.0'
38
+ version: '2.4'
42
39
  type: :runtime
43
40
  prerelease: false
44
41
  version_requirements: !ruby/object:Gem::Requirement
45
42
  requirements:
46
- - - ">="
43
+ - - "~>"
47
44
  - !ruby/object:Gem::Version
48
- version: '2.0'
49
- - - "<"
45
+ version: '2.4'
46
+ - !ruby/object:Gem::Dependency
47
+ name: ruby_llm
48
+ requirement: !ruby/object:Gem::Requirement
49
+ requirements:
50
+ - - '='
50
51
  - !ruby/object:Gem::Version
51
- version: '3.0'
52
+ version: 2.0.0
53
+ type: :runtime
54
+ prerelease: false
55
+ version_requirements: !ruby/object:Gem::Requirement
56
+ requirements:
57
+ - - '='
58
+ - !ruby/object:Gem::Version
59
+ version: 2.0.0
52
60
  - !ruby/object:Gem::Dependency
53
61
  name: rake
54
62
  requirement: !ruby/object:Gem::Requirement
@@ -138,6 +146,7 @@ files:
138
146
  - lib/agent_harness/api/attempt_report.rb
139
147
  - lib/agent_harness/api/chat_transport.rb
140
148
  - lib/agent_harness/api/ruby_llm_chat_adapter.rb
149
+ - lib/agent_harness/api/schema_response.rb
141
150
  - lib/agent_harness/authentication.rb
142
151
  - lib/agent_harness/command_executor.rb
143
152
  - lib/agent_harness/configuration.rb