agent-harness 0.44.40 → 0.44.41

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 19dfa53187f8dfe813b043cdc991b13eb9c82808b70efb6ea0563fc6254d34cc
4
- data.tar.gz: 24714b03c6fa0dba05162e921b13b655fb8442b0a5997b8ceeda82cf0c8dad6f
3
+ metadata.gz: b2db7196758b674a453ebbd3d80801a4a99a263cd2b882229b00ad35670df5a0
4
+ data.tar.gz: '0958fd2f98cbc1ed6bede11948648ab4f0e63484616fd820fc4ca150eb784754'
5
5
  SHA512:
6
- metadata.gz: f97f08f9aadfb858dc4a4e7b08e511fdeba130c268778bdea6a1a920b13cade8faac07d8fe09be7aa611fcf517ace06ba10c523a5c57b7ab466683f741c95555
7
- data.tar.gz: 5459443038a53a230ef47696194cc83fdc1ac176c493eeeaaf29921965f613ef5b9e6dda42755faa9aaf1f24330eb35941b0a0d2b360c8db69c9c45d5e8a9328
6
+ metadata.gz: d55dfa100bf3c7bb89d00800f35692610bf2910e343a4ddbb0b2d9a71792583c6b5811efa0cf454ce2eeb902175dbbe09bc65900df60f3562caf8da1120ab2c9
7
+ data.tar.gz: ce6d513734809e9f3a1c724c64b58afeb2854c34e1d70637eb36c4f51476df1ae3d9638b4adfefefd01c87d059e00ed1f119f62143f6647fba6c23b472982cee
@@ -1,3 +1,3 @@
1
1
  {
2
- ".": "0.44.40"
2
+ ".": "0.44.41"
3
3
  }
data/CHANGELOG.md CHANGED
@@ -1,5 +1,9 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ### Bug Fixes
4
+
5
+ * **api:** classify unusable empty chat responses, refusals, and truncations while preserving provider terminal diagnostics, partial tool state, usage, and cost ([#552](https://github.com/viamin/agent-harness/issues/552)).
6
+
3
7
  ### Features
4
8
 
5
9
  * support reasoning-capable OpenAI function-tool chat through the Responses API, including opaque reasoning state across tool-result continuation, streaming tool IDs and arguments, usage, cancellation, classified errors, an explicit compatible-provider protocol contract, and a gated live smoke test ([#499](https://github.com/viamin/agent-harness/issues/499)).
@@ -12,6 +16,13 @@
12
16
 
13
17
  * close the RDR-072 upstream epic from #437's retained-scope evidence while keeping unpublished capabilities and downstream adoption behind their explicit release and integration gates ([#430](https://github.com/viamin/agent-harness/issues/430)).
14
18
 
19
+ ## [0.44.41](https://github.com/viamin/agent-harness/compare/agent-harness/v0.44.40...agent-harness/v0.44.41) (2026-11-21)
20
+
21
+
22
+ ### Bug Fixes
23
+
24
+ * **api:** classify unusable chat outcomes ([#553](https://github.com/viamin/agent-harness/issues/553)) ([82637cd](https://github.com/viamin/agent-harness/commit/82637cd46c3db8ab4b51daffd912618fff78d750))
25
+
15
26
  ## [0.44.40](https://github.com/viamin/agent-harness/compare/agent-harness/v0.44.39...agent-harness/v0.44.40) (2026-11-20)
16
27
 
17
28
 
@@ -57,6 +57,23 @@ invokes a supplied tool; callers append completed tool results to a later
57
57
  request. A failed partial stream is terminal, so its content cannot be appended
58
58
  to a fallback response and its tool calls cannot be replayed automatically.
59
59
 
60
+ A chat result is usable only when it contains non-whitespace user-facing text
61
+ or at least one completed callable tool call. An otherwise successful physical
62
+ request returns `status: :failed` with retryable
63
+ `invalid_response/empty_response`; reasoning or opaque continuation state alone
64
+ does not make it a completed answer. Provider refusals are explicit,
65
+ non-retryable `invalid_response/refusal` outcomes. A provider-reported output
66
+ limit is `invalid_response/truncated_output`: visible text is retained with
67
+ `status: :partial`, while an empty truncation is failed. The result, terminal
68
+ event, and attempt report retain the supported provider `finish_reason` and
69
+ refusal flag. No stop reason is inferred when the provider supplies none.
70
+
71
+ Tool-only chat responses remain successful when they contain completed calls.
72
+ An interrupted stream retains visible text and both completed and incomplete
73
+ tool calls with `status: :partial`; incomplete calls MUST NOT be executed.
74
+ Schema success is determined by the locally validated parsed value and does not
75
+ require conversational prose.
76
+
60
77
  Streamed tool-call arguments are emitted as raw, appendable JSON fragments
61
78
  correlated to one `tool_call_started` event per provider call, and cumulative
62
79
  provider token counts are emitted as deduplicated `usage_updated` events. An
@@ -384,6 +401,12 @@ observer so durable accounting can persist an attempt even when no message is
384
401
  created. Callers deduplicate on `attempt_id`. Cost identifies its source as
385
402
  provider-reported or harness-estimated; unknown cost remains `nil`.
386
403
 
404
+ Attempt `status` describes whether that physical request produced a usable
405
+ operation result, not whether the provider returned a successful HTTP status.
406
+ Consequently, an unusable empty response has a failed attempt while retaining
407
+ the provider-reported usage and cost from the successful physical request.
408
+ Absent usage or cost remains unknown and is never synthesized as zero.
409
+
387
410
  The observer receives `attempt_completed` after every physical request, with
388
411
  the same report later returned in `result[:attempts]`. Delivery is at least
389
412
  once across process recovery: consumers MUST enforce a unique key on
@@ -401,7 +424,8 @@ requests are in this ledger. Existing CLI providers, `TextTransport`,
401
424
  `OpenAICompatibleTransport`, token trackers, embeddings, and other
402
425
  non-`Api::ChatTransport` paths remain outside it.
403
426
 
404
- Only errors classified `transient` are eligible for bounded request retry:
427
+ Errors classified `transient` and `invalid_response/empty_response` are
428
+ eligible for bounded request retry:
405
429
  connection failure, timeout before a partial stream, rate limit, server error,
406
430
  service unavailable, and overload. Authentication, authorization, billing,
407
431
  invalid request, unsupported capability, invalid schema, context length,
@@ -25,13 +25,14 @@ module AgentHarness
25
25
  private_class_method :symbolize
26
26
 
27
27
  def initialize(attempt_id:, request_id:, number:, provider:, model:, status:, started_at:, finished_at:,
28
- usage: nil, cost: nil, provider_reported: false, error: nil)
28
+ usage: nil, cost: nil, provider_reported: false, finish_reason: nil, refusal: false, error: nil)
29
29
  @attributes = {
30
30
  attempt_id: attempt_id.to_s, request_id: request_id.to_s, number: number,
31
31
  provider: provider.to_sym, model: model&.to_s, status: status.to_sym,
32
32
  started_at: timestamp(started_at), finished_at: timestamp(finished_at),
33
33
  usage: normalize_usage(usage), cost: normalize_cost(cost),
34
- provider_reported: provider_reported == true, error: error
34
+ provider_reported: provider_reported == true, finish_reason: finish_reason&.to_sym,
35
+ refusal: refusal == true, error: error
35
36
  }
36
37
  validate!
37
38
  deep_freeze(@attributes)
@@ -145,23 +145,49 @@ module AgentHarness
145
145
  def success(candidate, attempt_id, started_at, adapter_result, accounting)
146
146
  accounting ||= {usage: adapter_result[:usage], provider_reported: !adapter_result[:usage].nil?}
147
147
  schema_result = normalize_schema_response(adapter_result)
148
- status = schema_result[:error] ? :failed : :succeeded
149
- append_attempt(candidate, attempt_id, started_at, status, error: schema_result[:error], **accounting)
148
+ outcome = terminal_outcome(adapter_result, schema_result)
149
+ append_attempt(candidate, attempt_id, started_at, outcome[:status], error: outcome[:error],
150
+ finish_reason: adapter_result[:finish_reason], refusal: adapter_result[:refusal], **accounting)
150
151
  result = base_result(candidate).merge(
151
- status: status,
152
+ status: outcome[:status],
152
153
  content: schema_result[:content],
153
154
  parsed: schema_result[:parsed],
154
155
  tool_calls: normalize_tool_calls(adapter_result[:tool_calls]),
155
156
  provider_state: adapter_result[:provider_state],
156
157
  finish_reason: adapter_result[:finish_reason],
158
+ refusal: adapter_result[:refusal] == true,
157
159
  usage: aggregate_usage,
158
- error: schema_result[:error]
160
+ error: outcome[:error]
159
161
  )
160
- event = (status == :succeeded) ? :response_completed : :response_failed
162
+ event = (outcome[:status] == :succeeded) ? :response_completed : :response_failed
161
163
  emit(event, attempt_id:, result: result)
162
164
  result
163
165
  end
164
166
 
167
+ def terminal_outcome(adapter_result, schema_result)
168
+ return {status: schema_result[:error] ? :failed : :succeeded, error: schema_result[:error]} if schema_operation?
169
+ return chat_failure(adapter_result, :refusal, partial: false) if adapter_result[:refusal]
170
+ if adapter_result[:finish_reason]&.to_sym == :max_tokens
171
+ return chat_failure(adapter_result, :truncated_output, partial: adapter_result[:content].to_s.strip != "")
172
+ end
173
+ return {status: :succeeded, error: nil} if Array(adapter_result[:tool_calls]).any?
174
+ return {status: :succeeded, error: nil} unless adapter_result[:content].to_s.strip.empty?
175
+
176
+ chat_failure(adapter_result, :empty_response, partial: false, retryable: true)
177
+ end
178
+
179
+ def chat_failure(adapter_result, code, partial:, retryable: false)
180
+ error = {
181
+ category: :invalid_response,
182
+ code: code,
183
+ retryable: retryable,
184
+ message: "Chat response failed (invalid_response/#{code})",
185
+ finish_reason: adapter_result[:finish_reason],
186
+ refusal: adapter_result[:refusal] == true
187
+ }.compact
188
+ {status: partial ? :partial : :failed, error: error}
189
+ end
190
+
165
191
  def normalize_schema_response(adapter_result)
166
192
  return {content: adapter_result[:content] || "", parsed: nil, error: nil} unless schema_operation?
167
193
 
@@ -177,6 +203,7 @@ module AgentHarness
177
203
  content: partial ? @partial_content.dup : "",
178
204
  tool_calls: partial_tool_calls,
179
205
  finish_reason: nil,
206
+ refusal: false,
180
207
  usage: aggregate_usage,
181
208
  error: error
182
209
  )
@@ -296,17 +323,17 @@ module AgentHarness
296
323
 
297
324
  def failed_result(error, candidate)
298
325
  base_result(candidate).merge(status: :failed, content: "", tool_calls: [], finish_reason: nil,
299
- usage: aggregate_usage, error: error)
326
+ refusal: false, usage: aggregate_usage, error: error)
300
327
  end
301
328
 
302
329
  def cancelled_result(candidate = candidates.first)
303
330
  error = {category: :cancelled, code: :cancelled, retryable: false, message: "Request cancelled"}
304
331
  base_result(candidate).merge(status: :cancelled, content: "", tool_calls: [], finish_reason: nil,
305
- usage: aggregate_usage, error: error)
332
+ refusal: false, usage: aggregate_usage, error: error)
306
333
  end
307
334
 
308
335
  def append_attempt(candidate, attempt_id, started_at, status, error: nil, usage: nil, cost: nil,
309
- provider_reported: false)
336
+ provider_reported: false, finish_reason: nil, refusal: false)
310
337
  cost = cost.merge(priced_at: Time.now.utc) if cost
311
338
  report = AttemptReport.new(
312
339
  attempt_id: attempt_id,
@@ -320,6 +347,8 @@ module AgentHarness
320
347
  usage: usage,
321
348
  cost: cost,
322
349
  provider_reported: provider_reported,
350
+ finish_reason: finish_reason,
351
+ refusal: refusal,
323
352
  error: error
324
353
  ).attributes
325
354
  attempts << report
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module AgentHarness
4
- VERSION = "0.44.40"
4
+ VERSION = "0.44.41"
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: agent-harness
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.44.40
4
+ version: 0.44.41
5
5
  platform: ruby
6
6
  authors:
7
7
  - Bart Agapinan