agent-harness 0.44.40 → 0.44.41
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.release-please-manifest.json +1 -1
- data/CHANGELOG.md +11 -0
- data/docs/provider-neutral-api-execution-contract.md +25 -1
- data/lib/agent_harness/api/attempt_report.rb +3 -2
- data/lib/agent_harness/api/chat_transport.rb +37 -8
- data/lib/agent_harness/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b2db7196758b674a453ebbd3d80801a4a99a263cd2b882229b00ad35670df5a0
|
|
4
|
+
data.tar.gz: '0958fd2f98cbc1ed6bede11948648ab4f0e63484616fd820fc4ca150eb784754'
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d55dfa100bf3c7bb89d00800f35692610bf2910e343a4ddbb0b2d9a71792583c6b5811efa0cf454ce2eeb902175dbbe09bc65900df60f3562caf8da1120ab2c9
|
|
7
|
+
data.tar.gz: ce6d513734809e9f3a1c724c64b58afeb2854c34e1d70637eb36c4f51476df1ae3d9638b4adfefefd01c87d059e00ed1f119f62143f6647fba6c23b472982cee
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
### Bug Fixes
|
|
4
|
+
|
|
5
|
+
* **api:** classify unusable empty chat responses, refusals, and truncations while preserving provider terminal diagnostics, partial tool state, usage, and cost ([#552](https://github.com/viamin/agent-harness/issues/552)).
|
|
6
|
+
|
|
3
7
|
### Features
|
|
4
8
|
|
|
5
9
|
* support reasoning-capable OpenAI function-tool chat through the Responses API, including opaque reasoning state across tool-result continuation, streaming tool IDs and arguments, usage, cancellation, classified errors, an explicit compatible-provider protocol contract, and a gated live smoke test ([#499](https://github.com/viamin/agent-harness/issues/499)).
|
|
@@ -12,6 +16,13 @@
|
|
|
12
16
|
|
|
13
17
|
* close the RDR-072 upstream epic from #437's retained-scope evidence while keeping unpublished capabilities and downstream adoption behind their explicit release and integration gates ([#430](https://github.com/viamin/agent-harness/issues/430)).
|
|
14
18
|
|
|
19
|
+
## [0.44.41](https://github.com/viamin/agent-harness/compare/agent-harness/v0.44.40...agent-harness/v0.44.41) (2026-11-21)
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
### Bug Fixes
|
|
23
|
+
|
|
24
|
+
* **api:** classify unusable chat outcomes ([#553](https://github.com/viamin/agent-harness/issues/553)) ([82637cd](https://github.com/viamin/agent-harness/commit/82637cd46c3db8ab4b51daffd912618fff78d750))
|
|
25
|
+
|
|
15
26
|
## [0.44.40](https://github.com/viamin/agent-harness/compare/agent-harness/v0.44.39...agent-harness/v0.44.40) (2026-11-20)
|
|
16
27
|
|
|
17
28
|
|
|
@@ -57,6 +57,23 @@ invokes a supplied tool; callers append completed tool results to a later
|
|
|
57
57
|
request. A failed partial stream is terminal, so its content cannot be appended
|
|
58
58
|
to a fallback response and its tool calls cannot be replayed automatically.
|
|
59
59
|
|
|
60
|
+
A chat result is usable only when it contains non-whitespace user-facing text
|
|
61
|
+
or at least one completed callable tool call. An otherwise successful physical
|
|
62
|
+
request returns `status: :failed` with retryable
|
|
63
|
+
`invalid_response/empty_response`; reasoning or opaque continuation state alone
|
|
64
|
+
does not make it a completed answer. Provider refusals are explicit,
|
|
65
|
+
non-retryable `invalid_response/refusal` outcomes. A provider-reported output
|
|
66
|
+
limit is `invalid_response/truncated_output`: visible text is retained with
|
|
67
|
+
`status: :partial`, while an empty truncation is failed. The result, terminal
|
|
68
|
+
event, and attempt report retain the supported provider `finish_reason` and
|
|
69
|
+
refusal flag. No stop reason is inferred when the provider supplies none.
|
|
70
|
+
|
|
71
|
+
Tool-only chat responses remain successful when they contain completed calls.
|
|
72
|
+
An interrupted stream retains visible text and both completed and incomplete
|
|
73
|
+
tool calls with `status: :partial`; incomplete calls MUST NOT be executed.
|
|
74
|
+
Schema success is determined by the locally validated parsed value and does not
|
|
75
|
+
require conversational prose.
|
|
76
|
+
|
|
60
77
|
Streamed tool-call arguments are emitted as raw, appendable JSON fragments
|
|
61
78
|
correlated to one `tool_call_started` event per provider call, and cumulative
|
|
62
79
|
provider token counts are emitted as deduplicated `usage_updated` events. An
|
|
@@ -384,6 +401,12 @@ observer so durable accounting can persist an attempt even when no message is
|
|
|
384
401
|
created. Callers deduplicate on `attempt_id`. Cost identifies its source as
|
|
385
402
|
provider-reported or harness-estimated; unknown cost remains `nil`.
|
|
386
403
|
|
|
404
|
+
Attempt `status` describes whether that physical request produced a usable
|
|
405
|
+
operation result, not whether the provider returned a successful HTTP status.
|
|
406
|
+
Consequently, an unusable empty response has a failed attempt while retaining
|
|
407
|
+
the provider-reported usage and cost from the successful physical request.
|
|
408
|
+
Absent usage or cost remains unknown and is never synthesized as zero.
|
|
409
|
+
|
|
387
410
|
The observer receives `attempt_completed` after every physical request, with
|
|
388
411
|
the same report later returned in `result[:attempts]`. Delivery is at least
|
|
389
412
|
once across process recovery: consumers MUST enforce a unique key on
|
|
@@ -401,7 +424,8 @@ requests are in this ledger. Existing CLI providers, `TextTransport`,
|
|
|
401
424
|
`OpenAICompatibleTransport`, token trackers, embeddings, and other
|
|
402
425
|
non-`Api::ChatTransport` paths remain outside it.
|
|
403
426
|
|
|
404
|
-
|
|
427
|
+
Errors classified `transient` and `invalid_response/empty_response` are
|
|
428
|
+
eligible for bounded request retry:
|
|
405
429
|
connection failure, timeout before a partial stream, rate limit, server error,
|
|
406
430
|
service unavailable, and overload. Authentication, authorization, billing,
|
|
407
431
|
invalid request, unsupported capability, invalid schema, context length,
|
|
@@ -25,13 +25,14 @@ module AgentHarness
|
|
|
25
25
|
private_class_method :symbolize
|
|
26
26
|
|
|
27
27
|
def initialize(attempt_id:, request_id:, number:, provider:, model:, status:, started_at:, finished_at:,
|
|
28
|
-
usage: nil, cost: nil, provider_reported: false, error: nil)
|
|
28
|
+
usage: nil, cost: nil, provider_reported: false, finish_reason: nil, refusal: false, error: nil)
|
|
29
29
|
@attributes = {
|
|
30
30
|
attempt_id: attempt_id.to_s, request_id: request_id.to_s, number: number,
|
|
31
31
|
provider: provider.to_sym, model: model&.to_s, status: status.to_sym,
|
|
32
32
|
started_at: timestamp(started_at), finished_at: timestamp(finished_at),
|
|
33
33
|
usage: normalize_usage(usage), cost: normalize_cost(cost),
|
|
34
|
-
provider_reported: provider_reported == true,
|
|
34
|
+
provider_reported: provider_reported == true, finish_reason: finish_reason&.to_sym,
|
|
35
|
+
refusal: refusal == true, error: error
|
|
35
36
|
}
|
|
36
37
|
validate!
|
|
37
38
|
deep_freeze(@attributes)
|
|
@@ -145,23 +145,49 @@ module AgentHarness
|
|
|
145
145
|
def success(candidate, attempt_id, started_at, adapter_result, accounting)
|
|
146
146
|
accounting ||= {usage: adapter_result[:usage], provider_reported: !adapter_result[:usage].nil?}
|
|
147
147
|
schema_result = normalize_schema_response(adapter_result)
|
|
148
|
-
|
|
149
|
-
append_attempt(candidate, attempt_id, started_at, status, error:
|
|
148
|
+
outcome = terminal_outcome(adapter_result, schema_result)
|
|
149
|
+
append_attempt(candidate, attempt_id, started_at, outcome[:status], error: outcome[:error],
|
|
150
|
+
finish_reason: adapter_result[:finish_reason], refusal: adapter_result[:refusal], **accounting)
|
|
150
151
|
result = base_result(candidate).merge(
|
|
151
|
-
status: status,
|
|
152
|
+
status: outcome[:status],
|
|
152
153
|
content: schema_result[:content],
|
|
153
154
|
parsed: schema_result[:parsed],
|
|
154
155
|
tool_calls: normalize_tool_calls(adapter_result[:tool_calls]),
|
|
155
156
|
provider_state: adapter_result[:provider_state],
|
|
156
157
|
finish_reason: adapter_result[:finish_reason],
|
|
158
|
+
refusal: adapter_result[:refusal] == true,
|
|
157
159
|
usage: aggregate_usage,
|
|
158
|
-
error:
|
|
160
|
+
error: outcome[:error]
|
|
159
161
|
)
|
|
160
|
-
event = (status == :succeeded) ? :response_completed : :response_failed
|
|
162
|
+
event = (outcome[:status] == :succeeded) ? :response_completed : :response_failed
|
|
161
163
|
emit(event, attempt_id:, result: result)
|
|
162
164
|
result
|
|
163
165
|
end
|
|
164
166
|
|
|
167
|
+
def terminal_outcome(adapter_result, schema_result)
|
|
168
|
+
return {status: schema_result[:error] ? :failed : :succeeded, error: schema_result[:error]} if schema_operation?
|
|
169
|
+
return chat_failure(adapter_result, :refusal, partial: false) if adapter_result[:refusal]
|
|
170
|
+
if adapter_result[:finish_reason]&.to_sym == :max_tokens
|
|
171
|
+
return chat_failure(adapter_result, :truncated_output, partial: adapter_result[:content].to_s.strip != "")
|
|
172
|
+
end
|
|
173
|
+
return {status: :succeeded, error: nil} if Array(adapter_result[:tool_calls]).any?
|
|
174
|
+
return {status: :succeeded, error: nil} unless adapter_result[:content].to_s.strip.empty?
|
|
175
|
+
|
|
176
|
+
chat_failure(adapter_result, :empty_response, partial: false, retryable: true)
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
def chat_failure(adapter_result, code, partial:, retryable: false)
|
|
180
|
+
error = {
|
|
181
|
+
category: :invalid_response,
|
|
182
|
+
code: code,
|
|
183
|
+
retryable: retryable,
|
|
184
|
+
message: "Chat response failed (invalid_response/#{code})",
|
|
185
|
+
finish_reason: adapter_result[:finish_reason],
|
|
186
|
+
refusal: adapter_result[:refusal] == true
|
|
187
|
+
}.compact
|
|
188
|
+
{status: partial ? :partial : :failed, error: error}
|
|
189
|
+
end
|
|
190
|
+
|
|
165
191
|
def normalize_schema_response(adapter_result)
|
|
166
192
|
return {content: adapter_result[:content] || "", parsed: nil, error: nil} unless schema_operation?
|
|
167
193
|
|
|
@@ -177,6 +203,7 @@ module AgentHarness
|
|
|
177
203
|
content: partial ? @partial_content.dup : "",
|
|
178
204
|
tool_calls: partial_tool_calls,
|
|
179
205
|
finish_reason: nil,
|
|
206
|
+
refusal: false,
|
|
180
207
|
usage: aggregate_usage,
|
|
181
208
|
error: error
|
|
182
209
|
)
|
|
@@ -296,17 +323,17 @@ module AgentHarness
|
|
|
296
323
|
|
|
297
324
|
def failed_result(error, candidate)
|
|
298
325
|
base_result(candidate).merge(status: :failed, content: "", tool_calls: [], finish_reason: nil,
|
|
299
|
-
usage: aggregate_usage, error: error)
|
|
326
|
+
refusal: false, usage: aggregate_usage, error: error)
|
|
300
327
|
end
|
|
301
328
|
|
|
302
329
|
def cancelled_result(candidate = candidates.first)
|
|
303
330
|
error = {category: :cancelled, code: :cancelled, retryable: false, message: "Request cancelled"}
|
|
304
331
|
base_result(candidate).merge(status: :cancelled, content: "", tool_calls: [], finish_reason: nil,
|
|
305
|
-
usage: aggregate_usage, error: error)
|
|
332
|
+
refusal: false, usage: aggregate_usage, error: error)
|
|
306
333
|
end
|
|
307
334
|
|
|
308
335
|
def append_attempt(candidate, attempt_id, started_at, status, error: nil, usage: nil, cost: nil,
|
|
309
|
-
provider_reported: false)
|
|
336
|
+
provider_reported: false, finish_reason: nil, refusal: false)
|
|
310
337
|
cost = cost.merge(priced_at: Time.now.utc) if cost
|
|
311
338
|
report = AttemptReport.new(
|
|
312
339
|
attempt_id: attempt_id,
|
|
@@ -320,6 +347,8 @@ module AgentHarness
|
|
|
320
347
|
usage: usage,
|
|
321
348
|
cost: cost,
|
|
322
349
|
provider_reported: provider_reported,
|
|
350
|
+
finish_reason: finish_reason,
|
|
351
|
+
refusal: refusal,
|
|
323
352
|
error: error
|
|
324
353
|
).attributes
|
|
325
354
|
attempts << report
|