@modelprofile.com/flexharness-agent 17.0.0 → 18.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -151,10 +151,11 @@ Pass these callbacks to `runAgent` or `AgentSession.create()`:
151
151
  | `onToken(delta)` | Each streamed answer-text delta. |
152
152
  | `onReasoningStart`, `onReasoningDelta`, `onReasoningEnd` | Provider-supplied streamed reasoning summaries, when available. |
153
153
  | `onStepCommitted()` | Once each model step is committed: its response is in the session's events and, with an event store, saved. The text the step streamed through `onToken` is final. A step whose stream reports an error is a failed model call and is not committed. |
154
+ | `onToolCallStreamed(event)` | `toolCallId`, `toolName`, `input`: the response holds a complete local tool call, in stream order with `onToken`. What streams after it comes after it, and the call starts only after it (`onToolCallStart`), also when the SDK runs the call before the session read its chunk. A call the SDK could not parse or repair has `invalid: true` and the SDK's `error`, and never starts. |
154
155
  | `onToolCallStart(event)` | `toolCallId`, `toolName`, `input`; `providerExecuted: true` for a tool the provider ran, such as OpenAI's web search. |
155
156
  | `onToolCallUpdate(event)` | The call identity and a transient streamed `output`. |
156
157
  | `onToolCallFinish(event)` | The call identity plus either `success: true, output` or `success: false, error`. |
157
- | `onRetry(event)` | Before each wait to retry a model call: `attempt`, `maxAttempts`, `delayMs` and `reason` (`rate_limit`, `overloaded`, `unavailable`). The text and reasoning the failed call streamed since the last `onStepCommitted()` are not in the session; the retry streams its own. Tool calls it executed stay, with their results. |
158
+ | `onRetry(event)` | Before each wait to retry a model call: `attempt`, `maxAttempts`, `delayMs` and `reason` (`rate_limit`, `overloaded`, `unavailable`). The failed call keeps what it streamed since the last `onStepCommitted()` up to and including its last tool call that ran, named by `keptThrough`, with that call's result; what came after that call, or everything when no call ran, is not in the session, and the retry streams its own. |
158
159
  | `onUsage(event)` | Once per model call, as soon as its usage is known; see [Count the usage of every run](#count-the-usage-of-every-run). |
159
160
 
160
161
  Tool updates are transient; the finish callback carries the authoritative final output. A provider-executed call, whose call and result arrive in the model's response, gets a start and a finish callback with `providerExecuted: true` and is never executed locally. Use `subscribe()` for committed session changes (`committed`, `updated`, `archived`), and retain its returned unsubscribe function. Session listeners are delivered in order per listener, have bounded queues and timeouts, and are removed on failure. Streaming callbacks and session-change listeners serve different purposes.
@@ -381,7 +382,7 @@ The steps of one generation read each URL once; a read that fails fails the step
381
382
  - `contextCompactor(messages, events, { reason, abortSignal, reportUsage })` returns replacement model messages. Provide it to use `session.compact()` or automatic event retention. Report the usage of its model calls through `reportUsage`; see [What is counted](#what-is-counted).
382
383
  - A compactor also enables proactive compaction with reason `context-budget` before a generation's first inference, once the context reaches its compaction point in tokens. The point's ceiling is 90% of the served context window of the generation's model: `contextWindow` on the generation lease that names the model, or on the session options for the session model. The default is the ceiling, or `contextCompactionDefaultTokens` when lower; `contextCompaction` chooses `{ tokens }` (clamped to the ceiling, unclamped without a window), `{ percent }` (an integer from 1 through 90 of the window; none without one) or `false`. A lease's `contextCompaction` takes precedence over the session's. `resolveAgentContextCompactionPoint()` states the point. The context counts the input tokens of the latest transactional generation outcome's `contextUsage` while the events that call saw still build the same model messages and no compaction followed, plus an estimate of 4 UTF-8 JSON bytes per token for what came after; otherwise the whole context is estimated. Only settled history is covered, never the current unsettled generation. Budget compactions carry `automatic: true`; after transactional finalization their covered events are archived when the store supports it and no open transaction or uncertain tool intent remains. Manual compaction is not automatically archived.
383
384
  - `eventRetention: { maxEvents }` triggers compaction and archival when the active event count exceeds the threshold. It also requires an event store with `archive()` support.
384
- - Context overflow invokes the configured compactor, or the `onContextOverflow(messages, { abortSignal, reportUsage })` handler. Without either, generation throws `ContextOverflowError`. It throws it as well, without calling the model again, when the overflow compaction finds nothing to compact: compaction never covers the current generation, and uncertain tool intents block it. `maxContextOverflowRetries` defaults to `3`.
385
+ - Context overflow invokes the configured compactor, or the `onContextOverflow(messages, { abortSignal, reportUsage })` handler. A model call overflows when the provider refuses its input as too long: OpenAI's `context_length_exceeded`, Anthropic's `prompt is too long: ...`, or a refusal that names the context window, the maximum context length, too many tokens or an input that is too long. Without either, generation throws `ContextOverflowError`. It throws it as well, without calling the model again, when the overflow compaction finds nothing to compact: compaction never covers the current generation, and uncertain tool intents block it. `maxContextOverflowRetries` defaults to `3`.
385
386
  - Open transactions and uncertain tool intents constrain when compaction and archival can proceed. Resolve them before manual compaction.
386
387
 
387
388
  Compaction changes the active model context; archival moves covered events out of the active event set. Implement archive retention in your chosen store when you need a complete audit history.
@@ -404,7 +405,7 @@ const session = new AgentSession({
404
405
  });
405
406
  ```
406
407
 
407
- A read-only call that no call that writes precedes in its response starts as soon as its input is complete, while the model still streams the rest of the response. When the SDK executes the response's calls, the session adopts the running call instead of starting it again. A response that fails, is retried, or ends without executing the call discards it: its abort signal fires and no tool-call or tool-result event, no `onToolCallStart` and no part of it is recorded. Such a call gets `confirmed`, a promise in its execution options (`IAgentToolExecutionOptions`), which resolves when the response adopted it and rejects when it was discarded; work that must not happen for an unfinished call, such as asking for permission, waits for it. A transactional generation writes a read-only call's execution intent when the response adopts it, after the call started: read-only calls have no effect to recover.
408
+ A read-only call that no call that writes precedes in its response starts as soon as its input is complete, while the model still streams the rest of the response. When the SDK executes the response's calls, the session adopts the running call instead of starting it again. A response that fails, is retried, or ends without executing the call discards it: its abort signal fires, and the session records what the call did. A call that returned keeps its result, also in a retry; a call that has not returned is closed with an error result saying its effect is unknown, and its failed model call is not retried. Such a call gets `confirmed`, a promise in its execution options (`IAgentToolExecutionOptions`), which resolves when the response adopted it and rejects when it was discarded; work that must not happen for an unfinished call, such as asking for permission, waits for it. A transactional generation writes a read-only call's execution intent when the response adopts it, after the call started: read-only calls have no effect to recover.
408
409
 
409
410
  For generation-scoped resources, `generate({ prepare })` accepts a callback that returns an `IAgentGenerationLease`. Its model/tool/settings overrides take precedence over per-generation options and session defaults. The runtime calls the lease's `close()` after execution. The host must clean up partial acquisition if `prepare` throws; lease cleanup must tolerate repeated or overlapping attempts after a failure or timeout.
410
411
 
@@ -419,7 +420,7 @@ For generation-scoped resources, `generate({ prepare })` accepts a callback that
419
420
 
420
421
  Tools and resource callbacks must observe their supplied abort signals. A model call ends at its abort whether or not its transport observes the signal: a provider that heeds the abort ends the call itself, and a late finish it still sends is counted; a call still open `MODEL_ABORT_GRACE_MS` (1 s) after the abort fails with the signal's reason and its stream is cancelled (`endModelStreamAtAbort`). Background jobs have separate lifecycles; closing a session does not automatically terminate them. Cleanup failures are observable. `closeCleanupCompleted` reports whether the session has released its retryable cleanup ownership.
421
422
 
422
- The runtime retries a model call that failed with a rate limit (429), an overloaded provider (529) or an unavailable one (503) at most 8 times, waiting for the provider's `retry-after` delay but at least the backoff that doubles from 2 s to 30 s. A requested delay beyond 60 s, or one that would take the call's retries past 150 s in total, fails the generation: a rate limit (429) as a `ModelLimitError` of kind `rate_limit` carrying that retry time, a 503 or 529 with the provider error. A `ModelLimitError` of kind `usage_limit`, which the provider adapters raise for a spent quota, is never retried. Completed tool results are preserved during a generation's retry sequence. A ledger reuses an already observed execution for the same tool-call identity. Applications still need their own idempotency and recovery rules for external side effects.
423
+ The runtime retries a model call that failed with a rate limit (429), an overloaded provider (529) or an unavailable one (503) at most 8 times, waiting for the provider's `retry-after` delay but at least the backoff that doubles from 2 s to 30 s. A requested delay beyond 60 s, or one that would take the call's retries past 150 s in total, fails the generation: a rate limit (429) as a `ModelLimitError` of kind `rate_limit` carrying that retry time, a 503 or 529 with the provider error. A `ModelLimitError` of kind `usage_limit`, which the provider adapters raise for a spent quota, is never retried. Completed tool results are preserved during a generation's retry sequence. A retried call keeps what it streamed up to and including its last tool call that ran, which is finished output the retry continues after (Anthropic needs a tool-use turn's thinking back), and drops what came after it, or everything when no call ran. A tool call the failed call streamed but never ran is left out of a retried call, so the retry asks the model without it; a failed call that is not retried keeps it once, closed with the unknown-effect result. A ledger reuses an already observed execution for the same tool-call identity. Applications still need their own idempotency and recovery rules for external side effects.
423
424
 
424
425
  `maxSteps` has **no default limit**. It is an optional positive finite safe integer limiting completed model steps per generation; `0`, negatives, fractions, `NaN`, infinity and unsafe integers are invalid. Resolution precedence is prepared generation lease → per-generation option → session setting; only when all omit the cap is execution unbounded. Owner stops, aborts, permissions, real errors, usage accounting and cleanup remain effective without a cap. Prompt-cache handling defaults to `cache: 'auto'`; use `cache: false` to disable the runtime's cache defaults, or pass an explicit cache policy (`ISmartAiCacheOptions` of `@modelprofile.com/flexharness-models`). On OpenAI a generation's `prompt_cache_key` is derived from its instructions and tool definitions, so sessions with one prefix share the provider's cache; `keyScope`, `keyBuckets` and `key` adjust it.
425
426
 
@@ -81,15 +81,25 @@ const assistantToolResultIds = (event: IAssistantMessageEvent): string[] => {
81
81
  .map((part) => part.toolCallId);
82
82
  };
83
83
 
84
+ /** What an assistant message's tool parts are keyed by: its inference, or the message itself. */
85
+ const assistantOccurrence = (event: IAssistantMessageEvent): string => event.inferenceId ?? event.id;
86
+
84
87
  const assistantOccurrenceKey = (event: IAssistantMessageEvent, toolCallId: string): string =>
85
- `${event.inferenceId ?? event.id}:${toolCallId}`;
88
+ `${assistantOccurrence(event)}:${toolCallId}`;
86
89
 
90
+ /**
91
+ * A tool event a response's assistant message carries as a part is keyed like that part, so the
92
+ * part and the event are one occurrence: the model sees the call once.
93
+ */
87
94
  const toolOccurrenceKey = (
88
95
  event: IToolCallEvent | IToolResultEvent,
89
- assistantEventIds: ReadonlySet<string>,
96
+ assistantOccurrences: ReadonlyMap<string, string>,
90
97
  ): string => {
91
- const parentAssistantId = event.parentIds?.find((parentId) => assistantEventIds.has(parentId));
92
- return `${parentAssistantId ?? event.inferenceId ?? event.id}:${event.toolCallId}`;
98
+ const parentAssistantId = event.parentIds?.find((parentId) => assistantOccurrences.has(parentId));
99
+ const occurrence = parentAssistantId === undefined
100
+ ? event.inferenceId ?? event.id
101
+ : assistantOccurrences.get(parentAssistantId)!;
102
+ return `${occurrence}:${event.toolCallId}`;
93
103
  };
94
104
 
95
105
  const jsonValue = (value: unknown): unknown => {
@@ -126,7 +136,8 @@ const pushStandaloneToolCall = (
126
136
  && Array.isArray(previous.content)
127
137
  && previous.content.every((contentPart) => contentPart.type === 'tool-call')
128
138
  ) {
129
- previous.content.push(part);
139
+ // A new message: the previous one's content may be an event's, which building never changes.
140
+ messages[messages.length - 1] = { ...previous, content: [...previous.content, part] };
130
141
  return;
131
142
  }
132
143
  messages.push({ role: 'assistant', content: [part] });
@@ -198,10 +209,10 @@ const renderExternalRuntimeEvent = (event: IAgentRuntimeEventPayload): string =>
198
209
 
199
210
  export function buildModelMessages(events: readonly TAgentEvent[]): plugins.ModelMessage[] {
200
211
  const orderedEvents = orderEvents(applyLatestCompaction(filterModelVisibleAgentEvents(events)));
201
- const assistantEventIds = new Set(
212
+ const assistantOccurrences = new Map(
202
213
  orderedEvents
203
214
  .filter((event): event is IAssistantMessageEvent => event.type === 'assistant-message')
204
- .map((event) => event.id),
215
+ .map((event) => [event.id, assistantOccurrence(event)]),
205
216
  );
206
217
  const representedToolCalls = new Set<string>();
207
218
  const representedToolResults = new Set<string>();
@@ -252,12 +263,12 @@ export function buildModelMessages(events: readonly TAgentEvent[]): plugins.Mode
252
263
  });
253
264
  break;
254
265
  case 'tool-call':
255
- if (!representedToolCalls.has(toolOccurrenceKey(event, assistantEventIds))) {
266
+ if (!representedToolCalls.has(toolOccurrenceKey(event, assistantOccurrences))) {
256
267
  pushStandaloneToolCall(messages, event);
257
268
  }
258
269
  break;
259
270
  case 'tool-result':
260
- if (!representedToolResults.has(toolOccurrenceKey(event, assistantEventIds))) {
271
+ if (!representedToolResults.has(toolOccurrenceKey(event, assistantOccurrences))) {
261
272
  pushToolResult(messages, event);
262
273
  }
263
274
  break;
@@ -333,6 +333,20 @@ export interface IAgentSessionOptions {
333
333
  * stream reports an error is a failed model call and is not committed.
334
334
  */
335
335
  onStepCommitted?: () => void;
336
+ /**
337
+ * Called when the model's response holds a complete local tool call, with its parsed input, in
338
+ * stream order with `onToken` and before the call starts: what the response streams after it
339
+ * comes after it, and `onToolCallStart` follows it. A call the SDK could not parse or repair
340
+ * has `invalid: true` and the SDK's `error`, and never starts. A retry keeps what its failed
341
+ * call streamed up to its last call that ran (`IAgentRetryEvent.keptThrough`).
342
+ */
343
+ onToolCallStreamed?: (event: {
344
+ toolCallId: string;
345
+ toolName: string;
346
+ input: unknown;
347
+ invalid?: true;
348
+ error?: string;
349
+ }) => void;
336
350
  /**
337
351
  * Called when a tool call starts, with its stable AI SDK call id. A provider-executed call is
338
352
  * reported as well, with `providerExecuted`, when the model's response states it.
@@ -346,9 +360,10 @@ export interface IAgentSessionOptions {
346
360
  /** Called when a tool call finishes, with its stable AI SDK call id and success state. */
347
361
  onToolCallFinish?: (event: TAgentToolCallFinishEvent) => void;
348
362
  /**
349
- * Called before the engine waits to retry a rate-limited or unavailable model call. What the
350
- * failed call streamed since the last `onStepCommitted`, its text and reasoning, is not in the
351
- * session: the retry streams its own. Tool calls it executed stay, with their results.
363
+ * Called before the engine waits to retry a rate-limited or unavailable model call. The failed
364
+ * call keeps what it streamed since the last `onStepCommitted` up to and including its last tool
365
+ * call that ran (`keptThrough`), with that call's result; what came after that call, or
366
+ * everything when no call ran, is not in the session: the retry streams its own.
352
367
  */
353
368
  onRetry?: (event: IAgentRetryEvent) => void;
354
369
  /**
@@ -9,6 +9,12 @@ export interface IAgentRetryEvent {
9
9
  maxAttempts: number;
10
10
  delayMs: number;
11
11
  reason: TAgentRetryReason;
12
+ /**
13
+ * The failed call's last local tool call that ran: the session keeps what the call streamed up
14
+ * to and including it, with its result. Absent when none ran: the session keeps nothing the
15
+ * call streamed.
16
+ */
17
+ keptThrough?: string;
12
18
  }
13
19
 
14
20
  export type TAgentRetryDecision =