@modelprofile.com/flexharness-agent 17.0.1 → 18.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist_ts_agent/classes.sessionengine.js +125 -26
- package/dist_ts_agent/index.d.ts +1 -1
- package/dist_ts_agent/index.js +2 -2
- package/dist_ts_agent/smartagent.interfaces.d.ts +18 -3
- package/dist_ts_agent/smartagent.interfaces.js +1 -1
- package/dist_ts_agent/smartagent.retry.d.ts +6 -0
- package/dist_ts_agent/smartagent.retry.js +1 -1
- package/dist_ts_agent/smartagent.transactions.d.ts +6 -0
- package/dist_ts_agent/smartagent.transactions.js +15 -1
- package/package.json +2 -2
- package/readme.md +5 -4
- package/ts_agent/classes.sessionengine.ts +129 -26
- package/ts_agent/index.ts +1 -0
- package/ts_agent/readme.md +5 -4
- package/ts_agent/smartagent.interfaces.ts +18 -3
- package/ts_agent/smartagent.retry.ts +6 -0
- package/ts_agent/smartagent.transactions.ts +14 -0
package/ts_agent/readme.md
CHANGED
|
@@ -151,10 +151,11 @@ Pass these callbacks to `runAgent` or `AgentSession.create()`:
|
|
|
151
151
|
| `onToken(delta)` | Each streamed answer-text delta. |
|
|
152
152
|
| `onReasoningStart`, `onReasoningDelta`, `onReasoningEnd` | Provider-supplied streamed reasoning summaries, when available. |
|
|
153
153
|
| `onStepCommitted()` | Once each model step is committed: its response is in the session's events and, with an event store, saved. The text the step streamed through `onToken` is final. A step whose stream reports an error is a failed model call and is not committed. |
|
|
154
|
+
| `onToolCallStreamed(event)` | `toolCallId`, `toolName`, `input`: the response holds a complete local tool call, in stream order with `onToken`. What streams after it comes after it, and the call starts only after it (`onToolCallStart`), also when the SDK runs the call before the session read its chunk. A call the SDK could not parse or repair has `invalid: true` and the SDK's `error`, and never starts. |
|
|
154
155
|
| `onToolCallStart(event)` | `toolCallId`, `toolName`, `input`; `providerExecuted: true` for a tool the provider ran, such as OpenAI's web search. |
|
|
155
156
|
| `onToolCallUpdate(event)` | The call identity and a transient streamed `output`. |
|
|
156
157
|
| `onToolCallFinish(event)` | The call identity plus either `success: true, output` or `success: false, error`. |
|
|
157
|
-
| `onRetry(event)` | Before each wait to retry a model call: `attempt`, `maxAttempts`, `delayMs` and `reason` (`rate_limit`, `overloaded`, `unavailable`). The
|
|
158
|
+
| `onRetry(event)` | Before each wait to retry a model call: `attempt`, `maxAttempts`, `delayMs` and `reason` (`rate_limit`, `overloaded`, `unavailable`). The failed call keeps what it streamed since the last `onStepCommitted()` up to and including its last tool call that ran, named by `keptThrough`, with that call's result; what came after that call, or everything when no call ran, is not in the session, and the retry streams its own. |
|
|
158
159
|
| `onUsage(event)` | Once per model call, as soon as its usage is known; see [Count the usage of every run](#count-the-usage-of-every-run). |
|
|
159
160
|
|
|
160
161
|
Tool updates are transient; the finish callback carries the authoritative final output. A provider-executed call, whose call and result arrive in the model's response, gets a start and a finish callback with `providerExecuted: true` and is never executed locally. Use `subscribe()` for committed session changes (`committed`, `updated`, `archived`), and retain its returned unsubscribe function. Session listeners are delivered in order per listener, have bounded queues and timeouts, and are removed on failure. Streaming callbacks and session-change listeners serve different purposes.
|
|
@@ -280,7 +281,7 @@ export async function createReviewedDraft(
|
|
|
280
281
|
|
|
281
282
|
The `review` callback belongs to your application. While awaiting acceptance, `candidate.messages` includes the candidate, but `getModelMessages()` excludes it from ordinary future context. Acceptance includes the generation as successful. Rejection retains actual model/tool output and its prompt and applied input in future context with a truthful rejected outcome marker. Unsuccessful generations without output remain excluded. Generation IDs identify transactions within a session; use a distinct ID for each new transaction.
|
|
282
283
|
|
|
283
|
-
`generate({ transaction, stop })` takes a second signal for the owner's stop, the way a user stops a turn. It ends the generation like `abort`, but the interrupted outcome records the stop (`stopped: true`, also accepted by `finalizeGeneration(transaction, 'interrupted', { stopped: true })` for a generation stopped after its execution completed). A stopped generation stays in future context once its model produced output: its prompt, the input it took in, the text its running model step streamed and the tool calls it issued. Each tool call without a recorded result is closed with an error result saying its effect is unknown, so the next request is well-formed for every provider. A generation stopped before any model output leaves no trace, so its prompt can be sent again without appearing twice. `isGenerationInModelContext(events, generationId)` applies the same rule to a list of events, for a caller that needs to know whether a finished generation stayed in context. The same retention rule applies to aborted, failed, rejected and restart-interrupted generations once execution produced model/tool output, including recorded partial inference output. Model-visible outcome markers distinguish rejected, interrupted and stopped-by-owner work from successful completion. Synthetic unknown-effect results only close the model protocol: they are not evidence that an external effect happened or did not happen, and never trigger automatic tool replay. `stop` requires a transactional generation.
|
|
284
|
+
`generate({ transaction, stop })` takes a second signal for the owner's stop, the way a user stops a turn. It ends the generation like `abort`, but the interrupted outcome records the stop (`stopped: true`, also accepted by `finalizeGeneration(transaction, 'interrupted', { stopped: true })` for a generation stopped after its execution completed). A stopped generation stays in future context once its model produced output: its prompt, the input it took in, the text its running model step streamed and the tool calls it issued. Each tool call without a recorded result is closed with an error result saying its effect is unknown, so the next request is well-formed for every provider. A generation stopped before any model output leaves no trace, so its prompt can be sent again without appearing twice. `isGenerationInModelContext(events, generationId)` applies the same rule to a list of events, for a caller that needs to know whether a finished generation stayed in context. `getGenerationsInModelContext(events)` returns the set of every generation that did, from one pass over the events, for a caller that asks about many. The same retention rule applies to aborted, failed, rejected and restart-interrupted generations once execution produced model/tool output, including recorded partial inference output. Model-visible outcome markers distinguish rejected, interrupted and stopped-by-owner work from successful completion. Synthetic unknown-effect results only close the model protocol: they are not evidence that an external effect happened or did not happen, and never trigger automatic tool replay. `stop` requires a transactional generation.
|
|
284
285
|
|
|
285
286
|
`beginContinuation({ generationId })` on the built-in `AgentSession` claims a generation
|
|
286
287
|
without appending a user message, through the same transactional execution, scheduling
|
|
@@ -381,7 +382,7 @@ The steps of one generation read each URL once; a read that fails fails the step
|
|
|
381
382
|
- `contextCompactor(messages, events, { reason, abortSignal, reportUsage })` returns replacement model messages. Provide it to use `session.compact()` or automatic event retention. Report the usage of its model calls through `reportUsage`; see [What is counted](#what-is-counted).
|
|
382
383
|
- A compactor also enables proactive compaction with reason `context-budget` before a generation's first inference, once the context reaches its compaction point in tokens. The point's ceiling is 90% of the served context window of the generation's model: `contextWindow` on the generation lease that names the model, or on the session options for the session model. The default is the ceiling, or `contextCompactionDefaultTokens` when lower; `contextCompaction` chooses `{ tokens }` (clamped to the ceiling, unclamped without a window), `{ percent }` (an integer from 1 through 90 of the window; none without one) or `false`. A lease's `contextCompaction` takes precedence over the session's. `resolveAgentContextCompactionPoint()` states the point. The context counts the input tokens of the latest transactional generation outcome's `contextUsage` while the events that call saw still build the same model messages and no compaction followed, plus an estimate of 4 UTF-8 JSON bytes per token for what came after; otherwise the whole context is estimated. Only settled history is covered, never the current unsettled generation. Budget compactions carry `automatic: true`; after transactional finalization their covered events are archived when the store supports it and no open transaction or uncertain tool intent remains. Manual compaction is not automatically archived.
|
|
383
384
|
- `eventRetention: { maxEvents }` triggers compaction and archival when the active event count exceeds the threshold. It also requires an event store with `archive()` support.
|
|
384
|
-
- Context overflow invokes the configured compactor, or the `onContextOverflow(messages, { abortSignal, reportUsage })` handler. Without either, generation throws `ContextOverflowError`. It throws it as well, without calling the model again, when the overflow compaction finds nothing to compact: compaction never covers the current generation, and uncertain tool intents block it. `maxContextOverflowRetries` defaults to `3`.
|
|
385
|
+
- Context overflow invokes the configured compactor, or the `onContextOverflow(messages, { abortSignal, reportUsage })` handler. A model call overflows when the provider refuses its input as too long: OpenAI's `context_length_exceeded`, Anthropic's `prompt is too long: ...`, or a refusal that names the context window, the maximum context length, too many tokens or an input that is too long. Without either, generation throws `ContextOverflowError`. It throws it as well, without calling the model again, when the overflow compaction finds nothing to compact: compaction never covers the current generation, and uncertain tool intents block it. `maxContextOverflowRetries` defaults to `3`.
|
|
385
386
|
- Open transactions and uncertain tool intents constrain when compaction and archival can proceed. Resolve them before manual compaction.
|
|
386
387
|
|
|
387
388
|
Compaction changes the active model context; archival moves covered events out of the active event set. Implement archive retention in your chosen store when you need a complete audit history.
|
|
@@ -419,7 +420,7 @@ For generation-scoped resources, `generate({ prepare })` accepts a callback that
|
|
|
419
420
|
|
|
420
421
|
Tools and resource callbacks must observe their supplied abort signals. A model call ends at its abort whether or not its transport observes the signal: a provider that heeds the abort ends the call itself, and a late finish it still sends is counted; a call still open `MODEL_ABORT_GRACE_MS` (1 s) after the abort fails with the signal's reason and its stream is cancelled (`endModelStreamAtAbort`). Background jobs have separate lifecycles; closing a session does not automatically terminate them. Cleanup failures are observable. `closeCleanupCompleted` reports whether the session has released its retryable cleanup ownership.
|
|
421
422
|
|
|
422
|
-
The runtime retries a model call that failed with a rate limit (429), an overloaded provider (529) or an unavailable one (503) at most 8 times, waiting for the provider's `retry-after` delay but at least the backoff that doubles from 2 s to 30 s. A requested delay beyond 60 s, or one that would take the call's retries past 150 s in total, fails the generation: a rate limit (429) as a `ModelLimitError` of kind `rate_limit` carrying that retry time, a 503 or 529 with the provider error. A `ModelLimitError` of kind `usage_limit`, which the provider adapters raise for a spent quota, is never retried. Completed tool results are preserved during a generation's retry sequence. A tool call the failed call streamed but never ran is left out of a retried call, so the retry asks the model without it; a failed call that is not retried keeps it once, closed with the unknown-effect result. A ledger reuses an already observed execution for the same tool-call identity. Applications still need their own idempotency and recovery rules for external side effects.
|
|
423
|
+
The runtime retries a model call that failed with a rate limit (429), an overloaded provider (529) or an unavailable one (503) at most 8 times, waiting for the provider's `retry-after` delay but at least the backoff that doubles from 2 s to 30 s. A requested delay beyond 60 s, or one that would take the call's retries past 150 s in total, fails the generation: a rate limit (429) as a `ModelLimitError` of kind `rate_limit` carrying that retry time, a 503 or 529 with the provider error. A `ModelLimitError` of kind `usage_limit`, which the provider adapters raise for a spent quota, is never retried. Completed tool results are preserved during a generation's retry sequence. A retried call keeps what it streamed up to and including its last tool call that ran, which is finished output the retry continues after (Anthropic needs a tool-use turn's thinking back), and drops what came after it, or everything when no call ran. A tool call the failed call streamed but never ran is left out of a retried call, so the retry asks the model without it; a failed call that is not retried keeps it once, closed with the unknown-effect result. A ledger reuses an already observed execution for the same tool-call identity. Applications still need their own idempotency and recovery rules for external side effects.
|
|
423
424
|
|
|
424
425
|
`maxSteps` has **no default limit**. It is an optional positive finite safe integer limiting completed model steps per generation; `0`, negatives, fractions, `NaN`, infinity and unsafe integers are invalid. Resolution precedence is prepared generation lease → per-generation option → session setting; only when all omit the cap is execution unbounded. Owner stops, aborts, permissions, real errors, usage accounting and cleanup remain effective without a cap. Prompt-cache handling defaults to `cache: 'auto'`; use `cache: false` to disable the runtime's cache defaults, or pass an explicit cache policy (`ISmartAiCacheOptions` of `@modelprofile.com/flexharness-models`). On OpenAI a generation's `prompt_cache_key` is derived from its instructions and tool definitions, so sessions with one prefix share the provider's cache; `keyScope`, `keyBuckets` and `key` adjust it.
|
|
425
426
|
|
|
@@ -333,6 +333,20 @@ export interface IAgentSessionOptions {
|
|
|
333
333
|
* stream reports an error is a failed model call and is not committed.
|
|
334
334
|
*/
|
|
335
335
|
onStepCommitted?: () => void;
|
|
336
|
+
/**
|
|
337
|
+
* Called when the model's response holds a complete local tool call, with its parsed input, in
|
|
338
|
+
* stream order with `onToken` and before the call starts: what the response streams after it
|
|
339
|
+
* comes after it, and `onToolCallStart` follows it. A call the SDK could not parse or repair
|
|
340
|
+
* has `invalid: true` and the SDK's `error`, and never starts. A retry keeps what its failed
|
|
341
|
+
* call streamed up to its last call that ran (`IAgentRetryEvent.keptThrough`).
|
|
342
|
+
*/
|
|
343
|
+
onToolCallStreamed?: (event: {
|
|
344
|
+
toolCallId: string;
|
|
345
|
+
toolName: string;
|
|
346
|
+
input: unknown;
|
|
347
|
+
invalid?: true;
|
|
348
|
+
error?: string;
|
|
349
|
+
}) => void;
|
|
336
350
|
/**
|
|
337
351
|
* Called when a tool call starts, with its stable AI SDK call id. A provider-executed call is
|
|
338
352
|
* reported as well, with `providerExecuted`, when the model's response states it.
|
|
@@ -346,9 +360,10 @@ export interface IAgentSessionOptions {
|
|
|
346
360
|
/** Called when a tool call finishes, with its stable AI SDK call id and success state. */
|
|
347
361
|
onToolCallFinish?: (event: TAgentToolCallFinishEvent) => void;
|
|
348
362
|
/**
|
|
349
|
-
* Called before the engine waits to retry a rate-limited or unavailable model call.
|
|
350
|
-
*
|
|
351
|
-
*
|
|
363
|
+
* Called before the engine waits to retry a rate-limited or unavailable model call. The failed
|
|
364
|
+
* call keeps what it streamed since the last `onStepCommitted` up to and including its last tool
|
|
365
|
+
* call that ran (`keptThrough`), with that call's result; what came after that call, or
|
|
366
|
+
* everything when no call ran, is not in the session: the retry streams its own.
|
|
352
367
|
*/
|
|
353
368
|
onRetry?: (event: IAgentRetryEvent) => void;
|
|
354
369
|
/**
|
|
@@ -9,6 +9,12 @@ export interface IAgentRetryEvent {
|
|
|
9
9
|
maxAttempts: number;
|
|
10
10
|
delayMs: number;
|
|
11
11
|
reason: TAgentRetryReason;
|
|
12
|
+
/**
|
|
13
|
+
* The failed call's last local tool call that ran: the session keeps what the call streamed up
|
|
14
|
+
* to and including it, with its result. Absent when none ran: the session keeps nothing the
|
|
15
|
+
* call streamed.
|
|
16
|
+
*/
|
|
17
|
+
keptThrough?: string;
|
|
12
18
|
}
|
|
13
19
|
|
|
14
20
|
export type TAgentRetryDecision =
|
|
@@ -187,6 +187,20 @@ export const isGenerationInModelContext = (
|
|
|
187
187
|
&& isModelVisibleTransaction(transaction, generationsWithModelOutput(events));
|
|
188
188
|
};
|
|
189
189
|
|
|
190
|
+
/**
|
|
191
|
+
* The generations of `events` that contribute to later context: for every id, `has(id)` answers
|
|
192
|
+
* what `isGenerationInModelContext(events, id)` answers, from one pass over the events instead of
|
|
193
|
+
* one per id.
|
|
194
|
+
*/
|
|
195
|
+
export const getGenerationsInModelContext = (events: readonly TAgentEvent[]): Set<string> => {
|
|
196
|
+
const producedOutput = generationsWithModelOutput(events);
|
|
197
|
+
const visible = new Set<string>();
|
|
198
|
+
for (const [generationId, transaction] of getAgentGenerationTransactions(events)) {
|
|
199
|
+
if (isModelVisibleTransaction(transaction, producedOutput)) visible.add(generationId);
|
|
200
|
+
}
|
|
201
|
+
return visible;
|
|
202
|
+
};
|
|
203
|
+
|
|
190
204
|
const interruptedToolResult = (
|
|
191
205
|
toolCallId: string,
|
|
192
206
|
toolName: string,
|