@cubicecho/agent-core 4.16.0 → 4.18.0

package/README.md

@@ -213,6 +213,11 @@ is worth another attempt. The content-free `{"role":"assistant"}` most servers o
 does not set it: nothing has been shown to anybody yet, so an endpoint that primes the stream and
 then wedges is retried like one that never answered at all.
 
+`inFlight` is the other box `streamTurn` takes, for what an attempt had said once it has thrown:
+each call sets its `read` as it starts, so after a retry it reads the attempt that failed last and
+nothing of the one before. `runTurn` and `continueTurn` pass it through, the second with the
+answer it is continuing joined on, and `runAgentLoop` reads it into a failure's `partial`.
+
 `idleMs` is silence, not a deadline: the timer is rearmed on every chunk, so a model that is still
 talking is never cut off however long it takes, and one that has stopped answering raises
 `EndpointSilent` rather than hanging the run. `timeoutMs(config)` returns `undefined` for a
@@ -470,6 +475,7 @@ const { turn, messages, usage, loaded } = await runAgentLoop({
   messages: history,              // ending in the question; not written to
   tools,                          // every tool the run may reach
   catalog,                        // the same, name-only, for on-demand and proxied loading
+  hostTools,                      // the host's own, declared in every mode and never catalogued
   serverInstructions: true,       // the catalogue's `instructions` after `system`; off by default
   preselected,                    // from `preselect`, if a small model chose
   dispatch: ({ name, args }, signal) => pool.call(name, args, signal),
@@ -688,6 +694,27 @@ a `tool-call` and a `tool-result` event, told to `onToolCall` and `onToolResult`
 id is numbered by where the call lands in the transcript, so two questions do not share one. A name
 with no definition in `tools`, or outside the catalogue, is left out, and none left is no exchange.
 
+A host with tools of its own — a question for a person, a resource read — passes them as
+`hostTools`, and they are declared on every step in every mode: after `PROXY_TOOLS` proxied, after
+`load_tools` and ahead of anything loaded on demand, after `tools` in eager mode, and after the
+shortlist on a preselected first step. They reach `dispatch` like any other call and are kept out
+of discovery: taken out of `catalog`, `loaded` and `preselected` before the run starts, never
+counted as a load, and a `call_tool` naming one is refused with a pointer to call it directly.
+Proxied and eager, the array is one set for the whole run, so the cache holds whatever `toolOrder`
+does with it; on demand, `toolOrder: false` keeps them in the prefix every load appends after.
+
+```ts
+await runAgentLoop({
+  config: { ...config, toolDiscovery: "proxy" },
+  messages: history,
+  tools: pool.tools(),
+  catalog: pool.catalog(),
+  hostTools: [ASK_PERSON_DEFINITION], // declared beside load_tools and call_tool, on every step
+  dispatch: (call, signal) =>
+    call.name === ASK_PERSON ? ask(call.args) : pool.call(call.name, call.args, signal),
+});
+```
+
 ### What the servers said on the way
 
 A server can log while a call runs — a warning that the cache was stale, an error it recovered
@@ -924,7 +951,7 @@ try {
       store.add(sessionId, turn?.reasoning ? { ...message, reasoning_content: turn.reasoning } : message),
   });
 } catch (error) {
-  const run = failedRun(error); // messages, usage, toolCalls, loaded, used, preselected — or undefined
+  const run = failedRun(error); // messages, partial, usage, toolCalls, loaded, used, preselected — or undefined
   if (run) await store.keep(sessionId, { usage: run.usage, loaded: run.loaded });
   if (signal.aborted) return; // a stop, told by the signal as it always was
   if (error instanceof ContextOverflow) return compactAndRetry(run?.messages);
@@ -951,9 +978,14 @@ A failure while a step's tools are running leaves a transcript that can be sent
 that came back are kept, a call the run was stopped during is answered `Stopped before this call
 finished.`, and one the loop never reached `Not run: the run stopped first.` — a call with no
 result is a transcript an endpoint refuses. Those are announced through `onMessage` like any other
-result. The reply of a request that was in flight when the run stopped is not kept: the host has
-the deltas from its `output` and `thinking` events, and whether half an answer belongs in the
-history is its call.
+result. The reply of a request that was in flight when the run stopped or failed — a stop, the
+endpoint going silent, a stream that broke after its first token — is not in `messages`, because it
+is not a turn an endpoint will take back. It is on the failure as `partial`: its `content`, its
+`reasoning` and the `toolCalls` it had begun, assembled as far as they had arrived, with a
+continuation's piece joined onto the answer it was continuing and, after a retry, only the attempt
+that was in flight. A request that had said nothing leaves no `partial`, and neither does a
+failure while the tools run, since that step's reply is already in `messages`. Whether half an
+answer belongs in the history is the host's call.
 
 `resolveApiKey` is exported and not applied, because which key an endpoint gets is a rule a
 consumer states and a library guessing it could send one where it was not meant to go. The rule

package/dist/index.d.ts

@@ -37,7 +37,7 @@ export { type RunTurnOptions, runTurn } from './turn/run-turn.ts';
 export { type AskJsonOptions, ask, askJson, askReply, clean, listLines, parseJson, resetHints, type SideTask, type SideTaskInput, type SideTaskOptions, type SideTaskReply, taskCall, tryAsk, } from './turn/side-task.ts';
 export { AgentLoopError, type AgentLoopFailure, AgentLoopOverflow, errorMessage, failedRun, ToolIterationLimit, } from './wire/errors.ts';
 export { backoffMs, ContextOverflow, EndpointSilent, isModelLoading, isOverflow, isTransient, SMALLEST_LIKELY_WINDOW, } from './wire/retry.ts';
-export { CacheBreakReason, type Produced, type StreamTurnOptions, streamTurn, type Turn, type TurnUsage, } from './wire/stream.ts';
+export { CacheBreakReason, type InFlight, type PartialTurn, type Produced, type StreamTurnOptions, streamTurn, type Turn, type TurnUsage, } from './wire/stream.ts';
 export { ALL_FENCES, DEFAULT_FENCES, type Fence, FenceSplitter, type FenceSplitterOptions, SPLIT_OUTPUT, SPLIT_REASONING, type Split, stripThinking, THINK_FENCE, } from './wire/thinking.ts';
 export { type ContextBreakdown, type ContextBreakdownOptions, compact, contextChars, contextTokens, estimateTokens, messageTokens, requestChars, requestTokens, type TokenEstimateOptions, } from './wire/tokens.ts';
 export { ARGUMENTS_MALFORMED, ARGUMENTS_TRUNCATED, type EndedBy, parseToolArguments, type ResourceLink, type RunArtifact, recoverToolCalls, type ServerLogEntry, ServerLogLevel, type ToolAnswer, ToolArgumentsError, type ToolCall, type ToolCallOutcome, type ToolCallReport, type ToolCallRequest, type ToolCallResult, } from './wire/tool-calls.ts';

package/dist/run/agent-loop.d.ts

@@ -152,6 +152,23 @@ export interface AgentLoopOptions {
      * loaded so far, by name; proxied mode declares none, and answers a load with them instead.
      */
     tools?: OpenAI.ChatCompletionTool[];
+    /**
+     * The host's own tools — a question for a person, a resource read — declared on every step in
+     * every discovery mode and dispatched like any other call, but never catalogued, loaded, or
+     * reached through `call_tool`.
+     *
+     * @remarks
+     * Each sits after the loop's own definitions: after `PROXY_TOOLS` proxied, after `load_tools`
+     * and ahead of anything loaded on demand, after `tools` in eager mode, and after the shortlist
+     * on a preselected first step; `toolOrder` then orders the array as it always does. Proxied and
+     * eager, the array is one set for the whole run, so the cache holds in either order; on demand,
+     * `toolOrder: false` keeps them in the prefix every load appends after.
+     *
+     * A name of one is taken out of `catalog`, `loaded` and `preselected` before the run starts, is
+     * never counted as a load, and is declared once where `tools` or the loop has it too. A
+     * `call_tool` naming one is refused with a pointer to call it directly.
+     */
+    hostTools?: OpenAI.ChatCompletionTool[];
     /**
      * The same tools as a name-only catalogue. On-demand and proxied modes need it, and are eager
      * without it.
@@ -560,7 +577,8 @@ export interface AgentLoopResult {
  * @remarks
  * What it throws carries the run as it stood — `messages`, `usage`, `toolCalls`, `loaded`, `used`
  * and `preselected` — because the steps before a failure were real and a host that stores its
- * transcript has nothing else to store them from. A spent budget is a `ToolIterationLimit`, an
+ * transcript has nothing else to store them from. What the request in flight had said is on it
+ * as `partial`, and not in `messages`. A spent budget is a `ToolIterationLimit`, an
  * overflow is an `AgentLoopOverflow`, which is still a `ContextOverflow`, and anything else is an
  * `AgentLoopError` with what was caught as its `cause` and its message as its own; `failedRun`
  * reads the run off any of the three. A failure while a step's tools run leaves the transcript
@@ -595,6 +613,8 @@ export interface AgentLoopResult {
  * tool array to go into, so it is written into the transcript as a `load_tools` exchange after
  * the question. `call_tool` is honoured on demand too, unless the host has a tool of that name —
  * a session switched out of proxied mode still has it in its history, and the model copies it.
+ * The host's own `hostTools` are declared beside whatever the mode sends, in every mode, and are
+ * never catalogued.
  *
  * Neither side task is run unless asked for. `preselect` has the loop make the preselection and
  * `compact` has it fold the transcript, each by the task `config.tasks` carries for it and on

package/dist/run/agent-loop.js

@@ -77,7 +77,8 @@ function cacheDiagnosis(previous, next, usage) {
  * @remarks
  * What it throws carries the run as it stood — `messages`, `usage`, `toolCalls`, `loaded`, `used`
  * and `preselected` — because the steps before a failure were real and a host that stores its
- * transcript has nothing else to store them from. A spent budget is a `ToolIterationLimit`, an
+ * transcript has nothing else to store them from. What the request in flight had said is on it
+ * as `partial`, and not in `messages`. A spent budget is a `ToolIterationLimit`, an
  * overflow is an `AgentLoopOverflow`, which is still a `ContextOverflow`, and anything else is an
  * `AgentLoopError` with what was caught as its `cause` and its message as its own; `failedRun`
  * reads the run off any of the three. A failure while a step's tools run leaves the transcript
@@ -112,6 +113,8 @@ function cacheDiagnosis(previous, next, usage) {
  * tool array to go into, so it is written into the transcript as a `load_tools` exchange after
  * the question. `call_tool` is honoured on demand too, unless the host has a tool of that name —
  * a session switched out of proxied mode still has it in its history, and the model copies it.
+ * The host's own `hostTools` are declared beside whatever the mode sends, in every mode, and are
+ * never catalogued.
  *
  * Neither side task is run unless asked for. `preselect` has the loop make the preselection and
  * `compact` has it fold the transcript, each by the task `config.tasks` carries for it and on
@@ -133,9 +136,10 @@ export async function runAgentLoop(options) {
         }),
     };
     let result;
+    const apart = hostToolsApart(options);
     try {
-        pin = await pinSession(options);
-        result = await runSteps(options, standing, pin);
+        pin = await pinSession(apart);
+        result = await runSteps(apart, standing, pin);
     }
     catch (error) {
         // Wrapped rather than annotated: an abort's reason is one object shared by every run under
@@ -399,6 +403,43 @@ function recorder(heard) {
     };
     return { recorded, onEvent };
 }
+/**
+ * The names of the host's own tools.
+ *
+ * @param [tools] - The host's definitions. One that is not a function has no name, and is left out.
+ * @returns A new set, empty where there are none.
+ */
+const hostToolNames = (tools = []) => new Set(tools.flatMap((tool) => toolName(tool) ?? []));
+/**
+ * The run's options with the host's own tools kept out of discovery: off the catalogue, and out of
+ * what was carried and what was picked.
+ *
+ * @param options - What `runAgentLoop` was given. Not written to.
+ * @returns `options` itself where there are no host tools, and otherwise a copy with `catalog`,
+ * `loaded` and `preselected` filtered, a server that offers none of them kept as it was.
+ *
+ * @remarks
+ * Done once, here, so that preselection, `load_tools`, `call_tool` and the catalogue prompt all
+ * read one catalogue, and a host tool is never loadable however a name of one reaches the loop.
+ */
+function hostToolsApart(options) {
+    const own = hostToolNames(options.hostTools);
+    if (own.size === 0) {
+        return options;
+    }
+    const isCatalogued = (name) => own.has(name) === false;
+    const { catalog, loaded, preselected } = options;
+    const apart = (server) => server.tools.every((entry) => isCatalogued(entry.name))
+        ? server
+        : { ...server, tools: server.tools.filter((entry) => isCatalogued(entry.name)) };
+    return {
+        ...options,
+        ...(catalog ? { catalog: catalog.map(apart) } : {}),
+        ...(loaded ? { loaded: [...loaded].filter(isCatalogued) } : {}),
+        // Undefined stays undefined: an empty shortlist is a host saying it has already decided.
+        ...(preselected ? { preselected: preselected.filter(isCatalogued) } : {}),
+    };
+}
 /** What opens the notice naming the loop's own picks, ahead of the names. */
 export const PRESELECTED_NOTICE = 'tools picked before the run: ';
 /**
@@ -570,11 +611,17 @@ async function runSteps(options, standing, pin) {
     const summarise = ownSummariser(options, notice);
     const used = new Set();
     const definitions = definitionsByName(tools);
+    // Each once, by name, so one given twice is declared once.
+    const hostTools = loadedTools([], options.hostTools ?? []);
+    const hostNames = hostToolNames(hostTools);
+    // After the head the mode declares, which a host tool of the same name does not displace.
+    const withHost = (head) => (hostTools.length ? loadedTools(head, hostTools) : head);
     // Looked through unless the host has a tool by that name, which is then the host's to answer.
-    const proxies = onDemand && !definitions.has(CALL_TOOL);
+    const proxies = onDemand && !definitions.has(CALL_TOOL) && !hostNames.has(CALL_TOOL);
     // What a call the model wrote as text may name: the host's tools, and the loop's own on demand.
     const recoverable = [
         ...tools.map(toolName).filter((name) => name !== undefined),
+        ...hostNames,
         ...(onDemand ? [LOAD_TOOLS, ...(proxies ? [CALL_TOOL] : [])] : []),
     ];
     // In the order the names are given, not the order of `tools`: `loaded` is a set, which iterates
@@ -584,16 +631,23 @@ async function runSteps(options, standing, pin) {
     let usage = noUsage();
     const toolCalls = [];
     const artifacts = new Map();
-    standing.read = () => ({
-        messages,
-        usage,
-        toolCalls,
-        artifacts: [...artifacts.values()],
-        // As the result would have said it: proxied, nothing is loaded that the history does not hold.
-        loaded: proxied ? [] : [...loaded],
-        used: [...used],
-        preselected: [...preselected],
-    });
+    // A box per request, emptied once its turn is in `messages`: a failure while the tools run had
+    // no request in flight.
+    let inFlight = {};
+    standing.read = () => {
+        const partial = inFlight.read?.();
+        return {
+            messages,
+            ...(partial ? { partial } : {}),
+            usage,
+            toolCalls,
+            artifacts: [...artifacts.values()],
+            // As the result would have said it: proxied, nothing is loaded that the history does not hold.
+            loaded: proxied ? [] : [...loaded],
+            used: [...used],
+            preselected: [...preselected],
+        };
+    };
     const announce = announcer(onMessage);
     // Held by reference rather than by index, so a `beforeStep` that folds the head into a summary
     // moves the question without losing it — and one that summarises the question away takes the
@@ -626,6 +680,7 @@ async function runSteps(options, standing, pin) {
         proxied,
         proxies,
         definitions,
+        hostNames,
         loaded,
         loads,
         used,
@@ -691,15 +746,15 @@ async function runSteps(options, standing, pin) {
         // Ordered here rather than left to `buildBody`, so `names` below is what the request actually
         // declared — a diagnosis reading an order the server never saw calls an untouched tool array
         // `tools-changed`.
-        let offered = tools;
+        let offered = withHost(tools);
         if (routed) {
-            offered = byName(new Set(preselected));
+            offered = withHost(byName(new Set(preselected)));
         }
         else if (proxied) {
-            offered = [...PROXY_TOOLS];
+            offered = withHost([...PROXY_TOOLS]);
         }
         else if (onDemand) {
-            offered = loadedTools([LOAD_TOOLS_DEFINITION], byName(loaded));
+            offered = loadedTools(withHost([LOAD_TOOLS_DEFINITION]), byName(loaded));
         }
         const declared = orderTools(offered, toolOrder);
         // Unmarked, so the system prompt is the same text on every step and a load does not throw
@@ -717,8 +772,10 @@ async function runSteps(options, standing, pin) {
             ...withContext(messages, questionAt(), gathered.context, preface),
         ];
         const build = (supported, refused) => buildBody(sending, supported, refused, request, declared, toolOrder);
+        inFlight = {};
         const turnOptions = {
             model: config.model,
+            inFlight,
             droppable: Object.keys(extraBody ?? {}),
             ...(pin ? { headers: pin.headers } : {}),
             maxRetries,
@@ -772,6 +829,7 @@ async function runSteps(options, standing, pin) {
         const turn = maxContinuations > 0
             ? await continueTurn(client, supports, build, first, { ...turnOptions, maxContinuations })
             : first;
+        inFlight = {};
         previous = {
             messages: request,
             tools: names,
@@ -783,7 +841,7 @@ async function runSteps(options, standing, pin) {
         if (turn.finishReason === FinishReason.Length) {
             notice(`the model stopped at maxTokens (${config.maxTokens}); this turn is cut short`);
         }
-        const canRecover = recover && (tools.length > 0 || onDemand);
+        const canRecover = recover && (tools.length > 0 || hostTools.length > 0 || onDemand);
         const { calls, content } = turnCalls(turn, canRecover ? recoverable : undefined, notice);
         const shown = { ...turn, content, toolCalls: calls };
         onTurn?.(shown, step);

package/dist/run/run-calls.d.ts

@@ -86,6 +86,8 @@ export interface Calling {
     /** Whether a `call_tool` is looked through, which it is unless the host has a tool by that name. */
     proxies: boolean;
     definitions: ReadonlyMap<string, OpenAI.ChatCompletionTool>;
+    /** The host's own tools by name: declared on every step, and never reached through `call_tool`. */
+    hostNames: ReadonlySet<string>;
     loaded: Set<string>;
     loads: {
         toolsLoaded: number;

package/dist/run/run-calls.js

@@ -256,7 +256,12 @@ function prepare(run, entry) {
     // nothing a load could not, and the refusal is the call's answer.
     let inner = args ?? {};
     let refused;
-    if (proxy && args) {
+    if (proxy && args && run.hostNames.has(name)) {
+        // Declared already, so the model is pointed at the call it can make rather than told the
+        // tool does not exist.
+        refused = { error: new Error(`${name} is not in the catalogue; call it directly, not through ${CALL_TOOL}.`) };
+    }
+    else if (proxy && args) {
         try {
             inner = proxiedCall(args, catalog).input;
         }

package/dist/turn/continuation.d.ts

@@ -62,6 +62,8 @@ export declare const isContinuable: (turn: Turn) => boolean;
  * server that takes the request and ignores the prefill — hosted OpenAI among them — shows itself;
  * that check is only as good as a restart being word for word. Either way the answer so far is
  * kept, with a notice. So is it when the continuation fails any other way, since the tokens
- * already in hand are worth more than the error; only a stop is thrown.
+ * already in hand are worth more than the error; only a stop is thrown. An `inFlight` box reads
+ * the answer so far with the continuation's piece joined on, so a stop mid-continuation keeps it
+ * all.
  */
 export declare function continueTurn(client: OpenAI, supports: Capabilities, request: RequestBuilder, turn: Turn, { maxContinuations, ...options }?: ContinueTurnOptions): Promise<Turn>;

package/dist/turn/continuation.js

@@ -97,6 +97,22 @@ function joinUsage(first, next) {
     }
     return joined;
 }
+/**
+ * The answer so far with the piece a continuation in flight had added to it.
+ *
+ * @param sofar - The turn being continued, which may itself be a joined one. Not written to.
+ * @param [piece] - What the continuation had said. Absent adds nothing.
+ * @returns The two joined as `continueTurn` joins turns: content and reasoning in order, and the
+ * continuation's own calls. `reasoning` and `toolCalls` are left out where empty.
+ */
+function joinPartial(sofar, piece) {
+    const reasoning = sofar.reasoning + (piece?.reasoning ?? '');
+    return {
+        content: sofar.content + (piece?.content ?? ''),
+        ...(reasoning ? { reasoning } : {}),
+        ...(piece?.toolCalls ? { toolCalls: piece.toolCalls } : {}),
+    };
+}
 /**
  * Carries on an answer the token ceiling cut off, by sending the transcript again with the answer
  * so far as a trailing assistant message, and joins the pieces into one turn.
@@ -124,17 +140,28 @@ function joinUsage(first, next) {
  * server that takes the request and ignores the prefill — hosted OpenAI among them — shows itself;
  * that check is only as good as a restart being word for word. Either way the answer so far is
  * kept, with a notice. So is it when the continuation fails any other way, since the tokens
- * already in hand are worth more than the error; only a stop is thrown.
+ * already in hand are worth more than the error; only a stop is thrown. An `inFlight` box reads
+ * the answer so far with the continuation's piece joined on, so a stop mid-continuation keeps it
+ * all.
  */
 export async function continueTurn(client, supports, request, turn, { maxContinuations = 1, ...options } = {}) {
     const refused = options.model === undefined ? undefined : modelCapabilitiesFor(supports, options.model);
     const who = modelLabel(options.model);
+    // The caller's box reads the whole answer, not only the piece the continuation streams.
+    const outer = options.inFlight;
+    const piece = {};
+    const told = outer ? { inFlight: piece } : {};
     let joined = turn;
     for (let count = 0; count < maxContinuations && isContinuable(joined); count++) {
         if (refused?.assistantPrefill === false) {
             break;
         }
         const answer = joined.content;
+        if (outer) {
+            const sofar = joined;
+            piece.read = undefined;
+            outer.read = () => joinPartial(sofar, piece.read?.());
+        }
         let next;
         try {
             next = await runTurn(client, supports, (capabilities, forModel) => {
@@ -143,7 +170,7 @@ export async function continueTurn(client, supports, request, turn, { maxContinu
                     ...body,
                     messages: [...body.messages, { role: Role.Assistant, content: answer }],
                 };
-            }, { ...options, startInReasoning: false });
+            }, { ...options, ...told, startInReasoning: false });
         }
         catch (error) {
             if (options.signal?.aborted) {

package/dist/wire/errors.d.ts

@@ -1,6 +1,6 @@
 import type OpenAI from 'openai';
 import { ContextOverflow } from './retry.ts';
-import type { TurnUsage } from './stream.ts';
+import type { PartialTurn, TurnUsage } from './stream.ts';
 import type { RunArtifact, ToolCallOutcome } from './tool-calls.ts';
 /**
  * What went wrong, as a string.
@@ -27,9 +27,19 @@ export interface AgentLoopFailure {
     /**
      * The transcript, as the result's `messages` would have been: every assistant turn and tool
      * result the run added, and no system prompt. Well-formed — a call the run was stopped during
-     * has a result saying so. The reply of a request that was in flight is not in it.
+     * has a result saying so. The reply of a request that was in flight is not in it: see `partial`.
      */
     messages: OpenAI.ChatCompletionMessageParam[];
+    /**
+     * What the request in flight had said when the run stopped or failed — a stop, the endpoint
+     * going silent, a stream that broke — continuations joined on. Absent where no request was in
+     * flight or it had said nothing.
+     *
+     * @remarks
+     * Not in `messages`, because it is not a turn an endpoint will take back. It is what a person
+     * watching had been shown, and whether it belongs in the history is the host's call.
+     */
+    partial?: PartialTurn;
     /** Summed over the turns that came back. A request that failed or was stopped adds nothing. */
     usage: TurnUsage;
     /** Every call that was made, in order. One the run was stopped during is `ok: false`. */
@@ -59,6 +69,7 @@ export interface AgentLoopFailure {
 export declare class AgentLoopError extends Error implements AgentLoopFailure {
     readonly name: string;
     readonly messages: OpenAI.ChatCompletionMessageParam[];
+    readonly partial?: PartialTurn;
     readonly usage: TurnUsage;
     readonly toolCalls: ToolCallOutcome[];
     readonly artifacts: RunArtifact[];
@@ -93,6 +104,7 @@ export declare class ToolIterationLimit extends AgentLoopError {
  */
 export declare class AgentLoopOverflow extends ContextOverflow implements AgentLoopFailure {
     readonly messages: OpenAI.ChatCompletionMessageParam[];
+    readonly partial?: PartialTurn;
     readonly usage: TurnUsage;
     readonly toolCalls: ToolCallOutcome[];
     readonly artifacts: RunArtifact[];

package/dist/wire/errors.js

@@ -24,6 +24,7 @@ export const errorMessage = (error) => (error instanceof Error ? error.message :
 export class AgentLoopError extends Error {
     name = 'AgentLoopError';
     messages;
+    partial;
     usage;
     toolCalls;
     artifacts;
@@ -38,6 +39,9 @@ export class AgentLoopError extends Error {
     constructor(message, run, options) {
         super(message, options);
         this.messages = run.messages;
+        if (run.partial) {
+            this.partial = run.partial;
+        }
         this.usage = run.usage;
         this.toolCalls = run.toolCalls;
         this.artifacts = run.artifacts ?? [];
@@ -67,6 +71,7 @@ export class ToolIterationLimit extends AgentLoopError {
  */
 export class AgentLoopOverflow extends ContextOverflow {
     messages;
+    partial;
     usage;
     toolCalls;
     artifacts;
@@ -81,6 +86,9 @@ export class AgentLoopOverflow extends ContextOverflow {
     constructor(message, run, options) {
         super(message, options);
         this.messages = run.messages;
+        if (run.partial) {
+            this.partial = run.partial;
+        }
         this.usage = run.usage;
         this.toolCalls = run.toolCalls;
         this.artifacts = run.artifacts ?? [];

package/dist/wire/stream.d.ts

@@ -213,6 +213,39 @@ export interface Turn {
 export interface Produced {
     any: boolean;
 }
+/**
+ * What a turn had said when it ended without coming back: the answer, the scratchpad and the calls,
+ * as far as each had arrived.
+ *
+ * @remarks
+ * It is not a `Turn`, and not a message an endpoint will take back: it has no usage, no reason it
+ * ended, and a call in it may stop mid-JSON. It is what a person watching had been shown, for a
+ * host to keep or to show.
+ */
+export interface PartialTurn {
+    /** The answer so far, fences taken out; `""` where it had not begun. */
+    content: string;
+    /** The scratchpad so far, from its own field or a fence. Absent where none had arrived. */
+    reasoning?: string;
+    /**
+     * The calls begun, assembled as `Turn.toolCalls` would have been — a name may be missing and
+     * arguments may stop mid-JSON. Absent where none had begun.
+     */
+    toolCalls?: ToolCall[];
+}
+/**
+ * Where the request in flight can be read from, after it has thrown.
+ *
+ * @remarks
+ * A box for the reason `Produced` is one: a rejected promise carries no stream state, and the
+ * error is often the signal's reason, which is shared by every run under it. Each `streamTurn`
+ * sets `read` as it starts, so after a retry or a downgrade it reads the latest attempt and
+ * nothing of the one before.
+ */
+export interface InFlight {
+    /** What the stream had said, or `undefined` where it had said nothing. */
+    read?: () => PartialTurn | undefined;
+}
 /** What `streamTurn` takes besides the request body. */
 export interface StreamTurnOptions {
     /** Cancels the request and the stream being read from it. */
@@ -235,6 +268,11 @@ export interface StreamTurnOptions {
     firstChunkMs?: number;
     /** Set by the first chunk that carries anything, so a failed call knows if it can be retried. */
     produced?: Produced;
+    /**
+     * Given a `read` of what this call has said so far, as the call starts. Read it once the call has
+     * thrown; a turn that came back has it all, and the box is not cleared.
+     */
+    inFlight?: InFlight;
     /**
      * The fences that mark a scratchpad written into `content`.
      *
@@ -282,4 +320,4 @@ export interface StreamTurnOptions {
  * which run it is: `step` is the caller's flow concept, and wrapping these into an `emit` is one
  * line at the call site.
  */
-export declare function streamTurn(client: OpenAI, body: OpenAI.ChatCompletionCreateParamsStreaming, { signal, idleMs, firstChunkMs, produced, fences, startInReasoning, onThinking, onOutput, headers, }?: StreamTurnOptions): Promise<Turn>;
+export declare function streamTurn(client: OpenAI, body: OpenAI.ChatCompletionCreateParamsStreaming, { signal, idleMs, firstChunkMs, produced, inFlight, fences, startInReasoning, onThinking, onOutput, headers, }?: StreamTurnOptions): Promise<Turn>;

package/dist/wire/stream.js

@@ -188,6 +188,25 @@ function assembledCalls(calls) {
 }
 /** The largest delay a timer takes, which is as close to none as the SDK's timeout option goes. */
 const NO_SDK_TIMEOUT = 2_147_483_647;
+/**
+ * A partial turn from what a stream had put together, or nothing where it had put nothing.
+ *
+ * @param content - The answer so far.
+ * @param reasoning - The scratchpad so far.
+ * @param calls - The calls begun, as they arrived. Not written to.
+ * @returns `undefined` where all three are empty; otherwise the content, with `reasoning` and
+ * `toolCalls` only where they hold something.
+ */
+function partialTurn(content, reasoning, calls) {
+    if (content === '' && reasoning === '' && calls.length === 0) {
+        return undefined;
+    }
+    return {
+        content,
+        ...(reasoning ? { reasoning } : {}),
+        ...(calls.length ? { toolCalls: assembledCalls(calls) } : {}),
+    };
+}
 /**
  * Runs one turn as a stream, reporting tokens as they arrive and assembling them back into a
  * message.
@@ -208,7 +227,7 @@ const NO_SDK_TIMEOUT = 2_147_483_647;
  * which run it is: `step` is the caller's flow concept, and wrapping these into an `emit` is one
  * line at the call site.
  */
-export async function streamTurn(client, body, { signal, idleMs, firstChunkMs, produced, fences = DEFAULT_FENCES, startInReasoning, onThinking, onOutput, headers, } = {}) {
+export async function streamTurn(client, body, { signal, idleMs, firstChunkMs, produced, inFlight, fences = DEFAULT_FENCES, startInReasoning, onThinking, onOutput, headers, } = {}) {
     // Silence, not duration: the timer is rearmed on every chunk, so a model that is still
     // talking is never cut off however long it takes, and one that has stopped talking does not
     // hang the run until someone notices. A request that never answers at all is the same case
@@ -229,6 +248,10 @@ export async function streamTurn(client, body, { signal, idleMs, firstChunkMs, p
             idle = setTimeout(() => watchdog.abort(), ms);
         }
     };
+    if (inFlight) {
+        // Before the request, so a box an earlier attempt filled reads nothing of it.
+        inFlight.read = () => undefined;
+    }
     try {
         rearm(false);
         const started = Date.now();
@@ -271,6 +294,11 @@ export async function streamTurn(client, body, { signal, idleMs, firstChunkMs, p
         // In arrival order, sorted by index at the end; a call from a server that sent none keeps
         // its place in the order they arrived.
         const calls = [];
+        if (inFlight) {
+            // Lazy, so a stream nobody stops pays nothing for it. Text held back as a possible tag is
+            // left out: it is a few characters that may yet be a marker.
+            inFlight.read = () => partialTurn(splitter.output, reasoning.join('') + splitter.reasoning, calls);
+        }
         const usage = noUsage();
         // The usage block's own cache count wins over llama.cpp's, which is the same number in the
         // servers that send both; kept apart so the order the two arrive in does not decide.

package/llms.txt

@@ -387,6 +387,8 @@ Everything about a request failing that is not about what the request said.
 Reading one streamed turn back into a message.
 
 - `CacheBreakReason` — Any one of the reasons in `CacheBreakReason`.
+- `InFlight` (type) — Where the request in flight can be read from, after it has thrown.
+- `PartialTurn` (type) — What a turn had said when it ended without coming back: the answer, the scratchpad and the calls, as far as each had arrived.
 - `Produced` (type) — Whether the model has said anything a second attempt would say twice.
 - `StreamTurnOptions` (type) — What `streamTurn` takes besides the request body.
 - `streamTurn` — Runs one turn as a stream, reporting tokens as they arrive and assembling them back into a message.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cubicecho/agent-core",
-  "version": "4.16.0",
+  "version": "4.18.0",
   "description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
   "keywords": [
     "openai",