@cubicecho/agent-core 2.20.0 → 2.21.1

package/README.md

@@ -44,7 +44,7 @@ only, Node >=22.
 | `spec` | `parseSpec`, `resolveAgentSpec` and `exportSpec`: an agent as a versioned JSON document, layered into the flat config the loop takes. Imports nothing but types, and is published separately at `@cubicecho/agent-core/spec`. |
 | `reset` | `resetAll`: drops every cache and latch in one call, so a teardown cannot forget one. |
 | `tokens` | `estimateTokens`: characters over four, deliberately low, for everything here that has to guess at a window. |
-| `errors` | `errorMessage`: a caught `unknown` turned into something a run row can hold. |
+| `errors` | `errorMessage`: a caught `unknown` turned into something a run row can hold. And what `runAgentLoop` throws, with the run as it stood: `AgentLoopError`, `ToolIterationLimit`, `AgentLoopOverflow`, and `failedRun` to read it off any of them. |
 | `catalog` | `CatalogServer`: the name-only shape `tool-loading` reads a connected server as. |
 
 What is **not** here is the work: orchestration, prompts, and whatever the run is about. That
@@ -279,15 +279,24 @@ const turn = await runTurn(client, supports, build, {
 });
 ```
 
-`contextLimitFor` answers the operator's number when there is one. Otherwise it asks for the window
-the server is actually serving the model in (`servedWindow`): llama.cpp's `/props`
-(`default_generation_settings.n_ctx`) and LM Studio's `/api/v0/models` (`loaded_context_length`).
-That differs from the trained window in the case the guard exists for, a 256k model started at `-c
-16384`. A server with neither route is latched and not asked again. Failing both, it reads the
-`/v1/models` listing: `max_model_len` from vLLM, `context_length` from OpenRouter, and llama.cpp's
-`meta.n_ctx_train`, which is only the trained window. Ollama reports no window on any route this
-reads, and truncates an over-long prompt rather than refusing it, so on Ollama pass `contextLength`
-or there is no guard at all.
+`contextLimitFor` answers the operator's number when there is one. Otherwise it reads the
+`/v1/models` listing first. A window at the top of the model's entry (`context_length`,
+`max_context_window`, `max_model_len`, `context_window` or `n_ctx`: vLLM's `max_model_len`,
+OpenRouter's `context_length`) is taken as the one being served, and nothing else is asked.
+
+Where the listing does not settle it, the server's own API is asked (`servedWindow`): llama.cpp's
+`/props` (`default_generation_settings.n_ctx`), then LM Studio's `/api/v0/models`
+(`loaded_context_length`). That covers a model listed with no window, which is LM Studio, and one
+listed with only `meta.n_ctx_train`, which is llama.cpp. The trained window differs from the served
+one in the case the guard exists for, a 256k model started at `-c 16384`, so it is used only when
+neither native route gives a number. A server with neither route is latched and not asked again.
+`/api/v0/models` is also dropped on its own: for the life of the process once it answers 404, 405
+or 501, and for `listingMissMs` after any other error.
+
+A top-level listing key is trusted without a second opinion, so on a server that puts a trained
+window there, pass `contextLength`. Ollama reports no window on any route this reads, and truncates
+an over-long prompt rather than refusing it, so on Ollama pass `contextLength` or there is no guard
+at all.
 
 What is weighed is the prompt plus the reply ceiling the body carries, under whichever spelling
 was chosen, because that is what the endpoint weighs: a 30k prompt into a 32k window with
@@ -673,6 +682,50 @@ with pre-turn work to overlap with the hooks — compaction, a preselection —
 applies `withContext` to the current question as well, calls `notify` after, and passes no `hooks`
 (see [Hooks](#hooks)).
 
+A host that stores its transcript — a row per message, so a crash or a stop mid-run still leaves
+readable history — gets each message as the loop appends it, and the run as it stood when the loop
+throws:
+
+```ts
+try {
+  await runAgentLoop({
+    ...options,
+    // Awaited: the row is written before the tools run, and before the next request is sent.
+    onMessage: (message, step, turn) =>
+      store.add(sessionId, turn?.reasoning ? { ...message, reasoning_content: turn.reasoning } : message),
+  });
+} catch (error) {
+  const run = failedRun(error); // messages, usage, toolCalls, loaded, used — or undefined
+  if (run) await store.keep(sessionId, { usage: run.usage, loaded: run.loaded });
+  if (signal.aborted) return; // a stop, told by the signal as it always was
+  if (error instanceof ContextOverflow) return compactAndRetry(run?.messages);
+  if (error instanceof ToolIterationLimit) return outOfSteps();
+  throw error; // an AgentLoopError: the message is the cause's, and the cause is on `cause`
+}
+```
+
+`onMessage` hears the assistant message as it will be replayed — recovered calls folded in, and
+arguments repaired, which the turn `onTurn` is handed does not show — together with that `Turn`,
+and then each tool result, in transcript order. `onTurn` is not awaited and never sees a tool
+result; this is the one to store from. What it returns is ignored, since `beforeStep` is the one way to rewrite
+the transcript, and what it throws ends the run.
+
+Everything the loop throws carries the run. A spent `maxToolIterations` is a `ToolIterationLimit`,
+with the message it always had. A request too big for the window is an `AgentLoopOverflow`, which
+is a `ContextOverflow`, so a `catch` written for one still catches it. Anything else — a stop, a
+refused request, a dropped stream — is an `AgentLoopError` whose `cause` is what was caught and
+whose message is the cause's own. It is a wrapper and not fields on the error that was caught
+because a stop throws the signal's reason, one object for every run under that signal. A stop is
+still recognised by the signal the host aborted.
+
+A failure while a step's tools are running leaves a transcript that can be sent again. The results
+that came back are kept, a call the run was stopped during is answered `Stopped before this call
+finished.`, and one the loop never reached `Not run: the run stopped first.` — a call with no
+result is a transcript an endpoint refuses. Those are announced through `onMessage` like any other
+result. The reply of a request that was in flight when the run stopped is not kept: the host has
+the deltas from its `output` and `thinking` events, and whether half an answer belongs in the
+history is its call.
+
 `resolveApiKey` is exported and not applied, because which key an endpoint gets is a rule a
 consumer states and a library guessing it could send one where it was not meant to go. The rule
 it encodes is the conservative one: an endpoint's own key wins; one that names a base URL of its

package/dist/agent-loop.d.ts

@@ -354,6 +354,19 @@ export interface AgentLoopOptions {
     onRequest?: (request: AgentLoopRequest) => void;
     /** Each turn as it comes back, before its tools run. Recovered calls are in it as calls. */
     onTurn?: (turn: Turn, step: number) => void;
+    /**
+     * Each message as the loop appends it to the transcript, in transcript order, and awaited before
+     * the loop goes on — so a host that stores its transcript has written the assistant message
+     * before its tools run, and every tool result before the next request is sent.
+     *
+     * The assistant message is the one replayed on later requests — arguments repaired, recovered
+     * calls folded in — and comes with its `turn`, as `onTurn` was handed it, for the `reasoning` a
+     * host keeps beside the message. A tool result comes without one, and so do the two messages
+     * a proxied run writes for its `preselected` shortlist, told as step zero's before any request.
+     * What it returns is ignored: `beforeStep` is the one way to rewrite the transcript. What it
+     * throws ends the run, and it is not called again for that run.
+     */
+    onMessage?: (message: OpenAI.ChatCompletionMessageParam, step: number, turn?: Turn) => void | Promise<void>;
     /**
      * How many times an answer cut off at `maxTokens` is continued, zero — the default — for never.
      * Opt-in because it spends another request, and on a server that does not continue a trailing
@@ -423,6 +436,15 @@ export interface AgentLoopResult {
  * until a turn asks for none. Throws when `maxToolIterations` is spent, when stopped, and on
  * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
  *
+ * What it throws carries the run as it stood — `messages`, `usage`, `toolCalls`, `loaded` and
+ * `used` — because the steps before a failure were real and a host that stores its transcript
+ * has nothing else to store them from. A spent budget is a `ToolIterationLimit`, an overflow is
+ * an `AgentLoopOverflow`, which is still a `ContextOverflow`, and anything else is an
+ * `AgentLoopError` with what was caught as its `cause` and its message as its own; `failedRun`
+ * reads the run off any of the three. A failure while a step's tools run leaves the transcript
+ * well-formed: the results that came back are kept, and each call without one is answered with a
+ * line saying it was stopped or never run, since a call with no result is refused on replay.
+ *
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
  * prompt unchanged from step to step, a load adds to the tool array — which every request sends
  * in the stable order `toolOrder` asks for — and

package/dist/agent-loop.js

@@ -2,11 +2,11 @@ import { charsPerTokenFor } from "./calibration.js";
 import { capabilitiesFor, effortFor, modelCapabilitiesFor, } from "./capabilities.js";
 import { firstTokenMs, getClient, NO_KEY, timeoutMs } from "./client.js";
 import { continueTurn } from "./continuation.js";
-import { errorMessage } from "./errors.js";
+import { AgentLoopError, AgentLoopOverflow, errorMessage, ToolIterationLimit, } from "./errors.js";
 import { runMetrics } from "./events.js";
 import { configureHooks, gather, notify, turnIndex, turnMessages, withContext, } from "./hooks.js";
 import { rebaseLedger, recordRequest } from "./ledger.js";
-import { contextTokens, toolsChars } from "./retry.js";
+import { ContextOverflow, contextTokens, toolsChars } from "./retry.js";
 import { runTurn } from "./run-turn.js";
 import { relaxTools, sanitizeTools } from "./schema-compat.js";
 import { askJson, tryAsk } from "./side-task.js";
@@ -198,11 +198,24 @@ function cacheDiagnosis(previous, messages, tools, usage) {
         found.cacheBreakReason = breakReason(previous, messages, tools);
     return found;
 }
+/** What a tool call the run was stopped during is answered with, in place of a result. */
+const STOPPED_CALL = "Stopped before this call finished.";
+/** The same for a call of that step the loop never got to. */
+const UNRUN_CALL = "Not run: the run stopped first.";
 /**
  * Runs a question to its answer: one `runTurn` per step, the tools it asks for between them,
  * until a turn asks for none. Throws when `maxToolIterations` is spent, when stopped, and on
  * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
  *
+ * What it throws carries the run as it stood — `messages`, `usage`, `toolCalls`, `loaded` and
+ * `used` — because the steps before a failure were real and a host that stores its transcript
+ * has nothing else to store them from. A spent budget is a `ToolIterationLimit`, an overflow is
+ * an `AgentLoopOverflow`, which is still a `ContextOverflow`, and anything else is an
+ * `AgentLoopError` with what was caught as its `cause` and its message as its own; `failedRun`
+ * reads the run off any of the three. A failure while a step's tools run leaves the transcript
+ * well-formed: the results that came back are kept, and each call without one is answered with a
+ * line saying it was stopped or never run, since a call with no result is refused on replay.
+ *
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
  * prompt unchanged from step to step, a load adds to the tool array — which every request sends
  * in the stable order `toolOrder` asks for — and
@@ -231,8 +244,39 @@ function cacheDiagnosis(previous, messages, tools, usage) {
  * and cancellation. See `AgentLoopOptions`.
  */
 export async function runAgentLoop(options) {
+    // Until the steps have set themselves up there is only what the caller handed over.
+    const standing = {
+        read: () => ({
+            messages: [...options.messages],
+            usage: { prompt: 0, completion: 0, total: 0, cached: 0 },
+            toolCalls: [],
+            loaded: [],
+            used: [],
+        }),
+    };
+    let result;
+    try {
+        result = await runSteps(options, standing);
+    }
+    catch (error) {
+        // Wrapped rather than annotated: an abort's reason is one object shared by every run under
+        // the signal, so the run cannot be hung on the error itself. An overflow keeps its class,
+        // because `instanceof ContextOverflow` is how a caller knows to compact and try again.
+        throw error instanceof ContextOverflow
+            ? new AgentLoopOverflow(error.message, standing.read(), { cause: error })
+            : new AgentLoopError(errorMessage(error), standing.read(), { cause: error });
+    }
+    if (result)
+        return result;
+    throw new ToolIterationLimit(`Stopped after ${options.config.maxToolIterations} tool iterations.`, standing.read());
+}
+/**
+ * The steps of `runAgentLoop`, throwing what they caught as it was caught. Resolves to nothing
+ * when `maxToolIterations` is spent, and tells `standing` how to read the run either way.
+ */
+async function runSteps(options, standing) {
     const { config, system = "", tools = [], catalog = [], dispatch, hooks, signal } = options;
-    const { onRequest, onTurn, onToolCall, onToolResult, beforeStep, parallel = false, recoverToolCalls: recover = true, maxContinuations = 0, toolOrder = true, dedupeToolCalls = true, preselectRouting = "exclusive", } = options;
+    const { onRequest, onTurn, onMessage, onToolCall, onToolResult, beforeStep, parallel = false, recoverToolCalls: recover = true, maxContinuations = 0, toolOrder = true, dedupeToolCalls = true, preselectRouting = "exclusive", } = options;
     const dedupable = typeof dedupeToolCalls === "function" ? dedupeToolCalls : () => dedupeToolCalls;
     const started = Date.now();
     // What the loop emitted, less the token deltas, for `runMetrics` at the end. Stamped here rather
@@ -284,6 +328,30 @@ export async function runAgentLoop(options) {
     // in the order things were added, so a load appends and never reshuffles what went before.
     const byName = (names) => [...names].flatMap((name) => definitions.get(name) ?? []);
     let messages = [...options.messages];
+    const usage = { prompt: 0, completion: 0, total: 0, cached: 0 };
+    const toolCalls = [];
+    standing.read = () => ({
+        messages,
+        usage,
+        toolCalls,
+        // As the result would have said it: proxied, nothing is loaded that the history does not hold.
+        loaded: proxied ? [] : [...loaded],
+        used: [...used],
+    });
+    // Once `onMessage` has thrown the run is ending on that, and the results still to be written
+    // into the transcript are not offered to a host that has just failed to take one.
+    let heard = true;
+    const announce = async (message, step, turn) => {
+        if (!onMessage || !heard)
+            return;
+        try {
+            await onMessage(message, step, turn);
+        }
+        catch (error) {
+            heard = false;
+            throw error;
+        }
+    };
     // Held by reference rather than by index, so a `beforeStep` that folds the head into a summary
     // moves the question without losing it — and one that summarises the question away takes the
     // hooks' context with it, which is right.
@@ -297,8 +365,6 @@ export async function runAgentLoop(options) {
         : { context: "", notes: [] };
     // Read once, so what the result hands back is what every step of this run said.
     const preface = hooks?.preface ?? configureHooks().preface;
-    const usage = { prompt: 0, completion: 0, total: 0, cached: 0 };
-    const toolCalls = [];
     const loads = { toolsLoaded: 0, redundantLoads: 0, unknownToolNames: 0 };
     let previous;
     let ledger = options.ledger ?? [];
@@ -328,11 +394,19 @@ export async function runAgentLoop(options) {
         loads.toolsLoaded += names.length;
         for (const name of names)
             loaded.add(name);
-        messages.push({
-            role: "assistant",
-            content: null,
-            tool_calls: [{ id, type: "function", function: { name: LOAD_TOOLS, arguments: args } }],
-        }, { role: "tool", tool_call_id: id, content });
+        const exchange = [
+            {
+                role: "assistant",
+                content: null,
+                tool_calls: [{ id, type: "function", function: { name: LOAD_TOOLS, arguments: args } }],
+            },
+            { role: "tool", tool_call_id: id, content },
+        ];
+        // Both written before either is announced, so a host that throws on the first leaves a call
+        // with its result. Told as step zero's, with no turn: no request was made for them.
+        messages.push(...exchange);
+        for (const message of exchange)
+            await announce(message, 0);
     }
     for (let step = 0; step < config.maxToolIterations; step++) {
         // A stop aborts the request in flight, but a tool call already handed off runs to its own
@@ -476,7 +550,7 @@ export async function runAgentLoop(options) {
                 return { call, error, normal: "{}" };
             }
         });
-        messages.push({
+        const assistant = {
             role: "assistant",
             content: content || null,
             ...(parsed.length
@@ -487,7 +561,9 @@ export async function runAgentLoop(options) {
                     })),
                 }
                 : {}),
-        });
+        };
+        messages.push(assistant);
+        await announce(assistant, step, shown);
         if (!calls.length) {
             let afterTurn = Promise.resolve([]);
             if (hooks) {
@@ -531,13 +607,15 @@ export async function runAgentLoop(options) {
         // Per step, not per run: the answer to a call made two steps ago was true before the tools in
         // between ran, and the file the model read may be the file it has since written.
         const answered = new Map();
-        const run = async ({ call, args, error: unreadable, normal }) => {
+        // A `call_tool` is shown, counted and dispatched as the tool it names. Read from the repaired
+        // arguments where there are any, since the model's own may be almost-JSON.
+        const named = ({ call, args, normal }) => proxies && call.function.name === CALL_TOOL
+            ? shownCall(CALL_TOOL, args ? normal : call.function.arguments)
+            : { name: call.function.name, input: call.function.arguments };
+        const run = async (entry) => {
+            const { call, args, error: unreadable, normal } = entry;
             const proxy = proxies && call.function.name === CALL_TOOL;
-            // A `call_tool` is shown, counted and dispatched as the tool it names. Read from the repaired
-            // arguments where there are any, since the model's own may be almost-JSON.
-            const { name, input: raw } = proxy
-                ? shownCall(CALL_TOOL, args ? normal : call.function.arguments)
-                : { name: call.function.name, input: call.function.arguments };
+            const { name, input: raw } = named(entry);
             // Looked through before the call is announced, so a host is told of the tool it names with
             // that tool's arguments. A name outside the catalogue is refused here, so `call_tool` reaches
             // nothing a load could not, and the refusal is the call's answer below.
@@ -608,23 +686,65 @@ export async function runAgentLoop(options) {
             onToolResult?.(result);
             return result;
         };
-        const outcomes = [];
-        if (parallel) {
-            signal?.throwIfAborted();
-            outcomes.push(...(await Promise.all(parsed.map(run))));
-        }
-        else {
-            for (const call of parsed) {
+        // By position, so a call that came back while another was being stopped is still found.
+        const outcomes = parsed.map(() => undefined);
+        // How many of the step's calls were handed off, and how many have their result in the
+        // transcript. Both count from the front, since results are written in the order asked.
+        let begun = 0;
+        let kept = 0;
+        const keep = async (id, content) => {
+            const result = {
+                role: "tool",
+                tool_call_id: id,
+                content,
+            };
+            messages.push(result);
+            kept++;
+            await announce(result, step);
+        };
+        try {
+            if (parallel) {
                 signal?.throwIfAborted();
-                outcomes.push(await run(call));
+                begun = parsed.length;
+                await Promise.all(parsed.map(async (call, at) => {
+                    outcomes[at] = await run(call);
+                }));
+                for (const { id, name, ok, content } of outcomes.flatMap((outcome) => outcome ?? [])) {
+                    toolCalls.push({ id, name, ok });
+                    await keep(id, content);
+                }
+            }
+            else {
+                for (const [at, call] of parsed.entries()) {
+                    signal?.throwIfAborted();
+                    begun = at + 1;
+                    const { id, name, ok, content } = await run(call);
+                    toolCalls.push({ id, name, ok });
+                    await keep(id, content);
+                }
             }
         }
-        for (const { id, name, ok, content } of outcomes) {
-            toolCalls.push({ id, name, ok });
-            messages.push({ role: "tool", tool_call_id: id, content });
+        catch (error) {
+            // A call with no result is a transcript no endpoint takes back, so every call of the step
+            // is answered before the throw: with what it returned where it had, and otherwise with a
+            // line saying it was stopped, or never run. Only a call that was made counts in `toolCalls`.
+            for (const [at, entry] of parsed.entries()) {
+                const { call } = entry;
+                if (at < kept)
+                    continue;
+                const outcome = outcomes[at];
+                // Under the name it was announced by, which for a `call_tool` is the tool it names.
+                if (at < begun) {
+                    toolCalls.push({ id: call.id, name: named(entry).name, ok: outcome?.ok ?? false });
+                }
+                // The run is already ending on `error`; a host that cannot take this result does not
+                // get to replace it.
+                await keep(call.id, outcome?.content ?? (at < begun ? STOPPED_CALL : UNRUN_CALL)).catch(() => { });
+            }
+            throw error;
         }
     }
-    throw new Error(`Stopped after ${config.maxToolIterations} tool iterations.`);
+    return undefined;
 }
 /**
  * Makes a call at most once per key, sharing the in-flight promise so two identical calls in one

package/dist/client.d.ts

@@ -18,7 +18,8 @@ export interface ClientPoolOptions {
     maxClients?: number;
     /**
      * How long, in milliseconds, a model an endpoint did not list — or a server that answered
-     * without a served window — is taken at its word before the endpoint is asked again.
+     * without a served window, or whose `/api/v0/models` answered with an error other than a
+     * missing route — is taken at its word before the endpoint is asked again.
      */
     listingMissMs?: number;
 }
@@ -66,7 +67,12 @@ export declare const firstTokenMs: (config: Pick<Endpoint, "requestTimeoutSecond
  * @param config Where to send requests and how long to wait. An absent `apiKey` becomes `NO_KEY`.
  */
 export declare function getClient(config: Endpoint): OpenAI;
-/** A model an endpoint offers, and what it says the model will read. Zero means it did not say. */
+/**
+ * A model an endpoint offers, and what it says the model will read. Zero means it did not say.
+ *
+ * The number is whatever the listing carried, which from llama.cpp is the trained window rather
+ * than the served one; `contextLimitFor` is the lookup that knows the difference.
+ */
 export interface ModelInfo {
     id: string;
     contextLength: number;
@@ -109,7 +115,8 @@ export declare const endpointId: (config: {
     apiKey?: string;
 }) => string;
 /**
- * The window a local server is actually serving a model in, which its listing does not say.
+ * The window a local server's own API says it is serving a model in, for the servers whose
+ * listing does not say.
  *
  * llama.cpp reports it on `/props` as `default_generation_settings.n_ctx`, per slot, and LM
  * Studio on `/api/v0/models` as `loaded_context_length` while the model is loaded. Both differ
@@ -117,8 +124,12 @@ export declare const endpointId: (config: {
  * built for, and reading the trained one lets an overflow through the guard meant to catch it.
  * Ollama reports nothing on either; an operator there has to declare the window.
  *
- * Zero where no answer was found. A server without either route, which is every hosted API, is
- * latched and not asked again; one that could not be reached is not remembered at all.
+ * Zero where no answer was found, which is remembered for `listingMissMs`. A server without
+ * either route is latched and not asked again. `/api/v0/models` is also dropped on its own: until
+ * `resetClients` once it answers as a route that is not there, and for `listingMissMs` after any
+ * other error. A server that could not be reached is not remembered at all.
+ *
+ * The listing is not read here; `contextLimitFor` reads it first and comes here after.
  *
  * @param config The endpoint, plus the model whose window is wanted.
  */
@@ -139,11 +150,18 @@ export declare function listModels(config: Endpoint): Promise<ModelInfo[]>;
  * happily load a 256k model at `-c 16384` and go on listing it as 256k — and a run refused on
  * the honest-looking number is a run that fails at the endpoint instead.
  *
- * Otherwise the window the server is actually serving the model in, where it has an API that
- * says (`servedWindow`), and failing that the endpoint's listing — asked once, and again whenever
- * it does not name this model, since a model can arrive after the first listing was taken. A
- * server that will not list models still has to be able to run a turn: a failure here is an
- * unknown window, not a failed run.
+ * Otherwise the endpoint's listing — asked once, and again whenever it does not name this model,
+ * since a model can arrive after the first listing was taken. A window stated at the top of the
+ * model's entry is taken as the one being served, and nothing else is asked. Where the listing
+ * did not settle it — no window, only llama.cpp's trained one, no entry, no listing — the
+ * server's own API is asked (`servedWindow`), and what it says wins; the trained window is what
+ * is left when it says nothing. A server that will not list models still has to be able to run a
+ * turn: a failure here is an unknown window, not a failed run.
+ *
+ * The listing goes first because it is the one route every server here has, and the native ones
+ * are two more requests that a server stating its window in the listing never needed. Asked
+ * first, they were also asked again every `listingMissMs` for the life of the process on a server
+ * whose `/props` answers without a window, by any host that sizes the window per turn.
  *
  * @param config The endpoint, plus the model whose window is wanted.
  * @param declared The operator's own number. Above zero it wins and the endpoint is not asked.

package/dist/client.js

@@ -162,18 +162,26 @@ const CONTEXT_KEYS = [
     "n_ctx",
 ];
 const positive = (value) => (typeof value === "number" && value > 0 ? value : 0);
+/**
+ * A listing entry's window, and whether it is only the one the model was trained with.
+ *
+ * The two are told apart because they are trusted differently: a top-level key is taken as the
+ * window being served and settles the lookup, and a trained one is a ceiling the server may be
+ * running well under, so `contextLimitFor` still asks the native routes before settling for it.
+ */
 function contextLengthOf(model) {
     const record = model;
     for (const key of CONTEXT_KEYS) {
         const value = positive(record[key]);
         if (value)
-            return value;
+            return { contextLength: value, trained: false };
     }
     // llama.cpp's, and the window the model was trained with rather than the one it is served in:
     // a 256k model started at `-c 16384` lists 262144 here. Better than nothing, and why the
-    // served window is asked for first. `meta` is `null` while the model loads.
+    // served window is still asked for after it. `meta` is `null` while the model loads.
     const meta = record.meta;
-    return positive(meta?.n_ctx_train);
+    const contextLength = positive(meta?.n_ctx_train);
+    return { contextLength, trained: contextLength > 0 };
 }
 /**
  * The last listing from each endpoint, so a run can size its window without a round trip.
@@ -190,6 +198,13 @@ function contextLengthOf(model) {
  * whatever was there rather than emptying it.
  */
 const listings = new Map();
+/**
+ * The models in each endpoint's last listing whose window is only the trained one, by id.
+ *
+ * Kept beside `listings` rather than on `ModelInfo`, so what `listModels` hands back is the shape
+ * it has always been. Replaced with each listing that lands, and left alone by one that fails.
+ */
+const trainedOnly = new Map();
 /**
  * When an endpoint was last asked about a model it did not name, keyed on the two together.
  *
@@ -243,8 +258,25 @@ export const endpointId = (config) => createHash("sha256").update(endpointKey(co
  * is asked again after `listingMissMs`, like a listing that did not name the model.
  */
 const served = new Map();
-/** Endpoints that answered both probes with a refusal, and are not asked again. */
+/** Endpoints with neither native route, which are not asked for either again. */
 const unserved = new Set();
+/**
+ * Endpoints whose `/api/v0/models` answered with an error, and when.
+ *
+ * `gone` is a route the server does not have (`NOT_THERE`), which is a fact about the server and
+ * stands until `resetClients`. Any other error stands for `listingMissMs` only, because it may be
+ * one the server gets over, and LM Studio latched for good on a passing 5xx would have no window
+ * at all until the process restarted, its listing carrying none.
+ *
+ * Per endpoint and on its own, whatever `/props` said: a server whose `/props` answers without a
+ * window is never `unserved`, and without this its missing route was asked again with every miss.
+ */
+const refusals = new Map();
+/** Whether `/api/v0/models` is to be left alone on this endpoint for now. */
+const refused = (endpoint) => {
+    const refusal = refusals.get(endpoint);
+    return (refusal !== undefined && (refusal.gone || Date.now() - refusal.at < poolLimits.listingMissMs));
+};
 /** How long a probe may take before the window is taken from the listing instead. */
 const PROBE_TIMEOUT_MS = 10_000;
 /** The server root behind an OpenAI-compatible base URL, which is where the native APIs live. */
@@ -252,8 +284,9 @@ const rootOf = (baseUrl) => baseUrl.replace(/\/+$/, "").replace(/\/v1$/, "");
 /** A route this server does not have, rather than one that failed to answer. */
 const NOT_THERE = new Set([404, 405, 501]);
 /**
- * Asks one native endpoint. `missing` is a server without the route; `body` is absent for that
- * and for any other refusal, and a server that could not be reached throws.
+ * Asks one native endpoint. `failed` is any answer that was not a 2xx and `missing` the ones among
+ * them that mean a server without the route; `body` is absent for both, and for a 2xx that was not
+ * JSON. A server that could not be reached throws.
  */
 async function probe(config, path) {
     const apiKey = config.apiKey || undefined;
@@ -262,16 +295,17 @@ async function probe(config, path) {
         signal: AbortSignal.timeout(timeoutMs(config) ?? PROBE_TIMEOUT_MS),
     });
     if (!response.ok)
-        return { missing: NOT_THERE.has(response.status) };
+        return { failed: true, missing: NOT_THERE.has(response.status) };
     try {
-        return { missing: false, body: await response.json() };
+        return { failed: false, missing: false, body: await response.json() };
     }
     catch {
-        return { missing: false };
+        return { failed: false, missing: false };
     }
 }
 /**
- * The window a local server is actually serving a model in, which its listing does not say.
+ * The window a local server's own API says it is serving a model in, for the servers whose
+ * listing does not say.
  *
  * llama.cpp reports it on `/props` as `default_generation_settings.n_ctx`, per slot, and LM
  * Studio on `/api/v0/models` as `loaded_context_length` while the model is loaded. Both differ
@@ -279,8 +313,12 @@ async function probe(config, path) {
  * built for, and reading the trained one lets an overflow through the guard meant to catch it.
  * Ollama reports nothing on either; an operator there has to declare the window.
  *
- * Zero where no answer was found. A server without either route, which is every hosted API, is
- * latched and not asked again; one that could not be reached is not remembered at all.
+ * Zero where no answer was found, which is remembered for `listingMissMs`. A server without
+ * either route is latched and not asked again. `/api/v0/models` is also dropped on its own: until
+ * `resetClients` once it answers as a route that is not there, and for `listingMissMs` after any
+ * other error. A server that could not be reached is not remembered at all.
+ *
+ * The listing is not read here; `contextLimitFor` reads it first and comes here after.
  *
  * @param config The endpoint, plus the model whose window is wanted.
  */
@@ -300,13 +338,19 @@ export async function servedWindow(config) {
             ?.default_generation_settings;
         window = positive(settings?.n_ctx);
         if (!window) {
-            const lmstudio = await probe(config, "/api/v0/models");
-            const { data } = (lmstudio.body ?? {});
-            const entry = Array.isArray(data)
-                ? data.find((model) => model?.id === config.model)
-                : undefined;
-            window = positive(entry?.loaded_context_length);
-            if (props.missing && lmstudio.missing) {
+            if (!refused(endpoint)) {
+                const lmstudio = await probe(config, "/api/v0/models");
+                if (lmstudio.failed)
+                    refusals.set(endpoint, { at: Date.now(), gone: lmstudio.missing });
+                else
+                    refusals.delete(endpoint);
+                const { data } = (lmstudio.body ?? {});
+                const entry = Array.isArray(data)
+                    ? data.find((model) => model?.id === config.model)
+                    : undefined;
+                window = positive(entry?.loaded_context_length);
+            }
+            if (props.missing && refusals.get(endpoint)?.gone) {
                 unserved.add(endpoint);
                 return 0;
             }
@@ -325,10 +369,18 @@ export async function servedWindow(config) {
  */
 export async function listModels(config) {
     const { data } = await getClient(config).models.list();
+    const trained = new Set();
     const models = data
-        .map((model) => ({ id: model.id, contextLength: contextLengthOf(model) }))
+        .map((model) => {
+        const window = contextLengthOf(model);
+        if (window.trained)
+            trained.add(model.id);
+        return { id: model.id, contextLength: window.contextLength };
+    })
         .sort((a, b) => a.id.localeCompare(b.id));
-    listings.set(endpointKey(config), models);
+    const key = endpointKey(config);
+    listings.set(key, models);
+    trainedOnly.set(key, trained);
     return models;
 }
 /**
@@ -339,11 +391,18 @@ export async function listModels(config) {
  * happily load a 256k model at `-c 16384` and go on listing it as 256k — and a run refused on
  * the honest-looking number is a run that fails at the endpoint instead.
  *
- * Otherwise the window the server is actually serving the model in, where it has an API that
- * says (`servedWindow`), and failing that the endpoint's listing — asked once, and again whenever
- * it does not name this model, since a model can arrive after the first listing was taken. A
- * server that will not list models still has to be able to run a turn: a failure here is an
- * unknown window, not a failed run.
+ * Otherwise the endpoint's listing — asked once, and again whenever it does not name this model,
+ * since a model can arrive after the first listing was taken. A window stated at the top of the
+ * model's entry is taken as the one being served, and nothing else is asked. Where the listing
+ * did not settle it — no window, only llama.cpp's trained one, no entry, no listing — the
+ * server's own API is asked (`servedWindow`), and what it says wins; the trained window is what
+ * is left when it says nothing. A server that will not list models still has to be able to run a
+ * turn: a failure here is an unknown window, not a failed run.
+ *
+ * The listing goes first because it is the one route every server here has, and the native ones
+ * are two more requests that a server stating its window in the listing never needed. Asked
+ * first, they were also asked again every `listingMissMs` for the life of the process on a server
+ * whose `/props` answers without a window, by any host that sizes the window per turn.
  *
  * @param config The endpoint, plus the model whose window is wanted.
  * @param declared The operator's own number. Above zero it wins and the endpoint is not asked.
@@ -351,9 +410,6 @@ export async function listModels(config) {
 export async function contextLimitFor(config, declared = 0) {
     if (declared > 0)
         return declared;
-    const window = await servedWindow(config);
-    if (window > 0)
-        return window;
     const key = endpointKey(config);
     const listed = () => listings.get(key)?.find((model) => model.id === config.model);
     // The listing is asked for again when it does not name this model, rather than only when
@@ -369,26 +425,30 @@ export async function contextLimitFor(config, declared = 0) {
         // Asked again, but not on every call. A model that is never coming answers the same zero
         // however often the endpoint is asked, and a caller sizing a window per turn pays a round
         // trip for each of them; `listingMissMs` is how long that answer is allowed to stand.
-        if (asked !== undefined && Date.now() - asked < poolLimits.listingMissMs)
-            return 0;
-        // A failure is not remembered: an endpoint that was down when the last run started is not
-        // an endpoint with no models, and a window nobody could ask about is not a failed run.
-        try {
-            await listModels(config);
-        }
-        catch {
-            return 0;
+        if (asked === undefined || Date.now() - asked >= poolLimits.listingMissMs) {
+            // A failure is not remembered: an endpoint that was down when the last run started is not
+            // an endpoint with no models, and a window nobody could ask about is not a failed run.
+            const answered = await listModels(config).then(() => true, () => false);
+            // Recorded where the endpoint was actually asked, and only there. Stamping it on the calls
+            // that skipped the request would push the deadline out ahead of any caller polling faster
+            // than the interval, which is a memory that never expires rather than one that expires in
+            // half a minute.
+            if (answered) {
+                if (listed())
+                    misses.delete(missKey);
+                else
+                    misses.set(missKey, Date.now());
+            }
         }
-        // Recorded where the endpoint was actually asked, and only there. Stamping it on the calls
-        // that skipped the request would push the deadline out ahead of any caller polling faster
-        // than the interval, which is a memory that never expires rather than one that expires in
-        // half a minute.
-        if (listed())
-            misses.delete(missKey);
-        else
-            misses.set(missKey, Date.now());
     }
-    return listed()?.contextLength ?? 0;
+    const entry = listed();
+    const fromListing = entry?.contextLength ?? 0;
+    // A top-level key settles it. A trained window does not: llama.cpp lists the one the model was
+    // built with, and taking it over what `/props` says lets an overflow through the guard.
+    if (fromListing > 0 && !trainedOnly.get(key)?.has(config.model))
+        return fromListing;
+    const window = await servedWindow(config);
+    return window > 0 ? window : fromListing;
 }
 /**
  * Forgets every cached client and listing, and puts `configureClients` back to the defaults. For
@@ -399,6 +459,8 @@ export function resetClients() {
     clients.clear();
     listings.clear();
     misses.clear();
+    trainedOnly.clear();
     served.clear();
     unserved.clear();
+    refusals.clear();
 }

package/dist/errors.d.ts

@@ -1,3 +1,7 @@
+import type OpenAI from "openai";
+import type { ToolCallOutcome } from "./agent-loop.ts";
+import { ContextOverflow } from "./retry.ts";
+import type { TurnUsage } from "./stream.ts";
 /**
  * What went wrong, as a string.
  *
@@ -8,3 +12,88 @@
  * @param error Whatever a `catch` bound. Anything that is not an `Error` is stringified.
  */
 export declare const errorMessage: (error: unknown) => string;
+/**
+ * A run as it stood when `runAgentLoop` gave up on it.
+ *
+ * The loop works on its own copy of the transcript and hands it back only on success, so without
+ * this a spent budget, a stop or a refused request took every step before it along — steps whose
+ * assistant turns and tool results were real, and that a host storing its transcript has to keep.
+ */
+export interface AgentLoopFailure {
+    /**
+     * The transcript, as the result's `messages` would have been: every assistant turn and tool
+     * result the run added, and no system prompt. Well-formed — a call the run was stopped during
+     * has a result saying so. The reply of a request that was in flight is not in it.
+     */
+    messages: OpenAI.ChatCompletionMessageParam[];
+    /** Summed over the turns that came back. A request that failed or was stopped adds nothing. */
+    usage: TurnUsage;
+    /** Every call that was made, in order. One the run was stopped during is `ok: false`. */
+    toolCalls: ToolCallOutcome[];
+    /** What was loaded when the run failed, for `carryOver`. Empty in eager and proxied modes. */
+    loaded: string[];
+    /** The tools the model had called, `load_tools` excluded. */
+    used: string[];
+}
+/**
+ * What `runAgentLoop` throws, carrying the run as it stood and what was caught as `cause`.
+ *
+ * Its own class rather than fields hung on the error that was caught, because a stop throws the
+ * signal's reason and that is one object for every run under the signal. The message is the
+ * cause's own, so a host that logs `errorMessage(error)` logs what it always did. A stop is told
+ * the way it was before — by the signal the host aborted, not by the class of what came out.
+ */
+export declare class AgentLoopError extends Error implements AgentLoopFailure {
+    readonly name: string;
+    readonly messages: OpenAI.ChatCompletionMessageParam[];
+    readonly usage: TurnUsage;
+    readonly toolCalls: ToolCallOutcome[];
+    readonly loaded: string[];
+    readonly used: string[];
+    /**
+     * @param message What went wrong — the cause's own message, where there is a cause.
+     * @param run The run as it stood. Held, not copied.
+     * @param options The standard `cause`: what the loop caught, absent where the loop itself gave up.
+     */
+    constructor(message: string, run: AgentLoopFailure, options?: ErrorOptions);
+}
+/**
+ * `maxToolIterations` was spent with the model still asking for tools.
+ *
+ * Named so a host can tell it from a failure without matching the message, which is the same
+ * sentence it has always been. There is no `cause`: nothing was caught, the loop stopped itself.
+ */
+export declare class ToolIterationLimit extends AgentLoopError {
+    readonly name: string;
+}
+/**
+ * A `ContextOverflow` out of `runAgentLoop`, carrying the run as it stood.
+ *
+ * A subclass of `ContextOverflow` rather than an `AgentLoopError` around one, because
+ * `instanceof ContextOverflow` is how a caller knows to compact and send again, and wrapping
+ * would have made that test quietly false. The overflow the loop caught is the `cause`, and the
+ * endpoint's own error, where there was one, is that one's `cause` in turn.
+ */
+export declare class AgentLoopOverflow extends ContextOverflow implements AgentLoopFailure {
+    readonly messages: OpenAI.ChatCompletionMessageParam[];
+    readonly usage: TurnUsage;
+    readonly toolCalls: ToolCallOutcome[];
+    readonly loaded: string[];
+    readonly used: string[];
+    /**
+     * @param message The overflow's own message.
+     * @param run The run as it stood. Held, not copied.
+     * @param options The standard `cause`: the `ContextOverflow` the loop caught.
+     */
+    constructor(message: string, run: AgentLoopFailure, options?: ErrorOptions);
+}
+/**
+ * The run a failed `runAgentLoop` left behind, read off whatever it threw, or nothing.
+ *
+ * Two classes carry it — an overflow has to stay a `ContextOverflow` and so cannot also be an
+ * `AgentLoopError` — and a `catch` that only wants the transcript should not have to name both.
+ *
+ * @param error Whatever a `catch` around `runAgentLoop` bound. Anything the loop did not throw
+ * answers `undefined`.
+ */
+export declare const failedRun: (error: unknown) => AgentLoopFailure | undefined;

package/dist/errors.js

@@ -1,3 +1,4 @@
+import { ContextOverflow } from "./retry.js";
 /**
  * What went wrong, as a string.
  *
@@ -8,3 +9,79 @@
  * @param error Whatever a `catch` bound. Anything that is not an `Error` is stringified.
  */
 export const errorMessage = (error) => error instanceof Error ? error.message : String(error);
+/**
+ * What `runAgentLoop` throws, carrying the run as it stood and what was caught as `cause`.
+ *
+ * Its own class rather than fields hung on the error that was caught, because a stop throws the
+ * signal's reason and that is one object for every run under the signal. The message is the
+ * cause's own, so a host that logs `errorMessage(error)` logs what it always did. A stop is told
+ * the way it was before — by the signal the host aborted, not by the class of what came out.
+ */
+export class AgentLoopError extends Error {
+    name = "AgentLoopError";
+    messages;
+    usage;
+    toolCalls;
+    loaded;
+    used;
+    /**
+     * @param message What went wrong — the cause's own message, where there is a cause.
+     * @param run The run as it stood. Held, not copied.
+     * @param options The standard `cause`: what the loop caught, absent where the loop itself gave up.
+     */
+    constructor(message, run, options) {
+        super(message, options);
+        this.messages = run.messages;
+        this.usage = run.usage;
+        this.toolCalls = run.toolCalls;
+        this.loaded = run.loaded;
+        this.used = run.used;
+    }
+}
+/**
+ * `maxToolIterations` was spent with the model still asking for tools.
+ *
+ * Named so a host can tell it from a failure without matching the message, which is the same
+ * sentence it has always been. There is no `cause`: nothing was caught, the loop stopped itself.
+ */
+export class ToolIterationLimit extends AgentLoopError {
+    name = "ToolIterationLimit";
+}
+/**
+ * A `ContextOverflow` out of `runAgentLoop`, carrying the run as it stood.
+ *
+ * A subclass of `ContextOverflow` rather than an `AgentLoopError` around one, because
+ * `instanceof ContextOverflow` is how a caller knows to compact and send again, and wrapping
+ * would have made that test quietly false. The overflow the loop caught is the `cause`, and the
+ * endpoint's own error, where there was one, is that one's `cause` in turn.
+ */
+export class AgentLoopOverflow extends ContextOverflow {
+    messages;
+    usage;
+    toolCalls;
+    loaded;
+    used;
+    /**
+     * @param message The overflow's own message.
+     * @param run The run as it stood. Held, not copied.
+     * @param options The standard `cause`: the `ContextOverflow` the loop caught.
+     */
+    constructor(message, run, options) {
+        super(message, options);
+        this.messages = run.messages;
+        this.usage = run.usage;
+        this.toolCalls = run.toolCalls;
+        this.loaded = run.loaded;
+        this.used = run.used;
+    }
+}
+/**
+ * The run a failed `runAgentLoop` left behind, read off whatever it threw, or nothing.
+ *
+ * Two classes carry it — an overflow has to stay a `ContextOverflow` and so cannot also be an
+ * `AgentLoopError` — and a `catch` that only wants the transcript should not have to name both.
+ *
+ * @param error Whatever a `catch` around `runAgentLoop` bound. Anything the loop did not throw
+ * answers `undefined`.
+ */
+export const failedRun = (error) => error instanceof AgentLoopError || error instanceof AgentLoopOverflow ? error : undefined;

package/dist/index.d.ts

@@ -17,7 +17,7 @@ export { type ClientPoolOptions, configureClients, contextLimitFor, endpointId,
 export { applyCompaction, COMPACT_AT, type CompactionOptions, type CompactionPlan, type CompactionRecord, type CompactionRunOptions, compactTranscript, KEEP_RATIO, type PruneOptions, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.ts";
 export type { AgentConfig, Endpoint, ModelParams, RetryPolicy, ToolPolicy, } from "./config.ts";
 export { type ContinueTurnOptions, continueTurn, isContinuable, } from "./continuation.ts";
-export { errorMessage } from "./errors.ts";
+export { AgentLoopError, type AgentLoopFailure, AgentLoopOverflow, errorMessage, failedRun, ToolIterationLimit, } from "./errors.ts";
 export { configureEvents, type EventBusOptions, emit, endRun, fold, history, type RunEvent, type RunEventInput, type RunEventKind, type RunMetrics, type RunMetricsOptions, type RunUsage, resetEvents, runMetrics, type TurnReport, watch, } from "./events.ts";
 export { assembleContext, configureHooks, consult, type Gathered, gather, HOOK_CONTEXT_TOKENS, HOOK_EVENTS, HOOK_PREFACE, type HookContext, type HookEvent, type HookMessage, type HookNote, type HookOptions, type HookOutcome, type HookRunner, INJECT_EVENTS, notify, resetHooks, turnIndex, turnMessages, UNTRUSTED_PREFACE, untrusted, withContext, } from "./hooks.ts";
 export { estimateFrom, type LedgerEntry, type LedgerEstimateOptions, type LedgerRequest, rebaseLedger, recordRequest, type TokenLedger, tokensBetween, } from "./ledger.ts";

package/dist/index.js

@@ -15,7 +15,7 @@ export { capabilitiesFor, EFFORT_LADDER, effortFor, expireCapabilities, modelCap
 export { configureClients, contextLimitFor, endpointId, endpointKey, FIRST_TOKEN_FACTOR, firstTokenMs, getClient, listModels, NO_KEY, resetClients, servedWindow, timeoutMs, } from "./client.js";
 export { applyCompaction, COMPACT_AT, compactTranscript, KEEP_RATIO, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.js";
 export { continueTurn, isContinuable, } from "./continuation.js";
-export { errorMessage } from "./errors.js";
+export { AgentLoopError, AgentLoopOverflow, errorMessage, failedRun, ToolIterationLimit, } from "./errors.js";
 export { configureEvents, emit, endRun, fold, history, resetEvents, runMetrics, watch, } from "./events.js";
 export { assembleContext, configureHooks, consult, gather, HOOK_CONTEXT_TOKENS, HOOK_EVENTS, HOOK_PREFACE, INJECT_EVENTS, notify, resetHooks, turnIndex, turnMessages, UNTRUSTED_PREFACE, untrusted, withContext, } from "./hooks.js";
 export { estimateFrom, rebaseLedger, recordRequest, tokensBetween, } from "./ledger.js";

package/llms.txt

@@ -70,7 +70,7 @@ The contract between whatever holds the tools and the loop that offers them to a
 - `ModelInfo` (type) — A model an endpoint offers, and what it says the model will read.
 - `NO_KEY` — The SDK insists on a non-empty key even where the server will not look at it.
 - `resetClients` — Forgets every cached client and listing, and puts `configureClients` back to the defaults.
-- `servedWindow` — The window a local server is actually serving a model in, which its listing does not say.
+- `servedWindow` — The window a local server's own API says it is serving a model in, for the servers whose listing does not say.
 - `timeoutMs` — Zero, less, or absent means no limit, which the SDK spells as `undefined`.
 
 ### compaction
@@ -115,7 +115,12 @@ Picking up an answer the token ceiling cut off, instead of keeping half of it.
 
 ### errors
 
+- `AgentLoopError` — What `runAgentLoop` throws, carrying the run as it stood and what was caught as `cause`.
+- `AgentLoopFailure` (type) — A run as it stood when `runAgentLoop` gave up on it.
+- `AgentLoopOverflow` — A `ContextOverflow` out of `runAgentLoop`, carrying the run as it stood.
 - `errorMessage` — What went wrong, as a string.
+- `failedRun` — The run a failed `runAgentLoop` left behind, read off whatever it threw, or nothing.
+- `ToolIterationLimit` — `maxToolIterations` was spent with the model still asking for tools.
 
 ### events
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cubicecho/agent-core",
-  "version": "2.20.0",
+  "version": "2.21.1",
   "description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
   "keywords": [
     "openai",