@cubicecho/agent-core 2.19.0 → 2.21.0

package/README.md

@@ -23,7 +23,7 @@ only, Node >=22.
 | Module | What it does |
 | --- | --- |
 | `schema-compat` | Makes an MCP tool schema something a strict or grammar-constrained server will accept. `sanitizeTools`, `relaxTools`, `isGrammarError`. |
-| `tool-loading` | On-demand tool discovery: a name-only catalogue plus a `load_tools` meta-tool, so a run pays for the schemas it asks for instead of all of them. Plus `preselectByKeywords`, which picks from it without a model. |
+| `tool-loading` | On-demand tool discovery: a name-only catalogue plus a `load_tools` meta-tool, so a run pays for the schemas it asks for instead of all of them. Plus `preselectByKeywords`, which picks from it without a model, and the proxied form of the same thing — `PROXY_TOOLS`, `proxyLoadResult`, `proxiedCall` — for a server whose prompt cache a growing tool array throws away. |
 | `stream` | Reads one streamed turn back into a message: token callbacks, tool-call reassembly, fenced reasoning taken out of the answer, and the idle watchdog that turns a silent endpoint into `EndpointSilent`. |
 | `capabilities` | What an endpoint turned out not to support — and, under it, what one model on that endpoint did not — plus the loop that answers either when it says so. `capabilitiesFor`, `modelCapabilitiesFor`, `negotiate`. |
 | `thinking` | Tells a scratchpad fenced inside `content` from the answer: `FenceSplitter` for a stream, `stripThinking` for a whole reply, and the fence tables both read. |
@@ -44,7 +44,7 @@ only, Node >=22.
 | `spec` | `parseSpec`, `resolveAgentSpec` and `exportSpec`: an agent as a versioned JSON document, layered into the flat config the loop takes. Imports nothing but types, and is published separately at `@cubicecho/agent-core/spec`. |
 | `reset` | `resetAll`: drops every cache and latch in one call, so a teardown cannot forget one. |
 | `tokens` | `estimateTokens`: characters over four, deliberately low, for everything here that has to guess at a window. |
-| `errors` | `errorMessage`: a caught `unknown` turned into something a run row can hold. |
+| `errors` | `errorMessage`: a caught `unknown` turned into something a run row can hold. And what `runAgentLoop` throws, with the run as it stood: `AgentLoopError`, `ToolIterationLimit`, `AgentLoopOverflow`, and `failedRun` to read it off any of them. |
 | `catalog` | `CatalogServer`: the name-only shape `tool-loading` reads a connected server as. |
 
 What is **not** here is the work: orchestration, prompts, and whatever the run is about. That
@@ -385,10 +385,10 @@ import { runAgentLoop, emit } from "@cubicecho/agent-core";
 
 const { turn, messages, usage, loaded } = await runAgentLoop({
   config,                         // Endpoint & ModelParams & { maxToolIterations, toolDiscovery?, maxRetries?, loadingTimeoutSeconds?, contextLength? }
-  system,                         // sent as the first message; on-demand mode appends the catalogue
+  system,                         // sent as the first message; on-demand and proxied modes append the catalogue
   messages: history,              // ending in the question; not written to
   tools,                          // every tool the run may reach
-  catalog,                        // the same, name-only, for on-demand loading
+  catalog,                        // the same, name-only, for on-demand and proxied loading
   preselected,                    // from `preselect`, if a small model chose
   dispatch: ({ name, args }, signal) => pool.call(name, args, signal),
   hooks: { run, context: { session: { id } } },
@@ -483,6 +483,53 @@ tool array stays a strict prefix of the new one. With name order, the default, a
 is sorted in and moves every definition after it — the first step still matches the second, but
 the last turn's cache holds only as far as the first new name.
 
+`toolDiscovery: "proxy"` is the same catalogue behind a tool array that never changes. A chat
+template renders the tool array in the system turn, so on a server that caches by prefix —
+llama.cpp, and anything built on it — a `load_tools` that appends a definition changes the head of
+the prompt and the whole transcript is prefilled again, on the very step that was meant to save
+tokens. Proxied, every step declares `PROXY_TOOLS` and nothing else: `load_tools`, and `call_tool`,
+which takes `{ name, arguments }`. A load answers with each new tool's name, description and
+parameters as a line of JSON, since no request will ever declare them, and with a pointer back for
+one whose definition is already in the conversation. A `call_tool` is refused for a name outside
+the catalogue, takes `arguments` as an object or as JSON in a string, and reaches `dispatch` as the
+tool it names — the inner name and arguments, under the call's own id — so a dispatcher, a watcher
+and `used` see the same thing in every mode. The transcript keeps the `call_tool` the model wrote.
+
+```ts
+const { messages, used } = await runAgentLoop({
+  config: { ...config, toolDiscovery: "proxy" },
+  messages: history,
+  tools,                          // never declared; a load answers with them
+  catalog,
+  preselected,                    // written into the transcript, not into the tool array
+  dispatch: ({ name, args }, signal) => pool.call(name, args, signal),
+});
+```
+
+A preselection has no tool array to go into there, and no step of its own. It is written into the
+transcript after the question as a `load_tools` call the model did not make, and the result that
+call would have had:
+
+```ts
+{ role: "assistant", content: null, tool_calls: [{ id: "preselect-3", type: "function",
+    function: { name: "load_tools", arguments: '{"names":["files__read_file"]}' } }] },
+{ role: "tool", tool_call_id: "preselect-3", content: "Loaded 1 tool(s). Run them with `call_tool`.\n\n{...}" },
+```
+
+The pair is in the `messages` the loop hands back, to be stored like any other, and is reported as
+a `tool-call` and a `tool-result` event, told to `onToolCall` and `onToolResult`, and counted in
+`toolCalls` and in the `load_tools` metrics. The
+id is numbered by where the call lands in the transcript, so two questions do not share one. A name
+with no definition in `tools`, or outside the catalogue, is left out, and none left is no exchange.
+
+What is loaded there is what the history holds, so `loaded` comes back empty and the `loaded`
+option is not read: there is nothing to carry, and a definition a compaction folded away has to be
+loadable again. For the same reason a `beforeStep` that returns a different array makes the loop
+forget what it loaded, which costs a definition sent twice at worst. `call_tool` is honoured in
+`ondemand` mode as well — a session switched out of proxied mode still has it in its history, and
+a model copies what it has seen work — unless the host has a tool of that name, which is then the
+host's. Eager mode does not know it.
+
 That preselection costs a round trip to a model, which on a local box is a few seconds before the
 run has started, spent on a model doing term matching. `preselect(..., { keywords: true })` does
 the matching directly and spends the model only on what the words cannot settle:
@@ -626,6 +673,50 @@ with pre-turn work to overlap with the hooks — compaction, a preselection —
 applies `withContext` to the current question as well, calls `notify` after, and passes no `hooks`
 (see [Hooks](#hooks)).
 
+A host that stores its transcript — a row per message, so a crash or a stop mid-run still leaves
+readable history — gets each message as the loop appends it, and the run as it stood when the loop
+throws:
+
+```ts
+try {
+  await runAgentLoop({
+    ...options,
+    // Awaited: the row is written before the tools run, and before the next request is sent.
+    onMessage: (message, step, turn) =>
+      store.add(sessionId, turn?.reasoning ? { ...message, reasoning_content: turn.reasoning } : message),
+  });
+} catch (error) {
+  const run = failedRun(error); // messages, usage, toolCalls, loaded, used — or undefined
+  if (run) await store.keep(sessionId, { usage: run.usage, loaded: run.loaded });
+  if (signal.aborted) return; // a stop, told by the signal as it always was
+  if (error instanceof ContextOverflow) return compactAndRetry(run?.messages);
+  if (error instanceof ToolIterationLimit) return outOfSteps();
+  throw error; // an AgentLoopError: the message is the cause's, and the cause is on `cause`
+}
+```
+
+`onMessage` hears the assistant message as it will be replayed — recovered calls folded in, and
+arguments repaired, which the turn `onTurn` is handed does not show — together with that `Turn`,
+and then each tool result, in transcript order. `onTurn` is not awaited and never sees a tool
+result; this is the one to store from. What it returns is ignored, since `beforeStep` is the one way to rewrite
+the transcript, and what it throws ends the run.
+
+Everything the loop throws carries the run. A spent `maxToolIterations` is a `ToolIterationLimit`,
+with the message it always had. A request too big for the window is an `AgentLoopOverflow`, which
+is a `ContextOverflow`, so a `catch` written for one still catches it. Anything else — a stop, a
+refused request, a dropped stream — is an `AgentLoopError` whose `cause` is what was caught and
+whose message is the cause's own. It is a wrapper and not fields on the error that was caught
+because a stop throws the signal's reason, one object for every run under that signal. A stop is
+still recognised by the signal the host aborted.
+
+A failure while a step's tools are running leaves a transcript that can be sent again. The results
+that came back are kept, a call the run was stopped during is answered `Stopped before this call
+finished.`, and one the loop never reached `Not run: the run stopped first.` — a call with no
+result is a transcript an endpoint refuses. Those are announced through `onMessage` like any other
+result. The reply of a request that was in flight when the run stopped is not kept: the host has
+the deltas from its `output` and `thinking` events, and whether half an answer belongs in the
+history is its call.
+
 `resolveApiKey` is exported and not applied, because which key an endpoint gets is a rule a
 consumer states and a library guessing it could send one where it was not meant to go. The rule
 it encodes is the conservative one: an endpoint's own key wins; one that names a base URL of its
@@ -663,7 +754,9 @@ Two ways to make a transcript smaller, cheap first.
 `pruneToolResults(messages, { keepLast: 5, maxChars: 256 })` replaces every tool result but the
 latest five with a stub — `[result cleared, 10,412 chars]`. A 40k-character `read_file` is 10k
 tokens on every turn after it, and by then the model has usually taken what it wanted; the stub
-keeps the call answered and says how much was there.
+keeps the call answered and says how much was there. A proxied `load_tools` result that carries
+definitions is never cleared, however old: it is the only copy of the schema the model has, and
+`holdsDefinitions` is the test for a host with a prune of its own.
 
 `planCompaction(messages, { limit, used })` says where to fold the oldest stretch into a summary,
 once `used` (the last turn's prompt tokens, or the estimate) is past three quarters of the window.

package/dist/agent-loop.d.ts

@@ -98,7 +98,12 @@ export declare function preselect(config: Endpoint, model: string, catalog: Cata
      */
     keywords?: boolean | KeywordPreselectOptions;
 }): Promise<string[]>;
-/** One call the model made, as `dispatch` and `onToolCall` are handed it. */
+/**
+ * One call the model made, as `dispatch` and `onToolCall` are handed it.
+ *
+ * A `call_tool` arrives as the tool it names — that tool's name and arguments, under the
+ * `call_tool`'s id — so a dispatcher is the same in every discovery mode.
+ */
 export interface ToolCallRequest {
     id: string;
     name: string;
@@ -107,7 +112,10 @@ export interface ToolCallRequest {
      * read, which only `onToolCall` is ever handed.
      */
     args: Record<string, unknown>;
-    /** The arguments as the model wrote them, before any repair. */
+    /**
+     * The arguments as the model wrote them, before any repair. Through `call_tool`, the inner
+     * arguments as JSON text.
+     */
     raw: string;
 }
 /** What one tool call did, in the order the model asked. */
@@ -119,6 +127,7 @@ export interface ToolCallOutcome {
      * sets it; optional so an outcome built before it existed still compiles.
      */
     id?: string;
+    /** The tool that was called. For a `call_tool`, the tool it named. */
     name: string;
     /** False when the arguments did not parse, the tool threw, or `load_tools` loaded nothing. */
     ok: boolean;
@@ -192,20 +201,27 @@ export interface AgentLoopOptions {
      * The endpoint, what to ask the model for, and how long it may keep calling tools.
      * `toolDiscovery` absent is eager, `maxRetries` absent is none, and `contextLength` is handed
      * to `runTurn` as `contextLimit`, which sizes each request against the window before sending.
+     * `"proxy"` is on-demand loading behind a tool array that never changes; see `PROXY_TOOLS`.
      */
     config: Endpoint & ModelParams & Pick<ToolPolicy, "maxToolIterations"> & Partial<Pick<ToolPolicy, "toolDiscovery">> & Partial<RetryPolicy> & {
         contextLength?: number;
     };
-    /** The standing instruction, sent as the first message. On-demand mode appends the catalogue. */
+    /**
+     * The standing instruction, sent as the first message. On-demand and proxied modes append the
+     * catalogue, each in its own wording.
+     */
     system?: string;
     /** The transcript so far, ending in the question. Not written to; see the result's `messages`. */
     messages: OpenAI.ChatCompletionMessageParam[];
     /**
      * Every tool this run may reach. Eager mode sends all of them; on-demand mode sends the ones
-     * loaded so far, by name.
+     * loaded so far, by name; proxied mode declares none, and answers a load with them instead.
      */
     tools?: OpenAI.ChatCompletionTool[];
-    /** The same tools as a name-only catalogue. On-demand mode needs it, and is eager without it. */
+    /**
+     * The same tools as a name-only catalogue. On-demand and proxied modes need it, and are eager
+     * without it.
+     */
     catalog?: CatalogServer[];
     /**
      * How the declared tools are ordered before each request. By name unless told otherwise, so a
@@ -218,6 +234,13 @@ export interface AgentLoopOptions {
      * `load_tools` — because a model with the menu still in front of it shops: it reloads what it
      * has or picks a sibling. Everything comes back on the step after. `preselectRouting` trades
      * that for a first step shaped like the rest.
+     *
+     * Proxied, the tool array is fixed and there is no such step. The shortlist is written into the
+     * transcript instead, after the question, as a `load_tools` call the model did not make and the
+     * result that call would have had — an assistant message and a tool message, handed back in
+     * `messages` like any other, reported as a `tool-call` and a `tool-result` event and told to
+     * `onToolCall` and `onToolResult`. Names with no definition in `tools`, or outside the
+     * catalogue, are left out, and none left is no exchange.
      */
     preselected?: readonly string[];
     /**
@@ -233,11 +256,14 @@ export interface AgentLoopOptions {
      * Appended is before `toolOrder` has its say: sorted, a preselected tool lands at its name's
      * place and moves every definition after it, so only `toolOrder: false` keeps a carried array
      * a strict prefix. Either way the first step's system prompt is the second's, and so is its
-     * tool array until something else is loaded. Nothing to do without a preselection, or in eager
-     * mode.
+     * tool array until something else is loaded. Nothing to do without a preselection, in eager
+     * mode, or proxied, where no step is routed either way.
      */
     preselectRouting?: "exclusive" | "append";
-    /** Tools already loaded, carried from an earlier question. See `carryOver`. */
+    /**
+     * Tools already loaded, carried from an earlier question. See `carryOver`. Not read in proxied
+     * mode, where a tool is loaded only while its definition is in the history.
+     */
     loaded?: Iterable<string>;
     /** Runs one tool call and returns what the model reads. What it throws, the model reads too. */
     dispatch: (call: ToolCallRequest, signal?: AbortSignal) => Promise<string>;
@@ -328,6 +354,19 @@ export interface AgentLoopOptions {
     onRequest?: (request: AgentLoopRequest) => void;
     /** Each turn as it comes back, before its tools run. Recovered calls are in it as calls. */
     onTurn?: (turn: Turn, step: number) => void;
+    /**
+     * Each message as the loop appends it to the transcript, in transcript order, and awaited before
+     * the loop goes on — so a host that stores its transcript has written the assistant message
+     * before its tools run, and every tool result before the next request is sent.
+     *
+     * The assistant message is the one replayed on later requests — arguments repaired, recovered
+     * calls folded in — and comes with its `turn`, as `onTurn` was handed it, for the `reasoning` a
+     * host keeps beside the message. A tool result comes without one, and so do the two messages
+     * a proxied run writes for its `preselected` shortlist, told as step zero's before any request.
+     * What it returns is ignored: `beforeStep` is the one way to rewrite the transcript. What it
+     * throws ends the run, and it is not called again for that run.
+     */
+    onMessage?: (message: OpenAI.ChatCompletionMessageParam, step: number, turn?: Turn) => void | Promise<void>;
     /**
      * How many times an answer cut off at `maxTokens` is continued, zero — the default — for never.
      * Opt-in because it spends another request, and on a server that does not continue a trailing
@@ -343,11 +382,18 @@ export interface AgentLoopResult {
     messages: OpenAI.ChatCompletionMessageParam[];
     /** Summed over every turn of the run. */
     usage: TurnUsage;
-    /** Every call, `load_tools` included, in the order they were made, each under its call id. */
+    /**
+     * Every call, `load_tools` included, in the order they were made, each under its call id — a
+     * proxied preselection's among them, first.
+     */
     toolCalls: ToolCallOutcome[];
-    /** What is loaded at the end, for `carryOver`. Empty in eager mode. */
+    /**
+     * What is loaded at the end, for `carryOver`. Empty in eager mode, and in proxied mode: a
+     * definition loaded there is already in the history, and one a compaction folded away has to be
+     * loadable again rather than answered "already loaded".
+     */
     loaded: string[];
-    /** The tools the model actually called, `load_tools` excluded. */
+    /** The tools the model actually called, `load_tools` excluded and `call_tool` looked through. */
     used: string[];
     /** The hooks' notes from before the first request. */
     notes: HookNote[];
@@ -390,6 +436,15 @@ export interface AgentLoopResult {
  * until a turn asks for none. Throws when `maxToolIterations` is spent, when stopped, and on
  * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
  *
+ * What it throws carries the run as it stood — `messages`, `usage`, `toolCalls`, `loaded` and
+ * `used` — because the steps before a failure were real and a host that stores its transcript
+ * has nothing else to store them from. A spent budget is a `ToolIterationLimit`, an overflow is
+ * an `AgentLoopOverflow`, which is still a `ContextOverflow`, and anything else is an
+ * `AgentLoopError` with what was caught as its `cause` and its message as its own; `failedRun`
+ * reads the run off any of the three. A failure while a step's tools run leaves the transcript
+ * well-formed: the results that came back are kept, and each call without one is answered with a
+ * line saying it was stopped or never run, since a call with no result is refused on replay.
+ *
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
  * prompt unchanged from step to step, a load adds to the tool array — which every request sends
  * in the stable order `toolOrder` asks for — and
@@ -407,6 +462,13 @@ export interface AgentLoopResult {
  * continuation, and a step that reported none, or reported a cache count above its prompt, is not
  * recorded. The ledger comes back in the result and on every `usage` event; see `tokensBetween`.
  *
+ * Proxied discovery is the same catalogue behind a tool array that never moves: every step
+ * declares `PROXY_TOOLS` and nothing else, a load answers with the definitions themselves, and a
+ * `call_tool` is dispatched, counted and reported as the tool it names. A preselection has no
+ * tool array to go into, so it is written into the transcript as a `load_tools` exchange after
+ * the question. `call_tool` is honoured on demand too, unless the host has a tool of that name —
+ * a session switched out of proxied mode still has it in its history, and the model copies it.
+ *
  * @param options The config, transcript, tools and dispatcher, plus the optional hooks, events
  * and cancellation. See `AgentLoopOptions`.
  */