@cubicecho/agent-core 2.11.0 → 2.13.0

package/README.md

@@ -27,9 +27,11 @@ only, Node >=22.
 | `thinking` | Tells a scratchpad fenced inside `content` from the answer: `FenceSplitter` for a stream, `stripThinking` for a whole reply, and the fence tables both read. |
 | `side-task` | One-shot calls that support a run without being one — small prompt, short answer, no tools, never worth failing the run over. `askJson` holds the answer to a schema where the server can. |
 | `hooks` | The host's side of lifecycle hooks: `gather` before a request and `notify` after, the shared context budget, `withContext` to put what they add on the turn's question, `untrusted` to fence text nobody vouched for, and `turnMessages` to hand them a transcript. Running a hook is a runner the caller passes. |
-| `events` | The in-memory bus a watcher reads while a run happens: `emit`, `watch`, `history`, `fold`. A watcher's backlog is capped and reports its own gaps. |
+| `events` | The in-memory bus a watcher reads while a run happens: `emit`, `watch`, `history`, `fold`, and `runMetrics` for what a run cost. A watcher's backlog is capped and reports its own gaps. |
 | `client` | A pooled `OpenAI` client per endpoint, plus the context window: the served one where a local server says, the listed one otherwise, and their caches. |
 | `retry` | What to do when a request is lost, refused or too big: `isTransient`, `isModelLoading`, `backoffMs`, `ContextOverflow`, `EndpointSilent`, `requestTokens`. |
+| `calibration` | How many characters a token is worth on one model, learned from the prompt counts its endpoint reports: `charsPerTokenFor`, `calibrate`. |
+| `continuation` | `continueTurn`: carries on an answer the token ceiling cut off, by prefilling it as a trailing assistant message. |
 | `config` | The structural interfaces every function here asks for. |
 | `run-turn` | `runTurn`: one turn with the retry loop around the negotiation around the stream. The whole loop, for a caller that wants it rather than its parts. Sizes the request against an opt-in `contextLimit`. |
 | `agent-loop` | `runAgentLoop`: the loop above a turn — `runTurn` per step, the tools between, `load_tools` and preselection handled, until the model stops asking. Plus the parts it is made of: `buildBody`, `preselect`, `preview`, and `resolveApiKey` for a caller deciding which key an endpoint gets. |
@@ -87,6 +89,20 @@ truncated prose or — the case that bites — a tool call whose `arguments` sto
 caller meets a parse failure with nothing to attribute it to. `finishReason` is `"length"` there,
 `""` where the endpoint never said.
 
+`usage` is `prompt`, `completion`, `total` and `cached`, always there and zero where the server
+sent nothing. Everything else on it is there only when something measured it, and absent rather
+than zero otherwise:
+
+| Field | From |
+| --- | --- |
+| `uncached` | the prompt less `cached`, only where a cache count was reported, so a cold cache and no report read differently |
+| `reasoningTokens` | `completion_tokens_details.reasoning_tokens`; never estimated from the thinking stream |
+| `promptMs`, `promptTokensPerSecond`, `predictedMs`, `tokensPerSecond`, `draftTotal`, `draftAccepted` | llama.cpp's `timings`, which also stands in for `cached` (`cache_n`) where the usage has no cache count |
+| `firstTokenMs` | `streamTurn`, from the request to the first chunk that carried something |
+| `wallMs`, `retries`, `timeouts` | `runTurn`: the whole turn with its backoff, the lost requests sent again, and how many of those went silent |
+| `continuations` | `continueTurn` |
+| `cacheExpected`, `cacheBroken`, `cacheBreakReason`, `toolsDeclared`, `toolSchemaTokens` | `runAgentLoop`, below |
+
 `reasoning` is the scratchpad `onThinking` was told, kept because two common families want it
 back. gpt-oss and DeepSeek in thinking mode read the analysis behind a tool call off the assistant
 message on the next request: store it as `reasoning_content` on that message while it ends in a
@@ -150,6 +166,10 @@ and the newer spelling is exactly the one an older model or a llama.cpp-shaped e
 `modelCapabilitiesFor` starts a model at `legacyTokenLimit: true`, and an absent model has to read
 the same way it does.
 
+One more model flag is never answered by `negotiate`: `assistantPrefill`, whether the model
+carries on a trailing assistant message rather than answering afresh after it. Only `continueTurn`
+sends one, so only it latches the flag, and it is persisted in a snapshot like the rest.
+
 `runTurn` takes the same option and hands `request` the same second argument. Keying on
 `(endpoint, model)` rather than the model name alone is the part worth keeping: `gpt-4o` at
 OpenAI and `gpt-4o` behind a proxy need not be the same weights, and one that refused a reasoning
@@ -251,6 +271,30 @@ nothing, and the server gives the reply whatever the prompt leaves. A consumer t
 `maxTokens` from the limit itself before calling in no longer needs to, and doing both reserves
 the ceiling twice.
 
+How many tokens the body is, is a guess: characters over a divisor. Four is right for English prose
+and wrong for what a tool-using run is made of — JSON schemas and tool results pack closer to two or
+three characters a token — so a request the guard let through was refused anyway. The divisor is
+learned instead. Every turn `runTurn` answers comes back with the endpoint's exact prompt count for a
+body whose characters were already counted, and `calibrate` keeps that ratio per endpoint and model;
+the next request to the same model is sized by `charsPerTokenFor`, which is `CHARS_PER_TOKEN` (4)
+until a turn has reported one. It is the highest of the model's last four readings, which is the
+lowest token count: a count that comes out high refuses a run that would have fit, and one that
+comes out low only costs the round trip the guard was saving. A reading from a request carrying an
+image, or outside one to eight characters a token, is not taken.
+
+`requestTokens` and `messageTokens` take `{ charsPerToken }`, and so does `planCompaction`, so a
+caller sizing its own work can use the same number:
+
+```ts
+const charsPerToken = charsPerTokenFor(capabilitiesFor(config.baseUrl, config.apiKey), config.model);
+const plan = planCompaction(messages, { limit, used, charsPerToken });
+```
+
+The ratio is a measurement rather than a refusal, so it is not a capability latch and is not in
+`exportCapabilities`: it moves every turn, and a restarted process learns it again from its first
+one. llama.cpp's `/tokenize` would count exactly, but only after `/apply-template` renders the
+prompt — two round trips before every guarded request, on one server — so it is not used.
+
 The body is sized once, not per attempt: a downgraded request is strictly smaller than the one
 before it and the transcript does not change between retries. A `ContextOverflow` from this is
 neither a capability `negotiate` can answer nor something `isTransient` accepts, so it leaves
@@ -321,8 +365,17 @@ and — naming only tools that exist — a reply that is only a JSON call or hol
 Found calls are run as `call_recovered_0` onward, the text is what is left, `onTurn` and the
 result see the turn that way, and a notice says so, since the real fix is the server's parser.
 
+Every request declares its tools in name order, whichever way the caller assembled the array. A
+chat template renders the tool block ahead of the system prompt, so the tool array is the first
+thing a prompt cache has to match, and an array built from a map, from database rows, or from the
+order servers happened to connect in is a different array on the next boot — the same tools, the
+same run, and the cache for the whole transcript thrown away. Ordering by name makes it a property
+of the set instead. `toolOrder: false` sends the caller's order, for a host that means it — a model
+reads the array top to bottom — and a comparator orders it another way. `orderTools` is the same
+thing for a caller with its own loop, and `buildBody` takes the order as its last argument.
+
 With `toolDiscovery: "ondemand"` and a catalogue, the request declares `load_tools` and what has
-been loaded, appended in the order it was loaded, and the catalogue rides on the system prompt
+been loaded, and the catalogue rides on the system prompt
 unmarked, the same text on every step. Marking loads there rewrote the head of the prompt and lost
 the prompt cache for the whole transcript on each one; a model that loads a tool twice is told in
 the `load_tools` result that it already has it. A model that calls
@@ -331,6 +384,55 @@ run. A preselection shapes the first step alone: those tools, no catalogue, no `
 a model with the menu still in front of it shops, reloading what it has or picking a sibling —
 and everything is back from the second step on.
 
+A turn cut off at `maxTokens` is said so as a notice, and with `maxContinuations` above zero it is
+continued first. `continueTurn` is the same thing for a caller with its own loop:
+
+```ts
+let turn = await runTurn(client, supports, build, options);
+turn = await continueTurn(client, supports, build, turn, { ...options, maxContinuations: 2 });
+```
+
+It sends the transcript again with the answer so far as a trailing assistant message, which
+llama.cpp renders as a prefill: the model carries on from the last token, and the cache holds all
+of the prompt and most of the reply. vLLM only does so given `continue_final_message: true` and
+`add_generation_prompt: false` on that request, which nothing here sends yet. The pieces come
+back as one turn — content and reasoning joined, token counts summed, `continuations` counting the
+extra requests, timings summed or weighted, a field only one piece reported dropped. Only an answer
+begun and cut off is continued. A turn cut off in its scratchpad is left alone, since llama.cpp
+refuses a prefill outright on a template with thinking on, and so is one ending in a tool call,
+whose truncated arguments `parseToolArguments` already reports.
+
+Whether it works is latched per model as `assistantPrefill`. A 400 or 422 for the request latches it
+off, and so does a continuation that starts the answer over word for word — how hosted OpenAI, which
+takes a trailing assistant message and ignores it, shows itself. Either way, and on any other
+failure short of a stop, the cut-off answer is kept with a notice. The cap is one continuation for
+`continueTurn` and none for the loop, since it spends a request, and on a server that does not
+continue, one to find out.
+
+Every turn ends in a `usage` event, whether or not the endpoint reported tokens. Its totals are the
+run's so far; its `turn` is that turn's own `TurnUsage` and `finishReason`. The loop adds what only
+it can know: `toolsDeclared` and `toolSchemaTokens` for the tool block it sent, and, from the second
+request on, `cacheExpected` (the previous prompt plus its reply) and — where a cache count was
+reported — `cacheBroken`, a hit short of 90% of the previous prompt. A broken cache is given a
+`cacheBreakReason` read off the request against the one before: `tools-changed`, `system-changed`,
+`history-rewritten` (a compaction or a prune in `beforeStep`), or `none-known` where the new request
+only appended, which points at the server — a slot evicted, a template that re-renders the tail.
+
+`runMetrics(events)` adds a run up from those events — tokens, cache hit ratio and breaks by
+reason, prefill, decode and tool time, the slowest turn, mean time to first token, draft
+acceptance, the largest prompt against a `contextLength`, turns cut off, tool errors by name, and
+an `outcome`. The loop returns it as `metrics`, with the `load_tools` counts only it can see
+(`toolsLoaded`, `redundantLoads`, `unknownToolNames`). It is a sibling of `fold` rather than part
+of it, since `fold`'s blocks are for display and a summary is not one:
+
+```ts
+const metrics = runMetrics(history(runId), { contextLength: config.contextLength });
+```
+
+Counts are always present; every other field is absent where no turn reported what it is made of.
+What it cannot say: whether a failed run was stopped, errored or ran out of tool iterations, and
+whether the host compacted — neither is in the events.
+
 `beforeStep` is handed the transcript before each request and may return a replacement, which is
 where compaction goes (below). Hooks are gathered once, onto the question, and never written into
 the transcript that comes back; `afterTurn` is told the reply without the run waiting on it.
@@ -379,7 +481,7 @@ once `used` (the last turn's prompt tokens, or the estimate) is past three quart
 The kept tail fills at most 35% of it and starts on a user message, since a transcript resuming
 mid-exchange is one servers refuse; leading system prompts are never folded, and an earlier
 summary is continued rather than summarised. `compactTranscript` writes the summary and tells
-`beforeCompact` hooks what is going while it does. A hook cannot veto it. Because the cut lands on
+`beforeCompact` hooks what is going while it does. Because the cut lands on
 a user message, one long tool run under a single question has nothing to fold — pruning is what
 keeps that one going.
 
@@ -396,6 +498,21 @@ beforeStep: async (messages, step) => {
 },
 ```
 
+A hook can ask for a compaction not to happen — its runner sets `veto` on the outcome — and by default
+nobody listens: the hooks run beside the summary, so a slow one costs the run nothing. Pass
+`honourVeto: true` with the hooks and they run first — the summary waits on them — and a veto from
+any hook that ran leaves the transcript as it was, with a note naming the hook. One that failed
+vetoes nothing. A compaction passed `forced: true`, because a request was already refused as too
+big, goes ahead regardless: a veto there only trades the summary for a `ContextOverflow`.
+`consult` is the same wait for a host that compacts its own way.
+
+```ts
+const compacted = await compactTranscript(messages, plan, summarise, {
+  hooks: { run, context, onNote, honourVeto: true },
+  forced: retryingAfterOverflow, // the last request came back as a ContextOverflow
+});
+```
+
 **Both rewrite the prefix.** A prompt cache matches from the first token, so a transcript whose
 early messages change is re-processed whole — on a local server that is the entire prefill, every
 time. Run them rarely and together, at the point `planCompaction` says the window is filling, so
@@ -405,6 +522,51 @@ way to save tokens.
 `pruneToolResults` keeps the transcript's indexes, so a plan made before pruning still applies to
 what it returns, as above.
 
+### A fold you store, instead of a transcript you rewrite
+
+`compactTranscript` hands back a new array, which is the whole answer for a host whose transcript
+*is* that array. A host that keeps its messages append-only — rows in a database, every one still
+shown in the chat — wants the other half: what the fold was, as something to store on the session.
+`runCompaction` is `compactTranscript` without the rewrite. It returns `{ summary, through, at }`,
+or `undefined` when a hook vetoed or the summary came back empty, and `compactTranscript` is built
+out of it, so there is one summariser and one cut rather than two that drift.
+
+```ts
+const from = session.fold?.through ?? 0;
+const plan = planCompaction(session.messages, {
+  limit,
+  used,
+  from, // where the last fold ended, rather than scanning for it
+  previous: session.fold?.summary,
+});
+if (!plan) return;
+const fold = await runCompaction(session.messages, plan, summarise, { hooks: { run, context } });
+if (fold) await save(session.id, fold); // the messages themselves are never touched
+```
+
+`planCompaction` takes `from` and `previous` because its defaults are a *recovery*: it skips the
+leading `system` messages, and reads an earlier summary back out of a `SUMMARY_LEAD` message among
+them. A host whose system prompt is a separate argument and whose summary is a column has neither
+in the array, and knows both exactly. Given them, nothing is scanned.
+
+`applyCompaction(messages, fold)` is the way back — the summary as a `system` message, then
+everything from `through` — and it writes the same `SUMMARY_LEAD` `planCompaction` looks for, so
+the next fold continues those notes rather than summarising them a second time. No fold yet hands
+back the messages themselves.
+
+```ts
+const request = [systemMessage, ...applyCompaction(session.messages, session.fold)];
+```
+
+**Two numberings.** A stored transcript keeps its indexes and a folded request does not, so
+anything naming a position has to say which one it means. `requestIndex(index, fold)` maps the
+stored index onto the request — for `withContext`'s index, or a range being shown to a hook.
+`turnMessages` takes an `offset`, the stored index of the array's first message, so a message keeps
+the uuid it had before the fold and a memory server deduping on it files that turn once rather than
+twice; `turnIndex` takes one too, for the turns a fold took out of the array it is counting.
+Planning over the stored transcript, as above, sidesteps both: the plan's indexes are the host's
+already, and so are the ones `runCompaction` hands the `beforeCompact` hooks.
+
 ## Watching a run
 
 `watch` replays what the run has already emitted, then yields what happens next until `done`.
@@ -579,7 +741,8 @@ with no timeout to give now leaves it out rather than inventing a `0`.
 ## What is kept for the life of the process
 
 Four caches outlive any one run: the `OpenAI` clients, the model listings, the latched
-capabilities, and `side-task`'s no-thinking hints. All four are module-level and keyed on the same
+capabilities, and `side-task`'s no-thinking hints. A fifth, the characters per token each model was
+measured at, is keyed on the endpoint's capabilities object, so it goes with them. All four are module-level and keyed on the same
 notion of an endpoint — its base URL and its API key — and the clients' key carries the request
 timeout as well, since that changes how a request is sent.
 
@@ -619,7 +782,8 @@ URL and the key, an absent key read as `NO_KEY` — rather than rebuilding that
 cannot drift. `endpointKey` holds the key in the clear; `endpointId`, its SHA-256 digest, is the one
 that is safe to write down.
 
-`resetAll` drops all four, and `reset.ts` names each seam separately for a test that wants one.
+`resetAll` drops all five, and `reset.ts` names each seam separately for a test that wants one —
+`resetCalibration` for the measured ratios.
 
 The latches can outlive the process as well, because otherwise every restart spends one refused
 request per endpoint and model learning the same facts again. `exportCapabilities` returns every

package/dist/agent-loop.d.ts

@@ -2,9 +2,10 @@ import type OpenAI from "openai";
 import { type Capabilities, type ModelCapabilities } from "./capabilities.ts";
 import type { CatalogServer } from "./catalog.ts";
 import type { Endpoint, ModelParams, RetryPolicy, ToolPolicy } from "./config.ts";
-import type { RunEventInput } from "./events.ts";
+import { type RunEventInput, type RunMetrics } from "./events.ts";
 import { type HookContext, type HookEvent, type HookNote, type HookRunner } from "./hooks.ts";
 import type { Turn, TurnUsage } from "./stream.ts";
+import { type ToolOrder } from "./tool-loading.ts";
 /**
  * The one place a streamed request's body is decided from a config and what the endpoint and
  * the model have refused.
@@ -22,10 +23,13 @@ import type { Turn, TurnUsage } from "./stream.ts";
  * @param refused What the model has refused, as `negotiate` hands it over. Absent is a model
  * that has refused nothing.
  * @param messages The request's messages, system prompt included, sent as they are.
- * @param tools The tool definitions. Sanitised here — a lookup for a definition seen before —
- * and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
+ * @param tools The tool definitions. Ordered by name, sanitised here — a lookup for a definition
+ * seen before — and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
+ * @param order How to order them before sending. `true`, the default, is by name, which keeps the
+ * cache when the caller's array is assembled differently from one request to the next. See
+ * `orderTools`.
  */
-export declare function buildBody(config: ModelParams, supports: Capabilities, refused: ModelCapabilities | undefined, messages: OpenAI.ChatCompletionMessageParam[], tools?: OpenAI.ChatCompletionTool[]): OpenAI.ChatCompletionCreateParamsStreaming;
+export declare function buildBody(config: ModelParams, supports: Capabilities, refused: ModelCapabilities | undefined, messages: OpenAI.ChatCompletionMessageParam[], tools?: OpenAI.ChatCompletionTool[], order?: ToolOrder): OpenAI.ChatCompletionCreateParamsStreaming;
 /**
  * A long tool argument or result cut to what a watcher needs, with the full length said.
  *
@@ -131,6 +135,12 @@ export interface AgentLoopOptions {
     tools?: OpenAI.ChatCompletionTool[];
     /** The same tools as a name-only catalogue. On-demand mode needs it, and is eager without it. */
     catalog?: CatalogServer[];
+    /**
+     * How the declared tools are ordered before each request. By name unless told otherwise, so a
+     * run whose tool array was assembled in a different order than last time still meets its cache.
+     * `false` sends them as given. See `orderTools`.
+     */
+    toolOrder?: ToolOrder;
     /**
      * What `preselect` picked. The first step is sent these and nothing else — no catalogue, no
      * `load_tools` — because a model with the menu still in front of it shops: it reloads what it
@@ -168,6 +178,12 @@ export interface AgentLoopOptions {
     recoverToolCalls?: boolean;
     /** Each turn as it comes back, before its tools run. Recovered calls are in it as calls. */
     onTurn?: (turn: Turn, step: number) => void;
+    /**
+     * How many times an answer cut off at `maxTokens` is continued, zero — the default — for never.
+     * Opt-in because it spends another request, and on a server that does not continue a trailing
+     * assistant message it spends one to find that out. See `continueTurn`.
+     */
+    maxContinuations?: number;
 }
 /** What a finished loop hands back. */
 export interface AgentLoopResult {
@@ -185,6 +201,11 @@ export interface AgentLoopResult {
     used: string[];
     /** The hooks' notes from before the first request. */
     notes: HookNote[];
+    /**
+     * The run summed and derived: what `runMetrics` makes of the events this loop emitted, plus the
+     * `load_tools` findings only the loop sees, `wallMs` from the call to the return, and `outcome`.
+     */
+    metrics: RunMetrics;
 }
 /**
  * Runs a question to its answer: one `runTurn` per step, the tools it asks for between them,
@@ -192,10 +213,13 @@ export interface AgentLoopResult {
  * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
  *
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
- * prompt unchanged from step to step, loaded tools are appended to the tool array in load order,
+ * prompt unchanged from step to step, a load adds to the tool array — which every request sends
+ * in the stable order `toolOrder` asks for — and
  * a catalogued tool called without being loaded is loaded and run rather than refused, and a
  * preselection shapes the first step. A turn cut off at `maxTokens` is said so as a notice,
- * because it otherwise reads exactly like a finished one.
+ * because it otherwise reads exactly like a finished one — or, given `maxContinuations`, is
+ * continued first. Every turn ends in a `usage` event carrying the turn's own report, with the
+ * cache compared against the request before it; see `TurnUsage` and `runMetrics`.
  *
  * @param options The config, transcript, tools and dispatcher, plus the optional hooks, events
  * and cancellation. See `AgentLoopOptions`.

package/dist/agent-loop.js

@@ -1,12 +1,16 @@
+import { charsPerTokenFor } from "./calibration.js";
 import { capabilitiesFor } from "./capabilities.js";
 import { firstTokenMs, getClient, NO_KEY, timeoutMs } from "./client.js";
+import { continueTurn } from "./continuation.js";
 import { errorMessage } from "./errors.js";
+import { runMetrics } from "./events.js";
 import { gather, notify, turnIndex, turnMessages, withContext, } from "./hooks.js";
+import { toolsChars } from "./retry.js";
 import { runTurn } from "./run-turn.js";
 import { relaxTools, sanitizeTools } from "./schema-compat.js";
 import { askJson, tryAsk } from "./side-task.js";
 import { parseToolArguments, recoverToolCalls } from "./tool-calls.js";
-import { catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_PER_LOAD, PRESELECT_SCHEMA, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
+import { catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
 /**
  * The loop above a turn: send, run the tools the model asked for, send again, until it stops
  * asking.
@@ -36,11 +40,17 @@ const RESERVED = new Set(["model", "messages", "stream", "tools"]);
  * @param refused What the model has refused, as `negotiate` hands it over. Absent is a model
  * that has refused nothing.
  * @param messages The request's messages, system prompt included, sent as they are.
- * @param tools The tool definitions. Sanitised here — a lookup for a definition seen before —
- * and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
+ * @param tools The tool definitions. Ordered by name, sanitised here — a lookup for a definition
+ * seen before — and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
+ * @param order How to order them before sending. `true`, the default, is by name, which keeps the
+ * cache when the caller's array is assembled differently from one request to the next. See
+ * `orderTools`.
  */
-export function buildBody(config, supports, refused, messages, tools = []) {
-    const declared = supports.strictSchemas ? sanitizeTools(tools) : relaxTools(sanitizeTools(tools));
+export function buildBody(config, supports, refused, messages, tools = [], order = true) {
+    const sorted = orderTools(tools, order);
+    const declared = supports.strictSchemas
+        ? sanitizeTools(sorted)
+        : relaxTools(sanitizeTools(sorted));
     const effort = config.reasoningEffort;
     const extra = Object.entries(config.extraBody ?? {}).filter(([field]) => !RESERVED.has(field) && !refused?.refusedFields.has(field));
     return {
@@ -119,32 +129,107 @@ export async function preselect(config, model, catalog, prompt, { signal, onNoti
     const reply = await tryAsk("preselect", () => askJson(config, model, preselectSystem(maxPerLoad), preselectInput(catalog, prompt), PRESELECT_SCHEMA, { name: "preselection", maxTokens, signal, onNotice }), { onNotice });
     return preselection(reply, catalog, maxPerLoad);
 }
-/** Every numeric field of one usage added into another. */
+/**
+ * The four token counts of one usage added into another. Only those: the rest are measurements a
+ * sum of would mean nothing, or would mean something only `runMetrics` knows how to weigh.
+ */
 const accumulate = (total, turn) => {
-    for (const key of Object.keys(turn))
-        total[key] += turn[key] ?? 0;
+    total.prompt += turn.prompt;
+    total.completion += turn.completion;
+    total.total += turn.total;
+    total.cached += turn.cached;
+};
+/** Whether two messages say the same thing, by identity first since most of a transcript is. */
+const sameMessage = (a, b) => a === b || JSON.stringify(a) === JSON.stringify(b);
+/** The system messages a request opens with, which a template renders ahead of the history. */
+const leadingSystem = (messages) => {
+    const end = messages.findIndex((message) => message.role !== "system");
+    return messages.slice(0, end === -1 ? messages.length : end);
 };
+/**
+ * Where a request stopped matching the one before it, earliest in the rendered prompt first — the
+ * tool block, then the system prompt, then the history — or `none-known` where it only appended.
+ */
+function breakReason(previous, messages, tools) {
+    if (previous.tools.length !== tools.length ||
+        previous.tools.some((name, at) => name !== tools[at]))
+        return "tools-changed";
+    const before = leadingSystem(previous.messages);
+    const now = leadingSystem(messages);
+    if (before.length !== now.length || before.some((message, at) => !sameMessage(message, now[at])))
+        return "system-changed";
+    if (previous.messages.length > messages.length ||
+        previous.messages.some((message, at) => !sameMessage(message, messages[at])))
+        return "history-rewritten";
+    return "none-known";
+}
+/**
+ * The share of the previous prompt a cache that kept its prefix reports as hit. Short of it by
+ * more than this is a break, not the few tokens a template re-renders at the join.
+ */
+const CACHE_KEPT = 0.9;
+/**
+ * What one turn's cache should have been and whether it was, against the request before it.
+ * Nothing before the second request, or where the previous prompt was not reported.
+ */
+function cacheDiagnosis(previous, messages, tools, usage) {
+    if (!previous || !(previous.prompt > 0))
+        return {};
+    const found = { cacheExpected: previous.prompt + previous.completion };
+    if (usage.uncached === undefined)
+        return found;
+    found.cacheBroken = usage.cached < previous.prompt * CACHE_KEPT;
+    if (found.cacheBroken)
+        found.cacheBreakReason = breakReason(previous, messages, tools);
+    return found;
+}
 /**
  * Runs a question to its answer: one `runTurn` per step, the tools it asks for between them,
  * until a turn asks for none. Throws when `maxToolIterations` is spent, when stopped, and on
  * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
  *
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
- * prompt unchanged from step to step, loaded tools are appended to the tool array in load order,
+ * prompt unchanged from step to step, a load adds to the tool array — which every request sends
+ * in the stable order `toolOrder` asks for — and
  * a catalogued tool called without being loaded is loaded and run rather than refused, and a
  * preselection shapes the first step. A turn cut off at `maxTokens` is said so as a notice,
- * because it otherwise reads exactly like a finished one.
+ * because it otherwise reads exactly like a finished one — or, given `maxContinuations`, is
+ * continued first. Every turn ends in a `usage` event carrying the turn's own report, with the
+ * cache compared against the request before it; see `TurnUsage` and `runMetrics`.
  *
  * @param options The config, transcript, tools and dispatcher, plus the optional hooks, events
  * and cancellation. See `AgentLoopOptions`.
  */
 export async function runAgentLoop(options) {
     const { config, system = "", tools = [], catalog = [], dispatch, hooks, signal } = options;
-    const { onEvent, onTurn, beforeStep, parallel = false, recoverToolCalls: recover = true, } = options;
+    const { onTurn, beforeStep, parallel = false, recoverToolCalls: recover = true, maxContinuations = 0, toolOrder = true, } = options;
+    const started = Date.now();
+    // What the loop emitted, less the token deltas, for `runMetrics` at the end. Stamped here rather
+    // than by the bus, which the loop does not know about.
+    const recorded = [];
+    const record = (input) => {
+        if (input.kind === "thinking" || input.kind === "output")
+            return;
+        recorded.push({
+            runId: "",
+            seq: recorded.length + 1,
+            at: Date.now(),
+            text: "",
+            name: "",
+            step: "",
+            ok: null,
+            usage: null,
+            ...input,
+        });
+    };
+    const onEvent = (input) => {
+        record(input);
+        options.onEvent?.(input);
+    };
     const client = getClient(config);
     const supports = capabilitiesFor(config.baseUrl, config.apiKey);
     const maxRetries = Math.max(0, Number(config.maxRetries) || 0);
-    const notice = (text) => onEvent?.({ kind: "notice", text });
+    const notice = (text) => onEvent({ kind: "notice", text });
     const onDemand = config.toolDiscovery === "ondemand" && catalog.length > 0;
     const loaded = new Set(onDemand ? (options.loaded ?? []) : []);
     const preselected = onDemand ? [...(options.preselected ?? [])] : [];
@@ -175,18 +260,23 @@ export async function runAgentLoop(options) {
     const usage = { prompt: 0, completion: 0, total: 0, cached: 0 };
     const toolCalls = [];
     const answered = new Map();
+    const loads = { toolsLoaded: 0, redundantLoads: 0, unknownToolNames: 0 };
+    let previous;
     for (let step = 0; step < config.maxToolIterations; step++) {
         // A stop aborts the request in flight, but a tool call already handed off runs to its own
         // end — so the signal is read between steps as well.
         signal?.throwIfAborted();
         messages = (await beforeStep?.(messages, step)) ?? messages;
-        onEvent?.({ kind: "turn", text: `turn ${step + 1}` });
+        onEvent({ kind: "turn", text: `turn ${step + 1}` });
         const routed = preselected.length > 0 && step === 0;
-        const declared = routed
+        // Ordered here rather than left to `buildBody`, so `names` below is what the request actually
+        // declared — a diagnosis reading an order the server never saw calls an untouched tool array
+        // `tools-changed`.
+        const declared = orderTools(routed
             ? byName(new Set(preselected))
             : onDemand
                 ? loadedTools([LOAD_TOOLS_DEFINITION], byName(loaded))
-                : tools;
+                : tools, toolOrder);
         // Unmarked, so the system prompt is the same text on every step and a load does not throw
         // away the cache for the whole transcript. What is loaded is said in `declared` and in the
         // `load_tools` result instead. The preselected first step is the one exception, by design.
@@ -195,7 +285,8 @@ export async function runAgentLoop(options) {
             ...(prompt ? [{ role: "system", content: prompt }] : []),
             ...withContext(messages, question ? messages.indexOf(question) : -1, gathered.context, hooks?.preface),
         ];
-        const turn = await runTurn(client, supports, (supported, refused) => buildBody(config, supported, refused, request, declared), {
+        const build = (supported, refused) => buildBody(config, supported, refused, request, declared, toolOrder);
+        const turnOptions = {
             model: config.model,
             droppable: Object.keys(config.extraBody ?? {}),
             maxRetries,
@@ -207,20 +298,38 @@ export async function runAgentLoop(options) {
             idleMs: timeoutMs(config),
             firstChunkMs: firstTokenMs(config) ?? 0,
             onNotice: notice,
-            onThinking: (text) => onEvent?.({ kind: "thinking", text }),
-            onOutput: (text) => onEvent?.({ kind: "output", text }),
+            onThinking: (text) => onEvent({ kind: "thinking", text }),
+            onOutput: (text) => onEvent({ kind: "output", text }),
+        };
+        const first = await runTurn(client, supports, build, turnOptions);
+        const names = declared.map((tool) => (tool.type === "function" ? tool.function.name : ""));
+        // Compared before any continuation is joined on: the cache a request meets is the one its own
+        // prompt found, and a continuation's prompt is this request's plus the reply so far.
+        Object.assign(first.usage, cacheDiagnosis(previous, request, names, first.usage), {
+            toolsDeclared: declared.length,
+            toolSchemaTokens: Math.ceil(toolsChars(declared) / charsPerTokenFor(supports, config.model)),
         });
+        const firstPrompt = first.usage.prompt;
+        const turn = maxContinuations > 0
+            ? await continueTurn(client, supports, build, first, { ...turnOptions, maxContinuations })
+            : first;
+        previous = {
+            messages: request,
+            tools: names,
+            prompt: firstPrompt,
+            completion: turn.usage.completion,
+        };
         accumulate(usage, turn.usage);
-        if (turn.usage.total > 0 || turn.usage.prompt > 0 || turn.usage.completion > 0) {
-            onEvent?.({
-                kind: "usage",
-                usage: {
-                    promptTokens: usage.prompt,
-                    completionTokens: usage.completion,
-                    totalTokens: usage.total,
-                },
-            });
-        }
+        onEvent({
+            kind: "usage",
+            usage: {
+                promptTokens: usage.prompt,
+                completionTokens: usage.completion,
+                totalTokens: usage.total,
+                cachedTokens: usage.cached,
+                turn: { ...turn.usage, finishReason: turn.finishReason },
+            },
+        });
         if (turn.finishReason === "length") {
             notice(`the model stopped at maxTokens (${config.maxTokens}); this turn is cut short`);
         }
@@ -285,6 +394,7 @@ export async function runAgentLoop(options) {
                         : {}),
                 }, hooks.onNote);
             }
+            const metrics = runMetrics(recorded, { contextLength: config.contextLength });
             return {
                 turn: shown,
                 messages,
@@ -293,11 +403,17 @@ export async function runAgentLoop(options) {
                 loaded: [...loaded],
                 used: [...used],
                 notes: gathered.notes,
+                metrics: {
+                    ...metrics,
+                    ...(onDemand ? loads : {}),
+                    wallMs: Date.now() - started,
+                    outcome: turn.finishReason === "length" ? "truncated" : "answered",
+                },
             };
         }
         const run = async ({ call, args, error: unreadable, normal }) => {
             const { name, arguments: raw } = call.function;
-            onEvent?.({ kind: "tool-call", name, text: preview(raw) });
+            onEvent({ kind: "tool-call", name, text: preview(raw) });
             let content;
             let ok = true;
             try {
@@ -306,6 +422,9 @@ export async function runAgentLoop(options) {
                 if (onDemand && name === LOAD_TOOLS) {
                     const resolved = expandNames(requestedNames(args), catalog);
                     content = loadResult(resolved, catalog, loaded);
+                    for (const hit of resolved.matched)
+                        loads[loaded.has(hit) ? "redundantLoads" : "toolsLoaded"]++;
+                    loads.unknownToolNames += resolved.unknown.length;
                     for (const hit of resolved.matched)
                         loaded.add(hit);
                     ok = resolved.matched.length > 0;
@@ -318,7 +437,7 @@ export async function runAgentLoop(options) {
                     used.add(name);
                     const request = { id: call.id, name, args, raw };
                     content = parallel
-                        ? await once(answered, `${name}${normal}`, () => dispatch(request, signal))
+                        ? await once(answered, `${name}\0${normal}`, () => dispatch(request, signal))
                         : await dispatch(request, signal);
                 }
             }
@@ -328,7 +447,7 @@ export async function runAgentLoop(options) {
                 content = errorMessage(error);
                 ok = false;
             }
-            onEvent?.({ kind: "tool-result", name, ok, text: preview(content) });
+            onEvent({ kind: "tool-result", name, ok, text: preview(content) });
             return { id: call.id, name, ok, content };
         };
         const outcomes = [];