@cubicecho/agent-core 2.6.0 → 2.8.0

package/README.md

@@ -22,13 +22,14 @@ only, Node >=22.
 | --- | --- |
 | `schema-compat` | Makes an MCP tool schema something a strict or grammar-constrained server will accept. `sanitizeTools`, `relaxTools`, `isGrammarError`. |
 | `tool-loading` | On-demand tool discovery: a name-only catalogue plus a `load_tools` meta-tool, so a run pays for the schemas it asks for instead of all of them. |
-| `stream` | Reads one streamed turn back into a message: token callbacks, tool-call reassembly, and the idle watchdog that turns a silent endpoint into `EndpointSilent`. |
+| `stream` | Reads one streamed turn back into a message: token callbacks, tool-call reassembly, fenced reasoning taken out of the answer, and the idle watchdog that turns a silent endpoint into `EndpointSilent`. |
 | `capabilities` | What an endpoint turned out not to support — and, under it, what one model on that endpoint did not — plus the loop that answers either when it says so. `capabilitiesFor`, `modelCapabilitiesFor`, `negotiate`. |
+| `thinking` | Tells a scratchpad fenced inside `content` from the answer: `FenceSplitter` for a stream, `stripThinking` for a whole reply, and the fence tables both read. |
 | `side-task` | One-shot calls that support a run without being one — small prompt, short answer, no tools, never worth failing the run over. `askJson` holds the answer to a schema where the server can. |
 | `hooks` | The host's side of lifecycle hooks: `gather` before a request and `notify` after, the shared context budget, `withContext` to put what they add on the turn's question, `untrusted` to fence text nobody vouched for, and `turnMessages` to hand them a transcript. Running a hook is a runner the caller passes. |
 | `events` | The in-memory bus a watcher reads while a run happens: `emit`, `watch`, `history`, `fold`. A watcher's backlog is capped and reports its own gaps. |
-| `client` | A pooled `OpenAI` client per endpoint, plus the context-window listing and its cache. |
-| `retry` | What to do when a request is lost, refused or too big: `isTransient`, `backoffMs`, `ContextOverflow`, `EndpointSilent`, `requestTokens`. |
+| `client` | A pooled `OpenAI` client per endpoint, plus the context window: the served one where a local server says, the listed one otherwise, and their caches. |
+| `retry` | What to do when a request is lost, refused or too big: `isTransient`, `isModelLoading`, `backoffMs`, `ContextOverflow`, `EndpointSilent`, `requestTokens`. |
 | `config` | The structural interfaces every function here asks for. |
 | `run-turn` | `runTurn`: one turn with the retry loop around the negotiation around the stream. The whole loop, for a caller that wants it rather than its parts. Sizes the request against an opt-in `contextLimit`. |
 | `agent-loop` | `runAgentLoop`: the loop above a turn — `runTurn` per step, the tools between, `load_tools` and preselection handled, until the model stops asking. Plus the parts it is made of: `buildBody`, `preselect`, `preview`, and `resolveApiKey` for a caller deciding which key an endpoint gets. |
@@ -80,11 +81,28 @@ for: a router is free to send two keys to two different backends, and then what
 refused is not a fact about the other. Absent and empty read the same, so a local server with no
 key is one entry however its caller spells it.
 
-`Turn` is `content`, `toolCalls`, `usage` and `finishReason`. The last is worth reading: a turn
-cut off at the token ceiling comes back looking exactly like a finished one, with truncated prose
-or — the case that bites — a tool call whose `arguments` stop mid-JSON, so the caller meets a
-parse failure with nothing to attribute it to. `finishReason` is `"length"` there, `""` where the
-endpoint never said.
+`Turn` is `content`, `toolCalls`, `usage`, `finishReason` and `reasoning`. The fourth is worth
+reading: a turn cut off at the token ceiling comes back looking exactly like a finished one, with
+truncated prose or — the case that bites — a tool call whose `arguments` stop mid-JSON, so the
+caller meets a parse failure with nothing to attribute it to. `finishReason` is `"length"` there,
+`""` where the endpoint never said.
+
+`reasoning` is the scratchpad `onThinking` was told, kept because two common families want it
+back. gpt-oss and DeepSeek in thinking mode read the analysis behind a tool call off the assistant
+message on the next request: store it as `reasoning_content` on that message while it ends in a
+tool call, and drop it once the model has answered. Any other model is better off without it,
+since it is context paid for on every turn. `requestTokens` counts it either way.
+
+A server without a reasoning parser leaves the scratchpad in `content`, fenced, and then it is shown
+as output, stored and sent back. `streamTurn` routes text inside a fence to `onThinking` and
+`reasoning` instead, holding back the tail of a chunk that could be half a tag. `DEFAULT_FENCES` is
+`<think>`, gpt-oss harmony's analysis channel served raw, and Kimi's `◁think▷`, none of which a
+model writes as an answer; `ALL_FENCES` adds `<thinking>` and `<reasoning>`, which it can be
+quoting, and is what the side tasks use. Pass `fences: []` to read `content` as all answer. A reply
+cut off inside a fence has an empty `content`, not the deliberation promoted to one. A template that
+opens `<think>` in the prompt leaves only the closing tag, so everything before it is moved to
+`reasoning` when it arrives; `startInReasoning: true` says so up front, so `onOutput` is never told
+it at all.
 
 ## What the model refuses, rather than the server
 
@@ -145,11 +163,21 @@ is worth another attempt. The content-free `{"role":"assistant"}` most servers o
 does not set it: nothing has been shown to anybody yet, so an endpoint that primes the stream and
 then wedges is retried like one that never answered at all.
 
-`idleMs` is silence, not a deadline: the timer is rearmed on every chunk, so a model that is
-still talking is never cut off however long it takes, and one that has stopped answering raises
+`idleMs` is silence, not a deadline: the timer is rearmed on every chunk, so a model that is still
+talking is never cut off however long it takes, and one that has stopped answering raises
 `EndpointSilent` rather than hanging the run. `timeoutMs(config)` returns `undefined` for a
-`requestTimeoutSeconds` of zero or absent, which waits forever — what a local model answering
-slowly needs.
+`requestTimeoutSeconds` of zero or absent, which waits forever — what a local model answering slowly
+needs.
+
+The first chunk gets its own allowance, `firstChunkMs`, because the first wait is prefill: tens of
+seconds for a long prompt on a local GPU, minutes on a CPU, and longer again when the server is
+loading the model on demand. It holds until a chunk carries something, so an empty
+`{"role":"assistant"}` sent before the prompt is read does not start the idle clock.
+`firstTokenMs(config)` reads `firstTokenSeconds` off the endpoint, and five times
+`requestTimeoutSeconds` where that is absent. With a watchdog armed, the SDK's own timer is switched
+off for the stream; it runs until the headers arrive, which is the end of prefill, and used to
+abandon one at the idle number. `requestTimeoutSeconds` still bounds calls that do not stream, side
+tasks and model listings, the same way.
 
 ## Structured side tasks
 
@@ -191,6 +219,16 @@ const turn = await runTurn(client, supports, build, {
 });
 ```
 
+`contextLimitFor` answers the operator's number when there is one. Otherwise it asks for the window
+the server is actually serving the model in (`servedWindow`): llama.cpp's `/props`
+(`default_generation_settings.n_ctx`) and LM Studio's `/api/v0/models` (`loaded_context_length`).
+That differs from the trained window in the case the guard exists for, a 256k model started at `-c
+16384`. A server with neither route is latched and not asked again. Failing both, it reads the
+`/v1/models` listing: `max_model_len` from vLLM, `context_length` from OpenRouter, and llama.cpp's
+`meta.n_ctx_train`, which is only the trained window. Ollama reports no window on any route this
+reads, and truncates an over-long prompt rather than refusing it, so on Ollama pass `contextLength`
+or there is no guard at all.
+
 What is weighed is the prompt plus the reply ceiling the body carries, under whichever spelling
 was chosen, because that is what the endpoint weighs: a 30k prompt into a 32k window with
 `max_tokens: 4096` is refused there, so it is refused here. A body with no ceiling reserves
@@ -209,6 +247,14 @@ refuses instead — one round trip later — and `runTurn` reads that refusal ba
 original error as `cause`. A rate limit borrows those words and means the opposite ("Request too
 large for gpt-4o ... on tokens per min"); that is ruled out and waited through as the 429 it is.
 
+A server still loading the model is waited for on its own clock. llama.cpp answers 503 `Loading
+model` (type `unavailable_error`) until the weights are mapped, thirty to ninety seconds for a large
+model from a cold cache, and a router build says the same while it swaps models; `backoffMs` would
+give up inside fifteen. `isModelLoading` recognises it, and `runTurn` polls every `LOADING_POLL_MS`
+for up to `loadingTimeoutMs` (two minutes by default, zero to turn it off) without spending
+`maxRetries`, with one notice at the start. `runAgentLoop` reads it as `loadingTimeoutSeconds` off
+the config. A 503 that says nothing about loading stays on the ordinary backoff.
+
 ## The loop
 
 `runAgentLoop` is the part of an agent that three servers had each written, and that had drifted
@@ -221,7 +267,7 @@ did. What it does not know is what the run is for — the prompt, the tools, and
 import { runAgentLoop, emit } from "@cubicecho/agent-core";
 
 const { turn, messages, usage, loaded } = await runAgentLoop({
-  config,                         // Endpoint & ModelParams & { maxToolIterations, toolDiscovery?, maxRetries?, contextLength? }
+  config,                         // Endpoint & ModelParams & { maxToolIterations, toolDiscovery?, maxRetries?, loadingTimeoutSeconds?, contextLength? }
   system,                         // sent as the first message; on-demand mode appends the catalogue
   messages: history,              // ending in the question; not written to
   tools,                          // every tool the run may reach

package/dist/agent-loop.js

@@ -1,5 +1,5 @@
 import { capabilitiesFor } from "./capabilities.js";
-import { getClient, NO_KEY, timeoutMs } from "./client.js";
+import { firstTokenMs, getClient, NO_KEY, timeoutMs } from "./client.js";
 import { errorMessage } from "./errors.js";
 import { gather, notify, turnIndex, turnMessages, withContext, } from "./hooks.js";
 import { runTurn } from "./run-turn.js";
@@ -187,9 +187,13 @@ export async function runAgentLoop(options) {
             model: config.model,
             droppable: Object.keys(config.extraBody ?? {}),
             maxRetries,
+            ...(config.loadingTimeoutSeconds === undefined
+                ? {}
+                : { loadingTimeoutMs: Math.max(0, config.loadingTimeoutSeconds) * 1000 }),
             contextLimit: config.contextLength ?? 0,
             signal,
             idleMs: timeoutMs(config),
+            firstChunkMs: firstTokenMs(config) ?? 0,
             onNotice: notice,
             onThinking: (text) => onEvent?.({ kind: "thinking", text }),
             onOutput: (text) => onEvent?.({ kind: "output", text }),

package/dist/client.d.ts

@@ -12,6 +12,14 @@ export declare const NO_KEY = "agent-core";
  * @param config Read for `requestTimeoutSeconds` alone.
  */
 export declare const timeoutMs: (config: Pick<Endpoint, "requestTimeoutSeconds">) => number | undefined;
+/** How many idle windows the first chunk gets when `firstTokenSeconds` is not given. */
+export declare const FIRST_TOKEN_FACTOR = 5;
+/**
+ * The wait for a streamed turn's first chunk, in the SDK's spelling: `undefined` is no limit.
+ *
+ * @param config Read for `firstTokenSeconds`, and `requestTimeoutSeconds` where that is absent.
+ */
+export declare const firstTokenMs: (config: Pick<Endpoint, "requestTimeoutSeconds" | "firstTokenSeconds">) => number | undefined;
 /**
  * The client for an endpoint, built once and kept.
  *
@@ -67,6 +75,23 @@ export declare const endpointId: (config: {
     baseUrl: string;
     apiKey?: string;
 }) => string;
+/**
+ * The window a local server is actually serving a model in, which its listing does not say.
+ *
+ * llama.cpp reports it on `/props` as `default_generation_settings.n_ctx`, per slot, and LM
+ * Studio on `/api/v0/models` as `loaded_context_length` while the model is loaded. Both differ
+ * from the trained window in the case that matters, a model started in a smaller one than it was
+ * built for, and reading the trained one lets an overflow through the guard meant to catch it.
+ * Ollama reports nothing on either; an operator there has to declare the window.
+ *
+ * Zero where no answer was found. A server without either route, which is every hosted API, is
+ * latched and not asked again; one that could not be reached is not remembered at all.
+ *
+ * @param config The endpoint, plus the model whose window is wanted.
+ */
+export declare function servedWindow(config: Endpoint & {
+    model: string;
+}): Promise<number>;
 /**
  * Asks an endpoint what it serves, and remembers the answer.
  *
@@ -81,10 +106,11 @@ export declare function listModels(config: Endpoint): Promise<ModelInfo[]>;
  * happily load a 256k model at `-c 16384` and go on listing it as 256k — and a run refused on
  * the honest-looking number is a run that fails at the endpoint instead.
  *
- * Otherwise the endpoint's listing is asked — once, and again whenever it does not name this
- * model, since a model can arrive after the first listing was taken. A server that will not
- * list models still has to be able to run a turn: a failure here is an unknown window, not a
- * failed run.
+ * Otherwise the window the server is actually serving the model in, where it has an API that
+ * says (`servedWindow`), and failing that the endpoint's listing — asked once, and again whenever
+ * it does not name this model, since a model can arrive after the first listing was taken. A
+ * server that will not list models still has to be able to run a turn: a failure here is an
+ * unknown window, not a failed run.
  *
  * @param config The endpoint, plus the model whose window is wanted.
  * @param declared The operator's own number. Above zero it wins and the endpoint is not asked.

package/dist/client.js

@@ -47,6 +47,20 @@ const clients = new Map();
  * constraint.
  */
 const MAX_CLIENTS = 32;
+/** How many idle windows the first chunk gets when `firstTokenSeconds` is not given. */
+export const FIRST_TOKEN_FACTOR = 5;
+/**
+ * The wait for a streamed turn's first chunk, in the SDK's spelling: `undefined` is no limit.
+ *
+ * @param config Read for `firstTokenSeconds`, and `requestTimeoutSeconds` where that is absent.
+ */
+export const firstTokenMs = (config) => {
+    if (config.firstTokenSeconds === undefined) {
+        const idle = timeoutMs(config);
+        return idle === undefined ? undefined : idle * FIRST_TOKEN_FACTOR;
+    }
+    return config.firstTokenSeconds > 0 ? config.firstTokenSeconds * 1000 : undefined;
+};
 /**
  * The client for an endpoint, built once and kept.
  *
@@ -89,12 +103,13 @@ export function getClient(config) {
     return client;
 }
 /**
- * The context window, spelled every way a server spells it.
+ * The context window, spelled every way a server spells it at the top of a listing entry.
  *
  * None of these is in the OpenAI listing schema, so every server that says anything says it as
- * an extra key of its own: `context_length` is llama.cpp and LM Studio, `max_model_len` vLLM,
- * `n_ctx` the raw llama bindings. Whichever turns up first is taken — a server reporting two
- * of them is reporting the same number twice.
+ * an extra key of its own: `max_model_len` is vLLM, `context_length` OpenRouter, `n_ctx` the raw
+ * llama bindings. Whichever turns up first is taken — a server reporting two of them is
+ * reporting the same number twice. llama.cpp puts its number under `meta` instead, and LM Studio
+ * and Ollama put none on this route at all; see `servedWindow`.
  */
 const CONTEXT_KEYS = [
     "context_length",
@@ -103,14 +118,19 @@ const CONTEXT_KEYS = [
     "context_window",
     "n_ctx",
 ];
+const positive = (value) => (typeof value === "number" && value > 0 ? value : 0);
 function contextLengthOf(model) {
     const record = model;
     for (const key of CONTEXT_KEYS) {
-        const value = record[key];
-        if (typeof value === "number" && value > 0)
+        const value = positive(record[key]);
+        if (value)
             return value;
     }
-    return 0;
+    // llama.cpp's, and the window the model was trained with rather than the one it is served in:
+    // a 256k model started at `-c 16384` lists 262144 here. Better than nothing, and why the
+    // served window is asked for first. `meta` is `null` while the model loads.
+    const meta = record.meta;
+    return positive(meta?.n_ctx_train);
 }
 /**
  * The last listing from each endpoint, so a run can size its window without a round trip.
@@ -172,6 +192,88 @@ export const endpointKey = (config) => JSON.stringify([config.baseUrl, config.ap
  * @param config Read for `baseUrl` and `apiKey` alone, as `endpointKey` reads it.
  */
 export const endpointId = (config) => createHash("sha256").update(endpointKey(config)).digest("hex");
+/**
+ * Served windows found by asking a server's own API, keyed on endpoint and model together.
+ *
+ * A model's entry stays until `resetClients`, like a listing; a server that answered without one
+ * is asked again after `LISTING_MISS_MS`, like a listing that did not name the model.
+ */
+const served = new Map();
+/** Endpoints that answered both probes with a refusal, and are not asked again. */
+const unserved = new Set();
+/** How long a probe may take before the window is taken from the listing instead. */
+const PROBE_TIMEOUT_MS = 10_000;
+/** The server root behind an OpenAI-compatible base URL, which is where the native APIs live. */
+const rootOf = (baseUrl) => baseUrl.replace(/\/+$/, "").replace(/\/v1$/, "");
+/** A route this server does not have, rather than one that failed to answer. */
+const NOT_THERE = new Set([404, 405, 501]);
+/**
+ * Asks one native endpoint. `missing` is a server without the route; `body` is absent for that
+ * and for any other refusal, and a server that could not be reached throws.
+ */
+async function probe(config, path) {
+    const apiKey = config.apiKey || undefined;
+    const response = await fetch(`${rootOf(config.baseUrl)}${path}`, {
+        headers: apiKey ? { authorization: `Bearer ${apiKey}` } : {},
+        signal: AbortSignal.timeout(timeoutMs(config) ?? PROBE_TIMEOUT_MS),
+    });
+    if (!response.ok)
+        return { missing: NOT_THERE.has(response.status) };
+    try {
+        return { missing: false, body: await response.json() };
+    }
+    catch {
+        return { missing: false };
+    }
+}
+/**
+ * The window a local server is actually serving a model in, which its listing does not say.
+ *
+ * llama.cpp reports it on `/props` as `default_generation_settings.n_ctx`, per slot, and LM
+ * Studio on `/api/v0/models` as `loaded_context_length` while the model is loaded. Both differ
+ * from the trained window in the case that matters, a model started in a smaller one than it was
+ * built for, and reading the trained one lets an overflow through the guard meant to catch it.
+ * Ollama reports nothing on either; an operator there has to declare the window.
+ *
+ * Zero where no answer was found. A server without either route, which is every hosted API, is
+ * latched and not asked again; one that could not be reached is not remembered at all.
+ *
+ * @param config The endpoint, plus the model whose window is wanted.
+ */
+export async function servedWindow(config) {
+    const endpoint = endpointKey(config);
+    if (unserved.has(endpoint))
+        return 0;
+    const key = JSON.stringify([endpoint, config.model]);
+    const known = served.get(key);
+    if (known && (known.window > 0 || Date.now() - known.at < LISTING_MISS_MS))
+        return known.window;
+    let window = 0;
+    try {
+        // Named, for a llama.cpp router serving several models; a single-model server ignores it.
+        const props = await probe(config, `/props?model=${encodeURIComponent(config.model)}`);
+        const settings = props.body
+            ?.default_generation_settings;
+        window = positive(settings?.n_ctx);
+        if (!window) {
+            const lmstudio = await probe(config, "/api/v0/models");
+            const { data } = (lmstudio.body ?? {});
+            const entry = Array.isArray(data)
+                ? data.find((model) => model?.id === config.model)
+                : undefined;
+            window = positive(entry?.loaded_context_length);
+            if (props.missing && lmstudio.missing) {
+                unserved.add(endpoint);
+                return 0;
+            }
+        }
+    }
+    catch {
+        return 0;
+    }
+    served.set(key, { window, at: Date.now() });
+    return window;
+}
 /**
  * Asks an endpoint what it serves, and remembers the answer.
  *
@@ -193,10 +295,11 @@ export async function listModels(config) {
  * happily load a 256k model at `-c 16384` and go on listing it as 256k — and a run refused on
  * the honest-looking number is a run that fails at the endpoint instead.
  *
- * Otherwise the endpoint's listing is asked — once, and again whenever it does not name this
- * model, since a model can arrive after the first listing was taken. A server that will not
- * list models still has to be able to run a turn: a failure here is an unknown window, not a
- * failed run.
+ * Otherwise the window the server is actually serving the model in, where it has an API that
+ * says (`servedWindow`), and failing that the endpoint's listing — asked once, and again whenever
+ * it does not name this model, since a model can arrive after the first listing was taken. A
+ * server that will not list models still has to be able to run a turn: a failure here is an
+ * unknown window, not a failed run.
  *
  * @param config The endpoint, plus the model whose window is wanted.
  * @param declared The operator's own number. Above zero it wins and the endpoint is not asked.
@@ -204,6 +307,9 @@ export async function listModels(config) {
 export async function contextLimitFor(config, declared = 0) {
     if (declared > 0)
         return declared;
+    const window = await servedWindow(config);
+    if (window > 0)
+        return window;
     const key = endpointKey(config);
     const listed = () => listings.get(key)?.find((model) => model.id === config.model);
     // The listing is asked for again when it does not name this model, rather than only when
@@ -245,4 +351,6 @@ export function resetClients() {
     clients.clear();
     listings.clear();
     misses.clear();
+    served.clear();
+    unserved.clear();
 }

package/dist/config.d.ts

@@ -18,7 +18,13 @@ export interface Endpoint {
     /** Empty is normal — a local server ignores it. See `getClient` for what is sent instead. */
     apiKey: string;
     /**
-     * Zero, less, or absent means no limit — what a local model answering slowly needs.
+     * How long an endpoint may go quiet. Zero, less, or absent means no limit — what a local model
+     * answering slowly needs.
+     *
+     * Not the whole request, which may take as long as the model keeps talking. On a streamed turn
+     * it is the silence allowed between chunks, the idle watchdog; the wait for the first chunk is
+     * `firstTokenSeconds`. On a call that does not stream, a side task or a model listing, it is
+     * the SDK's timer, which runs until the response headers arrive.
      *
      * Optional because a consumer that has no timeout to give should not have to invent one. Two
      * of the three servers this was extracted from carry no such field, and requiring it made
@@ -26,6 +32,18 @@ export interface Endpoint {
      * number standing in for an absent one.
      */
     requestTimeoutSeconds?: number;
+    /**
+     * How long a streamed turn may wait for its first chunk. Absent is five times
+     * `requestTimeoutSeconds`; zero or less is no limit.
+     *
+     * Its own number because the first wait is prefill, and on a local server prefill of a long
+     * prompt is tens of seconds on a GPU and minutes on a CPU, where the gap between tokens is a
+     * fraction of a second. A server loading the model on demand, Ollama after `keep_alive` or LM
+     * Studio just in time, holds the request open for the same reason. One number for both waits
+     * was either too slow to notice a wedged stream or tight enough to abandon a prefill, and a
+     * retry pays for that prefill again from nothing.
+     */
+    firstTokenSeconds?: number;
 }
 /** What to ask the model for. */
 export interface ModelParams {
@@ -68,6 +86,12 @@ export interface ToolPolicy {
 /** How many times a lost or refused request is worth sending again. See `retry.ts`. */
 export interface RetryPolicy {
     maxRetries: number;
+    /**
+     * How long to wait for a local server that says it is still loading the model, absent two
+     * minutes and zero not at all. Separate from `maxRetries`, which is sized for a request that
+     * was lost rather than for weights being read off a disk. See `isModelLoading`.
+     */
+    loadingTimeoutSeconds?: number;
 }
 /**
  * A whole agent configuration — every part, plus the two fields that belong to no group.

package/dist/index.d.ts

@@ -12,19 +12,20 @@
 export { type AgentLoopHooks, type AgentLoopOptions, type AgentLoopResult, buildBody, preselect, preview, resolveApiKey, runAgentLoop, type ToolCallOutcome, type ToolCallRequest, } from "./agent-loop.ts";
 export { type Capabilities, capabilitiesFor, type ModelCapabilities, modelCapabilitiesFor, type NegotiateOptions, negotiate, resetCapabilities, } from "./capabilities.ts";
 export type { CatalogServer } from "./catalog.ts";
-export { contextLimitFor, getClient, listModels, type ModelInfo, NO_KEY, resetClients, timeoutMs, } from "./client.ts";
+export { contextLimitFor, FIRST_TOKEN_FACTOR, firstTokenMs, getClient, listModels, type ModelInfo, NO_KEY, resetClients, servedWindow, timeoutMs, } from "./client.ts";
 export { COMPACT_AT, type CompactionOptions, type CompactionPlan, compactTranscript, KEEP_RATIO, type PruneOptions, planCompaction, pruneToolResults, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.ts";
 export type { AgentConfig, Endpoint, ModelParams, RetryPolicy, ToolPolicy, } from "./config.ts";
 export { errorMessage } from "./errors.ts";
 export { configureEvents, type EventBusOptions, emit, endRun, fold, history, type RunEvent, type RunEventInput, type RunEventKind, type RunUsage, resetEvents, watch, } from "./events.ts";
 export { assembleContext, configureHooks, type Gathered, gather, HOOK_CONTEXT_TOKENS, HOOK_EVENTS, HOOK_PREFACE, type HookContext, type HookEvent, type HookMessage, type HookNote, type HookOptions, type HookOutcome, type HookRunner, INJECT_EVENTS, notify, resetHooks, turnIndex, turnMessages, UNTRUSTED_PREFACE, untrusted, withContext, } from "./hooks.ts";
 export { resetAll } from "./reset.ts";
-export { backoffMs, ContextOverflow, compact, EndpointSilent, isOverflow, isTransient, messageTokens, requestTokens, SMALLEST_LIKELY_WINDOW, sleep, } from "./retry.ts";
+export { backoffMs, ContextOverflow, compact, EndpointSilent, isModelLoading, isOverflow, isTransient, LOADING_POLL_MS, LOADING_TIMEOUT_MS, messageTokens, requestTokens, SMALLEST_LIKELY_WINDOW, sleep, } from "./retry.ts";
 export { type RunTurnOptions, runTurn } from "./run-turn.ts";
 export { isGrammarError, relaxTools, sanitizeTools } from "./schema-compat.ts";
 export { type AskJsonOptions, ask, askJson, clean, listLines, parseJson, resetHints, type SideTaskOptions, tryAsk, } from "./side-task.ts";
 export { CAPABILITY_SNAPSHOT_VERSION, type CapabilitySnapshot, type EndpointSnapshot, exportCapabilities, importCapabilities, type ModelSnapshot, } from "./snapshot.ts";
 export { type Produced, type StreamTurnOptions, streamTurn, type Turn, type TurnUsage, } from "./stream.ts";
+export { ALL_FENCES, DEFAULT_FENCES, type Fence, FenceSplitter, type FenceSplitterOptions, type Split, stripThinking, THINK_FENCE, } from "./thinking.ts";
 export { estimateTokens } from "./tokens.ts";
 export { parseToolArguments, recoverToolCalls, ToolArgumentsError, type ToolCall, } from "./tool-calls.ts";
 export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadResult, MAX_CARRIED, MAX_PER_LOAD, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.ts";

package/dist/index.js

@@ -11,18 +11,19 @@
  */
 export { buildBody, preselect, preview, resolveApiKey, runAgentLoop, } from "./agent-loop.js";
 export { capabilitiesFor, modelCapabilitiesFor, negotiate, resetCapabilities, } from "./capabilities.js";
-export { contextLimitFor, getClient, listModels, NO_KEY, resetClients, timeoutMs, } from "./client.js";
+export { contextLimitFor, FIRST_TOKEN_FACTOR, firstTokenMs, getClient, listModels, NO_KEY, resetClients, servedWindow, timeoutMs, } from "./client.js";
 export { COMPACT_AT, compactTranscript, KEEP_RATIO, planCompaction, pruneToolResults, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.js";
 export { errorMessage } from "./errors.js";
 export { configureEvents, emit, endRun, fold, history, resetEvents, watch, } from "./events.js";
 export { assembleContext, configureHooks, gather, HOOK_CONTEXT_TOKENS, HOOK_EVENTS, HOOK_PREFACE, INJECT_EVENTS, notify, resetHooks, turnIndex, turnMessages, UNTRUSTED_PREFACE, untrusted, withContext, } from "./hooks.js";
 export { resetAll } from "./reset.js";
-export { backoffMs, ContextOverflow, compact, EndpointSilent, isOverflow, isTransient, messageTokens, requestTokens, SMALLEST_LIKELY_WINDOW, sleep, } from "./retry.js";
+export { backoffMs, ContextOverflow, compact, EndpointSilent, isModelLoading, isOverflow, isTransient, LOADING_POLL_MS, LOADING_TIMEOUT_MS, messageTokens, requestTokens, SMALLEST_LIKELY_WINDOW, sleep, } from "./retry.js";
 export { runTurn } from "./run-turn.js";
 export { isGrammarError, relaxTools, sanitizeTools } from "./schema-compat.js";
 export { ask, askJson, clean, listLines, parseJson, resetHints, tryAsk, } from "./side-task.js";
 export { CAPABILITY_SNAPSHOT_VERSION, exportCapabilities, importCapabilities, } from "./snapshot.js";
 export { streamTurn, } from "./stream.js";
+export { ALL_FENCES, DEFAULT_FENCES, FenceSplitter, stripThinking, THINK_FENCE, } from "./thinking.js";
 export { estimateTokens } from "./tokens.js";
 export { parseToolArguments, recoverToolCalls, ToolArgumentsError, } from "./tool-calls.js";
 export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadResult, MAX_CARRIED, MAX_PER_LOAD, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";

package/dist/retry.d.ts

@@ -93,6 +93,21 @@ export declare const SMALLEST_LIKELY_WINDOW = 8192;
  * @param error The rejection, as caught. What is not an SDK error is not transient.
  */
 export declare function isTransient(error: unknown): boolean;
+/**
+ * Whether a failure is a local server still loading the model, rather than one failing to serve.
+ *
+ * llama.cpp answers 503 `Loading model` with type `unavailable_error` from the moment it starts
+ * until the weights are mapped, and a router build says the same while it swaps models. That is
+ * thirty to ninety seconds for a large model from a cold page cache, and `backoffMs` gives up
+ * inside fifteen: sized for a busy host, not for one reading a file. A plain 503 is not this.
+ *
+ * @param error The rejection, as caught.
+ */
+export declare function isModelLoading(error: unknown): boolean;
+/** How long to wait between asking a loading server again. */
+export declare const LOADING_POLL_MS = 3000;
+/** How long `runTurn` waits for a model to load unless told otherwise. */
+export declare const LOADING_TIMEOUT_MS = 120000;
 /**
  * Exponential, with jitter so several tasks failing at once do not return in lockstep.
  *

package/dist/retry.js

@@ -53,6 +53,16 @@ const TOOL_CALLS_KEY = 15;
 const TEXT_PART = 26;
 /** The same for `{"type":"refusal","refusal":""},` around a refusal part. */
 const REFUSAL_PART = 32;
+/**
+ * The same for `"reasoning_content":"",` around an assistant message's scratchpad.
+ *
+ * Not in the SDK's types, and passed back by the caller who keeps it: gpt-oss and DeepSeek in
+ * thinking mode want the analysis behind a tool call on the next request. Left uncounted, the
+ * guard came up short by the whole scratchpad on exactly the runs that follow that rule.
+ */
+const REASONING_KEY = 23;
+/** The same for `"reasoning":"",`, OpenRouter's spelling of it. */
+const REASONING_ALT_KEY = 15;
 /** The divisor behind `estimateTokens`, applied here to a character count rather than a string. */
 const CHARS_PER_TOKEN = 4;
 /** How many characters one message is worth: its keys, and its content in whichever shape. */
@@ -76,6 +86,11 @@ function messageChars(message) {
         chars += NAME_KEY + message.name.length;
     if ("tool_call_id" in message && typeof message.tool_call_id === "string")
         chars += TOOL_CALL_ID_KEY + message.tool_call_id.length;
+    const { reasoning_content: reasoning, reasoning: alternate } = message;
+    if (typeof reasoning === "string")
+        chars += REASONING_KEY + reasoning.length;
+    if (typeof alternate === "string")
+        chars += REASONING_ALT_KEY + alternate.length;
     if ("tool_calls" in message && Array.isArray(message.tool_calls)) {
         chars += TOOL_CALLS_KEY;
         for (const call of message.tool_calls) {
@@ -214,6 +229,27 @@ export function isTransient(error) {
     const { status } = error;
     return status === 408 || status === 409 || status === 429 || (status ?? 0) >= 500;
 }
+/**
+ * Whether a failure is a local server still loading the model, rather than one failing to serve.
+ *
+ * llama.cpp answers 503 `Loading model` with type `unavailable_error` from the moment it starts
+ * until the weights are mapped, and a router build says the same while it swaps models. That is
+ * thirty to ninety seconds for a large model from a cold page cache, and `backoffMs` gives up
+ * inside fifteen: sized for a busy host, not for one reading a file. A plain 503 is not this.
+ *
+ * @param error The rejection, as caught.
+ */
+export function isModelLoading(error) {
+    if (!(error instanceof OpenAI.APIError) || error.status !== 503)
+        return false;
+    const body = error.error;
+    return (body?.type === "unavailable_error" ||
+        /loading model|model is loading|unavailable_error/i.test(`${error.message} ${body?.message ?? ""}`));
+}
+/** How long to wait between asking a loading server again. */
+export const LOADING_POLL_MS = 3000;
+/** How long `runTurn` waits for a model to load unless told otherwise. */
+export const LOADING_TIMEOUT_MS = 120_000;
 /**
  * Exponential, with jitter so several tasks failing at once do not return in lockstep.
  *

package/dist/run-turn.d.ts

@@ -63,6 +63,15 @@ export interface RunTurnOptions extends Omit<StreamTurnOptions, "produced"> {
      * of `extraBody`, ordinarily. See `NegotiateOptions.droppable`; it needs `model` too.
      */
     droppable?: Iterable<string>;
+    /**
+     * How long to wait on a server answering that the model is still loading, `LOADING_TIMEOUT_MS`
+     * unless given; zero gives up on the first such answer like any other 503.
+     *
+     * Polled every `LOADING_POLL_MS` without spending `maxRetries`, and announced once rather than
+     * per poll. A consumer that starts alongside its llama.cpp, or asks a router for a model it
+     * has to swap in, meets this on its first request every time.
+     */
+    loadingTimeoutMs?: number;
 }
 /**
  * `request` is a callback rather than a body because the body has to be rebuilt from whatever
@@ -77,4 +86,4 @@ export interface RunTurnOptions extends Omit<StreamTurnOptions, "produced"> {
  * @param options Retry budget, context limit, the model to negotiate for, notices, and the
  * stream's own callbacks.
  */
-export declare function runTurn(client: OpenAI, supports: Capabilities, request: (supports: Capabilities, model: ModelCapabilities | undefined) => OpenAI.ChatCompletionCreateParamsStreaming, { maxRetries, onNotice, contextLimit, model, droppable, ...stream }?: RunTurnOptions): Promise<Turn>;
+export declare function runTurn(client: OpenAI, supports: Capabilities, request: (supports: Capabilities, model: ModelCapabilities | undefined) => OpenAI.ChatCompletionCreateParamsStreaming, { maxRetries, onNotice, contextLimit, model, droppable, loadingTimeoutMs, ...stream }?: RunTurnOptions): Promise<Turn>;

package/dist/run-turn.js

@@ -1,6 +1,6 @@
 import { negotiate } from "./capabilities.js";
 import { errorMessage } from "./errors.js";
-import { backoffMs, ContextOverflow, compact, isOverflow, isTransient, requestTokens, SMALLEST_LIKELY_WINDOW, sleep, } from "./retry.js";
+import { backoffMs, ContextOverflow, compact, isModelLoading, isOverflow, isTransient, LOADING_POLL_MS, LOADING_TIMEOUT_MS, requestTokens, SMALLEST_LIKELY_WINDOW, sleep, } from "./retry.js";
 import { streamTurn } from "./stream.js";
 /**
  * `request` is a callback rather than a body because the body has to be rebuilt from whatever
@@ -15,7 +15,7 @@ import { streamTurn } from "./stream.js";
  * @param options Retry budget, context limit, the model to negotiate for, notices, and the
  * stream's own callbacks.
  */
-export async function runTurn(client, supports, request, { maxRetries = 0, onNotice, contextLimit = 0, model, droppable, ...stream } = {}) {
+export async function runTurn(client, supports, request, { maxRetries = 0, onNotice, contextLimit = 0, model, droppable, loadingTimeoutMs = LOADING_TIMEOUT_MS, ...stream } = {}) {
     // Sized once rather than per build. `request` is called again for every downgrade and every
     // retry, but a downgraded body is strictly smaller than the one before it and the transcript
     // does not change between attempts — so the first body is the one worth measuring, and
@@ -42,6 +42,9 @@ export async function runTurn(client, supports, request, { maxRetries = 0, onNot
         }
         return body;
     };
+    // When the server first said it was loading. Unset until then, and never reset: a model that
+    // loads, fails and loads again has had its allowance.
+    let loadingSince;
     for (let attempt = 0;; attempt++) {
         const produced = { any: false };
         try {
@@ -66,6 +69,20 @@ export async function runTurn(client, supports, request, { maxRetries = 0, onNot
             if (!(error instanceof ContextOverflow) && isOverflow(errorMessage(error))) {
                 throw new ContextOverflow(errorMessage(error), { cause: error });
             }
+            if (isModelLoading(error) && loadingTimeoutMs > 0) {
+                const now = Date.now();
+                if (loadingSince === undefined) {
+                    loadingSince = now;
+                    onNotice?.(`${model ?? "the model"} is still loading — waiting up to ${compact(loadingTimeoutMs / 1000)}s`);
+                }
+                if (now - loadingSince < loadingTimeoutMs) {
+                    // Not an attempt: the request was never looked at, and a two-minute load would
+                    // otherwise have to be bought with a retry budget meant for dropped connections.
+                    attempt--;
+                    await sleep(LOADING_POLL_MS, stream.signal);
+                    continue;
+                }
+            }
             if (attempt >= maxRetries || !isTransient(error))
                 throw error;
             const wait = backoffMs(attempt);

package/dist/side-task.js

@@ -4,6 +4,7 @@ import { endpointId, getClient } from "./client.js";
 import { errorMessage } from "./errors.js";
 import { isTransient } from "./retry.js";
 import { relaxTools, sanitizeTools } from "./schema-compat.js";
+import { stripThinking } from "./thinking.js";
 /**
  * One-shot calls that support a run without being one: picking tools, naming a session,
  * summarising a transcript, proposing follow-ups. They share a shape — small prompt, short
@@ -64,23 +65,6 @@ function rejectedTheRequest(error) {
         return false;
     return error.status === 400 || error.status === 422;
 }
-/**
- * Reasoning models that ignore the hints still fence their scratchpad; drop it.
- *
- * Including the fence that never closes. A side task answers under a small `max_tokens`, so a
- * model that spends it deliberating is cut off mid-scratchpad and the closing tag never
- * arrives — and the whole deliberation was then returned to the caller as the answer.
- */
-const stripThinking = (text) => text
-    .replace(/<think>[\s\S]*?<\/think>/gi, "")
-    .replace(/<think>[\s\S]*$/i, "")
-    // And the fence that never opens. Several chat templates put the opening tag at the end of
-    // the prompt rather than leaving the model to write it, so what comes back is deliberation
-    // first and only the closing tag to mark where it stops. Neither pattern above matches that,
-    // and the whole scratchpad went to the caller as the answer — a session title, a tool
-    // preselection, a suggestion list. Every real `<think>` is gone by this point, so a `</think>`
-    // still here opened in the prompt; the first one is taken, which keeps the most text.
-    .replace(/^[\s\S]*?<\/think>/i, "");
 /**
  * Runs a side task and returns the reply text, thinking stripped. Throws like any request.
  *
@@ -159,6 +143,9 @@ async function complete(config, model, system, user, { maxTokens = 512, temperat
             modelCapabilitiesFor(supports, model).reasoningEffort = false;
     }
     const message = response.choices[0]?.message;
+    // Reasoning models that ignore the hints still fence their scratchpad. A side task answers
+    // under a small ceiling, so the fence often never closes, and a template that opened it in
+    // the prompt leaves only the close; either way the deliberation used to come back as the answer.
     const answer = stripThinking(message?.content ?? "").trim();
     // Nothing but scratchpad. Some servers put the deliberation in its own field and leave the
     // content genuinely empty, in which case there is no answer to find anywhere else.

package/dist/stream.d.ts

@@ -1,4 +1,5 @@
 import type OpenAI from "openai";
+import { type Fence } from "./thinking.ts";
 /**
  * Reading one streamed turn back into a message.
  *
@@ -46,6 +47,17 @@ export interface Turn {
      * handed over rather than raised.
      */
     finishReason: string;
+    /**
+     * The model's scratchpad, as `onThinking` was told it, `""` where it deliberated in silence or
+     * not at all.
+     *
+     * Kept because some models want it back. gpt-oss and DeepSeek in thinking mode read the
+     * analysis behind a tool call off the assistant message on the next request, so a caller
+     * talking to one stores it as `reasoning_content` on that message for as long as the message
+     * ends in a tool call, and drops it once the model has given a final answer. A model that
+     * does not ask for it is better off without it: it is context, paid for on every turn.
+     */
+    reasoning: string;
 }
 /**
  * Whether the model has said anything a second attempt would say twice.
@@ -75,9 +87,30 @@ export interface StreamTurnOptions {
      * needs.
      */
     idleMs?: number;
+    /**
+     * Silence allowed before the first chunk, `idleMs` unless given; zero waits forever.
+     *
+     * The first wait is prefill, or a server loading the model, and is routinely many times the
+     * gap between tokens. See `Endpoint.firstTokenSeconds` and `firstTokenMs`.
+     */
+    firstChunkMs?: number;
     /** Set by the first chunk that carries anything, so a failed call knows if it can be retried. */
     produced?: Produced;
-    /** The model's scratchpad, as it arrives. */
+    /**
+     * The fences that mark a scratchpad written into `content`, `DEFAULT_FENCES` unless given.
+     *
+     * Text inside one goes to `onThinking` and `reasoning` rather than `onOutput` and `content`.
+     * `ALL_FENCES` adds `<thinking>` and `<reasoning>`, which a model can also be quoting; an
+     * empty list reads `content` as all answer.
+     */
+    fences?: readonly Fence[];
+    /**
+     * The chat template opened the first fence in the prompt, so the reply starts inside it.
+     * Without this the scratchpad is still moved out of `content` once the closing tag arrives,
+     * but `onOutput` will have been told it first.
+     */
+    startInReasoning?: boolean;
+    /** The model's scratchpad, as it arrives, from its own field or from a fence in `content`. */
     onThinking?: (delta: string) => void;
     /** The model's answer, as it arrives. */
     onOutput?: (delta: string) => void;
@@ -98,4 +131,4 @@ export interface StreamTurnOptions {
  * @param body The request, which must set `stream: true`.
  * @param options Cancellation, the idle watchdog, and the token callbacks.
  */
-export declare function streamTurn(client: OpenAI, body: OpenAI.ChatCompletionCreateParamsStreaming, { signal, idleMs, produced, onThinking, onOutput }?: StreamTurnOptions): Promise<Turn>;
+export declare function streamTurn(client: OpenAI, body: OpenAI.ChatCompletionCreateParamsStreaming, { signal, idleMs, firstChunkMs, produced, fences, startInReasoning, onThinking, onOutput, }?: StreamTurnOptions): Promise<Turn>;

package/dist/stream.js

@@ -1,4 +1,7 @@
 import { EndpointSilent } from "./retry.js";
+import { DEFAULT_FENCES, FenceSplitter } from "./thinking.js";
+/** The largest delay a timer takes, which is as close to none as the SDK's timeout option goes. */
+const NO_SDK_TIMEOUT = 2 ** 31 - 1;
 /**
  * Runs one turn as a stream, reporting tokens as they arrive and assembling them back into a
  * message.
@@ -15,7 +18,7 @@ import { EndpointSilent } from "./retry.js";
  * @param body The request, which must set `stream: true`.
  * @param options Cancellation, the idle watchdog, and the token callbacks.
  */
-export async function streamTurn(client, body, { signal, idleMs, produced, onThinking, onOutput } = {}) {
+export async function streamTurn(client, body, { signal, idleMs, firstChunkMs, produced, fences = DEFAULT_FENCES, startInReasoning, onThinking, onOutput, } = {}) {
     // Silence, not duration: the timer is rearmed on every chunk, so a model that is still
     // talking is never cut off however long it takes, and one that has stopped talking does not
     // hang the run until someone notices. A request that never answers at all is the same case
@@ -23,14 +26,20 @@ export async function streamTurn(client, body, { signal, idleMs, produced, onThi
     const watchdog = new AbortController();
     const linked = signal ? AbortSignal.any([signal, watchdog.signal]) : watchdog.signal;
     let idle;
-    const rearm = () => {
-        if (!idleMs)
-            return;
+    const first = firstChunkMs ?? idleMs;
+    // Until the model says something the wait is still prefill, however many empty chunks a
+    // server sends first: some open the stream with `{"role":"assistant"}` before they have read
+    // the prompt, and switching to the idle allowance on that abandoned the prefill it preceded.
+    let talking = false;
+    const rearm = (carried) => {
+        talking ||= carried;
+        const ms = talking ? idleMs : first;
         clearTimeout(idle);
-        idle = setTimeout(() => watchdog.abort(), idleMs);
+        if (ms)
+            idle = setTimeout(() => watchdog.abort(), ms);
     };
     try {
-        rearm();
+        rearm(false);
         return await collect();
     }
     catch (error) {
@@ -39,7 +48,9 @@ export async function streamTurn(client, body, { signal, idleMs, produced, onThi
         // this backwards records a stopped run as an endpoint fault, which nobody notices until
         // they read the row and disbelieve it.
         if (watchdog.signal.aborted && !signal?.aborted) {
-            throw new EndpointSilent(`the model endpoint sent nothing for ${(idleMs ?? 0) / 1000}s`);
+            const waited = talking ? idleMs : first;
+            const before = talking || first === idleMs ? "" : " before its first token";
+            throw new EndpointSilent(`the model endpoint sent nothing for ${(waited ?? 0) / 1000}s${before}`);
         }
         throw error;
     }
@@ -47,15 +58,32 @@ export async function streamTurn(client, body, { signal, idleMs, produced, onThi
         clearTimeout(idle);
     }
     async function collect() {
-        const stream = await client.chat.completions.create(body, { signal: linked });
-        const content = [];
-        const calls = new Map();
+        // The SDK's own timer runs until the headers arrive, which for a stream is the end of
+        // prefill, and it was set from the idle number. Where a watchdog is armed it covers that
+        // wait already, with the allowance meant for it, so the SDK is told to leave it alone.
+        const watched = first || idleMs;
+        const stream = await client.chat.completions.create(body, {
+            signal: linked,
+            ...(watched ? { timeout: NO_SDK_TIMEOUT } : {}),
+        });
+        // The field's reasoning here, the fenced kind in the splitter, which can still move text
+        // already read as answer into reasoning when a closing tag turns up with no opening one.
+        const reasoning = [];
+        const splitter = new FenceSplitter(fences, { startInside: startInReasoning });
+        const report = (parts) => {
+            for (const part of parts)
+                (part.kind === "reasoning" ? onThinking : onOutput)?.(part.text);
+        };
+        // In arrival order, sorted by index at the end; a call from a server that sent none keeps
+        // its place in the order they arrived.
+        const calls = [];
         const usage = { prompt: 0, completion: 0, total: 0, cached: 0 };
         let finishReason = "";
         for await (const chunk of stream) {
             // Rearmed on every chunk, latched below on only some: a priming chunk is the endpoint
-            // being alive, which is all the watchdog is asking about.
-            rearm();
+            // being alive, which is all the watchdog is asking about. Which allowance it rearms with
+            // moves from the first chunk's to the idle one when a chunk carries something.
+            rearm(false);
             // Assigned rather than accumulated. `stream_options.include_usage` sends one final chunk
             // and the two agree there, but a server that reports cumulatively per chunk makes a sum
             // of sums out of an accumulator — and a token count wrong by a factor of the chunk count
@@ -87,44 +115,87 @@ export async function streamTurn(client, body, { signal, idleMs, produced, onThi
             // fragments count even though no callback reports them: a partial call is state the turn
             // has accumulated, and losing a retry is the safer half of that trade. Set before the
             // callbacks, so a watcher that throws mid-token cannot be told the same token twice.
-            if (produced && (thinking || delta.content || delta.tool_calls?.length))
+            const carried = Boolean(thinking || delta.content || delta.tool_calls?.length);
+            if (carried && !talking)
+                rearm(true);
+            if (produced && carried)
                 produced.any = true;
-            if (thinking)
+            if (thinking) {
+                reasoning.push(thinking);
                 onThinking?.(thinking);
-            if (delta.content) {
-                content.push(delta.content);
-                onOutput?.(delta.content);
             }
+            if (delta.content)
+                report(splitter.push(delta.content));
             // Tool calls arrive in pieces, keyed by position: the id in one chunk, the name in
             // another, the arguments spread across the next several.
             for (const part of delta.tool_calls ?? []) {
-                const call = calls.get(part.index) ?? { id: "", name: "", arguments: "" };
+                const call = fragmentOf(part);
                 if (part.id)
                     call.id = part.id;
                 if (part.function?.name)
                     call.name += part.function.name;
                 if (part.function?.arguments)
                     call.arguments += part.function.arguments;
-                calls.set(part.index, call);
             }
         }
+        /**
+         * The call a fragment belongs to.
+         *
+         * By `index` where the server sent a number, which the SDK types as required and servers have
+         * nonetheless left out: keyed on `undefined`, every call joined into one whose name and
+         * arguments were all of theirs run together. Without one, by `id`; failing that, a fragment
+         * naming a function opens a call and a bare run of arguments continues the latest. A server
+         * that sends whole calls one per chunk at index `0` makes the same mistake the other way, so
+         * a different id, or a name after arguments have begun, opens a new call under that index.
+         */
+        function fragmentOf(part) {
+            const index = typeof part.index === "number" ? part.index : undefined;
+            const name = part.function?.name;
+            const known = index !== undefined
+                ? calls.findLast((call) => call.index === index)
+                : part.id
+                    ? calls.find((call) => call.id === part.id)
+                    : name
+                        ? undefined
+                        : calls.at(-1);
+            const another = known && ((part.id && known.id && part.id !== known.id) || (name && known.arguments));
+            if (known && !another)
+                return known;
+            const call = { index, id: "", name: "", arguments: "" };
+            calls.push(call);
+            return call;
+        }
         // An aborted stream ends its iteration rather than throwing, so without this a turn cut off
         // halfway — by the watchdog or by someone stopping the run — comes back looking like a
         // complete one, and a truncated answer is recorded as the output. Nothing about the API
         // says you have to know this.
         linked.throwIfAborted();
+        // What was held back as a possible tag. A reply that ended inside a fence stays reasoning:
+        // cut off at the ceiling mid-scratchpad, it has no answer, and promoting the deliberation to
+        // one is how a truncated turn gets recorded as output.
+        report(splitter.finish());
+        const minted = new Set();
         return {
-            content: content.join(""),
-            toolCalls: [...calls.entries()]
-                .sort(([a], [b]) => a - b)
-                .map(([index, call]) => ({
-                // A server that streams a call without an id still needs one for the result to answer.
-                id: call.id || `call_${index}`,
-                type: "function",
-                function: { name: call.name, arguments: call.arguments },
-            })),
+            content: splitter.output,
+            toolCalls: calls
+                .map((call, order) => ({ call, order }))
+                .sort((a, b) => (a.call.index ?? a.order) - (b.call.index ?? b.order) || a.order - b.order)
+                .map(({ call }, position) => {
+                // A server that streams a call without an id still needs one for the result to answer,
+                // and two calls it put under one index must not be answered as one.
+                let id = call.id || `call_${call.index ?? position}`;
+                if (minted.has(id))
+                    id = `call_${position}_${minted.size}`;
+                minted.add(id);
+                return {
+                    id,
+                    type: "function",
+                    function: { name: call.name, arguments: call.arguments },
+                };
+            }),
             usage,
             finishReason,
+            reasoning: reasoning.join("") + splitter.reasoning,
         };
     }
 }

package/dist/thinking.d.ts

@@ -0,0 +1,85 @@
+/**
+ * Telling a model's scratchpad from its answer when both arrive in `content`.
+ *
+ * A server with a reasoning parser puts the deliberation in a field of its own. One without — a
+ * llama.cpp started with `--reasoning-format none`, a model whose template has no parser, most
+ * distills on any server — leaves it fenced inline, and then it is shown as output, stored in
+ * the transcript and sent back on the next turn, where it costs context and teaches the model to
+ * keep doing it. This is the one table of fences both the stream and the side tasks read.
+ */
+/** One way a model marks off its scratchpad inside `content`. */
+export interface Fence {
+    open: string;
+    close: string;
+}
+/** DeepSeek, Qwen3, QwQ and most distills. */
+export declare const THINK_FENCE: Fence;
+/**
+ * The fences nobody writes by accident, and so the ones `streamTurn` reads by default.
+ *
+ * `<think>` opening a reply is never meant as output, and the other two are made of tokens that
+ * only a model's template produces: gpt-oss's harmony analysis channel, served raw, and Kimi's.
+ */
+export declare const DEFAULT_FENCES: readonly Fence[];
+/**
+ * Every fence known, including the two plain-word ones some fine-tunes and prompt-instructed
+ * models use. Those can also be text the model is quoting, so they are opt-in for a stream,
+ * and on for a side task, whose answer is too short to be quoting anything.
+ */
+export declare const ALL_FENCES: readonly Fence[];
+/** A piece of `content`, said to be one or the other. */
+export interface Split {
+    kind: "reasoning" | "output";
+    text: string;
+}
+/** What `FenceSplitter` takes besides its fences. */
+export interface FenceSplitterOptions {
+    /**
+     * The template already opened the first fence in the prompt, so the reply starts inside it.
+     *
+     * Several chat templates end the prompt with `<think>` rather than leaving the model to write
+     * it. Without this the splitter still catches it once `</think>` arrives, and moves what came
+     * before into `reasoning`, but a watcher will have been shown it as output by then.
+     */
+    startInside?: boolean;
+}
+/**
+ * A state machine over a stream of `content` that routes fenced text to reasoning.
+ *
+ * The reference shape is Vercel's `extractReasoningMiddleware`. A tag can be split across chunks,
+ * so the tail of each push that could be the start of one is held until the next push settles
+ * it; `finish` releases it. A reply cut off mid-scratchpad ends with the fence still open and its
+ * text still reasoning, rather than promoted to the answer.
+ *
+ * A closing tag with no opening one is the template having opened it in the prompt. When no
+ * fence has been seen yet, everything before it becomes reasoning retroactively in `output` and
+ * `reasoning`, though what was already handed out as output cannot be taken back.
+ */
+export declare class FenceSplitter {
+    #private;
+    /** The answer so far, with every fence taken out. */
+    output: string;
+    /** Everything that was inside a fence so far. */
+    reasoning: string;
+    /**
+     * @param fences The fences to read, `DEFAULT_FENCES` unless given; an empty list passes
+     *   everything through as output.
+     * @param options Whether the reply starts inside the first fence.
+     */
+    constructor(fences?: readonly Fence[], { startInside }?: FenceSplitterOptions);
+    /**
+     * Reads one more piece of content, returning what it settled, in order.
+     *
+     * @param text The next delta.
+     */
+    push(text: string): Split[];
+    /** Releases whatever was held back as a possible tag, now that no more is coming. */
+    finish(): Split[];
+}
+/**
+ * What is left of a complete reply once every scratchpad is taken out of it.
+ *
+ * @param text The whole reply.
+ * @param fences The fences to read, every known one unless given.
+ */
+export declare function stripThinking(text: string, fences?: readonly Fence[]): string;

package/dist/thinking.js

@@ -0,0 +1,182 @@
+/**
+ * Telling a model's scratchpad from its answer when both arrive in `content`.
+ *
+ * A server with a reasoning parser puts the deliberation in a field of its own. One without — a
+ * llama.cpp started with `--reasoning-format none`, a model whose template has no parser, most
+ * distills on any server — leaves it fenced inline, and then it is shown as output, stored in
+ * the transcript and sent back on the next turn, where it costs context and teaches the model to
+ * keep doing it. This is the one table of fences both the stream and the side tasks read.
+ */
+/** DeepSeek, Qwen3, QwQ and most distills. */
+export const THINK_FENCE = { open: "<think>", close: "</think>" };
+/**
+ * The fences nobody writes by accident, and so the ones `streamTurn` reads by default.
+ *
+ * `<think>` opening a reply is never meant as output, and the other two are made of tokens that
+ * only a model's template produces: gpt-oss's harmony analysis channel, served raw, and Kimi's.
+ */
+export const DEFAULT_FENCES = [
+    THINK_FENCE,
+    { open: "<|channel|>analysis<|message|>", close: "<|end|>" },
+    { open: "◁think▷", close: "◁/think▷" },
+];
+/**
+ * Every fence known, including the two plain-word ones some fine-tunes and prompt-instructed
+ * models use. Those can also be text the model is quoting, so they are opt-in for a stream,
+ * and on for a side task, whose answer is too short to be quoting anything.
+ */
+export const ALL_FENCES = [
+    ...DEFAULT_FENCES,
+    { open: "<thinking>", close: "</thinking>" },
+    { open: "<reasoning>", close: "</reasoning>" },
+];
+/**
+ * Framing that is neither scratchpad nor answer, dropped wherever it turns up outside a fence.
+ *
+ * Only harmony has any: after the analysis channel closes, raw gpt-oss output announces the final
+ * channel before the answer and ends with a return token.
+ */
+const FRAMING = {
+    "<|channel|>analysis<|message|>": [
+        "<|start|>assistant",
+        "<|channel|>final<|message|>",
+        "<|return|>",
+    ],
+};
+/**
+ * A state machine over a stream of `content` that routes fenced text to reasoning.
+ *
+ * The reference shape is Vercel's `extractReasoningMiddleware`. A tag can be split across chunks,
+ * so the tail of each push that could be the start of one is held until the next push settles
+ * it; `finish` releases it. A reply cut off mid-scratchpad ends with the fence still open and its
+ * text still reasoning, rather than promoted to the answer.
+ *
+ * A closing tag with no opening one is the template having opened it in the prompt. When no
+ * fence has been seen yet, everything before it becomes reasoning retroactively in `output` and
+ * `reasoning`, though what was already handed out as output cannot be taken back.
+ */
+export class FenceSplitter {
+    /** The answer so far, with every fence taken out. */
+    output = "";
+    /** Everything that was inside a fence so far. */
+    reasoning = "";
+    #fences;
+    #markers;
+    #inside;
+    #seenFence;
+    #held = "";
+    /**
+     * @param fences The fences to read, `DEFAULT_FENCES` unless given; an empty list passes
+     *   everything through as output.
+     * @param options Whether the reply starts inside the first fence.
+     */
+    constructor(fences = DEFAULT_FENCES, { startInside } = {}) {
+        this.#fences = fences;
+        this.#markers = fences.flatMap((fence) => [
+            fence.open,
+            fence.close,
+            ...(FRAMING[fence.open] ?? []),
+        ]);
+        this.#inside = startInside ? fences[0] : undefined;
+        this.#seenFence = this.#inside !== undefined;
+    }
+    /**
+     * Reads one more piece of content, returning what it settled, in order.
+     *
+     * @param text The next delta.
+     */
+    push(text) {
+        const parts = [];
+        let rest = this.#held + text;
+        this.#held = "";
+        while (rest) {
+            const found = this.#next(rest);
+            if (!found) {
+                const keep = this.#partialTail(rest);
+                this.#emit(parts, rest.slice(0, rest.length - keep));
+                this.#held = rest.slice(rest.length - keep);
+                break;
+            }
+            this.#emit(parts, rest.slice(0, found.at));
+            rest = rest.slice(found.at + found.marker.length);
+            this.#take(found.marker);
+        }
+        return parts;
+    }
+    /** Releases whatever was held back as a possible tag, now that no more is coming. */
+    finish() {
+        const parts = [];
+        this.#emit(parts, this.#held);
+        this.#held = "";
+        return parts;
+    }
+    /** The earliest marker that means something in the current state. */
+    #next(text) {
+        const candidates = this.#inside ? [this.#inside.close] : this.#markers;
+        let best;
+        for (const marker of candidates) {
+            const at = text.indexOf(marker);
+            if (at === -1)
+                continue;
+            if (!best || at < best.at || (at === best.at && marker.length > best.marker.length))
+                best = { at, marker };
+        }
+        return best;
+    }
+    /** How much of the end of `text` could be the beginning of a marker. */
+    #partialTail(text) {
+        const candidates = this.#inside ? [this.#inside.close] : this.#markers;
+        let keep = 0;
+        for (const marker of candidates)
+            for (let length = Math.min(marker.length - 1, text.length); length > keep; length--)
+                if (text.endsWith(marker.slice(0, length))) {
+                    keep = length;
+                    break;
+                }
+        return keep;
+    }
+    #take(marker) {
+        if (this.#inside) {
+            this.#inside = undefined;
+            return;
+        }
+        const opened = this.#fences.find((fence) => fence.open === marker);
+        if (opened) {
+            this.#inside = opened;
+            this.#seenFence = true;
+            return;
+        }
+        // A close with no open. The first time, the template opened it in the prompt; after a fence
+        // has been read, it is a stray tag and only dropped.
+        const closes = this.#fences.some((fence) => fence.close === marker);
+        if (closes && !this.#seenFence) {
+            this.reasoning += this.output;
+            this.output = "";
+        }
+        if (closes)
+            this.#seenFence = true;
+    }
+    #emit(parts, text) {
+        if (!text)
+            return;
+        const kind = this.#inside ? "reasoning" : "output";
+        this[kind] += text;
+        const last = parts.at(-1);
+        if (last?.kind === kind)
+            last.text += text;
+        else
+            parts.push({ kind, text });
+    }
+}
+/**
+ * What is left of a complete reply once every scratchpad is taken out of it.
+ *
+ * @param text The whole reply.
+ * @param fences The fences to read, every known one unless given.
+ */
+export function stripThinking(text, fences = ALL_FENCES) {
+    const splitter = new FenceSplitter(fences);
+    splitter.push(text);
+    splitter.finish();
+    return splitter.output;
+}

package/llms.txt

@@ -45,11 +45,14 @@ The contract between whatever holds the tools and the loop that offers them to a
 ### client
 
 - `contextLimitFor` — How much a model will read, in tokens.
+- `FIRST_TOKEN_FACTOR` — How many idle windows the first chunk gets when `firstTokenSeconds` is not given.
+- `firstTokenMs` — The wait for a streamed turn's first chunk, in the SDK's spelling: `undefined` is no limit.
 - `getClient` — The client for an endpoint, built once and kept.
 - `listModels` — Asks an endpoint what it serves, and remembers the answer.
 - `ModelInfo` (type) — A model an endpoint offers, and what it says the model will read.
 - `NO_KEY` — The SDK insists on a non-empty key even where the server will not look at it.
 - `resetClients` — Forgets every cached client and listing.
+- `servedWindow` — The window a local server is actually serving a model in, which its listing does not say.
 - `timeoutMs` — Zero, less, or absent means no limit, which the SDK spells as `undefined`.
 
 ### compaction
@@ -139,8 +142,11 @@ Everything about a request failing that is not about what the request said.
 - `ContextOverflow` — The request was bigger than the model will read.
 - `compact` — 1234 → "1.2k".
 - `EndpointSilent` — The endpoint stopped answering mid-request.
+- `isModelLoading` — Whether a failure is a local server still loading the model, rather than one failing to serve.
 - `isOverflow` — Whether a refusal means the request was too big, rather than merely refused.
 - `isTransient` — Whether a failed request is worth trying again.
+- `LOADING_POLL_MS` — How long to wait between asking a loading server again.
+- `LOADING_TIMEOUT_MS` — How long `runTurn` waits for a model to load unless told otherwise.
 - `messageTokens` — One message's estimated tokens, by the same count `requestTokens` sums for a whole request.
 - `requestTokens` — What this request will cost the window, in tokens, near enough.
 - `SMALLEST_LIKELY_WINDOW` — The smallest window worth believing in, and the floor under `runTurn`'s guard.
@@ -196,6 +202,19 @@ Reading one streamed turn back into a message.
 - `Turn` (type) — One streamed turn, put back together into the shape a loop and a transcript work with.
 - `TurnUsage` (type) — What a turn cost.
 
+### thinking
+
+Telling a model's scratchpad from its answer when both arrive in `content`.
+
+- `ALL_FENCES` — Every fence known, including the two plain-word ones some fine-tunes and prompt-instructed models use.
+- `DEFAULT_FENCES` — The fences nobody writes by accident, and so the ones `streamTurn` reads by default.
+- `Fence` (type) — One way a model marks off its scratchpad inside `content`.
+- `FenceSplitter` — A state machine over a stream of `content` that routes fenced text to reasoning.
+- `FenceSplitterOptions` (type) — What `FenceSplitter` takes besides its fences.
+- `Split` (type) — A piece of `content`, said to be one or the other.
+- `stripThinking` — What is left of a complete reply once every scratchpad is taken out of it.
+- `THINK_FENCE` — DeepSeek, Qwen3, QwQ and most distills.
+
 ### tokens
 
 - `estimateTokens` — Rough token count.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cubicecho/agent-core",
-  "version": "2.6.0",
+  "version": "2.8.0",
   "description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
   "keywords": [
     "openai",