@cubicecho/agent-core 2.21.1 → 2.23.0

package/README.md

@@ -36,13 +36,14 @@ only, Node >=22.
 | `continuation` | `continueTurn`: carries on an answer the token ceiling cut off, by prefilling it as a trailing assistant message. |
 | `config` | The structural interfaces every function here asks for. |
 | `run-turn` | `runTurn`: one turn with the retry loop around the negotiation around the stream. The whole loop, for a caller that wants it rather than its parts. Sizes the request against an opt-in `contextLimit`. |
-| `agent-loop` | `runAgentLoop`: the loop above a turn — `runTurn` per step, the tools between, `load_tools` and preselection handled, until the model stops asking. Plus the parts it is made of: `buildBody`, `preselect`, `preview`, and `resolveApiKey` for a caller deciding which key an endpoint gets. |
+| `agent-loop` | `runAgentLoop`: the loop above a turn — `runTurn` per step, the tools between, `load_tools` and preselection handled, until the model stops asking. Plus the parts it is made of: `buildBody`, `preselect`, `preview`, `resolveApiKey` for a caller deciding which key an endpoint gets, and `taskCall` for running a side task on the settings an agent spec gives it. |
 | `tool-calls` | Reading what a model meant by a tool call it did not write cleanly: `parseToolArguments` repairs almost-JSON arguments and says when they were cut off, `recoverToolCalls` finds calls written into the reply as text. |
 | `compaction` | Keeping a long run inside its window: `pruneToolResults` clears stale tool results, `planCompaction` and `compactTranscript` fold the oldest stretch into a summary. |
 | `ledger` | What stretches of a transcript cost, read off the prompt counts the server reported rather than estimated: `recordRequest`, `tokensBetween`, `estimateFrom`, `rebaseLedger`. |
 | `snapshot` | `exportCapabilities` and `importCapabilities`: the latched refusals as a JSON blob a consumer stores, so a restart need not learn them again. |
 | `spec` | `parseSpec`, `resolveAgentSpec` and `exportSpec`: an agent as a versioned JSON document, layered into the flat config the loop takes. Imports nothing but types, and is published separately at `@cubicecho/agent-core/spec`. |
 | `reset` | `resetAll`: drops every cache and latch in one call, so a teardown cannot forget one. |
+| `runtime` | `createRuntime`: a second set of those caches, with the functions that use them as methods — for a host whose tenants must not share a client pool, a latch or an event bus. `defaultRuntime` is the one the top-level functions use. |
 | `tokens` | `estimateTokens`: characters over four, deliberately low, for everything here that has to guess at a window. |
 | `errors` | `errorMessage`: a caught `unknown` turned into something a run row can hold. And what `runAgentLoop` throws, with the run as it stood: `AgentLoopError`, `ToolIterationLimit`, `AgentLoopOverflow`, and `failedRun` to read it off any of them. |
 | `catalog` | `CatalogServer`: the name-only shape `tool-loading` reads a connected server as. |
@@ -564,7 +565,9 @@ have to score well below the ones it kept — a hit just under the line scoring
 one just over it means the ranking chose arbitrarily, which is the case a model is worth spending
 on. Matching nothing is not confident either, since the words cannot tell a request that needs no
 tools from one whose words are not in the catalogue. An empty `toolSelectModel` still means no
-preselection at all, words included.
+preselection at all, words included. `preselect` also takes a `temperature` and a
+`reasoningEffort`, read as `ask` reads them, and `runAgentLoop({ preselect: true })` makes the call
+itself — see "Its side tasks" below.
 
 A turn cut off at `maxTokens` is said so as a notice, and with `maxContinuations` above zero it is
 continued first. `continueTurn` is the same thing for a caller with its own loop:
@@ -1150,12 +1153,54 @@ a form can load it. Nothing in `src/` imports it — the same direction the conf
 what lets a document travel between hosts without the loop growing an opinion about where an agent
 comes from.
 
+### Its side tasks
+
+`tasks.<key>` configures a side task apart from the main model — its own model, endpoint, ceiling,
+temperature and reasoning effort — and resolution hands each back as a `ResolvedTask` holding only
+what a layer stated. `taskCall` turns one into the arguments `ask`, `askJson`, `summariser` and
+`preselect` already take, so a setting added to a task later arrives without the host's call
+changing:
+
+```ts
+const title = taskCall(config.tasks.title, { signal, onNotice }, config);
+const name = await ask(title.endpoint, title.model, "Name this chat.", firstMessage, title.options);
+```
+
+**An absent setting is the side task's own default, never the agent's.** A titler under an agent
+running at temperature 1.0 is still sent 0.3, and its ceiling is still `ask`'s 512, `summariser`'s
+1024 or `preselect`'s 256. What the task does state wins over the same field in the options beside
+it, so the options are where a host puts its own default for a task that names none. `maxTokens: 0`
+sends no ceiling, as it does on the main model.
+
+`reasoningEffort` is the one that reads differently from the main model's. A side task is sent the
+no-thinking hints by default, and absent or `"off"` keeps them; a level — `"low"`, `"high"` — is
+sent as `reasoning_effort` in their place, for a compaction model an operator pointed at a model
+that should deliberate.
+
+The third argument is the endpoint the main turn uses. Given it, a task on the same endpoint is
+sent the agent's spelling of the URL and, having no key of its own, the agent's key, so the two
+share one entry in `capabilitiesFor` and a refusal learned by either is known to both; a task on
+another endpoint is sent only a key of its own, as `resolveApiKey` has it.
+
+The loop reads two of them when asked to. `preselect: true` has `runAgentLoop` make the
+preselection itself with `tasks.toolSelect` — or with `toolSelectModel` on the agent's endpoint,
+which stays the flattened spelling for a host that is not on the spec — and `compact: true` has it
+fold the transcript before a step, once `planCompaction` says a `contextLength` window is filling,
+with `tasks.compaction` writing the summary and no falling back to the main model without it. Both
+are off by default: a host that already calls `preselect` or compacts in `beforeStep` is not sent
+a second request.
+
+```ts
+const result = await runAgentLoop({ config, preselect: { keywords: true }, compact: true, ... });
+```
+
 ## What is kept for the life of the process
 
 Four caches outlive any one run: the `OpenAI` clients, the model listings, the latched
 capabilities, and `side-task`'s no-thinking hints. A fifth, the characters per token each model was
-measured at, is keyed on the endpoint's capabilities object, so it goes with them. All four are module-level and keyed on the same
-notion of an endpoint — its base URL and its API key — and the clients' key carries the request
+measured at, is keyed on the endpoint's capabilities object, so it goes with them. All four are
+held once per runtime — once per process, unless you make another, [below](#a-runtime-of-its-own)
+— and keyed on the same notion of an endpoint — its base URL and its API key — and the clients' key carries the request
 timeout as well, since that changes how a request is sent.
 
 **They are keyed per deployment, not per request.** What belongs in them is an endpoint an
@@ -1166,8 +1211,8 @@ bounding them assumes that, and a consumer that mints an API key per *user* brea
 
 That is bounded rather than unbounded: the client pool is a 32-entry LRU, and an evicted client
 costs its connection pool and nothing else, since the next request through that endpoint builds
-another. But churning connection pools is not sharing them, and at that point a client of your
-own, built and held per tenant, is the better answer than this.
+another. But churning connection pools is not sharing them, and at that point a runtime per
+tenant, [below](#a-runtime-of-its-own), is the better answer than this.
 
 The listings cache also remembers, for half a minute, that an endpoint did not name a model — the
 case where a configured name never matches anything the server serves (a llama.cpp `-a` alias, an
@@ -1241,6 +1286,57 @@ carries it in the snapshot so an imported latch keeps its real age instead of be
 every boot. Importing takes the older of the two ages, and a snapshot written before this field
 existed reads as met now.
 
+### A runtime of its own
+
+Everything above is the *default runtime*: one set of caches, shared by every caller in the
+process. `createRuntime` makes another, with its own clients, listings, latches, hints, measured
+ratios, event bus and hook settings, and the functions that use them as methods of the same names
+and signatures:
+
+```ts
+import { createRuntime } from "@cubicecho/agent-core";
+
+const runtime = createRuntime({ clients: { maxClients: 8 }, events: { maxEvents: 200 } });
+
+const run = await runtime.runAgentLoop({ config, messages, dispatch });
+for await (const event of runtime.watch(runId)) send(event);
+runtime.resetAll();                 // this runtime's caches, and nobody else's
+```
+
+It is for the host the section above warns off — one that mints a key per tenant, where a shared
+pool has tenants evicting each other's clients and a shared bus makes one tenant's run id readable
+by another — and for tests, where a runtime per case needs no `resetAll` between cases and lets
+them run in parallel. A host with one deployment's worth of endpoints needs none: the top-level
+functions are the default runtime's methods, and nothing about them has changed.
+
+The runtime travels by async context (`AsyncLocalStorage`), not by argument. A method runs its
+function with the runtime current, and it stays current across every `await` beneath it — so
+`runtime.runAgentLoop` uses the runtime's clients and latches all the way down, and so does a
+top-level function your code calls from inside it: an `emit` in `onEvent`, an `ask` in `dispatch`,
+a `gather` in a hook. `runtime.run(fn)` does the same for code that calls the top-level functions
+itself:
+
+```ts
+await runtime.run(async () => {
+  const limit = await contextLimitFor(config);   // the runtime's listing cache
+  emit(runId, { kind: "output", text: "…" });    // the runtime's bus
+});
+```
+
+Outside a method or `run`, a top-level function is the default runtime's again. That is the one
+thing to keep straight: a request handler that watches or emits to a run a runtime started is not
+inside it, and calls `runtime.watch` or `runtime.emit`. What a method hands back keeps its runtime
+— the iterator from `runtime.watch`, the function from `runtime.summariser` — wherever it is read.
+
+A new runtime starts from the package's defaults plus what `createRuntime` was given; it does not
+inherit what `configureClients` and the others set on the default one. Nothing needs closing: a
+runtime is garbage once nothing holds it. `defaultRuntime` is the process's own as an object, for
+code that takes a `Runtime` and should be handed the shared one.
+
+The two schema caches (`sanitizeTools`, `relaxTools`) and `requestTokens`' tool-length cache stay
+process-wide. They are memoised pure functions keyed on the caller's own objects, so there is
+nothing in them for one tenant to learn about another.
+
 ## Where the merged behaviour came from
 
 - `schema-compat` — `kanban_server`/`task_server`'s version, which strips **every** sibling of a

package/dist/agent-loop.d.ts

@@ -1,10 +1,12 @@
 import type OpenAI from "openai";
 import { type Capabilities, type ModelCapabilities } from "./capabilities.ts";
 import type { CatalogServer } from "./catalog.ts";
+import { type CompactionOptions } from "./compaction.ts";
 import type { Endpoint, ModelParams, RetryPolicy, ToolPolicy } from "./config.ts";
 import { type RunEventInput, type RunMetrics } from "./events.ts";
 import { type HookContext, type HookEvent, type HookNote, type HookRunner } from "./hooks.ts";
 import { type TokenLedger } from "./ledger.ts";
+import { type SideTask, type SideTaskOptions } from "./side-task.ts";
 import type { Turn, TurnUsage } from "./stream.ts";
 import { type KeywordPreselectOptions, type ToolOrder } from "./tool-loading.ts";
 /**
@@ -64,6 +66,38 @@ export declare function resolveApiKey(own: {
     baseUrl: string;
     apiKey?: string;
 }, env?: Record<string, string | undefined>): string;
+/**
+ * A side task's settings as the call that honours them: its endpoint, its model, and the options
+ * `ask`, `askJson`, `summariser` and `preselect` take last.
+ *
+ * A helper rather than a second signature on each of the four, so a setting added to a task later
+ * arrives through `options` without a host's call changing. What the task states wins over the
+ * same field in `options`: the host's code is written once and the task is what an operator
+ * configured over it, so a host passes its own ceiling as the value for a task that names none.
+ * A setting the task leaves out and `options` does too is absent from the result, which is what
+ * leaves it to the entry point's own default — 0.3, not the agent's temperature.
+ *
+ * A resolved spec carries no key, so a task on the agent's own endpoint would otherwise go out
+ * unkeyed, and be remembered under a second capability entry beside the agent's. Given `agent`,
+ * such a task is sent the agent's `baseUrl` as the agent spells it and, having no key of its own,
+ * the agent's key, so `capabilitiesFor` answers both with one object; a task on another endpoint
+ * is sent only a key of its own, as `resolveApiKey` has it. The environment is not read: the
+ * agent's key is whatever the main turn is sent.
+ *
+ * @param task The task, ordinarily `resolved.tasks.<key>`.
+ * @param options What the host adds — cancellation, notices, `keywords` — and its own defaults
+ * for the settings a task may leave out.
+ * @param agent The endpoint the main turn uses, key included. Absent sends the task's endpoint
+ * as it stands.
+ */
+export declare function taskCall<Options extends SideTaskOptions = SideTaskOptions>(task: SideTask, options?: Options, agent?: {
+    baseUrl: string;
+    apiKey?: string;
+}): {
+    endpoint: Endpoint;
+    model: string;
+    options: Options;
+};
 /**
  * The tools a request is likely to need, picked by a small model before the run starts, or none.
  *
@@ -83,13 +117,18 @@ export declare function resolveApiKey(own: {
  * means by empty.
  * @param catalog The servers to choose from.
  * @param prompt The request being planned for. Only its head is read; see `preselectInput`.
- * @param options Cancellation, notices, the reply ceiling (256), the cap the choice is held to
+ * @param options Cancellation, notices, the reply ceiling (256), the temperature and reasoning
+ * effort as `ask` reads them (0.3 and none when absent), the cap the choice is held to
  * (`MAX_PER_LOAD`), and whether to try the words first.
  */
-export declare function preselect(config: Endpoint, model: string, catalog: CatalogServer[], prompt: string, { signal, onNotice, maxTokens, maxPerLoad, keywords, }?: {
+export declare function preselect(config: Endpoint, model: string, catalog: CatalogServer[], prompt: string, { signal, onNotice, maxTokens, temperature, reasoningEffort, maxPerLoad, keywords, }?: {
     signal?: AbortSignal;
     onNotice?: (message: string) => void;
     maxTokens?: number;
+    /** As `SideTaskOptions.temperature`: absent is 0.3. */
+    temperature?: number;
+    /** As `SideTaskOptions.reasoningEffort`: absent keeps the no-thinking hints. */
+    reasoningEffort?: string;
     maxPerLoad?: number;
     /**
      * Try `preselectByKeywords` first and spend the model only on what it cannot settle. `true`
@@ -202,9 +241,14 @@ export interface AgentLoopOptions {
      * `toolDiscovery` absent is eager, `maxRetries` absent is none, and `contextLength` is handed
      * to `runTurn` as `contextLimit`, which sizes each request against the window before sending.
      * `"proxy"` is on-demand loading behind a tool array that never changes; see `PROXY_TOOLS`.
+     *
+     * `tasks` is a resolved agent's, and is read only for the side tasks the loop was asked to run:
+     * `tasks.toolSelect` under `preselect` and `tasks.compaction` under `compact`. `toolSelectModel`
+     * is the flattened spelling of the first, for a host that is not on the spec.
      */
-    config: Endpoint & ModelParams & Pick<ToolPolicy, "maxToolIterations"> & Partial<Pick<ToolPolicy, "toolDiscovery">> & Partial<RetryPolicy> & {
+    config: Endpoint & ModelParams & Pick<ToolPolicy, "maxToolIterations"> & Partial<Pick<ToolPolicy, "toolDiscovery" | "toolSelectModel">> & Partial<RetryPolicy> & {
         contextLength?: number;
+        tasks?: Readonly<Record<string, SideTask>>;
     };
     /**
      * The standing instruction, sent as the first message. On-demand and proxied modes append the
@@ -260,6 +304,43 @@ export interface AgentLoopOptions {
      * mode, or proxied, where no step is routed either way.
      */
     preselectRouting?: "exclusive" | "append";
+    /**
+     * Has the loop make the preselection itself, before its first request, with the preselector
+     * the config names. Off by default: a host that passes a resolved agent and calls `preselect`
+     * on its own is not sent a second one.
+     *
+     * The preselector is `config.tasks.toolSelect` with everything it states — its endpoint, model,
+     * ceiling, temperature and reasoning effort, keyed as `taskCall` keys a task against `config` —
+     * and otherwise `config.toolSelectModel` on the config's own endpoint at `preselect`'s defaults.
+     * What it is asked about is the text of the last user message. Nothing is asked in eager mode,
+     * with neither preselector named, with no user message, or when `preselected` is given — an
+     * empty one included, which is how a host says it has already decided.
+     *
+     * `true` takes `preselect`'s defaults; an object passes `keywords` and `maxPerLoad` through.
+     * Its notices arrive as `notice` events, and a preselection that fails picks nothing rather
+     * than ending the run. It runs before the hooks gather, not beside them.
+     */
+    preselect?: boolean | {
+        keywords?: boolean | KeywordPreselectOptions;
+        maxPerLoad?: number;
+    };
+    /**
+     * Has the loop fold the transcript itself before a step, once `planCompaction` says the window
+     * is filling, with the summary written by `config.tasks.compaction` on its own settings. Off by
+     * default, and nothing is folded without that task — there is no falling back to the main model
+     * — or without a `config.contextLength` to plan against.
+     *
+     * It runs after `beforeStep`, on what that returned, and is `compactTranscript` and nothing
+     * more: the plan is made from the last request's reported prompt, or an estimate before the
+     * first step and after a `beforeStep` rewrite; `hooks`, when given, are told `beforeCompact`
+     * beside the summary; and a fold is said as a `notice` event. A summary that fails is a notice
+     * too and folds nothing. Tool results are not pruned — `beforeStep` is where a host does that.
+     *
+     * The result's `messages` then open with the summary in place of what was folded, so a host
+     * that keeps its transcript append-only wants `runCompaction` and a stored fold instead. `true`
+     * plans by `planCompaction`'s defaults; an object moves its thresholds.
+     */
+    compact?: boolean | Pick<CompactionOptions, "compactAt" | "keepRatio" | "target">;
     /**
      * Tools already loaded, carried from an earlier question. See `carryOver`. Not read in proxied
      * mode, where a tool is loaded only while its definition is in the history.
@@ -469,6 +550,11 @@ export interface AgentLoopResult {
  * the question. `call_tool` is honoured on demand too, unless the host has a tool of that name —
  * a session switched out of proxied mode still has it in its history, and the model copies it.
  *
+ * Neither side task is run unless asked for. `preselect` has the loop make the preselection and
+ * `compact` has it fold the transcript, each by the task `config.tasks` carries for it and on
+ * that task's own settings — a setting the task leaves out is the side task's default, never the
+ * run's `temperature` or `maxTokens`.
+ *
  * @param options The config, transcript, tools and dispatcher, plus the optional hooks, events
  * and cancellation. See `AgentLoopOptions`.
  */

package/dist/agent-loop.js

@@ -1,6 +1,7 @@
 import { charsPerTokenFor } from "./calibration.js";
 import { capabilitiesFor, effortFor, modelCapabilitiesFor, } from "./capabilities.js";
 import { firstTokenMs, getClient, NO_KEY, timeoutMs } from "./client.js";
+import { compactTranscript, planCompaction, summariser, } from "./compaction.js";
 import { continueTurn } from "./continuation.js";
 import { AgentLoopError, AgentLoopOverflow, errorMessage, ToolIterationLimit, } from "./errors.js";
 import { runMetrics } from "./events.js";
@@ -106,6 +107,52 @@ export function resolveApiKey(own, inherited, env = process.env) {
         return NO_KEY;
     return inherited?.apiKey || env.OPENAI_API_KEY || NO_KEY;
 }
+/**
+ * A side task's settings as the call that honours them: its endpoint, its model, and the options
+ * `ask`, `askJson`, `summariser` and `preselect` take last.
+ *
+ * A helper rather than a second signature on each of the four, so a setting added to a task later
+ * arrives through `options` without a host's call changing. What the task states wins over the
+ * same field in `options`: the host's code is written once and the task is what an operator
+ * configured over it, so a host passes its own ceiling as the value for a task that names none.
+ * A setting the task leaves out and `options` does too is absent from the result, which is what
+ * leaves it to the entry point's own default — 0.3, not the agent's temperature.
+ *
+ * A resolved spec carries no key, so a task on the agent's own endpoint would otherwise go out
+ * unkeyed, and be remembered under a second capability entry beside the agent's. Given `agent`,
+ * such a task is sent the agent's `baseUrl` as the agent spells it and, having no key of its own,
+ * the agent's key, so `capabilitiesFor` answers both with one object; a task on another endpoint
+ * is sent only a key of its own, as `resolveApiKey` has it. The environment is not read: the
+ * agent's key is whatever the main turn is sent.
+ *
+ * @param task The task, ordinarily `resolved.tasks.<key>`.
+ * @param options What the host adds — cancellation, notices, `keywords` — and its own defaults
+ * for the settings a task may leave out.
+ * @param agent The endpoint the main turn uses, key included. Absent sends the task's endpoint
+ * as it stands.
+ */
+export function taskCall(task, options, agent) {
+    const own = task.endpoint;
+    const shared = agent && (!own.baseUrl.trim() || sameUrl(own.baseUrl, agent.baseUrl));
+    const stated = {};
+    if (task.maxTokens !== undefined)
+        stated.maxTokens = task.maxTokens;
+    if (task.temperature !== undefined)
+        stated.temperature = task.temperature;
+    if (task.reasoningEffort !== undefined)
+        stated.reasoningEffort = task.reasoningEffort;
+    return {
+        endpoint: agent
+            ? {
+                ...own,
+                ...(shared ? { baseUrl: agent.baseUrl } : {}),
+                apiKey: resolveApiKey(own, agent, {}),
+            }
+            : own,
+        model: task.model,
+        options: { ...options, ...stated },
+    };
+}
 /**
  * The tools a request is likely to need, picked by a small model before the run starts, or none.
  *
@@ -125,10 +172,11 @@ export function resolveApiKey(own, inherited, env = process.env) {
  * means by empty.
  * @param catalog The servers to choose from.
  * @param prompt The request being planned for. Only its head is read; see `preselectInput`.
- * @param options Cancellation, notices, the reply ceiling (256), the cap the choice is held to
+ * @param options Cancellation, notices, the reply ceiling (256), the temperature and reasoning
+ * effort as `ask` reads them (0.3 and none when absent), the cap the choice is held to
  * (`MAX_PER_LOAD`), and whether to try the words first.
  */
-export async function preselect(config, model, catalog, prompt, { signal, onNotice, maxTokens = 256, maxPerLoad = MAX_PER_LOAD, keywords, } = {}) {
+export async function preselect(config, model, catalog, prompt, { signal, onNotice, maxTokens = 256, temperature, reasoningEffort, maxPerLoad = MAX_PER_LOAD, keywords, } = {}) {
     if (!model || !catalog.some((server) => server.tools.length > 0))
         return [];
     if (keywords) {
@@ -141,7 +189,7 @@ export async function preselect(config, model, catalog, prompt, { signal, onNoti
             return guess.names;
         }
     }
-    const reply = await tryAsk("preselect", () => askJson(config, model, preselectSystem(maxPerLoad), preselectInput(catalog, prompt), PRESELECT_SCHEMA, { name: "preselection", maxTokens, signal, onNotice }), { onNotice });
+    const reply = await tryAsk("preselect", () => askJson(config, model, preselectSystem(maxPerLoad), preselectInput(catalog, prompt), PRESELECT_SCHEMA, { name: "preselection", maxTokens, temperature, reasoningEffort, signal, onNotice }), { onNotice });
     return preselection(reply, catalog, maxPerLoad);
 }
 /**
@@ -240,6 +288,11 @@ const UNRUN_CALL = "Not run: the run stopped first.";
  * the question. `call_tool` is honoured on demand too, unless the host has a tool of that name —
  * a session switched out of proxied mode still has it in its history, and the model copies it.
  *
+ * Neither side task is run unless asked for. `preselect` has the loop make the preselection and
+ * `compact` has it fold the transcript, each by the task `config.tasks` carries for it and on
+ * that task's own settings — a setting the task leaves out is the side task's default, never the
+ * run's `temperature` or `maxTokens`.
+ *
  * @param options The config, transcript, tools and dispatcher, plus the optional hooks, events
  * and cancellation. See `AgentLoopOptions`.
  */
@@ -270,6 +323,15 @@ export async function runAgentLoop(options) {
         return result;
     throw new ToolIterationLimit(`Stopped after ${options.config.maxToolIterations} tool iterations.`, standing.read());
 }
+/** A message's words, as the preselector is asked about them: its string, or its text parts. */
+const userText = (message) => {
+    const content = message?.content;
+    if (typeof content === "string")
+        return content;
+    if (!Array.isArray(content))
+        return "";
+    return content.flatMap((part) => (part.type === "text" ? part.text : [])).join("\n");
+};
 /**
  * The steps of `runAgentLoop`, throwing what they caught as it was caught. Resolves to nothing
  * when `maxToolIterations` is spent, and tells `standing` how to read the run either way.
@@ -311,10 +373,44 @@ async function runSteps(options, standing) {
     const onDemand = proxied || (config.toolDiscovery === "ondemand" && catalog.length > 0);
     // Proxied, `loaded` is what this run has put a definition in the history for, and starts empty.
     const loaded = new Set(onDemand && !proxied ? (options.loaded ?? []) : []);
-    const preselected = onDemand ? [...(options.preselected ?? [])] : [];
+    // The loop's own preselection, where it was asked for and the host has not already decided: by
+    // the task where the config carries one, and by the flattened name on its own endpoint where
+    // it does not. `preselect` answers an empty model with nothing, so neither named asks nothing.
+    const choose = async () => {
+        const prompt = userText(options.messages.findLast((message) => message.role === "user"));
+        if (!prompt.trim())
+            return [];
+        const asked = {
+            ...(typeof options.preselect === "object" ? options.preselect : {}),
+            signal,
+            onNotice: notice,
+        };
+        const selector = config.tasks?.toolSelect;
+        if (!selector?.model) {
+            return preselect(config, config.toolSelectModel ?? "", catalog, prompt, asked);
+        }
+        const call = taskCall(selector, asked, config);
+        return preselect(call.endpoint, call.model, catalog, prompt, call.options);
+    };
+    const preselected = !onDemand
+        ? []
+        : options.preselect && options.preselected === undefined
+            ? await choose()
+            : [...(options.preselected ?? [])];
     if (!proxied)
         for (const name of preselected)
             loaded.add(name);
+    // The loop's own compaction, which is the task's or nothing: a summary the main model writes is
+    // a second full-price request the operator did not ask for.
+    const compactor = options.compact ? config.tasks?.compaction : undefined;
+    const summarise = (() => {
+        if (!compactor?.model)
+            return undefined;
+        const call = taskCall(compactor, { signal, onNotice: notice }, config);
+        const write = summariser(call.endpoint, call.model, call.options);
+        // A summary that fails folds nothing, and the step goes out as it was.
+        return async (text) => (await tryAsk("compaction", () => write(text), { onNotice: notice })) ?? "";
+    })();
     const used = new Set();
     const definitions = new Map();
     for (const tool of tools) {
@@ -418,11 +514,32 @@ async function runSteps(options, standing) {
             limit: config.contextLength ?? 0,
             ledger,
         });
+        let next = rewritten ?? messages;
+        if (summarise) {
+            const plan = planCompaction(next, {
+                ...(typeof options.compact === "object" ? options.compact : {}),
+                limit: config.contextLength ?? 0,
+                // The last request's prompt describes this transcript only while nothing has rewritten it.
+                ...(next === messages && previous && previous.prompt > 0 ? { used: previous.prompt } : {}),
+                charsPerToken: charsPerTokenFor(supports, config.model),
+                ...((compactor?.maxTokens ?? 0) > 0 ? { summaryTokens: compactor?.maxTokens } : {}),
+            });
+            if (plan) {
+                const folded = await compactTranscript(next, plan, summarise, {
+                    ...(hooks
+                        ? { hooks: { run: hooks.run, context: hooks.context, onNote: hooks.onNote } }
+                        : {}),
+                });
+                if (folded !== next)
+                    notice(`compacted ${plan.cut - plan.from} messages into a summary`);
+                next = folded;
+            }
+        }
         // A rewrite may have folded a definition away, and a load answered "already loaded" would
         // then point at nothing. Forgetting costs a definition sent twice at worst.
-        if (proxied && rewritten && rewritten !== messages)
+        if (proxied && next !== messages)
             loaded.clear();
-        messages = rewritten ?? messages;
+        messages = next;
         ledger = rebaseLedger(ledger, before, messages);
         onEvent({ kind: "turn", text: `turn ${step + 1}` });
         // Appended, the shortlist is already in `loaded` — after what was carried, and once — so the

package/dist/calibration.js

@@ -1,4 +1,5 @@
 import { CHARS_PER_TOKEN, requestChars, toolsChars } from "./retry.js";
+import { scoped } from "./scope.js";
 /**
  * How many characters a token is worth on one model, learned from what its endpoint reports.
  *
@@ -27,7 +28,7 @@ const PLAUSIBLE = { least: 1, most: 8 };
  * `resetCapabilities` takes its readings with it. Replaced rather than cleared by `resetCalibration`,
  * since a `WeakMap` has no `clear`.
  */
-let readings = new WeakMap();
+const readings = scoped(() => new WeakMap());
 /**
  * The characters per token to size a request to this model with, `CHARS_PER_TOKEN` until a turn
  * has reported one.
@@ -43,7 +44,7 @@ let readings = new WeakMap();
  * @param model The name the endpoint knows the model as, as it goes in the body.
  */
 export function charsPerTokenFor(supports, model) {
-    const known = readings.get(supports)?.get(model);
+    const known = readings().get(supports)?.get(model);
     return known?.length ? Math.max(...known) : CHARS_PER_TOKEN;
 }
 /** Whether any message carries a part whose tokens its characters do not count. */
@@ -69,10 +70,10 @@ export function calibrate(supports, body, promptTokens) {
     if (ratio < PLAUSIBLE.least || ratio > PLAUSIBLE.most) {
         return charsPerTokenFor(supports, body.model);
     }
-    let models = readings.get(supports);
+    let models = readings().get(supports);
     if (!models) {
         models = new Map();
-        readings.set(supports, models);
+        readings().set(supports, models);
     }
     const known = models.get(body.model) ?? [];
     known.push(ratio);
@@ -83,5 +84,5 @@ export function calibrate(supports, body, promptTokens) {
 }
 /** Forgets every reading, so the next request is sized at `CHARS_PER_TOKEN` again. */
 export function resetCalibration() {
-    readings = new WeakMap();
+    readings.reset();
 }

package/dist/capabilities.js

@@ -1,6 +1,7 @@
 import { endpointId } from "./client.js";
 import { errorMessage } from "./errors.js";
 import { isGrammarError } from "./schema-compat.js";
+import { scoped } from "./scope.js";
 /**
  * What each endpoint cannot do, remembered for the life of the process.
  *
@@ -19,7 +20,7 @@ import { isGrammarError } from "./schema-compat.js";
  * behind it, including the `models` map underneath, which is the level where two keys through
  * one host are most likely to differ at all.
  */
-const capabilities = new Map();
+const capabilities = scoped(() => new Map());
 /**
  * What this endpoint is known not to support. The same object every time, so what `negotiate`
  * latches off stays off.
@@ -64,16 +65,16 @@ export function modelCapabilitiesFor(supports, model) {
     return known;
 }
 /** Every endpoint's capabilities by `endpointId`, the live objects, for `exportCapabilities`. */
-export const knownCapabilities = () => capabilities;
+export const knownCapabilities = () => capabilities();
 /**
  * The capabilities of the endpoint with this `endpointId`, created optimistic if unseen — the way
  * `importCapabilities` reaches an endpoint it has only a digest for.
  */
 export function capabilitiesById(id) {
-    let known = capabilities.get(id);
+    let known = capabilities().get(id);
     if (!known) {
         known = { strictSchemas: true, usageInStream: true, models: new Map(), since: Date.now() };
-        capabilities.set(id, known);
+        capabilities().set(id, known);
     }
     return known;
 }
@@ -92,11 +93,11 @@ export function capabilitiesById(id) {
  */
 export function resetCapabilities(endpoint) {
     if (!endpoint) {
-        const held = capabilities.size > 0;
-        capabilities.clear();
+        const held = capabilities().size > 0;
+        capabilities().clear();
         return held;
     }
-    return capabilities.delete(endpointId(endpoint));
+    return capabilities().delete(endpointId(endpoint));
 }
 /**
  * Forgets every endpoint whose entry is older than this, so the next request finds out again.
@@ -117,10 +118,10 @@ export function resetCapabilities(endpoint) {
  */
 export function expireCapabilities(maxAgeMs, now = Date.now()) {
     let dropped = 0;
-    for (const [id, known] of capabilities) {
+    for (const [id, known] of capabilities()) {
         if (now - known.since < maxAgeMs)
             continue;
-        capabilities.delete(id);
+        capabilities().delete(id);
         dropped++;
     }
     return dropped;

package/dist/client.d.ts

@@ -27,10 +27,11 @@ export interface ClientPoolOptions {
  * Changes what the client pool is held to, for a process whose endpoints are not shaped like the
  * deployments these defaults were chosen for.
  *
- * Module-level for the same reason `configureEvents` is: the pool is one module-level thing, and
- * its size is a deployment's setting, said once at startup. A multi-tenant host keying on a key
- * per user raises `maxClients` so its tenants stop evicting each other's connection pools; a dev
- * box shortens `listingMissMs` so a model it has just pulled is seen sooner.
+ * One setting per runtime for the same reason `configureEvents` is: the pool is one thing per
+ * runtime, and its size is a deployment's setting, said once at startup. A multi-tenant host
+ * keying on a key per user raises `maxClients` so its tenants stop evicting each other's
+ * connection pools, or gives each tenant a `createRuntime` of its own; a dev box shortens
+ * `listingMissMs` so a model it has just pulled is seen sooner.
  *
  * A `maxClients` below the pool's current size evicts down to it at once, least recently asked
  * for first, as the next `getClient` would have. A shorter `listingMissMs` applies to misses