@cubicecho/agent-core 2.22.0 → 2.23.0

package/README.md

@@ -43,6 +43,7 @@ only, Node >=22.
 | `snapshot` | `exportCapabilities` and `importCapabilities`: the latched refusals as a JSON blob a consumer stores, so a restart need not learn them again. |
 | `spec` | `parseSpec`, `resolveAgentSpec` and `exportSpec`: an agent as a versioned JSON document, layered into the flat config the loop takes. Imports nothing but types, and is published separately at `@cubicecho/agent-core/spec`. |
 | `reset` | `resetAll`: drops every cache and latch in one call, so a teardown cannot forget one. |
+| `runtime` | `createRuntime`: a second set of those caches, with the functions that use them as methods — for a host whose tenants must not share a client pool, a latch or an event bus. `defaultRuntime` is the one the top-level functions use. |
 | `tokens` | `estimateTokens`: characters over four, deliberately low, for everything here that has to guess at a window. |
 | `errors` | `errorMessage`: a caught `unknown` turned into something a run row can hold. And what `runAgentLoop` throws, with the run as it stood: `AgentLoopError`, `ToolIterationLimit`, `AgentLoopOverflow`, and `failedRun` to read it off any of them. |
 | `catalog` | `CatalogServer`: the name-only shape `tool-loading` reads a connected server as. |
@@ -1197,8 +1198,9 @@ const result = await runAgentLoop({ config, preselect: { keywords: true }, compa
 
 Four caches outlive any one run: the `OpenAI` clients, the model listings, the latched
 capabilities, and `side-task`'s no-thinking hints. A fifth, the characters per token each model was
-measured at, is keyed on the endpoint's capabilities object, so it goes with them. All four are module-level and keyed on the same
-notion of an endpoint — its base URL and its API key — and the clients' key carries the request
+measured at, is keyed on the endpoint's capabilities object, so it goes with them. All four are
+held once per runtime — once per process, unless you make another, [below](#a-runtime-of-its-own)
+— and keyed on the same notion of an endpoint — its base URL and its API key — and the clients' key carries the request
 timeout as well, since that changes how a request is sent.
 
 **They are keyed per deployment, not per request.** What belongs in them is an endpoint an
@@ -1209,8 +1211,8 @@ bounding them assumes that, and a consumer that mints an API key per *user* brea
 
 That is bounded rather than unbounded: the client pool is a 32-entry LRU, and an evicted client
 costs its connection pool and nothing else, since the next request through that endpoint builds
-another. But churning connection pools is not sharing them, and at that point a client of your
-own, built and held per tenant, is the better answer than this.
+another. But churning connection pools is not sharing them, and at that point a runtime per
+tenant, [below](#a-runtime-of-its-own), is the better answer than this.
 
 The listings cache also remembers, for half a minute, that an endpoint did not name a model — the
 case where a configured name never matches anything the server serves (a llama.cpp `-a` alias, an
@@ -1284,6 +1286,57 @@ carries it in the snapshot so an imported latch keeps its real age instead of be
 every boot. Importing takes the older of the two ages, and a snapshot written before this field
 existed reads as met now.
 
+### A runtime of its own
+
+Everything above is the *default runtime*: one set of caches, shared by every caller in the
+process. `createRuntime` makes another, with its own clients, listings, latches, hints, measured
+ratios, event bus and hook settings, and the functions that use them as methods of the same names
+and signatures:
+
+```ts
+import { createRuntime } from "@cubicecho/agent-core";
+
+const runtime = createRuntime({ clients: { maxClients: 8 }, events: { maxEvents: 200 } });
+
+const run = await runtime.runAgentLoop({ config, messages, dispatch });
+for await (const event of runtime.watch(runId)) send(event);
+runtime.resetAll();                 // this runtime's caches, and nobody else's
+```
+
+It is for the host the section above warns off — one that mints a key per tenant, where a shared
+pool has tenants evicting each other's clients and a shared bus makes one tenant's run id readable
+by another — and for tests, where a runtime per case needs no `resetAll` between cases and lets
+them run in parallel. A host with one deployment's worth of endpoints needs none: the top-level
+functions are the default runtime's methods, and nothing about them has changed.
+
+The runtime travels by async context (`AsyncLocalStorage`), not by argument. A method runs its
+function with the runtime current, and it stays current across every `await` beneath it — so
+`runtime.runAgentLoop` uses the runtime's clients and latches all the way down, and so does a
+top-level function your code calls from inside it: an `emit` in `onEvent`, an `ask` in `dispatch`,
+a `gather` in a hook. `runtime.run(fn)` does the same for code that calls the top-level functions
+itself:
+
+```ts
+await runtime.run(async () => {
+  const limit = await contextLimitFor(config);   // the runtime's listing cache
+  emit(runId, { kind: "output", text: "…" });    // the runtime's bus
+});
+```
+
+Outside a method or `run`, a top-level function is the default runtime's again. That is the one
+thing to keep straight: a request handler that watches or emits to a run a runtime started is not
+inside it, and calls `runtime.watch` or `runtime.emit`. What a method hands back keeps its runtime
+— the iterator from `runtime.watch`, the function from `runtime.summariser` — wherever it is read.
+
+A new runtime starts from the package's defaults plus what `createRuntime` was given; it does not
+inherit what `configureClients` and the others set on the default one. Nothing needs closing: a
+runtime is garbage once nothing holds it. `defaultRuntime` is the process's own as an object, for
+code that takes a `Runtime` and should be handed the shared one.
+
+The two schema caches (`sanitizeTools`, `relaxTools`) and `requestTokens`' tool-length cache stay
+process-wide. They are memoised pure functions keyed on the caller's own objects, so there is
+nothing in them for one tenant to learn about another.
+
 ## Where the merged behaviour came from
 
 - `schema-compat` — `kanban_server`/`task_server`'s version, which strips **every** sibling of a

package/dist/calibration.js

@@ -1,4 +1,5 @@
 import { CHARS_PER_TOKEN, requestChars, toolsChars } from "./retry.js";
+import { scoped } from "./scope.js";
 /**
  * How many characters a token is worth on one model, learned from what its endpoint reports.
  *
@@ -27,7 +28,7 @@ const PLAUSIBLE = { least: 1, most: 8 };
  * `resetCapabilities` takes its readings with it. Replaced rather than cleared by `resetCalibration`,
  * since a `WeakMap` has no `clear`.
  */
-let readings = new WeakMap();
+const readings = scoped(() => new WeakMap());
 /**
  * The characters per token to size a request to this model with, `CHARS_PER_TOKEN` until a turn
  * has reported one.
@@ -43,7 +44,7 @@ let readings = new WeakMap();
  * @param model The name the endpoint knows the model as, as it goes in the body.
  */
 export function charsPerTokenFor(supports, model) {
-    const known = readings.get(supports)?.get(model);
+    const known = readings().get(supports)?.get(model);
     return known?.length ? Math.max(...known) : CHARS_PER_TOKEN;
 }
 /** Whether any message carries a part whose tokens its characters do not count. */
@@ -69,10 +70,10 @@ export function calibrate(supports, body, promptTokens) {
     if (ratio < PLAUSIBLE.least || ratio > PLAUSIBLE.most) {
         return charsPerTokenFor(supports, body.model);
     }
-    let models = readings.get(supports);
+    let models = readings().get(supports);
     if (!models) {
         models = new Map();
-        readings.set(supports, models);
+        readings().set(supports, models);
     }
     const known = models.get(body.model) ?? [];
     known.push(ratio);
@@ -83,5 +84,5 @@ export function calibrate(supports, body, promptTokens) {
 }
 /** Forgets every reading, so the next request is sized at `CHARS_PER_TOKEN` again. */
 export function resetCalibration() {
-    readings = new WeakMap();
+    readings.reset();
 }

package/dist/capabilities.js

@@ -1,6 +1,7 @@
 import { endpointId } from "./client.js";
 import { errorMessage } from "./errors.js";
 import { isGrammarError } from "./schema-compat.js";
+import { scoped } from "./scope.js";
 /**
  * What each endpoint cannot do, remembered for the life of the process.
  *
@@ -19,7 +20,7 @@ import { isGrammarError } from "./schema-compat.js";
  * behind it, including the `models` map underneath, which is the level where two keys through
  * one host are most likely to differ at all.
  */
-const capabilities = new Map();
+const capabilities = scoped(() => new Map());
 /**
  * What this endpoint is known not to support. The same object every time, so what `negotiate`
  * latches off stays off.
@@ -64,16 +65,16 @@ export function modelCapabilitiesFor(supports, model) {
     return known;
 }
 /** Every endpoint's capabilities by `endpointId`, the live objects, for `exportCapabilities`. */
-export const knownCapabilities = () => capabilities;
+export const knownCapabilities = () => capabilities();
 /**
  * The capabilities of the endpoint with this `endpointId`, created optimistic if unseen — the way
  * `importCapabilities` reaches an endpoint it has only a digest for.
  */
 export function capabilitiesById(id) {
-    let known = capabilities.get(id);
+    let known = capabilities().get(id);
     if (!known) {
         known = { strictSchemas: true, usageInStream: true, models: new Map(), since: Date.now() };
-        capabilities.set(id, known);
+        capabilities().set(id, known);
     }
     return known;
 }
@@ -92,11 +93,11 @@ export function capabilitiesById(id) {
  */
 export function resetCapabilities(endpoint) {
     if (!endpoint) {
-        const held = capabilities.size > 0;
-        capabilities.clear();
+        const held = capabilities().size > 0;
+        capabilities().clear();
         return held;
     }
-    return capabilities.delete(endpointId(endpoint));
+    return capabilities().delete(endpointId(endpoint));
 }
 /**
  * Forgets every endpoint whose entry is older than this, so the next request finds out again.
@@ -117,10 +118,10 @@ export function resetCapabilities(endpoint) {
  */
 export function expireCapabilities(maxAgeMs, now = Date.now()) {
     let dropped = 0;
-    for (const [id, known] of capabilities) {
+    for (const [id, known] of capabilities()) {
         if (now - known.since < maxAgeMs)
             continue;
-        capabilities.delete(id);
+        capabilities().delete(id);
         dropped++;
     }
     return dropped;

package/dist/client.d.ts

@@ -27,10 +27,11 @@ export interface ClientPoolOptions {
  * Changes what the client pool is held to, for a process whose endpoints are not shaped like the
  * deployments these defaults were chosen for.
  *
- * Module-level for the same reason `configureEvents` is: the pool is one module-level thing, and
- * its size is a deployment's setting, said once at startup. A multi-tenant host keying on a key
- * per user raises `maxClients` so its tenants stop evicting each other's connection pools; a dev
- * box shortens `listingMissMs` so a model it has just pulled is seen sooner.
+ * One setting per runtime for the same reason `configureEvents` is: the pool is one thing per
+ * runtime, and its size is a deployment's setting, said once at startup. A multi-tenant host
+ * keying on a key per user raises `maxClients` so its tenants stop evicting each other's
+ * connection pools, or gives each tenant a `createRuntime` of its own; a dev box shortens
+ * `listingMissMs` so a model it has just pulled is seen sooner.
  *
  * A `maxClients` below the pool's current size evicts down to it at once, least recently asked
  * for first, as the next `getClient` would have. A shorter `listingMissMs` applies to misses

package/dist/client.js

@@ -1,5 +1,6 @@
 import { createHash } from "node:crypto";
 import OpenAI from "openai";
+import { scoped } from "./scope.js";
 /**
  * The SDK insists on a non-empty key even where the server will not look at it. This is what it
  * gets. It also reads as "this endpoint has no key" at a call site, which an empty string does
@@ -31,7 +32,7 @@ export const timeoutMs = (config) => {
  * Bounded, because a cache is only a cache while its keys are. Insertion order is the whole of
  * the eviction: a `Map` iterates oldest-first, and `getClient` re-inserts on a hit.
  */
-const clients = new Map();
+const clients = scoped(() => new Map());
 /**
  * How many endpoints' clients are kept at once, until `configureClients` moves it.
  *
@@ -55,7 +56,7 @@ const CLIENT_DEFAULTS = {
     listingMissMs: LISTING_MISS_MS,
 };
 /** What is in force now. Read where it is used, so a change applies from the next call. */
-let poolLimits = { ...CLIENT_DEFAULTS };
+const poolLimits = scoped(() => ({ ...CLIENT_DEFAULTS }));
 /**
  * Drops the least recently asked-for clients until the pool fits. Nothing is closed on the way
  * out: the SDK holds no handle a caller can release, and an evicted client is garbage once
@@ -63,20 +64,21 @@ let poolLimits = { ...CLIENT_DEFAULTS };
  * of the eviction.
  */
 const evict = () => {
-    for (const oldest of clients.keys()) {
-        if (clients.size <= poolLimits.maxClients)
+    for (const oldest of clients().keys()) {
+        if (clients().size <= poolLimits().maxClients)
             break;
-        clients.delete(oldest);
+        clients().delete(oldest);
     }
 };
 /**
  * Changes what the client pool is held to, for a process whose endpoints are not shaped like the
  * deployments these defaults were chosen for.
  *
- * Module-level for the same reason `configureEvents` is: the pool is one module-level thing, and
- * its size is a deployment's setting, said once at startup. A multi-tenant host keying on a key
- * per user raises `maxClients` so its tenants stop evicting each other's connection pools; a dev
- * box shortens `listingMissMs` so a model it has just pulled is seen sooner.
+ * One setting per runtime for the same reason `configureEvents` is: the pool is one thing per
+ * runtime, and its size is a deployment's setting, said once at startup. A multi-tenant host
+ * keying on a key per user raises `maxClients` so its tenants stop evicting each other's
+ * connection pools, or gives each tenant a `createRuntime` of its own; a dev box shortens
+ * `listingMissMs` so a model it has just pulled is seen sooner.
  *
  * A `maxClients` below the pool's current size evicts down to it at once, least recently asked
  * for first, as the next `getClient` would have. A shorter `listingMissMs` applies to misses
@@ -91,11 +93,11 @@ const evict = () => {
 export function configureClients(options = {}) {
     for (const [name, value] of Object.entries(options)) {
         if (typeof value === "number" && value > 0) {
-            poolLimits[name] = value;
+            poolLimits()[name] = value;
         }
     }
     evict();
-    return { ...poolLimits };
+    return { ...poolLimits() };
 }
 /** How many idle windows the first chunk gets when `firstTokenSeconds` is not given. */
 export const FIRST_TOKEN_FACTOR = 5;
@@ -132,16 +134,16 @@ export function getClient(config) {
     // Stringified rather than joined on a separator: no character is impossible in a URL or a
     // key, and two different endpoints must never resolve to the same cached client.
     const key = JSON.stringify([config.baseUrl, apiKey, timeout]);
-    const existing = clients.get(key);
+    const existing = clients().get(key);
     if (existing) {
         // Re-inserted so it counts as the youngest. A `Map` keeps insertion order and hands the
         // oldest key over first, which is the whole of the LRU below.
-        clients.delete(key);
-        clients.set(key, existing);
+        clients().delete(key);
+        clients().set(key, existing);
         return existing;
     }
     const client = new OpenAI({ baseURL: config.baseUrl, apiKey, timeout, maxRetries: 0 });
-    clients.set(key, client);
+    clients().set(key, client);
     evict();
     return client;
 }
@@ -197,14 +199,14 @@ function contextLengthOf(model) {
  * It is only ever a cache of something asked for anyway, and a listing that fails leaves
  * whatever was there rather than emptying it.
  */
-const listings = new Map();
+const listings = scoped(() => new Map());
 /**
  * The models in each endpoint's last listing whose window is only the trained one, by id.
  *
  * Kept beside `listings` rather than on `ModelInfo`, so what `listModels` hands back is the shape
  * it has always been. Replaced with each listing that lands, and left alone by one that fails.
  */
-const trainedOnly = new Map();
+const trainedOnly = scoped(() => new Map());
 /**
  * When an endpoint was last asked about a model it did not name, keyed on the two together.
  *
@@ -219,7 +221,7 @@ const trainedOnly = new Map();
  * picked up within the minute instead of at the next restart. Bounded by the (endpoint, model)
  * pairs actually asked about, which is the bound the listings themselves have.
  */
-const misses = new Map();
+const misses = scoped(() => new Map());
 /**
  * What counts as one endpoint, everywhere in this package that has to remember something about
  * one — the model listings, `capabilities`, and the no-thinking hints in `side-task`.
@@ -257,9 +259,9 @@ export const endpointId = (config) => createHash("sha256").update(endpointKey(co
  * A model's entry stays until `resetClients`, like a listing; a server that answered without one
  * is asked again after `listingMissMs`, like a listing that did not name the model.
  */
-const served = new Map();
+const served = scoped(() => new Map());
 /** Endpoints with neither native route, which are not asked for either again. */
-const unserved = new Set();
+const unserved = scoped(() => new Set());
 /**
  * Endpoints whose `/api/v0/models` answered with an error, and when.
  *
@@ -271,11 +273,11 @@ const unserved = new Set();
  * Per endpoint and on its own, whatever `/props` said: a server whose `/props` answers without a
  * window is never `unserved`, and without this its missing route was asked again with every miss.
  */
-const refusals = new Map();
+const refusals = scoped(() => new Map());
 /** Whether `/api/v0/models` is to be left alone on this endpoint for now. */
 const refused = (endpoint) => {
-    const refusal = refusals.get(endpoint);
-    return (refusal !== undefined && (refusal.gone || Date.now() - refusal.at < poolLimits.listingMissMs));
+    const refusal = refusals().get(endpoint);
+    return (refusal !== undefined && (refusal.gone || Date.now() - refusal.at < poolLimits().listingMissMs));
 };
 /** How long a probe may take before the window is taken from the listing instead. */
 const PROBE_TIMEOUT_MS = 10_000;
@@ -324,11 +326,11 @@ async function probe(config, path) {
  */
 export async function servedWindow(config) {
     const endpoint = endpointKey(config);
-    if (unserved.has(endpoint))
+    if (unserved().has(endpoint))
         return 0;
     const key = JSON.stringify([endpoint, config.model]);
-    const known = served.get(key);
-    if (known && (known.window > 0 || Date.now() - known.at < poolLimits.listingMissMs))
+    const known = served().get(key);
+    if (known && (known.window > 0 || Date.now() - known.at < poolLimits().listingMissMs))
         return known.window;
     let window = 0;
     try {
@@ -341,17 +343,17 @@ export async function servedWindow(config) {
             if (!refused(endpoint)) {
                 const lmstudio = await probe(config, "/api/v0/models");
                 if (lmstudio.failed)
-                    refusals.set(endpoint, { at: Date.now(), gone: lmstudio.missing });
+                    refusals().set(endpoint, { at: Date.now(), gone: lmstudio.missing });
                 else
-                    refusals.delete(endpoint);
+                    refusals().delete(endpoint);
                 const { data } = (lmstudio.body ?? {});
                 const entry = Array.isArray(data)
                     ? data.find((model) => model?.id === config.model)
                     : undefined;
                 window = positive(entry?.loaded_context_length);
             }
-            if (props.missing && refusals.get(endpoint)?.gone) {
-                unserved.add(endpoint);
+            if (props.missing && refusals().get(endpoint)?.gone) {
+                unserved().add(endpoint);
                 return 0;
             }
         }
@@ -359,7 +361,7 @@ export async function servedWindow(config) {
     catch {
         return 0;
     }
-    served.set(key, { window, at: Date.now() });
+    served().set(key, { window, at: Date.now() });
     return window;
 }
 /**
@@ -379,8 +381,8 @@ export async function listModels(config) {
     })
         .sort((a, b) => a.id.localeCompare(b.id));
     const key = endpointKey(config);
-    listings.set(key, models);
-    trainedOnly.set(key, trained);
+    listings().set(key, models);
+    trainedOnly().set(key, trained);
     return models;
 }
 /**
@@ -411,7 +413,9 @@ export async function contextLimitFor(config, declared = 0) {
     if (declared > 0)
         return declared;
     const key = endpointKey(config);
-    const listed = () => listings.get(key)?.find((model) => model.id === config.model);
+    const listed = () => listings()
+        .get(key)
+        ?.find((model) => model.id === config.model);
     // The listing is asked for again when it does not name this model, rather than only when
     // there is no listing at all. Models arrive after a process starts — an `ollama pull` on a
     // box that has been up a week, a worker added to a router, a name the operator has only just
@@ -421,11 +425,11 @@ export async function contextLimitFor(config, declared = 0) {
     // the window instead of under it and fails against the endpoint's own refusal.
     if (!listed()) {
         const missKey = JSON.stringify([key, config.model]);
-        const asked = misses.get(missKey);
+        const asked = misses().get(missKey);
         // Asked again, but not on every call. A model that is never coming answers the same zero
         // however often the endpoint is asked, and a caller sizing a window per turn pays a round
         // trip for each of them; `listingMissMs` is how long that answer is allowed to stand.
-        if (asked === undefined || Date.now() - asked >= poolLimits.listingMissMs) {
+        if (asked === undefined || Date.now() - asked >= poolLimits().listingMissMs) {
             // A failure is not remembered: an endpoint that was down when the last run started is not
             // an endpoint with no models, and a window nobody could ask about is not a failed run.
             const answered = await listModels(config).then(() => true, () => false);
@@ -435,9 +439,9 @@ export async function contextLimitFor(config, declared = 0) {
             // half a minute.
             if (answered) {
                 if (listed())
-                    misses.delete(missKey);
+                    misses().delete(missKey);
                 else
-                    misses.set(missKey, Date.now());
+                    misses().set(missKey, Date.now());
             }
         }
     }
@@ -445,7 +449,7 @@ export async function contextLimitFor(config, declared = 0) {
     const fromListing = entry?.contextLength ?? 0;
     // A top-level key settles it. A trained window does not: llama.cpp lists the one the model was
     // built with, and taking it over what `/props` says lets an overflow through the guard.
-    if (fromListing > 0 && !trainedOnly.get(key)?.has(config.model))
+    if (fromListing > 0 && !trainedOnly().get(key)?.has(config.model))
         return fromListing;
     const window = await servedWindow(config);
     return window > 0 ? window : fromListing;
@@ -455,12 +459,12 @@ export async function contextLimitFor(config, declared = 0) {
  * tests, and for a settings change under test.
  */
 export function resetClients() {
-    poolLimits = { ...CLIENT_DEFAULTS };
-    clients.clear();
-    listings.clear();
-    misses.clear();
-    trainedOnly.clear();
-    served.clear();
-    unserved.clear();
-    refusals.clear();
+    poolLimits.reset();
+    clients().clear();
+    listings().clear();
+    misses().clear();
+    trainedOnly().clear();
+    served().clear();
+    unserved().clear();
+    refusals().clear();
 }

package/dist/events.d.ts

@@ -50,8 +50,10 @@ export interface EventBusOptions {
  * Changes what the bus keeps, for a process whose runs are not shaped like the ones these
  * defaults were chosen for.
  *
- * The bus is one module-level thing rather than an object a caller holds, so this is too: it is
+ * The bus is one thing per runtime rather than an object a caller holds, so this is too: it is
  * a deployment's setting, said once at startup, and not something to move around under a run.
+ * Called at the top level that is the process's bus; a runtime made by `createRuntime` has its
+ * own, set through its options or its own `configureEvents`.
  * What it costs is memory against how much of a run a late or slow watcher can still read —
  * a server with hundreds of concurrent runs wants a smaller backlog, and one whose tool calls
  * take an hour wants a longer `retainUnendedMs` than the thirty minutes assumed here.

package/dist/events.js

@@ -1,3 +1,4 @@
+import { scoped } from "./scope.js";
 import { LOAD_TOOLS } from "./tool-loading.js";
 /** The numbers a run of the shape this bus was written for wants. */
 const DEFAULTS = {
@@ -6,14 +7,15 @@ const DEFAULTS = {
     retainMs: 60_000,
     retainUnendedMs: 30 * 60_000,
 };
-/** What is in force now. Read where it is used, so a change applies from the next event. */
-let limits = { ...DEFAULTS };
+const bus = scoped(() => ({ limits: { ...DEFAULTS }, streams: new Map(), sweeping: null }));
 /**
  * Changes what the bus keeps, for a process whose runs are not shaped like the ones these
  * defaults were chosen for.
  *
- * The bus is one module-level thing rather than an object a caller holds, so this is too: it is
+ * The bus is one thing per runtime rather than an object a caller holds, so this is too: it is
  * a deployment's setting, said once at startup, and not something to move around under a run.
+ * Called at the top level that is the process's bus; a runtime made by `createRuntime` has its
+ * own, set through its options or its own `configureEvents`.
  * What it costs is memory against how much of a run a late or slow watcher can still read —
  * a server with hundreds of concurrent runs wants a smaller backlog, and one whose tool calls
  * take an hour wants a longer `retainUnendedMs` than the thirty minutes assumed here.
@@ -27,14 +29,14 @@ let limits = { ...DEFAULTS };
  * @returns Everything in force afterwards, including what this call did not change.
  */
 export function configureEvents(options = {}) {
+    const { limits } = bus();
     for (const [name, value] of Object.entries(options)) {
         if (typeof value === "number" && value > 0)
             limits[name] = value;
     }
     return { ...limits };
 }
-const streams = new Map();
-const streamFor = (runId) => {
+const streamFor = ({ streams }, runId) => {
     const existing = streams.get(runId);
     if (existing)
         return existing;
@@ -60,9 +62,9 @@ const streamFor = (runId) => {
  *
  * One timer for the whole map, rescheduled only while there is something in it to expire.
  */
-let sweeping = null;
-function sweep() {
-    sweeping = null;
+function sweep(held) {
+    const { limits, streams } = held;
+    held.sweeping = null;
     const now = Date.now();
     for (const [runId, stream] of streams) {
         if (stream.listeners.size)
@@ -70,13 +72,13 @@ function sweep() {
         if (stream.touched <= now - (stream.ended ? limits.retainMs : limits.retainUnendedMs))
             streams.delete(runId);
     }
-    scheduleSweep();
+    scheduleSweep(held);
 }
-function scheduleSweep() {
-    if (sweeping || streams.size === 0)
+function scheduleSweep(held) {
+    if (held.sweeping || held.streams.size === 0)
         return;
-    sweeping = setTimeout(sweep, limits.retainMs);
-    sweeping.unref?.();
+    held.sweeping = setTimeout(() => sweep(held), held.limits.retainMs);
+    held.sweeping.unref?.();
 }
 /**
  * Forgets a run that will not be emitting `done` — one whose process is tearing down, or whose
@@ -86,6 +88,7 @@ function scheduleSweep() {
  * @param runId The run to forget. An id nothing was emitted under is ignored.
  */
 export function endRun(runId) {
+    const { streams } = bus();
     const stream = streams.get(runId);
     if (!stream)
         return;
@@ -106,7 +109,9 @@ export function endRun(runId) {
  * @param input The event. `kind` is required; `runId` and `seq` are not a caller's to set.
  */
 export function emit(runId, input) {
-    const stream = streamFor(runId);
+    const held = bus();
+    const { limits } = held;
+    const stream = streamFor(held, runId);
     // One clock read, used for both the event and the sweep's bookkeeping.
     const at = Date.now();
     const event = {
@@ -130,7 +135,7 @@ export function emit(runId, input) {
     // then dropped: a finished run's record is the row, not this.
     if (event.kind === "done")
         stream.ended = true;
-    scheduleSweep();
+    scheduleSweep(held);
     for (const listener of stream.listeners) {
         // The guarantee above is the point of this: a listener that throws must not take out the
         // emitter, the listeners after it, or the bookkeeping already done above.
@@ -155,8 +160,14 @@ export function emit(runId, input) {
  * function it is suspended at an `await` rather than at a `yield`, and a `return()` there is
  * queued behind a promise only the next event can settle. An abort resolves that promise itself.
  */
-export async function* watch(runId, signal) {
-    const stream = streamFor(runId);
+export function watch(runId, signal) {
+    // Read here, not in the generator: its body runs when the first event is asked for, and by
+    // then the caller may be outside the runtime whose run this is.
+    return watching(bus(), runId, signal);
+}
+async function* watching(held, runId, signal) {
+    const { limits, streams } = held;
+    const stream = streamFor(held, runId);
     // A cursor rather than `shift()`. Draining a backlog an event at a time off the front of an
     // array is a copy of the whole array per event, which on the ten-thousand-delta run this bus
     // is built for is the one quadratic left in the file. The prefix behind the cursor is dropped
@@ -272,7 +283,7 @@ export async function* watch(runId, signal) {
  *
  * @param runId The run to read. An unknown or already-swept run gives an empty array.
  */
-export const history = (runId) => [...(streams.get(runId)?.events ?? [])];
+export const history = (runId) => [...(bus().streams.get(runId)?.events ?? [])];
 /**
  * Test seam: forget every run, so one test's events cannot be read by the next.
  *
@@ -286,11 +297,13 @@ export const history = (runId) => [...(streams.get(runId)?.events ?? [])];
  * teardown configures again, which is the same line it already wrote once.
  */
 export const resetEvents = () => {
-    streams.clear();
-    if (sweeping)
-        clearTimeout(sweeping);
-    sweeping = null;
-    limits = { ...DEFAULTS };
+    const held = bus();
+    held.streams.clear();
+    if (held.sweeping)
+        clearTimeout(held.sweeping);
+    held.sweeping = null;
+    // In place, so a watcher still open reads the defaults too and not the caps it began under.
+    Object.assign(held.limits, DEFAULTS);
 };
 /**
  * Consecutive tokens of one kind are one thing being said, not hundreds of things.

package/dist/hooks.d.ts

@@ -160,8 +160,8 @@ export interface HookOptions {
  * Changes what hooks are held to, for a process whose windows are not the size these defaults
  * were chosen for, or whose host wants its own name above the context.
  *
- * Module-level for the same reason `configureEvents` is: a budget and a preface are a deployment's
- * settings, said once at startup. A caller that sizes the budget per model or per agent — a 128k
+ * One setting per runtime for the same reason `configureEvents` is: a budget and a preface are a
+ * deployment's settings, said once at startup. A caller that sizes the budget per model or per agent — a 128k
  * window can afford more recall than an 8k one — passes `maxTokens` to `gather` instead, and a
  * `preface` passed to `withContext` or `runAgentLoop`'s hooks wins over this one the same way.
  *

package/dist/hooks.js

@@ -1,5 +1,6 @@
 import { createHash } from "node:crypto";
 import { errorMessage } from "./errors.js";
+import { scoped } from "./scope.js";
 import { estimateTokens } from "./tokens.js";
 /** Every event a hook can be bound to, in the order a session meets them. */
 export const HOOK_EVENTS = [
@@ -36,13 +37,13 @@ const HOOK_DEFAULTS = {
     preface: HOOK_PREFACE,
 };
 /** What is in force now. Read where it is used, so a change applies from the next request. */
-let hookSettings = { ...HOOK_DEFAULTS };
+const hookSettings = scoped(() => ({ ...HOOK_DEFAULTS }));
 /**
  * Changes what hooks are held to, for a process whose windows are not the size these defaults
  * were chosen for, or whose host wants its own name above the context.
  *
- * Module-level for the same reason `configureEvents` is: a budget and a preface are a deployment's
- * settings, said once at startup. A caller that sizes the budget per model or per agent — a 128k
+ * One setting per runtime for the same reason `configureEvents` is: a budget and a preface are a
+ * deployment's settings, said once at startup. A caller that sizes the budget per model or per agent — a 128k
  * window can afford more recall than an 8k one — passes `maxTokens` to `gather` instead, and a
  * `preface` passed to `withContext` or `runAgentLoop`'s hooks wins over this one the same way.
  *
@@ -55,11 +56,11 @@ let hookSettings = { ...HOOK_DEFAULTS };
 export function configureHooks(options = {}) {
     const { contextTokens, preface } = options;
     if (typeof contextTokens === "number" && contextTokens > 0) {
-        hookSettings.contextTokens = contextTokens;
+        hookSettings().contextTokens = contextTokens;
     }
     if (typeof preface === "string")
-        hookSettings.preface = preface;
-    return { ...hookSettings };
+        hookSettings().preface = preface;
+    return { ...hookSettings() };
 }
 /**
  * Test seam: puts `configureHooks` back to the defaults, so one test's budget or preface is not the
@@ -67,14 +68,14 @@ export function configureHooks(options = {}) {
  * `resetAll` calls it.
  */
 export const resetHooks = () => {
-    hookSettings = { ...HOOK_DEFAULTS };
+    hookSettings.reset();
 };
 /**
  * The budget a call is held to: its own when it gave a usable one, the process's otherwise. The
  * same rule `configureHooks` applies, so a `0` threaded through for "no opinion" does not quietly
  * turn every hook's context off.
  */
-const budget = (given) => typeof given === "number" && given > 0 ? given : hookSettings.contextTokens;
+const budget = (given) => typeof given === "number" && given > 0 ? given : hookSettings().contextTokens;
 const attribute = (text) => text.replaceAll("&", "&amp;").replaceAll('"', "&quot;").replaceAll("<", "&lt;");
 /**
  * Builds the context a set of outcomes adds and the notes that go with it.
@@ -143,7 +144,7 @@ export function assembleContext(outcomes, maxTokens) {
  * unless something moved it. Empty says nothing, rather than leaving a blank line where it was.
  * @returns `history` when there was nothing to add or nowhere to add it, otherwise a new array.
  */
-export function withContext(history, index, context, preface = hookSettings.preface) {
+export function withContext(history, index, context, preface = hookSettings().preface) {
     const message = history[index];
     if (!context || message?.role !== "user")
         return history;

package/dist/index.d.ts

@@ -24,6 +24,7 @@ export { estimateFrom, type LedgerEntry, type LedgerEstimateOptions, type Ledger
 export { resetAll } from "./reset.ts";
 export { backoffMs, CHARS_PER_TOKEN, type ContextBreakdown, type ContextBreakdownOptions, ContextOverflow, compact, contextChars, contextTokens, EndpointSilent, isModelLoading, isOverflow, isTransient, LOADING_POLL_MS, LOADING_TIMEOUT_MS, messageTokens, requestChars, requestTokens, SMALLEST_LIKELY_WINDOW, sleep, type TokenEstimateOptions, toolsChars, } from "./retry.ts";
 export { type RunTurnOptions, runTurn } from "./run-turn.ts";
+export { createRuntime, defaultRuntime, type Runtime, type RuntimeOptions } from "./runtime.ts";
 export { isGrammarError, relaxTools, sanitizeTools } from "./schema-compat.ts";
 export { type AskJsonOptions, ask, askJson, clean, listLines, parseJson, resetHints, type SideTask, type SideTaskInput, type SideTaskOptions, tryAsk, } from "./side-task.ts";
 export { CAPABILITY_SNAPSHOT_VERSION, type CapabilitySnapshot, type EndpointSnapshot, exportCapabilities, importCapabilities, type ModelSnapshot, } from "./snapshot.ts";

package/dist/index.js

@@ -22,6 +22,7 @@ export { estimateFrom, rebaseLedger, recordRequest, tokensBetween, } from "./led
 export { resetAll } from "./reset.js";
 export { backoffMs, CHARS_PER_TOKEN, ContextOverflow, compact, contextChars, contextTokens, EndpointSilent, isModelLoading, isOverflow, isTransient, LOADING_POLL_MS, LOADING_TIMEOUT_MS, messageTokens, requestChars, requestTokens, SMALLEST_LIKELY_WINDOW, sleep, toolsChars, } from "./retry.js";
 export { runTurn } from "./run-turn.js";
+export { createRuntime, defaultRuntime } from "./runtime.js";
 export { isGrammarError, relaxTools, sanitizeTools } from "./schema-compat.js";
 export { ask, askJson, clean, listLines, parseJson, resetHints, tryAsk, } from "./side-task.js";
 export { CAPABILITY_SNAPSHOT_VERSION, exportCapabilities, importCapabilities, } from "./snapshot.js";

package/dist/reset.d.ts

@@ -1,8 +1,9 @@
 /**
  * Forgets everything this package remembers between calls.
  *
- * Six modules here keep state for the life of the process, each for a good reason and each
- * with its own seam: the pooled clients and their model listings, the endpoints that turned
+ * Six modules here keep state for the life of the runtime — the process's, unless this is called
+ * as a method of one `createRuntime` made, which clears that one alone — each for a good reason
+ * and each with its own seam: the pooled clients and their model listings, the endpoints that turned
  * out not to take `stream_options` or a grammar, each model's measured characters per token,
  * the models that refused the no-thinking hints, the event bus, and the hooks' configured
  * budget and preface. `resetClients`, `resetCapabilities`, `resetCalibration`, `resetHints`,
@@ -18,5 +19,8 @@
  *
  * It is also the seam that does not need finding again. A seventh module with a cache is a seventh
  * line here, rather than an edit to the teardown of three consumers who will not all notice.
+ *
+ * A test that builds a `createRuntime` per case has nothing to tear down, and is the better
+ * answer where cases run in parallel: this clears state every concurrent caller is sharing.
  */
 export declare function resetAll(): void;

package/dist/reset.js

@@ -7,8 +7,9 @@ import { resetHints } from "./side-task.js";
 /**
  * Forgets everything this package remembers between calls.
  *
- * Six modules here keep state for the life of the process, each for a good reason and each
- * with its own seam: the pooled clients and their model listings, the endpoints that turned
+ * Six modules here keep state for the life of the runtime — the process's, unless this is called
+ * as a method of one `createRuntime` made, which clears that one alone — each for a good reason
+ * and each with its own seam: the pooled clients and their model listings, the endpoints that turned
  * out not to take `stream_options` or a grammar, each model's measured characters per token,
  * the models that refused the no-thinking hints, the event bus, and the hooks' configured
  * budget and preface. `resetClients`, `resetCapabilities`, `resetCalibration`, `resetHints`,
@@ -24,6 +25,9 @@ import { resetHints } from "./side-task.js";
  *
  * It is also the seam that does not need finding again. A seventh module with a cache is a seventh
  * line here, rather than an edit to the teardown of three consumers who will not all notice.
+ *
+ * A test that builds a `createRuntime` per case has nothing to tear down, and is the better
+ * answer where cases run in parallel: this clears state every concurrent caller is sharing.
  */
 export function resetAll() {
     resetClients();

package/dist/runtime.d.ts

@@ -0,0 +1,120 @@
+import { preselect, runAgentLoop } from "./agent-loop.ts";
+import { calibrate, charsPerTokenFor, resetCalibration } from "./calibration.ts";
+import { capabilitiesFor, expireCapabilities, resetCapabilities } from "./capabilities.ts";
+import { type ClientPoolOptions, configureClients, contextLimitFor, getClient, listModels, resetClients, servedWindow } from "./client.ts";
+import { compactTranscript, runCompaction, summariser } from "./compaction.ts";
+import { continueTurn } from "./continuation.ts";
+import { configureEvents, type EventBusOptions, emit, endRun, history, resetEvents, watch } from "./events.ts";
+import { assembleContext, configureHooks, consult, gather, type HookOptions, notify, resetHooks, withContext } from "./hooks.ts";
+import { resetAll } from "./reset.ts";
+import { runTurn } from "./run-turn.ts";
+import { ask, askJson, resetHints, tryAsk } from "./side-task.ts";
+import { exportCapabilities, importCapabilities } from "./snapshot.ts";
+/**
+ * Every exported function that reads or writes what a runtime remembers, itself or through what
+ * it calls. The rest of the package is pure, and is the same function whichever runtime it is
+ * called beside.
+ */
+declare const STATEFUL: {
+    runAgentLoop: typeof runAgentLoop;
+    preselect: typeof preselect;
+    runTurn: typeof runTurn;
+    continueTurn: typeof continueTurn;
+    ask: typeof ask;
+    askJson: typeof askJson;
+    tryAsk: typeof tryAsk;
+    summariser: typeof summariser;
+    runCompaction: typeof runCompaction;
+    compactTranscript: typeof compactTranscript;
+    getClient: typeof getClient;
+    listModels: typeof listModels;
+    contextLimitFor: typeof contextLimitFor;
+    servedWindow: typeof servedWindow;
+    configureClients: typeof configureClients;
+    capabilitiesFor: typeof capabilitiesFor;
+    expireCapabilities: typeof expireCapabilities;
+    exportCapabilities: typeof exportCapabilities;
+    importCapabilities: typeof importCapabilities;
+    calibrate: typeof calibrate;
+    charsPerTokenFor: typeof charsPerTokenFor;
+    emit: typeof emit;
+    endRun: typeof endRun;
+    history: typeof history;
+    watch: typeof watch;
+    configureEvents: typeof configureEvents;
+    configureHooks: typeof configureHooks;
+    gather: typeof gather;
+    notify: typeof notify;
+    consult: typeof consult;
+    assembleContext: typeof assembleContext;
+    withContext: typeof withContext;
+    resetClients: typeof resetClients;
+    resetCapabilities: typeof resetCapabilities;
+    resetCalibration: typeof resetCalibration;
+    resetHints: typeof resetHints;
+    resetEvents: typeof resetEvents;
+    resetHooks: typeof resetHooks;
+    resetAll: typeof resetAll;
+};
+/** What a runtime starts out configured with. Every field optional; see `createRuntime`. */
+export interface RuntimeOptions {
+    /** The client pool's bounds, as `configureClients` takes them. */
+    clients?: ClientPoolOptions;
+    /** The event bus's bounds, as `configureEvents` takes them. */
+    events?: EventBusOptions;
+    /** The hooks' budget and preface, as `configureHooks` takes them. */
+    hooks?: HookOptions;
+}
+/**
+ * One set of the package's caches, with the functions that use them as methods.
+ *
+ * Each method is the top-level function of the same name, with the same signature and doc
+ * comment, run against this runtime's state instead of the process's: its pooled clients and
+ * model listings, latched capabilities, measured characters per token, no-thinking hints, event
+ * bus, and hook settings. `resetAll` and the six narrower resets clear this runtime alone.
+ */
+export type Runtime = typeof STATEFUL & {
+    /**
+     * Runs `fn` with this runtime current, so every top-level function it calls — directly, after
+     * an `await`, or from a timer it sets — uses this runtime's state. What the methods do, for
+     * code that calls the top-level functions itself.
+     *
+     * It does not reach a callback that outlives `fn` and is called from elsewhere — a generator
+     * resumed later, a function handed back and called by someone else. Call a method there.
+     *
+     * @param fn What to run. Its return value, or what it throws, is handed straight back.
+     */
+    run<T>(fn: () => T): T;
+};
+/**
+ * A runtime of its own: separate clients, listings, capability latches, hints, event bus and hook
+ * settings from the process's and from every other runtime's.
+ *
+ * For a host that must not share them — one that mints a key per tenant, where a shared pool has
+ * tenants evicting each other's clients and a shared bus has one tenant's run id readable by
+ * another — and for tests, where a runtime per case needs no `resetAll` between them and can run
+ * in parallel. A host with one deployment's worth of endpoints does not need one: the top-level
+ * functions are the default runtime's methods.
+ *
+ * The runtime is carried by async context rather than by argument, so `runAgentLoop` called as a
+ * method uses this runtime all the way down, and so does a top-level function the host calls from
+ * inside it — from `dispatch`, a hook, `onEvent`. Outside a method or `run`, a top-level function
+ * is the default runtime's again; a request handler that emits to a run a runtime started calls
+ * `runtime.emit`.
+ *
+ * Nothing needs closing. A runtime is garbage once nothing holds it; its one timer, the event
+ * bus's sweep, is unreferenced and stops rescheduling when its last run has been swept.
+ *
+ * @param options What to configure it with before first use. A part left out keeps the
+ * defaults — not the default runtime's settings, which a new runtime does not inherit.
+ */
+export declare function createRuntime(options?: RuntimeOptions): Runtime;
+/**
+ * The runtime the top-level functions use when no other is running.
+ *
+ * `defaultRuntime.getClient(endpoint)` and `getClient(endpoint)` are the same call outside any
+ * runtime. Inside one they differ: the top-level function follows the runtime that is running,
+ * and this always reaches the process-wide state.
+ */
+export declare const defaultRuntime: Runtime;
+export {};

package/dist/runtime.js

@@ -0,0 +1,113 @@
+import { preselect, runAgentLoop } from "./agent-loop.js";
+import { calibrate, charsPerTokenFor, resetCalibration } from "./calibration.js";
+import { capabilitiesFor, expireCapabilities, resetCapabilities } from "./capabilities.js";
+import { configureClients, contextLimitFor, getClient, listModels, resetClients, servedWindow, } from "./client.js";
+import { compactTranscript, runCompaction, summariser } from "./compaction.js";
+import { continueTurn } from "./continuation.js";
+import { configureEvents, emit, endRun, history, resetEvents, watch, } from "./events.js";
+import { assembleContext, configureHooks, consult, gather, notify, resetHooks, withContext, } from "./hooks.js";
+import { resetAll } from "./reset.js";
+import { runTurn } from "./run-turn.js";
+import { inScope, rootScope } from "./scope.js";
+import { ask, askJson, resetHints, tryAsk } from "./side-task.js";
+import { exportCapabilities, importCapabilities } from "./snapshot.js";
+/**
+ * Every exported function that reads or writes what a runtime remembers, itself or through what
+ * it calls. The rest of the package is pure, and is the same function whichever runtime it is
+ * called beside.
+ */
+const STATEFUL = {
+    runAgentLoop,
+    preselect,
+    runTurn,
+    continueTurn,
+    ask,
+    askJson,
+    tryAsk,
+    summariser,
+    runCompaction,
+    compactTranscript,
+    getClient,
+    listModels,
+    contextLimitFor,
+    servedWindow,
+    configureClients,
+    capabilitiesFor,
+    expireCapabilities,
+    exportCapabilities,
+    importCapabilities,
+    calibrate,
+    charsPerTokenFor,
+    emit,
+    endRun,
+    history,
+    watch,
+    configureEvents,
+    configureHooks,
+    gather,
+    notify,
+    consult,
+    assembleContext,
+    withContext,
+    resetClients,
+    resetCapabilities,
+    resetCalibration,
+    resetHints,
+    resetEvents,
+    resetHooks,
+    resetAll,
+};
+/** Makes the methods of the runtime whose state is `scope`. */
+function bind(scope) {
+    const within = (fn) => (...args) => inScope(scope, () => fn(...args));
+    const methods = Object.fromEntries(Object.entries(STATEFUL).map(([name, fn]) => [
+        name,
+        within(fn),
+    ]));
+    return {
+        ...methods,
+        // What it hands back is called later, by `compactTranscript` or by the host, and asks then.
+        summariser: (...args) => within(summariser(...args)),
+        run: (fn) => inScope(scope, fn),
+    };
+}
+/**
+ * A runtime of its own: separate clients, listings, capability latches, hints, event bus and hook
+ * settings from the process's and from every other runtime's.
+ *
+ * For a host that must not share them — one that mints a key per tenant, where a shared pool has
+ * tenants evicting each other's clients and a shared bus has one tenant's run id readable by
+ * another — and for tests, where a runtime per case needs no `resetAll` between them and can run
+ * in parallel. A host with one deployment's worth of endpoints does not need one: the top-level
+ * functions are the default runtime's methods.
+ *
+ * The runtime is carried by async context rather than by argument, so `runAgentLoop` called as a
+ * method uses this runtime all the way down, and so does a top-level function the host calls from
+ * inside it — from `dispatch`, a hook, `onEvent`. Outside a method or `run`, a top-level function
+ * is the default runtime's again; a request handler that emits to a run a runtime started calls
+ * `runtime.emit`.
+ *
+ * Nothing needs closing. A runtime is garbage once nothing holds it; its one timer, the event
+ * bus's sweep, is unreferenced and stops rescheduling when its last run has been swept.
+ *
+ * @param options What to configure it with before first use. A part left out keeps the
+ * defaults — not the default runtime's settings, which a new runtime does not inherit.
+ */
+export function createRuntime(options = {}) {
+    const runtime = bind(new Map());
+    if (options.clients)
+        runtime.configureClients(options.clients);
+    if (options.events)
+        runtime.configureEvents(options.events);
+    if (options.hooks)
+        runtime.configureHooks(options.hooks);
+    return runtime;
+}
+/**
+ * The runtime the top-level functions use when no other is running.
+ *
+ * `defaultRuntime.getClient(endpoint)` and `getClient(endpoint)` are the same call outside any
+ * runtime. Inside one they differ: the top-level function follows the runtime that is running,
+ * and this always reaches the process-wide state.
+ */
+export const defaultRuntime = bind(rootScope);

package/dist/scope.d.ts

@@ -0,0 +1,35 @@
+/**
+ * Everything one runtime remembers: each module's state, under a key only that module holds.
+ *
+ * A map rather than a record of named caches, so a module that grows a cache declares it where
+ * it is used and `runtime.ts` does not have to hear about it.
+ */
+export type Scope = Map<symbol, unknown>;
+/** The default runtime's scope: what the top-level functions use when no runtime is running. */
+export declare const rootScope: Scope;
+/** The scope of the runtime whose method is running, or the default one outside any. */
+export declare const currentScope: () => Scope;
+/**
+ * Runs `fn` with `scope` as the current one, for it and for everything it awaits or schedules.
+ *
+ * @param scope The runtime's state. Nested calls replace it for their own duration only.
+ * @param fn What to run. Its return value, or what it throws, is handed straight back.
+ */
+export declare const inScope: <T>(scope: Scope, fn: () => T) => T;
+/** One module's state, read from whichever scope is current. See `scoped`. */
+export interface Scoped<T> {
+    /** The current scope's copy, made on first use. */
+    (): T;
+    /** Drops the current scope's copy, so the next read makes a fresh one. */
+    reset(): void;
+}
+/**
+ * Declares a piece of module state that every runtime has its own copy of.
+ *
+ * What used to be `const cache = new Map()` at the top of a module is `scoped(() => new Map())`,
+ * and each use reads `cache()`. The copy is made lazily, so a runtime pays only for the modules
+ * it touches.
+ *
+ * @param create Builds a fresh, empty copy. Called once per scope, and again after `reset`.
+ */
+export declare function scoped<T>(create: () => T): Scoped<T>;

package/dist/scope.js

@@ -0,0 +1,42 @@
+import { AsyncLocalStorage } from "node:async_hooks";
+/**
+ * Which scope the code running now belongs to, carried across every `await` beneath a runtime's
+ * method. Async context rather than a parameter, because the state is read five calls down —
+ * `runAgentLoop` to `runTurn` to `negotiate` to `capabilitiesFor` — and threading it would put a
+ * runtime argument on every exported function in between.
+ */
+const active = new AsyncLocalStorage();
+/** The default runtime's scope: what the top-level functions use when no runtime is running. */
+export const rootScope = new Map();
+/** The scope of the runtime whose method is running, or the default one outside any. */
+export const currentScope = () => active.getStore() ?? rootScope;
+/**
+ * Runs `fn` with `scope` as the current one, for it and for everything it awaits or schedules.
+ *
+ * @param scope The runtime's state. Nested calls replace it for their own duration only.
+ * @param fn What to run. Its return value, or what it throws, is handed straight back.
+ */
+export const inScope = (scope, fn) => active.run(scope, fn);
+/**
+ * Declares a piece of module state that every runtime has its own copy of.
+ *
+ * What used to be `const cache = new Map()` at the top of a module is `scoped(() => new Map())`,
+ * and each use reads `cache()`. The copy is made lazily, so a runtime pays only for the modules
+ * it touches.
+ *
+ * @param create Builds a fresh, empty copy. Called once per scope, and again after `reset`.
+ */
+export function scoped(create) {
+    const key = Symbol();
+    const read = () => {
+        const scope = currentScope();
+        if (!scope.has(key))
+            scope.set(key, create());
+        return scope.get(key);
+    };
+    return Object.assign(read, {
+        reset: () => {
+            currentScope().delete(key);
+        },
+    });
+}

package/dist/side-task.js

@@ -4,6 +4,7 @@ import { endpointId, getClient } from "./client.js";
 import { errorMessage } from "./errors.js";
 import { isTransient } from "./retry.js";
 import { relaxTools, sanitizeTools } from "./schema-compat.js";
+import { scoped } from "./scope.js";
 import { stripThinking } from "./thinking.js";
 /**
  * One-shot calls that support a run without being one: picking tools, naming a session,
@@ -39,13 +40,13 @@ const NO_THINKING = { chat_template_kwargs: { enable_thinking: false } };
  * Only the `chat_template_kwargs` half is here. `reasoning_effort` is `negotiate`'s to latch, on
  * the same (endpoint, model) pair, where a run on that model can read it too.
  */
-const noHints = new Set();
+const noHints = scoped(() => new Set());
 /** An (endpoint, model) pair as `noHints` holds it: `[endpointId, model]`, stringified. */
 export const hintKey = (endpoint, model) => JSON.stringify([endpoint, model]);
 /** The pairs that refused the hints, the live set, for `exportCapabilities` and `importCapabilities`. */
-export const refusedHints = () => noHints;
+export const refusedHints = () => noHints();
 /** Test seam, alongside `resetClients` and `resetAll`: forget which models refused the hints. */
-export const resetHints = () => noHints.clear();
+export const resetHints = () => noHints().clear();
 /**
  * Whether a failure is the server complaining about the request, rather than failing to answer.
  *
@@ -130,7 +131,7 @@ async function complete(config, model, system, user, { maxTokens = 512, temperat
     const key = hintKey(endpointId(config), model);
     // A chosen effort replaces the template hint rather than riding beside it: `enable_thinking:
     // false` next to `reasoning_effort: "high"` is a request that contradicts itself.
-    const hints = !level && !noHints.has(key);
+    const hints = !level && !noHints().has(key);
     let response;
     try {
         response = await attempt(hints, true);
@@ -153,7 +154,7 @@ async function complete(config, model, system, user, { maxTokens = 512, temperat
         // cannot latch is blamed; if it was the effort after all, the next call is left with only
         // the effort to drop and latches that instead.
         if (hints)
-            noHints.add(key);
+            noHints().add(key);
         else
             modelCapabilitiesFor(supports, model).reasoningEffort = false;
     }

package/llms.txt

@@ -222,6 +222,13 @@ One turn, given as many attempts as the caller allows.
 - `RunTurnOptions` (type) — A retry is not the same event as a downgrade, but a watcher wants to be told about both.
 - `runTurn` — `request` is a callback rather than a body because the body has to be rebuilt from whatever the last attempt latched off: the tools it sends depend on `strictSchemas`, and `relaxTools` has to apply to the schemas that were just sanitised.
 
+### runtime
+
+- `createRuntime` — A runtime of its own: separate clients, listings, capability latches, hints, event bus and hook settings from the process's and from every other runtime's.
+- `defaultRuntime` — The runtime the top-level functions use when no other is running.
+- `Runtime` (type) — One set of the package's caches, with the functions that use them as methods.
+- `RuntimeOptions` (type) — What a runtime starts out configured with.
+
 ### schema-compat
 
 JSON Schema compatibility for llama.cpp-backed servers.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cubicecho/agent-core",
-  "version": "2.22.0",
+  "version": "2.23.0",
   "description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
   "keywords": [
     "openai",