@cubicecho/agent-core 4.14.0 → 4.16.0

package/README.md

@@ -41,7 +41,7 @@ only, Node >=22.
 | `run-calls` | The tools between two turns: a step's calls read, run and written into the transcript, with every call answered whatever happens. `preview` cuts a long argument or result for a watcher. |
 | `preselect` | Choosing a run's tools before it starts: `preselect` asks a small model, `preselectByKeywords` scores the request's own words against the tool names without one, and `preselectSystem`, `preselectInput` and `preselection` are the prompt and the reading of the reply for a host that makes the call itself. |
 | `tool-calls` | Reading what a model meant by a tool call it did not write cleanly: `parseToolArguments` repairs almost-JSON arguments and says when they were cut off, `recoverToolCalls` finds calls written into the reply as text. Plus the shapes a call is handed round in: `ToolCallRequest`, `ToolCallResult`, `ToolCallOutcome`. |
-| `compaction` | Keeping a long run inside its window: `pruneToolResults` clears stale tool results, `planCompaction` and `compactTranscript` fold the oldest stretch into a summary. |
+| `compaction` | Keeping a long run inside its window: `pruneToolResults` clears stale tool results, `planPrune` moves a boundary for them that keeps the prompt cache, `planCompaction` and `compactTranscript` fold the oldest stretch into a summary. |
 | `ledger` | What stretches of a transcript cost, read off the prompt counts the server reported rather than estimated: `recordRequest`, `tokensBetween`, `estimateFrom`, `rebaseLedger`. |
 | `snapshot` | `exportCapabilities` and `importCapabilities`: the latched refusals as a JSON blob a consumer stores, so a restart need not learn them again. |
 | `spec` | `parseSpec`, `resolveAgentSpec` and `exportSpec`: an agent as a versioned JSON document, layered into the flat config the loop takes. Imports nothing but types, and is published separately at `@cubicecho/agent-core/spec`. `defineSpecColumns` maps a stored row onto a layer and back, at `@cubicecho/agent-core/spec/columns`. |
@@ -887,9 +887,29 @@ turn is remembered, awaits it once the answer is out.
 The loop gathers before its first request, beside its own preselection when `preselect` asked for
 one, and nothing else of the host's can run beside those. A host with pre-turn work to overlap with
 the hooks — a compaction — calls `gather` itself, applies `withContext` to the current question as
-well, calls `notify` after, and passes no `hooks` (see [Hooks](#hooks)). Such a host passes the
-question as it was typed as `preselectQuery` too, or the preselector is shown the context in front
-of it and picks tools for what was recalled rather than what was asked.
+well, and calls `notify` after (see [Hooks](#hooks)). Such a host passes the question as it was
+typed as `preselectQuery` too, or the preselector is shown the context in front of it and picks
+tools for what was recalled rather than what was asked.
+
+`hooks.fire` says which events the loop fires itself, and it fires no others: `sessionStart` and
+`beforeTurn` gathered before the first request, `beforeCompact` beside each fold `compact` makes,
+`afterTurn` once answered. Left out, it is all of them — `events` (`beforeTurn` by default), then
+`afterTurn` and `beforeCompact` — which is what the loop always did. A host that fires the turn's
+events around a step of its own still hands the loop its hooks for the fold:
+
+```ts
+await runAgentLoop({
+  ...options,
+  compact: true,
+  hooks: { run, context, fire: [HookEvent.BeforeCompact] }, // beforeTurn and afterTurn are the host's
+});
+```
+
+An event left out of `fire` is one the result has nothing for: `afterTurn` is resolved and empty
+without `afterTurn`, and `notes` and `context` are empty without an injecting event, since the
+host's own `gather` and `notify` hold those. `sessionEnd` and `sessionDelete` have no point in a
+loop and are ignored there. `events` is deprecated: it is the injecting half of `fire`, and is
+ignored once `fire` is given.
 
 A host that stores its transcript — a row per message, so a crash or a stop mid-run still leaves
 readable history — gets each message as the loop appends it, and the run as it stood when the loop
@@ -1047,6 +1067,38 @@ way to save tokens.
 `pruneToolResults` keeps the transcript's indexes, so a plan made before pruning still applies to
 what it returns, as above.
 
+### A prune boundary that keeps the cache
+
+`pruneToolResults` keeps a sliding window: applied on every step, each new result pushes an older
+one out and rewrites a message in the middle of the history, so the cache misses from there once
+per tool call. `planPrune` plans a boundary instead. The host stores its `through`, sends
+`pruneThrough(messages, through)` on every request, and between moves each request is the one
+before it with only its tail added.
+
+```ts
+const plan = planPrune(messages, config.contextLength ?? 0, { through: stored, used: lastPromptTokens });
+if (plan) {
+  stored = plan.through;
+  notice(pruneNotice(plan)); // cleared 4 old tool results (~9,120 tokens) as the window filled
+}
+const sent = pruneThrough(messages, stored);
+```
+
+A plan never moves the boundary backwards and never moves it for nothing. It moves to the earliest
+of the latest `keepLast` (5) results once the window is past `at` (`COMPACT_AT`, the fold's own
+line) — by the larger of `used` and the estimate, since a report does not count the results that
+came after it — and only when what it clears is at least `minShare` (`PRUNE_SHARE`, a tenth) of
+the window. `compacted: true`, beside a fold that has rewritten the head anyway, skips both gates:
+the cache is already lost, so the move is free. `from` leaves out results a fold has already taken
+out of the request. `at: 0` with a larger `minShare` is a pure hysteresis on what is won back.
+
+`runAgentLoop({ prune: true })` does this itself: after `beforeStep`, ahead of `compact`, which
+then plans from what is left — so a prune that brings the run back under the line spares it the
+summary — and again with `compacted` after a fold. Each move is a `notice` with the counts. Its
+stubs stay in the loop's transcript and in the result's `messages`, as a fold's summary does; a
+host that keeps the transcript whole calls `planPrune` in its own `beforeStep` and stores the
+boundary. An object moves `at`, `minShare`, `keepLast` and `maxChars`.
+
 ### What the messages cost, measured
 
 `planCompaction` decides where to cut by estimating each message, and a calibrated divisor fixes

package/dist/context/compaction.d.ts

@@ -63,6 +63,139 @@ export interface PruneOptions {
  * see the module comment on when to run it.
  */
 export declare function pruneToolResults(messages: Message[], { keepLast, maxChars }?: PruneOptions): Message[];
+/**
+ * The share of the window a prune has to win back on its own before it is worth the prompt cache
+ * it costs. Below it the request after a prune is barely smaller than the one before, and every
+ * token of it is processed again anyway.
+ */
+export declare const PRUNE_SHARE = 0.1;
+/** What `planPrune` takes beside the transcript and the window. */
+export interface PrunePlanOptions extends PruneOptions {
+    /**
+     * The boundary in force: results before it are already sent as stubs. A plan only ever moves it
+     * forward.
+     *
+     * @defaultValue `0`
+     */
+    through?: number;
+    /**
+     * The share of the window a move has to clear, in tokens, to be made on its own. Zero makes any
+     * move that clears something.
+     *
+     * @defaultValue `PRUNE_SHARE`
+     */
+    minShare?: number;
+    /**
+     * The share of the window in use before a move is considered at all. Zero considers one on every
+     * step, and the hysteresis is `minShare`'s alone.
+     *
+     * @defaultValue `COMPACT_AT`, so a prune is paid for when a fold would be
+     */
+    at?: number;
+    /**
+     * What the transcript costs now — the last request's reported prompt tokens. The fill weighed
+     * against `at` is the larger of it and the estimate, since a report does not count the results
+     * that arrived after it, which are what fills a window.
+     */
+    used?: number;
+    /**
+     * The first message still sent. Results before it were folded into a summary and are no saving
+     * of this one, so they are not counted.
+     *
+     * @defaultValue `0`
+     */
+    from?: number;
+    /**
+     * The head of the request was rewritten on this step anyway — a fold was just stored — so the
+     * cache is lost whatever a move does: `at` and `minShare` are not asked, and a move that clears
+     * anything is made, with a window known or not.
+     *
+     * @defaultValue `false`
+     */
+    compacted?: boolean;
+    /**
+     * One message's tokens.
+     *
+     * @defaultValue `messageTokens` divided by `charsPerToken`
+     */
+    estimate?: (message: Message) => number;
+    /**
+     * The divisor the default `estimate` uses. Ignored beside an `estimate` of the caller's own.
+     *
+     * @defaultValue `4`
+     */
+    charsPerToken?: number;
+}
+/** Where `planPrune` puts the boundary, and what moving it there clears. */
+export interface PrunePlan {
+    /**
+     * The new boundary: tool results before it are sent as stubs (`pruneThrough`), results at or
+     * after it whole. Always greater than the one in force. An index into the transcript the plan
+     * was made for, which the host stores.
+     */
+    through: number;
+    /** How many results the move stubs that were being sent whole. */
+    cleared: number;
+    /** Their estimated tokens, less the stubs', rounded. */
+    tokens: number;
+}
+/**
+ * The transcript with every tool result before `through` replaced by a stub.
+ *
+ * @param messages - The transcript. Not written to.
+ * @param through - The boundary, as a `PrunePlan` has it. Clamped to the transcript, so one left
+ * past its end cannot reach messages not yet written.
+ * @param [options] - `maxChars`, as `pruneToolResults` reads it.
+ * @returns `messages` itself when nothing was cleared, otherwise a new array of the same length in
+ * which each cleared result is a new message and every other one the same object.
+ *
+ * @remarks
+ * `pruneToolResults` over the head, keeping none of it — so a result that is short or carries
+ * definitions stays whole here as it does there. A function of the messages before `through` and
+ * nothing else: while the boundary stays put, what this sends for them does not change however
+ * many results arrive after it, which is what keeps the prompt cache.
+ */
+export declare function pruneThrough(messages: Message[], through: number, { maxChars }?: Pick<PruneOptions, 'maxChars'>): Message[];
+/**
+ * Where to move the prune boundary, or `undefined` when it should stay where it is.
+ *
+ * @param messages - The transcript as the host keeps it — whole, or with earlier stubs in it.
+ * @param window - The model's window, in tokens. Zero or less moves nothing on its own.
+ * @param [options] - The boundary in force, the gate, what to keep, and how to weigh a message.
+ * See `PrunePlanOptions`.
+ * @returns The plan, or `undefined`. `messages` is not written to.
+ *
+ * @remarks
+ * `pruneToolResults` keeps a sliding window of results: applied on every step, each new result
+ * pushes an older one out, rewrites a message in the middle of the history and loses the prompt
+ * cache from there on, once per tool call. A plan is a boundary instead. The host stores
+ * `through` and sends `pruneThrough(messages, through)` on every request, and between moves a
+ * request is the one before it with only its tail added.
+ *
+ * The rule, in the order it is asked:
+ *
+ * - **Never backwards, and never for nothing.** The target is the earliest of the latest
+ *   `keepLast` results. If that is not past `through`, or moving there would clear nothing still
+ *   being sent, there is no plan.
+ * - **With a fold, for free.** Given `compacted`, the move is made.
+ * - **Otherwise once the window is filling**, past `at` of it, and only when what it clears is at
+ *   least `minShare` of it. A run sitting just over the line would otherwise slide the boundary
+ *   forward and lose the cache on every step for a few hundred tokens each time.
+ *
+ * What is weighed is what the move would *clear*, by `estimate`: a short result and a proxied
+ * load's definitions stay whole wherever the boundary is, a stub is not cleared twice, and a
+ * result before `from` is not being sent.
+ */
+export declare function planPrune(messages: Message[], window: number, { through, minShare, at, used, from, compacted, keepLast, maxChars, charsPerToken, estimate, }?: PrunePlanOptions): PrunePlan | undefined;
+/**
+ * What a host says when a plan is applied, so every host says it the same way.
+ *
+ * @param plan - The plan that was applied.
+ * @param [compacted] - Whether it was made alongside a fold rather than because the window filled.
+ * @returns One line with the counts: `cleared 3 old tool results (~12,345 tokens) as the window
+ * filled`.
+ */
+export declare function pruneNotice({ cleared, tokens }: PrunePlan, compacted?: boolean): string;
 /** What `planCompaction` takes. */
 export interface CompactionOptions {
     /** The model's window, in tokens. Zero or less never compacts. */

package/dist/context/compaction.js

@@ -14,6 +14,8 @@ const SUMMARY_TOKENS = 1024;
 const SUMMARY_SLICE = 4000;
 /** The fewest messages worth folding: a summary of one costs a round trip and saves nothing. */
 const FEWEST_FOLDED = 2;
+/** How many of the latest tool results a prune leaves whole when the caller does not say. */
+const KEEP_LAST = 5;
 /** The locale a cleared result's length is written in, so the stub reads the same on every host. */
 const COUNT_LOCALE = 'en-US';
 /**
@@ -105,7 +107,7 @@ const systemHead = (messages) => {
  * few. Returns the same array when there was nothing to clear. Rewrites the prefix;
  * see the module comment on when to run it.
  */
-export function pruneToolResults(messages, { keepLast = 5, maxChars = 256 } = {}) {
+export function pruneToolResults(messages, { keepLast = KEEP_LAST, maxChars = 256 } = {}) {
     let kept = 0;
     let out;
     for (let at = messages.length - 1; at >= 0; at--) {
@@ -133,6 +135,132 @@ export function pruneToolResults(messages, { keepLast = 5, maxChars = 256 } = {}
     }
     return out ?? messages;
 }
+/**
+ * The share of the window a prune has to win back on its own before it is worth the prompt cache
+ * it costs. Below it the request after a prune is barely smaller than the one before, and every
+ * token of it is processed again anyway.
+ */
+export const PRUNE_SHARE = 0.1;
+/**
+ * The transcript with every tool result before `through` replaced by a stub.
+ *
+ * @param messages - The transcript. Not written to.
+ * @param through - The boundary, as a `PrunePlan` has it. Clamped to the transcript, so one left
+ * past its end cannot reach messages not yet written.
+ * @param [options] - `maxChars`, as `pruneToolResults` reads it.
+ * @returns `messages` itself when nothing was cleared, otherwise a new array of the same length in
+ * which each cleared result is a new message and every other one the same object.
+ *
+ * @remarks
+ * `pruneToolResults` over the head, keeping none of it — so a result that is short or carries
+ * definitions stays whole here as it does there. A function of the messages before `through` and
+ * nothing else: while the boundary stays put, what this sends for them does not change however
+ * many results arrive after it, which is what keeps the prompt cache.
+ */
+export function pruneThrough(messages, through, { maxChars } = {}) {
+    const end = clamp(through, messages.length);
+    if (end <= 0) {
+        return messages;
+    }
+    const head = messages.slice(0, end);
+    const stubbed = pruneToolResults(head, { keepLast: 0, ...(maxChars === undefined ? {} : { maxChars }) });
+    return stubbed === head ? messages : [...stubbed, ...messages.slice(end)];
+}
+/**
+ * Where the latest `keepLast` tool results begin, which is where a move puts the boundary.
+ *
+ * @param messages - The transcript.
+ * @param keepLast - How many of the latest results stay whole.
+ * @returns The index of the earliest of them; the length when `keepLast` is zero; and zero when
+ * there are not that many results, which is nowhere.
+ */
+const keepBoundary = (messages, keepLast) => {
+    if (keepLast <= 0) {
+        return messages.length;
+    }
+    let kept = 0;
+    for (let at = messages.length - 1; at >= 0; at--) {
+        if (messages[at].role === Role.Tool && ++kept === keepLast) {
+            return at;
+        }
+    }
+    return 0;
+};
+/**
+ * Where to move the prune boundary, or `undefined` when it should stay where it is.
+ *
+ * @param messages - The transcript as the host keeps it — whole, or with earlier stubs in it.
+ * @param window - The model's window, in tokens. Zero or less moves nothing on its own.
+ * @param [options] - The boundary in force, the gate, what to keep, and how to weigh a message.
+ * See `PrunePlanOptions`.
+ * @returns The plan, or `undefined`. `messages` is not written to.
+ *
+ * @remarks
+ * `pruneToolResults` keeps a sliding window of results: applied on every step, each new result
+ * pushes an older one out, rewrites a message in the middle of the history and loses the prompt
+ * cache from there on, once per tool call. A plan is a boundary instead. The host stores
+ * `through` and sends `pruneThrough(messages, through)` on every request, and between moves a
+ * request is the one before it with only its tail added.
+ *
+ * The rule, in the order it is asked:
+ *
+ * - **Never backwards, and never for nothing.** The target is the earliest of the latest
+ *   `keepLast` results. If that is not past `through`, or moving there would clear nothing still
+ *   being sent, there is no plan.
+ * - **With a fold, for free.** Given `compacted`, the move is made.
+ * - **Otherwise once the window is filling**, past `at` of it, and only when what it clears is at
+ *   least `minShare` of it. A run sitting just over the line would otherwise slide the boundary
+ *   forward and lose the cache on every step for a few hundred tokens each time.
+ *
+ * What is weighed is what the move would *clear*, by `estimate`: a short result and a proxied
+ * load's definitions stay whole wherever the boundary is, a stub is not cleared twice, and a
+ * result before `from` is not being sent.
+ */
+export function planPrune(messages, window, { through = 0, minShare = PRUNE_SHARE, at = COMPACT_AT, used, from = 0, compacted = false, keepLast = KEEP_LAST, maxChars, charsPerToken, estimate = (message) => messageTokens(message, { charsPerToken }), } = {}) {
+    const hasWindow = window > 0;
+    if (hasWindow === false && compacted === false) {
+        return undefined;
+    }
+    if (compacted === false) {
+        const estimated = messages.reduce((total, message) => total + estimate(message), 0);
+        const isFilling = Math.max(used ?? 0, estimated) >= window * at;
+        if (isFilling === false) {
+            return undefined;
+        }
+    }
+    const current = clamp(through, messages.length);
+    const target = keepBoundary(messages, keepLast);
+    if (target <= current) {
+        return undefined;
+    }
+    const sent = pruneThrough(messages, target, { maxChars });
+    let cleared = 0;
+    let tokens = 0;
+    for (let index = Math.max(current, from); index < target; index++) {
+        if (sent[index] !== messages[index]) {
+            cleared++;
+            tokens += estimate(messages[index]) - estimate(sent[index]);
+        }
+    }
+    if (cleared === 0) {
+        return undefined;
+    }
+    const isWorthIt = compacted || tokens >= window * minShare;
+    return isWorthIt ? { through: target, cleared, tokens: Math.round(tokens) } : undefined;
+}
+/**
+ * What a host says when a plan is applied, so every host says it the same way.
+ *
+ * @param plan - The plan that was applied.
+ * @param [compacted] - Whether it was made alongside a fold rather than because the window filled.
+ * @returns One line with the counts: `cleared 3 old tool results (~12,345 tokens) as the window
+ * filled`.
+ */
+export function pruneNotice({ cleared, tokens }, compacted = false) {
+    const results = cleared === 1 ? 'result' : 'results';
+    const why = compacted ? 'alongside a fold' : 'as the window filled';
+    return `cleared ${cleared} old tool ${results} (~${tokens.toLocaleString(COUNT_LOCALE)} tokens) ${why}`;
+}
 /**
  * An index held between the start of the transcript and `most`, for a caller's `from` that is
  * neither.

package/dist/index.d.ts

@@ -9,7 +9,7 @@
  * prompts, and whatever the run is about — because that is the caller's, and it is the part
  * that differs between one server and the next.
  */
-export { applyCompaction, COMPACT_AT, type CompactionOptions, type CompactionPlan, type CompactionRecord, type CompactionRunOptions, compactTranscript, KEEP_RATIO, type PruneOptions, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, } from './context/compaction.ts';
+export { applyCompaction, COMPACT_AT, type CompactionOptions, type CompactionPlan, type CompactionRecord, type CompactionRunOptions, compactTranscript, KEEP_RATIO, PRUNE_SHARE, type PruneOptions, type PrunePlan, type PrunePlanOptions, planCompaction, planPrune, pruneNotice, pruneThrough, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, } from './context/compaction.ts';
 export { estimateFrom, type LedgerEntry, type LedgerEstimateOptions, type LedgerRequest, rebaseLedger, recordRequest, type TokenLedger, tokensBetween, } from './context/ledger.ts';
 export { type AgentConfig, type Endpoint, type ModelParams, normalizeBaseUrl, type RetryPolicy, ToolDiscovery, type ToolPolicy, } from './core/config.ts';
 export { calibrate, charsPerTokenFor, resetCalibration } from './endpoint/calibration.ts';

package/dist/index.js

@@ -9,7 +9,7 @@
  * prompts, and whatever the run is about — because that is the caller's, and it is the part
  * that differs between one server and the next.
  */
-export { applyCompaction, COMPACT_AT, compactTranscript, KEEP_RATIO, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, } from './context/compaction.js';
+export { applyCompaction, COMPACT_AT, compactTranscript, KEEP_RATIO, PRUNE_SHARE, planCompaction, planPrune, pruneNotice, pruneThrough, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, } from './context/compaction.js';
 export { estimateFrom, rebaseLedger, recordRequest, tokensBetween, } from './context/ledger.js';
 export { normalizeBaseUrl, ToolDiscovery, } from './core/config.js';
 export { calibrate, charsPerTokenFor, resetCalibration } from './endpoint/calibration.js';

package/dist/run/agent-loop.d.ts

@@ -1,5 +1,5 @@
 import type OpenAI from 'openai';
-import { type CompactionOptions } from '../context/compaction.ts';
+import { type CompactionOptions, type PrunePlanOptions } from '../context/compaction.ts';
 import { type TokenLedger } from '../context/ledger.ts';
 import type { Endpoint, ModelParams, RetryPolicy, ToolPolicy } from '../core/config.ts';
 import { HookEvent } from '../hooks/hook-events.ts';
@@ -46,16 +46,38 @@ export interface AgentLoopRequest {
  * The loop gathers before its first request, beside its own preselection where it was asked for
  * one, and nothing else runs beside the two. A host with pre-turn work of its own to overlap with
  * the hooks — a compaction — calls `gather` itself, puts the context on with `withContext`, calls
- * `notify` after, passes no `hooks`, and hands `preselectQuery` the question as it was written.
+ * `notify` after, and hands `preselectQuery` the question as it was written. It still passes
+ * `hooks` when the loop folds for it, with `fire` naming only `beforeCompact`, so what it fires
+ * itself is not fired a second time.
  */
 export interface AgentLoopHooks {
     run: HookRunner;
     /** What the hooks are told. `reply` and `turn` are filled in for `afterTurn`. */
     context: HookContext;
     /**
-     * Run before the first request.
+     * The events the loop fires, and the only ones: `sessionStart` and `beforeTurn` gathered before
+     * the first request, in the order listed; `beforeCompact` beside each fold `compact` makes; and
+     * `afterTurn` once the run has its answer. `sessionEnd` and `sessionDelete` are the host's, since
+     * the loop has no point that is either, and are ignored here.
+     *
+     * @defaultValue `events`, then `afterTurn` and `beforeCompact` — everything the loop has a point
+     * for, as it fired before this existed.
+     *
+     * @remarks
+     * For a host that fires some of them around the loop rather than inside it: one that tells
+     * `beforeTurn` and `afterTurn` around a whole step of its own, and wants only the fold's
+     * `beforeCompact` from the loop, passes `[HookEvent.BeforeCompact]`. With `afterTurn` left out the
+     * result's `afterTurn` is resolved and empty, and `notes` and `context` are empty with no
+     * injecting event listed — the host's own `gather` and `notify` hold those instead.
+     */
+    fire?: readonly HookEvent[];
+    /**
+     * Run before the first request, when `fire` is not given. Ignored beside `fire`, which says the
+     * same thing with the other events in it.
      *
      * @defaultValue `[HookEvent.BeforeTurn]`
+     * @deprecated List them in `fire` instead, with `afterTurn` and `beforeCompact` where the loop
+     * should fire those too.
      */
     events?: readonly HookEvent[];
     /**
@@ -70,7 +92,7 @@ export interface AgentLoopHooks {
      * @defaultValue The preface `configureHooks` holds.
      */
     preface?: string;
-    /** Hears each note, from before the request and from `afterTurn`. */
+    /** Hears each note, from before the request, from `beforeCompact` and from `afterTurn`. */
     onNote?: OnNote;
     /**
      * What the run's first message is numbered as in the uuids hooks are told — the host's stored
@@ -244,13 +266,30 @@ export interface AgentLoopOptions {
      * more: the plan is made from the last request's reported prompt, or an estimate before the
      * first step and after a `beforeStep` rewrite; `hooks`, when given, are told `beforeCompact`
      * beside the summary; and a fold is said as a `notice` event. A summary that fails is a notice
-     * too and folds nothing. Tool results are not pruned — `beforeStep` is where a host does that.
+     * too and folds nothing. Tool results are not pruned by this — `prune` is.
      *
      * The result's `messages` then open with the summary in place of what was folded, so a host
      * that keeps its transcript append-only wants `runCompaction` and a stored fold instead. `true`
      * plans by `planCompaction`'s defaults; an object moves its thresholds.
      */
     compact?: boolean | Pick<CompactionOptions, 'compactAt' | 'keepRatio' | 'target'>;
+    /**
+     * Has the loop stub old tool results itself before a step, at a boundary `planPrune` moves, so
+     * the prompt cache is lost once per move rather than once per tool call. Off by default, and
+     * nothing moves without a `config.contextLength` — except alongside a fold.
+     *
+     * @remarks
+     * It runs after `beforeStep`, on what that returned, and before `compact`, which then plans from
+     * what is left: a prune that brings the run back under the line spares it the summary's round
+     * trip. Where a fold is made anyway, the boundary is moved again after it with `compacted`, since
+     * the cache is already lost. Each move is said as a `notice` event, with `pruneNotice`'s counts.
+     *
+     * The stubs are in the result's `messages`, as a fold is, so a host that keeps its transcript
+     * whole calls `planPrune` and `pruneThrough` in its own `beforeStep` and stores the boundary
+     * instead. `true` plans by `planPrune`'s defaults — past `COMPACT_AT` of the window, and only for
+     * at least `PRUNE_SHARE` of it; an object moves the gate and what is kept.
+     */
+    prune?: boolean | Pick<PrunePlanOptions, 'at' | 'minShare' | 'keepLast' | 'maxChars'>;
     /**
      * Tools already loaded, carried from an earlier question. See `carryOver`. Not read in proxied
      * mode, where a tool is loaded only while its definition is in the history.
@@ -326,7 +365,8 @@ export interface AgentLoopOptions {
      */
     onToolResult?: (result: ToolCallResult) => void;
     /**
-     * Hooks gathered onto the question before the first request, and told the reply after.
+     * Hooks gathered onto the question before the first request, and told the reply after — or
+     * whichever of those `fire` names.
      *
      * @remarks
      * Only this question is touched — the last user message of `messages` — and the context is on
@@ -466,12 +506,13 @@ export interface AgentLoopResult {
      * preselection picked it. Empty in eager mode, and where nothing was picked.
      */
     preselected: string[];
-    /** The hooks' notes from before the first request. */
+    /** The hooks' notes from before the first request. Empty where `fire` lists no injecting event. */
     notes: HookNote[];
     /**
      * The `<context>` blocks the hooks added to this question, as `withContext` takes them. Empty
-     * when no hook added any, and without `hooks`. Not in `messages`, so a host that wants the next
-     * turn's request to begin with this one's stores it beside the question; see `hooks`.
+     * when no hook added any, without `hooks`, and where `fire` lists no injecting event. Not in
+     * `messages`, so a host that wants the next turn's request to begin with this one's stores it
+     * beside the question; see `hooks`.
      */
     context: string;
     /**
@@ -486,8 +527,9 @@ export interface AgentLoopResult {
      * The `afterTurn` hooks' notes, once they have run — failures only, as `notify` returns them.
      * The loop does not wait for it, so the answer is never held for a hook; a host that stores the
      * notes with the turn, or must not exit before the turn is remembered, awaits it. Each note
-     * reaches `onNote` as well. Already resolved and empty without `hooks`, and rejects only if
-     * `onNote` throws.
+     * reaches `onNote` as well. Already resolved and empty without `hooks`, and where `fire` leaves
+     * `afterTurn` out — a host that tells it itself has the notes from its own `notify`. Rejects only
+     * if `onNote` throws.
      */
     afterTurn: Promise<HookNote[]>;
     /**

package/dist/run/agent-loop.js

@@ -1,4 +1,4 @@
-import { compactTranscript, planCompaction, summariser } from '../context/compaction.js';
+import { compactTranscript, planCompaction, planPrune, pruneNotice, pruneThrough, summariser, } from '../context/compaction.js';
 import { breakReason, rebaseLedger, recordRequest, } from '../context/ledger.js';
 import { ToolDiscovery } from '../core/config.js';
 import { counted } from '../core/guards.js';
@@ -6,7 +6,7 @@ import { charsPerTokenFor } from '../endpoint/calibration.js';
 import { capabilitiesFor, modelCapabilitiesFor, } from '../endpoint/capabilities.js';
 import { firstTokenMs, getClient, loadingMs, servedSlots, timeoutMs } from '../endpoint/client.js';
 import { CACHE_PROMPT_FIELD, SESSION_HEADER, SLOT_FIELD, slotAllocator } from '../endpoint/slots.js';
-import { HookEvent } from '../hooks/hook-events.js';
+import { HookEvent, INJECT_EVENTS } from '../hooks/hook-events.js';
 import { configureHooks, gather, notify, turnIndex, turnMessages, withContext, } from '../hooks/hooks.js';
 import { serverInstructionsPrompt } from '../tools/instructions.js';
 import { preselect } from '../tools/preselect.js';
@@ -265,6 +265,36 @@ function ownSummariser(options, notice) {
     // A summary that fails folds nothing, and the step goes out as it was.
     return async (text) => (await tryAsk('compaction', () => write(text), { onNotice: notice, signal })) ?? '';
 }
+/**
+ * A step's transcript with the loop's prune boundary moved, where a move was due.
+ *
+ * @param transcript - The step's messages. Not written to.
+ * @param through - The boundary in force, which only ever moves forward.
+ * @param context - The run's options, read for `prune`'s gate and the window; what the last
+ * request reported, where it still describes `transcript`; the characters per token an estimate is
+ * made at; whether a fold was just made; and `notice`, told of a move.
+ * @returns `transcript` itself and the boundary unmoved where nothing was due, otherwise the
+ * stubbed array, the new boundary and what it won back.
+ */
+function pruneBeforeStep(transcript, through, { options, used, charsPerToken, compacted = false, notice, }) {
+    const gate = typeof options.prune === 'object' ? options.prune : {};
+    const plan = planPrune(transcript, options.config.contextLength ?? 0, {
+        ...gate,
+        through,
+        ...(used === undefined ? {} : { used }),
+        charsPerToken,
+        compacted,
+    });
+    if (!plan) {
+        return { messages: transcript, through, tokens: 0 };
+    }
+    notice(pruneNotice(plan, compacted));
+    return {
+        messages: pruneThrough(transcript, plan.through, { maxChars: gate.maxChars }),
+        through: plan.through,
+        tokens: plan.tokens,
+    };
+}
 /**
  * A step's transcript after the loop's own compaction: folded where a fold was due and its summary
  * came back, and otherwise the array it was given.
@@ -281,6 +311,7 @@ function ownSummariser(options, notice) {
  */
 async function foldBeforeStep(transcript, reported, summarise, { options, charsPerToken, notice }) {
     const { config, hooks } = options;
+    const told = hooks && firing(hooks).beforeCompact ? hooks : undefined;
     const compactor = config.tasks?.compaction;
     const plan = planCompaction(transcript, {
         ...(typeof options.compact === 'object' ? options.compact : {}),
@@ -293,13 +324,37 @@ async function foldBeforeStep(transcript, reported, summarise, { options, charsP
         return transcript;
     }
     const folded = await compactTranscript(transcript, plan, summarise, {
-        ...(hooks ? { hooks: { run: hooks.run, context: hooks.context, onNote: hooks.onNote, offset: hooks.offset } } : {}),
+        ...(told ? { hooks: { run: told.run, context: told.context, onNote: told.onNote, offset: told.offset } } : {}),
     });
     if (folded !== transcript) {
         notice(`compacted ${plan.toSummarise.length} messages into a summary`);
     }
     return folded;
 }
+/**
+ * Which events the loop fires for these hooks, read from `fire`, or from `events` without it.
+ *
+ * @param hooks - The run's hooks.
+ * @returns The events to gather before the first request, and whether `afterTurn` and
+ * `beforeCompact` are the loop's to fire.
+ *
+ * @remarks
+ * Without `fire`, `events` goes to `gather` as it was given, unfiltered, so a host that had put
+ * something odd in it sees what it saw before. With `fire`, only the injecting events are gathered:
+ * a list that names every event is the whole of the loop's firing, and `afterTurn` gathered before
+ * the request would be the hooks told a reply that does not exist yet.
+ */
+function firing(hooks) {
+    if (!hooks.fire) {
+        return { before: hooks.events ?? [HookEvent.BeforeTurn], afterTurn: true, beforeCompact: true };
+    }
+    const fire = new Set(hooks.fire);
+    return {
+        before: [...fire].filter((event) => INJECT_EVENTS.has(event)),
+        afterTurn: fire.has(HookEvent.AfterTurn),
+        beforeCompact: fire.has(HookEvent.BeforeCompact),
+    };
+}
 /**
  * Tells the `afterTurn` hooks what was answered, with the turn it closed where the question is
  * still in the transcript.
@@ -499,8 +554,9 @@ async function runSteps(options, standing, pin) {
     // they cost the first request two round trips where one will do. A failing runner is a note, but
     // a host's `onNote` that throws rejects the gather, perhaps while the preselection is still out:
     // handled at once so that is no unhandled rejection, and still thrown below, where it always was.
+    const fired = hooks && firing(hooks);
     const gathering = hooks
-        ? gather(hooks.run, hooks.events ?? [HookEvent.BeforeTurn], hooks.context, {
+        ? gather(hooks.run, fired?.before ?? [], hooks.context, {
             signal,
             onNote: hooks.onNote,
             maxTokens: hooks.maxTokens,
@@ -549,6 +605,9 @@ async function runSteps(options, standing, pin) {
     const preface = hooks?.preface ?? configureHooks().preface;
     const loads = { toolsLoaded: 0, redundantLoads: 0, unknownToolNames: 0 };
     let previous;
+    // The loop's prune boundary. Its stubs stay in `messages`, so between moves each request only
+    // extends the last; a rewrite or a fold renumbers the transcript, and it starts again from zero.
+    let pruneBoundary = 0;
     let ledger = options.ledger ?? [];
     // The request the ledger's last entry came from, which is what the next one is measured against.
     // Nothing before this run's first: an earlier run's head is not something the loop can see.
@@ -596,14 +655,28 @@ async function runSteps(options, standing, pin) {
             ledger,
         });
         const given = rewritten ?? messages;
-        // The last request's prompt describes this transcript only while nothing has rewritten it.
-        const next = summarise
-            ? await foldBeforeStep(given, given === messages ? reported : {}, summarise, {
-                options,
-                charsPerToken: charsPerTokenFor(supports, config.model),
-                notice,
-            })
-            : given;
+        const isOwn = given === messages;
+        if (isOwn === false) {
+            pruneBoundary = 0;
+        }
+        // Read again after the turn: a reported prompt calibrates it.
+        const planRatio = charsPerTokenFor(supports, config.model);
+        // The last request's prompt describes this transcript only while nothing has rewritten it,
+        // and less what a prune has just won back of it.
+        const ownUsed = isOwn ? reported.used : undefined;
+        const trimmed = options.prune
+            ? pruneBeforeStep(given, pruneBoundary, { options, used: ownUsed, charsPerToken: planRatio, notice })
+            : { messages: given, through: pruneBoundary, tokens: 0 };
+        pruneBoundary = trimmed.through;
+        const foldUsed = ownUsed === undefined ? {} : { used: Math.max(0, ownUsed - trimmed.tokens) };
+        let next = summarise
+            ? await foldBeforeStep(trimmed.messages, foldUsed, summarise, { options, charsPerToken: planRatio, notice })
+            : trimmed.messages;
+        if (options.prune && next !== trimmed.messages) {
+            const after = pruneBeforeStep(next, 0, { options, charsPerToken: planRatio, compacted: true, notice });
+            next = after.messages;
+            pruneBoundary = after.through;
+        }
         // A rewrite may have folded a definition away, and a load answered "already loaded" would
         // then point at nothing. Forgetting costs a definition sent twice at worst.
         if (proxied && next !== messages) {
@@ -729,7 +802,7 @@ async function runSteps(options, standing, pin) {
         if (isAnswer || ended) {
             // Not awaited: the answer is ready, and remembering it is not something to hold it for.
             // Handed back instead, for a host that wants its notes.
-            const afterTurn = hooks ? tellAfterTurn(hooks, messages, questionAt(), turn.content) : Promise.resolve([]);
+            const afterTurn = hooks && fired?.afterTurn ? tellAfterTurn(hooks, messages, questionAt(), turn.content) : Promise.resolve([]);
             const metrics = runMetrics(recorded, { contextLength: config.contextLength });
             let outcome = RunOutcome.Answered;
             if (ended) {

package/llms.txt

@@ -21,8 +21,14 @@ Keeping a long run inside its window: stale tool results cleared, and the oldest
 - `CompactionRunOptions` (type) — What `runCompaction` and `compactTranscript` take beside the plan.
 - `compactTranscript` — The transcript with the plan's stretch replaced by one system message holding its summary.
 - `KEEP_RATIO` — The fraction of the window the kept tail may fill, leaving room for the run to grow again.
+- `PRUNE_SHARE` — The share of the window a prune has to win back on its own before it is worth the prompt cache it costs.
 - `PruneOptions` (type) — What `pruneToolResults` takes.
+- `PrunePlan` (type) — Where `planPrune` puts the boundary, and what moving it there clears.
+- `PrunePlanOptions` (type) — What `planPrune` takes beside the transcript and the window.
 - `planCompaction` — Where to fold a transcript that has grown into its window, or `undefined` when it should not be.
+- `planPrune` — Where to move the prune boundary, or `undefined` when it should stay where it is.
+- `pruneNotice` — What a host says when a plan is applied, so every host says it the same way.
+- `pruneThrough` — The transcript with every tool result before `through` replaced by a stub.
 - `pruneToolResults` — The transcript with every tool result but the latest few replaced by a one-line stub.
 - `requestIndex` — Where a stored index sits in the request `applyCompaction` builds, once a fold has shifted everything after it.
 - `runCompaction` — The hooks and the summariser for a plan, as a record to store rather than a transcript to send.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cubicecho/agent-core",
-  "version": "4.14.0",
+  "version": "4.16.0",
   "description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
   "keywords": [
     "openai",