@cubicecho/agent-core 2.12.0 → 2.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -365,8 +365,17 @@ and — naming only tools that exist — a reply that is only a JSON call or hol
365
365
  Found calls are run as `call_recovered_0` onward, the text is what is left, `onTurn` and the
366
366
  result see the turn that way, and a notice says so, since the real fix is the server's parser.
367
367
 
368
+ Every request declares its tools in name order, whichever way the caller assembled the array. A
369
+ chat template renders the tool block ahead of the system prompt, so the tool array is the first
370
+ thing a prompt cache has to match, and an array built from a map, from database rows, or from the
371
+ order servers happened to connect in is a different array on the next boot — the same tools, the
372
+ same run, and the cache for the whole transcript thrown away. Ordering by name makes it a property
373
+ of the set instead. `toolOrder: false` sends the caller's order, for a host that means it — a model
374
+ reads the array top to bottom — and a comparator orders it another way. `orderTools` is the same
375
+ thing for a caller with its own loop, and `buildBody` takes the order as its last argument.
376
+
368
377
  With `toolDiscovery: "ondemand"` and a catalogue, the request declares `load_tools` and what has
369
- been loaded, appended in the order it was loaded, and the catalogue rides on the system prompt
378
+ been loaded, and the catalogue rides on the system prompt
370
379
  unmarked, the same text on every step. Marking loads there rewrote the head of the prompt and lost
371
380
  the prompt cache for the whole transcript on each one; a model that loads a tool twice is told in
372
381
  the `load_tools` result that it already has it. A model that calls
@@ -513,6 +522,51 @@ way to save tokens.
513
522
  `pruneToolResults` keeps the transcript's indexes, so a plan made before pruning still applies to
514
523
  what it returns, as above.
515
524
 
525
+ ### A fold you store, instead of a transcript you rewrite
526
+
527
+ `compactTranscript` hands back a new array, which is the whole answer for a host whose transcript
528
+ *is* that array. A host that keeps its messages append-only — rows in a database, every one still
529
+ shown in the chat — wants the other half: what the fold was, as something to store on the session.
530
+ `runCompaction` is `compactTranscript` without the rewrite. It returns `{ summary, through, at }`,
531
+ or `undefined` when a hook vetoed or the summary came back empty, and `compactTranscript` is built
532
+ out of it, so there is one summariser and one cut rather than two that drift.
533
+
534
+ ```ts
535
+ const from = session.fold?.through ?? 0;
536
+ const plan = planCompaction(session.messages, {
537
+ limit,
538
+ used,
539
+ from, // where the last fold ended, rather than scanning for it
540
+ previous: session.fold?.summary,
541
+ });
542
+ if (!plan) return;
543
+ const fold = await runCompaction(session.messages, plan, summarise, { hooks: { run, context } });
544
+ if (fold) await save(session.id, fold); // the messages themselves are never touched
545
+ ```
546
+
547
+ `planCompaction` takes `from` and `previous` because its defaults are a *recovery*: it skips the
548
+ leading `system` messages, and reads an earlier summary back out of a `SUMMARY_LEAD` message among
549
+ them. A host whose system prompt is a separate argument and whose summary is a column has neither
550
+ in the array, and knows both exactly. Given them, nothing is scanned.
551
+
552
+ `applyCompaction(messages, fold)` is the way back — the summary as a `system` message, then
553
+ everything from `through` — and it writes the same `SUMMARY_LEAD` `planCompaction` looks for, so
554
+ the next fold continues those notes rather than summarising them a second time. No fold yet hands
555
+ back the messages themselves.
556
+
557
+ ```ts
558
+ const request = [systemMessage, ...applyCompaction(session.messages, session.fold)];
559
+ ```
560
+
561
+ **Two numberings.** A stored transcript keeps its indexes and a folded request does not, so
562
+ anything naming a position has to say which one it means. `requestIndex(index, fold)` maps the
563
+ stored index onto the request — for `withContext`'s index, or a range being shown to a hook.
564
+ `turnMessages` takes an `offset`, the stored index of the array's first message, so a message keeps
565
+ the uuid it had before the fold and a memory server deduping on it files that turn once rather than
566
+ twice; `turnIndex` takes one too, for the turns a fold took out of the array it is counting.
567
+ Planning over the stored transcript, as above, sidesteps both: the plan's indexes are the host's
568
+ already, and so are the ones `runCompaction` hands the `beforeCompact` hooks.
569
+
516
570
  ## Watching a run
517
571
 
518
572
  `watch` replays what the run has already emitted, then yields what happens next until `done`.
@@ -5,6 +5,7 @@ import type { Endpoint, ModelParams, RetryPolicy, ToolPolicy } from "./config.ts
5
5
  import { type RunEventInput, type RunMetrics } from "./events.ts";
6
6
  import { type HookContext, type HookEvent, type HookNote, type HookRunner } from "./hooks.ts";
7
7
  import type { Turn, TurnUsage } from "./stream.ts";
8
+ import { type ToolOrder } from "./tool-loading.ts";
8
9
  /**
9
10
  * The one place a streamed request's body is decided from a config and what the endpoint and
10
11
  * the model have refused.
@@ -22,10 +23,13 @@ import type { Turn, TurnUsage } from "./stream.ts";
22
23
  * @param refused What the model has refused, as `negotiate` hands it over. Absent is a model
23
24
  * that has refused nothing.
24
25
  * @param messages The request's messages, system prompt included, sent as they are.
25
- * @param tools The tool definitions. Sanitised here — a lookup for a definition seen before —
26
- * and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
26
+ * @param tools The tool definitions. Ordered by name, sanitised here — a lookup for a definition
27
+ * seen before — and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
28
+ * @param order How to order them before sending. `true`, the default, is by name, which keeps the
29
+ * cache when the caller's array is assembled differently from one request to the next. See
30
+ * `orderTools`.
27
31
  */
28
- export declare function buildBody(config: ModelParams, supports: Capabilities, refused: ModelCapabilities | undefined, messages: OpenAI.ChatCompletionMessageParam[], tools?: OpenAI.ChatCompletionTool[]): OpenAI.ChatCompletionCreateParamsStreaming;
32
+ export declare function buildBody(config: ModelParams, supports: Capabilities, refused: ModelCapabilities | undefined, messages: OpenAI.ChatCompletionMessageParam[], tools?: OpenAI.ChatCompletionTool[], order?: ToolOrder): OpenAI.ChatCompletionCreateParamsStreaming;
29
33
  /**
30
34
  * A long tool argument or result cut to what a watcher needs, with the full length said.
31
35
  *
@@ -131,6 +135,12 @@ export interface AgentLoopOptions {
131
135
  tools?: OpenAI.ChatCompletionTool[];
132
136
  /** The same tools as a name-only catalogue. On-demand mode needs it, and is eager without it. */
133
137
  catalog?: CatalogServer[];
138
+ /**
139
+ * How the declared tools are ordered before each request. By name unless told otherwise, so a
140
+ * run whose tool array was assembled in a different order than last time still meets its cache.
141
+ * `false` sends them as given. See `orderTools`.
142
+ */
143
+ toolOrder?: ToolOrder;
134
144
  /**
135
145
  * What `preselect` picked. The first step is sent these and nothing else — no catalogue, no
136
146
  * `load_tools` — because a model with the menu still in front of it shops: it reloads what it
@@ -203,7 +213,8 @@ export interface AgentLoopResult {
203
213
  * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
204
214
  *
205
215
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
206
- * prompt unchanged from step to step, loaded tools are appended to the tool array in load order,
216
+ * prompt unchanged from step to step, a load adds to the tool array which every request sends
217
+ * in the stable order `toolOrder` asks for — and
207
218
  * a catalogued tool called without being loaded is loaded and run rather than refused, and a
208
219
  * preselection shapes the first step. A turn cut off at `maxTokens` is said so as a notice,
209
220
  * because it otherwise reads exactly like a finished one — or, given `maxContinuations`, is
@@ -10,7 +10,7 @@ import { runTurn } from "./run-turn.js";
10
10
  import { relaxTools, sanitizeTools } from "./schema-compat.js";
11
11
  import { askJson, tryAsk } from "./side-task.js";
12
12
  import { parseToolArguments, recoverToolCalls } from "./tool-calls.js";
13
- import { catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_PER_LOAD, PRESELECT_SCHEMA, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
13
+ import { catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
14
14
  /**
15
15
  * The loop above a turn: send, run the tools the model asked for, send again, until it stops
16
16
  * asking.
@@ -40,11 +40,17 @@ const RESERVED = new Set(["model", "messages", "stream", "tools"]);
40
40
  * @param refused What the model has refused, as `negotiate` hands it over. Absent is a model
41
41
  * that has refused nothing.
42
42
  * @param messages The request's messages, system prompt included, sent as they are.
43
- * @param tools The tool definitions. Sanitised here — a lookup for a definition seen before —
44
- * and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
43
+ * @param tools The tool definitions. Ordered by name, sanitised here — a lookup for a definition
44
+ * seen before — and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
45
+ * @param order How to order them before sending. `true`, the default, is by name, which keeps the
46
+ * cache when the caller's array is assembled differently from one request to the next. See
47
+ * `orderTools`.
45
48
  */
46
- export function buildBody(config, supports, refused, messages, tools = []) {
47
- const declared = supports.strictSchemas ? sanitizeTools(tools) : relaxTools(sanitizeTools(tools));
49
+ export function buildBody(config, supports, refused, messages, tools = [], order = true) {
50
+ const sorted = orderTools(tools, order);
51
+ const declared = supports.strictSchemas
52
+ ? sanitizeTools(sorted)
53
+ : relaxTools(sanitizeTools(sorted));
48
54
  const effort = config.reasoningEffort;
49
55
  const extra = Object.entries(config.extraBody ?? {}).filter(([field]) => !RESERVED.has(field) && !refused?.refusedFields.has(field));
50
56
  return {
@@ -183,7 +189,8 @@ function cacheDiagnosis(previous, messages, tools, usage) {
183
189
  * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
184
190
  *
185
191
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
186
- * prompt unchanged from step to step, loaded tools are appended to the tool array in load order,
192
+ * prompt unchanged from step to step, a load adds to the tool array which every request sends
193
+ * in the stable order `toolOrder` asks for — and
187
194
  * a catalogued tool called without being loaded is loaded and run rather than refused, and a
188
195
  * preselection shapes the first step. A turn cut off at `maxTokens` is said so as a notice,
189
196
  * because it otherwise reads exactly like a finished one — or, given `maxContinuations`, is
@@ -195,7 +202,7 @@ function cacheDiagnosis(previous, messages, tools, usage) {
195
202
  */
196
203
  export async function runAgentLoop(options) {
197
204
  const { config, system = "", tools = [], catalog = [], dispatch, hooks, signal } = options;
198
- const { onTurn, beforeStep, parallel = false, recoverToolCalls: recover = true, maxContinuations = 0, } = options;
205
+ const { onTurn, beforeStep, parallel = false, recoverToolCalls: recover = true, maxContinuations = 0, toolOrder = true, } = options;
199
206
  const started = Date.now();
200
207
  // What the loop emitted, less the token deltas, for `runMetrics` at the end. Stamped here rather
201
208
  // than by the bus, which the loop does not know about.
@@ -262,11 +269,14 @@ export async function runAgentLoop(options) {
262
269
  messages = (await beforeStep?.(messages, step)) ?? messages;
263
270
  onEvent({ kind: "turn", text: `turn ${step + 1}` });
264
271
  const routed = preselected.length > 0 && step === 0;
265
- const declared = routed
272
+ // Ordered here rather than left to `buildBody`, so `names` below is what the request actually
273
+ // declared — a diagnosis reading an order the server never saw calls an untouched tool array
274
+ // `tools-changed`.
275
+ const declared = orderTools(routed
266
276
  ? byName(new Set(preselected))
267
277
  : onDemand
268
278
  ? loadedTools([LOAD_TOOLS_DEFINITION], byName(loaded))
269
- : tools;
279
+ : tools, toolOrder);
270
280
  // Unmarked, so the system prompt is the same text on every step and a load does not throw
271
281
  // away the cache for the whole transcript. What is loaded is said in `declared` and in the
272
282
  // `load_tools` result instead. The preselected first step is the one exception, by design.
@@ -275,7 +285,7 @@ export async function runAgentLoop(options) {
275
285
  ...(prompt ? [{ role: "system", content: prompt }] : []),
276
286
  ...withContext(messages, question ? messages.indexOf(question) : -1, gathered.context, hooks?.preface),
277
287
  ];
278
- const build = (supported, refused) => buildBody(config, supported, refused, request, declared);
288
+ const build = (supported, refused) => buildBody(config, supported, refused, request, declared, toolOrder);
279
289
  const turnOptions = {
280
290
  model: config.model,
281
291
  droppable: Object.keys(config.extraBody ?? {}),
@@ -427,7 +437,7 @@ export async function runAgentLoop(options) {
427
437
  used.add(name);
428
438
  const request = { id: call.id, name, args, raw };
429
439
  content = parallel
430
- ? await once(answered, `${name}${normal}`, () => dispatch(request, signal))
440
+ ? await once(answered, `${name}\0${normal}`, () => dispatch(request, signal))
431
441
  : await dispatch(request, signal);
432
442
  }
433
443
  }
@@ -70,6 +70,16 @@ export interface CompactionOptions {
70
70
  * an `estimate` of the caller's own.
71
71
  */
72
72
  charsPerToken?: number;
73
+ /**
74
+ * The first message that may be folded. Absent, the leading `system` messages are skipped and
75
+ * the fold starts after them.
76
+ */
77
+ from?: number;
78
+ /**
79
+ * The summary an earlier fold left, which this one continues. Absent, it is recovered from a
80
+ * `SUMMARY_LEAD` system message at the head, if there is one.
81
+ */
82
+ previous?: string;
73
83
  }
74
84
  /** Where to cut, as `compactTranscript` takes it. */
75
85
  export interface CompactionPlan {
@@ -92,10 +102,16 @@ export interface CompactionPlan {
92
102
  * than summarised as if it were conversation. No plan comes back when the window is not full
93
103
  * enough, or when the only legal cut folds too little to pay for the summary.
94
104
  *
105
+ * Both of those are recovered by reading the transcript, which is what a host whose array holds
106
+ * everything it sends has to do. A host that keeps its fold as a record beside an append-only
107
+ * transcript — its system prompt a separate argument, no summary message in the array at all —
108
+ * knows them exactly, and passes `from` and `previous` instead of hoping the scan agrees.
109
+ *
95
110
  * @param messages The transcript, system prompts included if the caller keeps them in it.
96
- * @param options The window, what is in use, and the ratios. See `CompactionOptions`.
111
+ * @param options The window, what is in use, the ratios, and where the last fold ended. See
112
+ * `CompactionOptions`.
97
113
  */
98
- export declare function planCompaction(messages: Message[], { limit, used, compactAt, keepRatio, charsPerToken, estimate, }: CompactionOptions): CompactionPlan | undefined;
114
+ export declare function planCompaction(messages: Message[], { limit, used, compactAt, keepRatio, charsPerToken, estimate, from: givenFrom, previous: givenPrevious, }: CompactionOptions): CompactionPlan | undefined;
99
115
  /**
100
116
  * What the summariser is handed for a plan: the earlier summary if there was one, then each
101
117
  * message as its role and at most 4000 characters of its text.
@@ -114,6 +130,93 @@ export declare function summaryInput(plan: CompactionPlan): string;
114
130
  export declare const summariser: (config: Endpoint, model: string, { system, maxTokens, ...options }?: SideTaskOptions & {
115
131
  system?: string;
116
132
  }) => (text: string) => Promise<string>;
133
+ /**
134
+ * One fold, as a host that keeps its transcript append-only stores it.
135
+ *
136
+ * The other half of `compactTranscript`: the same work, recorded rather than applied. A host that
137
+ * persists this beside an untouched transcript still shows the user every message, can undo a fold
138
+ * by dropping one row, and rebuilds the request with `applyCompaction` — where a host that keeps
139
+ * only the rewritten array has thrown the originals away.
140
+ */
141
+ export interface CompactionRecord {
142
+ /** The model's notes on everything before `through`. Trimmed, and never empty. */
143
+ summary: string;
144
+ /** Index into the transcript the plan was made for: the first message still sent whole. */
145
+ through: number;
146
+ /** ISO 8601, so the chat can show where history was folded and how stale the notes are. */
147
+ at: string;
148
+ }
149
+ /** What `runCompaction` and `compactTranscript` take beside the plan. */
150
+ export interface CompactionRunOptions {
151
+ /**
152
+ * Hooks to tell. `context` is extended with `compacting` and `range`, whose indexes are the
153
+ * plan's — and so the host's own, for a plan made over a stored transcript. `honourVeto` waits
154
+ * for the hooks and lets one stop the compaction; off by default, which adds no latency.
155
+ */
156
+ hooks?: {
157
+ run: HookRunner;
158
+ context: HookContext;
159
+ onNote?: (note: HookNote) => void;
160
+ honourVeto?: boolean;
161
+ };
162
+ /**
163
+ * The window is already exceeded — the caller caught a `ContextOverflow`, or is compacting to
164
+ * make a refused request fit — which overrides `honourVeto`.
165
+ */
166
+ forced?: boolean;
167
+ }
168
+ /**
169
+ * The hooks and the summariser for a plan, as a record to store rather than a transcript to send.
170
+ *
171
+ * What `compactTranscript` does before it rewrites anything, which is all a host needs when the
172
+ * fold lives on the session row and the messages stay where they are. Nothing here is persisted or
173
+ * logged — that is the host's, and so is deciding what to do with a fold that did not happen.
174
+ *
175
+ * @param messages The transcript the plan was made for. Read only, and only for the hooks.
176
+ * @param plan What `planCompaction` returned for it.
177
+ * @param summarise Writes the summary from `summaryInput`'s text. See `summariser`. Not called
178
+ * when a hook vetoes.
179
+ * @param options Hooks to tell and whether the window is already past. See `CompactionRunOptions`.
180
+ * @returns `undefined` when nothing was folded — a hook vetoed, or the summary came back empty —
181
+ * so the caller stores nothing and the transcript is still whole.
182
+ */
183
+ export declare function runCompaction(messages: Message[], plan: CompactionPlan, summarise: (text: string) => Promise<string>, { hooks, forced }?: CompactionRunOptions): Promise<CompactionRecord | undefined>;
184
+ /**
185
+ * The transcript as the server should see it: the folded head replaced by its summary.
186
+ *
187
+ * The inverse of storing a `CompactionRecord`, and the shape `planCompaction` expects to meet
188
+ * again — the same `SUMMARY_LEAD`, in a `system` message at the same place — so the next fold
189
+ * continues these notes rather than summarising them a second time. Any earlier summary message in
190
+ * the kept head is dropped, since the record's already contains it.
191
+ *
192
+ * @param messages The stored transcript, whole. Not written to.
193
+ * @param record The fold, or `undefined` for a session that has not been compacted, which hands
194
+ * back `messages` itself.
195
+ * @param options `from` is the first message the fold was allowed to take — the plan's, for a host
196
+ * that keeps its system prompts in the array; everything before it is kept ahead of the summary.
197
+ * Absent, the leading `system` messages are found by scanning, and zero of them is the ordinary
198
+ * case for a host whose system prompt is a separate argument.
199
+ */
200
+ export declare function applyCompaction(messages: Message[], record?: Pick<CompactionRecord, "summary" | "through">, { from }?: {
201
+ from?: number;
202
+ }): Message[];
203
+ /**
204
+ * Where a stored index sits in the request `applyCompaction` builds, once a fold has shifted
205
+ * everything after it.
206
+ *
207
+ * A transcript that stays append-only and a request that does not are two numberings of the same
208
+ * conversation, and anything that names a position — `withContext`'s index, a range handed to a
209
+ * hook — has to say which it is in. An index inside the folded stretch answers with the summary
210
+ * message that now stands for it.
211
+ *
212
+ * @param index The position in the stored transcript.
213
+ * @param record The fold in force, or `undefined` for a session that has none, which hands the
214
+ * index straight back.
215
+ * @param head How many messages the request keeps ahead of the summary — the leading system
216
+ * prompts, when the host keeps them in the array. Zero, the default, is the stored-fold case,
217
+ * where the summary is the request's first message.
218
+ */
219
+ export declare const requestIndex: (index: number, record?: Pick<CompactionRecord, "through">, head?: number) => number;
117
220
  /**
118
221
  * The transcript with the plan's stretch replaced by one system message holding its summary.
119
222
  *
@@ -127,25 +230,18 @@ export declare const summariser: (config: Endpoint, model: string, { system, max
127
230
  * `ContextOverflow`. An empty summary folds nothing either. Rewrites the prefix; see the module
128
231
  * comment on when to run it.
129
232
  *
233
+ * `runCompaction` and `applyCompaction` are its two halves, and it is nothing but the two in
234
+ * order, so a host that stores the fold instead of the array gets the same summary at the same
235
+ * cut rather than a second implementation that drifts from this one.
236
+ *
130
237
  * @param messages The transcript the plan was made for. Not written to.
131
238
  * @param plan What `planCompaction` returned for it.
132
239
  * @param summarise Writes the summary from `summaryInput`'s text. See `summariser`. Not called
133
240
  * when a hook vetoes.
134
- * @param options Hooks to tell. `context` is extended with `compacting` and `range`, whose
135
- * indexes are the plan's. `honourVeto` waits for the hooks and lets one stop the compaction; off
136
- * by default, which adds no latency. `forced` says the window is already exceeded — the caller
137
- * caught a `ContextOverflow`, or is compacting to make a refused request fit — and overrides
138
- * `honourVeto`.
241
+ * @param options Hooks to tell and whether the window is already past. See
242
+ * `CompactionRunOptions`.
139
243
  * @returns `messages` itself when nothing was folded — a veto or an empty summary — otherwise a
140
244
  * new array.
141
245
  */
142
- export declare function compactTranscript(messages: Message[], plan: CompactionPlan, summarise: (text: string) => Promise<string>, { hooks, forced, }?: {
143
- hooks?: {
144
- run: HookRunner;
145
- context: HookContext;
146
- onNote?: (note: HookNote) => void;
147
- honourVeto?: boolean;
148
- };
149
- forced?: boolean;
150
- }): Promise<Message[]>;
246
+ export declare function compactTranscript(messages: Message[], plan: CompactionPlan, summarise: (text: string) => Promise<string>, options?: CompactionRunOptions): Promise<Message[]>;
151
247
  export {};
@@ -37,6 +37,25 @@ const messageText = (message) => {
37
37
  return `${textOf(message.content)} ${calls}`.trim();
38
38
  };
39
39
  const isSummary = (message) => message.role === "system" && textOf(message.content).startsWith(SUMMARY_LEAD);
40
+ /** The summary as it sits in a transcript, which is the one shape `isSummary` recognises again. */
41
+ const summaryMessage = (summary) => ({
42
+ role: "system",
43
+ content: `${SUMMARY_LEAD}${summary.trim()}`,
44
+ });
45
+ /**
46
+ * The leading system messages a fold never touches, and the summary an earlier one left among
47
+ * them. What a caller who keeps its system prompt out of the array knows already, and passes.
48
+ */
49
+ const systemHead = (messages) => {
50
+ let from = 0;
51
+ let previous;
52
+ while (from < messages.length && messages[from].role === "system") {
53
+ if (isSummary(messages[from]))
54
+ previous = textOf(messages[from].content).slice(SUMMARY_LEAD.length);
55
+ from++;
56
+ }
57
+ return { from, previous };
58
+ };
40
59
  /**
41
60
  * The transcript with every tool result but the latest few replaced by a one-line stub.
42
61
  *
@@ -80,22 +99,24 @@ export function pruneToolResults(messages, { keepLast = 5, maxChars = 256 } = {}
80
99
  * than summarised as if it were conversation. No plan comes back when the window is not full
81
100
  * enough, or when the only legal cut folds too little to pay for the summary.
82
101
  *
102
+ * Both of those are recovered by reading the transcript, which is what a host whose array holds
103
+ * everything it sends has to do. A host that keeps its fold as a record beside an append-only
104
+ * transcript — its system prompt a separate argument, no summary message in the array at all —
105
+ * knows them exactly, and passes `from` and `previous` instead of hoping the scan agrees.
106
+ *
83
107
  * @param messages The transcript, system prompts included if the caller keeps them in it.
84
- * @param options The window, what is in use, and the ratios. See `CompactionOptions`.
108
+ * @param options The window, what is in use, the ratios, and where the last fold ended. See
109
+ * `CompactionOptions`.
85
110
  */
86
- export function planCompaction(messages, { limit, used, compactAt = COMPACT_AT, keepRatio = KEEP_RATIO, charsPerToken, estimate = (message) => messageTokens(message, { charsPerToken }), }) {
111
+ export function planCompaction(messages, { limit, used, compactAt = COMPACT_AT, keepRatio = KEEP_RATIO, charsPerToken, estimate = (message) => messageTokens(message, { charsPerToken }), from: givenFrom, previous: givenPrevious, }) {
87
112
  if (!(limit > 0))
88
113
  return undefined;
89
114
  const cost = used ?? messages.reduce((total, message) => total + estimate(message), 0);
90
115
  if (cost < limit * compactAt)
91
116
  return undefined;
92
- let from = 0;
93
- let previous;
94
- while (from < messages.length && messages[from].role === "system") {
95
- if (isSummary(messages[from]))
96
- previous = textOf(messages[from].content).slice(SUMMARY_LEAD.length);
97
- from++;
98
- }
117
+ const head = systemHead(messages);
118
+ const from = Math.min(Math.max(givenFrom ?? head.from, 0), messages.length);
119
+ const previous = givenPrevious ?? head.previous;
99
120
  const budget = limit * keepRatio;
100
121
  let kept = 0;
101
122
  let cut = messages.length;
@@ -139,31 +160,21 @@ export function summaryInput(plan) {
139
160
  */
140
161
  export const summariser = (config, model, { system = SUMMARY_PROMPT, maxTokens = 1024, ...options } = {}) => (text) => ask(config, model, system, text, { maxTokens, ...options });
141
162
  /**
142
- * The transcript with the plan's stretch replaced by one system message holding its summary.
163
+ * The hooks and the summariser for a plan, as a record to store rather than a transcript to send.
143
164
  *
144
- * `beforeCompact` is told what is being folded while the summary is written, beside it rather
145
- * than ahead of it a memory server filing it is not a rescue worth making the run wait for, and
146
- * `notify` never rejects. A host that wants its hooks able to stop a compaction sets
147
- * `honourVeto`, and then they run first and the summary waits on them: any `ok` outcome carrying
148
- * `veto` leaves the transcript as it was, and each vetoing hook is noted by name. A `forced`
149
- * compaction ignores a veto and runs the hooks beside the summary as before, because a run already
150
- * past its window has no better option — a veto there only trades the summary for a
151
- * `ContextOverflow`. An empty summary folds nothing either. Rewrites the prefix; see the module
152
- * comment on when to run it.
165
+ * What `compactTranscript` does before it rewrites anything, which is all a host needs when the
166
+ * fold lives on the session row and the messages stay where they are. Nothing here is persisted or
167
+ * logged that is the host's, and so is deciding what to do with a fold that did not happen.
153
168
  *
154
- * @param messages The transcript the plan was made for. Not written to.
169
+ * @param messages The transcript the plan was made for. Read only, and only for the hooks.
155
170
  * @param plan What `planCompaction` returned for it.
156
171
  * @param summarise Writes the summary from `summaryInput`'s text. See `summariser`. Not called
157
172
  * when a hook vetoes.
158
- * @param options Hooks to tell. `context` is extended with `compacting` and `range`, whose
159
- * indexes are the plan's. `honourVeto` waits for the hooks and lets one stop the compaction; off
160
- * by default, which adds no latency. `forced` says the window is already exceeded — the caller
161
- * caught a `ContextOverflow`, or is compacting to make a refused request fit — and overrides
162
- * `honourVeto`.
163
- * @returns `messages` itself when nothing was folded — a veto or an empty summary — otherwise a
164
- * new array.
173
+ * @param options Hooks to tell and whether the window is already past. See `CompactionRunOptions`.
174
+ * @returns `undefined` when nothing was folded a hook vetoed, or the summary came back empty —
175
+ * so the caller stores nothing and the transcript is still whole.
165
176
  */
166
- export async function compactTranscript(messages, plan, summarise, { hooks, forced = false, } = {}) {
177
+ export async function runCompaction(messages, plan, summarise, { hooks, forced = false } = {}) {
167
178
  const context = hooks && {
168
179
  ...hooks.context,
169
180
  compacting: turnMessages(hooks.context.session.id, messages, plan.from, plan.cut),
@@ -173,7 +184,7 @@ export async function compactTranscript(messages, plan, summarise, { hooks, forc
173
184
  if (hooks && context && hooks.honourVeto && !forced) {
174
185
  const { vetoed } = await consult(hooks.run, "beforeCompact", context, hooks.onNote);
175
186
  if (vetoed)
176
- return messages;
187
+ return undefined;
177
188
  summary = await summarise(summaryInput(plan));
178
189
  }
179
190
  else {
@@ -183,10 +194,85 @@ export async function compactTranscript(messages, plan, summarise, { hooks, forc
183
194
  ]);
184
195
  }
185
196
  if (!summary.trim())
197
+ return undefined;
198
+ return { summary: summary.trim(), through: plan.cut, at: new Date().toISOString() };
199
+ }
200
+ /**
201
+ * The transcript as the server should see it: the folded head replaced by its summary.
202
+ *
203
+ * The inverse of storing a `CompactionRecord`, and the shape `planCompaction` expects to meet
204
+ * again — the same `SUMMARY_LEAD`, in a `system` message at the same place — so the next fold
205
+ * continues these notes rather than summarising them a second time. Any earlier summary message in
206
+ * the kept head is dropped, since the record's already contains it.
207
+ *
208
+ * @param messages The stored transcript, whole. Not written to.
209
+ * @param record The fold, or `undefined` for a session that has not been compacted, which hands
210
+ * back `messages` itself.
211
+ * @param options `from` is the first message the fold was allowed to take — the plan's, for a host
212
+ * that keeps its system prompts in the array; everything before it is kept ahead of the summary.
213
+ * Absent, the leading `system` messages are found by scanning, and zero of them is the ordinary
214
+ * case for a host whose system prompt is a separate argument.
215
+ */
216
+ export function applyCompaction(messages, record, { from } = {}) {
217
+ if (!record?.summary.trim())
186
218
  return messages;
219
+ const head = Math.min(Math.max(from ?? systemHead(messages).from, 0), record.through);
187
220
  return [
188
- ...messages.slice(0, plan.from).filter((message) => !isSummary(message)),
189
- { role: "system", content: `${SUMMARY_LEAD}${summary.trim()}` },
190
- ...messages.slice(plan.cut),
221
+ ...messages.slice(0, head).filter((message) => !isSummary(message)),
222
+ summaryMessage(record.summary),
223
+ ...messages.slice(record.through),
191
224
  ];
192
225
  }
226
+ /**
227
+ * Where a stored index sits in the request `applyCompaction` builds, once a fold has shifted
228
+ * everything after it.
229
+ *
230
+ * A transcript that stays append-only and a request that does not are two numberings of the same
231
+ * conversation, and anything that names a position — `withContext`'s index, a range handed to a
232
+ * hook — has to say which it is in. An index inside the folded stretch answers with the summary
233
+ * message that now stands for it.
234
+ *
235
+ * @param index The position in the stored transcript.
236
+ * @param record The fold in force, or `undefined` for a session that has none, which hands the
237
+ * index straight back.
238
+ * @param head How many messages the request keeps ahead of the summary — the leading system
239
+ * prompts, when the host keeps them in the array. Zero, the default, is the stored-fold case,
240
+ * where the summary is the request's first message.
241
+ */
242
+ export const requestIndex = (index, record, head = 0) => {
243
+ if (!record)
244
+ return index;
245
+ return index < record.through ? head : index - record.through + head + 1;
246
+ };
247
+ /**
248
+ * The transcript with the plan's stretch replaced by one system message holding its summary.
249
+ *
250
+ * `beforeCompact` is told what is being folded while the summary is written, beside it rather
251
+ * than ahead of it — a memory server filing it is not a rescue worth making the run wait for, and
252
+ * `notify` never rejects. A host that wants its hooks able to stop a compaction sets
253
+ * `honourVeto`, and then they run first and the summary waits on them: any `ok` outcome carrying
254
+ * `veto` leaves the transcript as it was, and each vetoing hook is noted by name. A `forced`
255
+ * compaction ignores a veto and runs the hooks beside the summary as before, because a run already
256
+ * past its window has no better option — a veto there only trades the summary for a
257
+ * `ContextOverflow`. An empty summary folds nothing either. Rewrites the prefix; see the module
258
+ * comment on when to run it.
259
+ *
260
+ * `runCompaction` and `applyCompaction` are its two halves, and it is nothing but the two in
261
+ * order, so a host that stores the fold instead of the array gets the same summary at the same
262
+ * cut rather than a second implementation that drifts from this one.
263
+ *
264
+ * @param messages The transcript the plan was made for. Not written to.
265
+ * @param plan What `planCompaction` returned for it.
266
+ * @param summarise Writes the summary from `summaryInput`'s text. See `summariser`. Not called
267
+ * when a hook vetoes.
268
+ * @param options Hooks to tell and whether the window is already past. See
269
+ * `CompactionRunOptions`.
270
+ * @returns `messages` itself when nothing was folded — a veto or an empty summary — otherwise a
271
+ * new array.
272
+ */
273
+ export async function compactTranscript(messages, plan, summarise, options = {}) {
274
+ const record = await runCompaction(messages, plan, summarise, options);
275
+ if (!record)
276
+ return messages;
277
+ return applyCompaction(messages, record, { from: plan.from });
278
+ }
package/dist/hooks.d.ts CHANGED
@@ -256,26 +256,44 @@ export declare function untrusted(text: string, { source }?: {
256
256
  * two identical "ok"s one memory. The same message sent twice, after its turn and again when it
257
257
  * is compacted, is one.
258
258
  *
259
+ * That position is the array's, which is the host's only while the two agree. Once a compaction
260
+ * has folded the head of a request into one summary message, the same message sits at a lower
261
+ * index there than it does in a transcript the host kept whole — so the same turn, sent after it
262
+ * and again when it is compacted, would arrive under two uuids and be remembered twice. `offset`
263
+ * is what puts the numbering back on the host's own indexes; passing the stored transcript rather
264
+ * than the request avoids the question entirely.
265
+ *
259
266
  * @param sessionId Prefixes every uuid, so two sessions never share one.
260
267
  * @param messages The transcript, in whatever shape the host stores it, so long as each message
261
268
  * has an OpenAI-style `role` and `content`.
262
269
  * @param from The first index, inclusive. Below zero reads from the start.
263
270
  * @param to The end, exclusive. Absent, or past the end, reads to the end.
271
+ * @param options `offset` is what the array's first message is numbered as in the uuids — the
272
+ * stored index of `messages[0]`, when `messages` is a request a fold has shifted. Zero by default,
273
+ * which numbers by position as before. `from` and `to` stay array indexes either way.
264
274
  */
265
275
  export declare function turnMessages(sessionId: string, messages: readonly {
266
276
  role: string;
267
277
  content?: unknown;
268
- }[], from: number, to?: number): HookMessage[];
278
+ }[], from: number, to?: number, { offset }?: {
279
+ offset?: number;
280
+ }): HookMessage[];
269
281
  /**
270
282
  * Which turn of a session begins at a point, from 0: the user messages ahead of it.
271
283
  *
284
+ * Counted over what it is given, which is the session only while nothing has been folded away — a
285
+ * compacted request has lost the questions the summary now stands for, and turn eleven counting
286
+ * itself as turn two is the kind of thing a hook writes into a memory. Count over the stored
287
+ * transcript, or add what the fold took as `offset`.
288
+ *
272
289
  * @param messages The transcript.
273
290
  * @param before Where the turn begins. Absent is the end, which is the index of a turn whose
274
291
  * question has not been appended yet.
292
+ * @param offset Turns already folded away and so not in `messages`. Zero by default.
275
293
  */
276
294
  export declare const turnIndex: (messages: readonly {
277
295
  role: string;
278
- }[], before?: number) => number;
296
+ }[], before?: number, offset?: number) => number;
279
297
  /**
280
298
  * Runs the hooks ahead of a request and builds what they add to it.
281
299
  *
package/dist/hooks.js CHANGED
@@ -204,13 +204,23 @@ const textOf = (content) => {
204
204
  * two identical "ok"s one memory. The same message sent twice, after its turn and again when it
205
205
  * is compacted, is one.
206
206
  *
207
+ * That position is the array's, which is the host's only while the two agree. Once a compaction
208
+ * has folded the head of a request into one summary message, the same message sits at a lower
209
+ * index there than it does in a transcript the host kept whole — so the same turn, sent after it
210
+ * and again when it is compacted, would arrive under two uuids and be remembered twice. `offset`
211
+ * is what puts the numbering back on the host's own indexes; passing the stored transcript rather
212
+ * than the request avoids the question entirely.
213
+ *
207
214
  * @param sessionId Prefixes every uuid, so two sessions never share one.
208
215
  * @param messages The transcript, in whatever shape the host stores it, so long as each message
209
216
  * has an OpenAI-style `role` and `content`.
210
217
  * @param from The first index, inclusive. Below zero reads from the start.
211
218
  * @param to The end, exclusive. Absent, or past the end, reads to the end.
219
+ * @param options `offset` is what the array's first message is numbered as in the uuids — the
220
+ * stored index of `messages[0]`, when `messages` is a request a fold has shifted. Zero by default,
221
+ * which numbers by position as before. `from` and `to` stay array indexes either way.
212
222
  */
213
- export function turnMessages(sessionId, messages, from, to) {
223
+ export function turnMessages(sessionId, messages, from, to, { offset = 0 } = {}) {
214
224
  const end = Math.min(to ?? messages.length, messages.length);
215
225
  const out = [];
216
226
  for (let at = Math.max(0, from); at < end; at++) {
@@ -221,18 +231,28 @@ export function turnMessages(sessionId, messages, from, to) {
221
231
  if (!text)
222
232
  continue;
223
233
  const digest = createHash("sha256").update(`${message.role}\0${text}`).digest("hex");
224
- out.push({ speaker: message.role, text, uuid: `${sessionId}:${at}:${digest.slice(0, 12)}` });
234
+ out.push({
235
+ speaker: message.role,
236
+ text,
237
+ uuid: `${sessionId}:${at + offset}:${digest.slice(0, 12)}`,
238
+ });
225
239
  }
226
240
  return out;
227
241
  }
228
242
  /**
229
243
  * Which turn of a session begins at a point, from 0: the user messages ahead of it.
230
244
  *
245
+ * Counted over what it is given, which is the session only while nothing has been folded away — a
246
+ * compacted request has lost the questions the summary now stands for, and turn eleven counting
247
+ * itself as turn two is the kind of thing a hook writes into a memory. Count over the stored
248
+ * transcript, or add what the fold took as `offset`.
249
+ *
231
250
  * @param messages The transcript.
232
251
  * @param before Where the turn begins. Absent is the end, which is the index of a turn whose
233
252
  * question has not been appended yet.
253
+ * @param offset Turns already folded away and so not in `messages`. Zero by default.
234
254
  */
235
- export const turnIndex = (messages, before = messages.length) => messages.slice(0, before).filter((message) => message.role === "user").length;
255
+ export const turnIndex = (messages, before = messages.length, offset = 0) => offset + messages.slice(0, before).filter((message) => message.role === "user").length;
236
256
  /** A runner that rejected, as the one outcome its event can still be noted by. */
237
257
  const rejected = (event, error) => ({
238
258
  serverId: "",
package/dist/index.d.ts CHANGED
@@ -14,7 +14,7 @@ export { calibrate, charsPerTokenFor, resetCalibration } from "./calibration.ts"
14
14
  export { type Capabilities, capabilitiesFor, type ModelCapabilities, modelCapabilitiesFor, type NegotiateOptions, negotiate, resetCapabilities, } from "./capabilities.ts";
15
15
  export type { CatalogServer } from "./catalog.ts";
16
16
  export { type ClientPoolOptions, configureClients, contextLimitFor, endpointId, endpointKey, FIRST_TOKEN_FACTOR, firstTokenMs, getClient, listModels, type ModelInfo, NO_KEY, resetClients, servedWindow, timeoutMs, } from "./client.ts";
17
- export { COMPACT_AT, type CompactionOptions, type CompactionPlan, compactTranscript, KEEP_RATIO, type PruneOptions, planCompaction, pruneToolResults, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.ts";
17
+ export { applyCompaction, COMPACT_AT, type CompactionOptions, type CompactionPlan, type CompactionRecord, type CompactionRunOptions, compactTranscript, KEEP_RATIO, type PruneOptions, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.ts";
18
18
  export type { AgentConfig, Endpoint, ModelParams, RetryPolicy, ToolPolicy, } from "./config.ts";
19
19
  export { type ContinueTurnOptions, continueTurn, isContinuable, } from "./continuation.ts";
20
20
  export { errorMessage } from "./errors.ts";
@@ -30,4 +30,4 @@ export { type Produced, type StreamTurnOptions, streamTurn, type Turn, type Turn
30
30
  export { ALL_FENCES, DEFAULT_FENCES, type Fence, FenceSplitter, type FenceSplitterOptions, type Split, stripThinking, THINK_FENCE, } from "./thinking.ts";
31
31
  export { estimateTokens } from "./tokens.ts";
32
32
  export { parseToolArguments, recoverToolCalls, ToolArgumentsError, type ToolCall, } from "./tool-calls.ts";
33
- export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.ts";
33
+ export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, type ToolOrder, } from "./tool-loading.ts";
package/dist/index.js CHANGED
@@ -13,7 +13,7 @@ export { buildBody, preselect, preview, resolveApiKey, runAgentLoop, } from "./a
13
13
  export { calibrate, charsPerTokenFor, resetCalibration } from "./calibration.js";
14
14
  export { capabilitiesFor, modelCapabilitiesFor, negotiate, resetCapabilities, } from "./capabilities.js";
15
15
  export { configureClients, contextLimitFor, endpointId, endpointKey, FIRST_TOKEN_FACTOR, firstTokenMs, getClient, listModels, NO_KEY, resetClients, servedWindow, timeoutMs, } from "./client.js";
16
- export { COMPACT_AT, compactTranscript, KEEP_RATIO, planCompaction, pruneToolResults, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.js";
16
+ export { applyCompaction, COMPACT_AT, compactTranscript, KEEP_RATIO, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.js";
17
17
  export { continueTurn, isContinuable, } from "./continuation.js";
18
18
  export { errorMessage } from "./errors.js";
19
19
  export { configureEvents, emit, endRun, fold, history, resetEvents, runMetrics, watch, } from "./events.js";
@@ -28,4 +28,4 @@ export { streamTurn, } from "./stream.js";
28
28
  export { ALL_FENCES, DEFAULT_FENCES, FenceSplitter, stripThinking, THINK_FENCE, } from "./thinking.js";
29
29
  export { estimateTokens } from "./tokens.js";
30
30
  export { parseToolArguments, recoverToolCalls, ToolArgumentsError, } from "./tool-calls.js";
31
- export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
31
+ export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
@@ -56,16 +56,41 @@ export declare function catalogPrompt(catalog: CatalogServer[], loaded?: Readonl
56
56
  /**
57
57
  * A tool array with newly loaded definitions appended, in the order they were loaded.
58
58
  *
59
- * Never re-sorted and never rebuilt from a set. A template renders the tool array into the
60
- * prompt near its head, and a load that moved an earlier definition moved everything after it,
61
- * so the cache was lost from there on every load; appended, the definitions already sent stay
62
- * a prefix of the new array.
59
+ * Appended and never rebuilt from a set, so what a load adds is decided by the load and not by
60
+ * the shape of whatever collection the definitions came out of. Where the appended array ends up
61
+ * in the request is `orderTools`' business, which the loop applies after this.
63
62
  *
64
63
  * @param previous What the last request declared, `load_tools` included. Not written to.
65
64
  * @param matched The definitions to add. Ones whose name is already declared, here or earlier in
66
65
  * this list, are skipped rather than moved.
67
66
  */
68
67
  export declare function loadedTools(previous: readonly OpenAI.ChatCompletionTool[], matched: readonly OpenAI.ChatCompletionTool[]): OpenAI.ChatCompletionTool[];
68
+ /**
69
+ * How a tool array is ordered before it is sent: `true` by name, `false` as the caller built it,
70
+ * or a comparator over the two names.
71
+ */
72
+ export type ToolOrder = boolean | ((a: string, b: string) => number);
73
+ /**
74
+ * The tool array in a stable order, so the same set of tools renders the same way twice.
75
+ *
76
+ * A chat template renders the declared tools ahead of the system prompt, which makes the tool
77
+ * array the first thing a prompt cache has to match — and an array assembled from a map, from
78
+ * database rows, or from the order servers happened to connect in changes between processes and
79
+ * between reconnects. Every such change costs the cache for the whole transcript rather than for
80
+ * the tools alone, and nothing about the request the model sees is different. Ordering by name
81
+ * makes the array a property of the set instead of of how it was built, at the price of a load
82
+ * inserting rather than appending. Definitions come back by identity, so `sanitizeTools` still
83
+ * finds each one in its cache.
84
+ *
85
+ * `false` is for a caller that means its order: the model reads the array top to bottom, and a
86
+ * host may be putting what it wants reached for first at the front.
87
+ *
88
+ * @param tools The definitions to order. Not written to.
89
+ * @param order `true` for name order, `false` to leave it alone, or a comparator over the names.
90
+ * A tool that is not a function orders as the empty name.
91
+ * @returns `tools` itself when it is already in that order, so the common case copies nothing.
92
+ */
93
+ export declare function orderTools(tools: OpenAI.ChatCompletionTool[], order?: ToolOrder): OpenAI.ChatCompletionTool[];
69
94
  /**
70
95
  * The most a single `load_tools` call may pull in.
71
96
  *
@@ -107,10 +107,9 @@ const flatten = (catalog) => catalog.flatMap((server) => server.tools);
107
107
  /**
108
108
  * A tool array with newly loaded definitions appended, in the order they were loaded.
109
109
  *
110
- * Never re-sorted and never rebuilt from a set. A template renders the tool array into the
111
- * prompt near its head, and a load that moved an earlier definition moved everything after it,
112
- * so the cache was lost from there on every load; appended, the definitions already sent stay
113
- * a prefix of the new array.
110
+ * Appended and never rebuilt from a set, so what a load adds is decided by the load and not by
111
+ * the shape of whatever collection the definitions came out of. Where the appended array ends up
112
+ * in the request is `orderTools`' business, which the loop applies after this.
114
113
  *
115
114
  * @param previous What the last request declared, `load_tools` included. Not written to.
116
115
  * @param matched The definitions to add. Ones whose name is already declared, here or earlier in
@@ -129,6 +128,36 @@ export function loadedTools(previous, matched) {
129
128
  }
130
129
  return tools;
131
130
  }
131
+ /**
132
+ * The tool array in a stable order, so the same set of tools renders the same way twice.
133
+ *
134
+ * A chat template renders the declared tools ahead of the system prompt, which makes the tool
135
+ * array the first thing a prompt cache has to match — and an array assembled from a map, from
136
+ * database rows, or from the order servers happened to connect in changes between processes and
137
+ * between reconnects. Every such change costs the cache for the whole transcript rather than for
138
+ * the tools alone, and nothing about the request the model sees is different. Ordering by name
139
+ * makes the array a property of the set instead of of how it was built, at the price of a load
140
+ * inserting rather than appending. Definitions come back by identity, so `sanitizeTools` still
141
+ * finds each one in its cache.
142
+ *
143
+ * `false` is for a caller that means its order: the model reads the array top to bottom, and a
144
+ * host may be putting what it wants reached for first at the front.
145
+ *
146
+ * @param tools The definitions to order. Not written to.
147
+ * @param order `true` for name order, `false` to leave it alone, or a comparator over the names.
148
+ * A tool that is not a function orders as the empty name.
149
+ * @returns `tools` itself when it is already in that order, so the common case copies nothing.
150
+ */
151
+ export function orderTools(tools, order = true) {
152
+ if (order === false)
153
+ return tools;
154
+ const nameOf = (tool) => tool.type === "function" ? tool.function.name : "";
155
+ // Code-unit order rather than `localeCompare`, whose answer depends on the host's locale —
156
+ // which is the kind of instability this exists to remove.
157
+ const compare = typeof order === "function" ? order : (a, b) => (a < b ? -1 : a > b ? 1 : 0);
158
+ const sorted = [...tools].sort((a, b) => compare(nameOf(a), nameOf(b)));
159
+ return sorted.some((tool, at) => tool !== tools[at]) ? sorted : tools;
160
+ }
132
161
  /**
133
162
  * The most a single `load_tools` call may pull in.
134
163
  *
package/llms.txt CHANGED
@@ -71,14 +71,19 @@ The contract between whatever holds the tools and the loop that offers them to a
71
71
 
72
72
  Keeping a long run inside its window: stale tool results cleared, and the oldest stretch folded into a summary the model writes itself.
73
73
 
74
+ - `applyCompaction` — The transcript as the server should see it: the folded head replaced by its summary.
74
75
  - `COMPACT_AT` — The fraction of the window in use before a summary is worth its own round trip.
75
76
  - `CompactionOptions` (type) — What `planCompaction` takes.
76
77
  - `CompactionPlan` (type) — Where to cut, as `compactTranscript` takes it.
78
+ - `CompactionRecord` (type) — One fold, as a host that keeps its transcript append-only stores it.
79
+ - `CompactionRunOptions` (type) — What `runCompaction` and `compactTranscript` take beside the plan.
77
80
  - `compactTranscript` — The transcript with the plan's stretch replaced by one system message holding its summary.
78
81
  - `KEEP_RATIO` — The fraction of the window the kept tail may fill, leaving room for the run to grow again.
79
82
  - `PruneOptions` (type) — What `pruneToolResults` takes.
80
83
  - `planCompaction` — Where to fold a transcript that has grown into its window, or `undefined` when it should not be.
81
84
  - `pruneToolResults` — The transcript with every tool result but the latest few replaced by a one-line stub.
85
+ - `requestIndex` — Where a stored index sits in the request `applyCompaction` builds, once a fold has shifted everything after it.
86
+ - `runCompaction` — The hooks and the summariser for a plan, as a record to store rather than a transcript to send.
82
87
  - `SUMMARY_LEAD` — How a summary message opens, which is also how `planCompaction` knows one from a system prompt.
83
88
  - `SUMMARY_PROMPT` — The summariser's instruction when the caller gives none.
84
89
  - `summariser` — A summariser that asks `model` with `SUMMARY_PROMPT`, for `compactTranscript`.
@@ -271,9 +276,11 @@ Reading what a model meant by a tool call when it did not write one cleanly.
271
276
  - `loadResult` — What `load_tools` reports back: the descriptions, now that they are worth their tokens.
272
277
  - `MAX_CARRIED` — The most a conversation carries between turns.
273
278
  - `MAX_PER_LOAD` — The most a single `load_tools` call may pull in.
279
+ - `orderTools` — The tool array in a stable order, so the same set of tools renders the same way twice.
274
280
  - `PRESELECT_SCHEMA` — The shape a preselector's answer is held to where the server takes a schema: `{ tools: [...] }`.
275
281
  - `PRESELECT_SYSTEM` — The preselection system prompt at the default cap, for a caller that never changes it.
276
282
  - `preselectInput` — The user message for a preselection call: the catalogue, then the request.
277
283
  - `preselection` — Resolves a preselection against the catalogue: unknown names dropped, count capped.
278
284
  - `preselectSystem` — The system prompt a preselector is given, holding it to the cap its answer will be held to.
279
285
  - `requestedNames` — `load_tools` arguments, defensively — a model may send a bare string or a nested object.
286
+ - `ToolOrder` (type) — How a tool array is ordered before it is sent: `true` by name, `false` as the caller built it, or a comparator over the two names.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@cubicecho/agent-core",
3
- "version": "2.12.0",
3
+ "version": "2.13.0",
4
4
  "description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
5
5
  "keywords": [
6
6
  "openai",