@cubicecho/agent-core 2.12.0 → 2.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +55 -1
- package/dist/agent-loop.d.ts +15 -4
- package/dist/agent-loop.js +21 -11
- package/dist/compaction.d.ts +112 -16
- package/dist/compaction.js +118 -32
- package/dist/hooks.d.ts +20 -2
- package/dist/hooks.js +23 -3
- package/dist/index.d.ts +2 -2
- package/dist/index.js +2 -2
- package/dist/tool-loading.d.ts +29 -4
- package/dist/tool-loading.js +33 -4
- package/llms.txt +7 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -365,8 +365,17 @@ and — naming only tools that exist — a reply that is only a JSON call or hol
|
|
|
365
365
|
Found calls are run as `call_recovered_0` onward, the text is what is left, `onTurn` and the
|
|
366
366
|
result see the turn that way, and a notice says so, since the real fix is the server's parser.
|
|
367
367
|
|
|
368
|
+
Every request declares its tools in name order, whichever way the caller assembled the array. A
|
|
369
|
+
chat template renders the tool block ahead of the system prompt, so the tool array is the first
|
|
370
|
+
thing a prompt cache has to match, and an array built from a map, from database rows, or from the
|
|
371
|
+
order servers happened to connect in is a different array on the next boot — the same tools, the
|
|
372
|
+
same run, and the cache for the whole transcript thrown away. Ordering by name makes it a property
|
|
373
|
+
of the set instead. `toolOrder: false` sends the caller's order, for a host that means it — a model
|
|
374
|
+
reads the array top to bottom — and a comparator orders it another way. `orderTools` is the same
|
|
375
|
+
thing for a caller with its own loop, and `buildBody` takes the order as its last argument.
|
|
376
|
+
|
|
368
377
|
With `toolDiscovery: "ondemand"` and a catalogue, the request declares `load_tools` and what has
|
|
369
|
-
been loaded,
|
|
378
|
+
been loaded, and the catalogue rides on the system prompt
|
|
370
379
|
unmarked, the same text on every step. Marking loads there rewrote the head of the prompt and lost
|
|
371
380
|
the prompt cache for the whole transcript on each one; a model that loads a tool twice is told in
|
|
372
381
|
the `load_tools` result that it already has it. A model that calls
|
|
@@ -513,6 +522,51 @@ way to save tokens.
|
|
|
513
522
|
`pruneToolResults` keeps the transcript's indexes, so a plan made before pruning still applies to
|
|
514
523
|
what it returns, as above.
|
|
515
524
|
|
|
525
|
+
### A fold you store, instead of a transcript you rewrite
|
|
526
|
+
|
|
527
|
+
`compactTranscript` hands back a new array, which is the whole answer for a host whose transcript
|
|
528
|
+
*is* that array. A host that keeps its messages append-only — rows in a database, every one still
|
|
529
|
+
shown in the chat — wants the other half: what the fold was, as something to store on the session.
|
|
530
|
+
`runCompaction` is `compactTranscript` without the rewrite. It returns `{ summary, through, at }`,
|
|
531
|
+
or `undefined` when a hook vetoed or the summary came back empty, and `compactTranscript` is built
|
|
532
|
+
out of it, so there is one summariser and one cut rather than two that drift.
|
|
533
|
+
|
|
534
|
+
```ts
|
|
535
|
+
const from = session.fold?.through ?? 0;
|
|
536
|
+
const plan = planCompaction(session.messages, {
|
|
537
|
+
limit,
|
|
538
|
+
used,
|
|
539
|
+
from, // where the last fold ended, rather than scanning for it
|
|
540
|
+
previous: session.fold?.summary,
|
|
541
|
+
});
|
|
542
|
+
if (!plan) return;
|
|
543
|
+
const fold = await runCompaction(session.messages, plan, summarise, { hooks: { run, context } });
|
|
544
|
+
if (fold) await save(session.id, fold); // the messages themselves are never touched
|
|
545
|
+
```
|
|
546
|
+
|
|
547
|
+
`planCompaction` takes `from` and `previous` because its defaults are a *recovery*: it skips the
|
|
548
|
+
leading `system` messages, and reads an earlier summary back out of a `SUMMARY_LEAD` message among
|
|
549
|
+
them. A host whose system prompt is a separate argument and whose summary is a column has neither
|
|
550
|
+
in the array, and knows both exactly. Given them, nothing is scanned.
|
|
551
|
+
|
|
552
|
+
`applyCompaction(messages, fold)` is the way back — the summary as a `system` message, then
|
|
553
|
+
everything from `through` — and it writes the same `SUMMARY_LEAD` `planCompaction` looks for, so
|
|
554
|
+
the next fold continues those notes rather than summarising them a second time. No fold yet hands
|
|
555
|
+
back the messages themselves.
|
|
556
|
+
|
|
557
|
+
```ts
|
|
558
|
+
const request = [systemMessage, ...applyCompaction(session.messages, session.fold)];
|
|
559
|
+
```
|
|
560
|
+
|
|
561
|
+
**Two numberings.** A stored transcript keeps its indexes and a folded request does not, so
|
|
562
|
+
anything naming a position has to say which one it means. `requestIndex(index, fold)` maps the
|
|
563
|
+
stored index onto the request — for `withContext`'s index, or a range being shown to a hook.
|
|
564
|
+
`turnMessages` takes an `offset`, the stored index of the array's first message, so a message keeps
|
|
565
|
+
the uuid it had before the fold and a memory server deduping on it files that turn once rather than
|
|
566
|
+
twice; `turnIndex` takes one too, for the turns a fold took out of the array it is counting.
|
|
567
|
+
Planning over the stored transcript, as above, sidesteps both: the plan's indexes are the host's
|
|
568
|
+
already, and so are the ones `runCompaction` hands the `beforeCompact` hooks.
|
|
569
|
+
|
|
516
570
|
## Watching a run
|
|
517
571
|
|
|
518
572
|
`watch` replays what the run has already emitted, then yields what happens next until `done`.
|
package/dist/agent-loop.d.ts
CHANGED
|
@@ -5,6 +5,7 @@ import type { Endpoint, ModelParams, RetryPolicy, ToolPolicy } from "./config.ts
|
|
|
5
5
|
import { type RunEventInput, type RunMetrics } from "./events.ts";
|
|
6
6
|
import { type HookContext, type HookEvent, type HookNote, type HookRunner } from "./hooks.ts";
|
|
7
7
|
import type { Turn, TurnUsage } from "./stream.ts";
|
|
8
|
+
import { type ToolOrder } from "./tool-loading.ts";
|
|
8
9
|
/**
|
|
9
10
|
* The one place a streamed request's body is decided from a config and what the endpoint and
|
|
10
11
|
* the model have refused.
|
|
@@ -22,10 +23,13 @@ import type { Turn, TurnUsage } from "./stream.ts";
|
|
|
22
23
|
* @param refused What the model has refused, as `negotiate` hands it over. Absent is a model
|
|
23
24
|
* that has refused nothing.
|
|
24
25
|
* @param messages The request's messages, system prompt included, sent as they are.
|
|
25
|
-
* @param tools The tool definitions.
|
|
26
|
-
* and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
|
|
26
|
+
* @param tools The tool definitions. Ordered by name, sanitised here — a lookup for a definition
|
|
27
|
+
* seen before — and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
|
|
28
|
+
* @param order How to order them before sending. `true`, the default, is by name, which keeps the
|
|
29
|
+
* cache when the caller's array is assembled differently from one request to the next. See
|
|
30
|
+
* `orderTools`.
|
|
27
31
|
*/
|
|
28
|
-
export declare function buildBody(config: ModelParams, supports: Capabilities, refused: ModelCapabilities | undefined, messages: OpenAI.ChatCompletionMessageParam[], tools?: OpenAI.ChatCompletionTool[]): OpenAI.ChatCompletionCreateParamsStreaming;
|
|
32
|
+
export declare function buildBody(config: ModelParams, supports: Capabilities, refused: ModelCapabilities | undefined, messages: OpenAI.ChatCompletionMessageParam[], tools?: OpenAI.ChatCompletionTool[], order?: ToolOrder): OpenAI.ChatCompletionCreateParamsStreaming;
|
|
29
33
|
/**
|
|
30
34
|
* A long tool argument or result cut to what a watcher needs, with the full length said.
|
|
31
35
|
*
|
|
@@ -131,6 +135,12 @@ export interface AgentLoopOptions {
|
|
|
131
135
|
tools?: OpenAI.ChatCompletionTool[];
|
|
132
136
|
/** The same tools as a name-only catalogue. On-demand mode needs it, and is eager without it. */
|
|
133
137
|
catalog?: CatalogServer[];
|
|
138
|
+
/**
|
|
139
|
+
* How the declared tools are ordered before each request. By name unless told otherwise, so a
|
|
140
|
+
* run whose tool array was assembled in a different order than last time still meets its cache.
|
|
141
|
+
* `false` sends them as given. See `orderTools`.
|
|
142
|
+
*/
|
|
143
|
+
toolOrder?: ToolOrder;
|
|
134
144
|
/**
|
|
135
145
|
* What `preselect` picked. The first step is sent these and nothing else — no catalogue, no
|
|
136
146
|
* `load_tools` — because a model with the menu still in front of it shops: it reloads what it
|
|
@@ -203,7 +213,8 @@ export interface AgentLoopResult {
|
|
|
203
213
|
* whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
|
|
204
214
|
*
|
|
205
215
|
* On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
|
|
206
|
-
* prompt unchanged from step to step,
|
|
216
|
+
* prompt unchanged from step to step, a load adds to the tool array — which every request sends
|
|
217
|
+
* in the stable order `toolOrder` asks for — and
|
|
207
218
|
* a catalogued tool called without being loaded is loaded and run rather than refused, and a
|
|
208
219
|
* preselection shapes the first step. A turn cut off at `maxTokens` is said so as a notice,
|
|
209
220
|
* because it otherwise reads exactly like a finished one — or, given `maxContinuations`, is
|
package/dist/agent-loop.js
CHANGED
|
@@ -10,7 +10,7 @@ import { runTurn } from "./run-turn.js";
|
|
|
10
10
|
import { relaxTools, sanitizeTools } from "./schema-compat.js";
|
|
11
11
|
import { askJson, tryAsk } from "./side-task.js";
|
|
12
12
|
import { parseToolArguments, recoverToolCalls } from "./tool-calls.js";
|
|
13
|
-
import { catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_PER_LOAD, PRESELECT_SCHEMA, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
|
|
13
|
+
import { catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
|
|
14
14
|
/**
|
|
15
15
|
* The loop above a turn: send, run the tools the model asked for, send again, until it stops
|
|
16
16
|
* asking.
|
|
@@ -40,11 +40,17 @@ const RESERVED = new Set(["model", "messages", "stream", "tools"]);
|
|
|
40
40
|
* @param refused What the model has refused, as `negotiate` hands it over. Absent is a model
|
|
41
41
|
* that has refused nothing.
|
|
42
42
|
* @param messages The request's messages, system prompt included, sent as they are.
|
|
43
|
-
* @param tools The tool definitions.
|
|
44
|
-
* and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
|
|
43
|
+
* @param tools The tool definitions. Ordered by name, sanitised here — a lookup for a definition
|
|
44
|
+
* seen before — and relaxed where the endpoint needs it. Empty sends no `tools` field at all.
|
|
45
|
+
* @param order How to order them before sending. `true`, the default, is by name, which keeps the
|
|
46
|
+
* cache when the caller's array is assembled differently from one request to the next. See
|
|
47
|
+
* `orderTools`.
|
|
45
48
|
*/
|
|
46
|
-
export function buildBody(config, supports, refused, messages, tools = []) {
|
|
47
|
-
const
|
|
49
|
+
export function buildBody(config, supports, refused, messages, tools = [], order = true) {
|
|
50
|
+
const sorted = orderTools(tools, order);
|
|
51
|
+
const declared = supports.strictSchemas
|
|
52
|
+
? sanitizeTools(sorted)
|
|
53
|
+
: relaxTools(sanitizeTools(sorted));
|
|
48
54
|
const effort = config.reasoningEffort;
|
|
49
55
|
const extra = Object.entries(config.extraBody ?? {}).filter(([field]) => !RESERVED.has(field) && !refused?.refusedFields.has(field));
|
|
50
56
|
return {
|
|
@@ -183,7 +189,8 @@ function cacheDiagnosis(previous, messages, tools, usage) {
|
|
|
183
189
|
* whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
|
|
184
190
|
*
|
|
185
191
|
* On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
|
|
186
|
-
* prompt unchanged from step to step,
|
|
192
|
+
* prompt unchanged from step to step, a load adds to the tool array — which every request sends
|
|
193
|
+
* in the stable order `toolOrder` asks for — and
|
|
187
194
|
* a catalogued tool called without being loaded is loaded and run rather than refused, and a
|
|
188
195
|
* preselection shapes the first step. A turn cut off at `maxTokens` is said so as a notice,
|
|
189
196
|
* because it otherwise reads exactly like a finished one — or, given `maxContinuations`, is
|
|
@@ -195,7 +202,7 @@ function cacheDiagnosis(previous, messages, tools, usage) {
|
|
|
195
202
|
*/
|
|
196
203
|
export async function runAgentLoop(options) {
|
|
197
204
|
const { config, system = "", tools = [], catalog = [], dispatch, hooks, signal } = options;
|
|
198
|
-
const { onTurn, beforeStep, parallel = false, recoverToolCalls: recover = true, maxContinuations = 0, } = options;
|
|
205
|
+
const { onTurn, beforeStep, parallel = false, recoverToolCalls: recover = true, maxContinuations = 0, toolOrder = true, } = options;
|
|
199
206
|
const started = Date.now();
|
|
200
207
|
// What the loop emitted, less the token deltas, for `runMetrics` at the end. Stamped here rather
|
|
201
208
|
// than by the bus, which the loop does not know about.
|
|
@@ -262,11 +269,14 @@ export async function runAgentLoop(options) {
|
|
|
262
269
|
messages = (await beforeStep?.(messages, step)) ?? messages;
|
|
263
270
|
onEvent({ kind: "turn", text: `turn ${step + 1}` });
|
|
264
271
|
const routed = preselected.length > 0 && step === 0;
|
|
265
|
-
|
|
272
|
+
// Ordered here rather than left to `buildBody`, so `names` below is what the request actually
|
|
273
|
+
// declared — a diagnosis reading an order the server never saw calls an untouched tool array
|
|
274
|
+
// `tools-changed`.
|
|
275
|
+
const declared = orderTools(routed
|
|
266
276
|
? byName(new Set(preselected))
|
|
267
277
|
: onDemand
|
|
268
278
|
? loadedTools([LOAD_TOOLS_DEFINITION], byName(loaded))
|
|
269
|
-
: tools;
|
|
279
|
+
: tools, toolOrder);
|
|
270
280
|
// Unmarked, so the system prompt is the same text on every step and a load does not throw
|
|
271
281
|
// away the cache for the whole transcript. What is loaded is said in `declared` and in the
|
|
272
282
|
// `load_tools` result instead. The preselected first step is the one exception, by design.
|
|
@@ -275,7 +285,7 @@ export async function runAgentLoop(options) {
|
|
|
275
285
|
...(prompt ? [{ role: "system", content: prompt }] : []),
|
|
276
286
|
...withContext(messages, question ? messages.indexOf(question) : -1, gathered.context, hooks?.preface),
|
|
277
287
|
];
|
|
278
|
-
const build = (supported, refused) => buildBody(config, supported, refused, request, declared);
|
|
288
|
+
const build = (supported, refused) => buildBody(config, supported, refused, request, declared, toolOrder);
|
|
279
289
|
const turnOptions = {
|
|
280
290
|
model: config.model,
|
|
281
291
|
droppable: Object.keys(config.extraBody ?? {}),
|
|
@@ -427,7 +437,7 @@ export async function runAgentLoop(options) {
|
|
|
427
437
|
used.add(name);
|
|
428
438
|
const request = { id: call.id, name, args, raw };
|
|
429
439
|
content = parallel
|
|
430
|
-
? await once(answered, `${name}
|
|
440
|
+
? await once(answered, `${name}\0${normal}`, () => dispatch(request, signal))
|
|
431
441
|
: await dispatch(request, signal);
|
|
432
442
|
}
|
|
433
443
|
}
|
package/dist/compaction.d.ts
CHANGED
|
@@ -70,6 +70,16 @@ export interface CompactionOptions {
|
|
|
70
70
|
* an `estimate` of the caller's own.
|
|
71
71
|
*/
|
|
72
72
|
charsPerToken?: number;
|
|
73
|
+
/**
|
|
74
|
+
* The first message that may be folded. Absent, the leading `system` messages are skipped and
|
|
75
|
+
* the fold starts after them.
|
|
76
|
+
*/
|
|
77
|
+
from?: number;
|
|
78
|
+
/**
|
|
79
|
+
* The summary an earlier fold left, which this one continues. Absent, it is recovered from a
|
|
80
|
+
* `SUMMARY_LEAD` system message at the head, if there is one.
|
|
81
|
+
*/
|
|
82
|
+
previous?: string;
|
|
73
83
|
}
|
|
74
84
|
/** Where to cut, as `compactTranscript` takes it. */
|
|
75
85
|
export interface CompactionPlan {
|
|
@@ -92,10 +102,16 @@ export interface CompactionPlan {
|
|
|
92
102
|
* than summarised as if it were conversation. No plan comes back when the window is not full
|
|
93
103
|
* enough, or when the only legal cut folds too little to pay for the summary.
|
|
94
104
|
*
|
|
105
|
+
* Both of those are recovered by reading the transcript, which is what a host whose array holds
|
|
106
|
+
* everything it sends has to do. A host that keeps its fold as a record beside an append-only
|
|
107
|
+
* transcript — its system prompt a separate argument, no summary message in the array at all —
|
|
108
|
+
* knows them exactly, and passes `from` and `previous` instead of hoping the scan agrees.
|
|
109
|
+
*
|
|
95
110
|
* @param messages The transcript, system prompts included if the caller keeps them in it.
|
|
96
|
-
* @param options The window, what is in use, and the
|
|
111
|
+
* @param options The window, what is in use, the ratios, and where the last fold ended. See
|
|
112
|
+
* `CompactionOptions`.
|
|
97
113
|
*/
|
|
98
|
-
export declare function planCompaction(messages: Message[], { limit, used, compactAt, keepRatio, charsPerToken, estimate, }: CompactionOptions): CompactionPlan | undefined;
|
|
114
|
+
export declare function planCompaction(messages: Message[], { limit, used, compactAt, keepRatio, charsPerToken, estimate, from: givenFrom, previous: givenPrevious, }: CompactionOptions): CompactionPlan | undefined;
|
|
99
115
|
/**
|
|
100
116
|
* What the summariser is handed for a plan: the earlier summary if there was one, then each
|
|
101
117
|
* message as its role and at most 4000 characters of its text.
|
|
@@ -114,6 +130,93 @@ export declare function summaryInput(plan: CompactionPlan): string;
|
|
|
114
130
|
export declare const summariser: (config: Endpoint, model: string, { system, maxTokens, ...options }?: SideTaskOptions & {
|
|
115
131
|
system?: string;
|
|
116
132
|
}) => (text: string) => Promise<string>;
|
|
133
|
+
/**
|
|
134
|
+
* One fold, as a host that keeps its transcript append-only stores it.
|
|
135
|
+
*
|
|
136
|
+
* The other half of `compactTranscript`: the same work, recorded rather than applied. A host that
|
|
137
|
+
* persists this beside an untouched transcript still shows the user every message, can undo a fold
|
|
138
|
+
* by dropping one row, and rebuilds the request with `applyCompaction` — where a host that keeps
|
|
139
|
+
* only the rewritten array has thrown the originals away.
|
|
140
|
+
*/
|
|
141
|
+
export interface CompactionRecord {
|
|
142
|
+
/** The model's notes on everything before `through`. Trimmed, and never empty. */
|
|
143
|
+
summary: string;
|
|
144
|
+
/** Index into the transcript the plan was made for: the first message still sent whole. */
|
|
145
|
+
through: number;
|
|
146
|
+
/** ISO 8601, so the chat can show where history was folded and how stale the notes are. */
|
|
147
|
+
at: string;
|
|
148
|
+
}
|
|
149
|
+
/** What `runCompaction` and `compactTranscript` take beside the plan. */
|
|
150
|
+
export interface CompactionRunOptions {
|
|
151
|
+
/**
|
|
152
|
+
* Hooks to tell. `context` is extended with `compacting` and `range`, whose indexes are the
|
|
153
|
+
* plan's — and so the host's own, for a plan made over a stored transcript. `honourVeto` waits
|
|
154
|
+
* for the hooks and lets one stop the compaction; off by default, which adds no latency.
|
|
155
|
+
*/
|
|
156
|
+
hooks?: {
|
|
157
|
+
run: HookRunner;
|
|
158
|
+
context: HookContext;
|
|
159
|
+
onNote?: (note: HookNote) => void;
|
|
160
|
+
honourVeto?: boolean;
|
|
161
|
+
};
|
|
162
|
+
/**
|
|
163
|
+
* The window is already exceeded — the caller caught a `ContextOverflow`, or is compacting to
|
|
164
|
+
* make a refused request fit — which overrides `honourVeto`.
|
|
165
|
+
*/
|
|
166
|
+
forced?: boolean;
|
|
167
|
+
}
|
|
168
|
+
/**
|
|
169
|
+
* The hooks and the summariser for a plan, as a record to store rather than a transcript to send.
|
|
170
|
+
*
|
|
171
|
+
* What `compactTranscript` does before it rewrites anything, which is all a host needs when the
|
|
172
|
+
* fold lives on the session row and the messages stay where they are. Nothing here is persisted or
|
|
173
|
+
* logged — that is the host's, and so is deciding what to do with a fold that did not happen.
|
|
174
|
+
*
|
|
175
|
+
* @param messages The transcript the plan was made for. Read only, and only for the hooks.
|
|
176
|
+
* @param plan What `planCompaction` returned for it.
|
|
177
|
+
* @param summarise Writes the summary from `summaryInput`'s text. See `summariser`. Not called
|
|
178
|
+
* when a hook vetoes.
|
|
179
|
+
* @param options Hooks to tell and whether the window is already past. See `CompactionRunOptions`.
|
|
180
|
+
* @returns `undefined` when nothing was folded — a hook vetoed, or the summary came back empty —
|
|
181
|
+
* so the caller stores nothing and the transcript is still whole.
|
|
182
|
+
*/
|
|
183
|
+
export declare function runCompaction(messages: Message[], plan: CompactionPlan, summarise: (text: string) => Promise<string>, { hooks, forced }?: CompactionRunOptions): Promise<CompactionRecord | undefined>;
|
|
184
|
+
/**
|
|
185
|
+
* The transcript as the server should see it: the folded head replaced by its summary.
|
|
186
|
+
*
|
|
187
|
+
* The inverse of storing a `CompactionRecord`, and the shape `planCompaction` expects to meet
|
|
188
|
+
* again — the same `SUMMARY_LEAD`, in a `system` message at the same place — so the next fold
|
|
189
|
+
* continues these notes rather than summarising them a second time. Any earlier summary message in
|
|
190
|
+
* the kept head is dropped, since the record's already contains it.
|
|
191
|
+
*
|
|
192
|
+
* @param messages The stored transcript, whole. Not written to.
|
|
193
|
+
* @param record The fold, or `undefined` for a session that has not been compacted, which hands
|
|
194
|
+
* back `messages` itself.
|
|
195
|
+
* @param options `from` is the first message the fold was allowed to take — the plan's, for a host
|
|
196
|
+
* that keeps its system prompts in the array; everything before it is kept ahead of the summary.
|
|
197
|
+
* Absent, the leading `system` messages are found by scanning, and zero of them is the ordinary
|
|
198
|
+
* case for a host whose system prompt is a separate argument.
|
|
199
|
+
*/
|
|
200
|
+
export declare function applyCompaction(messages: Message[], record?: Pick<CompactionRecord, "summary" | "through">, { from }?: {
|
|
201
|
+
from?: number;
|
|
202
|
+
}): Message[];
|
|
203
|
+
/**
|
|
204
|
+
* Where a stored index sits in the request `applyCompaction` builds, once a fold has shifted
|
|
205
|
+
* everything after it.
|
|
206
|
+
*
|
|
207
|
+
* A transcript that stays append-only and a request that does not are two numberings of the same
|
|
208
|
+
* conversation, and anything that names a position — `withContext`'s index, a range handed to a
|
|
209
|
+
* hook — has to say which it is in. An index inside the folded stretch answers with the summary
|
|
210
|
+
* message that now stands for it.
|
|
211
|
+
*
|
|
212
|
+
* @param index The position in the stored transcript.
|
|
213
|
+
* @param record The fold in force, or `undefined` for a session that has none, which hands the
|
|
214
|
+
* index straight back.
|
|
215
|
+
* @param head How many messages the request keeps ahead of the summary — the leading system
|
|
216
|
+
* prompts, when the host keeps them in the array. Zero, the default, is the stored-fold case,
|
|
217
|
+
* where the summary is the request's first message.
|
|
218
|
+
*/
|
|
219
|
+
export declare const requestIndex: (index: number, record?: Pick<CompactionRecord, "through">, head?: number) => number;
|
|
117
220
|
/**
|
|
118
221
|
* The transcript with the plan's stretch replaced by one system message holding its summary.
|
|
119
222
|
*
|
|
@@ -127,25 +230,18 @@ export declare const summariser: (config: Endpoint, model: string, { system, max
|
|
|
127
230
|
* `ContextOverflow`. An empty summary folds nothing either. Rewrites the prefix; see the module
|
|
128
231
|
* comment on when to run it.
|
|
129
232
|
*
|
|
233
|
+
* `runCompaction` and `applyCompaction` are its two halves, and it is nothing but the two in
|
|
234
|
+
* order, so a host that stores the fold instead of the array gets the same summary at the same
|
|
235
|
+
* cut rather than a second implementation that drifts from this one.
|
|
236
|
+
*
|
|
130
237
|
* @param messages The transcript the plan was made for. Not written to.
|
|
131
238
|
* @param plan What `planCompaction` returned for it.
|
|
132
239
|
* @param summarise Writes the summary from `summaryInput`'s text. See `summariser`. Not called
|
|
133
240
|
* when a hook vetoes.
|
|
134
|
-
* @param options Hooks to tell
|
|
135
|
-
*
|
|
136
|
-
* by default, which adds no latency. `forced` says the window is already exceeded — the caller
|
|
137
|
-
* caught a `ContextOverflow`, or is compacting to make a refused request fit — and overrides
|
|
138
|
-
* `honourVeto`.
|
|
241
|
+
* @param options Hooks to tell and whether the window is already past. See
|
|
242
|
+
* `CompactionRunOptions`.
|
|
139
243
|
* @returns `messages` itself when nothing was folded — a veto or an empty summary — otherwise a
|
|
140
244
|
* new array.
|
|
141
245
|
*/
|
|
142
|
-
export declare function compactTranscript(messages: Message[], plan: CompactionPlan, summarise: (text: string) => Promise<string>,
|
|
143
|
-
hooks?: {
|
|
144
|
-
run: HookRunner;
|
|
145
|
-
context: HookContext;
|
|
146
|
-
onNote?: (note: HookNote) => void;
|
|
147
|
-
honourVeto?: boolean;
|
|
148
|
-
};
|
|
149
|
-
forced?: boolean;
|
|
150
|
-
}): Promise<Message[]>;
|
|
246
|
+
export declare function compactTranscript(messages: Message[], plan: CompactionPlan, summarise: (text: string) => Promise<string>, options?: CompactionRunOptions): Promise<Message[]>;
|
|
151
247
|
export {};
|
package/dist/compaction.js
CHANGED
|
@@ -37,6 +37,25 @@ const messageText = (message) => {
|
|
|
37
37
|
return `${textOf(message.content)} ${calls}`.trim();
|
|
38
38
|
};
|
|
39
39
|
const isSummary = (message) => message.role === "system" && textOf(message.content).startsWith(SUMMARY_LEAD);
|
|
40
|
+
/** The summary as it sits in a transcript, which is the one shape `isSummary` recognises again. */
|
|
41
|
+
const summaryMessage = (summary) => ({
|
|
42
|
+
role: "system",
|
|
43
|
+
content: `${SUMMARY_LEAD}${summary.trim()}`,
|
|
44
|
+
});
|
|
45
|
+
/**
|
|
46
|
+
* The leading system messages a fold never touches, and the summary an earlier one left among
|
|
47
|
+
* them. What a caller who keeps its system prompt out of the array knows already, and passes.
|
|
48
|
+
*/
|
|
49
|
+
const systemHead = (messages) => {
|
|
50
|
+
let from = 0;
|
|
51
|
+
let previous;
|
|
52
|
+
while (from < messages.length && messages[from].role === "system") {
|
|
53
|
+
if (isSummary(messages[from]))
|
|
54
|
+
previous = textOf(messages[from].content).slice(SUMMARY_LEAD.length);
|
|
55
|
+
from++;
|
|
56
|
+
}
|
|
57
|
+
return { from, previous };
|
|
58
|
+
};
|
|
40
59
|
/**
|
|
41
60
|
* The transcript with every tool result but the latest few replaced by a one-line stub.
|
|
42
61
|
*
|
|
@@ -80,22 +99,24 @@ export function pruneToolResults(messages, { keepLast = 5, maxChars = 256 } = {}
|
|
|
80
99
|
* than summarised as if it were conversation. No plan comes back when the window is not full
|
|
81
100
|
* enough, or when the only legal cut folds too little to pay for the summary.
|
|
82
101
|
*
|
|
102
|
+
* Both of those are recovered by reading the transcript, which is what a host whose array holds
|
|
103
|
+
* everything it sends has to do. A host that keeps its fold as a record beside an append-only
|
|
104
|
+
* transcript — its system prompt a separate argument, no summary message in the array at all —
|
|
105
|
+
* knows them exactly, and passes `from` and `previous` instead of hoping the scan agrees.
|
|
106
|
+
*
|
|
83
107
|
* @param messages The transcript, system prompts included if the caller keeps them in it.
|
|
84
|
-
* @param options The window, what is in use, and the
|
|
108
|
+
* @param options The window, what is in use, the ratios, and where the last fold ended. See
|
|
109
|
+
* `CompactionOptions`.
|
|
85
110
|
*/
|
|
86
|
-
export function planCompaction(messages, { limit, used, compactAt = COMPACT_AT, keepRatio = KEEP_RATIO, charsPerToken, estimate = (message) => messageTokens(message, { charsPerToken }), }) {
|
|
111
|
+
export function planCompaction(messages, { limit, used, compactAt = COMPACT_AT, keepRatio = KEEP_RATIO, charsPerToken, estimate = (message) => messageTokens(message, { charsPerToken }), from: givenFrom, previous: givenPrevious, }) {
|
|
87
112
|
if (!(limit > 0))
|
|
88
113
|
return undefined;
|
|
89
114
|
const cost = used ?? messages.reduce((total, message) => total + estimate(message), 0);
|
|
90
115
|
if (cost < limit * compactAt)
|
|
91
116
|
return undefined;
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
if (isSummary(messages[from]))
|
|
96
|
-
previous = textOf(messages[from].content).slice(SUMMARY_LEAD.length);
|
|
97
|
-
from++;
|
|
98
|
-
}
|
|
117
|
+
const head = systemHead(messages);
|
|
118
|
+
const from = Math.min(Math.max(givenFrom ?? head.from, 0), messages.length);
|
|
119
|
+
const previous = givenPrevious ?? head.previous;
|
|
99
120
|
const budget = limit * keepRatio;
|
|
100
121
|
let kept = 0;
|
|
101
122
|
let cut = messages.length;
|
|
@@ -139,31 +160,21 @@ export function summaryInput(plan) {
|
|
|
139
160
|
*/
|
|
140
161
|
export const summariser = (config, model, { system = SUMMARY_PROMPT, maxTokens = 1024, ...options } = {}) => (text) => ask(config, model, system, text, { maxTokens, ...options });
|
|
141
162
|
/**
|
|
142
|
-
* The
|
|
163
|
+
* The hooks and the summariser for a plan, as a record to store rather than a transcript to send.
|
|
143
164
|
*
|
|
144
|
-
* `
|
|
145
|
-
*
|
|
146
|
-
*
|
|
147
|
-
* `honourVeto`, and then they run first and the summary waits on them: any `ok` outcome carrying
|
|
148
|
-
* `veto` leaves the transcript as it was, and each vetoing hook is noted by name. A `forced`
|
|
149
|
-
* compaction ignores a veto and runs the hooks beside the summary as before, because a run already
|
|
150
|
-
* past its window has no better option — a veto there only trades the summary for a
|
|
151
|
-
* `ContextOverflow`. An empty summary folds nothing either. Rewrites the prefix; see the module
|
|
152
|
-
* comment on when to run it.
|
|
165
|
+
* What `compactTranscript` does before it rewrites anything, which is all a host needs when the
|
|
166
|
+
* fold lives on the session row and the messages stay where they are. Nothing here is persisted or
|
|
167
|
+
* logged — that is the host's, and so is deciding what to do with a fold that did not happen.
|
|
153
168
|
*
|
|
154
|
-
* @param messages The transcript the plan was made for.
|
|
169
|
+
* @param messages The transcript the plan was made for. Read only, and only for the hooks.
|
|
155
170
|
* @param plan What `planCompaction` returned for it.
|
|
156
171
|
* @param summarise Writes the summary from `summaryInput`'s text. See `summariser`. Not called
|
|
157
172
|
* when a hook vetoes.
|
|
158
|
-
* @param options Hooks to tell
|
|
159
|
-
*
|
|
160
|
-
*
|
|
161
|
-
* caught a `ContextOverflow`, or is compacting to make a refused request fit — and overrides
|
|
162
|
-
* `honourVeto`.
|
|
163
|
-
* @returns `messages` itself when nothing was folded — a veto or an empty summary — otherwise a
|
|
164
|
-
* new array.
|
|
173
|
+
* @param options Hooks to tell and whether the window is already past. See `CompactionRunOptions`.
|
|
174
|
+
* @returns `undefined` when nothing was folded — a hook vetoed, or the summary came back empty —
|
|
175
|
+
* so the caller stores nothing and the transcript is still whole.
|
|
165
176
|
*/
|
|
166
|
-
export async function
|
|
177
|
+
export async function runCompaction(messages, plan, summarise, { hooks, forced = false } = {}) {
|
|
167
178
|
const context = hooks && {
|
|
168
179
|
...hooks.context,
|
|
169
180
|
compacting: turnMessages(hooks.context.session.id, messages, plan.from, plan.cut),
|
|
@@ -173,7 +184,7 @@ export async function compactTranscript(messages, plan, summarise, { hooks, forc
|
|
|
173
184
|
if (hooks && context && hooks.honourVeto && !forced) {
|
|
174
185
|
const { vetoed } = await consult(hooks.run, "beforeCompact", context, hooks.onNote);
|
|
175
186
|
if (vetoed)
|
|
176
|
-
return
|
|
187
|
+
return undefined;
|
|
177
188
|
summary = await summarise(summaryInput(plan));
|
|
178
189
|
}
|
|
179
190
|
else {
|
|
@@ -183,10 +194,85 @@ export async function compactTranscript(messages, plan, summarise, { hooks, forc
|
|
|
183
194
|
]);
|
|
184
195
|
}
|
|
185
196
|
if (!summary.trim())
|
|
197
|
+
return undefined;
|
|
198
|
+
return { summary: summary.trim(), through: plan.cut, at: new Date().toISOString() };
|
|
199
|
+
}
|
|
200
|
+
/**
|
|
201
|
+
* The transcript as the server should see it: the folded head replaced by its summary.
|
|
202
|
+
*
|
|
203
|
+
* The inverse of storing a `CompactionRecord`, and the shape `planCompaction` expects to meet
|
|
204
|
+
* again — the same `SUMMARY_LEAD`, in a `system` message at the same place — so the next fold
|
|
205
|
+
* continues these notes rather than summarising them a second time. Any earlier summary message in
|
|
206
|
+
* the kept head is dropped, since the record's already contains it.
|
|
207
|
+
*
|
|
208
|
+
* @param messages The stored transcript, whole. Not written to.
|
|
209
|
+
* @param record The fold, or `undefined` for a session that has not been compacted, which hands
|
|
210
|
+
* back `messages` itself.
|
|
211
|
+
* @param options `from` is the first message the fold was allowed to take — the plan's, for a host
|
|
212
|
+
* that keeps its system prompts in the array; everything before it is kept ahead of the summary.
|
|
213
|
+
* Absent, the leading `system` messages are found by scanning, and zero of them is the ordinary
|
|
214
|
+
* case for a host whose system prompt is a separate argument.
|
|
215
|
+
*/
|
|
216
|
+
export function applyCompaction(messages, record, { from } = {}) {
|
|
217
|
+
if (!record?.summary.trim())
|
|
186
218
|
return messages;
|
|
219
|
+
const head = Math.min(Math.max(from ?? systemHead(messages).from, 0), record.through);
|
|
187
220
|
return [
|
|
188
|
-
...messages.slice(0,
|
|
189
|
-
|
|
190
|
-
...messages.slice(
|
|
221
|
+
...messages.slice(0, head).filter((message) => !isSummary(message)),
|
|
222
|
+
summaryMessage(record.summary),
|
|
223
|
+
...messages.slice(record.through),
|
|
191
224
|
];
|
|
192
225
|
}
|
|
226
|
+
/**
|
|
227
|
+
* Where a stored index sits in the request `applyCompaction` builds, once a fold has shifted
|
|
228
|
+
* everything after it.
|
|
229
|
+
*
|
|
230
|
+
* A transcript that stays append-only and a request that does not are two numberings of the same
|
|
231
|
+
* conversation, and anything that names a position — `withContext`'s index, a range handed to a
|
|
232
|
+
* hook — has to say which it is in. An index inside the folded stretch answers with the summary
|
|
233
|
+
* message that now stands for it.
|
|
234
|
+
*
|
|
235
|
+
* @param index The position in the stored transcript.
|
|
236
|
+
* @param record The fold in force, or `undefined` for a session that has none, which hands the
|
|
237
|
+
* index straight back.
|
|
238
|
+
* @param head How many messages the request keeps ahead of the summary — the leading system
|
|
239
|
+
* prompts, when the host keeps them in the array. Zero, the default, is the stored-fold case,
|
|
240
|
+
* where the summary is the request's first message.
|
|
241
|
+
*/
|
|
242
|
+
export const requestIndex = (index, record, head = 0) => {
|
|
243
|
+
if (!record)
|
|
244
|
+
return index;
|
|
245
|
+
return index < record.through ? head : index - record.through + head + 1;
|
|
246
|
+
};
|
|
247
|
+
/**
|
|
248
|
+
* The transcript with the plan's stretch replaced by one system message holding its summary.
|
|
249
|
+
*
|
|
250
|
+
* `beforeCompact` is told what is being folded while the summary is written, beside it rather
|
|
251
|
+
* than ahead of it — a memory server filing it is not a rescue worth making the run wait for, and
|
|
252
|
+
* `notify` never rejects. A host that wants its hooks able to stop a compaction sets
|
|
253
|
+
* `honourVeto`, and then they run first and the summary waits on them: any `ok` outcome carrying
|
|
254
|
+
* `veto` leaves the transcript as it was, and each vetoing hook is noted by name. A `forced`
|
|
255
|
+
* compaction ignores a veto and runs the hooks beside the summary as before, because a run already
|
|
256
|
+
* past its window has no better option — a veto there only trades the summary for a
|
|
257
|
+
* `ContextOverflow`. An empty summary folds nothing either. Rewrites the prefix; see the module
|
|
258
|
+
* comment on when to run it.
|
|
259
|
+
*
|
|
260
|
+
* `runCompaction` and `applyCompaction` are its two halves, and it is nothing but the two in
|
|
261
|
+
* order, so a host that stores the fold instead of the array gets the same summary at the same
|
|
262
|
+
* cut rather than a second implementation that drifts from this one.
|
|
263
|
+
*
|
|
264
|
+
* @param messages The transcript the plan was made for. Not written to.
|
|
265
|
+
* @param plan What `planCompaction` returned for it.
|
|
266
|
+
* @param summarise Writes the summary from `summaryInput`'s text. See `summariser`. Not called
|
|
267
|
+
* when a hook vetoes.
|
|
268
|
+
* @param options Hooks to tell and whether the window is already past. See
|
|
269
|
+
* `CompactionRunOptions`.
|
|
270
|
+
* @returns `messages` itself when nothing was folded — a veto or an empty summary — otherwise a
|
|
271
|
+
* new array.
|
|
272
|
+
*/
|
|
273
|
+
export async function compactTranscript(messages, plan, summarise, options = {}) {
|
|
274
|
+
const record = await runCompaction(messages, plan, summarise, options);
|
|
275
|
+
if (!record)
|
|
276
|
+
return messages;
|
|
277
|
+
return applyCompaction(messages, record, { from: plan.from });
|
|
278
|
+
}
|
package/dist/hooks.d.ts
CHANGED
|
@@ -256,26 +256,44 @@ export declare function untrusted(text: string, { source }?: {
|
|
|
256
256
|
* two identical "ok"s one memory. The same message sent twice, after its turn and again when it
|
|
257
257
|
* is compacted, is one.
|
|
258
258
|
*
|
|
259
|
+
* That position is the array's, which is the host's only while the two agree. Once a compaction
|
|
260
|
+
* has folded the head of a request into one summary message, the same message sits at a lower
|
|
261
|
+
* index there than it does in a transcript the host kept whole — so the same turn, sent after it
|
|
262
|
+
* and again when it is compacted, would arrive under two uuids and be remembered twice. `offset`
|
|
263
|
+
* is what puts the numbering back on the host's own indexes; passing the stored transcript rather
|
|
264
|
+
* than the request avoids the question entirely.
|
|
265
|
+
*
|
|
259
266
|
* @param sessionId Prefixes every uuid, so two sessions never share one.
|
|
260
267
|
* @param messages The transcript, in whatever shape the host stores it, so long as each message
|
|
261
268
|
* has an OpenAI-style `role` and `content`.
|
|
262
269
|
* @param from The first index, inclusive. Below zero reads from the start.
|
|
263
270
|
* @param to The end, exclusive. Absent, or past the end, reads to the end.
|
|
271
|
+
* @param options `offset` is what the array's first message is numbered as in the uuids — the
|
|
272
|
+
* stored index of `messages[0]`, when `messages` is a request a fold has shifted. Zero by default,
|
|
273
|
+
* which numbers by position as before. `from` and `to` stay array indexes either way.
|
|
264
274
|
*/
|
|
265
275
|
export declare function turnMessages(sessionId: string, messages: readonly {
|
|
266
276
|
role: string;
|
|
267
277
|
content?: unknown;
|
|
268
|
-
}[], from: number, to?: number
|
|
278
|
+
}[], from: number, to?: number, { offset }?: {
|
|
279
|
+
offset?: number;
|
|
280
|
+
}): HookMessage[];
|
|
269
281
|
/**
|
|
270
282
|
* Which turn of a session begins at a point, from 0: the user messages ahead of it.
|
|
271
283
|
*
|
|
284
|
+
* Counted over what it is given, which is the session only while nothing has been folded away — a
|
|
285
|
+
* compacted request has lost the questions the summary now stands for, and turn eleven counting
|
|
286
|
+
* itself as turn two is the kind of thing a hook writes into a memory. Count over the stored
|
|
287
|
+
* transcript, or add what the fold took as `offset`.
|
|
288
|
+
*
|
|
272
289
|
* @param messages The transcript.
|
|
273
290
|
* @param before Where the turn begins. Absent is the end, which is the index of a turn whose
|
|
274
291
|
* question has not been appended yet.
|
|
292
|
+
* @param offset Turns already folded away and so not in `messages`. Zero by default.
|
|
275
293
|
*/
|
|
276
294
|
export declare const turnIndex: (messages: readonly {
|
|
277
295
|
role: string;
|
|
278
|
-
}[], before?: number) => number;
|
|
296
|
+
}[], before?: number, offset?: number) => number;
|
|
279
297
|
/**
|
|
280
298
|
* Runs the hooks ahead of a request and builds what they add to it.
|
|
281
299
|
*
|
package/dist/hooks.js
CHANGED
|
@@ -204,13 +204,23 @@ const textOf = (content) => {
|
|
|
204
204
|
* two identical "ok"s one memory. The same message sent twice, after its turn and again when it
|
|
205
205
|
* is compacted, is one.
|
|
206
206
|
*
|
|
207
|
+
* That position is the array's, which is the host's only while the two agree. Once a compaction
|
|
208
|
+
* has folded the head of a request into one summary message, the same message sits at a lower
|
|
209
|
+
* index there than it does in a transcript the host kept whole — so the same turn, sent after it
|
|
210
|
+
* and again when it is compacted, would arrive under two uuids and be remembered twice. `offset`
|
|
211
|
+
* is what puts the numbering back on the host's own indexes; passing the stored transcript rather
|
|
212
|
+
* than the request avoids the question entirely.
|
|
213
|
+
*
|
|
207
214
|
* @param sessionId Prefixes every uuid, so two sessions never share one.
|
|
208
215
|
* @param messages The transcript, in whatever shape the host stores it, so long as each message
|
|
209
216
|
* has an OpenAI-style `role` and `content`.
|
|
210
217
|
* @param from The first index, inclusive. Below zero reads from the start.
|
|
211
218
|
* @param to The end, exclusive. Absent, or past the end, reads to the end.
|
|
219
|
+
* @param options `offset` is what the array's first message is numbered as in the uuids — the
|
|
220
|
+
* stored index of `messages[0]`, when `messages` is a request a fold has shifted. Zero by default,
|
|
221
|
+
* which numbers by position as before. `from` and `to` stay array indexes either way.
|
|
212
222
|
*/
|
|
213
|
-
export function turnMessages(sessionId, messages, from, to) {
|
|
223
|
+
export function turnMessages(sessionId, messages, from, to, { offset = 0 } = {}) {
|
|
214
224
|
const end = Math.min(to ?? messages.length, messages.length);
|
|
215
225
|
const out = [];
|
|
216
226
|
for (let at = Math.max(0, from); at < end; at++) {
|
|
@@ -221,18 +231,28 @@ export function turnMessages(sessionId, messages, from, to) {
|
|
|
221
231
|
if (!text)
|
|
222
232
|
continue;
|
|
223
233
|
const digest = createHash("sha256").update(`${message.role}\0${text}`).digest("hex");
|
|
224
|
-
out.push({
|
|
234
|
+
out.push({
|
|
235
|
+
speaker: message.role,
|
|
236
|
+
text,
|
|
237
|
+
uuid: `${sessionId}:${at + offset}:${digest.slice(0, 12)}`,
|
|
238
|
+
});
|
|
225
239
|
}
|
|
226
240
|
return out;
|
|
227
241
|
}
|
|
228
242
|
/**
|
|
229
243
|
* Which turn of a session begins at a point, from 0: the user messages ahead of it.
|
|
230
244
|
*
|
|
245
|
+
* Counted over what it is given, which is the session only while nothing has been folded away — a
|
|
246
|
+
* compacted request has lost the questions the summary now stands for, and turn eleven counting
|
|
247
|
+
* itself as turn two is the kind of thing a hook writes into a memory. Count over the stored
|
|
248
|
+
* transcript, or add what the fold took as `offset`.
|
|
249
|
+
*
|
|
231
250
|
* @param messages The transcript.
|
|
232
251
|
* @param before Where the turn begins. Absent is the end, which is the index of a turn whose
|
|
233
252
|
* question has not been appended yet.
|
|
253
|
+
* @param offset Turns already folded away and so not in `messages`. Zero by default.
|
|
234
254
|
*/
|
|
235
|
-
export const turnIndex = (messages, before = messages.length) => messages.slice(0, before).filter((message) => message.role === "user").length;
|
|
255
|
+
export const turnIndex = (messages, before = messages.length, offset = 0) => offset + messages.slice(0, before).filter((message) => message.role === "user").length;
|
|
236
256
|
/** A runner that rejected, as the one outcome its event can still be noted by. */
|
|
237
257
|
const rejected = (event, error) => ({
|
|
238
258
|
serverId: "",
|
package/dist/index.d.ts
CHANGED
|
@@ -14,7 +14,7 @@ export { calibrate, charsPerTokenFor, resetCalibration } from "./calibration.ts"
|
|
|
14
14
|
export { type Capabilities, capabilitiesFor, type ModelCapabilities, modelCapabilitiesFor, type NegotiateOptions, negotiate, resetCapabilities, } from "./capabilities.ts";
|
|
15
15
|
export type { CatalogServer } from "./catalog.ts";
|
|
16
16
|
export { type ClientPoolOptions, configureClients, contextLimitFor, endpointId, endpointKey, FIRST_TOKEN_FACTOR, firstTokenMs, getClient, listModels, type ModelInfo, NO_KEY, resetClients, servedWindow, timeoutMs, } from "./client.ts";
|
|
17
|
-
export { COMPACT_AT, type CompactionOptions, type CompactionPlan, compactTranscript, KEEP_RATIO, type PruneOptions, planCompaction, pruneToolResults, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.ts";
|
|
17
|
+
export { applyCompaction, COMPACT_AT, type CompactionOptions, type CompactionPlan, type CompactionRecord, type CompactionRunOptions, compactTranscript, KEEP_RATIO, type PruneOptions, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.ts";
|
|
18
18
|
export type { AgentConfig, Endpoint, ModelParams, RetryPolicy, ToolPolicy, } from "./config.ts";
|
|
19
19
|
export { type ContinueTurnOptions, continueTurn, isContinuable, } from "./continuation.ts";
|
|
20
20
|
export { errorMessage } from "./errors.ts";
|
|
@@ -30,4 +30,4 @@ export { type Produced, type StreamTurnOptions, streamTurn, type Turn, type Turn
|
|
|
30
30
|
export { ALL_FENCES, DEFAULT_FENCES, type Fence, FenceSplitter, type FenceSplitterOptions, type Split, stripThinking, THINK_FENCE, } from "./thinking.ts";
|
|
31
31
|
export { estimateTokens } from "./tokens.ts";
|
|
32
32
|
export { parseToolArguments, recoverToolCalls, ToolArgumentsError, type ToolCall, } from "./tool-calls.ts";
|
|
33
|
-
export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.ts";
|
|
33
|
+
export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, type ToolOrder, } from "./tool-loading.ts";
|
package/dist/index.js
CHANGED
|
@@ -13,7 +13,7 @@ export { buildBody, preselect, preview, resolveApiKey, runAgentLoop, } from "./a
|
|
|
13
13
|
export { calibrate, charsPerTokenFor, resetCalibration } from "./calibration.js";
|
|
14
14
|
export { capabilitiesFor, modelCapabilitiesFor, negotiate, resetCapabilities, } from "./capabilities.js";
|
|
15
15
|
export { configureClients, contextLimitFor, endpointId, endpointKey, FIRST_TOKEN_FACTOR, firstTokenMs, getClient, listModels, NO_KEY, resetClients, servedWindow, timeoutMs, } from "./client.js";
|
|
16
|
-
export { COMPACT_AT, compactTranscript, KEEP_RATIO, planCompaction, pruneToolResults, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.js";
|
|
16
|
+
export { applyCompaction, COMPACT_AT, compactTranscript, KEEP_RATIO, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, summaryInput, } from "./compaction.js";
|
|
17
17
|
export { continueTurn, isContinuable, } from "./continuation.js";
|
|
18
18
|
export { errorMessage } from "./errors.js";
|
|
19
19
|
export { configureEvents, emit, endRun, fold, history, resetEvents, runMetrics, watch, } from "./events.js";
|
|
@@ -28,4 +28,4 @@ export { streamTurn, } from "./stream.js";
|
|
|
28
28
|
export { ALL_FENCES, DEFAULT_FENCES, FenceSplitter, stripThinking, THINK_FENCE, } from "./thinking.js";
|
|
29
29
|
export { estimateTokens } from "./tokens.js";
|
|
30
30
|
export { parseToolArguments, recoverToolCalls, ToolArgumentsError, } from "./tool-calls.js";
|
|
31
|
-
export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
|
|
31
|
+
export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
|
package/dist/tool-loading.d.ts
CHANGED
|
@@ -56,16 +56,41 @@ export declare function catalogPrompt(catalog: CatalogServer[], loaded?: Readonl
|
|
|
56
56
|
/**
|
|
57
57
|
* A tool array with newly loaded definitions appended, in the order they were loaded.
|
|
58
58
|
*
|
|
59
|
-
*
|
|
60
|
-
*
|
|
61
|
-
*
|
|
62
|
-
* a prefix of the new array.
|
|
59
|
+
* Appended and never rebuilt from a set, so what a load adds is decided by the load and not by
|
|
60
|
+
* the shape of whatever collection the definitions came out of. Where the appended array ends up
|
|
61
|
+
* in the request is `orderTools`' business, which the loop applies after this.
|
|
63
62
|
*
|
|
64
63
|
* @param previous What the last request declared, `load_tools` included. Not written to.
|
|
65
64
|
* @param matched The definitions to add. Ones whose name is already declared, here or earlier in
|
|
66
65
|
* this list, are skipped rather than moved.
|
|
67
66
|
*/
|
|
68
67
|
export declare function loadedTools(previous: readonly OpenAI.ChatCompletionTool[], matched: readonly OpenAI.ChatCompletionTool[]): OpenAI.ChatCompletionTool[];
|
|
68
|
+
/**
|
|
69
|
+
* How a tool array is ordered before it is sent: `true` by name, `false` as the caller built it,
|
|
70
|
+
* or a comparator over the two names.
|
|
71
|
+
*/
|
|
72
|
+
export type ToolOrder = boolean | ((a: string, b: string) => number);
|
|
73
|
+
/**
|
|
74
|
+
* The tool array in a stable order, so the same set of tools renders the same way twice.
|
|
75
|
+
*
|
|
76
|
+
* A chat template renders the declared tools ahead of the system prompt, which makes the tool
|
|
77
|
+
* array the first thing a prompt cache has to match — and an array assembled from a map, from
|
|
78
|
+
* database rows, or from the order servers happened to connect in changes between processes and
|
|
79
|
+
* between reconnects. Every such change costs the cache for the whole transcript rather than for
|
|
80
|
+
* the tools alone, and nothing about the request the model sees is different. Ordering by name
|
|
81
|
+
* makes the array a property of the set instead of of how it was built, at the price of a load
|
|
82
|
+
* inserting rather than appending. Definitions come back by identity, so `sanitizeTools` still
|
|
83
|
+
* finds each one in its cache.
|
|
84
|
+
*
|
|
85
|
+
* `false` is for a caller that means its order: the model reads the array top to bottom, and a
|
|
86
|
+
* host may be putting what it wants reached for first at the front.
|
|
87
|
+
*
|
|
88
|
+
* @param tools The definitions to order. Not written to.
|
|
89
|
+
* @param order `true` for name order, `false` to leave it alone, or a comparator over the names.
|
|
90
|
+
* A tool that is not a function orders as the empty name.
|
|
91
|
+
* @returns `tools` itself when it is already in that order, so the common case copies nothing.
|
|
92
|
+
*/
|
|
93
|
+
export declare function orderTools(tools: OpenAI.ChatCompletionTool[], order?: ToolOrder): OpenAI.ChatCompletionTool[];
|
|
69
94
|
/**
|
|
70
95
|
* The most a single `load_tools` call may pull in.
|
|
71
96
|
*
|
package/dist/tool-loading.js
CHANGED
|
@@ -107,10 +107,9 @@ const flatten = (catalog) => catalog.flatMap((server) => server.tools);
|
|
|
107
107
|
/**
|
|
108
108
|
* A tool array with newly loaded definitions appended, in the order they were loaded.
|
|
109
109
|
*
|
|
110
|
-
*
|
|
111
|
-
*
|
|
112
|
-
*
|
|
113
|
-
* a prefix of the new array.
|
|
110
|
+
* Appended and never rebuilt from a set, so what a load adds is decided by the load and not by
|
|
111
|
+
* the shape of whatever collection the definitions came out of. Where the appended array ends up
|
|
112
|
+
* in the request is `orderTools`' business, which the loop applies after this.
|
|
114
113
|
*
|
|
115
114
|
* @param previous What the last request declared, `load_tools` included. Not written to.
|
|
116
115
|
* @param matched The definitions to add. Ones whose name is already declared, here or earlier in
|
|
@@ -129,6 +128,36 @@ export function loadedTools(previous, matched) {
|
|
|
129
128
|
}
|
|
130
129
|
return tools;
|
|
131
130
|
}
|
|
131
|
+
/**
|
|
132
|
+
* The tool array in a stable order, so the same set of tools renders the same way twice.
|
|
133
|
+
*
|
|
134
|
+
* A chat template renders the declared tools ahead of the system prompt, which makes the tool
|
|
135
|
+
* array the first thing a prompt cache has to match — and an array assembled from a map, from
|
|
136
|
+
* database rows, or from the order servers happened to connect in changes between processes and
|
|
137
|
+
* between reconnects. Every such change costs the cache for the whole transcript rather than for
|
|
138
|
+
* the tools alone, and nothing about the request the model sees is different. Ordering by name
|
|
139
|
+
* makes the array a property of the set instead of of how it was built, at the price of a load
|
|
140
|
+
* inserting rather than appending. Definitions come back by identity, so `sanitizeTools` still
|
|
141
|
+
* finds each one in its cache.
|
|
142
|
+
*
|
|
143
|
+
* `false` is for a caller that means its order: the model reads the array top to bottom, and a
|
|
144
|
+
* host may be putting what it wants reached for first at the front.
|
|
145
|
+
*
|
|
146
|
+
* @param tools The definitions to order. Not written to.
|
|
147
|
+
* @param order `true` for name order, `false` to leave it alone, or a comparator over the names.
|
|
148
|
+
* A tool that is not a function orders as the empty name.
|
|
149
|
+
* @returns `tools` itself when it is already in that order, so the common case copies nothing.
|
|
150
|
+
*/
|
|
151
|
+
export function orderTools(tools, order = true) {
|
|
152
|
+
if (order === false)
|
|
153
|
+
return tools;
|
|
154
|
+
const nameOf = (tool) => tool.type === "function" ? tool.function.name : "";
|
|
155
|
+
// Code-unit order rather than `localeCompare`, whose answer depends on the host's locale —
|
|
156
|
+
// which is the kind of instability this exists to remove.
|
|
157
|
+
const compare = typeof order === "function" ? order : (a, b) => (a < b ? -1 : a > b ? 1 : 0);
|
|
158
|
+
const sorted = [...tools].sort((a, b) => compare(nameOf(a), nameOf(b)));
|
|
159
|
+
return sorted.some((tool, at) => tool !== tools[at]) ? sorted : tools;
|
|
160
|
+
}
|
|
132
161
|
/**
|
|
133
162
|
* The most a single `load_tools` call may pull in.
|
|
134
163
|
*
|
package/llms.txt
CHANGED
|
@@ -71,14 +71,19 @@ The contract between whatever holds the tools and the loop that offers them to a
|
|
|
71
71
|
|
|
72
72
|
Keeping a long run inside its window: stale tool results cleared, and the oldest stretch folded into a summary the model writes itself.
|
|
73
73
|
|
|
74
|
+
- `applyCompaction` — The transcript as the server should see it: the folded head replaced by its summary.
|
|
74
75
|
- `COMPACT_AT` — The fraction of the window in use before a summary is worth its own round trip.
|
|
75
76
|
- `CompactionOptions` (type) — What `planCompaction` takes.
|
|
76
77
|
- `CompactionPlan` (type) — Where to cut, as `compactTranscript` takes it.
|
|
78
|
+
- `CompactionRecord` (type) — One fold, as a host that keeps its transcript append-only stores it.
|
|
79
|
+
- `CompactionRunOptions` (type) — What `runCompaction` and `compactTranscript` take beside the plan.
|
|
77
80
|
- `compactTranscript` — The transcript with the plan's stretch replaced by one system message holding its summary.
|
|
78
81
|
- `KEEP_RATIO` — The fraction of the window the kept tail may fill, leaving room for the run to grow again.
|
|
79
82
|
- `PruneOptions` (type) — What `pruneToolResults` takes.
|
|
80
83
|
- `planCompaction` — Where to fold a transcript that has grown into its window, or `undefined` when it should not be.
|
|
81
84
|
- `pruneToolResults` — The transcript with every tool result but the latest few replaced by a one-line stub.
|
|
85
|
+
- `requestIndex` — Where a stored index sits in the request `applyCompaction` builds, once a fold has shifted everything after it.
|
|
86
|
+
- `runCompaction` — The hooks and the summariser for a plan, as a record to store rather than a transcript to send.
|
|
82
87
|
- `SUMMARY_LEAD` — How a summary message opens, which is also how `planCompaction` knows one from a system prompt.
|
|
83
88
|
- `SUMMARY_PROMPT` — The summariser's instruction when the caller gives none.
|
|
84
89
|
- `summariser` — A summariser that asks `model` with `SUMMARY_PROMPT`, for `compactTranscript`.
|
|
@@ -271,9 +276,11 @@ Reading what a model meant by a tool call when it did not write one cleanly.
|
|
|
271
276
|
- `loadResult` — What `load_tools` reports back: the descriptions, now that they are worth their tokens.
|
|
272
277
|
- `MAX_CARRIED` — The most a conversation carries between turns.
|
|
273
278
|
- `MAX_PER_LOAD` — The most a single `load_tools` call may pull in.
|
|
279
|
+
- `orderTools` — The tool array in a stable order, so the same set of tools renders the same way twice.
|
|
274
280
|
- `PRESELECT_SCHEMA` — The shape a preselector's answer is held to where the server takes a schema: `{ tools: [...] }`.
|
|
275
281
|
- `PRESELECT_SYSTEM` — The preselection system prompt at the default cap, for a caller that never changes it.
|
|
276
282
|
- `preselectInput` — The user message for a preselection call: the catalogue, then the request.
|
|
277
283
|
- `preselection` — Resolves a preselection against the catalogue: unknown names dropped, count capped.
|
|
278
284
|
- `preselectSystem` — The system prompt a preselector is given, holding it to the cap its answer will be held to.
|
|
279
285
|
- `requestedNames` — `load_tools` arguments, defensively — a model may send a bare string or a nested object.
|
|
286
|
+
- `ToolOrder` (type) — How a tool array is ordered before it is sent: `true` by name, `false` as the caller built it, or a comparator over the two names.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cubicecho/agent-core",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.13.0",
|
|
4
4
|
"description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"openai",
|