@cubicecho/agent-core 2.19.0 → 2.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +53 -4
- package/dist/agent-loop.d.ts +51 -11
- package/dist/agent-loop.js +112 -29
- package/dist/compaction.d.ts +3 -1
- package/dist/compaction.js +8 -1
- package/dist/config.d.ts +4 -2
- package/dist/index.d.ts +1 -1
- package/dist/index.js +1 -1
- package/dist/spec.d.ts +5 -1
- package/dist/spec.js +4 -2
- package/dist/tool-loading.d.ts +98 -0
- package/dist/tool-loading.js +215 -0
- package/llms.txt +7 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -23,7 +23,7 @@ only, Node >=22.
|
|
|
23
23
|
| Module | What it does |
|
|
24
24
|
| --- | --- |
|
|
25
25
|
| `schema-compat` | Makes an MCP tool schema something a strict or grammar-constrained server will accept. `sanitizeTools`, `relaxTools`, `isGrammarError`. |
|
|
26
|
-
| `tool-loading` | On-demand tool discovery: a name-only catalogue plus a `load_tools` meta-tool, so a run pays for the schemas it asks for instead of all of them. Plus `preselectByKeywords`, which picks from it without a model. |
|
|
26
|
+
| `tool-loading` | On-demand tool discovery: a name-only catalogue plus a `load_tools` meta-tool, so a run pays for the schemas it asks for instead of all of them. Plus `preselectByKeywords`, which picks from it without a model, and the proxied form of the same thing — `PROXY_TOOLS`, `proxyLoadResult`, `proxiedCall` — for a server whose prompt cache a growing tool array throws away. |
|
|
27
27
|
| `stream` | Reads one streamed turn back into a message: token callbacks, tool-call reassembly, fenced reasoning taken out of the answer, and the idle watchdog that turns a silent endpoint into `EndpointSilent`. |
|
|
28
28
|
| `capabilities` | What an endpoint turned out not to support — and, under it, what one model on that endpoint did not — plus the loop that answers either when it says so. `capabilitiesFor`, `modelCapabilitiesFor`, `negotiate`. |
|
|
29
29
|
| `thinking` | Tells a scratchpad fenced inside `content` from the answer: `FenceSplitter` for a stream, `stripThinking` for a whole reply, and the fence tables both read. |
|
|
@@ -385,10 +385,10 @@ import { runAgentLoop, emit } from "@cubicecho/agent-core";
|
|
|
385
385
|
|
|
386
386
|
const { turn, messages, usage, loaded } = await runAgentLoop({
|
|
387
387
|
config, // Endpoint & ModelParams & { maxToolIterations, toolDiscovery?, maxRetries?, loadingTimeoutSeconds?, contextLength? }
|
|
388
|
-
system, // sent as the first message; on-demand
|
|
388
|
+
system, // sent as the first message; on-demand and proxied modes append the catalogue
|
|
389
389
|
messages: history, // ending in the question; not written to
|
|
390
390
|
tools, // every tool the run may reach
|
|
391
|
-
catalog, // the same, name-only, for on-demand loading
|
|
391
|
+
catalog, // the same, name-only, for on-demand and proxied loading
|
|
392
392
|
preselected, // from `preselect`, if a small model chose
|
|
393
393
|
dispatch: ({ name, args }, signal) => pool.call(name, args, signal),
|
|
394
394
|
hooks: { run, context: { session: { id } } },
|
|
@@ -483,6 +483,53 @@ tool array stays a strict prefix of the new one. With name order, the default, a
|
|
|
483
483
|
is sorted in and moves every definition after it — the first step still matches the second, but
|
|
484
484
|
the last turn's cache holds only as far as the first new name.
|
|
485
485
|
|
|
486
|
+
`toolDiscovery: "proxy"` is the same catalogue behind a tool array that never changes. A chat
|
|
487
|
+
template renders the tool array in the system turn, so on a server that caches by prefix —
|
|
488
|
+
llama.cpp, and anything built on it — a `load_tools` that appends a definition changes the head of
|
|
489
|
+
the prompt and the whole transcript is prefilled again, on the very step that was meant to save
|
|
490
|
+
tokens. Proxied, every step declares `PROXY_TOOLS` and nothing else: `load_tools`, and `call_tool`,
|
|
491
|
+
which takes `{ name, arguments }`. A load answers with each new tool's name, description and
|
|
492
|
+
parameters as a line of JSON, since no request will ever declare them, and with a pointer back for
|
|
493
|
+
one whose definition is already in the conversation. A `call_tool` is refused for a name outside
|
|
494
|
+
the catalogue, takes `arguments` as an object or as JSON in a string, and reaches `dispatch` as the
|
|
495
|
+
tool it names — the inner name and arguments, under the call's own id — so a dispatcher, a watcher
|
|
496
|
+
and `used` see the same thing in every mode. The transcript keeps the `call_tool` the model wrote.
|
|
497
|
+
|
|
498
|
+
```ts
|
|
499
|
+
const { messages, used } = await runAgentLoop({
|
|
500
|
+
config: { ...config, toolDiscovery: "proxy" },
|
|
501
|
+
messages: history,
|
|
502
|
+
tools, // never declared; a load answers with them
|
|
503
|
+
catalog,
|
|
504
|
+
preselected, // written into the transcript, not into the tool array
|
|
505
|
+
dispatch: ({ name, args }, signal) => pool.call(name, args, signal),
|
|
506
|
+
});
|
|
507
|
+
```
|
|
508
|
+
|
|
509
|
+
A preselection has no tool array to go into there, and no step of its own. It is written into the
|
|
510
|
+
transcript after the question as a `load_tools` call the model did not make, and the result that
|
|
511
|
+
call would have had:
|
|
512
|
+
|
|
513
|
+
```ts
|
|
514
|
+
{ role: "assistant", content: null, tool_calls: [{ id: "preselect-3", type: "function",
|
|
515
|
+
function: { name: "load_tools", arguments: '{"names":["files__read_file"]}' } }] },
|
|
516
|
+
{ role: "tool", tool_call_id: "preselect-3", content: "Loaded 1 tool(s). Run them with `call_tool`.\n\n{...}" },
|
|
517
|
+
```
|
|
518
|
+
|
|
519
|
+
The pair is in the `messages` the loop hands back, to be stored like any other, and is reported as
|
|
520
|
+
a `tool-call` and a `tool-result` event, told to `onToolCall` and `onToolResult`, and counted in
|
|
521
|
+
`toolCalls` and in the `load_tools` metrics. The
|
|
522
|
+
id is numbered by where the call lands in the transcript, so two questions do not share one. A name
|
|
523
|
+
with no definition in `tools`, or outside the catalogue, is left out, and none left is no exchange.
|
|
524
|
+
|
|
525
|
+
What is loaded there is what the history holds, so `loaded` comes back empty and the `loaded`
|
|
526
|
+
option is not read: there is nothing to carry, and a definition a compaction folded away has to be
|
|
527
|
+
loadable again. For the same reason a `beforeStep` that returns a different array makes the loop
|
|
528
|
+
forget what it loaded, which costs a definition sent twice at worst. `call_tool` is honoured in
|
|
529
|
+
`ondemand` mode as well — a session switched out of proxied mode still has it in its history, and
|
|
530
|
+
a model copies what it has seen work — unless the host has a tool of that name, which is then the
|
|
531
|
+
host's. Eager mode does not know it.
|
|
532
|
+
|
|
486
533
|
That preselection costs a round trip to a model, which on a local box is a few seconds before the
|
|
487
534
|
run has started, spent on a model doing term matching. `preselect(..., { keywords: true })` does
|
|
488
535
|
the matching directly and spends the model only on what the words cannot settle:
|
|
@@ -663,7 +710,9 @@ Two ways to make a transcript smaller, cheap first.
|
|
|
663
710
|
`pruneToolResults(messages, { keepLast: 5, maxChars: 256 })` replaces every tool result but the
|
|
664
711
|
latest five with a stub — `[result cleared, 10,412 chars]`. A 40k-character `read_file` is 10k
|
|
665
712
|
tokens on every turn after it, and by then the model has usually taken what it wanted; the stub
|
|
666
|
-
keeps the call answered and says how much was there.
|
|
713
|
+
keeps the call answered and says how much was there. A proxied `load_tools` result that carries
|
|
714
|
+
definitions is never cleared, however old: it is the only copy of the schema the model has, and
|
|
715
|
+
`holdsDefinitions` is the test for a host with a prune of its own.
|
|
667
716
|
|
|
668
717
|
`planCompaction(messages, { limit, used })` says where to fold the oldest stretch into a summary,
|
|
669
718
|
once `used` (the last turn's prompt tokens, or the estimate) is past three quarters of the window.
|
package/dist/agent-loop.d.ts
CHANGED
|
@@ -98,7 +98,12 @@ export declare function preselect(config: Endpoint, model: string, catalog: Cata
|
|
|
98
98
|
*/
|
|
99
99
|
keywords?: boolean | KeywordPreselectOptions;
|
|
100
100
|
}): Promise<string[]>;
|
|
101
|
-
/**
|
|
101
|
+
/**
|
|
102
|
+
* One call the model made, as `dispatch` and `onToolCall` are handed it.
|
|
103
|
+
*
|
|
104
|
+
* A `call_tool` arrives as the tool it names — that tool's name and arguments, under the
|
|
105
|
+
* `call_tool`'s id — so a dispatcher is the same in every discovery mode.
|
|
106
|
+
*/
|
|
102
107
|
export interface ToolCallRequest {
|
|
103
108
|
id: string;
|
|
104
109
|
name: string;
|
|
@@ -107,7 +112,10 @@ export interface ToolCallRequest {
|
|
|
107
112
|
* read, which only `onToolCall` is ever handed.
|
|
108
113
|
*/
|
|
109
114
|
args: Record<string, unknown>;
|
|
110
|
-
/**
|
|
115
|
+
/**
|
|
116
|
+
* The arguments as the model wrote them, before any repair. Through `call_tool`, the inner
|
|
117
|
+
* arguments as JSON text.
|
|
118
|
+
*/
|
|
111
119
|
raw: string;
|
|
112
120
|
}
|
|
113
121
|
/** What one tool call did, in the order the model asked. */
|
|
@@ -119,6 +127,7 @@ export interface ToolCallOutcome {
|
|
|
119
127
|
* sets it; optional so an outcome built before it existed still compiles.
|
|
120
128
|
*/
|
|
121
129
|
id?: string;
|
|
130
|
+
/** The tool that was called. For a `call_tool`, the tool it named. */
|
|
122
131
|
name: string;
|
|
123
132
|
/** False when the arguments did not parse, the tool threw, or `load_tools` loaded nothing. */
|
|
124
133
|
ok: boolean;
|
|
@@ -192,20 +201,27 @@ export interface AgentLoopOptions {
|
|
|
192
201
|
* The endpoint, what to ask the model for, and how long it may keep calling tools.
|
|
193
202
|
* `toolDiscovery` absent is eager, `maxRetries` absent is none, and `contextLength` is handed
|
|
194
203
|
* to `runTurn` as `contextLimit`, which sizes each request against the window before sending.
|
|
204
|
+
* `"proxy"` is on-demand loading behind a tool array that never changes; see `PROXY_TOOLS`.
|
|
195
205
|
*/
|
|
196
206
|
config: Endpoint & ModelParams & Pick<ToolPolicy, "maxToolIterations"> & Partial<Pick<ToolPolicy, "toolDiscovery">> & Partial<RetryPolicy> & {
|
|
197
207
|
contextLength?: number;
|
|
198
208
|
};
|
|
199
|
-
/**
|
|
209
|
+
/**
|
|
210
|
+
* The standing instruction, sent as the first message. On-demand and proxied modes append the
|
|
211
|
+
* catalogue, each in its own wording.
|
|
212
|
+
*/
|
|
200
213
|
system?: string;
|
|
201
214
|
/** The transcript so far, ending in the question. Not written to; see the result's `messages`. */
|
|
202
215
|
messages: OpenAI.ChatCompletionMessageParam[];
|
|
203
216
|
/**
|
|
204
217
|
* Every tool this run may reach. Eager mode sends all of them; on-demand mode sends the ones
|
|
205
|
-
* loaded so far, by name.
|
|
218
|
+
* loaded so far, by name; proxied mode declares none, and answers a load with them instead.
|
|
206
219
|
*/
|
|
207
220
|
tools?: OpenAI.ChatCompletionTool[];
|
|
208
|
-
/**
|
|
221
|
+
/**
|
|
222
|
+
* The same tools as a name-only catalogue. On-demand and proxied modes need it, and are eager
|
|
223
|
+
* without it.
|
|
224
|
+
*/
|
|
209
225
|
catalog?: CatalogServer[];
|
|
210
226
|
/**
|
|
211
227
|
* How the declared tools are ordered before each request. By name unless told otherwise, so a
|
|
@@ -218,6 +234,13 @@ export interface AgentLoopOptions {
|
|
|
218
234
|
* `load_tools` — because a model with the menu still in front of it shops: it reloads what it
|
|
219
235
|
* has or picks a sibling. Everything comes back on the step after. `preselectRouting` trades
|
|
220
236
|
* that for a first step shaped like the rest.
|
|
237
|
+
*
|
|
238
|
+
* Proxied, the tool array is fixed and there is no such step. The shortlist is written into the
|
|
239
|
+
* transcript instead, after the question, as a `load_tools` call the model did not make and the
|
|
240
|
+
* result that call would have had — an assistant message and a tool message, handed back in
|
|
241
|
+
* `messages` like any other, reported as a `tool-call` and a `tool-result` event and told to
|
|
242
|
+
* `onToolCall` and `onToolResult`. Names with no definition in `tools`, or outside the
|
|
243
|
+
* catalogue, are left out, and none left is no exchange.
|
|
221
244
|
*/
|
|
222
245
|
preselected?: readonly string[];
|
|
223
246
|
/**
|
|
@@ -233,11 +256,14 @@ export interface AgentLoopOptions {
|
|
|
233
256
|
* Appended is before `toolOrder` has its say: sorted, a preselected tool lands at its name's
|
|
234
257
|
* place and moves every definition after it, so only `toolOrder: false` keeps a carried array
|
|
235
258
|
* a strict prefix. Either way the first step's system prompt is the second's, and so is its
|
|
236
|
-
* tool array until something else is loaded. Nothing to do without a preselection,
|
|
237
|
-
* mode.
|
|
259
|
+
* tool array until something else is loaded. Nothing to do without a preselection, in eager
|
|
260
|
+
* mode, or proxied, where no step is routed either way.
|
|
238
261
|
*/
|
|
239
262
|
preselectRouting?: "exclusive" | "append";
|
|
240
|
-
/**
|
|
263
|
+
/**
|
|
264
|
+
* Tools already loaded, carried from an earlier question. See `carryOver`. Not read in proxied
|
|
265
|
+
* mode, where a tool is loaded only while its definition is in the history.
|
|
266
|
+
*/
|
|
241
267
|
loaded?: Iterable<string>;
|
|
242
268
|
/** Runs one tool call and returns what the model reads. What it throws, the model reads too. */
|
|
243
269
|
dispatch: (call: ToolCallRequest, signal?: AbortSignal) => Promise<string>;
|
|
@@ -343,11 +369,18 @@ export interface AgentLoopResult {
|
|
|
343
369
|
messages: OpenAI.ChatCompletionMessageParam[];
|
|
344
370
|
/** Summed over every turn of the run. */
|
|
345
371
|
usage: TurnUsage;
|
|
346
|
-
/**
|
|
372
|
+
/**
|
|
373
|
+
* Every call, `load_tools` included, in the order they were made, each under its call id — a
|
|
374
|
+
* proxied preselection's among them, first.
|
|
375
|
+
*/
|
|
347
376
|
toolCalls: ToolCallOutcome[];
|
|
348
|
-
/**
|
|
377
|
+
/**
|
|
378
|
+
* What is loaded at the end, for `carryOver`. Empty in eager mode, and in proxied mode: a
|
|
379
|
+
* definition loaded there is already in the history, and one a compaction folded away has to be
|
|
380
|
+
* loadable again rather than answered "already loaded".
|
|
381
|
+
*/
|
|
349
382
|
loaded: string[];
|
|
350
|
-
/** The tools the model actually called, `load_tools` excluded. */
|
|
383
|
+
/** The tools the model actually called, `load_tools` excluded and `call_tool` looked through. */
|
|
351
384
|
used: string[];
|
|
352
385
|
/** The hooks' notes from before the first request. */
|
|
353
386
|
notes: HookNote[];
|
|
@@ -407,6 +440,13 @@ export interface AgentLoopResult {
|
|
|
407
440
|
* continuation, and a step that reported none, or reported a cache count above its prompt, is not
|
|
408
441
|
* recorded. The ledger comes back in the result and on every `usage` event; see `tokensBetween`.
|
|
409
442
|
*
|
|
443
|
+
* Proxied discovery is the same catalogue behind a tool array that never moves: every step
|
|
444
|
+
* declares `PROXY_TOOLS` and nothing else, a load answers with the definitions themselves, and a
|
|
445
|
+
* `call_tool` is dispatched, counted and reported as the tool it names. A preselection has no
|
|
446
|
+
* tool array to go into, so it is written into the transcript as a `load_tools` exchange after
|
|
447
|
+
* the question. `call_tool` is honoured on demand too, unless the host has a tool of that name —
|
|
448
|
+
* a session switched out of proxied mode still has it in its history, and the model copies it.
|
|
449
|
+
*
|
|
410
450
|
* @param options The config, transcript, tools and dispatcher, plus the optional hooks, events
|
|
411
451
|
* and cancellation. See `AgentLoopOptions`.
|
|
412
452
|
*/
|
package/dist/agent-loop.js
CHANGED
|
@@ -11,7 +11,7 @@ import { runTurn } from "./run-turn.js";
|
|
|
11
11
|
import { relaxTools, sanitizeTools } from "./schema-compat.js";
|
|
12
12
|
import { askJson, tryAsk } from "./side-task.js";
|
|
13
13
|
import { parseToolArguments, recoverToolCalls } from "./tool-calls.js";
|
|
14
|
-
import { catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, preselectByKeywords, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
|
|
14
|
+
import { CALL_TOOL, catalogPrompt, expandNames, inCatalog, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PROXY_TOOLS, preselectByKeywords, preselectInput, preselection, preselectSystem, proxiedCall, proxyCatalogPrompt, proxyLoadResult, requestedNames, shownCall, } from "./tool-loading.js";
|
|
15
15
|
/**
|
|
16
16
|
* The loop above a turn: send, run the tools the model asked for, send again, until it stops
|
|
17
17
|
* asking.
|
|
@@ -220,6 +220,13 @@ function cacheDiagnosis(previous, messages, tools, usage) {
|
|
|
220
220
|
* continuation, and a step that reported none, or reported a cache count above its prompt, is not
|
|
221
221
|
* recorded. The ledger comes back in the result and on every `usage` event; see `tokensBetween`.
|
|
222
222
|
*
|
|
223
|
+
* Proxied discovery is the same catalogue behind a tool array that never moves: every step
|
|
224
|
+
* declares `PROXY_TOOLS` and nothing else, a load answers with the definitions themselves, and a
|
|
225
|
+
* `call_tool` is dispatched, counted and reported as the tool it names. A preselection has no
|
|
226
|
+
* tool array to go into, so it is written into the transcript as a `load_tools` exchange after
|
|
227
|
+
* the question. `call_tool` is honoured on demand too, unless the host has a tool of that name —
|
|
228
|
+
* a session switched out of proxied mode still has it in its history, and the model copies it.
|
|
229
|
+
*
|
|
223
230
|
* @param options The config, transcript, tools and dispatcher, plus the optional hooks, events
|
|
224
231
|
* and cancellation. See `AgentLoopOptions`.
|
|
225
232
|
*/
|
|
@@ -254,11 +261,16 @@ export async function runAgentLoop(options) {
|
|
|
254
261
|
const supports = capabilitiesFor(config.baseUrl, config.apiKey);
|
|
255
262
|
const maxRetries = Math.max(0, Number(config.maxRetries) || 0);
|
|
256
263
|
const notice = (text) => onEvent({ kind: "notice", text });
|
|
257
|
-
|
|
258
|
-
|
|
264
|
+
// On demand, but with a tool array that never changes: definitions come back as `load_tools`
|
|
265
|
+
// results and run through `call_tool`. `onDemand` is true of both.
|
|
266
|
+
const proxied = config.toolDiscovery === "proxy" && catalog.length > 0;
|
|
267
|
+
const onDemand = proxied || (config.toolDiscovery === "ondemand" && catalog.length > 0);
|
|
268
|
+
// Proxied, `loaded` is what this run has put a definition in the history for, and starts empty.
|
|
269
|
+
const loaded = new Set(onDemand && !proxied ? (options.loaded ?? []) : []);
|
|
259
270
|
const preselected = onDemand ? [...(options.preselected ?? [])] : [];
|
|
260
|
-
|
|
261
|
-
|
|
271
|
+
if (!proxied)
|
|
272
|
+
for (const name of preselected)
|
|
273
|
+
loaded.add(name);
|
|
262
274
|
const used = new Set();
|
|
263
275
|
const definitions = new Map();
|
|
264
276
|
for (const tool of tools) {
|
|
@@ -266,6 +278,8 @@ export async function runAgentLoop(options) {
|
|
|
266
278
|
definitions.set(tool.function.name, tool);
|
|
267
279
|
}
|
|
268
280
|
}
|
|
281
|
+
// Looked through unless the host has a tool by that name, which is then the host's to answer.
|
|
282
|
+
const proxies = onDemand && !definitions.has(CALL_TOOL);
|
|
269
283
|
// In the order the names are given, not the order of `tools`: `loaded` is a set, which iterates
|
|
270
284
|
// in the order things were added, so a load appends and never reshuffles what went before.
|
|
271
285
|
const byName = (names) => [...names].flatMap((name) => definitions.get(name) ?? []);
|
|
@@ -291,35 +305,71 @@ export async function runAgentLoop(options) {
|
|
|
291
305
|
// The request the ledger's last entry came from, which is what the next one is measured against.
|
|
292
306
|
// Nothing before this run's first: an earlier run's head is not something the loop can see.
|
|
293
307
|
let measured;
|
|
308
|
+
// Proxied, a shortlist has nowhere to go but the history: the tool array is fixed, so it is
|
|
309
|
+
// answered as though the model had loaded it, and the definitions sit after the question.
|
|
310
|
+
const shortlist = proxied
|
|
311
|
+
? byName(new Set(preselected)).filter((tool) => tool.type === "function" && inCatalog(catalog, tool.function.name))
|
|
312
|
+
: [];
|
|
313
|
+
if (shortlist.length) {
|
|
314
|
+
const names = shortlist.map((tool) => (tool.type === "function" ? tool.function.name : ""));
|
|
315
|
+
// Numbered by where the call lands, so two questions in one transcript do not share an id.
|
|
316
|
+
const id = `preselect-${messages.length}`;
|
|
317
|
+
const args = JSON.stringify({ names });
|
|
318
|
+
// Held to its own length rather than `MAX_PER_LOAD`: that cap is for a model choosing, and a
|
|
319
|
+
// host that shortlisted more has already chosen.
|
|
320
|
+
const content = proxyLoadResult(expandNames(names, catalog, names.length), catalog, shortlist);
|
|
321
|
+
// Announced as a call the model made would be, so a host pairing calls with results by id
|
|
322
|
+
// shows this one like the rest.
|
|
323
|
+
onEvent({ kind: "tool-call", id, name: LOAD_TOOLS, text: preview(args) });
|
|
324
|
+
onToolCall?.({ id, name: LOAD_TOOLS, args: { names }, raw: args });
|
|
325
|
+
onEvent({ kind: "tool-result", id, name: LOAD_TOOLS, ok: true, text: preview(content) });
|
|
326
|
+
onToolResult?.({ id, name: LOAD_TOOLS, ok: true, content });
|
|
327
|
+
toolCalls.push({ id, name: LOAD_TOOLS, ok: true });
|
|
328
|
+
loads.toolsLoaded += names.length;
|
|
329
|
+
for (const name of names)
|
|
330
|
+
loaded.add(name);
|
|
331
|
+
messages.push({
|
|
332
|
+
role: "assistant",
|
|
333
|
+
content: null,
|
|
334
|
+
tool_calls: [{ id, type: "function", function: { name: LOAD_TOOLS, arguments: args } }],
|
|
335
|
+
}, { role: "tool", tool_call_id: id, content });
|
|
336
|
+
}
|
|
294
337
|
for (let step = 0; step < config.maxToolIterations; step++) {
|
|
295
338
|
// A stop aborts the request in flight, but a tool call already handed off runs to its own
|
|
296
339
|
// end — so the signal is read between steps as well.
|
|
297
340
|
signal?.throwIfAborted();
|
|
298
341
|
const before = messages;
|
|
299
|
-
messages
|
|
300
|
-
(
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
342
|
+
const rewritten = await beforeStep?.(messages, step, {
|
|
343
|
+
...(previous && previous.prompt > 0 ? { used: previous.prompt } : {}),
|
|
344
|
+
limit: config.contextLength ?? 0,
|
|
345
|
+
ledger,
|
|
346
|
+
});
|
|
347
|
+
// A rewrite may have folded a definition away, and a load answered "already loaded" would
|
|
348
|
+
// then point at nothing. Forgetting costs a definition sent twice at worst.
|
|
349
|
+
if (proxied && rewritten && rewritten !== messages)
|
|
350
|
+
loaded.clear();
|
|
351
|
+
messages = rewritten ?? messages;
|
|
305
352
|
ledger = rebaseLedger(ledger, before, messages);
|
|
306
353
|
onEvent({ kind: "turn", text: `turn ${step + 1}` });
|
|
307
354
|
// Appended, the shortlist is already in `loaded` — after what was carried, and once — so the
|
|
308
|
-
// first step needs nothing of its own.
|
|
309
|
-
const routed = preselectRouting !== "append" && preselected.length > 0 && step === 0;
|
|
355
|
+
// first step needs nothing of its own. Proxied, it is in the history.
|
|
356
|
+
const routed = !proxied && preselectRouting !== "append" && preselected.length > 0 && step === 0;
|
|
310
357
|
// Ordered here rather than left to `buildBody`, so `names` below is what the request actually
|
|
311
358
|
// declared — a diagnosis reading an order the server never saw calls an untouched tool array
|
|
312
359
|
// `tools-changed`.
|
|
313
360
|
const declared = orderTools(routed
|
|
314
361
|
? byName(new Set(preselected))
|
|
315
|
-
:
|
|
316
|
-
?
|
|
317
|
-
:
|
|
362
|
+
: proxied
|
|
363
|
+
? [...PROXY_TOOLS]
|
|
364
|
+
: onDemand
|
|
365
|
+
? loadedTools([LOAD_TOOLS_DEFINITION], byName(loaded))
|
|
366
|
+
: tools, toolOrder);
|
|
318
367
|
// Unmarked, so the system prompt is the same text on every step and a load does not throw
|
|
319
368
|
// away the cache for the whole transcript. What is loaded is said in `declared` and in the
|
|
320
369
|
// `load_tools` result instead. The preselected first step is the one exception, by design,
|
|
321
370
|
// and `preselectRouting: "append"` is the way out of it.
|
|
322
|
-
const
|
|
371
|
+
const catalogue = proxied ? proxyCatalogPrompt(catalog) : catalogPrompt(catalog);
|
|
372
|
+
const prompt = onDemand && !routed ? `${system}\n\n${catalogue}`.trim() : system;
|
|
323
373
|
const request = [
|
|
324
374
|
...(prompt ? [{ role: "system", content: prompt }] : []),
|
|
325
375
|
...withContext(messages, question ? messages.indexOf(question) : -1, gathered.context, preface),
|
|
@@ -402,7 +452,7 @@ export async function runAgentLoop(options) {
|
|
|
402
452
|
if (recover && !calls.length && content && (tools.length > 0 || onDemand)) {
|
|
403
453
|
const names = tools.flatMap((tool) => (tool.type === "function" ? [tool.function.name] : []));
|
|
404
454
|
const recovered = recoverToolCalls(content, {
|
|
405
|
-
names: onDemand ? [...names, LOAD_TOOLS] : names,
|
|
455
|
+
names: onDemand ? [...names, LOAD_TOOLS, ...(proxies ? [CALL_TOOL] : [])] : names,
|
|
406
456
|
});
|
|
407
457
|
if (recovered.toolCalls.length) {
|
|
408
458
|
calls = recovered.toolCalls;
|
|
@@ -463,7 +513,7 @@ export async function runAgentLoop(options) {
|
|
|
463
513
|
messages,
|
|
464
514
|
usage,
|
|
465
515
|
toolCalls,
|
|
466
|
-
loaded: [...loaded],
|
|
516
|
+
loaded: proxied ? [] : [...loaded],
|
|
467
517
|
used: [...used],
|
|
468
518
|
notes: gathered.notes,
|
|
469
519
|
context: gathered.context,
|
|
@@ -482,10 +532,29 @@ export async function runAgentLoop(options) {
|
|
|
482
532
|
// between ran, and the file the model read may be the file it has since written.
|
|
483
533
|
const answered = new Map();
|
|
484
534
|
const run = async ({ call, args, error: unreadable, normal }) => {
|
|
485
|
-
const
|
|
535
|
+
const proxy = proxies && call.function.name === CALL_TOOL;
|
|
536
|
+
// A `call_tool` is shown, counted and dispatched as the tool it names. Read from the repaired
|
|
537
|
+
// arguments where there are any, since the model's own may be almost-JSON.
|
|
538
|
+
const { name, input: raw } = proxy
|
|
539
|
+
? shownCall(CALL_TOOL, args ? normal : call.function.arguments)
|
|
540
|
+
: { name: call.function.name, input: call.function.arguments };
|
|
541
|
+
// Looked through before the call is announced, so a host is told of the tool it names with
|
|
542
|
+
// that tool's arguments. A name outside the catalogue is refused here, so `call_tool` reaches
|
|
543
|
+
// nothing a load could not, and the refusal is the call's answer below.
|
|
544
|
+
let inner = args ?? {};
|
|
545
|
+
let refused;
|
|
546
|
+
if (proxy && args) {
|
|
547
|
+
try {
|
|
548
|
+
inner = proxiedCall(args, catalog).input;
|
|
549
|
+
}
|
|
550
|
+
catch (error) {
|
|
551
|
+
inner = {};
|
|
552
|
+
refused = { error };
|
|
553
|
+
}
|
|
554
|
+
}
|
|
486
555
|
// Built for every call rather than only the dispatched ones, so a host hears of the calls
|
|
487
556
|
// the loop answers itself in the same shape. Arguments that could not be read are none.
|
|
488
|
-
const request = { id: call.id, name, args:
|
|
557
|
+
const request = { id: call.id, name, args: inner, raw };
|
|
489
558
|
onEvent({ kind: "tool-call", id: call.id, name, text: preview(raw) });
|
|
490
559
|
onToolCall?.(request);
|
|
491
560
|
let content;
|
|
@@ -493,24 +562,38 @@ export async function runAgentLoop(options) {
|
|
|
493
562
|
try {
|
|
494
563
|
if (!args)
|
|
495
564
|
throw unreadable;
|
|
496
|
-
|
|
565
|
+
// By the name the model called, not the one a `call_tool` wraps: `load_tools` is not in
|
|
566
|
+
// the catalogue, and one reached through `call_tool` is refused below like any other.
|
|
567
|
+
if (onDemand && call.function.name === LOAD_TOOLS) {
|
|
497
568
|
const resolved = expandNames(requestedNames(args), catalog);
|
|
498
|
-
content =
|
|
499
|
-
|
|
569
|
+
content = proxied
|
|
570
|
+
? proxyLoadResult(resolved, catalog, byName(resolved.matched), loaded)
|
|
571
|
+
: loadResult(resolved, catalog, loaded);
|
|
572
|
+
// Proxied, a load is only of what the result could define: a catalogued name the host
|
|
573
|
+
// gave no definition for was not loaded, and is not counted or remembered as though it was.
|
|
574
|
+
const hits = proxied
|
|
575
|
+
? resolved.matched.filter((hit) => definitions.has(hit))
|
|
576
|
+
: resolved.matched;
|
|
577
|
+
for (const hit of hits)
|
|
500
578
|
loads[loaded.has(hit) ? "redundantLoads" : "toolsLoaded"]++;
|
|
501
579
|
loads.unknownToolNames += resolved.unknown.length;
|
|
502
|
-
for (const hit of
|
|
580
|
+
for (const hit of hits)
|
|
503
581
|
loaded.add(hit);
|
|
504
|
-
ok =
|
|
582
|
+
ok = hits.length > 0;
|
|
505
583
|
}
|
|
506
584
|
else {
|
|
585
|
+
if (refused)
|
|
586
|
+
throw refused.error;
|
|
507
587
|
// A model that skips `load_tools` and calls a catalogued tool by name is right about
|
|
508
|
-
// what it wants; load it and run it rather than refusing.
|
|
509
|
-
|
|
588
|
+
// what it wants; load it and run it rather than refusing. Not proxied, where loaded
|
|
589
|
+
// means its definition is in the history, and this call put none there.
|
|
590
|
+
if (onDemand && !proxied && inCatalog(catalog, name))
|
|
510
591
|
loaded.add(name);
|
|
511
592
|
used.add(name);
|
|
593
|
+
// Keyed on the inner call, so one made through `call_tool` and one made natively share.
|
|
594
|
+
const key = `${name}\0${proxy ? JSON.stringify(inner) : normal}`;
|
|
512
595
|
content = dedupable(request)
|
|
513
|
-
? await once(answered,
|
|
596
|
+
? await once(answered, key, () => dispatch(request, signal))
|
|
514
597
|
: await dispatch(request, signal);
|
|
515
598
|
}
|
|
516
599
|
}
|
package/dist/compaction.d.ts
CHANGED
|
@@ -42,7 +42,9 @@ export interface PruneOptions {
|
|
|
42
42
|
* after it, and by then the model has usually taken what it wanted from it; the stub keeps the
|
|
43
43
|
* call answered — a call with no result is a malformed transcript — and says how much was there,
|
|
44
44
|
* so a model that does need it again knows to ask. Short results are kept, since a stub saves
|
|
45
|
-
* nothing on them
|
|
45
|
+
* nothing on them, and so is a proxied `load_tools` result carrying definitions, which is the
|
|
46
|
+
* only copy of them the model has (`holdsDefinitions`); either still counts as one of the latest
|
|
47
|
+
* few. Returns the same array when there was nothing to clear. Rewrites the prefix;
|
|
46
48
|
* see the module comment on when to run it.
|
|
47
49
|
*
|
|
48
50
|
* @param messages The transcript. Not written to.
|
package/dist/compaction.js
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { consult, notify, turnMessages, } from "./hooks.js";
|
|
2
2
|
import { messageTokens } from "./retry.js";
|
|
3
3
|
import { ask } from "./side-task.js";
|
|
4
|
+
import { holdsDefinitions } from "./tool-loading.js";
|
|
4
5
|
/** The fraction of the window in use before a summary is worth its own round trip. */
|
|
5
6
|
export const COMPACT_AT = 0.75;
|
|
6
7
|
/** The fraction of the window the kept tail may fill, leaving room for the run to grow again. */
|
|
@@ -65,7 +66,9 @@ const systemHead = (messages) => {
|
|
|
65
66
|
* after it, and by then the model has usually taken what it wanted from it; the stub keeps the
|
|
66
67
|
* call answered — a call with no result is a malformed transcript — and says how much was there,
|
|
67
68
|
* so a model that does need it again knows to ask. Short results are kept, since a stub saves
|
|
68
|
-
* nothing on them
|
|
69
|
+
* nothing on them, and so is a proxied `load_tools` result carrying definitions, which is the
|
|
70
|
+
* only copy of them the model has (`holdsDefinitions`); either still counts as one of the latest
|
|
71
|
+
* few. Returns the same array when there was nothing to clear. Rewrites the prefix;
|
|
69
72
|
* see the module comment on when to run it.
|
|
70
73
|
*
|
|
71
74
|
* @param messages The transcript. Not written to.
|
|
@@ -83,6 +86,10 @@ export function pruneToolResults(messages, { keepLast = 5, maxChars = 256 } = {}
|
|
|
83
86
|
const text = textOf(message.content);
|
|
84
87
|
if (text.length <= maxChars || text.startsWith("[result cleared"))
|
|
85
88
|
continue;
|
|
89
|
+
// A proxied load's result is the schema itself; stubbed, the tool is one the model can still
|
|
90
|
+
// name and no longer call correctly.
|
|
91
|
+
if (holdsDefinitions(text))
|
|
92
|
+
continue;
|
|
86
93
|
out ??= [...messages];
|
|
87
94
|
out[at] = {
|
|
88
95
|
...message,
|
package/dist/config.d.ts
CHANGED
|
@@ -75,9 +75,11 @@ export interface ModelParams {
|
|
|
75
75
|
export interface ToolPolicy {
|
|
76
76
|
/**
|
|
77
77
|
* "eager" sends every tool definition on every request. "ondemand" sends a name-only
|
|
78
|
-
* catalogue and lets the model pull in the schemas it needs.
|
|
78
|
+
* catalogue and lets the model pull in the schemas it needs. "proxy" is the same catalogue
|
|
79
|
+
* behind a tool array that never changes — `load_tools` and `call_tool` — for a server whose
|
|
80
|
+
* prompt cache a growing tool array throws away. See `tool-loading.ts`.
|
|
79
81
|
*/
|
|
80
|
-
toolDiscovery: "eager" | "ondemand";
|
|
82
|
+
toolDiscovery: "eager" | "ondemand" | "proxy";
|
|
81
83
|
/** The model that does the preselection pass. Empty means don't preselect. */
|
|
82
84
|
toolSelectModel: string;
|
|
83
85
|
/** Hard stop on runaway tool loops. */
|
package/dist/index.d.ts
CHANGED
|
@@ -32,4 +32,4 @@ export { type Produced, type StreamTurnOptions, streamTurn, type Turn, type Turn
|
|
|
32
32
|
export { ALL_FENCES, DEFAULT_FENCES, type Fence, FenceSplitter, type FenceSplitterOptions, type Split, stripThinking, THINK_FENCE, } from "./thinking.ts";
|
|
33
33
|
export { estimateTokens } from "./tokens.ts";
|
|
34
34
|
export { parseToolArguments, recoverToolCalls, ToolArgumentsError, type ToolCall, } from "./tool-calls.ts";
|
|
35
|
-
export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, KEYWORD_DROPOFF, KEYWORD_MIN_SCORE, type KeywordPreselection, type KeywordPreselectOptions, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectByKeywords, preselectInput, preselection, preselectSystem, requestedNames, type ToolMatch, type ToolOrder, } from "./tool-loading.ts";
|
|
35
|
+
export { CALL_TOOL, carryOver, catalogList, catalogPrompt, expandNames, holdsDefinitions, inCatalog, KEYWORD_DROPOFF, KEYWORD_MIN_SCORE, type KeywordPreselection, type KeywordPreselectOptions, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PRESELECT_SYSTEM, PROXY_TOOLS, preselectByKeywords, preselectInput, preselection, preselectSystem, proxiedCall, proxyCatalogPrompt, proxyLoadResult, requestedNames, shownCall, type ToolMatch, type ToolOrder, } from "./tool-loading.ts";
|
package/dist/index.js
CHANGED
|
@@ -30,4 +30,4 @@ export { streamTurn, } from "./stream.js";
|
|
|
30
30
|
export { ALL_FENCES, DEFAULT_FENCES, FenceSplitter, stripThinking, THINK_FENCE, } from "./thinking.js";
|
|
31
31
|
export { estimateTokens } from "./tokens.js";
|
|
32
32
|
export { parseToolArguments, recoverToolCalls, ToolArgumentsError, } from "./tool-calls.js";
|
|
33
|
-
export { carryOver, catalogList, catalogPrompt, expandNames, inCatalog, KEYWORD_DROPOFF, KEYWORD_MIN_SCORE, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PRESELECT_SYSTEM, preselectByKeywords, preselectInput, preselection, preselectSystem, requestedNames, } from "./tool-loading.js";
|
|
33
|
+
export { CALL_TOOL, carryOver, catalogList, catalogPrompt, expandNames, holdsDefinitions, inCatalog, KEYWORD_DROPOFF, KEYWORD_MIN_SCORE, LOAD_TOOLS, LOAD_TOOLS_DEFINITION, loadedTools, loadResult, MAX_CARRIED, MAX_PER_LOAD, orderTools, PRESELECT_SCHEMA, PRESELECT_SYSTEM, PROXY_TOOLS, preselectByKeywords, preselectInput, preselection, preselectSystem, proxiedCall, proxyCatalogPrompt, proxyLoadResult, requestedNames, shownCall, } from "./tool-loading.js";
|
package/dist/spec.d.ts
CHANGED
|
@@ -105,7 +105,11 @@ export interface PromptPart {
|
|
|
105
105
|
}
|
|
106
106
|
/** How tools reach the model, and which servers this agent may reach at all. */
|
|
107
107
|
export interface ToolsSpec {
|
|
108
|
-
|
|
108
|
+
/**
|
|
109
|
+
* How tool definitions reach the model: all of them on every request, a name-only catalogue
|
|
110
|
+
* the model loads from, or that catalogue behind a fixed `load_tools` and `call_tool`.
|
|
111
|
+
*/
|
|
112
|
+
discovery?: "eager" | "ondemand" | "proxy";
|
|
109
113
|
/** The hard stop on a tool loop. At least one. */
|
|
110
114
|
maxIterations?: number;
|
|
111
115
|
/**
|
package/dist/spec.js
CHANGED
|
@@ -206,10 +206,12 @@ function parseTools(report, value) {
|
|
|
206
206
|
return void report.drop("tools", "must be an object");
|
|
207
207
|
let discovery;
|
|
208
208
|
if (!absent(value.discovery)) {
|
|
209
|
-
if (value.discovery === "eager" ||
|
|
209
|
+
if (value.discovery === "eager" ||
|
|
210
|
+
value.discovery === "ondemand" ||
|
|
211
|
+
value.discovery === "proxy")
|
|
210
212
|
discovery = value.discovery;
|
|
211
213
|
else
|
|
212
|
-
report.drop("tools.discovery", 'must be "eager" or "
|
|
214
|
+
report.drop("tools.discovery", 'must be "eager", "ondemand" or "proxy"');
|
|
213
215
|
}
|
|
214
216
|
let servers;
|
|
215
217
|
if (!absent(value.servers)) {
|
package/dist/tool-loading.d.ts
CHANGED
|
@@ -180,6 +180,104 @@ export declare const inCatalog: (catalog: CatalogServer[], name: string) => bool
|
|
|
180
180
|
* @param args The tool call's arguments, exactly as the model sent them.
|
|
181
181
|
*/
|
|
182
182
|
export declare function requestedNames(args: Record<string, unknown>): string[];
|
|
183
|
+
/**
|
|
184
|
+
* The one tool a proxied run reaches every catalogued tool through.
|
|
185
|
+
*
|
|
186
|
+
* Proxied discovery is on-demand loading with a tool array that never changes. On-demand mode
|
|
187
|
+
* declares each tool as it is loaded, and a chat template renders the tool array in the system
|
|
188
|
+
* turn, ahead of the whole conversation — so appending one definition there moves every token
|
|
189
|
+
* after it, and on a model llama.cpp can only rewind to a saved checkpoint a single load
|
|
190
|
+
* re-prefills the transcript from the first token. Proxied, the array is `load_tools` and
|
|
191
|
+
* `call_tool` for the life of the session: a load answers with the definitions as its result, at
|
|
192
|
+
* the end of the history where the cache already stops, and the model runs a loaded tool through
|
|
193
|
+
* this one. The price is a level of indirection the model has to get right, which a small model
|
|
194
|
+
* does less reliably than a native call.
|
|
195
|
+
*/
|
|
196
|
+
export declare const CALL_TOOL = "call_tool";
|
|
197
|
+
/**
|
|
198
|
+
* The whole tool array of a proxied run, `load_tools` then `call_tool`, frozen because it is
|
|
199
|
+
* shared.
|
|
200
|
+
*
|
|
201
|
+
* The load keeps `LOAD_TOOLS` as its name, so `requestedNames` and `expandNames` read its
|
|
202
|
+
* arguments as they do on demand; only what it promises differs, which is why it is not
|
|
203
|
+
* `LOAD_TOOLS_DEFINITION`. `call_tool`'s `arguments` says `additionalProperties: true` and has to:
|
|
204
|
+
* `sanitizeTools` gives an object with no properties an empty property list, a grammar-constrained
|
|
205
|
+
* server compiles that to `{}`, and without the keyword the model is left no way to pass an
|
|
206
|
+
* argument at all.
|
|
207
|
+
*/
|
|
208
|
+
export declare const PROXY_TOOLS: readonly OpenAI.ChatCompletionTool[];
|
|
209
|
+
/**
|
|
210
|
+
* The catalogue block for a proxied run's system prompt, worded for `call_tool` rather than for a
|
|
211
|
+
* tool list.
|
|
212
|
+
*
|
|
213
|
+
* `catalogPrompt` tells the model a loaded tool is in its tool list, which proxied is never true —
|
|
214
|
+
* a model told so looks for it there and calls it natively. The listing itself is the same, and
|
|
215
|
+
* like that one it is never marked, so the head of the prompt is one text for the whole session.
|
|
216
|
+
*
|
|
217
|
+
* @param catalog The connected servers. A catalogue with no tools in it produces an empty string.
|
|
218
|
+
*/
|
|
219
|
+
export declare function proxyCatalogPrompt(catalog: CatalogServer[]): string;
|
|
220
|
+
/**
|
|
221
|
+
* Whether a tool result is a proxied load carrying definitions, which is the only copy of them
|
|
222
|
+
* the model has.
|
|
223
|
+
*
|
|
224
|
+
* On demand the definitions are in the tool array and a load's result only repeats their
|
|
225
|
+
* descriptions, so clearing it loses nothing. Proxied, the result is the schema, and
|
|
226
|
+
* `pruneToolResults` asks this before it stubs one. Told by how `proxyLoadResult` opens, the way
|
|
227
|
+
* a summary is told by `SUMMARY_LEAD`, because a `tool` message carries no tool name of its own.
|
|
228
|
+
*
|
|
229
|
+
* @param result The tool message's text. A load that only pointed back or refused is not one.
|
|
230
|
+
*/
|
|
231
|
+
export declare const holdsDefinitions: (result: string) => boolean;
|
|
232
|
+
/**
|
|
233
|
+
* What a proxied `load_tools` answers: each new tool's whole definition as a line of JSON, since
|
|
234
|
+
* the result is the only place the model will ever see it.
|
|
235
|
+
*
|
|
236
|
+
* A name loaded before this call is answered with a pointer back rather than its definition a
|
|
237
|
+
* second time, and one the catalogue lists but `definitions` does not hold is said to have none,
|
|
238
|
+
* so the model is not left waiting on a schema that is not coming. The refusals — too broad, over
|
|
239
|
+
* the per-call cap, not in the catalogue, nothing asked for — are `loadResult`'s, worded the same
|
|
240
|
+
* in both modes.
|
|
241
|
+
*
|
|
242
|
+
* @param resolved What the call asked for, from `expandNames`.
|
|
243
|
+
* @param catalog The servers, read for the refusals' wording.
|
|
244
|
+
* @param definitions The definitions of `resolved.matched`, as many as the host has. Ones that
|
|
245
|
+
* are not functions are skipped.
|
|
246
|
+
* @param loaded What was loaded before this call. Absent reports every match as newly loaded.
|
|
247
|
+
*/
|
|
248
|
+
export declare function proxyLoadResult(resolved: ReturnType<typeof expandNames>, catalog: CatalogServer[], definitions: readonly OpenAI.ChatCompletionTool[], loaded?: ReadonlySet<string>): string;
|
|
249
|
+
/**
|
|
250
|
+
* The tool a `call_tool` names and the arguments to run it with, or a throw the model can read.
|
|
251
|
+
*
|
|
252
|
+
* `arguments` arrives as an object when the model follows the schema and as a JSON string when it
|
|
253
|
+
* copies the shape of a native call instead; both are taken, since the intent is the same. A name
|
|
254
|
+
* outside the catalogue is refused here rather than handed to a dispatcher, which is what keeps
|
|
255
|
+
* `call_tool` from reaching anything the catalogue does not offer.
|
|
256
|
+
*
|
|
257
|
+
* @param args The `call_tool` call's own arguments, parsed. An absent `arguments` is no arguments.
|
|
258
|
+
* @param catalog What may be called. The name is trimmed and then matched exactly.
|
|
259
|
+
*/
|
|
260
|
+
export declare function proxiedCall(args: Record<string, unknown>, catalog: CatalogServer[]): {
|
|
261
|
+
name: string;
|
|
262
|
+
input: Record<string, unknown>;
|
|
263
|
+
};
|
|
264
|
+
/**
|
|
265
|
+
* A tool call as a watcher should see it: a `call_tool` as the tool it ran, anything else as it
|
|
266
|
+
* is.
|
|
267
|
+
*
|
|
268
|
+
* The transcript keeps the `call_tool` the model wrote, since the next request has to repeat it
|
|
269
|
+
* word for word to stay in the cache; only the display looks through it. Anything that does not
|
|
270
|
+
* parse as a proxied call is left as it came.
|
|
271
|
+
*
|
|
272
|
+
* @param name The name the model called.
|
|
273
|
+
* @param input The call's arguments as JSON text.
|
|
274
|
+
* @returns The inner name and its arguments as JSON text — the string itself where the model
|
|
275
|
+
* passed `arguments` as one.
|
|
276
|
+
*/
|
|
277
|
+
export declare function shownCall(name: string, input: string): {
|
|
278
|
+
name: string;
|
|
279
|
+
input: string;
|
|
280
|
+
};
|
|
183
281
|
/**
|
|
184
282
|
* The system prompt a preselector is given, holding it to the cap its answer will be held to.
|
|
185
283
|
*
|
package/dist/tool-loading.js
CHANGED
|
@@ -340,6 +340,221 @@ export function requestedNames(args) {
|
|
|
340
340
|
return value.filter((item) => typeof item === "string");
|
|
341
341
|
return [];
|
|
342
342
|
}
|
|
343
|
+
/**
|
|
344
|
+
* The one tool a proxied run reaches every catalogued tool through.
|
|
345
|
+
*
|
|
346
|
+
* Proxied discovery is on-demand loading with a tool array that never changes. On-demand mode
|
|
347
|
+
* declares each tool as it is loaded, and a chat template renders the tool array in the system
|
|
348
|
+
* turn, ahead of the whole conversation — so appending one definition there moves every token
|
|
349
|
+
* after it, and on a model llama.cpp can only rewind to a saved checkpoint a single load
|
|
350
|
+
* re-prefills the transcript from the first token. Proxied, the array is `load_tools` and
|
|
351
|
+
* `call_tool` for the life of the session: a load answers with the definitions as its result, at
|
|
352
|
+
* the end of the history where the cache already stops, and the model runs a loaded tool through
|
|
353
|
+
* this one. The price is a level of indirection the model has to get right, which a small model
|
|
354
|
+
* does less reliably than a native call.
|
|
355
|
+
*/
|
|
356
|
+
export const CALL_TOOL = "call_tool";
|
|
357
|
+
/**
|
|
358
|
+
* The whole tool array of a proxied run, `load_tools` then `call_tool`, frozen because it is
|
|
359
|
+
* shared.
|
|
360
|
+
*
|
|
361
|
+
* The load keeps `LOAD_TOOLS` as its name, so `requestedNames` and `expandNames` read its
|
|
362
|
+
* arguments as they do on demand; only what it promises differs, which is why it is not
|
|
363
|
+
* `LOAD_TOOLS_DEFINITION`. `call_tool`'s `arguments` says `additionalProperties: true` and has to:
|
|
364
|
+
* `sanitizeTools` gives an object with no properties an empty property list, a grammar-constrained
|
|
365
|
+
* server compiles that to `{}`, and without the keyword the model is left no way to pass an
|
|
366
|
+
* argument at all.
|
|
367
|
+
*/
|
|
368
|
+
export const PROXY_TOOLS = deepFreeze([
|
|
369
|
+
{
|
|
370
|
+
type: "function",
|
|
371
|
+
function: {
|
|
372
|
+
name: LOAD_TOOLS,
|
|
373
|
+
description: "Get the full definitions of tools listed in the tool catalogue: what each does and the " +
|
|
374
|
+
"arguments it takes. Pass the exact names you need, or a trailing wildcard like " +
|
|
375
|
+
"`server__group__*` for a whole group. Then run them with `call_tool`. Load only what " +
|
|
376
|
+
"the task actually needs.",
|
|
377
|
+
parameters: {
|
|
378
|
+
type: "object",
|
|
379
|
+
properties: {
|
|
380
|
+
names: {
|
|
381
|
+
type: "array",
|
|
382
|
+
items: { type: "string" },
|
|
383
|
+
description: "Tool names from the catalogue. Wildcards may end with `*`.",
|
|
384
|
+
},
|
|
385
|
+
},
|
|
386
|
+
required: ["names"],
|
|
387
|
+
additionalProperties: false,
|
|
388
|
+
},
|
|
389
|
+
},
|
|
390
|
+
},
|
|
391
|
+
{
|
|
392
|
+
type: "function",
|
|
393
|
+
function: {
|
|
394
|
+
name: CALL_TOOL,
|
|
395
|
+
description: "Run a tool from the catalogue. Load it with `load_tools` first to learn its arguments, " +
|
|
396
|
+
"then pass its exact name and an arguments object matching its parameters.",
|
|
397
|
+
parameters: {
|
|
398
|
+
type: "object",
|
|
399
|
+
properties: {
|
|
400
|
+
name: { type: "string", description: "The tool's exact name from the catalogue." },
|
|
401
|
+
arguments: {
|
|
402
|
+
type: "object",
|
|
403
|
+
description: "The tool's arguments, as its definition describes them.",
|
|
404
|
+
additionalProperties: true,
|
|
405
|
+
},
|
|
406
|
+
},
|
|
407
|
+
required: ["name", "arguments"],
|
|
408
|
+
},
|
|
409
|
+
},
|
|
410
|
+
},
|
|
411
|
+
]);
|
|
412
|
+
/**
|
|
413
|
+
* The catalogue block for a proxied run's system prompt, worded for `call_tool` rather than for a
|
|
414
|
+
* tool list.
|
|
415
|
+
*
|
|
416
|
+
* `catalogPrompt` tells the model a loaded tool is in its tool list, which proxied is never true —
|
|
417
|
+
* a model told so looks for it there and calls it natively. The listing itself is the same, and
|
|
418
|
+
* like that one it is never marked, so the head of the prompt is one text for the whole session.
|
|
419
|
+
*
|
|
420
|
+
* @param catalog The connected servers. A catalogue with no tools in it produces an empty string.
|
|
421
|
+
*/
|
|
422
|
+
export function proxyCatalogPrompt(catalog) {
|
|
423
|
+
const list = catalogList(catalog);
|
|
424
|
+
if (!list)
|
|
425
|
+
return "";
|
|
426
|
+
return [
|
|
427
|
+
"# Tool catalogue",
|
|
428
|
+
"",
|
|
429
|
+
"These tools exist. Call `load_tools` with the names you need to get their definitions, then",
|
|
430
|
+
"run them with `call_tool`. Names are descriptive; load a tool to see its parameters. A tool",
|
|
431
|
+
"whose definition is already in this conversation does not need loading again. Do not load",
|
|
432
|
+
"tools the task does not need, and do not mention this mechanism in your answer.",
|
|
433
|
+
"",
|
|
434
|
+
list,
|
|
435
|
+
].join("\n");
|
|
436
|
+
}
|
|
437
|
+
/** How a proxied load result that carries definitions opens, with the count left open. */
|
|
438
|
+
const PROXY_LOADED = /^Loaded \d+ tool\(s\)\. Run them with `call_tool`\./;
|
|
439
|
+
/**
|
|
440
|
+
* Whether a tool result is a proxied load carrying definitions, which is the only copy of them
|
|
441
|
+
* the model has.
|
|
442
|
+
*
|
|
443
|
+
* On demand the definitions are in the tool array and a load's result only repeats their
|
|
444
|
+
* descriptions, so clearing it loses nothing. Proxied, the result is the schema, and
|
|
445
|
+
* `pruneToolResults` asks this before it stubs one. Told by how `proxyLoadResult` opens, the way
|
|
446
|
+
* a summary is told by `SUMMARY_LEAD`, because a `tool` message carries no tool name of its own.
|
|
447
|
+
*
|
|
448
|
+
* @param result The tool message's text. A load that only pointed back or refused is not one.
|
|
449
|
+
*/
|
|
450
|
+
export const holdsDefinitions = (result) => PROXY_LOADED.test(result);
|
|
451
|
+
/**
|
|
452
|
+
* What a proxied `load_tools` answers: each new tool's whole definition as a line of JSON, since
|
|
453
|
+
* the result is the only place the model will ever see it.
|
|
454
|
+
*
|
|
455
|
+
* A name loaded before this call is answered with a pointer back rather than its definition a
|
|
456
|
+
* second time, and one the catalogue lists but `definitions` does not hold is said to have none,
|
|
457
|
+
* so the model is not left waiting on a schema that is not coming. The refusals — too broad, over
|
|
458
|
+
* the per-call cap, not in the catalogue, nothing asked for — are `loadResult`'s, worded the same
|
|
459
|
+
* in both modes.
|
|
460
|
+
*
|
|
461
|
+
* @param resolved What the call asked for, from `expandNames`.
|
|
462
|
+
* @param catalog The servers, read for the refusals' wording.
|
|
463
|
+
* @param definitions The definitions of `resolved.matched`, as many as the host has. Ones that
|
|
464
|
+
* are not functions are skipped.
|
|
465
|
+
* @param loaded What was loaded before this call. Absent reports every match as newly loaded.
|
|
466
|
+
*/
|
|
467
|
+
export function proxyLoadResult(resolved, catalog, definitions, loaded) {
|
|
468
|
+
const lines = [];
|
|
469
|
+
const again = resolved.matched.filter((name) => loaded?.has(name));
|
|
470
|
+
const fresh = definitions.flatMap((tool) => tool.type === "function" && !loaded?.has(tool.function.name) ? [tool.function] : []);
|
|
471
|
+
const defined = new Set(fresh.map((tool) => tool.name));
|
|
472
|
+
const missing = resolved.matched.filter((name) => !loaded?.has(name) && !defined.has(name));
|
|
473
|
+
if (fresh.length) {
|
|
474
|
+
lines.push(`Loaded ${fresh.length} tool(s). Run them with \`${CALL_TOOL}\`.`);
|
|
475
|
+
for (const { name, description, parameters } of fresh)
|
|
476
|
+
lines.push("", JSON.stringify({ name, description, parameters }));
|
|
477
|
+
}
|
|
478
|
+
if (again.length) {
|
|
479
|
+
if (lines.length)
|
|
480
|
+
lines.push("");
|
|
481
|
+
lines.push(`Already loaded earlier in this conversation: ${again.join(", ")}. Run them with ` +
|
|
482
|
+
`\`${CALL_TOOL}\`; do not load them again.`);
|
|
483
|
+
}
|
|
484
|
+
if (missing.length) {
|
|
485
|
+
if (lines.length)
|
|
486
|
+
lines.push("");
|
|
487
|
+
lines.push(`No definition is available for: ${missing.join(", ")}.`);
|
|
488
|
+
}
|
|
489
|
+
const { overBroad, deferred, unknown, matched } = resolved;
|
|
490
|
+
if (overBroad.length || deferred.length || unknown.length || !matched.length) {
|
|
491
|
+
if (lines.length)
|
|
492
|
+
lines.push("");
|
|
493
|
+
lines.push(loadResult({ ...resolved, matched: [] }, catalog));
|
|
494
|
+
}
|
|
495
|
+
return lines.join("\n");
|
|
496
|
+
}
|
|
497
|
+
/**
|
|
498
|
+
* The tool a `call_tool` names and the arguments to run it with, or a throw the model can read.
|
|
499
|
+
*
|
|
500
|
+
* `arguments` arrives as an object when the model follows the schema and as a JSON string when it
|
|
501
|
+
* copies the shape of a native call instead; both are taken, since the intent is the same. A name
|
|
502
|
+
* outside the catalogue is refused here rather than handed to a dispatcher, which is what keeps
|
|
503
|
+
* `call_tool` from reaching anything the catalogue does not offer.
|
|
504
|
+
*
|
|
505
|
+
* @param args The `call_tool` call's own arguments, parsed. An absent `arguments` is no arguments.
|
|
506
|
+
* @param catalog What may be called. The name is trimmed and then matched exactly.
|
|
507
|
+
*/
|
|
508
|
+
export function proxiedCall(args, catalog) {
|
|
509
|
+
const name = typeof args.name === "string" ? args.name.trim() : "";
|
|
510
|
+
if (!name)
|
|
511
|
+
throw new Error(`${CALL_TOOL} needs a name; pass one from the tool catalogue.`);
|
|
512
|
+
if (!inCatalog(catalog, name))
|
|
513
|
+
throw new Error(`Not in the catalogue: ${name}. Check the name and try again.`);
|
|
514
|
+
let input = args.arguments ?? {};
|
|
515
|
+
if (typeof input === "string") {
|
|
516
|
+
const text = input;
|
|
517
|
+
try {
|
|
518
|
+
input = text.trim() ? JSON.parse(text) : {};
|
|
519
|
+
}
|
|
520
|
+
catch {
|
|
521
|
+
throw new Error(`${CALL_TOOL} arguments for ${name} are not valid JSON: ${text.slice(0, 200)}`);
|
|
522
|
+
}
|
|
523
|
+
}
|
|
524
|
+
if (!input || typeof input !== "object" || Array.isArray(input))
|
|
525
|
+
throw new Error(`${CALL_TOOL} arguments for ${name} must be an object.`);
|
|
526
|
+
return { name, input: input };
|
|
527
|
+
}
|
|
528
|
+
/**
|
|
529
|
+
* A tool call as a watcher should see it: a `call_tool` as the tool it ran, anything else as it
|
|
530
|
+
* is.
|
|
531
|
+
*
|
|
532
|
+
* The transcript keeps the `call_tool` the model wrote, since the next request has to repeat it
|
|
533
|
+
* word for word to stay in the cache; only the display looks through it. Anything that does not
|
|
534
|
+
* parse as a proxied call is left as it came.
|
|
535
|
+
*
|
|
536
|
+
* @param name The name the model called.
|
|
537
|
+
* @param input The call's arguments as JSON text.
|
|
538
|
+
* @returns The inner name and its arguments as JSON text — the string itself where the model
|
|
539
|
+
* passed `arguments` as one.
|
|
540
|
+
*/
|
|
541
|
+
export function shownCall(name, input) {
|
|
542
|
+
if (name !== CALL_TOOL)
|
|
543
|
+
return { name, input };
|
|
544
|
+
try {
|
|
545
|
+
const args = JSON.parse(input);
|
|
546
|
+
if (typeof args.name !== "string" || !args.name.trim())
|
|
547
|
+
return { name, input };
|
|
548
|
+
const inner = args.arguments ?? {};
|
|
549
|
+
return {
|
|
550
|
+
name: args.name.trim(),
|
|
551
|
+
input: typeof inner === "string" ? inner : JSON.stringify(inner),
|
|
552
|
+
};
|
|
553
|
+
}
|
|
554
|
+
catch {
|
|
555
|
+
return { name, input };
|
|
556
|
+
}
|
|
557
|
+
}
|
|
343
558
|
/**
|
|
344
559
|
* Where a request is cut for the preselector, in characters.
|
|
345
560
|
*
|
package/llms.txt
CHANGED
|
@@ -316,10 +316,12 @@ Reading what a model meant by a tool call when it did not write one cleanly.
|
|
|
316
316
|
|
|
317
317
|
### tool-loading
|
|
318
318
|
|
|
319
|
+
- `CALL_TOOL` — The one tool a proxied run reaches every catalogued tool through.
|
|
319
320
|
- `carryOver` — The tools to start the next turn with: last turn's where they were, then what this turn used.
|
|
320
321
|
- `catalogList` — The catalogue as a plain grouped listing of names, loaded ones marked if asked.
|
|
321
322
|
- `catalogPrompt` — The catalogue block appended to the system prompt.
|
|
322
323
|
- `expandNames` — Resolves requested names against the catalogue, expanding trailing `*` wildcards.
|
|
324
|
+
- `holdsDefinitions` — Whether a tool result is a proxied load carrying definitions, which is the only copy of them the model has.
|
|
323
325
|
- `inCatalog` — Whether the catalogue holds a tool by this name.
|
|
324
326
|
- `KEYWORD_DROPOFF` — How far the best unpicked tool must fall below the last picked one for the cut to count clean.
|
|
325
327
|
- `KEYWORD_MIN_SCORE` — The least a best match may score and still be acted on without a model.
|
|
@@ -334,10 +336,15 @@ Reading what a model meant by a tool call when it did not write one cleanly.
|
|
|
334
336
|
- `orderTools` — The tool array in a stable order, so the same set of tools renders the same way twice.
|
|
335
337
|
- `PRESELECT_SCHEMA` — The shape a preselector's answer is held to where the server takes a schema: `{ tools: [...] }`.
|
|
336
338
|
- `PRESELECT_SYSTEM` — The preselection system prompt at the default cap, for a caller that never changes it.
|
|
339
|
+
- `PROXY_TOOLS` — The whole tool array of a proxied run, `load_tools` then `call_tool`, frozen because it is shared.
|
|
337
340
|
- `preselectByKeywords` — The tools a request's own words point at, ranked, and whether they point clearly enough.
|
|
338
341
|
- `preselectInput` — The user message for a preselection call: the catalogue, then the request.
|
|
339
342
|
- `preselection` — Resolves a preselection against the catalogue: unknown names dropped, count capped.
|
|
340
343
|
- `preselectSystem` — The system prompt a preselector is given, holding it to the cap its answer will be held to.
|
|
344
|
+
- `proxiedCall` — The tool a `call_tool` names and the arguments to run it with, or a throw the model can read.
|
|
345
|
+
- `proxyCatalogPrompt` — The catalogue block for a proxied run's system prompt, worded for `call_tool` rather than for a tool list.
|
|
346
|
+
- `proxyLoadResult` — What a proxied `load_tools` answers: each new tool's whole definition as a line of JSON, since the result is the only place the model will ever see it.
|
|
341
347
|
- `requestedNames` — `load_tools` arguments, defensively — a model may send a bare string or a nested object.
|
|
348
|
+
- `shownCall` — A tool call as a watcher should see it: a `call_tool` as the tool it ran, anything else as it is.
|
|
342
349
|
- `ToolMatch` (type) — One tool's score against a request.
|
|
343
350
|
- `ToolOrder` (type) — How a tool array is ordered before it is sent: `true` by name, `false` as the caller built it, or a comparator over the two names.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cubicecho/agent-core",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.20.0",
|
|
4
4
|
"description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"openai",
|