@cubicecho/agent-core 4.10.0 → 4.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -30,11 +30,11 @@ only, Node >=22.
30
30
  | `side-task` | One-shot calls that support a run without being one — small prompt, short answer, no tools, never worth failing the run over. `askJson` holds the answer to a schema where the server can, and `taskCall` turns the settings an agent spec gives a task into the call that honours them. |
31
31
  | `hooks` | The host's side of lifecycle hooks: `gather` before a request and `notify` after, the shared context budget, `withContext` to put what they add on the turn's question, `untrusted` to fence text nobody vouched for, and `turnMessages` to hand them a transcript. Running a hook is a runner the caller passes. |
32
32
  | `events` | The in-memory bus a watcher reads while a run happens: `emit`, `watch`, `history`, `fold`, and `runMetrics` for what a run cost. A watcher's backlog is capped and reports its own gaps. |
33
- | `client` | A pooled `OpenAI` client per endpoint, plus the context window: the served one where a local server says, the listed one otherwise, and their caches. And `resolveApiKey`, for a caller deciding which key an endpoint gets. |
33
+ | `client` | A pooled `OpenAI` client per endpoint, plus the context window: the served one where a local server says, the listed one otherwise, and their caches. And `resolveApiKey`, for a caller deciding which key an endpoint gets, and `probeEndpoint`, for listing one a form has not saved yet. |
34
34
  | `retry` | What to do when a request is lost, refused or too big: `isTransient`, `isModelLoading`, `backoffMs`, `ContextOverflow`, `EndpointSilent`. |
35
35
  | `calibration` | How many characters a token is worth on one model, learned from the prompt counts its endpoint reports: `charsPerTokenFor`, `calibrate`. |
36
36
  | `continuation` | `continueTurn`: carries on an answer the token ceiling cut off, by prefilling it as a trailing assistant message. |
37
- | `config` | The structural interfaces every function here asks for. |
37
+ | `config` | The structural interfaces every function here asks for, and `normalizeBaseUrl`, the one rule for when two base URLs are one endpoint. |
38
38
  | `run-turn` | `runTurn`: one turn with the retry loop around the negotiation around the stream. The whole loop, for a caller that wants it rather than its parts. Sizes the request against an opt-in `contextLimit`. |
39
39
  | `agent-loop` | `runAgentLoop`: the loop above a turn — `runTurn` per step, the tools between, `load_tools` and preselection handled, until the model stops asking. |
40
40
  | `request-body` | `buildBody`: the one place a streamed request's body is decided from a config and what the endpoint and the model have refused. |
@@ -306,7 +306,7 @@ const turn = await runTurn(client, supports, build, {
306
306
 
307
307
  `contextLimitFor` answers the operator's number when there is one. Otherwise it reads the
308
308
  `/v1/models` listing first. A window at the top of the model's entry (`context_length`,
309
- `max_context_window`, `max_model_len`, `context_window` or `n_ctx`: vLLM's `max_model_len`,
309
+ `max_context_length`, `max_context_window`, `max_model_len`, `context_window` or `n_ctx`: vLLM's `max_model_len`,
310
310
  OpenRouter's `context_length`) is taken as the one being served, and nothing else is asked.
311
311
 
312
312
  Where the listing does not settle it, the server's own API is asked (`servedWindow`): llama.cpp's
@@ -316,7 +316,10 @@ listed with only `meta.n_ctx_train`, which is llama.cpp. The trained window diff
316
316
  one in the case the guard exists for, a 256k model started at `-c 16384`, so it is used only when
317
317
  neither native route gives a number. A server with neither route is latched and not asked again.
318
318
  `/api/v0/models` is also dropped on its own: for the life of the process once it answers 404, 405
319
- or 501, and for `listingMissMs` after any other error.
319
+ or 501, and for `listingMissMs` after any other error. A 200 that is not JSON counts as a missing
320
+ route: it is what a proxy with a web app of its own (llm-router) answers on every path it does not
321
+ route. Neither route is sent `NO_KEY` as a bearer; an endpoint with no key is asked with no
322
+ `Authorization` header at all.
320
323
 
321
324
  A host that sizes the window before each run passes the run's signal as the third argument, so a
322
325
  stopped run does not wait on the lookup:
@@ -335,6 +338,33 @@ nothing would otherwise hold the lookup for that long, or for ever where there i
335
338
  bound `contextLimitFor` answers zero, as it does for a server it could not reach, and
336
339
  `listModels` rejects with a `TimeoutError`.
337
340
 
341
+ ### Listing an endpoint that is not saved yet
342
+
343
+ `listModels` is for an endpoint that is configured: it pools a client per URL and remembers what
344
+ came back. A settings form asking at each keystroke would leave a client behind for every
345
+ half-typed URL, so it asks `probeEndpoint` instead, which makes one plain request and remembers
346
+ nothing:
347
+
348
+ ```ts
349
+ import { EndpointProbeError, ProbeFailure, probeEndpoint } from "@cubicecho/agent-core";
350
+
351
+ try {
352
+ const models = await probeEndpoint({ baseUrl: typed, apiKey: typedKey }, { signal });
353
+ } catch (error) {
354
+ if (error instanceof EndpointProbeError) {
355
+ // error.kind: invalid-url, unreachable, timeout, unauthorized, redirected, not-openai, failed.
356
+ // error.message is a sentence naming the endpoint by its origin, fit to show under the field.
357
+ return { problem: error.message };
358
+ }
359
+ throw error; // the signal's own reason
360
+ }
361
+ ```
362
+
363
+ It refuses a URL `normalizeBaseUrl` refuses before asking anybody, follows no redirect (so a typed
364
+ key is never carried on to a host nobody typed), reads both OpenAI's `{ data: [...] }` and
365
+ Ollama's native `{ models: [...] }` (by `name` where an entry has no `id`), and hands back each id
366
+ once, sorted, at most `limit` of them (500 unless told). It is bounded as a lookup is.
367
+
338
368
  A top-level listing key is trusted without a second opinion, so on a server that puts a trained
339
369
  window there, pass `contextLength`. Ollama reports no window on any route this reads, and truncates
340
370
  an over-long prompt rather than refusing it, so on Ollama pass `contextLength` or there is no guard
@@ -532,6 +562,41 @@ its calls made on the error too, read off with `failedRun`.
532
562
  A link meant for the model alone — one whose MCP `audience` leaves out `"user"` — is the
533
563
  dispatcher's to drop, since that is MCP's rule and this package knows nothing of MCP.
534
564
 
565
+ ### A call that ends the run
566
+
567
+ A tool whose call is the end of the job — handing off, asking a person — says so with `end` on
568
+ its `ToolAnswer`, `true` or `{ reason }`. The loop answers every call of that step, sends no
569
+ further request, and resolves with `ended`; aborting the run's signal from inside `dispatch` was
570
+ the way out before, and that is a failure the host then has to dig the run back out of.
571
+
572
+ ```ts
573
+ const result = await runAgentLoop({
574
+ // ...
575
+ dispatch: async ({ name, args }, signal) =>
576
+ name === "ask_person"
577
+ ? { text: "Asked. This run ends here.", end: { reason: String(args.question) } }
578
+ : pool.call(name, args, { signal }),
579
+ });
580
+ if (result.ended) {
581
+ // { callId, name: "ask_person", reason }
582
+ const { callId, name, reason = "" } = result.ended;
583
+ emit(runId, { kind: RunEventKind.Done, ok: true, id: callId, name, text: reason });
584
+ }
585
+ ```
586
+
587
+ The step's other calls still run, whichever side of the end the model asked for them and however
588
+ `parallel` batches them: they were asked for in the same turn, and a model that says "write the
589
+ file, then hand off" wants the file written in either order. Where two calls end it, `ended` is
590
+ the first in the order asked. Everything else is a finished run's — `usage`, `metrics`,
591
+ `messages`, `artifacts`, the `afterTurn` hooks — with `metrics.outcome` `ended` and a notice
592
+ saying which tool ended it. `turn` is the turn that made the call, so its `content` is whatever
593
+ the model wrote beside its calls, often nothing; the run's answer is what the ending tool took.
594
+ The tool, its definition and its wording stay the host's: this is only the exit.
595
+
596
+ A host's `done` that carries the call — `id`, `name` the tool, `text` the reason — is read by
597
+ `runMetrics` as `ended` too, so the outcome is the same whether it is read off the result or the
598
+ events.
599
+
535
600
  Arguments go through `parseToolArguments`, which is lenient where the model's meaning is plain:
536
601
  JSON held in a string is opened, and the almost-JSON local models write — single quotes, Python's
537
602
  `True` and `None`, bare keys, a trailing comma — is repaired, without touching what is inside a
@@ -874,6 +939,12 @@ it encodes is the conservative one: an endpoint's own key wins; one that names a
874
939
  own, different from the settings it inherits from, gets `NO_KEY` rather than the operator's key or
875
940
  `$OPENAI_API_KEY`; anything else inherits.
876
941
 
942
+ "Different" is decided by `normalizeBaseUrl`, the same rule `endpointKey` and every cache here
943
+ compares by: http or https, no credentials, query or fragment, scheme and host in lower case, no
944
+ default port and no trailing slash. A host storing a base URL should store what it returns, and
945
+ refuse a URL it answers `null` for. A URL it refuses still equals itself, trimmed of whitespace
946
+ and trailing slashes, so a row saved before the rule still finds its own key.
947
+
877
948
  ## Fields this interface cannot spell
878
949
 
879
950
  `ModelParams` has `temperature`, `maxTokens` and `reasoningEffort`. Everything else a model card or
@@ -1438,7 +1509,7 @@ call returns everything in force. Lowering `maxClients` below the pool's current
1438
1509
  it at once, least recently used first. `resetClients` (and `resetAll`) puts the defaults back.
1439
1510
 
1440
1511
  A host that keeps per-endpoint state of its own should key it on `endpointKey` — the JSON of the base
1441
- URL and the key, an absent key read as `NO_KEY` — rather than rebuilding that string, so the two
1512
+ URL as `normalizeBaseUrl` writes it and the key, an absent key read as `NO_KEY` — rather than rebuilding that string, so the two
1442
1513
  cannot drift. `endpointKey` holds the key in the clear; `endpointId`, its SHA-256 digest, is the one
1443
1514
  that is safe to write down.
1444
1515
 
@@ -57,11 +57,44 @@ export interface Endpoint {
57
57
  * not in it, since a different patience is not a different server.
58
58
  */
59
59
  export interface EndpointIdentity {
60
- /** The base URL as given. Compared as written, so a trailing slash is another endpoint. */
60
+ /**
61
+ * The base URL as given. Compared as `baseUrlKey` has it, so a trailing slash, a host's case and
62
+ * a default port are the same endpoint, and a path's case and a `/v1` are not.
63
+ */
61
64
  baseUrl: string;
62
65
  /** Absent and empty mean the same: no key, which `endpointKey` reads as `NO_KEY`. */
63
66
  apiKey?: string;
64
67
  }
68
+ /**
69
+ * A base URL worth storing: http or https, no credentials, query or fragment, no trailing slash.
70
+ *
71
+ * @param raw - The URL as typed. Surrounding whitespace is trimmed.
72
+ * @returns The URL as the WHATWG parser writes it, less its trailing slashes — scheme and host in
73
+ * lower case, a default port dropped — or `null` for anything that is not such a URL.
74
+ *
75
+ * @remarks
76
+ * One rule for every host, because "a stored key only goes to its own endpoint" is decided by
77
+ * comparing two of these, and four hosts writing four rules had begun to disagree about it. A key
78
+ * in the URL is a credential nobody meant to store; a query or a fragment is one more thing two
79
+ * copies of the same endpoint could differ in, and the SDK appends its paths after it.
80
+ *
81
+ * A host that stores a base URL stores this, and a form that refuses one refuses on `null`, so what
82
+ * is saved is already in the form every comparison here reads.
83
+ */
84
+ export declare function normalizeBaseUrl(raw: string): string | null;
85
+ /**
86
+ * A base URL in the form two of them are compared in, valid or not.
87
+ *
88
+ * @param raw - The URL as configured.
89
+ * @returns `normalizeBaseUrl`'s answer where it has one, and otherwise the URL trimmed of
90
+ * whitespace and trailing slashes, which is the rule every comparison here used before.
91
+ *
92
+ * @remarks
93
+ * A URL the normaliser refuses is still somebody's endpoint — an old settings row, a key written
94
+ * into the URL — and it still has to equal itself, so it falls back to the older, looser rule
95
+ * rather than to "unequal to everything".
96
+ */
97
+ export declare const baseUrlKey: (raw: string) => string;
65
98
  /**
66
99
  * The request fields `extraBody` may not set, because the loop's request is built around them.
67
100
  *
@@ -11,6 +11,51 @@
11
11
  * it structurally. Nothing imports a config *type* from a consumer, and no consumer has to grow
12
12
  * a field it has no use for in order to call `getClient`.
13
13
  */
14
+ /**
15
+ * A base URL worth storing: http or https, no credentials, query or fragment, no trailing slash.
16
+ *
17
+ * @param raw - The URL as typed. Surrounding whitespace is trimmed.
18
+ * @returns The URL as the WHATWG parser writes it, less its trailing slashes — scheme and host in
19
+ * lower case, a default port dropped — or `null` for anything that is not such a URL.
20
+ *
21
+ * @remarks
22
+ * One rule for every host, because "a stored key only goes to its own endpoint" is decided by
23
+ * comparing two of these, and four hosts writing four rules had begun to disagree about it. A key
24
+ * in the URL is a credential nobody meant to store; a query or a fragment is one more thing two
25
+ * copies of the same endpoint could differ in, and the SDK appends its paths after it.
26
+ *
27
+ * A host that stores a base URL stores this, and a form that refuses one refuses on `null`, so what
28
+ * is saved is already in the form every comparison here reads.
29
+ */
30
+ export function normalizeBaseUrl(raw) {
31
+ let url;
32
+ try {
33
+ url = new URL(raw.trim());
34
+ }
35
+ catch {
36
+ return null;
37
+ }
38
+ if (url.protocol !== 'http:' && url.protocol !== 'https:') {
39
+ return null;
40
+ }
41
+ if (url.username || url.password || url.search || url.hash) {
42
+ return null;
43
+ }
44
+ return url.toString().replace(/\/+$/, '');
45
+ }
46
+ /**
47
+ * A base URL in the form two of them are compared in, valid or not.
48
+ *
49
+ * @param raw - The URL as configured.
50
+ * @returns `normalizeBaseUrl`'s answer where it has one, and otherwise the URL trimmed of
51
+ * whitespace and trailing slashes, which is the rule every comparison here used before.
52
+ *
53
+ * @remarks
54
+ * A URL the normaliser refuses is still somebody's endpoint — an old settings row, a key written
55
+ * into the URL — and it still has to equal itself, so it falls back to the older, looser rule
56
+ * rather than to "unequal to everything".
57
+ */
58
+ export const baseUrlKey = (raw) => normalizeBaseUrl(raw) ?? raw.trim().replace(/\/+$/, '');
14
59
  /**
15
60
  * The request fields `extraBody` may not set, because the loop's request is built around them.
16
61
  *
@@ -1,5 +1,5 @@
1
1
  import OpenAI from 'openai';
2
- import type { Endpoint, EndpointIdentity, RetryPolicy } from '../core/config.ts';
2
+ import { type Endpoint, type EndpointIdentity, type RetryPolicy } from '../core/config.ts';
3
3
  /**
4
4
  * The SDK insists on a non-empty key even where the server will not look at it. This is what it
5
5
  * gets. It also reads as "this endpoint has no key" at a call site, which an empty string does
@@ -112,13 +112,15 @@ export interface ModelInfo {
112
112
  * `Endpoint` requires it — `capabilitiesFor` is handed a URL and maybe a key rather than a whole
113
113
  * config, and absent and empty already mean the same thing. The timeout is deliberately not in
114
114
  * it; see `listings`.
115
- * @returns The URL and the key as one JSON string, the key in the clear.
115
+ * @returns The URL as `baseUrlKey` writes it and the key, as one JSON string, the key in the clear.
116
116
  *
117
117
  * @remarks
118
118
  * The URL and the key together, because the key is part of what is on the other end rather than
119
119
  * only how it is paid for: a router is free to send two keys to two different backends, and then
120
120
  * what one of them refused is not a fact about the other. Absent reads as `NO_KEY`, so an
121
- * endpoint with no key and one that passes `undefined` are one entry rather than two.
121
+ * endpoint with no key and one that passes `undefined` are one entry rather than two. The URL is
122
+ * normalised for the same reason: `…/v1/` and `…/v1` are one server, and two entries for it would
123
+ * learn every refusal twice.
122
124
  *
123
125
  * Stringified rather than joined on a separator, for the reason `getClient` gives: no character
124
126
  * is impossible in a URL or a key, and two endpoints must never collide on one entry. A host
@@ -207,6 +209,72 @@ export declare function servedSlots(config: Endpoint & {
207
209
  * request's own error otherwise.
208
210
  */
209
211
  export declare function listModels(config: Endpoint, { signal }?: LookupOptions): Promise<ModelInfo[]>;
212
+ /** Why `probeEndpoint` could not list an endpoint's models, in words a form can branch on. */
213
+ export declare const ProbeFailure: {
214
+ /** Not an http or https URL, or one with credentials, a query or a fragment in it. */
215
+ readonly InvalidUrl: 'invalid-url';
216
+ /** Nothing answered: no such host, a refused connection, a TLS failure. */
217
+ readonly Unreachable: 'unreachable';
218
+ /** Something answered, but not inside the lookup's bound. */
219
+ readonly Timeout: 'timeout';
220
+ /** A 401 or a 403: the key is wrong, or one is needed. */
221
+ readonly Unauthorized: 'unauthorized';
222
+ /** A redirect, which is not followed, so the key is never carried to a host nobody typed. */
223
+ readonly Redirected: 'redirected';
224
+ /** An answer that is not a model listing: no such route, a web page, JSON of another shape. */
225
+ readonly NotOpenAi: 'not-openai';
226
+ /** Any other error status, which is on the error as `status`. */
227
+ readonly Failed: 'failed';
228
+ };
229
+ export type ProbeFailure = (typeof ProbeFailure)[keyof typeof ProbeFailure];
230
+ /**
231
+ * `probeEndpoint` could not list an endpoint's models; `kind` says why.
232
+ *
233
+ * @remarks
234
+ * The message is a sentence for a person, naming the endpoint by its origin only, so it can go
235
+ * under a form field as it is. `kind` is for a form that words it itself.
236
+ */
237
+ export declare class EndpointProbeError extends Error {
238
+ readonly name: string;
239
+ readonly kind: ProbeFailure;
240
+ /** The HTTP status, where the endpoint answered with one. */
241
+ readonly status?: number;
242
+ /**
243
+ * @param kind - Why the listing failed.
244
+ * @param message - The same, as a sentence.
245
+ * @param [status] - The status the endpoint answered with, where it answered.
246
+ */
247
+ constructor(kind: ProbeFailure, message: string, status?: number);
248
+ }
249
+ /** What `probeEndpoint` takes besides the endpoint. */
250
+ export interface ProbeOptions extends LookupOptions {
251
+ /** How many models come back at most, the first by id. Defaults to `PROBE_MODEL_LIMIT`. */
252
+ limit?: number;
253
+ }
254
+ /**
255
+ * Asks an endpoint nobody has saved yet which models it serves, for a form a person is typing it
256
+ * into.
257
+ *
258
+ * @param endpoint - The endpoint as typed. Its key goes as a bearer token where it has one other
259
+ * than `NO_KEY`; its `requestTimeoutSeconds` bounds the request where that is shorter than
260
+ * `lookupTimeoutMs`.
261
+ * @param [options] - Cancellation, and how many models to hand back. See `ProbeOptions`.
262
+ * @returns The models in order of id, one entry per id, at most `options.limit` of them. Rejects
263
+ * with the reason of `options.signal` where that stopped it, and with an `EndpointProbeError`
264
+ * for every other failure.
265
+ *
266
+ * @remarks
267
+ * `listModels` is for an endpoint that is configured: it pools a client per URL and remembers the
268
+ * listing, and a form asking at every keystroke would leave a client and a listing behind for
269
+ * each half-typed URL. This makes one plain request and remembers nothing, so asking again asks
270
+ * again.
271
+ *
272
+ * Redirects are not followed, so a typed key is never carried on to a host nobody typed. Both
273
+ * listing shapes are read: OpenAI's `{ data: [...] }` by `id`, and Ollama's native
274
+ * `{ models: [...] }` by `name` where an entry has no `id`. A window is read from the keys
275
+ * `contextLimitFor` reads, and an id listed twice keeps the window one of its entries stated.
276
+ */
277
+ export declare function probeEndpoint(endpoint: EndpointIdentity & Pick<Endpoint, 'requestTimeoutSeconds'>, { signal, limit }?: ProbeOptions): Promise<ModelInfo[]>;
210
278
  /**
211
279
  * How much a model will read, in tokens. Zero means nobody knows.
212
280
  *
@@ -245,12 +313,12 @@ export declare function contextLimitFor(config: Endpoint & {
245
313
  */
246
314
  export declare function resetClients(): void;
247
315
  /**
248
- * A base URL as two settings rows would agree on it: trimmed, without the trailing slash.
316
+ * Whether two base URLs are one endpoint, by `baseUrlKey`.
249
317
  *
250
318
  * @param a - The URL as one row has it.
251
319
  * @param b - The URL as the other has it.
252
- * @returns `true` where the two differ in nothing but surrounding whitespace and trailing slashes.
253
- * Case counts, and so does a `/v1`.
320
+ * @returns `true` where the two differ in nothing but surrounding whitespace, trailing slashes, the
321
+ * case of the scheme and host, and a default port. A path's case counts, and so does a `/v1`.
254
322
  */
255
323
  export declare const sameUrl: (a: string, b: string) => boolean;
256
324
  /**
@@ -1,4 +1,5 @@
1
1
  import OpenAI from 'openai';
2
+ import { baseUrlKey, normalizeBaseUrl, } from '../core/config.js';
2
3
  import { digestOf } from '../core/digest.js';
3
4
  import { byCodeUnit, isPositive } from '../core/guards.js';
4
5
  import { MS_PER_SECOND } from '../core/platform.js';
@@ -192,7 +193,14 @@ export function getClient(config) {
192
193
  * reporting the same number twice. llama.cpp puts its number under `meta` instead, and LM Studio
193
194
  * and Ollama put none on this route at all; see `servedWindow`.
194
195
  */
195
- const CONTEXT_KEYS = ['context_length', 'max_context_window', 'max_model_len', 'context_window', 'n_ctx'];
196
+ const CONTEXT_KEYS = [
197
+ 'context_length',
198
+ 'max_context_length',
199
+ 'max_context_window',
200
+ 'max_model_len',
201
+ 'context_window',
202
+ 'n_ctx',
203
+ ];
196
204
  /**
197
205
  * A number above zero as itself and anything else as zero, which here means "not stated".
198
206
  *
@@ -276,20 +284,22 @@ const misses = scoped(() => new Map());
276
284
  * `Endpoint` requires it — `capabilitiesFor` is handed a URL and maybe a key rather than a whole
277
285
  * config, and absent and empty already mean the same thing. The timeout is deliberately not in
278
286
  * it; see `listings`.
279
- * @returns The URL and the key as one JSON string, the key in the clear.
287
+ * @returns The URL as `baseUrlKey` writes it and the key, as one JSON string, the key in the clear.
280
288
  *
281
289
  * @remarks
282
290
  * The URL and the key together, because the key is part of what is on the other end rather than
283
291
  * only how it is paid for: a router is free to send two keys to two different backends, and then
284
292
  * what one of them refused is not a fact about the other. Absent reads as `NO_KEY`, so an
285
- * endpoint with no key and one that passes `undefined` are one entry rather than two.
293
+ * endpoint with no key and one that passes `undefined` are one entry rather than two. The URL is
294
+ * normalised for the same reason: `…/v1/` and `…/v1` are one server, and two entries for it would
295
+ * learn every refusal twice.
286
296
  *
287
297
  * Stringified rather than joined on a separator, for the reason `getClient` gives: no character
288
298
  * is impossible in a URL or a key, and two endpoints must never collide on one entry. A host
289
299
  * keeping its own per-endpoint state keys it on this rather than on a copy of it, so the two
290
300
  * cannot drift apart. It holds the key in the clear; `endpointId` is the one to write down.
291
301
  */
292
- export const endpointKey = (config) => JSON.stringify([config.baseUrl, config.apiKey || NO_KEY]);
302
+ export const endpointKey = (config) => JSON.stringify([baseUrlKey(config.baseUrl), config.apiKey || NO_KEY]);
293
303
  /**
294
304
  * One model on one endpoint, as every per-model cache keys it: stringified, so neither half runs
295
305
  * into the other.
@@ -402,23 +412,39 @@ const NOT_THERE = new Set([
402
412
  HttpStatus.MethodNotAllowed,
403
413
  HttpStatus.NotImplemented,
404
414
  ]);
415
+ /**
416
+ * The headers that carry an endpoint's key, and none where it has none.
417
+ *
418
+ * @param apiKey - The key as configured. Empty, absent and `NO_KEY` all mean there is none.
419
+ * @returns A bearer `authorization` header, or no headers at all.
420
+ *
421
+ * @remarks
422
+ * `NO_KEY` is the SDK's placeholder, not a credential, and a request made outside the SDK has no
423
+ * reason to send it: a server that checks keys refuses `Bearer agent-core` as it refuses a wrong
424
+ * one, where it might have answered an unauthenticated route.
425
+ */
426
+ const authHeaders = (apiKey) => apiKey && apiKey !== NO_KEY ? { authorization: `Bearer ${apiKey}` } : {};
405
427
  /**
406
428
  * Asks one native endpoint.
407
429
  *
408
- * @param config - The endpoint. Its key goes as a bearer token where it has one.
430
+ * @param config - The endpoint. Its key goes as a bearer token where it has one other than `NO_KEY`.
409
431
  * @param path - The route and its query, from the server root rather than from the base URL.
410
432
  * @param within - The lookup's signal. Fired, the request throws as an unreachable server does.
411
433
  * @returns How the server answered, with `body` only for a 2xx that parsed as JSON.
412
434
  *
413
435
  * @remarks
414
- * `failed` is any answer that was not a 2xx and `missing` the ones among them that mean a server
415
- * without the route; `body` is absent for both, and for a 2xx that was not JSON. A server that
416
- * could not be reached throws.
436
+ * `failed` is any answer that was not a 2xx, or was not JSON, and `missing` the ones among them
437
+ * that mean a server without the route; `body` is absent for all of them. A server that could not
438
+ * be reached throws.
439
+ *
440
+ * A 2xx that is not JSON is a missing route rather than a route with nothing to say. It is what a
441
+ * proxy with a web app of its own answers on every path it does not route — llm-router serves its
442
+ * `index.html` for `/props` with a 200 — and read as "no window" it was asked again every
443
+ * `listingMissMs` instead of being latched as a server without the route.
417
444
  */
418
445
  async function probe(config, path, within) {
419
- const apiKey = config.apiKey || undefined;
420
446
  const response = await fetch(`${rootOf(config.baseUrl)}${path}`, {
421
- headers: apiKey ? { authorization: `Bearer ${apiKey}` } : {},
447
+ headers: authHeaders(config.apiKey),
422
448
  signal: within,
423
449
  });
424
450
  if (!response.ok) {
@@ -428,7 +454,9 @@ async function probe(config, path, within) {
428
454
  return { failed: false, missing: false, body: await response.json() };
429
455
  }
430
456
  catch {
431
- return { failed: false, missing: false };
457
+ // A fired signal surfaces here too, mid-body, and is a server that stopped answering.
458
+ within.throwIfAborted();
459
+ return { failed: true, missing: true };
432
460
  }
433
461
  }
434
462
  /**
@@ -594,6 +622,144 @@ async function readListing(config, within) {
594
622
  trainedOnly().set(key, trained);
595
623
  return models;
596
624
  }
625
+ /** Why `probeEndpoint` could not list an endpoint's models, in words a form can branch on. */
626
+ export const ProbeFailure = {
627
+ /** Not an http or https URL, or one with credentials, a query or a fragment in it. */
628
+ InvalidUrl: 'invalid-url',
629
+ /** Nothing answered: no such host, a refused connection, a TLS failure. */
630
+ Unreachable: 'unreachable',
631
+ /** Something answered, but not inside the lookup's bound. */
632
+ Timeout: 'timeout',
633
+ /** A 401 or a 403: the key is wrong, or one is needed. */
634
+ Unauthorized: 'unauthorized',
635
+ /** A redirect, which is not followed, so the key is never carried to a host nobody typed. */
636
+ Redirected: 'redirected',
637
+ /** An answer that is not a model listing: no such route, a web page, JSON of another shape. */
638
+ NotOpenAi: 'not-openai',
639
+ /** Any other error status, which is on the error as `status`. */
640
+ Failed: 'failed',
641
+ };
642
+ /**
643
+ * `probeEndpoint` could not list an endpoint's models; `kind` says why.
644
+ *
645
+ * @remarks
646
+ * The message is a sentence for a person, naming the endpoint by its origin only, so it can go
647
+ * under a form field as it is. `kind` is for a form that words it itself.
648
+ */
649
+ export class EndpointProbeError extends Error {
650
+ name = 'EndpointProbeError';
651
+ kind;
652
+ /** The HTTP status, where the endpoint answered with one. */
653
+ status;
654
+ /**
655
+ * @param kind - Why the listing failed.
656
+ * @param message - The same, as a sentence.
657
+ * @param [status] - The status the endpoint answered with, where it answered.
658
+ */
659
+ constructor(kind, message, status) {
660
+ super(message);
661
+ this.kind = kind;
662
+ if (status !== undefined) {
663
+ this.status = status;
664
+ }
665
+ }
666
+ }
667
+ /** How many models `probeEndpoint` hands back unless told otherwise: enough for any picker. */
668
+ const PROBE_MODEL_LIMIT = 500;
669
+ /** The statuses that mean the key, not the server, is what is wrong. */
670
+ const UNAUTHORIZED = new Set([HttpStatus.Unauthorized, HttpStatus.Forbidden]);
671
+ /** The range a redirect is answered in. */
672
+ const REDIRECT_MIN = 300;
673
+ const REDIRECT_MAX = 399;
674
+ /**
675
+ * Asks an endpoint nobody has saved yet which models it serves, for a form a person is typing it
676
+ * into.
677
+ *
678
+ * @param endpoint - The endpoint as typed. Its key goes as a bearer token where it has one other
679
+ * than `NO_KEY`; its `requestTimeoutSeconds` bounds the request where that is shorter than
680
+ * `lookupTimeoutMs`.
681
+ * @param [options] - Cancellation, and how many models to hand back. See `ProbeOptions`.
682
+ * @returns The models in order of id, one entry per id, at most `options.limit` of them. Rejects
683
+ * with the reason of `options.signal` where that stopped it, and with an `EndpointProbeError`
684
+ * for every other failure.
685
+ *
686
+ * @remarks
687
+ * `listModels` is for an endpoint that is configured: it pools a client per URL and remembers the
688
+ * listing, and a form asking at every keystroke would leave a client and a listing behind for
689
+ * each half-typed URL. This makes one plain request and remembers nothing, so asking again asks
690
+ * again.
691
+ *
692
+ * Redirects are not followed, so a typed key is never carried on to a host nobody typed. Both
693
+ * listing shapes are read: OpenAI's `{ data: [...] }` by `id`, and Ollama's native
694
+ * `{ models: [...] }` by `name` where an entry has no `id`. A window is read from the keys
695
+ * `contextLimitFor` reads, and an id listed twice keeps the window one of its entries stated.
696
+ */
697
+ export async function probeEndpoint(endpoint, { signal, limit = PROBE_MODEL_LIMIT } = {}) {
698
+ const baseUrl = normalizeBaseUrl(endpoint.baseUrl);
699
+ if (baseUrl === null) {
700
+ throw new EndpointProbeError(ProbeFailure.InvalidUrl, 'A base URL is http:// or https://, with no credentials, query or fragment in it.');
701
+ }
702
+ const url = new URL(`${baseUrl}/models`);
703
+ const within = lookupSignal(endpoint, signal);
704
+ /** The failure for a request that threw: the caller's stop, the bound, or no answer at all. */
705
+ const thrown = () => {
706
+ signal?.throwIfAborted();
707
+ return within.aborted
708
+ ? new EndpointProbeError(ProbeFailure.Timeout, `${url.origin} did not answer in time.`)
709
+ : new EndpointProbeError(ProbeFailure.Unreachable, `Could not reach ${url.origin}.`);
710
+ };
711
+ let response;
712
+ try {
713
+ response = await fetch(url, { headers: authHeaders(endpoint.apiKey), signal: within, redirect: 'manual' });
714
+ }
715
+ catch {
716
+ throw thrown();
717
+ }
718
+ const { status } = response;
719
+ // A browser hands a refused redirect back as an opaque response with status zero.
720
+ if (response.type === 'opaqueredirect' || (status >= REDIRECT_MIN && status <= REDIRECT_MAX)) {
721
+ const location = response.headers.get('location');
722
+ throw new EndpointProbeError(ProbeFailure.Redirected, `${url.origin} redirected the request${location ? ` to ${location}` : ''}; use the URL it redirects to.`, status || undefined);
723
+ }
724
+ if (UNAUTHORIZED.has(status)) {
725
+ throw new EndpointProbeError(ProbeFailure.Unauthorized, `${url.origin} refused the key (${status}).`, status);
726
+ }
727
+ if (NOT_THERE.has(status)) {
728
+ throw new EndpointProbeError(ProbeFailure.NotOpenAi, `${url.origin} has no model listing there (${status}); is the base URL missing a /v1?`, status);
729
+ }
730
+ if (response.ok === false) {
731
+ throw new EndpointProbeError(ProbeFailure.Failed, `${url.origin} answered ${status} when asked for its models.`, status);
732
+ }
733
+ let body;
734
+ try {
735
+ body = await response.json();
736
+ }
737
+ catch {
738
+ if (within.aborted) {
739
+ throw thrown();
740
+ }
741
+ body = undefined;
742
+ }
743
+ const { data, models } = (body ?? {});
744
+ const rows = Array.isArray(data) ? data : Array.isArray(models) ? models : undefined;
745
+ if (rows === undefined) {
746
+ throw new EndpointProbeError(ProbeFailure.NotOpenAi, `${url.origin} did not answer with a list of models.`);
747
+ }
748
+ const found = new Map();
749
+ for (const row of rows) {
750
+ if (row === null || typeof row !== 'object') {
751
+ continue;
752
+ }
753
+ const entry = row;
754
+ const id = typeof entry.id === 'string' ? entry.id : typeof entry.name === 'string' ? entry.name : '';
755
+ if (id === '') {
756
+ continue;
757
+ }
758
+ const { contextLength } = contextLengthOf(row);
759
+ found.set(id, { id, contextLength: contextLength || (found.get(id)?.contextLength ?? 0) });
760
+ }
761
+ return [...found.values()].sort((a, b) => byCodeUnit(a.id, b.id)).slice(0, limit);
762
+ }
597
763
  /**
598
764
  * How much a model will read, in tokens. Zero means nobody knows.
599
765
  *
@@ -692,14 +858,14 @@ export function resetClients() {
692
858
  refusals().clear();
693
859
  }
694
860
  /**
695
- * A base URL as two settings rows would agree on it: trimmed, without the trailing slash.
861
+ * Whether two base URLs are one endpoint, by `baseUrlKey`.
696
862
  *
697
863
  * @param a - The URL as one row has it.
698
864
  * @param b - The URL as the other has it.
699
- * @returns `true` where the two differ in nothing but surrounding whitespace and trailing slashes.
700
- * Case counts, and so does a `/v1`.
865
+ * @returns `true` where the two differ in nothing but surrounding whitespace, trailing slashes, the
866
+ * case of the scheme and host, and a default port. A path's case counts, and so does a `/v1`.
701
867
  */
702
- export const sameUrl = (a, b) => a.trim().replace(/\/+$/, '') === b.trim().replace(/\/+$/, '');
868
+ export const sameUrl = (a, b) => baseUrlKey(a) === baseUrlKey(b);
703
869
  /**
704
870
  * The key to send, where an endpoint may inherit one from the settings it overrides.
705
871
  *
@@ -1,3 +1,4 @@
1
+ import { baseUrlKey } from '../core/config.js';
1
2
  import { scoped } from '../core/scope.js';
2
3
  import { modelKey } from './client.js';
3
4
  /**
@@ -22,14 +23,14 @@ const pools = scoped(() => ({ slots: new Map(), clock: 0 }));
22
23
  *
23
24
  * @param endpoint - The server, read for `baseUrl` alone.
24
25
  * @param model - The model, which a llama.cpp router serves from a process with slots of its own.
25
- * @returns The key: the base URL less its trailing slashes, and the model.
26
+ * @returns The key: the base URL as `baseUrlKey` writes it, and the model.
26
27
  *
27
28
  * @remarks
28
29
  * The API key is left out where every other per-endpoint cache keys on it. Two keys reaching
29
30
  * different servers through one URL cost only a slot left unused; two keys reaching the same
30
31
  * server under two allocators hand one slot to two runs, which is the collision this exists to stop.
31
32
  */
32
- const poolKey = (endpoint, model) => modelKey(endpoint.baseUrl.trim().replace(/\/+$/, ''), model);
33
+ const poolKey = (endpoint, model) => modelKey(baseUrlKey(endpoint.baseUrl), model);
33
34
  /**
34
35
  * The slot allocator for one endpoint and model, sized to what its `/props` said.
35
36
  *
package/dist/index.d.ts CHANGED
@@ -11,10 +11,10 @@
11
11
  */
12
12
  export { applyCompaction, COMPACT_AT, type CompactionOptions, type CompactionPlan, type CompactionRecord, type CompactionRunOptions, compactTranscript, KEEP_RATIO, type PruneOptions, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, } from './context/compaction.ts';
13
13
  export { estimateFrom, type LedgerEntry, type LedgerEstimateOptions, type LedgerRequest, rebaseLedger, recordRequest, type TokenLedger, tokensBetween, } from './context/ledger.ts';
14
- export { type AgentConfig, type Endpoint, type ModelParams, type RetryPolicy, ToolDiscovery, type ToolPolicy, } from './core/config.ts';
14
+ export { type AgentConfig, type Endpoint, type ModelParams, normalizeBaseUrl, type RetryPolicy, ToolDiscovery, type ToolPolicy, } from './core/config.ts';
15
15
  export { calibrate, charsPerTokenFor, resetCalibration } from './endpoint/calibration.ts';
16
16
  export { type Capabilities, capabilitiesFor, EFFORT_LADDER, EFFORT_NONE, EFFORT_OFF, effortFor, expireCapabilities, type ModelCapabilities, modelCapabilitiesFor, type NegotiateOptions, negotiate, resetCapabilities, } from './endpoint/capabilities.ts';
17
- export { type ClientPoolOptions, configureClients, contextLimitFor, endpointId, endpointKey, firstTokenMs, getClient, type LookupOptions, listModels, type ModelInfo, NO_KEY, resetClients, resolveApiKey, servedSlots, servedWindow, timeoutMs, } from './endpoint/client.ts';
17
+ export { type ClientPoolOptions, configureClients, contextLimitFor, EndpointProbeError, endpointId, endpointKey, firstTokenMs, getClient, type LookupOptions, listModels, type ModelInfo, NO_KEY, ProbeFailure, type ProbeOptions, probeEndpoint, resetClients, resolveApiKey, servedSlots, servedWindow, timeoutMs, } from './endpoint/client.ts';
18
18
  export { isGrammarError, relaxSchema, relaxTools, sanitizeSchema, sanitizeTools, } from './endpoint/schema-compat.ts';
19
19
  export { CACHE_PROMPT_FIELD, resetSlots, SESSION_HEADER, SLOT_FIELD, type SlotAllocator, slotAllocator, } from './endpoint/slots.ts';
20
20
  export { HOOK_EVENTS, HookEvent, INJECT_EVENTS } from './hooks/hook-events.ts';
@@ -39,5 +39,5 @@ export { backoffMs, ContextOverflow, EndpointSilent, isModelLoading, isOverflow,
39
39
  export { CacheBreakReason, type Produced, type StreamTurnOptions, streamTurn, type Turn, type TurnUsage, } from './wire/stream.ts';
40
40
  export { ALL_FENCES, DEFAULT_FENCES, type Fence, FenceSplitter, type FenceSplitterOptions, SPLIT_OUTPUT, SPLIT_REASONING, type Split, stripThinking, THINK_FENCE, } from './wire/thinking.ts';
41
41
  export { type ContextBreakdown, type ContextBreakdownOptions, compact, contextChars, contextTokens, estimateTokens, messageTokens, requestChars, requestTokens, type TokenEstimateOptions, } from './wire/tokens.ts';
42
- export { ARGUMENTS_MALFORMED, ARGUMENTS_TRUNCATED, parseToolArguments, type ResourceLink, type RunArtifact, recoverToolCalls, type ServerLogEntry, ServerLogLevel, type ToolAnswer, ToolArgumentsError, type ToolCall, type ToolCallOutcome, type ToolCallReport, type ToolCallRequest, type ToolCallResult, } from './wire/tool-calls.ts';
42
+ export { ARGUMENTS_MALFORMED, ARGUMENTS_TRUNCATED, type EndedBy, parseToolArguments, type ResourceLink, type RunArtifact, recoverToolCalls, type ServerLogEntry, ServerLogLevel, type ToolAnswer, ToolArgumentsError, type ToolCall, type ToolCallOutcome, type ToolCallReport, type ToolCallRequest, type ToolCallResult, } from './wire/tool-calls.ts';
43
43
  export { FinishReason, FUNCTION_TOOL, JSON_SCHEMA_FORMAT, PartType, Role, SchemaType } from './wire/wire.ts';
package/dist/index.js CHANGED
@@ -11,10 +11,10 @@
11
11
  */
12
12
  export { applyCompaction, COMPACT_AT, compactTranscript, KEEP_RATIO, planCompaction, pruneToolResults, requestIndex, runCompaction, SUMMARY_LEAD, SUMMARY_PROMPT, summariser, } from './context/compaction.js';
13
13
  export { estimateFrom, rebaseLedger, recordRequest, tokensBetween, } from './context/ledger.js';
14
- export { ToolDiscovery, } from './core/config.js';
14
+ export { normalizeBaseUrl, ToolDiscovery, } from './core/config.js';
15
15
  export { calibrate, charsPerTokenFor, resetCalibration } from './endpoint/calibration.js';
16
16
  export { capabilitiesFor, EFFORT_LADDER, EFFORT_NONE, EFFORT_OFF, effortFor, expireCapabilities, modelCapabilitiesFor, negotiate, resetCapabilities, } from './endpoint/capabilities.js';
17
- export { configureClients, contextLimitFor, endpointId, endpointKey, firstTokenMs, getClient, listModels, NO_KEY, resetClients, resolveApiKey, servedSlots, servedWindow, timeoutMs, } from './endpoint/client.js';
17
+ export { configureClients, contextLimitFor, EndpointProbeError, endpointId, endpointKey, firstTokenMs, getClient, listModels, NO_KEY, ProbeFailure, probeEndpoint, resetClients, resolveApiKey, servedSlots, servedWindow, timeoutMs, } from './endpoint/client.js';
18
18
  export { isGrammarError, relaxSchema, relaxTools, sanitizeSchema, sanitizeTools, } from './endpoint/schema-compat.js';
19
19
  export { CACHE_PROMPT_FIELD, resetSlots, SESSION_HEADER, SLOT_FIELD, slotAllocator, } from './endpoint/slots.js';
20
20
  export { HOOK_EVENTS, HookEvent, INJECT_EVENTS } from './hooks/hook-events.js';
@@ -10,7 +10,7 @@ import { type KeywordPreselectOptions } from '../tools/preselect.ts';
10
10
  import { type ToolOrder } from '../tools/tool-loading.ts';
11
11
  import { type SideTask } from '../turn/side-task.ts';
12
12
  import { type Turn, type TurnUsage } from '../wire/stream.ts';
13
- import { type RunArtifact, type ToolCallOutcome, type ToolCallRequest, type ToolCallResult } from '../wire/tool-calls.ts';
13
+ import { type EndedBy, type RunArtifact, type ToolCallOutcome, type ToolCallRequest, type ToolCallResult } from '../wire/tool-calls.ts';
14
14
  import { type RunEventInput, type RunMetrics } from './events.ts';
15
15
  import { type ToolDispatch } from './run-calls.ts';
16
16
  /**
@@ -243,7 +243,8 @@ export interface AgentLoopOptions {
243
243
  * @remarks
244
244
  * A `ToolAnswer` in place of the string adds what the call produced: its `text` is the whole of
245
245
  * what goes into the transcript, and its `links` become the result's `artifacts`, an `artifact`
246
- * event the first time each uri is seen, and `links` on what `onToolResult` is handed.
246
+ * event the first time each uri is seen, and `links` on what `onToolResult` is handed. Its `end`
247
+ * ends the run once the step's calls are answered; see `AgentLoopResult.ended`.
247
248
  */
248
249
  dispatch: ToolDispatch;
249
250
  /**
@@ -395,8 +396,29 @@ export interface AgentLoopOptions {
395
396
  }
396
397
  /** What a finished loop hands back. */
397
398
  export interface AgentLoopResult {
398
- /** The last turn: the one that asked for no tools, as `onTurn` was handed it. */
399
+ /**
400
+ * The last turn, as `onTurn` was handed it: the one that asked for no tools, or — where a call
401
+ * `ended` the run — the one that made that call.
402
+ *
403
+ * @remarks
404
+ * Ended by a call, its `content` is what the model wrote beside its calls, which is often
405
+ * nothing, and its `toolCalls` are not empty. The run's answer is then whatever the ending tool
406
+ * took — the question it asked, the work it handed off — and not this text.
407
+ */
399
408
  turn: Turn;
409
+ /**
410
+ * The call whose answer said `end`, where one did: the first in the order the model asked, once
411
+ * every call of its step was answered. Absent where the run ended on a turn that asked for no
412
+ * tools.
413
+ *
414
+ * @remarks
415
+ * Every other field is as a finished run has it: `messages` close on the step's tool results
416
+ * with no assistant turn after them, the hooks are told `afterTurn` with `turn.content`, and
417
+ * `metrics.outcome` is `ended`. No `beforeStep` or `compact` runs after the end, since no request
418
+ * follows it. A host saying `done` passes the call on it — `id` the call, `name` the tool, `text`
419
+ * the reason — which is how `runMetrics` reads the same outcome off the events.
420
+ */
421
+ ended?: EndedBy;
400
422
  /** The transcript, with every assistant turn and tool result the run added. No system prompt. */
401
423
  messages: OpenAI.ChatCompletionMessageParam[];
402
424
  /** Summed over every turn of the run. */
@@ -463,14 +485,16 @@ export interface AgentLoopResult {
463
485
  }
464
486
  /**
465
487
  * Runs a question to its answer: one `runTurn` per step, the tools it asks for between them,
466
- * until a turn asks for none. Throws when `maxToolIterations` is spent, when stopped, and on
467
- * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
488
+ * until a turn asks for none or a call's answer ends the run. Throws when `maxToolIterations` is
489
+ * spent, when stopped, and on whatever `runTurn` throws — `ContextOverflow` among them, however it
490
+ * was found out.
468
491
  *
469
492
  * @param options - The config, transcript, tools and dispatcher, plus the optional hooks, events
470
493
  * and cancellation. See `AgentLoopOptions`.
471
- * @returns The turn that asked for no tools and the run around it: the transcript with what the
472
- * run added, usage, every call, what is loaded and used, the hooks' notes and context, metrics
473
- * and the ledger. See `AgentLoopResult`.
494
+ * @returns The turn that asked for no tools — or made the call that ended the run, with `ended`
495
+ * naming it — and the run around it: the transcript with what the run added, usage, every call,
496
+ * what is loaded and used, the hooks' notes and context, metrics and the ledger. See
497
+ * `AgentLoopResult`.
474
498
  *
475
499
  * @remarks
476
500
  * What it throws carries the run as it stood — `messages`, `usage`, `toolCalls`, `loaded`, `used`
@@ -482,6 +506,11 @@ export interface AgentLoopResult {
482
506
  * well-formed: the results that came back are kept, and each call without one is answered with a
483
507
  * line saying it was stopped or never run, since a call with no result is refused on replay.
484
508
  *
509
+ * A tool that finishes the job — a hand-off, a question for a person — ends the run with a
510
+ * `ToolAnswer` whose `end` is set, rather than by aborting it: the step's calls are all run and
511
+ * answered, no further request is sent, and the loop resolves with `ended`. An end in the last step
512
+ * `maxToolIterations` allows resolves too, since nothing was left undone.
513
+ *
485
514
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
486
515
  * prompt unchanged from step to step, a load adds to the tool array — which every request sends
487
516
  * in the stable order `toolOrder` asks for — and
@@ -63,14 +63,16 @@ function cacheDiagnosis(previous, next, usage) {
63
63
  }
64
64
  /**
65
65
  * Runs a question to its answer: one `runTurn` per step, the tools it asks for between them,
66
- * until a turn asks for none. Throws when `maxToolIterations` is spent, when stopped, and on
67
- * whatever `runTurn` throws — `ContextOverflow` among them, however it was found out.
66
+ * until a turn asks for none or a call's answer ends the run. Throws when `maxToolIterations` is
67
+ * spent, when stopped, and on whatever `runTurn` throws — `ContextOverflow` among them, however it
68
+ * was found out.
68
69
  *
69
70
  * @param options - The config, transcript, tools and dispatcher, plus the optional hooks, events
70
71
  * and cancellation. See `AgentLoopOptions`.
71
- * @returns The turn that asked for no tools and the run around it: the transcript with what the
72
- * run added, usage, every call, what is loaded and used, the hooks' notes and context, metrics
73
- * and the ledger. See `AgentLoopResult`.
72
+ * @returns The turn that asked for no tools — or made the call that ended the run, with `ended`
73
+ * naming it — and the run around it: the transcript with what the run added, usage, every call,
74
+ * what is loaded and used, the hooks' notes and context, metrics and the ledger. See
75
+ * `AgentLoopResult`.
74
76
  *
75
77
  * @remarks
76
78
  * What it throws carries the run as it stood — `messages`, `usage`, `toolCalls`, `loaded`, `used`
@@ -82,6 +84,11 @@ function cacheDiagnosis(previous, next, usage) {
82
84
  * well-formed: the results that came back are kept, and each call without one is answered with a
83
85
  * line saying it was stopped or never run, since a call with no result is refused on replay.
84
86
  *
87
+ * A tool that finishes the job — a hand-off, a question for a person — ends the run with a
88
+ * `ToolAnswer` whose `end` is set, rather than by aborting it: the step's calls are all run and
89
+ * answered, no further request is sent, and the loop resolves with `ended`. An end in the last step
90
+ * `maxToolIterations` allows resolves too, since nothing was left undone.
91
+ *
85
92
  * On-demand loading is handled here, `load_tools` and all: the catalogue rides on the system
86
93
  * prompt unchanged from step to step, a load adds to the tool array — which every request sends
87
94
  * in the stable order `toolOrder` asks for — and
@@ -471,8 +478,8 @@ function turnCalls(turn, recoverable, notice) {
471
478
  * @param standing - Written to: once the steps have set themselves up, its `read` is theirs, so the
472
479
  * caller can read the run as it stands whether they return or throw.
473
480
  * @param [pin] - What the run's session adds to each request. Absent adds nothing.
474
- * @returns The result of the turn that asked for no tools, or `undefined` when `maxToolIterations`
475
- * is spent.
481
+ * @returns The result of the turn that asked for no tools or whose calls ended the run, or
482
+ * `undefined` when `maxToolIterations` is spent.
476
483
  */
477
484
  async function runSteps(options, standing, pin) {
478
485
  const { config, system = '', tools = [], catalog = [], dispatch, hooks, signal } = options;
@@ -706,13 +713,27 @@ async function runSteps(options, standing, pin) {
706
713
  const assistant = assistantMessage(content, parsed);
707
714
  messages.push(assistant);
708
715
  await announce(assistant, step, shown);
709
- if (!calls.length) {
716
+ const isAnswer = calls.length === 0;
717
+ const ended = isAnswer ? undefined : await runCalls(calling, parsed, messages, step);
718
+ if (ended) {
719
+ const by = ended.reason ? `: ${ended.reason}` : '';
720
+ notice(`${ended.name} ended the run${by}`);
721
+ }
722
+ if (isAnswer || ended) {
710
723
  // Not awaited: the answer is ready, and remembering it is not something to hold it for.
711
724
  // Handed back instead, for a host that wants its notes.
712
725
  const afterTurn = hooks ? tellAfterTurn(hooks, messages, questionAt(), turn.content) : Promise.resolve([]);
713
726
  const metrics = runMetrics(recorded, { contextLength: config.contextLength });
727
+ let outcome = RunOutcome.Answered;
728
+ if (ended) {
729
+ outcome = RunOutcome.Ended;
730
+ }
731
+ else if (turn.finishReason === FinishReason.Length) {
732
+ outcome = RunOutcome.Truncated;
733
+ }
714
734
  return {
715
735
  turn: shown,
736
+ ...(ended ? { ended } : {}),
716
737
  messages,
717
738
  usage,
718
739
  toolCalls,
@@ -728,12 +749,11 @@ async function runSteps(options, standing, pin) {
728
749
  ...metrics,
729
750
  ...(onDemand ? loads : {}),
730
751
  wallMs: Date.now() - started,
731
- outcome: turn.finishReason === FinishReason.Length ? RunOutcome.Truncated : RunOutcome.Answered,
752
+ outcome,
732
753
  },
733
754
  ledger,
734
755
  };
735
756
  }
736
- await runCalls(calling, parsed, messages, step);
737
757
  }
738
758
  return undefined;
739
759
  }
@@ -109,7 +109,10 @@ export declare const RunEventKind: {
109
109
  * measured either way.
110
110
  */
111
111
  readonly Usage: 'usage';
112
- /** The run ended. Always last, and always sent. */
112
+ /**
113
+ * The run ended. Always last, and always sent. A run a tool call ended carries that call: `id`
114
+ * the call's, `name` the tool, and `text` the reason, as `AgentLoopResult.ended` has them.
115
+ */
113
116
  readonly Done: 'done';
114
117
  };
115
118
  /** Any one of the kinds in `RunEventKind`. */
@@ -171,7 +174,7 @@ export interface RunEvent {
171
174
  kind: RunEventKind;
172
175
  /** The delta, the arguments, the result, or the reason — whatever the kind carries. */
173
176
  text: string;
174
- /** Tool name on the tool kinds, otherwise empty. */
177
+ /** Tool name on the tool kinds, and on a `done` the tool that ended the run; otherwise empty. */
175
178
  name: string;
176
179
  /**
177
180
  * The call's id on the tool kinds, which pairs a `tool-result` with its `tool-call` when one
@@ -390,6 +393,11 @@ export declare const RunOutcome: {
390
393
  readonly Answered: 'answered';
391
394
  /** The run did not fail, and its last turn is an answer the reply ceiling cut off. */
392
395
  readonly Truncated: 'truncated';
396
+ /**
397
+ * The run did not fail, and a tool call ended it: `AgentLoopResult.ended` says which, and so
398
+ * does a `done` that names the tool.
399
+ */
400
+ readonly Ended: 'ended';
393
401
  /**
394
402
  * `done` said the run was not ok. Whether that was an error, a stop or the tool budget is in
395
403
  * the host's own `done` text and not here.
@@ -76,7 +76,10 @@ export const RunEventKind = {
76
76
  * measured either way.
77
77
  */
78
78
  Usage: 'usage',
79
- /** The run ended. Always last, and always sent. */
79
+ /**
80
+ * The run ended. Always last, and always sent. A run a tool call ended carries that call: `id`
81
+ * the call's, `name` the tool, and `text` the reason, as `AgentLoopResult.ended` has them.
82
+ */
80
83
  Done: 'done',
81
84
  };
82
85
  /**
@@ -471,6 +474,11 @@ export const RunOutcome = {
471
474
  Answered: 'answered',
472
475
  /** The run did not fail, and its last turn is an answer the reply ceiling cut off. */
473
476
  Truncated: 'truncated',
477
+ /**
478
+ * The run did not fail, and a tool call ended it: `AgentLoopResult.ended` says which, and so
479
+ * does a `done` that names the tool.
480
+ */
481
+ Ended: 'ended',
474
482
  /**
475
483
  * `done` said the run was not ok. Whether that was an error, a stop or the tool budget is in
476
484
  * the host's own `done` text and not here.
@@ -480,15 +488,21 @@ export const RunOutcome = {
480
488
  /**
481
489
  * How a run that reported `done` ended.
482
490
  *
483
- * @param ok - What the `done` event said. Only `false` is a failure; anything else is read off the turn.
491
+ * @param done - The `done` event. Only an `ok` of `false` is a failure; one that names a tool was
492
+ * ended by it; anything else is read off the turn.
484
493
  * @param last - The last turn a `usage` event reported, if any did.
485
- * @returns `failed` for a `done` that was not ok, `truncated` where the last turn ran into the
486
- * reply ceiling, and `answered` otherwise.
494
+ * @returns `failed` for a `done` that was not ok, `ended` for one that names the tool that ended
495
+ * the run, `truncated` where the last turn ran into the reply ceiling, and `answered` otherwise.
487
496
  */
488
- function outcomeOf(ok, last) {
497
+ function outcomeOf({ ok, name }, last) {
489
498
  if (ok === false) {
490
499
  return RunOutcome.Failed;
491
500
  }
501
+ // Read as a truthy string, so an event put together by hand without one is not ended.
502
+ const isEnded = Boolean(name);
503
+ if (isEnded) {
504
+ return RunOutcome.Ended;
505
+ }
492
506
  if (last?.finishReason === FinishReason.Length) {
493
507
  return RunOutcome.Truncated;
494
508
  }
@@ -614,7 +628,7 @@ export function runMetrics(events, { contextLength } = {}) {
614
628
  }
615
629
  }
616
630
  else if (event.kind === RunEventKind.Done) {
617
- metrics.outcome = outcomeOf(event.ok, last);
631
+ metrics.outcome = outcomeOf(event, last);
618
632
  }
619
633
  }
620
634
  if (events.length > 1) {
@@ -1,7 +1,7 @@
1
1
  import type OpenAI from 'openai';
2
2
  import type { CatalogServer } from '../tools/catalog.ts';
3
3
  import type { Turn } from '../wire/stream.ts';
4
- import { type RunArtifact, type ToolAnswer, type ToolCall, type ToolCallOutcome, type ToolCallReport, type ToolCallRequest, type ToolCallResult } from '../wire/tool-calls.ts';
4
+ import { type EndedBy, type RunArtifact, type ToolAnswer, type ToolCall, type ToolCallOutcome, type ToolCallReport, type ToolCallRequest, type ToolCallResult } from '../wire/tool-calls.ts';
5
5
  import type { RunEventInput } from './events.ts';
6
6
  /**
7
7
  * The tools between two turns: a step's calls read, run and written into the transcript.
@@ -114,13 +114,21 @@ export interface Calling {
114
114
  * @param parsed - The step's calls, with their arguments read.
115
115
  * @param messages - The transcript the results are written into.
116
116
  * @param step - Which step this is, for the host told of each result.
117
+ * @returns The first call, in the order asked, whose answer ended the run, or `undefined` where
118
+ * none did.
117
119
  *
118
120
  * @remarks
119
121
  * A failure part-way leaves the transcript well-formed before it is rethrown: a call with no
120
122
  * result is a transcript no endpoint takes back, so every call of the step is answered — with what
121
123
  * it returned where it had, and otherwise with a line saying it was stopped, or never run.
124
+ *
125
+ * An answer that ends the run does not end the step. The calls after it still run, in whatever
126
+ * batch `parallel` puts them, because the model asked for them in the same breath and how a host
127
+ * schedules calls is not what they mean: a reply that says "write the file, then hand off" in one
128
+ * turn wants the file written whichever order it named the two in, and a call batched beside the
129
+ * end was running already. A tool that must have nothing after it says so in its description.
122
130
  */
123
- export declare function runCalls(run: Calling, parsed: ReadCall[], messages: OpenAI.ChatCompletionMessageParam[], step: number): Promise<void>;
131
+ export declare function runCalls(run: Calling, parsed: ReadCall[], messages: OpenAI.ChatCompletionMessageParam[], step: number): Promise<EndedBy | undefined>;
124
132
  /**
125
133
  * Writes a proxied preselection into the transcript as a `load_tools` exchange: the tool array is
126
134
  * fixed, so a shortlist has nowhere else to go, and it is answered as though the model had loaded
@@ -128,9 +128,19 @@ function reportFor(run, id, name) {
128
128
  * A dispatcher's answer in the one shape, whichever it gave.
129
129
  *
130
130
  * @param answer - What `dispatch` resolved to.
131
- * @returns The text and the links, a string answer's being none.
131
+ * @returns The text, the links and whether it ends the run, a string answer having no links and
132
+ * ending nothing.
132
133
  */
133
- const answerOf = (answer) => typeof answer === 'string' ? { text: answer, links: [] } : { text: answer.text, links: answer.links ?? [] };
134
+ function answerOf(answer) {
135
+ if (typeof answer === 'string') {
136
+ return { text: answer, links: [], end: false };
137
+ }
138
+ const { text, links = [], end = false } = answer;
139
+ if (typeof end === 'object') {
140
+ return { text, links, end: { reason: end.reason } };
141
+ }
142
+ return { text, links, end: end ? {} : false };
143
+ }
134
144
  /**
135
145
  * What a call is shown, counted and dispatched as: a `call_tool` as the tool it names.
136
146
  *
@@ -322,9 +332,9 @@ function batchEnd(run, ready, from, total, answered) {
322
332
  * @param prepared - The call, with its request built.
323
333
  * @param answered - The step's calls already made, by tool and arguments, for `dedupeToolCalls`.
324
334
  * @param step - Which step this is, for the artifacts the call's links become.
325
- * @returns The call's id, the name it was shown under, whether it worked, the whole of what the
326
- * model reads, and the links its answer carried, where it carried any. Rejects where the run was
327
- * stopped or a host callback threw.
335
+ * @returns The result — the call's id, the name it was shown under, whether it worked, the whole
336
+ * of what the model reads, and the links its answer carried, where it carried any — and whether
337
+ * the answer ended the run. Rejects where the run was stopped or a host callback threw.
328
338
  */
329
339
  async function runCall(run, prepared, answered, step) {
330
340
  const { catalog, onDemand, proxied, signal } = run;
@@ -336,6 +346,7 @@ async function runCall(run, prepared, answered, step) {
336
346
  const { report, close } = reportFor(run, call.id, name);
337
347
  let content;
338
348
  let links = [];
349
+ let end = false;
339
350
  let ok = true;
340
351
  // Answered from an earlier identical call of the step, whose links are already counted.
341
352
  let shared = false;
@@ -364,7 +375,7 @@ async function runCall(run, prepared, answered, step) {
364
375
  const answer = dedupable
365
376
  ? await once(answered, key, () => run.dispatch(request, signal, report))
366
377
  : await run.dispatch(request, signal, report);
367
- ({ text: content, links } = answerOf(answer));
378
+ ({ text: content, links, end } = answerOf(answer));
368
379
  }
369
380
  }
370
381
  catch (error) {
@@ -384,7 +395,7 @@ async function runCall(run, prepared, answered, step) {
384
395
  }
385
396
  const result = { id: call.id, name, ok, content, ...(links.length ? { links } : {}) };
386
397
  run.onToolResult?.(result);
387
- return result;
398
+ return { result, end };
388
399
  }
389
400
  /**
390
401
  * Runs a step's calls and writes each result into the transcript, in the order asked.
@@ -393,11 +404,19 @@ async function runCall(run, prepared, answered, step) {
393
404
  * @param parsed - The step's calls, with their arguments read.
394
405
  * @param messages - The transcript the results are written into.
395
406
  * @param step - Which step this is, for the host told of each result.
407
+ * @returns The first call, in the order asked, whose answer ended the run, or `undefined` where
408
+ * none did.
396
409
  *
397
410
  * @remarks
398
411
  * A failure part-way leaves the transcript well-formed before it is rethrown: a call with no
399
412
  * result is a transcript no endpoint takes back, so every call of the step is answered — with what
400
413
  * it returned where it had, and otherwise with a line saying it was stopped, or never run.
414
+ *
415
+ * An answer that ends the run does not end the step. The calls after it still run, in whatever
416
+ * batch `parallel` puts them, because the model asked for them in the same breath and how a host
417
+ * schedules calls is not what they mean: a reply that says "write the file, then hand off" in one
418
+ * turn wants the file written whichever order it named the two in, and a call batched beside the
419
+ * end was running already. A tool that must have nothing after it says so in its description.
401
420
  */
402
421
  export async function runCalls(run, parsed, messages, step) {
403
422
  const { signal, toolCalls } = run;
@@ -406,6 +425,7 @@ export async function runCalls(run, parsed, messages, step) {
406
425
  const answered = new Map();
407
426
  // By position, so a call that came back while another was being stopped is still found.
408
427
  const outcomes = parsed.map(() => undefined);
428
+ const ends = parsed.map(() => false);
409
429
  // How many of the step's calls were handed off, and how many have their result in the
410
430
  // transcript. Both count from the front, since results are written in the order asked.
411
431
  let begun = 0;
@@ -436,7 +456,9 @@ export async function runCalls(run, parsed, messages, step) {
436
456
  const to = batchEnd(run, ready, start, parsed.length, answered);
437
457
  begun = to;
438
458
  await Promise.all(parsed.slice(start, to).map(async (_, offset) => {
439
- outcomes[start + offset] = await runCall(run, ready(start + offset), answered, step);
459
+ const { result, end } = await runCall(run, ready(start + offset), answered, step);
460
+ outcomes[start + offset] = result;
461
+ ends[start + offset] = end;
440
462
  }));
441
463
  for (const { id, name, ok, content } of outcomes.slice(from, to).flatMap((outcome) => outcome ?? [])) {
442
464
  toolCalls.push({ id, name, ok });
@@ -444,6 +466,15 @@ export async function runCalls(run, parsed, messages, step) {
444
466
  }
445
467
  from = to;
446
468
  }
469
+ // The first by position rather than by finishing: a batch's calls end in no order worth
470
+ // reporting, and a deduplicated repeat shares its answer's `end` with the call before it.
471
+ const at = ends.findIndex((end) => end !== false);
472
+ const end = ends[at];
473
+ const outcome = outcomes[at];
474
+ if (end === undefined || end === false || outcome === undefined) {
475
+ return undefined;
476
+ }
477
+ return { callId: outcome.id, name: outcome.name, ...(end.reason === undefined ? {} : { reason: end.reason }) };
447
478
  }
448
479
  catch (error) {
449
480
  // Only a call that was made counts in `toolCalls`.
@@ -188,6 +188,27 @@ export interface ToolAnswer {
188
188
  text: string;
189
189
  /** What the call produced, in the order the tool gave them. Never sent to the model. */
190
190
  links?: ResourceLink[];
191
+ /**
192
+ * Ends the run once this step's calls are answered: no further request is sent, and the loop
193
+ * resolves with `ended` rather than throwing. An object says why, for the host and its `done`.
194
+ *
195
+ * @remarks
196
+ * The step's other calls still run and are answered, so the transcript stays one an endpoint
197
+ * will take back. For a tool whose call is the end of the job — handing off, asking a person —
198
+ * where the alternative is an abort, which the loop reads as a failure.
199
+ */
200
+ end?: boolean | {
201
+ reason: string;
202
+ };
203
+ }
204
+ /** The call that ended a run through its answer's `end`, as `AgentLoopResult.ended` says it. */
205
+ export interface EndedBy {
206
+ /** The call's id. */
207
+ callId: string;
208
+ /** The tool, under the name it was announced by: for a `call_tool`, the tool it names. */
209
+ name: string;
210
+ /** Why, where the answer said: absent for `end: true`. */
211
+ reason?: string;
191
212
  }
192
213
  /**
193
214
  * The levels a server's log message is sent at, quietest first: MCP's, which are syslog's.
@@ -68,6 +68,7 @@ export declare const HttpStatus: {
68
68
  readonly Ok: 200;
69
69
  readonly BadRequest: 400;
70
70
  readonly Unauthorized: 401;
71
+ readonly Forbidden: 403;
71
72
  readonly NotFound: 404;
72
73
  readonly MethodNotAllowed: 405;
73
74
  readonly RequestTimeout: 408;
package/dist/wire/wire.js CHANGED
@@ -60,6 +60,7 @@ export const HttpStatus = {
60
60
  Ok: 200,
61
61
  BadRequest: 400,
62
62
  Unauthorized: 401,
63
+ Forbidden: 403,
63
64
  NotFound: 404,
64
65
  MethodNotAllowed: 405,
65
66
  RequestTimeout: 408,
package/llms.txt CHANGED
@@ -50,6 +50,7 @@ What this package needs to know about a caller's configuration.
50
50
  - `AgentConfig` (type) — A whole agent configuration — every part, plus the two fields that belong to no group.
51
51
  - `Endpoint` (type) — Where to send a request and how long to wait.
52
52
  - `ModelParams` (type) — What to ask the model for.
53
+ - `normalizeBaseUrl` — A base URL worth storing: http or https, no credentials, query or fragment, no trailing slash.
53
54
  - `RetryPolicy` (type) — How many times a lost or refused request is worth sending again.
54
55
  - `ToolDiscovery` — Any one of the modes in `ToolDiscovery`.
55
56
  - `ToolPolicy` (type) — How tools reach the model, and how long it may keep calling them.
@@ -84,6 +85,7 @@ What an endpoint turned out not to support, and answering it when it says so.
84
85
  - `ClientPoolOptions` (type) — What the client pool is held to across a process.
85
86
  - `configureClients` — Changes what the client pool is held to, for a process whose endpoints are not shaped like the deployments these defaults were chosen for.
86
87
  - `contextLimitFor` — How much a model will read, in tokens.
88
+ - `EndpointProbeError` — `probeEndpoint` could not list an endpoint's models; `kind` says why.
87
89
  - `endpointId` — `endpointKey` hashed, for the remembered facts that can leave the process.
88
90
  - `endpointKey` — What counts as one endpoint, everywhere in this package that has to remember something about one — the model listings, `capabilities`, and the no-thinking hints in `side-task`.
89
91
  - `firstTokenMs` — The wait for a streamed turn's first chunk, in the SDK's spelling: `undefined` is no limit.
@@ -92,6 +94,9 @@ What an endpoint turned out not to support, and answering it when it says so.
92
94
  - `listModels` — Asks an endpoint what it serves, and remembers the answer.
93
95
  - `ModelInfo` (type) — A model an endpoint offers, and what it says the model will read.
94
96
  - `NO_KEY` — The SDK insists on a non-empty key even where the server will not look at it.
97
+ - `ProbeFailure` — Why `probeEndpoint` could not list an endpoint's models, in words a form can branch on.
98
+ - `ProbeOptions` (type) — What `probeEndpoint` takes besides the endpoint.
99
+ - `probeEndpoint` — Asks an endpoint nobody has saved yet which models it serves, for a form a person is typing it into.
95
100
  - `resetClients` — Forgets every cached client and listing, and puts `configureClients` back to the defaults.
96
101
  - `resolveApiKey` — The key to send, where an endpoint may inherit one from the settings it overrides.
97
102
  - `servedSlots` — How many slots a llama.cpp server runs a model in, for pinning a session to one of them.
@@ -163,7 +168,7 @@ The loop above a turn: send, run the tools the model asked for, send again, unti
163
168
  - `PRESELECT_APPEND` — The `preselectRouting` that loads the shortlist after what was carried, and sends the first step like any other.
164
169
  - `PRESELECT_EXCLUSIVE` — The `preselectRouting` that sends the first step the shortlist alone.
165
170
  - `PRESELECTED_NOTICE` — What opens the notice naming the loop's own picks, ahead of the names.
166
- - `runAgentLoop` — Runs a question to its answer: one `runTurn` per step, the tools it asks for between them, until a turn asks for none.
171
+ - `runAgentLoop` — Runs a question to its answer: one `runTurn` per step, the tools it asks for between them, until a turn asks for none or a call's answer ends the run.
167
172
  - `StepWindow` (type) — How full the window is as a step is about to be sent, as `beforeStep` is told it.
168
173
 
169
174
  ### events
@@ -402,6 +407,7 @@ Reading what a model meant by a tool call when it did not write one cleanly.
402
407
 
403
408
  - `ARGUMENTS_MALFORMED` — The `ToolArgumentsError` kind for a call the model wrote wrongly.
404
409
  - `ARGUMENTS_TRUNCATED` — The `ToolArgumentsError` kind for a call cut off at the reply ceiling.
410
+ - `EndedBy` (type) — The call that ended a run through its answer's `end`, as `AgentLoopResult.ended` says it.
405
411
  - `parseToolArguments` — A tool call's arguments as the object the tool is handed.
406
412
  - `ResourceLink` (type) — A handle to something a tool call made or found — a document it wrote, a file it saved.
407
413
  - `RunArtifact` (type) — One thing a run's tool calls produced: the first link to a uri, and where it came from.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@cubicecho/agent-core",
3
- "version": "4.10.0",
3
+ "version": "4.12.0",
4
4
  "description": "The endpoint-agnostic half of an OpenAI-compatible agent loop: tool-schema compatibility, on-demand tool loading, one-shot side tasks, run events, and a pooled client.",
5
5
  "keywords": [
6
6
  "openai",