@juno-ai/bind 3.0.0 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +205 -62
- package/contracts/index.d.ts +1 -1
- package/contracts/index.js +1 -1
- package/contracts/turn.d.ts +26 -2
- package/contracts/turn.js +45 -0
- package/index.d.ts +9 -6
- package/index.js +9 -6
- package/loop/index.d.ts +1 -0
- package/loop/index.js +1 -0
- package/loop/tool-loop.d.ts +260 -0
- package/loop/tool-loop.js +276 -0
- package/package.json +6 -2
- package/run/children.d.ts +204 -0
- package/run/children.js +226 -0
- package/run/index.d.ts +1 -0
- package/run/index.js +1 -0
package/README.md
CHANGED
|
@@ -36,7 +36,8 @@ is; you do not need it to use the package.*
|
|
|
36
36
|
|
|
37
37
|
### What a harness is, and what it is not
|
|
38
38
|
|
|
39
|
-
A harness owns the parts of an agent loop that are the same for everyone:
|
|
39
|
+
A harness owns the parts of an agent loop that are the same for everyone: the
|
|
40
|
+
iteration itself — call the model, run what it asked for, repeat — plus
|
|
40
41
|
deciding which provider to call and what to do when it fails, bounding a run in
|
|
41
42
|
wall-clock time, keeping a transcript in a shape providers accept, rewriting
|
|
42
43
|
tool schemas that strict validators reject, and tracking which tools are
|
|
@@ -106,8 +107,15 @@ losing it.
|
|
|
106
107
|
Transports' request construction and wire-error classification, credentials and
|
|
107
108
|
environment parsing, your routing policy configuration, billing accounting
|
|
108
109
|
(persistence and charging), inference logging, authorization, prompt rendering,
|
|
109
|
-
and run orchestration — the queue a run is scheduled on, and
|
|
110
|
-
|
|
110
|
+
and run orchestration — the queue a run is scheduled on, and the enqueuing and
|
|
111
|
+
storage behind any child runs it spawns. The harness decides whether a child is
|
|
112
|
+
*allowed*; putting it on a queue is yours (see
|
|
113
|
+
[Spawning child runs](#spawning-child-runs-sub-agents)).
|
|
114
|
+
|
|
115
|
+
The loop is here, but the **driver** around it is not: starting a run, recording
|
|
116
|
+
what it did, delivering its output, and deciding when to run it again. That is
|
|
117
|
+
where a runtime's identity, storage, and product behaviour live, and it is why
|
|
118
|
+
`runToolLoop` takes a dozen observers instead of doing any of it.
|
|
111
119
|
|
|
112
120
|
---
|
|
113
121
|
|
|
@@ -246,6 +254,93 @@ progressive tool disclosure with
|
|
|
246
254
|
*Goal-oriented. Each answers one question and assumes you know roughly what you
|
|
247
255
|
are doing.*
|
|
248
256
|
|
|
257
|
+
### How to run the loop
|
|
258
|
+
|
|
259
|
+
`runToolLoop` is the engine: it calls the model, runs the tools the model asks
|
|
260
|
+
for, and repeats until the model stops asking, a tool suspends the run, a caller
|
|
261
|
+
stops it, or `maxIterations` is reached. Everything that *happens* as a result is
|
|
262
|
+
a callback you supply.
|
|
263
|
+
|
|
264
|
+
```ts
|
|
265
|
+
import { runToolLoop, type ToolLoopState } from "@juno-ai/bind/loop";
|
|
266
|
+
|
|
267
|
+
const state: ToolLoopState = {
|
|
268
|
+
messages: [systemMessage, userMessage],
|
|
269
|
+
inputTokens: 0,
|
|
270
|
+
outputTokens: 0,
|
|
271
|
+
costCents: 0,
|
|
272
|
+
lastPromptTokens: 0,
|
|
273
|
+
lastOutputTokens: 0,
|
|
274
|
+
hasFreshTokenCount: false,
|
|
275
|
+
toolCalls: 0,
|
|
276
|
+
};
|
|
277
|
+
|
|
278
|
+
await runToolLoop({
|
|
279
|
+
state,
|
|
280
|
+
activePlugins,
|
|
281
|
+
maxIterations: 30,
|
|
282
|
+
|
|
283
|
+
callModel: (messages, tools) => llm.complete({ messages, tools }),
|
|
284
|
+
buildTools: () => registry.toolDefinitions([...activePlugins]),
|
|
285
|
+
runToolCall: (call) => dispatch(call),
|
|
286
|
+
activatePlugins: (names) => names.forEach((n) => activePlugins.add(n)),
|
|
287
|
+
activateSkills: (refs) => loadInstructions(refs),
|
|
288
|
+
});
|
|
289
|
+
|
|
290
|
+
// `state` is mutated in place — read totals off it after, or mid-run from a
|
|
291
|
+
// heartbeat.
|
|
292
|
+
console.log(state.inputTokens, state.outputTokens, state.toolCalls);
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
`state` is mutated rather than returned so a heartbeat can read live totals while
|
|
296
|
+
the run is still going; a returned result could not report anything until the
|
|
297
|
+
run ended.
|
|
298
|
+
|
|
299
|
+
### How to make a tool take effect mid-batch
|
|
300
|
+
|
|
301
|
+
A model can request several tools at once, and one of them may change what the
|
|
302
|
+
others can do — activating a plugin, loading an instruction module. Those must
|
|
303
|
+
run first, alone, or a dependent call in the same batch executes against the old
|
|
304
|
+
tool surface.
|
|
305
|
+
|
|
306
|
+
```ts
|
|
307
|
+
runsSerially: (call) =>
|
|
308
|
+
call.type === "function" && ACTIVATION_TOOLS.has(call.function.name),
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
The loop runs those one at a time, applies each outcome immediately, then fans
|
|
312
|
+
the rest out concurrently (pooled per tool name). Outcomes are reassembled in the
|
|
313
|
+
model's original order either way — every `tool_call_id` gets its answer in the
|
|
314
|
+
sequence the provider expects.
|
|
315
|
+
|
|
316
|
+
Unwired, nothing is serial. That is correct for a host whose tools do not reshape
|
|
317
|
+
the tool surface, and wrong the moment one does.
|
|
318
|
+
|
|
319
|
+
### How to decide which tool failures kill the run
|
|
320
|
+
|
|
321
|
+
By default a thrown tool becomes a tool error the model can read and recover
|
|
322
|
+
from, which is what you want for an isolated failure. Two kinds are not that:
|
|
323
|
+
|
|
324
|
+
```ts
|
|
325
|
+
isFatalToolError: (error) =>
|
|
326
|
+
error instanceof RunCancelledError || // must abort, not be answered
|
|
327
|
+
error instanceof PersistenceError, // we could not RECORD the outcome
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
Cancellation has to propagate even when no `ensureNotCancelled` observer is
|
|
331
|
+
wired. A persistence failure matters for a subtler reason: synthesizing "the tool
|
|
332
|
+
failed" over a write you could not record tells the model a lie about work that
|
|
333
|
+
may well have happened.
|
|
334
|
+
|
|
335
|
+
Everything not fatal is answered and observed:
|
|
336
|
+
|
|
337
|
+
```ts
|
|
338
|
+
onToolCallRejected: (toolCallId, error) => log.warn("tool rejected", { toolCallId, error }),
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
Wire it. The model sees these failures either way; without the observer, nothing
|
|
342
|
+
else does.
|
|
343
|
+
|
|
249
344
|
### How to pin a model to one provider
|
|
250
345
|
|
|
251
346
|
Use a hard `only` fence. Plan-time filtering and runtime traversal both respect
|
|
@@ -428,8 +523,9 @@ keeps a consumer who only wants routing from pulling in the rest.
|
|
|
428
523
|
| Import | Owns | Reach for it when |
|
|
429
524
|
|---|---|---|
|
|
430
525
|
| `@juno-ai/bind/routing` | Route plans, the planner, the failure taxonomy, the plan executor, the circuit breaker, billing-basis arithmetic, config degradation | You call more than one provider or model, or you want retries and fallback governed by one table |
|
|
431
|
-
| `@juno-ai/bind/contracts` | Turn vocabulary — `TurnFn`, `ModelTurnResult`, `StopReason`, `RunStats` | You want one seam between your loop and any LLM client, and comparable per-run metrics |
|
|
432
|
-
| `@juno-ai/bind/
|
|
526
|
+
| `@juno-ai/bind/contracts` | Turn vocabulary — `TurnFn`, `ModelTurnResult`, `StopReason`, `RunStats` and its folds | You want one seam between your loop and any LLM client, and comparable per-run metrics |
|
|
527
|
+
| `@juno-ai/bind/loop` | `runToolLoop` — the iteration engine: model turn, two-phase tool batch, activation, compaction, interrupts, suspend | You want the agent loop itself, not just the pieces to build one |
|
|
528
|
+
| `@juno-ai/bind/run` | Wall-clock deadline, failure classification, coalesced heartbeat, tool-batch pooling, child-run lineage and admission, poll backoff | A run must be bounded, observable, and able to say *why* it stopped — or it can spawn runs of its own |
|
|
433
529
|
| `@juno-ai/bind/transcript` | `validateAndHealMessages` | You send transcripts to more than one provider, or you build them across turns |
|
|
434
530
|
| `@juno-ai/bind/tools` | `sanitizeToolSchema` | Any tool schema reaches a provider — especially third-party ones |
|
|
435
531
|
| `@juno-ai/bind/plugins` | Tool/plugin vocabulary, the registry factory, progressive-disclosure activation | You have more tools than fit comfortably in one prompt |
|
|
@@ -462,6 +558,20 @@ tenant's revoked key opens the circuit for every tenant sharing that
|
|
|
462
558
|
60 s ceiling including `Retry-After` extensions. Resolve a half-open probe that
|
|
463
559
|
ended without a recordable outcome via `releaseProbe`, or the slot sticks.
|
|
464
560
|
|
|
561
|
+
**Run — a child is admitted before it exists, never after.** `admitChildRun`
|
|
562
|
+
judges counts you supply; call it ahead of enqueuing. A chain bounded only once
|
|
563
|
+
its runs are on the queue is not bounded, it is billed. A rule whose measurement
|
|
564
|
+
is `NaN` or `Infinity` refuses rather than admits — every comparison is false
|
|
565
|
+
against `NaN`, so the naive reading of a broken count is an unbounded chain.
|
|
566
|
+
|
|
567
|
+
**Run — chain lineage is "null means me".** A root run's `rootRunId` and
|
|
568
|
+
`parentRunId` are both `null`, because a chain's origin has no id to point at
|
|
569
|
+
until its own row exists. Read any chain's root as `chain.rootRunId ?? runId`;
|
|
570
|
+
`descendChain` does that when it hands the id down, so every descendant carries
|
|
571
|
+
a concrete root. Adopting the parent's id at *every* level instead makes each
|
|
572
|
+
generation a fresh chain, and every per-chain bound then counts the wrong set
|
|
573
|
+
and silently never fires.
|
|
574
|
+
|
|
465
575
|
**Run — whoever classifies the outcome owns the deadline.** `timedOut` reflects
|
|
466
576
|
*that* deadline firing, and stays false when only a combined external signal
|
|
467
577
|
aborts — which is what separates a timeout from a cancellation. Handle
|
|
@@ -865,14 +975,13 @@ try {
|
|
|
865
975
|
|
|
866
976
|
### Spawning child runs (sub-agents)
|
|
867
977
|
|
|
868
|
-
> **Not in the harness yet.** There is no sub-agent, spawn, or child-run concept
|
|
869
|
-
> in `bind` today — the vocabulary below is yours to define. The admission
|
|
870
|
-
> policy is a planned addition; see [Roadmap](#roadmap).
|
|
871
|
-
|
|
872
978
|
A sub-agent is not a special kind of thing. It is **a run that another run
|
|
873
979
|
asked for**, so everything the harness already gives a run applies unchanged:
|
|
874
980
|
its own deadline, its own heartbeat, its own `RunStats`, its own route plan.
|
|
875
|
-
|
|
981
|
+
Three things are genuinely new — lineage, admission, and waiting — and
|
|
982
|
+
`@juno-ai/bind/run` owns the decision in each. It owns none of the measuring:
|
|
983
|
+
counting runs in a chain needs your database, and judging whether a count is
|
|
984
|
+
too high does not.
|
|
876
985
|
|
|
877
986
|
**1. The spawn tool is an ordinary plugin.** Nothing special is needed here —
|
|
878
987
|
the tool vocabulary is already shared.
|
|
@@ -892,10 +1001,10 @@ const orchestration: ToolPlugin<Ctx> = {
|
|
|
892
1001
|
async execute(toolName, args, ctx) {
|
|
893
1002
|
const { objective, plugins } = spawnArgsSchema.parse(args);
|
|
894
1003
|
|
|
895
|
-
const admission =
|
|
896
|
-
{
|
|
897
|
-
|
|
898
|
-
);
|
|
1004
|
+
const admission = admitChildRun([
|
|
1005
|
+
{ kind: "depth", parentDepth: ctx.chain.depth, maxDepth: 5 },
|
|
1006
|
+
{ kind: "chain_budget", runsInChain: await countRuns(ctx.chain), maxRuns: 50 },
|
|
1007
|
+
]);
|
|
899
1008
|
if (!admission.admitted) {
|
|
900
1009
|
return { success: false, kind: "validation", error: admission.reason };
|
|
901
1010
|
}
|
|
@@ -903,9 +1012,7 @@ const orchestration: ToolPlugin<Ctx> = {
|
|
|
903
1012
|
const childRunId = await queue.enqueueRun({
|
|
904
1013
|
objective,
|
|
905
1014
|
plugins,
|
|
906
|
-
|
|
907
|
-
depth: ctx.chainDepth + 1,
|
|
908
|
-
parentRunId: ctx.runId,
|
|
1015
|
+
chain: descendChain(ctx.runId, ctx.chain),
|
|
909
1016
|
});
|
|
910
1017
|
return { success: true, data: { childRunId } };
|
|
911
1018
|
},
|
|
@@ -914,34 +1021,47 @@ const orchestration: ToolPlugin<Ctx> = {
|
|
|
914
1021
|
|
|
915
1022
|
**2. Admission is the part worth getting right.** An agent that can spawn can
|
|
916
1023
|
spawn agents that spawn. Bound it *before* enqueuing, not after — an unbounded
|
|
917
|
-
chain is a runaway spend, and the failure mode is silent.
|
|
918
|
-
your database; the deciding does not.
|
|
1024
|
+
chain is a runaway spend, and the failure mode is silent.
|
|
919
1025
|
|
|
920
|
-
|
|
921
|
-
|
|
1026
|
+
Each rule carries its limit **and** the measurement it judges, so a bound you
|
|
1027
|
+
configure but never wired a count for is not expressible. That shape exists
|
|
1028
|
+
because the alternative fails quietly: a bounds object beside a facts object
|
|
1029
|
+
lets a limit sit in config and never fire, and nothing looks wrong.
|
|
922
1030
|
|
|
923
|
-
|
|
924
|
-
|
|
925
|
-
|
|
926
|
-
|
|
927
|
-
|
|
928
|
-
|
|
929
|
-
}
|
|
930
|
-
|
|
931
|
-
|
|
932
|
-
|
|
933
|
-
|
|
1031
|
+
```ts
|
|
1032
|
+
import { admitChildRun, descendChain, type ChainRule } from "@juno-ai/bind/run";
|
|
1033
|
+
|
|
1034
|
+
const rules: ChainRule[] = [
|
|
1035
|
+
{ kind: "depth", parentDepth: chain.depth, maxDepth: 5 },
|
|
1036
|
+
{ kind: "chain_budget", runsInChain: await countRunsInChain(chain), maxRuns: 50 },
|
|
1037
|
+
{ kind: "pair_cooldown", msSinceLastSpawn: await msSinceLastSpawn(runId), cooldownMs: 30_000 },
|
|
1038
|
+
{ kind: "tenant_ceiling", activeRuns: await countActiveRuns(tenantId), maxActiveRuns: 200 },
|
|
1039
|
+
];
|
|
1040
|
+
|
|
1041
|
+
const admission = admitChildRun(rules);
|
|
1042
|
+
if (!admission.admitted) {
|
|
1043
|
+
log.warn("child run refused", { rule: admission.rule });
|
|
1044
|
+
return { success: false, kind: "validation", error: admission.reason };
|
|
934
1045
|
}
|
|
935
1046
|
```
|
|
936
1047
|
|
|
937
|
-
|
|
938
|
-
|
|
939
|
-
|
|
940
|
-
|
|
941
|
-
|
|
1048
|
+
Rules are evaluated in order and the first refusal wins, so you choose which
|
|
1049
|
+
reason the model sees. Pick the set against your own cost model: depth caps
|
|
1050
|
+
runaway recursion, a chain budget caps a chain that stays shallow but keeps
|
|
1051
|
+
fanning out, a pair cooldown stops two runs ping-ponging, and a tenant ceiling
|
|
1052
|
+
covers what none of the chain rules can — someone starting a thousand
|
|
1053
|
+
independent chains. A broken measurement (`NaN`, `Infinity`) refuses rather
|
|
1054
|
+
than admits: every comparison is false against `NaN`, so the naive reading
|
|
1055
|
+
would turn a broken count into an unbounded chain.
|
|
942
1056
|
|
|
943
|
-
|
|
944
|
-
|
|
1057
|
+
`descendChain` handles the lineage arithmetic, including the root-id fallback
|
|
1058
|
+
that is easy to get backwards — a first-generation child adopts its parent's
|
|
1059
|
+
*id* as the chain root, later generations keep the root the parent already
|
|
1060
|
+
carries. Getting that wrong makes every generation its own chain, and every
|
|
1061
|
+
per-chain bound then counts the wrong set and never fires.
|
|
1062
|
+
|
|
1063
|
+
**3. Waiting.** Two shapes work. Suspend the parent and let child completion
|
|
1064
|
+
wake it, which frees the worker slot:
|
|
945
1065
|
|
|
946
1066
|
```ts
|
|
947
1067
|
return {
|
|
@@ -952,27 +1072,51 @@ return {
|
|
|
952
1072
|
```
|
|
953
1073
|
|
|
954
1074
|
…or poll inside the tool, which keeps the parent's transcript intact but holds
|
|
955
|
-
its slot — so
|
|
1075
|
+
its slot — so budget the poll well under the parent's own deadline.
|
|
1076
|
+
`createPollSchedule` is a doubling backoff bounded by that budget; it clamps
|
|
1077
|
+
the final delay so a sleep can never overshoot the deadline you promised.
|
|
1078
|
+
|
|
1079
|
+
```ts
|
|
1080
|
+
import { createPollSchedule } from "@juno-ai/bind/run";
|
|
1081
|
+
|
|
1082
|
+
const startedAt = Date.now();
|
|
1083
|
+
const poll = createPollSchedule({
|
|
1084
|
+
initialDelayMs: 500,
|
|
1085
|
+
maxDelayMs: 30_000,
|
|
1086
|
+
budgetMs: 10 * 60_000,
|
|
1087
|
+
});
|
|
1088
|
+
|
|
1089
|
+
while (true) {
|
|
1090
|
+
const children = await loadChildRuns(childRunIds);
|
|
1091
|
+
if (children.every((c) => c.finished)) return { success: true, data: { children } };
|
|
956
1092
|
|
|
957
|
-
|
|
958
|
-
|
|
959
|
-
|
|
1093
|
+
const step = poll.next(Date.now() - startedAt);
|
|
1094
|
+
if (step.kind === "expired") {
|
|
1095
|
+
return { success: true, data: { children, timedOut: true } };
|
|
1096
|
+
}
|
|
1097
|
+
await sleep(step.delayMs, signal);
|
|
1098
|
+
}
|
|
1099
|
+
```
|
|
1100
|
+
|
|
1101
|
+
Elapsed time is an argument rather than something the schedule reads off a
|
|
1102
|
+
clock, so a test can drive the whole backoff without waiting for any of it.
|
|
1103
|
+
|
|
1104
|
+
**4. Rolling results up.** `accumulateTurn` and `accumulateToolCall` fold a
|
|
1105
|
+
run's *own* activity; `accumulateRun` folds one whole run into another, which
|
|
1106
|
+
is what a chain's totals are made of.
|
|
960
1107
|
|
|
961
1108
|
```ts
|
|
962
|
-
|
|
963
|
-
|
|
964
|
-
|
|
965
|
-
toolCalls: acc.toolCalls + s.toolCalls,
|
|
966
|
-
inputTokens: acc.inputTokens + s.inputTokens,
|
|
967
|
-
outputTokens: acc.outputTokens + s.outputTokens,
|
|
968
|
-
costCents: acc.costCents + s.costCents,
|
|
969
|
-
modelTimeMs: acc.modelTimeMs + s.modelTimeMs,
|
|
970
|
-
toolTimeMs: acc.toolTimeMs + s.toolTimeMs,
|
|
971
|
-
}), emptyRunStats());
|
|
1109
|
+
import { accumulateRun, emptyRunStats } from "@juno-ai/bind/contracts";
|
|
1110
|
+
|
|
1111
|
+
const chainTotals = childStats.reduce(accumulateRun, parentStats);
|
|
972
1112
|
```
|
|
973
1113
|
|
|
974
|
-
|
|
975
|
-
|
|
1114
|
+
The fold is associative, so a chain reduces in whatever order its children
|
|
1115
|
+
finish. Two properties are worth expecting rather than debugging: `modelTimeMs`
|
|
1116
|
+
will exceed the chain's wall-clock once children run in parallel (the sum is
|
|
1117
|
+
what the chain *cost*, not how long it took), and `outputTokensPerSecond` is
|
|
1118
|
+
recomputed from the merged totals rather than averaged across runs — averaging
|
|
1119
|
+
two rates weights a 10-token run like a 10,000-token one.
|
|
976
1120
|
|
|
977
1121
|
---
|
|
978
1122
|
|
|
@@ -981,13 +1125,12 @@ correct, and it is why the two are tracked separately.
|
|
|
981
1125
|
Named, not scheduled. Listed so a consumer can tell a deliberate omission from
|
|
982
1126
|
an oversight.
|
|
983
1127
|
|
|
984
|
-
- **
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
989
|
-
-
|
|
990
|
-
not have to be summed by hand.
|
|
1128
|
+
- **One `RunStats` per turn.** The loop accounts usage with `ToolLoopTurn`
|
|
1129
|
+
(tokens and cost) while `ModelTurnResult` carries timings too. They converge
|
|
1130
|
+
when the loop folds `RunStats` directly; today a host that wants throughput
|
|
1131
|
+
metrics accumulates them alongside.
|
|
1132
|
+
- **A streaming turn contract.** `callModel` reports an output-token estimate
|
|
1133
|
+
mid-stream; the assistant message itself still arrives whole.
|
|
991
1134
|
|
|
992
1135
|
---
|
|
993
1136
|
|
package/contracts/index.d.ts
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export { emptyRunStats, accumulateTurn, accumulateToolCall, type TranscriptMessage, type AssistantTurnMessage, type WireToolDefinition, type WireToolCall, type TurnTimings, type TurnUsage, type ModelTurnResult, type TurnFn, type StopReason, type RunStats, } from "./turn.js";
|
|
1
|
+
export { emptyRunStats, accumulateTurn, accumulateToolCall, accumulateRun, type TranscriptMessage, type AssistantTurnMessage, type WireToolDefinition, type WireToolCall, type TurnTimings, type TurnUsage, type ModelTurnResult, type TurnFn, type StopReason, type RunStats, } from "./turn.js";
|
package/contracts/index.js
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export { emptyRunStats, accumulateTurn, accumulateToolCall, } from "./turn.js";
|
|
1
|
+
export { emptyRunStats, accumulateTurn, accumulateToolCall, accumulateRun, } from "./turn.js";
|
package/contracts/turn.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import type OpenAI from "openai";
|
|
2
2
|
/**
|
|
3
|
-
* Turn vocabulary — the shared language between the turn kernel
|
|
4
|
-
*
|
|
3
|
+
* Turn vocabulary — the shared language between the turn kernel
|
|
4
|
+
* (`@juno-ai/bind/loop`), LLM transports, and hosts.
|
|
5
5
|
*
|
|
6
6
|
* The declared wire format is the OpenAI chat-completions message shape,
|
|
7
7
|
* consumed as **types only** (`openai` is a peer used purely for its type
|
|
@@ -77,3 +77,27 @@ export declare function emptyRunStats(): RunStats;
|
|
|
77
77
|
export declare function accumulateTurn(stats: RunStats, turn: ModelTurnResult): RunStats;
|
|
78
78
|
/** Fold one dispatched tool call's duration into cumulative run stats. */
|
|
79
79
|
export declare function accumulateToolCall(stats: RunStats, toolName: string, durationMs: number): RunStats;
|
|
80
|
+
/**
|
|
81
|
+
* Fold a completed run's totals into another run's — the roll-up for a chain
|
|
82
|
+
* that spawned child runs (`@juno-ai/bind/run`).
|
|
83
|
+
*
|
|
84
|
+
* Two things follow from summing across runs rather than within one, and both
|
|
85
|
+
* are correct rather than artifacts:
|
|
86
|
+
*
|
|
87
|
+
* - **`modelTimeMs` can exceed the chain's wall-clock**, because children that
|
|
88
|
+
* ran concurrently each contribute their own. That is precisely why model
|
|
89
|
+
* time and wall-clock are separate numbers; a chain's *cost* is the sum, its
|
|
90
|
+
* *latency* is not.
|
|
91
|
+
* - **`outputTokensPerSecond` is recomputed from the merged totals**, not
|
|
92
|
+
* averaged from the parts. An average of two rates weights a 10-token run
|
|
93
|
+
* the same as a 10,000-token one and reports a throughput neither run
|
|
94
|
+
* achieved.
|
|
95
|
+
*
|
|
96
|
+
* The fold is associative and order-independent, so a chain reduces cleanly in
|
|
97
|
+
* whatever order its children finish:
|
|
98
|
+
*
|
|
99
|
+
* ```ts
|
|
100
|
+
* const chainTotals = childStats.reduce(accumulateRun, parentStats);
|
|
101
|
+
* ```
|
|
102
|
+
*/
|
|
103
|
+
export declare function accumulateRun(stats: RunStats, run: RunStats): RunStats;
|
package/contracts/turn.js
CHANGED
|
@@ -37,3 +37,48 @@ export function accumulateToolCall(stats, toolName, durationMs) {
|
|
|
37
37
|
},
|
|
38
38
|
};
|
|
39
39
|
}
|
|
40
|
+
/**
|
|
41
|
+
* Fold a completed run's totals into another run's — the roll-up for a chain
|
|
42
|
+
* that spawned child runs (`@juno-ai/bind/run`).
|
|
43
|
+
*
|
|
44
|
+
* Two things follow from summing across runs rather than within one, and both
|
|
45
|
+
* are correct rather than artifacts:
|
|
46
|
+
*
|
|
47
|
+
* - **`modelTimeMs` can exceed the chain's wall-clock**, because children that
|
|
48
|
+
* ran concurrently each contribute their own. That is precisely why model
|
|
49
|
+
* time and wall-clock are separate numbers; a chain's *cost* is the sum, its
|
|
50
|
+
* *latency* is not.
|
|
51
|
+
* - **`outputTokensPerSecond` is recomputed from the merged totals**, not
|
|
52
|
+
* averaged from the parts. An average of two rates weights a 10-token run
|
|
53
|
+
* the same as a 10,000-token one and reports a throughput neither run
|
|
54
|
+
* achieved.
|
|
55
|
+
*
|
|
56
|
+
* The fold is associative and order-independent, so a chain reduces cleanly in
|
|
57
|
+
* whatever order its children finish:
|
|
58
|
+
*
|
|
59
|
+
* ```ts
|
|
60
|
+
* const chainTotals = childStats.reduce(accumulateRun, parentStats);
|
|
61
|
+
* ```
|
|
62
|
+
*/
|
|
63
|
+
export function accumulateRun(stats, run) {
|
|
64
|
+
const outputTokens = stats.outputTokens + run.outputTokens;
|
|
65
|
+
const modelTimeMs = stats.modelTimeMs + run.modelTimeMs;
|
|
66
|
+
const toolTimeBreakdownMs = {
|
|
67
|
+
...stats.toolTimeBreakdownMs,
|
|
68
|
+
};
|
|
69
|
+
for (const [toolName, durationMs] of Object.entries(run.toolTimeBreakdownMs)) {
|
|
70
|
+
toolTimeBreakdownMs[toolName] =
|
|
71
|
+
(toolTimeBreakdownMs[toolName] ?? 0) + durationMs;
|
|
72
|
+
}
|
|
73
|
+
return {
|
|
74
|
+
turns: stats.turns + run.turns,
|
|
75
|
+
toolCalls: stats.toolCalls + run.toolCalls,
|
|
76
|
+
inputTokens: stats.inputTokens + run.inputTokens,
|
|
77
|
+
outputTokens,
|
|
78
|
+
costCents: stats.costCents + run.costCents,
|
|
79
|
+
modelTimeMs,
|
|
80
|
+
toolTimeMs: stats.toolTimeMs + run.toolTimeMs,
|
|
81
|
+
outputTokensPerSecond: modelTimeMs > 0 ? (outputTokens / modelTimeMs) * 1000 : null,
|
|
82
|
+
toolTimeBreakdownMs,
|
|
83
|
+
};
|
|
84
|
+
}
|
package/index.d.ts
CHANGED
|
@@ -5,13 +5,15 @@
|
|
|
5
5
|
* completion into tool effects into the next turn's context. This package is
|
|
6
6
|
* the harness that runs that chain.
|
|
7
7
|
*
|
|
8
|
-
* Current surface: the
|
|
8
|
+
* Current surface: the tool-calling turn kernel (`src/loop/` — the iteration
|
|
9
|
+
* engine itself), the deterministic LLM provider-routing core, the turn
|
|
9
10
|
* vocabulary, the run mechanics (deadline, coalesced heartbeat, failure
|
|
10
|
-
* classification, tool-batch pooling
|
|
11
|
-
* tool-schema sanitization, and the
|
|
12
|
-
*
|
|
13
|
-
* host's invocation context
|
|
14
|
-
* the
|
|
11
|
+
* classification, tool-batch pooling, child-run lineage and admission),
|
|
12
|
+
* transcript validation/healing, provider tool-schema sanitization, and the
|
|
13
|
+
* plugin/tool vocabulary with its registry and progressive-disclosure
|
|
14
|
+
* activation — generic over the host's invocation context. What is NOT here is
|
|
15
|
+
* the run driver: starting a run, recording what it did, and delivering its
|
|
16
|
+
* output. See the README for the rest of what is deliberately absent.
|
|
15
17
|
*/
|
|
16
18
|
export * from "./routing/index.js";
|
|
17
19
|
export * from "./contracts/index.js";
|
|
@@ -19,3 +21,4 @@ export * from "./run/index.js";
|
|
|
19
21
|
export * from "./transcript/index.js";
|
|
20
22
|
export * from "./tools/index.js";
|
|
21
23
|
export * from "./plugins/index.js";
|
|
24
|
+
export * from "./loop/index.js";
|
package/index.js
CHANGED
|
@@ -5,13 +5,15 @@
|
|
|
5
5
|
* completion into tool effects into the next turn's context. This package is
|
|
6
6
|
* the harness that runs that chain.
|
|
7
7
|
*
|
|
8
|
-
* Current surface: the
|
|
8
|
+
* Current surface: the tool-calling turn kernel (`src/loop/` — the iteration
|
|
9
|
+
* engine itself), the deterministic LLM provider-routing core, the turn
|
|
9
10
|
* vocabulary, the run mechanics (deadline, coalesced heartbeat, failure
|
|
10
|
-
* classification, tool-batch pooling
|
|
11
|
-
* tool-schema sanitization, and the
|
|
12
|
-
*
|
|
13
|
-
* host's invocation context
|
|
14
|
-
* the
|
|
11
|
+
* classification, tool-batch pooling, child-run lineage and admission),
|
|
12
|
+
* transcript validation/healing, provider tool-schema sanitization, and the
|
|
13
|
+
* plugin/tool vocabulary with its registry and progressive-disclosure
|
|
14
|
+
* activation — generic over the host's invocation context. What is NOT here is
|
|
15
|
+
* the run driver: starting a run, recording what it did, and delivering its
|
|
16
|
+
* output. See the README for the rest of what is deliberately absent.
|
|
15
17
|
*/
|
|
16
18
|
export * from "./routing/index.js";
|
|
17
19
|
export * from "./contracts/index.js";
|
|
@@ -19,3 +21,4 @@ export * from "./run/index.js";
|
|
|
19
21
|
export * from "./transcript/index.js";
|
|
20
22
|
export * from "./tools/index.js";
|
|
21
23
|
export * from "./plugins/index.js";
|
|
24
|
+
export * from "./loop/index.js";
|
package/loop/index.d.ts
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export { runToolLoop, type ToolLoopParams, type ToolLoopState, type ToolLoopTurn, type ToolCallOutcome, type CompactionApplied, type RunStatus, } from "./tool-loop.js";
|
package/loop/index.js
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export { runToolLoop, } from "./tool-loop.js";
|