@juno-ai/bind 8.0.0 → 10.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +405 -58
- package/completion/index.d.ts +1 -0
- package/completion/index.js +1 -0
- package/completion/text-stream.d.ts +311 -0
- package/completion/text-stream.js +273 -0
- package/contracts/index.d.ts +1 -1
- package/contracts/index.js +1 -1
- package/contracts/turn.d.ts +77 -2
- package/contracts/turn.js +35 -2
- package/loop/index.d.ts +2 -1
- package/loop/index.js +1 -1
- package/loop/tool-loop.d.ts +131 -12
- package/loop/tool-loop.js +285 -49
- package/package.json +5 -1
- package/plugins/dispatch.d.ts +130 -0
- package/plugins/dispatch.js +241 -0
- package/plugins/index.d.ts +2 -0
- package/plugins/index.js +2 -0
- package/plugins/tool-message.d.ts +23 -0
- package/plugins/tool-message.js +31 -0
- package/plugins/tool.d.ts +1 -1
- package/run/index.d.ts +1 -1
- package/run/index.js +1 -1
- package/run/tool-batch.d.ts +51 -1
- package/run/tool-batch.js +59 -1
- package/testing/index.d.ts +153 -0
- package/testing/index.js +188 -0
- package/tools/control-chars.d.ts +23 -0
- package/tools/control-chars.js +35 -0
package/README.md
CHANGED
|
@@ -29,6 +29,86 @@ constraints that will fail CI if you break them.
|
|
|
29
29
|
|
|
30
30
|
---
|
|
31
31
|
|
|
32
|
+
## Changelog
|
|
33
|
+
|
|
34
|
+
### Unreleased
|
|
35
|
+
|
|
36
|
+
**Breaking**
|
|
37
|
+
|
|
38
|
+
- `runToolLoop` now returns `ToolLoopResult` (`{ stopReason, stats }`) instead
|
|
39
|
+
of `void`. A caller that ignores the return value is unchanged, but a
|
|
40
|
+
wrapper *annotated* `Promise<void>` no longer typechecks — widen it to
|
|
41
|
+
`Promise<ToolLoopResult>`.
|
|
42
|
+
- `RunStats` gained `cachedInputTokens`. `emptyRunStats()` and the folds set it;
|
|
43
|
+
code that hand-builds a `RunStats` literal must add the field. `ToolLoopTurn`
|
|
44
|
+
and `CompactionApplied` gained a matching optional `cachedInputTokens`, so
|
|
45
|
+
the loop can actually populate it — return it from `callModel` if your
|
|
46
|
+
transport reports one.
|
|
47
|
+
- `MissingActivationPortError` — a tool outcome that asks the loop to activate
|
|
48
|
+
a plugin or skill while the matching port is unwired is now reported through
|
|
49
|
+
`onToolCallRejected` instead of being dropped in silence. The run still
|
|
50
|
+
completes; the wiring bug is no longer invisible.
|
|
51
|
+
- `defineTool` no longer applies `normalizeArgs` inside `execute`.
|
|
52
|
+
Normalization is the dispatcher's step (it runs before the idempotency
|
|
53
|
+
hash), and applying it in both places applied it twice.
|
|
54
|
+
- **`StopReason` no longer has `suspended`.** It split into
|
|
55
|
+
`waiting_for_reply` (a tool asked a human a question; nothing happens until
|
|
56
|
+
someone answers) and `resuming_later` (a tool scheduled its own resume).
|
|
57
|
+
Removing a union member breaks any exhaustive `switch`, so map both new
|
|
58
|
+
values wherever you handled `suspended` — and note that the old single value
|
|
59
|
+
could not tell them apart at all, which is why it was split.
|
|
60
|
+
- **`@juno-ai/bind/plugins` now loads `zod` at runtime.** The barrel re-exports
|
|
61
|
+
`defineTool` / `pluginFromTools`, and `toolWireDefinition` needs
|
|
62
|
+
`z.toJSONSchema`. Every other module in the subpath was previously
|
|
63
|
+
runtime-zod-free, so a consumer importing only `createToolRegistry` while
|
|
64
|
+
ignoring the peer-dependency warning now fails to resolve. `zod` is a
|
|
65
|
+
required (non-optional) peer, so a correctly-installed consumer is
|
|
66
|
+
unaffected. `@juno-ai/bind/loop` is deliberately still zod-free — which is
|
|
67
|
+
why `toolResultMessage` lives in its own type-only module.
|
|
68
|
+
|
|
69
|
+
**Added**
|
|
70
|
+
|
|
71
|
+
- `ToolLoopResult.stopReason` — `done` / `waiting_for_reply` / `resuming_later`
|
|
72
|
+
/ `iteration_limit` / `aborted`, one per exit, replacing the two or three
|
|
73
|
+
loop-state flags every host was combining differently. The two pause reasons
|
|
74
|
+
are separate values on purpose: one needs a person to act and the other does
|
|
75
|
+
not, and that is the distinction a host most needs to surface. `deadline` stays in the `StopReason` union
|
|
76
|
+
for a host classifying a thrown `RunTimeoutError`; the loop cannot return it,
|
|
77
|
+
and the type's doc comment says why.
|
|
78
|
+
- `ToolLoopResult.stats` — a `RunStats` the loop folds itself: turns,
|
|
79
|
+
dispatched tool calls, tokens, cost, and the model-time/tool-time split with
|
|
80
|
+
a per-tool breakdown. A compaction contributes its spend but not a turn
|
|
81
|
+
(`accumulateAuxiliarySpend`, also new), so `stats.turns` stays comparable
|
|
82
|
+
with `maxIterations`. `ToolLoopParams.now` injects the clock.
|
|
83
|
+
- `@juno-ai/bind/testing` — the scripted-model fixtures this package's own
|
|
84
|
+
cross-module suites use: `loopHarness`, `scriptedModel`, `toolCallTurn`,
|
|
85
|
+
`finalAnswer`, `toolCall`, `freshState`, `recordingSink`, `steppingClock`.
|
|
86
|
+
Credential-free multi-turn, multi-tool tests without mocking a chat client.
|
|
87
|
+
- `defineTool` / `pluginFromTools` (`@juno-ai/bind/plugins`) — author a tool as
|
|
88
|
+
`{ name, description, schema, execute }` and get argument parsing, a
|
|
89
|
+
`validation` failure the model can act on, and typed `args` in `execute`.
|
|
90
|
+
Plus `toolWireDefinition` (zod → sanitized JSON Schema) and
|
|
91
|
+
`toolResultMessage` (the `role:"tool"` encoding the loop itself uses).
|
|
92
|
+
- `ToolLoopParams.activePlugins`, `activatePlugins` and `activateSkills` are
|
|
93
|
+
now optional. A host with a fixed tool surface had been required to supply
|
|
94
|
+
empty functions; `activePlugins` was never read by the loop at all.
|
|
95
|
+
|
|
96
|
+
- `runToolCallsPooledByTool` accepts an optional `signal`, and
|
|
97
|
+
`AbortedToolCallError` / `ToolBatchOptions` are exported from
|
|
98
|
+
`@juno-ai/bind/run`. Additive — a caller that passes nothing is unchanged, and
|
|
99
|
+
a test pins that. A batch that *is* given one stops claiming queued calls once
|
|
100
|
+
it aborts, and those come back `rejected` with an `AbortedToolCallError`
|
|
101
|
+
(`kind: "not_run"` once the loop synthesizes them), so a caller relying on
|
|
102
|
+
every call always executing simply does not pass a signal.
|
|
103
|
+
|
|
104
|
+
- `createTurnTextStream` (`@juno-ai/bind/completion`) — streams a turn's
|
|
105
|
+
assistant text to a live surface and repairs it across routing retries. A
|
|
106
|
+
retractable surface keeps the whole fallback chain; a permanent one clamps as
|
|
107
|
+
before. See "How to stream tokens to a user without breaking fallback".
|
|
108
|
+
- `ToolLoopParams.signal` — bounds the tool batch. Without it the deadline and
|
|
109
|
+
cancellation ports are only consulted between iterations, so a budget that
|
|
110
|
+
expired during the model call still let the batch run its side effects.
|
|
111
|
+
|
|
32
112
|
## Explanation
|
|
33
113
|
|
|
34
114
|
*Understanding-oriented. Read this to know why the package is shaped the way it
|
|
@@ -275,26 +355,165 @@ const state: ToolLoopState = {
|
|
|
275
355
|
toolCalls: 0,
|
|
276
356
|
};
|
|
277
357
|
|
|
278
|
-
await runToolLoop({
|
|
358
|
+
const { stopReason, stats } = await runToolLoop({
|
|
279
359
|
state,
|
|
280
|
-
activePlugins,
|
|
281
360
|
maxIterations: 30,
|
|
282
361
|
|
|
283
362
|
callModel: (messages, tools) => llm.complete({ messages, tools }),
|
|
284
363
|
buildTools: () => registry.toolDefinitions([...activePlugins]),
|
|
285
364
|
runToolCall: (call) => dispatch(call),
|
|
365
|
+
|
|
366
|
+
// Only if your tools can change the tool surface mid-run:
|
|
286
367
|
activatePlugins: (names) => names.forEach((n) => activePlugins.add(n)),
|
|
287
368
|
activateSkills: (refs) => loadInstructions(refs),
|
|
288
369
|
});
|
|
289
370
|
|
|
290
|
-
// `state` is mutated in place — read totals off it
|
|
291
|
-
// heartbeat.
|
|
371
|
+
// `state` is mutated in place — read totals off it mid-run from a heartbeat.
|
|
292
372
|
console.log(state.inputTokens, state.outputTokens, state.toolCalls);
|
|
373
|
+
|
|
374
|
+
// The result is the run's conclusion, which only exists once it is over.
|
|
375
|
+
await persistRun({ stopReason, ...stats });
|
|
293
376
|
```
|
|
294
377
|
|
|
295
|
-
`state` is mutated rather than returned so a
|
|
296
|
-
the run is still going;
|
|
297
|
-
run ended.
|
|
378
|
+
The two halves are deliberate. `state` is mutated rather than returned so a
|
|
379
|
+
heartbeat can read live totals while the run is still going; `ToolLoopResult`
|
|
380
|
+
is what could not exist until the run ended.
|
|
381
|
+
|
|
382
|
+
`stopReason` is the whole outcome, in one word:
|
|
383
|
+
|
|
384
|
+
| Reason | The model | What a host should say |
|
|
385
|
+
|---|---|---|
|
|
386
|
+
| `done` | Stopped calling tools | It answered |
|
|
387
|
+
| `waiting_for_reply` | A tool asked a human a question | It needs you to answer — nothing happens until you do |
|
|
388
|
+
| `resuming_later` | A tool scheduled its own resume | It paused on purpose and will come back |
|
|
389
|
+
| `iteration_limit` | Still calling tools at the ceiling | It ran out of room, mid-task |
|
|
390
|
+
| `aborted` | Cut off by `shouldStop` or the batch `signal` | It was stopped |
|
|
391
|
+
|
|
392
|
+
The two pause reasons are separate values because their consequences differ
|
|
393
|
+
more than any other pair: one needs a person to act, the other needs nobody to.
|
|
394
|
+
Collapsed into one `suspended`, a host that wanted to say which had to go back
|
|
395
|
+
and read `state.suspended` — the loop-state-flag reconstruction this return
|
|
396
|
+
value exists to replace.
|
|
397
|
+
|
|
398
|
+
`deadline` is in the `StopReason` union but never returned: the wall-clock port
|
|
399
|
+
(`throwIfTimedOut`) is throw-based, so an expired budget leaves the loop as a
|
|
400
|
+
`RunTimeoutError` your catch block maps — `classifyRunFailure` recognises the
|
|
401
|
+
same condition. The loop returns `aborted` for a fired signal because once a
|
|
402
|
+
deadline and a cancellation are combined into one `AbortSignal` it genuinely
|
|
403
|
+
cannot tell which one fired.
|
|
404
|
+
|
|
405
|
+
`stats` is a `RunStats`: turns, dispatched tool calls, tokens (including
|
|
406
|
+
provider-reported cached input, when your `callModel` returns a
|
|
407
|
+
`cachedInputTokens`), cost, and model time vs tool time with a per-tool
|
|
408
|
+
breakdown. Three things are worth reading carefully:
|
|
409
|
+
|
|
410
|
+
- **`stats` is this invocation's contribution; `state` is whatever you seeded
|
|
411
|
+
plus that.** They match only if you seeded zeros. A host resuming a run seeds
|
|
412
|
+
`state` from the stored totals, and then `state.costCents` is the run's
|
|
413
|
+
lifetime cost while `stats.costCents` is this leg's — bill from whichever you
|
|
414
|
+
mean, and don't substitute one for the other.
|
|
415
|
+
|
|
416
|
+
- **`stats.toolCalls` counts calls that *ran*; `state.toolCalls` counts calls
|
|
417
|
+
the model *requested*.** They differ by exactly the work an aborted batch
|
|
418
|
+
prevented, which is why they are two numbers.
|
|
419
|
+
- **A compaction contributes tokens, cost and model time but not a turn**, so
|
|
420
|
+
`stats.turns` stays comparable with `maxIterations`.
|
|
421
|
+
|
|
422
|
+
Both are lost if the loop throws: a deadline, a cancellation, or a fatal tool
|
|
423
|
+
error leaves no return value, so `state` — mutated in place — is the only
|
|
424
|
+
accounting that survives those exits.
|
|
425
|
+
|
|
426
|
+
Pass `now` to make either measurement deterministic in a test; it defaults to
|
|
427
|
+
`Date.now`.
|
|
428
|
+
|
|
429
|
+
### How to test an agent without a provider
|
|
430
|
+
|
|
431
|
+
`@juno-ai/bind/testing` ships the fixtures this package's own cross-module
|
|
432
|
+
suites use. A scripted model is a queue of prepared turns, so a multi-turn,
|
|
433
|
+
multi-tool test needs no credential, no network, and no mocked chat client.
|
|
434
|
+
|
|
435
|
+
```ts
|
|
436
|
+
import { runToolLoop } from "@juno-ai/bind/loop";
|
|
437
|
+
import {
|
|
438
|
+
loopHarness, toolCall, toolCallTurn, finalAnswer,
|
|
439
|
+
} from "@juno-ai/bind/testing";
|
|
440
|
+
|
|
441
|
+
const h = loopHarness([
|
|
442
|
+
toolCallTurn([toolCall("search", { q: "bind" }), toolCall("read_file")]),
|
|
443
|
+
toolCallTurn([toolCall("write_file", { path: "out.md" })]),
|
|
444
|
+
finalAnswer("Done."),
|
|
445
|
+
]);
|
|
446
|
+
|
|
447
|
+
const { stopReason, stats } = await runToolLoop(h.params);
|
|
448
|
+
|
|
449
|
+
expect(stopReason).toBe("done");
|
|
450
|
+
expect(stats.turns).toBe(3);
|
|
451
|
+
expect(h.ran).toEqual(["search", "read_file", "write_file"]);
|
|
452
|
+
```
|
|
453
|
+
|
|
454
|
+
Every field of `h.params` is overridable, which is how you reach the
|
|
455
|
+
interesting states — `{ runToolCall }` to make one tool fail, `{ signal }` to
|
|
456
|
+
abort mid-batch, `{ now }` to make the timing figures deterministic
|
|
457
|
+
(`steppingClock()` for a fixed tick, or your own closure advanced inside
|
|
458
|
+
`runToolCall` when you want to *choose* each interval).
|
|
459
|
+
`h.ran` records what the loop *dispatched* and `h.sideEffects` what actually
|
|
460
|
+
*completed*; the gap between them is the answer to every cancellation question.
|
|
461
|
+
|
|
462
|
+
Two failure modes are deliberate. A script that runs out throws rather than
|
|
463
|
+
returning an empty turn — otherwise `maxIterations` absorbs the mistake and the
|
|
464
|
+
test passes while describing a run that never happened. And a duplicate
|
|
465
|
+
tool-call id throws at construction, rather than producing a transcript the
|
|
466
|
+
provider rejects ten frames deep in the loop.
|
|
467
|
+
|
|
468
|
+
### How to author a tool without writing dispatch by hand
|
|
469
|
+
|
|
470
|
+
A `ToolPlugin` dispatches by tool name, which is right for a bundle with shared
|
|
471
|
+
setup and pure ceremony for a flat list of independent tools. `defineTool` does
|
|
472
|
+
the mechanical half — parse the arguments, turn a parse failure into something
|
|
473
|
+
the model can act on — and `pluginFromTools` bundles the result.
|
|
474
|
+
|
|
475
|
+
```ts
|
|
476
|
+
import { defineTool, pluginFromTools } from "@juno-ai/bind/plugins";
|
|
477
|
+
|
|
478
|
+
const search = defineTool({
|
|
479
|
+
name: "search",
|
|
480
|
+
description: "Search the corpus.",
|
|
481
|
+
schema: z.object({ query: z.string().min(1), limit: z.number().default(10) }),
|
|
482
|
+
// `args` is typed from the schema; `limit` has already defaulted.
|
|
483
|
+
execute: async (args, ctx: Ctx) => ({
|
|
484
|
+
success: true,
|
|
485
|
+
data: await corpus.search(args.query, args.limit, ctx.tenantId),
|
|
486
|
+
}),
|
|
487
|
+
});
|
|
488
|
+
|
|
489
|
+
const plugin = pluginFromTools<Ctx>({
|
|
490
|
+
name: "corpus",
|
|
491
|
+
description: "Corpus tools.",
|
|
492
|
+
tools: [search],
|
|
493
|
+
});
|
|
494
|
+
```
|
|
495
|
+
|
|
496
|
+
The schema is the single source of truth: it is converted to JSON Schema for
|
|
497
|
+
the model *and* used to validate what comes back, so the two cannot drift.
|
|
498
|
+
|
|
499
|
+
Bad arguments come back as `{ success: false, kind: "validation", error }`
|
|
500
|
+
naming the offending path — **returned, not thrown**. That distinction is the
|
|
501
|
+
bug this replaces: a thrown parse error turns a recoverable "you passed the
|
|
502
|
+
wrong field" into a dead run. An unknown tool name is likewise a returned
|
|
503
|
+
`not_found`, because a resumed session's history can reference a tool you have
|
|
504
|
+
since retired.
|
|
505
|
+
|
|
506
|
+
The validation lives on the tool, not on the bundle, so a host with its own
|
|
507
|
+
dispatcher can call `search.execute(rawArgs, ctx)` directly and get the same
|
|
508
|
+
guarantee — `pluginFromTools` only resolves names.
|
|
509
|
+
|
|
510
|
+
Two encoders come with it. `toolWireDefinition(tool, wireName?)` converts a
|
|
511
|
+
tool to what the provider is shown, through `sanitizeToolSchema` — `wireName`
|
|
512
|
+
because tool naming is host policy (Monad encodes `plugin__tool` to route a
|
|
513
|
+
call back to its plugin). `toolResultMessage(callId, result)` produces the
|
|
514
|
+
`role:"tool"` message, using the same encoding the loop synthesizes for a
|
|
515
|
+
failed or refused call — so a model never has to learn two error formats in one
|
|
516
|
+
transcript.
|
|
298
517
|
|
|
299
518
|
### How to make a tool take effect mid-batch
|
|
300
519
|
|
|
@@ -572,54 +791,160 @@ never happened.
|
|
|
572
791
|
|
|
573
792
|
### How to stream tokens to a user without breaking fallback
|
|
574
793
|
|
|
575
|
-
|
|
576
|
-
*your* functions, so forwarding deltas from inside them is already supported and
|
|
577
|
-
needs nothing from here. What the package gives you is three granularities and
|
|
578
|
-
one rule.
|
|
794
|
+
Three granularities reach you, and the third is the one with a retry problem.
|
|
579
795
|
|
|
580
796
|
| You want | Use |
|
|
581
797
|
|---|---|
|
|
582
798
|
| Each assistant message as it lands | `onAssistantMessage(content)` on the loop — fires per message, including text emitted alongside tool calls, rather than batching at turn end |
|
|
583
799
|
| A live "N tokens so far" indicator | `onOutputProgress` into `callModel` → `onProgressUpdate(outputTokens, toolCalls)` — a count, never content |
|
|
584
|
-
| Individual tokens in a UI |
|
|
800
|
+
| Individual tokens in a UI | `createTurnTextStream` from `@juno-ai/bind/completion`, driven from inside your `AttemptFn` |
|
|
585
801
|
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
your
|
|
802
|
+
The problem the third one has: routing's answer to a mid-stream failure is to
|
|
803
|
+
try again — same endpoint, next provider, fallback model — and each of those
|
|
804
|
+
re-renders a turn your user is already reading. `createTurnTextStream` solves it
|
|
805
|
+
by asking one question about your surface. **Can it be told to discard what it
|
|
806
|
+
rendered?**
|
|
590
807
|
|
|
591
808
|
```ts
|
|
592
|
-
|
|
593
|
-
|
|
809
|
+
const stream = createTurnTextStream({
|
|
810
|
+
turnId: messageId,
|
|
811
|
+
sink: { retractable: true, emit: (event) => socket.send(JSON.stringify(event)) },
|
|
812
|
+
});
|
|
813
|
+
|
|
814
|
+
const attempt = async (candidate, cursor) => {
|
|
815
|
+
stream.beginAttempt(); // once per attempt, including the first
|
|
594
816
|
try {
|
|
595
817
|
const value = await callProvider(candidate, {
|
|
596
|
-
onDelta: (text) =>
|
|
818
|
+
onDelta: (text) => stream.observe(text), // content only — see below
|
|
597
819
|
});
|
|
598
820
|
return { kind: "success", value };
|
|
599
821
|
} catch (error) {
|
|
600
|
-
return {
|
|
822
|
+
return {
|
|
823
|
+
kind: "failure",
|
|
824
|
+
error: classify(error, candidate, cursor),
|
|
825
|
+
producedOutput: stream.producedOutput, // never hand-rolled again
|
|
826
|
+
};
|
|
601
827
|
}
|
|
602
|
-
}
|
|
828
|
+
};
|
|
829
|
+
|
|
830
|
+
const result = await executeRoutePlan({ plan, attempt, breaker });
|
|
831
|
+
stream.finish(result.ok ? "succeeded" : "failed");
|
|
832
|
+
return result;
|
|
833
|
+
```
|
|
834
|
+
|
|
835
|
+
Two lifetime rules, and neither is enforceable from inside the package:
|
|
836
|
+
|
|
837
|
+
**Construct it once per turn** — outside the executor and outside your
|
|
838
|
+
structured-output retry loop. One per *attempt* never sees a second attempt, so
|
|
839
|
+
it never resets and `producedOutput` is never true; the single-attempt path is
|
|
840
|
+
indistinguishable from correct, and the bug shows up only under fallback, as two
|
|
841
|
+
partial answers glued together.
|
|
842
|
+
|
|
843
|
+
**Always call `finish()`, and tell it how the turn ended.** A reset still armed
|
|
844
|
+
at the finish line means an earlier attempt's text is on screen, and the two
|
|
845
|
+
outcomes want opposite things. `finish("succeeded")` flushes it — the retry
|
|
846
|
+
succeeded with `content: null` plus tool calls, so nothing triggered the lazy
|
|
847
|
+
reset and that narration belongs to a turn that never said it.
|
|
848
|
+
`finish("failed")` drops it — every attempt failed, so the partial text is the
|
|
849
|
+
best thing the reader is going to get, and wiping it hands them a blank space
|
|
850
|
+
plus an error instead.
|
|
851
|
+
|
|
852
|
+
With `retractable: true`, `producedOutput` stays `false`, so the plan keeps
|
|
853
|
+
every stage. On the retry's first byte the sink receives
|
|
854
|
+
`{ kind: "reset", epoch, seq, reason }` and re-renders from scratch. With
|
|
855
|
+
`retractable: false` — a message already posted through a third-party API, an
|
|
856
|
+
email, a webhook, an append-only row — it latches on the first byte and routing
|
|
857
|
+
clamps to propagate-only, which is the old behaviour and the right one.
|
|
858
|
+
|
|
859
|
+
Your client needs three lines to be correct under a transport that can reorder or
|
|
860
|
+
duplicate, because an in-flight delta from attempt 1 can arrive *after* attempt
|
|
861
|
+
2's reset and text alone cannot be told apart from stale text:
|
|
862
|
+
|
|
863
|
+
```ts
|
|
864
|
+
// Per turn: { lastSeq: -1, newestEpoch: -1, rendered: "", held: new Map() }
|
|
865
|
+
// `lastSeq` starts at -1 — `seq` starts at 0, so seeding it to 0 silently drops
|
|
866
|
+
// the first delta of every turn, on the single-attempt path that is almost all
|
|
867
|
+
// traffic.
|
|
868
|
+
const turn = state.getOrCreate(event.turnId); // scope everything to the turn
|
|
869
|
+
if (event.seq <= turn.lastSeq) return; // replay or duplicate, drop
|
|
870
|
+
if (event.seq > turn.lastSeq + 1) return buffer(turn, event); // arrived early
|
|
871
|
+
turn.lastSeq = event.seq;
|
|
872
|
+
if (event.epoch < turn.newestEpoch) return; // stale attempt, drop
|
|
873
|
+
turn.newestEpoch = event.epoch;
|
|
874
|
+
if (event.kind === "reset") turn.rendered = "";
|
|
875
|
+
else turn.rendered += event.text;
|
|
876
|
+
drainBuffered(turn); // apply anything that was early
|
|
603
877
|
```
|
|
604
878
|
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
879
|
+
Both keys are load-bearing and they do different jobs. **`seq`** is monotonic
|
|
880
|
+
within the turn and never restarts, so it is what makes the stream tolerant of a
|
|
881
|
+
transport that duplicates or reorders — drop anything at or below the last seq
|
|
882
|
+
applied, hold anything that arrives ahead of it. **`epoch`** identifies the
|
|
883
|
+
attempt, so it is what tells a *fresh* delta from a stale one after a reset.
|
|
884
|
+
Neither substitutes for the other: without `seq` a duplicated text event appends
|
|
885
|
+
twice and same-epoch chunks concatenate in arrival order; without `epoch` a
|
|
886
|
+
delta from the wiped attempt is indistinguishable from the retry's.
|
|
887
|
+
|
|
888
|
+
**Bound the hold buffer, and define when a turn ends.** The rule above holds an
|
|
889
|
+
early event until its gap fills — and on a reconnect the gap never fills, because
|
|
890
|
+
the events that would have closed it were dropped. Left alone, the surface then
|
|
891
|
+
freezes on whatever the *retracted* attempt rendered, which is the exact outcome
|
|
892
|
+
this module exists to prevent, and the buffer grows one entry per token. So: cap
|
|
893
|
+
the buffer (a count or a timeout), and on overflow resync from the persisted
|
|
894
|
+
message rather than continuing to hold. For the same reason the client needs a
|
|
895
|
+
turn-final signal — `onAssistantMessage`, or the persisted row landing — at which
|
|
896
|
+
it drops the turn's state entirely. Nothing on this wire tells it; that is the
|
|
897
|
+
host's to define.
|
|
898
|
+
|
|
899
|
+
If your transport already guarantees ordered exactly-once delivery to the client
|
|
900
|
+
(a single WebSocket with no replay window, say), the `seq` half collapses to a
|
|
901
|
+
no-op and the two `epoch` lines are enough — but say so deliberately rather than
|
|
902
|
+
discovering it under load.
|
|
903
|
+
|
|
904
|
+
**In React**, the recipe above mutates in place. Dropped into a store as written,
|
|
905
|
+
`getSnapshot` returns an identity-stable object and `useSyncExternalStore` never
|
|
906
|
+
re-renders — the stream looks dead. Publish a fresh snapshot per applied event.
|
|
907
|
+
And key the rendered element by `turnId`, never by `epoch`: keying by epoch turns
|
|
908
|
+
every reset into an unmount, discarding focus, selection and scroll position when
|
|
909
|
+
the contract only ever needed a content update.
|
|
910
|
+
|
|
911
|
+
**`turnId` is not decoration either.** `epoch` and `seq` both restart each turn,
|
|
912
|
+
so a client that carried "newest epoch" across turns drops every event after the
|
|
913
|
+
first turn that retried, and a later turn's reset tells it to wipe an earlier,
|
|
914
|
+
committed message.
|
|
915
|
+
|
|
916
|
+
Three things stay yours:
|
|
917
|
+
|
|
918
|
+
- **Reasoning deltas.** The watchdog gates its tight budget on the first
|
|
919
|
+
*answer* token, so reasoning can flow well before one. Whether "thinking…"
|
|
920
|
+
counts as output the user has seen is a product call — express it by choosing
|
|
921
|
+
what you pass to `observe`.
|
|
922
|
+
- **Tool-call deltas are not output** and should not go through `observe`. A
|
|
923
|
+
tool call is not a side effect until it is *dispatched*, which happens after
|
|
924
|
+
the turn — so a turn that dies having streamed only tool-call bytes changed
|
|
925
|
+
nothing anyone can see, and clamping it forfeits a fallback for free. If you
|
|
926
|
+
have a genuine mid-attempt effect, call `stream.markProducedOutput()`.
|
|
927
|
+
- **How much flicker is acceptable.** A wipe is not only a flicker — it collapses
|
|
928
|
+
the message's height mid-stream, so an auto-scrolled transcript lurches, and it
|
|
929
|
+
destroys any text selection inside that message. If the body is an `aria-live`
|
|
930
|
+
region, every wipe re-announces the whole answer from the top; keep it
|
|
931
|
+
`aria-busy` while streaming and announce once at the end instead.
|
|
932
|
+
`maxResets` is opt-in with no default: a
|
|
933
|
+
plan with three stages, three candidates and two defect retries can legally
|
|
934
|
+
wipe the screen more than twenty times. Spending the budget latches
|
|
935
|
+
`producedOutput` so routing stops traversing — it does *not* stop emission,
|
|
936
|
+
because an attempt already in flight may be the one that succeeds and its
|
|
937
|
+
answer still has to reach the reader. Note the budget therefore also shapes how
|
|
938
|
+
many endpoints record a failure against the circuit breaker for one turn.
|
|
939
|
+
|
|
940
|
+
- **Which surface gets which callback.** `onAssistantMessage` on the loop fires
|
|
941
|
+
once per completed assistant message; these events stream one attempt of one
|
|
942
|
+
message. Wiring both to the same UI element delivers the text twice. Deltas
|
|
943
|
+
drive the live view; `onAssistantMessage` drives the permanent record.
|
|
944
|
+
|
|
945
|
+
Structured-output retries run in your loop, outside the executor, and get the
|
|
946
|
+
same treatment — open them with `stream.beginAttempt("structured_output_retry")`
|
|
947
|
+
so the user sees the re-ask replace the malformed JSON rather than follow it.
|
|
623
948
|
|
|
624
949
|
### How to map your transport errors onto the routing taxonomy
|
|
625
950
|
|
|
@@ -795,7 +1120,9 @@ client error), call `releaseProbe` so the slot cannot stick.
|
|
|
795
1120
|
your editor. This section covers what the types cannot say: which entry point
|
|
796
1121
|
to reach for, and the contracts that hold between calls.*
|
|
797
1122
|
|
|
798
|
-
Every export is re-exported from the package root
|
|
1123
|
+
Every export is re-exported from the package root **except `@juno-ai/bind/testing`,
|
|
1124
|
+
which is subpath-only so fixtures never reach a production bundle** — but prefer
|
|
1125
|
+
the subpath either way, since it
|
|
799
1126
|
keeps a consumer who only wants routing from pulling in the rest.
|
|
800
1127
|
|
|
801
1128
|
| Import | Owns | Reach for it when |
|
|
@@ -807,7 +1134,8 @@ keeps a consumer who only wants routing from pulling in the rest.
|
|
|
807
1134
|
| `@juno-ai/bind/run` | Wall-clock deadline, failure classification, coalesced heartbeat, tool-batch pooling, child-run lineage and admission, poll backoff, tool-call receipts for retry-safe side effects | A run must be bounded, observable, and able to say *why* it stopped — or it can spawn runs of its own |
|
|
808
1135
|
| `@juno-ai/bind/transcript` | `validateAndHealMessages` | You send transcripts to more than one provider, or you build them across turns |
|
|
809
1136
|
| `@juno-ai/bind/tools` | `sanitizeToolSchema` | Any tool schema reaches a provider — especially third-party ones |
|
|
810
|
-
| `@juno-ai/bind/plugins` | Tool/plugin vocabulary, the registry factory, progressive-disclosure activation | You have more
|
|
1137
|
+
| `@juno-ai/bind/plugins` | Tool/plugin vocabulary, `defineTool` / `pluginFromTools`, the wire-definition and tool-result encoders, the registry factory, progressive-disclosure activation | You are authoring tools, or you have more of them than fit comfortably in one prompt |
|
|
1138
|
+
| `@juno-ai/bind/testing` | Scripted-model fixtures — `loopHarness`, `scriptedModel`, `toolCallTurn`, `finalAnswer`, `freshState`, `recordingSink`, `steppingClock` | You want multi-turn, multi-tool tests without a credential or a mocked chat client |
|
|
811
1139
|
|
|
812
1140
|
### Contracts the types do not carry
|
|
813
1141
|
|
|
@@ -1160,8 +1488,14 @@ const turn: TurnFn = async (messages, tools, signal) => {
|
|
|
1160
1488
|
|
|
1161
1489
|
### Accumulating run statistics
|
|
1162
1490
|
|
|
1491
|
+
**If you use `runToolLoop`, you do not need this** — it folds `RunStats` itself
|
|
1492
|
+
and returns it. This is the shape for a host driving turns by hand.
|
|
1493
|
+
|
|
1163
1494
|
Fold each turn and each tool call as they complete; `RunStats` keeps model time
|
|
1164
|
-
and tool time separate so a slow tool never looks like a slow model.
|
|
1495
|
+
and tool time separate so a slow tool never looks like a slow model. Model
|
|
1496
|
+
calls that are not agent turns — a compaction pass — go through
|
|
1497
|
+
`accumulateAuxiliarySpend` instead, which charges the tokens and the time
|
|
1498
|
+
without counting a turn.
|
|
1165
1499
|
|
|
1166
1500
|
```ts
|
|
1167
1501
|
let stats = emptyRunStats();
|
|
@@ -1273,7 +1607,13 @@ try {
|
|
|
1273
1607
|
stats = accumulateTurn(stats, result);
|
|
1274
1608
|
await heartbeat.beat();
|
|
1275
1609
|
if (!result.message.tool_calls?.length) break;
|
|
1276
|
-
|
|
1610
|
+
const suspend = await runToolBatch(result.message.tool_calls);
|
|
1611
|
+
if (suspend) {
|
|
1612
|
+
// The two pause reasons are separate values — see the stop-reason table.
|
|
1613
|
+
stopReason =
|
|
1614
|
+
suspend.resumeKind === "answer" ? "waiting_for_reply" : "resuming_later";
|
|
1615
|
+
break;
|
|
1616
|
+
}
|
|
1277
1617
|
}
|
|
1278
1618
|
} catch (error) {
|
|
1279
1619
|
stopReason = classifyRunFailure(deadline, error) === "timed_out" ? "deadline" : "aborted";
|
|
@@ -1451,21 +1791,28 @@ two rates weights a 10-token run like a 10,000-token one.
|
|
|
1451
1791
|
Named, not scheduled. Listed so a consumer can tell a deliberate omission from
|
|
1452
1792
|
an oversight.
|
|
1453
1793
|
|
|
1454
|
-
- **
|
|
1455
|
-
|
|
1456
|
-
|
|
1457
|
-
|
|
1458
|
-
|
|
1459
|
-
|
|
1460
|
-
|
|
1461
|
-
|
|
1462
|
-
|
|
1463
|
-
|
|
1464
|
-
|
|
1465
|
-
|
|
1466
|
-
|
|
1467
|
-
|
|
1468
|
-
|
|
1794
|
+
- **Time-to-first-token in `RunStats`.** The loop now folds `RunStats` itself,
|
|
1795
|
+
so `modelTimeMs`, `toolTimeMs` and the per-tool breakdown come for free — but
|
|
1796
|
+
`ttftMs` lives on `TurnTimings` and only the transport can see the first byte.
|
|
1797
|
+
A `callModel` that reported its own timings back would close the gap;
|
|
1798
|
+
`ToolLoopTurn` would have to grow, which is a change to the shape every host
|
|
1799
|
+
already implements.
|
|
1800
|
+
- **A streaming turn contract.** Half of this landed:
|
|
1801
|
+
`createTurnTextStream` owns the emit/retry interaction for assistant *text*,
|
|
1802
|
+
arms `producedOutput` itself, and repairs a retractable surface between
|
|
1803
|
+
attempts ([how-to](#how-to-stream-tokens-to-a-user-without-breaking-fallback)).
|
|
1804
|
+
What has not landed is the *contract*: `TurnFn` still returns one finished
|
|
1805
|
+
`ModelTurnResult`, so `runToolLoop` cannot see the stream and a host must
|
|
1806
|
+
thread it through its own `AttemptFn`. A loop-level streaming turn — where the
|
|
1807
|
+
kernel wires the stream and the reset lands without host cooperation — needs
|
|
1808
|
+
`TurnFn` to grow a streaming variant, and that is a breaking change to the
|
|
1809
|
+
package's central type. Deferred for that reason, not for lack of a design.
|
|
1810
|
+
- **Streaming for tool calls and reasoning.** `createTurnTextStream` handles
|
|
1811
|
+
text only, deliberately. Reasoning is a product judgement the host expresses
|
|
1812
|
+
by what it forwards; tool-call deltas are not output at all until dispatch.
|
|
1813
|
+
Neither has a natural primitive yet, and a host that renders a tool call as it
|
|
1814
|
+
is being assembled reads `assembled()` off `createToolCallAccumulator`
|
|
1815
|
+
mid-stream today, which works.
|
|
1469
1816
|
|
|
1470
1817
|
---
|
|
1471
1818
|
|
package/completion/index.d.ts
CHANGED
|
@@ -2,3 +2,4 @@ export { createStreamWatchdog, DEFAULT_TIME_TO_FIRST_TOKEN_MS, DEFAULT_INTER_CHU
|
|
|
2
2
|
export { detectCompletionDefect, expectsJsonOutput, jsonParses, structuredOutputParses, DEFAULT_STRUCTURED_OUTPUT_MAX_RETRIES, DEFAULT_COMPLETION_DEFECT_MAX_RETRIES, type AssembledCompletion, type CompletionDefect, type CompletionOutputs, type ResponseFormatShape, type StreamedToolCall, } from "./defects.js";
|
|
3
3
|
export { parseToolCallArguments, toolCallArgumentsAbsent, type DispatchableToolCall, type ToolCallArguments, } from "./tool-calls.js";
|
|
4
4
|
export { createToolCallAccumulator, type ToolCallDelta, type AssembledToolCall, type ToolCallAccumulator, } from "./stream-assembly.js";
|
|
5
|
+
export { createTurnTextStream, type TurnResetReason, type TurnStreamEvent, type TurnStreamSink, type TurnTextStream, type TurnTextStreamOptions, } from "./text-stream.js";
|
package/completion/index.js
CHANGED
|
@@ -2,3 +2,4 @@ export { createStreamWatchdog, DEFAULT_TIME_TO_FIRST_TOKEN_MS, DEFAULT_INTER_CHU
|
|
|
2
2
|
export { detectCompletionDefect, expectsJsonOutput, jsonParses, structuredOutputParses, DEFAULT_STRUCTURED_OUTPUT_MAX_RETRIES, DEFAULT_COMPLETION_DEFECT_MAX_RETRIES, } from "./defects.js";
|
|
3
3
|
export { parseToolCallArguments, toolCallArgumentsAbsent, } from "./tool-calls.js";
|
|
4
4
|
export { createToolCallAccumulator, } from "./stream-assembly.js";
|
|
5
|
+
export { createTurnTextStream, } from "./text-stream.js";
|