@juno-ai/bind 1.0.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1009 -64
- package/contracts/index.d.ts +1 -1
- package/contracts/index.js +1 -1
- package/contracts/turn.d.ts +5 -5
- package/index.d.ts +13 -5
- package/index.js +13 -5
- package/package.json +18 -2
- package/plugins/activation.d.ts +67 -0
- package/plugins/activation.js +61 -0
- package/plugins/index.d.ts +3 -0
- package/plugins/index.js +3 -0
- package/plugins/registry.d.ts +52 -0
- package/plugins/registry.js +54 -0
- package/plugins/tool.d.ts +164 -0
- package/plugins/tool.js +9 -0
- package/routing/billing-basis.d.ts +48 -0
- package/routing/billing-basis.js +67 -0
- package/routing/circuit-breaker.d.ts +2 -2
- package/routing/errors.d.ts +1 -1
- package/routing/executor.d.ts +3 -3
- package/routing/executor.js +1 -1
- package/routing/index.d.ts +11 -9
- package/routing/index.js +11 -9
- package/routing/plan-degradation.d.ts +34 -0
- package/routing/plan-degradation.js +38 -0
- package/routing/plan.d.ts +2 -2
- package/routing/planner.d.ts +4 -4
- package/routing/planner.js +1 -1
- package/routing/policy.d.ts +1 -1
- package/routing/policy.js +1 -1
- package/routing/transport.d.ts +2 -2
- package/run/harness.d.ts +94 -0
- package/run/harness.js +140 -0
- package/run/index.d.ts +2 -0
- package/run/index.js +2 -0
- package/run/tool-batch.d.ts +16 -0
- package/run/tool-batch.js +83 -0
- package/tools/index.d.ts +1 -0
- package/tools/index.js +1 -0
- package/tools/sanitize-schema.d.ts +150 -0
- package/tools/sanitize-schema.js +683 -0
- package/transcript/index.d.ts +1 -0
- package/transcript/index.js +1 -0
- package/transcript/validate.d.ts +54 -0
- package/transcript/validate.js +226 -0
package/README.md
CHANGED
|
@@ -3,93 +3,1038 @@
|
|
|
3
3
|
**`@juno-ai/bind` — an agent harness.**
|
|
4
4
|
|
|
5
5
|
The agent loop is a bind chain: each turn sequences a model completion into
|
|
6
|
-
tool effects into the next turn's context. `bind` is the harness that runs
|
|
7
|
-
|
|
8
|
-
|
|
6
|
+
tool effects into the next turn's context. `bind` is the harness that runs that
|
|
7
|
+
chain — the runtime-agnostic core of a production agent loop, extracted from
|
|
8
|
+
[Monad](https://onmonad.ai).
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
This document is organised on the [Diátaxis](https://diataxis.fr) axes.
|
|
11
|
+
**Tutorial** and **How-to** are practical; **Reference** and **Explanation** are
|
|
12
|
+
theoretical. If you are a coding agent working against this package, read the
|
|
13
|
+
exported types for signatures — they are the specification — and use
|
|
14
|
+
[Reference](#reference) for the contracts those types cannot express,
|
|
15
|
+
[Usage scenarios](#usage-scenarios) for porting shapes, and
|
|
16
|
+
[Rules for automated contributors](#rules-for-automated-contributors) for the
|
|
17
|
+
constraints that will fail CI if you break them.
|
|
11
18
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
19
|
+
| I want to… | Go to |
|
|
20
|
+
|---|---|
|
|
21
|
+
| Understand what this is and whether I need it | [Explanation](#explanation) |
|
|
22
|
+
| Get something running end to end | [Tutorial](#tutorial-route-one-completion) |
|
|
23
|
+
| Solve one specific problem | [How-to guides](#how-to-guides) |
|
|
24
|
+
| Know which entry point to use, and what holds between calls | [Reference](#reference) |
|
|
25
|
+
| Know how versions work before installing | [Versioning](#versioning) |
|
|
26
|
+
| Port an existing agent runtime onto this | [Usage scenarios](#usage-scenarios) |
|
|
27
|
+
| Know what is deliberately not here yet | [Roadmap](#roadmap) |
|
|
28
|
+
| Change this package safely | [Rules for automated contributors](#rules-for-automated-contributors) |
|
|
16
29
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Explanation
|
|
33
|
+
|
|
34
|
+
*Understanding-oriented. Read this to know why the package is shaped the way it
|
|
35
|
+
is; you do not need it to use the package.*
|
|
36
|
+
|
|
37
|
+
### What a harness is, and what it is not
|
|
38
|
+
|
|
39
|
+
A harness owns the parts of an agent loop that are the same for everyone:
|
|
40
|
+
deciding which provider to call and what to do when it fails, bounding a run in
|
|
41
|
+
wall-clock time, keeping a transcript in a shape providers accept, rewriting
|
|
42
|
+
tool schemas that strict validators reject, and tracking which tools are
|
|
43
|
+
currently loaded.
|
|
44
|
+
|
|
45
|
+
A *runtime* owns the parts that are yours: identity, authorization, persistence,
|
|
46
|
+
transports, prompt voice, and product behaviour. `bind` deliberately contains
|
|
47
|
+
none of that. It never reads the environment, never touches a filesystem, and
|
|
48
|
+
holds no secrets — which is what lets the same code run on Bun, Node, and edge
|
|
49
|
+
runtimes such as Cloudflare workerd.
|
|
50
|
+
|
|
51
|
+
### The fences, and why they exist
|
|
52
|
+
|
|
53
|
+
Four constraints, enforced by lint:
|
|
54
|
+
|
|
55
|
+
- **No `@/*` application imports.** Anything the harness needs from the host
|
|
56
|
+
arrives through an injected function or value — a port — never a direct
|
|
57
|
+
import.
|
|
58
|
+
- **No Node builtins, no `process`.** No filesystem, no environment reads.
|
|
59
|
+
Configuration is explicit input. Clocks are injectable, with a `Date.now`
|
|
60
|
+
default as the one sanctioned exception.
|
|
61
|
+
- **No framework imports.**
|
|
62
|
+
- **Peer dependencies only** (`zod`, plus `openai` as an optional *type-only*
|
|
63
|
+
peer). The host supplies the instances, so schemas never split across
|
|
64
|
+
duplicate copies — a failure that stays invisible until two zod instances
|
|
65
|
+
disagree about the same schema at runtime.
|
|
66
|
+
|
|
67
|
+
The fences are not stylistic. They are the reason a Cloudflare Worker and a
|
|
68
|
+
long-lived Node server can share this code unmodified.
|
|
69
|
+
|
|
70
|
+
### Why the plugin types are generic
|
|
71
|
+
|
|
72
|
+
`ToolPlugin` is generic over the invocation context (`TCtx`) rather than
|
|
73
|
+
shipping a concrete one. Almost nothing in a real tool context is common: the
|
|
74
|
+
abort signal is, and the rest is the host's own identity model, its
|
|
75
|
+
authorization, and its product features. Two applications comparing notes here
|
|
76
|
+
will typically find they share one field. A concrete context would therefore be
|
|
77
|
+
either a lowest-common-denominator or the union of several products' identity
|
|
78
|
+
models — so the context is a type parameter you supply, and you add your own
|
|
79
|
+
plugin fields by ordinary interface extension.
|
|
80
|
+
|
|
81
|
+
`ToolResult` is separately generic over the content-part type for a smaller
|
|
82
|
+
reason: the wire shape is the provider's, but turning bytes into a multimodal
|
|
83
|
+
part needs runtime-specific APIs that differ between Node and the edge. The
|
|
84
|
+
harness carries parts through without interpreting them, which keeps those APIs
|
|
85
|
+
— and the dependencies they imply — out of a package that must run on workerd.
|
|
86
|
+
|
|
87
|
+
### Why the registry is a factory
|
|
88
|
+
|
|
89
|
+
`createToolRegistry` returns an instance rather than exposing a module-level
|
|
90
|
+
map. Module state is per-isolate on edge runtimes and its lifetime is not the
|
|
91
|
+
host's; it also makes tests share state implicitly. A host that wants singleton
|
|
92
|
+
ergonomics wraps one instance in a module — the choice belongs to the host.
|
|
93
|
+
|
|
94
|
+
### Why a restored activation set is a hint
|
|
95
|
+
|
|
96
|
+
Progressive tool disclosure persists plugin **names**; implementations resolve
|
|
97
|
+
at load time. Between two runs a plugin can be renamed, gated off, or (if it is
|
|
98
|
+
dynamically connected) fail to reconnect. `rehydrateActivation` therefore
|
|
99
|
+
re-validates every persisted entry against current reality and **drops** what no
|
|
100
|
+
longer resolves, returning the drops with a reason rather than failing the run.
|
|
101
|
+
Reporting a plugin as active when its tools cannot be called is worse than
|
|
102
|
+
losing it.
|
|
103
|
+
|
|
104
|
+
### What stays with your application
|
|
105
|
+
|
|
106
|
+
Transports' request construction and wire-error classification, credentials and
|
|
107
|
+
environment parsing, your routing policy configuration, billing accounting
|
|
108
|
+
(persistence and charging), inference logging, authorization, prompt rendering,
|
|
109
|
+
and run orchestration — the queue a run is scheduled on, and any child runs it
|
|
110
|
+
spawns (see [Spawning child runs](#spawning-child-runs-sub-agents)).
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## Tutorial: route one completion
|
|
115
|
+
|
|
116
|
+
*Learning-oriented. Follow these steps in order on a scratch file; the goal is a
|
|
117
|
+
working mental model, not production code.*
|
|
118
|
+
|
|
119
|
+
You will plan a route across two providers, execute it against a fake
|
|
120
|
+
transport, and watch the failure policy fall over to the second provider.
|
|
121
|
+
|
|
122
|
+
**1. Install.** `zod` is a peer dependency — bring your own v4 instance. Add
|
|
123
|
+
`openai` too if you use `/run`, `/transcript`, `/contracts`, or the package
|
|
124
|
+
root: they reference its message types (type-only, erased at runtime).
|
|
125
|
+
|
|
126
|
+
```sh
|
|
127
|
+
bun add @juno-ai/bind zod openai
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
**2. Describe your providers.** A `PlannerTransport` answers two questions:
|
|
131
|
+
are you available, and can you serve this model? It returns a candidate, a
|
|
132
|
+
recorded skip, or `unserved`.
|
|
48
133
|
|
|
49
134
|
```ts
|
|
50
135
|
import {
|
|
51
|
-
buildRoutePlan,
|
|
52
|
-
executeRoutePlan,
|
|
53
|
-
createCircuitBreaker,
|
|
54
136
|
canonicalModelIdSchema,
|
|
137
|
+
providerIdSchema,
|
|
138
|
+
type PlannerTransport,
|
|
55
139
|
} from "@juno-ai/bind/routing";
|
|
56
140
|
|
|
141
|
+
const model = canonicalModelIdSchema.parse("openai/gpt-example");
|
|
142
|
+
const primaryId = providerIdSchema.parse("primary");
|
|
143
|
+
const backupId = providerIdSchema.parse("backup");
|
|
144
|
+
|
|
145
|
+
function fakeTransport(id: typeof primaryId): PlannerTransport {
|
|
146
|
+
return {
|
|
147
|
+
id,
|
|
148
|
+
getAvailability: () => ({ available: true }),
|
|
149
|
+
resolveCandidate: () => ({
|
|
150
|
+
kind: "candidate",
|
|
151
|
+
candidate: {
|
|
152
|
+
providerId: id,
|
|
153
|
+
canonicalModelId: model,
|
|
154
|
+
providerInvocationModel: "gpt-example",
|
|
155
|
+
credentialSource: "platform",
|
|
156
|
+
creditEligible: true,
|
|
157
|
+
capabilities: new Set(["chat_completions"]),
|
|
158
|
+
maxCompletionTokens: null,
|
|
159
|
+
pricingBasis: { kind: "provider_reported" },
|
|
160
|
+
bindingFingerprint: `${id}:gpt-example`,
|
|
161
|
+
},
|
|
162
|
+
}),
|
|
163
|
+
};
|
|
164
|
+
}
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
**3. Build the plan.** Policy order is the only ordering input. The result is
|
|
168
|
+
frozen, secret-free, and safe to log or snapshot in a test.
|
|
169
|
+
|
|
170
|
+
```ts
|
|
171
|
+
import { buildRoutePlan } from "@juno-ai/bind/routing";
|
|
172
|
+
|
|
57
173
|
const { plan, skips } = buildRoutePlan({
|
|
58
|
-
primaryModel:
|
|
174
|
+
primaryModel: model,
|
|
59
175
|
fallbackModel: null,
|
|
60
|
-
requirements: {
|
|
61
|
-
|
|
62
|
-
|
|
176
|
+
requirements: {
|
|
177
|
+
capabilities: new Set(["chat_completions"]),
|
|
178
|
+
requestedMaxCompletionTokens: null,
|
|
179
|
+
},
|
|
180
|
+
policyFor: () => ({ mode: "ordered", providers: [primaryId, backupId] }),
|
|
181
|
+
transports: new Map([
|
|
182
|
+
[primaryId, fakeTransport(primaryId)],
|
|
183
|
+
[backupId, fakeTransport(backupId)],
|
|
184
|
+
]),
|
|
63
185
|
});
|
|
64
186
|
|
|
187
|
+
console.log(plan.stages[0].candidates.map((c) => c.providerId)); // primary, backup
|
|
188
|
+
console.log(skips); // [] — nothing was filtered out
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
**4. Execute it.** Your `attempt` function performs the real call and
|
|
192
|
+
classifies any failure into facts. It never decides route order — that is the
|
|
193
|
+
executor's job.
|
|
194
|
+
|
|
195
|
+
```ts
|
|
196
|
+
import { executeRoutePlan, createCircuitBreaker } from "@juno-ai/bind/routing";
|
|
197
|
+
|
|
65
198
|
const result = await executeRoutePlan({
|
|
66
199
|
plan,
|
|
67
200
|
breaker: createCircuitBreaker(),
|
|
68
|
-
attempt: async (candidate, cursor) =>
|
|
201
|
+
attempt: async (candidate, cursor) => {
|
|
202
|
+
if (candidate.providerId === primaryId) {
|
|
203
|
+
return {
|
|
204
|
+
kind: "failure",
|
|
205
|
+
error: {
|
|
206
|
+
kind: "http",
|
|
207
|
+
category: "server_error",
|
|
208
|
+
statusCode: 503,
|
|
209
|
+
retryAfterMs: null,
|
|
210
|
+
target: {
|
|
211
|
+
cursor,
|
|
212
|
+
providerId: candidate.providerId,
|
|
213
|
+
canonicalModelId: candidate.canonicalModelId,
|
|
214
|
+
providerInvocationModel: candidate.providerInvocationModel,
|
|
215
|
+
durationMs: 12,
|
|
216
|
+
},
|
|
217
|
+
cause: new Error("upstream unavailable"),
|
|
218
|
+
},
|
|
219
|
+
};
|
|
220
|
+
}
|
|
221
|
+
return { kind: "success", value: "hello from backup" };
|
|
222
|
+
},
|
|
223
|
+
});
|
|
224
|
+
|
|
225
|
+
if (result.ok) {
|
|
226
|
+
console.log(result.value); // "hello from backup"
|
|
227
|
+
console.log(result.served.providerId); // backup
|
|
228
|
+
console.log(result.fallbackKind); // "provider"
|
|
229
|
+
}
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
**What you just saw.** A 503 classifies as a retriable transport failure, so the
|
|
233
|
+
disposition table allows traversal to the next provider and records a breaker
|
|
234
|
+
failure against the first endpoint. You did not write that logic, and you cannot
|
|
235
|
+
accidentally reorder it from inside a transport.
|
|
236
|
+
|
|
237
|
+
**5. Next.** Add a wall-clock budget with
|
|
238
|
+
[`createRunDeadline`](#how-to-bound-a-run-in-wall-clock-time), or add
|
|
239
|
+
progressive tool disclosure with
|
|
240
|
+
[`createToolRegistry`](#how-to-add-progressive-tool-disclosure).
|
|
241
|
+
|
|
242
|
+
---
|
|
243
|
+
|
|
244
|
+
## How-to guides
|
|
245
|
+
|
|
246
|
+
*Goal-oriented. Each answers one question and assumes you know roughly what you
|
|
247
|
+
are doing.*
|
|
248
|
+
|
|
249
|
+
### How to pin a model to one provider
|
|
250
|
+
|
|
251
|
+
Use a hard `only` fence. Plan-time filtering and runtime traversal both respect
|
|
252
|
+
it — a pinned provider that fails at request time is never retried elsewhere.
|
|
253
|
+
|
|
254
|
+
```ts
|
|
255
|
+
policyFor: () => ({ mode: "only", provider: complianceProviderId });
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
Useful for evals (reproducible plans) and for compliance routing.
|
|
259
|
+
|
|
260
|
+
### How to bound a run in wall-clock time
|
|
261
|
+
|
|
262
|
+
Create the deadline where you classify the outcome, and dispose it in a
|
|
263
|
+
`finally`.
|
|
264
|
+
|
|
265
|
+
```ts
|
|
266
|
+
import { createRunDeadline, classifyRunFailure } from "@juno-ai/bind/run";
|
|
267
|
+
|
|
268
|
+
const deadline = createRunDeadline({ timeoutMs: 60 * 60 * 1000, label: "agent run" });
|
|
269
|
+
try {
|
|
270
|
+
for (;;) {
|
|
271
|
+
deadline.throwIfTimedOut();
|
|
272
|
+
await callModel({ signal: deadline.withExternal(cancellationSignal) });
|
|
273
|
+
}
|
|
274
|
+
} catch (error) {
|
|
275
|
+
const status = classifyRunFailure(deadline, error); // "timed_out" | "failed"
|
|
276
|
+
} finally {
|
|
277
|
+
deadline.dispose();
|
|
278
|
+
}
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
**Ownership rule:** whoever needs to read `deadline.timedOut` must create and
|
|
282
|
+
own the deadline. A loop handed one must not dispose it; a loop given none
|
|
283
|
+
should mint its own, so a turn is never unbounded. Handle caller-driven
|
|
284
|
+
cancellation *before* calling `classifyRunFailure` — a cancellation is not a
|
|
285
|
+
timeout, and `timedOut` stays false when only a combined external signal fires.
|
|
286
|
+
|
|
287
|
+
### How to stop a run's progress writes from stampeding
|
|
288
|
+
|
|
289
|
+
```ts
|
|
290
|
+
import { createCoalescedHeartbeat } from "@juno-ai/bind/run";
|
|
291
|
+
|
|
292
|
+
const heartbeat = createCoalescedHeartbeat({
|
|
293
|
+
coalesceMs: 10_000,
|
|
294
|
+
flush: () => db.bumpRunRow(runId),
|
|
295
|
+
onError: (error) => log.warn("heartbeat flush failed", { error }),
|
|
296
|
+
});
|
|
297
|
+
|
|
298
|
+
await heartbeat.beat(); // coalesced
|
|
299
|
+
await heartbeat.beat({ force: true }); // always flushes, resolves after the write
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
Flush errors go to `onError` and are swallowed, so a transient database hiccup
|
|
303
|
+
never aborts a run. Flushes drain one at a time, so two overlapping writes can
|
|
304
|
+
never land out of order.
|
|
305
|
+
|
|
306
|
+
### How to keep a provider from rejecting your whole tool list
|
|
307
|
+
|
|
308
|
+
Strict validators reject the **entire** request — every tool — on the first
|
|
309
|
+
schema violation. Run each tool's JSON Schema through the sanitizer before it
|
|
310
|
+
reaches the model.
|
|
311
|
+
|
|
312
|
+
```ts
|
|
313
|
+
import { sanitizeToolSchema } from "@juno-ai/bind/tools";
|
|
314
|
+
|
|
315
|
+
const wireTools = tools.map((tool) => ({
|
|
316
|
+
type: "function",
|
|
317
|
+
function: {
|
|
318
|
+
name: tool.name,
|
|
319
|
+
description: tool.description,
|
|
320
|
+
parameters: sanitizeToolSchema(tool.inputSchema),
|
|
321
|
+
},
|
|
322
|
+
}));
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
Every transform is correctness-preserving and was bisected against live
|
|
326
|
+
inference. It fixes, among others: a `type` array where a provider requires a
|
|
327
|
+
scalar; `enum` on a non-string type; `required` entries with no matching
|
|
328
|
+
property; a boolean `additionalProperties: false` on a nested object; and a
|
|
329
|
+
parameter literally named `properties`. Run it on third-party (e.g. MCP) tool
|
|
330
|
+
schemas too — those are where the violations usually come from.
|
|
331
|
+
|
|
332
|
+
### How to repair a transcript before sending it
|
|
333
|
+
|
|
334
|
+
```ts
|
|
335
|
+
import { validateAndHealMessages } from "@juno-ai/bind/transcript";
|
|
336
|
+
|
|
337
|
+
const { messages, issues } = validateAndHealMessages(transcript);
|
|
338
|
+
if (issues.length > 0) log.warn("healed transcript", { issues });
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
Detects and repairs orphan tool results, dangling unanswered tool calls, empty
|
|
342
|
+
assistant messages mid-conversation, and a trailing assistant turn — which is
|
|
343
|
+
prefill on one provider's native API and a hard rejection through another.
|
|
344
|
+
|
|
345
|
+
### How to add progressive tool disclosure
|
|
346
|
+
|
|
347
|
+
Model tool-selection accuracy degrades past a few dozen tools, and every tool's
|
|
348
|
+
schema is resent on every turn. Load a small core set, announce the rest as a
|
|
349
|
+
catalog, and activate on demand.
|
|
350
|
+
|
|
351
|
+
```ts
|
|
352
|
+
import {
|
|
353
|
+
createToolRegistry,
|
|
354
|
+
initialActivePlugins,
|
|
355
|
+
partitionPluginCatalog,
|
|
356
|
+
} from "@juno-ai/bind/plugins";
|
|
357
|
+
|
|
358
|
+
const registry = createToolRegistry<MyPlugin>({
|
|
359
|
+
corePlugins: ["messaging", "memory"],
|
|
360
|
+
aliases: { "old-name": "new-name" },
|
|
361
|
+
onRegister: (plugin) => indexPluginSkills(plugin),
|
|
69
362
|
});
|
|
363
|
+
|
|
364
|
+
registry.register(myPlugin);
|
|
365
|
+
|
|
366
|
+
const active = new Set(initialActivePlugins(registry.corePlugins()));
|
|
367
|
+
const { active: shown, loadable } = partitionPluginCatalog(active, registry.summaries());
|
|
70
368
|
```
|
|
71
369
|
|
|
72
|
-
|
|
370
|
+
`aliases` are permanent: persisted activation state stores names, so an alias is
|
|
371
|
+
how a rename avoids silently stripping capabilities from live sessions without a
|
|
372
|
+
data migration.
|
|
373
|
+
|
|
374
|
+
`partitionPluginCatalog` returns **data, not prose** — catalog wording is your
|
|
375
|
+
system prompt's business, and re-rendering it byte-identically for unchanged
|
|
376
|
+
inputs is what preserves a provider's prompt-cache prefix.
|
|
377
|
+
|
|
378
|
+
### How to restore a persisted activation set
|
|
379
|
+
|
|
380
|
+
```ts
|
|
381
|
+
import { rehydrateActivation } from "@juno-ai/bind/plugins";
|
|
382
|
+
|
|
383
|
+
const { active, dropped } = await rehydrateActivation(persistedNames, {
|
|
384
|
+
canonicalizeName: (name) => registry.canonicalizeName(name),
|
|
385
|
+
resolve: async (name) => {
|
|
386
|
+
if (isGatedOffThisRun(name)) return "unavailable";
|
|
387
|
+
if (!isDynamic(name)) return registry.get(name) ? true : "unknown";
|
|
388
|
+
return (await connect(name)) ? true : "unreachable";
|
|
389
|
+
},
|
|
390
|
+
});
|
|
391
|
+
|
|
392
|
+
for (const drop of dropped) log.warn("activation dropped", drop);
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
The walk canonicalizes, de-duplicates (two legacy names collapsing to one plugin
|
|
396
|
+
resolve **once**, so a reconnect is not paid twice), and drops rather than
|
|
397
|
+
throws. `resolve` is async precisely so a reconnect can happen inside it.
|
|
398
|
+
|
|
399
|
+
### How to scope the circuit breaker per tenant
|
|
400
|
+
|
|
401
|
+
Pass a non-secret `credentialScope`. Without it, one tenant's revoked
|
|
402
|
+
bring-your-own key opens the circuit for every tenant sharing the same
|
|
403
|
+
`credentialSource`.
|
|
404
|
+
|
|
405
|
+
```ts
|
|
406
|
+
breaker.recordFailure({
|
|
407
|
+
providerId,
|
|
408
|
+
invocationModel,
|
|
409
|
+
credentialSource: "tenant",
|
|
410
|
+
credentialScope: tenantTag, // opaque, non-secret — it lands in state keys
|
|
411
|
+
});
|
|
412
|
+
```
|
|
413
|
+
|
|
414
|
+
If a half-open probe ends without a recordable outcome (an abort, a propagated
|
|
415
|
+
client error), call `releaseProbe` so the slot cannot stick.
|
|
416
|
+
|
|
417
|
+
---
|
|
418
|
+
|
|
419
|
+
## Reference
|
|
420
|
+
|
|
421
|
+
*Information-oriented. The exported types are the specification — read them in
|
|
422
|
+
your editor. This section covers what the types cannot say: which entry point
|
|
423
|
+
to reach for, and the contracts that hold between calls.*
|
|
424
|
+
|
|
425
|
+
Every export is re-exported from the package root, but prefer the subpath — it
|
|
426
|
+
keeps a consumer who only wants routing from pulling in the rest.
|
|
427
|
+
|
|
428
|
+
| Import | Owns | Reach for it when |
|
|
429
|
+
|---|---|---|
|
|
430
|
+
| `@juno-ai/bind/routing` | Route plans, the planner, the failure taxonomy, the plan executor, the circuit breaker, billing-basis arithmetic, config degradation | You call more than one provider or model, or you want retries and fallback governed by one table |
|
|
431
|
+
| `@juno-ai/bind/contracts` | Turn vocabulary — `TurnFn`, `ModelTurnResult`, `StopReason`, `RunStats` | You want one seam between your loop and any LLM client, and comparable per-run metrics |
|
|
432
|
+
| `@juno-ai/bind/run` | Wall-clock deadline, failure classification, coalesced heartbeat, tool-batch pooling | A run must be bounded, observable, and able to say *why* it stopped |
|
|
433
|
+
| `@juno-ai/bind/transcript` | `validateAndHealMessages` | You send transcripts to more than one provider, or you build them across turns |
|
|
434
|
+
| `@juno-ai/bind/tools` | `sanitizeToolSchema` | Any tool schema reaches a provider — especially third-party ones |
|
|
435
|
+
| `@juno-ai/bind/plugins` | Tool/plugin vocabulary, the registry factory, progressive-disclosure activation | You have more tools than fit comfortably in one prompt |
|
|
436
|
+
|
|
437
|
+
### Contracts the types do not carry
|
|
438
|
+
|
|
439
|
+
**Routing — attempt order is fixed.** Structured-output attempt → model stage →
|
|
440
|
+
provider candidate → same-endpoint retry. Nothing else reorders it: not
|
|
441
|
+
transport registration order, not map iteration, not the clock.
|
|
442
|
+
|
|
443
|
+
**Routing — transports classify, they never decide.** An `AttemptFn` reports a
|
|
444
|
+
fact (`aborted` / `completion_defect` / `network` / `http` + category);
|
|
445
|
+
`failureDisposition` alone decides retry, next-provider, fallback-model, or
|
|
446
|
+
propagate. Putting routing logic in a transport is the one way to break the
|
|
447
|
+
guarantee that identical inputs produce identical plans.
|
|
448
|
+
|
|
449
|
+
**Routing — a healthy endpoint must not be punished for a bad request.**
|
|
450
|
+
Request-shaped rejections (a 400, a moderation refusal) traverse to another
|
|
451
|
+
provider but record **no** breaker failure. Otherwise one caller's malformed or
|
|
452
|
+
flagged prompt degrades a shared endpoint for everyone.
|
|
453
|
+
|
|
454
|
+
**Routing — a fallback model equal to the primary is ignored**, so you can pass
|
|
455
|
+
a configured fallback through unconditionally without producing a duplicate
|
|
456
|
+
stage.
|
|
457
|
+
|
|
458
|
+
**Routing — breaker keys must stay secret-free.** `credentialScope` lands in
|
|
459
|
+
state keys; pass an opaque tag, never key material. Omitting it means one
|
|
460
|
+
tenant's revoked key opens the circuit for every tenant sharing that
|
|
461
|
+
`credentialSource`. Defaults: open after 3 consecutive failures, 60 s cooldown,
|
|
462
|
+
60 s ceiling including `Retry-After` extensions. Resolve a half-open probe that
|
|
463
|
+
ended without a recordable outcome via `releaseProbe`, or the slot sticks.
|
|
464
|
+
|
|
465
|
+
**Run — whoever classifies the outcome owns the deadline.** `timedOut` reflects
|
|
466
|
+
*that* deadline firing, and stays false when only a combined external signal
|
|
467
|
+
aborts — which is what separates a timeout from a cancellation. Handle
|
|
468
|
+
cancellation before calling `classifyRunFailure`. A loop handed a deadline must
|
|
469
|
+
not dispose it; a loop given none should mint its own, so a turn is never
|
|
470
|
+
unbounded.
|
|
471
|
+
|
|
472
|
+
**Run — pooling preserves input order and answers everything.**
|
|
473
|
+
`runToolCallsPooledByTool` caps concurrent calls to the *same* tool at 5 while
|
|
474
|
+
different tools fan out fully, and returns `PromiseSettledResult`s in input
|
|
475
|
+
order. A rejection still needs a synthesized error tool message, or the next
|
|
476
|
+
request carries an unanswered `tool_call_id`.
|
|
477
|
+
|
|
478
|
+
**Routing — a policy that names a provider you have not configured degrades,
|
|
479
|
+
it does not fail.** `buildRoutePlanWithConfigDegradation` drops such a stage at
|
|
480
|
+
*plan time* and records why, so a missing key costs you that provider rather
|
|
481
|
+
than making the model unavailable. Runtime failures keep the fence: a configured
|
|
482
|
+
`only` provider that errors at request time is still never retried elsewhere.
|
|
483
|
+
|
|
484
|
+
**Run — heartbeat flushes drain one at a time**, so two overlapping writes
|
|
485
|
+
cannot land out of order. Flush errors go to `onError` and are swallowed; a
|
|
486
|
+
transient storage failure never aborts a run.
|
|
487
|
+
|
|
488
|
+
**Contracts — model time and tool time stay separate.** Task wall-clock
|
|
489
|
+
conflates provider inference speed with tool execution. `RunStats` reports both,
|
|
490
|
+
plus a per-tool breakdown; field shapes follow the [StirrupJS](https://github.com/stirrupjs/stirrup)
|
|
491
|
+
`speedStats` methodology, so numbers stay comparable with published benchmarks
|
|
492
|
+
and a slow tool never reads as a slow model.
|
|
493
|
+
|
|
494
|
+
**Transcript — the second argument is an assertion, not a silence.** Ids passed
|
|
495
|
+
as `allowedOpenToolCallIds` are *legitimately* open (a suspended call awaiting a
|
|
496
|
+
human answer). Because this is the universal heal site and a correct transcript
|
|
497
|
+
never reaches it with a suspended call unpaired, an allowed id arriving unpaired
|
|
498
|
+
means resume failed — and it fails loudly rather than being quietly kept.
|
|
499
|
+
|
|
500
|
+
**Tools — sanitize third-party schemas too.** Strict validators reject the
|
|
501
|
+
*entire* request, every tool, on the first violation, and third-party schemas
|
|
502
|
+
are where violations usually originate. Every transform is
|
|
503
|
+
correctness-preserving and was bisected against live inference.
|
|
504
|
+
|
|
505
|
+
**Plugins — `rawJsonSchema` overrides, it does not replace.** `parameters` is
|
|
506
|
+
always required; a host authoring tools as raw JSON Schema supplies a
|
|
507
|
+
placeholder there and puts the real schema on `rawJsonSchema`, which is what
|
|
508
|
+
reaches the model.
|
|
509
|
+
|
|
510
|
+
**Plugins — availability is evaluated per call.** `get()` returns `undefined`
|
|
511
|
+
for an unavailable plugin while `all()` still includes it, so a plugin can stay
|
|
512
|
+
registered for configuration purposes while being gated off for a run. A
|
|
513
|
+
connection going unhealthy mid-process takes its tools out of the catalog with
|
|
514
|
+
no re-registration.
|
|
515
|
+
|
|
516
|
+
**Plugins — aliases are permanent.** Activation state persists *names*, so an
|
|
517
|
+
alias is how a rename avoids stripping capabilities from live sessions without a
|
|
518
|
+
data migration. Removing one silently downgrades every session that still stores
|
|
519
|
+
the old name.
|
|
73
520
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
521
|
+
**Plugins — `hidden: true` keeps a tool callable but unadvertised**, for resumed
|
|
522
|
+
sessions whose history references a tool you have retired from the prompt.
|
|
523
|
+
|
|
524
|
+
**Plugins — rehydration drops, it never throws.** `resolve` returns `true` to
|
|
525
|
+
keep or a drop reason to discard, and may be async so a reconnect can happen
|
|
526
|
+
inside it. The walk canonicalizes and de-duplicates first, so two legacy names
|
|
527
|
+
collapsing to one plugin resolve once — a reconnect is not paid twice.
|
|
528
|
+
|
|
529
|
+
**Plugins — catalog partitioning returns data, not prose.** Wording belongs to
|
|
530
|
+
your system prompt, where re-rendering byte-identically for unchanged inputs is
|
|
531
|
+
what preserves a provider's prompt-cache prefix.
|
|
532
|
+
|
|
533
|
+
### Suspend semantics
|
|
534
|
+
|
|
535
|
+
A first-party tool returns a `SuspendDirective` on a **successful** result to end
|
|
536
|
+
the run with its call recorded as awaiting resolution. The resume kind decides
|
|
537
|
+
what happens to the tool message:
|
|
538
|
+
|
|
539
|
+
- `"answer"` — the reply becomes this call's `role:"tool"` result, so the loop
|
|
540
|
+
**withholds** the message. At most one may be open per run.
|
|
541
|
+
- `"wake"` — a time or event resume that re-enters via a prompt and **keeps** the
|
|
542
|
+
message.
|
|
543
|
+
|
|
544
|
+
`request` is an opaque render/route payload, validated by the consumer and never
|
|
545
|
+
inspected by the loop.
|
|
546
|
+
|
|
547
|
+
---
|
|
548
|
+
|
|
549
|
+
## Usage scenarios
|
|
550
|
+
|
|
551
|
+
*Sketches, not runnable files — they name types loosely and omit imports. Each
|
|
552
|
+
shows the shape of one migration an existing agent runtime typically has to
|
|
553
|
+
make. Read the [Reference](#reference) for exact signatures.*
|
|
554
|
+
|
|
555
|
+
### Tools authored as raw JSON Schema, not zod
|
|
556
|
+
|
|
557
|
+
`ToolDef.parameters` is a zod schema, but a host whose tools are already JSON
|
|
558
|
+
Schema does not need to rewrite them — set `rawJsonSchema` and it is used
|
|
559
|
+
verbatim. Keep `parameters` as a permissive placeholder if you validate
|
|
560
|
+
elsewhere.
|
|
561
|
+
|
|
562
|
+
```ts
|
|
563
|
+
const toolDef: ToolDef = {
|
|
564
|
+
name: "search",
|
|
565
|
+
description: "Search one page of results.",
|
|
566
|
+
parameters: z.unknown(), // unused when rawJsonSchema is present
|
|
567
|
+
rawJsonSchema: existingJsonSchema, // your hand-written schema, as-is
|
|
568
|
+
annotations: { readOnlyHint: true },
|
|
569
|
+
};
|
|
570
|
+
|
|
571
|
+
// Sanitize on the way to the model, regardless of which form it came from.
|
|
572
|
+
const wire = {
|
|
573
|
+
type: "function",
|
|
574
|
+
function: {
|
|
575
|
+
name: toolDef.name,
|
|
576
|
+
description: toolDef.description,
|
|
577
|
+
parameters: sanitizeToolSchema(toolDef.rawJsonSchema ?? z.toJSONSchema(toolDef.parameters)),
|
|
578
|
+
},
|
|
579
|
+
};
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
### One plugin per integration
|
|
583
|
+
|
|
584
|
+
A natural plugin boundary is the external system a group of tools talks to.
|
|
585
|
+
|
|
586
|
+
`isAvailable` gates *whether the deployment has this integration at all* — a
|
|
587
|
+
build flag, a missing binding, an edition that does not ship it. It must return
|
|
588
|
+
the same answer for the whole life of the process: the catalog is re-rendered
|
|
589
|
+
several times per run, and a value that flips mid-run changes the system prompt
|
|
590
|
+
between turns, which costs the prompt-cache prefix and can strand a tool call
|
|
591
|
+
the model already issued.
|
|
592
|
+
|
|
593
|
+
Health that changes at runtime belongs at *invocation* instead. Let the call
|
|
594
|
+
fail, and return the reason:
|
|
595
|
+
|
|
596
|
+
```ts
|
|
597
|
+
function integrationPlugin(adapter: Adapter, conn: Connection): ToolPlugin<Ctx> {
|
|
598
|
+
return {
|
|
599
|
+
name: adapter.id, // "calendar", "issues", "docs", …
|
|
600
|
+
description: adapter.catalogLine, // one line; it lands in the prompt catalog
|
|
601
|
+
isAvailable: () => adapter.installed, // constant for this process
|
|
602
|
+
tools: adapter.tools.filter((t) => conn.grants.has(t.capability)),
|
|
603
|
+
execute: async (toolName, args, ctx) => {
|
|
604
|
+
if (conn.status !== "healthy") {
|
|
605
|
+
// The model can retry, or route around it — a dropped catalog entry
|
|
606
|
+
// tells it nothing.
|
|
607
|
+
return { content: `${adapter.id} is unreachable right now.`, isError: true };
|
|
608
|
+
}
|
|
609
|
+
return adapter.invoke(toolName, args, ctx);
|
|
610
|
+
},
|
|
611
|
+
};
|
|
612
|
+
}
|
|
613
|
+
|
|
614
|
+
for (const conn of connections) {
|
|
615
|
+
registry.register(integrationPlugin(adapterFor(conn), conn));
|
|
616
|
+
}
|
|
617
|
+
```
|
|
618
|
+
|
|
619
|
+
For a plugin whose *connection* is established per run rather than per process,
|
|
620
|
+
restoring a persisted activation is where the reconnect belongs — see
|
|
621
|
+
[How to restore a persisted activation set](#how-to-restore-a-persisted-activation-set).
|
|
622
|
+
|
|
623
|
+
Splitting one large integration into several plugins (`mail`, `calendar`,
|
|
624
|
+
`files`) is usually worth it: the unit of activation should match the unit of
|
|
625
|
+
intent, and a 20-tool plugin is a large thing to load for one call.
|
|
626
|
+
|
|
627
|
+
### A per-request registry on an edge runtime
|
|
628
|
+
|
|
629
|
+
Module-level state is per-isolate and outlives a single request unpredictably.
|
|
630
|
+
Build the registry inside the request instead — the factory is cheap.
|
|
631
|
+
|
|
632
|
+
```ts
|
|
633
|
+
export default {
|
|
634
|
+
async fetch(request: Request, env: Env) {
|
|
635
|
+
const registry = createToolRegistry<MyPlugin>({
|
|
636
|
+
corePlugins: ["core", "discovery"],
|
|
637
|
+
});
|
|
638
|
+
for (const plugin of await loadPluginsFor(env, request)) {
|
|
639
|
+
registry.register(plugin);
|
|
640
|
+
}
|
|
641
|
+
return handle(request, registry);
|
|
642
|
+
},
|
|
643
|
+
};
|
|
644
|
+
```
|
|
645
|
+
|
|
646
|
+
### A single-provider transport
|
|
647
|
+
|
|
648
|
+
You do not need multiple providers to benefit from the plan/execute split — one
|
|
649
|
+
transport still gets you the failure taxonomy, the retry budget, and the
|
|
650
|
+
breaker.
|
|
651
|
+
|
|
652
|
+
```ts
|
|
653
|
+
const transport: PlannerTransport = {
|
|
654
|
+
id: providerIdSchema.parse("openrouter"),
|
|
655
|
+
getAvailability: () =>
|
|
656
|
+
env.API_KEY ? { available: true } : { available: false, reason: "no key" },
|
|
657
|
+
resolveCandidate: (model, requirements) => {
|
|
658
|
+
if (!supports(model, requirements.capabilities)) {
|
|
659
|
+
return { kind: "skip", skip: { canonicalModelId: model, providerId: id, reason: "capability_mismatch" } };
|
|
660
|
+
}
|
|
661
|
+
return { kind: "candidate", candidate: candidateFor(model) };
|
|
662
|
+
},
|
|
663
|
+
};
|
|
664
|
+
```
|
|
665
|
+
|
|
666
|
+
### Replacing a hand-rolled retry/fallback loop
|
|
667
|
+
|
|
668
|
+
A typical hand-rolled loop is `for (model) for (attempt)` with an ad-hoc
|
|
669
|
+
`retry | fallback | terminal` decision. Move the decision to the taxonomy: your
|
|
670
|
+
transport reports **facts**, the executor decides order.
|
|
671
|
+
|
|
672
|
+
```ts
|
|
673
|
+
// Before: the transport decided what to do next.
|
|
674
|
+
// if (status === 429 || status >= 500) retry();
|
|
675
|
+
// else if (status === 401) throw;
|
|
676
|
+
// else tryNextModel();
|
|
677
|
+
|
|
678
|
+
// After: the transport only classifies.
|
|
679
|
+
const attempt: AttemptFn<Completion> = async (candidate, cursor) => {
|
|
680
|
+
const startedAt = performance.now();
|
|
681
|
+
try {
|
|
682
|
+
return { kind: "success", value: await callProvider(candidate) };
|
|
683
|
+
} catch (cause) {
|
|
684
|
+
const target = {
|
|
685
|
+
cursor,
|
|
686
|
+
providerId: candidate.providerId,
|
|
687
|
+
canonicalModelId: candidate.canonicalModelId,
|
|
688
|
+
providerInvocationModel: candidate.providerInvocationModel,
|
|
689
|
+
durationMs: Math.round(performance.now() - startedAt),
|
|
690
|
+
};
|
|
691
|
+
if (isAbort(cause)) return { kind: "failure", error: { kind: "aborted", target, cause } };
|
|
692
|
+
if (!cause.status) return { kind: "failure", error: { kind: "network", target, cause } };
|
|
693
|
+
return {
|
|
694
|
+
kind: "failure",
|
|
695
|
+
error: {
|
|
696
|
+
kind: "http",
|
|
697
|
+
category: categorizeHttpStatus(cause.status),
|
|
698
|
+
statusCode: cause.status,
|
|
699
|
+
retryAfterMs: parseRetryAfter(cause),
|
|
700
|
+
target,
|
|
701
|
+
cause,
|
|
702
|
+
},
|
|
703
|
+
};
|
|
704
|
+
}
|
|
705
|
+
};
|
|
706
|
+
```
|
|
707
|
+
|
|
708
|
+
Two behaviours you get for free and probably did not have: a request-shaped
|
|
709
|
+
rejection (a 400, a moderation refusal) traverses to another provider **without**
|
|
710
|
+
opening the breaker, and an empty or truncated completion is retried on the same
|
|
711
|
+
endpoint before any fallback.
|
|
712
|
+
|
|
713
|
+
### Adapting a non-SDK client to `TurnFn`
|
|
714
|
+
|
|
715
|
+
`TurnFn` is the seam between a loop and any client. If you hand-roll SSE, adapt
|
|
716
|
+
at this boundary and the rest of the harness does not care.
|
|
717
|
+
|
|
718
|
+
```ts
|
|
719
|
+
const turn: TurnFn = async (messages, tools, signal) => {
|
|
720
|
+
const startedAt = performance.now();
|
|
721
|
+
let ttftMs: number | null = null;
|
|
722
|
+
|
|
723
|
+
const { message, usage } = await streamCompletion({
|
|
724
|
+
messages, tools, signal,
|
|
725
|
+
onFirstToken: () => { ttftMs ??= Math.round(performance.now() - startedAt); },
|
|
726
|
+
});
|
|
727
|
+
|
|
728
|
+
return {
|
|
729
|
+
message,
|
|
730
|
+
usage: {
|
|
731
|
+
inputTokens: usage.prompt_tokens ?? 0,
|
|
732
|
+
outputTokens: usage.completion_tokens ?? 0,
|
|
733
|
+
cachedInputTokens: usage.cached_tokens ?? null,
|
|
734
|
+
costCents: usage.cost != null ? usage.cost * 100 : null,
|
|
735
|
+
},
|
|
736
|
+
timings: { ttftMs, generationMs: Math.round(performance.now() - startedAt) },
|
|
737
|
+
};
|
|
738
|
+
};
|
|
739
|
+
```
|
|
740
|
+
|
|
741
|
+
### Accumulating run statistics
|
|
742
|
+
|
|
743
|
+
Fold each turn and each tool call as they complete; `RunStats` keeps model time
|
|
744
|
+
and tool time separate so a slow tool never looks like a slow model.
|
|
745
|
+
|
|
746
|
+
```ts
|
|
747
|
+
let stats = emptyRunStats();
|
|
748
|
+
|
|
749
|
+
for (const turn of turns) {
|
|
750
|
+
const result = await turnFn(messages, tools, signal);
|
|
751
|
+
stats = accumulateTurn(stats, result);
|
|
752
|
+
|
|
753
|
+
for (const call of result.message.tool_calls ?? []) {
|
|
754
|
+
const startedAt = performance.now();
|
|
755
|
+
await dispatch(call);
|
|
756
|
+
stats = accumulateToolCall(stats, call.function.name, performance.now() - startedAt);
|
|
757
|
+
}
|
|
758
|
+
}
|
|
759
|
+
|
|
760
|
+
await persistRun({ ...stats, stopReason: "done" satisfies StopReason });
|
|
761
|
+
```
|
|
762
|
+
|
|
763
|
+
### Batching a turn's tool calls
|
|
764
|
+
|
|
765
|
+
Reads can overlap; writes usually should not. Pool the reads and keep the
|
|
766
|
+
transcript in the model's original call order.
|
|
767
|
+
|
|
768
|
+
```ts
|
|
769
|
+
const reads = calls.filter((c) => isReadOnly(c));
|
|
770
|
+
const writes = calls.filter((c) => !isReadOnly(c));
|
|
771
|
+
|
|
772
|
+
const readResults = await runToolCallsPooledByTool(reads, (call) => dispatch(call));
|
|
773
|
+
|
|
774
|
+
const writeResults: Outcome[] = [];
|
|
775
|
+
for (const call of writes) writeResults.push(await dispatch(call));
|
|
776
|
+
|
|
777
|
+
// Re-emit in the order the model asked for, so every tool_call_id is answered.
|
|
778
|
+
for (const call of calls) {
|
|
779
|
+
messages.push(toolMessageFor(call, resultFor(call, readResults, writeResults)));
|
|
780
|
+
}
|
|
781
|
+
```
|
|
782
|
+
|
|
783
|
+
`runToolCallsPooledByTool` returns `PromiseSettledResult`s — a rejection still
|
|
784
|
+
needs a synthesized error tool message, or the next request has an unanswered
|
|
785
|
+
call.
|
|
786
|
+
|
|
787
|
+
### Pausing a run for human input
|
|
788
|
+
|
|
789
|
+
A tool that needs an answer returns a `suspend` directive on a **successful**
|
|
790
|
+
result. Pick the resume kind by whether the answer becomes the tool's result.
|
|
791
|
+
|
|
792
|
+
```ts
|
|
793
|
+
// The run resumes later via a prompt; the tool message is kept.
|
|
794
|
+
return {
|
|
795
|
+
success: true,
|
|
796
|
+
data: { askedAt: nowIso },
|
|
797
|
+
suspend: { reason: "awaiting user selection", resumeKind: "wake", request: formSpec },
|
|
798
|
+
};
|
|
799
|
+
|
|
800
|
+
// The human's reply IS this call's tool result; the loop withholds the message.
|
|
801
|
+
return {
|
|
802
|
+
success: true,
|
|
803
|
+
data: null,
|
|
804
|
+
suspend: { reason: "awaiting answer", resumeKind: "answer", request: formSpec },
|
|
805
|
+
};
|
|
806
|
+
```
|
|
807
|
+
|
|
808
|
+
If your runtime ends the whole run and starts a fresh one on reply, `wake` is
|
|
809
|
+
the kind you want — `answer` only pays off when you thread the reply back into
|
|
810
|
+
the same transcript as the matching `role:"tool"` message.
|
|
811
|
+
|
|
812
|
+
### Carrying disclosure across a run boundary
|
|
813
|
+
|
|
814
|
+
When a run ends and a later one continues the same conversation, persist the
|
|
815
|
+
**names** and re-validate on the way back in.
|
|
816
|
+
|
|
817
|
+
```ts
|
|
818
|
+
// End of run: names only, never plugin objects.
|
|
819
|
+
await db.saveActivation(conversationId, [...activePlugins]);
|
|
820
|
+
|
|
821
|
+
// Start of the next run.
|
|
822
|
+
const persisted = await db.loadActivation(conversationId);
|
|
823
|
+
const { active, dropped } = await rehydrateActivation(persisted, {
|
|
824
|
+
canonicalizeName: (n) => registry.canonicalizeName(n),
|
|
825
|
+
resolve: async (n) => (registry.get(n) ? true : "unknown"),
|
|
826
|
+
});
|
|
827
|
+
// Seed core, then add what survived — core plugins load unconditionally.
|
|
828
|
+
const activePlugins = new Set([...initialActivePlugins(registry.corePlugins()), ...active]);
|
|
829
|
+
for (const drop of dropped) log.info("plugin not restored", drop);
|
|
830
|
+
```
|
|
831
|
+
|
|
832
|
+
Without this, every boundary silently resets the agent to core-only and it
|
|
833
|
+
re-discovers from scratch — which costs a round-trip per resumption.
|
|
834
|
+
|
|
835
|
+
### Assembling a bounded run
|
|
836
|
+
|
|
837
|
+
The pieces compose; nothing here knows about the others.
|
|
838
|
+
|
|
839
|
+
```ts
|
|
840
|
+
const deadline = createRunDeadline({ timeoutMs: RUN_BUDGET_MS, label: "run" });
|
|
841
|
+
const heartbeat = createCoalescedHeartbeat({
|
|
842
|
+
coalesceMs: 10_000,
|
|
843
|
+
flush: () => db.touchRun(runId),
|
|
844
|
+
});
|
|
845
|
+
let stats = emptyRunStats();
|
|
846
|
+
let stopReason: StopReason = "done";
|
|
847
|
+
|
|
848
|
+
try {
|
|
849
|
+
for (let i = 0; i < maxIterations; i++) {
|
|
850
|
+
deadline.throwIfTimedOut();
|
|
851
|
+
const { messages: healed } = validateAndHealMessages(transcript);
|
|
852
|
+
const result = await turnFn(healed, wireTools, deadline.withExternal(cancelSignal));
|
|
853
|
+
stats = accumulateTurn(stats, result);
|
|
854
|
+
await heartbeat.beat();
|
|
855
|
+
if (!result.message.tool_calls?.length) break;
|
|
856
|
+
if (await runToolBatch(result.message.tool_calls)) { stopReason = "suspended"; break; }
|
|
857
|
+
}
|
|
858
|
+
} catch (error) {
|
|
859
|
+
stopReason = classifyRunFailure(deadline, error) === "timed_out" ? "deadline" : "aborted";
|
|
860
|
+
} finally {
|
|
861
|
+
await heartbeat.beat({ force: true });
|
|
862
|
+
deadline.dispose();
|
|
863
|
+
}
|
|
864
|
+
```
|
|
865
|
+
|
|
866
|
+
### Spawning child runs (sub-agents)
|
|
867
|
+
|
|
868
|
+
> **Not in the harness yet.** There is no sub-agent, spawn, or child-run concept
|
|
869
|
+
> in `bind` today — the vocabulary below is yours to define. The admission
|
|
870
|
+
> policy is a planned addition; see [Roadmap](#roadmap).
|
|
871
|
+
|
|
872
|
+
A sub-agent is not a special kind of thing. It is **a run that another run
|
|
873
|
+
asked for**, so everything the harness already gives a run applies unchanged:
|
|
874
|
+
its own deadline, its own heartbeat, its own `RunStats`, its own route plan.
|
|
875
|
+
Only three things are genuinely new, and all three are currently yours.
|
|
876
|
+
|
|
877
|
+
**1. The spawn tool is an ordinary plugin.** Nothing special is needed here —
|
|
878
|
+
the tool vocabulary is already shared.
|
|
879
|
+
|
|
880
|
+
```ts
|
|
881
|
+
const orchestration: ToolPlugin<Ctx> = {
|
|
882
|
+
name: "orchestration",
|
|
883
|
+
description: "Delegate focused work to a child run.",
|
|
884
|
+
icon: "share",
|
|
885
|
+
tools: [
|
|
886
|
+
{
|
|
887
|
+
name: "spawn",
|
|
888
|
+
description: "Run one focused objective as a child and return its id.",
|
|
889
|
+
parameters: spawnArgsSchema, // z.object({ objective: z.string(), plugins: z.array(z.string()) })
|
|
890
|
+
},
|
|
891
|
+
],
|
|
892
|
+
async execute(toolName, args, ctx) {
|
|
893
|
+
const { objective, plugins } = spawnArgsSchema.parse(args);
|
|
894
|
+
|
|
895
|
+
const admission = await admitChildRun(
|
|
896
|
+
{ depth: ctx.chainDepth, chainId: ctx.chainId, parentKey: ctx.runKey },
|
|
897
|
+
BOUNDS,
|
|
898
|
+
);
|
|
899
|
+
if (!admission.admitted) {
|
|
900
|
+
return { success: false, kind: "validation", error: admission.reason };
|
|
901
|
+
}
|
|
902
|
+
|
|
903
|
+
const childRunId = await queue.enqueueRun({
|
|
904
|
+
objective,
|
|
905
|
+
plugins,
|
|
906
|
+
chainId: ctx.chainId,
|
|
907
|
+
depth: ctx.chainDepth + 1,
|
|
908
|
+
parentRunId: ctx.runId,
|
|
909
|
+
});
|
|
910
|
+
return { success: true, data: { childRunId } };
|
|
911
|
+
},
|
|
912
|
+
};
|
|
913
|
+
```
|
|
914
|
+
|
|
915
|
+
**2. Admission is the part worth getting right.** An agent that can spawn can
|
|
916
|
+
spawn agents that spawn. Bound it *before* enqueuing, not after — an unbounded
|
|
917
|
+
chain is a runaway spend, and the failure mode is silent. The counting needs
|
|
918
|
+
your database; the deciding does not.
|
|
919
|
+
|
|
920
|
+
```ts
|
|
921
|
+
type Admission = { admitted: true } | { admitted: false; reason: string };
|
|
922
|
+
|
|
923
|
+
async function admitChildRun(chain: Chain, bounds: Bounds): Promise<Admission> {
|
|
924
|
+
if (chain.depth >= bounds.maxDepth) {
|
|
925
|
+
return { admitted: false, reason: `chain depth limit reached (${chain.depth}/${bounds.maxDepth})` };
|
|
926
|
+
}
|
|
927
|
+
if (await countRunsInChain(chain.chainId) >= bounds.maxRunsPerChain) {
|
|
928
|
+
return { admitted: false, reason: "chain run budget exhausted" };
|
|
929
|
+
}
|
|
930
|
+
if (Date.now() - (await lastSpawnBetween(chain.parentKey)) < bounds.pairCooldownMs) {
|
|
931
|
+
return { admitted: false, reason: "spawn cooldown active" };
|
|
932
|
+
}
|
|
933
|
+
return { admitted: true };
|
|
934
|
+
}
|
|
935
|
+
```
|
|
936
|
+
|
|
937
|
+
Pick bounds against your own cost model. Depth caps runaway recursion; a
|
|
938
|
+
per-chain run budget caps total spend once a chain starts; a pair cooldown stops
|
|
939
|
+
two agents ping-ponging. A separate per-tenant ceiling on *directly triggered*
|
|
940
|
+
runs is worth having too — depth limits do not constrain someone starting a
|
|
941
|
+
thousand independent runs.
|
|
942
|
+
|
|
943
|
+
**3. Waiting, and rolling results up.** Two shapes work. Suspend the parent and
|
|
944
|
+
let child completion wake it, which frees the worker slot:
|
|
945
|
+
|
|
946
|
+
```ts
|
|
947
|
+
return {
|
|
948
|
+
success: true,
|
|
949
|
+
data: { childRunIds },
|
|
950
|
+
suspend: { reason: "awaiting child runs", resumeKind: "wake", request: { childRunIds } },
|
|
951
|
+
};
|
|
952
|
+
```
|
|
953
|
+
|
|
954
|
+
…or poll inside the tool, which keeps the parent's transcript intact but holds
|
|
955
|
+
its slot — so cap the poll well under the parent's deadline.
|
|
956
|
+
|
|
957
|
+
For accounting, note that `accumulateTurn` and `accumulateToolCall` fold a
|
|
958
|
+
run's *own* activity; there is no built-in roll-up across child runs. Sum them
|
|
959
|
+
yourself if a chain's total cost is a number you report:
|
|
960
|
+
|
|
961
|
+
```ts
|
|
962
|
+
const chainTotals = [parentStats, ...childStats].reduce((acc, s) => ({
|
|
963
|
+
...acc,
|
|
964
|
+
turns: acc.turns + s.turns,
|
|
965
|
+
toolCalls: acc.toolCalls + s.toolCalls,
|
|
966
|
+
inputTokens: acc.inputTokens + s.inputTokens,
|
|
967
|
+
outputTokens: acc.outputTokens + s.outputTokens,
|
|
968
|
+
costCents: acc.costCents + s.costCents,
|
|
969
|
+
modelTimeMs: acc.modelTimeMs + s.modelTimeMs,
|
|
970
|
+
toolTimeMs: acc.toolTimeMs + s.toolTimeMs,
|
|
971
|
+
}), emptyRunStats());
|
|
972
|
+
```
|
|
973
|
+
|
|
974
|
+
Model time will exceed wall-clock once children run in parallel — that is
|
|
975
|
+
correct, and it is why the two are tracked separately.
|
|
976
|
+
|
|
977
|
+
---
|
|
978
|
+
|
|
979
|
+
## Roadmap
|
|
980
|
+
|
|
981
|
+
Named, not scheduled. Listed so a consumer can tell a deliberate omission from
|
|
982
|
+
an oversight.
|
|
983
|
+
|
|
984
|
+
- **The turn kernel** — `runAgent` / `advanceTurn` over `TurnFn`. The loop
|
|
985
|
+
itself is still yours.
|
|
986
|
+
- **Sub-agent admission** — the depth / chain-budget / cooldown policy
|
|
987
|
+
sketched [above](#spawning-child-runs-sub-agents), as a pure decision over
|
|
988
|
+
injected counts. The queue, the persistence, and the bounds stay yours.
|
|
989
|
+
- **Child-run stats roll-up** — an `accumulateRun` fold so a chain's totals do
|
|
990
|
+
not have to be summed by hand.
|
|
991
|
+
|
|
992
|
+
---
|
|
993
|
+
|
|
994
|
+
## Rules for automated contributors
|
|
995
|
+
|
|
996
|
+
*Invariants of this package. Read this before changing anything under `src/`;
|
|
997
|
+
each is enforced by lint, typecheck, or CI where it is developed.*
|
|
998
|
+
|
|
999
|
+
1. **Never import `@/*`, a Node builtin, `process`, or a framework.** Take a
|
|
1000
|
+
port instead. An ESLint block scoped to `packages/bind/**` enforces this.
|
|
1001
|
+
2. **Never add a runtime dependency.** New third-party code must be a peer
|
|
1002
|
+
dependency, and only with a strong reason. `openai` is type-only.
|
|
1003
|
+
3. **Keep every module I/O-free.** If a change needs a clock, a random source,
|
|
1004
|
+
the environment, or a network call, take it as an explicit input. A
|
|
1005
|
+
`Date.now` default on an injectable clock is the only sanctioned exception.
|
|
1006
|
+
4. **Stay deterministic.** Identical inputs must produce identical plans.
|
|
1007
|
+
Nothing about registration order, map iteration, or wall-clock time may
|
|
1008
|
+
reorder candidates.
|
|
1009
|
+
5. **Do not add module-level mutable state.** Export a factory. Module state is
|
|
1010
|
+
per-isolate on edge runtimes and leaks between tests.
|
|
1011
|
+
6. **Do not collapse a generic into a concrete host type.** `TCtx` and
|
|
1012
|
+
`TContentPart` are parameters because hosts genuinely diverge there; binding
|
|
1013
|
+
them to one application's types would fork the package.
|
|
1014
|
+
7. **Version the policy, do not mutate it.** If routing order or meaning
|
|
1015
|
+
changes, bump `ROUTE_POLICY_VERSION`.
|
|
1016
|
+
8. **Put tests in `src/**/__tests__/`.** They run under bare `bun test` with no
|
|
1017
|
+
DOM and no database. Anything needing either belongs in the host.
|
|
1018
|
+
9. **Do not edit `version` in `package.json`.** It is a placeholder; the
|
|
1019
|
+
published version is stamped at release time.
|
|
1020
|
+
|
|
1021
|
+
---
|
|
1022
|
+
|
|
1023
|
+
## Versioning
|
|
1024
|
+
|
|
1025
|
+
**Versioning is not semver.** Each published release increments the major and
|
|
1026
|
+
resets the rest — `1.0.0`, `2.0.0`, `3.0.0` — so the major is a release counter,
|
|
1027
|
+
not a compatibility signal, and a bump does not by itself mean the surface
|
|
1028
|
+
changed. Pin an exact version and read the changes between releases until this
|
|
1029
|
+
stabilizes.
|
|
83
1030
|
|
|
84
1031
|
## Development
|
|
85
1032
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
releases
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
npm release has. Issues are welcome here; code changes land in the canonical
|
|
92
|
-
repo and flow out with the next export.
|
|
1033
|
+
The source lives in Monad's canonical repository. `juno-ai-labs/agent-harness`
|
|
1034
|
+
on GitHub is a read-only **archive mirror** of it, exported via Copybara, and
|
|
1035
|
+
npm releases are published from the canonical repository rather than from either
|
|
1036
|
+
mirror. Issues are welcome on the GitHub mirror; code changes land in the
|
|
1037
|
+
canonical repo and flow out with the next export.
|
|
93
1038
|
|
|
94
1039
|
Run the tests with [Bun](https://bun.sh):
|
|
95
1040
|
|