@superdurable/dex-agents 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +598 -45
- package/dist/src/agent-context-compaction.d.ts +1 -1
- package/dist/src/agent-framework-runner.d.ts +210 -10
- package/dist/src/agent-framework-runner.js +30 -0
- package/dist/src/agent-framework-runner.js.map +1 -1
- package/dist/src/agent-local-directory-media-store.d.ts +111 -0
- package/dist/src/agent-local-directory-media-store.js +417 -0
- package/dist/src/agent-local-directory-media-store.js.map +1 -0
- package/dist/src/agent-media.d.ts +279 -0
- package/dist/src/agent-media.js +457 -0
- package/dist/src/agent-media.js.map +1 -0
- package/dist/src/agent-session-codecs.d.ts +3 -1
- package/dist/src/agent-session-codecs.js +53 -4
- package/dist/src/agent-session-codecs.js.map +1 -1
- package/dist/src/agent-session-failure.d.ts +4 -2
- package/dist/src/agent-session-failure.js +6 -2
- package/dist/src/agent-session-failure.js.map +1 -1
- package/dist/src/agent-session-flow-graph.d.ts +46 -0
- package/dist/src/agent-session-flow-graph.js +51 -0
- package/dist/src/agent-session-flow-graph.js.map +1 -1
- package/dist/src/agent-session-flow-graph.json +98 -81
- package/dist/src/agent-session-flow.d.ts +5 -3
- package/dist/src/agent-session-flow.js +10 -1
- package/dist/src/agent-session-flow.js.map +1 -1
- package/dist/src/agent-session-start.d.ts +1 -1
- package/dist/src/agent-session-start.js +2 -2
- package/dist/src/agent-session-start.js.map +1 -1
- package/dist/src/agent-session-steps.d.ts +9 -0
- package/dist/src/agent-session-steps.js +219 -34
- package/dist/src/agent-session-steps.js.map +1 -1
- package/dist/src/agent-session-tracer.js +1 -1
- package/dist/src/agent-session-tracer.js.map +1 -1
- package/dist/src/agent-session-transitions.d.ts +4 -0
- package/dist/src/agent-session-transitions.js +29 -1
- package/dist/src/agent-session-transitions.js.map +1 -1
- package/dist/src/agent-session-values.d.ts +112 -1
- package/dist/src/agent-session-values.js +6 -0
- package/dist/src/agent-session-values.js.map +1 -1
- package/dist/src/agent-session-wait-steps.js +17 -5
- package/dist/src/agent-session-wait-steps.js.map +1 -1
- package/dist/src/ai-sdk/ai-sdk-model-client.d.ts +15 -3
- package/dist/src/ai-sdk/ai-sdk-model-client.js +36 -7
- package/dist/src/ai-sdk/ai-sdk-model-client.js.map +1 -1
- package/dist/src/ai-sdk/index.d.ts +1 -1
- package/dist/src/ai-sdk/index.js +1 -1
- package/dist/src/ai-sdk/index.js.map +1 -1
- package/dist/src/{stepwise → builtin-loop}/agent-definition.d.ts +120 -5
- package/dist/src/builtin-loop/agent-definition.js.map +1 -0
- package/dist/src/builtin-loop/builtin-agent-session-flow.d.ts +69 -0
- package/dist/src/{stepwise/stepwise-agent-session-flow.js → builtin-loop/builtin-agent-session-flow.js} +15 -9
- package/dist/src/builtin-loop/builtin-agent-session-flow.js.map +1 -0
- package/dist/src/{stepwise/stepwise-agent-runner.d.ts → builtin-loop/builtin-loop-agent-runner.d.ts} +85 -26
- package/dist/src/{stepwise/stepwise-agent-runner.js → builtin-loop/builtin-loop-agent-runner.js} +320 -213
- package/dist/src/builtin-loop/builtin-loop-agent-runner.js.map +1 -0
- package/dist/src/{stepwise/stepwise-input-items.d.ts → builtin-loop/builtin-loop-input-items.d.ts} +7 -3
- package/dist/src/{stepwise/stepwise-input-items.js → builtin-loop/builtin-loop-input-items.js} +57 -10
- package/dist/src/builtin-loop/builtin-loop-input-items.js.map +1 -0
- package/dist/src/builtin-loop/function-tool.d.ts +282 -0
- package/dist/src/{stepwise → builtin-loop}/function-tool.js +63 -3
- package/dist/src/builtin-loop/function-tool.js.map +1 -0
- package/dist/src/builtin-loop/index.d.ts +8 -0
- package/dist/src/builtin-loop/index.js +13 -0
- package/dist/src/builtin-loop/index.js.map +1 -0
- package/dist/src/builtin-loop/json-schema-validation.js.map +1 -0
- package/dist/src/{stepwise → builtin-loop}/model-client.d.ts +36 -7
- package/dist/src/builtin-loop/model-client.js.map +1 -0
- package/dist/src/{stepwise → builtin-loop}/scripted-model-client.d.ts +8 -5
- package/dist/src/{stepwise → builtin-loop}/scripted-model-client.js +8 -5
- package/dist/src/builtin-loop/scripted-model-client.js.map +1 -0
- package/dist/src/{stepwise → builtin-loop}/tool-source.d.ts +13 -3
- package/dist/src/builtin-loop/tool-source.js.map +1 -0
- package/dist/src/connectors/connector-input-schema.d.ts +19 -0
- package/dist/src/connectors/connector-input-schema.js +228 -0
- package/dist/src/connectors/connector-input-schema.js.map +1 -0
- package/dist/src/connectors/connector-step-context.d.ts +94 -0
- package/dist/src/connectors/connector-step-context.js +180 -0
- package/dist/src/connectors/connector-step-context.js.map +1 -0
- package/dist/src/connectors/connector-tool-source.d.ts +401 -0
- package/dist/src/connectors/connector-tool-source.js +628 -0
- package/dist/src/connectors/connector-tool-source.js.map +1 -0
- package/dist/src/connectors/index.d.ts +3 -0
- package/dist/src/connectors/index.js +11 -0
- package/dist/src/connectors/index.js.map +1 -0
- package/dist/src/index.d.ts +7 -3
- package/dist/src/index.js +6 -3
- package/dist/src/index.js.map +1 -1
- package/dist/src/mcp/index.d.ts +1 -0
- package/dist/src/mcp/index.js +1 -0
- package/dist/src/mcp/index.js.map +1 -1
- package/dist/src/mcp/mcp-dex-conventions.d.ts +57 -0
- package/dist/src/mcp/mcp-dex-conventions.js +90 -0
- package/dist/src/mcp/mcp-dex-conventions.js.map +1 -0
- package/dist/src/mcp/mcp-image-content.d.ts +20 -0
- package/dist/src/mcp/mcp-image-content.js +82 -0
- package/dist/src/mcp/mcp-image-content.js.map +1 -0
- package/dist/src/mcp/mcp-tool-source.d.ts +178 -17
- package/dist/src/mcp/mcp-tool-source.js +237 -34
- package/dist/src/mcp/mcp-tool-source.js.map +1 -1
- package/dist/src/s3/index.d.ts +1 -0
- package/dist/src/s3/index.js +9 -0
- package/dist/src/s3/index.js.map +1 -0
- package/dist/src/s3/s3-media-store.d.ts +138 -0
- package/dist/src/s3/s3-media-store.js +255 -0
- package/dist/src/s3/s3-media-store.js.map +1 -0
- package/dist/src/stepwise/index.d.ts +41 -8
- package/dist/src/stepwise/index.js +35 -5
- package/dist/src/stepwise/index.js.map +1 -1
- package/dist/src/testing/agent-session-test-client.d.ts +6 -3
- package/dist/src/testing/agent-session-test-client.js +10 -3
- package/dist/src/testing/agent-session-test-client.js.map +1 -1
- package/dist/src/testing/scripted-model-client.d.ts +1 -1
- package/dist/src/testing/scripted-model-client.js +2 -2
- package/dist/src/testing/scripted-model-client.js.map +1 -1
- package/dist/src/tool-source-router.d.ts +211 -0
- package/dist/src/tool-source-router.js +380 -0
- package/dist/src/tool-source-router.js.map +1 -0
- package/dist/src/ui-stream/agent-event-json.d.ts +2 -1
- package/dist/src/ui-stream/agent-event-json.js +3 -1
- package/dist/src/ui-stream/agent-event-json.js.map +1 -1
- package/dist/src/ui-stream/agent-ui-messages.d.ts +11 -3
- package/dist/src/ui-stream/agent-ui-messages.js +29 -5
- package/dist/src/ui-stream/agent-ui-messages.js.map +1 -1
- package/package.json +25 -3
- package/dist/src/stepwise/agent-definition.js.map +0 -1
- package/dist/src/stepwise/function-tool.d.ts +0 -130
- package/dist/src/stepwise/function-tool.js.map +0 -1
- package/dist/src/stepwise/json-schema-validation.js.map +0 -1
- package/dist/src/stepwise/model-client.js.map +0 -1
- package/dist/src/stepwise/scripted-model-client.js.map +0 -1
- package/dist/src/stepwise/stepwise-agent-runner.js.map +0 -1
- package/dist/src/stepwise/stepwise-agent-session-flow.d.ts +0 -35
- package/dist/src/stepwise/stepwise-agent-session-flow.js.map +0 -1
- package/dist/src/stepwise/stepwise-input-items.js.map +0 -1
- package/dist/src/stepwise/tool-source.js.map +0 -1
- /package/dist/src/{stepwise → builtin-loop}/agent-definition.js +0 -0
- /package/dist/src/{stepwise → builtin-loop}/json-schema-validation.d.ts +0 -0
- /package/dist/src/{stepwise → builtin-loop}/json-schema-validation.js +0 -0
- /package/dist/src/{stepwise → builtin-loop}/model-client.js +0 -0
- /package/dist/src/{stepwise → builtin-loop}/tool-source.js +0 -0
package/README.md
CHANGED
|
@@ -1,15 +1,15 @@
|
|
|
1
1
|
# @superdurable/dex-agents
|
|
2
2
|
|
|
3
3
|
Durable agent sessions for Dex, shared by the TypeScript agent framework
|
|
4
|
-
adapters. One Flow
|
|
4
|
+
adapters. One Flow run is one agent session, and the Flow ID is the
|
|
5
5
|
session ID. An adapter package supplies a framework runner; this package owns
|
|
6
6
|
the Steps, persistence, RPCs, live events, tool approval, and crash recovery.
|
|
7
7
|
It runs in Step mode: each model call and each tool call is its own Dex Step,
|
|
8
8
|
and the tool calls of one model response run in parallel.
|
|
9
9
|
|
|
10
|
-
The package also ships the built-in agent loop
|
|
11
|
-
defined in Dex terms without a framework (`@superdurable/dex-agents/
|
|
12
|
-
and its AI SDK model client (`@superdurable/dex-agents/ai-sdk`).
|
|
10
|
+
The package also ships the built-in agent loop for agents
|
|
11
|
+
defined in Dex terms without a framework (`@superdurable/dex-agents/builtin-loop`),
|
|
12
|
+
and its AI SDK model client (`@superdurable/dex-agents/ai-sdk`). The built-in loop runs
|
|
13
13
|
on the same session Flow as every adapter, with the same Steps, RPCs, and
|
|
14
14
|
events. The session also handles two built-in tools under any runner:
|
|
15
15
|
`ask_user` asks the user a question and waits in `AwaitQuestionAnswer`, and
|
|
@@ -27,10 +27,10 @@ Applications usually install an adapter next to this package. To build a
|
|
|
27
27
|
runner of your own, install the core with the Dex SDK it supports:
|
|
28
28
|
|
|
29
29
|
```bash
|
|
30
|
-
npm i @superdurable/dex-agents @superdurable/dex@1.
|
|
30
|
+
npm i @superdurable/dex-agents @superdurable/dex@1.45.0
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
`@superdurable/dex` 1.
|
|
33
|
+
`@superdurable/dex` 1.45.0 is a peer dependency, so the application, the core,
|
|
34
34
|
and every adapter share one copy. The Dex Registry matches Flow classes and
|
|
35
35
|
RPC registrations by identity, so a second copy makes it reject the Flow
|
|
36
36
|
(`steps must be a StepList`). Adapters list this package as a peer dependency
|
|
@@ -49,11 +49,13 @@ the published core. It is for tests only; it needs `DEXCLI` and
|
|
|
49
49
|
| Subpath | Contents | Extra dependency |
|
|
50
50
|
| --- | --- | --- |
|
|
51
51
|
| `@superdurable/dex-agents` | Session Flow, runner contract, built-in tool definitions | none |
|
|
52
|
-
| `@superdurable/dex-agents/
|
|
52
|
+
| `@superdurable/dex-agents/builtin-loop` | The built-in agent loop: `createBuiltinAgentSessionFlow`, `functionTool`, `ToolSource`, `ModelClient`, `ScriptedModelClient` | none |
|
|
53
53
|
| `@superdurable/dex-agents/ai-sdk` | `createAiSdkModelClient` for any AI SDK `LanguageModel` | optional peer `ai@^7.0.159`, the Vercel AI SDK adapter's range |
|
|
54
54
|
| `@superdurable/dex-agents/mcp` | MCP servers' tools as agent tools; see [MCP tools](#mcp-tools-superdurabledex-agentsmcp) | optional peer `@modelcontextprotocol/sdk@^1.32.2` |
|
|
55
|
+
| `@superdurable/dex-agents/connectors` | Reviewed Connector operations as agent tools; see [Connector tools](#connector-tools-superdurabledex-agentsconnectors) | optional peer `@superdurable/connectors-sdk` |
|
|
55
56
|
| `@superdurable/dex-agents/ui-stream` | Browser streams of live sessions; see [UI streams](#ui-streams) | none |
|
|
56
|
-
| `@superdurable/dex-agents/
|
|
57
|
+
| `@superdurable/dex-agents/s3` | `S3MediaStore`, image bytes in an S3-compatible bucket; see [Images](#images) | optional peer `@aws-sdk/client-s3@^3.400.0` |
|
|
58
|
+
| `@superdurable/dex-agents/testing` | Test engine, Worker processes, waits, and `ScriptedModelClient` (the same class as the builtin-loop entry point's) | none |
|
|
57
59
|
|
|
58
60
|
Install `ai` and a provider package only for `./ai-sdk`; importing that
|
|
59
61
|
subpath without `ai` fails with `Cannot find package 'ai'`.
|
|
@@ -127,17 +129,17 @@ Enable built-in tools and call `answer_question` only after every Worker of the
|
|
|
127
129
|
Flow type runs 0.2.0, because a 0.1.0 Worker cannot decode the new non-final
|
|
128
130
|
statuses.
|
|
129
131
|
|
|
130
|
-
##
|
|
132
|
+
## Built-in agent loop
|
|
131
133
|
|
|
132
|
-
|
|
133
|
-
and its model client; `
|
|
134
|
-
with a `
|
|
134
|
+
The built-in agent loop runs an agent without a framework. Declare the agent, its function tools,
|
|
135
|
+
and its model client; `createBuiltinAgentSessionFlow` builds the session Flow
|
|
136
|
+
with a `BuiltinLoopAgentRunner` and derives the session options from the
|
|
135
137
|
definition.
|
|
136
138
|
|
|
137
139
|
```ts
|
|
138
140
|
import { openai } from "@ai-sdk/openai";
|
|
139
141
|
import { createAiSdkModelClient } from "@superdurable/dex-agents/ai-sdk";
|
|
140
|
-
import {
|
|
142
|
+
import { createBuiltinAgentSessionFlow, functionTool } from "@superdurable/dex-agents/builtin-loop";
|
|
141
143
|
|
|
142
144
|
const lookupOrder = functionTool<{ order_id: string }>({
|
|
143
145
|
name: "lookup_order",
|
|
@@ -154,7 +156,7 @@ const refundOrder = functionTool<{ order_id: string }>({
|
|
|
154
156
|
execute: async ({ order_id }, context) => payments.refund(order_id, { idempotencyKey: context.idempotencyKey }),
|
|
155
157
|
});
|
|
156
158
|
|
|
157
|
-
export const supportAgentFlow =
|
|
159
|
+
export const supportAgentFlow = createBuiltinAgentSessionFlow(
|
|
158
160
|
{
|
|
159
161
|
name: "support",
|
|
160
162
|
instructions: "You help customers with their orders.",
|
|
@@ -180,6 +182,7 @@ export const supportAgentFlow = createStepwiseAgentSessionFlow(
|
|
|
180
182
|
| `maxModelCallsPerTurn` | 100 | After this many model calls in one turn, the next call offers no tools, so the model must answer. If it still asks for tools, the turn ends and each call gets the result `The tool call was not run: this turn reached its limit of <n> model calls.` |
|
|
181
183
|
| `planModeEnabled` | `false` | Accept `planMode` user messages; requires `write_todos` in `builtInTools`. See [Plans and plan mode](#plans-and-plan-mode) |
|
|
182
184
|
| `interruptDeadlineMs`, `eventStreamCapacityBytes` | as for the session | Passed to the session options |
|
|
185
|
+
| `decideTurnContinuation` | none | A function that ends or continues the turn at the end of each loop iteration; see [Ending or continuing a turn](#ending-or-continuing-a-turn). Since 0.5.0 |
|
|
183
186
|
|
|
184
187
|
Each `RunModel` makes at most one model call, built from committed state only:
|
|
185
188
|
the instructions, the stored messages, the plan note from `buildPlanNote`
|
|
@@ -187,7 +190,8 @@ before the turn's messages when a plan exists or the turn plans, and the
|
|
|
187
190
|
results of the previous model response from the turn's final records. The runner stores the turn's
|
|
188
191
|
transcript as it goes: `ToolCallsRequested.newMessages` carries the previous
|
|
189
192
|
round's tool results and the new model output, so `FrameworkRunState` holds
|
|
190
|
-
only `{"model_calls": <n>}
|
|
193
|
+
only `{"model_calls": <n>}`, plus `"consecutive_continuations": <n>` after a
|
|
194
|
+
continued answer. The input builder pairs every stored tool call
|
|
191
195
|
with a result, and a missing one gets the outcome-unknown text, so an orphaned
|
|
192
196
|
call never reaches a provider.
|
|
193
197
|
|
|
@@ -196,9 +200,19 @@ A function tool declares its effect, which picks `CallTool` or
|
|
|
196
200
|
model's arguments against the tool's JSON Schema before `execute` runs, and a
|
|
197
201
|
mismatch or an unknown tool reaches the model as a tool failure. `execute`
|
|
198
202
|
receives the arguments and a `ToolCallContext` with the idempotency key,
|
|
199
|
-
attempt, events, heartbeat,
|
|
200
|
-
`
|
|
201
|
-
|
|
203
|
+
attempt, events, heartbeat, the answers to the tool's own questions, and
|
|
204
|
+
`effect`: the call's effect after the tool policies, which chose its tool Step.
|
|
205
|
+
A wrapper that turns errors into `ToolExecutionError` lets them propagate for a
|
|
206
|
+
`non_idempotent_write` or `unclassified` effect, and for an absent one, which
|
|
207
|
+
only a hand-built context has. `prepareArguments`, a synchronous and deterministic function, converts the
|
|
208
|
+
decoded arguments before the schema check, such as an argument shape an earlier
|
|
209
|
+
version of the tool took; a throw fails the call like a schema violation, a
|
|
210
|
+
cancellation passes through, and a Promise result is refused. The
|
|
211
|
+
stored tool call and the approval interrupt keep the model's original
|
|
212
|
+
arguments. A
|
|
213
|
+
`string` result is the output, a `functionToolResult` adds images (see
|
|
214
|
+
[Images from function tools](#images-from-function-tools)), and a
|
|
215
|
+
`ToolInputRequested` result asks the user first. Any other result is JSON with the separators of Python's `json.dumps`,
|
|
202
216
|
so the model sees the same text as from the Python core; the one exception is a
|
|
203
217
|
whole-number float such as `1.0`, which JavaScript cannot tell from `1`. The
|
|
204
218
|
argument check and its messages match the Python core, including Python's `==`
|
|
@@ -206,7 +220,7 @@ for `enum`, so `true` matches `1`.
|
|
|
206
220
|
|
|
207
221
|
The session's tool policies, `turn.toolPolicies`, win over a function tool's own
|
|
208
222
|
effect and `requiresApproval`, whether they come from the agent definition or
|
|
209
|
-
from options passed to `createAgentSessionFlow` with a `
|
|
223
|
+
from options passed to `createAgentSessionFlow` with a `BuiltinLoopAgentRunner`. A
|
|
210
224
|
built-in tool enabled only in those options must not share a function tool's
|
|
211
225
|
name; the first `RunModel` fails before it calls the model.
|
|
212
226
|
|
|
@@ -221,6 +235,74 @@ SDK returned them. It is replayed unchanged to a model of the same provider;
|
|
|
221
235
|
any other provider gets its text and tool calls only, so a session can change
|
|
222
236
|
model between turns.
|
|
223
237
|
|
|
238
|
+
### Ending or continuing a turn
|
|
239
|
+
|
|
240
|
+
`decideTurnContinuation` lets the application decide, at the end of each loop
|
|
241
|
+
iteration, whether the turn ends or continues. An iteration is one model call
|
|
242
|
+
and the tool calls it requested. The function receives a
|
|
243
|
+
`TurnContinuationContext`: `flowId`, `turnId`, `sessionSettings`, the
|
|
244
|
+
iteration's `modelOutput` items, and its `toolResults` (empty when the model
|
|
245
|
+
answered without tools). It returns `"end"`, `"default"`, or a
|
|
246
|
+
`ContinueTurn` (`{ action: "continue", message }`), directly or as a Promise.
|
|
247
|
+
|
|
248
|
+
| The iteration | `"end"` | `{ action: "continue", message }` | `"default"` or no hook |
|
|
249
|
+
| --- | --- | --- | --- |
|
|
250
|
+
| The model called tools | The turn completes without another model call | The message follows the tool results, and the model is called with both | The model is called with the tool results |
|
|
251
|
+
| The model answered without tools | The turn completes | The answer and the message are stored, and the model is called again in the same turn | The turn completes |
|
|
252
|
+
|
|
253
|
+
```ts
|
|
254
|
+
const approvalsAgentFlow = createBuiltinAgentSessionFlow(
|
|
255
|
+
{
|
|
256
|
+
name: "approvals",
|
|
257
|
+
instructions: "Review the request, then call submit_decision.",
|
|
258
|
+
model,
|
|
259
|
+
tools: [submitDecision],
|
|
260
|
+
decideTurnContinuation: ({ toolResults, modelOutput }) => {
|
|
261
|
+
if (toolResults.some((result) => result.toolName === "submit_decision")) {
|
|
262
|
+
return "end";
|
|
263
|
+
}
|
|
264
|
+
const answer = modelOutput.map((item) => item.text ?? "").join("");
|
|
265
|
+
if (toolResults.length === 0 && !answer.includes("DECISION:")) {
|
|
266
|
+
return { action: "continue", message: "Finish with a line that starts with DECISION:." };
|
|
267
|
+
}
|
|
268
|
+
return "default";
|
|
269
|
+
},
|
|
270
|
+
},
|
|
271
|
+
"ApprovalsAgent",
|
|
272
|
+
);
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
- After tools, the hook runs in the next `RunModel` Step, before its model
|
|
276
|
+
call. `"end"` completes the turn there with the tool results as its last
|
|
277
|
+
messages and zero model calls in that Step's usage. The snapshot, the
|
|
278
|
+
`turn_completed` event, and the UI stream show a finished turn whose last
|
|
279
|
+
part is the tool result. That `RunModel` still runs the session's
|
|
280
|
+
compaction and spend-limit checks first, and still emits a `chat` span. The
|
|
281
|
+
UI stream starts a step for every `RunModel` Step, so it sends one more
|
|
282
|
+
`start-step` and `finish-step` with no visible part; the AI SDK message has
|
|
283
|
+
no part for them and matches `convertAgentSnapshotToUIMessages`.
|
|
284
|
+
- After an answer, the hook runs in the same `RunModel` Step, after the model
|
|
285
|
+
call. A continuation returns the `TurnContinued` outcome: the session stores
|
|
286
|
+
the answer and the message as user text, then moves to `RunModel` again.
|
|
287
|
+
The next model request therefore ends with a user message, which every
|
|
288
|
+
provider accepts. Each model call stays its own `RunModel` Step, so a crash
|
|
289
|
+
or retry repeats only that call. Each continued call counts toward
|
|
290
|
+
`maxModelCallsPerTurn`.
|
|
291
|
+
- The message is part of the turn's transcript: the model sees it as user
|
|
292
|
+
text, and snapshots hold it as a `user` message of the same turn.
|
|
293
|
+
`convertAgentSnapshotToUIMessages` keeps a user message between two model
|
|
294
|
+
calls of one turn out of the chat and starts the next model call's step, as
|
|
295
|
+
the UI stream does.
|
|
296
|
+
- At most `maxConsecutiveTurnContinuations` (8) continuations in a row may
|
|
297
|
+
follow answers without tools; a tool call resets the count. The next one
|
|
298
|
+
ends the turn with its answer stored, `lastTurnError` set to `the turn
|
|
299
|
+
reached its limit of 8 consecutive continuations without a tool call`, and a
|
|
300
|
+
`turn_failed` event.
|
|
301
|
+
- A thrown error, or any other value, including a continuation with a blank
|
|
302
|
+
message, fails the `RunModel` attempt, and its retry policy runs it again.
|
|
303
|
+
After an answer, the retry repeats the model call too. Keep the function
|
|
304
|
+
free of side effects.
|
|
305
|
+
|
|
224
306
|
### Tool sources
|
|
225
307
|
|
|
226
308
|
`toolSources` takes tools that are listed at run time, such as an MCP server's
|
|
@@ -235,7 +317,7 @@ const files = new McpToolSource(
|
|
|
235
317
|
[{ transport: "stdio", name: "files", command: "mcp-server-files", args: ["/srv/reports"] }],
|
|
236
318
|
{ resourceAndPromptToolsEnabled: true },
|
|
237
319
|
);
|
|
238
|
-
export const reportsAgentFlow =
|
|
320
|
+
export const reportsAgentFlow = createBuiltinAgentSessionFlow(
|
|
239
321
|
{
|
|
240
322
|
name: "reports",
|
|
241
323
|
instructions: "You manage reports.",
|
|
@@ -285,7 +367,7 @@ runner's last listing of that source, as the Python core does.
|
|
|
285
367
|
|
|
286
368
|
### Demo model without a key
|
|
287
369
|
|
|
288
|
-
`ScriptedModelClient` from `@superdurable/dex-agents/
|
|
370
|
+
`ScriptedModelClient` from `@superdurable/dex-agents/builtin-loop` answers each
|
|
289
371
|
call from a function of the `ModelRequest`, so an agent runs without a model
|
|
290
372
|
key: a local demo mode, a tutorial, or a test. The function returns text,
|
|
291
373
|
reasoning, and tool calls, sync or async; `ScriptedModelClient.userTexts` and
|
|
@@ -295,7 +377,7 @@ core's. Keep the script deterministic, because a crashed `RunModel` reruns
|
|
|
295
377
|
it. `@superdurable/dex-agents/testing` exports the same class.
|
|
296
378
|
|
|
297
379
|
```ts
|
|
298
|
-
import { ScriptedModelClient, type ModelRequest, type ScriptedModelReply } from "@superdurable/dex-agents/
|
|
380
|
+
import { ScriptedModelClient, type ModelRequest, type ScriptedModelReply } from "@superdurable/dex-agents/builtin-loop";
|
|
299
381
|
|
|
300
382
|
function replyAsDemoModel(request: ModelRequest): ScriptedModelReply {
|
|
301
383
|
const results = ScriptedModelClient.toolResults(request);
|
|
@@ -308,7 +390,7 @@ function replyAsDemoModel(request: ModelRequest): ScriptedModelReply {
|
|
|
308
390
|
const model = isDemo ? new ScriptedModelClient(replyAsDemoModel) : createAiSdkModelClient(openai("gpt-5-mini"));
|
|
309
391
|
```
|
|
310
392
|
|
|
311
|
-
The stored `framework_item` of a
|
|
393
|
+
The stored `framework_item` of a built-in loop message is one of
|
|
312
394
|
`{"type":"user_text","text"}`,
|
|
313
395
|
`{"type":"model_output","provider","api","item","text","tool_calls"}`, or
|
|
314
396
|
`{"type":"tool_result","tool_call_id","tool_name","output"}`, the same as the
|
|
@@ -361,7 +443,7 @@ response`; invalid arguments fail with `invalid todos: <reason>`.
|
|
|
361
443
|
|
|
362
444
|
Plan mode reviews a plan before it runs. Enable it with
|
|
363
445
|
`planModeEnabled: true`, which needs `write_todos` in `builtInTools` and a
|
|
364
|
-
runner whose `supportsPlanMode` is `true`, such as the
|
|
446
|
+
runner whose `supportsPlanMode` is `true`, such as the built-in loop runner.
|
|
365
447
|
`AgentDefinition.planningToolChoice` is `"auto"` by default: a planning turn
|
|
366
448
|
offers only `write_todos`, the plan note tells the model to write the plan with
|
|
367
449
|
it, and a turn that answers in text, such as with a clarifying question, ends
|
|
@@ -407,7 +489,7 @@ if (snapshot.plan?.status === "draft") {
|
|
|
407
489
|
user approved plan revision <n>. Execute it and keep the task statuses
|
|
408
490
|
current with write_todos.`.
|
|
409
491
|
- `buildPlanNote(plan, interactionMode)` returns the note that shows the model
|
|
410
|
-
the plan.
|
|
492
|
+
the plan. The built-in loop adds it as user text before the turn's messages in every
|
|
411
493
|
model call, so the model always sees the latest plan.
|
|
412
494
|
|
|
413
495
|
## Steering and turn cancellation
|
|
@@ -492,7 +574,7 @@ at once.
|
|
|
492
574
|
|
|
493
575
|
Compaction keeps the model input under the context window. Pass
|
|
494
576
|
`compaction: { contextWindowTokens, triggerRatio?, retainedRatio? }` to
|
|
495
|
-
`createAgentSessionFlow`, or as `AgentDefinition.compaction` for
|
|
577
|
+
`createAgentSessionFlow`, or as `AgentDefinition.compaction` for the built-in loop. The
|
|
496
578
|
ratios default to 0.85 and 0.3, as in the Python core.
|
|
497
579
|
|
|
498
580
|
```ts
|
|
@@ -516,7 +598,7 @@ in `turn.messages` and the summary in `turn.contextSummary`.
|
|
|
516
598
|
Summarized messages stay stored until retention deletes them; see
|
|
517
599
|
[Message storage and retention](#message-storage-and-retention). A compaction policy needs a runner with all
|
|
518
600
|
three optional methods, and `createAgentSessionFlow` throws a `TypeError`
|
|
519
|
-
otherwise. `
|
|
601
|
+
otherwise. `BuiltinLoopAgentRunner` implements them: its input starts with
|
|
520
602
|
`contextSummaryNoteHeader` (`[Dex] Summary of the earlier conversation:`) and
|
|
521
603
|
the summary, it cuts only between model responses whose tool calls all have
|
|
522
604
|
stored results, so a turn with many tool calls compacts mid-turn without
|
|
@@ -559,7 +641,7 @@ const models: Record<string, ModelClient> = {
|
|
|
559
641
|
"gpt-5-mini": createAiSdkModelClient(openai("gpt-5-mini")),
|
|
560
642
|
"gpt-5": createAiSdkModelClient(openai("gpt-5")),
|
|
561
643
|
};
|
|
562
|
-
const flow =
|
|
644
|
+
const flow = createBuiltinAgentSessionFlow(
|
|
563
645
|
{
|
|
564
646
|
name: "support",
|
|
565
647
|
instructions: "You help customers with their orders.",
|
|
@@ -587,7 +669,7 @@ await client.invokeRPC(flow.updateSessionSettings, sessionId, { sessionSettings:
|
|
|
587
669
|
turn left reads as no settings.
|
|
588
670
|
- A runner reads `AgentTurnContext.sessionSettings`, `{}` when there are
|
|
589
671
|
none, such as in a session started on 0.1.0. `ContextSummaryRequest` carries
|
|
590
|
-
the same value.
|
|
672
|
+
the same value. The built-in loop's `AgentDefinition.resolveModel` maps it to the
|
|
591
673
|
`ModelClient` of each model call, including summary calls and token
|
|
592
674
|
estimates, and `InstructionContext.sessionSettings` passes it to an
|
|
593
675
|
instructions function.
|
|
@@ -615,6 +697,182 @@ await client.invokeRPC(flow.updateSessionSettings, sessionId, { sessionSettings:
|
|
|
615
697
|
- Start sessions with settings, and call `update_session_settings`, only after
|
|
616
698
|
every Worker of the Flow type runs 0.2.0. A 0.1.0 Worker drops the start
|
|
617
699
|
settings and starts turns without them.
|
|
700
|
+
- `AgentSessionStartInput.startEnvironment` (`startAgentSession(..., { startEnvironment })`) is different: string
|
|
701
|
+
keys and values that say where the starting program ran. It stays in the Flow's start input and the session never
|
|
702
|
+
reads it, so it is not a setting and cannot change.
|
|
703
|
+
|
|
704
|
+
## Images
|
|
705
|
+
|
|
706
|
+
A user message and a tool result can carry images. Session state holds only an `AgentImageReference` for each image;
|
|
707
|
+
the bytes live in a media store, keyed by their SHA-256. The wire form is the same as the Python core's:
|
|
708
|
+
|
|
709
|
+
```json
|
|
710
|
+
{"type": "image", "uri": "s3://agent-media/sessions/sha256/9f2c...", "mime_type": "image/png", "sha256": "9f2c...", "size_bytes": 48213}
|
|
711
|
+
```
|
|
712
|
+
|
|
713
|
+
Upload first, then send the reference:
|
|
714
|
+
|
|
715
|
+
```ts
|
|
716
|
+
import { sendAgentUserMessage } from "@superdurable/dex-agents";
|
|
717
|
+
import { S3MediaStore } from "@superdurable/dex-agents/s3";
|
|
718
|
+
|
|
719
|
+
const store = new S3MediaStore({ bucket: "agent-media", prefix: "sessions" }); // credentials from the AWS chain
|
|
720
|
+
const reference = await store.putImage(await readFile("chart.png"));
|
|
721
|
+
await sendAgentUserMessage(client, flow, "session-42", { text: "What does this chart show?", images: [reference] });
|
|
722
|
+
```
|
|
723
|
+
|
|
724
|
+
A tool source returns images the same way, uploading inside its call. This is a tool source's
|
|
725
|
+
`executeDeferredToolCall` result:
|
|
726
|
+
|
|
727
|
+
```ts
|
|
728
|
+
const reference = await store.putImage(screenshotPng);
|
|
729
|
+
return { output: "Took a screenshot.", images: [reference] };
|
|
730
|
+
```
|
|
731
|
+
|
|
732
|
+
A function tool returns `functionToolResult({ output, images })` instead, because the runner serializes its plain
|
|
733
|
+
objects as JSON; see [Images from function tools](#images-from-function-tools).
|
|
734
|
+
|
|
735
|
+
- **One object per image.** The object key is `<prefix>/sha256/<sha256>`, and the reference's `uri` ends in
|
|
736
|
+
`/sha256/<sha256>`. `S3MediaStore.putImage` uploads with `If-None-Match: *` and treats "already exists" (412) as
|
|
737
|
+
success, so a tool Step that a crash retries, a client that sends again, or two Workers at once write one object.
|
|
738
|
+
A 409 conflict retries the upload twice; when every attempt conflicts, a `HEAD` must find the object with the
|
|
739
|
+
image's size, or `putImage` throws `MediaStoreError`. On a
|
|
740
|
+
service that refuses conditional writes it checks with `HEAD` first; two concurrent uploads may then both write the
|
|
741
|
+
same content, so deduplication holds for the key and its final content. A replay always reads the bytes the
|
|
742
|
+
reference was made from: `getImage` reads only references under its own bucket and prefix, refuses an object whose
|
|
743
|
+
`Content-Length` differs, reads at most one byte past `sizeBytes`, and `verifyImageBytes` checks the SHA-256, the
|
|
744
|
+
size, and that the bytes are a well-formed image of `mimeType`.
|
|
745
|
+
- **Types and sizes.** PNG, JPEG, GIF, and WebP, detected from the bytes with a cheap structural check (PNG signature
|
|
746
|
+
and `IHDR`; JPEG SOI, a marker, and EOI at the end; GIF header and screen descriptor; WebP RIFF size and a `VP8`,
|
|
747
|
+
`VP8L`, or `VP8X` chunk), not a full decode, up to `maxImageBytes` (3,750,000 bytes) each. Its base64 text is 5,000,000 bytes, which fits the per-image limits of Amazon Bedrock (5 MB of base64) and the
|
|
748
|
+
Claude API (10 MB of base64). One message or tool
|
|
749
|
+
result carries at most `maxImagesPerMessage` (20). `putImage` throws `ImageValidationError` for other bytes. A `uri`
|
|
750
|
+
is at most `maxImageUriLength` (1024) bytes as UTF-8 in every core, and a store refuses a bucket, prefix, or name
|
|
751
|
+
that would make longer ones. `validateImageReferences` checks references, and `sendAgentUserMessage` checks them
|
|
752
|
+
before it sends.
|
|
753
|
+
- **Where they are stored.** `AgentUserMessage.images`, the session copy on the stored `AgentMessage.images` for
|
|
754
|
+
display, `ToolExecutionResult.images`, and `AgentToolCall.images`. A runner also keeps the references, never the
|
|
755
|
+
bytes, in its `frameworkItem`, the replay truth. Decoded values omit `images` when there are none. The browser
|
|
756
|
+
snapshot JSON includes the `images` of messages and tool calls.
|
|
757
|
+
- **The runner.** `send_user_message` returns `false` for images unless the runner implements
|
|
758
|
+
`buildUserMessageWithImages`, which the session calls instead of `buildUserMessage` for a message with images.
|
|
759
|
+
When it builds model input, the runner reads the bytes with `loadImages(store, references)`, choosing the
|
|
760
|
+
references from its replayed items or from `collectTurnImages(turn)`. The session sets the stored user message's
|
|
761
|
+
`images` itself; for a tool result message the runner sets `FrameworkMessage.images` from the record. A tool result
|
|
762
|
+
with an invalid reference fails the tool Step attempt. The session checks the `images` of every `FrameworkMessage` a
|
|
763
|
+
runner returns before it stores any: an invalid one fails the model call like any runner error, and one from
|
|
764
|
+
`buildUnfinishedTurnMessages` leaves only the turn's note.
|
|
765
|
+
- **Stores.** `new S3MediaStore({ bucket, prefix, endpoint, region, forcePathStyle, credentials })` or
|
|
766
|
+
`S3MediaStore.fromStoreUri("s3://bucket/prefix")` for AWS S3 or any S3-compatible service; call `close()` when done.
|
|
767
|
+
Credentials come from the standard AWS chain by default; pass `credentials` (`accessKeyId`, `secretAccessKey`, and
|
|
768
|
+
optionally `sessionToken`) only for a bucket whose keys are not secret, such as a local development bucket with
|
|
769
|
+
fixed loopback keys. The store never logs them. `InMemoryMediaStore` is for tests in one process, and returns a copy
|
|
770
|
+
on each read. A store of your own implements `MediaStore`: check its location with `requireShortImageUris`, build
|
|
771
|
+
each reference with `buildImageReference(data, storeUri)`, and check reads with `requireStoreImageReference` and
|
|
772
|
+
`verifyImageBytes`.
|
|
773
|
+
- **Local directory.** `new LocalDirectoryMediaStore(root)`, from the main export, keeps images on one machine, for a
|
|
774
|
+
client and Workers that share a file system, such as a command-line tool in local mode; use `S3MediaStore` across
|
|
775
|
+
machines. `root` must be an absolute path. An image's file is `<root>/sha256/<sha256>`, and its `uri` is
|
|
776
|
+
`file://<root>/sha256/<sha256>`: the root is made absolute as text, without resolving symbolic links or normalizing
|
|
777
|
+
Unicode, and every byte other than ASCII letters, digits, `-._~`, and `/` is percent-encoded, the same in every core.
|
|
778
|
+
`putImage` copies the bytes into a temporary file in the same directory and renames it into place; an existing file
|
|
779
|
+
with the same bytes is success. Directories it creates get mode `0700` and files `0600`. A write syncs the image file
|
|
780
|
+
before the rename. Before every `putImage` returns, even when an identical file already existed and nothing was
|
|
781
|
+
written, it syncs the `sha256` directory, the root, and the root's parent; it also syncs the parent of each directory
|
|
782
|
+
it creates. Directories above the root's parent must already be durable; a concurrent creator of a deeper missing
|
|
783
|
+
ancestor is out of scope. It refuses a symbolic link, a special file, or an image file
|
|
784
|
+
with more than one hard link under the root with `MediaStoreError`. Node.js has no `openat`, so it opens files by path
|
|
785
|
+
and checks before and after each operation: `lstat` on the `sha256` directory and `O_NOFOLLOW` on the file before,
|
|
786
|
+
then the directory's real path and the file's device and inode after. A read that fails the check returns nothing;
|
|
787
|
+
a write that fails it removes the file it wrote when it can and throws. A process of the same user that can write the
|
|
788
|
+
`0700` root can still race a directory swap between these checks, so this store does not guarantee that it never
|
|
789
|
+
follows a symbolic link. The Python core opens every path relative to an open directory and has no such window.
|
|
790
|
+
Reads check the file's size first, read at most one byte past `sizeBytes`, and run `verifyImageBytes`. It needs a POSIX file system. Deleting a
|
|
791
|
+
session deletes no file.
|
|
792
|
+
- **Built-in runners.** The built-in loop and the framework adapters show images to their model when built with a
|
|
793
|
+
`mediaStore`: they implement `buildUserMessageWithImages`, keep references in their stored items, and load the bytes
|
|
794
|
+
for each model call. Without a store they do not, and their sessions refuse images. For the built-in loop, pass it
|
|
795
|
+
to `createBuiltinAgentSessionFlow(definition, flowType, { mediaStore })` or `new BuiltinLoopAgentRunner(definition,
|
|
796
|
+
mediaStore)`; see [Images on the built-in loop](#images-on-the-built-in-loop).
|
|
797
|
+
- **MCP tools.** `new McpToolSource(servers, { mediaStore })` uploads the `image` content of a successful result
|
|
798
|
+
inside the tool Step and returns the references in `ToolExecutionResult.images`. Give it the store the runner
|
|
799
|
+
reads.
|
|
800
|
+
- **Retention.** Deleting a session deletes no object. A runner reads the images again whenever a model call replays
|
|
801
|
+
them, so a bucket lifecycle rule must keep each object at least as long as any session that references it can
|
|
802
|
+
still run a turn or replay its history. When an object is gone, the turn that needs it fails with `MediaStoreError`
|
|
803
|
+
in `lastTurnError`, after its user message, with nothing else stored; later turns that replay the image fail the
|
|
804
|
+
same way.
|
|
805
|
+
- Send images only after every Worker of the Flow type runs 0.5.0: an earlier Worker drops them.
|
|
806
|
+
|
|
807
|
+
### Images on the built-in loop
|
|
808
|
+
|
|
809
|
+
```ts
|
|
810
|
+
import { S3MediaStore } from "@superdurable/dex-agents/s3";
|
|
811
|
+
|
|
812
|
+
const mediaStore = S3MediaStore.fromStoreUri("s3://agent-media/sessions");
|
|
813
|
+
const flow = createBuiltinAgentSessionFlow(supportAgent, "SupportAgent", { mediaStore });
|
|
814
|
+
```
|
|
815
|
+
|
|
816
|
+
- **Stored items.** The `user_text` item of a message with images, and the `tool_result` item of a call whose record
|
|
817
|
+
has images, add an `images` list of snake_case references after their other fields, the same JSON as the Python
|
|
818
|
+
core. An item without images has no `images` key, so its bytes are unchanged. The tool result message also shows
|
|
819
|
+
the references in `images`. Session state never holds image bytes.
|
|
820
|
+
- **Model calls.** `ModelUserTextItem.images` and `ModelToolResultItem.images` carry the references into the
|
|
821
|
+
`ModelRequest`. Right before each model call, the runner reads the bytes of every image in the request with
|
|
822
|
+
`loadImages` and passes them in `ModelRequest.imageBytes`, keyed by SHA-256. A retried `RunModel` attempt, or one on
|
|
823
|
+
a new Worker after a crash, builds the same request from the same references and bytes. A store read that fails
|
|
824
|
+
fails that attempt, which retries; after the last attempt the turn records the error in `lastTurnError`.
|
|
825
|
+
- **Summaries and estimates.** The model call that writes a context summary sees the text of the summarized items,
|
|
826
|
+
without their images. `estimateInputTokens` gets requests without image bytes; the AI SDK client counts
|
|
827
|
+
`estimatedImageInputTokens` (1600) for each image.
|
|
828
|
+
- **Without a store.** A tool source result's references are still stored for display, but the model sees only the
|
|
829
|
+
text. A function tool's images fail the call instead, without a retry.
|
|
830
|
+
- **AI SDK.** `createAiSdkModelClient` sends each image after the item's text as an AI SDK `file` part with its bytes:
|
|
831
|
+
`{ type: "file", mediaType, data: { type: "data", data } }`. A user message's content is a text part and one file
|
|
832
|
+
part per image. A tool result with images has `content` output, a text part and one file part per image; the
|
|
833
|
+
provider package converts it, and a provider that takes no images in a tool result handles them as its package
|
|
834
|
+
does.
|
|
835
|
+
- **Your own model client.** Convert each reference in `images` to the provider's image part with its bytes from
|
|
836
|
+
`ModelRequest.imageBytes`, and throw when they are missing. A client that ignores `images` shows the model the text
|
|
837
|
+
only.
|
|
838
|
+
- **Which tools return images.** A tool source's `ToolExecutionResult.images`, such as the image content of a
|
|
839
|
+
`new McpToolSource(servers, { mediaStore })` result. Give the source the store the runner reads. A function tool
|
|
840
|
+
returns `functionToolResult`; see [Images from function tools](#images-from-function-tools).
|
|
841
|
+
|
|
842
|
+
### Images from function tools
|
|
843
|
+
|
|
844
|
+
A function tool gets the runner's media store as `ToolCallContext.mediaStore`, or `undefined` when the runner has
|
|
845
|
+
none. It uploads the bytes inside its call and returns the references with its text through `functionToolResult`:
|
|
846
|
+
|
|
847
|
+
```ts
|
|
848
|
+
import { functionTool, functionToolResult } from "@superdurable/dex-agents/builtin-loop";
|
|
849
|
+
|
|
850
|
+
const takeScreenshot = functionTool<Record<string, never>>({
|
|
851
|
+
name: "take_screenshot",
|
|
852
|
+
description: "Take a screenshot of the screen.",
|
|
853
|
+
parameters: { type: "object", properties: {} },
|
|
854
|
+
effect: "read",
|
|
855
|
+
execute: async (_args, context) => {
|
|
856
|
+
if (context.mediaStore === undefined) {
|
|
857
|
+
return "Screenshots need a media store.";
|
|
858
|
+
}
|
|
859
|
+
const reference = await context.mediaStore.putImage(await screen.capturePng());
|
|
860
|
+
return functionToolResult({ output: "Took a screenshot.", images: [reference] });
|
|
861
|
+
},
|
|
862
|
+
});
|
|
863
|
+
```
|
|
864
|
+
|
|
865
|
+
- The runner returns the `FunctionToolResult` that `functionToolResult` created as a `ToolExecutionResult` with the
|
|
866
|
+
same output and images, so the images are stored, checked, and shown to the model exactly like a tool source's.
|
|
867
|
+
Only the object `functionToolResult` returned counts: a plain object, or a copy, with the same fields is still
|
|
868
|
+
serialized as JSON.
|
|
869
|
+
- More than `maxImagesPerMessage` images, or a reference that `validateImageReference` refuses, fails the attempt
|
|
870
|
+
with `ImageValidationError`. A failed attempt of a `read` or `idempotent_write` tool retries; a single-attempt call
|
|
871
|
+
records `outcome_unknown`.
|
|
872
|
+
- Images from a runner without a media store throw `ToolExecutionError`, because no retry could show them. The
|
|
873
|
+
session records `failed` once, without a retry, and the model sees the error text.
|
|
874
|
+
- A retried attempt uploads the same bytes to the same object, so uploading again is safe. A replay reads the stored
|
|
875
|
+
references, so the tool does not run again.
|
|
618
876
|
|
|
619
877
|
## Message storage and retention
|
|
620
878
|
|
|
@@ -737,10 +995,22 @@ interface AgentFrameworkRunner<ToolOutcome = ToolExecutionResult> {
|
|
|
737
995
|
runModelStep(turn: AgentTurnContext): Promise<AgentTurnOutcome>;
|
|
738
996
|
executeDeferredToolCall(call: DeferredToolExecution): Promise<ToolOutcome>;
|
|
739
997
|
buildUnfinishedTurnMessages?(turn: AgentTurnContext): FrameworkMessage[]; // optional
|
|
998
|
+
buildUserMessageWithImages?(text: string, images: AgentImageReference[]): FrameworkMessage; // optional; see Images
|
|
740
999
|
}
|
|
741
1000
|
|
|
742
|
-
type AgentTurnOutcome = TurnCompleted | ToolCallsRequested;
|
|
743
|
-
interface TurnCompleted {
|
|
1001
|
+
type AgentTurnOutcome = TurnCompleted | ToolCallsRequested | TurnContinued;
|
|
1002
|
+
interface TurnCompleted {
|
|
1003
|
+
kind: "completed";
|
|
1004
|
+
newMessages: FrameworkMessage[];
|
|
1005
|
+
usage: AgentUsage;
|
|
1006
|
+
turnError?: string; // optional since 0.5.0: end the turn as failed, without a retry
|
|
1007
|
+
}
|
|
1008
|
+
interface TurnContinued { // since 0.5.0
|
|
1009
|
+
kind: "turn_continued";
|
|
1010
|
+
frameworkRunState: string; // non-empty, opaque to the session
|
|
1011
|
+
newMessages: FrameworkMessage[]; // stored now; requires buildUnfinishedTurnMessages
|
|
1012
|
+
usage: AgentUsage;
|
|
1013
|
+
}
|
|
744
1014
|
interface ToolCallsRequested {
|
|
745
1015
|
kind: "tool_calls_requested";
|
|
746
1016
|
frameworkRunState: string; // non-empty, opaque to the session
|
|
@@ -755,6 +1025,7 @@ interface FrameworkMessage {
|
|
|
755
1025
|
text: string | null; toolCallId: string | null; toolName: string | null;
|
|
756
1026
|
toolArguments: string | null; toolOutput: string | null;
|
|
757
1027
|
frameworkItem: string; // framework-native JSON, replayed unchanged
|
|
1028
|
+
images?: AgentImageReference[]; // optional display projection; see Images
|
|
758
1029
|
}
|
|
759
1030
|
```
|
|
760
1031
|
|
|
@@ -775,7 +1046,7 @@ calls it for the turn failure note.
|
|
|
775
1046
|
| --- | --- |
|
|
776
1047
|
| `flowId`, `turnId`, `attempt` | Session ID, `turn-<n>`, one-based `RunModel` attempt |
|
|
777
1048
|
| `messages` | Every stored message in sequence order, including this turn's user message |
|
|
778
|
-
| `frameworkRunState`, `isResuming` | Run state from this turn's last `ToolCallsRequested`, or `null` for the turn's first model call |
|
|
1049
|
+
| `frameworkRunState`, `isResuming` | Run state from this turn's last `ToolCallsRequested` or `TurnContinued`, or `null` for the turn's first model call |
|
|
779
1050
|
| `toolCalls`, `recordedToolCall(id)` | This turn's tool call records, in deferral order; all final when `runModelStep` runs |
|
|
780
1051
|
| `resolveToolPolicy(name, frameworkEffect?)` | Effective `ResolvedToolPolicy { effect, requiresApproval }` |
|
|
781
1052
|
| `events` | `AgentEventWriter` bound to this turn and attempt |
|
|
@@ -811,13 +1082,20 @@ The runner must:
|
|
|
811
1082
|
every message of the turn the session has not stored yet, in order: model
|
|
812
1083
|
output, tool requests, tool results, and the answer. Without incremental
|
|
813
1084
|
messages, that is every message the turn produced after its user message.
|
|
814
|
-
6.
|
|
1085
|
+
6. To make another model call within the same turn, return `TurnContinued`
|
|
1086
|
+
with the run state and the messages to store now, ending with a user
|
|
1087
|
+
message when the provider needs the request to end with one. The session
|
|
1088
|
+
stores them and runs `RunModel` again. A runner that never needs this
|
|
1089
|
+
ignores it. To end the turn as failed without a retry, such as at a
|
|
1090
|
+
limit the runner enforces, return `TurnCompleted` with `turnError`; the
|
|
1091
|
+
session stores `newMessages` and sets `lastTurnError` to that text.
|
|
1092
|
+
7. Report only this model call's usage. Use `emptyAgentUsage()` and
|
|
815
1093
|
`addAgentUsage()` when a framework reports several usage parts. Optionally
|
|
816
1094
|
return `modelCallMetadata: { requestModel, responseModel, providerName,
|
|
817
1095
|
responseId, finishReasons }` in either outcome. The session never stores it;
|
|
818
1096
|
it names the `chat` span (see [Tracing](#tracing)). Omit a field the
|
|
819
1097
|
framework does not expose.
|
|
820
|
-
|
|
1098
|
+
8. Throw for an unexpected failure. The model call reruns from the same
|
|
821
1099
|
committed state, up to 3 attempts within 30 minutes; then the turn fails.
|
|
822
1100
|
|
|
823
1101
|
**Incremental turn messages (optional).** A runner may return the turn's
|
|
@@ -1006,7 +1284,32 @@ const waiting = await client.searchFlows('ActiveStepTypes = "AwaitToolApproval"'
|
|
|
1006
1284
|
`AwaitQuestionAnswer` finds questions the same way. A session leaves the search when it moves on. `Client.startFlow`
|
|
1007
1285
|
does not turn this on; pass `configOverride: { activeStepSearchMode: agentSessionActiveStepSearchMode }` there, or use
|
|
1008
1286
|
`Client.updateFlowConfig` for a running session. See
|
|
1009
|
-
[Find sessions that need a person](https://docs.superdurable.io/
|
|
1287
|
+
[Find sessions that need a person](https://docs.superdurable.io/reference/agents/approvals#find-sessions).
|
|
1288
|
+
|
|
1289
|
+
## Showing sessions in Dex Web
|
|
1290
|
+
|
|
1291
|
+
Dex Web draws a session and offers its Actions from the session Flow
|
|
1292
|
+
Definition Graph. Let the Worker upload it: pass
|
|
1293
|
+
`sessionFlowDefinitionDocument({ flowType, actionPermission })` in
|
|
1294
|
+
`WorkerOptions.flowDefinitions`.
|
|
1295
|
+
|
|
1296
|
+
```ts
|
|
1297
|
+
import { Registry, Worker } from "@superdurable/dex";
|
|
1298
|
+
import { sessionFlowDefinitionDocument } from "@superdurable/dex-agents";
|
|
1299
|
+
|
|
1300
|
+
const worker = new Worker(new Registry([flow]), blobCache, {
|
|
1301
|
+
flowDefinitions: [
|
|
1302
|
+
sessionFlowDefinitionDocument({ flowType: "SupportAgent", actionPermission: "support-agent.operate" }),
|
|
1303
|
+
],
|
|
1304
|
+
});
|
|
1305
|
+
```
|
|
1306
|
+
|
|
1307
|
+
The upload needs server-zd 0.1.26 or later. The document's `generatedAt` is
|
|
1308
|
+
the time the graph template last changed, the same in both cores, so a restart
|
|
1309
|
+
writes nothing and an upgrade replaces the graph. For an older server, render
|
|
1310
|
+
a file with `renderSessionFlowGraph` or the `dex-agents-session-graph` command
|
|
1311
|
+
and pass its directory to Dex Web with `--flow-rendering-dir`. See
|
|
1312
|
+
[Show sessions in Dex Web](https://docs.superdurable.io/reference/agents/dex-web).
|
|
1010
1313
|
|
|
1011
1314
|
## Spend limits
|
|
1012
1315
|
|
|
@@ -1014,7 +1317,7 @@ does not turn this on; pass `configOverride: { activeStepSearchMode: agentSessio
|
|
|
1014
1317
|
across many sessions. The session asks it before each model call and reports what
|
|
1015
1318
|
the call used. Counters and limits belong to the application. Without one, the
|
|
1016
1319
|
session makes no checks. See the interface's documentation and
|
|
1017
|
-
[Spend limits](https://docs.superdurable.io/
|
|
1320
|
+
[Spend limits](https://docs.superdurable.io/reference/agents/spend-limits).
|
|
1018
1321
|
|
|
1019
1322
|
## Flow shape
|
|
1020
1323
|
|
|
@@ -1190,12 +1493,13 @@ cannot defer it.
|
|
|
1190
1493
|
|
|
1191
1494
|
`@superdurable/dex-agents/mcp` turns the tools of MCP servers into tools of an
|
|
1192
1495
|
agent session. Every MCP tool call runs in its own tool Step, with an effect
|
|
1193
|
-
class from the tool's annotations
|
|
1496
|
+
class from the tool's annotations or from the [Dex MCP
|
|
1497
|
+
conventions](#dex-mcp-conventions), so a crash never repeats a write silently.
|
|
1194
1498
|
It needs `@modelcontextprotocol/sdk` 1.x, an optional peer dependency; the main
|
|
1195
1499
|
entry point never imports it:
|
|
1196
1500
|
|
|
1197
1501
|
```bash
|
|
1198
|
-
npm i @superdurable/dex-agents @superdurable/dex@1.
|
|
1502
|
+
npm i @superdurable/dex-agents @superdurable/dex@1.45.0 @modelcontextprotocol/sdk zod
|
|
1199
1503
|
```
|
|
1200
1504
|
|
|
1201
1505
|
Configure the servers on the Worker, never from Flow input, a user message, or
|
|
@@ -1215,7 +1519,7 @@ const tools = new McpToolSource([
|
|
|
1215
1519
|
]);
|
|
1216
1520
|
```
|
|
1217
1521
|
|
|
1218
|
-
A
|
|
1522
|
+
A built-in loop agent takes the source in `toolSources`; see
|
|
1219
1523
|
[Tool sources](#tool-sources). Another runner offers the tools and defers every
|
|
1220
1524
|
call; the tool Step runs it:
|
|
1221
1525
|
|
|
@@ -1257,6 +1561,16 @@ characters shortened with a hash. When two tools get the same name, the first
|
|
|
1257
1561
|
in server order is offered and the other is left out with a warning. Key tool
|
|
1258
1562
|
policies by that name.
|
|
1259
1563
|
|
|
1564
|
+
`executeDeferredToolCall(call, { requestMeta })` adds entries to the
|
|
1565
|
+
`tools/call` request's `_meta`, such as the identity the caller expects the
|
|
1566
|
+
tool to have; `callTool` takes the same `requestMeta` option. The idempotency
|
|
1567
|
+
key entry wins on a name clash. `willResumeToolTaskWithoutSending(call)` is
|
|
1568
|
+
`true` only when `executeDeferredToolCall` will return a session-polled task
|
|
1569
|
+
that an earlier attempt started and can never send `tools/call` for the call;
|
|
1570
|
+
a task polled inside the tool Step, such as one bound to a protocol session,
|
|
1571
|
+
may be called again when the server no longer knows it. A check that only
|
|
1572
|
+
guards a new send can be skipped when it is `true`. Since 0.5.0.
|
|
1573
|
+
|
|
1260
1574
|
### Resources and prompts
|
|
1261
1575
|
|
|
1262
1576
|
With `resourceAndPromptToolsEnabled: true`, `listTools` also offers three
|
|
@@ -1286,10 +1600,11 @@ or list-change notifications. The names are exported as
|
|
|
1286
1600
|
|
|
1287
1601
|
### Effects
|
|
1288
1602
|
|
|
1289
|
-
`toolEffectFromMcpToolAnnotations` classifies
|
|
1603
|
+
A tool with the Dex `effect` key keeps the declared effect; `toolEffectFromMcpToolAnnotations` classifies the others:
|
|
1290
1604
|
|
|
1291
1605
|
| MCP annotations | Effect | Tool Step | Approval by default |
|
|
1292
1606
|
| --- | --- | --- | --- |
|
|
1607
|
+
| any, with the Dex `effect` key in `_meta` | the declared effect; see [Dex MCP conventions](#dex-mcp-conventions) | by effect | Only with `approval: recommended` |
|
|
1293
1608
|
| `readOnlyHint: true` | `read` | `CallTool` | No |
|
|
1294
1609
|
| `idempotentHint: true`, whatever `destructiveHint` says | `idempotent_write` | `CallTool` | No |
|
|
1295
1610
|
| anything else, or none | `defaultToolEffect`, `unclassified` by default | `CallSingleAttemptTool` | Yes |
|
|
@@ -1316,7 +1631,10 @@ signal aborts the request.
|
|
|
1316
1631
|
| What happens | Recorded | Re-issued? |
|
|
1317
1632
|
| --- | --- | --- |
|
|
1318
1633
|
| A result | `succeeded`: text blocks joined with newlines, other content as JSON | - |
|
|
1634
|
+
| A result with `image` content, on a source built with `mediaStore` | `succeeded`: each image uploaded and recorded in `images`; the text shows `[image <n>]` in its place, counting from 1. More than 20 images, base64 that is invalid or not canonical, or bytes that are not a supported image: `failed` (`ToolExecutionError`) | - |
|
|
1635
|
+
| An `isError`, `rejected`, or `unknown` result with `image` content | As above, with `[image <n> omitted]` for each image in the text, with or without a store, so the recorded error holds no image bytes | As above |
|
|
1319
1636
|
| An `isError` result, or a JSON-RPC error response other than `-32603` | `failed` with the text (`ToolExecutionError`) | No |
|
|
1637
|
+
| A result or error with the Dex `outcome` key | `outcome_unknown` or `failed`; see [Dex MCP conventions](#dex-mcp-conventions) | No |
|
|
1320
1638
|
| The call was never sent: its server is not configured, or 3 connection attempts failed | `failed` | No |
|
|
1321
1639
|
| The connection broke, the server exited, or the request timed out after it was sent; an internal error (`-32603`), an HTTP error status, or an unreadable body, which a gateway can return after the server ran the call | `McpToolCallInterruptedError`, a Step failure | `read` and `idempotent_write`: `CallTool` sends a new request, same idempotency key, up to 3 attempts. Other effects: `outcome_unknown`, never sent again |
|
|
1322
1640
|
|
|
@@ -1346,7 +1664,7 @@ only delays the next poll. A server that no longer knows the task, or cannot be
|
|
|
1346
1664
|
reached for three poll attempts, records `failed` for a `read` tool and
|
|
1347
1665
|
`outcome_unknown` otherwise. `cancel_turn` abandons the task, and
|
|
1348
1666
|
`cancelToolTask` sends `tasks/cancel`; forward it from your runner's
|
|
1349
|
-
`cancelToolTask`, as the
|
|
1667
|
+
`cancelToolTask`, as the built-in loop runner does. Any answer from the server ends
|
|
1350
1668
|
the cancellation, so a retried one is harmless.
|
|
1351
1669
|
|
|
1352
1670
|
The source requests such a task's TTL as `taskTtlMs`, 24 hours by default. The
|
|
@@ -1378,13 +1696,117 @@ task's input request into a session question. The source declares no
|
|
|
1378
1696
|
elicitation, sampling, or roots capability, so a server cannot send those
|
|
1379
1697
|
requests directly.
|
|
1380
1698
|
|
|
1699
|
+
### Dex MCP conventions
|
|
1700
|
+
|
|
1701
|
+
An MCP server built for Dex, such as one that serves Dex connectors, carries a
|
|
1702
|
+
connector's guarantees across MCP with `_meta` keys under the
|
|
1703
|
+
`io.superdurable.dex/` prefix. A server that sets none of them behaves exactly
|
|
1704
|
+
as described above. Both cores follow the same rules, and
|
|
1705
|
+
[`agents/conformance/mcp_dex_conventions.json`](../../conformance/mcp_dex_conventions.json)
|
|
1706
|
+
fixes them case by case. Since 0.5.0.
|
|
1707
|
+
|
|
1708
|
+
| Key | Where | Values |
|
|
1709
|
+
| --- | --- | --- |
|
|
1710
|
+
| `io.superdurable.dex/effect` (`mcpToolEffectMetaKey`) | tool definition `_meta` in `tools/list` | `read`, `idempotent_write`, `non_idempotent_write` |
|
|
1711
|
+
| `io.superdurable.dex/approval` (`mcpToolApprovalMetaKey`) | tool definition `_meta` in `tools/list` | `recommended` |
|
|
1712
|
+
| `io.superdurable.dex/outcome` (`mcpToolOutcomeMetaKey`) | `tools/call` result `_meta`, or JSON-RPC `error.data._meta` | `unknown`, `rejected` |
|
|
1713
|
+
| `io.superdurable.dex/idempotencyKey` (`mcpIdempotencyKeyMetaKey`) | `tools/call` request `_meta`, sent by the source | `<flowId>/<turnId>/<toolCallId>` |
|
|
1714
|
+
|
|
1715
|
+
**Tool definitions.** A tool with the `effect` key is declared, as a connector
|
|
1716
|
+
manifest declares an operation for `ConnectorToolSource`:
|
|
1717
|
+
|
|
1718
|
+
- Its effect class is the declared value, whatever its annotations say:
|
|
1719
|
+
`read` and `idempotent_write` run in `CallTool`, `non_idempotent_write` in
|
|
1720
|
+
`CallSingleAttemptTool`. A declared write runs in the model's order after the
|
|
1721
|
+
turn's earlier writes, and a write after one that did not succeed is skipped.
|
|
1722
|
+
- Without a session policy, its calls wait for approval only with `approval:
|
|
1723
|
+
recommended`, like `agent.approval: recommended` in a manifest. Any other
|
|
1724
|
+
`approval` value, `null` included, also asks, so an unknown value fails
|
|
1725
|
+
closed.
|
|
1726
|
+
- A session policy in `toolPolicies` for the tool may keep or raise its
|
|
1727
|
+
safety, never lower it. The call gets the more conservative of the declared
|
|
1728
|
+
effect and the policy's effect, ranked `read`, `idempotent_write`,
|
|
1729
|
+
`non_idempotent_write`, then `unclassified`. So a policy
|
|
1730
|
+
`{ effect: "non_idempotent_write" }` on a tool declared `read` runs it once in
|
|
1731
|
+
`CallSingleAttemptTool`, in the model's order, after approval. A declared
|
|
1732
|
+
`non_idempotent_write` stays one with a `read` policy.
|
|
1733
|
+
- The policy's `requiresApproval`, when set, wins. When it is left out, a call
|
|
1734
|
+
waits for approval when the tool recommends it or the policy's effect asks
|
|
1735
|
+
for approval by default, as `non_idempotent_write` does.
|
|
1736
|
+
- A framework adapter gets the same rule from
|
|
1737
|
+
`tools.mergeToolPolicies(yourPolicies)`. Do not spread your policies after
|
|
1738
|
+
`tools.toolPolicies()`: yours would replace the declaration and could lower
|
|
1739
|
+
its effect.
|
|
1740
|
+
- `approval` without `effect` is ignored. An `effect` value outside the three
|
|
1741
|
+
above gives the tool the session's `defaultToolEffect` and ignores its
|
|
1742
|
+
annotations, with a warning log.
|
|
1743
|
+
- `trustsToolAnnotations: false` also ignores both keys.
|
|
1744
|
+
|
|
1745
|
+
**Results.** The `outcome` key reports what a call did:
|
|
1746
|
+
|
|
1747
|
+
| `outcome` | Recorded | The model sees | Sent again? |
|
|
1748
|
+
| --- | --- | --- | --- |
|
|
1749
|
+
| `unknown` | `outcome_unknown` (`ToolOutcomeUnknownError`), with the result's text as `error` | `outcomeUnknownToolResult` | Never, for any effect class |
|
|
1750
|
+
| `rejected` | `failed` (`ToolExecutionError`) | `The tool call failed: <text>` | Never |
|
|
1751
|
+
| absent | as in "What a call does" | | |
|
|
1752
|
+
|
|
1753
|
+
- The key wins over `isError` and over the JSON-RPC error code. Set
|
|
1754
|
+
`isError: true` with either value, so a client that does not know these
|
|
1755
|
+
conventions never reads the result as a success.
|
|
1756
|
+
- The text is the result's text: text blocks joined with newlines, other
|
|
1757
|
+
content as JSON. A result without text gets a fixed text. Write a message
|
|
1758
|
+
that is safe to show the model and the user.
|
|
1759
|
+
- In the JSON-RPC error form, put the key in `error.data._meta`. The text is
|
|
1760
|
+
`MCP error <code>: <message>`. An internal error (`-32603`) with the key is
|
|
1761
|
+
not re-issued.
|
|
1762
|
+
- A task's completed result follows the same rule, whether the tool Step or
|
|
1763
|
+
`PollToolTask` reads it.
|
|
1764
|
+
- Other values are reserved; the source ignores them with a warning log.
|
|
1765
|
+
|
|
1766
|
+
**Idempotency key.** Every `tools/call` from a tool Step carries the session's
|
|
1767
|
+
idempotency key. It is the same on every attempt of one tool call, including a
|
|
1768
|
+
`CallTool` retry and an attempt after a Worker restart, and it is unique per
|
|
1769
|
+
tool call. Treat it as opaque. A server that follows these conventions derives
|
|
1770
|
+
its own call ID from it, such as a hash, and keeps a fence per call ID: a
|
|
1771
|
+
retried attempt gets the first attempt's result and never applies the write a
|
|
1772
|
+
second time. When the server cannot tell whether an earlier attempt took
|
|
1773
|
+
effect, it answers `outcome: unknown`. It should refuse a declared write that
|
|
1774
|
+
arrives without the key, because it cannot recognize a retry of that call.
|
|
1775
|
+
Task requests (`tasks/get`, `tasks/update`, `tasks/cancel`) carry no key; the
|
|
1776
|
+
task ID names the call there.
|
|
1777
|
+
|
|
1778
|
+
A declared tool, as `tools/list` returns it:
|
|
1779
|
+
|
|
1780
|
+
```json
|
|
1781
|
+
{
|
|
1782
|
+
"name": "send_payment",
|
|
1783
|
+
"description": "Send a payment.",
|
|
1784
|
+
"inputSchema": {"type": "object", "properties": {"amount": {"type": "integer"}}},
|
|
1785
|
+
"annotations": {"idempotentHint": true},
|
|
1786
|
+
"_meta": {
|
|
1787
|
+
"io.superdurable.dex/effect": "idempotent_write",
|
|
1788
|
+
"io.superdurable.dex/approval": "recommended"
|
|
1789
|
+
}
|
|
1790
|
+
}
|
|
1791
|
+
```
|
|
1792
|
+
|
|
1793
|
+
A result whose outcome is unknown:
|
|
1794
|
+
|
|
1795
|
+
```json
|
|
1796
|
+
{
|
|
1797
|
+
"content": [{"type": "text", "text": "the payment provider timed out"}],
|
|
1798
|
+
"isError": true,
|
|
1799
|
+
"_meta": {"io.superdurable.dex/outcome": "unknown"}
|
|
1800
|
+
}
|
|
1801
|
+
```
|
|
1802
|
+
|
|
1381
1803
|
### With a framework adapter
|
|
1382
1804
|
|
|
1383
1805
|
The runner above is the core's runner-agnostic path. A framework adapter uses
|
|
1384
1806
|
the source when it accepts tools defined by a JSON Schema: wrap each `McpTool`
|
|
1385
1807
|
as a framework tool that calls `tools.callTool(name, input, { idempotencyKey })`
|
|
1386
1808
|
with the key from the adapter's tool context, and pass
|
|
1387
|
-
`
|
|
1809
|
+
`tools.mergeToolPolicies(yourPolicies)` as the adapter's `toolPolicies`.
|
|
1388
1810
|
No adapter package changes for this.
|
|
1389
1811
|
|
|
1390
1812
|
| Adapter | Schema-only tool | Key inside the tool |
|
|
@@ -1399,6 +1821,136 @@ Steps; use the source there for the shared idempotency key and annotation
|
|
|
1399
1821
|
handling. The OpenAI Agents SDK runs its own MCP servers inline in `RunModel`,
|
|
1400
1822
|
so a wrapper is its only path to tool Steps.
|
|
1401
1823
|
|
|
1824
|
+
## Connector tools (`@superdurable/dex-agents/connectors`)
|
|
1825
|
+
|
|
1826
|
+
`@superdurable/dex-agents/connectors` offers reviewed Connector operations to a
|
|
1827
|
+
built-in loop agent as tools. `ConnectorToolSource` implements `ToolSource`, like
|
|
1828
|
+
`McpToolSource`. Each call runs one operation in its own tool Step through the
|
|
1829
|
+
Connector SDK's `runQuery` or `runMutation`, so the connector's Call ID,
|
|
1830
|
+
idempotency key, credential handling, and single-dispatch fence apply. Install
|
|
1831
|
+
the Connector SDK, an optional peer dependency, and each Connector package the
|
|
1832
|
+
agent uses:
|
|
1833
|
+
|
|
1834
|
+
```bash
|
|
1835
|
+
npm i @superdurable/connectors-sdk @superdurable/connector-stripe
|
|
1836
|
+
```
|
|
1837
|
+
|
|
1838
|
+
The subpath needs Connector SDK 0.9.0, the first release that exports
|
|
1839
|
+
`throwDexRetry`, and Connector packages that export `./dex-connector.json`.
|
|
1840
|
+
|
|
1841
|
+
```ts
|
|
1842
|
+
import manifest from "@superdurable/connector-stripe/dex-connector.json" with { type: "json" };
|
|
1843
|
+
import { CreateRefundInputSchema, newProjectConnection } from "@superdurable/connector-stripe";
|
|
1844
|
+
import { ConnectorToolSource, connectorTool } from "@superdurable/dex-agents/connectors";
|
|
1845
|
+
|
|
1846
|
+
const stripe = newProjectConnection(project, "billing-stripe");
|
|
1847
|
+
const source = new ConnectorToolSource([
|
|
1848
|
+
connectorTool(stripe.operations.createRefund(), stripe.reference, {
|
|
1849
|
+
manifest,
|
|
1850
|
+
inputValidator: CreateRefundInputSchema,
|
|
1851
|
+
requiresApproval: (args) => Number(args["amount"]) > 5000,
|
|
1852
|
+
}),
|
|
1853
|
+
]);
|
|
1854
|
+
```
|
|
1855
|
+
|
|
1856
|
+
Nothing is exposed by default. `connectorTool` takes the package's parsed
|
|
1857
|
+
`dex-connector.json` and the input validator the package exports, and refuses
|
|
1858
|
+
an operation whose entry lacks `agent.exposed: true`, an `agent` value it does
|
|
1859
|
+
not know, an `agent.branches` key that is not an optional branch (or an
|
|
1860
|
+
`unknown` one on a Query), or an `inputSchema` keyword it cannot check. The
|
|
1861
|
+
tool name defaults to `<connectorId>_<operation in snake case>`; the
|
|
1862
|
+
description and argument schema come from the manifest. The snake case is
|
|
1863
|
+
connectorctl's, which also names the Python Step factories: an acronym run
|
|
1864
|
+
stays one word, so `listStoryIDs` becomes `list_story_ids`.
|
|
1865
|
+
`agents/conformance/connector_tools.json` pins it. A Mutation is any
|
|
1866
|
+
operation with an `idempotencyKey` method.
|
|
1867
|
+
|
|
1868
|
+
| Operation | Effect | Tool Step | Retries |
|
|
1869
|
+
| --- | --- | --- | --- |
|
|
1870
|
+
| Query | `read` | `CallTool` | The operation's Execute retry policy, at most three attempts |
|
|
1871
|
+
| Mutation with `agent.retry: safe` | `idempotent_write` | `CallTool` | The same |
|
|
1872
|
+
| Any other Mutation | `non_idempotent_write` | `CallSingleAttemptTool` | None |
|
|
1873
|
+
|
|
1874
|
+
The source gives each call's tool Step the operation's `executeMethodTimeoutMs`
|
|
1875
|
+
and retry policy through `toolCallStepOptions`, a movement-scoped option. A
|
|
1876
|
+
session `toolPolicies` entry for a connector tool changes only its approval;
|
|
1877
|
+
the effect always comes from the manifest. Approval follows
|
|
1878
|
+
`agent.approval: recommended`; `requiresApproval` overrides it with a boolean
|
|
1879
|
+
or a per-call predicate over the decoded arguments, and a predicate that throws
|
|
1880
|
+
asks for approval.
|
|
1881
|
+
|
|
1882
|
+
A call checks the arguments against `inputSchema`, then decodes them with the
|
|
1883
|
+
input validator; either failure is a `ToolExecutionError` the model reads, and
|
|
1884
|
+
the operation never runs. The Call ID comes from
|
|
1885
|
+
`deriveCallID(flowId, stepExecutionId, operation)` over
|
|
1886
|
+
`DeferredToolExecution.context`, the same on every attempt. The operation sees
|
|
1887
|
+
a `ConnectorStepContext`, which stores each heartbeat value, such as the
|
|
1888
|
+
fence's marker, as the tool Step's text checkpoint in the Python core's JSON
|
|
1889
|
+
form, so the session's heartbeats repeat it and the next attempt reads it
|
|
1890
|
+
back. A `RetryError` goes through `throwDexRetry`, except for a single-attempt write
|
|
1891
|
+
whose fence saved `notSent` or `canResend` in this attempt: that records `failed` with
|
|
1892
|
+
`connectorNotSentRetryLater` instead of `outcome_unknown`.
|
|
1893
|
+
|
|
1894
|
+
The first matching row wins. The rows before the listing can only make an
|
|
1895
|
+
outcome unknown:
|
|
1896
|
+
|
|
1897
|
+
| Branch | Record | The model reads |
|
|
1898
|
+
| --- | --- | --- |
|
|
1899
|
+
| `uncertain`, whatever the listing or the required branches say | `outcome_unknown` (`ToolOutcomeUnknownError`) | `outcomeUnknownToolResult` |
|
|
1900
|
+
| A Mutation's `defect`, whatever the listing or the required branches say | `outcome_unknown` | The same text |
|
|
1901
|
+
| A required (success) branch other than `defect` | `succeeded` | The value as compact JSON, keys sorted |
|
|
1902
|
+
| Any Mutation branch when its fence cannot prove that nothing was sent | `outcome_unknown` | The same text |
|
|
1903
|
+
| Listed `rejected` under `agent.branches` | `failed` (`ToolExecutionError`) | `<connector> <operation> returned <branch>: <kind>: <message>` |
|
|
1904
|
+
| Listed `unknown`, or a Mutation branch not listed | `outcome_unknown` | `outcomeUnknownToolResult` |
|
|
1905
|
+
| A Query branch not listed, including `defect` | `failed` | `<connector> <operation> returned <branch>: <kind>: <message>` |
|
|
1906
|
+
|
|
1907
|
+
A Mutation's `defect` is not always local: the Connector SDK selects it for
|
|
1908
|
+
some attempts that already called the provider, such as an invalid attempt or
|
|
1909
|
+
an undeclared branch, and the source cannot tell those apart. A Query's
|
|
1910
|
+
`defect` follows its listing.
|
|
1911
|
+
|
|
1912
|
+
A Mutation that uses the single-dispatch fence gets one more check, the same
|
|
1913
|
+
one the Go Connector SDK makes for Go agent hosts. The source observes every
|
|
1914
|
+
fence marker the attempt saves through its Step context, bound to the call's
|
|
1915
|
+
Call ID and attempt. Once the fence claims a send, or records a provider ID,
|
|
1916
|
+
the attempt may have sent. Only a later `canResend` marker of the same call and
|
|
1917
|
+
attempt, after a claim without a provider ID, lowers that, as the fence's own
|
|
1918
|
+
`recordCanResend` does. The operation's own later checkpoints never lower it.
|
|
1919
|
+
On a retry that reads the earlier checkpoint, a missing checkpoint, one that is
|
|
1920
|
+
not a fence marker, a malformed one, another call's, or one that does not allow
|
|
1921
|
+
this attempt a send also means the request may have been sent. While the
|
|
1922
|
+
request may have been sent, every branch but the success branch records outcome
|
|
1923
|
+
unknown, whatever the listing says, and a Retry is never proof that nothing was
|
|
1924
|
+
sent. The core cannot see inside the fence, so it reads the markers instead of
|
|
1925
|
+
the fence's own events: a forged `canResend` marker of the same call would
|
|
1926
|
+
count. A Mutation that never reads its checkpoint and never saves a fence
|
|
1927
|
+
marker gets no such check and relies on its listing.
|
|
1928
|
+
`agents/conformance/connector_tools.json` pins these cases in
|
|
1929
|
+
`fenceObservationCases`.
|
|
1930
|
+
|
|
1931
|
+
A result over `maxResultBytes` (default `defaultConnectorResultMaxBytes`,
|
|
1932
|
+
64 KiB of UTF-8 text, set per source or per tool) is cut, followed by
|
|
1933
|
+
`connectorResultTruncatedNote`, and `AgentToolCall.truncatedOutputBytes` holds
|
|
1934
|
+
the full size.
|
|
1935
|
+
|
|
1936
|
+
Writes are ordered: the source marks every connector write
|
|
1937
|
+
`DeferredToolCall.ordered`, so the writes of one model response run one at a
|
|
1938
|
+
time in the model's order while reads run in parallel. `RunModel` starts the
|
|
1939
|
+
next queued write (`AgentToolCall.writeQueued`) before any model call and raises
|
|
1940
|
+
its approval only then. Once a write is not `succeeded`, the response's later
|
|
1941
|
+
writes record `failed` with `skipped: true` and the error
|
|
1942
|
+
`` `${toolCallNotRunErrorPrefix}: ${toolCallSkippedAfterWriteDetail}` ``.
|
|
1943
|
+
Steering and `cancel_turn` cancel queued writes as they cancel unstarted calls.
|
|
1944
|
+
|
|
1945
|
+
The Vercel AI, OpenAI Agents, and Mastra adapters take tool sources, such as
|
|
1946
|
+
`ConnectorToolSource`, in their own `toolSources` option; see each adapter's
|
|
1947
|
+
README. An adapter serves them through `ToolSourceRouter`, exported from the
|
|
1948
|
+
package root: it lists the sources in each `RunModel`, refuses name
|
|
1949
|
+
collisions, lets the source build each deferred call, so the effect, the
|
|
1950
|
+
approval, and the write order come from the source, and routes the tool Step,
|
|
1951
|
+
its Step options, and polls back to that source. The built-in loop runner uses it
|
|
1952
|
+
too.
|
|
1953
|
+
|
|
1402
1954
|
## UI streams
|
|
1403
1955
|
|
|
1404
1956
|
`@superdurable/dex-agents/ui-stream` lets a browser render a live session
|
|
@@ -1624,7 +2176,7 @@ The snapshot JSON is the `get_agent_snapshot` wire format without the fields
|
|
|
1624
2176
|
only the session's Steps use. Messages leave out `framework_item`, the
|
|
1625
2177
|
framework's replay data, which can hold the agent's instructions. Tool calls
|
|
1626
2178
|
leave out `tool_source_reference`, `trace_context`, `input_continuation`,
|
|
1627
|
-
`input_answers`, and `
|
|
2179
|
+
`input_answers`, `task_answer_pending`, and `runner_tool_state`. Read them with `getAgentSnapshot` on the server when needed. It
|
|
1628
2180
|
includes `session_settings` as stored, so keep anything a browser must not see out of session settings, or
|
|
1629
2181
|
remove the key.
|
|
1630
2182
|
|
|
@@ -1788,7 +2340,7 @@ real Dex Server with no model key:
|
|
|
1788
2340
|
| `AgentSessionTestClient` | Typed RPC calls (`resolveInterrupt` retries lock conflicts) and Worker-free waits on committed Attributes |
|
|
1789
2341
|
| `readStepHistory(serverAddress, flowId)` | Completed Step methods from `dexcli flow history`, to assert a turn's Step shape |
|
|
1790
2342
|
| `FileGate`, `InvocationLog` | Pause Worker code at an exact point; count real tool runs across processes |
|
|
1791
|
-
| `ScriptedModelClient` |
|
|
2343
|
+
| `ScriptedModelClient` | Built-in loop `ModelClient` that answers from a script, like the Python core's; the same class as `@superdurable/dex-agents/builtin-loop` exports. See [Demo model without a key](#demo-model-without-a-key) |
|
|
1792
2344
|
| `pollUntil(probe, options)` | Deadline polling where the Client has no direct wait |
|
|
1793
2345
|
| `findFreeLoopbackPorts(count)` | Distinct free loopback ports, chosen with every listener held open until all are picked |
|
|
1794
2346
|
|
|
@@ -1881,7 +2433,7 @@ use a scripted runner (`test/fixtures/scripted-agent-runner.ts`) and cover:
|
|
|
1881
2433
|
- two tools of one response that ask at once, both answerable, and a sibling
|
|
1882
2434
|
approval resolved while a tool runs that never reappears as pending.
|
|
1883
2435
|
|
|
1884
|
-
The
|
|
2436
|
+
The built-in loop tests (`test/integ/stepwise-*.integration.test.ts`) run the
|
|
1885
2437
|
built-in loop on the AI SDK's `MockLanguageModelV4`, scripted from its prompt
|
|
1886
2438
|
(`test/fixtures/stepwise-scripted-language-model.ts`), and cover:
|
|
1887
2439
|
|
|
@@ -2013,6 +2565,7 @@ built-in loop on the AI SDK's `MockLanguageModelV4`, scripted from its prompt
|
|
|
2013
2565
|
attempts, on every `listTools` call until it answers. A server that binds
|
|
2014
2566
|
tasks to its protocol session, such as a stateful server on the SDK's task
|
|
2015
2567
|
store, cannot resume a task across attempts; the retry calls the tool again.
|
|
2568
|
+
- Images: the media stores keep no local read cache, so each `loadImages` reads the store.
|
|
2016
2569
|
- Tracing covers the session's Steps only. The Dex SDK emits no spans, so Dex
|
|
2017
2570
|
Server and Worker internals do not appear in the traces.
|
|
2018
2571
|
- The adapters report only the model information their framework exposes, so
|