@statelyai/agent 2.0.0-alpha.34 → 2.0.0-alpha.36
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +124 -0
- package/dist/act-B9hLQls6.mjs +19 -0
- package/dist/act-CrX0JwWY.cjs +42 -0
- package/dist/ai-sdk.cjs +63 -7
- package/dist/ai-sdk.d.cts +13 -3
- package/dist/ai-sdk.d.mts +13 -3
- package/dist/ai-sdk.mjs +63 -7
- package/dist/{decision-BEsATcr1.cjs → decision-CMeuNZ1g.cjs} +262 -4
- package/dist/{decision-COkJNjOW.mjs → decision-m5hGmHpF.mjs} +239 -5
- package/dist/decision-object-B5zGSPFp.d.mts +15 -0
- package/dist/decision-object-DBcLWVvv.d.cts +15 -0
- package/dist/{run-agent-s5D32w99.cjs → effects-B23bFZEs.cjs} +948 -102
- package/dist/{run-agent-Ct_GZw_f.mjs → effects-CuOUNYtZ.mjs} +914 -104
- package/dist/{event-log-C7jeH0y8.mjs → event-log-DkVlk3Rh.mjs} +30 -15
- package/dist/{event-log-BWSSLUff.cjs → event-log-Dkc-yIYu.cjs} +41 -14
- package/dist/{event-log-store-BzXehP7R.d.cts → event-log-store-BBS8so8a.d.mts} +2 -2
- package/dist/{event-log-store-W49TCJXi.mjs → event-log-store-Brx9tOHt.mjs} +1 -1
- package/dist/{event-log-store-CleWpsp5.cjs → event-log-store-Cl-YWDnq.cjs} +1 -1
- package/dist/{event-log-store-CWjN6lmM.d.mts → event-log-store-bc4ICGcS.d.cts} +2 -2
- package/dist/index.cjs +660 -286
- package/dist/index.d.cts +168 -9
- package/dist/index.d.mts +168 -9
- package/dist/index.mjs +639 -271
- package/dist/internal-C3Lqu538.d.cts +43 -0
- package/dist/internal-DUcdhk-i.d.mts +43 -0
- package/dist/log.cjs +2 -2
- package/dist/log.d.cts +1 -1
- package/dist/log.d.mts +1 -1
- package/dist/log.mjs +2 -2
- package/dist/machines.cjs +457 -41
- package/dist/machines.d.cts +114 -43
- package/dist/machines.d.mts +114 -43
- package/dist/machines.mjs +457 -42
- package/dist/{model-source-DmhTMK39.cjs → model-source-B4WQ8Idp.cjs} +1 -1
- package/dist/{model-source-CK3F4YuU.mjs → model-source-kjGwqAV8.mjs} +1 -1
- package/dist/openai.cjs +149 -36
- package/dist/openai.d.cts +28 -10
- package/dist/openai.d.mts +28 -10
- package/dist/openai.mjs +149 -36
- package/dist/otel.d.cts +1 -1
- package/dist/otel.d.mts +1 -1
- package/dist/{run-agent-BGdx5913.d.mts → run-agent-BcQ-9NRT.d.cts} +91 -9
- package/dist/{run-agent-B1-Eqviq.d.cts → run-agent-C86drZtN.d.mts} +91 -9
- package/dist/{setup-agent-05uw4dPZ.mjs → setup-agent-BDmNr6PC.mjs} +9 -3
- package/dist/{setup-agent-BtfeFfF6.d.cts → setup-agent-BIKcbVXV.d.mts} +4 -4
- package/dist/{setup-agent-CXEBURdo.cjs → setup-agent-CmEToB8y.cjs} +9 -3
- package/dist/{setup-agent-C6nqzv_X.d.mts → setup-agent-jlo4vtaZ.d.cts} +4 -4
- package/dist/tanstack-ai.cjs +62 -4
- package/dist/tanstack-ai.d.cts +18 -6
- package/dist/tanstack-ai.d.mts +18 -6
- package/dist/tanstack-ai.mjs +62 -4
- package/dist/testing.cjs +536 -67
- package/dist/testing.d.cts +197 -180
- package/dist/testing.d.mts +197 -180
- package/dist/testing.mjs +532 -67
- package/dist/{text-logic-CV_4KGr6.d.cts → text-logic-BqxalgMr.d.mts} +87 -4
- package/dist/{text-logic-Ckh-rJFV.d.mts → text-logic-CSf9bqpP.d.cts} +87 -4
- package/dist/{types-Dp5_Ru4R.d.cts → types-DQUOOaZO.d.cts} +1 -1
- package/dist/{types-Dp5_Ru4R.d.mts → types-DQUOOaZO.d.mts} +1 -1
- package/dist/{usage-BDiJKyv9.d.cts → usage-DoP0Oi84.d.mts} +1 -1
- package/dist/{usage-DOoVUGVq.d.mts → usage-HtnrUsl9.d.cts} +1 -1
- package/dist/utils-7lSa4H-i.d.cts +403 -0
- package/dist/utils-CBbKw2fM.d.mts +403 -0
- package/dist/validate.cjs +44 -12
- package/dist/validate.d.cts +6 -6
- package/dist/validate.d.mts +6 -6
- package/dist/validate.mjs +44 -12
- package/dist/verify-Bink-Drw.cjs +2304 -0
- package/dist/{verify-BQpE8hUz.d.cts → verify-Buh8XSUr.d.mts} +208 -14
- package/dist/{verify-CqeuEmIk.d.mts → verify-CciG8UP-.d.cts} +208 -14
- package/dist/verify-Dpz1u5Pe.mjs +2251 -0
- package/dist/with-model-calls-B_WLmOSl.cjs +248 -0
- package/dist/with-model-calls-BkOwdroS.mjs +213 -0
- package/docs/act.md +102 -0
- package/docs/advanced.md +105 -0
- package/docs/any-stack.md +62 -0
- package/docs/choosing-a-run-mode.md +55 -0
- package/docs/debugging.md +29 -0
- package/docs/decisions.md +473 -0
- package/docs/environment-and-policy.md +188 -0
- package/docs/evals.md +509 -0
- package/docs/event-log.md +311 -0
- package/docs/from-a-loop.md +528 -0
- package/docs/generate-machines.md +110 -0
- package/docs/hosts.md +414 -0
- package/docs/human-in-the-loop.md +177 -0
- package/docs/index.md +48 -0
- package/docs/judgments.md +281 -0
- package/docs/langgraph-comparison.md +198 -0
- package/docs/machines-as-data.md +454 -0
- package/docs/machines-presets.md +198 -0
- package/docs/machines.md +497 -0
- package/docs/messages.md +67 -0
- package/docs/meta.json +52 -0
- package/docs/models-and-providers.md +268 -0
- package/docs/multi-agent.md +68 -0
- package/docs/observability.md +109 -0
- package/docs/patterns.md +31 -0
- package/docs/persistence.md +136 -0
- package/docs/quickstart.md +121 -0
- package/docs/roadmap.md +22 -0
- package/docs/scope.md +75 -0
- package/docs/statechart-policy-examples.md +69 -0
- package/docs/steps.md +54 -0
- package/docs/strangler-fig.md +323 -0
- package/docs/text-requests.md +250 -0
- package/docs/thinking-in-state-machines.md +324 -0
- package/docs/tools.md +249 -0
- package/docs/upgrading.md +37 -0
- package/docs/usage-and-budgets.md +516 -0
- package/docs/validate-and-repair.md +253 -0
- package/docs/verify.md +487 -0
- package/docs/xstate-4-and-5-apps.md +161 -0
- package/llms.txt +53 -0
- package/package.json +10 -3
- package/readme.md +30 -15
- package/skills/add-evals/SKILL.md +3 -1
- package/skills/add-traces/SKILL.md +6 -1
- package/skills/build-machine/SKILL.md +81 -0
- package/skills/generate-machine/SKILL.md +1 -1
- package/skills/graph-engineering/SKILL.md +13 -3
- package/skills/improve-machine/SKILL.md +21 -2
- package/skills/loop-to-machine/SKILL.md +147 -190
- package/skills/loop-to-machine/reference.md +260 -0
- package/dist/utils-BnXdWih3.d.mts +0 -108
- package/dist/utils-ebVpCYoF.d.cts +0 -108
- package/dist/verify-Bpahmr34.mjs +0 -1241
- package/dist/verify-DCLZa9R0.cjs +0 -1288
- package/dist/with-model-calls-BXYdBdcu.cjs +0 -45
- package/dist/with-model-calls-Bv4V4EHC.mjs +0 -34
|
@@ -0,0 +1,528 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Migrating from a hand-rolled loop
|
|
3
|
+
description: Convert a while-loop agent into an agent machine one step at a time, then improve a factory loop's triage from its traces.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
> **Alpha:** `@statelyai/agent` 2.0 is in alpha. APIs can change between releases; pin an exact version. Feedback: [github.com/statelyai/agent](https://github.com/statelyai/agent/issues).
|
|
7
|
+
|
|
8
|
+
This page ports a working `while`-loop agent to an agent machine without rewriting your model calls. Your SDK calls, tools, and retry logic become the [executors](hosts.md). The machine replaces only the control flow.
|
|
9
|
+
|
|
10
|
+
Part 1 (Steps 1–5) ports a support agent and ends on the tests the port makes possible. [Part 2](#part-2-a-factory-loop-that-improves) (Steps 6–11) ports a software-factory loop and improves its triage from traces without loosening its rules.
|
|
11
|
+
|
|
12
|
+
Part 1's code is in `examples/port-walkthrough`:
|
|
13
|
+
|
|
14
|
+
- [`before.ts`](https://github.com/statelyai/agent/blob/next/examples/port-walkthrough/before.ts): the loop, runnable with `OPENAI_API_KEY=... npx tsx examples/port-walkthrough/before.ts`.
|
|
15
|
+
- [`index.ts`](https://github.com/statelyai/agent/blob/next/examples/port-walkthrough/index.ts): the machine, and a `runSupportAgent` with the loop's signature and result type.
|
|
16
|
+
- [`index.test.ts`](https://github.com/statelyai/agent/blob/next/examples/port-walkthrough/index.test.ts): four tests that run without an API key.
|
|
17
|
+
|
|
18
|
+
The loop's model call is this:
|
|
19
|
+
|
|
20
|
+
```ts no-check
|
|
21
|
+
const { toolCalls, text, response } = await generateText({
|
|
22
|
+
// Model IDs here are illustrative; substitute your provider's current models.
|
|
23
|
+
model: openai("gpt-6-luna"),
|
|
24
|
+
messages,
|
|
25
|
+
tools: { search, sendEmail },
|
|
26
|
+
});
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
The port's call site is this:
|
|
30
|
+
|
|
31
|
+
```ts no-check
|
|
32
|
+
const result = await runAgent(supportLoopMachine, {
|
|
33
|
+
input: { request },
|
|
34
|
+
executors,
|
|
35
|
+
});
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
The `generateText` call still exists. It moves into the executors, and the loop around it becomes the machine. The result carries `result.output`, the live `result.snapshot`, native `result.persist()`, and aggregated `result.usage`.
|
|
39
|
+
|
|
40
|
+
To move a workflow over one phase at a time instead, with the original running unchanged until each phase is carved, read [Strangler-fig retrofit](strangler-fig.md).
|
|
41
|
+
|
|
42
|
+
For the design work behind this refactor, read [Thinking in state machines](thinking-in-state-machines.md). It covers how to find the states in a loop before you write any of them down.
|
|
43
|
+
|
|
44
|
+
## What stays the same, what changes
|
|
45
|
+
|
|
46
|
+
The model work moves into executors unchanged. The control flow around it changes in six places.
|
|
47
|
+
|
|
48
|
+
| Area | In the loop | With `runAgent` | Why | Read more |
|
|
49
|
+
| ------------------------ | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
|
|
50
|
+
| Same: SDK call | `generateText` with your options | The same call inside an executor, written by hand or through an adapter such as `createAiSdkExecutors` | Executors are host code; the machine never calls a provider | [Hosts](hosts.md) |
|
|
51
|
+
| Same: tools | The SDK runs the tool-call loop | The request's `tools` reach the executor, and the SDK still runs the tool-call loop. With the AI SDK adapter, `maxSteps` becomes `stopWhen` | A tool loop inside one state is one request | [Hosts](hosts.md#ai-sdk-adapter) |
|
|
52
|
+
| Same: messages | Your message array | A request's `messages` reach the executor without conversion | Messages stay framework-native | [Messages](messages.md) |
|
|
53
|
+
| Same: retries and errors | Your retry and `try`/`catch` code | The same code inside the executor. Core does not retry a text request | Retry policy belongs to the SDK or the executor | [Hosts](hosts.md#executor-owned-structured-retries) |
|
|
54
|
+
| Same: model | A provider model object | The same object, registered in `models` and named by key in the request | The machine names a model; the host resolves it | [Models and providers](models-and-providers.md) |
|
|
55
|
+
| Same: token usage | Read off each response | Each executor returns `usage`; `result.usage` sums it for the run, plus `modelCalls` | Core reads only what the executor reports | [Usage and budgets](usage-and-budgets.md#the-usage-result) |
|
|
56
|
+
| Changes: human wait | `await approve(...)` blocks the process | The run settles `{ status: "idle" }`. Resume with `runAgent(machine, { snapshot, event })`, or pass `onIdle` to keep the blocking shape | A pause can outlive the process; the persisted snapshot is plain JSON | [Human in the loop](human-in-the-loop.md) |
|
|
57
|
+
| Changes: call cap | Your own step counter, or none | `maxModelCalls` defaults to `100`. Pass a higher value to raise it. The overrun reaches the invoke's `onError`, or settles `{ status: "error", cause: "max-model-calls" }` | A runaway loop stops at a fixed budget | [Usage and budgets](usage-and-budgets.md#the-global-backstop-maxmodelcalls) |
|
|
58
|
+
| Changes: timers | `setTimeout` or a sleep in the loop | Root `after` timers run in process by default, and the run waits for them. A durable host passes `timers: { schedule, cancel }` and delivers `{ type: "xstate.timer", id }` when one fires | A timer can outlive the process | [The runtime loop](steps.md#host-owned-timers) |
|
|
59
|
+
| Changes: unknown events | Your code decides: a `switch` falls through or throws | An event the current state does not handle is ignored. The run settles normally and returns the event as `result.ignored` | A stale or duplicate answer is not a failure | [Persistence](persistence.md#resume-with-an-event-off-the-wire) |
|
|
60
|
+
| Changes: input | Passed to the first prompt unchecked | Validated against the machine's input schema before the run starts. Invalid input throws `AgentError` with code `invalid-machine-input` | Bad input fails before any model call | [Machines](machines.md#schema-declarations) |
|
|
61
|
+
| Changes: executor errors | A thrown SDK error rejects the loop's promise | The error goes to the invoke's `onError`. With no `onError`, the run settles `{ status: "error", cause: "machine" }` with the error and the snapshot | The machine routes failures as transitions; the result keeps the snapshot | [Machines](machines.md#request-and-actor-invokes) |
|
|
62
|
+
|
|
63
|
+
## Starting point: a hand-rolled loop
|
|
64
|
+
|
|
65
|
+
The starting point is a support agent written as a `while` loop against the AI SDK. The model calls `search` freely, and it can call `sendEmail`, which a human must approve. The full loop is in [`before.ts`](https://github.com/statelyai/agent/blob/next/examples/port-walkthrough/before.ts). Its control flow is these lines:
|
|
66
|
+
|
|
67
|
+
```ts no-check
|
|
68
|
+
if (steps++ >= 8) return { sent, stopped: "step-limit" };
|
|
69
|
+
// ...
|
|
70
|
+
if (!toolCalls?.length) return { sent, reply: text };
|
|
71
|
+
|
|
72
|
+
for (const call of toolCalls) {
|
|
73
|
+
if (call.toolName === "sendEmail") {
|
|
74
|
+
if (!(await approve(call.input))) return { sent, pending: true }; // loop state is lost here
|
|
75
|
+
sent = true;
|
|
76
|
+
}
|
|
77
|
+
}
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
The loop returns one of three shapes. `before.ts` names them so the port can be checked against them:
|
|
81
|
+
|
|
82
|
+
```ts
|
|
83
|
+
export type SupportResult =
|
|
84
|
+
| { sent: boolean; stopped: "step-limit" }
|
|
85
|
+
| { sent: boolean; reply: string }
|
|
86
|
+
| { sent: boolean; pending: true };
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
This loop has four problems:
|
|
90
|
+
|
|
91
|
+
- `generateText` runs a tool's `execute` in the same call that returns `toolCalls`. By the time the loop asks `approve`, `sendEmail` has already run.
|
|
92
|
+
- The approval rule is an `if` statement inside the tool loop, so no test can list the paths around it.
|
|
93
|
+
- The human pause returns `{ pending: true }` and discards the loop's state, so the run cannot resume.
|
|
94
|
+
- The step cap is a local counter. A test reaches it only by spending eight model calls.
|
|
95
|
+
|
|
96
|
+
## Step 1: implicit phases as explicit states
|
|
97
|
+
|
|
98
|
+
Each line of the loop becomes a state in [`index.ts`](https://github.com/statelyai/agent/blob/next/examples/port-walkthrough/index.ts):
|
|
99
|
+
|
|
100
|
+
| In the loop | In the machine |
|
|
101
|
+
| ---------------------------------------------- | -------------------------------------------------- |
|
|
102
|
+
| `if (steps++ >= 8)` | The `checkingSteps` choice state |
|
|
103
|
+
| `generateText` with `search` | The `assisting` state, which invokes `assist` |
|
|
104
|
+
| `if (!toolCalls?.length)` | The `deciding` state, which invokes `agent.decide` |
|
|
105
|
+
| `await approve(call.input)` | The idle `awaitingApproval` state |
|
|
106
|
+
| `sendEmail`'s `execute` | The `sending` state, which invokes `sendEmail` |
|
|
107
|
+
| `return { sent, reply }`, `pending`, `stopped` | The final states `replied`, `declined`, `failed` |
|
|
108
|
+
|
|
109
|
+
The loop's first line becomes the machine's initial state. A choice state compares the counter against an exported constant:
|
|
110
|
+
|
|
111
|
+
```ts no-check
|
|
112
|
+
// The loop's first line: `if (steps++ >= 8) return { sent, stopped: "step-limit" }`.
|
|
113
|
+
checkingSteps: {
|
|
114
|
+
type: "choice",
|
|
115
|
+
choice: ({ context }) =>
|
|
116
|
+
context.steps >= MAX_STEPS ? { target: "failed" } : { target: "assisting" },
|
|
117
|
+
},
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
`setupAgent` declares the context, events, request, and actor. Its `output` schema lists the loop's three returns, one per final state:
|
|
121
|
+
|
|
122
|
+
```ts no-check
|
|
123
|
+
// The loop's three returns, one per final state.
|
|
124
|
+
output: z.union([
|
|
125
|
+
z.object({ sent: z.boolean(), stopped: z.literal("step-limit") }),
|
|
126
|
+
z.object({ sent: z.boolean(), reply: z.string() }),
|
|
127
|
+
z.object({ sent: z.boolean(), pending: z.literal(true) }),
|
|
128
|
+
]),
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
A tool the machine must gate is not a tool any more. `sendEmail` becomes an actor the machine invokes, written with XState's `createAsyncLogic`. `createAsyncLogic` takes `{ schemas, run }`. Annotate `run`'s return type when it resolves an array or another literal; the schema type parameters are `const`, so an unannotated array literal infers as a readonly tuple.
|
|
132
|
+
|
|
133
|
+
<!-- viz: support loop machine: checkingSteps -> assisting (text request with the search tool) -> deciding (agent.decide) -> awaitingApproval (SEND_EMAIL) -> sending (APPROVE) -> checkingSteps; deciding -> replied (REPLY); deciding -> checkingSteps (CONTINUE); awaitingApproval -> declined (REJECT); checkingSteps -> failed (MAX_STEPS) -->
|
|
134
|
+
|
|
135
|
+
## Step 2: free tools stay tools, the gated action becomes a decision
|
|
136
|
+
|
|
137
|
+
A converted loop splits its tools in two, and the split is the whole refactor:
|
|
138
|
+
|
|
139
|
+
| In the loop | In the machine |
|
|
140
|
+
| ------------------------------------- | ---------------------------------------------------------------------------------------------------- |
|
|
141
|
+
| A tool with no side effect you gate | Stays a tool on a [text request](text-requests.md). The host runs the tool loop inside one state. |
|
|
142
|
+
| A tool a human or a rule must approve | Becomes an event the model proposes through `agent.decide`, plus a state that invokes the real work. |
|
|
143
|
+
|
|
144
|
+
> **Note:** This walkthrough changes `sendEmail`'s call shape in the first port, so one page shows the finished split. On your own code, port the call unchanged first: keep `sendEmail` a tool on the text request, guard the returned call's shape, and prove the port matches the loop (a tool with `execute` still runs inside the SDK before the request returns, so that port keeps the loop's timing; the approval gate arrives with the decision step). Then turn it into a decision in a separate step, and list the differences that step causes as intended. Changing both at once leaves a behavior difference with two possible causes. The `loop-to-machine` skill follows that order.
|
|
145
|
+
|
|
146
|
+
`search` is the first kind. `index.ts` imports it from `before.ts` and passes it to the `assist` request as `tools: { search }`. The machine stays in `assisting` for every search turn and sees one `onDone`.
|
|
147
|
+
|
|
148
|
+
`sendEmail` is the second kind. The model no longer calls it. In `deciding`, the model chooses one of three events: `REPLY` ends the run, `CONTINUE` starts another round, and `SEND_EMAIL` proposes a draft:
|
|
149
|
+
|
|
150
|
+
```ts no-check
|
|
151
|
+
on: {
|
|
152
|
+
REPLY: { target: "replied" },
|
|
153
|
+
CONTINUE: { target: "checkingSteps" },
|
|
154
|
+
SEND_EMAIL: ({ event }) => ({
|
|
155
|
+
target: "awaitingApproval",
|
|
156
|
+
context: { draft: { to: event.to, subject: event.subject, body: event.body } },
|
|
157
|
+
}),
|
|
158
|
+
},
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
A tool-carrying request and a decision are two different states. The request is one model call that may run many tool turns inside the host. The decision is one model call that returns exactly one event the current state accepts.
|
|
162
|
+
|
|
163
|
+
Rules that were `if` statements can move into the transition instead of a separate state. A transition function that returns `undefined` rejects the choice, and the decision retries with typed feedback. See [transitions](machines.md#transitions) and [Decisions](decisions.md).
|
|
164
|
+
|
|
165
|
+
## Step 3: the pause as an idle state
|
|
166
|
+
|
|
167
|
+
The loop's `return { sent, pending: true }` becomes `awaitingApproval`, a state with no invoke. `runAgent` settles as `idle` in that state instead of discarding the run. The snapshot is plain JSON, so you can persist it anywhere.
|
|
168
|
+
|
|
169
|
+
`APPROVE` is the only way into `sending`, and it is the only writer of `approved`:
|
|
170
|
+
|
|
171
|
+
```ts no-check
|
|
172
|
+
on: {
|
|
173
|
+
APPROVE: () => ({ target: "sending", context: { approved: true } }),
|
|
174
|
+
REJECT: { target: "declined" },
|
|
175
|
+
},
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
`REJECT` lands in `declined`, whose output is the loop's `{ sent, pending: true }`, so callers do not change. After `sending`, the machine returns to `checkingSteps`, as the loop continued after a send.
|
|
179
|
+
|
|
180
|
+
The state's `meta.interaction` block is what a host renders for the pause. `setupAgent` types it against the machine's events, so a choice naming an undeclared event does not compile. See [Human in the loop](human-in-the-loop.md).
|
|
181
|
+
|
|
182
|
+
## Step 4: the run with `runAgent`
|
|
183
|
+
|
|
184
|
+
The loop ran N turns. Its replacement is one `runAgent` call with `onIdle`. Each time the machine pauses on a human, `runAgent` calls `onIdle` with the idle snapshot, then resumes from the persisted snapshot with the event `onIdle` returns. The loop's `approve` callback becomes `onIdle`:
|
|
185
|
+
|
|
186
|
+
```ts no-check
|
|
187
|
+
const result = await runAgent(supportLoopMachine, {
|
|
188
|
+
input: { request },
|
|
189
|
+
executors,
|
|
190
|
+
onIdle: async (snapshot) =>
|
|
191
|
+
(await approve(snapshot.context.draft))
|
|
192
|
+
? { type: "APPROVE" as const }
|
|
193
|
+
: { type: "REJECT" as const },
|
|
194
|
+
});
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
With `onIdle`, the call returns only `done` or `error`. `runSupportAgent` in [`index.ts`](https://github.com/statelyai/agent/blob/next/examples/port-walkthrough/index.ts) wraps this call with the loop's signature: it returns `result.output` and rethrows a run that settles `error`. The invokes have no `onError`, so an SDK error still rejects the call, as it did in the loop.
|
|
198
|
+
|
|
199
|
+
For a single leg that hands the pause back to a caller, leave out `onIdle` and resume later with a second `runAgent` call that passes `{ snapshot, event }`. This is the form an HTTP handler wants, because the pause outlives the request. The third test below runs it. [Use in any stack](any-stack.md) runs one machine in local, Express, and Cloudflare hosts. See [Choosing a run mode](choosing-a-run-mode.md) for the full set.
|
|
200
|
+
|
|
201
|
+
The executors hold your existing model code:
|
|
202
|
+
|
|
203
|
+
- The `createAiSdkExecutors` adapter wraps the AI SDK. `runSupportAgent` uses it by default.
|
|
204
|
+
- Another SDK, or a raw `fetch`, is three plain functions that return `{ result }`. See [Hosts](hosts.md).
|
|
205
|
+
- The tools, retry logic, and provider calls you already wrote move across unchanged.
|
|
206
|
+
|
|
207
|
+
A host that drives the transition loop itself uses `createAgentRuntime`. See [Write your own host loop](hosts.md#write-your-own-host-loop).
|
|
208
|
+
|
|
209
|
+
## Step 5: the tests
|
|
210
|
+
|
|
211
|
+
[`index.test.ts`](https://github.com/statelyai/agent/blob/next/examples/port-walkthrough/index.test.ts) has four tests. None of them needs an API key or network access. Each one starts with a comment naming what the `while` loop could not test.
|
|
212
|
+
|
|
213
|
+
The model is `examples/mock-model.ts`, a file in the repository, not in the package: the AI SDK's `MockLanguageModelV3` behind the real `createAiSdkExecutors` adapter. Answers are keyed by request name, `assist` for the text request and `chooseAction` for the decision.
|
|
214
|
+
|
|
215
|
+
### The port runs with no key
|
|
216
|
+
|
|
217
|
+
The loop builds its model inside the function, so it runs only against a live provider. The port takes executors, so the test passes the mock:
|
|
218
|
+
|
|
219
|
+
```ts no-check
|
|
220
|
+
const executors = createMockModelExecutors({
|
|
221
|
+
text: { assist: "Your duplicate charge is refunded." },
|
|
222
|
+
decisions: { chooseAction: { type: "REPLY" } },
|
|
223
|
+
});
|
|
224
|
+
|
|
225
|
+
const output = await runSupportAgent(request, never, executors);
|
|
226
|
+
|
|
227
|
+
expectTypeOf(output).toEqualTypeOf<Awaited<ReturnType<typeof runLoop>>>();
|
|
228
|
+
expect(output).toEqual({ sent: false, reply: "Your duplicate charge is refunded." });
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
`runLoop` is `before.ts`'s `runSupportAgent`, imported as a type only. The `expectTypeOf` line fails `pnpm run typecheck:examples` if the port's result type drifts from the loop's.
|
|
232
|
+
|
|
233
|
+
### The step limit settles `done`
|
|
234
|
+
|
|
235
|
+
The decision answers `CONTINUE` every time, so the run spends its whole budget:
|
|
236
|
+
|
|
237
|
+
```ts no-check
|
|
238
|
+
const result = await runAgent(supportLoopMachine, { input: { request }, executors });
|
|
239
|
+
|
|
240
|
+
expect(result.status).toBe("done");
|
|
241
|
+
if (result.status !== "done") return;
|
|
242
|
+
expect(result.snapshot.value).toBe("failed");
|
|
243
|
+
expect(result.output).toEqual({ sent: false, stopped: "step-limit" });
|
|
244
|
+
expect(executors.calls.filter((call) => call.name === "assist")).toHaveLength(MAX_STEPS);
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
The run ends in the `failed` final state with the loop's `stopped: "step-limit"` output. It does not throw. The test also counts the `assist` calls, which the loop's local counter never exposed.
|
|
248
|
+
|
|
249
|
+
### The approval pauses and resumes
|
|
250
|
+
|
|
251
|
+
The decision proposes `SEND_EMAIL`, and the run settles `idle`. The test round-trips the snapshot through JSON and resumes with `APPROVE`:
|
|
252
|
+
|
|
253
|
+
```ts no-check
|
|
254
|
+
const paused = await runAgent(supportLoopMachine, { input: { request }, executors });
|
|
255
|
+
expect(paused.status).toBe("idle");
|
|
256
|
+
expect(paused.snapshot.value).toBe("awaitingApproval");
|
|
257
|
+
|
|
258
|
+
const restored = JSON.parse(JSON.stringify(paused.persist()));
|
|
259
|
+
const resumed = await runAgent(supportLoopMachine, {
|
|
260
|
+
executors,
|
|
261
|
+
snapshot: restored,
|
|
262
|
+
event: { type: "APPROVE" },
|
|
263
|
+
});
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
The resumed run sends the email, takes another round, and ends with `{ sent: true, reply }`. The loop returned `{ pending: true }` at this point and had nothing to resume.
|
|
267
|
+
|
|
268
|
+
### No path sends without approval
|
|
269
|
+
|
|
270
|
+
`canReach` enumerates every decision branch and every human answer with no model calls. The target is a snapshot predicate: the machine is in `sending` and `approved` is `false`.
|
|
271
|
+
|
|
272
|
+
```ts no-check
|
|
273
|
+
const sendsUnapproved = await canReach(
|
|
274
|
+
supportLoopMachine,
|
|
275
|
+
(snapshot) => snapshot.matches("sending") && !snapshot.context.approved,
|
|
276
|
+
everyPath,
|
|
277
|
+
);
|
|
278
|
+
|
|
279
|
+
expect(sendsUnapproved.reachable).toBe(false);
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
`everyPath` scripts the `assist` text and the `sendEmail` output, and sets `maxDepth` and `maxPaths` high enough that all 766 paths through the eight rounds are explored. The loop's approval was an `if` inside the tool loop, and no test could list the paths around it. Here the claim holds for every path. See [Testing and verification](verify.md) for `explorePaths`, `simulateAgent`, and `lintAgentMachine`.
|
|
283
|
+
|
|
284
|
+
### Recorded runs and parity
|
|
285
|
+
|
|
286
|
+
To check a port against the loop's actual effects, record the loop's model and tool calls with `createRecorder`, replay the machine against the recordings with `compareRecordings`, and label outcomes with `compareRuns`. [`examples/retrofit-parity`](https://github.com/statelyai/agent/blob/next/examples/retrofit-parity) does all three with no API key. See [Parity with the code you replace](evals.md#parity-with-the-code-you-replace).
|
|
287
|
+
|
|
288
|
+
A loop on a client other than the AI SDK or `openai` (Anthropic, Gemini, a framework's model client) records through `recorder.client`. See [Any model client](evals.md#any-model-client).
|
|
289
|
+
|
|
290
|
+
## Part 2: a factory loop that improves
|
|
291
|
+
|
|
292
|
+
> **Experimental:** self-improvement is experimental. The gates are stable, but the round itself may change, and on a loop with no decision point it mostly proposes wording, cap, and approval-rule changes. Code-level causes come back as change requests for a person.
|
|
293
|
+
|
|
294
|
+
Part 2 starts from a working software factory. An author writes a patch, a self-check and an independent review run in parallel, and the task's visible tests run. Then the loop ships, retries, or asks a person.
|
|
295
|
+
|
|
296
|
+
The loop works. Part 2 makes its triage improve from its own runs, and keeps its rules fixed: nothing ships without passing checks and approval, three autonomous rounds at most, and a task whose spec is broken goes to a person.
|
|
297
|
+
|
|
298
|
+
The code is in `examples/code-review-loop`:
|
|
299
|
+
|
|
300
|
+
- [`before.ts`](https://github.com/statelyai/agent/blob/next/examples/code-review-loop/before.ts): the factory as a `while` loop with plain injectable dependencies. [`before.test.ts`](https://github.com/statelyai/agent/blob/next/examples/code-review-loop/before.test.ts) runs it with fakes.
|
|
301
|
+
- [`environment.ts`](https://github.com/statelyai/agent/blob/next/examples/code-review-loop/environment.ts): the same factory as a machine.
|
|
302
|
+
- [`policies/`](https://github.com/statelyai/agent/blob/next/examples/code-review-loop/policies): the triage policy versions, as JSON configs. [`policy.ts`](https://github.com/statelyai/agent/blob/next/examples/code-review-loop/policy.ts) runs one as the triage decider.
|
|
303
|
+
- [`run.ts`](https://github.com/statelyai/agent/blob/next/examples/code-review-loop/run.ts): runs the tasks on Claude Code in headless mode. `--improve` runs one improvement round.
|
|
304
|
+
|
|
305
|
+
The loop's body is this:
|
|
306
|
+
|
|
307
|
+
```ts no-check
|
|
308
|
+
while (round < 3) {
|
|
309
|
+
patch = await deps.writePatch({ task, round, previous: patch, feedback });
|
|
310
|
+
round++;
|
|
311
|
+
const [self, review] = await Promise.all([
|
|
312
|
+
deps.selfCheck({ task, patch }),
|
|
313
|
+
deps.reviewPatch({ task, patch }),
|
|
314
|
+
]);
|
|
315
|
+
const checks = await deps.runChecks({ taskId: task.id, file: task.file, patch });
|
|
316
|
+
|
|
317
|
+
if (review.issues.some((issue) => issue.severity === "novel")) {
|
|
318
|
+
return escalate("the reviewer marked an issue novel");
|
|
319
|
+
}
|
|
320
|
+
if (checks.passed && review.approved && self.ok) {
|
|
321
|
+
return { outcome: "shipped", rounds: round, humanApproved: false, patch };
|
|
322
|
+
}
|
|
323
|
+
if (!checks.passed && !checks.failingTests.some((name) => task.visibleTestNames.includes(name))) {
|
|
324
|
+
return escalate("the checks failed outside every visible test");
|
|
325
|
+
}
|
|
326
|
+
// feedback = the failing tests with their output, the review issues, and the self-check concerns
|
|
327
|
+
}
|
|
328
|
+
return escalate("round cap");
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
Each dependency takes the input and returns the output of the matching machine request or actor. `escalate` calls `askHuman` and returns the person's answer as the result.
|
|
332
|
+
|
|
333
|
+
This loop has three problems:
|
|
334
|
+
|
|
335
|
+
- The rules and the triage share one function. To change when the loop escalates, you edit the function that holds the ship rule, and no test lists the paths around it.
|
|
336
|
+
- A thrown dependency ends the whole run. In `before.test.ts`, a reviewer outage in round 2 throws out of `runFactory`. The patch and the feedback lived in local variables, and no person is asked.
|
|
337
|
+
- `escalate` blocks the process on `askHuman`, as `approve` did in Part 1.
|
|
338
|
+
|
|
339
|
+
## Step 6: the factory as a machine
|
|
340
|
+
|
|
341
|
+
Port the loop as in Steps 1–4. Each line of `before.ts` becomes one construct in `environment.ts`:
|
|
342
|
+
|
|
343
|
+
| In `before.ts` | In `codeReviewLoopMachine` |
|
|
344
|
+
| ------------------------------------------------------- | ------------------------------------------------------------------------------------- |
|
|
345
|
+
| `await deps.writePatch(...)` | The `generating` state, which invokes the `writePatch` request |
|
|
346
|
+
| `await Promise.all([selfCheck(...), reviewPatch(...)])` | The `verifying` parallel state: one region per invoke, and `onDone` after both |
|
|
347
|
+
| `await deps.runChecks(...)` | The `checking` state, which invokes the `runChecks` actor |
|
|
348
|
+
| `while (round < 3)` | `MAX_ROUNDS` as `context.roundLimit`, read by `triaging` and `retryBlocker` |
|
|
349
|
+
| `if (checks.passed && review.approved && self.ok)` | The `triaging` choice state, through `autoShippable` |
|
|
350
|
+
| `if (review.issues.some(... "novel"))` | `retryBlocker`, the guard on `RETRY` |
|
|
351
|
+
| `feedback = ...` | The `feedback` of a `RETRY` event, built by `triagePayloads` |
|
|
352
|
+
| `await deps.askHuman(...)` | The idle `humanReview` state, with `meta.interaction` |
|
|
353
|
+
| `return { outcome: ... }` | The final states `shipped`, `rejected`, `failed`, and the `illegallyShipped` sentinel |
|
|
354
|
+
|
|
355
|
+
The ship check is a choice state. It runs before any model is asked to triage:
|
|
356
|
+
|
|
357
|
+
```ts no-check
|
|
358
|
+
triaging: {
|
|
359
|
+
type: "choice",
|
|
360
|
+
choice: ({ context }) => {
|
|
361
|
+
if (autoShippable(context)) return { target: "checkingShip" };
|
|
362
|
+
if (context.round >= context.roundLimit) {
|
|
363
|
+
return { target: "humanReview", context: { escalation: `Round cap: ...` } };
|
|
364
|
+
}
|
|
365
|
+
return { target: "deciding" };
|
|
366
|
+
},
|
|
367
|
+
},
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
- An invoke failure is a transition. A failed review records `approved: false` with a `novel` issue, so the same outage leaves the run idle in `humanReview` with the round-2 patch. The last test in `before.test.ts` runs the outage through both.
|
|
371
|
+
- `checkingShip` checks the ship rule a second time, written independently of `autoShippable`. A ship that breaks it lands in `illegallyShipped`, a final state that exists to be proven unreachable in Step 10.
|
|
372
|
+
|
|
373
|
+
## Step 7: rules as guards, judgment as a decision
|
|
374
|
+
|
|
375
|
+
Split what must always hold from what a model can learn:
|
|
376
|
+
|
|
377
|
+
| Kind | In the loop | In the machine |
|
|
378
|
+
| -------- | ------------------------------------------ | ------------------------------------------------------------------------------------ |
|
|
379
|
+
| Rule | Ship only with passing checks, approval and no novel issue | `autoShippable` in `triaging`, re-checked by `checkingShip`; or a person's `APPROVE` |
|
|
380
|
+
| Rule | At most 3 rounds | `roundLimit`, read by `triaging` and `retryBlocker` |
|
|
381
|
+
| Rule | A novel issue is not retried | `retryBlocker`, the guard on `RETRY` |
|
|
382
|
+
| Judgment | Fixed: retry until the cap | `deciding`: `RETRY` with which feedback, or `ESCALATE` with which reason |
|
|
383
|
+
|
|
384
|
+
`retryBlocker` returns why `RETRY` is refused, or `undefined`:
|
|
385
|
+
|
|
386
|
+
```ts no-check
|
|
387
|
+
export function retryBlocker(facts: Facts): string | undefined {
|
|
388
|
+
if (facts.round >= facts.roundLimit) {
|
|
389
|
+
return `round cap reached (${facts.round} of ${facts.roundLimit})`;
|
|
390
|
+
}
|
|
391
|
+
if (hasNovelIssue(facts)) {
|
|
392
|
+
return "the reviewer marked an issue novel";
|
|
393
|
+
}
|
|
394
|
+
if (unattributedFailure(facts)) return "the checks failed outside every visible test";
|
|
395
|
+
return undefined;
|
|
396
|
+
}
|
|
397
|
+
```
|
|
398
|
+
|
|
399
|
+
The decision is a [closed choice](decisions.md#closed-choices). `triageInput` passes model `"triage"`, `allowedEvents: ["RETRY", "ESCALATE"]`, and `payloads` from `triagePayloads`: one `RETRY` per feedback focus, none while `retryBlocker` refuses it, and one `ESCALATE` per fixed reason. The `RETRY` transition checks the guard again:
|
|
400
|
+
|
|
401
|
+
```ts no-check
|
|
402
|
+
deciding: {
|
|
403
|
+
invoke: { id: "triage", src: "agent.decide", input: ({ context }) => triageInput(context) },
|
|
404
|
+
on: {
|
|
405
|
+
RETRY: ({ context, event }) =>
|
|
406
|
+
retryBlocker(context)
|
|
407
|
+
? undefined
|
|
408
|
+
: { target: "generating", context: { feedback: event.feedback } },
|
|
409
|
+
ESCALATE: ({ event }) => ({
|
|
410
|
+
target: "humanReview",
|
|
411
|
+
context: { escalation: `Escalated: ${event.reason}.` },
|
|
412
|
+
}),
|
|
413
|
+
},
|
|
414
|
+
},
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
- Shipping is not an event the decider can pick. No triage answer ships a patch.
|
|
418
|
+
- Model `"triage"` resolves to the policy: a JSON `AgentWorkflowConfig` that `createPolicyDecide` runs as the decide executor. `v0` retries until the cap. `v1` asks a `classifyIssue` judgment whether the task itself is broken and escalates early. See [Environment and policy](environment-and-policy.md).
|
|
419
|
+
|
|
420
|
+
## Step 8: the goal and the score
|
|
421
|
+
|
|
422
|
+
The goal is the machine's `description`. [`describeMachine`](machines-as-data.md#machine-descriptions-for-models) renders it first, so a model that proposes a policy reads the goal before the states.
|
|
423
|
+
|
|
424
|
+
```ts no-check
|
|
425
|
+
description: REVIEW_GOAL, // "Ship a patch for each coding task that passes its visible checks, ..."
|
|
426
|
+
```
|
|
427
|
+
|
|
428
|
+
The objective is `scoreEpisode` in `bench.ts`. It reads ground truth: the final state, and hidden tests that run only on what shipped.
|
|
429
|
+
|
|
430
|
+
| Outcome | Score |
|
|
431
|
+
| --------------------------------------------------------- | ----- |
|
|
432
|
+
| Shipped, hidden tests pass (a person's approval included) | 1 |
|
|
433
|
+
| Escalated a task labeled `novel-expected` | 1 |
|
|
434
|
+
| Escalated a task labeled `resolvable` | 0.5 |
|
|
435
|
+
| Shipped, hidden tests fail | 0 |
|
|
436
|
+
| Failed without escalating | 0 |
|
|
437
|
+
|
|
438
|
+
Each round subtracts 0.05, floored at 0. A batch with any run in `illegallyShipped` scores `-Infinity`.
|
|
439
|
+
|
|
440
|
+
- Escalating a novel task scores the same as a clean ship after the same number of rounds. A policy is never penalized for asking a person; it pays only for the rounds it spent first.
|
|
441
|
+
- People write the task labels and the hidden tests. No model grades a run.
|
|
442
|
+
|
|
443
|
+
## Step 9: one trace per task
|
|
444
|
+
|
|
445
|
+
Pass a store and a thread id to the runtime. `runEpisode` in `bench.ts` names each thread `<split>/<policyVersion>/<taskId>`:
|
|
446
|
+
|
|
447
|
+
```ts no-check
|
|
448
|
+
createAgentRuntime(codeReviewLoopMachine, {
|
|
449
|
+
executors: options.executors,
|
|
450
|
+
actors,
|
|
451
|
+
store, // run.ts passes createFileEventLogStore(tracesDir): one JSONL file per thread
|
|
452
|
+
threadId, // `${task.split}/${policyVersion}/${task.id}`
|
|
453
|
+
});
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
- Every model output, check result, and human answer is an entry in the thread. A person's answer resumes the run from the thread.
|
|
457
|
+
- `getDecisionAttempts(entries)` lists the triage answers the machine refused before one landed. See [Decision attempts](event-log.md#decision-attempts).
|
|
458
|
+
- `listThreads({ prefix: "train/<version>/" })` returns one split. The improvement round reads train threads only. See [Stores](event-log.md#stores).
|
|
459
|
+
|
|
460
|
+
## Step 10: the ship proof
|
|
461
|
+
|
|
462
|
+
`canReach` searches every path to the `illegallyShipped` sentinel. The proof in `environment.test.ts` forks every invoke over a table of outputs and a failure, every triage answer, and every human answer:
|
|
463
|
+
|
|
464
|
+
```ts no-check
|
|
465
|
+
const violation = await canReach(codeReviewLoopMachine, "illegallyShipped", {
|
|
466
|
+
input: { taskId: "slugify" },
|
|
467
|
+
payloads: () => ({
|
|
468
|
+
RETRY: [{ feedback: "explore", focus: "all" }] /* REQUEST_CHANGES, REJECT */,
|
|
469
|
+
}),
|
|
470
|
+
outputs: {
|
|
471
|
+
reviewPatch: [
|
|
472
|
+
{ approved: true, issues: [] },
|
|
473
|
+
{ approved: false, issues: [issue("major")] },
|
|
474
|
+
{ approved: true, issues: [issue("novel")] },
|
|
475
|
+
// ...
|
|
476
|
+
],
|
|
477
|
+
runChecks: [/* passed, a visible test failed, an unknown test failed, a crash */],
|
|
478
|
+
},
|
|
479
|
+
errors: { reviewPatch: failure, runChecks: failure, "agent.decide": failure /* ... */ },
|
|
480
|
+
dedupeBy: projection,
|
|
481
|
+
maxDepth: 6 * MAX_TOTAL_ROUNDS + 6,
|
|
482
|
+
});
|
|
483
|
+
|
|
484
|
+
expect(violation).toMatchObject({ reachable: false, complete: true });
|
|
485
|
+
```
|
|
486
|
+
|
|
487
|
+
- The outputs are not tied to a patch, so the proof covers every author, reviewer, and test result in the table, not only the fixtures. `complete: true` makes `reachable: false` a proof within the round ceiling. See [Reachability checks](verify.md#reachability-checks).
|
|
488
|
+
- The machine's guards judge every event a policy proposes, so the proof holds for every policy. A promoted policy needs no new proof.
|
|
489
|
+
- A copy of the machine whose ship check forgets the reviewer fails the same search, with a witness path that ends in a passing `runChecks`. The loop's `if` has no such test.
|
|
490
|
+
|
|
491
|
+
## Step 11: an improvement round
|
|
492
|
+
|
|
493
|
+
Improvement is an explicit step. No policy changes while tasks run. Run one round:
|
|
494
|
+
|
|
495
|
+
```bash
|
|
496
|
+
pnpm exec tsx examples/code-review-loop/run.ts --improve --policy v0
|
|
497
|
+
```
|
|
498
|
+
|
|
499
|
+
`improvePolicy` in `improve.ts` runs these steps:
|
|
500
|
+
|
|
501
|
+
1. Run the train and held-out tasks with the current policy. The traces go to the store.
|
|
502
|
+
2. Reflect: one `proposePolicy` request with the goal, the objective, both machines as `describeMachine` text, and the train trails. It returns one candidate config.
|
|
503
|
+
3. Gate: `verifyAgentConfig` runs `validate`, `vocabulary`, `lower`, `lint`, and `explore`. An `outputs` probe then checks that every answer is an offered option. A failing gate's `detail` goes back as a repair prompt, up to 3 times.
|
|
504
|
+
4. Replay the train and held-out tasks with the candidate.
|
|
505
|
+
5. Promote only with zero violations on both splits, a held-out objective no lower than before, and a combined objective strictly higher. `run.ts` writes a promoted policy to `policies/current.json`.
|
|
506
|
+
|
|
507
|
+
Inside Claude Code, the [`improve-machine` skill](https://github.com/statelyai/agent/blob/next/skills/improve-machine/SKILL.md) runs the same round with Claude Code as the improver:
|
|
508
|
+
|
|
509
|
+
```bash
|
|
510
|
+
mkdir -p .claude/skills && cp -R skills/improve-machine .claude/skills/
|
|
511
|
+
pnpm exec tsx examples/code-review-loop/run.ts --policy v0 --no-human
|
|
512
|
+
claude -p "Use the improve-machine skill on examples/code-review-loop. ..."
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
The full prompt and its allowed tools are in the [example's README](https://github.com/statelyai/agent/blob/next/examples/code-review-loop/README.md#inside-claude-code-with-the-improve-machine-skill).
|
|
516
|
+
|
|
517
|
+
- One change per round. People own the environment, the goal, the objective, and the task split.
|
|
518
|
+
- With the mock executors in `improve.test.ts`, one round from `v0` promotes `v1`: the combined objective rises from 0.900 to 0.925, and the held-out objective from 0.900 to 0.933.
|
|
519
|
+
|
|
520
|
+
## Related
|
|
521
|
+
|
|
522
|
+
- [retrofit](https://github.com/statelyai/agent/blob/next/examples/retrofit/index.ts): a larger loop refactored one step at a time, in the order `before.ts`, `step1.ts`, `step2.ts`, `step3.ts`, and `index.ts`.
|
|
523
|
+
- [Strangler-fig retrofit](strangler-fig.md): wrap the workflow unchanged with `wrapWorkflow`, then carve one phase at a time, checked with `checkParity`.
|
|
524
|
+
- [Overview](index.md): what the machine gives you over the loop.
|
|
525
|
+
- [Human in the loop](human-in-the-loop.md): idle states, `meta.interaction`, and resuming.
|
|
526
|
+
- [Environment and policy](environment-and-policy.md): the fixed environment, the generated policy, the objective, and the improvement loop.
|
|
527
|
+
- [code-review-loop](https://github.com/statelyai/agent/blob/next/examples/code-review-loop): Part 2's factory: `before.ts`, the machine, the policies, and the improvement round on Claude Code.
|
|
528
|
+
- [`skills/improve-machine`](https://github.com/statelyai/agent/blob/next/skills/improve-machine/SKILL.md): one improvement round from traces, run by an agent.
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Generating machines with an LLM
|
|
3
|
+
description: Have a model author the agent workflow itself, then run the generated config through every pre-run gate before a single model call runs inside it.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
> **Alpha:** `@statelyai/agent` 2.0 is in alpha. APIs can change between releases; pin an exact version. Feedback: [github.com/statelyai/agent](https://github.com/statelyai/agent/issues).
|
|
7
|
+
|
|
8
|
+
This page covers having a model author an agent machine config, then checking that config with `verifyAgentConfig` before it runs.
|
|
9
|
+
|
|
10
|
+
An agent machine is [data](machines-as-data.md), so a model can write one. A generated machine can be checked before it runs, at several gates.
|
|
11
|
+
|
|
12
|
+
<!-- static lint diagnostics from src/verify.ts -->
|
|
13
|
+
|
|
14
|
+
- The config's shape is fixed by a JSON Schema that ships with the package. Expressions are dot paths, not code, so a generated config cannot do anything a hand-authored machine could not do.
|
|
15
|
+
- `verifyAgentConfig` runs every pre-run gate in one call and returns a record per gate. It is exported from `@statelyai/agent/validate`, which uses `ajv` as an optional peer dependency, so install `ajv` to use it.
|
|
16
|
+
- The gates check the config's shape, the names it references, the lowering, Agent lint, and every explored path. Use [`canReach` and `explorePaths`](verify.md) directly for other reachability questions.
|
|
17
|
+
- Replay against real scenarios and scoring are domain-specific. They stay with the caller, after the gates pass.
|
|
18
|
+
|
|
19
|
+
## The loop
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
generate → verifyAgentConfig (validate → vocabulary → lower → lint → explore) → run
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
<!-- viz: generation pipeline: generate -> verifyAgentConfig gates (validate -> vocabulary -> lower -> lint -> explore) -> run, with each failing gate feeding back into generate as a repair prompt, capped at 3 attempts -->
|
|
26
|
+
|
|
27
|
+
| Gate | Checks with | Fails on |
|
|
28
|
+
| ------------ | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
|
29
|
+
| `validate` | `validateAgentConfig` | malformed config shape |
|
|
30
|
+
| `vocabulary` | the `allow` lists | an invoke src, guard, action, `agent.decide` event, or model ref outside the vocabulary |
|
|
31
|
+
| `lower` | `setupAgent.fromConfig` | unresolved named guards/actions, `onDone` on a decision |
|
|
32
|
+
| `lint` | `lintAgentMachine` | decisions with no handled events; warnings too with `lint: { warnings: true }` |
|
|
33
|
+
| `explore` | `explorePaths`, once per `explore.inputs` item | a path ending outside `explore.terminals`, a loop past `maxDepth`, a `maxPaths`/`maxNodes` cap, untried payloads (`unexpandedEvents`) |
|
|
34
|
+
|
|
35
|
+
Run the gates on the config the model returned:
|
|
36
|
+
|
|
37
|
+
```ts no-check
|
|
38
|
+
import { verifyAgentConfig } from "@statelyai/agent/validate";
|
|
39
|
+
|
|
40
|
+
const result = await verifyAgentConfig(candidate, {
|
|
41
|
+
compileSchema,
|
|
42
|
+
guards: { isLarge: ({ context }) => context.amount > 100 },
|
|
43
|
+
allow: {
|
|
44
|
+
actors: [],
|
|
45
|
+
requests: ["draftReply"],
|
|
46
|
+
guards: ["isLarge"],
|
|
47
|
+
actions: [],
|
|
48
|
+
events: ["APPROVE", "DENY"],
|
|
49
|
+
models: ["openai/gpt-6-luna"],
|
|
50
|
+
},
|
|
51
|
+
explore: {
|
|
52
|
+
inputs: [{ amount: 5 }, { amount: 500 }],
|
|
53
|
+
outputs: { draftReply: [{ reply: "stub" }] },
|
|
54
|
+
},
|
|
55
|
+
});
|
|
56
|
+
|
|
57
|
+
if (!result.ok) {
|
|
58
|
+
const failed = result.gates.find((gate) => !gate.ok)!;
|
|
59
|
+
// Feed this back to the model as a repair prompt.
|
|
60
|
+
feedback.push(`gate '${failed.name}' failed: ${failed.detail}`);
|
|
61
|
+
}
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
`compileSchema` is the JSON Schema compiler `fromConfig` uses for the config's schemas. Build it from `ajv`. See [Machines as data](machines-as-data.md).
|
|
65
|
+
|
|
66
|
+
The result behaves as follows.
|
|
67
|
+
|
|
68
|
+
- `result.gates` lists the gates that ran, in order. Each is `{ name, ok, detail? }`. The gates stop at the first failure, so a failing gate is always the last entry.
|
|
69
|
+
- `detail` names the offending path, state, or name, one problem per line. A passing `lint` gate carries its warnings in `detail`.
|
|
70
|
+
- `result.config` is the typed config once `validate` passes. `result.machine` is the lowered machine once `lower` passes.
|
|
71
|
+
- A config that throws while lowering is reported as a failed `lower` gate. `verifyAgentConfig` does not throw on a bad config.
|
|
72
|
+
- An omitted `allow` key is unrestricted. Omit `allow` to skip the `vocabulary` gate, and omit `explore` to skip the `explore` gate. An empty `explore.inputs` list fails the `explore` gate.
|
|
73
|
+
- An invoke can name only an `agent.*` builtin or a key the config declares under `requests` or `actors`. Any other src fails the `lower` gate, even when `actors` binds it. `allow.requests` limits the declared request keys, and `allow.actors` limits the declared actor keys. With `allow.actors` listed, every non-builtin src must be in `allow.actors` or `allow.requests`, or be a declared request when `allow.requests` is omitted.
|
|
74
|
+
- With `allow` given, a raw xstate expression slot (`{ "@expr": ... }`) anywhere in the config fails the `vocabulary` gate. xstate evaluates the slot in place of its literal siblings, so it could hide a model ref or `allowedEvents`. Write `{{ }}` templates instead.
|
|
75
|
+
- `explore.terminals` defaults to `['done']`. Other `explore` keys, such as `outputs`, `errors`, `payloads`, `dedupeBy`, `maxPaths`, and `maxNodes`, pass through to `explorePaths`. Script text requests and invoked actors with `outputs`. See [Payloads and invoke outputs](verify.md#payloads-and-invoke-outputs).
|
|
76
|
+
- `actors` are bound with `machine.provide({ actors })` after lowering. `guards` and `actions` go to `fromConfig`.
|
|
77
|
+
- No gate needs an API key.
|
|
78
|
+
- `validateAgentConfig(config)` runs the `validate` gate alone. It returns `{ valid, errors }`, and each error's `path` is a JSON Pointer into the config.
|
|
79
|
+
|
|
80
|
+
## The repair loop
|
|
81
|
+
|
|
82
|
+
A failed gate's `detail` is the repair prompt. Send it back to the model with the config it rejected, and run the gates again on the new config.
|
|
83
|
+
|
|
84
|
+
- Cap the repair loop at about 3 attempts. A config that fails three schema-shaped repairs usually asks for something the data form cannot express. In that case, [author it in TypeScript](machines.md) instead.
|
|
85
|
+
- Replay the passing config against your own scenarios before promoting it. `simulateAgent` plays one scripted path with canned responses and no API key.
|
|
86
|
+
|
|
87
|
+
## The skill
|
|
88
|
+
|
|
89
|
+
The full authoring procedure ships as an agent skill. It covers the environment and policy split, the schema location, the rules a config must follow, a few-shot config, the Ajv adapter, `verifyAgentConfig`, the repair loop, and persisting traces.
|
|
90
|
+
|
|
91
|
+
**[`skills/generate-machine/SKILL.md`](https://github.com/statelyai/agent/blob/next/skills/generate-machine/SKILL.md)**
|
|
92
|
+
|
|
93
|
+
Copy that directory into your coding agent's skills directory. For Claude Code this is `.claude/skills/`. Other harnesses use their own location. The agent then loads the skill on requests such as "author an agent machine for X" and runs the gates itself.
|
|
94
|
+
|
|
95
|
+
To improve a generated policy from its run traces, use [`skills/improve-machine`](https://github.com/statelyai/agent/blob/next/skills/improve-machine/SKILL.md). See [Environment and policy](environment-and-policy.md).
|
|
96
|
+
|
|
97
|
+
## Limits
|
|
98
|
+
|
|
99
|
+
Generated machines have these constraints.
|
|
100
|
+
|
|
101
|
+
- Named guards and actions are resolved by the host. A config cannot carry functions. A model can emit `guard: "isReady"`, but the implementation lives in the `guards` option passed to `fromConfig(...)`, and an unresolved name throws at build time. List the names your host implements in the prompt, or forbid named references entirely.
|
|
102
|
+
- Schema validity is not semantic validity. The schema constrains shape, not meaning. A config can validate and still reference a request that does not exist, or route to a state that does not make progress. The `vocabulary`, `lint`, and `explore` gates cover those cases.
|
|
103
|
+
- Exploration covers the inputs you give it. List inputs that reach every branch you care about in `explore.inputs`.
|
|
104
|
+
- Model refs resolve at run time, in the host. `allow.models` restricts them to literal strings you list; tool names stay unchecked. Generated machines carry string refs, so pass `createAiSdkExecutors({ resolveModel })`. See [Models and providers](models-and-providers.md). The ref `"openai/gpt-6-luna"` used on this page is illustrative. Substitute your provider's current models.
|
|
105
|
+
|
|
106
|
+
## Related
|
|
107
|
+
|
|
108
|
+
- [Machines as data](machines-as-data.md): the config format, its expressions, and its limits.
|
|
109
|
+
- [Testing and verification](verify.md): every check and simulation API in full.
|
|
110
|
+
- [examples/json-agent](https://github.com/statelyai/agent/blob/next/examples/json-agent/index.ts): a hand-written config, run.
|