@statelyai/agent 2.0.0-alpha.38 → 2.0.0-alpha.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +45 -0
- package/dist/{act-CrX0JwWY.cjs → act-B0CVbmo4.cjs} +1 -1
- package/dist/{act-B9hLQls6.mjs → act-Bsz-hUfD.mjs} +1 -1
- package/dist/ai-sdk.cjs +4 -4
- package/dist/ai-sdk.d.cts +2 -3
- package/dist/ai-sdk.d.mts +2 -3
- package/dist/ai-sdk.mjs +4 -4
- package/dist/{decision-m5hGmHpF.mjs → decision-B41YBx5U.mjs} +1 -1
- package/dist/{decision-CMeuNZ1g.cjs → decision-D8bcIGXQ.cjs} +1 -1
- package/dist/{decision-object-DBcLWVvv.d.cts → decision-object-BB8XV5Ec.d.cts} +1 -1
- package/dist/{decision-object-B5zGSPFp.d.mts → decision-object-CH_sE694.d.mts} +1 -1
- package/dist/effects-B9IvUPVh.mjs +23 -0
- package/dist/effects-DrWcUGgc.cjs +34 -0
- package/dist/evals.cjs +810 -0
- package/dist/evals.d.cts +595 -0
- package/dist/evals.d.mts +595 -0
- package/dist/evals.mjs +801 -0
- package/dist/{event-log-DkVlk3Rh.mjs → event-log-CEq_bVXQ.mjs} +8 -4
- package/dist/{event-log-Dkc-yIYu.cjs → event-log-dSOnXHuG.cjs} +7 -15
- package/dist/{event-log-store-Brx9tOHt.mjs → event-log-store-DLAfNM34.mjs} +1 -1
- package/dist/{event-log-store-BBS8so8a.d.mts → event-log-store-DYun526m.d.mts} +15 -14
- package/dist/{event-log-store-Cl-YWDnq.cjs → event-log-store-Dey9ufA4.cjs} +1 -1
- package/dist/{event-log-store-bc4ICGcS.d.cts → event-log-store-aFXK6yZX.d.cts} +15 -14
- package/dist/index.cjs +131 -93
- package/dist/index.d.cts +73 -134
- package/dist/index.d.mts +73 -134
- package/dist/index.mjs +112 -70
- package/dist/internal-BzlpzSux.d.cts +14 -0
- package/dist/internal-BzlpzSux.d.mts +14 -0
- package/dist/log.cjs +2 -3
- package/dist/log.d.cts +2 -2
- package/dist/log.d.mts +2 -2
- package/dist/log.mjs +3 -3
- package/dist/machines.cjs +1 -719
- package/dist/machines.d.cts +2 -424
- package/dist/machines.d.mts +2 -424
- package/dist/machines.mjs +2 -709
- package/dist/{model-source-kjGwqAV8.mjs → model-source-BrsLDvHL.mjs} +1 -1
- package/dist/{model-source-B4WQ8Idp.cjs → model-source-YpxjmWuX.cjs} +1 -1
- package/dist/openai.cjs +3 -3
- package/dist/openai.d.cts +2 -3
- package/dist/openai.d.mts +2 -3
- package/dist/openai.mjs +3 -3
- package/dist/otel.d.cts +1 -1
- package/dist/otel.d.mts +1 -1
- package/dist/policy.cjs +77 -0
- package/dist/policy.d.cts +143 -0
- package/dist/policy.d.mts +143 -0
- package/dist/policy.mjs +75 -0
- package/dist/{effects-BT_cSY0j.mjs → run-agent-BE7DOWD6.mjs} +150 -184
- package/dist/{run-agent-DlPzBWj-.d.mts → run-agent-DHlq87G7.d.mts} +18 -34
- package/dist/{effects-DDseYtJk.cjs → run-agent-De3xVFGw.cjs} +149 -201
- package/dist/{run-agent-30mH2moE.d.cts → run-agent-Dq0oSFRm.d.cts} +18 -34
- package/dist/{setup-agent-BDmNr6PC.mjs → setup-agent-CLm6QVGy.mjs} +3 -3
- package/dist/{setup-agent-BaNyz6Q3.d.cts → setup-agent-Cv8Hk3mX.d.cts} +12 -11
- package/dist/{setup-agent-C1CG_nfX.d.mts → setup-agent-DgFAhaPJ.d.mts} +12 -11
- package/dist/{setup-agent-CmEToB8y.cjs → setup-agent-Dv5VpuHe.cjs} +3 -3
- package/dist/tanstack-ai.cjs +4 -4
- package/dist/tanstack-ai.d.cts +2 -3
- package/dist/tanstack-ai.d.mts +2 -3
- package/dist/tanstack-ai.mjs +4 -4
- package/dist/testing.cjs +122 -996
- package/dist/testing.d.cts +36 -695
- package/dist/testing.d.mts +36 -695
- package/dist/testing.mjs +112 -977
- package/dist/{text-logic-BqxalgMr.d.mts → text-logic-x17uLBwA.d.cts} +238 -2
- package/dist/{text-logic-CSf9bqpP.d.cts → text-logic-x17uLBwA.d.mts} +238 -2
- package/dist/trajectory-BFjXaooB.d.cts +96 -0
- package/dist/trajectory-BFjXaooB.d.mts +96 -0
- package/dist/trajectory-BUXj_1Ue.mjs +152 -0
- package/dist/trajectory-CoZSv4wF.cjs +157 -0
- package/dist/{usage-Bm5nvpow.d.cts → usage-B4B-TQSV.d.mts} +1 -1
- package/dist/{usage-BnikdOca.d.mts → usage-DvGcoK6x.d.cts} +1 -1
- package/dist/{utils-7lSa4H-i.d.cts → utils-B8k-YPYI.d.mts} +23 -17
- package/dist/{utils-CBbKw2fM.d.mts → utils-DAclF-kw.d.cts} +23 -17
- package/dist/validate.cjs +4 -4
- package/dist/validate.d.cts +2 -2
- package/dist/validate.d.mts +2 -2
- package/dist/validate.mjs +4 -4
- package/dist/{verify-DJHfEfm6.d.cts → verify-BM0sGmsf.d.cts} +42 -6
- package/dist/{verify-ChyX6jq0.mjs → verify-CWlg3j7B.mjs} +153 -45
- package/dist/{verify-CwlombbC.cjs → verify-Dek6gHCS.cjs} +153 -45
- package/dist/{verify-i9Hiwc5g.d.mts → verify-H3Pr2gNw.d.mts} +42 -6
- package/dist/{with-model-calls-B_WLmOSl.cjs → with-model-calls-D50Ca6qW.cjs} +2 -2
- package/dist/{with-model-calls-BkOwdroS.mjs → with-model-calls-uY-KKliA.mjs} +2 -2
- package/docs/advanced.md +5 -5
- package/docs/choosing-a-run-mode.md +7 -7
- package/docs/debugging.md +6 -8
- package/docs/decisions.md +5 -4
- package/docs/environment-and-policy.md +1 -1
- package/docs/evals.md +57 -43
- package/docs/event-log.md +13 -15
- package/docs/from-a-loop.md +3 -3
- package/docs/generate-machines.md +4 -4
- package/docs/hosts.md +20 -27
- package/docs/human-in-the-loop.md +5 -8
- package/docs/index.md +4 -4
- package/docs/judgments.md +5 -7
- package/docs/langgraph-comparison.md +6 -6
- package/docs/machines-as-data.md +7 -12
- package/docs/machines-presets.md +17 -120
- package/docs/machines.md +13 -13
- package/docs/models-and-providers.md +19 -25
- package/docs/multi-agent.md +1 -1
- package/docs/observability.md +6 -8
- package/docs/persistence.md +10 -20
- package/docs/quickstart.md +3 -3
- package/docs/statechart-policy-examples.md +1 -1
- package/docs/steps.md +5 -6
- package/docs/strangler-fig.md +72 -7
- package/docs/text-requests.md +10 -12
- package/docs/thinking-in-state-machines.md +8 -8
- package/docs/tools.md +10 -12
- package/docs/upgrading.md +1 -1
- package/docs/usage-and-budgets.md +51 -56
- package/docs/validate-and-repair.md +9 -14
- package/docs/verify.md +53 -7
- package/docs/xstate-4-and-5-apps.md +19 -28
- package/llms.txt +1 -1
- package/package.json +31 -6
- package/readme.md +32 -24
- package/dist/internal-C3Lqu538.d.cts +0 -43
- package/dist/internal-DUcdhk-i.d.mts +0 -43
- package/dist/types-DQUOOaZO.d.cts +0 -238
- package/dist/types-DQUOOaZO.d.mts +0 -238
- package/skills/add-evals/SKILL.md +0 -152
- package/skills/add-traces/SKILL.md +0 -152
- package/skills/build-machine/SKILL.md +0 -84
- package/skills/generate-machine/SKILL.md +0 -335
- package/skills/graph-engineering/SKILL.md +0 -43
- package/skills/improve-machine/SKILL.md +0 -261
- package/skills/improve-machine/reference.md +0 -101
- package/skills/loop-to-machine/SKILL.md +0 -205
- package/skills/loop-to-machine/policy.md +0 -154
- package/skills/loop-to-machine/reference.md +0 -260
|
@@ -1,261 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: improve-machine
|
|
3
|
-
description: Experimental. Improve a @statelyai/agent policy machine (a TypeScript setupAgent machine or a JSON AgentWorkflowConfig) from its run traces toward its goal, inside a fixed environment machine, with gates and a held-out promotion rule, and hand the result to a person as a pull request. Use when asked to "improve this agent", "self-improve", "optimize the playbook from traces", "tune the policy", "make the agent cheaper or more accurate", or "why is my agent failing" for a machine whose runs are persisted to an event log store.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Improve an agent machine
|
|
7
|
-
|
|
8
|
-
> **Experimental:** self-improvement is experimental. The gates are stable, but the round itself may change, and on a loop with no decision point it mostly proposes wording, cap, and approval-rule changes. Code-level causes come back as change requests for a person. Tell the user this skill is experimental before the first round.
|
|
9
|
-
|
|
10
|
-
You improve a **policy**: the machine that decides how the agent works toward its goal, written in TypeScript or as a JSON `AgentWorkflowConfig`. It runs inside an **environment**: a TypeScript machine whose states and guards encode the rules of the world and the business invariants. One round changes the policy once and measures it:
|
|
11
|
-
|
|
12
|
-
```
|
|
13
|
-
locate inputs → read traces → diagnose → propose one diff → gate (≤ 3 repairs) → replay train → replay held-out → recommend or discard → record → pull request
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
This is step 4 of 4, after `loop-to-machine`, `add-traces`, and `add-evals`. To author a JSON policy from scratch, use the `generate-machine` skill. Concept page: [Environment and policy](../../docs/environment-and-policy.md).
|
|
17
|
-
|
|
18
|
-
## Hard rules
|
|
19
|
-
|
|
20
|
-
- **Never edit the environment.** Not its states, guards, invariants, event schemas, or the vocabulary it exposes. When the goal is unreachable without an environment change, write a change request for a person (template in [reference.md](reference.md)) and stop.
|
|
21
|
-
- **Never edit a machine while it runs.** A promoted policy applies to new runs only.
|
|
22
|
-
- **The objective comes from ground truth.** Score outcomes from the traces (terminal states, outputs) or from effects recorded in the world. Never ask a model to grade its own runs.
|
|
23
|
-
- **One cause per round, smallest diff.** A diff that fixes two things cannot tell you which one helped.
|
|
24
|
-
- **Diagnose from training scenarios only.** Held-out scenarios are for the promotion decision. Never read their failures to shape a change, and never move a scenario between splits. Eval files may list baseline failures for every split; read only the training rows.
|
|
25
|
-
- **Stop after 3 rounds without a promotion** (or the limit the user gives).
|
|
26
|
-
- **Propose, never promote.** A round ends in a recommendation. A recommended candidate goes to a person as a pull request with its scorecard; merging it is the promotion. Never merge, deploy, or switch the running policy yourself.
|
|
27
|
-
|
|
28
|
-
## Policy format
|
|
29
|
-
|
|
30
|
-
The rounds are the same for both formats. What differs is what the diff may touch and how it is gated.
|
|
31
|
-
|
|
32
|
-
| | TypeScript policy | JSON policy |
|
|
33
|
-
| --- | --- | --- |
|
|
34
|
-
| The policy | A `setupAgent` machine in its own file(s), with a declared `version` | An `AgentWorkflowConfig` file |
|
|
35
|
-
| May change | The policy file(s) and their `CHANGELOG.md`; once, the line in the runner that routes the decision to the policy; tests that pin measured numbers (see the pull request) | The config's `states` and `context` |
|
|
36
|
-
| Gates | `tsc`, `lintAgentMachine`, `explorePaths` on the policy, the environment check, the overfit check | `verifyAgentConfig` |
|
|
37
|
-
| Vocabulary | What the policy file imports and the events the environment offers | The `allow` lists |
|
|
38
|
-
|
|
39
|
-
**No policy yet.** `loop-to-machine` leaves a single model decision inside the environment when the old code had no policy logic. Then:
|
|
40
|
-
|
|
41
|
-
- Record `v0` in a new `CHANGELOG.md` next to the environment (in a `policy/` directory): "the model decides directly". This entry goes in the working tree.
|
|
42
|
-
- The first recommended candidate introduces the policy as `v1`: a policy file that reproduces the current behavior (it passes the environment's prompt and facts through to the model) plus the one rule the diagnosis found, and the routing line in the runner (`withPolicy(modelExecutors)`). The policy runs as the decider through `createPolicyDecider` from `@statelyai/agent`; the pattern is in `../loop-to-machine/policy.md`.
|
|
43
|
-
- Its `description` is the environment's goal with measurable success criteria. If the environment's goal has none, draft them from the confirmed objective and list the draft under questions for the user.
|
|
44
|
-
- If the policy needs a fact the environment does not pass in the decision's `metadata.facts`, that is an environment change request.
|
|
45
|
-
|
|
46
|
-
For a TypeScript policy, before the first round:
|
|
47
|
-
|
|
48
|
-
- **List the environment files** (the environment machine, its guards, the world and its tools) and record a hash of each. Any change to them fails the round. The runner is not one of them, but its diff may contain only the routing line.
|
|
49
|
-
- **Keep the policy a machine.** A diff may add states, `choice` branches, guards over facts, helpers in the policy file, or change a prompt or a model ref. It may not turn the policy into a free function, call tools the environment owns, or import from the environment anything but its schemas, types, constants, and exported guard functions.
|
|
50
|
-
|
|
51
|
-
## No decision points
|
|
52
|
-
|
|
53
|
-
A tool loop or a scheduler has no `agent.decide`: the model picks tools inside one text request and the machine only gates. There is no policy to diff. Traces still drive change, as proposals a person reviews:
|
|
54
|
-
|
|
55
|
-
| Proposal | Evidence from traces | Goes to |
|
|
56
|
-
| --- | --- | --- |
|
|
57
|
-
| An approval rule ("always allow `read_file` under `src/`", "always block `rm -rf`") | Approve/reject verdicts grouped by tool and argument pattern, counted from the log's approval events; `foldTraces` rows for the approval state; `traceCoverage` for approval handlers no run exercised (propose nothing for those) | Environment change request |
|
|
58
|
-
| A cap or threshold (retry cap, max steps, batch size) | Runs ending in the cap's failure final, and how close passing runs came to it | Environment change request |
|
|
59
|
-
| Prompt or tool-description wording of the text request | The rounds below, scored by the objective | A versioned request config the improver may edit |
|
|
60
|
-
|
|
61
|
-
- **Approval rules and caps are environment.** Write each as a change request ([reference.md](reference.md)) with the counts, the pattern, and thread ids. Never write a persisted "always allow" rule yourself. Propose a rule only from a pattern with no rejection and no violation in the training runs, and say how many runs it rests on.
|
|
62
|
-
- **The request config is the policy.** Move the request's system prompt and tool descriptions into one file with a `version` (`requests/agent.ts`, or JSON), and stamp that version as the decider version (`add-traces`). The move itself is a one-time environment edit, like the routing line when a policy is first introduced: it changes the request's declaration to import the config and nothing else, it is behavior-preserving (replay the training runs before and after), and it goes in the same pull request as `v0` of the config. Re-record the environment hash after it. From then on, one round changes the config once, through the same rounds, replays, and pull request as a policy.
|
|
63
|
-
- **Gates for a request-config diff:** environment files hash as recorded; the diff changes wording only, never tool names, input schemas, or the offered tool set (that changes the call shape, so it is an environment change); the overfit check; the rule evals from `add-evals` still pass.
|
|
64
|
-
- **Prompt text in code you may not edit** (a TypeScript `private` method or any other method looked up on the instance): install the config by overriding that method on the instance; a native `#private` method cannot be overridden, so it needs an injection point in the code (an environment change request), and make `v0` byte-identical to the old text. Event-log replay does not re-run model calls, so prove it from recorded calls: `checkParity` (or `createRecorder` with `compareRecordings`) between the old method and `v0` must show no `call-args` divergence.
|
|
65
|
-
- **A value lost between states.** `foldTraces` counts visits and outcomes per state, not what context carried. When one state collects most failures, compare the context entering it with what the state before produced. A value dropped on the way is an environment change request, often the biggest win of the round.
|
|
66
|
-
- **Propose, never promote** applies to all three.
|
|
67
|
-
|
|
68
|
-
## 1. Locate the inputs
|
|
69
|
-
|
|
70
|
-
Find all six before changing anything. If one is missing, stop and ask.
|
|
71
|
-
|
|
72
|
-
| Input | Where to look | If missing |
|
|
73
|
-
| -------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
|
|
74
|
-
| Goal | Root `description` of the policy config or the environment machine | Draft one with measurable success criteria; ask the user to confirm it |
|
|
75
|
-
| Objective | A scorer over outcomes (an eval file, a `score*` function, a benchmark) | Write one from the goal's success criteria; **ask the user to confirm it** |
|
|
76
|
-
| Traces | The `store` passed to `createAgentRuntime` / `runAgent`; its `listThreads`, or recorded ids | Wire a store first (the `add-traces` skill) and collect runs |
|
|
77
|
-
| Scenarios | A scenario list with expected outcomes, split into train and held-out | Propose a fixed split (about 70/30, every category in both); ask to confirm |
|
|
78
|
-
| Vocabulary | The actors, requests, named guards, actions, decision events, and model refs the host implements | Read them off the host code; never invent names |
|
|
79
|
-
| Policy history | The policy files and a `CHANGELOG.md` next to them | Create the changelog with the current policy as `v0` |
|
|
80
|
-
|
|
81
|
-
The objective defines "better". Do not start a round until the user has confirmed it, including how two versions compare when it has more than one field. A good objective has:
|
|
82
|
-
|
|
83
|
-
- a primary metric from the goal (accuracy against expected outcomes and effects, resolution rate). With several, state the rule: one primary, or none worse and one better,
|
|
84
|
-
- a hard constraint: invariant violations, which must be 0,
|
|
85
|
-
- a cost term (model calls, tokens, guard rejections) used as a tie-breaker or a stated budget. Count the calls that reach a model executor, not the environment's decision requests: a policy that answers with a rule makes no model call, and a cost term that counts requests cannot see the saving.
|
|
86
|
-
|
|
87
|
-
## 2. Read
|
|
88
|
-
|
|
89
|
-
Read both machines as text. `describeMachine` puts the goal first:
|
|
90
|
-
|
|
91
|
-
```ts
|
|
92
|
-
import { describeMachine, setupAgent } from "@statelyai/agent";
|
|
93
|
-
|
|
94
|
-
// A TypeScript policy: import it. A JSON policy: lower it.
|
|
95
|
-
const { machine: policy } = setupAgent.fromConfig(policyConfig, { compileSchema });
|
|
96
|
-
const environmentText = describeMachine(environment);
|
|
97
|
-
const policyText = describeMachine(policy);
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
Then find the training threads and compute one outcome row per episode:
|
|
101
|
-
|
|
102
|
-
```ts
|
|
103
|
-
import { getDecisionAttempts, getUsageFromEvents, replay } from "@statelyai/agent/log";
|
|
104
|
-
|
|
105
|
-
// Thread ids name the split and the decider version, as add-traces set up: "train:<deciderVersion>:<scenario>:r<k>".
|
|
106
|
-
// With a policy, the decider version is the policy's version ("v3"); before one exists, the model's label.
|
|
107
|
-
const prefix = `train:${deciderVersion}:`;
|
|
108
|
-
const threadIds: string[] = [];
|
|
109
|
-
let cursor: string | undefined;
|
|
110
|
-
do {
|
|
111
|
-
const page = await store.listThreads!({ prefix, limit: 100, cursor });
|
|
112
|
-
threadIds.push(...page.threadIds);
|
|
113
|
-
cursor = page.cursor;
|
|
114
|
-
} while (cursor !== undefined);
|
|
115
|
-
|
|
116
|
-
const rows = [];
|
|
117
|
-
for (const threadId of threadIds) {
|
|
118
|
-
const scenario = scenarioOf(threadId);
|
|
119
|
-
const entries = await store.read(threadId);
|
|
120
|
-
const { snapshot } = replay(environment, entries); // the recording machine; pure fold, no model calls
|
|
121
|
-
rows.push({
|
|
122
|
-
threadId,
|
|
123
|
-
expected: scenario.expected,
|
|
124
|
-
state: snapshot.value,
|
|
125
|
-
status: snapshot.status,
|
|
126
|
-
output: snapshot.output,
|
|
127
|
-
steps: entries.length,
|
|
128
|
-
tokens: getUsageFromEvents(entries).totalTokens ?? 0,
|
|
129
|
-
failedAttempts: getDecisionAttempts(entries).flatMap(({ attempts }) => attempts),
|
|
130
|
-
});
|
|
131
|
-
}
|
|
132
|
-
```
|
|
133
|
-
|
|
134
|
-
- `store.listThreads({ prefix, limit, cursor })` pages through the threads that hold at least one entry. `cursor` is present only while more ids remain. It is optional on a custom `AgentEventLogStore`; `createInMemoryEventLogStore` implements it. Without it, use the thread ids recorded with each scenario.
|
|
135
|
-
- Prefix thread ids with the split and version so a `listThreads` prefix returns one version's training runs and never a held-out thread. `scenarioOf` is your mapping from a thread id back to its scenario.
|
|
136
|
-
- `store.read(threadId)` returns a thread's entries in log order.
|
|
137
|
-
- `replay(machine, entries)` needs the machine, at the version, that recorded the thread. Here that is the environment: its runtime records every thread, and the policy runs only as a decider at its boundary, so never replay with the policy machine. A thread recorded by a different environment version throws `AgentMachineVersionMismatchError`. Score each thread with the version that recorded it.
|
|
138
|
-
- `getDecisionAttempts(entries)` returns `{ index, eventType, attempts }` for each entry whose decision needed retries: the failed tries before the event landed, or every try of an exhausted decision (`eventType` is then `xstate.error.actor.*`). Each attempt has `failure` (`unknown-event`, `invalid-payload`, or `rejected-by-guard`), `reason`, and the attempted `event` when there was one. A first-try decision records nothing.
|
|
139
|
-
- `agent.decide` invokes journal their attempts on their own. With `decideNext` at the boundary, the host must pass them with the event, `runToQuiescence(runtime, { event, attempts })` or `runtime.transition(state, event, { attempts })`, or they never reach the log.
|
|
140
|
-
- Compute the objective over the rows. This is the baseline for the round.
|
|
141
|
-
|
|
142
|
-
## 3. Diagnose
|
|
143
|
-
|
|
144
|
-
Group the failing training episodes by cause. Check these clusters in order:
|
|
145
|
-
|
|
146
|
-
1. **Invariant violations.** Any violation is a stop: the environment is wrong, not the policy. Write a change request.
|
|
147
|
-
2. **Wrong terminal state.** The episode ended in `escalated` when the label says `issued`, or the reverse. Look at which decision led there.
|
|
148
|
-
3. **Right terminal state, wrong effect.** The episode ended `issued` with the wrong amount, or reached the right state by a costlier path. Compare the chosen event with the offered options at each decision.
|
|
149
|
-
4. **Guard rejections.** The policy keeps proposing events the environment refuses: `rejected-by-guard` entries in the rows' `failedAttempts`. The policy lacks a check the environment already makes.
|
|
150
|
-
5. **Loops and budget exhaustion.** Repeated state values, or a terminal reached because a turn or retry budget ran out.
|
|
151
|
-
6. **Cost hotspots.** Correct episodes that spend model calls where a `choice` over known facts, or a cheap judgment, would decide.
|
|
152
|
-
|
|
153
|
-
Pick the cluster with the largest effect on the objective. Write the hypothesis down before you edit anything (template in [reference.md](reference.md)): cause, evidence (thread ids), the change, and which metric should move.
|
|
154
|
-
|
|
155
|
-
## 4. Propose
|
|
156
|
-
|
|
157
|
-
Make the smallest policy diff that addresses the one cause.
|
|
158
|
-
|
|
159
|
-
- Typical diffs: add a `choice` branch over facts already in context; replace a model call with a judgment actor from the vocabulary; narrow an `agent.decide` `allowedEvents` list; turn an open decision into a closed choice with `payloads`; tighten a prompt; add an `onError` path; lower a model ref to a cheaper one in the vocabulary.
|
|
160
|
-
- TypeScript policy: compare facts directly in a guard or `choice` instead of adding a named guard; keep helpers in the policy file.
|
|
161
|
-
- JSON policy: use only names in the vocabulary. A named guard the host does not implement fails the `lower` gate.
|
|
162
|
-
- Keep the machine `id`. Bump `version`.
|
|
163
|
-
- Keep the root `description` (the goal) unchanged. Changing the goal is the user's decision.
|
|
164
|
-
|
|
165
|
-
## 5. Gate
|
|
166
|
-
|
|
167
|
-
### TypeScript policy
|
|
168
|
-
|
|
169
|
-
Run every check; a failure's output is the repair prompt. Cap repairs at 3.
|
|
170
|
-
|
|
171
|
-
1. **Environment unchanged:** every environment file hashes as recorded. A change is never repaired; discard the candidate.
|
|
172
|
-
2. **Types:** `tsc --noEmit` passes.
|
|
173
|
-
3. **Lint:** `lintAgentMachine(policy)` from `@statelyai/agent/testing` reports no errors.
|
|
174
|
-
4. **Paths:** for probe inputs built from recorded decision requests (`getDecisionRequests(trace, { name })` from `@statelyai/agent/testing`, one per training scenario that reaches the decision: its facts and offered options), `canReach(policy, (snapshot) => snapshot.status === "done" && !isOfferedEvent(snapshot.output), { input })` is `{ reachable: false, complete: true }`: every final output is one of the offered events. Pass `outputs` for any actor the policy invokes, and `errors: { "agent.decide": new Error("down") }` so the fallback path is checked too.
|
|
175
|
-
5. **Source rules:** the policy file is still one `setupAgent` machine; its imports from the environment are only schemas, types, constants, and exported guard functions; it names no actor the environment owns. These are source checks: read the diff, or grep its imports and actor names.
|
|
176
|
-
6. **Overfit:** the diff contains no literal from the scenarios (ids, ticket text, customer ids, order ids). A branch that matches one scenario's data is memorization; discard it.
|
|
177
|
-
7. **Existing evals:** the rule and decision evals from `add-evals` still pass. Tests that pin measured numbers of the current version (a baseline failure list, a call count) are expected to change; update them in the same pull request and list each change in its body.
|
|
178
|
-
|
|
179
|
-
The environment's own `canReach` proofs still cover every policy, because the environment's guards judge every proposed event.
|
|
180
|
-
|
|
181
|
-
### JSON policy
|
|
182
|
-
|
|
183
|
-
Run `verifyAgentConfig` from `@statelyai/agent/validate` with the vocabulary and probe inputs. Probe inputs are one input per training scenario category, chosen to reach every branch the diff touches.
|
|
184
|
-
|
|
185
|
-
```ts
|
|
186
|
-
import { verifyAgentConfig } from "@statelyai/agent/validate";
|
|
187
|
-
|
|
188
|
-
const verified = await verifyAgentConfig(candidate, {
|
|
189
|
-
compileSchema,
|
|
190
|
-
guards: hostGuards,
|
|
191
|
-
actors: hostActors,
|
|
192
|
-
allow: { actors, requests, guards, actions, events, models },
|
|
193
|
-
explore: {
|
|
194
|
-
inputs: probeInputs,
|
|
195
|
-
terminals: ["done"],
|
|
196
|
-
// Both outcomes, so the branches on either side of the classifier are probed.
|
|
197
|
-
outputs: {
|
|
198
|
-
classifyTicket: [
|
|
199
|
-
{ isRefundRequest: true, confidence: 0.9 },
|
|
200
|
-
{ isRefundRequest: false, confidence: 0.9 },
|
|
201
|
-
],
|
|
202
|
-
},
|
|
203
|
-
errors: { classifyTicket: new Error("down") },
|
|
204
|
-
},
|
|
205
|
-
});
|
|
206
|
-
const failed = verified.gates.find((gate) => !gate.ok);
|
|
207
|
-
```
|
|
208
|
-
|
|
209
|
-
- Gates run in order and stop at the first failure: `validate`, `vocabulary`, `lower`, `lint`, `explore`.
|
|
210
|
-
- A failed gate's `detail` names the offending path, state, or name. Send it back verbatim with the rejected config and regenerate. Cap repairs at 3. After 3, discard the candidate, record it, and move to the next cause.
|
|
211
|
-
- `explore` keys besides `inputs` and `terminals` pass through to `explorePaths`: `outputs` (output lists per src: one element is a canned output, two or more fork one branch each), `errors`, `payloads` (one branch per payload), `dedupeBy` (explore each snapshot key once; the key must cover every field a guard reads), `maxDepth`, `maxPaths`.
|
|
212
|
-
- The environment's own `canReach` proofs cover every policy, because the environment's guards judge every proposed event. You did not change the environment, so you do not re-prove it.
|
|
213
|
-
|
|
214
|
-
## 6. Replay
|
|
215
|
-
|
|
216
|
-
Run the candidate on the training scenarios, then on the held-out scenarios. Write every replay to the store under new thread ids that name the split and version, in the scheme `add-traces` set up (`train:v3:<scenario>:r1`, `held-out:v3:<scenario>:r1`), so the next round can list them by prefix. With no persistent store, write each thread's `result.events` to a file.
|
|
217
|
-
|
|
218
|
-
- Run the current policy and the candidate on the same scenarios, the same world state, and the same model refs.
|
|
219
|
-
- Model calls are nondeterministic. Run each scenario k times (k ≥ 3 when calls are not cached) with `repeatRuns` (from `@statelyai/agent/testing`), and compare `passHat(k)` as well as the mean. A candidate that raises the mean but lowers pass^k made the agent less reliable.
|
|
220
|
-
- Compute the objective, invariant violations, and cost per split, for both versions.
|
|
221
|
-
|
|
222
|
-
## 7. Recommend or discard
|
|
223
|
-
|
|
224
|
-
State the tolerance before you replay. Recommend only when every condition holds:
|
|
225
|
-
|
|
226
|
-
- invariant violations are 0 on both splits,
|
|
227
|
-
- the objective on the held-out split is better than the current policy's,
|
|
228
|
-
- no single scenario regresses beyond the tolerance (for example, a scenario that passed in all k runs now fails in any),
|
|
229
|
-
- cost stays within the stated budget, if there is one.
|
|
230
|
-
|
|
231
|
-
Otherwise discard the candidate and keep the current policy. Either way, append an entry to the `CHANGELOG.md` next to the policies: version, hypothesis, diff summary, gate results, before and after per split, and the decision. Record discarded candidates too; they stop the next round from retrying the same idea.
|
|
232
|
-
|
|
233
|
-
### The pull request
|
|
234
|
-
|
|
235
|
-
A recommended candidate becomes one pull request (or, with no remote, one branch and a diff file), opened against the repository's integration branch:
|
|
236
|
-
|
|
237
|
-
- **The diff:** the policy file(s) and the new `CHANGELOG.md` entry; the runner's routing line, when this candidate introduces the policy; and updated pinned measurements, each listed in the body. Nothing else.
|
|
238
|
-
- **The body:** the hypothesis, the scorecard (objective, violations, and cost before and after, train and held-out separately, k runs each), every scenario that changed outcome, the gate results, and the thread ids a reviewer can replay.
|
|
239
|
-
- **One candidate per pull request.** Leave your working tree on the current policy. Later rounds build on the current policy, not on an unmerged candidate.
|
|
240
|
-
|
|
241
|
-
## 8. Stop and report
|
|
242
|
-
|
|
243
|
-
Stop when any of these holds:
|
|
244
|
-
|
|
245
|
-
- 3 rounds in a row without a promotion,
|
|
246
|
-
- the goal's success criteria are met on the held-out split,
|
|
247
|
-
- the remaining failures need an environment change (write a change request),
|
|
248
|
-
- the vocabulary lacks a capability the fix needs (ask for it; name the actor, request, or guard and its input and output).
|
|
249
|
-
|
|
250
|
-
Report to the user: rounds run, each candidate with its hypothesis and decision, the pull request link (or, with no remote, the branch and diff file), the objective before and after on both splits, violations, cost, and any change requests.
|
|
251
|
-
|
|
252
|
-
## Worked example
|
|
253
|
-
|
|
254
|
-
[`examples/self-improving-refunds`](https://github.com/statelyai/agent/blob/next/examples/self-improving-refunds) in the `statelyai/agent` repo (not in the installed package) runs this loop end to end for a JSON policy:
|
|
255
|
-
|
|
256
|
-
- the refund desk is the environment: business rules as guards, and an `illegallyIssued` sentinel proven unreachable,
|
|
257
|
-
- `playbooks/*.json` are policy versions, with the vocabulary listed in `playbooks/README.md`,
|
|
258
|
-
- `improve.ts` is the loop: propose, gate, replay, promote,
|
|
259
|
-
- `benchmark.ts` compares a raw tool loop, the desk with a raw model deciding, and the desk with a playbook deciding.
|
|
260
|
-
|
|
261
|
-
Read the directory before you start. File contents change between releases.
|
|
@@ -1,101 +0,0 @@
|
|
|
1
|
-
# Improve-machine templates
|
|
2
|
-
|
|
3
|
-
Templates for the `improve-machine` skill: a round checklist, the hypothesis, the proposer prompt, the changelog entry, and the environment change request.
|
|
4
|
-
|
|
5
|
-
## Round checklist
|
|
6
|
-
|
|
7
|
-
```
|
|
8
|
-
[ ] Goal confirmed (root description, measurable success criteria)
|
|
9
|
-
[ ] Objective confirmed by the user (primary metric, violations = 0, cost term)
|
|
10
|
-
[ ] Train / held-out split fixed; tolerance stated
|
|
11
|
-
[ ] Vocabulary listed (actors, requests, guards, actions, events, models)
|
|
12
|
-
[ ] Baseline: objective, violations, cost per split for the current version
|
|
13
|
-
[ ] Failures clustered (train only); one cause picked
|
|
14
|
-
[ ] Hypothesis written
|
|
15
|
-
[ ] Candidate: smallest diff, id kept, version bumped, description unchanged (or, for a new policy: v0 recorded, v1 reproduces current behavior plus one rule)
|
|
16
|
-
[ ] Gates passed: TS (env unchanged, tsc, lint, paths, source rules, overfit, evals) or JSON (verifyAgentConfig) (repairs used: _/3)
|
|
17
|
-
[ ] Replayed train, then held-out, k runs each, new thread ids
|
|
18
|
-
[ ] Promotion rule applied; CHANGELOG entry written
|
|
19
|
-
[ ] Recommended candidate opened as one pull request with its scorecard; not merged
|
|
20
|
-
[ ] Stop condition checked
|
|
21
|
-
```
|
|
22
|
-
|
|
23
|
-
## Hypothesis
|
|
24
|
-
|
|
25
|
-
```
|
|
26
|
-
Cause: <one sentence, the mechanism, not the symptom>
|
|
27
|
-
Evidence: <thread ids and what each shows>
|
|
28
|
-
Change: <the one diff, in words>
|
|
29
|
-
Expected effect: <metric> moves from <x> to about <y>; nothing else changes
|
|
30
|
-
Risk: <scenarios that could regress, and why>
|
|
31
|
-
```
|
|
32
|
-
|
|
33
|
-
## Proposer prompt
|
|
34
|
-
|
|
35
|
-
Use this when a model proposes the candidate. Fill every slot from real data.
|
|
36
|
-
|
|
37
|
-
```
|
|
38
|
-
You improve a policy machine. You may change only the policy config below.
|
|
39
|
-
|
|
40
|
-
{describeMachine(environment)}
|
|
41
|
-
|
|
42
|
-
Current policy (version {n}):
|
|
43
|
-
{the policy file's source, or JSON.stringify(policyConfig, null, 2)}
|
|
44
|
-
|
|
45
|
-
Vocabulary (use no other names):
|
|
46
|
-
- actors: {actors}
|
|
47
|
-
- requests: {requests}
|
|
48
|
-
- named guards: {guards}
|
|
49
|
-
- named actions: {actions}
|
|
50
|
-
- decision events: {events}
|
|
51
|
-
- model refs: {models}
|
|
52
|
-
|
|
53
|
-
Objective on the training scenarios: {objective summary}
|
|
54
|
-
Diagnosed cause: {hypothesis.cause}
|
|
55
|
-
Evidence: {failing rows: scenario, expected, reached state, rejections, tokens}
|
|
56
|
-
|
|
57
|
-
Return the complete policy: the TypeScript file, or one AgentWorkflowConfig as
|
|
58
|
-
JSON. Make the smallest change that addresses the diagnosed cause. Keep "id" and
|
|
59
|
-
"description". Set "version" to {n + 1}. Never match on a scenario's data.
|
|
60
|
-
For a TypeScript policy, keep the result one `setupAgent` machine. Import from
|
|
61
|
-
the environment only schemas, types, constants, and exported guard functions.
|
|
62
|
-
Do not invoke tools the environment owns.
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
On a failed gate, append:
|
|
66
|
-
|
|
67
|
-
```
|
|
68
|
-
Your previous config was rejected.
|
|
69
|
-
Gate '{gate.name}' failed:
|
|
70
|
-
{gate.detail}
|
|
71
|
-
Return a corrected config.
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
## Changelog entry
|
|
75
|
-
|
|
76
|
-
```md
|
|
77
|
-
## v{n} — {recommended | discarded}
|
|
78
|
-
|
|
79
|
-
- Hypothesis: {cause} → {change}
|
|
80
|
-
- Diff: {states or transitions touched}
|
|
81
|
-
- Gates: {TS: environment ✓ tsc ✓ lint ✓ paths ✓ overfit ✓ evals ✓ | JSON: validate ✓ vocabulary ✓ lower ✓ lint ✓ explore ✓} (repairs: {r})
|
|
82
|
-
- Train: objective {a} → {b}, violations {v}, cost {c0} → {c1}
|
|
83
|
-
- Held-out: objective {a} → {b}, violations {v}, cost {c0} → {c1}
|
|
84
|
-
- Regressions: {none | scenario ids}
|
|
85
|
-
- Decision: {why}
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
## Environment change request
|
|
89
|
-
|
|
90
|
-
Write this instead of editing the environment.
|
|
91
|
-
|
|
92
|
-
```md
|
|
93
|
-
# Environment change request: {short title}
|
|
94
|
-
|
|
95
|
-
- Goal criterion blocked: {the success criterion no policy can meet}
|
|
96
|
-
- Evidence: {thread ids; the state and guard that block it}
|
|
97
|
-
- Policies tried: {versions and why each could not get there}
|
|
98
|
-
- Proposed change: {the state, guard, or event to add or change, in words}
|
|
99
|
-
- Invariants affected: {which, and the canReach check to rerun}
|
|
100
|
-
- Owner: a person reviews and merges this; the improver does not.
|
|
101
|
-
```
|
|
@@ -1,205 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: loop-to-machine
|
|
3
|
-
description: Convert existing agent code (a while-loop around a model, a tool loop, a tool-call scheduler, ad-hoc if/else control flow) into an XState v6 state machine with @statelyai/agent, and prove with a parity report that it behaves like the code it replaces. Use when asked to "turn this agent into a state machine", "port my loop to xstate", "make this a statechart", "refactor this agent into a machine", or "split environment and policy" for code that already runs.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Convert agent code to a machine
|
|
7
|
-
|
|
8
|
-
You replace the control flow of working agent code with a machine, and show where it behaves like the old code and where it differs on purpose. Model calls, tools, and retry code move across unchanged.
|
|
9
|
-
|
|
10
|
-
```
|
|
11
|
-
inventory → pin the baseline → environment → (policy) → runner → proofs → parity → hand back
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
## Default track: wrap first, carve per commit
|
|
15
|
-
|
|
16
|
-
For an async workflow (not a hand-rolled loop you are asked to replace wholesale), start here, and use the sections below for each carved phase.
|
|
17
|
-
|
|
18
|
-
1. **Step 0: wrap and record.** Move every side effect behind one `effects` object of async functions (the only edit to the old code). Then `wrapWorkflow({ id, effects, run: (input, effects) => old(input, effects) })` from `@statelyai/agent`. Pin the baseline with `checkParity` (`@statelyai/agent/testing`): `{ scenarios, effects: (s) => fakes, before: old, after: (input, fx) => step0.run(input, { effects: fx }) }`; every row `equivalent`. Ship it: inspection, traces, the event log, `signal`, a recorded baseline.
|
|
19
|
-
2. **Carve one phase per commit.** One phase becomes states (§3); the rest stays the old helper, invoked as a `createAsyncLogic` actor built per run from `effects`. Keep the `(input, effects)` runner signature with a third `options` argument passed through to `runAgent` (`threadId`, `store`, `metadata`, `onTrace`, `inspect`), so traces and inspection never need a copy of the runner. Move any prompt or warning text the carved phase sends into a versioned request config as you carve it: that file is what `improve-machine` may later edit. Run `checkParity` against the old code after each carve. Add the §6 proofs for the carved phase only.
|
|
20
|
-
|
|
21
|
-
A wrapped workflow is a black box: one state, no mid-function resume, no `canReach` about its inside. Do not report proofs before the phase is carved. Worked example: `examples/strangler-fig`. A class whose loop calls its own private model/tool methods: replace those methods on the instance inside `run` so they call `effects` (no edit to the class). Effect-TS generators, Redux thunks, LangGraph graphs, closures in DI factories, and step loops inside the model SDK have their own seams: [docs/strangler-fig.md](../../docs/strangler-fig.md#seams-for-other-shapes). Effects bound to one call's live objects: create the workflow once, pass `run(input, { effects })` per call. Scrub generated ids and timestamps: `compare.normalize` for effect arguments, `normalizeOutput` for the output, `normalizeError` for thrown errors.
|
|
22
|
-
|
|
23
|
-
**A phase in the middle of a private method:** copy only that phase's lines into the carved actor, list the copied line ranges in a comment, and let `checkParity` call-args catch drift. When a public per-turn hook exists, replace that method on the instance instead and let the machine decide at that boundary ([reference.md](reference.md#a-phase-in-the-middle-of-a-method)).
|
|
24
|
-
|
|
25
|
-
This is step 1 of 4. Next: `add-traces`, then `add-evals`, then `improve-machine`. To author a machine with no existing code, use `generate-machine`. Recording, replay, and the full parity test code are in [reference.md](reference.md).
|
|
26
|
-
|
|
27
|
-
## Hard rules
|
|
28
|
-
|
|
29
|
-
- **Pin the baseline before you change anything.** The parity test runs against the old code, which stays in the repo, untouched, until a person deletes it. When the old code cannot run, the baseline is a written behavioral spec (see "No model in the loop").
|
|
30
|
-
- **Never promise "behaves exactly like".** Promise what the parity evidence shows, every difference labeled.
|
|
31
|
-
- **Never copy a rule break.** Where the old code broke a rule, the machine blocks it. That difference is `intended`, not a regression.
|
|
32
|
-
- **Rules are code, not prompt text.** A prompt-only rule becomes a guard; the prompt line stays only as context.
|
|
33
|
-
- **Business rules belong to people.** Ambiguous rule (which threshold, which order)? Ask; never pick silently.
|
|
34
|
-
- **Port call shapes unchanged first.** An SDK tool call stays a text request with the same tools. Changing it (to `agent.decide` or `agent.act`) is a separate step with its own `INTENDED` divergences.
|
|
35
|
-
|
|
36
|
-
## 0. Before you write code
|
|
37
|
-
|
|
38
|
-
- Text requests need exactly one non-empty `prompt` or `messages`. `setupAgent` throws `invalid-request` when both are omitted; resolving neither or both at run time fails validation.
|
|
39
|
-
- Actors: `createAsyncLogic<Output, Input>({ run })` from `xstate` (no `fromPromise` in v6); without the generics `onDone`'s `output` is untyped.
|
|
40
|
-
- `onIdle` return events need `as const` on `type`, or the union does not narrow.
|
|
41
|
-
- In `onError`, the error is `event.error`; there is no `error` argument.
|
|
42
|
-
- Context, input, events, outputs: JSON only. No `Date`, `Error`, `Map`, class instances, functions; store ISO strings and `String(error)`.
|
|
43
|
-
- Every executor slot is optional in the type; assert the ones you call.
|
|
44
|
-
- `request.metadata.facts` is `unknown`; parse it with the exported facts schema.
|
|
45
|
-
- A `type: "choice"` state has no `enq` or actions; when routing must emit, use `always` on a normal state.
|
|
46
|
-
- `compareRuns` scenarios reserve `expected: string` for the outcome; name other expectations differently.
|
|
47
|
-
- `explorePaths` `errors[src]`: an array forks one failing branch per element (`[[...]]` for an array-valued failure). `outputs` and `errors` branches are alternatives, not a sequence: drive "fails twice, then succeeds" from a context counter.
|
|
48
|
-
|
|
49
|
-
### Inspect one event
|
|
50
|
-
|
|
51
|
-
`transition(machine, snapshot, event)` from `xstate` is pure: it returns the next snapshot and the effects it would run, and runs none of them. Use it to answer "what happens if this event arrives here?" before writing a test or running the loop.
|
|
52
|
-
|
|
53
|
-
```ts
|
|
54
|
-
import { initialTransition, transition } from "xstate";
|
|
55
|
-
|
|
56
|
-
const [s0] = initialTransition(machine, { maxTurns: 2 });
|
|
57
|
-
const actorId = Object.keys(s0.children)[0]; // the running invoke
|
|
58
|
-
const [s1, effects] = transition(machine, s0, {
|
|
59
|
-
type: "xstate.done.actor",
|
|
60
|
-
actorId,
|
|
61
|
-
output: { done: false },
|
|
62
|
-
});
|
|
63
|
-
s1.value; // next state
|
|
64
|
-
s1.context; // next context
|
|
65
|
-
effects; // what would run (spawn, start, stop, ...); nothing ran
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
- Same snapshot object back and no effects: the event is unhandled there.
|
|
69
|
-
- Fail an invoke with `{ type: "xstate.error.actor", actorId, error }`.
|
|
70
|
-
- To start mid-run, pass a live snapshot: `restoreActor(machine, persisted).getSnapshot()` (`xstate/persist`) for a persisted one, or the snapshot `replay` (`@statelyai/agent/log`) folds from a journal. A raw persisted snapshot throws.
|
|
71
|
-
|
|
72
|
-
## 1. Inventory the code
|
|
73
|
-
|
|
74
|
-
Read the loop and fill this table before writing anything. Each row maps to a machine element.
|
|
75
|
-
|
|
76
|
-
| In the code | Becomes |
|
|
77
|
-
| --- | --- |
|
|
78
|
-
| A tool with a side effect someone must gate (refund, send) | An actor invoked by one state, entered only through a guarded transition |
|
|
79
|
-
| A read-only tool (search, lookup) | An actor the machine invokes, or a tool on a text request |
|
|
80
|
-
| A rule: an `if`, a prompt line, a check after the fact | A guard on the transition it protects |
|
|
81
|
-
| A model call that produces content | A text request in `setupAgent({ requests })`, invoked by one state |
|
|
82
|
-
| One SDK tool call that picks the action and carries its content (`send_email({ … })`) | A text request with those tools, then shape guards on the returned call: exactly one, offered, schema-valid. A tool that keeps `execute` runs inside the SDK (old timing); to gate it, drop `execute` and invoke the real work after the verdict. Not `agent.decide` yet: it hides your tool names and descriptions |
|
|
83
|
-
| N tool calls per model turn, each needing approval | First port: the request returns the calls; a state holds them in context and waits for a verdict per call. Later step: `agent.act` ([docs/act.md](../../docs/act.md)), each call an event the state queues |
|
|
84
|
-
| A persisted "always allow" rule | Data loaded into input or context, read by a guard. Saving a rule is a recorded effect |
|
|
85
|
-
| A tool the browser runs (no `execute`) | The request ends with the tool calls; an idle state waits for `TOOL_RESULTS` |
|
|
86
|
-
| A point where the model picks what to do | A decision state: `agent.decide`, usually a closed choice |
|
|
87
|
-
| A check on model output, then retry or repair | A checking actor or guard, and a `choice` state routing to repair, next candidate, or a failure final |
|
|
88
|
-
| Provider retry/backoff around the SDK call | Stays in the executor, unchanged |
|
|
89
|
-
| Context overflow caught → compact → retry | A compaction request state, entered from the call's `onError` on an overflow error, with a retry cap |
|
|
90
|
-
| Recursive re-entry (`this.run()` inside the loop) | A transition back to the first state, under the recursion's cap (or a new one, `intended`) |
|
|
91
|
-
| A value pulled out of the input | Code when the format is fixed; a text request when not |
|
|
92
|
-
| A step every correct run takes in order | A state the machine enters itself |
|
|
93
|
-
| A person in the loop (`await approve()`, a returned `pending`, an ask-a-person tool) | An idle state with `meta.interaction` and `onIdle`, or an invoked wait actor (§5) |
|
|
94
|
-
| `AbortSignal` or a cancel method | `runAgent`'s `signal`, or a `CANCEL` event (§5) |
|
|
95
|
-
| Hooks or bus emissions the UI depends on | `enq.emit(...)` (declared under `emitted`) or `onTransition`, same payloads |
|
|
96
|
-
| Message history in a class field | Context when a guard or prompt reads it; else host-owned, passed as `messages`. Write down which |
|
|
97
|
-
| A streaming iterator | The `streamText` executor keeps it and pipes to the old consumer. In a wrapped workflow, an effect returning it is observed: `effect.done` carries the chunks once read, a mid-stream throw is `effect.error` |
|
|
98
|
-
| A step loop inside the model SDK (`stopWhen`, `maxSteps`) | One executor or effect, unchanged. Carve the control flow around it: handoff, approval, caps |
|
|
99
|
-
| A synchronous host callback, a diagnostic logger | Not an effect: the machine factory closes over it |
|
|
100
|
-
| An effect result that is a live object, a write to host state by reference | A name, id, or change in the result, with a host-side registry. Parity cannot see the live object or the write |
|
|
101
|
-
| Mutable state on the class instance | JSON context fields, or host state passed as input |
|
|
102
|
-
| A cap on a loop the machine keeps / removed | A `choice` state against an exported constant / nothing, with the reason recorded |
|
|
103
|
-
| A class-shaped runtime (`run`/`continue`/`abort`/`restore`/`snapshot`) | A subclass overriding the loop methods to call `runAgent`, inheriting the rest. The conversation may stay host-owned at first (a snapshot then cannot resume alone); moving it into context is a later labeled step |
|
|
104
|
-
| A function that returns a live stream at once (`streamText`-shaped) | Return at once from a `TransformStream` the run writes into: an `emit` effect writes each chunk, the run closes the writer when it succeeds and aborts it with the error when it fails, and the result's promise fields resolve from the run. The machine owns the loop; the caller reads the stream as before |
|
|
105
|
-
| A live receiver the effects need (a class instance, an open stream, a cancellation token) | A host registry keyed by id: context and events carry the id, each effect looks the object up |
|
|
106
|
-
| Each distinct return | A final state |
|
|
107
|
-
| A terminal tool with a side effect (`escalate`) | A state that invokes it, then the final; `onError` to a `failed` final |
|
|
108
|
-
|
|
109
|
-
Number the rules in your notes. Each ends as a guard, a state ordering, or a written reason it was dropped.
|
|
110
|
-
|
|
111
|
-
A loop with no decision point is still worth converting: caps, rules, and failure paths become provable. Skip step 4 and the decision parts of step 5.
|
|
112
|
-
|
|
113
|
-
## 2. Pin the baseline
|
|
114
|
-
|
|
115
|
-
The parity test exists before the machine does.
|
|
116
|
-
|
|
117
|
-
1. **Scenarios.** Reuse the code's tests, fixtures, or benchmarks: an `id` each and, where known, the `expected` outcome. If none exist, draft 6–10 covering every terminal and rule; ask the user to confirm them.
|
|
118
|
-
2. **Deterministic stand-in models.** The code's mocks, or a function from the request (prompt or `name`) to a fixed answer ([`examples/mock-model.ts`](https://github.com/statelyai/agent/blob/next/examples/mock-model.ts), repo only). Two: one follows the rules, one breaks them. A seeded model seeds from the scenario id, fresh per run.
|
|
119
|
-
3. **Ground truth.** Outcomes and effects from the world, not the agent's report. `effects` holds **every write the world records**: refunds, approvals, verifications, questions asked, escalations. Model-written text: compare count or presence. Violations come from the world, or a plain function over the effects.
|
|
120
|
-
4. **A baseline with no function to call.** Loop inline in a framework handler: call the handler with stand-in model and storage. Do not extract the loop first.
|
|
121
|
-
5. **Record golden runs.** `createRecorder` (`@statelyai/agent/testing`) wraps model and tools; run the old code per scenario and stand-in, save JSON fixtures. AI SDK model → `recorder.model`; `openai` client → `recorder.openai`; anything else → `recorder.client` on the SDK object (`methods: ["messages.create"]`), or `recorder.fn` on a function. Code, streams, and `redact`/`serialize`/`revive`: [reference.md](reference.md#any-other-model-client).
|
|
122
|
-
6. **The harness check.** The old code on both sides of `compareRuns`, per stand-in: baseline equals candidate in every row; all `same` with the rule-following model, `regression` by design with the rule-breaker. [reference.md](reference.md#the-parity-harness).
|
|
123
|
-
|
|
124
|
-
### No model in the loop
|
|
125
|
-
|
|
126
|
-
For code with no model call (a tool scheduler, an approval queue), and old code that cannot run (a class wired to a bus, no callable entry). A tool runner whose text request picks tools has a model in the loop: keep recorders and replay (§7); use the spec baseline only where the old code cannot run.
|
|
127
|
-
|
|
128
|
-
- **Skip recorders only when nothing calls a model.** Then replay (§7) does not apply.
|
|
129
|
-
- **Baseline as spec.** When the old code cannot run, `baseline: async (s) => SPEC[s.id]`: outcome, effects, and violations a person confirmed from the old code and its docs, before the machine exists. Skip the harness check; the hand-back says the baseline is a spec, not a run.
|
|
130
|
-
- **Stand-ins become scripted worlds:** the proposed calls, the order approvals arrive in, which tool fails.
|
|
131
|
-
|
|
132
|
-
## 3. Author the environment
|
|
133
|
-
|
|
134
|
-
The environment is the machine: states, guards that encode the rules, side-effecting actors, an invariant sentinel. Use `setupAgent` (from `@statelyai/agent`) with schemas for `input`, `context`, `output`, and every event. A zod `z.object` strips keys it does not list at the boundary; list every field the old code reads, or use `z.looseObject`. It takes any Standard Schema library (zod, valibot, arktype); the package ships none, so install one. Pass the schemas as top-level fields, `setupAgent({ context, input, output, events })` (`context` is required), or as one `schemas: createAgentSchemas({ ... })`; a partial object under `schemas` does not typecheck. Export the event schemas as one object for a later policy. A gated action (closed-choice `agent.decide`, guard in the transition) is in [reference.md](reference.md#a-gated-action).
|
|
135
|
-
|
|
136
|
-
- **Do the deterministic steps yourself.** Identity, lookup, ownership: states in order. A model answers only what needs judgment.
|
|
137
|
-
- **Write simple guards inline** (`context.turn < context.maxTurns`): `describeMachine(machine, { source: true })` prints them. Export a guard as a plain function only when a test must loosen it, build the machine in a factory (`createDeskMachine({ refundIsLegal })`), and pass the same functions as `helpers` so the read-back still shows their bodies: `describeMachine(createDeskMachine(guards), { source: true, helpers: guards })`.
|
|
138
|
-
- **Add an invariant sentinel** for the rule that must never break: a final state described `SENTINEL: …`, entered from an `always` on the side-effect state (`paying`), which runs before the invoke starts. The check is its own function, not the guard, so a loosened guard does not loosen it.
|
|
139
|
-
- **A person's step** is an idle state whose `meta.interaction` tells a host what to render: `{ label, events: { APPROVE: { label: "Approve" }, REJECT: { label: "Reject" } } }`, or `textEvent: { type: "CUSTOMER_REPLIED", field: "reply" }` for free text.
|
|
140
|
-
- **Actors are placeholders** in `setupAgent({ actors })`; pass the real ones with `runAgent(machine, { actors })`.
|
|
141
|
-
- **Final outputs carry the result** (`{ outcome, orderId, amount }`); the runner rebuilds the old return type from it and the world.
|
|
142
|
-
- **Declare `version`** in `createMachine({ id, version: "1", ... })` on every recorded machine; bump it on every behavior change. Traces stamp it (`add-traces` §1).
|
|
143
|
-
|
|
144
|
-
## 4. A policy, only if the code already has one
|
|
145
|
-
|
|
146
|
-
Keep the decision in the environment as one `agent.decide`, with no policy machine, unless the old code has decision logic beyond one model call: a rule ladder, strategy heuristics, a cheap check before the model. `improve-machine` introduces a policy when traces show a rule worth adding. If the code has that logic, port it as a policy machine that answers the decision: [policy.md](policy.md).
|
|
147
|
-
|
|
148
|
-
## 5. The runner
|
|
149
|
-
|
|
150
|
-
Write `runMachine` with the old function's signature and result type.
|
|
151
|
-
|
|
152
|
-
- **The existing SDK call becomes the executor.** `generateText` (or `streamText`) in `executors`, as the old call: same model, system prompt, tools, options, retry code; return `{ result }`. Pass the conversation as `messages: ({ input }) => input.messages`.
|
|
153
|
-
- **A streaming route keeps its response, not its signature.** `createAiSdkExecutors().streamText` forwards text chunks only, so wrap the old call: pipe its stream to the old consumer, then return `{ result: finalValue, messages: responseMessages }` so later requests can rebuild the assistant and tool history from `output.messages`.
|
|
154
|
-
- `runAgent(machine, { input, executors, actors, onIdle })`. `onIdle` answers each idle state with the old code's call.
|
|
155
|
-
- `executors.decide` reads `request.options` (each `{ event }`) and `request.metadata?.facts`.
|
|
156
|
-
- JSON-safe input and events: with a store, an unloggable value fails the run.
|
|
157
|
-
- Rebuild the old result from `result.output` and the world. A field with no meaning now keeps its type and a comment.
|
|
158
|
-
|
|
159
|
-
**Approvals off a message bus.** The rule: nothing else runs while the answer is awaited → idle state + `onIdle`. Other calls in flight, or answers in any order → never idle, so invoke a wait actor: `createCallbackLogic` that subscribes to the bus, sends `APPROVED`/`REJECTED` with the call id to its parent, and unsubscribes on exit. For N calls per batch (per-call approve/reject/always, a barrier before execute, cancel-all), use `createApprovalBatchMachine` from `@statelyai/agent/machines` ([docs/machines-presets.md](../../docs/machines-presets.md)). Only when it does not fit, hand-write a child machine per batch.
|
|
160
|
-
|
|
161
|
-
**Host events mid-run** (cancel, abort, a user message while a request runs):
|
|
162
|
-
|
|
163
|
-
- A legacy body that owns abort (it returns its own `stopped` result) during a partial carve: pass the signal to the legacy body only, not to `runAgent`, so its result stands and reaches the machine as a normal final.
|
|
164
|
-
- Abort: `runAgent`'s `signal` alone is a hard stop: the run settles `{ status: "error", cause: "aborted" }` and the machine sees nothing. Add `abortWith: { event, timeout? }` when the machine should handle it (a `cancelled` final, cleanup invokes); it falls back to `aborted` if the event is unhandled, if the machine is still active once the event has been applied and the run settles, or on timeout: the handled path must reach a final or idle state before you can rely on its cleanup.
|
|
165
|
-
- A cancel the machine handles (cleanup, a `cancelled` final): a `CANCEL` event on the parent state. With `runAgent`, pass `inbox: createAgentInbox()` and call `inbox.send(event)`; an inbox serves one run. A hand-written `createAgentRuntime` loop uses `runtime.send(event)` ([docs/hosts.md](../../docs/hosts.md)).
|
|
166
|
-
|
|
167
|
-
## 6. Proofs that need no model
|
|
168
|
-
|
|
169
|
-
From `@statelyai/agent/testing` (example options in [reference.md](reference.md#proof-options)):
|
|
170
|
-
|
|
171
|
-
- `lintAgentMachine`: no errors. An invoke whose error must reach the caller unchanged has no `onError` (the runner rethrows `result.error`); pass `disable: ["invoke-without-on-error"]` for that machine. For one invoke that cannot fail (a callback bridge), pass `ignore: [{ code: "invoke-without-on-error", path: "<state>" }]` instead.
|
|
172
|
-
- `canReach(machine, "<sentinel>", options)` returns `reachable: false` **and** `complete: true`, for each distinct input shape (with and without an order id; loop over them).
|
|
173
|
-
- `options.payloads(snapshot)` returns payload lists per event type, on both sides of every guard boundary (`0`, the threshold and one over, each producible amount and one over); without it `complete: true` proves nothing about boundaries. A person's free-text event needs one payload per branch.
|
|
174
|
-
- `outputs`: every output each actor can return; `errors`: each actor fails. An actor that never settles (a callback bridge, a long-lived listener) gets `outputs: { src: [] }`: its invoke stays pending and only events move the machine.
|
|
175
|
-
- **A loop** needs `options.dedupeBy`, or the search hits `maxDepth` with `complete: false`. Key on the state value plus the fields read at the loop head (counters, flags); drop fields that only change within one pass. A search over a few seconds: cut the key first. A counter cap of N needs `maxDepth` above N times the events per pass; with the default the search ends `complete: false`.
|
|
176
|
-
- A state passed through within one step (left at once by `always`, or a `choice` state) counts as reached: `canReach` returns the path through it and `explorePaths` lists it in `reachedStates`, so it works as a sentinel.
|
|
177
|
-
- **A machine that spawns or invokes children**: pass `children: "coupled"`, `childEvents` (only the types the host sends a child directly; the parent's sends need no entry), and `hostEvents` when child status events must not be forged by the host. A witness is then a run the machines can make together. Read `vacuous`. Use `interleave: "pending"` only for the invariant that needs host events raced against in-flight work, and for a long-lived invoke that hands calls back through an inbox: without it, events that arrive while the invoke runs are never explored.
|
|
178
|
-
- A loosened guard makes `canReach` true; test that too.
|
|
179
|
-
- Each normal final is reachable (the search is not vacuous).
|
|
180
|
-
|
|
181
|
-
Type options with `ExplorePathsOptions` and callback snapshots with `ExploreSnapshot` (`@statelyai/agent/testing`), not your app's `AnyMachineSnapshot`.
|
|
182
|
-
|
|
183
|
-
## 7. Parity
|
|
184
|
-
|
|
185
|
-
Switch the candidate to `runMachine`; run per stand-in (or against the spec). Code in [reference.md](reference.md#the-parity-harness).
|
|
186
|
-
|
|
187
|
-
- Every `regression` row: fix the machine, or ask the user when the old behavior was wrong but broke no rule. Differing outcomes: the answer may become `expected`. Differing effects only: list the row in a reviewed-differences map, exact difference asserted, with who accepted it. Never edit a scenario to make a row pass.
|
|
188
|
-
- **Run the old suite against the port**, the strongest evidence when the old code has tests: mock the old module (`vi.mock('../old-module', ...)`) with a subclass whose loop methods call `runAgent`, plus a canary counter the machine increments. Assert the exact count (the N runs the suite drives), not `> 0`. A test that spies on internals the port no longer calls is a structural failure: mark it `it.fails` with a comment naming why; never delete it. Snapshots are stored under the importing file's name, so a `toMatchSnapshot` there writes a new snapshot and passes: copy the old `__snapshots__/<old>.snap` to the swap file's name and run with `CI=true`, so a missing snapshot fails.
|
|
189
|
-
- **Swap pitfalls.** A mock factory that imports the port deadlocks when the port imports the mocked module: load the port with a dynamic import in `beforeAll` and call it from inside the factory's functions. A mock that misses the code's own import of the module: patch the class prototype from a setup file that imports the module dynamically. Code in [reference.md](reference.md#old-suite-swap).
|
|
190
|
-
- **Timing.** `runAgent` starts asynchronously, so fire-and-forget work the old code starts at the same time may settle in another order. An old test asserting exact action order across concurrent tasks is structural only after you confirm the actions commute and their order is not part of the contract; an order users or storage see is behavioral.
|
|
191
|
-
- Every `intended` row: its reason is a real rule. A fixed rule break with the same outcome is `intended` too.
|
|
192
|
-
- With recordings, replay each against the machine (`createRecorder({ replay })`, `compareRecordings`): no key, no side effects, every divergence in `INTENDED` with who accepted it, including a guard that blocks a rule break. Semantics in [reference.md](reference.md#replay-equivalence).
|
|
193
|
-
- With no model in the loop, parity is the whole proof: the spec baseline plus §6. Say so.
|
|
194
|
-
|
|
195
|
-
## 8. Hand back
|
|
196
|
-
|
|
197
|
-
Report, in this order:
|
|
198
|
-
|
|
199
|
-
1. The files (environment, runner, tests, any policy); the old code untouched.
|
|
200
|
-
2. The rule list, each mapped to a guard, a state, or why it was dropped.
|
|
201
|
-
3. The parity table per stand-in model: `same / intended / regression`, one line per `intended` row with its reason. Say whether the baseline was a run or a spec.
|
|
202
|
-
4. The `canReach` results and the loosened-guard test.
|
|
203
|
-
5. The machine rendered: `describeMachine(machine)` output, and how to watch it live in the Stately inspector (install `@statelyai/sdk` and its optional peer `ws` for a local relay, and pass `inspector.inspect` as `runAgent`'s `inspect`; see [docs/observability.md](../../docs/observability.md)).
|
|
204
|
-
6. Anything you had to decide that the user should confirm, including where message history lives.
|
|
205
|
-
7. Next step: `add-traces`.
|