@statelyai/agent 2.0.0-alpha.37 → 2.0.0-alpha.39

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (135) hide show
  1. package/CHANGELOG.md +51 -0
  2. package/dist/{act-CrX0JwWY.cjs → act-B0CVbmo4.cjs} +1 -1
  3. package/dist/{act-B9hLQls6.mjs → act-Bsz-hUfD.mjs} +1 -1
  4. package/dist/ai-sdk.cjs +4 -4
  5. package/dist/ai-sdk.d.cts +2 -3
  6. package/dist/ai-sdk.d.mts +2 -3
  7. package/dist/ai-sdk.mjs +4 -4
  8. package/dist/{decision-m5hGmHpF.mjs → decision-B41YBx5U.mjs} +1 -1
  9. package/dist/{decision-CMeuNZ1g.cjs → decision-D8bcIGXQ.cjs} +1 -1
  10. package/dist/{decision-object-DBcLWVvv.d.cts → decision-object-BB8XV5Ec.d.cts} +1 -1
  11. package/dist/{decision-object-B5zGSPFp.d.mts → decision-object-CH_sE694.d.mts} +1 -1
  12. package/dist/effects-B9IvUPVh.mjs +23 -0
  13. package/dist/effects-DrWcUGgc.cjs +34 -0
  14. package/dist/evals.cjs +810 -0
  15. package/dist/evals.d.cts +595 -0
  16. package/dist/evals.d.mts +595 -0
  17. package/dist/evals.mjs +801 -0
  18. package/dist/{event-log-DkVlk3Rh.mjs → event-log-CEq_bVXQ.mjs} +8 -4
  19. package/dist/{event-log-Dkc-yIYu.cjs → event-log-dSOnXHuG.cjs} +7 -15
  20. package/dist/{event-log-store-Brx9tOHt.mjs → event-log-store-DLAfNM34.mjs} +1 -1
  21. package/dist/{event-log-store-BBS8so8a.d.mts → event-log-store-DYun526m.d.mts} +15 -14
  22. package/dist/{event-log-store-Cl-YWDnq.cjs → event-log-store-Dey9ufA4.cjs} +1 -1
  23. package/dist/{event-log-store-bc4ICGcS.d.cts → event-log-store-aFXK6yZX.d.cts} +15 -14
  24. package/dist/index.cjs +136 -98
  25. package/dist/index.d.cts +73 -134
  26. package/dist/index.d.mts +73 -134
  27. package/dist/index.mjs +117 -75
  28. package/dist/internal-BzlpzSux.d.cts +14 -0
  29. package/dist/internal-BzlpzSux.d.mts +14 -0
  30. package/dist/log.cjs +2 -3
  31. package/dist/log.d.cts +2 -2
  32. package/dist/log.d.mts +2 -2
  33. package/dist/log.mjs +3 -3
  34. package/dist/machines.cjs +1 -719
  35. package/dist/machines.d.cts +2 -424
  36. package/dist/machines.d.mts +2 -424
  37. package/dist/machines.mjs +2 -709
  38. package/dist/{model-source-kjGwqAV8.mjs → model-source-BrsLDvHL.mjs} +1 -1
  39. package/dist/{model-source-B4WQ8Idp.cjs → model-source-YpxjmWuX.cjs} +1 -1
  40. package/dist/openai.cjs +3 -3
  41. package/dist/openai.d.cts +2 -3
  42. package/dist/openai.d.mts +2 -3
  43. package/dist/openai.mjs +3 -3
  44. package/dist/otel.d.cts +1 -1
  45. package/dist/otel.d.mts +1 -1
  46. package/dist/policy.cjs +77 -0
  47. package/dist/policy.d.cts +143 -0
  48. package/dist/policy.d.mts +143 -0
  49. package/dist/policy.mjs +75 -0
  50. package/dist/{effects-BT_cSY0j.mjs → run-agent-BE7DOWD6.mjs} +150 -184
  51. package/dist/{run-agent-DlPzBWj-.d.mts → run-agent-DHlq87G7.d.mts} +18 -34
  52. package/dist/{effects-DDseYtJk.cjs → run-agent-De3xVFGw.cjs} +149 -201
  53. package/dist/{run-agent-30mH2moE.d.cts → run-agent-Dq0oSFRm.d.cts} +18 -34
  54. package/dist/{setup-agent-BDmNr6PC.mjs → setup-agent-CLm6QVGy.mjs} +3 -3
  55. package/dist/{setup-agent-BaNyz6Q3.d.cts → setup-agent-Cv8Hk3mX.d.cts} +12 -11
  56. package/dist/{setup-agent-C1CG_nfX.d.mts → setup-agent-DgFAhaPJ.d.mts} +12 -11
  57. package/dist/{setup-agent-CmEToB8y.cjs → setup-agent-Dv5VpuHe.cjs} +3 -3
  58. package/dist/tanstack-ai.cjs +4 -4
  59. package/dist/tanstack-ai.d.cts +2 -3
  60. package/dist/tanstack-ai.d.mts +2 -3
  61. package/dist/tanstack-ai.mjs +4 -4
  62. package/dist/testing.cjs +122 -996
  63. package/dist/testing.d.cts +36 -695
  64. package/dist/testing.d.mts +36 -695
  65. package/dist/testing.mjs +112 -977
  66. package/dist/{text-logic-BqxalgMr.d.mts → text-logic-x17uLBwA.d.cts} +238 -2
  67. package/dist/{text-logic-CSf9bqpP.d.cts → text-logic-x17uLBwA.d.mts} +238 -2
  68. package/dist/trajectory-BFjXaooB.d.cts +96 -0
  69. package/dist/trajectory-BFjXaooB.d.mts +96 -0
  70. package/dist/trajectory-BUXj_1Ue.mjs +152 -0
  71. package/dist/trajectory-CoZSv4wF.cjs +157 -0
  72. package/dist/{usage-Bm5nvpow.d.cts → usage-B4B-TQSV.d.mts} +1 -1
  73. package/dist/{usage-BnikdOca.d.mts → usage-DvGcoK6x.d.cts} +1 -1
  74. package/dist/{utils-7lSa4H-i.d.cts → utils-B8k-YPYI.d.mts} +23 -17
  75. package/dist/{utils-CBbKw2fM.d.mts → utils-DAclF-kw.d.cts} +23 -17
  76. package/dist/validate.cjs +4 -4
  77. package/dist/validate.d.cts +2 -2
  78. package/dist/validate.d.mts +2 -2
  79. package/dist/validate.mjs +4 -4
  80. package/dist/{verify-DJHfEfm6.d.cts → verify-BM0sGmsf.d.cts} +42 -6
  81. package/dist/{verify-ChyX6jq0.mjs → verify-CWlg3j7B.mjs} +153 -45
  82. package/dist/{verify-CwlombbC.cjs → verify-Dek6gHCS.cjs} +153 -45
  83. package/dist/{verify-i9Hiwc5g.d.mts → verify-H3Pr2gNw.d.mts} +42 -6
  84. package/dist/{with-model-calls-B_WLmOSl.cjs → with-model-calls-D50Ca6qW.cjs} +2 -2
  85. package/dist/{with-model-calls-BkOwdroS.mjs → with-model-calls-uY-KKliA.mjs} +2 -2
  86. package/docs/advanced.md +5 -5
  87. package/docs/choosing-a-run-mode.md +7 -7
  88. package/docs/debugging.md +6 -8
  89. package/docs/decisions.md +6 -5
  90. package/docs/environment-and-policy.md +1 -1
  91. package/docs/evals.md +57 -43
  92. package/docs/event-log.md +14 -16
  93. package/docs/from-a-loop.md +3 -3
  94. package/docs/generate-machines.md +4 -4
  95. package/docs/hosts.md +22 -29
  96. package/docs/human-in-the-loop.md +5 -8
  97. package/docs/index.md +4 -4
  98. package/docs/judgments.md +5 -7
  99. package/docs/langgraph-comparison.md +6 -6
  100. package/docs/machines-as-data.md +12 -17
  101. package/docs/machines-presets.md +17 -120
  102. package/docs/machines.md +13 -13
  103. package/docs/models-and-providers.md +19 -25
  104. package/docs/multi-agent.md +1 -1
  105. package/docs/observability.md +6 -8
  106. package/docs/persistence.md +15 -25
  107. package/docs/quickstart.md +4 -4
  108. package/docs/statechart-policy-examples.md +1 -1
  109. package/docs/steps.md +5 -6
  110. package/docs/strangler-fig.md +72 -7
  111. package/docs/text-requests.md +10 -12
  112. package/docs/thinking-in-state-machines.md +8 -8
  113. package/docs/tools.md +10 -12
  114. package/docs/upgrading.md +1 -1
  115. package/docs/usage-and-budgets.md +51 -56
  116. package/docs/validate-and-repair.md +9 -14
  117. package/docs/verify.md +53 -7
  118. package/docs/xstate-4-and-5-apps.md +19 -28
  119. package/llms.txt +1 -1
  120. package/package.json +31 -6
  121. package/readme.md +32 -24
  122. package/dist/internal-C3Lqu538.d.cts +0 -43
  123. package/dist/internal-DUcdhk-i.d.mts +0 -43
  124. package/dist/types-DQUOOaZO.d.cts +0 -238
  125. package/dist/types-DQUOOaZO.d.mts +0 -238
  126. package/skills/add-evals/SKILL.md +0 -152
  127. package/skills/add-traces/SKILL.md +0 -152
  128. package/skills/build-machine/SKILL.md +0 -84
  129. package/skills/generate-machine/SKILL.md +0 -335
  130. package/skills/graph-engineering/SKILL.md +0 -43
  131. package/skills/improve-machine/SKILL.md +0 -261
  132. package/skills/improve-machine/reference.md +0 -101
  133. package/skills/loop-to-machine/SKILL.md +0 -205
  134. package/skills/loop-to-machine/policy.md +0 -154
  135. package/skills/loop-to-machine/reference.md +0 -260
@@ -1,335 +0,0 @@
1
- ---
2
- name: generate-machine
3
- description: Author a @statelyai/agent workflow as JSON and check it before it runs. Use when asked to generate or author an agent machine, create a workflow config, write an AgentWorkflowConfig, or produce an LLM-authored state machine for an agent — and when a generated config fails verifyAgentConfig, Ajv validation, or agent lint and needs repair.
4
- ---
5
-
6
- # Generate an agent machine
7
-
8
- An agent machine is data. You author a JSON `AgentWorkflowConfig`, then run it through `verifyAgentConfig`, whose gates all work with no API key: validate → vocabulary → lower → lint → explore. Do not hand back a config that has not passed every gate.
9
-
10
- ```
11
- split environment/policy → author (description = goal) → verifyAgentConfig → repair (≤ 3) → persist traces → hand back
12
- ```
13
-
14
- To improve an existing policy from its traces, use the `improve-machine` skill instead.
15
-
16
- ## 0. Split environment and policy
17
-
18
- Decide what is fixed before you write any JSON.
19
-
20
- - **Environment** (TypeScript, fixed): the rules of the world and the business invariants, as states and guards. People author it and verify it with `canReach` / `explorePaths` from `@statelyai/agent/testing`. You never generate it, and no generated config may change it.
21
- - **Policy** (JSON, generated): how the agent works toward the goal: which step next, which tool, when to ask a person, cheap judgment vs model call. This is the `AgentWorkflowConfig` you author.
22
- - The policy plugs into the environment at its decision points and outputs one event; the environment's guards decide whether to accept it.
23
- - No split needed for a single judgment inside a fixed workflow, or for a workflow where every step is dictated by rules. Author one machine.
24
-
25
- If the request asks you to generate a business rule (an approval threshold, a payout limit), stop: that belongs in the environment, in TypeScript, reviewed by a person. Say so.
26
-
27
- Before editing an existing machine, read it as text:
28
-
29
- ```ts
30
- import { describeMachine } from "@statelyai/agent";
31
- const text = describeMachine(machine); // "Machine <id>", then "Goal: …", states, transitions, events
32
- ```
33
-
34
- Pass `{ snapshot }` to mark the active states with `*`. Concept page: [Environment and policy](../../docs/environment-and-policy.md).
35
-
36
- ## 1. Read the schema
37
-
38
- The config shape is fixed by a shipped JSON Schema. Read it first; it is the contract.
39
-
40
- - Consuming the package: `node_modules/@statelyai/agent/schemas/agent-workflow.json` (importable as `@statelyai/agent/agent-workflow.json`)
41
- - In the `statelyai/agent` repo: `schemas/agent-workflow.json`
42
-
43
- ## 2. Author the config
44
-
45
- Start with the root `description`: it is the machine's **goal**, the overall objective plus measurable success criteria. `describeMachine` renders it first as `Goal: …`, and an improver later climbs toward it. Write criteria a scorer can compute from outcomes (terminal states, effects in the world, cost), not "be helpful":
46
-
47
- ```json
48
- "description": "Resolve each support ticket in one session. Success: escalations only for tickets that need a human; every reply approved before sending; at most 2 model calls per ticket."
49
- ```
50
-
51
- Rules the runtime enforces. Each one maps to a build error or a lint diagnostic:
52
-
53
- - Every value is a JSON literal or a whole-string `"{{ }}"` expression reading a dot path on `input`, `context`, or `event`. No JavaScript.
54
- - Guard expressions are truthy-only: no comparisons, operators, or negation.
55
- - A model call is either a named `requests` entry invoked with `src: "<requestName>"`, or an inline invoke with `src: "agent.decide"` plus an `allowedEvents` list. A `requests` entry's templates read only its own `input` (`"{{ input.ticket }}"`); the invoke's `input` maps context into it.
56
- - A state invoking `agent.decide` MUST handle every allowed event in its `on`.
57
- - An `agent.decide` invoke has no `onDone` (a decision produces no output). Use `onError` for the retries-exhausted path.
58
- - A request invoke resolves with `{ result, messages }`, so `onDone.assign` reads its result from `"{{ event.output.result.<field> }}"` (`"{{ event.output.result }}"` for a plain-text request). A non-request `actors` invoke reads `"{{ event.output.<field> }}"` directly.
59
- - Every path must reach a `"type": "final"` state, and each final state needs an `output` when `schemas.output` is declared.
60
- - Do not invent guard or action names. Only `"{{ }}"` guards, `assign`, and `emit` exist unless the host tells you which named guards/actions it implements.
61
- - Model refs are strings (`"openai/gpt-6-luna"`); the host resolves them.
62
- - To make a decision a closed choice, add `payloads` to the `agent.decide` input: one payload list per event type, without `type` (`"payloads": { "REFUND": [{ "amount": 20 }, { "amount": 50 }], "ESCALATE": [] }`). The model then picks one listed event instead of writing a payload. A type with no key is offered as `{ type }`; `[]` withholds it.
63
- - An idle human step can carry `meta.interaction` for the host to render with `getInteraction`: a `label`, `events` (one button per event type), `choices` (one button per full event, `{ "label": "Refund $20", "event": { "type": "REFUND", "amount": 20 } }`, or a context path string holding such a list), and `textEvent` for free text. A bare `"textEvent": "REJECT"` puts the text in a `text` field; `{ "type": "REJECT", "field": "reason" }` names the field. `meta` is not template-evaluated.
64
-
65
- Reference config — decision, text request, idle human step, one final state:
66
-
67
- ```json
68
- {
69
- "id": "support-ticket",
70
- "description": "Resolve each support ticket in one session. Success: escalations only for tickets that need a human; every reply approved before sending; at most 2 model calls per ticket.",
71
- "schemas": {
72
- "input": {
73
- "type": "object",
74
- "properties": { "ticket": { "type": "string" } },
75
- "required": ["ticket"]
76
- },
77
- "context": {
78
- "type": "object",
79
- "properties": {
80
- "ticket": { "type": "string" },
81
- "reply": { "type": "string" },
82
- "resolution": { "type": "string" }
83
- },
84
- "required": ["ticket"]
85
- },
86
- "events": {
87
- "ESCALATE": {
88
- "type": "object",
89
- "properties": { "reason": { "type": "string" } },
90
- "required": ["reason"]
91
- },
92
- "REPLY": { "type": "object", "properties": {} },
93
- "APPROVE": { "type": "object", "properties": {} },
94
- "REJECT": { "type": "object", "properties": {} }
95
- },
96
- "output": {
97
- "type": "object",
98
- "properties": { "resolution": { "type": "string" }, "reply": { "type": "string" } },
99
- "required": ["resolution"]
100
- }
101
- },
102
- "context": { "ticket": "{{ input.ticket }}" },
103
- "requests": {
104
- "draftReply": {
105
- "model": "openai/gpt-6-luna",
106
- "system": "Draft a short, courteous support reply.",
107
- "prompt": "{{ input.ticket }}",
108
- "input": {
109
- "type": "object",
110
- "properties": { "ticket": { "type": "string" } },
111
- "required": ["ticket"]
112
- },
113
- "output": {
114
- "type": "object",
115
- "properties": { "reply": { "type": "string" } },
116
- "required": ["reply"]
117
- }
118
- }
119
- },
120
- "initial": "triaging",
121
- "states": {
122
- "triaging": {
123
- "invoke": {
124
- "id": "triageDecision",
125
- "src": "agent.decide",
126
- "input": {
127
- "model": "openai/gpt-6-luna",
128
- "system": "Decide whether this ticket needs escalation or a drafted reply.",
129
- "prompt": "{{ context.ticket }}",
130
- "allowedEvents": ["ESCALATE", "REPLY"],
131
- "maxRetries": 2
132
- },
133
- "onError": { "target": "resolved", "assign": { "resolution": "escalated" } }
134
- },
135
- "on": {
136
- "ESCALATE": { "target": "resolved", "assign": { "resolution": "escalated" } },
137
- "REPLY": { "target": "drafting" }
138
- }
139
- },
140
- "drafting": {
141
- "invoke": {
142
- "id": "draft",
143
- "src": "draftReply",
144
- "input": { "ticket": "{{ context.ticket }}" },
145
- "onDone": {
146
- "target": "awaitingApproval",
147
- "assign": { "reply": "{{ event.output.result.reply }}" }
148
- }
149
- }
150
- },
151
- "awaitingApproval": {
152
- "description": "Idle: nothing to do until a human approves or rejects the draft.",
153
- "on": {
154
- "APPROVE": { "target": "resolved", "assign": { "resolution": "replied" } },
155
- "REJECT": { "target": "resolved", "assign": { "resolution": "escalated" } }
156
- }
157
- },
158
- "resolved": {
159
- "type": "final",
160
- "output": { "resolution": "{{ context.resolution }}", "reply": "{{ context.reply }}" }
161
- }
162
- }
163
- }
164
- ```
165
-
166
- ## 3. Verify with `verifyAgentConfig`
167
-
168
- `verifyAgentConfig(config, options)` from `@statelyai/agent/validate` runs every pre-run gate and stops at the first failure. It never throws on a bad config; each gate is a record `{ name, ok, detail? }`. It needs `ajv` installed (an optional peer of the package).
169
-
170
- | Gate | Checks with | Fails on |
171
- | ------------ | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
172
- | `validate` | `validateAgentConfig` (shipped schema) | malformed config shape |
173
- | `vocabulary` | `allow` lists | an invoke src, named guard, named action, `agent.decide` event, or model ref not listed |
174
- | `lower` | `setupAgent.fromConfig` | unresolved named guard/action, `onDone` on an `agent.decide` invoke |
175
- | `lint` | `lintAgentMachine` | error findings (warnings too with `lint: { warnings: true }`) |
176
- | `explore` | `explorePaths`, once per input | a path ending outside `explore.terminals` (default `['done']`), a loop past `maxDepth`, a `maxPaths`/`maxNodes` cap, untried payloads (`unexpandedEvents`) |
177
-
178
- `compileSchema` turns the JSON Schemas _inside_ the config into Standard Schema validators for `fromConfig`. The library bundles no JSON Schema engine:
179
-
180
- ```ts
181
- import Ajv from "ajv";
182
- import type { SchemaCompiler, StandardSchemaV1 } from "@statelyai/agent";
183
-
184
- const ajv = new Ajv({ strict: false });
185
-
186
- export const compileSchema: SchemaCompiler = (jsonSchema, name): StandardSchemaV1 => {
187
- const validate = ajv.compile(jsonSchema);
188
- return {
189
- "~standard": {
190
- version: 1,
191
- vendor: "ajv",
192
- validate: (value) =>
193
- validate(value)
194
- ? { value }
195
- : {
196
- issues: (validate.errors ?? []).map((e) => ({
197
- message: `${name}${e.instancePath} ${e.message}`,
198
- })),
199
- },
200
- // Expose the source JSON Schema so lint and describeMachine can read the shape.
201
- jsonSchema: { input: () => jsonSchema },
202
- },
203
- };
204
- };
205
- ```
206
-
207
- Run the gates. List the host's real names in `allow`; an omitted key is unrestricted:
208
-
209
- ```ts
210
- import { verifyAgentConfig } from "@statelyai/agent/validate";
211
-
212
- const result = await verifyAgentConfig(candidate, {
213
- compileSchema,
214
- allow: {
215
- actors: [],
216
- requests: ["draftReply"],
217
- guards: [],
218
- actions: [],
219
- events: ["ESCALATE", "REPLY", "APPROVE", "REJECT"],
220
- models: ["openai/gpt-6-luna"],
221
- },
222
- explore: {
223
- inputs: [{ ticket: "Export downloads a 0-byte CSV on Safari." }],
224
- outputs: { draftReply: [{ reply: "" }] },
225
- // ESCALATE's `reason` is a free-form string: give exploration one to try.
226
- payloads: () => ({ ESCALATE: [{ reason: "needs a human" }] }),
227
- },
228
- });
229
- // result.ok, result.gates, result.config (after validate), result.machine (after lower)
230
- ```
231
-
232
- - Exploration forks past idle waits on every event the waiting state accepts, so a human step still ends `done`. Add `terminals: ["done", "idle"]` only when a path may legitimately stop at a wait with nothing to accept.
233
- - `guards` / `actions`: named implementations, passed to `fromConfig`. `actors`: bound with `machine.provide({ actors })` after lowering.
234
- - `explore` keys other than `inputs` and `terminals` pass through to `explorePaths`: `outputs` (output lists per request or actor src: one element is a canned output, two or more fork one branch each), `errors` (fork a failing branch per src), `payloads` (fork one branch per payload at decisions and idle waits), `dedupeBy` (explore each snapshot key once; the key must cover every field a guard reads), `maxDepth` (default 8), `maxPaths` (default 200), `maxNodes` (default 100,000).
235
- - Stub each request's output from its declared `output` schema (`""` for string, `0` for number, `[]` for array, recurse on `properties`). Add an `errors` entry for every src whose `onError` path matters.
236
- - Pick `explore.inputs` that reach every branch the goal cares about. Exploration covers only the inputs you give it.
237
- - A passing `lint` gate carries its warnings in `detail`. Report them.
238
-
239
- ## 4. Repair loop
240
-
241
- A failed gate's `detail` names the offending path, state, or name, one problem per line. Feed it back verbatim with the rejected config, and run the gates again:
242
-
243
- ```ts
244
- let prompt = originalTask;
245
- for (let attempt = 0; attempt < 3; attempt++) {
246
- const candidate = await author({ system, prompt });
247
- const result = await verifyAgentConfig(candidate, verifyOptions);
248
- if (result.ok) return result;
249
- const failed = result.gates.find((gate) => !gate.ok)!;
250
- prompt = `${originalTask}\n\nYour previous config:\n${JSON.stringify(candidate)}\n\nGate '${failed.name}' failed:\n${failed.detail}\nReturn a corrected config.`;
251
- }
252
- ```
253
-
254
- Keep the cap at 3. A config that fails three repairs is usually asking for something the data form cannot express: author it in TypeScript with `setupAgent` instead, or report that the vocabulary is missing something.
255
-
256
- ## 5. Manual fallback
257
-
258
- Without `verifyAgentConfig` (older package, or a gate you need to run alone), run the same gates by hand:
259
-
260
- ```ts
261
- import { setupAgent, type AgentWorkflowConfig } from "@statelyai/agent";
262
- import { validateAgentConfig } from "@statelyai/agent/validate";
263
- import { explorePaths, lintAgentMachine, simulateAgent } from "@statelyai/agent/testing";
264
-
265
- const validation = validateAgentConfig(candidate); // { valid, errors: [{ path, message }] }, path is a JSON Pointer
266
- if (!validation.valid) {
267
- // Stop here: feed the errors back as the repair prompt. Never lower an invalid config.
268
- throw new Error(validation.errors.map((e) => `${e.path} ${e.message}`).join("\n"));
269
- }
270
- const { machine } = setupAgent.fromConfig(candidate as AgentWorkflowConfig, { compileSchema }); // throws on unresolved names
271
- lintAgentMachine(machine, { throw: true }); // throws AgentLintError on error findings
272
- const dryRun = await simulateAgent(machine, {
273
- input: { ticket: "" },
274
- script: {
275
- decisions: { "agent.decide": [{ type: "REPLY" }] },
276
- text: { draftReply: [{ reply: "" }] },
277
- },
278
- }); // dryRun.status: 'done' | 'idle' | 'exhausted' ('exhausted' = loops, reject)
279
- const report = await explorePaths(machine, {
280
- input: { ticket: "" },
281
- outputs: { draftReply: [{ reply: "" }] },
282
- payloads: () => ({ ESCALATE: [{ reason: "needs a human" }] }),
283
- }); // report.unexpandedEvents lists types that still need a payloads entry
284
- ```
285
-
286
- The lint check that bites most often is `decide-without-events`: an `allowedEvents` list with no matching `on` transitions. Do not disable checks.
287
-
288
- ## 6. Persist traces from day one
289
-
290
- An improver needs data. Wire an event log store and a thread id per run before the first real run, so every run is a trace:
291
-
292
- ```ts
293
- import { runAgent } from "@statelyai/agent";
294
- import { createInMemoryEventLogStore } from "@statelyai/agent/log";
295
-
296
- const store = createInMemoryEventLogStore(); // tests only; implement AgentEventLogStore on your database
297
- const result = await runAgent(machine, {
298
- input,
299
- executors,
300
- store,
301
- threadId: `tickets:${ticketId}`,
302
- });
303
- ```
304
-
305
- - `threadId` is required with `store`. One thread per run.
306
- - Give thread ids a prefix an improver can list by, such as `tickets:<ticketId>`. `store.listThreads({ prefix, limit, cursor })` pages through them. It is optional on a custom `AgentEventLogStore`, so with a store that lacks it, record each thread id with its scenario or ticket.
307
- - Failed decision attempts are journaled on the entry they explain; `getDecisionAttempts(entries)` from `@statelyai/agent/log` reads them back.
308
- - `replay(machine, await store.read(threadId))` from `@statelyai/agent/log` rebuilds the final state with no model calls. Pass the machine that recorded the thread: with an environment and a policy, the environment; `getUsageFromEvents(entries)` sums token usage.
309
-
310
- ## 7. Hand back
311
-
312
- Deliver the config JSON plus how to run it:
313
-
314
- ```ts
315
- import { runAgent } from "@statelyai/agent";
316
- import { createAiSdkExecutors } from "@statelyai/agent/ai-sdk";
317
-
318
- const run = await runAgent(result.machine!, {
319
- input: { ticket: "Export downloads a 0-byte CSV on Safari." },
320
- executors: createAiSdkExecutors({ resolveModel }),
321
- store,
322
- threadId,
323
- });
324
- ```
325
-
326
- Say which gates passed (list `result.gates`), list any lint warnings, state the goal and its success criteria, and say where traces are written. Model refs are strings, so the host supplies `resolveModel`.
327
-
328
- ## Known limits
329
-
330
- - **Named guards and actions are host-resolved.** A config carries no functions. `guard: "isReady"` only works if the host passes an `isReady` implementation to `fromConfig`; an unresolved name is a build-time throw. Either list the host's names in the prompt or forbid named guards/actions entirely.
331
- - **Expressions are dot-path templates only.** `"{{ context.a.b }}"` — no comparisons, no arithmetic, no method calls, no partial interpolation inside a larger string.
332
- - **Ajv validity is not semantic validity.** A config can validate and still invoke a request that does not exist or route to a state that solves nothing. That is what the `vocabulary`, `lint`, and `explore` gates are for.
333
- - **Exploration covers the inputs you give it.** A branch no `explore.inputs` entry reaches is unverified.
334
- - **Model refs are checked only with `allow.models`.** Tool names stay unchecked strings that resolve at run time, in the host.
335
- - **The gates check structure, not quality.** Whether a config is better than another is measured by replaying scenarios against an objective. See the `improve-machine` skill.
@@ -1,43 +0,0 @@
1
- ---
2
- name: graph-engineering
3
- description: Route an agent-reliability request to the right @statelyai/agent skill and keep the four steps in order — make it a state machine, add traces, add evals, self-improve. Use when asked to "make my agent reliable", "graph engineering", "turn my agent into a state machine and improve it", "where do I start with this agent", or any request that spans more than one of those steps.
4
- ---
5
-
6
- # Graph engineering
7
-
8
- Four steps, in order. Each one needs the one before it.
9
-
10
- | Step | Skill | Done when |
11
- | --- | --- | --- |
12
- | 1. Make it a state machine | `loop-to-machine` (existing code), `build-machine` (from scratch, TypeScript), or `generate-machine` (from scratch, when the spec fits JSON) | Rules are guards and the sentinel is unreachable (`canReach`). From existing code: the parity report has no unreviewed regressions. From scratch: every invariant is proved complete or listed as not proved, or every `verifyAgentConfig` gate passes |
13
- | 2. Add traces | `add-traces` | Every run leaves a stamped log thread that replays to its final state |
14
- | 3. Add evals | `add-evals` | Evals per state at the cheapest level, an objective, baseline numbers per split |
15
- | 4. Self-improve (experimental) | `improve-machine` | A pull request with one policy change and its held-out scorecard, or a report that none helped |
16
-
17
- ## Find where the project is
18
-
19
- Check in this order and start at the first step that is not done:
20
-
21
- 1. Does the code path production actually runs go through a machine (`setupAgent`, `createMachine`, an `AgentWorkflowConfig`)? Follow the route or entry point to the model call; a machine that nothing calls does not count. If the path is a loop or ad-hoc code, step 1.
22
- 2. Do runs persist to an event log store with a declared machine `version` and a `metadata` stamp? If not, step 2.
23
- 3. Is there an objective over ground truth, and evals that name the states they cover? If not, step 3.
24
- 4. Otherwise, step 4, which is experimental: say so when you route there. A machine with no decision point (a tool loop, a scheduler: the model picks tools inside one text request) still goes to step 4: `improve-machine`'s "No decision points" branch, which turns traces into approval-rule, cap, and request-wording proposals.
25
-
26
- From scratch, pick `build-machine` when a rule needs a threshold, a counter, spawned fan-out, a timer, or an any-time event: the JSON config's guards are truthy-only. Pick `generate-machine` when every rule is a truthy check.
27
-
28
- Tell the user which step you are starting at and why, in one line, then load that skill.
29
-
30
- ## Across all steps
31
-
32
- - **XState v6, not v5.** Most training data is v5; v6 differs:
33
- - A transition is a plain function: `({ context, event }) => ({ target, context })`. There is no `assign`; return the new context.
34
- - Guards are inline JS in that function. Named guards are optional, not required.
35
- - Targets are objects (`{ target: "b" }`), never bare strings.
36
- - Actors: `createAsyncLogic`, not `fromPromise`. Done/error events are `{ type: "xstate.done.actor", actorId, output }` and `{ type: "xstate.error.actor", actorId, error }`.
37
- - **Keep targets in the transition function.** `describeMachine` parses each function and lists every target it can return. A target from a helper (`=> resolveRound(context)`) or a variable reads as `→ ?`: have the helper return data and pick the target inline. Pass `{ source: true }` to read each function under its line, and `helpers: { name: fn }` for the functions they call by name.
38
- - **To see what one event does, call `transition`.** It is pure and runs nothing; see [loop-to-machine](../loop-to-machine/SKILL.md#inspect-one-event).
39
- - **The environment is people's code.** Rules, guards, invariants, and the sentinel change only by human review. No step rewrites them on its own.
40
- - **No decision point is not a dead end.** Steps 2–4 still apply: traces stamp the request config or approval-rules version, evals are rule checks and per-state stats, and improvement proposes changes for a person to review.
41
- - **Ground truth scores everything.** Outcomes and effects in the world, labeled by people. A model never grades its own runs.
42
- - **Ask for labels and rules; decide the mechanics.** Expected outcomes, thresholds, the objective, and the train/held-out split are the user's. File layout, test names, and helper code are yours.
43
- - **Hand back between steps.** Finish one step's hand-back before starting the next, unless the user asked for several in one go.