@statelyai/agent 2.0.0-alpha.34 → 2.0.0-alpha.36

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (130) hide show
  1. package/CHANGELOG.md +124 -0
  2. package/dist/act-B9hLQls6.mjs +19 -0
  3. package/dist/act-CrX0JwWY.cjs +42 -0
  4. package/dist/ai-sdk.cjs +63 -7
  5. package/dist/ai-sdk.d.cts +13 -3
  6. package/dist/ai-sdk.d.mts +13 -3
  7. package/dist/ai-sdk.mjs +63 -7
  8. package/dist/{decision-BEsATcr1.cjs → decision-CMeuNZ1g.cjs} +262 -4
  9. package/dist/{decision-COkJNjOW.mjs → decision-m5hGmHpF.mjs} +239 -5
  10. package/dist/decision-object-B5zGSPFp.d.mts +15 -0
  11. package/dist/decision-object-DBcLWVvv.d.cts +15 -0
  12. package/dist/{run-agent-s5D32w99.cjs → effects-B23bFZEs.cjs} +948 -102
  13. package/dist/{run-agent-Ct_GZw_f.mjs → effects-CuOUNYtZ.mjs} +914 -104
  14. package/dist/{event-log-C7jeH0y8.mjs → event-log-DkVlk3Rh.mjs} +30 -15
  15. package/dist/{event-log-BWSSLUff.cjs → event-log-Dkc-yIYu.cjs} +41 -14
  16. package/dist/{event-log-store-BzXehP7R.d.cts → event-log-store-BBS8so8a.d.mts} +2 -2
  17. package/dist/{event-log-store-W49TCJXi.mjs → event-log-store-Brx9tOHt.mjs} +1 -1
  18. package/dist/{event-log-store-CleWpsp5.cjs → event-log-store-Cl-YWDnq.cjs} +1 -1
  19. package/dist/{event-log-store-CWjN6lmM.d.mts → event-log-store-bc4ICGcS.d.cts} +2 -2
  20. package/dist/index.cjs +660 -286
  21. package/dist/index.d.cts +168 -9
  22. package/dist/index.d.mts +168 -9
  23. package/dist/index.mjs +639 -271
  24. package/dist/internal-C3Lqu538.d.cts +43 -0
  25. package/dist/internal-DUcdhk-i.d.mts +43 -0
  26. package/dist/log.cjs +2 -2
  27. package/dist/log.d.cts +1 -1
  28. package/dist/log.d.mts +1 -1
  29. package/dist/log.mjs +2 -2
  30. package/dist/machines.cjs +457 -41
  31. package/dist/machines.d.cts +114 -43
  32. package/dist/machines.d.mts +114 -43
  33. package/dist/machines.mjs +457 -42
  34. package/dist/{model-source-DmhTMK39.cjs → model-source-B4WQ8Idp.cjs} +1 -1
  35. package/dist/{model-source-CK3F4YuU.mjs → model-source-kjGwqAV8.mjs} +1 -1
  36. package/dist/openai.cjs +149 -36
  37. package/dist/openai.d.cts +28 -10
  38. package/dist/openai.d.mts +28 -10
  39. package/dist/openai.mjs +149 -36
  40. package/dist/otel.d.cts +1 -1
  41. package/dist/otel.d.mts +1 -1
  42. package/dist/{run-agent-BGdx5913.d.mts → run-agent-BcQ-9NRT.d.cts} +91 -9
  43. package/dist/{run-agent-B1-Eqviq.d.cts → run-agent-C86drZtN.d.mts} +91 -9
  44. package/dist/{setup-agent-05uw4dPZ.mjs → setup-agent-BDmNr6PC.mjs} +9 -3
  45. package/dist/{setup-agent-BtfeFfF6.d.cts → setup-agent-BIKcbVXV.d.mts} +4 -4
  46. package/dist/{setup-agent-CXEBURdo.cjs → setup-agent-CmEToB8y.cjs} +9 -3
  47. package/dist/{setup-agent-C6nqzv_X.d.mts → setup-agent-jlo4vtaZ.d.cts} +4 -4
  48. package/dist/tanstack-ai.cjs +62 -4
  49. package/dist/tanstack-ai.d.cts +18 -6
  50. package/dist/tanstack-ai.d.mts +18 -6
  51. package/dist/tanstack-ai.mjs +62 -4
  52. package/dist/testing.cjs +536 -67
  53. package/dist/testing.d.cts +197 -180
  54. package/dist/testing.d.mts +197 -180
  55. package/dist/testing.mjs +532 -67
  56. package/dist/{text-logic-CV_4KGr6.d.cts → text-logic-BqxalgMr.d.mts} +87 -4
  57. package/dist/{text-logic-Ckh-rJFV.d.mts → text-logic-CSf9bqpP.d.cts} +87 -4
  58. package/dist/{types-Dp5_Ru4R.d.cts → types-DQUOOaZO.d.cts} +1 -1
  59. package/dist/{types-Dp5_Ru4R.d.mts → types-DQUOOaZO.d.mts} +1 -1
  60. package/dist/{usage-BDiJKyv9.d.cts → usage-DoP0Oi84.d.mts} +1 -1
  61. package/dist/{usage-DOoVUGVq.d.mts → usage-HtnrUsl9.d.cts} +1 -1
  62. package/dist/utils-7lSa4H-i.d.cts +403 -0
  63. package/dist/utils-CBbKw2fM.d.mts +403 -0
  64. package/dist/validate.cjs +44 -12
  65. package/dist/validate.d.cts +6 -6
  66. package/dist/validate.d.mts +6 -6
  67. package/dist/validate.mjs +44 -12
  68. package/dist/verify-Bink-Drw.cjs +2304 -0
  69. package/dist/{verify-BQpE8hUz.d.cts → verify-Buh8XSUr.d.mts} +208 -14
  70. package/dist/{verify-CqeuEmIk.d.mts → verify-CciG8UP-.d.cts} +208 -14
  71. package/dist/verify-Dpz1u5Pe.mjs +2251 -0
  72. package/dist/with-model-calls-B_WLmOSl.cjs +248 -0
  73. package/dist/with-model-calls-BkOwdroS.mjs +213 -0
  74. package/docs/act.md +102 -0
  75. package/docs/advanced.md +105 -0
  76. package/docs/any-stack.md +62 -0
  77. package/docs/choosing-a-run-mode.md +55 -0
  78. package/docs/debugging.md +29 -0
  79. package/docs/decisions.md +473 -0
  80. package/docs/environment-and-policy.md +188 -0
  81. package/docs/evals.md +509 -0
  82. package/docs/event-log.md +311 -0
  83. package/docs/from-a-loop.md +528 -0
  84. package/docs/generate-machines.md +110 -0
  85. package/docs/hosts.md +414 -0
  86. package/docs/human-in-the-loop.md +177 -0
  87. package/docs/index.md +48 -0
  88. package/docs/judgments.md +281 -0
  89. package/docs/langgraph-comparison.md +198 -0
  90. package/docs/machines-as-data.md +454 -0
  91. package/docs/machines-presets.md +198 -0
  92. package/docs/machines.md +497 -0
  93. package/docs/messages.md +67 -0
  94. package/docs/meta.json +52 -0
  95. package/docs/models-and-providers.md +268 -0
  96. package/docs/multi-agent.md +68 -0
  97. package/docs/observability.md +109 -0
  98. package/docs/patterns.md +31 -0
  99. package/docs/persistence.md +136 -0
  100. package/docs/quickstart.md +121 -0
  101. package/docs/roadmap.md +22 -0
  102. package/docs/scope.md +75 -0
  103. package/docs/statechart-policy-examples.md +69 -0
  104. package/docs/steps.md +54 -0
  105. package/docs/strangler-fig.md +323 -0
  106. package/docs/text-requests.md +250 -0
  107. package/docs/thinking-in-state-machines.md +324 -0
  108. package/docs/tools.md +249 -0
  109. package/docs/upgrading.md +37 -0
  110. package/docs/usage-and-budgets.md +516 -0
  111. package/docs/validate-and-repair.md +253 -0
  112. package/docs/verify.md +487 -0
  113. package/docs/xstate-4-and-5-apps.md +161 -0
  114. package/llms.txt +53 -0
  115. package/package.json +10 -3
  116. package/readme.md +30 -15
  117. package/skills/add-evals/SKILL.md +3 -1
  118. package/skills/add-traces/SKILL.md +6 -1
  119. package/skills/build-machine/SKILL.md +81 -0
  120. package/skills/generate-machine/SKILL.md +1 -1
  121. package/skills/graph-engineering/SKILL.md +13 -3
  122. package/skills/improve-machine/SKILL.md +21 -2
  123. package/skills/loop-to-machine/SKILL.md +147 -190
  124. package/skills/loop-to-machine/reference.md +260 -0
  125. package/dist/utils-BnXdWih3.d.mts +0 -108
  126. package/dist/utils-ebVpCYoF.d.cts +0 -108
  127. package/dist/verify-Bpahmr34.mjs +0 -1241
  128. package/dist/verify-DCLZa9R0.cjs +0 -1288
  129. package/dist/with-model-calls-BXYdBdcu.cjs +0 -45
  130. package/dist/with-model-calls-Bv4V4EHC.mjs +0 -34
@@ -0,0 +1,188 @@
1
+ ---
2
+ title: Environment and policy
3
+ description: Split an agent into a fixed environment that people author and verify, and a policy that a model can write and improve inside it.
4
+ ---
5
+
6
+ > **Alpha:** `@statelyai/agent` 2.0 is in alpha. APIs can change between releases; pin an exact version. Feedback: [github.com/statelyai/agent](https://github.com/statelyai/agent/issues).
7
+
8
+ This page explains how to split an agent into two machines: an environment that people own, and a policy that a model can write.
9
+
10
+ An agent does two kinds of work. It follows the rules of its domain, and it chooses how to make progress within those rules. The two change at different rates and need different owners, so they can live in different machines.
11
+
12
+ - The **environment** is the rules of the world and the business invariants. People author it in TypeScript, as states and guards, and verify it with [`canReach` and `explorePaths`](verify.md). It is fixed.
13
+ - The **policy** is how the agent works toward the goal: which step comes next, which tool to call, when to ask a person, and when a cheap judgment is enough instead of a model call. It is a machine a model can write: a TypeScript `setupAgent` machine, or a JSON `AgentWorkflowConfig` when it must change without a deploy. It is gated before it runs and promoted only on a measured improvement.
14
+
15
+ The environment is code you own and can verify. The policy is something a model can write inside it.
16
+
17
+ ## The two layers
18
+
19
+ <!-- viz: two nested boxes: environment (TypeScript, fixed: states, guards, invariant sentinel) asks for one event at each decision point -> policy (TypeScript or JSON machine, improvable: steps, tools, judgments) returns one event -> environment guards accept it or not; traces flow from the environment to an event log store -->
20
+
21
+ ```text
22
+ environment (TypeScript, fixed) policy (TS or JSON, improvable)
23
+ states, guards, invariants ── asks for one event ──▶ steps, tools, judgments
24
+ accepts the event or not ◀── returns one event ─── ends in a final state
25
+ ```
26
+
27
+ | Layer | Authored by | Format | Changes | Checked with |
28
+ | ----------- | ----------------------------- | -------------------------------------------------------------- | ------------------------------- | ---------------------------------------------------------------------------------------------- |
29
+ | Environment | People | TypeScript machine | By code review, rarely | `canReach`, `explorePaths`, unit tests |
30
+ | Policy | A model, reviewed by a person | TypeScript `setupAgent` machine, or JSON `AgentWorkflowConfig` | Between runs, on measured gains | TS: `tsc`, `lintAgentMachine`, replay, objective; JSON: `verifyAgentConfig`, replay, objective |
31
+
32
+ - The environment exposes decision points: states where it asks for one event. Its guards decide whether to accept whatever the policy proposes, so a policy cannot take an action the environment forbids.
33
+ - The policy answers at those decision points. It can be a single model call, a ladder of `choice` states over the facts, a judgment, or a request to ask the customer. Its final state outputs the event it proposes. The host runs the policy as the environment's `decide` executor: `createPolicyDecider(policyMachine, { executors })` starts one policy run per decision and returns its output event. See [A policy as the decider](decisions.md#a-policy-as-the-decider).
34
+ - An invariant sentinel, a state that is entered only when a rule is broken, stays unreachable for every policy. `canReach` proves it once, against the environment, for every event sequence and payload variant you list. See [Payloads and invoke outputs](verify.md#payloads-and-invoke-outputs).
35
+
36
+ The environment can be a plain XState machine. [`decideNext`](decisions.md#decisions-at-the-boundary-of-a-plain-machine) asks a model to pick one event that a plain XState machine accepts right now, and [closed choices](decisions.md#closed-choices) list the concrete events a decider may pick.
37
+
38
+ ### The event vocabulary
39
+
40
+ The keys of `machine.schemas.events` are the machine's event vocabulary: every event type it declares, in any state. `schemas` is optional in the machine type, so read it with `?.`.
41
+
42
+ ```ts
43
+ import { createMachine } from "xstate";
44
+ import { z } from "zod";
45
+
46
+ const refundDesk = createMachine({
47
+ schemas: {
48
+ events: {
49
+ "desk.refund": z.object({ amount: z.number() }),
50
+ "desk.escalate": z.object({}),
51
+ },
52
+ },
53
+ initial: "reviewing",
54
+ states: { reviewing: {} },
55
+ });
56
+
57
+ const vocabulary = Object.keys(refundDesk.schemas?.events ?? {});
58
+ // ["desk.refund", "desk.escalate"]
59
+ ```
60
+
61
+ - `getAcceptedEvents(snapshot)` lists what one snapshot accepts. The vocabulary is machine-wide and needs no snapshot.
62
+ - Gate a policy to it: pass it as `allow.events` to `verifyAgentConfig`, and a candidate whose `agent.decide` lists any other event fails the `vocabulary` gate.
63
+ - A `setupAgent` machine also declares the reserved `@agent.usage` event. Drop it with `.filter((type) => !type.startsWith("@agent."))`.
64
+
65
+ ## The goal and the objective
66
+
67
+ A policy is improved toward something. The machine's `description` states that goal.
68
+
69
+ - **Goal:** the overall objective in words, with measurable success criteria. For example: "Resolve each refund ticket correctly in one session. Success: the refund issued matches policy, no refund above $100 issues without approval, at most 4 proposals per ticket."
70
+ - **Objective:** a function that scores a batch of runs against those criteria. It reads outcomes from ground truth: the environment's terminal states in the traces, or the effects recorded in the world. A model never grades its own runs.
71
+ - **Held-out scenarios:** a set of scenarios the improver never sees while it proposes changes. A change is promoted only when it improves the objective on them, so the policy cannot overfit the training scenarios.
72
+
73
+ `describeMachine` puts the goal first in the text it renders for a model, so a model proposing a change reads the goal before the states. See [Machine descriptions for models](machines-as-data.md#machine-descriptions-for-models).
74
+
75
+ ## Traces
76
+
77
+ The improver reads past runs. Persist every run to an [event log store](event-log.md) from the first run.
78
+
79
+ ```ts
80
+ import { runAgent } from "@statelyai/agent";
81
+ import { createInMemoryEventLogStore } from "@statelyai/agent/log";
82
+
83
+ const store = createInMemoryEventLogStore();
84
+
85
+ const result = await runAgent(machine, {
86
+ input,
87
+ executors,
88
+ store,
89
+ threadId: "ticket-1001",
90
+ });
91
+ ```
92
+
93
+ - Each run writes to its own thread. Name threads by split and scenario, such as `train:ticket-1001`, and `store.listThreads({ prefix: "train:" })` pages through one split. `listThreads` is optional on a custom store; without it, record the thread ids per scenario. See [Stores](event-log.md#stores).
94
+ - `replay(machine, entries)` from `@statelyai/agent/log` rebuilds the state at the end of a thread, with no model calls. `getUsageFromEvents(entries)` sums its token usage.
95
+ - The log also keeps the decision attempts the machine did not accept, on the entry they explain. `getDecisionAttempts(entries)` from `@statelyai/agent/log` lists them with their reasons, so the events the guards refused are countable from the log alone. A host that decides with `decideNext` passes its `attempts` with the event. See [Decision attempts](event-log.md#decision-attempts).
96
+ - `createInMemoryEventLogStore` is for tests. In production, implement `AgentEventLogStore` against your database. See [Persistence](persistence.md).
97
+
98
+ An objective reads the threads and scores them. This one counts correct terminal states, invariant violations, and tokens:
99
+
100
+ ```ts
101
+ import { getUsageFromEvents, replay, type AgentEventLogStore } from "@statelyai/agent/log";
102
+ import type { AnyStateMachine } from "xstate";
103
+
104
+ interface Scenario {
105
+ threadId: string;
106
+ expected: "issued" | "escalated" | "closed";
107
+ }
108
+
109
+ async function scoreRuns(
110
+ environment: AnyStateMachine,
111
+ store: AgentEventLogStore,
112
+ scenarios: Scenario[],
113
+ ) {
114
+ let correct = 0;
115
+ let violations = 0;
116
+ let tokens = 0;
117
+ for (const scenario of scenarios) {
118
+ const entries = await store.read(scenario.threadId);
119
+ const { snapshot } = replay(environment, entries);
120
+ if (snapshot.matches("illegallyIssued")) violations++;
121
+ else if (snapshot.matches(scenario.expected)) correct++;
122
+ tokens += getUsageFromEvents(entries).totalTokens ?? 0;
123
+ }
124
+ return { accuracy: correct / scenarios.length, violations, tokens };
125
+ }
126
+ ```
127
+
128
+ The expected outcome per scenario is a label people write. The score never comes from a model's opinion of the run.
129
+
130
+ ## The improvement loop
131
+
132
+ > **Experimental:** self-improvement is experimental. The gates are stable, but the round itself may change, and on a loop with no decision point it mostly proposes wording, cap, and approval-rule changes. Code-level causes come back as change requests for a person.
133
+
134
+ ```text
135
+ traces → objective → diagnose → propose a policy diff → verifyAgentConfig → replay train → replay held-out → promote or discard
136
+ ```
137
+
138
+ <!-- viz: improvement loop: read traces from the event log store -> compute the objective -> diagnose failure clusters -> propose one policy diff -> verifyAgentConfig gates (validate -> vocabulary -> lower -> lint -> explore), failing gate detail feeds back as a repair prompt up to 3 times -> replay train then held-out scenarios -> promote when violations are 0 and held-out improves, else discard; the environment box is outside the loop -->
139
+
140
+ Each round changes the policy once, then measures it.
141
+
142
+ 1. **Diagnose.** Compute per-run outcomes and the objective. Group the failures by cause: the terminal state reached, guard rejections (`getDecisionAttempts`), loops, and cost hotspots. [`foldTraces`](evals.md#per-state-stats) counts visits, failures, and withheld options per state across runs.
143
+ 2. **Propose.** A model writes the smallest policy change that addresses one cause, with its hypothesis written down.
144
+ 3. **Gate.** [`verifyAgentConfig`](generate-machines.md) runs `validate`, `vocabulary`, `lower`, `lint`, and `explore` on the candidate. A failing gate's `detail` is the repair prompt. Cap repairs at 3.
145
+ 4. **Replay.** Run the training scenarios with the candidate, then the held-out scenarios.
146
+ 5. **Promote or discard.** Promote only when invariant violations are 0, the objective improves on the held-out scenarios, and no scenario regresses beyond a stated tolerance. Otherwise keep the current policy.
147
+
148
+ The `vocabulary` gate keeps a candidate inside the environment's contract: it lists the actors, requests, named guards, decision events, and model refs a policy may reference. A candidate that names anything else fails before it runs.
149
+
150
+ > **Warning:** No machine is edited while it runs. A promoted policy applies to new runs. Runs in progress finish on the version they started with.
151
+
152
+ ## What stays human
153
+
154
+ People own everything that defines "better" and everything the policy cannot touch.
155
+
156
+ - The environment: its states, guards, invariants, and the `canReach` proofs behind them.
157
+ - The events a person sends. Tag their transitions `meta: { actor: "human" }` so no decision offers them and `verifyAgentConfig` fails a policy that lists them. See [Events a person sends](decisions.md#events-a-person-sends).
158
+ - The goal, its success criteria, and the objective function. A model can draft an objective from the goal; a person confirms it before it is used.
159
+ - The vocabulary a policy may reference.
160
+ - The scenario set, the held-out split, and the regression tolerance.
161
+ - Environment changes. When the goal is unreachable without one, the improver writes a change request for a person instead of editing the environment.
162
+
163
+ Self-improvement is an experimental pattern with stable gates. The library supplies the gates; the loop, the objective, and the promotion rule stay with the host.
164
+
165
+ ### Machines with no decision point
166
+
167
+ A pure environment still improves through its approval rules and caps. [`proposeApprovalRules` and `proposeCaps`](evals.md#proposals-from-traces) read many runs' traces and propose an `allow` or `deny` rule per tool-and-arguments pattern and a value per cap.
168
+ They are proposals for a person to review; nothing promotes them.
169
+
170
+ ## When not to split
171
+
172
+ The split pays off when there is a policy worth learning. Skip it in these cases.
173
+
174
+ - **A single judgment.** One classification or yes/no check inside a fixed workflow does not need its own machine. Use a [judgment](judgments.md) or a [decision](decisions.md) in the environment.
175
+ - **A pure environment.** When every step is dictated by the rules, such as an approval flow or a compensation sequence, there is nothing for a policy to choose. Author one TypeScript machine and verify it.
176
+ - **No ground truth.** Without outcomes you can score, improvement has nothing to climb. Collect traces and labels first.
177
+
178
+ ## Related
179
+
180
+ - [Decisions](decisions.md): closed choices, decision models, and `decideNext` for plain machines.
181
+ - [Generating machines with an LLM](generate-machines.md): `verifyAgentConfig` and the repair loop.
182
+ - [Machines as data](machines-as-data.md): the config format and `describeMachine`.
183
+ - [Testing and verification](verify.md): `canReach`, `explorePaths`, and payload variants.
184
+ - [Event log](event-log.md) and [Persistence](persistence.md): stores, threads, and replay.
185
+ - The [`graph-engineering`](https://github.com/statelyai/agent/blob/next/skills/graph-engineering/SKILL.md) skill routes existing code to [`loop-to-machine`](https://github.com/statelyai/agent/blob/next/skills/loop-to-machine/SKILL.md), which converts the loop and checks parity with `compareRuns`. It routes new policies to [`generate-machine`](https://github.com/statelyai/agent/blob/next/skills/generate-machine/SKILL.md). Then [`add-traces`](https://github.com/statelyai/agent/blob/next/skills/add-traces/SKILL.md) persists runs, [`add-evals`](https://github.com/statelyai/agent/blob/next/skills/add-evals/SKILL.md) writes evaluations for each state, and [`improve-machine`](https://github.com/statelyai/agent/blob/next/skills/improve-machine/SKILL.md) proposes a pull request from traces, or reports that no candidate beat the current policy.
186
+ - [examples/self-improving-refunds](https://github.com/statelyai/agent/blob/next/examples/self-improving-refunds): a refund desk as the environment, JSON playbooks as the policy, an improvement loop, and a benchmark against a raw tool loop.
187
+ - [examples/code-review-loop](https://github.com/statelyai/agent/blob/next/examples/code-review-loop): review gates as the environment, a triage policy improved from traces against hidden tests, run with Claude Code in headless mode.
188
+ - [examples/tic-tac-toe](https://github.com/statelyai/agent/blob/next/examples/tic-tac-toe): a game as the environment, a policy over board features, and a `canReach` proof that it never loses as the bar for promotion.