@statelyai/agent 2.0.0-alpha.29 → 2.0.0-alpha.31

package/skills/generate-machine/SKILL.md

@@ -31,13 +31,13 @@ import { describeMachine } from "@statelyai/agent";
 const text = describeMachine(machine); // "Machine <id>", then "Goal: …", states, transitions, events
 ```
 
-Pass `{ snapshot }` to mark the active states with `*`. Concept page: `docs/environment-and-policy.md`.
+Pass `{ snapshot }` to mark the active states with `*`. Concept page: [Environment and policy](https://github.com/statelyai/agent/blob/next/docs/environment-and-policy.md).
 
 ## 1. Read the schema
 
 The config shape is fixed by a shipped JSON Schema. Read it first; it is the contract.
 
-- Consuming the package: `node_modules/@statelyai/agent/agent-workflow.json` (importable as `@statelyai/agent/agent-workflow.json`)
+- Consuming the package: `node_modules/@statelyai/agent/schemas/agent-workflow.json` (importable as `@statelyai/agent/agent-workflow.json`)
 - In the `statelyai/agent` repo: `schemas/agent-workflow.json`
 
 ## 2. Author the config
@@ -58,7 +58,7 @@ Rules the runtime enforces. Each one maps to a build error or a lint diagnostic:
 - A request invoke resolves with `{ result, messages }`, so `onDone.assign` reads its result from `"{{ event.output.result.<field> }}"` (`"{{ event.output.result }}"` for a plain-text request). A non-request `actors` invoke reads `"{{ event.output.<field> }}"` directly.
 - Every path must reach a `"type": "final"` state, and each final state needs an `output` when `schemas.output` is declared.
 - Do not invent guard or action names. Only `"{{ }}"` guards, `assign`, and `emit` exist unless the host tells you which named guards/actions it implements.
-- Model refs are strings (`"openai/gpt-5.4-mini"`); the host resolves them.
+- Model refs are strings (`"openai/gpt-6-luna"`); the host resolves them.
 - To make a decision a closed choice, add `payloads` to the `agent.decide` input: one payload list per event type, without `type` (`"payloads": { "REFUND": [{ "amount": 20 }, { "amount": 50 }], "ESCALATE": [] }`). The model then picks one listed event instead of writing a payload. A type with no key is offered as `{ type }`; `[]` withholds it.
 - An idle human step can carry `meta.interaction` for the host to render with `getInteraction`: a `label`, `events` (one button per event type), `choices` (one button per full event, `{ "label": "Refund $20", "event": { "type": "REFUND", "amount": 20 } }`, or a context path string holding such a list), and `textEvent` for free text. A bare `"textEvent": "REJECT"` puts the text in a `text` field; `{ "type": "REJECT", "field": "reason" }` names the field. `meta` is not template-evaluated.
 
@@ -102,7 +102,7 @@ Reference config — decision, text request, idle human step, one final state:
   "context": { "ticket": "{{ input.ticket }}" },
   "requests": {
     "draftReply": {
-      "model": "openai/gpt-5.4-mini",
+      "model": "openai/gpt-6-luna",
       "system": "Draft a short, courteous support reply.",
       "prompt": "{{ input.ticket }}",
       "input": {
@@ -124,7 +124,7 @@ Reference config — decision, text request, idle human step, one final state:
         "id": "triageDecision",
         "src": "agent.decide",
         "input": {
-          "model": "openai/gpt-5.4-mini",
+          "model": "openai/gpt-6-luna",
           "system": "Decide whether this ticket needs escalation or a drafted reply.",
           "prompt": "{{ context.ticket }}",
           "allowedEvents": ["ESCALATE", "REPLY"],
@@ -217,7 +217,7 @@ const result = await verifyAgentConfig(candidate, {
     guards: [],
     actions: [],
     events: ["ESCALATE", "REPLY", "APPROVE", "REJECT"],
-    models: ["openai/gpt-5.4-mini"],
+    models: ["openai/gpt-6-luna"],
   },
   explore: {
     inputs: [{ ticket: "Export downloads a 0-byte CSV on Safari." }],

package/skills/graph-engineering/SKILL.md

@@ -0,0 +1,33 @@
+---
+name: graph-engineering
+description: Route an agent-reliability request to the right @statelyai/agent skill and keep the four steps in order — make it a state machine, add traces, add evals, self-improve. Use when asked to "make my agent reliable", "graph engineering", "turn my agent into a state machine and improve it", "where do I start with this agent", or any request that spans more than one of those steps.
+---
+
+# Graph engineering
+
+Four steps, in order. Each one needs the one before it.
+
+| Step | Skill | Done when |
+| --- | --- | --- |
+| 1. Make it a state machine | `loop-to-machine` (existing code) or `generate-machine` (from scratch) | Rules are guards and the sentinel is unreachable (`canReach`). From existing code: the parity report has no unreviewed regressions. From scratch: every `verifyAgentConfig` gate passes |
+| 2. Add traces | `add-traces` | Every run leaves a stamped log thread that replays to its final state |
+| 3. Add evals | `add-evals` | Evals per state at the cheapest level, an objective, baseline numbers per split |
+| 4. Self-improve | `improve-machine` | A pull request with one policy change and its held-out scorecard, or a report that none helped |
+
+## Find where the project is
+
+Check in this order and start at the first step that is not done:
+
+1. Does the code path production actually runs go through a machine (`setupAgent`, `createMachine`, an `AgentWorkflowConfig`)? Follow the route or entry point to the model call; a machine that nothing calls does not count. If the path is a loop or ad-hoc code, step 1.
+2. Do runs persist to an event log store with a declared machine `version` and a `metadata` stamp? If not, step 2.
+3. Is there an objective over ground truth, and evals that name the states they cover? If not, step 3.
+4. Otherwise, step 4.
+
+Tell the user which step you are starting at and why, in one line, then load that skill.
+
+## Across all steps
+
+- **The environment is people's code.** Rules, guards, invariants, and the sentinel change only by human review. No step rewrites them on its own.
+- **Ground truth scores everything.** Outcomes and effects in the world, labeled by people. A model never grades its own runs.
+- **Ask for labels and rules; decide the mechanics.** Expected outcomes, thresholds, the objective, and the train/held-out split are the user's. File layout, test names, and helper code are yours.
+- **Hand back between steps.** Finish one step's hand-back before starting the next, unless the user asked for several in one go.

package/skills/improve-machine/SKILL.md

@@ -1,17 +1,17 @@
 ---
 name: improve-machine
-description: Improve a @statelyai/agent policy machine (a JSON AgentWorkflowConfig) from its run traces toward its goal, inside a fixed environment machine, with gates and a held-out promotion rule. Use when asked to "improve this agent", "self-improve", "optimize the playbook from traces", "tune the policy", "make the agent cheaper or more accurate", or "why is my agent failing" for a machine whose runs are persisted to an event log store.
+description: Improve a @statelyai/agent policy machine (a TypeScript setupAgent machine or a JSON AgentWorkflowConfig) from its run traces toward its goal, inside a fixed environment machine, with gates and a held-out promotion rule, and hand the result to a person as a pull request. Use when asked to "improve this agent", "self-improve", "optimize the playbook from traces", "tune the policy", "make the agent cheaper or more accurate", or "why is my agent failing" for a machine whose runs are persisted to an event log store.
 ---
 
 # Improve an agent machine
 
-You improve a **policy**: the JSON `AgentWorkflowConfig` that decides how the agent works toward its goal. It runs inside an **environment**: a TypeScript machine whose states and guards encode the rules of the world and the business invariants. One round changes the policy once and measures it:
+You improve a **policy**: the machine that decides how the agent works toward its goal, written in TypeScript or as a JSON `AgentWorkflowConfig`. It runs inside an **environment**: a TypeScript machine whose states and guards encode the rules of the world and the business invariants. One round changes the policy once and measures it:
 
 ```
-locate inputs → read traces → diagnose → propose one diff → verifyAgentConfig (≤ 3 repairs) → replay train → replay held-out → promote or discard → record
+locate inputs → read traces → diagnose → propose one diff → gate (≤ 3 repairs) → replay train → replay held-out → recommend or discard → record → pull request
 ```
 
-To author a policy from scratch, use the `generate-machine` skill. Concept page: `docs/environment-and-policy.md`.
+This is step 4 of 4, after `loop-to-machine`, `add-traces`, and `add-evals`. To author a JSON policy from scratch, use the `generate-machine` skill. Concept page: [Environment and policy](https://github.com/statelyai/agent/blob/next/docs/environment-and-policy.md).
 
 ## Hard rules
 
@@ -19,8 +19,32 @@ To author a policy from scratch, use the `generate-machine` skill. Concept page:
 - **Never edit a machine while it runs.** A promoted policy applies to new runs only.
 - **The objective comes from ground truth.** Score outcomes from the traces (terminal states, outputs) or from effects recorded in the world. Never ask a model to grade its own runs.
 - **One cause per round, smallest diff.** A diff that fixes two things cannot tell you which one helped.
-- **Diagnose from training scenarios only.** Held-out scenarios are for the promotion decision. Never read their failures to shape a change, and never move a scenario between splits.
+- **Diagnose from training scenarios only.** Held-out scenarios are for the promotion decision. Never read their failures to shape a change, and never move a scenario between splits. Eval files may list baseline failures for every split; read only the training rows.
 - **Stop after 3 rounds without a promotion** (or the limit the user gives).
+- **Propose, never promote.** A round ends in a recommendation. A recommended candidate goes to a person as a pull request with its scorecard; merging it is the promotion. Never merge, deploy, or switch the running policy yourself.
+
+## Policy format
+
+The rounds are the same for both formats. What differs is what the diff may touch and how it is gated.
+
+| | TypeScript policy | JSON policy |
+| --- | --- | --- |
+| The policy | A `setupAgent` machine in its own file(s), with a declared `version` | An `AgentWorkflowConfig` file |
+| May change | The policy file(s) and their `CHANGELOG.md`; once, the line in the runner that routes the decision to the policy; tests that pin measured numbers (see the pull request) | The config's `states` and `context` |
+| Gates | `tsc`, `lintAgentMachine`, `explorePaths` on the policy, the environment check, the overfit check | `verifyAgentConfig` |
+| Vocabulary | What the policy file imports and the events the environment offers | The `allow` lists |
+
+**No policy yet.** `loop-to-machine` leaves a single model decision inside the environment when the old code had no policy logic. Then:
+
+- Record `v0` in a new `CHANGELOG.md` next to the environment (in a `policy/` directory): "the model decides directly". This entry goes in the working tree.
+- The first recommended candidate introduces the policy as `v1`: a policy file that reproduces the current behavior (it passes the environment's prompt and facts through to the model) plus the one rule the diagnosis found, and the routing line in the runner (`withPolicy(modelExecutors)`). The policy runs as the decider through `createPolicyDecider` from `@statelyai/agent`; the pattern is in `../loop-to-machine/policy.md`.
+- Its `description` is the environment's goal with measurable success criteria. If the environment's goal has none, draft them from the confirmed objective and list the draft under questions for the user.
+- If the policy needs a fact the environment does not pass in the decision's `metadata.facts`, that is an environment change request.
+
+For a TypeScript policy, before the first round:
+
+- **List the environment files** (the environment machine, its guards, the world and its tools) and record a hash of each. Any change to them fails the round. The runner is not one of them, but its diff may contain only the routing line.
+- **Keep the policy a machine.** A diff may add states, `choice` branches, guards over facts, helpers in the policy file, or change a prompt or a model ref. It may not turn the policy into a free function, call tools the environment owns, or import from the environment anything but its schemas, types, constants, and exported guard functions.
 
 ## 1. Locate the inputs
 
@@ -30,16 +54,16 @@ Find all six before changing anything. If one is missing, stop and ask.
 | -------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
 | Goal           | Root `description` of the policy config or the environment machine                               | Draft one with measurable success criteria; ask the user to confirm it      |
 | Objective      | A scorer over outcomes (an eval file, a `score*` function, a benchmark)                          | Write one from the goal's success criteria; **ask the user to confirm it**  |
-| Traces         | The `store` passed to `createAgentRuntime` / `runAgent`; its `listThreads`, or recorded ids      | Wire a store first (see `generate-machine` step 6) and collect runs         |
+| Traces         | The `store` passed to `createAgentRuntime` / `runAgent`; its `listThreads`, or recorded ids      | Wire a store first (the `add-traces` skill) and collect runs                |
 | Scenarios      | A scenario list with expected outcomes, split into train and held-out                            | Propose a fixed split (about 70/30, every category in both); ask to confirm |
 | Vocabulary     | The actors, requests, named guards, actions, decision events, and model refs the host implements | Read them off the host code; never invent names                             |
 | Policy history | The policy files and a `CHANGELOG.md` next to them                                               | Create the changelog with the current policy as `v0`                        |
 
-The objective defines "better". Do not start a round until the user has confirmed it. A good objective has:
+The objective defines "better". Do not start a round until the user has confirmed it, including how two versions compare when it has more than one field. A good objective has:
 
-- a primary metric from the goal (accuracy against expected outcomes, resolution rate),
+- a primary metric from the goal (accuracy against expected outcomes and effects, resolution rate). With several, state the rule: one primary, or none worse and one better,
 - a hard constraint: invariant violations, which must be 0,
-- a cost term (model calls, tokens, guard rejections) used as a tie-breaker or a stated budget.
+- a cost term (model calls, tokens, guard rejections) used as a tie-breaker or a stated budget. Count the calls that reach a model executor, not the environment's decision requests: a policy that answers with a rule makes no model call, and a cost term that counts requests cannot see the saving.
 
 ## 2. Read
 
@@ -48,6 +72,7 @@ Read both machines as text. `describeMachine` puts the goal first:
 ```ts
 import { describeMachine, setupAgent } from "@statelyai/agent";
 
+// A TypeScript policy: import it. A JSON policy: lower it.
 const { machine: policy } = setupAgent.fromConfig(policyConfig, { compileSchema });
 const environmentText = describeMachine(environment);
 const policyText = describeMachine(policy);
@@ -58,8 +83,9 @@ Then find the training threads and compute one outcome row per episode:
 ```ts
 import { getDecisionAttempts, getUsageFromEvents, replay } from "@statelyai/agent/log";
 
-// Thread ids name the split and policy version: "train:v<n>:<scenario>:r<k>".
-const prefix = `train:v${currentVersion}:`;
+// Thread ids name the split and the decider version, as add-traces set up: "train:<deciderVersion>:<scenario>:r<k>".
+// With a policy, the decider version is the policy's version ("v3"); before one exists, the model's label.
+const prefix = `train:${deciderVersion}:`;
 const threadIds: string[] = [];
 let cursor: string | undefined;
 do {
@@ -100,9 +126,10 @@ Group the failing training episodes by cause. Check these clusters in order:
 
 1. **Invariant violations.** Any violation is a stop: the environment is wrong, not the policy. Write a change request.
 2. **Wrong terminal state.** The episode ended in `escalated` when the label says `issued`, or the reverse. Look at which decision led there.
-3. **Guard rejections.** The policy keeps proposing events the environment refuses: `rejected-by-guard` entries in the rows' `failedAttempts`. The policy lacks a check the environment already makes.
-4. **Loops and budget exhaustion.** Repeated state values, or a terminal reached because a turn or retry budget ran out.
-5. **Cost hotspots.** Correct episodes that spend model calls where a `choice` over known facts, or a cheap judgment, would decide.
+3. **Right terminal state, wrong effect.** The episode ended `issued` with the wrong amount, or reached the right state by a costlier path. Compare the chosen event with the offered options at each decision.
+4. **Guard rejections.** The policy keeps proposing events the environment refuses: `rejected-by-guard` entries in the rows' `failedAttempts`. The policy lacks a check the environment already makes.
+5. **Loops and budget exhaustion.** Repeated state values, or a terminal reached because a turn or retry budget ran out.
+6. **Cost hotspots.** Correct episodes that spend model calls where a `choice` over known facts, or a cheap judgment, would decide.
 
 Pick the cluster with the largest effect on the objective. Write the hypothesis down before you edit anything (template in [reference.md](reference.md)): cause, evidence (thread ids), the change, and which metric should move.
 
@@ -111,12 +138,29 @@ Pick the cluster with the largest effect on the objective. Write the hypothesis
 Make the smallest policy diff that addresses the one cause.
 
 - Typical diffs: add a `choice` branch over facts already in context; replace a model call with a judgment actor from the vocabulary; narrow an `agent.decide` `allowedEvents` list; turn an open decision into a closed choice with `payloads`; tighten a prompt; add an `onError` path; lower a model ref to a cheaper one in the vocabulary.
-- Use only names in the vocabulary. A named guard the host does not implement fails the `lower` gate.
-- Keep the config `id`. Bump `version`.
+- TypeScript policy: compare facts directly in a guard or `choice` instead of adding a named guard; keep helpers in the policy file.
+- JSON policy: use only names in the vocabulary. A named guard the host does not implement fails the `lower` gate.
+- Keep the machine `id`. Bump `version`.
 - Keep the root `description` (the goal) unchanged. Changing the goal is the user's decision.
 
 ## 5. Gate
 
+### TypeScript policy
+
+Run every check; a failure's output is the repair prompt. Cap repairs at 3.
+
+1. **Environment unchanged:** every environment file hashes as recorded. A change is never repaired; discard the candidate.
+2. **Types:** `tsc --noEmit` passes.
+3. **Lint:** `lintAgentMachine(policy)` from `@statelyai/agent/testing` reports no errors.
+4. **Paths:** for probe inputs built from recorded decision requests (`getDecisionRequests(trace, { name })` from `@statelyai/agent/testing`, one per training scenario that reaches the decision: its facts and offered options), `canReach(policy, (snapshot) => snapshot.status === "done" && !isOfferedEvent(snapshot.output), { input })` is `{ reachable: false, complete: true }`: every final output is one of the offered events. Pass `outputs` for any actor the policy invokes, and `errors: { "agent.decide": new Error("down") }` so the fallback path is checked too.
+5. **Source rules:** the policy file is still one `setupAgent` machine; its imports from the environment are only schemas, types, constants, and exported guard functions; it names no actor the environment owns. These are source checks: read the diff, or grep its imports and actor names.
+6. **Overfit:** the diff contains no literal from the scenarios (ids, ticket text, customer ids, order ids). A branch that matches one scenario's data is memorization; discard it.
+7. **Existing evals:** the rule and decision evals from `add-evals` still pass. Tests that pin measured numbers of the current version (a baseline failure list, a call count) are expected to change; update them in the same pull request and list each change in its body.
+
+The environment's own `canReach` proofs still cover every policy, because the environment's guards judge every proposed event.
+
+### JSON policy
+
 Run `verifyAgentConfig` from `@statelyai/agent/validate` with the vocabulary and probe inputs. Probe inputs are one input per training scenario category, chosen to reach every branch the diff touches.
 
 ```ts
@@ -150,15 +194,15 @@ const failed = verified.gates.find((gate) => !gate.ok);
 
 ## 6. Replay
 
-Run the candidate on the training scenarios, then on the held-out scenarios. Write every replay to the store under new thread ids that name the split and version (`train:v3:<scenario>:r1`, `heldout:v3:<scenario>:r1`), so the next round can list them by prefix.
+Run the candidate on the training scenarios, then on the held-out scenarios. Write every replay to the store under new thread ids that name the split and version, in the scheme `add-traces` set up (`train:v3:<scenario>:r1`, `held-out:v3:<scenario>:r1`), so the next round can list them by prefix. With no persistent store, write each thread's `result.events` to a file.
 
 - Run the current policy and the candidate on the same scenarios, the same world state, and the same model refs.
 - Model calls are nondeterministic. Run each scenario k times (k ≥ 3 when calls are not cached) and compare means.
 - Compute the objective, invariant violations, and cost per split, for both versions.
 
-## 7. Promote or discard
+## 7. Recommend or discard
 
-State the tolerance before you replay. Promote only when every condition holds:
+State the tolerance before you replay. Recommend only when every condition holds:
 
 - invariant violations are 0 on both splits,
 - the objective on the held-out split is better than the current policy's,
@@ -167,6 +211,14 @@ State the tolerance before you replay. Promote only when every condition holds:
 
 Otherwise discard the candidate and keep the current policy. Either way, append an entry to the `CHANGELOG.md` next to the policies: version, hypothesis, diff summary, gate results, before and after per split, and the decision. Record discarded candidates too; they stop the next round from retrying the same idea.
 
+### The pull request
+
+A recommended candidate becomes one pull request (or, with no remote, one branch and a diff file), opened against the repository's integration branch:
+
+- **The diff:** the policy file(s) and the new `CHANGELOG.md` entry; the runner's routing line, when this candidate introduces the policy; and updated pinned measurements, each listed in the body. Nothing else.
+- **The body:** the hypothesis, the scorecard (objective, violations, and cost before and after, train and held-out separately, k runs each), every scenario that changed outcome, the gate results, and the thread ids a reviewer can replay.
+- **One candidate per pull request.** Leave your working tree on the current policy. Later rounds build on the current policy, not on an unmerged candidate.
+
 ## 8. Stop and report
 
 Stop when any of these holds:
@@ -176,11 +228,11 @@ Stop when any of these holds:
 - the remaining failures need an environment change (write a change request),
 - the vocabulary lacks a capability the fix needs (ask for it; name the actor, request, or guard and its input and output).
 
-Report to the user: rounds run, each candidate with its hypothesis and decision, the objective before and after on both splits, violations, cost, and any change requests.
+Report to the user: rounds run, each candidate with its hypothesis and decision, the pull request link (or, with no remote, the branch and diff file), the objective before and after on both splits, violations, cost, and any change requests.
 
 ## Worked example
 
-`examples/self-improving-refunds` in the `statelyai/agent` repo runs this loop end to end:
+[`examples/self-improving-refunds`](https://github.com/statelyai/agent/blob/next/examples/self-improving-refunds) in the `statelyai/agent` repo (not in the installed package) runs this loop end to end for a JSON policy:
 
 - the refund desk is the environment: business rules as guards, and an `illegallyIssued` sentinel proven unreachable,
 - `playbooks/*.json` are policy versions, with the vocabulary listed in `playbooks/README.md`,

package/skills/improve-machine/reference.md

@@ -12,10 +12,11 @@ Templates for the `improve-machine` skill: a round checklist, the hypothesis, th
 [ ] Baseline: objective, violations, cost per split for the current version
 [ ] Failures clustered (train only); one cause picked
 [ ] Hypothesis written
-[ ] Candidate: smallest diff, id kept, version bumped, description unchanged
-[ ] verifyAgentConfig passed (repairs used: _/3)
+[ ] Candidate: smallest diff, id kept, version bumped, description unchanged (or, for a new policy: v0 recorded, v1 reproduces current behavior plus one rule)
+[ ] Gates passed: TS (env unchanged, tsc, lint, paths, source rules, overfit, evals) or JSON (verifyAgentConfig) (repairs used: _/3)
 [ ] Replayed train, then held-out, k runs each, new thread ids
 [ ] Promotion rule applied; CHANGELOG entry written
+[ ] Recommended candidate opened as one pull request with its scorecard; not merged
 [ ] Stop condition checked
 ```
 
@@ -39,7 +40,7 @@ You improve a policy machine. You may change only the policy config below.
 {describeMachine(environment)}
 
 Current policy (version {n}):
-{JSON.stringify(policyConfig, null, 2)}
+{the policy file's source, or JSON.stringify(policyConfig, null, 2)}
 
 Vocabulary (use no other names):
 - actors: {actors}
@@ -53,8 +54,12 @@ Objective on the training scenarios: {objective summary}
 Diagnosed cause: {hypothesis.cause}
 Evidence: {failing rows: scenario, expected, reached state, rejections, tokens}
 
-Return one complete AgentWorkflowConfig as JSON. Make the smallest change that
-addresses the diagnosed cause. Keep "id" and "description". Set "version" to {n + 1}.
+Return the complete policy: the TypeScript file, or one AgentWorkflowConfig as
+JSON. Make the smallest change that addresses the diagnosed cause. Keep "id" and
+"description". Set "version" to {n + 1}. Never match on a scenario's data.
+For a TypeScript policy, keep the result one `setupAgent` machine. Import from
+the environment only schemas, types, constants, and exported guard functions.
+Do not invoke tools the environment owns.
 ```
 
 On a failed gate, append:
@@ -69,11 +74,11 @@ Return a corrected config.
 ## Changelog entry
 
 ```md
-## v{n} — {promoted | discarded}
+## v{n} — {recommended | discarded}
 
 - Hypothesis: {cause} → {change}
 - Diff: {states or transitions touched}
-- Gates: validate ✓ vocabulary ✓ lower ✓ lint ✓ explore ✓ (repairs: {r})
+- Gates: {TS: environment ✓ tsc ✓ lint ✓ paths ✓ overfit ✓ evals ✓ | JSON: validate ✓ vocabulary ✓ lower ✓ lint ✓ explore ✓} (repairs: {r})
 - Train: objective {a} → {b}, violations {v}, cost {c0} → {c1}
 - Held-out: objective {a} → {b}, violations {v}, cost {c0} → {c1}
 - Regressions: {none | scenario ids}

package/skills/loop-to-machine/SKILL.md

@@ -0,0 +1,210 @@
+---
+name: loop-to-machine
+description: Convert existing agent code (a while-loop around a model, a tool loop, ad-hoc if/else control flow) into an XState v6 state machine with @statelyai/agent, and prove with a parity report that it behaves like the code it replaces. Use when asked to "turn this agent into a state machine", "port my loop to xstate", "make this a statechart", "refactor this agent into a machine", or "split environment and policy" for code that already runs.
+---
+
+# Convert agent code to a machine
+
+You replace the control flow of working agent code with a machine, and show where the machine behaves like the old code and where it differs on purpose. The model calls, tools, and retry code move across unchanged.
+
+```
+inventory → pin the baseline → environment → (policy) → runner → proofs → parity → hand back
+```
+
+This is step 1 of 4. Next: `add-traces`, then `add-evals`, then `improve-machine`. To author a machine with no existing code, use `generate-machine`.
+
+## Hard rules
+
+- **Pin the baseline before you change anything.** The parity test runs against the old code, which stays in the repo, untouched, until a person deletes it.
+- **Never promise "behaves exactly like".** Promise what the parity report shows: same outcome and effects on these scenarios, with every difference labeled.
+- **Never copy a rule break.** Where the old code broke a rule, the machine blocks it. That difference is `intended`, not a regression.
+- **Rules are code, not prompt text.** A rule that lived only in a system prompt becomes a guard. Keep the prompt line only as context for the model.
+- **Business rules belong to people.** If a rule is ambiguous (which threshold, which order of checks), ask. Never pick one silently.
+
+## 1. Inventory the code
+
+Read the loop and fill this table before writing anything. Each row maps to a machine element.
+
+| In the code                                                                  | Becomes                                                                  |
+| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
+| A tool with a side effect someone must gate (refund, send)                   | An actor invoked by one state, entered only through a guarded transition |
+| A read-only tool (search, lookup)                                            | An actor the machine invokes itself, or a tool on a text request         |
+| A rule: an `if`, a prompt line, a check after the fact                       | A guard on the transition it protects                                    |
+| A model call that produces content (a draft, generated code, a summary)      | A text request in `setupAgent({ requests })`, invoked by one state       |
+| One SDK tool call that both picks the action and carries its content (`edit_graph({ … })`, `send_email({ … })`) | A text request with those tools, then guards on the returned call: exactly one, a tool that was offered, input that passes the tool's schema. Keep it a text request: `agent.decide` would rename the tools to `send_event_<TYPE>` with the description "Choose the '<TYPE>' move.", so the model would no longer see your tool names and descriptions |
+| A tool the browser runs (no `execute`; the client sends results back)        | The request ends with the tool calls; an idle state waits for a `TOOL_RESULTS` event the route sends when the client posts results |
+| A point where the model picks what to do                                     | A decision state: `agent.decide`, usually a closed choice                |
+| A check on the model's output, then a retry or a repair call                 | An actor or guard that checks it, and a `choice` state that routes to the repair request, the next candidate, or a failure final |
+| A value the model pulled out of the input (an order id, an amount)           | Code when the format is fixed (a regex); a text request when it is not   |
+| A step every correct run takes in the same order (verify, then look up)      | A state the machine enters itself; the model no longer chooses it        |
+| A person in the loop: `await approve()`, a returned `pending`, or a tool that asks a person (`askCustomer`, `requestApproval`), even a synchronous one | An idle state with `meta.interaction`, answered by `onIdle` in the runner |
+| A cap (`turns < 8`, a retry counter) on a loop the machine keeps             | A `choice` state against an exported constant                            |
+| A cap on a loop the machine removed                                          | Nothing. Record that it is gone and why                                  |
+| Each distinct return                                                         | A final state                                                            |
+| A terminal tool with a side effect (`escalate`, `close`)                     | A state that invokes it, then the final state; its `onError` goes to a `failed` final |
+
+List the rules as a numbered list in your notes. Every one must end as a guard, a state ordering, or a written reason it was dropped.
+
+A loop with no decision point (generate, validate, repair) is still worth converting: the machine makes the retry cap, the validation rules, and every failure path explicit and provable. It has no `agent.decide`, so step 4 and the decision parts of step 5 do not apply.
+
+## 2. Pin the baseline
+
+The parity test exists before the machine does.
+
+1. **Scenarios.** Reuse the code's tests, fixtures, or benchmark cases. Each has an `id` and, where people know it, the `expected` outcome. If there are none, draft 6–10 that cover every terminal and every rule, and ask the user to confirm them.
+2. **Deterministic stand-in models.** Parity with a live model drifts between runs. Use the code's scripted or mock models if it has them; otherwise write one: a function from the request (its prompt, or its `name`) to a fixed answer. ([`examples/mock-model.ts`](https://github.com/statelyai/agent/blob/next/examples/mock-model.ts) in the `statelyai/agent` repo is one, built on the AI SDK's `MockLanguageModelV3`; it is not in the package.) You want two: one that follows the rules, and one that breaks them. A seeded random model needs its seed derived from the scenario id and a fresh instance per run, or every scenario replays the same draws.
+3. **Ground truth.** Read outcomes and effects from the world, not from the agent's own report. `effects` holds **every write the world records**: the refunds, and also the approvals, the verifications, the questions asked, the escalations. A refund issued without its approval looks correct if `effects` holds only refunds. Text a model wrote (an escalation reason) differs on every run: compare its count or presence, not the text. If the world can check rules after the fact, return those violations. If it cannot, write the check as a plain function over the effects.
+4. **A baseline with no function to call.** When the loop lives inline in a framework handler (a tRPC procedure, a route), call the handler itself in the test, with its model client and storage replaced by the stand-ins (a module mock, or the framework's test caller). Do not extract the loop into a function first: that changes the code you are trying to pin.
+5. **The harness check.** Run the old code on both sides with each stand-in model. Each row's baseline must equal its candidate (the runs are deterministic), and with the rule-following model every row must be `same`. With a rule-breaking model, rows are `regression` by design: the candidate breaks rules.
+
+```ts
+import { compareRuns, type ParityResult, type ParityRun } from "@statelyai/agent/testing";
+
+function toParityRun(run: { outcome: string; world: World }): ParityRun {
+  return {
+    outcome: run.outcome,
+    // Every write the world records; model-written text as a count.
+    effects: { refunds: run.world.refunds, approvals: run.world.approvals, verified: [...run.world.verified], escalations: run.world.escalations.length },
+    violations: run.world.violations, // rules broken, from ground truth
+  };
+}
+
+// A model factory, called once per scenario and per side. Construct a stateful or seeded
+// model inside it (seeded from s.id); returning a shared stateless object is fine.
+const runWith = (runner: typeof runLoop, model: (s: Scenario) => Model) => async (s: Scenario) =>
+  toParityRun(await runner({ ...s, model: model(s), world: createWorld(s) }));
+
+const MODELS: Array<[string, (s: Scenario) => Model]> = [
+  ["careful", () => carefulModel],
+  ["sloppy", () => sloppyModel],
+];
+
+test.each(MODELS)("harness: the loop matches itself (%s)", async (_, model) => {
+  const report = await compareRuns({ scenarios: SCENARIOS, baseline: runWith(runLoop, model), candidate: runWith(runLoop, model) });
+  for (const row of report.scenarios) expect(row.candidate, row.id).toEqual(row.baseline);
+});
+```
+
+## 3. Author the environment
+
+The environment is the machine: states, guards that encode the rules, side-effecting actors, an invariant sentinel. Use `setupAgent` (from `@statelyai/agent`) with zod schemas for `input`, `context`, `output`, and every event. Export the event schemas as one object, so a policy can reuse them later.
+
+A gated action:
+
+```ts
+deciding: {
+  tags: ["decision"],
+  invoke: {
+    src: "agent.decide",
+    input: ({ context }) => ({
+      model: "decider", // a key the host's executors resolve
+      name: "chooseResolution", // tests, traces, and a later policy find it by this name
+      prompt: `Ticket: ${context.ticket}\nOrder: ${JSON.stringify(context.order)}`,
+      allowedEvents: ["REFUND", "ESCALATE", "CLOSE"],
+      // A closed choice: the decider picks one of these. The runtime drops the ones the guards refuse.
+      payloads: { REFUND: refundAmounts(context).map((amount) => ({ amount })) },
+      // The facts a decider reads. Arrives as `request.metadata.facts`.
+      metadata: { facts: { order: context.order, requestedAmount: context.requestedAmount } },
+    }),
+    onError: { target: "escalating" }, // retries exhausted, or no option legal
+  },
+  on: {
+    REFUND: ({ context, event }) =>
+      refundIsLegal(context, event.amount)
+        ? { target: event.amount > APPROVAL_THRESHOLD ? "awaitingApproval" : "paying", context: { amount: event.amount } }
+        : undefined, // undefined rejects: the decision retries with feedback
+    // `escalate` and `close` are tools with side effects: states that invoke them, then the final states.
+    ESCALATE: { target: "escalating" },
+    CLOSE: { target: "closing" },
+  },
+},
+```
+
+- **Do the deterministic steps yourself.** Identity, lookup, the ownership check: states the machine runs in order. A model only answers what genuinely needs judgment.
+- **Export the guards** as plain functions (`refundIsLegal`) so tests can call them, and build the machine in a factory (`createDeskMachine({ refundIsLegal })`) so a test can pass a loosened guard.
+- **Add an invariant sentinel** for the rule that must never break: a final state with a description starting `SENTINEL:`, entered from an `always` on the state that invokes the side effect (`paying`). An `always` is taken before the state's invoke starts, so the effect never runs. Write the check as its own function, not a call to the guard, so loosening the guard does not loosen the check.
+- **A person's step** is an idle state whose `meta.interaction` tells a host what to render: `{ label: "Approve a refund of ${amount}?", events: { APPROVE: { label: "Approve" }, REJECT: { label: "Reject" } } }`, or `textEvent: { type: "CUSTOMER_REPLIED", field: "reply" }` for free text.
+- **Actors are placeholders.** Declare each tool as `createAsyncLogic<Output, Input>({ run })` from `xstate` in `setupAgent({ actors })` (XState v6 has no `fromPromise`). Give the type parameters, or `onDone`'s `output` is untyped. Pass the real implementations at run time with `runAgent(machine, { actors })`.
+- **Final outputs carry the machine's result** (`{ outcome, orderId, amount }`), JSON only. The runner rebuilds the old return type from it and from the world (step 5).
+- **Declare `version`** on `createMachine` (`version: "1"`). Behavior lives in transition functions, which the structural hash cannot see; traces stamp this string.
+
+## 4. A policy, only if the code already has one
+
+Keep the decision in the environment as one `agent.decide`, and do not write a policy machine, unless the old code already has decision logic beyond one model call: a rule ladder, heuristics that pick a strategy, a cheap check before the model. A policy that only passes the options through to a model is boilerplate. `improve-machine` introduces a policy when the traces show a rule worth adding.
+
+If the code does have that logic, port it as a policy machine that answers `chooseResolution`. The pattern, with code that compiles, is in [policy.md](policy.md).
+
+## 5. The runner
+
+Write `runMachine` with the old function's signature and result type, so callers and the parity test change one line.
+
+- **The existing SDK call becomes the executor.** Write `generateText` (or `streamText`) in `executors` as the old call, with the same model, system prompt, tools, and options, and return `{ result }`. The request still needs `prompt` or `messages` in `setupAgent({ requests })` (`setupAgent` throws `invalid-request` otherwise): pass the conversation as `messages: ({ input }) => input.messages` and read `request.messages` in the executor.
+- **A streaming route keeps its response, not its function signature.** `createAiSdkExecutors().streamText` forwards text chunks only, so a UI stream with tool calls cannot come out of it. Write the `streamText` executor around the old call, pipe its UI stream to the route's response yourself, and return the final messages as `{ result }`.
+
+- `runAgent(machine, { input, executors, actors, onIdle })`.
+- `onIdle` answers each idle state with the old code's call (`tools.askCustomer(...)`, `tools.requestApproval(...)`). Write the returned events with `as const` on `type`, or the event union does not narrow.
+- `executors.decide` adapts the code's model to the decision: the closed choice arrives as `request.options` (each `{ event }`), and the facts as `request.metadata?.facts`, typed `unknown`: parse it with the exported facts schema. Every executor slot is optional in the type; check or assert the ones you call.
+- In an `onError` function, the error is `event.error`: `onError: ({ event }) => ({ target: "failed", context: { error: String(event.error) } })`. There is no `error` argument.
+- Make input and events JSON-safe at the boundary: no `Date`, `Map`, class instances, or functions (fields set to `undefined` are dropped). With a store, a value the log cannot hold fails the run; without one, the log stops with a warning.
+- Rebuild the old result from `result.output` and the world. A field that has no meaning anymore (a turn count with no loop) keeps its type and gets a comment saying what it now holds.
+
+## 6. Proofs that need no model
+
+From `@statelyai/agent/testing`:
+
+- `lintAgentMachine(machine)`: no errors.
+- `canReach(machine, "<sentinel>", options)` returns `reachable: false` **and** `complete: true`, for each distinct input shape (an input with an order id and one without lead to different paths; loop over them).
+- `options.payloads` is a function of the snapshot that returns payload lists per event type. List values on both sides of every guard boundary: `0`, the threshold and one over, and each order amount the outputs below can produce and one over. Without it, a closed choice is explored only with its own `payloads`, and `complete: true` then proves nothing about the boundaries.
+- A free-text event from a person's step (`CUSTOMER_REPLIED`) needs an entry too, one payload per branch it can take, or the search cannot send it.
+
+```ts
+const proofOptions: ExplorePathsOptions = {
+  payloads: () => ({
+    REFUND: [0, 40, 41, 100, 101, 240, 241, 800, 801].map((amount) => ({ amount })),
+    CUSTOMER_REPLIED: [{ reply: "It's ORD-1001" }, { reply: "no idea" }],
+  }),
+  outputs: { lookupOrder: [ownedOrder, otherCustomersOrder, refundedOrder, { found: false }] },
+  errors: { lookupOrder: new Error("down"), "agent.decide": new Error("exhausted") },
+};
+```
+- `options.outputs` lists every output each actor can return (found and not found, owned and not, refunded and not); `options.errors` makes each actor fail once.
+- **A machine with a loop** (a repair cycle, a resubmit) needs `options.dedupeBy`, or the search runs to `maxDepth` and reports `complete: false`. Return every context field a guard or choice reads: `dedupeBy: (s) => JSON.stringify([s.value, s.context.attempts, s.context.lastError])`. `outputs` are canned per actor src, not per call: to explore "fails twice, then succeeds", let the guard read a counter from context instead of the output.
+- `explorePaths(...).reachedStates` lists only states a run settles in; a `choice` state it passes through is not listed.
+- A loosened guard makes `canReach` true. Write this test too, or the proof proves nothing.
+- Each normal final state is reachable, so the search is not vacuous.
+
+Type the options with `ExplorePathsOptions`, and a `payloads`, `dedupeBy`, or predicate callback's snapshot with `ExploreSnapshot`, both from `@statelyai/agent/testing`. Do not use `AnyMachineSnapshot` from your app's own `xstate`, which may be another version. `errors` takes one error per actor src, and `"agent.decide"` is a valid key.
+
+## 7. Parity
+
+Switch the parity test's candidate to `runMachine`, and run it with each stand-in model:
+
+```ts
+test.each(MODELS)("parity with the loop: %s", async (_, model) => {
+  const report = await compareRuns({ scenarios: SCENARIOS, baseline: runWith(runLoop, model), candidate: runWith(runMachine, model) });
+  const unreviewed = report.scenarios.filter((row) => row.label === "regression" && !REVIEWED[row.id]);
+  expect(unreviewed).toEqual([]);
+  // A reviewed difference still has to be exactly the difference a person accepted.
+  for (const row of report.scenarios.filter((row) => REVIEWED[row.id])) REVIEWED[row.id]!(row);
+});
+
+// Effect-only differences a person accepted: who, and the exact difference.
+const REVIEWED: Record<string, (row: ParityResult<Scenario>) => void> = {
+  // "no-order-id-no-reply": (row) => { /* accepted by <name>: asks once, not three times */ expect(row.candidate.outcome).toBe(row.baseline.outcome); ... },
+};
+```
+
+- Every `regression` row: fix the machine, or, when the old behavior was wrong but broke no rule, ask the user. If the outcomes differ, the answer may become the scenario's `expected`. If only effects differ (the loop asked three times, the machine once), list the row in a reviewed-differences list in the test, with the exact difference asserted and the person who accepted it. Never edit a scenario to make a row pass.
+- Every `intended` row: check that its reason is a real rule. A candidate that fixed a rule break with the same outcome is `intended` too.
+- Run once with a live model if a key is available. Report it as one sample, not a guarantee.
+
+## 8. Hand back
+
+Report, in this order:
+
+1. The files: environment, runner, tests (and a policy, if the code had one). The old code is untouched.
+2. The rule list from step 1, each mapped to its guard, its state, or the reason it was dropped.
+3. The parity table per stand-in model: `same / intended / regression`, and one line per `intended` row with its reason.
+4. The `canReach` results and the loosened-guard test.
+5. Anything you had to decide that the user should confirm.
+6. Next step: `add-traces`.