@statelyai/agent 2.0.0-alpha.28 → 2.0.0-alpha.30

package/skills/graph-engineering/SKILL.md

@@ -0,0 +1,33 @@
+---
+name: graph-engineering
+description: Route an agent-reliability request to the right @statelyai/agent skill and keep the four steps in order — make it a state machine, add traces, add evals, self-improve. Use when asked to "make my agent reliable", "graph engineering", "turn my agent into a state machine and improve it", "where do I start with this agent", or any request that spans more than one of those steps.
+---
+
+# Graph engineering
+
+Four steps, in order. Each one needs the one before it.
+
+| Step | Skill | Done when |
+| --- | --- | --- |
+| 1. Make it a state machine | `loop-to-machine` (existing code) or `generate-machine` (from scratch) | Rules are guards and the sentinel is unreachable (`canReach`). From existing code: the parity report has no unreviewed regressions. From scratch: every `verifyAgentConfig` gate passes |
+| 2. Add traces | `add-traces` | Every run leaves a stamped log thread that replays to its final state |
+| 3. Add evals | `add-evals` | Evals per state at the cheapest level, an objective, baseline numbers per split |
+| 4. Self-improve | `improve-machine` | A pull request with one policy change and its held-out scorecard, or a report that none helped |
+
+## Find where the project is
+
+Check in this order and start at the first step that is not done:
+
+1. Is the agent's control flow a machine (`setupAgent`, `createMachine`, an `AgentWorkflowConfig`)? If it is a loop or ad-hoc code, step 1.
+2. Do runs persist to an event log store with a declared machine `version` and a `metadata` stamp? If not, step 2.
+3. Is there an objective over ground truth, and evals that name the states they cover? If not, step 3.
+4. Otherwise, step 4.
+
+Tell the user which step you are starting at and why, in one line, then load that skill.
+
+## Across all steps
+
+- **The environment is people's code.** Rules, guards, invariants, and the sentinel change only by human review. No step rewrites them on its own.
+- **Ground truth scores everything.** Outcomes and effects in the world, labeled by people. A model never grades its own runs.
+- **Ask for labels and rules; decide the mechanics.** Expected outcomes, thresholds, the objective, and the train/held-out split are the user's. File layout, test names, and helper code are yours.
+- **Hand back between steps.** Finish one step's hand-back before starting the next, unless the user asked for several in one go.

package/skills/improve-machine/SKILL.md

@@ -0,0 +1,241 @@
+---
+name: improve-machine
+description: Improve a @statelyai/agent policy machine (a TypeScript setupAgent machine or a JSON AgentWorkflowConfig) from its run traces toward its goal, inside a fixed environment machine, with gates and a held-out promotion rule, and hand the result to a person as a pull request. Use when asked to "improve this agent", "self-improve", "optimize the playbook from traces", "tune the policy", "make the agent cheaper or more accurate", or "why is my agent failing" for a machine whose runs are persisted to an event log store.
+---
+
+# Improve an agent machine
+
+You improve a **policy**: the machine that decides how the agent works toward its goal, written in TypeScript or as a JSON `AgentWorkflowConfig`. It runs inside an **environment**: a TypeScript machine whose states and guards encode the rules of the world and the business invariants. One round changes the policy once and measures it:
+
+```
+locate inputs → read traces → diagnose → propose one diff → gate (≤ 3 repairs) → replay train → replay held-out → recommend or discard → record → pull request
+```
+
+This is step 4 of 4, after `loop-to-machine`, `add-traces`, and `add-evals`. To author a JSON policy from scratch, use the `generate-machine` skill. Concept page: [Environment and policy](https://github.com/statelyai/agent/blob/main/docs/environment-and-policy.md).
+
+## Hard rules
+
+- **Never edit the environment.** Not its states, guards, invariants, event schemas, or the vocabulary it exposes. When the goal is unreachable without an environment change, write a change request for a person (template in [reference.md](reference.md)) and stop.
+- **Never edit a machine while it runs.** A promoted policy applies to new runs only.
+- **The objective comes from ground truth.** Score outcomes from the traces (terminal states, outputs) or from effects recorded in the world. Never ask a model to grade its own runs.
+- **One cause per round, smallest diff.** A diff that fixes two things cannot tell you which one helped.
+- **Diagnose from training scenarios only.** Held-out scenarios are for the promotion decision. Never read their failures to shape a change, and never move a scenario between splits. Eval files may list baseline failures for every split; read only the training rows.
+- **Stop after 3 rounds without a promotion** (or the limit the user gives).
+- **Propose, never promote.** A round ends in a recommendation. A recommended candidate goes to a person as a pull request with its scorecard; merging it is the promotion. Never merge, deploy, or switch the running policy yourself.
+
+## Policy format
+
+The rounds are the same for both formats. What differs is what the diff may touch and how it is gated.
+
+| | TypeScript policy | JSON policy |
+| --- | --- | --- |
+| The policy | A `setupAgent` machine in its own file(s), with a declared `version` | An `AgentWorkflowConfig` file |
+| May change | The policy file(s) and their `CHANGELOG.md`; once, the line in the runner that routes the decision to the policy; tests that pin measured numbers (see the pull request) | The config's `states` and `context` |
+| Gates | `tsc`, `lintAgentMachine`, `explorePaths` on the policy, the environment check, the overfit check | `verifyAgentConfig` |
+| Vocabulary | What the policy file imports and the events the environment offers | The `allow` lists |
+
+**No policy yet.** `loop-to-machine` leaves a single model decision inside the environment when the old code had no policy logic. Then:
+
+- Record `v0` in a new `CHANGELOG.md` next to the environment (in a `policy/` directory): "the model decides directly". This entry goes in the working tree.
+- The first recommended candidate introduces the policy as `v1`: a policy file that reproduces the current behavior (it passes the environment's prompt and facts through to the model) plus the one rule the diagnosis found, and the routing line in the runner (`withPolicy(modelExecutors)`). The pattern is in `../loop-to-machine/policy.md`.
+- Its `description` is the environment's goal with measurable success criteria. If the environment's goal has none, draft them from the confirmed objective and list the draft under questions for the user.
+- If the policy needs a fact the environment does not pass in the decision's `metadata.facts`, that is an environment change request.
+
+For a TypeScript policy, before the first round:
+
+- **List the environment files** (the environment machine, its guards, the world and its tools) and record a hash of each. Any change to them fails the round. The runner is not one of them, but its diff may contain only the routing line.
+- **Keep the policy a machine.** A diff may add states, `choice` branches, guards over facts, helpers in the policy file, or change a prompt or a model ref. It may not turn the policy into a free function, call tools the environment owns, or import from the environment anything but its schemas, types, constants, and exported guard functions.
+
+## 1. Locate the inputs
+
+Find all six before changing anything. If one is missing, stop and ask.
+
+| Input          | Where to look                                                                                    | If missing                                                                  |
+| -------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
+| Goal           | Root `description` of the policy config or the environment machine                               | Draft one with measurable success criteria; ask the user to confirm it      |
+| Objective      | A scorer over outcomes (an eval file, a `score*` function, a benchmark)                          | Write one from the goal's success criteria; **ask the user to confirm it**  |
+| Traces         | The `store` passed to `createAgentRuntime` / `runAgent`; its `listThreads`, or recorded ids      | Wire a store first (the `add-traces` skill) and collect runs                |
+| Scenarios      | A scenario list with expected outcomes, split into train and held-out                            | Propose a fixed split (about 70/30, every category in both); ask to confirm |
+| Vocabulary     | The actors, requests, named guards, actions, decision events, and model refs the host implements | Read them off the host code; never invent names                             |
+| Policy history | The policy files and a `CHANGELOG.md` next to them                                               | Create the changelog with the current policy as `v0`                        |
+
+The objective defines "better". Do not start a round until the user has confirmed it, including how two versions compare when it has more than one field. A good objective has:
+
+- a primary metric from the goal (accuracy against expected outcomes and effects, resolution rate). With several, state the rule: one primary, or none worse and one better,
+- a hard constraint: invariant violations, which must be 0,
+- a cost term (model calls, tokens, guard rejections) used as a tie-breaker or a stated budget. Count the calls that reach a model executor, not the environment's decision requests: a policy that answers with a rule makes no model call, and a cost term that counts requests cannot see the saving.
+
+## 2. Read
+
+Read both machines as text. `describeMachine` puts the goal first:
+
+```ts
+import { describeMachine, setupAgent } from "@statelyai/agent";
+
+// A TypeScript policy: import it. A JSON policy: lower it.
+const { machine: policy } = setupAgent.fromConfig(policyConfig, { compileSchema });
+const environmentText = describeMachine(environment);
+const policyText = describeMachine(policy);
+```
+
+Then find the training threads and compute one outcome row per episode:
+
+```ts
+import { getDecisionAttempts, getUsageFromEvents, replay } from "@statelyai/agent/log";
+
+// Thread ids name the split and policy version: "train:v<n>:<scenario>:r<k>".
+const prefix = `train:v${currentVersion}:`;
+const threadIds: string[] = [];
+let cursor: string | undefined;
+do {
+  const page = await store.listThreads!({ prefix, limit: 100, cursor });
+  threadIds.push(...page.threadIds);
+  cursor = page.cursor;
+} while (cursor !== undefined);
+
+const rows = [];
+for (const threadId of threadIds) {
+  const scenario = scenarioOf(threadId);
+  const entries = await store.read(threadId);
+  const { snapshot } = replay(environment, entries); // the recording machine; pure fold, no model calls
+  rows.push({
+    threadId,
+    expected: scenario.expected,
+    state: snapshot.value,
+    status: snapshot.status,
+    output: snapshot.output,
+    steps: entries.length,
+    tokens: getUsageFromEvents(entries).totalTokens ?? 0,
+    failedAttempts: getDecisionAttempts(entries).flatMap(({ attempts }) => attempts),
+  });
+}
+```
+
+- `store.listThreads({ prefix, limit, cursor })` pages through the threads that hold at least one entry. `cursor` is present only while more ids remain. It is optional on a custom `AgentEventLogStore`; `createInMemoryEventLogStore` implements it. Without it, use the thread ids recorded with each scenario.
+- Prefix thread ids with the split and version so a `listThreads` prefix returns one version's training runs and never a held-out thread. `scenarioOf` is your mapping from a thread id back to its scenario.
+- `store.read(threadId)` returns a thread's entries in log order.
+- `replay(machine, entries)` needs the machine, at the version, that recorded the thread. Here that is the environment: its runtime records every thread, and the policy runs only as a decider at its boundary, so never replay with the policy machine. A thread recorded by a different environment version throws `AgentMachineVersionMismatchError`. Score each thread with the version that recorded it.
+- `getDecisionAttempts(entries)` returns `{ index, eventType, attempts }` for each entry whose decision needed retries: the failed tries before the event landed, or every try of an exhausted decision (`eventType` is then `xstate.error.actor.*`). Each attempt has `failure` (`unknown-event`, `invalid-payload`, or `rejected-by-guard`), `reason`, and the attempted `event` when there was one. A first-try decision records nothing.
+- `agent.decide` invokes journal their attempts on their own. With `decideNext` at the boundary, the host must pass them with the event, `runToQuiescence(runtime, { event, attempts })` or `runtime.transition(state, event, { attempts })`, or they never reach the log.
+- Compute the objective over the rows. This is the baseline for the round.
+
+## 3. Diagnose
+
+Group the failing training episodes by cause. Check these clusters in order:
+
+1. **Invariant violations.** Any violation is a stop: the environment is wrong, not the policy. Write a change request.
+2. **Wrong terminal state.** The episode ended in `escalated` when the label says `issued`, or the reverse. Look at which decision led there.
+3. **Right terminal state, wrong effect.** The episode ended `issued` with the wrong amount, or reached the right state by a costlier path. Compare the chosen event with the offered options at each decision.
+4. **Guard rejections.** The policy keeps proposing events the environment refuses: `rejected-by-guard` entries in the rows' `failedAttempts`. The policy lacks a check the environment already makes.
+5. **Loops and budget exhaustion.** Repeated state values, or a terminal reached because a turn or retry budget ran out.
+6. **Cost hotspots.** Correct episodes that spend model calls where a `choice` over known facts, or a cheap judgment, would decide.
+
+Pick the cluster with the largest effect on the objective. Write the hypothesis down before you edit anything (template in [reference.md](reference.md)): cause, evidence (thread ids), the change, and which metric should move.
+
+## 4. Propose
+
+Make the smallest policy diff that addresses the one cause.
+
+- Typical diffs: add a `choice` branch over facts already in context; replace a model call with a judgment actor from the vocabulary; narrow an `agent.decide` `allowedEvents` list; turn an open decision into a closed choice with `payloads`; tighten a prompt; add an `onError` path; lower a model ref to a cheaper one in the vocabulary.
+- TypeScript policy: compare facts directly in a guard or `choice` instead of adding a named guard; keep helpers in the policy file.
+- JSON policy: use only names in the vocabulary. A named guard the host does not implement fails the `lower` gate.
+- Keep the machine `id`. Bump `version`.
+- Keep the root `description` (the goal) unchanged. Changing the goal is the user's decision.
+
+## 5. Gate
+
+### TypeScript policy
+
+Run every check; a failure's output is the repair prompt. Cap repairs at 3.
+
+1. **Environment unchanged:** every environment file hashes as recorded. A change is never repaired; discard the candidate.
+2. **Types:** `tsc --noEmit` passes.
+3. **Lint:** `lintAgentMachine(policy)` from `@statelyai/agent/testing` reports no errors.
+4. **Paths:** for probe inputs built from recorded decision requests (one per training scenario that reaches the decision: its facts and offered options), `canReach(policy, (snapshot) => snapshot.status === "done" && !isOfferedEvent(snapshot.output), { input })` is `{ reachable: false, complete: true }`: every final output is one of the offered events. Pass `outputs` for any actor the policy invokes, and `errors: { "agent.decide": new Error("down") }` so the fallback path is checked too.
+5. **Source rules:** the policy file is still one `setupAgent` machine; its imports from the environment are only schemas, types, constants, and exported guard functions; it names no actor the environment owns. These are source checks: read the diff, or grep its imports and actor names.
+6. **Overfit:** the diff contains no literal from the scenarios (ids, ticket text, customer ids, order ids). A branch that matches one scenario's data is memorization; discard it.
+7. **Existing evals:** the rule and decision evals from `add-evals` still pass. Tests that pin measured numbers of the current version (a baseline failure list, a call count) are expected to change; update them in the same pull request and list each change in its body.
+
+The environment's own `canReach` proofs still cover every policy, because the environment's guards judge every proposed event.
+
+### JSON policy
+
+Run `verifyAgentConfig` from `@statelyai/agent/validate` with the vocabulary and probe inputs. Probe inputs are one input per training scenario category, chosen to reach every branch the diff touches.
+
+```ts
+import { verifyAgentConfig } from "@statelyai/agent/validate";
+
+const verified = await verifyAgentConfig(candidate, {
+  compileSchema,
+  guards: hostGuards,
+  actors: hostActors,
+  allow: { actors, requests, guards, actions, events, models },
+  explore: {
+    inputs: probeInputs,
+    terminals: ["done"],
+    // Both outcomes, so the branches on either side of the classifier are probed.
+    outputs: {
+      classifyTicket: [
+        { isRefundRequest: true, confidence: 0.9 },
+        { isRefundRequest: false, confidence: 0.9 },
+      ],
+    },
+    errors: { classifyTicket: new Error("down") },
+  },
+});
+const failed = verified.gates.find((gate) => !gate.ok);
+```
+
+- Gates run in order and stop at the first failure: `validate`, `vocabulary`, `lower`, `lint`, `explore`.
+- A failed gate's `detail` names the offending path, state, or name. Send it back verbatim with the rejected config and regenerate. Cap repairs at 3. After 3, discard the candidate, record it, and move to the next cause.
+- `explore` keys besides `inputs` and `terminals` pass through to `explorePaths`: `outputs` (output lists per src: one element is a canned output, two or more fork one branch each), `errors`, `payloads` (one branch per payload), `dedupeBy` (explore each snapshot key once; the key must cover every field a guard reads), `maxDepth`, `maxPaths`.
+- The environment's own `canReach` proofs cover every policy, because the environment's guards judge every proposed event. You did not change the environment, so you do not re-prove it.
+
+## 6. Replay
+
+Run the candidate on the training scenarios, then on the held-out scenarios. Write every replay to the store under new thread ids that name the split and version, in the scheme `add-traces` set up (`train:v3:<scenario>:r1`, `held-out:v3:<scenario>:r1`), so the next round can list them by prefix. With no persistent store, write each thread's `result.events` to a file.
+
+- Run the current policy and the candidate on the same scenarios, the same world state, and the same model refs.
+- Model calls are nondeterministic. Run each scenario k times (k ≥ 3 when calls are not cached) and compare means.
+- Compute the objective, invariant violations, and cost per split, for both versions.
+
+## 7. Recommend or discard
+
+State the tolerance before you replay. Recommend only when every condition holds:
+
+- invariant violations are 0 on both splits,
+- the objective on the held-out split is better than the current policy's,
+- no single scenario regresses beyond the tolerance (for example, a scenario that passed in all k runs now fails in any),
+- cost stays within the stated budget, if there is one.
+
+Otherwise discard the candidate and keep the current policy. Either way, append an entry to the `CHANGELOG.md` next to the policies: version, hypothesis, diff summary, gate results, before and after per split, and the decision. Record discarded candidates too; they stop the next round from retrying the same idea.
+
+### The pull request
+
+A recommended candidate becomes one pull request (or, with no remote, one branch and a diff file), opened against the repository's integration branch:
+
+- **The diff:** the policy file(s) and the new `CHANGELOG.md` entry; the runner's routing line, when this candidate introduces the policy; and updated pinned measurements, each listed in the body. Nothing else.
+- **The body:** the hypothesis, the scorecard (objective, violations, and cost before and after, train and held-out separately, k runs each), every scenario that changed outcome, the gate results, and the thread ids a reviewer can replay.
+- **One candidate per pull request.** Leave your working tree on the current policy. Later rounds build on the current policy, not on an unmerged candidate.
+
+## 8. Stop and report
+
+Stop when any of these holds:
+
+- 3 rounds in a row without a promotion,
+- the goal's success criteria are met on the held-out split,
+- the remaining failures need an environment change (write a change request),
+- the vocabulary lacks a capability the fix needs (ask for it; name the actor, request, or guard and its input and output).
+
+Report to the user: rounds run, each candidate with its hypothesis and decision, the pull request link (or, with no remote, the branch and diff file), the objective before and after on both splits, violations, cost, and any change requests.
+
+## Worked example
+
+[`examples/self-improving-refunds`](https://github.com/statelyai/agent/blob/main/examples/self-improving-refunds) in the `statelyai/agent` repo (not in the installed package) runs this loop end to end for a JSON policy:
+
+- the refund desk is the environment: business rules as guards, and an `illegallyIssued` sentinel proven unreachable,
+- `playbooks/*.json` are policy versions, with the vocabulary listed in `playbooks/README.md`,
+- `improve.ts` is the loop: propose, gate, replay, promote,
+- `benchmark.ts` compares a raw tool loop, the desk with a raw model deciding, and the desk with a playbook deciding.
+
+Read the directory before you start. File contents change between releases.

package/skills/improve-machine/reference.md

@@ -0,0 +1,101 @@
+# Improve-machine templates
+
+Templates for the `improve-machine` skill: a round checklist, the hypothesis, the proposer prompt, the changelog entry, and the environment change request.
+
+## Round checklist
+
+```
+[ ] Goal confirmed (root description, measurable success criteria)
+[ ] Objective confirmed by the user (primary metric, violations = 0, cost term)
+[ ] Train / held-out split fixed; tolerance stated
+[ ] Vocabulary listed (actors, requests, guards, actions, events, models)
+[ ] Baseline: objective, violations, cost per split for the current version
+[ ] Failures clustered (train only); one cause picked
+[ ] Hypothesis written
+[ ] Candidate: smallest diff, id kept, version bumped, description unchanged (or, for a new policy: v0 recorded, v1 reproduces current behavior plus one rule)
+[ ] Gates passed: TS (env unchanged, tsc, lint, paths, source rules, overfit, evals) or JSON (verifyAgentConfig) (repairs used: _/3)
+[ ] Replayed train, then held-out, k runs each, new thread ids
+[ ] Promotion rule applied; CHANGELOG entry written
+[ ] Recommended candidate opened as one pull request with its scorecard; not merged
+[ ] Stop condition checked
+```
+
+## Hypothesis
+
+```
+Cause: <one sentence, the mechanism, not the symptom>
+Evidence: <thread ids and what each shows>
+Change: <the one diff, in words>
+Expected effect: <metric> moves from <x> to about <y>; nothing else changes
+Risk: <scenarios that could regress, and why>
+```
+
+## Proposer prompt
+
+Use this when a model proposes the candidate. Fill every slot from real data.
+
+```
+You improve a policy machine. You may change only the policy config below.
+
+{describeMachine(environment)}
+
+Current policy (version {n}):
+{the policy file's source, or JSON.stringify(policyConfig, null, 2)}
+
+Vocabulary (use no other names):
+- actors: {actors}
+- requests: {requests}
+- named guards: {guards}
+- named actions: {actions}
+- decision events: {events}
+- model refs: {models}
+
+Objective on the training scenarios: {objective summary}
+Diagnosed cause: {hypothesis.cause}
+Evidence: {failing rows: scenario, expected, reached state, rejections, tokens}
+
+Return the complete policy: the TypeScript file, or one AgentWorkflowConfig as
+JSON. Make the smallest change that addresses the diagnosed cause. Keep "id" and
+"description". Set "version" to {n + 1}. Never match on a scenario's data.
+For a TypeScript policy, keep the result one `setupAgent` machine. Import from
+the environment only schemas, types, constants, and exported guard functions.
+Do not invoke tools the environment owns.
+```
+
+On a failed gate, append:
+
+```
+Your previous config was rejected.
+Gate '{gate.name}' failed:
+{gate.detail}
+Return a corrected config.
+```
+
+## Changelog entry
+
+```md
+## v{n} — {recommended | discarded}
+
+- Hypothesis: {cause} → {change}
+- Diff: {states or transitions touched}
+- Gates: {TS: environment ✓ tsc ✓ lint ✓ paths ✓ overfit ✓ evals ✓ | JSON: validate ✓ vocabulary ✓ lower ✓ lint ✓ explore ✓} (repairs: {r})
+- Train: objective {a} → {b}, violations {v}, cost {c0} → {c1}
+- Held-out: objective {a} → {b}, violations {v}, cost {c0} → {c1}
+- Regressions: {none | scenario ids}
+- Decision: {why}
+```
+
+## Environment change request
+
+Write this instead of editing the environment.
+
+```md
+# Environment change request: {short title}
+
+- Goal criterion blocked: {the success criterion no policy can meet}
+- Evidence: {thread ids; the state and guard that block it}
+- Policies tried: {versions and why each could not get there}
+- Proposed change: {the state, guard, or event to add or change, in words}
+- Invariants affected: {which, and the canReach check to rerun}
+- Owner: a person reviews and merges this; the improver does not.
+```

package/skills/loop-to-machine/SKILL.md

@@ -0,0 +1,183 @@
+---
+name: loop-to-machine
+description: Convert existing agent code (a while-loop around a model, a tool loop, ad-hoc if/else control flow) into an XState v6 state machine with @statelyai/agent, and prove with a parity report that it behaves like the code it replaces. Use when asked to "turn this agent into a state machine", "port my loop to xstate", "make this a statechart", "refactor this agent into a machine", or "split environment and policy" for code that already runs.
+---
+
+# Convert agent code to a machine
+
+You replace the control flow of working agent code with a machine, and show where the machine behaves like the old code and where it differs on purpose. The model calls, tools, and retry code move across unchanged.
+
+```
+inventory → pin the baseline → environment → (policy) → runner → proofs → parity → hand back
+```
+
+This is step 1 of 4. Next: `add-traces`, then `add-evals`, then `improve-machine`. To author a machine with no existing code, use `generate-machine`.
+
+## Hard rules
+
+- **Pin the baseline before you change anything.** The parity test runs against the old code, which stays in the repo, untouched, until a person deletes it.
+- **Never promise "behaves exactly like".** Promise what the parity report shows: same outcome and effects on these scenarios, with every difference labeled.
+- **Never copy a rule break.** Where the old code broke a rule, the machine blocks it. That difference is `intended`, not a regression.
+- **Rules are code, not prompt text.** A rule that lived only in a system prompt becomes a guard. Keep the prompt line only as context for the model.
+- **Business rules belong to people.** If a rule is ambiguous (which threshold, which order of checks), ask. Never pick one silently.
+
+## 1. Inventory the code
+
+Read the loop and fill this table before writing anything. Each row maps to a machine element.
+
+| In the code                                                                  | Becomes                                                                  |
+| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
+| A tool with a side effect someone must gate (refund, send)                   | An actor invoked by one state, entered only through a guarded transition |
+| A read-only tool (search, lookup)                                            | An actor the machine invokes itself, or a tool on a text request         |
+| A rule: an `if`, a prompt line, a check after the fact                       | A guard on the transition it protects                                    |
+| A point where the model picks what to do                                     | A decision state: `agent.decide`, usually a closed choice                |
+| A value the model pulled out of the input (an order id, an amount)           | Code when the format is fixed (a regex); a text request when it is not   |
+| A step every correct run takes in the same order (verify, then look up)      | A state the machine enters itself; the model no longer chooses it        |
+| A person in the loop: `await approve()`, a returned `pending`, or a tool that asks a person (`askCustomer`, `requestApproval`), even a synchronous one | An idle state with `meta.interaction`, answered by `onIdle` in the runner |
+| A cap (`turns < 8`, a retry counter) on a loop the machine keeps             | A `choice` state against an exported constant                            |
+| A cap on a loop the machine removed                                          | Nothing. Record that it is gone and why                                  |
+| Each distinct return                                                         | A final state                                                            |
+| A terminal tool with a side effect (`escalate`, `close`)                     | A state that invokes it, then the final state; its `onError` goes to a `failed` final |
+
+List the rules as a numbered list in your notes. Every one must end as a guard, a state ordering, or a written reason it was dropped.
+
+## 2. Pin the baseline
+
+The parity test exists before the machine does.
+
+1. **Scenarios.** Reuse the code's tests, fixtures, or benchmark cases. Each has an `id` and, where people know it, the `expected` outcome. If there are none, draft 6–10 that cover every terminal and every rule, and ask the user to confirm them.
+2. **Deterministic stand-in models.** Parity with a live model drifts between runs. Use the code's scripted or mock models if it has them; otherwise write one keyed on the input (see [`examples/mock-model.ts`](https://github.com/statelyai/agent/blob/main/examples/mock-model.ts) in the `statelyai/agent` repo). You want two: one that follows the rules, and one that breaks them. A seeded random model needs its seed derived from the scenario id and a fresh instance per run, or every scenario replays the same draws.
+3. **Ground truth.** Read outcomes and effects from the world, not from the agent's own report. `effects` holds **every write the world records**: the refunds, and also the approvals, the verifications, the questions asked, the escalations. A refund issued without its approval looks correct if `effects` holds only refunds. Text a model wrote (an escalation reason) differs on every run: compare its count or presence, not the text. If the world can check rules after the fact, return those violations. If it cannot, write the check as a plain function over the effects.
+4. **The harness check.** Run the old code on both sides with each stand-in model. Each row's baseline must equal its candidate (the runs are deterministic), and with the rule-following model every row must be `same`. With a rule-breaking model, rows are `regression` by design: the candidate breaks rules.
+
+```ts
+import { compareRuns, type ParityResult, type ParityRun } from "@statelyai/agent/testing";
+
+function toParityRun(run: { outcome: string; world: World }): ParityRun {
+  return {
+    outcome: run.outcome,
+    // Every write the world records; model-written text as a count.
+    effects: { refunds: run.world.refunds, approvals: run.world.approvals, verified: [...run.world.verified], escalations: run.world.escalations.length },
+    violations: run.world.violations, // rules broken, from ground truth
+  };
+}
+
+// A model factory, called once per scenario and per side. Construct a stateful or seeded
+// model inside it (seeded from s.id); returning a shared stateless object is fine.
+const runWith = (runner: typeof runLoop, model: (s: Scenario) => Model) => async (s: Scenario) =>
+  toParityRun(await runner({ ...s, model: model(s), world: createWorld(s) }));
+
+const MODELS: Array<[string, (s: Scenario) => Model]> = [
+  ["careful", () => carefulModel],
+  ["sloppy", () => sloppyModel],
+];
+
+test.each(MODELS)("harness: the loop matches itself (%s)", async (_, model) => {
+  const report = await compareRuns({ scenarios: SCENARIOS, baseline: runWith(runLoop, model), candidate: runWith(runLoop, model) });
+  for (const row of report.scenarios) expect(row.candidate, row.id).toEqual(row.baseline);
+});
+```
+
+## 3. Author the environment
+
+The environment is the machine: states, guards that encode the rules, side-effecting actors, an invariant sentinel. Use `setupAgent` (from `@statelyai/agent`) with zod schemas for `input`, `context`, `output`, and every event. Export the event schemas as one object, so a policy can reuse them later.
+
+A gated action:
+
+```ts
+deciding: {
+  tags: ["decision"],
+  invoke: {
+    src: "agent.decide",
+    input: ({ context }) => ({
+      model: "decider", // a key the host's executors resolve
+      name: "chooseResolution", // tests, traces, and a later policy find it by this name
+      prompt: `Ticket: ${context.ticket}\nOrder: ${JSON.stringify(context.order)}`,
+      allowedEvents: ["REFUND", "ESCALATE", "CLOSE"],
+      // A closed choice: the decider picks one of these. The runtime drops the ones the guards refuse.
+      payloads: { REFUND: refundAmounts(context).map((amount) => ({ amount })) },
+      // The facts a decider reads. Arrives as `request.metadata.facts`.
+      metadata: { facts: { order: context.order, requestedAmount: context.requestedAmount } },
+    }),
+    onError: { target: "escalated" }, // retries exhausted, or no option legal
+  },
+  on: {
+    REFUND: ({ context, event }) =>
+      refundIsLegal(context, event.amount)
+        ? { target: event.amount > APPROVAL_THRESHOLD ? "awaitingApproval" : "paying", context: { amount: event.amount } }
+        : undefined, // undefined rejects: the decision retries with feedback
+    ESCALATE: { target: "escalated" },
+    CLOSE: { target: "closed" },
+  },
+},
+```
+
+- **Do the deterministic steps yourself.** Identity, lookup, the ownership check: states the machine runs in order. A model only answers what genuinely needs judgment.
+- **Export the guards** as plain functions (`refundIsLegal`) so tests can call them, and build the machine in a factory (`createDeskMachine({ refundIsLegal })`) so a test can pass a loosened guard.
+- **Add an invariant sentinel** for the rule that must never break: a final state with a description starting `SENTINEL:`, entered from an `always` on the state that invokes the side effect (`paying`). An `always` is taken before the state's invoke starts, so the effect never runs. Write the check as its own function, not a call to the guard, so loosening the guard does not loosen the check.
+- **A person's step** is an idle state whose `meta.interaction` tells a host what to render: `{ label: "Approve a refund of ${amount}?", events: { APPROVE: { label: "Approve" }, REJECT: { label: "Reject" } } }`, or `textEvent: { type: "CUSTOMER_REPLIED", field: "reply" }` for free text.
+- **Actors are placeholders.** Declare each tool as `createAsyncLogic<Output, Input>({ run })` from `xstate` in `setupAgent({ actors })` (XState v6 has no `fromPromise`). Give the type parameters, or `onDone`'s `output` is untyped. Pass the real implementations at run time with `runAgent(machine, { actors })`.
+- **Final outputs carry the machine's result** (`{ outcome, orderId, amount }`), JSON only. The runner rebuilds the old return type from it and from the world (step 5).
+- **Declare `version`** on `createMachine` (`version: "1"`). Behavior lives in transition functions, which the structural hash cannot see; traces stamp this string.
+
+## 4. A policy, only if the code already has one
+
+Keep the decision in the environment as one `agent.decide`, and do not write a policy machine, unless the old code already has decision logic beyond one model call: a rule ladder, heuristics that pick a strategy, a cheap check before the model. A policy that only passes the options through to a model is boilerplate. `improve-machine` introduces a policy when the traces show a rule worth adding.
+
+If the code does have that logic, port it as a policy machine that answers `chooseResolution`. The pattern, with code that compiles, is in [policy.md](policy.md).
+
+## 5. The runner
+
+Write `runMachine` with the old function's signature and result type, so callers and the parity test change one line.
+
+- `runAgent(machine, { input, executors, actors, onIdle })`.
+- `onIdle` answers each idle state with the old code's call (`tools.askCustomer(...)`, `tools.requestApproval(...)`). Write the returned events with `as const` on `type`, or the event union does not narrow.
+- `executors.decide` adapts the code's model to the decision: the closed choice arrives as `request.options` (each `{ event }`), and the facts as `request.metadata?.facts`, typed `unknown`: parse it with the exported facts schema. Every executor slot is optional in the type; check or assert the ones you call.
+- Rebuild the old result from `result.output` and the world. A field that has no meaning anymore (a turn count with no loop) keeps its type and gets a comment saying what it now holds.
+
+## 6. Proofs that need no model
+
+From `@statelyai/agent/testing`:
+
+- `lintAgentMachine(machine)`: no errors.
+- `canReach(machine, "<sentinel>", options)` returns `reachable: false` **and** `complete: true`, for each distinct input shape (an input with an order id and one without lead to different paths; loop over them).
+- `options.payloads` lists values on both sides of every guard boundary (`0, 40, 41, 100, 101, 240, 241, 1000`). Without it, a closed choice is explored only with its own `payloads`, and `complete: true` then proves nothing about the boundaries.
+- `options.outputs` lists every output each actor can return (found and not found, owned and not, refunded and not); `options.errors` makes each actor fail once.
+- A loosened guard makes `canReach` true. Write this test too, or the proof proves nothing.
+- Each normal final state is reachable, so the search is not vacuous.
+
+Type the options with `ExplorePathsOptions` from `@statelyai/agent/testing`. `errors` takes one error per actor src, and `"agent.decide"` is a valid key.
+
+## 7. Parity
+
+Switch the parity test's candidate to `runMachine`, and run it with each stand-in model:
+
+```ts
+test.each(MODELS)("parity with the loop: %s", async (_, model) => {
+  const report = await compareRuns({ scenarios: SCENARIOS, baseline: runWith(runLoop, model), candidate: runWith(runMachine, model) });
+  const unreviewed = report.scenarios.filter((row) => row.label === "regression" && !REVIEWED[row.id]);
+  expect(unreviewed).toEqual([]);
+  // A reviewed difference still has to be exactly the difference a person accepted.
+  for (const row of report.scenarios.filter((row) => REVIEWED[row.id])) REVIEWED[row.id]!(row);
+});
+
+// Effect-only differences a person accepted: who, and the exact difference.
+const REVIEWED: Record<string, (row: ParityResult<Scenario>) => void> = {
+  // "no-order-id-no-reply": (row) => { /* accepted by <name>: asks once, not three times */ expect(row.candidate.outcome).toBe(row.baseline.outcome); ... },
+};
+```
+
+- Every `regression` row: fix the machine, or, when the old behavior was wrong but broke no rule, ask the user. If the outcomes differ, the answer may become the scenario's `expected`. If only effects differ (the loop asked three times, the machine once), list the row in a reviewed-differences list in the test, with the exact difference asserted and the person who accepted it. Never edit a scenario to make a row pass.
+- Every `intended` row: check that its reason is a real rule. A candidate that fixed a rule break with the same outcome is `intended` too.
+- Run once with a live model if a key is available. Report it as one sample, not a guarantee.
+
+## 8. Hand back
+
+Report, in this order:
+
+1. The files: environment, runner, tests (and a policy, if the code had one). The old code is untouched.
+2. The rule list from step 1, each mapped to its guard, its state, or the reason it was dropped.
+3. The parity table per stand-in model: `same / intended / regression`, and one line per `intended` row with its reason.
+4. The `canReach` results and the loosened-guard test.
+5. Anything you had to decide that the user should confirm.
+6. Next step: `add-traces`.