ravensight-playtest 0.3.1 → 0.4.0

package/README.md

@@ -4,6 +4,11 @@ AI personas play your game, then file a report you can read, mark up and argue
 with. The personas run on your machine, against your build; Ravensight stores
 the reports and renders them.
 
+Simulation Runner adds seeded headless batches for mechanics, economy and
+progression checks through your own game adapter. It runs locally without
+model calls and saves bounded results to the same Playtest workspace. See
+[Simulation Runner](docs/SIMULATION.md) for the held candidate and setup.
+
 You need a Ravensight account and Node 22 or newer. You do not need an
 Anthropic API key: model calls go through Ravensight under the same token you
 log in with, so there is one credential on your machine and you can revoke it
@@ -45,9 +50,32 @@ Credential Manager, libsecret on Linux). With no keychain available it falls
 back to a file with 0600 permissions under your home directory, and says so.
 In CI, set `RAVENSIGHT_TOKEN` instead and skip `login` entirely.
 
+## Upgrading to 0.3.2
+
+- **A plain `run` plays one persona, not two.** The default persona set is now
+  `curious-kid` alone, so a first run costs one persona run. Name more with
+  `--personas curious-kid,grumpy-veteran` (or `personas` in
+  `.ravensight/config.json`); `ravensight-playtest personas` lists the slugs.
+  A script that relied on two personas by default has to name them.
+- **`run` checks the build before it prices anything.** A web build's URL has
+  to answer 2xx or 3xx, or the `preflight` URL in `.ravensight/config.json` has
+  to answer what you set (see `run`). A failure exits 1 with no job registered.
+  `--skip-preflight` skips it, as it already skipped the native probe.
+- **`brief` and `check` quote the run that `run` would start**: its modules and
+  its personas, the default one when you name none, each unit with its own
+  price. They used to quote a game profile with no personas.
+- **Two new brief fields stay on this machine**: `expected_noise` and
+  `test_account` (see the `brief --from` file). `init` adds
+  `.ravensight/brief.local.json` to `.gitignore`.
+- **Out of range report values are normalised before upload** instead of
+  failing the run (see how the report step reads what happened, under `run`).
+- **A CSS selector is no longer a target.** A target is a ref, coordinates, a
+  line copied from the screen, or text on the screen; anything else is refused
+  at once instead of searched for.
+
 ## Commands
 
-Twelve commands. Flags below are the ones the code accepts, not the ones the
+Thirteen commands. Flags below are the ones the code accepts, not the ones the
 prose elsewhere describes; `--help` on any command prints the same list.
 
 Every command takes these four:
@@ -89,9 +117,26 @@ to), `project binding` (the game and environment this directory is linked to,
 and whether that game is in the token's scope), `brief` (present, complete, and
 which of goals, success criteria, controls and how to start are empty), `build`
 (the URL, Godot project or native build a run would use), `automation server`
-(native drivers only, see below), then the balance against the next job's price,
-the content pack verified by sha256, disk, recommended concurrency, and the two
-optional tools.
+(native drivers only, see below), then the content pack verified by sha256, the
+balance against the price of the run `run` would start (its modules and
+personas, the default persona when none is named, each unit with its price),
+disk, recommended concurrency, and the two optional tools. A `--personas` slug
+the pack does not have fails a `personas` row that lists the valid slugs, and
+nothing is priced.
+
+For a web build linked by URL (`--build-url` is a `run` flag; `check` reads the
+URL from `.ravensight/config.json`), the `browser` row also loads the first
+screen headless and a `first screen` row measures it the way a persona would
+read it. When the whole text is over the 4,000 characters a persona reads per
+step, or the screen has a list of 50 or more look-alike items, the row is a
+warning that suggests `run --long` with the reason ("first screen is 18,400
+characters, about 4.6x the 4,000 a persona reads per step") and both prices,
+read from the server's free estimate. A light screen passes with its size. A
+first screen that does not load within 20 seconds is a warning, never a
+failure. The page is marked as a playtest before the game's scripts run (the
+same `window.__RAVENSIGHT_PLAYTEST__` a run sets, with the token `check`), so a
+build shipping the Ravensight web SDK has its session refused rather than
+recorded as a real player.
 
 Every row is `pass`, `warn` or `fail`, and every row that is not `pass` prints
 its fix. Exit code 3 when any row fails; a warning never fails, so CI does not
@@ -118,7 +163,7 @@ setup with the expected rows at each step.
   "ok": true,
   "reserves_units": false,
   "note": "check reserves no units and starts no job: ...",
-  "cli": { "installed": "0.3.1", "latest": "0.3.1", "min_supported": "0.1.0" },
+  "cli": { "installed": "0.3.2", "latest": "0.3.2", "min_supported": "0.1.0" },
   "login": { "present": true, "valid": true, "games": [{ "gameId": "g1", "name": "Pocket Knight" }] },
   "project": { "game": "g1", "environment": null, "in_scope": true },
   "brief": { "present": true, "complete": true, "missing": [] },
@@ -156,8 +201,8 @@ automatically. `check` shows that same warning as a row when it applies.
 | Flag | Effect |
 |---|---|
 | `--game <gameId>` | Check against this game instead of the linked one. |
-| `--modules <list>` | Price this module list, comma separated. |
-| `--personas <list>` | Price this persona list. Without it the balance row prices no personas, so the quote is lower than a real job's. |
+| `--modules <list>` | Price this module list, comma separated. Default the modules `run` registers, `persona_playtest,aggregate_report`. |
+| `--personas <list>` | Price this persona list. Without it, a module list with `persona_playtest` prices the repo config's personas or the default one, as `run` would. |
 | `--godot-path <path>` | Where Godot is, if it is not on the PATH. |
 | `--web` | Check the browser even when this repo is not linked to a web build. |
 | `--driver <name>` | Check the setup for this driver instead of the configured one. |
@@ -167,7 +212,8 @@ automatically. `check` shows that same warning as a row when it applies.
 
 Links this repo to a game and writes `.ravensight/config.json`. Refuses a game
 your token cannot see, so the first job does not fail with a 404. Adds
-`.ravensight/jobs/` to an existing `.gitignore`.
+`.ravensight/jobs/` and `.ravensight/brief.local.json` to an existing
+`.gitignore`.
 
 Without `--game`, one visible game is linked as is. With several, `init` lists
 them numbered and asks `Which game? [1-N]` when it is running in a terminal; in
@@ -180,11 +226,18 @@ login).
 | `--game <gameId>` | The game to link. Optional with one visible game, or in a terminal. |
 | `--force` | Relink a repo that is already linked. |
 
+### personas
+
+Lists the persona slugs `--personas` accepts, from the content pack this game
+gets: each one's name, action budget (its patience) and play style, with the
+default marked. A free read. Takes `--game <gameId>` and `--json`.
+
 ### brief
 
 Writes the expectations brief: your goals, what success looks like, how the game
-is controlled, where a persona starts. Then prints what a job would cost,
-whether your balance covers it, and, past 8 personas, the same wall-clock
+is controlled, where a persona starts. Then prints what the run `run` would
+start costs ("A run of 1 persona (curious-kid) would cost $1.00: a persona run
+$1.00 each"; the game profile is a separate job), whether your balance covers it, and, past 8 personas, the same wall-clock
 warning `run` shows. A brief is required before `run`; the server answers 409
 `brief_required` without one.
 
@@ -195,7 +248,7 @@ warning `run` shows. A brief is required before `run`; the server answers 409
 | `--draft` | Save as a draft even when it looks complete. |
 | `--from <file>` | Read the fields from a JSON file instead of asking. |
 | `--game <gameId>` | The game, when this repo is not linked. |
-| `--modules <list>`, `--personas <list>` | Price this module and persona list in the estimate it prints. |
+| `--modules <list>`, `--personas <list>` | Price this module and persona list in the estimate it prints. Every unit is named with its own price, such as "profile job $4.00, a persona run $1.00 each". |
 
 #### The `brief --from` file
 
@@ -211,13 +264,43 @@ are what makes a brief complete; the rest are optional.
 | `build_notes.how_to_start` | string | Where a persona begins: the URL, the menu path, the first thing to press. |
 | `design_intent_notes` | array of strings | Deliberate choices a persona must not report as bugs. |
 | `known_issues` | array of strings | Issues you already know about, so they are not reported again. |
+| `expected_noise` | array of strings | Console errors and failed requests that are expected, such as `"401 /auth/me before login"`. Never a finding. Kept on this machine. |
+| `test_account` | `{ "email", "password", "how_to_sign_in" }` | A throwaway account the persona signs in with, so a run starts past sign-up. The password must be at least 8 characters. Kept on this machine. |
+
+`expected_noise` and `test_account` never go to the server: its brief schema
+refuses keys it does not list, and a password does not belong in a record the
+studio shares. `brief --from` splits them into `.ravensight/brief.local.json`,
+readable by you only, which `init` adds to `.gitignore`; `brief --show` prints
+them with the password masked, and `run` and `resume` read them from there.
+
+An `expected_noise` pattern matches an automatic console note when every status
+code (`401`) and every path (`/auth/me`) in it appears in the note; the other
+words describe it for the reader. A pattern with neither is matched as text,
+and `/.../` is a regular expression. One rule is built in: a 401 on a who am I
+check, a request whose last path segment is `/me`, `/whoami`, `/session` or
+`/current_user` (such as `/auth/me`), is what a signed out page does and is not
+a finding. A 401 anywhere else (`/me/inventory`, `/user/42/save`, `/token`,
+`/login`) is left to the report step. A matching note is marked "expected
+noise" in the report prompt, and the runner drops a finding whose evidence is
+only expected noise, listing it in `report.md`. A note the persona wrote itself
+is never noise.
+
+**The test account password crosses the model proxy during play only**: it is
+in the persona's prompt on every play turn, because that is how a persona can
+type it. Use an account made for testing, with nothing behind it you would mind
+losing. The report step never sees it: its prompt has no test account, and the
+password is replaced by `[test password]`, wherever it stands as a whole token, in the notes, the action log, the
+transcript, and in `report.md` and `report.json` before anything is uploaded. A
+password field's value is never shown in an observation.
 
 ```json
 {
   "goals": [{ "goal": "onboarding", "priority": 1 }, { "goal": "find_bugs", "priority": 2 }],
   "success_criteria": "A new player reaches level 2\nNobody gets stuck in the menu",
   "build_notes": { "controls": "Arrow keys to move, space to jump", "how_to_start": "Press Start on the title screen" },
-  "design_intent_notes": ["The fog on level 1 is intentional"]
+  "design_intent_notes": ["The fog on level 1 is intentional"],
+  "expected_noise": ["401 /auth/me before login"],
+  "test_account": { "email": "qa@example.test", "password": "a-throwaway-password", "how_to_sign_in": "Log in from the top right" }
 }
 ```
 
@@ -230,7 +313,36 @@ run options below.
 ### run
 
 Registers a job, plays it, writes the reports, uploads them, finishes the job.
-Prints the price and asks before spending.
+Prints the price and asks before spending. Before any charge, even with
+`--yes`, it prints one bold line with the persona count and the total ("This
+run: 1 persona (curious-kid) on playwright_web, $1.00 in total: a persona run
+$1.00 each"), the balance, and what a persona run buys: the action budget (the
+persona's patience in the pack, lowered by `--max-actions` or the server's
+ceiling) and that sign-up alone can take 8 of those actions. A `test_account`
+in the brief lets the persona sign in instead. An unknown `--personas` slug is
+refused with the valid slugs before anything is priced.
+
+`--long` makes every persona run a long run: twice the actions, model budget
+and wall time of a standard run, for a game whose screens are heavy or whose
+goal is far in. It costs twice a standard run. The price is the server's
+estimate for the long run, printed and confirmed like any other, never a figure
+built into the CLI. Locally a long run uses the run's own action limit when the
+server's register answer carries one, otherwise twice the standard ceilings
+(the persona's patience and the per run ceiling); `--max-actions` still only
+lowers it. `check` suggests `--long` when the first screen is heavy.
+
+Before the estimate, `run` checks the build is up, so a down server is never
+paid for. Set `preflight` in `.ravensight/config.json` to name what to check:
+
+```json
+{ "preflight": { "url": "/api/health", "expect_status": 200 } }
+```
+
+`url` may be relative to the build URL; `expect_status` is a number or a list
+of numbers, and without it any 2xx or 3xx passes (redirects are not followed).
+With no `preflight`, a web build's own URL has to answer 2xx or 3xx; other
+drivers are not checked unless a `preflight.url` is set. A failure prints the
+status it saw (or the network error), exits 1, and registers nothing.
 
 With a native driver, `run` first runs the `build` and `automation server` rows
 of `check`. When either fails it exits 1 before the estimate, the confirmation
@@ -252,14 +364,15 @@ splitting the job in two is what gets everything played.
 | Flag | Effect |
 |---|---|
 | `--game <gameId>` | The game, when this repo is not linked. |
-| `--personas <list>` | Comma separated persona slugs. Defaults to the pack's set. |
+| `--personas <list>` | Comma separated persona slugs. Defaults to one, `curious-kid`; `ravensight-playtest personas` lists them. |
 | `--driver <name>` | `playwright_web`, `godot_driver`, `cli_stdio`, or a native driver: `desktop_driver`, `unity_driver`, `unreal_driver`, `android_driver`, `ios_driver`. |
 | `--build-url <url>` | The web build to play. |
 | `--godot-project <dir>` | The Godot project directory. |
 | `--driver-config <file>` | The native driver config, or the JSON-RPC bridge config for `cli_stdio`. |
-| `--skip-preflight` | Native drivers only: register without running the free automation probe first. The job is registered and charged even if the automation server turns out to be down. |
+| `--skip-preflight` | Register without the free checks first: the build health check (any driver) and the automation probe (native drivers). The job is registered and charged even if the build or the automation server turns out to be down. |
 | `--cli <command>` | The command that starts a text build. |
 | `--max-actions <n>` | Lower the actions one persona may take. It can only lower the persona's own patience (and the server's per run ceiling), never raise it. |
+| `--long` | A long persona run: twice the actions, model budget and wall time of a standard run, at twice the price. The price is read from the estimate and confirmed before any charge. |
 | `--concurrency <n>` | Personas at once. Default 2; `check` recommends a number for this machine. |
 | `--upload-video` | Upload `session.webm` as well as the reports. |
 | `--upload-transcript` | Upload `transcript.jsonl` as well as the reports. |
@@ -304,6 +417,30 @@ message, the run is completed as `failed`, and the job exits 1.
 
 How the report step reads what happened:
 
+- Values the report schema does not allow are normalised before upload, so a
+  run is not lost to them when the model has no budget left for a repair: a
+  severity that is a category word (`"severity": "ux"`) moves to `category`
+  when the category is missing or not allowed and the severity becomes
+  `minor`; any other unknown severity becomes `minor`; an unknown category
+  takes a close match the schema has (`ui` is `ux`, `crash` is `bug`), else
+  `other` when the schema has it, else `ux` (the server's schema has no
+  catch-all today); a value that differs only in case is respelled; any other
+  enum field takes the schema's default. Each change is a runner note at the
+  end of the finding's description and is listed in `report.md` under
+  "Normalised by the runner". The server's schema stays strict.
+- After the report is written, each blocker or major (at most 3) is checked
+  against its own evidence: the screenshot it cites taken nearest after the
+  observation it cites is sent with its claim, and the question "does this
+  image contradict the claim: yes, no, unclear". A yes downgrades it one step,
+  with a note naming the screenshot, in both report files. The cost is 3
+  calls at most, each one screenshot in and 32 tokens out, drawn from the
+  run's own model budget (the one the 80 percent warning keeps room in for
+  the report), never charged separately; a refusal ends the checks and keeps
+  the report as written.
+- Console notes that match the brief's `expected_noise`, or the built in rule
+  (a 401 on a who am I check while signed out), are expected noise
+  and never a finding; see the `brief --from` file.
+
 - Console errors are recorded as automatic notes. A 4xx on a request (`Failed
   to load resource`, `HTTP 404`, `status: 401`, `401 Unauthorized` and similar)
   is a network note: context, not a bug. On the web a `Failed to load
@@ -369,11 +506,26 @@ clicks and types by ref, which names exactly one element even when two share
 a name; two such elements are both shown, each with where it is ("in header
 strip", "in main"). A name that matches twice is answered with the refs to
 choose from, and a target written as locator code (`getByRole(...)`,
-`page.locator(...)`, `css=`, `xpath=`) is refused at once. Failures like these
+`page.locator(...)`, `css=`, `xpath=`) is refused at once. A whole line copied
+from the screen (`button "Run the week" [e12] @(1000,303)`, or any part of one)
+is clicked by its ref, else its `@(x,y)` (refused like a stale ref when the page
+has loaded a new document since); a ref or point inside a quoted name is part
+of the name, and a bare `e4` is text when something on screen is named exactly
+that. A target that is
+none of a ref, coordinates or text on the screen is refused at once rather than
+searched for. A name that matches one element on screen is clicked by that
+element's ref, and a text search skips hidden elements, so a mobile only copy
+hidden by CSS is never the one clicked. Failures like these
 are the driver's, not the game's: they are recorded as `driver_error`, and a
 blocker or major resting only on them is dropped before upload, as is any
 finding resting only on the persona's "waiting for X to finish" notes.
 
+An action counts as changing the screen when the URL changed, the page loaded
+a new document, the visible tree (field values included) changed, or for typing,
+the field's own value changed, with one short second look for a navigation
+still starting. That is the "screen changed" the report step's action log and
+the downgrade rule read.
+
 Every field, checkbox, radio, slider and select the persona sees carries what
 it holds now, read from the page rather than from the accessibility text:
 `value="..."` (cut at 80 characters), `checked` or `not checked`, or
@@ -382,7 +534,9 @@ the page could not answer for shows no value at all, so a typed password never
 reaches the model.
 
 On a long web page the persona's text view of the screen is capped at 4,000
-characters. Over the cap, buttons, links and fields are kept first, a long run
+characters. A run of 20 or more look-alike items is always shown as its first
+two and "and N more like it", under the cap too; a shorter run under the cap is
+shown as it is. Over the cap, buttons, links and fields are kept first, a long run
 of look-alike items (a list of 249 cards) is shown as its first two and "and
 247 more like it", and what changed since the last look comes next, so a form
 after a long list still reaches the persona.
@@ -394,7 +548,8 @@ reached. Every run gets its own identity to sign up with, an email
 short run id, so a second run never hits "that email is taken" from the first.
 `example.test` is a reserved name and never delivers mail. Point `--build-url` at a guest or demo path, or say where to start in
 the brief's `how_to_start` note (for example "choose Play as guest"), when the
-sign-up is not what you want tested.
+sign-up is not what you want tested, or give the brief a `test_account` the
+persona signs in with instead.
 
 ### fix
 
@@ -586,6 +741,7 @@ In your repo:
 ```
 .ravensight/
   config.json                 the game link and your defaults
+  brief.local.json            expected_noise and test_account, this machine only (0600)
   jobs/<jobId>/
     state.json                the resumable journal
     upload-queue.jsonl        what was uploaded, and what is still owed
@@ -602,8 +758,8 @@ and a local `transcript.jsonl` of the apply loop (which tool, on what, how
 much came back) that is never uploaded and is how an attempt that wrote
 nothing gets explained afterwards.
 
-Add `.ravensight/jobs/` to your `.gitignore`; `init` does it for you when you
-already have one.
+Add `.ravensight/jobs/` and `.ravensight/brief.local.json` to your
+`.gitignore`; `init` does it for you when you already have one.
 
 `fix` also leaves a branch on your working copy, checked out, with the change in
 your working tree and nothing committed. It is the only thing in this CLI that
@@ -633,6 +789,16 @@ pins it equal to the server's file when a checkout is on the same machine.
 Findings cite a file and a line range. The source text behind that line is not
 uploaded.
 
+`.ravensight/brief.local.json` is never uploaded. Its `expected_noise` goes into
+the persona and report prompts. Its `test_account`, password included, goes into
+the persona's prompt during play only, through the model proxy; the report step
+never gets it, and the password is scrubbed to `[test password]` from the
+transcript, the notes and both report files before upload.
+
+`run`'s build health check is one GET to your own build (or the `preflight`
+URL you set), made from your machine.
+
+
 A `fix` uploads `changeset.json` and `report.md`. The changeset carries the
 finding id, the branch name, the commit it branched from, the file paths it
 changed with their added and removed line counts, the verify verdict, and the
@@ -723,3 +889,17 @@ feedback and timing-sensitive gameplay.
 ## Build adapters
 
 Browser and Godot runners share the paid playtest lifecycle with desktop, Unity/Unreal export, Android/iOS and custom JSON-RPC adapters. Native adapters require a locally configured automation server and remain experimental until validated against your build. See [adapter setup and validation boundaries](docs/ADAPTERS.md) and the [native runner starter](docs/NATIVE_RUNNER.md); `check` probes the automation server for free before a paid native run.
+
+## Simulation Runner
+
+Available in held local candidate `0.4.0`; not yet published. Simulation Runner requires macOS or Linux (including WSL); native Windows is refused before purchase.
+
+Run seeded mechanics tests through a local game adapter, compare explicit strategies and replay selected traces. A batch uses the existing standard Playtest purchase of 5 AI units from prepaid balance, confirmed against the server quote. Only the summary is uploaded; game execution and replay traces stay local. It does not automatically call a model or replace testing the player interface.
+
+```sh
+ravensight-playtest simulate --config simulation.json --driver-config bridge.json
+ravensight-playtest simulate --resume .ravensight/simulations/BATCH_ID/state.json
+ravensight-playtest simulation-replay trace-00000.json --driver-config bridge.json
+```
+
+See [Simulation Runner setup, adapter protocol and limits](docs/SIMULATION.md). The package includes `examples/simulation/economy.mjs` as a deterministic starting adapter. Local replay creates no new paid job.

package/docs/NATIVE_RUNNER.md

@@ -60,7 +60,7 @@ Every device command times out after 30 seconds and responses are capped at
 ## Step 1: the account side
 
 ```bash
-npm install -g ravensight-playtest@0.3.1
+npm install -g ravensight-playtest@0.3.2
 ravensight-playtest login
 cd path/to/your/game
 ravensight-playtest init --game <gameId>
@@ -70,7 +70,7 @@ ravensight-playtest check --driver ios_driver
 Expected rows (your email, game and version differ):
 
 ```
-pass  cli version        installed 0.3.1; server latest 0.3.1, min_supported 0.1.0
+pass  cli version        installed 0.3.2; server latest 0.3.2, min_supported 0.1.0
 pass  login              you@example.com, valid, scoped to 1 game: Pocket Knight (g1)
 pass  project binding    linked to Pocket Knight (g1), in the token's scope
 fail  brief              this game has no expectations brief, and a persona run needs one

package/docs/SIMULATION.md

@@ -0,0 +1,91 @@
+# Simulation Runner
+
+Held local candidate: `0.4.0`. This package has not been published; use a reviewed local tarball with the matching held server implementation.
+
+Simulation Runner exercises game mechanics through a developer-owned local adapter. It can run seeded games across explicit strategies, summarize outcomes and metrics, and save selected replay traces. Use it for questions such as whether an economy supports several viable strategies or whether a progression rule traps a simulated player. It does not judge animation, interface usability, feel or subjective fun.
+
+A batch uses the existing standard Playtest purchase: **5 AI units from prepaid balance**. The CLI obtains the current server quote and asks for confirmation before registration. One batch is one purchase, not one purchase per simulated game. It does not consume the monthly shared AI allowance. Local replay is a debugging operation with no new job or purchase. Automatic AI interpretation is not included; an agent can use the existing report tools when explicitly asked to interpret the summary.
+
+## Setup and run
+
+Run Simulation Runner on macOS or Linux, including WSL. Native Windows is rejected during preflight before quoting or registering a purchase because this feature requires POSIX process-group cleanup. This restriction applies to Simulation Runner, not every Playtest adapter. The recorded real-process and 10,000-game validation was run on macOS; native Windows and WSL execution were not validated in this review.
+
+Sign in and initialize your Ravensight project as usual. Implement the protocol below, or start with the packaged `examples/simulation/economy.mjs`. The example is deterministic and contains no networking or telemetry.
+
+Create `bridge.json` with absolute paths for the Node executable and adapter:
+
+```json
+{
+  "command": "/absolute/path/to/node",
+  "args": ["/absolute/path/to/economy.mjs"],
+  "cwd": "/absolute/path/to/game"
+}
+```
+
+The adapter is trusted local code, **not a sandbox**. No shell is used to spawn it. Account/provider credential environment variables are stripped. Paid execution injects a synthetic Playtest token and run/job IDs; preflight and replay inject an invalid token sentinel so an inherited token cannot accidentally mark the session as production.
+
+Create `simulation.json`:
+
+```json
+{
+  "version": 1,
+  "seed": "economy-review-1",
+  "runs": 100,
+  "max_steps": 12,
+  "workers": 4,
+  "strategies": [
+    { "name": "random", "policy": "random" },
+    { "name": "collector", "policy": "scored" },
+    { "name": "saver", "policy": "scripted", "actions": ["earn", "earn", "earn", "earn", "earn", "earn", "earn", "earn"] }
+  ]
+}
+```
+
+```sh
+ravensight-playtest simulate --config simulation.json --driver-config bridge.json
+```
+
+A `driver_config` path inside the simulation file may replace `--driver-config`. The CLI probes the adapter before purchasing. A probe only starts a game and validates its first observation; it does not run the batch. Keep adapter initialization free of destructive side effects.
+
+Strategy names start with a lowercase letter and contain at most 64 lowercase letters, digits, underscores or hyphens. Strategies are assigned round-robin. Each run receives a seed derived from the root seed and its zero-based run index. Worker count changes concurrency, not action selection. `random` chooses among legal action IDs deterministically. `scripted` follows supplied IDs and fails on an illegal action or exhaustion before a terminal state. `scored` chooses the highest score supplied by the game for that strategy name, with a deterministic action-ID tie break. Names such as "aggressive" have no built-in gameplay meaning.
+
+## Adapter protocol
+
+Use one JSON-RPC 2.0 JSON object per line on stdout. Send diagnostics to stderr. The runner starts a fresh subprocess for each game, with one request outstanding at a time.
+
+```json
+{"jsonrpc":"2.0","id":1,"method":"simulation.start","params":{"version":1,"seed":"derived-seed","strategy":"collector","max_steps":12}}
+{"jsonrpc":"2.0","id":1,"result":{"done":false,"state_hash":"state-0","actions":[{"id":"earn","scores":{"collector":1}},{"id":"buy","scores":{"collector":3}}],"metrics":{"coins":4}}}
+{"jsonrpc":"2.0","id":2,"method":"simulation.step","params":{"action":"buy"}}
+{"jsonrpc":"2.0","id":2,"result":{"done":true,"state_hash":"state-1","actions":[],"metrics":{"coins":2,"inventory":1},"outcome":"collected"}}
+```
+
+The game owns state hashing and must include every state component relevant to reproducibility. Return `done` as a boolean, a nonempty `state_hash` of at most 128 characters, and at most 256 unique action IDs of at most 128 characters. Nonterminal observations need legal actions. Optional action scores contain at most 16 finite numeric values keyed by strategy. Optional metrics contain at most 32 finite numeric values between -1e15 and 1e15 with keys of at most 80 characters. Optional outcomes are nonempty strings of at most 80 characters. Reserved names `__proto__`, `constructor` and `prototype` are rejected.
+
+## Limits and results
+
+- 1 to 10,000 requested games, 1 to 10,000 actions per game, 1 to 16 workers and 1 to 16 strategies.
+- At most 1,000,000 actions across a batch and at most 15 minutes of execution, subject to the registered server budget.
+- At most 30 seconds per response and 256 KiB per response. Cancellation terminates the subprocess group, escalating from SIGTERM to SIGKILL.
+- At most ten selected traces: the first ten run indices, not an exhaustive failure archive. Each trace is bounded to 4 MiB and 10,000 actions.
+- Per-strategy summaries retain at most 32 distinct metric names and 128 outcome buckets. Additional outcome names are grouped as `other`; keep metric names stable between games.
+
+`duration_ms` reports execution time within the batch deadline, excluding any cleanup or post-stop serialization beyond that deadline. `results.jsonl` streams one local result per requested run. `summary.json` contains completed, failed and truncated totals, total actions, terminal outcome counts and terminal metric count/min/max/mean by strategy. Metrics are run-level terminal values, not a mean across every observation. A terminal result without an outcome uses `unspecified`. Reaching `max_steps` is **truncated**, never a successful terminal outcome. Games not started because of cancellation or a batch limit count as failed, with the stop reason retained in the local report.
+
+Only the bounded summary is uploaded to the existing Playtest job. Adapter scripts, raw states, per-game rows, replay traces and scripted action lists remain local. The public configuration hash covers the registered configuration and strategy names/policies; trace metadata also includes a separate full local configuration hash covering scripts.
+
+## Recovery and replay
+
+The CLI prints a durable recovery journal path. Recover the same purchase rather than starting another batch:
+
+```sh
+ravensight-playtest simulate --resume .ravensight/simulations/BATCH_ID/state.json
+```
+
+Resume preserves the API host, game, configuration and purchase identity. Do not combine it with a new config or bridge argument. A changed bridge configuration is rejected. Keep the adapter build and its dependencies unchanged for reproducible recovery. After a process interruption, any restarted local attempt has its own output directory. If the summary was already saved, resume uploads that summary without executing the games again. A terminal server job is not restarted.
+
+```sh
+ravensight-playtest simulation-replay /absolute/path/to/trace-00000.json --driver-config bridge.json
+```
+
+Replay checks each recorded state hash, terminal flag, outcome and metrics and reports the first divergence. A matching replay verifies that recording with the current adapter; it does not prove the game is deterministic. External services, clocks, platform differences and ignored seeds can cause divergence. A trace from a max-step-truncated game only covers the recorded window, not a finished game. Size-capped or failed recordings can verify their captured prefix, reported as `matched_prefix: true` and `fully_replayed: false`. A recorded failed protocol action is retried after that prefix; `failure_reproduced` states whether the same sanitized protocol error recurred. A failed initial start has no state prefix to verify. Policy failures such as an exhausted script only support prefix replay, not recreation of the policy failure. These results never claim a fully reproduced terminal game.

package/examples/simulation/economy.mjs

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+// Trusted local adapter example. It has no networking or telemetry.
+import readline from 'node:readline';
+import { createHash } from 'node:crypto';
+let seed, turn, coins, inventory;
+function observe() {
+  const done = turn >= 8;
+  return { done, state_hash: createHash('sha256').update(JSON.stringify([seed, turn, coins, inventory])).digest('hex'),
+    actions: done ? [] : [{ id: 'earn', scores: { saver: 2, collector: 1 } }, ...(coins >= 2 ? [{ id: 'buy', scores: { saver: 0, collector: 3 } }] : [])],
+    metrics: { coins, inventory }, ...(done ? { outcome: inventory >= 2 ? 'collected' : 'saved' } : {}) };
+}
+for await (const line of readline.createInterface({ input: process.stdin })) {
+  let request;
+  try {
+    request = JSON.parse(line);
+    if (request.method === 'simulation.start') { seed = request.params.seed; turn = 0; inventory = 0; coins = parseInt(createHash('sha256').update(seed).digest('hex').slice(0, 4), 16) % 3; }
+    else if (request.method === 'simulation.step' && !observe().done && observe().actions.some(a => a.id === request.params.action)) {
+      turn++; if (request.params.action === 'earn') coins++; else { coins -= 2; inventory++; }
+    } else throw new Error('Illegal simulation request');
+    process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result: observe() }) + '\n');
+  } catch { process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request?.id, error: { code: -32602, message: 'Invalid request' } }) + '\n'); }
+}

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "ravensight-playtest",
-  "version": "0.3.1",
-  "description": "AI personas play your game and file reports to Ravensight.",
+  "version": "0.4.0",
+  "description": "Local AI playtests and seeded game simulations with reports in Ravensight.",
   "license": "MIT",
   "homepage": "https://ravensight.io/docs/#playtest",
   "bugs": {
@@ -12,6 +12,8 @@
     "playtest",
     "game-analytics",
     "ai-personas",
+    "simulation",
+    "game-balance",
     "ravensight"
   ],
   "type": "module",
@@ -27,6 +29,8 @@
     "addons",
     "docs/ADAPTERS.md",
     "docs/NATIVE_RUNNER.md",
+    "docs/SIMULATION.md",
+    "examples/simulation",
     "README.md",
     "LICENSE",
     "!src/**/*.test.js",
@@ -53,6 +57,6 @@
   },
   "optionalDependencies": {
     "keytar": "^7.9.0",
-    "playwright": "^1.49.0"
+    "playwright": "^1.51.0"
   }
 }

package/src/api/client.js

@@ -189,9 +189,15 @@ export function createClient(options = {}) {
   };
 
   const jobs = {
-    estimate: (gameId, { modules = [], personas = [] } = {}) =>
-      request('POST', `${gameBase(gameId)}/estimate`, { body: { modules, personas } })
-        .then(answer => answer.body),
+    /**
+     * Price a job without charging. `runLength: 'long'` asks for the long
+     * persona run's price; anything else leaves `run_length` out, which the
+     * server reads as a standard run.
+     */
+    estimate: (gameId, { modules = [], personas = [], runLength = null, simulation } = {}) =>
+      request('POST', `${gameBase(gameId)}/estimate`, {
+        body: { modules, personas, ...(runLength === 'long' ? { run_length: 'long' } : {}), ...(simulation ? { simulation } : {}) }
+      }).then(answer => answer.body),
 
     /**
      * Register a job: this is the request that moves money.