ravensight-playtest 0.3.2 → 0.5.0

package/README.md

@@ -4,6 +4,11 @@ AI personas play your game, then file a report you can read, mark up and argue
 with. The personas run on your machine, against your build; Ravensight stores
 the reports and renders them.
 
+Simulation Runner adds seeded headless batches for mechanics, economy and
+progression checks through your own game adapter. It runs locally without
+model calls and saves bounded results to the same Playtest workspace. See
+[Simulation Runner](docs/SIMULATION.md) for setup.
+
 You need a Ravensight account and Node 22 or newer. You do not need an
 Anthropic API key: model calls go through Ravensight under the same token you
 log in with, so there is one credential on your machine and you can revoke it
@@ -369,7 +374,10 @@ splitting the job in two is what gets everything played.
 | `--max-actions <n>` | Lower the actions one persona may take. It can only lower the persona's own patience (and the server's per run ceiling), never raise it. |
 | `--long` | A long persona run: twice the actions, model budget and wall time of a standard run, at twice the price. The price is read from the estimate and confirmed before any charge. |
 | `--concurrency <n>` | Personas at once. Default 2; `check` recommends a number for this machine. |
-| `--upload-video` | Upload `session.webm` as well as the reports. |
+| `--controller <name>` | One playthrough using `standard` or an available System One controller. The server must advertise and quote that exact choice. |
+| `--compare-with jev` | Separate agent/skill and Jev playthroughs for each chosen persona. Shows their combined quote before purchase. Comparisons run one playthrough at a time. |
+| `--record-video` | Capture the selected web or graphical Godot game locally. Does not upload the video. |
+| `--upload-video` | Record and explicitly upload `session.webm` or `session.mp4` with the reports. |
 | `--upload-transcript` | Upload `transcript.jsonl` as well as the reports. |
 | `--dry-upload` | List every file and byte that would be sent, and send nothing. |
 | `--key-modifier <meta\|control>` | The modifier the browser driver uses for editing shortcuts such as select all. Defaults to this machine's own (Command on macOS, Control elsewhere); set it when the browser's platform is not this machine's. `RAVENSIGHT_KEY_MODIFIER` does the same. |
@@ -775,7 +783,7 @@ to it. The short version:
 
 Uploaded by default: `report.md`, `report.json`, `usage.json`, `changeset.json`,
 the aggregate and capability reports, and `screenshots/`. Uploaded only when you ask:
-`transcript.jsonl` (`--upload-transcript`) and `session.webm`
+`transcript.jsonl` (`--upload-transcript`) and `session.webm` or `session.mp4`
 (`--upload-video`). **Source code never, at any path.** The upload allowlist has
 no entry for it, the CLI refuses it locally, and the server refuses anything not
 on its own copy of the list. `src/upload/allowlist.js` is that copy, and a test
@@ -876,6 +884,59 @@ before registering a paid job. No provider key belongs in the CLI. Selection cos
 share the existing persona-run budget and do not create another wallet charge.
 Resume preserves the controller recorded in the job journal.
 
+### Recorded playthroughs and comparisons
+
+The `0.5.0` candidate adds local recording and explicit comparisons. It requires a
+matching server release for controller plans; an older server is refused before
+purchase. Rune is recognized as a future controller but is unavailable until
+provider access, pricing and acceptance are complete. These commands do not
+activate graphical AWS hosting.
+
+To keep one playthrough's video on your computer:
+
+```sh
+ravensight-playtest run --driver playwright_web --build-url https://your-game.example/ --record-video
+```
+
+To request two reports, one from the agent/skill controller and one from Jev:
+
+```sh
+ravensight-playtest run --driver playwright_web --build-url https://your-game.example/ --personas curious-kid --compare-with jev --record-video
+```
+
+The server quotes every persona/controller run before the CLI asks for payment.
+Each keeps its own existing model budget and settlement rules. Two standard runs
+at the current $1 unit price total $2; the displayed server quote is authoritative.
+No comparison model call or extra recording fee is added. Use `--upload-video`
+instead of `--record-video` only when you want the videos uploaded for private
+dashboard playback. Recording is silent and captures only the selected game,
+not your desktop. Browser popups and navigation to another origin are blocked
+while recording.
+
+Web recording produces WebM. Graphical Godot recording needs an existing FFmpeg
+installation and produces MP4; a headless Godot run or missing encoder is reported
+as unsupported. Capture is bounded to 15 minutes, 200 MiB of video, 1280 by 720
+pixels and 10 frames per second for the Godot encoder. Godot raw frames have a
+separate 256 MiB bound. This is evidence capture, not a promise of smooth video
+at the game's original frame rate. Cancellation, crashes and limits can leave a
+playable partial video; local `recording.json` records that outcome. Neither that
+file nor raw frames are uploaded as artifacts.
+
+With `--upload-video`, completion also sends a bounded capture status, duration
+and stop reason. This is the runner's account of capture, separate from verified
+upload or report success. Without video-upload consent, no capture status is sent.
+Older recordings and later byte-only uploads can show status as unreported.
+Malformed local status is omitted so it cannot prevent an existing report from
+being completed.
+
+Each controller starts a separate game session and produces its own report.
+Saved progress and random events may still differ; this is not a guaranteed
+identical-state benchmark. The aggregate compares evidence-bearing findings and
+marks missing, failed or partial playthroughs as inconclusive. Agreement between
+reports is evidence to review, not confirmation that a bug exists. In the
+dashboard, choose a playthrough and its Recording tab; timestamp links refer only
+to that run's uploaded video. Headless Simulation Runner has no rendered video.
+
 For Jump Bunny, use `https://reality-software-entertainment.github.io/poc-jump-bunny/`
 and the public player label **Ravensight Playtester**. This is a local browser run,
 not a hosted browser service. Human playtesting is still needed for subjective
@@ -884,3 +945,17 @@ feedback and timing-sensitive gameplay.
 ## Build adapters
 
 Browser and Godot runners share the paid playtest lifecycle with desktop, Unity/Unreal export, Android/iOS and custom JSON-RPC adapters. Native adapters require a locally configured automation server and remain experimental until validated against your build. See [adapter setup and validation boundaries](docs/ADAPTERS.md) and the [native runner starter](docs/NATIVE_RUNNER.md); `check` probes the automation server for free before a paid native run.
+
+## Simulation Runner
+
+Available since `0.4.0`. Simulation Runner requires macOS or Linux (including WSL); native Windows is refused before purchase.
+
+Run seeded mechanics tests through a local game adapter, compare explicit strategies and replay selected traces. A batch uses the existing standard Playtest purchase of 5 AI units from prepaid balance, confirmed against the server quote. Only the summary is uploaded; game execution and replay traces stay local. It does not automatically call a model or replace testing the player interface.
+
+```sh
+ravensight-playtest simulate --config simulation.json --driver-config bridge.json
+ravensight-playtest simulate --resume .ravensight/simulations/BATCH_ID/state.json
+ravensight-playtest simulation-replay trace-00000.json --driver-config bridge.json
+```
+
+See [Simulation Runner setup, adapter protocol and limits](docs/SIMULATION.md). The package includes `examples/simulation/economy.mjs` as a deterministic starting adapter. Local replay creates no new paid job.

package/docs/SIMULATION.md

@@ -0,0 +1,91 @@
+# Simulation Runner
+
+Available since `0.4.0`, with the matching released Ravensight server.
+
+Simulation Runner exercises game mechanics through a developer-owned local adapter. It can run seeded games across explicit strategies, summarize outcomes and metrics, and save selected replay traces. Use it for questions such as whether an economy supports several viable strategies or whether a progression rule traps a simulated player. It does not judge animation, interface usability, feel or subjective fun.
+
+A batch uses the existing standard Playtest purchase: **5 AI units from prepaid balance**. The CLI obtains the current server quote and asks for confirmation before registration. One batch is one purchase, not one purchase per simulated game. It does not consume the monthly shared AI allowance. Local replay is a debugging operation with no new job or purchase. Automatic AI interpretation is not included; an agent can use the existing report tools when explicitly asked to interpret the summary.
+
+## Setup and run
+
+Run Simulation Runner on macOS or Linux, including WSL. Native Windows is rejected during preflight before quoting or registering a purchase because this feature requires POSIX process-group cleanup. This restriction applies to Simulation Runner, not every Playtest adapter. The recorded real-process and 10,000-game validation was run on macOS; native Windows and WSL execution were not validated in this review.
+
+Sign in and initialize your Ravensight project as usual. Implement the protocol below, or start with the packaged `examples/simulation/economy.mjs`. The example is deterministic and contains no networking or telemetry.
+
+Create `bridge.json` with absolute paths for the Node executable and adapter:
+
+```json
+{
+  "command": "/absolute/path/to/node",
+  "args": ["/absolute/path/to/economy.mjs"],
+  "cwd": "/absolute/path/to/game"
+}
+```
+
+The adapter is trusted local code, **not a sandbox**. No shell is used to spawn it. Account/provider credential environment variables are stripped. Paid execution injects a synthetic Playtest token and run/job IDs; preflight and replay inject an invalid token sentinel so an inherited token cannot accidentally mark the session as production.
+
+Create `simulation.json`:
+
+```json
+{
+  "version": 1,
+  "seed": "economy-review-1",
+  "runs": 100,
+  "max_steps": 12,
+  "workers": 4,
+  "strategies": [
+    { "name": "random", "policy": "random" },
+    { "name": "collector", "policy": "scored" },
+    { "name": "saver", "policy": "scripted", "actions": ["earn", "earn", "earn", "earn", "earn", "earn", "earn", "earn"] }
+  ]
+}
+```
+
+```sh
+ravensight-playtest simulate --config simulation.json --driver-config bridge.json
+```
+
+A `driver_config` path inside the simulation file may replace `--driver-config`. The CLI probes the adapter before purchasing. A probe only starts a game and validates its first observation; it does not run the batch. Keep adapter initialization free of destructive side effects.
+
+Strategy names start with a lowercase letter and contain at most 64 lowercase letters, digits, underscores or hyphens. Strategies are assigned round-robin. Each run receives a seed derived from the root seed and its zero-based run index. Worker count changes concurrency, not action selection. `random` chooses among legal action IDs deterministically. `scripted` follows supplied IDs and fails on an illegal action or exhaustion before a terminal state. `scored` chooses the highest score supplied by the game for that strategy name, with a deterministic action-ID tie break. Names such as "aggressive" have no built-in gameplay meaning.
+
+## Adapter protocol
+
+Use one JSON-RPC 2.0 JSON object per line on stdout. Send diagnostics to stderr. The runner starts a fresh subprocess for each game, with one request outstanding at a time.
+
+```json
+{"jsonrpc":"2.0","id":1,"method":"simulation.start","params":{"version":1,"seed":"derived-seed","strategy":"collector","max_steps":12}}
+{"jsonrpc":"2.0","id":1,"result":{"done":false,"state_hash":"state-0","actions":[{"id":"earn","scores":{"collector":1}},{"id":"buy","scores":{"collector":3}}],"metrics":{"coins":4}}}
+{"jsonrpc":"2.0","id":2,"method":"simulation.step","params":{"action":"buy"}}
+{"jsonrpc":"2.0","id":2,"result":{"done":true,"state_hash":"state-1","actions":[],"metrics":{"coins":2,"inventory":1},"outcome":"collected"}}
+```
+
+The game owns state hashing and must include every state component relevant to reproducibility. Return `done` as a boolean, a nonempty `state_hash` of at most 128 characters, and at most 256 unique action IDs of at most 128 characters. Nonterminal observations need legal actions. Optional action scores contain at most 16 finite numeric values keyed by strategy. Optional metrics contain at most 32 finite numeric values between -1e15 and 1e15 with keys of at most 80 characters. Optional outcomes are nonempty strings of at most 80 characters. Reserved names `__proto__`, `constructor` and `prototype` are rejected.
+
+## Limits and results
+
+- 1 to 10,000 requested games, 1 to 10,000 actions per game, 1 to 16 workers and 1 to 16 strategies.
+- At most 1,000,000 actions across a batch and at most 15 minutes of execution, subject to the registered server budget.
+- At most 30 seconds per response and 256 KiB per response. Cancellation terminates the subprocess group, escalating from SIGTERM to SIGKILL.
+- At most ten selected traces: the first ten run indices, not an exhaustive failure archive. Each trace is bounded to 4 MiB and 10,000 actions.
+- Per-strategy summaries retain at most 32 distinct metric names and 128 outcome buckets. Additional outcome names are grouped as `other`; keep metric names stable between games.
+
+`duration_ms` reports execution time within the batch deadline, excluding any cleanup or post-stop serialization beyond that deadline. `results.jsonl` streams one local result per requested run. `summary.json` contains completed, failed and truncated totals, total actions, terminal outcome counts and terminal metric count/min/max/mean by strategy. Metrics are run-level terminal values, not a mean across every observation. A terminal result without an outcome uses `unspecified`. Reaching `max_steps` is **truncated**, never a successful terminal outcome. Games not started because of cancellation or a batch limit count as failed, with the stop reason retained in the local report.
+
+Only the bounded summary is uploaded to the existing Playtest job. Adapter scripts, raw states, per-game rows, replay traces and scripted action lists remain local. The public configuration hash covers the registered configuration and strategy names/policies; trace metadata also includes a separate full local configuration hash covering scripts.
+
+## Recovery and replay
+
+The CLI prints a durable recovery journal path. Recover the same purchase rather than starting another batch:
+
+```sh
+ravensight-playtest simulate --resume .ravensight/simulations/BATCH_ID/state.json
+```
+
+Resume preserves the API host, game, configuration and purchase identity. Do not combine it with a new config or bridge argument. A changed bridge configuration is rejected. Keep the adapter build and its dependencies unchanged for reproducible recovery. After a process interruption, any restarted local attempt has its own output directory. If the summary was already saved, resume uploads that summary without executing the games again. A terminal server job is not restarted.
+
+```sh
+ravensight-playtest simulation-replay /absolute/path/to/trace-00000.json --driver-config bridge.json
+```
+
+Replay checks each recorded state hash, terminal flag, outcome and metrics and reports the first divergence. A matching replay verifies that recording with the current adapter; it does not prove the game is deterministic. External services, clocks, platform differences and ignored seeds can cause divergence. A trace from a max-step-truncated game only covers the recorded window, not a finished game. Size-capped or failed recordings can verify their captured prefix, reported as `matched_prefix: true` and `fully_replayed: false`. A recorded failed protocol action is retried after that prefix; `failure_reproduced` states whether the same sanitized protocol error recurred. A failed initial start has no state prefix to verify. Policy failures such as an exhausted script only support prefix replay, not recreation of the policy failure. These results never claim a fully reproduced terminal game.

package/docs/godot-driver.md

@@ -0,0 +1,310 @@
+# The Godot driver
+
+`godot_driver` lets a playtest persona observe and control a native Godot game
+by launching the developer's own Godot on their own machine. It has two halves:
+
+- `addons/ravensight_driver/` (GDScript, Godot 4.2+): a TCP server that answers
+  JSON-RPC requests from inside the running game.
+- `src/run/drivers/godot.js` (Node): starts Godot, speaks that protocol, turns
+  the scene tree into an observation, and writes screenshots.
+
+Spec 07 is the contract for both halves. This document is how the two sides fit
+together and what the driver is and is not able to do.
+
+## The driver never reaches players
+
+The addon refuses to start unless every one of these holds:
+
+1. The build is not an export template, **or** it was exported with the custom
+   feature tag `ravensight_driver`. A normal release export never activates it.
+2. **And** either `--ravensight-driver` is in the user args (after the `--`
+   separator) or `RAVENSIGHT_DRIVER=1` is set.
+3. It binds `127.0.0.1` only. There is no code path that binds any other
+   address, and the Node client refuses to dial a non loopback host.
+4. If `RAVENSIGHT_DRIVER_TOKEN` is set, the first request on a connection must
+   be `hello{token}`. A wrong token gets `-32002` and the connection closes.
+5. One client at a time. A second connection is answered with `-32001 busy` and
+   hung up on.
+
+When any of that fails the autoload calls `queue_free()` on itself in
+`_ready()`, so an inactive driver is not a node that idles in `_process`, it is
+not in the tree at all.
+
+On top of refusing to run, `export_plugin.gd` **strips the whole addon** from
+any export whose preset was not built with the `ravensight_driver` feature tag,
+so the usual release build does not contain the driver's code.
+
+One file does not simply vanish. A developer who installs the addon in their own
+project gets `RavensightDriver` written into `project.godot`, and `project.godot`
+ships in the export. Deleting `driver.gd` outright would leave that autoload
+pointing at a file that is not there, and the exported game would greet the
+player with a load error before its first frame. So the strip leaves a four line
+inert stub at `driver.gd`'s path: `extends Node` and a `_ready()` that calls
+`queue_free()`. The autoload resolves, nothing listens, and no driver code
+ships. The alternative, failing the export with a message, was rejected because
+it punishes a developer for having installed the addon at all.
+
+This is covered by a test, not an assertion of intent: `godot-real.test.js`
+exports the same project twice with `--export-pack` (which needs no export
+templates) and checks that the plain pack contains the stub and none of the
+driver's source, while a pack built with the feature tag contains all of it.
+
+## How the addon gets into the project
+
+The CLI never writes to the developer's project. `prepareProjectCopy` copies the
+project to a temp directory (mode `0700`, `.git` and `node_modules` excluded,
+refused above 2 GB), drops the addon into `addons/ravensight_driver/` in the
+copy, and registers the autoload **twice** in that copy:
+
+- in `override.cfg`, which is the mechanism spec 07 describes, and
+- in the copy's own `project.godot`.
+
+Spec 07's open question 1 asks whether an `override.cfg` autoload works for
+every project layout. Rather than leave that unanswered at runtime, both are
+written. They name the same script, so whichever Godot honors, the result is
+identical. A project that already ships an `override.cfg` or already registers
+other autoloads keeps them: only the one `RavensightDriver` key is added.
+
+**Symlinks are the trap here.** `project.godot`, `override.cfg` and `addons/`
+can each be a symlink pointing outside the project, which is what a developer
+gets from a shared settings repo or a vendored addons checkout. Writing to a
+symlink writes to its target, so a naive copy would put the injection into the
+developer's real files, permanently, breaking the one promise spec 07 makes
+about the project directory. Three things prevent it:
+
+- the copy runs with `dereference: true`, so a symlink becomes a real file or
+  directory in the copy rather than a pointer back out of it;
+- every path this module writes to is materialized first, component by
+  component, replacing any surviving link with a real copy of its content;
+- `measureProject` follows symlinked directories when it sizes the project,
+  because sizing that ignored them reported a few hundred bytes for a tree the
+  copy then dereferenced into gigabytes, and the 2 GB cap did nothing.
+
+Dereferencing brings its own failure mode, so the copy filter drops two shapes
+outright. A link resolving to the project directory or any ancestor of it
+(`selfref -> .`, `up -> ..`) is dropped: dereferenced, the copy walks the project
+into itself, finds the link again, and repeats until `PATH_MAX`, which one such
+link turned into 33 nested copies. And a directory already dereferenced once is
+not dereferenced again, because `measureProject` counts a shared target once and
+the two have to agree or the cap stops meaning anything.
+
+Both rules compare **realpaths**, not `path.resolve()` output. On macOS
+`os.tmpdir()` is `/var/folders/...`, a symlink to `/private/var/folders/...`, so
+a project living there never matches its own symlink target unless both sides are
+fully resolved, and the rule quietly does nothing. That is a test that caught it,
+not a hypothetical.
+
+A failed `prepareProjectCopy` deletes the temp root it created, so a project that
+cannot be prepared does not leave a half written copy on disk. A `tempRoot` the
+caller passed in is left alone: it is theirs.
+
+A dangling symlink is dropped from the copy instead of failing the launch:
+`fs.cp` throws on one it cannot dereference, and one broken link is not a reason
+a project cannot be playtested.
+
+## Launch
+
+```
+<godot> --path <temp copy> --resolution 1280x720 -- --ravensight-driver --ravensight-driver-port=<port>
+```
+
+The binary is found the way spec 17 describes: `--godot-path`, then `$GODOT`,
+then `PATH` (`godot`, `godot4`), then the platform's usual install locations.
+When the project declares an engine version in `config/features`, a binary with
+that major.minor wins over a newer one. Anything below 4.2 is refused.
+
+The environment matters as much as the arguments:
+
+| Variable | Read by | Why |
+|---|---|---|
+| `RAVENSIGHT_DRIVER=1` | the addon | activation |
+| `RAVENSIGHT_DRIVER_PORT` | the addon | loopback port, default 47800 |
+| `RAVENSIGHT_DRIVER_TOKEN` | the addon | per run handshake secret |
+| `RAVENSIGHT_PLAYTEST_TOKEN` | the game's own `Ravensight.gd` | playtest session auth |
+| `RAVENSIGHT_PLAYTEST_RUN_ID` | the game's own `Ravensight.gd` | `pt_run` tag, throwaway device id |
+| `RAVENSIGHT_PLAYTEST_JOB_ID` | the game's own `Ravensight.gd` | `pt_job` tag |
+| `RAVENSIGHT_PLAYTEST_PERSONA` | the game's own `Ravensight.gd` | `persona` tag |
+
+Only those, plus a fixed allowlist of variables a process needs to start at all
+(`PATH`, `HOME`, the `XDG_*`, `DISPLAY`, `LD_LIBRARY_PATH` and
+`LIBGL_ALWAYS_SOFTWARE` set that the Xvfb path needs, the Windows equivalents),
+reach Godot. The environment is **not** inherited wholesale: the game is the
+customer's own code running on the developer's machine, and it has no business
+seeing `ANTHROPIC_API_KEY`, `RAVENSIGHT_TOKEN`, cloud credentials or CI secrets.
+`ENV_ALLOWLIST` is the list, and `extraEnv` is the deliberate escape hatch for a
+caller that needs one more variable through.
+
+The last four are the whole reason a driven run shows up correctly in the
+dashboard: the SDK reads them in `_ready()` and every event the game already
+tracks gets `synthetic: true`, `persona`, `pt_job`, `pt_run` and `pt_tier`
+without the developer writing anything. The driver sets them and otherwise
+stays out of the SDK's way. When the SDK autoload is present the driver also
+connects to its `event_tracked` signal and mirrors each tracked event to the
+runner as a `ravensight_event` notification.
+
+The runner then dials `127.0.0.1`, retrying every 250 ms for up to 60 s, sends
+`hello`, and calls `ping`. Godot's stdout and stderr go to `<run dir>/godot.log`
+and are scanned for `SCRIPT ERROR` / `ERROR:` lines, which `consoleErrors()`
+returns with the `at:` source line attached. That is spec 07's stdout fallback:
+the engine's `Logger` hook is not available on every 4.x version the CLI
+supports, so it is not used.
+
+## The driver interface
+
+`src/run/drivers/driver.js` does not exist on this branch, so `godot.js` writes
+the five method shape out in its own header instead of importing it:
+
+```js
+launch({ recordVideo, viewport })  // -> {}
+observe({ vision })                // -> Observation
+act(action)                        // -> { ok, error?, code?, data?, changed }
+screenshot(name)                   // -> { path } | { path: null, skipped }
+stop({ reason })                   // -> { videoPath, recording, frameCount, chapters, logPath, exitCode }
+```
+
+plus the two optional extras spec 03 lists, `consoleErrors()` and
+`chapter(title)`.
+
+`act` maps the spec 03 action kinds onto driver methods:
+
+| Action | Driver calls |
+|---|---|
+| `key` | `key` press, `wait_frames{1}`, `key` release |
+| `type` | one press and release per character, each carrying its unicode, then Enter when `submit` |
+| `click` with a string target | `click{node_path}` (the center of the Control's global rect) |
+| `click` with coordinates | `click{x,y}` |
+| `press_action` | `press_action{action, duration_ms}` |
+| `wait` | `wait_frames`, milliseconds converted at 60 fps |
+| `set_time_scale` | `set_time_scale` |
+| `quit` | `quit` |
+
+`changed` compares the hash of the scene tree outline before and after the
+action. Two details make that answer honest:
+
+- input reaches the game a frame after it is parsed, and a handler that updates
+  a label runs a frame after that, so `act` waits `settleFrames` (default 2)
+  before taking the comparison snapshot;
+- `observe()` always takes its own `get_state`. An earlier version reused the
+  snapshot `act()` had just taken, to save a round trip, and that handed the
+  persona a screen one action out of date, which is the opposite of what
+  `observe()` is for. Two `get_state` calls a turn is the price of the
+  observation being current.
+
+A driver level error is returned, not thrown: an undefined input action comes
+back as `{ ok: false, code: -32012, data: { available_actions } }` so the
+persona can pick a real one instead of the run ending.
+
+## Observations cite files
+
+Every node in a `get_state` snapshot carries its `path`, and also its `scene`
+(`scene_file_path`) and `script` (the script's `resource_path`) when it has
+them. `buildObservation` collects all of those and puts them at the top of the
+observation text:
+
+```
+scene: res://main.tscn
+scene node: /root/Main
+scripts: res://ravensight_stub.gd, res://main.gd
+
+Window root
+  Node2D Main @(0,0) [groups=ravensight_state] {res://main.tscn} <res://main.gd>
+    CanvasLayer HUD
+      Label TrustLabel "Trust: 4" @(24,16)
+      Button BeginButton "Begin" @(600,400) [focus]
+
+game state:
+  /root/Main: {"phase":"HEALING","trust":4}
+```
+
+That is what lets a finding say which scene and which script to open. The
+driver's own autoload is filtered out of the snapshot, so its script path never
+appears in that list.
+
+Subtrees with nothing a player could see or press are dropped, the outline is
+capped at 12k characters, and `needsVision()` asks for a screenshot when the
+remaining outline is under 200 characters, which is the canvas drawn or 3D case.
+
+Spec 07's `extensions/navigation.gd` (`nav_sample`, `nav_path`, `teleport`,
+`raycast`) is **not** part of this addon: those methods exist for spec 06 level
+probing, not for a persona run, and they can be added later through
+`register_method()` without touching the protocol. `list_actions` is the one
+extension method implemented here, because a persona that gets `-32012` needs to
+know what it may press instead.
+
+The author hook is unchanged from the spec: a node in the `ravensight_state`
+group with a `_ravensight_state() -> Dictionary` method. Vectors become arrays,
+recursion stops at 8 levels, each result is capped at 64 KB, and a hook that
+returns something other than a Dictionary gets `-32020` naming the node path.
+A broken hook is recorded as a console error against the game, not treated as a
+driver failure.
+
+## Headless is for smoke tests, not for playtests
+
+Verified against Godot on macOS: in `--headless` mode the engine routes no
+synthesized mouse or GUI input at all. A `click` on a Button whose rect is right
+there does nothing, and `screenshot` returns `-32010` because there is no
+viewport. The same clicks work windowed.
+
+So `--headless` is only useful for a game driven entirely by InputMap actions
+that manages its own focus. `launch()` pushes a console entry saying so when it
+detects a headless run, and the local CLI default stays windowed.
+
+`--record-video` opts into local capture of this game's viewport; it does not
+consent to upload. `--upload-video` still enables both recording and optional
+video upload. Audio capture is off. Godot samples up to ten frames per second,
+records their actual timestamps, and uses optional local FFmpeg to create
+`session.mp4`. Set `RAVENSIGHT_FFMPEG` to an existing executable when it is not
+on PATH. No package or encoder is downloaded automatically.
+
+Recording is capped at 15 minutes, 1280 by 720 pixels, 256 MiB of temporary
+frames and the existing 200 MiB final video limit. Temporary frames are removed
+after encoding. A missing encoder or headless viewport is reported as
+`unsupported` in local `recording.json`; frames are never labeled as video.
+Cancellation, a game crash, or a recording limit can produce a playable partial
+video, explicitly marked with its stop reason. Encoding failure produces no
+uploadable video. Normal stop and concurrent cancellation share one bounded
+finalization. A hard kill of the CLI or machine cannot guarantee finalization.
+
+`recording.json` stays local and includes timestamps, duration, size, status,
+reason and `audio: false`. When optional FFprobe is available, duration is read
+from the actual silent media stream; otherwise Godot uses frame timestamps and
+Playwright uses capture clock time, labeled by `durationSource`. This metadata
+is not part of the upload allowlist. Video upload remains separately opted in,
+with the existing per-run limits and server retention unchanged.
+
+## Verifying a change
+
+The GDScript is parse checked with a real Godot binary, which is the only thing
+that catches an undeclared identifier:
+
+```bash
+/Users/.../game_tracker/scripts/check-gdscript.sh \
+  /Applications/Godot.app/Contents/MacOS/Godot \
+  addons/ravensight_driver/driver.gd \
+  addons/ravensight_driver/plugin.gd \
+  addons/ravensight_driver/export_plugin.gd
+```
+
+It works on this standalone package: `--check-only --script` needs no
+`project.godot`, and `res://` resolves against the working directory, so
+`plugin.gd`'s `preload` of `export_plugin.gd` is checked too. Run it from the
+package root.
+
+Most Node tests (`npm test`) run against `testing/fake-godot.js`, a fake driver
+socket that speaks the same wire protocol and enforces the same single client,
+hello before anything, and keycode resolution rules. No Godot binary is involved
+there, so those tests prove the Node side's shapes and nothing about the
+GDScript.
+
+`godot-real.test.js` covers what only a real engine can: the activation matrix
+(nothing listens without the flag, a wrong token is refused, a second client is
+refused, an unknown method and a bad param get the right codes), the author hook,
+typing a space, and the export strip. It drives the fixture project at
+`test/fixtures/godot-project/` and **skips itself** when no Godot 4.2+ binary is
+found, the same way the Ravensight repo's ClickHouse integration tests skip
+without a server. A skip there is expected, not a failure to fix, but it does
+mean a change to `driver.gd` is unverified until it has run on a machine with
+Godot installed.
+
+The fixture is never written to: every case copies it to a temp directory first,
+so `--import` and `--export-pack` leave no `.godot/` or `out/` behind.

package/examples/simulation/economy.mjs

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+// Trusted local adapter example. It has no networking or telemetry.
+import readline from 'node:readline';
+import { createHash } from 'node:crypto';
+let seed, turn, coins, inventory;
+function observe() {
+  const done = turn >= 8;
+  return { done, state_hash: createHash('sha256').update(JSON.stringify([seed, turn, coins, inventory])).digest('hex'),
+    actions: done ? [] : [{ id: 'earn', scores: { saver: 2, collector: 1 } }, ...(coins >= 2 ? [{ id: 'buy', scores: { saver: 0, collector: 3 } }] : [])],
+    metrics: { coins, inventory }, ...(done ? { outcome: inventory >= 2 ? 'collected' : 'saved' } : {}) };
+}
+for await (const line of readline.createInterface({ input: process.stdin })) {
+  let request;
+  try {
+    request = JSON.parse(line);
+    if (request.method === 'simulation.start') { seed = request.params.seed; turn = 0; inventory = 0; coins = parseInt(createHash('sha256').update(seed).digest('hex').slice(0, 4), 16) % 3; }
+    else if (request.method === 'simulation.step' && !observe().done && observe().actions.some(a => a.id === request.params.action)) {
+      turn++; if (request.params.action === 'earn') coins++; else { coins -= 2; inventory++; }
+    } else throw new Error('Illegal simulation request');
+    process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result: observe() }) + '\n');
+  } catch { process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request?.id, error: { code: -32602, message: 'Invalid request' } }) + '\n'); }
+}

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "ravensight-playtest",
-  "version": "0.3.2",
-  "description": "AI personas play your game and file reports to Ravensight.",
+  "version": "0.5.0",
+  "description": "Local AI playtests and seeded game simulations with reports in Ravensight.",
   "license": "MIT",
   "homepage": "https://ravensight.io/docs/#playtest",
   "bugs": {
@@ -12,6 +12,8 @@
     "playtest",
     "game-analytics",
     "ai-personas",
+    "simulation",
+    "game-balance",
     "ravensight"
   ],
   "type": "module",
@@ -27,6 +29,9 @@
     "addons",
     "docs/ADAPTERS.md",
     "docs/NATIVE_RUNNER.md",
+    "docs/godot-driver.md",
+    "docs/SIMULATION.md",
+    "examples/simulation",
     "README.md",
     "LICENSE",
     "!src/**/*.test.js",

package/src/api/client.js

@@ -194,9 +194,9 @@ export function createClient(options = {}) {
      * persona run's price; anything else leaves `run_length` out, which the
      * server reads as a standard run.
      */
-    estimate: (gameId, { modules = [], personas = [], runLength = null } = {}) =>
+    estimate: (gameId, { modules = [], personas = [], runLength = null, simulation, controllers, driver } = {}) =>
       request('POST', `${gameBase(gameId)}/estimate`, {
-        body: { modules, personas, ...(runLength === 'long' ? { run_length: 'long' } : {}) }
+        body: { modules, personas, ...(controllers ? { controllers, driver } : {}), ...(runLength === 'long' ? { run_length: 'long' } : {}), ...(simulation ? { simulation } : {}) }
       }).then(answer => answer.body),
 
     /**

package/src/cli.js

@@ -54,7 +54,9 @@ export const RUNNER_HELP = `
 Run options (parsed by the runner, see src/run/args.js):
   --personas <list>         comma separated persona slugs, default one (curious-kid); "personas" lists them
   --driver <name>           playwright_web, godot_driver, desktop_driver, unity_driver, unreal_driver, android_driver, ios_driver or cli_stdio
-  --controller <name>       standard or jev action selection (default standard)
+  --controller <name>       one controller: standard, jev or rune (if enabled)
+  --compare-with <names>    separate standard + jev and/or rune playthroughs; quotes every run
+  --record-video           record this game locally; --upload-video also uploads it
   --build-url <url>         the web build to play
   --godot-project <dir>     the Godot project directory
   --driver-config <file>    local WebDriver or JSON-RPC bridge configuration
@@ -316,6 +318,24 @@ export function buildProgram() {
       return run(flagsFrom(options, { game: options.game, argv: command.args }));
     }));
 
+  common(program.command('simulate').description('run a paid, seeded batch against a local headless game'))
+    .option('--game <gameId>', 'the game, when this repo is not linked')
+    .option('--config <file>', 'simulation JSON: seed, runs, max_steps, workers and strategies')
+    .option('--driver-config <file>', 'local JSON-RPC executable configuration')
+    .option('--resume <file>', 'resume the same paid batch from its state.json, without another charge')
+    .action(capture(slot, async options => {
+      const { simulateCommand } = await import('./commands/simulate.js');
+      return simulateCommand(flagsFrom(options, { game: options.game, configFile: options.config, driverConfig: options.driverConfig, resume: options.resume }));
+    }));
+
+  common(program.command('simulation-replay').description('verify a recorded simulation trace locally; creates no paid job'))
+    .argument('<trace>', 'a selected trace.json written by simulate')
+    .requiredOption('--driver-config <file>', 'local JSON-RPC executable configuration')
+    .action(capture(slot, async (trace, options) => {
+      const { replayCommand } = await import('./commands/simulate.js');
+      return replayCommand(trace, flagsFrom(options, { driverConfig: options.driverConfig }));
+    }));
+
   return program;
 }