ravensight-playtest 0.2.4 → 0.2.5

package/README.md

@@ -87,6 +87,18 @@ content pack verified by sha256, disk, recommended concurrency, and the two
 optional tools. Exit code 3 when any row fails; a missing optional tool is a
 warning and never a failure, so CI does not break on it.
 
+For a web game there is also a `browser` row: `check` imports Playwright,
+launches a headless Chromium and closes it, which is exactly how a web run
+starts. A missing browser fails the check when the linked build is a URL or you
+pass `--web`. When no build is linked yet and the configured driver is the
+default `playwright_web`, it is only a warning, so a Godot repo that never
+changed the default is not failed over a browser it will not use. A repo linked
+to a Godot project or a driver config gets no browser row. The fix is:
+
+```bash
+npm i -D playwright && npx playwright install chromium
+```
+
 Past `--personas` in `POST /jobs`, requesting more than 100 personas or the same
 slug twice is refused with 400 `too_many_personas`, and a slug this game does
 not have is 400 `unknown_persona`; `run` checks the count against the same
@@ -101,6 +113,7 @@ automatically. `check` shows that same warning as a row when it applies.
 | `--modules <list>` | Price this module list, comma separated. |
 | `--personas <list>` | Price this persona list. Without it the balance row prices no personas, so the quote is lower than a real job's. |
 | `--godot-path <path>` | Where Godot is, if it is not on the PATH. |
+| `--web` | Check the browser even when this repo is not linked to a web build. |
 
 ### init
 
@@ -108,9 +121,15 @@ Links this repo to a game and writes `.ravensight/config.json`. Refuses a game
 your token cannot see, so the first job does not fail with a 404. Adds
 `.ravensight/jobs/` to an existing `.gitignore`.
 
+Without `--game`, one visible game is linked as is. With several, `init` lists
+them numbered and asks `Which game? [1-N]` when it is running in a terminal; in
+CI or a pipe it prints the list and asks you to pass `--game` instead. A token
+that can see no games is told so, with the fix (approve a game for the token at
+login).
+
 | Flag | Effect |
 |---|---|
-| `--game <gameId>` | The game to link. With one visible game it is optional. |
+| `--game <gameId>` | The game to link. Optional with one visible game, or in a terminal. |
 | `--force` | Relink a repo that is already linked. |
 
 ### brief
@@ -130,6 +149,30 @@ warning `run` shows. A brief is required before `run`; the server answers 409
 | `--game <gameId>` | The game, when this repo is not linked. |
 | `--modules <list>`, `--personas <list>` | Price this module and persona list in the estimate it prints. |
 
+#### The `brief --from` file
+
+A JSON object of brief fields, or the same object wrapped as
+`{ "fields": { ... } }`, which is how a saved brief carries them. The first four
+are what makes a brief complete; the rest are optional.
+
+| Field | Shape | Notes |
+|---|---|---|
+| `goals` | `[{ "goal": "...", "priority": 1 }]` | At least one. `goal` is one of `find_bugs`, `onboarding`, `balance`, `level_flow`, `narrative`, `accessibility`, `performance`. `priority` is 1 (highest) to 5. |
+| `success_criteria` | string | What success looks like, one criterion per line. The aggregate report scores each line. |
+| `build_notes.controls` | string | How to play: keys, buttons, the verbs a persona needs. |
+| `build_notes.how_to_start` | string | Where a persona begins: the URL, the menu path, the first thing to press. |
+| `design_intent_notes` | array of strings | Deliberate choices a persona must not report as bugs. |
+| `known_issues` | array of strings | Issues you already know about, so they are not reported again. |
+
+```json
+{
+  "goals": [{ "goal": "onboarding", "priority": 1 }, { "goal": "find_bugs", "priority": 2 }],
+  "success_criteria": "A new player reaches level 2\nNobody gets stuck in the menu",
+  "build_notes": { "controls": "Arrow keys to move, space to jump", "how_to_start": "Press Start on the title screen" },
+  "design_intent_notes": ["The fog on level 1 is intentional"]
+}
+```
+
 ### profile
 
 Builds the game profile and the capability report: what your game is, what can
@@ -158,7 +201,7 @@ splitting the job in two is what gets everything played.
 | `--build-url <url>` | The web build to play. |
 | `--godot-project <dir>` | The Godot project directory. |
 | `--cli <command>` | The command that starts a text build. |
-| `--max-actions <n>` | Cap the actions one persona may take. |
+| `--max-actions <n>` | Lower the actions one persona may take. It can only lower the persona's own patience (and the server's per run ceiling), never raise it. |
 | `--concurrency <n>` | Personas at once. Default 2; `check` recommends a number for this machine. |
 | `--upload-video` | Upload `session.webm` as well as the reports. |
 | `--upload-transcript` | Upload `transcript.jsonl` as well as the reports. |
@@ -166,10 +209,24 @@ splitting the job in two is what gets everything played.
 | `--headed`, `--headless` | Show or hide the browser. |
 | `--verbose` | Print each turn. |
 
-Four flags are accepted and reported as not applied by this release, rather than
-silently ignored: `--budget-usd` and `--quality` (the budget and the model tier
-are the server's, enforced at the proxy), `--offline-upload` (uploads already
-queue) and `--build-dir` (serve the build and pass `--build-url`).
+Three flags are accepted and reported as not applied by this release, rather
+than silently ignored: `--quality` (the model tier is the server's, enforced at
+the proxy), `--offline-upload` (uploads already queue) and `--build-dir` (serve
+the build and pass `--build-url`). There is no spending flag: each run's model
+budget is set by the server from what the job was charged, and `--budget-usd` is
+an unknown option (the refusal says where the budget comes from).
+
+Each run hands its playtest identity to the game, so the game's own Ravensight
+events from a playtest are marked synthetic rather than counted as a player. A
+Godot game reads it from its environment. A web game's JavaScript SDK reads it
+from the page: the browser driver sets `window.__RAVENSIGHT_PLAYTEST__` and
+`localStorage['ravensight.playtest']` before the game's scripts run. It is never
+sent as a request header, which would reach your game's own API too.
+
+When a run reaches its model budget before the report step, nothing more is
+asked of the model. The persona's notes are written to a partial `report.json`
+(`partial: true`, one unreviewed finding per note with the nearest screenshot as
+evidence) and uploaded, so the job's aggregate still counts them.
 
 ### fix
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ravensight-playtest",
-  "version": "0.2.4",
+  "version": "0.2.5",
   "description": "AI personas play your game and file reports to Ravensight.",
   "license": "MIT",
   "homepage": "https://ravensight.io/docs/#playtest",

package/src/cli.js

@@ -50,7 +50,7 @@ async function loadRunner(name, exportName) {
  * Kept in the same order as `VALUED` and `BOOLEAN` in src/run/args.js, which is
  * the list it has to agree with.
  */
-const RUNNER_HELP = `
+export const RUNNER_HELP = `
 Run options (parsed by the runner, see src/run/args.js):
   --personas <list>         comma separated persona slugs, default the pack's set
   --driver <name>           playwright_web, godot_driver, desktop_driver, unity_driver, unreal_driver, android_driver, ios_driver or cli_stdio
@@ -58,7 +58,7 @@ Run options (parsed by the runner, see src/run/args.js):
   --build-url <url>         the web build to play
   --godot-project <dir>     the Godot project directory
   --driver-config <file>    local WebDriver or JSON-RPC bridge configuration
-  --max-actions <n>         cap the actions one persona may take
+  --max-actions <n>         lower the actions one persona may take; it can only lower the persona's own patience, never raise it
   --concurrency <n>         personas at once, default 2
   --upload-video            upload session.webm as well as the reports
   --upload-transcript       upload transcript.jsonl as well as the reports
@@ -68,7 +68,10 @@ Run options (parsed by the runner, see src/run/args.js):
   --verbose                 print each turn
 
 Accepted and reported as not applied by this release:
-  --budget-usd, --quality, --offline-upload, --build-dir
+  --quality, --offline-upload, --build-dir
+
+There is no spending flag: each run's model budget is set by the server from
+what the job was charged.
 `;
 
 /** Flags every command shares. */
@@ -157,13 +160,15 @@ export function buildProgram() {
     .addOption(new Option('--modules <list>', 'price this module list').argParser(list))
     .addOption(new Option('--personas <list>', 'price this persona list').argParser(list))
     .option('--godot-path <path>', 'where Godot is, if it is not on the PATH')
+    .option('--web', 'check the browser even when this repo is not linked to a web build')
     .action(capture(slot, async options => {
       const { check } = await import('./commands/check.js');
       return check(flagsFrom(options, {
         game: options.game,
         modules: options.modules,
         personas: options.personas,
-        godotPath: options.godotPath
+        godotPath: options.godotPath,
+        web: Boolean(options.web)
       }));
     }));
 

package/src/commands/check.js

@@ -19,6 +19,75 @@ const row = (name, status, detail, fix = null) => ({ name, status, detail, fix }
 /** "1 persona", "3 personas". A doctor that cannot count reads like a draft. */
 const plural = (count, noun) => `${count} ${noun}${count === 1 ? '' : 's'}`;
 
+/** The two commands that make the playwright_web driver work. */
+export const BROWSER_FIX = 'npm i -D playwright && npx playwright install chromium';
+
+/**
+ * The repo config, for deciding whether this is a web game. A config that
+ * cannot be read is reported by the rows that need it, not here.
+ *
+ * @param {Object} flags
+ * @returns {Promise<Object|null>}
+ */
+async function configForBrowser(flags) {
+  if (flags.config !== undefined) return flags.config;
+  try {
+    return await loadConfig({ repoRoot: flags.repoRoot });
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * How much a missing browser matters here, or null when the row is skipped.
+ *
+ * `fail` when `--web` asks for it or the linked build is a URL: a web run
+ * cannot start without one. `warn` when only the configured driver says
+ * `playwright_web` and no build is linked yet, because that is the untouched
+ * default `init` writes, and a Godot developer who never changed it must not
+ * get exit 3 over a browser they will never launch. Skipped otherwise.
+ *
+ * @param {Object} flags
+ * @param {Object|null} config
+ * @returns {'fail'|'warn'|null}
+ */
+export function browserSeverity(flags, config) {
+  if (flags.web) return 'fail';
+  if (!config) return null;
+  const kind = config.build && config.build.kind;
+  if (kind === 'url') return 'fail';
+  if (kind) return null;
+  return config.driver === 'playwright_web' ? 'warn' : null;
+}
+
+/**
+ * Import Playwright, launch headless Chromium, close it. The same three steps
+ * a web run starts with, so a pass here means the run will get a browser.
+ *
+ * @param {(specifier: string) => Promise<any>} [importer] test seam
+ * @returns {Promise<{ok: boolean, detail: string}>}
+ */
+export async function probeBrowser(importer) {
+  const load = importer || (specifier => import(specifier));
+  let browser = null;
+  try {
+    const playwright = await load('playwright');
+    browser = await playwright.chromium.launch({ headless: true });
+    const version = typeof browser.version === 'function' ? browser.version() : '';
+    return { ok: true, detail: `chromium ${version}`.trim() + ' launched headless' };
+  } catch (error) {
+    return { ok: false, detail: String(error && error.message ? error.message : error).split('\n')[0] };
+  } finally {
+    if (browser) {
+      try {
+        await browser.close();
+      } catch {
+        // A browser that will not close still launched, which is what was asked.
+      }
+    }
+  }
+}
+
 /**
  * Every environment check, as data.
  *
@@ -37,6 +106,7 @@ const plural = (count, noun) => `${count} ${noun}${count === 1 ? '' : 's'}`;
  * @param {string[]} [flags.modules]
  * @param {string[]} [flags.personas]
  * @param {string} [flags.repoRoot]
+ * @param {boolean} [flags.web] probe the browser whatever the config says
  * @param {Object} [deps]
  * @returns {Promise<{rows: Array, ok: boolean}>}
  */
@@ -243,6 +313,17 @@ export async function runChecks(flags = {}, deps = {}) {
     'needed only to re-encode a session video before upload'
   ));
 
+  const browserNeed = browserSeverity(flags, await configForBrowser(flags));
+  if (browserNeed) {
+    const probe = await probeBrowser(deps.importer);
+    rows.push(row(
+      'browser',
+      probe.ok ? 'ok' : browserNeed,
+      probe.detail,
+      probe.ok ? null : BROWSER_FIX
+    ));
+  }
+
   const godot = await (deps.inspectGodot || inspectGodot)(
     { repoRoot, godotPath: flags.godotPath },
     deps

package/src/commands/fix.js

@@ -1,7 +1,7 @@
 import { randomUUID } from 'node:crypto';
 import { appendFile, mkdir, readdir, readFile, writeFile } from 'node:fs/promises';
 import path from 'node:path';
-import { optionsFrom } from '../run/args.js';
+import { optionsFrom, unknownOptionNotes } from '../run/args.js';
 import { getDeps } from '../run/deps.js';
 import { createDriver } from '../run/drivers/index.js';
 import EXIT from '../run/exit.js';
@@ -15,7 +15,7 @@ import { createRepoTools, PROFILE_TOOLS } from '../run/repoTools.js';
 import { assertNoSecrets } from '../run/secretScan.js';
 import * as stateFile from '../run/state.js';
 import { createUsageLedger, writeUsageFile } from '../run/usage.js';
-import { createInterruptHandler, deliver, exitCodeFor, explainRegisterRefusal, formatCents, resolveSetup, resolveTarget, skillFor } from './run.js';
+import { createInterruptHandler, deliver, driverOptions, exitCodeFor, explainRegisterRefusal, formatCents, resolveSetup, resolveTarget, skillFor } from './run.js';
 
 /**
  * `ravensight-playtest fix <target>`: take one finding, change the code on a
@@ -155,6 +155,7 @@ async function fix(target, flags, injected, deps, log) {
   const { options, unknown } = optionsFrom(flags);
   if (unknown.length > 0) {
     log(`I do not know these options: ${unknown.join(', ')}. Run "ravensight-playtest fix --help".`);
+    for (const line of unknownOptionNotes(unknown)) log(line);
     return EXIT.ENVIRONMENT;
   }
   const parsed = parseTarget(target);
@@ -506,6 +507,7 @@ async function fix(target, flags, injected, deps, log) {
         repoRoot,
         entry: verifyRun,
         pack,
+        brief,
         finding,
         driverName,
         target: buildTarget,
@@ -752,6 +754,7 @@ export async function runVerify({
   repoRoot,
   entry,
   pack,
+  brief = null,
   finding,
   driverName,
   target,
@@ -769,7 +772,7 @@ export async function runVerify({
     return null;
   }
   const headless = driverName === 'godot_driver' ? options.headless === true : options.headed !== true;
-  const driver = await makeDriver(driverName, { headless });
+  const driver = await makeDriver(driverName, driverOptions({ headless, driverName, target, directory, entry, jobId }));
   const ledger = createUsageLedger();
   const heartbeat = createHeartbeat({
     api,
@@ -796,7 +799,8 @@ export async function runVerify({
       target,
       runDirectory: directory,
       skill: skillFor(pack, 'persona_playtest'),
-      brief: verifyBrief(pack.brief || null, finding),
+      // The brief fetched above, as `run` does: the pack does not always carry one.
+      brief: verifyBrief(brief || pack.brief || null, finding),
       intentNotes: pack.intent_notes || [],
       schemas: pack.schemas || [],
       maxActions,

package/src/commands/init.js

@@ -37,7 +37,8 @@ export async function ensureGitignore(repoRoot) {
 /**
  * `ravensight-playtest init --game <gameId>`
  *
- * Link this repo to a Ravensight game.
+ * Link this repo to a Ravensight game. Without `--game`, one visible game is
+ * used as is, and several are offered as a numbered list on a terminal.
  *
  * The game is checked against `whoami` before anything is written, because the
  * alternative is a config that looks right and fails on the first job with a
@@ -45,7 +46,7 @@ export async function ensureGitignore(repoRoot) {
  * authority: an unapproved game is invisible to it, by design.
  *
  * @param {{game?: string, apiUrl?: string, repoRoot?: string, force?: boolean, json?: boolean}} flags
- * @param {Object} [deps]
+ * @param {{context?: Object, choose?: Function}} [deps]
  * @returns {Promise<number>}
  */
 export async function init(flags = {}, deps = {}) {
@@ -58,15 +59,39 @@ export async function init(flags = {}, deps = {}) {
   // Every API call in this command goes through the one 401 handler, so a
   // refused credential is cleared and explained here the same way it is
   // everywhere else.
-  return withAuthHandling(context, () => linkRepo(context, flags, repoRoot));
+  return withAuthHandling(context, () => linkRepo(context, flags, repoRoot, deps.choose || ui.choose));
 }
 
-async function linkRepo(context, flags, repoRoot) {
+/**
+ * @param {Object} context
+ * @param {Object} flags
+ * @param {string} repoRoot
+ * @param {(question: string, labels: Array<string>) => Promise<number|null>} choose
+ *   the picker; answers null when there is no terminal to ask on
+ */
+async function linkRepo(context, flags, repoRoot, choose) {
   const whoami = await context.api.cli.whoami();
 
   const games = whoami.games || [];
   let gameId = flags.game;
 
+  if (!gameId && games.length === 0) {
+    throw new CliError(
+      'This token can see no games, so there is nothing to link.',
+      1,
+      { hint: 'Create a game in the dashboard, then run login again and approve it for this token.' }
+    );
+  }
+
+  // Never under --json: whatever reads that output is a script, not a person.
+  if (!gameId && games.length > 1 && !flags.json) {
+    // On a terminal, ask. Anywhere else (CI, a pipe) the picker answers null
+    // and the command falls through to the list and the --game error, which
+    // is what a script needs to see.
+    const picked = await choose(`Which game? [1-${games.length}]`, games.map(game => `${game.name || game.gameId} (${game.gameId})`));
+    if (Number.isInteger(picked) && games[picked]) gameId = games[picked].gameId;
+  }
+
   if (!gameId) {
     if (games.length === 1) {
       gameId = games[0].gameId;

package/src/commands/profile.js

@@ -1,6 +1,6 @@
 import { randomUUID } from 'node:crypto';
 import { mkdir } from 'node:fs/promises';
-import { optionsFrom } from '../run/args.js';
+import { optionsFrom, unknownOptionNotes } from '../run/args.js';
 import { getDeps } from '../run/deps.js';
 import EXIT from '../run/exit.js';
 import { createHeartbeat } from '../run/heartbeat.js';
@@ -55,6 +55,7 @@ async function profile(flags, injected, deps, log) {
   const { options, unknown } = optionsFrom(flags);
   if (unknown.length > 0) {
     log(`I do not know these options: ${unknown.join(', ')}.`);
+    for (const line of unknownOptionNotes(unknown)) log(line);
     return EXIT.ENVIRONMENT;
   }
 

package/src/commands/run.js

@@ -2,7 +2,7 @@ import { randomUUID } from 'node:crypto';
 import { mkdir, readFile, stat } from 'node:fs/promises';
 import path from 'node:path';
 import { buildAggregate, writeAggregate } from '../run/aggregate.js';
-import { optionsFrom } from '../run/args.js';
+import { optionsFrom, unknownOptionNotes } from '../run/args.js';
 import { getDeps } from '../run/deps.js';
 import { createDriver } from '../run/drivers/index.js';
 import EXIT from '../run/exit.js';
@@ -80,6 +80,7 @@ async function run(flags, injected, deps, log) {
   const { options, unknown } = optionsFrom(flags);
   if (unknown.length > 0) {
     log(`I do not know these options: ${unknown.join(', ')}. Run "ravensight-playtest run --help".`);
+    for (const line of unknownOptionNotes(unknown)) log(line);
     return EXIT.ENVIRONMENT;
   }
   for (const line of unusedFlagWarnings(options)) log(line);
@@ -294,17 +295,17 @@ export function personaCountRefusal(count, maxPersonas) {
 /**
  * Flags this release parses and then does not act on.
  *
- * Saying so is the point: a developer who passes `--budget-usd 1` and is not
- * told it did nothing will believe they capped the spend.
+ * Saying so is the point: a developer who passes a flag and is not told it
+ * did nothing will believe it did something. `--budget-usd` is not here
+ * because it is not parsed at all any more: the model budget is the server's,
+ * derived from what the job was charged and enforced per call at the proxy,
+ * so a spending flag the CLI cannot honour is refused rather than accepted.
  *
  * @param {Object} options
  * @returns {Array<string>}
  */
 export function unusedFlagWarnings(options) {
   const warnings = [];
-  if (options.budgetUsd !== undefined) {
-    warnings.push('--budget-usd is not applied by this release: the model budget is the server\'s, derived from what the job was charged, and it is enforced per call at the proxy.');
-  }
   if (options.quality !== undefined) {
     warnings.push('--quality is not applied by this release: the model tier is the server\'s decision, and the CLI is not allowed to name one.');
   }
@@ -392,6 +393,7 @@ export async function playJob({
           journal,
           inFlight,
           makeDriver,
+          brief,
           log
         }));
       }
@@ -474,6 +476,7 @@ export async function playPersonaRun({
   journal,
   inFlight = null,
   makeDriver,
+  brief = null,
   log
 }) {
   const runId = entry.run_id;
@@ -486,7 +489,7 @@ export async function playPersonaRun({
   // Headless is right for a browser and wrong for Godot: a headless Godot
   // routes no GUI input, so it is a smoke-test mode rather than a way to play.
   const headless = driverName === 'godot_driver' ? options.headless === true : options.headed !== true;
-  const driver = await makeDriver(driverName, { headless });
+  const driver = await makeDriver(driverName, driverOptions({ headless, driverName, target, directory, entry, jobId }));
   // The ledger is created here rather than inside the loop so the heartbeat
   // can feed it the server's own figure for what this run has spent. Prices
   // are server side only, so that answer is the only real one.
@@ -535,7 +538,9 @@ export async function playPersonaRun({
       target,
       runDirectory: directory,
       skill,
-      brief: pack.brief || null,
+      // The brief this command fetched and checked is complete, not the
+      // pack's copy, which a pack bundle does not always carry.
+      brief: brief || pack.brief || null,
       intentNotes: pack.intent_notes || [],
       schemas: pack.schemas || [],
       maxActions,
@@ -587,10 +592,10 @@ export async function playPersonaRun({
  * Upload the run directory and complete the run, through cli-core's
  * `finalizeRun` so the 422 `upload_unverified` repair pass lives in one place.
  *
- * A budget-capped or cancelled run sends no `report`: the route accepts that,
- * and it is the honest answer. Inventing a sentiment score nobody gave so a
- * placeholder report would validate would put a fabricated verdict in front of
- * the developer.
+ * A cancelled run sends no `report`. A budget-capped run sends the partial
+ * report built from its notes (`partial: true`, placeholder sentiment the
+ * report itself marks as no verdict), because the notes are what the run
+ * bought before the money ran out.
  *
  * A refused delivery does not take the job down. `finish` still settles it, the
  * artifacts are on disk, and `resume` can retry.
@@ -632,15 +637,27 @@ export async function deliver({ deps, api, gameId, jobId, runId, directory, outc
   }
 
   try {
-    const answer = await deps.finalizeRun(directory, {
-      api,
-      gameId,
-      jobId,
-      runId,
-      includeVideo: Boolean(options.uploadVideo),
-      includeTranscript: Boolean(options.uploadTranscript),
-      body
-    });
+    let answer;
+    try {
+      answer = await deps.finalizeRun(directory, {
+        api,
+        gameId,
+        jobId,
+        runId,
+        includeVideo: Boolean(options.uploadVideo),
+        includeTranscript: Boolean(options.uploadTranscript),
+        body
+      });
+    } catch (error) {
+      // A server older than partial reports refuses one as `invalid_report`.
+      // The uploads are already done by then, so the run is completed
+      // without it rather than left open: the notes stay on disk in
+      // report.json either way.
+      if (!(body.report && body.report.partial && error && error.status === 422 && error.code === 'invalid_report')) throw error;
+      log(`run ${runId}: the server refused the partial report (${String(error.message)}), so the run is completed without it. The notes are in ${directory}.`);
+      const { report: ignored, report_md: alsoIgnored, ...withoutReport } = body;
+      answer = await api.runs.complete(gameId, jobId, runId, withoutReport);
+    }
     if (journal) await stateFile.patchRun(repoRoot, journal, runId, { uploaded: true, completed: true });
     if (answer && answer.dropped_findings > 0) {
       log(`run ${runId}: the server dropped ${answer.dropped_findings} finding(s) with no usable evidence.`);
@@ -919,12 +936,41 @@ export async function resolveSetup(deps, options, log) {
   };
 }
 
-async function loadRunReports(repoRoot, jobId, personaRuns) {
+/**
+ * What a driver is created with. The playtest identity is the run's own: the
+ * Godot driver passes it into the game's environment and the web driver into
+ * the page, and the game's SDK sends the token on `POST /session`, which is
+ * what makes the server force the device id and mark the session synthetic.
+ * Without it the game's own events from a playtest count as a real player.
+ *
+ * @returns {Object}
+ */
+export function driverOptions({ headless, driverName, target, directory, entry, jobId }) {
+  return {
+    headless,
+    playtest: {
+      token: entry.playtest_token || null,
+      runId: entry.run_id,
+      jobId,
+      persona: entry.persona || null
+    },
+    projectDir: driverName === 'godot_driver' ? target : undefined,
+    runDir: directory
+  };
+}
+
+/**
+ * The persona reports on disk, for the aggregate. A partial report (written
+ * from the notes when the money ran out) is loaded as a `budget_exceeded`
+ * run: its findings count, and the aggregate says the job is partial.
+ */
+export async function loadRunReports(repoRoot, jobId, personaRuns) {
   const out = [];
   for (const entry of personaRuns) {
     const file = path.join(runDirFor(repoRoot, jobId, entry.run_id), RUN_FILES.reportJson);
     try {
-      out.push({ runId: entry.run_id, persona: entry.persona, state: 'succeeded', report: JSON.parse(await readFile(file, 'utf8')) });
+      const report = JSON.parse(await readFile(file, 'utf8'));
+      out.push({ runId: entry.run_id, persona: entry.persona, state: report && report.partial === true ? 'budget_exceeded' : 'succeeded', report });
     } catch {
       out.push({ runId: entry.run_id, persona: entry.persona, state: 'failed', report: null });
     }

package/src/run/aggregate.js

@@ -344,7 +344,9 @@ export async function buildAggregate({
   const issues = clusters.flatMap(cluster => toIssues(cluster, verdicts.get(cluster.signature)));
   const ranked = rankIssues(issues, personasRun);
 
-  const partial = runs.some(run => !run.report || ['failed', 'canceled', 'budget_exceeded'].includes(run.state));
+  // A run whose report is partial (the CLI wrote it from the notes when the
+  // money ran out) counts toward the issues, and makes the job partial.
+  const partial = runs.some(run => !run.report || run.report.partial === true || ['failed', 'canceled', 'budget_exceeded'].includes(run.state));
   const judgement = aggregateRunId
     ? await judgeAggregate({ client, jobId, runId: aggregateRunId, ranked, inputs, brief, intentNotes, ledger, log })
     : null;
@@ -374,8 +376,11 @@ export async function buildAggregate({
       state: run.state,
       actions_taken: run.report.actions_taken,
       quit_reason: run.report.quit_reason,
-      sentiment: run.report.sentiment,
-      would_continue: run.report.would_continue
+      // A partial report's sentiment is the schema's placeholder, not a
+      // persona verdict, so it is not shown as one.
+      sentiment: run.report.partial ? null : run.report.sentiment,
+      would_continue: run.report.partial ? null : run.report.would_continue,
+      ...(run.report.partial ? { partial: true } : {})
     })),
     issues: applied.issues,
     praise: ranked.praise,
@@ -450,7 +455,7 @@ export async function judgeAggregate({ client, jobId, runId, ranked, inputs, bri
     '',
     `issues:\n${ranked.issues.map(issue => `  ${issue.issue_id} [${issue.severity}/${issue.category}] ${issue.title}: ${String(issue.description || '').slice(0, 300)} (personas: ${issue.personas_affected.join(', ') || 'none'})`).join('\n') || '  (none)'}`,
     `praise:\n${ranked.praise.map(issue => `  ${issue.issue_id} ${issue.title}`).join('\n') || '  (none)'}`,
-    `runs:\n${inputs.map(run => `  ${run.persona}: ${run.report.actions_taken} actions, ${run.report.quit_reason}, would_continue=${run.report.would_continue}`).join('\n') || '  (none)'}`
+    `runs:\n${inputs.map(run => `  ${run.persona}: ${run.report.actions_taken} actions, ${run.report.quit_reason}, ${run.report.partial ? 'partial (out of budget, no verdict)' : `would_continue=${run.report.would_continue}`}`).join('\n') || '  (none)'}`
   ].join('\n');
 
   try {
@@ -582,7 +587,7 @@ export function renderAggregateMarkdown(json) {
   lines.push('|---|---|---|---|---|---|---|');
   for (const run of json.runs) {
     const sentiment = run.sentiment || {};
-    lines.push(`| ${run.persona} | ${run.actions_taken} | ${run.quit_reason} | ${sentiment.fun ?? '-'} | ${sentiment.clarity ?? '-'} | ${sentiment.frustration ?? '-'} | ${run.would_continue ? 'yes' : 'no'} |`);
+    lines.push(`| ${run.persona} | ${run.actions_taken} | ${run.quit_reason} | ${sentiment.fun ?? '-'} | ${sentiment.clarity ?? '-'} | ${sentiment.frustration ?? '-'} | ${run.would_continue === null || run.would_continue === undefined ? '-' : run.would_continue ? 'yes' : 'no'} |`);
   }
   lines.push('');
   lines.push('## Top issues (ranked)');

package/src/run/args.js

@@ -22,7 +22,6 @@ const VALUED = new Map([
   ['--cli', 'cli'],
   ['--max-actions', 'maxActions'],
   ['--concurrency', 'concurrency'],
-  ['--budget-usd', 'budgetUsd'],
   ['--quality', 'quality']
 ]);
 
@@ -41,7 +40,7 @@ const BOOLEAN = new Map([
 ]);
 
 /** Options whose value is a number. */
-const NUMERIC = new Set(['maxActions', 'concurrency', 'budgetUsd']);
+const NUMERIC = new Set(['maxActions', 'concurrency']);
 
 /**
  * @param {string[]} argv the leftovers commander handed over
@@ -77,6 +76,21 @@ export function parseRunArgv(argv = []) {
   return { options, unknown };
 }
 
+/**
+ * Extra lines for unknown options that used to exist, so a removed flag is
+ * explained rather than just refused.
+ *
+ * @param {string[]} unknown
+ * @returns {string[]}
+ */
+export function unknownOptionNotes(unknown = []) {
+  const notes = [];
+  if (unknown.includes('--budget-usd')) {
+    notes.push('--budget-usd was removed: the model budget is set by the server for each run, from what the job was charged, and enforced at the proxy.');
+  }
+  return notes;
+}
+
 /**
  * Merge the flags cli-core parsed with the ones this file parsed, flags
  * winning, and normalise the two names for the repository root.
@@ -115,4 +129,4 @@ function stripUndefined(value) {
   return Object.fromEntries(Object.entries(value).filter(([, entry]) => entry !== undefined));
 }
 
-export default { parseRunArgv, optionsFrom };
+export default { parseRunArgv, optionsFrom, unknownOptionNotes };

package/src/run/context.js

@@ -31,6 +31,13 @@ export const MAX_IMAGES_IN_CONTEXT = 1;
 /** How many full observations stay in context (spec 03's K). */
 export const KEEP_OBSERVATIONS = 6;
 
+/**
+ * What an observation identical to the one before it is sent as. The driver
+ * marks it (`observation.unchanged`) and ./tools.js writes this in place of
+ * the screen, keeping the `step N` line so it is still an observation.
+ */
+export const UNCHANGED_OBSERVATION_TEXT = '(unchanged)';
+
 const DROPPED_IMAGE_TEXT = '[screenshot dropped from context; only the newest screen is kept]';
 const DROPPED_OBSERVATION_TEXT = '[older observation dropped from context; ask for a snapshot if you need to look again]';
 
@@ -138,7 +145,11 @@ export function pruneObservations(messages, keep = KEEP_OBSERVATIONS) {
     message.content.forEach((block, j) => {
       if (block && block.type === 'tool_result' && Array.isArray(block.content)) {
         const first = block.content.find(child => child && child.type === 'text');
-        if (first && /^(?:[^\n]*\n)?step \d+/.test(String(first.text))) indexes.push([i, j]);
+        // An unchanged marker is not counted: it is a few bytes, and counting
+        // it would let a run of identical screens push the last REAL screen
+        // out of context, leaving the persona nothing to act on.
+        const text = String(first && first.text);
+        if (first && /^(?:[^\n]*\n)?step \d+/.test(text) && !text.endsWith(`\n${UNCHANGED_OBSERVATION_TEXT}`)) indexes.push([i, j]);
       }
     });
   });

package/src/run/drivers/web.js

@@ -2,7 +2,6 @@ import { mkdir, rename, stat, readdir } from 'node:fs/promises';
 import path from 'node:path';
 import {
   DriverError,
-  OBSERVATION_MAX_CHARS,
   THIN_OBSERVATION_CHARS,
   observationHash,
   screenshotName,
@@ -36,6 +35,87 @@ export const MAX_WAIT_MS = 10000;
 /** How long a single Playwright call may take before it is an error, not a hang. */
 export const ACTION_TIMEOUT_MS = 15000;
 
+/**
+ * The web observation ceiling. An aria snapshot is re-sent on every turn the
+ * observation stays in context, and the run's model budget is fixed, so a busy
+ * page at the shared 12,000 character ceiling spends the budget on markup the
+ * persona never needed. Compaction (below) usually lands well under this.
+ */
+export const WEB_OBSERVATION_MAX_CHARS = 4000;
+
+/**
+ * Full observations kept in context for a web run. The newest screen is what
+ * the persona acts on; one more is enough to compare against, and every older
+ * one is a stub.
+ */
+export const WEB_KEEP_OBSERVATIONS = 2;
+
+/** Roles a player can act on. Kept even with no name, since they are still clickable. */
+const INTERACTIVE_ROLES = new Set([
+  'button', 'link', 'textbox', 'searchbox', 'checkbox', 'radio', 'combobox', 'listbox', 'option',
+  'slider', 'spinbutton', 'switch', 'tab', 'menuitem', 'menuitemcheckbox', 'menuitemradio', 'treeitem',
+  'canvas'
+]);
+
+/**
+ * Keep the nodes a player could act on or read, and drop the structure in
+ * between. An aria snapshot prints every wrapper (`- generic:`, `- list:`, a
+ * nameless `- img`) and the attribute lines under links (`- /url: ...`), none
+ * of which tell a persona anything, and all of which cost tokens every turn.
+ *
+ * Indentation is dropped too: the persona targets nodes by role and name, not
+ * by depth. Text that is not an aria snapshot (the innerText fallback for a
+ * thin page) is returned unchanged.
+ *
+ * @param {string} text
+ * @returns {string}
+ */
+export function compactAriaSnapshot(text) {
+  const value = String(text == null ? '' : text);
+  const lines = value.split('\n');
+  if (!lines.some(line => /^\s*- /.test(line))) return value;
+  const kept = [];
+  for (const line of lines) {
+    const trimmed = line.trim();
+    if (!trimmed.startsWith('- ')) continue;
+    const body = unquoteKey(trimmed.slice(2));
+    if (body.startsWith('/')) continue;
+    const role = (body.match(/^([a-z]+)/) || [])[1] || '';
+    const named = /^[a-z]+ "[^"]*"/.test(body);
+    const inlineText = /^[a-z]+(?: "[^"]*")?(?: \[[^\]]*\])*: \S/.test(body);
+    if (INTERACTIVE_ROLES.has(role) || named || inlineText) kept.push(`- ${body}`);
+  }
+  return kept.join('\n');
+}
+
+/**
+ * Playwright writes a node key as a single quoted YAML scalar when the name
+ * contains `: `, ` #`, a brace or a backtick: `- 'button "Player #1"'`. Strip
+ * that outer pair (a doubled `''` inside is one quote) so the node is
+ * classified and shown like any other, keeping whatever followed the closing
+ * quote (a `:` for a node with children, or `: text`).
+ *
+ * @param {string} body the line after its `- `
+ * @returns {string}
+ */
+export function unquoteKey(body) {
+  if (!body.startsWith("'")) return body;
+  let out = '';
+  for (let i = 1; i < body.length; i += 1) {
+    if (body[i] === "'") {
+      if (body[i + 1] === "'") {
+        out += "'";
+        i += 1;
+        continue;
+      }
+      return out + body.slice(i + 1);
+    }
+    out += body[i];
+  }
+  // No closing quote: not a quoted key after all.
+  return body;
+}
+
 const KNOWN_SELECTOR_ENGINES = /^(role|text|css|xpath|id|data-testid|internal:)=/;
 
 /**
@@ -127,12 +207,33 @@ export function resolveLocator(page, target) {
   return page.getByText(value, { exact: false }).first();
 }
 
+/**
+ * Hand the run's playtest identity to the page. Runs in the browser, before
+ * any of the game's own scripts, so the Ravensight JS SDK finds it at
+ * construction. Serialised by Playwright, so it must not close over anything.
+ *
+ * @param {{token: string, runId: string, jobId: string, persona: string}} identity
+ */
+function injectPlaytestIdentity(identity) {
+  window.__RAVENSIGHT_PLAYTEST__ = identity;
+  try {
+    localStorage.setItem('ravensight.playtest', JSON.stringify(identity));
+  } catch {
+    // Storage can be refused (a sandboxed frame, a privacy mode). The window
+    // global is the first thing the SDK reads, so this is only the fallback.
+  }
+}
+
 /**
  * @param {Object} opts
  * @param {(specifier: string) => Promise<any>} [opts.importer] test seam
+ * @param {boolean} [opts.headless]
+ * @param {{token?: string, runId?: string, jobId?: string, persona?: string}} [opts.playtest]
+ *   the run's playtest identity, handed to the page so the game's own SDK
+ *   session is marked synthetic
  * @returns {Object} a driver
  */
-export function createWebDriver({ importer, headless = true } = {}) {
+export function createWebDriver({ importer, headless = true, playtest = null } = {}) {
   let browser = null;
   let context = null;
   let page = null;
@@ -142,6 +243,7 @@ export function createWebDriver({ importer, headless = true } = {}) {
   let step = 0;
   let shots = 0;
   let lastHash = null;
+  let lastObservedHash = null;
   const consoleBuffer = [];
 
   /** @returns {any} */
@@ -150,9 +252,15 @@ export function createWebDriver({ importer, headless = true } = {}) {
     return page;
   }
 
+  /**
+   * @returns {Promise<{text: string, aria: boolean}>} the screen text, and
+   *   whether it is the aria snapshot (compacted) or the innerText fallback
+   *   (never compacted: a bullet list in the page is not snapshot structure)
+   */
   async function snapshotText() {
     const current = requirePage();
     let text = '';
+    let aria = true;
     const body = current.locator('body');
     if (typeof body.ariaSnapshot === 'function') {
       try {
@@ -167,17 +275,31 @@ export function createWebDriver({ importer, headless = true } = {}) {
           const el = globalThis.document && globalThis.document.body;
           return el ? el.innerText : '';
         });
-        if (String(inner).trim().length > text.trim().length) text = String(inner);
+        if (String(inner).trim().length > text.trim().length) {
+          text = String(inner);
+          aria = false;
+        }
       } catch {
         // A page mid-navigation has no body to read. The thin observation
         // is the honest answer, and the screenshot fallback below covers it.
       }
     }
-    return text;
+    return { text, aria };
+  }
+
+  /**
+   * What the model is shown for a raw snapshot. `observe` and `act` both use
+   * this, so the two hashes compare like with like.
+   * @param {{text: string, aria: boolean}} snap
+   * @returns {string}
+   */
+  function shownText(snap) {
+    return truncateObservation(snap.aria ? compactAriaSnapshot(snap.text) : snap.text, WEB_OBSERVATION_MAX_CHARS);
   }
 
   return {
     name: 'playwright_web',
+    keepObservations: WEB_KEEP_OBSERVATIONS,
 
     /**
      * @param {Object} opts
@@ -200,6 +322,18 @@ export function createWebDriver({ importer, headless = true } = {}) {
         viewport,
         ...(videoRequested ? { recordVideo: { dir: runDir, size: viewport } } : {})
       });
+      // The playtest identity goes in as an init script, not as
+      // `extraHTTPHeaders`: a context header rides on every request the page
+      // makes, including to the game's own API, and the token is only for
+      // Ravensight. The SDK reads it from the page and sends it itself.
+      if (playtest && playtest.token) {
+        await context.addInitScript(injectPlaytestIdentity, {
+          token: playtest.token,
+          runId: playtest.runId || null,
+          jobId: playtest.jobId || null,
+          persona: playtest.persona || null
+        });
+      }
       page = await context.newPage();
       page.on('console', message => {
         if (message.type() === 'error') consoleBuffer.push({ type: 'console', text: message.text() });
@@ -219,10 +353,11 @@ export function createWebDriver({ importer, headless = true } = {}) {
       const current = requirePage();
       const started = Date.now();
       step += 1;
-      const text = await snapshotText();
+      const snap = await snapshotText();
+      const text = snap.text;
       const observation = {
         step,
-        text: truncateObservation(text, OBSERVATION_MAX_CHARS),
+        text: shownText(snap),
         url: current.url(),
         elapsedMs: Date.now() - started
       };
@@ -244,6 +379,11 @@ export function createWebDriver({ importer, headless = true } = {}) {
       // point would otherwise read as changed to `act` and identical to
       // anything reading the observation.
       lastHash = observationHash(observation);
+      // The same screen twice is sent as a marker rather than in full
+      // (./tools.js), which is most of what a stuck persona would otherwise
+      // pay for.
+      if (lastHash === lastObservedHash) observation.unchanged = true;
+      lastObservedHash = lastHash;
       return observation;
     },
 
@@ -293,9 +433,9 @@ export function createWebDriver({ importer, headless = true } = {}) {
       try {
         // The same truncation observe() applies, so the two hashes are
         // comparable. Without it a long screen is always "changed".
-        const text = truncateObservation(await snapshotText(), OBSERVATION_MAX_CHARS);
-        const after = { text };
-        if (text.trim().length < THIN_OBSERVATION_CHARS) {
+        const snap = await snapshotText();
+        const after = { text: shownText(snap) };
+        if (String(snap.text).trim().length < THIN_OBSERVATION_CHARS) {
           const frame = await current.screenshot({ type: 'png', timeout: ACTION_TIMEOUT_MS });
           after.screenshotBase64 = frame.toString('base64');
         }

package/src/run/personaLoop.js

@@ -3,6 +3,7 @@ import { mkdir, writeFile } from 'node:fs/promises';
 import path from 'node:path';
 import { prepareMessages, MAX_IMAGES_IN_CONTEXT, KEEP_OBSERVATIONS, shrinkImages } from './context.js';
 import { observationHash } from './drivers/driver.js';
+import { withRollingCacheBreakpoint } from './fixLoop.js';
 import { callStepWithRetry, ProxyError, STEPS } from './model.js';
 import { RUN_FILES } from './paths.js';
 import { checkEvidence, createReportValidator, lintMarkdownSections, repairInstruction, REQUIRED_MARKDOWN_SECTIONS, stampReport, writeReportFiles } from './report.js';
@@ -32,11 +33,30 @@ import { createUsageLedger, writeUsageFile } from './usage.js';
  *   up, so the report (which is the product) gets written while there is
  *   still budget to write it with.
  * - A 402 means the money is gone. Nothing further is asked of the model,
- *   including the report: a partial `report.md` is written locally from the
- *   notes, `complete` is sent with `state: budget_exceeded` and no
- *   `report`, and the process exits 2.
+ *   including the report: a partial `report.md` and `report.json` are
+ *   written locally from the notes (`partial: true`, each note an
+ *   unreviewed finding), `complete` is sent with `state: budget_exceeded`
+ *   and that report, and the process exits 2. The notes are the one thing
+ *   the run bought before the money ran out, so they are not thrown away.
  */
 
+/**
+ * Order across notes and screenshots, so a partial report can attach the
+ * screenshot nearest to each note. Module wide and only ever compared within
+ * one run, where it is monotonic.
+ */
+let ORDER = 0;
+const nextOrder = () => {
+  ORDER += 1;
+  return ORDER;
+};
+
+/** `Failed to load resource: ... status of 4xx`: a missing asset or a refused call, not a crash. */
+const RESOURCE_4XX = /Failed to load resource: the server responded with a status of 4\d\d/;
+
+/** A persona note kind as a report finding category. */
+export const NOTE_CATEGORY = Object.freeze({ bug: 'bug', confusing: 'ux', suggestion: 'ux', good: 'ux', reaction: 'ux' });
+
 /** Stop injecting the "nothing changed" hint after this many identical screens. */
 export const STUCK_HINT_AFTER = 5;
 
@@ -89,7 +109,9 @@ export async function runPersona(args) {
     schemas,
     maxActions,
     maxImages = MAX_IMAGES_IN_CONTEXT,
-    keepObservations = KEEP_OBSERVATIONS,
+    // A driver may say how many observations it needs in context: a web
+    // snapshot is big and re-sent every turn, a Godot scene summary is not.
+    keepObservations = (driver && Number.isInteger(driver.keepObservations) ? driver.keepObservations : KEEP_OBSERVATIONS),
     heartbeat = null,
     onProgress = () => {},
     log = () => {},
@@ -188,7 +210,15 @@ export async function runPersona(args) {
       outcome.quitReason = 'cost_cap';
       // Nothing more is asked of the model: there is no budget left to ask
       // with, and a refused call would just be another 402.
-      await writePartialMarkdown({ runDirectory, persona, driverName, target, notes, screenshots, play, reason: 'The run reached its model budget before the report step. What the persona saw is below; there is no report.json for this run.' });
+      // A report that cannot be written (a secret shaped note, a full disk)
+      // must not turn an out of budget run into a failed one.
+      try {
+        await writePartialReport({ outcome, runDirectory, persona, driverName, target, notes, screenshots, play, skill, ledger, runId, jobId, startedAt, endedAt });
+      } catch (writeError) {
+        outcome.report = null;
+        outcome.reportMarkdown = null;
+        log(`the partial report could not be written: ${String(writeError && writeError.message)}`);
+      }
       return finish(outcome, ledger, runDirectory, transcript, heartbeat);
     }
 
@@ -230,6 +260,14 @@ export async function runPersona(args) {
     if (error instanceof ProxyError && error.isBudget) {
       outcome.state = 'budget_exceeded';
       outcome.quitReason = 'cost_cap';
+      // The money ran out in the report step itself: the notes are still
+      // worth keeping, by the same rule as a 402 during play.
+      try {
+        const play = { actionsTaken: outcome.actionsTaken, quitReason: 'cost_cap', quitDetail: String(error.message || '') };
+        await writePartialReport({ outcome, runDirectory, persona, driverName, target, notes, screenshots, play, skill, ledger, runId, jobId, startedAt, endedAt: new Date().toISOString() });
+      } catch (writeError) {
+        log(`the partial report could not be written: ${String(writeError && writeError.message)}`);
+      }
     } else if (error instanceof ProxyError && error.isClosed) {
       outcome.state = 'canceled';
       outcome.quitReason = 'error';
@@ -420,11 +458,11 @@ async function playLoop({
           // (bug, confusing, good, suggestion, reaction) would otherwise
           // overwrite the transcript line's `kind`, and every note would
           // read as a different sort of line.
-          notes.push({ ...note, at_action: actions });
+          notes.push({ ...note, at_action: actions, order: nextOrder() });
           transcript.write({ kind: 'note', tool: toolUse.name, note });
         },
         onScreenshot: shot => {
-          screenshots.push(shot);
+          screenshots.push({ ...shot, order: nextOrder() });
           transcript.write({ kind: 'screenshot', path: shot.relativePath, reason: shot.reason });
         }
       });
@@ -466,13 +504,17 @@ async function playLoop({
 
     const consoleErrors = typeof driver.consoleErrors === 'function' ? await driver.consoleErrors() : [];
     if (consoleErrors.length > 0) {
-      // An uncaught error in the game is a finding whether or not the
+      // An uncaught error in the game is worth recording whether or not the
       // persona noticed it, so it is logged and evidenced automatically
-      // (spec 03). It is the model's job to decide the severity.
+      // (spec 03). It carries no severity: the report step decides that,
+      // with the persona's view of the screen in front of it. A 4xx on a
+      // resource is a `network` note, which the report step reads as context
+      // rather than as a bug (a missing favicon is not a broken game).
       const shot = await safeScreenshot(driver, 'console-error', screenshots, transcript);
       for (const entry of consoleErrors) {
-        const note = { kind: 'bug', severity: 'major', text: `${entry.type}: ${entry.text}`, automatic: true };
-        notes.push(note);
+        const kind = RESOURCE_4XX.test(String(entry.text)) ? 'network' : 'bug';
+        const note = { kind, text: `${entry.type}: ${entry.text}`, automatic: true };
+        notes.push({ ...note, at_action: actions, order: nextOrder() });
         transcript.write({ kind: 'note', note });
       }
       asides.push(`The game logged ${consoleErrors.length} error(s): ${consoleErrors.map(entry => entry.text).join(' | ').slice(0, 800)}${shot ? ` A screenshot is saved at ${shot.relativePath}.` : ''}`);
@@ -560,6 +602,8 @@ async function reportPhase({
         'Notes you logged, in order:',
         notes.length > 0 ? notes.map((note, i) => `${i + 1}. [${note.kind}${note.severity ? `/${note.severity}` : ''}] ${note.text}`).join('\n') : '(none)',
         '',
+        'Notes marked automatic were recorded by the runner from the console, with no severity: decide one from what the persona saw. Network notes are context, not bugs: a failed request is a finding only when the persona saw a broken screen because of it.',
+        '',
         'Screenshots available as evidence (use these exact paths in evidence refs):',
         screenshots.length > 0 ? screenshots.map(shot => `- ${shot.relativePath}${shot.reason ? ` (${shot.reason})` : ''}`).join('\n') : '(none)',
         '',
@@ -685,7 +729,10 @@ async function reportPhase({
 async function callWithImageRetry({ imageBudget, keepObservations, messages, onImageBudget, ...rest }) {
   let budget = imageBudget;
   for (let attempt = 0; attempt < 3; attempt += 1) {
-    const prepared = prepareMessages(messages, { maxImages: budget, keepObservations });
+    // One moving cache breakpoint on the newest block, as the fix loop does:
+    // the prefix up to it is what the next turn re-sends, so the next turn
+    // reads it from the cache instead of paying for it again.
+    const prepared = withRollingCacheBreakpoint(prepareMessages(messages, { maxImages: budget, keepObservations }));
     try {
       return await callStepWithRetry({ ...rest, messages: prepared });
     } catch (error) {
@@ -801,7 +848,7 @@ function redactToolInput(toolUse) {
 async function safeScreenshot(driver, name, screenshots, transcript) {
   try {
     const shot = await driver.screenshot(name);
-    screenshots.push({ ...shot, reason: name });
+    screenshots.push({ ...shot, reason: name, order: nextOrder() });
     transcript.write({ kind: 'screenshot', path: shot.relativePath, reason: name });
     return shot;
   } catch (error) {
@@ -893,6 +940,104 @@ export function renderFallbackMarkdown({ persona, driverName, target, notes, scr
   ].join('\n');
 }
 
+/**
+ * The screenshot nearest a note, by the order they happened in; a tie goes
+ * to the later one, since a persona usually notes something and then
+ * captures it.
+ *
+ * @param {Object} note
+ * @param {Array<Object>} screenshots
+ * @returns {Object|null}
+ */
+export function nearestScreenshot(note, screenshots) {
+  let best = null;
+  let bestDistance = Infinity;
+  for (const shot of screenshots) {
+    if (!shot || !shot.relativePath || !Number.isFinite(shot.order) || !Number.isFinite(note.order)) continue;
+    const distance = Math.abs(shot.order - note.order);
+    if (distance < bestDistance || (distance === bestDistance && best && shot.order > best.order)) {
+      best = shot;
+      bestDistance = distance;
+    }
+  }
+  return best || screenshots.find(shot => shot && shot.relativePath) || null;
+}
+
+/**
+ * The persona's notes as report findings, for a run with no report step.
+ *
+ * Nothing is judged here, because there is no model turn to judge with:
+ * the persona's own kind becomes the category, its own severity is kept
+ * when it gave one and is `minor` otherwise (`praise` for a `good` note), and
+ * a network note is left out, since it is context rather than a finding.
+ *
+ * @param {Array<Object>} notes
+ * @param {Array<Object>} screenshots
+ * @returns {Array<Object>}
+ */
+export function notesToFindings(notes, screenshots) {
+  const findings = [];
+  for (const note of notes) {
+    const category = NOTE_CATEGORY[note.kind];
+    if (!category || !note.text) continue;
+    const text = String(note.text);
+    const shot = nearestScreenshot(note, screenshots);
+    findings.push({
+      id: `F${String(findings.length + 1).padStart(3, '0')}`,
+      severity: note.kind === 'good' ? 'praise' : (note.severity && note.severity !== 'praise' ? note.severity : 'minor'),
+      category,
+      title: text.length > 120 ? `${text.slice(0, 117)}...` : text,
+      description: text,
+      repro_steps: [`Seen after ${Number(note.at_action) || 0} action(s) in this run.`],
+      evidence: shot
+        ? [{ type: 'screenshot', ref: shot.relativePath }]
+        : [{ type: 'transcript_step', ref: `action ${Number(note.at_action) || 0}` }]
+    });
+  }
+  return findings;
+}
+
+/**
+ * The report for a run the money ran out on: the notes as findings,
+ * `partial: true`, and both files written. The sentiment is the scale's
+ * midpoint and `would_continue` is false, because the schema requires both
+ * and no persona verdict was recorded; `partial` is what tells a reader not
+ * to read them as one.
+ */
+async function writePartialReport({ outcome, runDirectory, persona, driverName, target, notes, screenshots, play, skill, ledger, runId, jobId, startedAt, endedAt }) {
+  const report = stampReport({
+    findings: notesToFindings(notes, screenshots),
+    sentiment: { fun: 3, clarity: 3, frustration: 3 },
+    would_continue: false,
+    partial: true
+  }, {
+    runId,
+    jobId,
+    persona: persona.slug,
+    driver: driverName,
+    target,
+    startedAt,
+    endedAt,
+    actionsTaken: play.actionsTaken,
+    usage: ledger.toReportUsage(),
+    promptVersion: promptVersionOf(skill) || 'unknown',
+    quitReason: 'cost_cap',
+    quitDetail: play.quitDetail
+  });
+  const markdown = renderFallbackMarkdown({
+    persona,
+    driverName,
+    target,
+    notes,
+    screenshots,
+    play,
+    reason: 'The run reached its model budget before the report step. What the persona saw is below, and report.json carries the notes as unreviewed findings.'
+  });
+  await writeReportFiles({ runDirectory, markdown, report });
+  outcome.report = report;
+  outcome.reportMarkdown = markdown;
+}
+
 async function writePartialMarkdown({ runDirectory, persona, driverName, target, notes, screenshots, play, reason }) {
   const markdown = renderFallbackMarkdown({ persona, driverName, target, notes, screenshots, play, reason });
   assertNoSecrets('report.md', markdown);
@@ -911,4 +1056,4 @@ export function promptVersionOf(skill) {
   return match ? match[1] : null;
 }
 
-export default { runPersona, buildSystem, promptVersionOf, shouldEscalate, renderFallbackMarkdown };
+export default { runPersona, buildSystem, promptVersionOf, shouldEscalate, renderFallbackMarkdown, notesToFindings, nearestScreenshot };

package/src/run/tools.js

@@ -1,3 +1,4 @@
+import { UNCHANGED_OBSERVATION_TEXT } from './context.js';
 import { OBSERVATION_MAX_CHARS, truncateObservation } from './drivers/driver.js';
 
 /**
@@ -300,13 +301,17 @@ export async function executeTool({ driver, toolUse, onNote, onScreenshot }) {
  * @param {boolean} [isError]
  */
 export function observationResult(id, observation, prefix = '', isError = false) {
+  // The same screen as last time is a marker, not a resend: the persona
+  // already has it in context, and a stuck run would otherwise pay for the
+  // whole snapshot (and its image) on every turn it stays stuck.
+  const body = observation.unchanged ? UNCHANGED_OBSERVATION_TEXT : truncateObservation(observation.text, OBSERVATION_MAX_CHARS);
   const content = [
     {
       type: 'text',
-      text: `${prefix}step ${observation.step}${observation.url ? ` (${observation.url})` : ''}${citationLine(observation)}\n${truncateObservation(observation.text, OBSERVATION_MAX_CHARS)}`
+      text: `${prefix}step ${observation.step}${observation.url ? ` (${observation.url})` : ''}${citationLine(observation)}\n${body}`
     }
   ];
-  if (observation.screenshotBase64) {
+  if (observation.screenshotBase64 && !observation.unchanged) {
     content.push({
       type: 'image',
       source: {

package/src/ui/index.js

@@ -157,6 +157,33 @@ export async function confirm(question, options = {}) {
   }
 }
 
+/**
+ * Ask the person to pick one of a numbered list. Like `confirm`, it never
+ * asks without a terminal: with no TTY (CI, a pipe) it answers null at once,
+ * and so does an answer that is not one of the numbers shown.
+ *
+ * @param {string} question e.g. "Which game? [1-3]"
+ * @param {Array<string>} labels one line per choice, printed numbered from 1
+ * @param {{input?: object, output?: object}} [options]
+ * @returns {Promise<number|null>} the zero based index, or null
+ */
+export async function choose(question, labels, options = {}) {
+  const input = options.input || process.stdin;
+  if (!input.isTTY) return null;
+  const output = options.output || stream;
+
+  labels.forEach((label, index) => output.write(`  ${index + 1}. ${label}\n`));
+  const rl = createInterface({ input, output });
+  try {
+    const answer = (await rl.question(`${question} `)).trim();
+    if (!/^\d+$/.test(answer)) return null;
+    const index = Number(answer) - 1;
+    return index >= 0 && index < labels.length ? index : null;
+  } finally {
+    rl.close();
+  }
+}
+
 /** Cents as dollars, the one place that formatting exists. */
 export function money(cents) {
   if (cents === null || cents === undefined) return 'unknown';
@@ -190,6 +217,7 @@ export const ui = {
   table,
   spinner,
   confirm,
+  choose,
   money,
   bytes
 };