ravensight-playtest 0.2.6 → 0.2.8

package/README.md

@@ -206,6 +206,7 @@ splitting the job in two is what gets everything played.
 | `--upload-video` | Upload `session.webm` as well as the reports. |
 | `--upload-transcript` | Upload `transcript.jsonl` as well as the reports. |
 | `--dry-upload` | List every file and byte that would be sent, and send nothing. |
+| `--key-modifier <meta\|control>` | The modifier the browser driver uses for editing shortcuts such as select all. Defaults to this machine's own (Command on macOS, Control elsewhere); set it when the browser's platform is not this machine's. `RAVENSIGHT_KEY_MODIFIER` does the same. |
 | `--headed`, `--headless` | Show or hide the browser. |
 | `--verbose` | Print each turn. |
 
@@ -223,11 +224,73 @@ from the page: the browser driver sets `window.__RAVENSIGHT_PLAYTEST__` and
 `localStorage['ravensight.playtest']` before the game's scripts run. It is never
 sent as a request header, which would reach your game's own API too.
 
+With no ingest key configured, a Godot run says "no ingest key configured, so
+no synthetic telemetry was sent". A web run says "synthetic telemetry comes from
+the game's own SDK on the page, sent with the run's playtest token" instead,
+because the page's SDK already sends the game's events marked synthetic.
+
 When a run reaches its model budget before the report step, nothing more is
 asked of the model. The persona's notes are written to a partial `report.json`
 (`partial: true`, one unreviewed finding per note with the nearest screenshot as
 evidence) and uploaded, so the job's aggregate still counts them.
 
+When the budget runs out during the report step itself, the same partial report
+is uploaded and the run is completed as `budget_exceeded`. The job does not exit 2 for it, and the CLI
+prints "Run ran out of budget; the partial report was uploaded and the refund
+decision is the server's". When the server's answer says a refund is
+expected, the line ends "a refund of $X.XX is expected when the job settles"
+instead, and "the run was refunded" once the job has settled and the run was
+refunded. If the server refused the partial report, it says so and where the
+notes are instead. An older API that refuses that completion prints its own
+message, the run is completed as `failed`, and the job exits 1.
+
+How the report step reads what happened:
+
+- Console errors are recorded as automatic notes. A 4xx on a request (`Failed
+  to load resource`, `HTTP 404`, `status: 401`, `401 Unauthorized` and similar)
+  is a network note: context, not a bug. A finding whose only evidence is
+  automatic network notes is capped at `minor` by the CLI, and its description
+  says so.
+- Every note carries `after`, the URL and title of the screen that followed
+  it. A blocker or major finding has to cite the observation where the failure
+  is visible and must not be contradicted by that next screen or by a later
+  persona note.
+- The model is told never to write calendar dates. `report.md` carries the
+  CLI's own `Generated at` timestamp under its title, and the aggregate report's
+  header uses its `generated_at`.
+- A template slot the model copied instead of filling (`**Date:** <run date>`,
+  `<driver>`, `<N actions>`) is filled from the CLI's own facts, or "not
+  recorded" when it has none, in `report.md`, the prose of `report.json` and
+  the aggregate. Other angle brackets are left as written: keys such as
+  `<Enter>`, elements such as `<form>`, types such as `Array<string>`, links,
+  code and quoted game text.
+- The server keeps a finding only when one of its evidence references points
+  at a file the run uploaded. After delivery the CLI marks the findings it
+  dropped (by the server's `dropped_finding_ids`, or by applying the same rule
+  to the files it uploaded when its count matches): `dropped: true` in
+  `report.json`, out of the summary in `report.md` and into a section headed
+  "Dropped by the server (no usable evidence)", and out of the aggregate and
+  the synthetic telemetry, so the local report counts what the dashboard
+  counts. When it cannot tell which, `report.md` says how many and that the
+  dashboard is the record. The rewritten files are journalled as local edits,
+  and `upload <job>` never re-sends a completed run's report files.
+
+### What personas do well
+
+A persona plays like a first time player at human speed: menus, onboarding,
+the first few minutes, unclear text, screens that do not respond, and whether
+it would come back. On the web, when a persona replaces a field's value (the
+`type` tool with `replace`), the driver clears it with `fill`, the same way on
+macOS, Windows and Linux; ordinary typing is keystrokes. An editing shortcut
+such as select all uses the platform's own modifier (Command on a Mac, Control
+elsewhere, or `--key-modifier`).
+
+Each persona starts at the game's URL as a fresh player with no account, so a
+sign-up form with validation can use up a whole unit before the game itself is
+reached. Point `--build-url` at a guest or demo path, or say where to start in
+the brief's `how_to_start` note (for example "choose Play as guest"), when the
+sign-up is not what you want tested.
+
 ### fix
 
 Takes one finding, attempts the change on a branch of its own, replays the
@@ -383,7 +446,7 @@ start avoids that.
 |---|---|
 | 0 | Success |
 | 1 | Failure |
-| 2 | Budget exceeded. Partial results were written and uploaded. |
+| 2 | Budget exceeded during play. Partial results were written and uploaded. A run that ran out during the report step and was completed does not make the job exit 2. |
 | 3 | Environment check failed |
 | 4 | Authentication failed |
 | 5 | Model API error after retries |
@@ -399,6 +462,7 @@ CI can treat 2 as a success with a warning.
 | `RAVENSIGHT_API_URL` | The API base URL. Defaults to `https://api.ravensight.io`. |
 | `RAVENSIGHT_DASHBOARD_URL` | Where the dashboard is, if it is not derivable from the API URL. |
 | `RAVENSIGHT_GAME_ID` | The game, when there is no linked repo. |
+| `RAVENSIGHT_KEY_MODIFIER` | `meta` or `control`: the browser driver's modifier for editing shortcuts, as `--key-modifier`. The flag wins when both are set. |
 | `RAVENSIGHT_INGEST_KEY` | The game's ingest key, so a web run can emit its synthetic session into your event stream. `ingest_key` in `.ravensight/config.json` does the same. Without either the run still reports; it just sends no telemetry. |
 | `RAVENSIGHT_GODOT` | Path to the Godot binary. Tried first, then `$GODOT`, then `$GODOT_PATH`, then the PATH, then the platform's usual install locations. `--godot-path` beats all of them. |
 | `RAVENSIGHT_FFMPEG` | Path to ffmpeg. |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ravensight-playtest",
-  "version": "0.2.6",
+  "version": "0.2.8",
   "description": "AI personas play your game and file reports to Ravensight.",
   "license": "MIT",
   "homepage": "https://ravensight.io/docs/#playtest",

package/src/cli.js

@@ -63,6 +63,7 @@ Run options (parsed by the runner, see src/run/args.js):
   --upload-video            upload session.webm as well as the reports
   --upload-transcript       upload transcript.jsonl as well as the reports
   --dry-upload              list every file and byte that would be sent, send nothing
+  --key-modifier <name>     meta or control: the modifier for editing shortcuts in the browser, default the platform's own
   --headed                  show the browser
   --headless                hide it
   --verbose                 print each turn

package/src/commands/fakeCore.js

@@ -18,13 +18,17 @@ const SKILL = ['---', 'name: persona-playtest', 'prompt_version: persona-playtes
 const PROFILE_SKILL = ['---', 'name: game-profile', 'prompt_version: game-profile@0.1.0', '---', '# Game Profile'].join('\n');
 const FIX_SKILL = ['---', 'name: fix', 'prompt_version: fix@0.1.0', '---', '# Fix'].join('\n');
 
-/** The server's table, verbatim. */
+/**
+ * The server's table, verbatim, including the change that lets a run out of
+ * money in its report step end as budget_exceeded from `reporting` or
+ * `uploading`.
+ */
 export const RUN_TRANSITIONS = Object.freeze({
   queued: ['launching', 'canceled', 'skipped', 'failed', 'interrupted'],
   launching: ['playing', 'failed', 'canceled', 'interrupted'],
   playing: ['reporting', 'failed', 'canceled', 'budget_exceeded', 'interrupted'],
-  reporting: ['uploading', 'failed', 'interrupted'],
-  uploading: ['succeeded', 'failed', 'interrupted'],
+  reporting: ['uploading', 'failed', 'budget_exceeded', 'interrupted'],
+  uploading: ['succeeded', 'failed', 'budget_exceeded', 'interrupted'],
   interrupted: ['launching', 'playing', 'reporting', 'uploading', 'failed', 'canceled'],
   succeeded: [],
   failed: [],
@@ -33,6 +37,13 @@ export const RUN_TRANSITIONS = Object.freeze({
   skipped: []
 });
 
+/** An API from before that change, which refuses budget_exceeded after `playing`. */
+export const OLDER_RUN_TRANSITIONS = Object.freeze({
+  ...RUN_TRANSITIONS,
+  reporting: ['uploading', 'failed', 'interrupted'],
+  uploading: ['succeeded', 'failed', 'interrupted']
+});
+
 export const RUN_TERMINAL_STATES = Object.freeze(['succeeded', 'failed', 'canceled', 'budget_exceeded', 'skipped']);
 
 /** Shaped like cli-core's `ApiError`: `status`, `code`, `body`. */
@@ -234,7 +245,10 @@ export function fakeCore({
   findings = [fakeFinding()],
   triage = { verdict: 'bug', confidence: 0.9, source: 'brief' },
   graphContext = { levels: [{ name: 'title', exits: ['level_1'] }] },
-  verifyVerdict = { reproduced: 0.1, source: 'judge' }
+  verifyVerdict = { reproduced: 0.1, source: 'judge' },
+  transitions = RUN_TRANSITIONS,
+  completeExtras = {},
+  runRefund = {}
 } = {}) {
   const calls = [];
   const record = (name, payload) => calls.push({ name, ...payload });
@@ -250,7 +264,7 @@ export function fakeCore({
 
   const move = (runId, to) => {
     const from = runStates.get(runId) || 'queued';
-    if (!(RUN_TRANSITIONS[from] || []).includes(to)) {
+    if (!(transitions[from] || []).includes(to)) {
       throw new FakeApiError({
         status: 409,
         code: 'invalid_transition',
@@ -347,8 +361,23 @@ export function fakeCore({
         if (RUN_TERMINAL_STATES.includes(from)) {
           throw new FakeApiError({ status: 409, code: 'run_already_complete', message: 'That run has already finished.', body: { state: from } });
         }
-        runStates.set(runId, body.state);
-        return { run: { run_id: runId, state: body.state }, dropped_findings: 0, report_lint: [] };
+        // The server's complete reads the table too, after advancing a
+        // `reporting` run that is succeeding to `uploading` itself.
+        const state = body.state || 'succeeded';
+        const effective = from === 'reporting' && state === 'succeeded' ? 'uploading' : from;
+        if (!(transitions[effective] || []).includes(state)) {
+          throw new FakeApiError({ status: 409, code: 'invalid_transition', message: `A run cannot go from "${effective}" to "${state}".`, body: { from: effective, to: state } });
+        }
+        runStates.set(runId, state);
+        // The server's refund fields ride on `run`. Before the job settles
+        // `refunded` is false and `refund_expected` says whether settling
+        // would give the run's unit back.
+        return {
+          run: { run_id: runId, state, refunded: false, refunded_cents: 0, refund_expected: false, refund_expected_cents: 0, ...runRefund },
+          dropped_findings: 0,
+          report_lint: [],
+          ...completeExtras
+        };
       },
       async list(gameId, jobId) {
         record('list', { jobId });

package/src/commands/run.js

@@ -13,6 +13,7 @@ import { jobDir, runDir as runDirFor, JOB_FILES, RUN_FILES } from '../run/paths.
 import { readBridgeConfig } from "../run/drivers/stdio.js";
 import { WEBDRIVER_NAMES, readDriverConfig } from '../run/drivers/webdriver.js';
 import { runPersona } from '../run/personaLoop.js';
+import { reconcileDroppedFindings } from '../run/dropped.js';
 import * as stateFile from '../run/state.js';
 import { emitRunEvents } from '../run/synthetic.js';
 
@@ -90,6 +91,8 @@ async function run(flags, injected, deps, log) {
   const { context, gameId, repoRoot, apiUrl, cliVersion, api } = setup;
 
   if (options.controller && !['standard', 'jev'].includes(options.controller)) { log('Choose --controller standard or --controller jev.'); return EXIT.ENVIRONMENT; }
+  const keyModifier = resolveKeyModifier(options.keyModifier);
+  if (keyModifier.error) { log(keyModifier.error); return EXIT.ENVIRONMENT; }
   const driverName = options.driver || (context.config && context.config.driver) || 'playwright_web';
   const target = resolveTarget(options, context.config, driverName);
   if (!target) {
@@ -433,7 +436,14 @@ export async function playJob({
     log
   });
 
+  // A run that ran out of money in the report step, after its partial report
+  // was uploaded and the run completed, delivered what it bought: it does not
+  // make the job exit 2. One the server refused to complete exits 1, never 2,
+  // because 2 promises the partial results were shipped.
+  const playBudgetHit = results.some(result => result.state === 'budget_exceeded' && !result.reportStepBudget);
   const budgetHit = results.some(result => result.state === 'budget_exceeded');
+  const reportBudgetRefused = results.some(result => result.reportStepBudget === 'refused');
+  const reportBudgetDelivered = results.some(result => result.reportStepBudget === 'delivered' || result.reportStepBudget === 'dry_upload');
   const canceled = results.some(result => result.state === 'canceled');
   const anySucceeded = results.some(result => result.state === 'succeeded');
 
@@ -449,8 +459,9 @@ export async function playJob({
   log(`Reports: ${directory}`);
 
   if (canceled) return EXIT.CANCELED;
-  if (budgetHit) return EXIT.BUDGET;
-  return anySucceeded ? EXIT.OK : EXIT.FAILED;
+  if (reportBudgetRefused) return EXIT.FAILED;
+  if (playBudgetHit) return EXIT.BUDGET;
+  return anySucceeded || reportBudgetDelivered ? EXIT.OK : EXIT.FAILED;
 }
 
 /**
@@ -489,7 +500,7 @@ export async function playPersonaRun({
   // Headless is right for a browser and wrong for Godot: a headless Godot
   // routes no GUI input, so it is a smoke-test mode rather than a way to play.
   const headless = driverName === 'godot_driver' ? options.headless === true : options.headed !== true;
-  const driver = await makeDriver(driverName, driverOptions({ headless, driverName, target, directory, entry, jobId }));
+  const driver = await makeDriver(driverName, driverOptions({ headless, driverName, target, directory, entry, jobId, keyModifier: options.keyModifier }));
   // The ledger is created here rather than inside the loop so the heartbeat
   // can feed it the server's own figure for what this run has spent. Prices
   // are server side only, so that answer is the only real one.
@@ -554,7 +565,47 @@ export async function playPersonaRun({
     if (inFlight) inFlight.delete(runId);
     const failed = error.outcome || { state: 'failed', quitReason: 'error', actionsTaken: 0, report: null, reportLint: [], reachedState: 'playing' };
     await stateFile.patchRun(repoRoot, journal, runId, { state: failed.state, error: String(error && error.message), quit_reason: failed.quitReason });
-    await deliver({ deps, api, gameId, jobId, runId, directory, outcome: failed, options, log });
+    const delivery = {};
+    const answer = await deliver({ deps, api, gameId, jobId, runId, directory, outcome: failed, options, log, repoRoot, delivery });
+    if (isReportStepBudget(failed)) {
+      // The money ran out while the report was being written. The partial
+      // report went up and the run is completed as budget_exceeded from
+      // `uploading`; what happens to the charge is the server's decision.
+      if (answer) {
+        await stateFile.patchRun(repoRoot, journal, runId, { uploaded: true, completed: true });
+        // "Uploaded" only when it was: the server can refuse the partial
+        // report (and the run is completed without it), and the report may
+        // never have been written.
+        const message = delivery.reportRefused || !failed.report
+          ? `Run ran out of budget; the server refused the partial report; the persona's notes are in ${directory}`
+          : reportBudgetMessage(answer);
+        log(`run ${runId} (${persona.slug}): ${message}`);
+        return { runId, persona: persona.slug, state: failed.state, quitReason: failed.quitReason, reportStepBudget: 'delivered' };
+      }
+      if (options.dryUpload) {
+        // Nothing was sent, so the server still holds the run where it was.
+        // Not a refusal, so the job does not exit 1 for it.
+        log(`run ${runId} (${persona.slug}) ran out of budget while writing its report: dry upload, run not completed on the server.`);
+        return { runId, persona: persona.slug, state: failed.state, quitReason: failed.quitReason, reportStepBudget: 'dry_upload' };
+      }
+      log(`run ${runId} (${persona.slug}) ran out of budget while writing its report, and the server would not complete it: ${delivery.message || 'no reason given'}`);
+      // An older API refuses budget_exceeded after `playing`. The CLI moved
+      // the run to `uploading` itself, so it does not leave it there: `failed`
+      // is allowed from `uploading` on every table.
+      try {
+        await api.runs.complete(gameId, jobId, runId, {
+          state: 'failed',
+          reason: `ran out of budget while writing the report; the server refused budget_exceeded (${delivery.message || 'no reason given'})`,
+          actions_taken: failed.actionsTaken,
+          quit_reason: failed.quitReason
+        });
+        await stateFile.patchRun(repoRoot, journal, runId, { completed: true, state: 'failed' });
+        log(`run ${runId} (${persona.slug}): completed as failed instead.`);
+      } catch (fallbackError) {
+        log(`run ${runId} (${persona.slug}): could not be completed as failed either (${String(fallbackError && fallbackError.message)}). "ravensight-playtest resume" can retry it.`);
+      }
+      return { runId, persona: persona.slug, state: failed.state, quitReason: failed.quitReason, reportStepBudget: 'refused', error };
+    }
     log(`run ${runId} (${persona.slug}) failed: ${String(error && error.message)}`);
     return { runId, persona: persona.slug, state: failed.state, quitReason: failed.quitReason, error };
   }
@@ -576,6 +627,9 @@ export async function playPersonaRun({
       apiUrl,
       ingestKey: (config && config.ingest_key) || process.env.RAVENSIGHT_INGEST_KEY || null,
       playtestToken: entry.playtest_token,
+      // The web driver injects the playtest token into the page (see
+      // driverOptions), so the game's own SDK sends with it.
+      pageSdk: driverName === 'playwright_web' && Boolean(entry.playtest_token),
       context: { jobId, runId, persona: persona.slug, gameVersion: (config && config.game_version) || 'unknown' },
       report: outcome.report,
       driver: driverName,
@@ -604,7 +658,7 @@ export async function playPersonaRun({
  * uploaded and completed in exactly this order, including the `uploading`
  * transition the server's table allows from `reporting` and nowhere else.
  */
-export async function deliver({ deps, api, gameId, jobId, runId, directory, outcome, options, log, journal, repoRoot }) {
+export async function deliver({ deps, api, gameId, jobId, runId, directory, outcome, options, log, journal, repoRoot, delivery = null }) {
   const body = {
     state: outcome.state,
     reason: outcome.quitDetail || '',
@@ -655,23 +709,90 @@ export async function deliver({ deps, api, gameId, jobId, runId, directory, outc
       // report.json either way.
       if (!(body.report && body.report.partial && error && error.status === 422 && error.code === 'invalid_report')) throw error;
       log(`run ${runId}: the server refused the partial report (${String(error.message)}), so the run is completed without it. The notes are in ${directory}.`);
+      if (delivery) delivery.reportRefused = true;
       const { report: ignored, report_md: alsoIgnored, ...withoutReport } = body;
       answer = await api.runs.complete(gameId, jobId, runId, withoutReport);
     }
     if (journal) await stateFile.patchRun(repoRoot, journal, runId, { uploaded: true, completed: true });
     if (answer && answer.dropped_findings > 0) {
       log(`run ${runId}: the server dropped ${answer.dropped_findings} finding(s) with no usable evidence.`);
+      await markDropped({ directory, jobId, runId, repoRoot, outcome, droppedCount: answer.dropped_findings, droppedIds: answer.dropped_finding_ids, options, log });
     }
     if (answer && Array.isArray(answer.report_lint) && answer.report_lint.length > 0) {
       log(`run ${runId}: report lint: ${answer.report_lint.join('; ')}`);
     }
     return answer;
   } catch (error) {
+    if (delivery) {
+      delivery.message = String(error && error.message);
+      delivery.status = error && error.status;
+      delivery.code = error && error.code;
+    }
     log(`run ${runId}: upload or complete was refused (${String(error && error.message)}). The artifacts are in the run directory and "ravensight-playtest resume" can retry it.`);
     return null;
   }
 }
 
+/**
+ * Bring the local report into line with what the server kept, so report.md,
+ * report.json, the aggregate and the telemetry count what the dashboard
+ * counts. A failure here never fails the delivery: the run is already
+ * complete on the server.
+ */
+async function markDropped({ directory, jobId, runId, repoRoot, outcome, droppedCount, droppedIds, options, log }) {
+  try {
+    const result = await reconcileDroppedFindings({ directory, droppedCount, droppedIds, options, journal: { jobId, runId, repoRoot } });
+    if (!result) return;
+    if (outcome) {
+      outcome.report = result.report;
+      outcome.reportMarkdown = result.markdown;
+    }
+    log(result.identified
+      ? `run ${runId}: marked ${result.dropped.join(', ')} as dropped in report.md and report.json, under "Dropped by the server".`
+      : `run ${runId}: could not tell which finding(s) the server dropped, so report.md says so and the dashboard is the record.`);
+  } catch (error) {
+    log(`run ${runId}: could not mark the dropped finding(s) in the local report (${String(error && error.message)}).`);
+  }
+}
+
+/**
+ * Whether a run ran out of money in the report step: it is budget_exceeded
+ * and the server had already accepted `reporting` (or `uploading`) for it.
+ *
+ * @param {Object} outcome
+ * @returns {boolean}
+ */
+export function isReportStepBudget(outcome) {
+  return Boolean(outcome) && outcome.state === 'budget_exceeded' && ['reporting', 'uploading'].includes(outcome.reachedState);
+}
+
+/**
+ * What the developer is told when a report-step budget run was completed.
+ * The refund is the server's decision; the CLI only repeats what the
+ * complete answer says about it. Before the job settles the answer carries
+ * `refund_expected` and `refund_expected_cents`; `refunded` is true only once
+ * settlement gave the unit back. Each may be on the answer or on its `run`.
+ *
+ * @param {Object} answer the complete response
+ * @returns {string}
+ */
+export function reportBudgetMessage(answer) {
+  const read = name => {
+    if (!answer) return undefined;
+    if (answer[name] !== undefined) return answer[name];
+    return answer.run ? answer.run[name] : undefined;
+  };
+  const prefix = 'Run ran out of budget; the partial report was uploaded and';
+  if (read('refunded') === true) return `${prefix} the run was refunded`;
+  if (read('refund_expected') === true) {
+    const cents = read('refund_expected_cents');
+    return Number.isFinite(Number(cents)) && cents !== null && Number(cents) > 0
+      ? `${prefix} a refund of ${formatCents(cents)} is expected when the job settles`
+      : `${prefix} a refund is expected when the job settles`;
+  }
+  return `${prefix} the refund decision is the server's`;
+}
+
 /**
  * Build the aggregate, and drive the `aggregate_report` run's own state machine
  * while doing it.
@@ -945,9 +1066,12 @@ export async function resolveSetup(deps, options, log) {
  *
  * @returns {Object}
  */
-export function driverOptions({ headless, driverName, target, directory, entry, jobId }) {
+export function driverOptions({ headless, driverName, target, directory, entry, jobId, keyModifier }) {
+  // Only the browser driver sends key chords with a modifier.
+  const modifier = driverName === 'playwright_web' ? resolveKeyModifier(keyModifier).modifier : null;
   return {
     headless,
+    ...(modifier ? { modifier } : {}),
     playtest: {
       token: entry.playtest_token || null,
       runId: entry.run_id,
@@ -959,6 +1083,25 @@ export function driverOptions({ headless, driverName, target, directory, entry,
   };
 }
 
+/**
+ * The modifier the web driver uses for editing shortcuts, from
+ * `--key-modifier meta|control`, else `RAVENSIGHT_KEY_MODIFIER`, else the
+ * platform's own (Command on macOS, Control elsewhere, decided in the driver).
+ * For a remote or containerised browser whose platform is not this machine's.
+ *
+ * @param {string} [value] the flag's value
+ * @param {Object} [env]
+ * @returns {{modifier: 'Meta'|'Control'|null, error?: string}}
+ */
+export function resolveKeyModifier(value, env = process.env) {
+  const raw = value !== undefined && value !== null && value !== '' ? value : env.RAVENSIGHT_KEY_MODIFIER;
+  if (raw === undefined || raw === null || raw === '') return { modifier: null };
+  const key = String(raw).trim().toLowerCase();
+  if (['meta', 'cmd', 'command'].includes(key)) return { modifier: 'Meta' };
+  if (['control', 'ctrl'].includes(key)) return { modifier: 'Control' };
+  return { modifier: null, error: `Choose --key-modifier meta or --key-modifier control (or set RAVENSIGHT_KEY_MODIFIER), not "${raw}".` };
+}
+
 /**
  * The persona reports on disk, for the aggregate. A partial report (written
  * from the notes when the money ran out) is loaded as a `budget_exceeded`

package/src/commands/upload.js

@@ -8,6 +8,9 @@ import { pending } from '../upload/queue.js';
 import { uploadRunDir } from '../upload/index.js';
 import { JOB_UPLOAD_PATHS } from '../upload/allowlist.js';
 import { ui } from '../ui/index.js';
+
+/** Report files never re-sent for a run the journal marks completed. */
+const COMPLETED_RUN_REPORTS = Object.freeze(['report.md', 'report.json']);
 import { CliError, ExitCode } from '../errors.js';
 
 /**
@@ -73,11 +76,18 @@ async function drainJob(context, jobId, flags) {
   const results = [];
 
   for (const runId of runIds) {
+    // A completed run's report went to the server with `complete`, which
+    // validated it; the copy on disk may since have been marked locally
+    // (dropped findings), and sending it again would replace the validated
+    // artifact, or be refused once the job is closed.
+    const completed = Boolean(state && state.runs && state.runs[runId] && state.runs[runId].completed);
     const result = await uploadRunDir(paths.runDir(jobId, runId, { repoRoot }), {
       api: context.api,
       gameId,
       jobId,
       runId,
+      exclude: completed ? COMPLETED_RUN_REPORTS : null,
+      excludeReason: 'the run is complete; its report was delivered with complete',
       includeVideo,
       includeTranscript,
       dryRun: Boolean(flags.dryRun),

package/src/run/aggregate.js

@@ -1,4 +1,5 @@
 import { createHash } from 'node:crypto';
+import { fillPlaceholders } from './placeholders.js';
 import { writeFile } from 'node:fs/promises';
 import path from 'node:path';
 import { parseJsonArray, parseJsonObject } from './json.js';
@@ -331,7 +332,10 @@ export async function buildAggregate({
     persona: run.persona,
     state: run.state,
     report: run.report,
-    findings: run.report.findings || []
+    // A finding the server dropped for want of evidence is marked on disk
+    // after delivery (./dropped.js) and kept out of the aggregate, so the
+    // job report counts what the dashboard counts.
+    findings: (run.report.findings || []).filter(finding => !(finding && finding.dropped === true))
   }));
   const personasRun = new Set(inputs.map(run => run.persona).filter(Boolean)).size;
 
@@ -447,6 +451,7 @@ export async function judgeAggregate({ client, jobId, runId, ranked, inputs, bri
     ' "known":["iss_..."], "excluded_by_intent":[{"issue_id":"iss_...","intent":"<the stated intent it matches>"}],',
     ' "tldr":["up to five short bullets: the most important fixes and the strongest positive signal"]}',
     '',
+    'Never write calendar dates or times in the tldr or anywhere else; the runner stamps generated_at on the report.',
     'Rules. A criterion no run touched is "not testable"; never invent evidence for one. An issue is "known" only when the brief already describes it. An issue is excluded by intent only when the persona experience is CONSISTENT with the stated intent; when it CONTRADICTS the intent, leave it in the list, because that is the valuable case.',
     '',
     `success criteria:\n${criteria.length > 0 ? criteria.map(row => `  ${row.criterion_id}: ${row.text}`).join('\n') : '  (none stated)'}`,
@@ -623,7 +628,9 @@ export function renderAggregateMarkdown(json) {
     ? `Observations dropped for lack of evidence:\n${json.observations.map(o => `- ${o.persona}: ${o.text}`).join('\n')}`
     : 'No dropped observations.');
   lines.push('');
-  return lines.join('\n');
+  // The synthesis writes the TL;DR, and a model can copy a template slot into
+  // it as easily as into a persona report.
+  return fillPlaceholders(lines.join('\n'), { generatedAt: json.generated_at, game: json.game_id });
 }
 
 /**

package/src/run/args.js

@@ -22,7 +22,8 @@ const VALUED = new Map([
   ['--cli', 'cli'],
   ['--max-actions', 'maxActions'],
   ['--concurrency', 'concurrency'],
-  ['--quality', 'quality']
+  ['--quality', 'quality'],
+  ['--key-modifier', 'keyModifier']
 ]);
 
 /** Options that are just present or absent. */

package/src/run/drivers/godot.js

@@ -360,6 +360,11 @@ export class GodotDriver {
     if (!action || typeof action.kind !== 'string') {
       return { ok: false, error: 'action needs a kind', changed: false };
     }
+    // Typing more characters is not replacing a value, and claiming it was
+    // would join the old text to the new one without saying so.
+    if (action.kind === 'type' && (action.replace || action.target)) {
+      return { ok: false, error: 'replace (empty a field, then type) and a typing target are only available on playwright_web; this driver can only type more text into what has focus', changed: false };
+    }
     try {
       await this._perform(action);
     } catch (error) {

package/src/run/drivers/stdio.js

@@ -79,6 +79,7 @@ export function createStdioDriver({ spawn: spawnImpl = spawn, timeoutMs = 30000
     observe,
     async act(action) {
       if (!['type', 'key', 'click', 'wait'].includes(action.kind)) return { ok: false, changed: false, error: 'Unsupported bridge action.' };
+      if (action.kind === 'type' && (action.replace || action.target)) return { ok: false, changed: false, error: 'replace (empty a field, then type) and a typing target are only available on playwright_web; this driver can only type more text into what has focus' };
       const before = lastHash;
       const safe = action.kind === 'type' ? { kind: 'type', text: String(action.text || '').slice(0, 2000), submit: Boolean(action.submit) }
         : action.kind === 'key' ? { kind: 'key', key: String(action.key || '').slice(0, 64) }